最近项目上用HANA开发的比较多,之前我是bw用的比较多,就不会有这种问题。我们这个项目很多都是开发的计算视图,但最近做acdoca的逻辑时,计算视图在生产环境执行的时候报错,报的内存溢出,acdoca上亿的数据,直接用计算视图确实跑不出来,跑出来也要跑很久,内存溢出的报错如下这种。
想解决内存溢出和查询慢的这个问题,只有把中间过程落地存储成数据库表了,有点类似BW ADSO的分层,但是HANA分层直接插入数据库表的时候,插入数据的时候也是直接去处理acdoca全部的数据,依然报错,内存溢出。
BW ADSO建模为什么不报错呢。因为adso存储的时候执行DTP。是分包跑的,就是acdoca几亿的 数据量,我也是根据包大小去分开上千个包执行落地的,这次的解决方案就是模仿这种逻辑
定义参数:对HANA最底层视图定义参数,在HANA中间层视图也创建一个相同的参数
参数传递:
在中间层把中间层的视图传递给最底层,
创建存储过程时根据定义的游标,按字段的不通去for循环分开循环执行,字段的值会传递到中间层参数,从而传递到最底层视图的参数,限制最底层数据,实现类似的分包处理。
上面想到了解决办法,怎么实现呢,大体的思路就是分公司或者分期间,去循环执行存储过程,这样可以有效减少最底层处理的数据量,解决内存溢出的问题,执行这个的时候一定要注意,你的计算视图逻辑里,有没有跨公司的逻辑,或者跨年的逻辑,如果有,就不能这样分开去处理,我这里用我项目上的为例,是分开按年份执行的,这样可以在内存允许的范围内减少循环次数。
传参:涉及到减少acdoca最底层的计算数据量,那么我中间层按年份执行的变量参数怎么传给最底层呢,这里涉及到定义HANA参数,以及HANA参数的传递,这是我本次需求的最底层视图和中间层定义的参数和参数限制,最底层为图形化视图,主要是acdoca的明细数据,中间层为计算视图,是通用逻辑处理。
首先看参数的创建和传递
基础数据层视图
最底层为ACDOCA模型,如图创建参数
参数限制写在过滤中
中间层视图
中间层为脚本视图,定义参数及参数传递如下
这里参数传递的语法如下:
PLACEHOLDER."$$参数"=>:需要传递的参数
存储过程中的传参
后面会把存储过程的代码贴出来,可以参考
游标有些没接触过的比较陌生,先说下根据我百度到的和个人理解的概念
什么是数据库游标?
可以将数据库游标视为指向查询结果中特定行的指针。指针可以从一行移动到下一行。根据游标的类型,您甚至可以将其移动到上一行。
可以这样想:SQL结果就像一个袋子,你可以一次保存一大堆行,但不能单独保存任何一行;而游标就像一把镊子。有了它,你可以把手伸进包里抓一排,然后移到下一个。
这样理解游标是不是很像abap定义内表,定义工作区,for循环就是去循环抓取数据。
特殊说明一下游标的语法
--定义游标表及赋值,游标表的数据为select的字段及数据(类似abap定义内表,内表赋值),
DECLARE CURSOR? 游标表?FOR
SELECT DISTINCT 表字段?FROM 查询表";
循环读取数据到R工作区)
FOR 工作区?AS 游标表DO
****数据处理****
FROM 表名(PLACEHOLDER."$$变量$$" =>:?工作区.字段?)--R工作区字段循环传参赋值
END FOR;--结束循环
实现存储过程的代码如下,定义游标关键步骤的注释我都写的很详细,不特殊解释了
create procedure "SAPHANADB".ZPR_insert_ZBWT_DT_BPC as
/********* Begin Procedure Script ************/
BEGIN
--定义游标表ZGJAHR,游标表的数据为select的字段及数据(类似abap定义内表,内表赋值)
DECLARE CURSOR ZGJAHR FOR
SELECT DISTINCT GJAHR FROM "SAPHANADB"."/BIC/AZEFI00012";
truncate table "SAPHANADB"."ZBWT_DT_BPC";
--执行FOR循环,根据游标数据循环读取到R中,(定义的R我理解有点类似定义abap的工作区,循环读取数据到R工作区)
FOR R AS ZGJAHR DO
insert into "SAPHANADB"."ZBWT_DT_BPC"
SELECT
"TIME" || ROW_NUMBER()OVER(ORDER BY "TIME","ENTITY") AS "ID",
"TIME",
CASE WHEN "ACCOUNT" IS NULL THEN '' ELSE "ACCOUNT" END AS "ACCOUNT",
"ENTITY",
"ENTITY_TEXT",
"INTERCO",
"INTERCO_TEXT",
"AUDITTRAIL",
"CATEGORY",
"SCOPE",
"RHCUR",
"RWCUR",
"TYPE",
"MISC",
"KUNNR",
"KIDNO",
"ZZYLIU6",
"ZZYLIU11",
"VTWEG",
CASE WHEN "FLOW" IS NULL THEN '' ELSE "FLOW" END AS "FLOW",
CASE WHEN "ZHBHB" IS NULL THEN '' ELSE "ZHBHB" END AS "ZHBHB",
CASE WHEN "ZLOGIC_CODE" IS NULL THEN '' ELSE "ZLOGIC_CODE" END AS "ZLOGIC_CODE",
CASE WHEN "ZLOGIC_VALUE" IS NULL THEN '' ELSE "ZLOGIC_VALUE" END AS "ZLOGIC_VALUE",
"RSTGR",
"RFAREA",
"DRCRK",
"RASSC",
"MATKL",
CASE WHEN "MATKL_HB" IS NULL THEN '' ELSE "MATKL_HB" END AS "MATKL_HB",
CASE WHEN "CODE_QD_D" IS NULL THEN '' ELSE "CODE_QD_D" END AS "CODE_QD_D",
CASE WHEN "CODE_QD_E" IS NULL THEN '' ELSE "CODE_QD_E" END AS "CODE_QD_E",
"RACCT",
"RMVCT",
CASE WHEN "LAND1" IS NULL THEN '' ELSE "LAND1" END AS "LAND1",
CASE WHEN "REGION" IS NULL THEN '' ELSE "REGION" END AS "REGION",
"ZZYLIU10",
"MATNR",
CASE WHEN "MATNR_TEXT" IS NULL THEN '' ELSE "MATNR_TEXT" END AS "MATNR_TEXT",
"BTTYPE",
"ZDATE",
CASE WHEN "PSPID" IS NULL THEN '' ELSE "PSPID" END AS "PSPID",
CASE WHEN "AUFNR" IS NULL THEN '' ELSE "AUFNR" END AS "AUFNR",
CASE WHEN "ZZYLIU9" IS NULL THEN '' ELSE "ZZYLIU9" END AS "ZZYLIU9",
CASE WHEN "ATTRIBUTE1" IS NULL THEN '' ELSE "ATTRIBUTE1" END AS "ATTRIBUTE1",
CASE WHEN "ATTRIBUTE2" IS NULL THEN '' ELSE "ATTRIBUTE2" END AS "ATTRIBUTE2",
CASE WHEN "ATTRIBUTE3" IS NULL THEN '' ELSE "ATTRIBUTE3" END AS "ATTRIBUTE3",
CASE WHEN "HKONT_SR" IS NULL THEN '' ELSE "HKONT_SR" END AS "HKONT_SR",
CASE WHEN "HKONT_CB" IS NULL THEN '' ELSE "HKONT_CB" END AS "HKONT_CB",
"PS_POSID",
"ZSJLY",
"ZSFJS",
"ZCHBJ",
"WERKS",
"NPLNR",
"ZDXJT",
sum("HSL") AS "HSL",
sum("WSL") AS "WSL",
sum("MSL") AS "MSL"
FROM "_SYS_BIC"."ZP_DFYH_BW.ZP_LL/ZCV_DT_BPC"(PLACEHOLDER."$$IP_GJAHR$$" =>:R.GJAHR)--R工作区字段循环传参赋值
GROUP BY "TIME",
CASE WHEN "ACCOUNT" IS NULL THEN '' ELSE "ACCOUNT" END,
"ENTITY",
"ENTITY_TEXT",
"INTERCO",
"INTERCO_TEXT",
"AUDITTRAIL",
"CATEGORY",
"SCOPE",
"RHCUR",
"RWCUR",
"TYPE",
"MISC",
"KUNNR",
"KIDNO",
"ZZYLIU6",
"ZZYLIU11",
"VTWEG",
CASE WHEN "FLOW" IS NULL THEN '' ELSE "FLOW" END,
CASE WHEN "ZHBHB" IS NULL THEN '' ELSE "ZHBHB" END,
CASE WHEN "ZLOGIC_CODE" IS NULL THEN '' ELSE "ZLOGIC_CODE" END,
CASE WHEN "ZLOGIC_VALUE" IS NULL THEN '' ELSE "ZLOGIC_VALUE" END,
"RSTGR",
"RFAREA",
"DRCRK",
"RASSC",
"MATKL",
CASE WHEN "MATKL_HB" IS NULL THEN '' ELSE "MATKL_HB" END,
CASE WHEN "CODE_QD_D" IS NULL THEN '' ELSE "CODE_QD_D" END,
CASE WHEN "CODE_QD_E" IS NULL THEN '' ELSE "CODE_QD_E" END,
"RACCT",
"RMVCT",
CASE WHEN "LAND1" IS NULL THEN '' ELSE "LAND1" END,
CASE WHEN "REGION" IS NULL THEN '' ELSE "REGION" END,
"ZZYLIU10",
"MATNR",
CASE WHEN "MATNR_TEXT" IS NULL THEN '' ELSE "MATNR_TEXT" END,
"BTTYPE",
"ZDATE",
CASE WHEN "PSPID" IS NULL THEN '' ELSE "PSPID" END,
CASE WHEN "AUFNR" IS NULL THEN '' ELSE "AUFNR" END,
CASE WHEN "ZZYLIU9" IS NULL THEN '' ELSE "ZZYLIU9" END,
CASE WHEN "ATTRIBUTE1" IS NULL THEN '' ELSE "ATTRIBUTE1" END,
CASE WHEN "ATTRIBUTE2" IS NULL THEN '' ELSE "ATTRIBUTE2" END,
CASE WHEN "ATTRIBUTE3" IS NULL THEN '' ELSE "ATTRIBUTE3" END,
CASE WHEN "HKONT_SR" IS NULL THEN '' ELSE "HKONT_SR" END,
CASE WHEN "HKONT_CB" IS NULL THEN '' ELSE "HKONT_CB" END,
"PS_POSID",
"ZSJLY",
"ZSFJS",
"ZCHBJ",
"WERKS",
"NPLNR",
"ZDXJT";
END FOR;--结束循环
END
以上方式处理后,再执行存储过程,就会分包处理,不会报错了,如果对大家有帮助希望点赞一下,谢谢,也欢迎评论交流。