2057046 - Faq Sap Hana Delta Merges

2017-01-23 Page 1/222057046 - FAQ: SAP HANA Delta Merges Version 93 Type SAP Knowledge Base Article Language English Master Language English Release Status Released to Customer Category How To Component HAN-DB (SAP HANA Database) Released On 17.12.2016 Please find the original document at https://launchpad.support.sap.com/#/notes/2057046 Symptom You want to understand merges in SAP HANA environments. Environment SAP HANA Cause 1. What are delta merges in SAP HANA environments? 2. Where can I find more detailed information related to delta merges? 3. Which indications exist for problems in the delta merge area? 4. Is the delta merge an online operation? 5. How is a merge triggered? 6. How can the history of temporal tables be merged manually? 7. How can a manual delta merge be triggered for individual partitions? 8. How can delta merges and the delta storage be monitored? 9. How can peaks of high load due to concurrent or parallelized delta merges be prevented? 10. How can the load, priority and decision functions be interpreted? 11. Can merge related parameters be adjusted? 12. Why can large delta storages be critical? 13. What should be done in case of large delta storages? 14. How can I trace the merge activities in order to analyze problems with merges? 15. Why is more memory and disk space required during the delta merge operation? 16. What can be done to limit the memory requirements during delta merges? 17. How can I check for which tables auto merges are disabled? 18. How can I check for which tables persistent merges are disabled? 19. In which situations are critical merges executed? 20. Why do I see auto merges on tables although auto merges are disabled on table level? 21. What is the meaning of the different TYPE values in table M_DELTA_MERGE_STATISTICS? 22. What do the shortcuts mean which are used in the decision and cost functions? 23. In which scenarios can it be useful to reduce or disable auto merges? 24. Is the default auto merge decision function already optimal? 25. Is the default critical merge decision function already optimal? 26. What needs to be done in case of merge related errors? 27. Is a restart required when changing delta merge related parameters? 28. Is the delta storage empty after a merge? 29. What are reasons why auto merges aren't executed as expected? 30. How can I temporarily disable merges for a critical table? 31. Does M_DELTA_MERGE_STATISTICS also contain merge information from the secondary site of a system © 2017 SAP SE or an SAP affiliate company. All rights reserved 2017-01-23 Page 2/22 replication scenario? 32. For which services is the mergedog required? Resolution 1. What are delta merges in SAP HANA environments? Changes (INSERT, UPDATE, DELETE) to SAP HANA column store tables are not directly updated in the main storage. Instead the change information is written to a dedicated delta storage: From time to time the main storage needs to be updated with the content of the delta storage. This operation is called delta merge. The rough execution steps of a delta merge are illustrated in the following picture: You can see that during a delta merge two separate main and delta storages exist. 2. Where can I find more detailed information related to delta merges? More detailed information about the delta merge operation can be found in the SAP HANA Administration Guide. The SAP HANA Troubleshooting and Performance Analysis Guide provides troubleshooting details for problems in the delta merge area. 3. Which indications exist for problems in the delta merge area? The following SAP HANA alerts indicate problems in the delta merge area: Alert Name Description © 2017 SAP SE or an SAP affiliate company. All rights reserved column-store tables Size of delta storage of 29 Determines the size of the delta storage of column tables. tables SQL: "HANA_Configuration_MiniChecks" (SAP Note 1969700. column-store tables Auto merge for 88 column-store Determines if the delta merge of a table was executed successfully or not. last day) 541 Failing delta merges (error messages. see SAP Note 2098801 for more information End of merge: Switch from source to target storages 5. All rights reserved . last day) 542 Auto merge tables with delta storage > 5 GB 543 Auto merge tables with many delta records 544 Non-auto merge tables with delta storage > 5 GB 545 Non-auto merge tables with many delta records 546 Delta size of small auto merge tables (GB) 547 Non BW tables with disabled auto merge 548 Tables with disabled persistent merge 4. 1999993) returns a potentially critical issue (C = 'X') for one of the following individual checks: Check ID Details 538 Delta merges > 900 s (last day) 540 Failing delta merges (info messages. Is the delta merge an online operation? Most steps of the delta merge are performed online. The following picture provides a rough overview: © 2017 SAP SE or an SAP affiliate company. forced and conditioned. mergedog is the system process that periodically checks 10 (mergedog) column tables to determine whether or not a delta merge operation needs to be configuration executed. Record count of delta storage of 19 Determines the number of records in the delta storage of column-store tables. so concurrent read and write accesses to the table aren't blocked.automatically and manually. How is a merge triggered? Merges can be triggered in the following ways .2017-01-23 Page 3/22 Determines whether or not the 'active' parameter in the 'mergedog' section of system Delta merge configuration file(s) is 'yes'. Only during the following typically short periods of time a lock is acquired blocking change operations: Beginning of merge: Move of uncommitted rows from the source delta storage (Delta1) to the target delta storage (Delta2). Detailed automatic merge conditions are maintained using the following parameter: indexserver.ini -> [mergedog] -> auto_merge_decision_func Auto automatic.in the following contexts: Recovery Log replay Automatic merges are performed by the mergedog which is configured with the following parameters: Parameter Value true: Activates mergedog (default) indexserver.ini -> [mergedog] .<frequency_ms>: Defines the interval (in ms) of mergedog > check_interval invocations (default: 60000. All rights reserved . If required it can be activated and deactivated on table level using the following SQL commands: ALTER TABLE "<table_name>" ENABLE AUTOMERGE ALTER TABLE "<table_name>" DISABLE AUTOMERGE © 2017 SAP SE or an SAP affiliate company.e.ini -> [mergedog] - > active false: Deactivates mergedog indexserver.independent of mergedog and manual triggers . Normal system operation merge conditioned Per default the auto merge option is active for each table. 1 minute) Further details for the merge types can be found in the table below: Merge Classification Usage Scenarios Details Type Automatic merges are performed by the mergedog process when certain conditions are fulfilled.2017-01-23 Page 4/22 Additionally there can be implicit merges . i. They are either executed immediately or . instead it makes sure that the delta log on disk is cleaned immediately (see SAP Note 2144274).2017-01-23 Page 5/22 Critical merges are performed by the mergedog process in order to avoid system instabilities.in case of high concurrent merge activity . Alternatively you can use SAP HANA Studio for that Exceptional situations Hard manual. During times of critical Critical automatic..g. purpose: (e. Hard merges are triggered with the following SQL command: MERGE DELTA OF "<table_name>" [FORCE REBUILD] The FORCE REBUILD option doesn't indicate a forced merge.at a later point in time. because reasonable merge operations are performed by the mergedog or by smart merges controlled by the application." In exceptional cases you can trigger a hard merge of all tables using the following Python script delivered as part of the SAP HANA installation: /usr/sap/<sid>/HDB<inst_id> /exe/python_support/mergeAllDeltaTables. manual. problems with auto merge command: immediate merge or specific tables) MERGE DELTA OF "<table_name>" WITH PARAMETERS ('FORCED_MERGE' = 'ON') © 2017 SAP SE or an SAP affiliate company. All rights reserved . (e.py Be aware that this action can be responsible for CPU and memory bottlenecks and there is normally no need to execute it.. Exceptional situations Forced Forced merges are triggered with the following unconditioned. problems with auto merge unconditioned merge or specific tables) SAP HANA Studio -><system> -> Catalog -><schema> - ><table> -> "Perform Delta Merge. SAP HANA memory The critical merge conditions are maintained using the merge conditioned allocation following parameter: indexserver. A forced merge is a variant of a hard merge which is generally executed immediately.ini -> [mergedog] -> critical_merge_decision_func Hard merges are unconditioned merges controlled by the application or administrator.g. application (e. If required it can be activated and deactivated on table level using the following SQL commands: ALTER TABLE "<table_name>" ENABLE PERSISTENT MERGE ALTER TABLE "<table_name>" DISABLE PERSISTENT MERGE 6.2017-01-23 Page 6/22 Smart merges are conditioned merges controlled by the application. It makes sure that auto merge operations don't interfer negatively with the business activities. (e. How can the history of temporal tables be merged manually? © 2017 SAP SE or an SAP affiliate company.even if the configuration may still indicate that they should be done. The merge only happens in memory with the advantage of reduced disk I/O and the disadvantage of a longer delta log and longer recovery times.ini -> [mergedog] -> smart_merge_decision_func Smart merges are triggered with the following command: MERGE DELTA OF "<table_name>" WITH PARAMETERS ('SMART_MERGE' = 'ON') In SAP ABAP environments you can use the function module TREX_EXT_MERGE_DELTA_INDEX for triggering smart merges from application side. A memory merge is a variant of a hard merge which is not persisted. This approach can be useful during specific application scenarios like BW data loads. All rights reserved . Afterwards they are no longer possible . Smart merges can be controlled by setting the following parameter to 'true' (enabled.g.ini -> [mergedog] -> Standard approach for smart_merge_enabled merges controlled by Smart manual. to avoid disk I/O (SPS < not persisted bottlenecks) = 08) MERGE DELTA OF "<table_name>" WITH PARAMETERS ('MEMORY_MERGE' = 'ON') Per default the persistent merge option is active for each table. default) or 'false' (disabled): indexserver. Memory Memory merges are triggered with the following manual. in order When a smart merge is triggered by the application. the merge conditioned to avoid interference of following parameter controls if a merge is actually auto merges with data executed: load activities) indexserver. Memory merges are only available up to SPS 08.g. Exceptional situations merge command: unconditioned. merge type. merge duration. Example output: © 2017 SAP SE or an SAP affiliate company..2017-01-23 Page 7/22 The history part of temporal tables can be merged by adding the key word HISTORY after MERGE: MERGE HISTORY DELTA OF "<table_name>" . 7. number of rows merged and merge errors. It provides information like execution time. Example output: The current utilization of the delta storage can be analyzed using SQL: "HANA_Tables_ColumnStore_DeltaStorage" (SAP Note 1969700). How can a manual delta merge be triggered for individual partitions? An individual table partition can be merged by including the PART<part_id> clause in the MERGE command: MERGE DELTA OF "<table_name>" PART <part_id> . All rights reserved . How can delta merges and the delta storage be monitored? Delta merge activities are tracked in table M_DELTA_MERGE_STATISTICS and its history in HOST_DELTA_MERGE_STATISTICS... See SAP Note 2088971 for details how to configure the amount of data available in monitoring view M_DELTA_MERGE_STATISTICS. 8. table name. You can use SQL: "HANA_Tables_ColumnStore_Merges" (SAP Note 1969700) to retrieve information.. How can peaks of high load due to concurrent or parallelized delta merges be prevented? All merge operations apart from forced merges can only start if a so-called merge token is assigned. When a merge is triggered. the CPU utilization and the heap memory situation. SAP HANA calculates the number of required merge tokens. Typical symptoms are "Semaphore Wait" locks on TRexAPI::TableMergeRequest::TableMergeRequest as described in SAP Note 1999998. Se "HANA_Threads_ThreadSamples_AggregationPerTimeSlice" 2114710 for Starting with the view M_J contains info current M_JOB_PROGRESS SQL: "HANA_Jobs_JobProgress" M_DELTA_M related activi 'BW F Fact T 'Delta Merge Compression 9.ini -> [mergedog] -> load_balancing_func The value calculation includes information like the number of available CPUs. the merges have to wait. the value defined by the following parameter is used (default: 2): indexserver.2017-01-23 Page 8/22 Furthermore merge activities can be monitored in the following ways: Tables SQL statement (SAP Note 1969700) Merge relate SQL: "HANA_Threads_CurrentThreads" can provide i M_SERVICE_THREADS SQL: are currently M_SERVICE_THREAD_SAMPLES "HANA_Threads_ThreadSamples_FilterAndAggregation" kind of detail HOST_SERVICE_THREAD_SAMPLES SQL: executed.ini -> [mergedog] -> token_per_table Furthermore there are priority parameters that are used to calculate priorities of merge requests so that they can be processed in an optimized order: © 2017 SAP SE or an SAP affiliate company. If this calculation is not possible. All rights reserved . The total number of available merge tokens is calculated via the following parameter: indexserver. If no merge token is available. 2017-01-23 Page 9/22 indexserver. With this information it is possible to understand better how the execution of merges depend on factors like CPU.001 Deleted main rows > 1000 ) ) ) or ( THM < 256000 and Less than 256 GB of heap ( DMS > 50 or Delta memory size > 50 MB DCC > 8 or Delta cell count > 8 million (rows * columns) DLS > 100 Delta log size > 100 MB ) ) ) and ( DUC < 0. How can the load.001 Deleted main rows > 1000 ) ) and ( DUC<0.2*MRC and Deleted main rows > 20% of main rows and DMR > 0.1 or Uncommitted delta row count < 100000 0.2 * MRC and Deleted main rows > 20% of main rows and DMR > 0. All rights reserved . priority and decision functions be interpreted? The load.ini -> [mergedog] -> hard_merge_priority_func indexserver. e. It is not trivial to understand their meaning. In the following a typical auto merge decision function (SPS <= 08) is illustrated: ( ( THM >= 256000 and At least 256 GB of heap ( ( ( DMS > 100 or Delta memory size > 100 MB DCC > 100 or Delta cell count > 100 million (rows * columns) DLS > 1000 Delta log size > 1 GB ) and DRC > MRC / 100 Delta row count more than 1% of main row count ) or ( DMR > 0.ini -> [mergedog] -> smart_merge_priority_func 10. Can merge related parameters be adjusted? © 2017 SAP SE or an SAP affiliate company.ini -> [mergedog] -> auto_merge_priority_func indexserver.05*DRC>=DUC Uncommitted delta row count <= 5% of delta row count ) It is possible to include further general conditions into these functions.g. to change the behavior during specific times or for specific tables.1 or Uncommitted delta row count < 100000 0.05% of process allocation limit DCC>100 Delta cell count > 100 million ) and DRC > MRC/100 Delta row count more than 1% of main row count ) or ( DMR>0. Example: (decision function including time dependent rules) DMS > 1000 or DMS > 42 and weekday(now())=6 and secondtime(now())> secondtime('01:00') and secondtime(now())<secondtime('02:00') 11.05 * DRC >= DUC Uncommitted delta row count <= 5% of delta row count ) With SPS 09 a different default for the auto merge decision function is in place: ( ( ( DMS>PAL/2000 or Delta memory size > 0. The SAP HANA Administration Guide describes the shortcuts. priority and decision function parameters contain quite complex logical expressions with several 3 character shortcuts. heap memory or the delta memory size. 12. What should be done in case of large delta storages? At first you should check if the large delta storage really has to be treated as a problem. If you suffer from large delta storages you can proceed according to SAP Note 1977314 in order to understand and resolve the root cause. 14. More precisely: size of source main + size of target main + size of source delta. Therefore it is recommended to adjust merge related parameters only in exceptional situations. times Performance Database requests can take longer because the delta storage is not read-optimized. Why is more memory and disk space required during the delta merge operation? A delta merge needs about twice the size of the table (partition) in memory and on disk to perform the merge. Results will be written to the standard service trace file. See SAP Note Increased 2014987 which describes a situation where a large disk size can be reduced by performing a disk size delta merge operation. Additionally a temporary working space is needed which is about the size of the largest column of the table (partition).g. 13. See SAP Note 2119087 for more details. you would need about 100 GB * 2 + 40 GB = 240 GB while the merge is executed. © 2017 SAP SE or an SAP affiliate company. All rights reserved . but also the disk size interferes with delta merges. degradations Not only the memory size. See SAP Note 2043152 for more information. as part of a migration activity) it is important to undo the changes after the activity has finished. Why can large delta storages be critical? Large delta storages can be critical for the following reasons: Problem Details Large delta storages increase the memory requirements because: Increased memory They are compressed less efficiently than the main storages consumption Not only UPDATE and INSERT. How can I trace the merge activities in order to analyze problems with merges? To find out details about merge activities you can increase the level of the database trace for "mergedog" and "mergemonitor" to INFO. In case you have a table of 100 GB with 40 GB being the size of largest column. but also DELETE operations result in a growing delta storage. For example it is normal that there are large delta storages during data loads in BW. If parameters are adjusted for a specific task (e. SAP Note 2039810 describes a scenario where auto merges are no longer executed after a table was renamed.2017-01-23 Page 10/22 Due to the complexity of the merge operation changes to the parameters can lead to unforeseen effects like increased system load. 15. Memory can only be reclaimed with a delta merge (see SAP Note 2088183) Longer recovery Large delta storages result in large delta logs which can increase the recovery times. higher memory consumption or performance degradations. because BW often suppresses auto merges and uses smart merges at the end. Manual In individual cases you can execute manual merges before the delta storage has grown to critical merges sizes. reduction of indexes. Disabled auto merges on table level can be a consequence of table moves from row store to column store using SAP ABAP functionality. reduction of CONCAT attributes. A typical default value is: UPT > 43200 and Indexserver uptime at least 43200 seconds (12 hours) ( ( MMS < 10000 and Main memory size < 10 GB DMS > 1000 and Delta memory size > 1 GB TMD > 86400 Table merge delay (time since last merge. See SAP Note 1999997 ("Which options exist to reduce the risk of SAP HANA memory footprint issues?") for more information Reduction Check from application side if you can reduce the amount of INSERT / UPDATE / DELETE of changes operations. 19. The following approaches exist to reduce the memory requirements during delta merges: Area Details Partition large tables so that smaller delta storages on partition level are used instead of a large Partitioning table-wide delta storage. 17.2017-01-23 Page 11/22 16. In which situations are critical merges executed? Critical merges are controlled by the critical_merge_decision_func parameter. Merge parameters In individual cases you can adjust merge parameters so that merges happen before the delta storage has grown to critical sizes. How can I check for which tables auto merges are disabled? You can use SQL: "HANA_Tables_ColumnStore_AutoMergeDisabled" (SAP Note 1969700) in order to check for tables with disabled auto merges. How can I check for which tables persistent merges are disabled? You can use SQL: "HANA_Tables_ColumnStore_PersistentMergeDisabled" (SAP Note 1969700) in order to check for tables with disabled persistent merges. Make sure that the merge related parameters are set to reasonable values (optimally default values) and avoid settings resulting in merges of large delta storages. Reduce the memory requirements of the table by actions like cleanup of technical tables. 18. All rights reserved . data aging and extended of memory storage. Reduction archiving. What can be done to limit the memory requirements during delta merges? The memory requirements during a merge depend both on the overall table size and the delta storage size. in seconds) ) © 2017 SAP SE or an SAP affiliate company. See SAP Note 2196615 for more information. in seconds) ) or TMD>604800 Table merge delay (time since last merge. the table delta size is greater than 1 GB and the table wasn't merged since at least one day The table wasn't merged since around 7 days So we can see that . The following TYPE values exist in M_DELTA_MERGE_STATISTICS: Type Merge Details FACT No BW fact table compression (BW_F_FACT_TABLE_COMPRESSION) Smart merge request triggered by application. Why do I see auto merges on tables although auto merges are disabled on table level? Up to SAP HANA Rev. 21. Therefore you can see a merge categorized as 'AUTO' when a critical merge was executed (e. 122 critical merges are recorded with MOTIVATION = 'CRITICAL' in M_DELTA_MERGE_STATISTICS.if merge is indicated . What do the shortcuts mean which are used in the decision and cost functions? The most important shortcuts used in decision and cost functions are: Shortcut Unit Details AHM MB Available heap memory BASENAME Table name (SAP HANA SPS >= 12) CLA % CPU load average © 2017 SAP SE or an SAP affiliate company.g. 121 critical merges are registered with MOTIVATION = 'AUTO' in views like M_DELTA_MERGE_STATISTICS and HOST_DELTA_MERGE_STATISTICS. All rights reserved . SPARSE No can also be triggered manually Not restricted to SPARSE compression.2017-01-23 Page 12/22 This means that critical merges are only executed when the indexserver runs already for at least 12 hour and additionally at least one of the following conditions is fulfilled: The table main size is smaller than 10 GB. because no successful merge happened within the last 7 days). so 'COMPRESS' would be a clearer name for this type 22.among others .a subsequent MERGE step is performed MERGE Yes Delta merge RECLAIM No Garbage collection in delta storage Compression optimization (see SAP Note 2112604) Typically executed as subsequent step of an auto merge. 20. it depends on the TYPE.an unconditioned merge of tables is performed when the last merge happened more than 7 days ago. smart merge or critical merge. What is the meaning of the different TYPE values in table M_DELTA_MERGE_STATISTICS? In fact not all entries in M_DELTA_MERGE_STATISTICS actually reflect merge operations. smart_merge_decision_func is evaluated HINT No and . Starting with SAP HANA Rev. < the first occurrence of <sub_string> in <full_string> is returned sub_string>) LCC Logical CPU count LOADED TRUE if table is loaded. < Returns <arg2> if <arg1> is fullfilled (i. true or not 0).2017-01-23 Page 13/22 CRCSOC million Changed row count since optimize compression DCC million Delta cell count (number of rows in delta multiplied with number of table columns) DLS MB Delta log size DMR million Deleted main rows DMS MB Delta memory size DRC million Delta row count (uncommitted rows are not always taken into account) DUC million Uncommitted delta row count Internal table name (SAP HANA SPS >= 12): FULLNAME Table partitioned: _SYS_SPLIT/<table_name>~<part_id> Table not partitioned: <table_name> GAL MB Global allocation limit IF(<arg1>. otherwise <arg3> arg2>. <arg3>) INSTR(< Returns 0 if <sub_string> isn't contained in <full_string>. otherwise FALSE MMS MB Main memory size MMU million Main max UDIV MRC million Main row count Internal table name: Table partitioned: _SYS_SPLIT/<table_name>~<part_id> Table not partitioned: <table_name> NAME Due to the special naming convention for partitioned tables you have to use BASENAME (SAP HANA SPS >= 12) or the following approach if you want to refer to a partitioned table: INSTR(NAME. All rights reserved .e. '<table_name>') > 0 NMU million Main max UDIV (>= SPS 09) NOW() Current date and time (localtime of the server timezone) OCRC million (Last) optimize compression row count PAL MB Process allocation limit QDW s Queuing delay wait (wait time for assignment of token) © 2017 SAP SE or an SAP affiliate company. otherwise the position of full_string>. 05*DRC>=DUC)' WITH RECONFIGURE Be aware that you should always take the default value for the auto merge decision function in the system as a basis and only add the additional term at the beginning. so that e. data loads (many INSERTs) or archiving (many DELETEs).2*MRC and DMR>0. Is the default auto merge decision function already optimal? The default auto merge decision function has changed a few times.g. The effect of reducing or disabling auto merges depends on several factors like the COMMIT frequency and the actual DML activities. Therefore a general rule of thumb is not available and you have to test the different options if you perform time-critical mass changes. BW relies on smart merges triggered on application side. 24. 'SYSTEM') SET ('mergedog'. In which scenarios can it be useful to reduce or disable auto merges? Disabling auto merges is mainly useful in scenarios of mass changes.1 or 0. otherwise false SCHEMA Schema name THM MB Total heap memory TMD s Time since end of last merge of table UPT s Uptime of indexserver 23. If you upgrade to a later SAP HANA revision.2017-01-23 Page 14/22 RHM MB Estimated required heap memory to complete table optimization RP boolean TRUE if table is range partitioned.g. because the threshold for delta storage sizes is 100 MB or higher. e. you © 2017 SAP SE or an SAP affiliate company. a small table with the same number of records in main and delta storage will typically be merged after 1 hour.ini'. Several standard functionalities already work in this way: In BW environments auto merges are explicitly disabled for certain table types in order to speed up operations like data loads.0001 )) or This formula compares the delta row count with the main row count and additionally takes the time since the last merge into account. an auto merge is typically not triggered for a table with 20 MB main storage and 90 MB delta storage. Regardless of these changes it tends to merge small tables with large delta storages too rarely with SPS 09 and below.'auto_merge_decision_func')= '((DRC*TMD>3600*(MRC+0. Example (Rev. 82): ALTER SYSTEM ALTER CONFIGURATION ('indexserver. All rights reserved . Furthermore it is a trade-off between performance and memory consumption. In order to eliminate this issue. Migration tools disable auto merges or set the auto_merge_decision_func to values that result in less auto merges.0001)) or (THM>=256000 and (((DMS>100 or DCC>100 or DLS> 1000) and DRC>MRC/100) or (DMR>0.001))) or (THM<256000 and (DMS>50 or DCC>8 or DLS>100))) and (DUC<0. As a consequence the memory allocation is higher than necessary and the performance can suffer. The fewer merges are done. For example. the larger is the memory requirement. you can adjust the auto_merge_decision_func parameter by adding the following condition after the first bracket in its value: (DRC * TMD > 3600 * ( MRC + 0. 1)) indexserver.010240 and (CRCSOC>=50 or if(OCRC!=0. so that the delta storage size exceeds 1 GB. CRCSOC/OCRC>0.1)) indexserver.1 Starting with SAP HANA SPS 11 this change is incorporated per default.04: indexserver.ini -> [mergedog] -> auto_merge_decision_func = ((DRC*TMD> 3600*(MRC+0. 120 to 122.ini -> [optimize_compression] -> critical_decision_func = UPT>43200 and TOCD>604800 and MMU>0. When moving the uncommitted changes to the new delta storage. All rights reserved . With SPS 10 and higher the default parameter value has been adjusted so that also small tables are properly considered. DML operations on the table are blocked (SAP Note 2098801). A bug with SAP HANA Rev. Example: Be aware that error details aren't available in earlier revisions. CRCSOC/OCRC> 0. Thus.2. For some time M_DELTA_MERGE_STATISTICS contained the ERROR_DESCRIPTION. you have to remove this special configuration when you uses SPS 10 or higher. Is the default critical merge decision function already optimal? Up to SAP HANA SPS 10 the default value of the critical_merge_decision_func parameter is: UPT>43200 and ((MMS<10000 and DMS>1000 and TMD>86400) or TMD>604800) When a table wasn't merged for more than one day and a high amount of changes are executed without COMMIT.2. 26.0001)) or ((DMS>PAL/2000 or DMS>1000 or DCC>100) and DRC>MRC/100) or (TMD>86400 and DMR>0. Therefore it is useful to adjust the critical_merge_decision_func parameter so that the critical merge is only executed when the amount of uncommitted changes is on a reasonably low level: UPT>43200 and ((MMS<10000 and DMS>1000 and TMD>86400) or TMD>604800) and DUC<0.2017-01-23 Page 15/22 should revisit this parameter setting and adjust it if the default has changed. What needs to be done in case of merge related errors? Delta merges can fail with various errors.2.ini -> [optimize_compression] -> auto_decision_func = MMU>0.04 can result in permanent merges of specific tables because UDIV gaps (that can only be eliminated with optimize compression) are erroneously considered as deleted main rows and so the decision function may indicate the need for a merge on a permanent basis. 120 to 122. You can check for the top error messages using SQL: "HANA_Tables_ColumnStore_Merges" (SAP Note 1969700). © 2017 SAP SE or an SAP affiliate company.ini -> [optimize_compression] -> smart_decision_func = MMU>0. As a workaround the following merge and optimize compression decision functions are suggested for Rev. and the details are written to columns LAST_ERROR and ERROR_DESCRIPTION in monitoring view M_DELTA_MERGE_STATISTICS and its history HOST_DELTA_MERGE_STATISTICS. This can take significant time in case of a high amount of uncommitted changes. but HOST_DELTA_MERGE_STATISTICS not.1)) indexserver.001)) 25. the critical merge will kick in. CRCSOC/OCRC>0.010240 and (CRCSOC>=50 or if(OCRC!=0.010240 and (CRCSOC>=50 or if(OCRC!=0.2*MRC and DMR>0. The "Description" of an error code depends on the SAP HANA patch level and so you find more than one description for some of the error codes.2017-01-23 Page 16/22 Typical errors and required actions are described below. The term "table optimization" in some of the error messages covers merges and compression optimizations. Error Description Recommendations This error typically indicates an inconsistency. 120) can be responsible. as cancel was manually requested by a kill session call © 2017 SAP SE or an SAP affiliate company. Check the indexserver trace file for more details.02. For SAP HANA Revisions <= 61 see SAP Note 1895207. If the problem happens on newer Revisions. This error can happen when merges are executed on temporal tables. OOM situations. Rev. See SAP Note 2009 Memory allocation failed 1999997 and make sure that sufficient memory is available. 110 . Error during merge of 2450 delta index occurred Don't execute memory merges on tables with LOB columns ("main memory merge not supported on tables containing disk lobs!").g. e. See trace file for more information If the error is reported for merges on secondary sites of system replication scenarios (SAP Note 1999880). The underlying meaning is nevertheless the same. see SAP Note 1999997 for more details. 1527 replay. general error during In case the history table is a SAP HANA-optimized DSO you can convert 2454 historisation of data this DSO to a optimized standard DSO (see SAP Note 1849497 and 1849498) without keeping change log. the bug described in SAP Note 2348480 (Rev. information available) This error is issued when no more memory is available. All rights reserved . General error (no further 1999 Check the indexserver trace file for more details. In case of memory problems. Open a SAP incident if you are not able to resolve the issue. aborted as cancel requested 2458 Table delta merge aborted If it happens repeatedly: Check why sessions are cancelled manually.112. check the trace files for content. Check the trace file for further details and proceed according to SAP Note 2116157 in order to exception during deltalog analyze and resolve the corruption. The following application SAP Notes can help to reduce the number of unnecessary smart merge request in specific scenarios: © 2017 SAP SE or an SAP affiliate company. optimizeCompression occurred This error is issued if during recovery a merge isn't able to proceed due to a lack of merge tokens.04 and 120 to 122. If you nevertheless want to reduce it. The table in question is already being merged. Normally this error isn't critical because a merge will be repeated at a later time when the current number of tokens is not sufficient. Less smart merges for classic BW DSO tables with RFC 2243673 calls 2245078 Reduction of smart merges in Bank Analyzer scenarios Usually this error is harmless and can be ignored (SAP Note 2250715). 110 to 112. Also with newer Revisions this error is still possible.2017-01-23 Page 17/22 2461 error during Check the trace file for more details about the underlying error. You can use SQL: "HANA_Tables_ColumnStore_Merges_TokenOwners" (SAP Note 1969700) to understand which activities occupy how many merge tokens. same table optimization The following application SAP Notes can help to reduce the number of requested multiple times unnecessary smart merge request in specific scenarios: 2481 There are already other SAP smart merge requests for Details Note this table in the queue. SAP Note 2351926 describes a bug that can result in these errors with SAP HANA Rev. you can increase the number of merge tokens as a not enough merge tokens 2465 workaround by adjusting indexserver. If it happens repeatedly: Check why concurrent smart merge requests are issued against the same table. All rights reserved .ini -> [mergedog] -> for delta merge load_balancing_func and reconfiguring the secondary site as described in SAP Note 2351926. same table optimization is ongoing If it happens repeatedly: Check why concurrent merge requests are 2480 issued against the same table. This typically happens in system replication environments (SAP Note 1999880) with a configured logreplay replication mode. but typically harmless.00. rolled back If it happens repeatedly: Check why merge operations are started table optimization did not although no row can be processed. (parameter smart_merge_enabled=no) not enough memory for table optimization Analyze and resolve memory shortages or misconfigurations (see SAP 2484 Memory required to Note 1999997 for more details). All rights reserved . SAP 2594 General partitioning error HANA bugs in the area of paged attributes (SAP Note 1871386) can be responsible for this error. Compression failed Internal error 6900 Check the indexserver trace file for more details. Check the indexserver trace file for more details. © 2017 SAP SE or an SAP affiliate company. Check the indexserver trace file for more details. smart_merge_decision_func is set properly (optimally: default value) and if smart merge logic on application side works correctly.ini -> [mergedog] -> Smart merge is not active smart_merge_enabled. optimize table exceeds heap limit table optimization was 2485 Check the indexserver trace file for more details. caught exception during This error can be introduced by a SAP HANA bug with SPS 12. Open a SAP metadata changes incident on component HAN-DB if your require assistance from SAP. See SAP 2490 merge of history table Note 2403758 for more information. table optimization was disabled Check why smart merges are disabled globally and activate them if 2483 possible by unsetting parameter indexserver.2017-01-23 Page 18/22 SAP Details table optimization was not Note 2482 indicated Less smart merges for classic BW DSO tables with RFC 2243673 The delta storage is empty calls or the evaluation of the 2245078 Reduction of smart merges in Bank Analyzer scenarios smart merge cost function indicated that a merge is In case of doubts you can check if the parameter not necessary. Perform a consistency table optimization was not check using CHECK_TABLE_CONSISTENCY in order to identify 2487 possible due to open possible underlying corruptions (see SAP Note 1977584). have an effect because all 2486 rows in delta were not This situation can happen if all records in the delta storage have to be optimizable moved to the new delta storage because they are still uncommitted. Among others. BW F Fact Table 2953 Check the indexserver trace file for more details. Open without calling prepare() a SAP incident on component HAN-DB if your require assistance from first SAP. File <file_id> is not opened or already closed This error can be returned if the underlying table is corrupted or has reached the 2 billion record limit.. OOM situations. In 6923 Attribute load failed case of memory problems. This error can be the consequence of a SAP HANA bug that can result in AttributeEngine: commit() an inconsistency between existing columns and metadata information. Perform a consistency check using 999999 message not found CHECK_TABLE_CONSISTENCY in order to identify possible underlying corruptions (see SAP Note 1977584). No restart is required. When an auto merge fails due to an error. or rollback() was called 6939 Upgrade to SAP HANA SPS 09 or higher in order to fix this issue. e. e. see SAP Note 1999997 for more details. Check the indexserver trace file for more details. ltt::exception caught while This error is related to the persistence layer. the mergedog waits for some time (typically 1 hour) before trying the same auto merge again. AttributeEngine: not This error is issued when no more memory is available. All rights reserved .2017-01-23 Page 19/22 Attribute engine failed Check the indexserver trace file for more details. In 6924 Attribute save failed case of memory problems. OOM situations. Is the delta storage empty after a merge? No. Check the indexserver trace file for more details. yet. Is a restart required when changing delta merge related parameters? Delta merge parameters can be changed online and immediately take effect. Uncommitted records remain in delta storage. Check the indexserver trace 29020 operating on file for more details. $STORAGEOBJECT$ Check the indexserver trace file for more details. 28.g. It is a good practice (also considered by the auto merge decision function) to avoid merges if a high fraction of records in delta storage is not committed. see SAP Note 1999997 for more details.. Changes performed during the delta merge will also appear in the new delta storage. not necessarily: Only committed data is moved from delta storage to main storage. See SAP Note 6952 enough memory 1999997 and make sure that sufficient memory is available. © 2017 SAP SE or an SAP affiliate company. Open a SAP incident on component HAN-DB if your require assistance from SAP.g. 27. Transaction canceled The following error after a CANCEL attempt of the merge session is 29063 during persistence harmless and caused by an inadequate error handling: operation file write error: exception 1: . 'SYSTEM') UNSET ('mergedog'.g scriptserver).'SYSTEM') RECONFIGURE SET ('mergedog'. 'critical_merge_decision_func') = CRITICAL 'INSTR(NAME. to wrong service no auto merge will happen. Auto merges Check via SQL: "HANA_Tables_ColumnStore_AutoMergeDisabled" (SAP Note 1969700) if deactivated on auto merges are disabled on table level and enable them if required.g. 30. How can I temporarily disable merges for a critical table? In some situations (e. All rights reserved . ALTER SYSTEM ALTER CONFIGURATION 'critical_merge_decision_func') W ('indexserver. table level Make sure that Mergedog indexserver. What are reasons why auto merges aren't executed as expected? If tables aren't automatically merged although the delta storage is filled significantly.ini'. ''<table_name>'') = 0 AND < If system specific value is used: original_critical_merge_decision_func_value ALTER SYSTEM ALTER CONFIGURATION >' ('indexserver.'SYSTEM') WITH RECONFIGURE © 2017 SAP SE or an SAP affiliate company. This can be achieved in the following way: Merge Deactivation command Activation command type ALTER TABLE "<table_name>" DISABLE ALTER TABLE "<table_name>" ENABLE AUTO AUTOMERGE AUTOMERGE If DEFAULT value is used: ALTER SYSTEM ALTER CONFIGURATION ('indexserver. The mergedog only considers tables related to the services with own persistent tables like Tables assigned indexserver and statisticsserver. you can check the following potential reasons: Reason Details Make sure that the auto_merge_decision_func parameter is default or set to a reasonable Inadequate auto value.ini'.ini'.2017-01-23 Page 20/22 29. in case of a corrupted table) it can be useful to disable both auto and critical merge for a specific table on a temporary basis and enable it again once the problem is fixed. because this would disable the mergedog and no more auto merges are performed. If tables are assigned to other services (e. merge decision function Check if the conditions of the tables in question fulfill the auto_merge_decision_func.ini -> [mergedog] -> active deactivated isn't explicitly switched off ('false'). 111 the view M_DELTA_MERGE_STATISTICS can also contain merge information from the secondary site of a system replication scenario (SAP Note 1999880). platform edition all versions This document refers to SAP Note/KBA Title 2250715 Column Store Error: [2482] table optimization was not indicated 2119087 How-To: Configuring SAP HANA Traces © 2017 SAP SE or an SAP affiliate company. This can in general be considered as harmless. This new behavior can result in some confusion.: Merge errors 2465 ("not enough merge tokens for delta merge") reported: See "What needs to be done in case of merge related errors?" for more information.g. 32. Certain conditions like identical SAP HANA patch level needs to be fulfilled for this feature. SAP Note 2240059 describes a statistics server bug that can result in false positive alerts if the mergedog is (correctly) disabled for the dpserver service and suggests to activate the mergedog for the dpserver as a temporary workaround. Long merge durations due to the fact that a merge on the secondary site needs to synchronize with the recovery queue and so it can't be committed immediately. e. All rights reserved . For which services is the mergedog required? The mergedog is only activated for services with column store tables: indexserver (standalone) statistics server For other services the mergedog is intentionally disabled. 'critical_merge_decision_func') = '< original_critical_merge_decision_ >' WITH RECONFIGURE 31.2017-01-23 Page 21/22 SET ('mergedog'. Keywords SAP HANA delta merge M_DELTA_MERGE_STATISTICS optimzable [sic] Products SAP HANA. Does M_DELTA_MERGE_STATISTICS also contain merge information from the secondary site of a system replication scenario? Starting with SAP HANA Rev. 2017-01-23 Page 22/22 2116157 FAQ: SAP HANA Consistency Checks and Corruptions 2114710 FAQ: SAP HANA Threads and Thread Samples 2112604 FAQ: SAP HANA Compression 2088971 How-To: Controlling the Amount of Records in SAP HANA Monitoring Views 2088183 Memory usage not decreasing after the deletion of entries in a table 1999997 FAQ: SAP HANA Memory 1999993 How-To: Interpreting SAP HANA Mini Check Results 1999880 FAQ: SAP HANA System Replication 1977314 How to handle HANA Alert 29: 'Size of delta storage of column-store tables' 1909742 How to handle HANA Alert 19: &#x2018.02 can lead to a disk full event 1977584 Technical Consistency Checks for SAP HANA Databases 1969700 SQL Statement Collection for SAP HANA 1895207 Delta merge of history tables fails with error 2454 1871386 SAP HANA: Paged Attributes 1849498 SAP HANA: Reconversion of SAP HANA-optimized DataStores 1849497 SAP HANA: Optimizing standard DataStore objects SAP HANA Administration Guide SAP HANA Troubleshooting and Performance Analysis Guide Terms of use | Copyright | Trademark | Legal Disclosure | Privacy © 2017 SAP SE or an SAP affiliate company.&#x2018. All rights reserved .Delta Merge (mergedog) Configuration&#x2019. 2403758 HANA DB: Merge failed .Caught exception during merge of history table 2351926 Many "Delta Merge was not executed successfully" Alerts For Secondary Site 2348480 Delta merges on secondary fail repeatedly 2245078 High number of smart merges on DSO table 2243673 DataStore objects (classic) for direct writing: Adjustment of smart merge behavior for RFC calls 2240059 SAP HANA DB: False Positive Alert 10 2196615 AUTO_MERGE_ON deactivated after change from row store to column store 2144274 R3load migration to SAP HANA database Revision 85.Record Count of Delta Storage of Column-Store Tables' 1909641 How to handle HANA Alert 10 .