2399996 HANA Cleanup

2017-11-17 Page 1/82399996 - How-To: Configuring automatic SAP HANA Cleanup with SAP HANACleaner Version 30 Type SAP Knowledge Base Article Language English Master Language English Release Status Released to Customer Category How To Component HAN-DB (SAP HANA Database) Released On 06.10.2017 Please find the original document at https://launchpad.support.sap.com/#/notes/2399996 Symptom You are interested in scheduling regular SAP HANA cleanup activities automatically. Environment SAP HANA Cause Certain SAP HANA cleanup tasks like purging the backup catalog or deleting old trace files (SAP Note 2119087 ) need to be implemented individually. SAP HANACleaner is now available to perform these tasks automatically SAP HANACleaner is implemented via Python script. This script is an expert tool designed by SAP support. You are allowed to use it, but SAP doesn't take over any responsibility for problems originating from the use of this tool. Resolution SAP HANACleaner can be used for the following cleanup tasks: Task SAP Note Cleanup of backup catalog entries 2096851 Cleanup of backups 1642148 Cleanup of trace files 2380176 Cleanup of backup.log and backint.log 1642148 Cleanup of audit logs 2159014 Cleanup of SAP HANA alerts 2147247 Cleanup of free log segments 2083715 Cleanup of internal events 2147247 Cleanup of multiple row store containers 2222277 Cleanup of data file fragmentation 1870858 Cleanup of SAP HANACleaner logs 2399996 Optimize compression of tables not compressed 2112604 © 2017 SAP SE or an SAP affiliate company. All rights reserved py") no actions are performed. displays backup catalog before and after the cleanup. decides Note: As mentioned in SAP Note 1812057 backup entries made via backint cannot be recovered.e.2017-11-17 Page 2/8 Optimize compression of tables with columns not compressed 2112604 Optimize compression of tables with large UDIV overhead 2112604 You can install SAP HANACleaner in the following way: Download the attached script hanacleaner. so you should either download the file directly to the database server or make sure that you use a file-based copy approach that doesn't modify the file content. complete data backups and data snapshots) entries in the catalog [days]. default: false © 2017 SAP SE or an SAP affiliate company.py Copy it to a directory on your SAP HANA database server Attention: Text-based "copy and paste" can result in unforeseen issues. i. use -be and -bd with care if you want to be able to recover from older data backups (it is possible to recover from a specific data backup without the backup catalog) -bb delete backups also [true/false]. i. default: false -bo output catalog [true/false]. this number of entries of data backups will remain in the backup catalog. default -1 (not used) Note: if both -be and -bd is used. the flag that removes the least number entries.e. The following command line options exist to adjust the behavior: ---. The following command provides you with an overview of the way how SAP HANACleaner works and the available configuration options: python hanacleaner. backups are deleted when the related backup catalog entries are deleted with BACKUP CATALOG DELETE BACKUP_ID <id> COMPLETE (see SQL reference for more info).e. complete data backups and data snapshots) entries in the catalog. the youngest successful data backup entry in the backup catalog that is older than this number of days is the oldest successful data backup entry not removed from the backup catalog. the most conservative. You always have to specify specific options that suit your needs.e. All rights reserved . all older log backup entries will also be removed with BACKUP CATALOG DELETE BACKUP_ID <id> (see SQL reference for more info) default: -1 (not used) -bd min retained days of data backup (i. "python hanacleaner. Once it is installed.BACKUP ENTRIES in BACKUP CATALOG (and possibly BACKUPS) ---- -be minimum retained number of data backup (i.e. you can start it.py --help When SAP HANACleaner is called without additional options (i. log and backint. that are older than this number of days are removed (except for the currently opened trace files). trace file content older than these number of days is removed from (almost) all trace files in all hosts (even currently opened tracefiles).log and backint.log or backint. default: -1 (not used) -ao output alerts [true/false]. backup. in all hosts. default: false ---. ---. default: false -zo print zipped backup logs.2017-11-17 Page 3/8 -br output removed catalog entries [true/false]. .log that were zipped.DUMP FILES ---- -dr retention days for dump files [days]. All rights reserved . displays backup catalog entries that were removed.log file (see -zp below) that is bigger than this size limit.trc. default: false ---. default: false -ad output deleted alerts [true/false]. specifies the path (and all subdirectories) where to look for the backup. please see -zb and -zp instead. manually created dump files (a.BACKUP LOGS ---- -zb backup logs compression size limit [mb]. displays trace files that were deleted. displays trace files before and after the cleanup. default is the directory specified by the alias cdtrace -zl zip links [true/false]. if there are any backup. displays statistics server alerts that were deleted. min age (today not included) of retained statistics server alerts. fullysytem dumps and runtime dumps) that are older than this number of days are removed. default: -1 (not used) -to output traces [true/false]. displays statistics server alerts before and after the cleanup.log files.log etc are taken into account.a.TRACE FILES ---- -tc retention days for trace file content [days]. default: -1 (not used) ---. default: -1 (not used) -tf retention days for trace files [days]. are excepted.OBJECT LOCKS ENTRIES with UNKOWN OBJECT NAME ---- © 2017 SAP SE or an SAP affiliate company. display the backup. trace files. default: -1 (not used) -zp zip path.log. default: false ---.k. then it is compressed and renamed.log and backint. default: false -td output deleted traces [true/false].ALERTS ---- -ar min retained alerts days [days]. specifies if symbolic links should be followed searching for backup logs in subdirectories of the directory defined by zp (or by alias cdtrace). only files with certain extensions like . default: false Note: Please do not use -bo and -br if your catalog is huge (>10000) entries. 2017-11-17 Page 4/8 -kr min retained unknown object lock days [days].LOG SEGMENTS ---- -lr max free logsegments per service [number logsegments]. minimum retained days for events. this is done for all hosts. default: -1 (not used) -fo output fragmentation [true/false]. displays how much memory was cleaned up from object history table. min age (today not included) of retained object lock entries with unknown object name. events that are older are removed by first being handled and acknowledged and then deleted. default: false ---. this is done for all hosts. of any service. default: -1 (not used) ---. default: false Note: Unfortunately there is NO nice way to give privileges to the DB User to be allowed to do this. displays data volume statistics before and after defragmentation. handled events that are older are removed by first being acknowledged and then deleted. default: -1 (not used) -oo output cleaned memory from object table [true/false]. minimum retained days for the handled events. in accordance with SAP Note 2147247. if more free logsegments exist for a service the statement ALTER SYSTEM RECLAIM LOG is executed. if the table _SYS_REPO.AUDIT LOG ---- -ur retention days for audit log table [days]. audit log content older than these number of days is removed.OBJECT_HISTORY is bigger than this threshold this table will be cleaned up according to SAP Note 2479702. switch to clean up multiple row store table containers.OBJECT HISTORY ---- -om object history table max size [mb]. default: -1 (not used) ---. default: -1 (not used) ---.DATA VOLUMES FRAGMENTATION ---- -fl fragmentation limit [%]. All rights reserved . before defragmentation of that service is started: ALTER SYSTEM RECLAIM DATAVOLUME '<host>:<port>' 120 DEFRAGMENT.MULTIPLE ROW STORE TABLE CONTAINERS ---- -rc row store containers cleanup [true/false]. default: -1 (not used) ---. default: false ---. maximum fragmentation of data volume files. Either you can run hanacleaner as SYSTEM user (NOT recommended) or grant DATA ADMIN to the user (NOT recommended) © 2017 SAP SE or an SAP affiliate company. default: -1 (not used) -eu min retained days for unhandled events [day].EVENTS ---- -eh min retained days for handled events [day]. i. All rights reserved . 100000 Following three flags are general. -cc. if a column has more then this number UDIVs --> compress if -cq. and -ce.. 4. last_compressed_record_count = 0): -cc max allowed raw main records.. -cs. switch to print all tables that were compression re-optimized. 500 -cd min allowed distinct count [%]. if a column is larger --> compress if -cr&-cd.COMPRESSION OPTIMIZATION ---- 1.02) or PREFIXED then this table should be compression re-optimized.2017-11-17 Page 5/8 -ro output row containers [true/false]. if estimated size is larger --> compress if -cc.e. default: false -co output compressed tables [true/false]. default: -1 (not used) e. Both following two flags.g. switch to consider flags above per partition instead of per column. -cr. must be > 0 to control the force compression optimization on tables whose UDIV quota is too large. 10000000 4. 1. Both following two flags. default -1 (not used) e.g.g.g. switch to perform a delta merge on the tables before compression. if table has more raw main rows --> compress if -ce.g. if a column has less distinct quota --> compress if -cr&-cs. default: -1 (not used) e.g. Flag -cb must be > 0 to control the force compression optimization on tables with columns with SPARSE (<122. All following three flags.. 10000000 -cs max allowed size [MB]. default: false -cm merge before compress [true/false]. 2. default: false ---.e. compression optimization possibilities above -cp per partition [true/false]. default: false ---. default -1 (not used) e.g. they control all three. #UDIVs/(#raw main + #raw delta) -cq max allowed UDIV quota [%]. if a column has more rows --> compress if -cs&-cd. must be > 0 to control the force compression optimization on tables that never was compression re-optimized (i. no additional compression algorithm in main) -cr max allowed rows. 3. default: -1 (not used) e.. 1 2. -cu and -cq.02) or PREFIXED and a BLOCK index -cb max allowed rows.g. default: -1 (not used) e. default: -1 (not used) e. 5 3. if the table has larger UDIV quota --> compress if -cu. default: -1 (not used) e. must be > 0 to control the force compression optimization on tables with columns with compression type 'DEFAULT' (i.e. displays row store tables with more than one container before cleanup. 150 -cu max allowed UDIVs. if a column has more rows and a BLOCK index and SPARSE (< 122.INTERVALL ---- © 2017 SAP SE or an SAP affiliate company. and -cd. 10000000 -ce max allowed estimated size [GB]. TENANT1KEY. default: '' ---. makes it possible to use SAP HANA Cleaner despite SSL. default: false -op output path. prints all crucial housekeeping tasks (useful for debugging with -es=false).: SYSTEMKEY.SERVER FULL CHECK ---- -fs file system.INPUT ---- -ff flag file. full path of the folder for the output logs (if not exists it will be created).2017-11-17 Page 6/8 -hci hana cleaner interval [days].EXECUTE ---- -es execute sql [true/false].USER KEY ---- -k DB user key. default: SYSTEMKEY It could also be a list of comma seperated userkeys (useful in MDC environments). default: blank. path to server to check for disk full situation before hanacleaner runs. execute all crucial housekeeping tasks (useful to turn off for investigation with -os=true). default = "" (not used) -or output retention days. number days that hanacleaner waits before it restarts. df -h is used Could also be used to specify a couple of servers with e. default: true ---.g. this one has to be maintained in hdbuserstore. default: -1 (exits after 1 cycle) NOTE: Do NOT use if you run hanacleaner in a cron job! ---. full path to a file that contains input flags. this flag makes it possible to ignore some filesystems. each flag in a new line.TENANT2KEY The following table lists some examples how to call SAP HANACleaner for different purposes: Command Details python hanacleaner.OUTPUT ---- -os output sql [true/false]. All rights reserved .SSL ---- -ssl turns on ssl certificate [true/false]. default: '' (not used) ---. from the -df h command. with comma seperated list. default: -1 (not used) -so standard out switch [true/false]. e.py No execution of actions ("hanacleaner needs input arguments") © 2017 SAP SE or an SAP affiliate company. -fs "|grep sapmnt" -if ignore filesystems. if this flag is used no other flags should be given. before hanacleaner starts it checks that there is no disk full situation in any of the filesystems.e. all lines in the file that does not start with a flag are considered comments. logs in the path specified with -op are only saved for this number of days.g. switch to write to standard out. i. default: false ---.e. i. default: true ---. as <sid>adm do > hdbuserstore SET <DB USER KEY><ENV><USERNAME><PASSWORD> . pdf. Keywords HANA housekeeping cleanup HANACleaner Products SAP HANA. handle / acknowledge events after 2 / 42 days More details and examples can be found in the attached presentation hanacleaner_intro.py -tc Clean up statistics server alerts.2017-11-17 Page 7/8 python hanacleaner. platform edition all versions This document refers to SAP Note/KBA Title 2380176 FAQ: SAP HANA Database Trace 2222277 FAQ: SAP HANA Column Store and Row Store 2159014 FAQ: SAP HANA Security 2147247 FAQ: SAP HANA Statistics Server 2119087 How-To: Configuring SAP HANA Traces 2112604 FAQ: SAP HANA Compression 2096851 Backup Catalog Housekeeping within HANA DB 1870858 HANA Alerts related to file system utilization 2083715 Analyzing log volume full situations 1642148 FAQ: SAP HANA Database Backup & Recovery Attachments © 2017 SAP SE or an SAP affiliate company.py Clean up backup catalog entries and backups that are older than 30 days and that -be 10 -bd 30 -td true don't be long to the ten newest backups python hanacleaner.log when size exceeds 50 50 -eh 2 -eu 42 MB. traces and backup catalog entries older than 42 42 -tf 42 -ar 42 -bd 42 -zb days. rename and compress backup. For feedback and suggestions you can contact Christian Hansen.log and backint. All rights reserved . pdf 1680 application/pdf hanacleaner.py 102 text/plain Terms of use | Copyright | Trademark | Legal Disclosure | Privacy © 2017 SAP SE or an SAP affiliate company.2017-11-17 Page 8/8 File Name File Size Mime Type hanacleaner_intro. All rights reserved .