BRSPACE Online Reorganizations

BRSPACE Online ReorganizationsTop Ten Pitfalls and Problems Martin Frauendorfer, SAP Active Global Support [email protected] November 2009 Agenda 1. Overview 2. Pitfalls and Problems © SAP 2009 / Page 2 EXPDP / IMPDP. Several types of reorganization exist: Index reorganization (e. …) – SAP functionality (ICNV. SE14. export / import) – Oracle tools (DBMS_REDEFINITION. © SAP 2009 / Page 3 . MOVE.Overview (1) Reorganizations in general A reorganization in a database context is the recreation or cleanup of one or several segments. mainly used to reduce index fragmentation Table reorganization with tools like: – BRSPACE (online. offline. via REBUILD or COALESCE). EXP / IMP.g. …) Tablespace reorganization (reorganization of all tables in the tablespace and recreation of the tablespace) In this presentation we concentrate on the BRSPACE online reorganization of tables (which also includes tablespace reorganizations). Overview (2) In which situations can a table reorganization be useful? Reduction of table size Cleanup of table fragmentation Change of table storage parameters (e. INITRANS) Move of table to a different tablespace Reduction of chained and migrated rows Reduction of number of allocated extents Transition from LONG to LOB columns (Oracle >= 10g) Reduction of hot spots on disk level Activation of Transparent Data Encryption Change of table structure © SAP 2009 / Page 4 .g. Overview (3) In which situations can a tablespace reorganization be useful? Activation of LMTS Activation of ASSM Cleanup of freespace block corruption Reduction of database size Reduction of backup size Reduction of filesystem size Reduction of fragmentation Reduction of number of datafiles Increase of number of datafiles Transition to new tablespace layout Other change to tablespace layout Optimization of filesystem layout © SAP 2009 / Page 5 . Creation of target indexes with naming convention <source_index>#$ Import of source table CBO statistics to target table based on DBMS_STATS. The „#$“ suffix of the target table indexes is removed.GET_DDL Creation of target table with naming convention <source_table>#$ Export of source table CBO statistics based on DBMS_STATS.Overview (4) How does a BRSPACE online reorganization work? Determination of CREATE commands for target table and indexes based on DBMS_METADATA.IMPORT_TABLE_STATS Finalizing of the online reorganization using DBMS_REDEFINITION. © SAP 2009 / Page 6 .FINISH_REDEF_TABLE. In this step all changes in the materialized log are applied to the target table and the names of source and target table are exchanged. The source table is dropped. This package will implicitly create a materialized view log MLOG$_<source_table> to track all changes that are performed on the source table while the reorganization is running.START_REDEF_TABLE.EXPORT_TABLE_STATS Copy of source table data to target table based on DBMS_REDEFINITION. © SAP 2009 / Page 7 . It performs a lot of useful activities implicitly that would require a lot of manual efforts and impose the risk of errors if DBMS_REDEFINITION is used directly.Overview (5) What are the advantages of a BRSPACE Online Reorganization? It uses the advantages of DBMS_REDEFINITION: The reorganization can be performed in parallel to production operation. It has already proven to be a reliable reorganization tool at many customers. If anything goes wrong during the reorganization the source table still exists unchanged and no restore / recovery is necessary. Compared to an offline reorganization via export / import all data has to be copied only once and not twice It provides possibilities for client and server side parallelism in order to speed up the reorganization if sufficient system resources exist and the reorganization should finish within a specific time window. This means the table is accessible and records can be inserted / updated / deleted. 40 is used). As of 10g it is possible to convert them during an online reorganization on the fly to LOB columns (if SAP >= 6. It cannot handle tables with LONG or LONG RAW columns.Overview (6) What are the restrictions of a BRSPACE Online Reorganization? It is only available as of Oracle 9i. Temporarily twice the space is needed because both source and target table and indexes exist in parallel. ROWID column). The setup of the materialized view is an overhead that can negatively impact the performance of the reorganization of many very small tables. If a table has no primary key constraint (or no unique index with NOT NULL columns) a ROWID based online reorganization is necessary that requires overhead (ROWID index. Structure changes on the source table and its indexes must not be performed during the reorganization (as they would be lost). © SAP 2009 / Page 8 . g.Overview (7) How is a normal BRSPACE Online Reorganization started? Table reorganization: brspace -f tbreorg -t <table_name> Tablespace reorganization: brspace -f tbreorg -s <source_tsp> -t "*" -n <target_tsp> Additional useful options exist.: -p <degree> BRSPACE parallelism degree -e <degree> PX parallelism degree -l <category> Initial extent size category -i <target_ind_tsp> Definition of target index tablespace (if different from target table tablespace) For more detailed information see note 646681 and the BRSPACE online documentation. e. © SAP 2009 / Page 9 . 2. What is a good parameter and redo log configuration? What has to be considered for PSAPTEMP? What about PSAPUNDO? And why is the DEFAULT tablespace sometimes used significantly? Why do tables sometimes grow during reorganization? What about SAP technical settings? Is it useful to convert from LONG to LOB? What is a good way of parallelism? Why should I look for invalid objects? Why does a DROP sometimes take ages? © SAP 2009 / Page 10 . 7. 3. 6. 4.Overview Top Pitfalls and Problems 1. 10. 5. 8. 9.Pitfalls and Problems . Database Configuration A good online reorganization performance needs a slightly different database configuration than normal system operation.Pitfalls and Problems .1. PROCESSES: Sufficiently large to cope with the increased PARALLEL_MAX_SERVERS value DB_CACHE_SIZE >= 1 GB Large buffer pool doesn't provide significant advantage © SAP 2009 / Page 11 . Configuration 1. Oracle parameters: DB_FILE_MULTIBLOCK_READ_COUNT >= 128 Fast full table scan on the source table when creating the target table PGA_AGGREGATE_TARGET as large as possible Fast index creations in PGA with minimum PSAPTEMP accesses If Oracle parallel execution is used: – PARALLEL_EXECUTION_MESSAGE_SIZE >= 16384 – PARALLEL_MAX_SERVERS >= 2 * BRSPACE parallelism degree * Oracle parallelism degree – PARALLEL_ADAPTIVE_MULTI_USER = FALSE Otherwise Oracle may reduce the desired parallelism degree – SESSIONS. because in case of recovery across the reorganization time the table data is lost! Only useful during a dedicated reorganization time window and with a good backup after the reorganization Redo logs should be created large enough to avoid log switches within less than one minute Avoids unnecessary checkpoint and log switch load ARCHIVELOG mode can be disabled in case of a dedicated reorganization time window and a good backup after the reorganization. © SAP 2009 / Page 12 . Almost no redo log information created at all Dangerous. Configuration 1.Pitfalls and Problems .1. Database Configuration (2) Redo Log Configuration Usually all changes of an online reorganization are recorded in the redo logs. so there can be a significant amount of additional redo logs generated With current BRSPACE patches you can use the option "-NBR" to create the target indexes with NOLOGGING No redo log information is created for the indexes Theoretically it is possible to switch off logging for the target table. too. PSAPTEMP recommendations: Define PSAPTEMP as LMTS/T tablespace (see SAP Note 659946) in order to avoid delays due to space transactions Make sure that PSAPTEMP is assigned as temporary tablespace to the SAP user and to SYS. PSAPTEMP 2. If BRSPACE parallelism is used ("-p <degree>" option). PSAPTEMP must be large enough to hold the table data. In case of a sorted reorganization ("-r <index>" option). PSAPTEMP must be large enough to hold the size of the largest reorganized index. PSAPTEMP must be large enough for the <degree> largest indexes that are reorganized in parallel. Make sure that there are no I/O bottlenecks when accessing PSAPTEMP.2. PSAPTEMP Problems with PSAPTEMP can result in a long reorganization runtime or terminations.Pitfalls and Problems . Consider AUTOEXTEND or RESUMABLE to be on the save side © SAP 2009 / Page 13 . Concurrent DML operations logged in the MLOG$ table and finally applied to the target table generate undo. PSAPUNDO recommendations: Use Automatic Undo Management (AUM. PSAPUNDO PSAPUNDO is only used to a minor extent: The data transfer into the target table works with APPEND.Pitfalls and Problems . © SAP 2009 / Page 14 . SAP Note 600141) Usually no increase of PSAPUNDO necessary Larger PSAPUNDO may be required if reorganization is done in parallel to significant DML operations (in general not recommended).3. so no undo is generated. PSAPUNDO 3. The creation of the indexes also doesn't create undo. 4. Default tablespace recommendations: Make sure that a default tablespace with sufficient free space or AUTOEXTEND is assigned to both the SAP user and SYS. Default Tablespace The default tablespace is used in two situations: The MLOG$ table with the recorded DML operations is created in the default tablespace.. © SAP 2009 / Page 15 . Default Tablespace 4. Use LMTS or make sure that NEXT and MAXEXTENTS is set sufficiently high in order to avoid max extents errors. a ROWID index (I_SNAP$.Pitfalls and Problems . If a table has no primary index..) is created in the default tablespace. g. MOVE based reorganization of BW fact tables) A large INITIAL_EXTENT value is used – E. Table Growth 5. because of a former SAPDBA reorg with Compress Extents – Use BRSPACE option "-l 2" to set the initial extent to 64K. cleanup of QCM tables.Pitfalls and Problems . Table Growth Normally a reorganization will reduce the table size or at least keep it on the same level. This column is hidden after the reorganization. but it still allocates space ( table increase up to 30 %) – Typically happens with SAP conversion tables (QCM tables) and BW fact tables – Should be avoided whenever possible (e. © SAP 2009 / Page 16 .g. Sometimes the size reduction is not as high as expected or there is even a size increase The most typical reasons are: The reorganized table has no primary index.5. – In this case a ROWID based reorganization rather than a PK constraint reorganization is done and a ROWID column has to be added to the target table. during a transport of a new table or new index or during a system copy).Pitfalls and Problems . Otherwise segments may be created in wrong tablespaces. Technical Settings 6. you should consider the TABART settings. If an existing table or index is modified. © SAP 2009 / Page 17 . Technical Settings The SAP technical settings contain a data class (TABART) that (TABART) maps a table to a tablespace: Table DBSTATC DD09L TABART SPROT TAORA / IAORA Tablespace PSAPPROTD The TABART is used whenever a table or index is newly created (e.6. BRSPACE is TABART aware.g. the current tablespace is used regardless of the TABART. Whenever you move tables and indexes to different tablespaces. Technical Settings (2) Recommendations: Use the "-l" option to define an appropriate existing or new TABART whenever creating a new tablespace using "brspace -f tscreate": – "-l <tabart>": New tabart <tabart> – "-l all": All standard TABARTs – "-l <source_tsp>": TABART of tablespace <source_tsp> Maintain TABARTs also in the other systems of the system landscape. RSTS. RSDS and RSDCHABAS. In BW TABARTs are also maintained in special BW tables like RSDCUBE. © SAP 2009 / Page 18 .Pitfalls and Problems . Technical Settings 6. See SAP Note 771191 for more information. These entries also have to be adapted. RSDODSO. Follow SAP Notes 778784 and 777615 in order to avoid problems during system copies.6. TABART can point to different tablespaces in different systems. LONG2LOB 7.unless an implicit LONG2LOB conversion is performed (BRSPACE parameter "-a long2lob") In general it is useful to convert to LOB columns. 1226314 and 1171043) Small performance impact due to LOBs reported (see SAP Note 835552).Pitfalls and Problems . but there are some restrictions: Oracle >= 10g. 1257979. can usually be neglected © SAP 2009 / Page 19 .1x (for bugfixes see SAP Notes 1329293. 1269704.7. SAP >= 6.40 ORA-00932 errors possible on SAP side (for bugfix see SAP Note 1223716) ORA-00600 errors possible if PX is used (for bugfix see SAP Note 1292525) High water mark enqueues possible with LOBs in ASSM (for bugfix see SAP Note 1166242). LONG2LOB Conversion Tables with LONG and LONG RAW columns can't be reorganized online . Several LOB related DBSL problems in SAP 7. Often I/O is the bottleneck and not CPU. Parallelism Settings Two possibilities of parallelism: "-p <degree>": BRSPACE parallelism degree. reorganization of multiple different tables in parallel "-e <degree>": Oracle parallel execution (PX) degree.g. several Oracle PX slaves work on the reorganization of one table in parallel Reorganization in parallel to production operation System resources (I/O. CPU) as good as possible.8.Pitfalls and Problems . "-p 6 -e 4" for tablespace reorganizations or "-e 8" for individual table reorganization © SAP 2009 / Page 20 .g. Parallelism 8. Time is less critical Use no or a rather small parallelism setting (e. Starting point can be e.e 2") Reorganization in a dedicated time window with low or no parallel production operation Use the system resources (I/O. CPU) mustn't be exceeded. but I/O capacities are often hard to determine. "-p 2 . Perform a rebuild if required. DBMS_STATS and underlying objects. Invalid objects Different kinds of invalid objects can result in errors and terminations during online reorganizations.g. duplicate keys found“) errors when creating the unique index on the target table and the whole table reorganization is terminated. Invalid Objects 9. Make sure that no invalid triggers exist on the tables to be reorganized (e.Pitfalls and Problems . Make sure that the source tables are consistent (SAP Note 23345). Drop or validate these triggers. Make sure that no SYS objects exist in the Oracle DDIC with status = INVALID. /BIC/05… BW triggers). © SAP 2009 / Page 21 .9. DBMS_METADATA. Therefore the following should be checked in advance: Make sure that no index or index partition of the tables to be reorganized is in status UNUSABLE. Particularly important are packages like DBMS_REDEFINITION. As an example the REBUILD ONLINE bug described in SAP Note 1413928 can result in ORA-01452 („cannot CREATE UNIQUE INDEX. © SAP 2009 / Page 22 . DROP Performance Usually it is a matter of seconds (or less) to drop a table or an index. But: Under certain circumstances dropping the source table (or the source tablespace) can take longer than the whole reorganization.10. Even worse: As space transactions require the unique space transaction (ST) enqueue. other extent operations in the system may be blocked for a long time. In real life we have already seen situations where a DROP of a single table took several days! The reason are typically space transactions in dictionary managed tablespaces . Due to an inadequate Oracle index design on the DDIC tables FET$ and UET$ the cleanup has a square complexity (4 times higher runtime for twice the number of extents).Pitfalls and Problems .every formerly allocated extent needs to be freed individually. DROP Performance 10. g. by copying and truncating the table with REUSE SPACE and doing ALTER TABLE … DEALLOCATE UNUSED KEEP <size> with <size> decreasing over time) – Reorganize the critical tables individually during certain defined timeframes. More than 100.Pitfalls and Problems . DROP Performance (2) Recommendations Check for critical segments in dictionary managed tablespaces with more than 5000 extents. the more critical. If segments are returned.000 extents is definitly a big problem. © SAP 2009 / Page 23 . DROP Performance 10. – Test how long the DROP takes and provide a sufficient time window for the reorganization in production.10. you have several options: – Reduce the number of extents in smaller portions in advance (e. See SAP Note 745639 (section TYPE = ST) for more information. The higher the number of extents. – In extreme situations it can even make sense to recreate the database with LMTS using R3load. Thank you! © SAP 2009 / Page 24 . die Eigentum anderer Softwarehersteller sind. aber nicht ausschließlich. Einige von der SAP AG und deren Vertriebspartnern vertriebene Softwareprodukte können Softwarekomponenten umfassen. Die in diesem Dokument enthaltenen Informationen sind Eigentum von SAP. text. ByDesign. including but not limited to the implied warranties of merchantability.com. SAP has no control over the information that you may access through the use of hot links contained in these materials and does not endorse your use of third-party Web pages nor provide any warranty whatsoever relating to third-party Web pages Weitergabe und Vervielfältigung dieser Publikation oder von Teilen daraus sind. or other items contained within this material. indirekte oder Folgeschäden im Zusammenhang mit der Verwendung dieser Unterlagen. zu welchem Zweck und in welcher Form auch immer. R/3. SAP. or non-infringement. graphics. National product specifications may vary. SAP does not warrant the accuracy or completeness of the information. spezielle.. SAP übernimmt keine Haftung für Schäden jeglicher Art. auf die Sie möglicherweise über die in diesem Material enthaltenen Hotlinks zugreifen. Produkte können länderspezifische Unterschiede aufweisen. xApp. und SAP unterstützt nicht die Nutzung von Internetseiten Dritter durch Sie und gibt keinerlei Gewährleistungen oder Zusagen über Internetseiten Dritter ab. Dies gilt u. xApps. SAP shall have no liability for damages of any kind including without limitation direct. PartnerEdge und andere in diesem Dokument erwähnte SAP-Produkte und Services sowie die dazugehörigen Logos sind Marken oder eingetragene Marken der SAP AG in Deutschland und in mehreren anderen Ländern weltweit.Copyright 2008 SAP AG All rights reserved No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP AG. In dieser Publikation enthaltene Informationen können ohne vorherige Ankündigung geändert werden. and/or development. Diese Publikation wird ohne jegliche Gewähr. Dieses Dokument ist eine Vorabversion und unterliegt nicht Ihrer Lizenzvereinbarung oder einer anderen Vereinbarung mit SAP. xApp. eine Produktstrategie bzw. The information contained herein may be changed without prior notice. Dieses Dokument enthält nur vorgesehene Strategien. Die gesetzliche Haftung bei Personenschäden oder die Produkthaftung bleibt unberührt. Alle Rechte vorbehalten. Texte. The statutory liability for personal injury and defective products is not affected. All other product and service names mentioned and associated logos displayed are the trademarks of their respective companies. mySAP. Alle anderen in diesem Dokument erwähnten Namen von Produkten und Services sowie die damit verbundenen Firmenlogos sind Marken der jeweiligen Unternehmen. Duet. unterliegen nicht dem Einfluss von SAP. links.com. Business ByDesign. Diese Einschränkung gilt nicht bei Vorsatz oder grober Fahrlässigkeit. This document contains only intended strategies. Data contained in this document serves informational purposes only. This document is provided without a warranty of any kind. mySAP. weder ausdrücklich noch stillschweigend. special. Die Informationen. indirect. Links oder anderer in diesen Materialien enthaltenen Elemente. developments. R/3. SAP NetWeaver. SAP. Business ByDesign. a. xApps. hinsichtlich der Gewährleistung der Marktgängigkeit und der Eignung für einen bestimmten Zweck sowie für die Gewährleistung der Nichtverletzung geltenden Rechts. Entwicklungen und Funktionen des SAP®-Produkts und ist für SAP nicht bindend. The information in this document is proprietary to SAP. -entwicklung einzuschlagen. Some software products marketed by SAP AG and its distributors contain proprietary software components of other software vendors. product strategy. Grafiken. SAP übernimmt keine Verantwortung für Fehler oder Auslassungen in diesen Materialien. This limitation shall not apply in cases of intent or gross negligence. fitness for a particular purpose. einschließlich und ohne Einschränkung für direkte. either express or implied. einen bestimmten Geschäftsweg. ohne die ausdrückliche schriftliche Genehmigung durch SAP AG nicht gestattet. SAP garantiert nicht die Richtigkeit oder Vollständigkeit der Informationen. mySAP. or consequential damages that may result from the use of these materials. and functionalities of the SAP® product and is not intended to be binding upon SAP to any particular course of business. mySAP. SAP NetWeaver. Duet. Die Angaben im Text sind unverbindlich und dienen lediglich zu Informationszwecken. bereitgestellt. This document is a preliminary version and not subject to your license agreement or any other agreement with SAP. SAP assumes no responsibility for errors or omissions in this document. © SAP 2009 / Page 25 . PartnerEdge and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP AG in Germany and in several other countries all over the world. ByDesign.