SAP on RAC UpdateKurt Broeg Principal Member of Technical Staff Jan Klokkers Senior Director SAP Development Server Technologies Oracle Corporation Topics Introduction Certification Status Mandatory SAP Requirements for RAC SAP Certified Technology Stack for RAC 9.2 Necessary Technology Stack for SAP with RAC 10.2 Introduction of Net Services SAPCTL Introduction More than 18 years of SAP – Oracle cooperation More than 12,000 SAP customers use Oracle 2 out of 3 SAP installations run on Oracle Database Joint SAP & Oracle development/platform team in Walldorf Joint support teams in Rot (Germany) & US SAP & Oracle Partner/Presales teams in Walldorf Certification Status – Supported Platforms of SAP for Oracle IBM AIX5L HP-UX PA-RISC HP-UX IA64 HP Tru64 Linux x86 Linux x64 Linux IA64 Linux Power Solaris Sparc-64 Solaris x64 (AMD64) Windows x86 Windows x64 Windows IA64 1.2 availability planned for Q4CY2006 SAP versions 4. ERP2004 Only Single Instance Only for SAP ERP2005 Single Instance available RAC 10.7.6C.7.6C. 4.2.0.x – – Single Instance and RAC SAP versions 4.x (x>=2) – – – .Certification Status – Supported Oracle Versions Oracle 9. ERP2004 and ERP2005 Oracle 10.x (x>=4) – – Oracle 10.2.0. 4. sapbackup. sapcheck .Mandatory SAP Requirements for RAC Shared Oracle Home – No support for local Oracle Homes No support for Raw Devices No support for ASM (planned for Oracle11) Database on Shared Filesystem – – Shared Archive Destination on Shared Filesystem Shared SAP Directories on Shared Filesystem – saparch. saptrace. sapreorg. 1 HACMP 4.23 5.5/5. 9.2 Configurations for SAP OS Version HP-UX HP Tru64 IBM AIX Linux Linux Solaris (Fujitsu Siemens) Solaris (SUN) Windows 11.2/2.3 Oracle Clusterware and OCFS 1.1+ QFS or or Oracle Clusterware + OCFS 9.Certified RAC 9.14 or NetApp NFS Oracle Clusterware and OCFS2 1. 9. RHEL3 SLES9. 10 2003 Enterprise Server Cluster Version and CFS Type ServiceGuard with SGCFS TruCluster Server 5.2. RHEL4 8. 10 8.0.2/3 SLES8.0/5 SUN Cluster 3.7/8 .1 + GPFS 2.2.0.1b 5.0/5 PrimeCluster + Net App Filer Veritas Foundation Suite 4.x or NetApp NFS Veritas Foundation Suite 4. 1+ QFS or Oracle Clusterware + NetApp NFS Oracle Clusterware + OCFS 10.x or NetApp NFS Veritas Foundation Suite 4.2 or 9. RHEL4 9.0/5 or SUN Cluster 3.23 5.0/5 or PrimeCluster + Net App Filer or Oracle Clusterware + NetApp NFS Veritas Foundation Suite 4.1b 5.2 Configurations SAP OS Version HP-UX HP Tru64 IBM AIX Linux Solaris (Fujitsu Siemens) Solaris (SUN) 11. 10 Windows 2003 Enterprise Server .2.Required RAC 10.3 Oracle Clusterware and OCFS2 1.1 HACMP 5L + GPFS 2. 10 Cluster Version and CFS Type ServiceGuard with SGCFS TruCluster Server 5.3 Oracle Clusterware + GPFS 2.2/3 SLES9. Q U E S T I O N S A N S W E R S . Providing High Availability for SAP Resources . systems. Oracle provides a tool sapctl which uses the capabilities of Oracle Clusterware to support the SAP Replicated Enqueue feature. Especially for SAP. A key new feature for Oracle Database 10g Release 2 – Oracle Clusterware is now made available for customers and partners to protect 3rd Party Applications. This presentation will cover the implementation and use of sapctl in SAP HA . .Abstract Oracle Clusterware is the component that’s responsible for restarting the Oracle Real Application Cluster Instance and Listeners on process failure & relocating Database Virtual IP’s on Node failure. The SAP Enqueue Service must be available whenever an SAP installation uses more than one SAP R/3 instance. Whenever the active node fails or is shut down. the VIP and the Enqueue Service fail over to the next node. this copy is kept in a shared memory segment during runtime. SAP uses an active/passive approach to enable high availability for the Enqueue Service.Concept of SAP Enqueue Replication The SAP Enqueue Service provides distributed lock management for SAP R/3 instances. The Replication Service maintains a copy of the Enqueue Service´s lock table. . The node that supports the Enqueue Service also hosts a virtual IP address (VIP) for communication. There is exactly one Enqueue Service running in the SAP system. The Replication Service run on the node on which the Enqueue Service runs in the event of a failure or planned outage. Normal Operation.Control flow within SAPCTL to support SAP Enqueue Replication Host 1 TCP Connection Host 2 Lock Table Enqueue Server Lock State Transfer VIP Replication Server Lock Replica Table 0. CRS checks healthiness of resources on all hosts in the cluster . CRS detects a failure of the node or in the Enqueue Server .Control flow within SAPCTL to support SAP Enqueue Replication Host 1 Host 2 Crash Lock Table Enqueue Server Replication Server Lock Replica Table VIP 1. CRS switches the VIP to the node where Replication Server is running .Control flow within SAPCTL to support SAP Enqueue Replication Host 1 Host 2 Replication Server Lock Replica Table VIP 2. CRS starts a new Enqueue Server on the node supporting the VIP .Control flow within SAPCTL to support SAP Enqueue Replication Host 1 Host 2 Lock Table Enqueue Server Replication Server Lock Replica Table VIP 3. Enqueue Server reads data from Lock Replica Table .Control flow within SAPCTL to support SAP Enqueue Replication Host 1 Host 2 Lock Table Enqueue Server fill Enq. Table with replica Data Replication Server Lock Replica Table VIP 4. Control flow within SAPCTL to support SAP Enqueue Replication Host 1 Host 2 Lock Table Enqueue Server VIP 5. CRS stops Replication Server after Lock Table is rebuild from Lock Replica Table . Control flow within SAPCTL to support SAP Enqueue Replication Host 1N Host Host 2 Lock Table Enqueue Server Replication Server Lock Replica Table VIP 6. CRS starts Replication Server on another available node in the cluster . Back to normal operation.Control flow within SAPCTL to support SAP Enqueue Replication Host 1N Host Host 2 Connect Lock Table Enqueue Server Replication Server Lock Replica Table Lock State Transfer VIP 7. CRS checks healthiness of resources on all hosts in the cluster . listener database ORACLE_HOME ORA_CRS_HOME Oracle Clusterware HA Agent for SAP is protecting SAP Enqueue Server and SAP Enqueue Replication Server It is also providing resources to the RAC database running on nodes 1 & 2 .Possible Configuration SAP Enqueue Server listener cluster database ORACLE_HOME ORA_CRS_HOME SAP Enqueue Repl. listener cluster database ORACLE_HOME ORA_CRS_HOME ORA_CRS_HOME SAP Enqueue Repl. agent agent will be invoked by Oracle Clusterware with one of three commands – – – start stop check Stores information relating to how Oracle Clusterware interacts and manages the application Oracle Cluster Registry – .The Oracle Clusterware Framework Calls SAP HA . Oracle Clusterware Framework Architecture Framework Please Start Agent ‘start’ ‘check’ How are you ? Please Stop ‘stop’ . How the HA Agent for SAP is called by the Framework The ‘start’ command Would invoke the sapstart command startsap r3 ASCS startsap r3 ENR The ‘check’ command There are a number of things that could be checked Uses ensmon to obtain the status of the replication But: Does not check the message server The ‘stop’ command Would invoke the stopsap command stopsap r3 ASCS stopsap r3 ENR ‘start’ ‘check’ ‘stop’ . registers resources .shows configuration .shows status of resources .delete configuration .stops resources .Overview: SAPCTL commands for use with SAP R/3 Enqueue Replication sapctl create sapctl config sapctl status sapctl start sapctl stop sapctl update sapctl remove .modfy configuration .starts resources . IP address VIP <.255.0 -ip 140.name of the SAP System <.242.Central Instance <.network interface <.254.$HOME of SAP admin <.86.list of nodes <.SAPCTL command create sapctl create -sapsid C11 -if eth0 -nm 255.Enqueue replication .63 -u c11adm -n “node1 node2” -s /saphome/c11adm -e ASCS01 -r ENR01 <.netmask <.SAP admin <. SAPCTL command config sapctl config shows list of protected SAP systems sapctl config enqueue –sapsid C11 shows configuration of Central Instance sapctl config replication –sapsid C11 shows configuration of Replication Server sapctl config all –sapsid C11 shows complete configuration for SAP system . SAPCTL command status sapctl status enqueue –sapsid C11 shows status of Central Instance sapctl status replication –sapsid C11 shows status of Replication Server sapctl status all –sapsid C11 shows status of all SAP resources . SAPCTL command start sapctl start enqueue –sapsid C11 starts SAP Central Instance sapctl start replication –sapsid C11 starts SAP Replication Server sapctl start all –sapsid C11 starts all SAP resources . SAPCTL command stop sapctl stop enqueue –sapsid C11 stops SAP Central Instance sapctl stop replication –sapsid C11 stops SAP Replication Server sapctl stop all –sapsid C11 stops all SAP resources . list of nodes to add <.242.86.SAPCTL command update sapctl update -sapsid C11 -ip 140.63 -a “node1 node2” -d “node1 node2” <.name of the SAP System <.list of nodes to delete .IP address VIP <. `root` permission required .only remove of all SAP resources possible .SAPCTL command remove sapctl remove all –sapsid C11 removal of all SAP resources . as other tools or scripts relying on these commands still work. Consider this as either a very convenient feature or a big advantage. startsap r3 ASCS01 stopsap r3 ASCS01 startsap r3 ENR01 stopsap r3 ENR01 .SAP start/stop commands still usable Commonly used scripts startsap and stopsap trigger the start / stop of CRS resources as well.stops Replication Server Restrictions apply: User must use this command on the right node! .starts Replication Server .starts Enqueue and Message Server .stops Enqueue and Message Server . SAP changes required to use SAPCTL startsap script: • add instance type ENR / ASCS / SCS • add service type ER • call wrapper to notify Oracle Clusterware in function start_instance if service is “CS” or “ER”: sap2crs start \ $SAPSYSTEMNAME \ $SERVICE \ $INSTANCE . SAP changes required to use SAPCTL stopsap script: • add instance type ENR / ASCS / SCS • add service type ER • call wrapper to notify Oracle Clusterware in function stop_instance if service is “CS” or “ER”: sap2crs stop \ $SAPSYSTEMNAME \ $SERVICE \ $INSTANCE . SAP changes required to use SAP HA • Instance profile Standalone Enqueue Server (ENQ + MSG) • Instance profile Enqueue Replication Server (ENR) enque/process_location=LOCAL enque/server/replication=true enque/server/internal_replication= true • Start profile Standalone Enqueue Server • Start profile Enqueue replication Server As recommended by SAP HA documentation . C11_ASCS01_node-n The hostname is not virtualized by Oracle Clusterware! . START_ASCS01_node-n.SAP changes required to use SAP HA • All scripts exist multiple times: START_ASCS01_node-1. C11_ASCS01_node-1 START_ASCS01_node-2.. C11_ASCS01_node-2 …. Demonstration . SAP NetWeaver with Oracle 10gR2 RAC (UNIX & Linux) Oracle Location: http://www.com/sap -> Oracle on RAC section SAP Location: http://service.Further Information available: Oracle Whitepapers: 1.sap.com/dbaora ->Oracle Database Administration -> Media Library -> Oracle RAC . Technical White Paper: Providing High Availability for SAP Resources 2.oracle. Q U E S T I O N S A N S W E R S .