Build Oracle RAC 11.2.0.3 on Oracle Solaris 11 11.11 using Oracle VirtualBox In the article you will have a look at how to use some Oracle VirtualBox features to build two node Oracle 11gR2 11.2.0.3 RAC system on Oracle Solaris 11 11.11 x86-64. For information about a similar two node Oracle RAC 11.2 setup on Solaris 10 x86-64 using VirtualBox click here. The article will emphasize on the Solaris 11 management options and configurations required to meet Oracle 11.2.0.3 RAC installation prerequisites. The following software will be used: 1. Oracle 11.2.0.3 for Solaris (x86-64)- patch 10404530. Download from MOS here. 2. Oracle Solaris 11 (x86-64). I used Oracle Solaris 11 11/11 Live Media for x86. Download from here. 3. Oracle VM VirtualBox 4.1.8 Download from here. There will be two virtual machines Sol1 and Sol2, each of them will be configured with 4GB RAM 160GB bootable disk NIC – NAT: for access to Internet NIC - bridged for public interface in RAC with address 192.168.2.21/22 (first IP 192.168.2.21 on sol1 and second IP 192.168.2.22 on node sol2). These are public interface in RAC. NIC – bridged for private interface in RAC with address 10.10.2.21/22 (first IP 10.10.2.21 on sol1 and second IP 10.10.2.22 on node sol2). These are private interface in RAC. NIC – Host Only : for FTP/SSH/telnet/SCP access from the host OS to the guest OS. (first IP 192.168.56.51 on sol1 and second IP 192.168.56.51on node sol2). 5 10GB attached shared disks for the ASM storage. Sol1 VM will run Solaris 11 guest with hostname sol1. Sol2 VM will run Solaris 11 guest with hostname sol2. Note. For access to the Guest OS (Solaris) from the Host (MS Windows Vista 64) via ssh/scp/telnet/ftp. For the host OS to be able to access the guest OS a Host Only adapter is required and the corresponding IP on the guest OS should be within the IP subnet of the VirtualBox Host-Only adapter. On the host (Windows in this case) we have. Ethernet adapter VirtualBox Host-Only Network: Connection-specific Description . . . . Physical Address. . DHCP Enabled. . . . DNS . . . . . . Suffix . . . . . . . . . . . . . . . . : : VirtualBox Host-Only Ethernet Adapter : 08-00-27-00-3C-B7 : No Autoconfiguration Enabled . . . . : Yes IPv4 Address. . . . . . . . . . . : 192.168.56.1(Preferred) Subnet Mask . . . . . . . . . . . : 255.255.255.0 Default Gateway . . . . . . . . . : NetBIOS over Tcpip. . . . . . . . : Enabled Thus, the IP, corresponding to the Host-Only NIC, inside the guest (Solaris in this case) should be within the 192.168.56.* subnet. Should the VirtualBox Host-Only adapter IP change, make sure that the IP within the guest OS is within the same subnet as the IP of the VirtualBox Host-Only adapter. Examples of accessing Solaris 11 from a cygwin on Windows using ssh/scp. bash-3.2$ uname -a CYGWIN_NT-6.0-WOW64 userpc 1.7.5(0.225/5/3) 2010-04-12 19:07 i686 Cygwin bash-3.2$ bash-3.2$ ssh -X
[email protected] The authenticity of host '192.168.56.52 (192.168.56.52)' can't be established. RSA key fingerprint is 1b:f8:c2:74:cf:29:4b:e8:0d:a6:d8:f6:d9:51:92:72. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added '192.168.56.52' (RSA) to the list of known hosts. Password: Warning: untrusted X11 forwarding setup failed: xauth key data not generated Warning: No xauth data; using fake authentication data for X11 forwarding. Last login: Mon Feb 6 22:22:49 2012 Oracle Corporation SunOS 5.11 11.0 November 2011 gjilevski@sol2:~$ uname -a SunOS sol2 5.11 11.0 i86pc i386 i86pc gjilevski@sol2:~$ ls /u01/sh.sh /u01/sh.sh gjilevski@sol2:~$ exit logout Connection to 192.168.56.52 closed. bash-3.2$ scp
[email protected]:/u01/sh.sh . Password: sh.sh 100% 682 0.7KB/s 00:00 bash-3.2$ uname -a CYGWIN_NT-6.0-WOW64 userpc 1.7.5(0.225/5/3) 2010-04-12 19:07 i686 Cygwin bash-3.2$ Same approach can be used for telnet or ftp access from host OS if ftp or telnet services are enabled on the guest Solaris OS. First you will create the first virtual machine and configure network interfaces and install Solaris software and meet the Oracle RAC installation prerequisites. You will than attach the shared disks to the first virtual machine and prepare the shared storage for ASM. Last you will clone the boot disk of the first VM to create the second VM and plug the shared storage and change the IPs and hostname. In this article you will have role separation with two different accounts one (grid) for ASM storage management and the second (oracle) for RDBMS installation. In order to support role separation you will create additional groups for (asmadmin, asmdba, asmoper, oinstall) and ( dba. oinstall, oper). At the end you will install GI, RDBMS and create a two node RAC database. You will see how to fix the problems that were encountered during the installation. 1. Create the first VM Sol1 Oracle Solaris 11 was released late 2011. In order to obtain detailed information for installation options click here or access Solaris 11 documentation click here. I am using Oracle Solaris 11 11/11 Live Media for x86. It performs default installation using Automatic network management. After Solaris 11 installation, I will switch to Manual network management and modify IP addresses and hostnames. Select New and click Next. Enter the name of the VM (Sol1) and press Next. Select 4096 MB for the RAM of the VM and press Next to continue. Select create a new disk for the boot disk and press Next to continue. Select Dynamically expanding storage and press Next to continue. Select 160GB (not 16GB) and press Next to continue. (16GB is not sufficient to install Solaris and Oracle GI and RDBMS). . Press Next to continue. vdi --size 10240 --format VDI --variant Fixed d:\vb\asm10. Create the disks to be used as shared. bridged and HostOnly. Note that for a disk to be shared it must be fixed. VBoxManage storageattach Sol1 --storagectl "SATA Controller" --port 1 --device 0 -type hdd --medium d:\vb\asm1.vdi d:\vb\asm3.vdi --size 10240 --format VDI --variant Fixed d:\vb\asm8.vdi --mtype shareable . First to be NAT.vdi d:\vb\asm5. After that select the VM click settings and add the four NIC as specified.Press Finish.vdi d:\vb\asm4. bridged.vdi --size 10240 --format VDI --variant Fixed d:\vb\asm7.vdi --size 10240 --format VDI --variant Fixed Attach the shared disks to the VM and mark them as shared.vdi --size --size --size --size --size 10240 10240 10240 10240 10240 --format --format --format --format --format VDI VDI VDI VDI VDI --variant --variant --variant --variant --variant Fixed Fixed Fixed Fixed Fixed d:\vb\asm6.vdi d:\vb\asm2. This concludes the VM creation.vdi --size 10240 --format VDI --variant Fixed d:\vb\asm9. VBoxManage VBoxManage VBoxManage VBoxManage VBoxManage VBoxManage VBoxManage VBoxManage VBoxManage VBoxManage createhd createhd createhd createhd createhd createhd createhd createhd createhd createhd --filename --filename --filename --filename --filename --filename --filename --filename --filename --filename d:\vb\asm1.vdi --mtype shareable VBoxManage storageattach Sol1 --storagectl "SATA Controller" --port 3 --device 0 -type hdd --medium d:\vb\asm3.vdi --mtype shareable VBoxManage storageattach Sol1 --storagectl "SATA Controller" --port 2 --device 0 -type hdd --medium d:\vb\asm2. vdi --mtype shareable VBoxManage storageattach Sol1 --storagectl "SATA Controller" --type hdd --medium d:\vb\asm10.vdi --mtype shareable VBoxManage storageattach Sol1 --storagectl "SATA Controller" -type hdd --medium d:\vb\asm8.vdi --type shareable d:\vb\asm10.vdi --type shareable d:\vb\asm2.VBoxManage storageattach Sol1 --storagectl "SATA Controller" -type hdd --medium d:\vb\asm4.vdi --mtype shareable VBoxManage storageattach Sol1 --storagectl "SATA Controller" -type hdd --medium d:\vb\asm7.vdi --type shareable d:\vb\asm7. .vdi --type shareable d:\vb\asm4.vdi --mtype shareable VBoxManage storageattach Sol1 --storagectl "SATA Controller" -type hdd --medium d:\vb\asm9.vdi --type shareable The configuration will look like the graphical image below.vdi --mtype shareable VBoxManage storageattach Sol1 --storagectl "SATA Controller" -type hdd --medium d:\vb\asm6.vdi --mtype shareable VBoxManage storageattach Sol1 --storagectl "SATA Controller" -type hdd --medium d:\vb\asm5.vdi --type shareable d:\vb\asm8.vdi --type shareable d:\vb\asm5.vdi --type shareable d:\vb\asm6.vdi --mtype shareable --port 4 --device 0 --port 5 --device 0 --port 6 --device 0 --port 7 --device 0 --port 8 --device 0 --port 9 --device 0 --port 10 --device 0 VBoxManage VBoxManage VBoxManage VBoxManage VBoxManage VBoxManage VBoxManage VBoxManage VBoxManage VBoxManage modifyhd modifyhd modifyhd modifyhd modifyhd modifyhd modifyhd modifyhd modifyhd modifyhd d:\vb\asm1.vdi --type shareable d:\vb\asm3.vdi --type shareable d:\vb\asm9. In this case the iso media is on a Windows directory and you press on the yellow folder with green arrow to select it and press Next to continue. . After the VM is created and started specify the media for Solaris installation.Now that the shared disks are attached and the virtual NICs are specified let’s start the VM for Solaris installation. The iso image file can be either staged in a directory on the host OS or on DVD. Note that although the image is for Solaris 10 but you must use Solaris 11 iso file and will work the same way. Start the VM and specify the installation media containing the Solaris iso. . Press Finish to start.Press Next to continue. Once the system boots up Select Solaris 11 and press Return. .Select keyboard. Select language. . In order to start the real installation click Install Oracle Solaris. Note that Automatic Network is selected.Wait until the virtual OS is launched in memory. . .Press Next. I did not partition the disk but later added in swap space for the OUI to succeed. The default swap is 1GB and OUI asks for minimum 4GB in order to install 11.Select the 160GB disk.0.2. .3 RAC. .Select your time zone. Starting with Solaris 11 root is defined as a role and you need a user to log in. . Specify the hostname.Create an account with administrative privileges. A role cannot log you in. Review and press Install. . .Wait for the install to completes. . Reboot once installation completes. . .Select Solaris 11. .Login in with the previously created user. .Select login session. .Finally you are complete the Solaris 11 installation. Installing Guest Additions After login as yourself. Select OK.2. on the VirtualBox console select Devices->Install Guest Additions… The Virtual Editions media gets mounted as a virtual CD. . use the default (solaris) which is expired and need a change.Here you have a chance to change the root password. . Initially. 3. Shared Folders configuration Shared folders are configured as follows. .Reboot the Solaris guest VM. .VirtualBox allows you to mount on the guest OS folders from the host OS.1 and later you can automatically add shared folders and VirtualBox mounts them for you. If using latest VirtualBox 4. /OracleVMServer .yes vboxfs . pfexec mount -F vboxfs Solaris10 /whatever-you-want In my case I created two directories /software and /OracleVMServer and mounted the shared folders as follow. software OracleVMServer . pfexec mount -F vboxfs pfexec mount -F vboxfs software /software /OracleVMServer OracleVMServer To create a permanent mount points that persist a reboot add the following line to the /etc/vfstab file./software vboxfs .In case you are on a previous VirtualBox release. as root user create directories for each folder and use the following command to mount.yes - . The flow of commands is as follow.0/pci8086.2829@d/
[email protected]/pci8086.0/pci8086. You need to format and prepare them for the ASM with privileges (0660) and ownership (grid:asmadmin). c1t5d0. Using the following commands drop and recreate the swap with desired size 5GB.0 2. Each disk is formatted and a partition is created starting from 1 cylinder..0/pci8086.2829@d/disk@a. c1t1d0 <DEFAULT cyl 1303 alt 2 hd 255 sec 63> /pci@0. root@sol1:~# swap -d /dev/zvol/dsk/rpool/swap root@sol1:~# zfs volsize=5G rpool/swap root@sol1:~# swap -a /dev/zvol/dsk/rpool/swap root@sol1:~# swap -s total: 259856k bytes allocated + 35808k reserved = 295664k used. Than for each disk you invoke fdisk and accept Solaris as default type of partition.2829@d/
[email protected] 4.0/pci8086. The available disks can be seen using the format command as shown below.0 7. c1t6d0 <DEFAULT cyl 1302 alt 2 hd 255 sec 63> /
[email protected]@d/
[email protected] Specify disk (enter its number): 4 selecting c1t4d0 [disk formatted] . c1t8d0.4. c1t2d0. c1t9d0 and c1t10d0. c1t7d0 <DEFAULT cyl 1302 alt 2 hd 255 sec 63> /
[email protected] installation 4GB minimum is required.0 5.0 6.95GB (remember 10GB was the original size) and print the partition table. c1t9d0 <DEFAULT cyl 1302 alt 2 hd 255 sec 63> /pci@0. c1t4d0 <DEFAULT cyl 1303 alt 2 hd 255 sec 63> /pci@0. Then you partition with partition command and select 0 partition to start from first cylinder and enter size 9.0/pci8086. Setting the shared storage for ASM and SWAP The default configured swap space was 1GB. root@sol1:~# format Searching for disks.done AVAILABLE DISK SELECTIONS: 0. c1t8d0 <DEFAULT cyl 1302 alt 2 hd 255 sec 63> /
[email protected] 8.0 10.2829@d/disk@1. Those disks are visible as c1t1d0. c1t0d0 <DEFAULT cyl 20883 alt 2 hd 255 sec 63> /
[email protected] 3. For Oracle 11. c1t2d0 <DEFAULT cyl 1303 alt 2 hd 255 se c 63> /
[email protected]@d/
[email protected] 8 10485752 10485752 root@sol1:~# There are ten shared disks attached to the Solaris VM.2829@d/
[email protected]@d/
[email protected]/pci8086.c1t3d0.0/pci8086. c1t6d0.2829@d/
[email protected]/pci8086. c1t5d0 <DEFAULT cyl 1303 alt 2 hd 255 sec 63> /pci@0. 6669044k available root@sol1:~# swap -l swapfile dev swaplo blocks free /dev/zvol/dsk/rpool/swap 195. c1t7d0.2829@d/
[email protected]/pci8086.0 1.2829@d/
[email protected] 9. You repeat the procedure for all ten shared disks. c1t10d0 <DEFAULT cyl 1302 alt 2 hd 255 sec 63> /pci@0. c1t3d0 <DEFAULT cyl 1303 alt 2 hd 255 sec 63> /
[email protected]/pci8086. defect list management backup .show vendor.surface analysis defect . y format> format> partition PARTITION MENU: 0 1 2 3 4 5 6 7 select modify name print label !<cmd> quit partition> 0 Part Tag 0 unassigned otherwise type "n" to edit the change `0' partition change `1' partition change `2' partition change `3' partition change `4' partition change `5' partition change `6' partition change `7' partition select a predefined table modify a predefined partition table name the current table display the current table write partition map and label to the disk execute <cmd>.execute <cmd>. 1e.select (define) a disk type partition . The default partition for the disk is: a 100% "SOLARIS System" partition Type "y" to accept the default partition.describe the current disk format . then return quit format> format> format> fdisk No fdisk table exists. 0.00mb.read and display labels save .95GB 0 9.00gb]: 9.search for backup labels verify .1301 0 0 0 0 0 0 0 Size 9.select (define) a partition table current .95g partition> print Current partition table (unnamed): Total disk cylinders available: 1302 + 2 (reserved cylinders) Part Tag 0 unassigned 1 unassigned 2 backup 3 unassigned 4 unassigned 5 unassigned 6 unassigned 7 unassigned 8 boot Flag wm wm wu wm wm wm wm wm wu Cylinders 1 .write label to the disk analyze .1299 0 0 . product and revision volname . 0.select a disk type .set 8-character volume name !<cmd> .format and analyze the disk fdisk .97GB 0 0 0 0 0 7. partition table. then return Flag wm Cylinders 0 Size 0 Blocks (0/0/0) 0 Enter partition id tag[unassigned]: Enter partition permission flags[wm]: Enter new starting cyl[0]: 1 Enter partition size[0b.repair a defective sector label . 0c.save new disk/partition definitions inquiry .run the fdisk program repair .84MB Blocks (1299/0/0) 20868435 (0/0/0) 0 (1302/0/0) 20916630 (0/0/0) 0 (0/0/0) 0 (0/0/0) 0 (0/0/0) 0 (0/0/0) 0 (1/0/0) 16065 .FORMAT MENU: disk . Repeat the same steps for all remaining nine shared disks. The oracle user is a member of dba.dba. continue? y partition>quit In this case the whole disk c1t4d0 was partitioned as one partition and made available for ASM. asmoper and the primary group is oinstall. However each admin has different groups. The idea is Oracle GI will have a user grid as ASM administrator and Oracle RDBMS will have a user oracle as RDBMS administrator. Users and groups are created with the following commands.asmadmin. asmdba.asmdba -d /export/home/oracle -m oracle . groupadd -g 1000 oinstall groupadd -g 1020 asmadmin groupadd -g 1021 asmdba groupadd -g 1022 asmoper groupadd -g 1031 dba groupadd -g 1032 oper useradd -u 1100 -g oinstall -G asmoper. Creating Solaris users and groups Oracle starting with 11gR2 is enabling a role separation when installing Oracle GI and Oracle RDBMS. 5.asmdba -d /export/home/grid -m grid useradd -u 1101 -g oinstall -G oper. Since slice 0 was used you need to make those available to ASM by setting the permissions and ownership as show below. asmdba and oper and the primary group is oinstall. The grid user is a member of asmadmin.9 unassigned wm 0 0 (0/0/0) 0 partition> partition> label Ready to label disk. chown chmod chown chmod chown chmod chown chmod chown chmod chown chmod chown chmod chown chmod chown chmod chown chmod grid:asmadmin /dev/rdsk/c1t1d0s0 660 /dev/rdsk/c1t1d0s0 grid:asmadmin /dev/rdsk/c1t10d0s0 660 /dev/rdsk/c1t10d0s0 grid:asmadmin /dev/rdsk/c1t9d0s0 660 /dev/rdsk/c1t9d0s0 grid:asmadmin /dev/rdsk/c1t8d0s0 660 /dev/rdsk/c1t8d0s0 grid:asmadmin /dev/rdsk/c1t7d0s0 660 /dev/rdsk/c1t7d0s0 grid:asmadmin /dev/rdsk/c1t6d0s0 660 /dev/rdsk/c1t6d0s0 grid:asmadmin /dev/rdsk/c1t5d0s0 660 /dev/rdsk/c1t5d0s0 grid:asmadmin /dev/rdsk/c1t4d0s0 660 /dev/rdsk/c1t4d0s0 grid:asmadmin /dev/rdsk/c1t3d0s0 660 /dev/rdsk/c1t3d0s0 grid:asmadmin /dev/rdsk/c1t2d0s0 660 /dev/rdsk/c1t2d0s0 For more information refer to Oracle documentation here. Both users share oinstall group so that both Oracle GI and Oracle RDBMS share the same inventory. lines for the TCP and UDP parameters to tm::sysinit:/usr/sbin/ndd -set /dev/tcp tcp_smallest_anon_port 9000 > /dev/console tm::sysinit:/usr/sbin/ndd -set /dev/tcp tcp_largest_anon_port 65500 > /dev/console .grid -U oracle -K "project. Set the TCP and UDP kernel parameters /usr/sbin/ndd /usr/sbin/ndd /usr/sbin/ndd /usr/sbin/ndd -set -set -set -set /dev/tcp /dev/tcp /dev/udp /dev/udp tcp_smallest_anon_port 9000 tcp_largest_anon_port 65500 udp_smallest_anon_port 9000 udp_largest_anon_port 65500 Put in /etc/inittab the following persist across reboot.512.deny)" user. projadd projmod projmod projmod projmod projadd projmod projmod projmod projmod -U grid -K "project. 6.max-sem-nsems=(priv.deny)" user.128.512.oracle -sK "project.grid -sK "project.2.128.max-sem-ids=(priv.deny)" user.deny)" user. Create the directories for ORACLE_BASE and ORACLE_HOME for Oracle GI and Oracle RDBMS mkdir mkdir chown mkdir chown chmod mkdir chown chmod mkdir chown chmod -p /u01/app/11.0/db_1 -R oracle:oinstall /u01/app/oracle -R 775 /u01/app/oracle 7.6g.2.6g.grid If the max file descriptors are not set properly there might be errors starting OUI.grid -sK "project.max-shm-memory=(priv.max-shm-memory=(priv.128.65536.grid -sK "project.max-shm-memory=(priv.max-sem-ids=(priv.deny)" user.2.2.0/grid grid:oinstall /u01/app/11.6g.deny)" user.max-file-descriptor=(priv.deny)" user.deny)" user.oracle -sK "project.oracle /usr/sbin/projmod -sK "process.User grid and user oracle separate the responsibilities between storage administrator and Oracle DBA.oracle -sK "project.0/grid -p /u01/app/grid -R grid:oinstall /u01 -p /u01/app/oracle oracle:oinstall /u01/app/oracle -R 775 /u01 -p /u01/app/11.max-shm-ids=(priv. Create two projects for Oracle GI and Oracle RDBMS users respectively.128.oracle /usr/sbin/projmod -sK "process.oracle -sK "project.max-shm-memory=(priv.0/grid -R 775 /u01/app/11.grid -sK "project.max-file-descriptor=(priv.deny)" user.6g.deny)" user.deny)" user.max-sem-nsems=(priv.65536.2.max-shm-ids=(priv. Set the share memory parameters.0/grid -p /u01/app/oracle/product/11.deny)" user. Setup Solaris kernel parameters. %n.%p. And enable core dumps as show below.core -e global -e global-setid -e log -d process -d proc-setid Verify the result. Automatic SSH configuration Oracle recommends using OUI to setup ssh user equivalence where users can connect across the node of the cluster without a password. To check if core dumps are enabled use coreadm command as follows.tm::sysinit:/usr/sbin/ndd -set /dev/udp udp_smallest_anon_port 9000 > /dev/console tm::sysinit:/usr/sbin/ndd -set /dev/udp udp_largest_anon_port 65500 > /dev/console 8.%p.%n.%t. # coreadm global global init init core file pattern: core file content: core file pattern: core file content: global core dumps: per-process core dumps: global setid core dumps: per-process setid core dumps: global core dump logging: /var/cores/%f. To avoid errors while attaching $OH when the remote node closes a connection prematurely set in /etc/ssh/sshd_config file LoginGraceTime 0. # coreadm global global init init core file pattern: core file content: core file pattern: core file content: global core dumps: per-process core dumps: global setid core dumps: per-process setid core dumps: global core dump logging: default core default disabled enabled disabled disabled disabled # As root user make the following directory.core default core default enabled disabled enabled disabled enabled .%t. Enable Core file creation Make sure that Core Dumps are enabled. mkdir -p /var/cores coreadm -g /var/cores/%f. After changing the file restart ssh service. svcadm restart ssh 9. Network Time protocol Settings You have two options for time synchronization: an operating system configured network time protocol (NTP).2. If you do not have NTP daemons. Disable NTP and rely entirely on CTSS: As root execute the command below: /usr/sbin/svcadm disable ntp B. If you use NTP. then the Oracle Cluster Time Synchronization daemon (ctssd) starts up in observer mode. Configure NTP and use it together with CTSS: As root edit the /etc/inet/ntp. or Oracle Cluster Time Synchronization Service. Create the following profiles For user grid umask 022 ORACLE_BASE=/u01/app/grid ORACLE_HOME=/u01/app/11. then stty intr ^C fi For user oracle umask 022 ORACLE_BASE=/u01/app/oracle ORACLE_HOME=/u01/app/oracle/product/11. NTP is disabled in the installation reflected in article. restart xntpd using the command /usr/sbin/svcadm restart ntp. After you make these changes.0/grid ORACLE_SID=+ASM1 LD_LIBRARY_PATH=$ORACLE_HOME/lib PATH=$PATH:/usr/local/bin:/usr/sbin:/usr/bin:/usr/openwin/bin:/usr/ucb:$ORACLE_H OME/bin export ORACLE_BASE ORACLE_HOME ORACLE_SID LD_LIBRARY_PATH PATH TEMP=/tmp TMPDIR=/tmp export TEMP TMPDIR ulimit ulimit ulimit ulimit ulimit -t -f -d -s -v unlimited unlimited unlimited unlimited unlimited if [ -t 0 ]. Oracle Cluster Time Synchronization Service is designed for organizations whose cluster servers are unable to access NTP services. then ctssd starts up in active mode and synchronizes time among cluster members without contacting an external time server.2. So there are two options A.# 10. 11.0/db_1 .conf file to add "slewalways yes" and "disable pll" to the file. or refreshing publisher information: SUNWi1of SUNW SUNWxwfnt root@sol2:~# .2] Since I am on Solaris 11 I did root@sol2:~# pkg install SUNWarc SUNWbtool SUNWhea SUNWlibC SUNWlibm SUNWlibms SUNW SUNWtoo SUNWi1of SUNWi1cs SUNWi15cs SUNWxwfnt SUNWcsl Creating Plan pkg install: The following pattern(s) did not match any allowable packages. SUNWarc SUNWbtool SUNWcsl SUNWhea SUNWlibC SUNWlibm SUNWlibms SUNWsprot SUNWtoo SUNWi1of (ISO8859-1) SUNWi1cs (ISO8859-15) SUNWi15cs SUNWxwfnt 119961-05 or later 119964-14 or later 120754-06 or later 139556-08 or later 139575-03 or later 141415-04 or later 141445-09 or later [11.2. Require packages and patches For Solaris 10 x86-64 the following packages and patches are required. then stty intr ^C fi 12.0.ORACLE_SID=D11G LD_LIBRARY_PATH=$ORACLE_HOME/lib PATH=$PATH:/usr/local/bin:/usr/sbin:/usr/bin:/usr/openwin/bin:/usr/ucb:$ORACLE_HOME/bin export ORACLE_BASE ORACLE_HOME ORACLE_SID LD_LIBRARY_PATH PATH TEMP=/tmp TMPDIR=/tmp export TEMP TMPDIR ulimit -t unlimited ulimit -f unlimited ulimit -d unlimited ulimit -s unlimited ulimit -v unlimited if [ -t 0 ]. Try using a different matching pattern. Set up file descriptors Edit the /etc/system to add the following parameters. # Hard limit on file descriptors for single process set rlim_fd_max = 65536 # Soft limit on the file descriptors for a single process set rlim_fd_cur = 65536 13. 0.168.0.21/24 net1/addr ipadm create-ip net2 ipadm create-addr -T static -a local=10.3 RAC installation on Solaris 11.2.51/24 net3/addr At the end I have: root@sol1:~# dladm show-phys LINK MEDIA net1 Ethernet net2 Ethernet net0 Ethernet net3 Ethernet root@sol1:~# STATE up up up up SPEED 1000 1000 1000 1000 DUPLEX full full f ull full DEVICE e1000g1 e1000g2 e1000g0 e1000g3 Hostname was configured during Solaris 11 installation.3.168. gjilevski@sol1:~$ LINK net1 net2 net0 net3 gjilevski@sol1:~$ dladm show-phys MEDIA Ethernet Ethernet Ethernet Ethernet STATE unknown unknown unknown unknown SPEED 0 0 0 0 DUPLEX unknown unknown unknown unknown DEVICE e1000g1 e1000g2 e1000g0 e1000g3 In Solaris 11 IP addresses are set as follows: ipadm create-ip net0 ipadm create-addr -T dhcp net0/addr ipadm create-ip net1 ipadm create-addr -T static -a local=192. Despite. Modify /etc/hosts file to have entries for the public.3 OUI confirmed later that installed packages and patches suffice for Oracle 11.2. root@sol1:~# netadm enable -p ncp DefaultFixed Enabling ncp 'DefaultFixed' root@sol1:~# After switch to manual Network Management I have.2.2. that I did not find information about packages and patches for Oracle 11.2.21/24 net2/addr ipadm create-ip net3 ipadm create-addr -T static -a local=192. In order to manually set up IP addresses I switched to Manual Network management using the following command. private and scan addresses similarly to the output below . Configure IP addresses and confirm hostname The performed Oracle Solaris 11 11/11 Live Media installation provides Automatic Network Management. 14.Solaris 11 is supported and certified for Oracle 11.10.0.56. 15. .gj.gj.0.21 sol1 sol1..10. Attach the shared disks to the second VM using commands as described below.gj....30%.gjilevski@sol2:~$ cat /etc/hosts # # Copyright 2009 Sun Microsystems. d:\VB>VBoxManage createhd --filename d:\vb\asm_5.21 sol1-priv sol1-priv..vdi --mtype Sol2 --storagectl "SATA Controller" -hdd --medium d:\vb\asm4.gj.20%..com 10.gj.2.gj.56.22 sol2 sol2.10.0.gj. UUID: 0df463ae-587a-4f62-bcd1-ac7dbab86f8f d:\VB> Create a second VM called Sol2 with the same resource as Sol1 but specify the cloned disk as a bootable disk.com 192. All rights reserved.168..vdi --mtype Sol2 --storagectl "SATA Controller" -hdd --medium d:\vb\asm3.2.. VBoxManage storageattach port 1 --device 0 --type shareable VBoxManage storageattach port 2 --device 0 --type shareable VBoxManage storageattach port 3 --device 0 --type shareable VBoxManage storageattach port 4 --device 0 --type shareable Sol2 --storagectl "SATA Controller" -hdd --medium d:\vb\asm1..gj.com 192.60%.vdi --mtype .2.31 sol1-vip sol1-vip.22 sol2-priv sol2-priv.40%..com 10..vdi --mtype Sol2 --storagectl "SATA Controller" -hdd --medium d:\vb\asm2.vdi --size 10240 --format VDI -variant Fixed 0%..com 192.51 scan-sol scan-sol..com 192.. Inc.1 localhost 192..168.100% Disk image created.168.168.90%.com gjilevski@sol2:~$ 16.com 192.2..2.80%.2.168.10%. # Use is subject to license terms. Clone the boot disk of Sol1 VM to create a second VM corresponding to the second node.168.70%.gj.52 sol2-exp sol2-exp.56.com 192.168.2. # # Internet host table # ::1 localhost 127.51 sol1-exp sol1-exp.32 sol2-vip sol2-vip..50%... vdi --type shareable VBoxManage modifyhd d:\vb\asm3.vdi --type shareable VBoxManage modifyhd d:\vb\asm6. ipadm create-ip net0 ipadm create-addr -T dhcp net0/addr .vdi --mtype shareable VBoxManage storageattach Sol2 --storagectl "SATA Controller" -port 7 --device 0 --type hdd --medium d:\vb\asm7.vdi --type shareable VBoxManage modifyhd d:\vb\asm8. As root on Sol2 VM execute the following commands.vdi --type shareable VBoxManage modifyhd d:\vb\asm7.vdi --mtype shareable VBoxManage storageattach Sol2 --storagectl "SATA Controller" -port 8 --device 0 --type hdd --medium d:\vb\asm8.VBoxManage storageattach Sol2 --storagectl "SATA Controller" -port 5 --device 0 --type hdd --medium d:\vb\asm5.vdi --type shareable VBoxManage modifyhd d:\vb\asm2. Initially both machines apparently have same IPs and hostname.vdi --type shareable VBoxManage modifyhd d:\vb\asm4.vdi --type shareable VBoxManage modifyhd d:\vb\asm5. svccfg -s svc:/system/identity:node setp rop config/nodename =sol2 svcadm refresh svc:/system/id entity:node svcadm restart svc:/system/identity:node ipadm delete-ip net0 ipadm delete-ip net1 ipadm delete-ip net2 ipadm delete-ip net3 After restart as root execute the following commands to setup the IP addresses.vdi --type shareable VBoxManage modifyhd d:\vb\asm10.vdi --type shareable 17.vdi --mtype shareable VBoxManage modifyhd d:\vb\asm1.vdi --mtype shareable VBoxManage storageattach Sol2 --storagectl "SATA Controller" -port 10 --device 0 --type hdd --medium d:\vb\asm10. Modify the hostname and IPs on the cloned VM Sol2 In order to avoid collision with IP start only the second cloned VM and change the IP addresses and hostname.vdi --type shareable VBoxManage modifyhd d:\vb\asm9.vdi --mtype shareable VBoxManage storageattach Sol2 --storagectl "SATA Controller" -port 6 --device 0 --type hdd --medium d:\vb\asm6.vdi --mtype shareable VBoxManage storageattach Sol2 --storagectl "SATA Controller" -port 9 --device 0 --type hdd --medium d:\vb\asm9. In order to modify the hostname to sol2 execute the following command to change the host and delete the interfaces and reboot the node. lang.2.Runtime.loadLibrary0(Runtime. .10.ipadm create-ip net1 ipadm create-addr -T static -a local=192.168.ClassLoader$NativeLibra ry. Run cluvfy to verify the prerequisites.0.56.sol2 –verbose /runcluvfy.load0(Runtime.grid@sol1:/u01/stage/grid$ Exception in thread "main" java. AWT_TOOLKIT be Oracle 11.so .1: java: fatal: libXm.22/24 net1/addr ipadm create-ip net2 ipadm create-addr -T static -a local=10.UnsatisfiedLinkError: /tmp/Ora Install2012-0204_05-50-46AM/jdk/jre/lib/amd64/mo tif21/libma wt.loadLibrary(ClassLoader. Reboot the cloned Sol2 and fire up the Sol1 19.System.22/24 net2/addr ipadm create-ip net3 ipadm create-addr -T static -a local=192.ClassLoader.lang..168.ClassLoader$NativeLibra ry./runcluvfy.3 on Solaris 11 requires that variable exported.java :968) at java.52/24 net3/addr root@sol2:~# dladm show-phys LINK MEDIA STATE net1 Ethernet up net2 Ethernet up net0 Ethernet up net3 Ethernet up root@sol2:~# SPEED DUPLEX DEVICE 1000 full e1000g1 1000 full e1000g2 1000 full e1000g0 1000 full e1000g3 18.java:1753) at java.load(Native Method) at java.loadLibrary0(ClassLoader.lang.java:822) at java.sh stage -pre crsinst -n sol1.4: open failed : No su ch file o r directory at java.. export AWT_TOOLKIT=XToolkit Failure to do so will produce the following error. Preparing to launch O racle Universal Installer fro m /tmp/Ora Install2012 -02-04_05-50-46AM.sol2 -verbose 21.lang.load(Native Method) at java.loadLibrary(ClassLoader.ClassLoader.ClassLoader. Please wait .java :1670) at java.lang.2.java:1753) at java.2.sh stage -post hwos -n sol1.lang.load(System. Start OUI for Oracle GI installation.lang. Use OUI to setup ssh user equivalence for user grid and oracle. Use cluvfy from the stage directory to make sure that prerequisites are met.System.so: ld.ClassLoader.loadLibrary(System.lang. 20.lang.Runtime.lang.java :1649) at java.so .java:769) at java.lang.loadLibrary0(ClassLoader.java:993) . looks.install.lang.jgoodies.isLowResolution(LookUtils.java :348) at oracle. Select Skip software update and press Next to continue.LookUtils.LookUtils.co mmons.CRSInstaller.Toolkit.crs.AccessController.awt.Class.driver.fo rName0(Native Method) at java.swing.jgoodies.looks.driver.java:1779) at javax.LoadLibraryAction.util.loadLib raries(Toolkit.<clinit>(Toolkit.flow.install.install.security.java:105) After issuing xhost + from a root connected terminal session and setting a DISPLAY=:0 and AWT_TOOLKIT=XToolkit in a session logged as grid run OUI.SwingUtilities.Installer.CRSInstaller.java:50) at java.fo rName(Class.startup(Application.common.at sun.java:135) at java.FlowApplica tion.driver.Toolkit.setLookAndFeel(UIManager.ivw.sta rtup(Flo wApplicatio n.main(CRSInstaller.jgoodies.action.sta rtup(CRSInstaller.java:780) at oracle.sta rtup(In staller.java :453) at oracle.<clinit>(LookUtils.java :165) at oracle.plastic.Class.co mmons.looks.java:98) at oracle.FlowApplica tion.java:242) at javax.<clinit>(PlasticLookAndFeel.sta rtup(Flo wApplication.java:1530) at com. .base.co mmons.PlasticLookAndFeel.flow.install.ivw.UIManager.Application.security.co mmons.awt.loadSystemCla ss(Swing Utilities.java :249) at com.install.install.crs.java:484) at com.java:1509) at java.java :182) at oracle.run(LoadLibra ryAction.swing.lang.doPrivileged(Native Method) at java. Select Install and configure GI Select typical installation. . Specify the SCAN name and enter data for the second node. . .Make sure that interfaces are properly selected. .Select ASM as storage option. redundancy and disks. .Specify dusk group name. I selected HIGH redundancy. .Specify inventory location. .Examine the prerequisites check. Review the summary and press Install. . .Wait for the OUI to prompt for the scripts to be run as root. Now product-specific root actions will be performed.2. No need to overwrite.2.root@sol1:/u01/app/11. No need to overwrite.0/grid# . No need to overwrite.sh Performing root user operation for Oracle 11g The following environment variables are set as: ORACLE_OWNER= grid ORACLE_HOME= /u01/app/11.0/grid Enter the full pathname of the local bin directory: [/usr/local/bin]: The contents of "dbhome" have not changed.2. Entries will be added to the /var/opt/oracle/oratab file as needed by Database Configuration Assistant when a database is created Finished running generic part of root script.successful root wallet root wallet cert root cert export ./root. The contents of "coraenv" have not changed.0/grid/crs/install/crsconfig_params User ignored Prerequisites during installation OLR initialization . The contents of "oraenv" have not changed. Using configuration parameter file: /u01/app/11. ONLINE 3a99cb045bd94f5ebf038d4b4584228c (/dev/rdsk/c4t3d0s0) [DATA] 4.cssdmonitor' on 'sol1' succeeded CRS-2672: Attempting to start 'ora. CRS-4256: Updating the profile Successful addition of voting disk 054f135ad3644fbabff3f058ef28ff75. Successful addition of voting disk 3a99cb045bd94f5ebf038d4b4584228c.cssd' on 'sol1' CRS-2672: Attempting to start 'ora.cssd' on 'sol1' succeeded ASM created and started successfully. .gpnpd' on 'sol1' succeeded CRS-2672: Attempting to start 'ora.diskmon' on 'sol1' succeeded CRS-2676: Start of 'ora.gpnpd' on 'sol1' CRS-2676: Start of 'ora.. CRS-4256: Updating the profile CRS-4266: Voting file(s) successfully replaced ## STATE File Universal Id File Name Disk group -.----------------------------. privgrp 'root'.peer wallet profile reader wallet pa wallet peer wallet keys pa wallet keys peer cert request pa cert request peer cert pa cert peer root cert TP profile reader root cert TP pa root cert TP peer pa cert TP pa peer cert TP profile reader pa cert TP profile reader peer cert TP peer user cert pa user cert Adding Clusterware entries to inittab CRS-2672: Attempting to start 'ora. Successful addition of voting disk 90b3a7e49b004ff3bf58d1aa6eb91bbb. ONLINE 054f135ad3644fbabff3f058ef28ff75 (/dev/rdsk/c4t1d0s0) [DATA] 2. ONLINE 834ba35a0f6c4fe0bf04700b9edc2bb2 (/dev/rdsk/c4t2d0s0) [DATA] 3.cssdmonitor' on 'sol1' CRS-2672: Attempting to start 'ora. ONLINE 133e3427001c4f0dbfa957587866693a (/dev/rdsk/c4t4d0s0) [DATA] 5.gipcd' on 'sol1' CRS-2676: Start of 'ora.gipcd' on 'sol1' succeeded CRS-2676: Start of 'ora. Successful addition of voting disk 834ba35a0f6c4fe0bf04700b9edc2bb2.--------1. clscfg: -install mode specified Successfully accumulated necessary OCR keys. Successfully replaced voting disk group with +DATA.mdnsd' on 'sol1' CRS-2676: Start of 'ora.diskmon' on 'sol1' CRS-2676: Start of 'ora.mdnsd' on 'sol1' succeeded CRS-2672: Attempting to start 'ora. Creating OCR keys for user 'root'. Operation successful. Successful addition of voting disk 133e3427001c4f0dbfa957587866693a. Disk Group DATA created successfully. ONLINE 90b3a7e49b004ff3bf58d1aa6eb91bbb (/dev/rdsk/c4t5d0s0) [DATA] Located 5 voting disk(s). .0/grid# . ..dg' on 'sol1' CRS-2676: Start of 'ora.successful Adding Clusterware entries to inittab CRS-4402: The CSS daemon was started in exclusive mode but found an active CSS daemon on node sol1.0/grid Enter the full pathname of the local bin directory: [/usr/local/bin]: Creating /usr/local/bin directory. Creating /var/opt/oracle/oratab file.. and is terminating An active cluster was found during exclusive startup.asm' on 'sol1' succeeded CRS-2672: Attempting to start 'ora.2.2.DATA../root....asm' on 'sol1' CRS-2676: Start of 'ora.. number 1.2. restarting to join the cluster Configure Oracle Grid Infrastructure for a Cluster .CRS-2672: Attempting to start 'ora.0/grid# root@sol2:/u01/app/11.... succeeded root@sol1:/u01/app/11.sh Performing root user operation for Oracle 11g The following environment variables are set as: ORACLE_OWNER= grid ORACLE_HOME= /u01/app/11.DATA.2. Copying oraenv to /usr/local/bin . Now product-specific root actions will be performed.0/grid# At Final configuration step CLUVFY fails. Entries will be added to the /var/opt/oracle/oratab file as needed by Database Configuration Assistant when a database is created Finished running generic part of root script. Copying coraenv to /usr/local/bin .dg' on 'sol1' succeeded Configure Oracle Grid Infrastructure for a Cluster . succeeded root@sol2:/u01/app/11. Copying dbhome to /usr/local/bin .. Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_params Creating trace directory User ignored Prerequisites during installation OLR initialization ... 2.INFO: ERROR: INFO: PRVG-1101 : SCAN name "scan-sol" failed to resolve INFO: ERROR: INFO: PRVF-4657 : Name resolution setup check for "scan-sol" (IP address: 192. Exit the OUI.51) failed INFO: ERROR: INFO: PRVF-4664 : Found inconsistent name resolution entries for SCAN name "scan-sol" As this is the only error in the log I ignored it. The error is due to not properly defined SCAN in DNS. .168. LISTENER.2.0/grid/log/sol2$ crsctl check cluster -all ************************************************************** sol1: CRS-4537: Cluster Ready Services is online CRS-4529: Cluster Synchronization Services is online CRS-4533: Event Manager is online ************************************************************** sol2: CRS-4537: Cluster Ready Services is online CRS-4529: Cluster Synchronization Services is online CRS-4533: Event Manager is online ************************************************************** grid@sol2:/u01/app/11. grid@sol2:/u01/app/11.Verify that GI is properly installed and configured.DATA.lsnr ONLINE ONLINE sol1 ONLINE ONLINE sol2 ora.dg ONLINE ONLINE sol1 ONLINE ONLINE sol2 ora.2.0/grid/log/sol2$ crsctl stat res -t -------------------------------------------------------------------------------NAME TARGET STATE SERVER STATE_DETAILS -------------------------------------------------------------------------------Local Resources -------------------------------------------------------------------------------ora.asm ONLINE ONLINE sol1 Started . Ignore the E-mail notification and press Next to continue.evmd 1 ONLINE ONLINE sol2 ora.gipcd 1 ONLINE ONLINE sol2 ora.0/grid/log/sol2$ 22.vip 1 ONLINE ONLINE sol1 ora.mdnsd 1 ONLINE ONLINE sol2 grid@sol2:/u01/app/11.crf 1 ONLINE ONLINE sol2 ora.scan1.sol2. export AWT_TOOLKIT=XToolkit export DISPLAY=:0.cvu 1 ONLINE ONLINE sol1 ora.vip 1 ONLINE ONLINE sol1 ora.cluster_interconnect.net1.diskmon 1 OFFLINE OFFLINE ora.gpnpd 1 ONLINE ONLINE sol2 ora.lsnr 1 ONLINE ONLINE sol1 ora.ons ONLINE ONLINE sol1 ONLINE ONLINE sol2 -------------------------------------------------------------------------------Cluster Resources -------------------------------------------------------------------------------ora.sol1.crsd 1 ONLINE ONLINE sol2 ora. Oracle RDBMS installation only Log in as oracle user.cssd 1 ONLINE ONLINE sol2 ora.0 Run the installer. Set and export the following variables.2.ctssd 1 ONLINE ONLINE sol2 ACTIVE:0 ora.haip 1 ONLINE ONLINE sol2 ora. .2.asm 1 ONLINE ONLINE sol2 Started ora.network ONLINE ONLINE sol1 ONLINE ONLINE sol2 ora.0/grid/log/sol2$ crsctl stat res -t -init -------------------------------------------------------------------------------NAME TARGET STATE SERVER STATE_DETAILS -------------------------------------------------------------------------------Cluster Resources -------------------------------------------------------------------------------ora.cssdmonitor 1 ONLINE ONLINE sol2 ora.ONLINE ora.oc4j 1 ONLINE ONLINE sol1 ora.LISTENER_SCAN1.vip 1 ONLINE ONLINE sol2 grid@sol2:/u01/app/11.gsd ONLINE sol2 Started OFFLINE OFFLINE sol1 OFFLINE OFFLINE sol2 ora. Select skip software updates. . Select install software only. . Select both nodes and Oracle RAC database. . Select languages. . Select EE. . .Select location. .Select groups. .Examine the checks. network TYPE=ora.network./crsctl stat res ora.0/grid/bin# .type TARGET=ONLINE .net1.sol2.2.2.vip NAME=ora.0/grid/bin# Ignore examine Summary and run installation.network NAME=ora.vip NAME=ora.2.net1.0/grid/bin# . ONLINE STATE=ONLINE on sol1.cluster_vip_net1.sol2./crsctl stat res ora. root@sol1:/u01/app/11.sol1.vip TYPE=ora.cluster_vip_net1. ONLINE on sol2 root@sol1:/u01/app/11./crsctl stat res ora.type TARGET=ONLINE STATE=ONLINE on sol1 root@sol1:/u01/app/11.type TARGET=ONLINE STATE=ONLINE on sol2 root@sol1:/u01/app/11.Separate checks confirmed that I can continue.vip TYPE=ora.sol1.0/grid/bin# .2. . Run the scripts. Exit the installer. . In my case I created the following disk groups.23. . Create a cluster database using dbca If you intend to have separate disk groups as FRA create it. .While logged in as oracle user and properly set environment variables run dbca. Select RAC database. Select Create a database. . Select a template. . .Select Admin managed and both nodes. .Select OEM option. .Specify password(s). Error! . 1] Turned out permission issue.Look at: Dbca Does Not Show ASM Diskgroup Information [ID 1286434. eg: -rwsr-s—x Select a Disk Group.1] ASM Diskgroup Can Not Be Shown When Creating Database With DBCA [ID 1269734. Add setuid and setgid bit for oracle binary under $GRID_HOME/bin: $ chmod 6755 $GRID_HOME/bin/oracle The oracle binary for both GRID_HOME/bin and RDBMS ORACLE_HOME/bin should have 6755 permission. . . .Select FRA and Archiving options. .Select sample schemas. .Select parameters. Press Next. . .Select Create. Review the summary. Modify the .profile to include ORACLE_UNQNAME and start dbconsole. Modify passwords and take note of the OEM DC URL. At the end database is up and running. gjilevski@sol1:/u01/app/11.2.0/grid/bin$ ./crsctl stat res -t -------------------------------------------------------------------------------NAME TARGET STATE SERVER STATE_DETAILS -------------------------------------------------------------------------------Local Resources -------------------------------------------------------------------------------ora.DATA.dg ONLINE ONLINE sol1 ONLINE ONLINE sol2 ora.DATADG.dg ONLINE ONLINE sol1 ONLINE ONLINE sol2 ora.LISTENER.lsnr ONLINE ONLINE sol1 ONLINE ONLINE sol2 ora.asm ONLINE ONLINE sol1 Started ONLINE ONLINE sol2 Started ora.gsd OFFLINE OFFLINE sol1 OFFLINE OFFLINE sol2 ora.net1.network ONLINE ONLINE sol1 ONLINE ONLINE sol2 ora.ons ONLINE ONLINE sol1 ONLINE ONLINE sol2 -------------------------------------------------------------------------------Cluster Resources gj.sol2.oc4j 1 ONLINE ONLINE sol2 ora.d11g.vip 1 ONLINE ONLINE sol1 ora.LISTENER_SCAN1.3. Connected to: Oracle Database 11g Enterprise Edition Release 11.gj. OLAP. . Automatic Storage Management. Real Application Clusters. 2011.scan1.0 Production on Mon Feb 6 18:24:05 2012 Copyright (c) 1982. Oracle.0/grid/bin$ oracle@sol2:~$ sqlplus / as sysdba SQL*Plus: Release 11. All rights reserved.2.cvu 1 ONLINE ONLINE sol2 ora. Data Mining and Real Application Testing options SQL> select * from v$active_instances.0.2.vip 1 ONLINE ONLINE sol2 gjilevski@sol1:/u01/app/11.0 .db 1 ONLINE ONLINE sol1 Open 2 ONLINE ONLINE sol2 Open ora. INST_NUMBER ----------1 2 SQL> SQL> INST_NAME -----------------------------------------------------------sol1.lsnr 1 ONLINE ONLINE sol2 ora.com:D11G2 OEM status is as follows for the cluster database.64bit Production With the Partitioning.-------------------------------------------------------------------------------ora.0.com:D11G1 sol2.2.sol1.vip 1 ONLINE ONLINE sol2 ora.3. .OEM status is as follows for the cluster.