3 Cloning Oracle Clusterware to Create a Cluster

Step 1: Install Oracle Clusterware

Use the detailed instructions in the Oracle Grid Infrastructure Installation Guide to perform the following steps on the source node:

1. Install Oracle Clusterware 11g release 2 (11.2). This installation puts Oracle Cluster Registry (OCR) and the voting disk on Oracle Automatic Storage Management (Oracle ASM).

   Note:

   Either install and configure the grid infrastructure for a cluster or install just the Oracle Clusterware software, as described in your platform-specific Oracle Grid Infrastructure Installation Guide.

   To install and configure the grid infrastructure for a cluster, stop Oracle Clusterware before performing the cloning procedures. If you install just Oracle Clusterware, then you do not have to stop Oracle Clusterware.

2. Install any patches that are required (for example, 11.2.0.n), if necessary.

3. Apply one-off patches, if necessary.

   See Also:

   Oracle Grid Infrastructure Installation Guide for Oracle Clusterware installation instructions

Step 2: Shut Down Running Software

Before copying the source Oracle grid infrastructure home, shut down all of the services, databases, listeners, applications, Oracle Clusterware, and Oracle ASM instances that run on the node. Use the Server Control (SRVCTL) and Oracle Clusterware Control (CRSCTL) utilities to shut down these components.

Step 3: Create a Copy of the Oracle Grid Infrastructure Home

To keep the installed Oracle grid infrastructure home as a working home, make a full copy of the source Oracle grid infrastructure home for cloning. Because the Oracle grid infrastructure home contains files that are relevant only to the source node, you can optionally remove the unnecessary files from the copy.

Use one of the following methods to create a compressed copy of the Oracle grid infrastructure home.

Method 1: Create a copy of the Oracle grid infrastructure home and remove the unnecessary files from the copy:

1. On the source node, create a copy of the Oracle grid infrastructure home. To keep the installed Oracle grid infrastructure home as a working home, make a full copy of the source Oracle grid infrastructure home and remove the unnecessary files from the copy. For example, as root on Linux systems, run the cp command:

   # cp -prf Grid_home location_of_the_copy_of_Grid_home
   

2. Delete unnecessary files from the copy.

   The Oracle grid infrastructure home contains files that are relevant only to the source node, so you can remove the unnecessary files from the copy in the log, crs/init, and cdata directories. The following example for Linux and UNIX systems shows the commands to run to remove the unnecessary files from the copy of the Oracle grid infrastructure home:

   [root@node1 root]# cd /opt/oracle/product/11g/crs
   [root@node1 crs]# rm -rf /opt/oracle/product/11g/crs/log/host_name
   [root@node1 crs]# rm -rf crs/init
   [root@node1 crs]# rm -rf cdata
   [root@node1 crs]# rm -rf gpnp/*
   [root@node1 crs]# rm -rf network/admin/*.ora
   [root@node1 crs]# find . -name '*.ouibak' -exec rm {} \;
   [root@node1 crs]# find . -name '*.ouibak.1' -exec rm {} \;
   [root@node1 crs]# rm -rf root.sh*
   [root@node1 crs]# rm -rf Grid_home/inventory/ContentsXML/oraclehomeproperties.xml
   [root@node1 crs]# cd cfgtoollogs
   [root@node1 cfgtoollogs]# find . -type f -exec rm -f {} \;
   

3. Create a compressed copy of the previously copied Oracle grid infrastructure home using tar or gzip on Linux and UNIX systems. Ensure that the tool you use preserves the permissions and file timestamps. For example:

   On Linux and UNIX systems:

   [root@node1 root]# cd /opt/oracle/product/11g/crs/
   [root@node1 crs]# tar -zcvf /path_name/gridHome.tgz .
   

   In the example, the cd command changes the location to the Oracle grid infrastructure home, and the tar command creates the copy named crs11101.tgz. In the tar command, path_name represents the location of the file.

   On AIX or HPUX systems:

   tar cpf - . | compress -fv > temp_dir/gridHome.tar.Z
   

Method 2: Create a compressed copy of the Oracle grid infrastructure home using the -X option:

1. Create a file that lists the unnecessary files in the Oracle grid infrastructure home. For example, list the following file names, using the asterisk (*) wildcard, in a file called excludeFileList:

   ./opt/oracle/product/11g/crs/log/host_name
   ./opt/oracle/product/11g/crs/root.sh*
   ./opt/oracle/products/11g/crs/gpnp
   ./opt/oracle/products/11g/crs/network/admin/*.ora
   

2. Use the tar command or Winzip to create a compressed copy of the Oracle grid infrastructure home. For example, on Linux and UNIX systems, run the following command to archive and compress the source Oracle grid infrastructure home:

   tar cpfX - excludeFileList . | compress -fv > temp_dir/gridHome.tar.Z
   

   Note:

   Do not use the jar utility to copy and compress the Oracle grid infrastructure home.

Step 1: Prepare the New Cluster Nodes

On each destination node, perform the following preinstallation steps:

• Specify the kernel parameters

• Configure block devices for Oracle Clusterware devices

• Ensure that you have set the block device permissions correctly

• Use short, nondomain-qualified names for all of the names in the Hosts file

• Test whether the interconnect interfaces are reachable using the ping command

• Verify that the VIP addresses are not active at the start of the cloning process by using the ping command (the ping command of the VIP address must fail)

• Delete all files in the Grid_home/gpnp folder

  Note:

  If the Grid_home/gpnp folder contains any files, then cluster creation fails. All resources are added to the existing cluster, instead.

• Run CVU to verify your hardware and operating system environment

Refer to your platform-specific Oracle Clusterware installation guide for the complete preinstallation checklist.

Step 2: Deploy Oracle Clusterware on the Destination Nodes

Before you begin the cloning procedure that is described in this section, ensure that you have completed the prerequisite tasks to create a copy of the Oracle grid infrastructure home, as described in the section titled "Preparing the Oracle Grid Infrastructure Home for Cloning".

1. On each destination node, deploy the copy of the Oracle grid infrastructure home that you created in "Step 3: Create a Copy of the Oracle Grid Infrastructure Home", as follows:

   If you do not have a shared Oracle grid infrastructure home, then restore the copy of the Oracle grid infrastructure home on each node in the destination cluster. Use the equivalent directory structure as the directory structure that was used in the Oracle grid infrastructure home on the source node. Skip this step if you have a shared Oracle grid infrastructure home.

   For example, on Linux or UNIX systems, run commands similar to the following:

   [root@node1 root]# mkdir -p /u01/app/11.2.0/grid
   [root@node1 root]# cd /u01/app/11.2.0/grid
   [root@node1 crs]# tar -zxvf /path_name/gridHome.tgz
   

   In this example, path_name represents the directory structure in which you want to install the Oracle grid infrastructure home. Note that you can change the Grid Home location as part of the clone process

2. Change the ownership of all of the files to belong to the oracle:oinstall group, and create a directory for the Oracle Inventory. The following example shows the commands to do this on a Linux system:

   [root@node1 crs]# chown -R oracle:oinstall /u01/app/11.2.0/grid
   [root@node1 crs]# mkdir -p /u01/app/oraInventory
   [root@node1 crs]# chown oracle:oinstall /u01/app/oracle/oraInventory
   

3. It is important to remove any Oracle network files from the /u01/app/11.2.0/grid/network/admin directory on both nodes before continuing. For example, remove any tnsnames.ora, listener.ora or sqlnet.ora files.

   See Also:

   "Locating and Viewing Log Files Generated During Cloning"

Step 3: Run the clone.pl Script on Each Destination Node

To set up the new Oracle Clusterware environment, the clone.pl script requires you to provide several setup values for the script. You can provide the variable values by either supplying input on the command line when you run the clone.pl script, or by creating a file in which you can assign values to the cloning variables. The following discussions describe these options.

$ perl clone.pl -silent ORACLE_BASE=/u01/app/oracle ORACLE_HOME=
/u01/app/1.2.0/grid ORACLE_HOME_NAME=OraHome1Grid \
INVENTORY_LOCATION=/u01/app/oraInventory -O'"CLUSTER_NODES={node1, node2}"'
-O'"LOCAL_NODE=node1"'

#!/bin/sh
ORACLE_BASE=/u01/app/oracle
GRID_HOME=/u01/app/11.2.0/grid
THIS_NODE=`hostname -s`
 
E01=ORACLE_BASE=${ORACLE_BASE}
E02=ORACLE_HOME=${ORACLE_HOME}
E03=ORACLE_HOME_NAME=OraGridHome1
E04=INVENTORY_LOCATION=${ORACLE_BASE}/oraInventory
 
#C00="-O'-debug'"
C01="'-O\"CLUSTER_NODES={node1,node2}\"'"
C02="'-O\"LOCAL_NODE=${THIS_NODE}\"'"
 
perl ${GRID_HOME}/clone/bin/clone.pl -silent $E01 $E02 $E03 $E04 $C01 $C02 

Step 4: Prepare the crsconfig_params File on All Nodes

Prepare the /u01/app/11.2.0/grid/install/crsconfig_params file on all of the nodes in the cluster. You can copy the file from one node to all of the other nodes.

To use GNS, Oracle recommends that you add GNS to the cluster after your cloned cluster is running. Add GNS using the crsctl add gns command.

Step 5: Run the orainstRoot.sh Script on Each Node

In the Central Inventory directory on each destination node, run the orainstRoot.sh script as the operating system user that installed Oracle Clusterware. This script populates the /etc/oraInst.loc directory with the location of the central inventory. You can run the script on each node simultaneously. For example:

[root@node1 root]# /opt/oracle/oraInventory/orainstRoot.sh 

Ensure that the orainstRoot.sh script has completed on each destination node before proceeding to the next step.

Step 6: Run the GRID_HOME/root.sh and rootcrs.pl Scripts

On each destination node, run the GRID_HOME/root.sh script, followed by the rootcrs.pl script. You must run these scripts on only one node at a time. The following example is for a Linux or UNIX system. On the first node, run the following command:

[root@node1 root]# /u01/app/11.2.0/grid/root.sh 

Then run the following command:

[root@node1 root]# /u01/app/11.2.0/grid/perl/bin/perl \
-I/u01/app/11.2.0/grid/perl/lib -I/u01/app/11.2.0/grid/crs/install \
/u01/app/11.2.0/grid/crs/install/rootcrs.pl 

Ensure that the root.sh and rootcrs.pl scripts have completed on the first node before running them on the second node and subsequent nodes. On each subsequent node, run the following command:

[root@node2 root]# /u01/app/11.2.0/grid/root.sh 

Then run the following command:

[root@node1 root]# /u01/app/11.2.0/grid/perl/bin/perl \
-I/u01/app/11.2.0/grid/perl/lib -I/u01/app/11.2.0/grid/crs/install \ 
/u01/app/11.2.0/grid/crs/install/rootcrs.pl 

The root.sh script automatically configures the following node applications:

• Global Services Daemon (GSD)

• Oracle Notification Service (ONS)

• Enhanced ONS (eONS)

• Virtual IP (VIP) resources in the Oracle Cluster Registry (OCR)

• Single Client Access Name (SCAN) VIPs and SCAN listeners

• Oracle ASM

Step 7: Run the Configuration Assistants and the Oracle Cluster Verification Utility

1. At the end of the Oracle Clusterware installation, on each new node, manually run the configuration assistants and CVU. The following commands should be run from the first node only.

   [oracle] $ /u01/app/11.2.0/grid/bin/netca \
                   /orahome /u01/app/11.2.0/grid \
                   /orahnam OraGridHome1 \
                   /instype typical \
                   /inscomp client,oraclenet,javavm,server\
                   /insprtcl tcp \
                   /cfg local \
                   /authadp NO_VALUE \
                   /responseFile /u01/app/11.2.0/grid/network/install/netca_ typ.rsp \
                    /silent
   

2. To complete ASM configuration, run the following command:

   [oracle] $ /u01/app/11.2.0/grid/bin/asmca -silent -postConfigureASM
   -sysAsmPassword oracle -asmsnmpPassword oracle
   

3. If you a plan to run a pre-11g release 2 (11.2) database on this cluster, then you should run oifcfg as described in the Oracle Database 11g release 2 (11.2) documentation.

4. To use IPMI, use the crsctl command to configure IPMI on each node.

   See Also:

   "Configuration and Installation for IPMI Node Fencing" for information about configuring IPMI

5. Run a final CVU check to confirm that your grid infrastructure home has been cloned correctly.

   [oracle] $ /u01/app/11.2.0/grid/bin/cluvfy  stage -post  crsinst -n  node1,node2