环境说明：
OS:Redhat 7.5 DB:11.2.0.4.0 RAC ASM:NFS
磁盘初始化
检查磁盘大小
root@chenj-ch-05:/oradata3#df -h /oradata* Filesystem Size Used Avail Use% Mounted on 10.1.10.3:/test_ora19c_nfsB1 100G 710M 100G 1% /oradata1 10.1.10.3:/test_ora19c_01_nfsB1 30G 40M 30G 1% /oradata2 10.1.10.3:/test_ora19c_02_nfsB1 30G 38M 30G 1% /oradata3
检查磁盘挂载情况
root@chenj-ch-05:/oradata3# mount 10.1.10.3:/test_ora19c_nfsB1 on /oradata1 type nfs (rw,relatime,vers=3,rsize=32768,wsize=32768,namlen=255,acregmin=0,acregmax=0,acdirmin=0,acdirmax=0,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=10.1.10.3,mountvers=3,mountport=635,mountproto=tcp,local_lock=none,addr=10.1.10.3) 10.1.10.3:/test_ora19c_01_nfsB1 on /oradata2 type nfs (rw,relatime,vers=3,rsize=32768,wsize=32768,namlen=255,acregmin=0,acregmax=0,acdirmin=0,acdirmax=0,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=10.1.10.3,mountvers=3,mountport=635,mountproto=tcp,local_lock=none,addr=10.1.10.3) 10.1.10.3:/test_ora19c_02_nfsB1 on /oradata3 type nfs (rw,relatime,vers=3,rsize=32768,wsize=32768,namlen=255,acregmin=0,acregmax=0,acdirmin=0,acdirmax=0,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=10.1.10.3,mountvers=3,mountport=635,mountproto=tcp,local_lock=none,addr=10.1.10.3)
创建共享磁盘文件
dd if=/dev/zero of=/oradata1/test_asm/ocr1 bs=1024k count=2048 dd if=/dev/zero of=/oradata1/test_asm/ocr2 bs=1024k count=2048 dd if=/dev/zero of=/oradata1/test_asm/ocr3 bs=1024k count=2048 dd if=/dev/zero of=/oradata1/test_asm/data1 bs=1024k count=10240 dd if=/dev/zero of=/oradata1/test_asm/data2 bs=1024k count=10240 dd if=/dev/zero of=/oradata1/test_asm/data3 bs=1024k count=10240 dd if=/dev/zero of=/oradata1/test_asm/data4 bs=1024k count=10240 dd if=/dev/zero of=/oradata1/test_asm/data5 bs=1024k count=10240
修改权限
chown grid.oinstall /oradata1/test_asm -R chmod 660 /oradata1/test_asm/*
静默安装grid
grid@chenj-ch-05:/home/grid$nohup sh /oracle/tmp/grid/runInstaller -silent -noconfig -waitForCompletion -ignorePrereq -ignoreInternalDriverError -showProgress -responseFile /home/grid/grid_install.rsp > /home/grid/install_0304.log &
静默安装grid日志
Starting Oracle Universal Installer... Checking Temp space: must be greater than 120 MB. Actual 3949 MB Passed Checking swap space: must be greater than 150 MB. Actual 16338 MB Passed Preparing to launch Oracle Universal Installer from /tmp/OraInstall2022-03-04_02-34-22PM. Please wait ...[WARNING] [INS-30011] The SYS password entered does not conform to the Oracle recommended standards. CAUSE: Oracle recommends that the password entered should be at least 8 characters in length, contain at least 1 uppercase character, 1 lower case character and 1 digit [0-9]. ACTION: Provide a password that conforms to the Oracle recommended standards. [WARNING] [INS-30011] The ASMSNMP password entered does not conform to the Oracle recommended standards. CAUSE: Oracle recommends that the password entered should be at least 8 characters in length, contain at least 1 uppercase character, 1 lower case character and 1 digit [0-9]. ACTION: Provide a password that conforms to the Oracle recommended standards. [WARNING] [INS-32016] The selected Oracle home contains directories or files. CAUSE: The selected Oracle home contained directories or files. ACTION: To start with an empty Oracle home, either remove its contents or choose another location. You can find the log of this install session at: /oracle/oraINventory/logs/installActions2022-03-04_02-34-22PM.log Prepare in progress. .................................................. 9% Done. Prepare successful. Copy files in progress. .................................................. 15% Done. .................................................. 20% Done. .................................................. 25% Done. .................................................. 30% Done. .................................................. 35% Done. .................................................. 40% Done. .................................................. 45% Done. ........................................ Copy files successful. Link binaries in progress. Link binaries successful. .................................................. 62% Done. Setup files in progress. Setup files successful. .................................................. 76% Done. Perform remote operations in progress. .................................................. 89% Done. Perform remote operations successful. The installation of Oracle Grid Infrastructure 11g was successful. Please check '/oracle/oraINventory/logs/silentInstall2022-03-04_02-34-22PM.log' for more details. .................................................. 94% Done. Execute Root Scripts in progress. As a root user, execute the following script(s): 1. /oracle/oraINventory/orainstRoot.sh 2. /oracle/grid/product/11.2/root.sh Execute /oracle/oraINventory/orainstRoot.sh on the following nodes: [chenj-ch-05, chenj-ch-06] Execute /oracle/grid/product/11.2/root.sh on the following nodes: [chenj-ch-05, chenj-ch-06] .................................................. 100% Done. Execute Root Scripts successful. As install user, execute the following script to complete the configuration. 1. /oracle/grid/product/11.2/cfgtoollogs/configToolAllCommands RESPONSE_FILE=Note: 1. This script must be run on the same host from where installer was run. 2. This script needs a small password properties file for configuration assistants that require passwords (refer to install guide documentation). Successfully Setup Software.
节点1 root.sh执行日志
The following environment variables are set as: ORACLE_OWNER= grid ORACLE_HOME= /oracle/grid/product/11.2 Copying dbhome to /usr/local/bin ... Copying oraenv to /usr/local/bin ... Copying coraenv to /usr/local/bin ... Creating /etc/oratab file... Entries will be added to the /etc/oratab file as needed by Database Configuration Assistant when a database is created Finished running generic part of root script. Now product-specific root actions will be performed. Using configuration parameter file: /oracle/grid/product/11.2/crs/install/crsconfig_params Creating trace directory User ignored Prerequisites during installation Installing Trace File Analyzer OLR initialization - successful root wallet root wallet cert root cert export peer wallet profile reader wallet pa wallet peer wallet keys pa wallet keys peer cert request pa cert request peer cert pa cert peer root cert TP profile reader root cert TP pa root cert TP peer pa cert TP pa peer cert TP profile reader pa cert TP profile reader peer cert TP peer user cert pa user cert Adding Clusterware entries to inittab CRS-2672: Attempting to start 'ora.mdnsd' on 'chenj-ch-05' CRS-2676: Start of 'ora.mdnsd' on 'chenj-ch-05' succeeded CRS-2672: Attempting to start 'ora.gpnpd' on 'chenj-ch-05' CRS-2676: Start of 'ora.gpnpd' on 'chenj-ch-05' succeeded CRS-2672: Attempting to start 'ora.cssdmonitor' on 'chenj-ch-05' CRS-2672: Attempting to start 'ora.gipcd' on 'chenj-ch-05' CRS-2676: Start of 'ora.cssdmonitor' on 'chenj-ch-05' succeeded CRS-2676: Start of 'ora.gipcd' on 'chenj-ch-05' succeeded CRS-2672: Attempting to start 'ora.cssd' on 'chenj-ch-05' CRS-2672: Attempting to start 'ora.diskmon' on 'chenj-ch-05' CRS-2676: Start of 'ora.diskmon' on 'chenj-ch-05' succeeded CRS-2676: Start of 'ora.cssd' on 'chenj-ch-05' succeeded ASM created and started successfully. Disk Group GD_TEST_OCR created successfully. clscfg: -install mode specified Successfully accumulated necessary OCR keys. Creating OCR keys for user 'root', privgrp 'root'.. Operation successful. CRS-4256: Updating the profile Successful addition of voting disk 2c78cf2ab1ac4f99bfbeafadec4caa01. Successful addition of voting disk 003f25c63a7e4f15bfe6ce115377e75d. Successful addition of voting disk 083b82cd51cc4fb0bf44ba96af95c99c. Successfully replaced voting disk group with +GD_TEST_OCR. CRS-4256: Updating the profile CRS-4266: Voting file(s) successfully replaced ## STATE File Universal Id File Name Disk group -- ----- ----------------- --------- --------- 1. ONLINE 2c78cf2ab1ac4f99bfbeafadec4caa01 (/oradata1/test_asm/ocr1) [GD_TEST_OCR] 2. ONLINE 003f25c63a7e4f15bfe6ce115377e75d (/oradata1/test_asm/ocr2) [GD_TEST_OCR] 3. ONLINE 083b82cd51cc4fb0bf44ba96af95c99c (/oradata1/test_asm/ocr3) [GD_TEST_OCR] Located 3 voting disk(s). CRS-2672: Attempting to start 'ora.asm' on 'chenj-ch-05' CRS-2676: Start of 'ora.asm' on 'chenj-ch-05' succeeded CRS-2672: Attempting to start 'ora.GD_TEST_OCR.dg' on 'chenj-ch-05' CRS-2676: Start of 'ora.GD_TEST_OCR.dg' on 'chenj-ch-05' succeeded FirstNode configuration failed at /oracle/grid/product/11.2/crs/install/crsconfig_lib.pm line 9379. /oracle/grid/product/11.2/perl/bin/perl -I/oracle/grid/product/11.2/perl/lib -I/oracle/grid/product/11.2/crs/install /oracle/grid/product/11.2/crs/install/rootcrs.pl execution failed
执行节点2 root.sh
完整日志：
Creating /etc/oratab file... Entries will be added to the /etc/oratab file as needed by Database Configuration Assistant when a database is created Finished running generic part of root script. Now product-specific root actions will be performed. Using configuration parameter file: /oracle/grid/product/11.2/crs/install/crsconfig_params Creating trace directory User ignored Prerequisites during installation Installing Trace File Analyzer OLR initialization - successful Adding Clusterware entries to inittab ohasd failed to start Failed to start the Clusterware. Last 20 lines of the alert log follow: 2022-03-04 15:24:05.980: [client(55011)]CRS-2101:The OLR was formatted using version 3. CRS-4402: The CSS daemon was started in exclusive mode but found an active CSS daemon on node chenj-ch-05, number 1, and is terminating An active cluster was found during exclusive startup, restarting to join the cluster Configure Oracle Grid Infrastructure for a Cluster ... succeeded
查看crs
grid@chenj-ch-06:/home/grid$crsctl stat res -t -------------------------------------------------------------------------------- NAME TARGET STATE SERVER STATE_DETAILS -------------------------------------------------------------------------------- Local Resources -------------------------------------------------------------------------------- ora.GD_TEST_OCR.dg ONLINE ONLINE chenj-ch-05 ONLINE ONLINE chenj-ch-06 ora.asm ONLINE ONLINE chenj-ch-05 Started ONLINE ONLINE chenj-ch-06 Started ora.gsd OFFLINE OFFLINE chenj-ch-05 OFFLINE OFFLINE chenj-ch-06 ora.net1.network ONLINE ONLINE chenj-ch-05 ONLINE ONLINE chenj-ch-06 ora.ons ONLINE ONLINE chenj-ch-05 ONLINE ONLINE chenj-ch-06 -------------------------------------------------------------------------------- Cluster Resources -------------------------------------------------------------------------------- ora.LISTENER_SCAN1.lsnr 1 ONLINE ONLINE chenj-ch-05 ora.cvu 1 OFFLINE OFFLINE ora.chenj-ch-05.vip 1 ONLINE ONLINE chenj-ch-05 ora.chenj-ch-06.vip 1 ONLINE ONLINE chenj-ch-06 ora.oc4j 1 ONLINE ONLINE chenj-ch-06 ora.scan1.vip 1 ONLINE ONLINE chenj-ch-05
节点1：执行configToolAllCommands
生成rsp文件
su - grid vi /home/grid/cfgrsp.properties oracle.assistants.asm|S_ASMPASSWORD=oracle oracle.assistants.asm|S_ASMMONITORPASSWORD=oracle oracle.crs|S_BMCPASSWORD=oracle
开始执行
/oracle/ggrid@chenj-ch-05:/home/grid$/oracle/grid/product/11.2/cfgtoollogs/configToolAllCommands RESPONSE_FILE=/home/grid/cfgrsp.properties Setting the invPtrLoc to /oracle/grid/product/11.2/oraInst.loc perform - mode is starting for action: configure Mar 4, 2022 3:40:46 PM oracle.install.driver.oui.config.GenericInternalPlugIn invoke INFO: Executing ASMCA Mar 4, 2022 3:40:46 PM oracle.install.driver.oui.config.GenericInternalPlugIn invoke INFO: Command /oracle/grid/product/11.2/bin/asmca -silent -postConfigureASM -oui_internal Mar 4, 2022 3:40:46 PM oracle.install.driver.oui.config.GenericInternalPlugIn handleProcess INFO: ... GenericInternalPlugIn.handleProcess() entered. Mar 4, 2022 3:40:46 PM oracle.install.driver.oui.config.GenericInternalPlugIn handleProcess INFO: ... GenericInternalPlugIn: getting configAssistantParmas. Mar 4, 2022 3:40:46 PM oracle.install.driver.oui.config.GenericInternalPlugIn handleProcess INFO: ... GenericInternalPlugIn: checking secretArguments. Mar 4, 2022 3:40:46 PM oracle.install.driver.oui.config.GenericInternalPlugIn handleProcess INFO: ... GenericInternalPlugIn: starting read loop. Mar 4, 2022 3:40:49 PM oracle.install.driver.oui.config.GenericInternalPlugIn handleProcess INFO: Read: SYS_PASSWORD_PROMPT Mar 4, 2022 3:40:49 PM oracle.install.driver.oui.config.GenericInternalPlugIn handleProcess INFO: Processing: SYS_PASSWORD_PROMPT for argument tag -sysAsmPassword Mar 4, 2022 3:40:49 PM oracle.install.driver.oui.config.GenericInternalPlugIn handleProcess INFO: Read: ASMSNMP_PASSWORD_PROMPT Mar 4, 2022 3:40:49 PM oracle.install.driver.oui.config.GenericInternalPlugIn handleProcess INFO: Processing: ASMSNMP_PASSWORD_PROMPT for argument tag -asmMonitorPassword Mar 4, 2022 3:40:49 PM oracle.install.driver.oui.config.GenericInternalPlugIn handleProcess INFO: End of argument passing to stdin Mar 4, 2022 3:40:50 PM oracle.install.driver.oui.config.GenericInternalPlugIn handleProcess INFO: Read: Mar 4, 2022 3:40:50 PM oracle.install.driver.oui.config.GenericInternalPlugIn handleProcess INFO: Read: PostConfiguration completed successfully Mar 4, 2022 3:40:50 PM oracle.install.driver.oui.config.GenericInternalPlugIn handleProcess INFO: Read: perform - mode finished for action: configure You can see the log file: /oracle/grid/product/11.2/cfgtoollogs/oui/configActions2022-03-04_03-39-48-PM.logrid/product/11.2/cfgtoollogs/configToolAllCommands RESPONSE_FILE=/home/grid/cfgrsp.properties
查看执行日志
实际调用如下命令：
/oracle/grid/product/11.2/oui/bin/runInstaller -nowait -noconsole -waitforcompletion -ignoreSysPrereqs -updateNodeList -silent CRS=true "CLUSTER_NODES={chenj-ch-05,chenj-ch-06}" ORACLE_HOME=/oracle/grid/product/11.2
日志：
vi /oracle/grid/product/11.2/cfgtoollogs/oui/configActions2022-03-04_03-39-48-PM.logrid/product/11.2/cfgtoollogs/configToolAllCommands RESPONSE_FILE=/home/grid/cfgrsp.properties
###################################################
The action configuration is performing
------------------------------------------------------
The plug-in Update Inventory is running
/oracle/grid/product/11.2/oui/bin/runInstaller -nowait -noconsole -waitforcompletion -ignoreSysPrereqs -updateNodeList -silent CRS=true "CLUSTER_NODES={chenj-ch-05,chenj-ch-06}" ORACLE_HOME=/oracle/grid/product/11.2
Starting Oracle Universal Installer...
Checking swap space: must be greater than 500 MB. Actual 16338 MB Passed
The inventory pointer is located at /etc/oraInst.loc
The inventory is located at /oracle/oraINventory
The plug-in Update Inventory has successfully been performed
------------------------------------------------------
------------------------------------------------------
The plug-in Oracle Net Configuration Assistant is running
Parsing command line arguments:
Parameter "orahome" = /oracle/grid/product/11.2
Parameter "orahnam" = Ora11g_gridinfrahome1
Parameter "instype" = typical
Parameter "inscomp" = client,oraclenet,javavm,server,ano
Parameter "insprtcl" = tcp
Parameter "cfg" = local
Parameter "authadp" = NO_VALUE
Parameter "responsefile" = /oracle/grid/product/11.2/network/install/netca_typ.rsp
Parameter "silent" = true
Parameter "silent" = true
Done parsing command line arguments.
Oracle Net Services Configuration:
Profile configuration complete.
Profile configuration complete.
chenj-ch-05...
chenj-ch-06...
Oracle Net Listener Startup:
Listener started successfully.
Listener configuration complete
......
Post-check for cluster services setup was unsuccessful on all the nodes.
The plug-in Oracle Cluster Verification Utility has failed its perform method
------------------------------------------------------
The action configuration has failed its perform method
###################################################
Oracle软件安装
root@chenj-ch-05:/oracle#cp -r oraINventory oraINventory_bak
静默安装
nohup sh /oracle/tmp/database/runInstaller -silent -noconfig -waitForCompletion -ignorePrereq -ignoreInternalDriverError -showProgress -responseFile /home/oracle/rsp/db_install.rsp > /home/oracle/install_0304.log &
安装日志
Starting Oracle Universal Installer... Checking Temp space: must be greater than 120 MB. Actual 3949 MB Passed Checking swap space: must be greater than 150 MB. Actual 16338 MB Passed Preparing to launch Oracle Universal Installer from /tmp/OraInstall2022-03-04_03-47-30PM. Please wait ... You can find the log of this install session at: /oracle/oraINventory/logs/installActions2022-03-04_03-47-30PM.log Prepare in progress. .................................................. 9% Done. Prepare successful. Copy files in progress. .................................................. 15% Done. .................................................. 20% Done. .................................................. 25% Done. .................................................. 30% Done. .................................................. 36% Done. .................................................. 41% Done. .................................................. 46% Done. .................................................. 51% Done. .................................................. 56% Done. Copy files successful. .......... Link binaries in progress. Link binaries successful. .................................................. 77% Done. Setup files in progress. .................................................. 94% Done. Setup files successful. The installation of Oracle Database 11g was successful. Please check '/oracle/oraINventory/logs/silentInstall2022-03-04_03-47-30PM.log' for more details. Execute Root Scripts in progress. As a root user, execute the following script(s): 1. /oracle/db/product/11.2/root.sh Execute /oracle/db/product/11.2/root.sh on the following nodes: [chenj-ch-05, chenj-ch-06] .................................................. 100% Done. Execute Root Scripts successful. Successfully Setup Software.
执行脚本
root@chenj-ch-05:/root#/oracle/db/product/11.2/root.sh Check /oracle/db/product/11.2/install/root_chenj-ch-05_2022-03-04_15-56-21.log for the output of root script root@chenj-ch-05:/root#vi /oracle/db/product/11.2/install/root_chenj-ch-05_2022-03-04_15-56-21.log root@chenj-ch-06:/oracle#/oracle/db/product/11.2/root.sh Check /oracle/db/product/11.2/install/root_chenj-ch-06_2022-03-04_15-57-10.log for the output of root script root@chenj-ch-06:/oracle#vi /oracle/db/product/11.2/install/root_chenj-ch-06_2022-03-04_15-57-10.log
日志如下：
Performing root user operation for Oracle 11g The following environment variables are set as: ORACLE_OWNER= oracle ORACLE_HOME= /oracle/db/product/11.2 Copying dbhome to /usr/local/bin ... Copying oraenv to /usr/local/bin ... Copying coraenv to /usr/local/bin ... Entries will be added to the /etc/oratab file as needed by Database Configuration Assistant when a database is created Finished running generic part of root script. Now product-specific root actions will be performed. Finished product-specific root actions. Finished product-specific root actions.
检查安装
oracle@chenj-ch-05:/home/oracle$sqlplus / as sysdba SQL*Plus: Release 11.2.0.4.0 Production on Fri Mar 4 15:55:38 2022 Copyright (c) 1982, 2013, Oracle. All rights reserved. Connected to an idle instance. SQL>
创建磁盘组
SQL> select name,total_mb,free_mb from v$asm_diskgroup; NAME TOTAL_MB FREE_MB ------------------------------ ---------- ---------- GD_TEST_OCR 6144 5218 SQL> CREATE DISKGROUP GD_TEST_DATA EXTERNAL REDUNDANCY DISK '/oradata1/test_asm/data1','/oradata1/test_asm/data2','/oradata1/test_asm/data3'; SQL> select name,state,type from v$asm_diskgroup; NAME STATE TYPE ------------------------------ ----------- ------ GD_TEST_OCR MOUNTED NORMAL GD_TEST_DATA MOUNTED EXTERN
节点2：
SQL> select name,state,type from v$asm_diskgroup; NAME STATE TYPE ------------------------------ ----------- ------ GD_TEST_OCR MOUNTED NORMAL GD_TEST_DATA DISMOUNTED SQL> alter diskgroup GD_TEST_DATA mount; Diskgroup altered. SQL> select name,state,type from v$asm_diskgroup; NAME STATE TYPE ------------------------------ ----------- ------ GD_TEST_OCR MOUNTED NORMAL GD_TEST_DATA MOUNTED EXTERN
DBCA建库
oracle@chenj-ch-05:/home/oracle/rsp$dbca -silent -createdatabase -responseFile /home/oracle/rsp/dbca.rsp
日志如下：
Cleaning up failed steps 4% complete Copying database files 5% complete 6% complete 7% complete 13% complete 19% complete 24% complete 30% complete 33% complete Creating and starting Oracle instance 35% complete 39% complete 43% complete 46% complete 47% complete 48% complete 50% complete 52% complete Creating cluster database views 54% complete 71% complete Completing Database Creation 74% complete 77% complete 85% complete 94% complete 100% complete Look at the log file "/oracle/db/cfgtoollogs/dbca/test/test5.log" for further details.
RAC检查
oracle@chenj-ch-05:/home/oracle/rsp$srvctl config database test oracle@chenj-ch-05:/home/oracle/rsp$srvctl status database -d test Instance test1 is running on node chenj-ch-05 Instance test2 is running on node chenj-ch-06 grid@chenj-ch-05:/home/grid$crsctl stat res -t -------------------------------------------------------------------------------- NAME TARGET STATE SERVER STATE_DETAILS -------------------------------------------------------------------------------- Local Resources -------------------------------------------------------------------------------- ora.GD_TEST_DATA.dg ONLINE ONLINE chenj-ch-05 ONLINE ONLINE chenj-ch-06 ora.GD_TEST_OCR.dg ONLINE ONLINE chenj-ch-05 ONLINE ONLINE chenj-ch-06 ora.LISTENER.lsnr ONLINE ONLINE chenj-ch-05 ONLINE ONLINE chenj-ch-06 ora.asm ONLINE ONLINE chenj-ch-05 Started ONLINE ONLINE chenj-ch-06 Started ora.gsd OFFLINE OFFLINE chenj-ch-05 OFFLINE OFFLINE chenj-ch-06 ora.net1.network ONLINE ONLINE chenj-ch-05 ONLINE ONLINE chenj-ch-06 ora.ons ONLINE ONLINE chenj-ch-05 ONLINE ONLINE chenj-ch-06 -------------------------------------------------------------------------------- Cluster Resources -------------------------------------------------------------------------------- ora.LISTENER_SCAN1.lsnr 1 ONLINE ONLINE chenj-ch-05 ora.cvu 1 OFFLINE OFFLINE ora.chenj-ch-05.vip 1 ONLINE ONLINE chenj-ch-05 ora.chenj-ch-06.vip 1 ONLINE ONLINE chenj-ch-06 ora.oc4j 1 ONLINE ONLINE chenj-ch-06 ora.scan1.vip 1 ONLINE ONLINE chenj-ch-05 ora.test.db 1 ONLINE ONLINE chenj-ch-05 Open 2 ONLINE ONLINE chenj-ch-06 Open
数据测试
节点1：
SQL> create user cjc identified by a; User created. SQL> grant dba to cjc; Grant succeeded. SQL> conn cjc/a Connected. SQL> SQL> SQL> create table t1 as select level as id from dual connect by level <=10; Table created. SQL> select * from t1; ID ---------- 1 2 3 4 5 6 7 8 9 10 10 rows selected.
节点2：
oracle@chenj-ch-06:/home/oracle$sqlplus cjc/a SQL*Plus: Release 11.2.0.4.0 Production on Fri Mar 4 16:17:40 2022 Copyright (c) 1982, 2013, Oracle. All rights reserved. Connected to: Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP, Data Mining and Real Application Testing options SQL> select * from t1; ID ---------- 1 2 3 4 5 6 7 8 9 10 10 rows selected.
重启：
oracle@chenj-ch-05:/home/oracle$srvctl stop database -d test oracle@chenj-ch-05:/home/oracle$srvctl start database -d test
重启crs
停crs
root@chenj-ch-06:/oracle/grid/product/11.2/bin#./crsctl stop crs CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'chenj-ch-06' CRS-2673: Attempting to stop 'ora.crsd' on 'chenj-ch-06' CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'chenj-ch-06' CRS-2673: Attempting to stop 'ora.LISTENER_SCAN1.lsnr' on 'chenj-ch-06' CRS-2673: Attempting to stop 'ora.GD_TEST_OCR.dg' on 'chenj-ch-06' CRS-2673: Attempting to stop 'ora.test.db' on 'chenj-ch-06' CRS-2673: Attempting to stop 'ora.oc4j' on 'chenj-ch-06' CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'chenj-ch-06' CRS-2677: Stop of 'ora.LISTENER_SCAN1.lsnr' on 'chenj-ch-06' succeeded CRS-2673: Attempting to stop 'ora.scan1.vip' on 'chenj-ch-06' CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'chenj-ch-06' succeeded CRS-2673: Attempting to stop 'ora.chenj-ch-06.vip' on 'chenj-ch-06' CRS-2677: Stop of 'ora.test.db' on 'chenj-ch-06' succeeded CRS-2673: Attempting to stop 'ora.GD_TEST_DATA.dg' on 'chenj-ch-06' CRS-2677: Stop of 'ora.chenj-ch-06.vip' on 'chenj-ch-06' succeeded CRS-2672: Attempting to start 'ora.chenj-ch-06.vip' on 'chenj-ch-05' CRS-2677: Stop of 'ora.GD_TEST_DATA.dg' on 'chenj-ch-06' succeeded CRS-2677: Stop of 'ora.scan1.vip' on 'chenj-ch-06' succeeded CRS-2672: Attempting to start 'ora.scan1.vip' on 'chenj-ch-05' CRS-2677: Stop of 'ora.oc4j' on 'chenj-ch-06' succeeded CRS-2672: Attempting to start 'ora.oc4j' on 'chenj-ch-05' CRS-2676: Start of 'ora.chenj-ch-06.vip' on 'chenj-ch-05' succeeded CRS-2676: Start of 'ora.scan1.vip' on 'chenj-ch-05' succeeded CRS-2672: Attempting to start 'ora.LISTENER_SCAN1.lsnr' on 'chenj-ch-05' CRS-2676: Start of 'ora.LISTENER_SCAN1.lsnr' on 'chenj-ch-05' succeeded CRS-2677: Stop of 'ora.GD_TEST_OCR.dg' on 'chenj-ch-06' succeeded CRS-2673: Attempting to stop 'ora.asm' on 'chenj-ch-06' CRS-2677: Stop of 'ora.asm' on 'chenj-ch-06' succeeded CRS-2674: Start of 'ora.oc4j' on 'chenj-ch-05' failed CRS-2679: Attempting to clean 'ora.oc4j' on 'chenj-ch-05' CRS-2681: Clean of 'ora.oc4j' on 'chenj-ch-05' succeeded CRS-2673: Attempting to stop 'ora.ons' on 'chenj-ch-06' CRS-2677: Stop of 'ora.ons' on 'chenj-ch-06' succeeded CRS-2673: Attempting to stop 'ora.net1.network' on 'chenj-ch-06' CRS-2677: Stop of 'ora.net1.network' on 'chenj-ch-06' succeeded CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'chenj-ch-06' has completed CRS-2677: Stop of 'ora.crsd' on 'chenj-ch-06' succeeded CRS-2673: Attempting to stop 'ora.mdnsd' on 'chenj-ch-06' CRS-2673: Attempting to stop 'ora.crf' on 'chenj-ch-06' CRS-2673: Attempting to stop 'ora.ctssd' on 'chenj-ch-06' CRS-2673: Attempting to stop 'ora.evmd' on 'chenj-ch-06' CRS-2673: Attempting to stop 'ora.asm' on 'chenj-ch-06' CRS-2677: Stop of 'ora.mdnsd' on 'chenj-ch-06' succeeded CRS-2677: Stop of 'ora.evmd' on 'chenj-ch-06' succeeded CRS-2677: Stop of 'ora.crf' on 'chenj-ch-06' succeeded CRS-2677: Stop of 'ora.asm' on 'chenj-ch-06' succeeded CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'chenj-ch-06' CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'chenj-ch-06' succeeded CRS-2677: Stop of 'ora.ctssd' on 'chenj-ch-06' succeeded CRS-2673: Attempting to stop 'ora.cssd' on 'chenj-ch-06' CRS-2677: Stop of 'ora.cssd' on 'chenj-ch-06' succeeded CRS-2673: Attempting to stop 'ora.gipcd' on 'chenj-ch-06' CRS-2677: Stop of 'ora.gipcd' on 'chenj-ch-06' succeeded CRS-2673: Attempting to stop 'ora.gpnpd' on 'chenj-ch-06' CRS-2677: Stop of 'ora.gpnpd' on 'chenj-ch-06' succeeded CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'chenj-ch-06' has completed CRS-4133: Oracle High Availability Services has been stopped.
启动crs过程
root@chenj-ch-06:/oracle/grid/product/11.2/bin#./crsctl start crs 2022-03-04 16:50:50.088: [ohasd(69993)]CRS-2112:The OLR service started on node chenj-ch-06. 2022-03-04 16:50:50.093: [ohasd(69993)]CRS-1301:Oracle High Availability Service started on node chenj-ch-06. 2022-03-04 16:50:50.093: [ohasd(69993)]CRS-8017:location: /etc/oracle/lastgasp has 2 reboot advisory log files, 0 were announced and 0 errors occurred 2022-03-04 16:50:53.370: [/oracle/grid/product/11.2/bin/orarootagent.bin(70033)]CRS-2302:Cannot get GPnP profile. Error CLSGPNP_NO_DAEMON (GPNPD daemon is not running). 2022-03-04 16:50:57.737: [gpnpd(70128)]CRS-2328:GPNPD started on node chenj-ch-06. 2022-03-04 16:51:00.056: [cssd(70196)]CRS-1713:CSSD daemon is started in clustered mode 2022-03-04 16:51:01.902: [ohasd(69993)]CRS-2767:Resource state recovery not attempted for 'ora.diskmon' as its target state is OFFLINE 2022-03-04 16:51:01.902: [ohasd(69993)]CRS-2769:Unable to failover resource 'ora.diskmon'. 2022-03-04 16:51:20.542: [cssd(70196)]CRS-1707:Lease acquisition for node chenj-ch-06 number 2 completed 2022-03-04 16:51:21.814: [cssd(70196)]CRS-1605:CSSD voting file is online: /oradata1/test_asm/ocr1; details in /oracle/grid/product/11.2/log/chenj-ch-06/cssd/ocssd.log. 2022-03-04 16:51:21.858: [cssd(70196)]CRS-1605:CSSD voting file is online: /oradata1/test_asm/ocr2; details in /oracle/grid/product/11.2/log/chenj-ch-06/cssd/ocssd.log. 2022-03-04 16:51:21.898: [cssd(70196)]CRS-1605:CSSD voting file is online: /oradata1/test_asm/ocr3; details in /oracle/grid/product/11.2/log/chenj-ch-06/cssd/ocssd.log. 2022-03-04 16:51:25.529: [cssd(70196)]CRS-1601:CSSD Reconfiguration complete. Active nodes are chenj-ch-05 chenj-ch-06 . 2022-03-04 16:51:27.063: [ctssd(70307)]CRS-2403:The Cluster Time Synchronization Service on host chenj-ch-06 is in observer mode. 2022-03-04 16:51:27.309: [ctssd(70307)]CRS-2407:The new Cluster Time Synchronization Service reference node is host chenj-ch-05. 2022-03-04 16:51:27.310: [ctssd(70307)]CRS-2401:The Cluster Time Synchronization Service started on host chenj-ch-06. 2022-03-04 16:51:29.059: [ohasd(69993)]CRS-2767:Resource state recovery not attempted for 'ora.diskmon' as its target state is OFFLINE 2022-03-04 16:51:29.059: [ohasd(69993)]CRS-2769:Unable to failover resource 'ora.diskmon'. 2022-03-04 16:51:47.564: [crsd(70414)]CRS-1012:The OCR service started on node chenj-ch-06. 2022-03-04 16:51:47.572: [evmd(70329)]CRS-1401:EVMD started on node chenj-ch-06. 2022-03-04 16:51:49.614: [crsd(70414)]CRS-1201:CRSD started on node chenj-ch-06. 2022-03-04 16:51:57.716: [client(70701)]CRS-4743:File /oracle/grid/product/11.2/oc4j/j2ee/home/OC4J_DBWLM_config/system-jazn-data.xml was updated from OCR(Size: 13365(New), 13378(Old) bytes)
查看会话
SQL> select inst_id,count(*) from gv$session group by inst_id; INST_ID COUNT(*) ---------- ---------- 1 41 2 39
排错过程
问题一： crsconfig_lib.pm line 9379
报错：
FirstNode configuration failed at /oracle/grid/product/11.2/crs/install/crsconfig_lib.pm line 9379. /oracle/grid/product/11.2/perl/bin/perl -I/oracle/grid/product/11.2/perl/lib -I/oracle/grid/product/11.2/crs/install /oracle/grid/product/11.2/crs/install/rootcrs.pl execution failed
手动执行：
/oracle/grid/product/11.2/perl/bin/perl -I/oracle/grid/product/11.2/perl/lib -I/oracle/grid/product/11.2/crs/install /oracle/grid/product/11.2/crs/install/rootcrs.pl ... PRCR-1079 : Failed to start resource ora.LISTENER_SCAN1.lsnr CRS-5702: Resource 'ora.LISTENER_SCAN1.lsnr' is already running on 'chenj-ch-05' FirstNode configuration failed at /oracle/grid/product/11.2/crs/install/crsconfig_lib.pm line 9379.
检查crsd.log
grid@chenj-ch-05:/oracle/grid/product/11.2/log/chenj-ch-05/crsd$vi crsd.log
2022-03-04 15:06:08.395: [UiServer][1295947520]{1:56663:491} Container [ Name: ORDER
MESSAGE:
TextMessage[CRS-2674: Start of 'ora.oc4j' on 'chenj-ch-05' failed]
MSGTYPE:
TextMessage[1]
OBJID:
TextMessage[ora.oc4j]
WAIT:
TextMessage[0]
]
...
2022-03-04 15:11:15.902: [ CRSPE][1298048768]{1:56663:598} Op 0x7fddfc1266d0 has 3 WOs
2022-03-04 15:11:15.903: [ CRSPE][1298048768]{1:56663:598} ICE has queued an operation. Details: Operation [START of [ora.oc4j 1 1] on [chenj-ch-05] : local=0, unplanned=00x7fddfc1266d0] cannot run cause it needs W lock for: WO for Placement Path RI:[ora.oc4j 1 1] server [chenj-ch-05] target states [ONLINE INTERMEDIATE ], locked by op [START of [ora.oc4j 1 1] on [chenj-ch-05] : local=0, unplanned=00x7fddfc123e60]. Owner: CRS-2682: It is locked by 'grid' for command 'Start Resource' issued from 'chenj-ch-05
手动启动
grid@chenj-ch-05:/oracle/grid/product/11.2/log/chenj-ch-05/crsd$srvctl start oc4j -v OC4J could not be started PRCR-1079 : Failed to start resource ora.oc4j CRS-2674: Start of 'ora.oc4j' on 'chenj-ch-05' failed CRS-2632: There are no more servers to try to place resource 'ora.oc4j' on that would satisfy its placement policy
先跳过，不处理，ora.oc4不影响安装
问题二:
gpnptool rc=256
节点1执行root.sh遇到的另一个错误：
错误：
Failed to sign a peer profile for Oracle Cluster GPnP. gpnptool rc=256 Creation of Oracle GPnP peer profile failed for chenj-ch-05 Failed to create GPnP peer profile for chenj-ch-05 at /oracle/grid/product/11.2/crs/install/crsconfig_lib.pm line 5013. /oracle/grid/product/11.2/perl/bin/perl -I/oracle/grid/product/11.2/perl/lib -I/oracle/grid/product/11.2/crs/install /oracle/grid/product/11.2/crs/install/rootcrs.pl execution failed
问题原因：
在安装grid阶段，执行root.sh前，更新了31537677 Opatch，导致root.sh执行失败。
回退OPatch
opatch rollback -id 31537677
问题三： CRS-2101
执行节点2 root.sh
日志：
报错，hang住
ohasd failed to start Failed to start the Clusterware. Last 20 lines of the alert log follow: 2022-03-04 15:24:05.980: [client(55011)]CRS-2101:The OLR was formatted using version 3.
解决方案：
root@chenj-ch-06:/root#cd /var/tmp/.oracle/ root@chenj-ch-06:/var/tmp/.oracle#ls npohasd root@chenj-ch-06:/var/tmp/.oracle#ll -rth total 0 prw-r--r-- 1 root root 0 Mar 3 21:41 npohasd root@chenj-ch-06:/var/tmp/.oracle#rm npohasd rm: remove fifo ‘npohasd’? y root@chenj-ch-06:/var/tmp/.oracle#touch npohasd root@chenj-ch-06:/var/tmp/.oracle#chmod 755 npohasd root@chenj-ch-06:/var/tmp/.oracle#ll -rth total 0 -rwxr-xr-x 1 root root 0 Mar 3 21:48 npohasd
问题四: Oracle Grid Infrastructure is not installed
###执行configToolAllCommands
必须执行configToolAllCommands，如果不执行，后面安装oracle时，会提示下面错误：
CAUSE: Before you can install Oracle RAC, you must install Oracle Grid Infrastructure on all servers (Oracle Clusterware and Oracle ASM) to create a cluster. ACTION: Oracle Grid Infrastructure is not installed. Install it either from the separate installation media included in your media pack, or install it by downloading it from Electronic Product Delivery (EPD) or the Oracle Technology Network (OTN). Oracle Grid Infrastructure normally is installed by a different operating system user than the one used for Oracle Database. It may need to be installed by your system administrator. See the installation guide for more details.
问题五: ORA-12545
安装oracle软件时遇到如下错误：
执行root.sh时，提示rootadd_rdbms.sh、rootadd_filemap.sh 文件找不到，sqlplus登录时提示ORA-12545
Performing root user operation for Oracle 11g The following environment variables are set as: ORACLE_OWNER= oracle ORACLE_HOME= /oracle/db/product/11.2 Copying dbhome to /usr/local/bin ... Copying oraenv to /usr/local/bin ... Copying coraenv to /usr/local/bin ... Entries will be added to the /etc/oratab file as needed by Database Configuration Assistant when a database is created Finished running generic part of root script. Now product-specific root actions will be performed. Finished product-specific root actions. /oracle/db/product/11.2/root.sh: line 14: /oracle/db/product/11.2/rdbms/install/rootadd_rdbms.sh: No such file or directory /oracle/db/product/11.2/root.sh: line 16: /oracle/db/product/11.2/rdbms/install/rootadd_filemap.sh: No such file or directory
sqlplus登录报错ORA-12545
oracle@chenj-ch-05:/home/oracle$source .bash_profile oracle@chenj-ch-05:/home/oracle$sqlplus / as sysdba SQL*Plus: Release 11.2.0.4.0 Production on Fri Mar 4 10:04:57 2022 Copyright (c) 1982, 2013, Oracle. All rights reserved. ERROR: ORA-12545: Connect failed because target host or object does not exist
通过history发现，在安装oracle时，解决软件包只解压了1个，第2个包居然忘解压了，静默安装时没有报错，安装后发现少了很多文件。
卸载Oracle软件，重新安装，具体需要删除$ORACLE_BASE，$ORACLE_HOME，/etc/ora*，/var/tmp/.oracle/*等文件。
#####chenjuchao 20220304 20:50#####