Oracle RAC 第二节点 root.sh 报错 Timed out waiting for the CRS stack to start

在VBox 上安装11.2.0.1的RAC。 这里打算安装11.2.0.1是因为从11.2.0.2以后的版本对public 和private 网卡网段区分很严格，这个网卡必须配置在不同的网段。 而在11.2.0.1中则没有这个限制，所以在虚拟机上安装，用同网段要相对方便一些。

在11.2.0.1的grid安装中有个很麻烦的东西，就是在执行root.sh 时会报错：rootcrs.pl line 433. 解决方法参考：
Oracle 11gRAC ohasd failed to start at /u01/app/11.2.0/grid/crs/install/rootcrs.pl line443 解决方法
/article/1448705.html

今天在安装时，在节点2执行root.s，又遇到了Timedout waiting for the CRS stack to start的错误，其实这个错误在之前的安装过程中也遇到过，所以看了以前的记录，发现出入很大。

以前的记录：
Oracle RACroot.sh 报错 Timed out waiting for the CRSstack to start 解决方法
/article/1448707.html



相关记录如下：
[root@rac2 grid]# /u01/grid/root.sh
Running Oracle 11g root.sh script...

The following environment variables are setas:
ORACLE_OWNER= grid
ORACLE_HOME= /u01/grid

Enter the full pathname of the local bindirectory: [/usr/local/bin]:
The file "dbhome" already existsin /usr/local/bin. Overwrite it? (y/n)

: y
Copying dbhome to /usr/local/bin ...
The file "oraenv" already existsin /usr/local/bin. Overwrite it? (y/n)

: y
Copying oraenv to /usr/local/bin ...
The file "coraenv" already existsin /usr/local/bin. Overwrite it? (y/n)

: y
Copying coraenv to /usr/local/bin ...

Entries will be added to the /etc/oratabfile as needed by
Database Configuration Assistant when adatabase is created
Finished running generic part of root.shscript.
Now product-specific root actions will beperformed.
2012-10-11 01:58:50: Parsing the host name
2012-10-11 01:58:50: Checking for superuser privileges
2012-10-11 01:58:50: User has super userprivileges
Using configuration parameter file:/u01/grid/crs/install/crsconfig_params
Creating trace directory
LOCAL ADD MODE
Creating OCR keys for user 'root', privgrp'root'..
Operation successful.
Adding daemon to inittab
CRS-4123: Oracle High Availability Serviceshas been started.
ohasd is starting
ADVM/ACFS is not supported onoraclelinux-release-6Server-1.0.2.x86_64



CRS-4402: The CSS daemon was started inexclusive mode but found an active CSS daemon on node rac1, number 1, and isterminating
An active cluster was found duringexclusive startup, restarting to join the cluster
CRS-2672: Attempting to start 'ora.mdnsd' on'rac2'
CRS-2676: Start of 'ora.mdnsd' on 'rac2'succeeded
CRS-2672: Attempting to start 'ora.gipcd'on 'rac2'
CRS-2676: Start of 'ora.gipcd' on 'rac2'succeeded
CRS-2672: Attempting to start 'ora.gpnpd'on 'rac2'
CRS-2676: Start of 'ora.gpnpd' on 'rac2'succeeded
CRS-2672: Attempting to start'ora.cssdmonitor' on 'rac2'
CRS-2676: Start of 'ora.cssdmonitor' on'rac2' succeeded
CRS-2672: Attempting to start 'ora.cssd' on'rac2'
CRS-2672: Attempting to start 'ora.diskmon'on 'rac2'
CRS-2676: Start of 'ora.diskmon' on 'rac2'succeeded
CRS-2676: Start of 'ora.cssd' on 'rac2'succeeded
CRS-2672: Attempting to start 'ora.ctssd'on 'rac2'
CRS-2676: Start of 'ora.ctssd' on 'rac2'succeeded
CRS-2672: Attempting to start 'ora.asm' on'rac2'
CRS-2676: Start of 'ora.asm' on 'rac2'succeeded
CRS-2672: Attempting to start 'ora.crsd' on'rac2'
CRS-2676: Start of 'ora.crsd' on 'rac2'succeeded
CRS-2672: Attempting to start 'ora.evmd' on'rac2'
CRS-2676: Start of 'ora.evmd' on 'rac2'succeeded
Timed out waiting for the CRS stack to start.


Crsd的log 日志：
[root@rac2 crsd]# tail -20f crsd.log
ORA-17503: ksfdopn:DGOpenFile05 Failed toopen file +OCRVOTING.255.4294967295
ORA-17503: ksfdopn:2 Failed to open file+OCRVOTING.255.4294967295
ORA-15001: disk

2012-10-11 01:40:00.597: [ OCRASM][170809120]proprasmo: kgfoCheckMountreturned [6]
2012-10-11 01:40:00.598: [ OCRASM][170809120]proprasmo: The ASM diskgroup OCRVOTING is not found or not mounted
2012-10-11 01:40:00.604: [ OCRRAW][170809120]proprioo: Failed to open[+OCRVOTING]. Returned proprasmo() with [26]. Marking location as UN***AILABLE.
2012-10-11 01:40:00.604:[ OCRRAW][170809120]proprioo: No OCR/OLRdevices are usable
2012-10-11 01:40:00.604: [ OCRASM][170809120]proprasmcl: asmhandle isNULL
2012-10-11 01:40:00.605: [ OCRRAW][170809120]proprinit: Could not openraw device
2012-10-11 01:40:00.605: [ OCRASM][170809120]proprasmcl: asmhandle isNULL
2012-10-11 01:40:00.606: [ OCRAPI][170809120]a_init:16!: Backend initunsuccessful : [26]
2012-10-11 01:40:00.608: [ CRSOCR][170809120] OCR context initfailure. Error: PROC-26: Error whileaccessing the physical storage ASM error [SLOS: cat=8, opn=kgfoOpenFile01,dep=15056, loc=kgfokge
ORA-17503: ksfdopn:DGOpenFile05 Failed toopen file +OCRVOTING.255.4294967295
ORA-17503: ksfdopn:2 Failed to open file+OCRVOTING.255.4294967295
ORA-15001: disk
] [8]
2012-10-11 01:40:00.608:[ CRSD][170809120][PANIC] CRSDexiting: Could not init OCR, code: 26
2012-10-11 01:40:00.609: [ CRSD][170809120] Done.


这里的提示信息将问题指向了防火墙。 具体在MOS的文档有说明：
GridInfrastructure 11gR2 ROOT.SH Fails on Second Node due to Firewall [ID 1103313.1]


实际情况是，在安装GI之前，我就已经把防火墙关闭了：
[grid@rac2 crsd]$ /sbin/chkconfig --listiptables
iptables 0:off 1:off 2:off 3:off 4:off 5:off 6:off
[grid@rac2 crsd]$ /sbin/lsmod |grepip_tables


但根据CRSD的信息，问题又指向了防火墙，尝试把节点2重启一下。

重启节点2之后，清楚之前的信息：
[root@rac2 ~]# /u01/grid/crs/install/rootcrs.pl-deconfig -verbose -force
2012-10-11 02:11:06: Parsing the host name
2012-10-11 02:11:06: Checking for superuser privileges
2012-10-11 02:11:06: User has super userprivileges
Using configuration parameter file:/u01/grid/crs/install/crsconfig_params
PRCR-1035 : Failed to look up CRS resourceora.cluster_vip.type for 1
PRCR-1068 : Failed to query resources
Cannot communicate with crsd
PRCR-1070 : Failed to check if resourceora.gsd is registered
Cannot communicate with crsd
PRCR-1070 : Failed to check if resourceora.ons is registered
Cannot communicate with crsd
PRCR-1070 : Failed to check if resourceora.eons is registered
Cannot communicate with crsd

ADVM/ACFS is not supported onoraclelinux-release-6Server-1.0.2.x86_64

ACFS-9201: Not Supported
CRS-4133: Oracle High Availability Serviceshas been stopped.
error: package cvuqdisk is not installed
Successfully deconfigured Oracleclusterware stack on this node


再次执行root.sh:
[root@rac2 ~]# /u01/grid/root.sh
Running Oracle 11g root.sh script...

The following environment variables are setas:
ORACLE_OWNER= grid
ORACLE_HOME= /u01/grid

Enter the full pathname of the local bindirectory: [/usr/local/bin]:
The file "dbhome" already existsin /usr/local/bin. Overwrite it? (y/n)

:
The file "oraenv" already existsin /usr/local/bin. Overwrite it? (y/n)

:
The file "coraenv" already existsin /usr/local/bin. Overwrite it? (y/n)

:

Entries will be added to the /etc/oratabfile as needed by
Database Configuration Assistant when adatabase is created
Finished running generic part of root.shscript.
Now product-specific root actions will beperformed.
2012-10-11 02:15:36: Parsing the host name
2012-10-11 02:15:36: Checking for superuser privileges
2012-10-11 02:15:36: User has super userprivileges
Using configuration parameter file:/u01/grid/crs/install/crsconfig_params
LOCAL ADD MODE
Creating OCR keys for user 'root', privgrp'root'..
Operation successful.
Adding daemon to inittab
CRS-4123: Oracle High Availability Serviceshas been started.
ohasd is starting
ADVM/ACFS is not supported onoraclelinux-release-6Server-1.0.2.x86_64



CRS-4402: The CSS daemon was started inexclusive mode but found an active CSS daemon on node rac1, number 1, and isterminating
An active cluster was found during exclusivestartup, restarting to join the cluster
CRS-2672: Attempting to start 'ora.mdnsd'on 'rac2'
CRS-2676: Start of 'ora.mdnsd' on 'rac2'succeeded
CRS-2672: Attempting to start 'ora.gipcd'on 'rac2'
CRS-2676: Start of 'ora.gipcd' on 'rac2'succeeded
CRS-2672: Attempting to start 'ora.gpnpd'on 'rac2'
CRS-2676: Start of 'ora.gpnpd' on 'rac2'succeeded
CRS-2672: Attempting to start'ora.cssdmonitor' on 'rac2'
CRS-2676: Start of 'ora.cssdmonitor' on'rac2' succeeded
CRS-2672: Attempting to start 'ora.cssd' on'rac2'
CRS-2672: Attempting to start 'ora.diskmon'on 'rac2'
CRS-2676: Start of 'ora.diskmon' on 'rac2'succeeded
CRS-2676: Start of 'ora.cssd' on 'rac2'succeeded
CRS-2672: Attempting to start 'ora.ctssd'on 'rac2'
CRS-2676: Start of 'ora.ctssd' on 'rac2'succeeded
CRS-2672: Attempting to start 'ora.asm' on'rac2'
CRS-2676: Start of 'ora.asm' on 'rac2'succeeded
CRS-2672: Attempting to start 'ora.crsd' on'rac2'
CRS-2676: Start of 'ora.crsd' on 'rac2'succeeded
CRS-2672: Attempting to start 'ora.evmd' on'rac2'
CRS-2676: Start of 'ora.evmd' on 'rac2'succeeded

rac2 2012/10/11 02:38:36 /u01/grid/cdata/rac2/backup_20121011_023836.olr
Preparing packages for installation...
cvuqdisk-1.0.7-1
Configure Oracle Grid Infrastructure for aCluster ... succeeded
Updating inventory properties forclusterware
Starting Oracle Universal Installer...

Checking swap space: must be greater than500 MB. Actual 1999 MB Passed
The inventory pointer is located at/etc/oraInst.loc
The inventory is located at/u01/grid/oraInventory
'UpdateNodeList' was successful.
[root@rac2 ~]#

这次成功执行。







--------------------------------------------------------------------------------------------版权所有，文章允许转载，但必须以链接方式注明源地址，否则追究法律责任!Skype: tianlesoftwareQQ: tianlesoftware@gmail.comEmail: tianlesoftware@gmail.comBlog: http://blog.csdn.net/tianlesoftwareWeibo: http://weibo.com/tianlesoftwareTwitter: http://twitter.com/tianlesoftwareFacebook: http://www.facebook.com/tianlesoftwareLinkedin: http://cn.linkedin.com/in/tianlesoftware