rac root.sh on node2 Died at /u01/grid/product/crs/install/crsinstall.pm line 914错误解决

/u01/grid/product/root.sh 

--省略部分

CRS-2676: Start of 'ora.diskmon' on 'rac2' succeeded

CRS-2883: Resource 'ora.evmd' failed during Clusterware stack start.

CRS-4406: Oracle High Availability Services synchronous start failed.

CRS-4000: Command Start failed, or completed with errors.

2017/04/13 13:23:38 CLSRSC-117: Failed to start Oracle Clusterware stack

Died at /u01/grid/product/crs/install/crsinstall.pm line 914.

The command '/u01/grid/product/perl/bin/perl -I/u01/grid/product/perl/lib -I/u01/grid/product/crs/install /u01/grid/product/crs/install/rootcrs.pl ' execution failed

第一次执行root.sh报错如下，查看/u01/grid/product/cfgtoollogs/crsconfig下crs日志也只是CRS-2883: Resource 'ora.evmd' failed during Clusterware stack start.未得到可用信息，决定杀掉ohas相关进程

[root@rac2 u01]# ps -ef|grep has

root      4404     1  1 12:00 ?        00:00:58 /u01/grid/product/bin/ohasd.bin reboot

root     12309     1  0 13:21 ?        00:00:00 /bin/sh /etc/init.d/init.ohasd run

root     13490     1  1 13:22 ?        00:00:09 /u01/grid/product/bin/ohasd.bin reboot _ORA_BLOCKING_STACK_LOCALE=AMERICAN_AMERICA.AL32UTF8

root     18380  4281  0 13:33 pts/1    00:00:00 grep has

[root@rac2 u01]# kill -9 4404

[root@rac2 u01]# kill -9 12309

[root@rac2 u01]# kill -9 13490

[root@rac2 u01]# ps -ef|grep has

root     18490     1  0 13:33 ?        00:00:00 /bin/sh /etc/init.d/init.ohasd run

root     18540 18490 14 13:33 ?        00:00:00 /u01/grid/product/bin/ohasd.bin restart

root     18678  4281  0 13:33 pts/1    00:00:00 grep has

[root@rac2 u01]# /u01/grid/product/crs/install/rootcrs.pl -deconfig

Using configuration parameter file: /u01/grid/product/crs/install/crsconfig_params

Network 1 exists

Subnet IPv4: 192.168.80.0/255.255.255.0/em1, dhcp

Subnet IPv6: 

Ping Targets: 

Network is enabled

Network is individually enabled on nodes: 

Network is individually disabled on nodes: 

VIP exists: network number 1, hosting node rac1

VIP IPv4 Address: -/rac1-vip/192.168.80.222

VIP IPv6 Address: 

VIP is enabled.

VIP is individually enabled on nodes: 

VIP is individually disabled on nodes: 

VIP exists: network number 1, hosting node rac2

VIP Name: rac2

VIP IPv4 Address: 

VIP IPv6 Address: 

VIP is enabled.

VIP is individually enabled on nodes: 

VIP is individually disabled on nodes: 

ONS exists: Local port 6100, remote port 6200, EM port 2016, Uses SSL false

ONS is enabled

ONS is individually enabled on nodes: 

ONS is individually disabled on nodes: 

PRCR-1014 : Failed to stop resource ora.net1.network

PRCR-1065 : Failed to stop resource ora.net1.network

CRS-2529: Unable to act on 'ora.net1.network' because that would require stopping or relocating 'ora.scan1.vip', but the force option was not specified

CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rac2'

CRS-2673: Attempting to stop 'ora.crsd' on 'rac2'

CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'rac2'

CRS-2673: Attempting to stop 'ora.LISTENER_SCAN1.lsnr' on 'rac2'

CRS-2673: Attempting to stop 'ora.DATA.dg' on 'rac2'

CRS-2677: Stop of 'ora.DATA.dg' on 'rac2' succeeded

CRS-2673: Attempting to stop 'ora.asm' on 'rac2'

CRS-2677: Stop of 'ora.LISTENER_SCAN1.lsnr' on 'rac2' succeeded

CRS-2673: Attempting to stop 'ora.scan1.vip' on 'rac2'

CRS-2677: Stop of 'ora.scan1.vip' on 'rac2' succeeded

CRS-2672: Attempting to start 'ora.scan1.vip' on 'rac1'

CRS-2676: Start of 'ora.scan1.vip' on 'rac1' succeeded

CRS-2672: Attempting to start 'ora.LISTENER_SCAN1.lsnr' on 'rac1'

CRS-2677: Stop of 'ora.asm' on 'rac2' succeeded

CRS-2673: Attempting to stop 'ora.ASMNET1LSNR_ASM.lsnr' on 'rac2'

CRS-2677: Stop of 'ora.ASMNET1LSNR_ASM.lsnr' on 'rac2' succeeded

CRS-2676: Start of 'ora.LISTENER_SCAN1.lsnr' on 'rac1' succeeded

CRS-2673: Attempting to stop 'ora.net1.network' on 'rac2'

CRS-2677: Stop of 'ora.net1.network' on 'rac2' succeeded

CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'rac2' has completed

CRS-2677: Stop of 'ora.crsd' on 'rac2' succeeded

CRS-2673: Attempting to stop 'ora.evmd' on 'rac2'

CRS-2673: Attempting to stop 'ora.storage' on 'rac2'

CRS-2673: Attempting to stop 'ora.mdnsd' on 'rac2'

CRS-2673: Attempting to stop 'ora.gpnpd' on 'rac2'

CRS-2677: Stop of 'ora.storage' on 'rac2' succeeded

CRS-2673: Attempting to stop 'ora.asm' on 'rac2'

CRS-2677: Stop of 'ora.evmd' on 'rac2' succeeded

CRS-2677: Stop of 'ora.mdnsd' on 'rac2' succeeded

CRS-2677: Stop of 'ora.gpnpd' on 'rac2' succeeded

CRS-2677: Stop of 'ora.asm' on 'rac2' succeeded

CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'rac2'

CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'rac2' succeeded

CRS-2673: Attempting to stop 'ora.ctssd' on 'rac2'

CRS-2677: Stop of 'ora.ctssd' on 'rac2' succeeded

CRS-2673: Attempting to stop 'ora.cssd' on 'rac2'

CRS-2677: Stop of 'ora.cssd' on 'rac2' succeeded

CRS-2673: Attempting to stop 'ora.crf' on 'rac2'

CRS-2677: Stop of 'ora.crf' on 'rac2' succeeded

CRS-2673: Attempting to stop 'ora.gipcd' on 'rac2'

CRS-2677: Stop of 'ora.gipcd' on 'rac2' succeeded

CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'rac2' has completed

CRS-4133: Oracle High Availability Services has been stopped.

2017/04/13 13:34:56 CLSRSC-4006: Removing Oracle Trace File Analyzer (TFA) Collector.

2017/04/13 13:35:18 CLSRSC-4007: Successfully removed Oracle Trace File Analyzer (TFA) Collector.

error: package cvuqdisk is not installed
2017/04/13 13:35:18 CLSRSC-336: Successfully deconfigured Oracle Clusterware stack on this node

[root@rac2 u01]# /u01/grid/product/root.sh 

Performing root user operation.

The following environment variables are set as:

    ORACLE_OWNER= grid

    ORACLE_HOME=  /u01/grid/product

Enter the full pathname of the local bin directory: [/usr/local/bin]: 

The contents of "dbhome" have not changed. No need to overwrite.

The contents of "oraenv" have not changed. No need to overwrite.

The contents of "coraenv" have not changed. No need to overwrite.

Entries will be added to the /etc/oratab file as needed by

Database Configuration Assistant when a database is created

Finished running generic part of root script.

Now product-specific root actions will be performed.

Using configuration parameter file: /u01/grid/product/crs/install/crsconfig_params

2017/04/13 13:35:51 CLSRSC-4001: Installing Oracle Trace File Analyzer (TFA) Collector.

2017/04/13 13:36:14 CLSRSC-4002: Successfully installed Oracle Trace File Analyzer (TFA) Collector.

2017/04/13 13:36:15 CLSRSC-363: User ignored prerequisites during installation

OLR initialization - successful

2017/04/13 13:37:45 CLSRSC-330: Adding Clusterware entries to file 'oracle-ohasd.conf'

CRS-4133: Oracle High Availability Services has been stopped.

CRS-4123: Oracle High Availability Services has been started.

CRS-4133: Oracle High Availability Services has been stopped.

CRS-4123: Oracle High Availability Services has been started.

CRS-4133: Oracle High Availability Services has been stopped.

CRS-4123: Starting Oracle High Availability Services-managed resources

CRS-2672: Attempting to start 'ora.mdnsd' on 'rac2'

CRS-2672: Attempting to start 'ora.evmd' on 'rac2'

CRS-2676: Start of 'ora.mdnsd' on 'rac2' succeeded

CRS-2676: Start of 'ora.evmd' on 'rac2' succeeded

CRS-2672: Attempting to start 'ora.gpnpd' on 'rac2'

CRS-2676: Start of 'ora.gpnpd' on 'rac2' succeeded

CRS-2672: Attempting to start 'ora.gipcd' on 'rac2'

CRS-2676: Start of 'ora.gipcd' on 'rac2' succeeded

CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rac2'

CRS-2676: Start of 'ora.cssdmonitor' on 'rac2' succeeded

CRS-2672: Attempting to start 'ora.cssd' on 'rac2'

CRS-2672: Attempting to start 'ora.diskmon' on 'rac2'

CRS-2676: Start of 'ora.diskmon' on 'rac2' succeeded

CRS-2676: Start of 'ora.cssd' on 'rac2' succeeded

CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'rac2'

CRS-2672: Attempting to start 'ora.ctssd' on 'rac2'

CRS-2676: Start of 'ora.ctssd' on 'rac2' succeeded

CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'rac2' succeeded

CRS-2672: Attempting to start 'ora.asm' on 'rac2'

CRS-2676: Start of 'ora.asm' on 'rac2' succeeded

CRS-2672: Attempting to start 'ora.storage' on 'rac2'

CRS-2676: Start of 'ora.storage' on 'rac2' succeeded

CRS-2672: Attempting to start 'ora.crf' on 'rac2'

CRS-2676: Start of 'ora.crf' on 'rac2' succeeded

CRS-2672: Attempting to start 'ora.crsd' on 'rac2'

CRS-2676: Start of 'ora.crsd' on 'rac2' succeeded

CRS-6017: Processing resource auto-start for servers: rac2

CRS-2672: Attempting to start 'ora.ASMNET1LSNR_ASM.lsnr' on 'rac2'

CRS-2672: Attempting to start 'ora.net1.network' on 'rac2'

CRS-2676: Start of 'ora.net1.network' on 'rac2' succeeded

CRS-2672: Attempting to start 'ora.ons' on 'rac2'

CRS-2676: Start of 'ora.ASMNET1LSNR_ASM.lsnr' on 'rac2' succeeded

CRS-2672: Attempting to start 'ora.asm' on 'rac2'

CRS-2676: Start of 'ora.ons' on 'rac2' succeeded

CRS-2673: Attempting to stop 'ora.LISTENER_SCAN1.lsnr' on 'rac1'

CRS-2677: Stop of 'ora.LISTENER_SCAN1.lsnr' on 'rac1' succeeded

CRS-2673: Attempting to stop 'ora.scan1.vip' on 'rac1'

CRS-2677: Stop of 'ora.scan1.vip' on 'rac1' succeeded

CRS-2672: Attempting to start 'ora.scan1.vip' on 'rac2'

CRS-2676: Start of 'ora.scan1.vip' on 'rac2' succeeded

CRS-2672: Attempting to start 'ora.LISTENER_SCAN1.lsnr' on 'rac2'

CRS-2676: Start of 'ora.LISTENER_SCAN1.lsnr' on 'rac2' succeeded

CRS-2676: Start of 'ora.asm' on 'rac2' succeeded

CRS-6016: Resource auto-start has completed for server rac2

CRS-6024: Completed start of Oracle Cluster Ready Services-managed resources

CRS-4123: Oracle High Availability Services has been started.

2017/04/13 13:40:23 CLSRSC-343: Successfully started Oracle Clusterware stack

Preparing packages for installation...

cvuqdisk-1.0.9-1
2017/04/13 13:40:40 CLSRSC-325: Configure Oracle Grid Infrastructure for a Cluster ... succeeded