oracle11g rac 实例1不能启动 ORA-01078

数据库版本：11.2.0.3 RAC

操作系统：red hat 5

问题：起初客户端连接不上实例1，后来检查后，最终重启了服务器，也没找到问题原因，但是今天下午的时候，发现实例1连接不上了，

           后来经过查看，发现实例1没启动，然后登录sqlplus，启动实例1报错：

SQL> startup

ORA-01078: failure in processing system parameters

ORA-01565: error in identifying file '+DATA/gzwlfp/spfilegzwlfp.ora'

ORA-17503: ksfdopn:2 Failed to open file +DATA/gzwlfp/spfilegzwlfp.ora

ORA-01034: ORACLE not available

ORA-27123: unable to attach to shared memory segment

Linux-x86_64 Error: 13: Permission denied

经过从网上搜索方法，解决了，解决方法如下：

Symptoms
1. When attempting to startup database,the following error occurs.（类似下面的报错会出现）
ORA-01078: failure in processing system parameters
ORA-01565: error in identifying file '+DATA1/orcl/spfileorcl.ora'
ORA-17503: ksfdopn:2 Failed to open file +DATA1/orcl/spfileorcl.ora
ORA-12547: TNS:lost contact

2. DB alert log:
ERROR: Failed to connect with connect string: (DESCRIPTION=(ADDRESS=(PROTOCOL=beq)(PROGRAM=/opt/11.2.0/grid/bin/oracle)(ARGV0=oracle+ASM_pmon_orcl)(ENVS='ORACLE_HOME=/opt/11.2.0/grid,ORACLE_SID=+ASM')(ARGS='(DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq)))'))(enable=setuser))
WARNING: ASM communication error: op 0 state 0x0 (15055)
ERROR: direct connection failure with ASM
NOTE: Deferred communication with ASM instance
Errors in file /opt/oracle/diag/rdbms/orcl/orcl/trace/orcl_pmon_26924.trc:
ORA-15055: unable to connect to ASM instance
ORA-12547: TNS:lost contact

3. ASM instance and required diskgroups are online and no error in ASM alert:
crsctl stat res -t
--------------------------------------------------------------------------------
NAME TARGET STATE SERVER STATE_DETAILS Local Resources
--------------------------------------------------------------------------------
ora.DATA1.dg ONLINE ONLINE prod 
ora.DATA2.dg ONLINE ONLINE prod 
ora.asm ONLINE ONLINE prod Started 

Cause
The issue is caused by incorrect permission of $GRID_HOME/bin/oracle and $ORACLE_HOME/bin/oracle, which lead to connection failure to ASM instance:

查看两个实例的文件权限确认不一样
ls -al $GRID_HOME/bin/oracle
-rwxr-x--x 1 grid oinstall 200678464 Feb 28 14:54 oracle 

ls -al $ORACLE_HOME/bin/oracle
-rwxr-x--x 1 oracle asmadmin 228886191 Feb 28 15:41 oracle 

Solution
Oracle binary should have permission of 6751. To correct the permission, as owner of oracle binary:

后来修改正确的权限：

cd $GRID_HOME/bin
chmod 6751 oracle

cd $ORACLE_HOME/bin
chmod 6751 oracle

After the change, permission should show:

ls -l oracle
-rwsr-s--x 1 grid oinstall 152400480 Nov 24 15:49 oracle

修改完成后，重新startup实例1，问题解决。
Once the permission is fixed, if database still fails to come up, restart Grid Infrastructure.

不知道为什么这个的权限变了，貌似也没有人更改。