RAC在OCR磁盘故障且有备份时的恢复操作

问题描述：

CRS启动失败，并且RAC进程全部退出。

查看messages日志：

... ...

Apr 10 15:58:31 node74 logger: Cluster Ready Services waiting on dependencies. Diagnostics in /tmp/crsctl.7510.

Apr 10 15:58:31 node74 logger: Cluster Ready Services waiting on dependencies. Diagnostics in /tmp/crsctl.7264.

Apr 10 15:58:31 node74 logger: Cluster Ready Services waiting on dependencies. Diagnostics in /tmp/crsctl.7174.

Apr 10 15:59:31 node74 su: (to oracle) root on none

Apr 10 15:59:31 node74 su: (to oracle) root on none

Apr 10 15:59:31 node74 su: (to oracle) root on none

Apr 10 15:59:31 node74 logger: Cluster Ready Services waiting on dependencies. Diagnostics in /tmp/crsctl.7510.

Apr 10 15:59:31 node74 logger: Cluster Ready Services waiting on dependencies. Diagnostics in /tmp/crsctl.7174.

Apr 10 15:59:31 node74 logger: Cluster Ready Services waiting on dependencies. Diagnostics in /tmp/crsctl.7264.

Apr 10 16:00:26 node74 su: (to oracle) root on /dev/pts/0

Apr 10 16:00:31 node74 su: (to oracle) root on none

Apr 10 16:00:31 node74 su: (to oracle) root on none

Apr 10 16:00:31 node74 su: (to oracle) root on none

Apr 10 16:00:31 node74 logger: Cluster Ready Services waiting on dependencies. Diagnostics in /tmp/crsctl.7264.

Apr 10 16:00:31 node74 logger: Cluster Ready Services waiting on dependencies. Diagnostics in /tmp/crsctl.7510.

Apr 10 16:00:31 node74 logger: Cluster Ready Services waiting on dependencies. Diagnostics in /tmp/crsctl.7174.

... ...

取其中一个文件下来，/tmp/crsctl.7174：

Oracle Cluster Registry initialization failed with invalid format: PROC-22: The OCR backend has an invalid format

怀疑是OCR损坏。运行crsctl check boot命令，与上面的日志报错是一致的：

oracle@node74:~> crsctl check boot

Oracle Cluster Registry initialization failed with invalid format: PROC-22: The OCR backend has an invalid format

问题处理：

查看是否有OCR自动备份，很不幸，没有结果输出：

oracle@node74:~> ocrconfig -showbackup

---空

幸好之前做OCR迁移实验时，有做过OCR的手动备份。

node74:/home/database/oracle/oracrs/bin # ll /home/oracle/ocr.exp

-rw-r--r-- 1 root root 87246 Dec 29 16:45 /home/oracle/ocr.exp

导入之前备份的OCR：

node74:/home/database/oracle/oracrs/bin # ./ocrconfig -import /home/oracle/ocr.exp

检查OCR：

node74:/home/database/oracle/oracrs/bin # ./ocrcheck

Status of Oracle Cluster Registry is as follows :

         Version                  :          2

         Total space (kbytes)     :     975584

         Used space (kbytes)      :       3844

         Available space (kbytes) :     971740

         ID                       :  811637621

         Device/File Name         : /dev/raw/raw1

                                    Device/File integrity check succeeded

                                    Device/File not configured

         Cluster registry integrity check succeeded

         Logical corruption check succeeded

---修复成功

启动CRS：         

node74:/home/database/oracle/oracrs/bin # /etc/init.d/init.crs start

所有RAC进程启动成功：

node74:~ # su - oracle

oracle@node74:~> crs_stat -t

Name           Type           Target    State     Host        

------------------------------------------------------------

ora....SM1.asm application    ONLINE    ONLINE    node74      

ora....74.lsnr application    ONLINE    ONLINE    node74      

ora.node74.gsd application    ONLINE    ONLINE    node74      

ora.node74.ons application    ONLINE    ONLINE    node74      

ora.node74.vip application    ONLINE    ONLINE    node74      

ora....SM2.asm application    ONLINE    ONLINE    node76      

ora....76.lsnr application    ONLINE    ONLINE    node76      

ora.node76.gsd application    ONLINE    ONLINE    node76      

ora.node76.ons application    ONLINE    ONLINE    node76      

ora.node76.vip application    ONLINE    ONLINE    node76      

ora.orcl.db    application    ONLINE    ONLINE    node74      

ora....l1.inst application    ONLINE    ONLINE    node74      
ora....l2.inst application    ONLINE    ONLINE    node76   

--问题解决