RAC由于归档表空间满而无法启动实例的解决

今天想测试点东西，登录测试库；发现实例是关闭的；
SQL> startup
ORACLE instance started.

Total System Global Area 2722467840 bytes
Fixed Size 2231472 bytes
Variable Size 1476395856 bytes
Database Buffers 1241513984 bytes
Redo Buffers 2326528 bytes
Database mounted.
ORA-03113: end-of-file on communication channel
Process ID: 23281
Session ID: 1 Serial number: 5

查看日志

SUCCESS: diskgroup ORAARCH was mounted
ARCH: Error 19504 Creating archive log file to '+ORAARCH'
Errors in file /u01/app/oracle/diag/rdbms/xhdb/xhdb1/trace/xhdb1_ora_23565.trc:
ORA-16038: log 5 sequence# 345 cannot be archived
ORA-19504: failed to create file ""
ORA-00312: online log 5 thread 1: '+ORADATA/xhdb/onlinelog/group_5.272.857422319 '
USER (ospid: 23565): terminating the instance due to error 16038
System state dump requested by (instance=1, osid=23565), summary=[abnormal insta nce termination].
System State dumped to trace file /u01/app/oracle/diag/rdbms/xhdb/xhdb1/trace/xh db1_diag_23451.trc
Dumping diagnostic data in directory=[cdmp_20141218102056], requested by (instan ce=1, osid=23565), summary=[abnormal instance termination].
Instance terminated by USER, pid = 23565

查看(fdisk)查看磁盘都在；切换到grid集群也在运行，asm磁盘组都在；可能是归档空间不够了；把节点1的数据库实例启动到挂载状态下；

SQL> startup mount;
ORACLE instance started.

Total System Global Area 2722467840 bytes
Fixed Size 2231472 bytes
Variable Size 1476395856 bytes
Database Buffers 1241513984 bytes
Redo Buffers 2326528 bytes
Database mounted.
SQL> archive log list;
Database log mode Archive Mode
Automatic archival Enabled
Archive destination +ORAARCH
Oldest online log sequence 345
Next log sequence to archive 345

SQL> select group_number,name,state,total_mb,free_mb from v$asm_diskgroup;

GROUP_NUMBER NAME STATE TOTAL_MB FREE_MB
------------ ------------------------------ ----------- ---------- ----------
1 CRS MOUNTED 998 600
2 ORAARCH MOUNTED 8189 11
3 ORADATA CONNECTED 20473 16198

发现是oraarch 归档空间不够了,节点1登录rman,把归档日志删除；在节点1和节点2启动数据库实例，正常；

RMAN> crosscheck archivelog all;

RMAN> delete archivelog all;

SQL> select group_number,name,state,total_mb,free_mb from v$asm_diskgroup;

GROUP_NUMBER NAME STATE TOTAL_MB FREE_MB
------------ ------------------------------ ----------- ---------- ----------
1 CRS MOUNTED 998 600
2 ORAARCH CONNECTED 8189 5123
3 ORADATA CONNECTED 20473 16198

SQL> alter database open;

Database altered.

节点2 也能正常打开
[oracle@rac2 ~]$ sqlplus / as sysdba

SQL*Plus: Release 11.2.0.3.0 Production on Thu Dec 18 10:56:54 2014

Copyright (c) 1982, 2011, Oracle. All rights reserved.

Connected to an idle instance.

SQL> startup
ORACLE instance started.

Total System Global Area 2722467840 bytes
Fixed Size 2231472 bytes
Variable Size 1191183184 bytes
Database Buffers 1526726656 bytes
Redo Buffers 2326528 bytes
Database mounted.
Database opened.