ORACLE ADG 最大可用模式下实例启动失败分析

一、系统环境

服务器:X86两节点,HP存储

操作系统:redhat6.9

数据库版本:11.2.0.4


二、故障介绍

RAC生产库一号节点物理机宕机后,无法启动实例。二号节点服务正常运行,Dataguard同步正常,未中断。

故障节点硬重启后,GI服务正常启动,但数据库实例启动失败

告警如下:

ARC3 started with pid=38, OS id=9473

ARC1: Archival started

ARC2: Archival started

ARC1: Becoming the 'no FAL' ARCH

ARC1: Becoming the 'no SRL' ARCH

ARC2: Becoming the heartbeat ARCH

LGWR: Primary database is in MAXIMUM AVAILABILITY mode

LGWR: Destination LOG_ARCHIVE_DEST_1 is not serviced by LGWR

ARC3: Archival started

ARC0: STARTING ARCH PROCESSES COMPLETE

Destination LOG_ARCHIVE_DEST_2 is UNSYNCHRONIZED

LGWR: Minimum of 1 applicable standby database required

Errors in file /oracle/app/diag/rdbms/orcl/orcl2/trace/orcl2_lgwr_9361.trc:

ORA-16072: a minimum of one standby database destination is required

LGWR (ospid: 9361): terminating the instance due to error 16072

Thu Apr 14 20:58:24 2022

System state dump requested by (instance=2, osid=9361 (LGWR)), summary=[abnormal instance termination].

System State dumped to trace file /oracle/app/diag/rdbms/orcl/orcl2/trace/orcl2_diag_9335_20220414205824.trc

Dumping diagnostic data in directory=[cdmp_20220414205824], requested by (instance=2, osid=9361 (LGWR)), summary=[abnormal instance termination].

Instance terminated by LGWR, pid = 9361


三、处理过程

可从报错

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

LGWR: Primary database is in MAXIMUM AVAILABILITY mode

LGWR: Destination LOG_ARCHIVE_DEST_1 is not serviced by LGWR

Destination LOG_ARCHIVE_DEST_2 is UNSYNCHRONIZED

LGWR: Minimum of 1 applicable standby database required

Errors in file /oracle/app/diag/rdbms/orcl/orcl2/trace/orcl2_lgwr_9361.trc:

ORA-16072: a minimum of one standby database destination is required

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

中看到当前Dataguard运行在MAXIMUM AVAILABILITY模式,LOG_ARCHIVE_DEST_2为非同步状态,要求到少有一个standby database destination是正常的。当前条件不满足导致实例启动时被LGWR中断。

但此问题比较诡异,当前实例2正常在运行,dataguard同步是正常的,未中断,按道理实例1可正常启动并加入集群。

为尽快恢复业务,临时处理方法如下:

1)将dataguard同步模式降级为MAXIMIZE PERFORMANCE

ALTER DATABASE SET STANDBY TO MAXIMIZE PERFORMANCE;

2)启动故障实例

startup


完成上述修改后,实例1正常启动并加入集群


四、 MAXIMUM AVAILABILITY下实例启动故障分析

检查生产DG相关参数如下:

log_archive_config string

log_archive_dest_2 string SERVICE=orclstd LGWR SYNC VALI

D_FOR=(ONLINE_LOGFILES,PRIMARY

_ROLE)

dg_broker_config_file1 string /oracle/app/product/11.2.0/db_

1/dbs/dr1orcl.dat

dg_broker_config_file2 string /oracle/app/product/11.2.0/db_

1/dbs/dr2orcl.dat


五、疑问:

log_archive_config 参数未按规范配置是否是导致MAXIMUM AVAILABILITY模式下RAC其中一节点重启后不能再正常启动的原因?


六、测试:

测试一:

1)在测试环境中,去掉log_archive_config参数,并修改DG同步模式为MAXIMUM AVAILABILITY

2)重启其中一个实例遇到相同故障

测试二:

1)配置上log_archive_config DG_CONFIG=(orcl,orclstd),并修改DG同步模式为MAXIMUM AVAILABILITY

2)重启其中一个实例,可正常启动。


七、总结

初始化参数LOG_ARCHIVE_CONFIG用于控制发送归档日志到远程位置、接收远程归档日志,搭建ADG环境时,可以配置也可以不配置,ADG都可正常运行。

但当同步模式为MAXIMUM AVAILABILITY模式时,

1)RAC环境会遇到实例重启后无法正常启动的情况。

2)无法在线从MAXIMIZE PERFORMANCE切换为MAXIMIZE AVAILABILITY,提示需要mount状态进行模式切换。



请使用浏览器的分享功能分享到微信等