故障现象:
收到zabbix告警邮件,具体内容如下:
告警内容:
ZABBIX-监控系统:
------------------------------------
报警内容: Found failed scheduler jobs
------------------------------------
报警级别: PROBLEM
------------------------------------
监控项目: app_job_fail_cnt:1
------------------------------------
报警时间:2016.11.04-10:43:05
故障原因:
由于有个项目的DB下线,而这个db的部分表数据需要同步到另一数据库(统计库),统计库有定时任务定期同步数据,一旦下线DB关闭,统计库定时任务同步数据失败,从而触发邮件告警。
故障处理过程:
查看失败Job信息
select log_id,log_date,owner,job_name,status from dba_scheduler_job_log where status = 'FAILED' ;
626593 04-11月-16 10.17.52.065773 上午 +08:00 ASTA_STAT SYN_SERINFO FAILED
禁用失败job
exec dbms_scheduler.disable('DBUSER.SYN_SERINFO') ;
删除失败日志:(由于dba_scheduler_job_log是视图,根据其SQL找到对应的物理表,然后删除数据)
SQL> delete from dba_scheduler_job_log where status = 'FAILED' ;
delete from dba_scheduler_job_log where status = 'FAILED'
*
ERROR at line 1:
ORA-01752: cannot delete from view without exactly one key-preserved table
select owner,object_type from dba_objects where object_name=upper('dba_scheduler_job_log');
SYS VIEW
PUBLIC SYNONYM
CREATE OR REPLACE VIEW SYS.DBA_SCHEDULER_JOB_LOG AS
(SELECT
LOG_ID, LOG_DATE, OWNER,
DECODE(instr(e.NAME,'"'),0, e.NAME,substr(e.NAME,1,instr(e.NAME,'"')-1)),
DECODE(instr(e.NAME,'"'),0,NULL,substr(e.NAME,instr(e.NAME,'"')+1)),
co.NAME, OPERATION,e.STATUS, USER_NAME, CLIENT_ID, GUID,
decode(e.credential, NULL, NULL,
substr(e.credential, 1, instr(e.credential, '"')-1)),
decode(e.credential, NULL, NULL,
substr(e.credential, instr(e.credential, '"')+1,
length(e.credential) - instr(e.credential, '"'))),
decode(bitand(e.flags, 1), 0, NULL,
substr(e.destination, 1, instr(e.destination, '"')-1)),
decode(bitand(e.flags, 1), 0, e.destination,
substr(e.destination, instr(e.destination, '"')+1,
length(e.destination) - instr(e.destination, '"'))),
ADDITIONAL_INFO
FROM scheduler$_event_log e, obj$ co
WHERE e.type# = 66 and e.dbid is null and e.class_id = co.obj#(+));
根据物理表删除日志
select object_name,object_type ,owner from dba_objects where object_name=upper('scheduler$_event_log');
select * from sys.SCHEDULER$_EVENT_LOG where status='FAILED';
DELETE FROM sys.SCHEDULER$_EVENT_LOG where status='FAILED';
commit;
扩展:
oracle 定时任务管理,请参考之前的文章
http://blog.itpub.net/27067062/viewspace-2127806/