环境：
操作系统：linux 2.6.18-371.el5 x86_64
数据库：ORACLE 10.2.0.5 rac

最近，一oracle数据库运行报错如下:
/u01/app/oracle/admin/bjcvsdb/udump/bjcvsdb1_ora_19040.trc
Oracle Database 10g Enterprise Edition Release 10.2.0.5.0 - 64bit Production
With the Partitioning, Real Application Clusters, OLAP, Data Mining
and Real Application Testing options
ORACLE_HOME = /u01/app/oracle/product/10.2.0/db_1
System name: Linux
Node name: bjcvsdb1
Release: 2.6.18-371.el5
Version: #1 SMP Fri Jul 18 06:33:06 UTC 2014
Machine: x86_64
Instance name: bjcvsdb1
Redo thread mounted by this instance: 1
Oracle process number: 203
Unix process pid: 19040, image: oracle@bjcvsdb1 (TNS V1-V3)
*** 2018-03-14 17:01:16.366
*** ACTION NAME:() 2018-03-14 17:01:16.366
*** MODULE NAME:(sqlplus@bjcvsdb1 (TNS V1-V3)) 2018-03-14 17:01:16.366
*** SERVICE NAME:(SYS$USERS) 2018-03-14 17:01:16.366
*** SESSION ID:(3560.19301) 2018-03-14 17:01:16.366
Exception signal: 11 (SIGSEGV), code: 1 (Address not mapped to object), addr: 0x8, PC: [0x2b15abbd22e4, joet_create_root_thread_group()+140]
*** 2018-03-14 17:01:16.367
ksedmp: internal or fatal error
ORA-07445: exception encountered: core dump [joet_create_root_thread_group()+140] [SIGSEGV] [Address not mapped to object] [0x000000008] [] []
Current SQL statement for this session:
BEGIN
/* NOP UNLESS A TABLE OBJECT */
IF dictionary_obj_type = 'TABLE'
THEN
sys.dbms_cdc_ipublish.change_table_trigger(dictionary_obj_owner,dictionary_obj_name,'LOCK');
END IF;
END;
----- PL/SQL Call Stack -----
object line object
handle number name
0x25d8acfbb0 12 package body SYS.DBMS_CDC_IPUBLISH
0x25d85031a8 5 anonymous block
----- Call Stack Trace -----
calling call entry argument values in hex
location type point (? means dubious value)
-------------------- -------- -------------------- ----------------------------
ksedst()+31 call ksedst1() 000000000 ? 000000001 ?
2B15ACA04D50 ? 2B15ACA04DB0 ?
2B15ACA04CF0 ? 000000000 ?
ksedmp()+610 call ksedst() 000000000 ? 000000001 ?
2B15ACA04D50 ? 2B15ACA04DB0 ?
2B15ACA04CF0 ? 000000000 ?
ssexhd()+629 call ksedmp() 000000003 ? 000000001 ?
2B15ACA04D50 ? 2B15ACA04DB0 ?
2B15ACA04CF0 ? 000000000 ?
__restore_rt()+0 call ssexhd() 00000000B ? 2B15ACA05D70 ?
2B15ACA05C40 ? 2B15ACA04DB0 ?
2B15ACA04CF0 ? 000000000 ?
joet_create_root_th signal __restore_rt() 2B15ACEC5A4F ? 2B15ACE77F78 ?
read_group()+140 2B15ACF00C88 ? 2B15ACE71F08 ?
2B15ACE73EB8 ? 2B15ACE77F78 ?
joet_aux_thread_mai call joet_create_root_th 2B15ACEC5A4F ? 2B15ACE77F78 ?
n()+7523 read_group() 2B15ACF00C88 ? 2B15ACE71F08 ?
2B15ACE73EB8 ? 2B15ACE77F78 ?
seoa_note_stack_out call joet_aux_thread_mai 7FFF18A8EBC0 ? 7FFF18A8EBA0 ?
side()+26 n() 2B15ACED476B ? 2B15ACF07342 ?
2B15ACF06242 ? 2B15ACE77F78 ?
joet_thread_main()+ call seoa_note_stack_out 7FFF18A8EBC0 ? 2B15ACED476B ?
32 side() 2B15ACED476B ? 2B15ACF07342 ?
2B15ACF06242 ? 2B15ACE77F78 ?
从Oracle官网mos上查询到文档 ID 1940340.1，看报错提示一致，但是涉及的数据库版本不一致，根据文档 ID 1940340.1提
示，原因可能是集群应用了PSU、CPU补丁集引入的新BUG，相关的处理见文档 ID 1940340.1。
文档 ID 1940340.1原文如下（仅供参考）

Applies to:

Oracle Database - Enterprise Edition - Version 11.2.0.3 to 11.2.0.3 [Release 11.2]
Information in this document applies to any platform.

Symptoms

ORA-07445: exception encountered: core dump [joet_create_root_thread_group()+145] is seen at startup.

The issue happens in CDC environments and the following statement is seen in the alert log:

----- Current SQL Statement for this session (sql_id=f03bykh6st1m5) -----
BEGIN
      /* NOP UNLESS A TABLE OBJECT */
      IF dictionary_obj_type = 'TABLE'
      THEN
        sys.dbms_cdc_ipublish.change_table_trigger(dictionary_obj_owner,dictionary_obj_name,sysevent);
      END IF;
      END;
----- PL/SQL Stack -----
----- PL/SQL Call Stack -----
object      line object
handle    number name
0xa7dd71c0        12 package body SYS.DBMS_CDC_IPUBLISH

The error can also be seen from MMON_SLAVE with an action of : JAVAVM JIT slave action without any SQL statement referenced.

The call stack is:

joet_create_root_th <- read_group <- joet_aux_thread_mai
<- seoa_note_stack_out <- side <- joet_thread_main <- sjontlo_initialize <- joe_enter_vm
<- ioei_execute <- ioesub_EXECUTE <- seoa_note_stack_out <- side <- ioe_execute
<- side <- jox_ioe_execute <- joxgrovel_jitsomeme <- thods <- joxgroveldrv
<- thods <- joxjitexe <- joxrdp_ <- joxrdp <- kebm_slave_main
<- ksvrdp <- opirip <- opidrv <- sou2o <- opimai_real
<- ssthrdmain <- main <- libc_start_main <- start

joet_create_root_thread_group <- joet_aux_thread_main
<- seoa_note_stack_outside <- joet_thread_main <- sjontlo_initialize <- joe_enter_vm

Changes

Apply Patch 19791427 : COMBO OF OJVM COMPONENT 11.2.0.3.1 DB PSU + DB PSU 11.2.0.3.12 (OCT2014)

OR

Patch 19282015: ORACLE JAVAVM COMPONENT 11.2.0.3.1 DATABASE PSU (OCT2014) +
Patch 19121548: DATABASE PATCH SET UPDATE 11.2.0.3.12 (INCLUDES CPUOCT2014)

OR ONLY

Patch 19282015 "ORACLE JAVAVM COMPONENT 11.2.0.3.1 DATABASE PSU (OCT2014)

Cause

The cause is a problem within the Patch and it is investigated by Development in active bugs:

BUG 19904315 - RECEIVE ORA-07445[JOET_CREATE_ROOT_THREAD_GROUP()] AFTER APPLYING PATCH 19791427

Bug 19838078 - ORA-7445 [JOET_CREATE_ROOT_THREAD_GROUP()+136] ON SMON CACHE RECOVERY

Solution

To workaround this issue perform the next steps:

1/ Apply patch 19791427 or Patch 19282015 till the end of Installation steps and 3.3.1.

2/ Before running Postinstallation step (3.3.2), set the following init parameters so that JIT and job process do not start.

If spfile is used:

SQL> alter system set java_jit_enabled = FALSE;
SQL> alter system set "_system_trig_enabled"=FALSE;
SQL> alter system set JOB_QUEUE_PROCESSES=0;

3/ Startup instance in restricted mode and run postinstallation step.

SQL> startup restrict

4/ Run Postinstallation steps (3.3.2)

5/ Reset modified init parameters
SQL> alter system set java_jit_enabled = true;
SQL> alter system set "_system_trig_enabled"=TRUE;
SQL> alter system set JOB_QUEUE_PROCESSES=10;
-- or original JOB_QUEUE_PROCESSES value

5/ Restart instance as normal

References