问题描述：开发人员在往数据库中插入（仅一张表插入数据有次问题，其他表都正常）数据时出现异常信息，无法插入数据，具体报错信息见后文。

告警日志内容：

Wed Oct 25 11:42:47 2017

Errors in file /u01/app/oracle/diag/rdbms/sjjhdb/sjjhdb1/trace/sjjhdb1_ora_11477.trc  (incident=1128284):

ORA-00600: 内部错误代码, 参数: [ktspfmdb:objdchk_kcbnew_3], [1], [217404], [4], [], [], [], [], [], [], [], []

Use ADRCI or Support Workbench to package the incident.

See Note 411.1 at My Oracle Support for error and packaging details.

Wed Oct 25 11:43:03 2017

Sweep [inc][1128284]: completed

 

Trace文件内容：

[root@zwdb1 ~]# more /u01/app/oracle/diag/rdbms/sjjhdb/sjjhdb1/trace/sjjhdb1_ora_9990.trc

DDE: Problem Key 'ORA 600 [ktspfmdb:objdchk_kcbnew_3]' was flood controlled (0x6) (incident: 1128272)

ORA-00600: 内部错误代码, 参数: [ktspfmdb:objdchk_kcbnew_3], [1], [217404], [4], [], [], [], [], [], [], [], []

 

*** 2017-10-25 10:48:28.311

BH (0x4a78b1818) file#: 15 rdba: 0x03d8c801 (15/1624065) class: 8 ba: 0x4a4582000

  set: 91 pool: 3 bsz: 8192 bsi: 0 sflg: 1 pwc: 0,25

  dbwrid: 2 obj: 229898 objn: 229898 tsn: 12 afn: 15 hint: f

  hash: [0x8ecba7f98,0x8ecba7f98] lru: [0x76fb0efa8,0x3f7a7d3c8]

  lru-flags: on_auxiliary_list

  ckptq: [NULL] fileq: [NULL] objq: [0xb8383098,0xb8383098] objaq: [0xb8383088,0xb8383088]

  st: XCURRENT md: NULL fpin: 'ktspfwh17: ktspfundo' tch: 0 le: 0x44ffbfe10

  flags: block_written_once redo_since_read

  LRBA: [0x0.0.0] LSCN: [0x0.0] HSCN: [0xffff.ffffffff] HSUB: [103]

Dump of buffer cache at level 10 for tsn=12 rdba=64538625

BH (0x4a78b1818) file#: 15 rdba: 0x03d8c801 (15/1624065) class: 8 ba: 0x4a4582000

  set: 91 pool: 3 bsz: 8192 bsi: 0 sflg: 1 pwc: 0,25

  dbwrid: 2 obj: 229898 objn: 229898 tsn: 12 afn: 15 hint: f

  hash: [0x8ecba7f98,0x8ecba7f98] lru: [0x76fb0efa8,0x3f7a7d3c8]

  lru-flags: on_auxiliary_list

  ckptq: [NULL] fileq: [NULL] objq: [0xb8383098,0xb8383098] objaq: [0xb8383088,0xb8383088]

  st: XCURRENT md: NULL fpin: 'ktspfwh17: ktspfundo' tch: 0 le: 0x44ffbfe10

  flags: block_written_once redo_since_read

  LRBA: [0x0.0.0] LSCN: [0x0.0] HSCN: [0xffff.ffffffff] HSUB: [103]

GLOBAL CACHE ELEMENT DUMP (address: 0x44ffbfe10):

  id1: 0x18c801 id2: 0xf pkey: OBJ#229898 block: (15/1624065)

  lock: X rls: 0x0 acq: 0x0 latch: 12

  flags: 0x20 fair: 0 recovery: 0 fpin: 'ktspfwh17: ktspfundo'

  bscn: 0x4.9577b4bf bctx: (nil) write: 0 scan: 0x0

  lcp: (nil) lnk: [NULL] lch: [0x4a78b1950,0x4a78b1950]

  seq: 32299 hist: 113 238 180 67 143:0 208 352 197 48 121 113 238 180 113 238 180

    113

  LIST OF BUFFERS LINKED TO THIS GLOBAL CACHE ELEMENT:

    flg: 0x02200000 lflg: 0x4 state: XCURRENT tsn: 12 tsh: 0

      addr: 0x4a78b1818 obj: 229898 cls: 1ST LEVEL BMB bscn: 0x4.9577b4bf

  buffer tsn: 12 rdba: 0x03d8c801 (15/1624065)

  scn: 0x0004.9577b4bf seq: 0x01 flg: 0x04 tail: 0xb4bf2001

  frmt: 0x02 chkval: 0xabc8 type: 0x20=FIRST LEVEL BITMAP BLOCK

 

应用开发插入数据时的报错信息：

2017/10/25 10:54:53 - 插入 / 更新.0 -

2017/10/25 10:54:53 - 插入 / 更新.0 - at org.pentaho.di.core.database.Database.insertRow(Database.java:1266)

2017/10/25 10:54:53 - 插入 / 更新.0 - at org.pentaho.di.core.database.Database.insertRow(Database.java:1190)

2017/10/25 10:54:53 - 插入 / 更新.0 - at org.pentaho.di.core.database.Database.insertRow(Database.java:1178)

2017/10/25 10:54:53 - 插入 / 更新.0 - at org.pentaho.di.core.database.Database.insertRow(Database.java:1166)

2017/10/25 10:54:53 - 插入 / 更新.0 - at org.pentaho.di.trans.steps.insertupdate.InsertUpdate.lookupValues(InsertUpdate.java:114)

2017/10/25 10:54:53 - 插入 / 更新.0 - at org.pentaho.di.trans.steps.insertupdate.InsertUpdate.processRow(InsertUpdate.java:299)

2017/10/25 10:54:53 - 插入 / 更新.0 - ... 2 more

2017/10/25 10:54:53 - 插入 / 更新.0 - Caused by: java.sql.SQLException: ORA-00600: 内部错误代码, 参数: [ktspfmdb:objdchk_kcbnew_3], [1], [217404], [4], [], [], [], [], [], [], [], []

 

2017/10/25 10:54:53 - 插入 / 更新.0 - at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:439)

2017/10/25 10:54:53 - 插入 / 更新.0 - at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:395)

2017/10/25 10:54:53 - 插入 / 更新.0 - at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:802)

2017/10/25 10:54:53 - 插入 / 更新.0 - at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:436)

2017/10/25 10:54:53 - 插入 / 更新.0 - at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:186)

2017/10/25 10:54:53 - 插入 / 更新.0 - at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:521)

2017/10/25 10:54:53 - 插入 / 更新.0 - at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:205)

2017/10/25 10:54:53 - 插入 / 更新.0 - at oracle.jdbc.driver.T4CPreparedStatement.executeForRows(T4CPreparedStatement.java:1008)

2017/10/25 10:54:53 - 插入 / 更新.0 - at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1307)

2017/10/25 10:54:53 - 插入 / 更新.0 - at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3449)

2017/10/25 10:54:53 - 插入 / 更新.0 - at oracle.jdbc.driver.OraclePreparedStatement.executeUpdate(OraclePreparedStatement.java:3530)

2017/10/25 10:54:53 - 插入 / 更新.0 - at oracle.jdbc.driver.OraclePreparedStatementWrapper.executeUpdate(OraclePreparedStatementWrapper.java:1062)

2017/10/25 10:54:53 - 插入 / 更新.0 - at org.pentaho.di.core.database.Database.insertRow(Database.java:1233)

2017/10/25 10:54:53 - 插入 / 更新.0 - ... 7 more

2017/10/25 10:54:53 - 插入 / 更新.0 - ERROR (version 6.0-SNAPSHOT, build 1 from 2015-12-30 09.40.21 by huyongliang) : Unable to commit connection(s) :org.pentaho.di.core.exception.KettleDatabaseException:

2017/10/25 10:54:53 - 插入 / 更新.0 - Unable to get database metadata from this database connection

2017/10/25 10:54:53 - 插入 / 更新.0 - 关闭的连接

2017/10/25 10:54:53 - 插入 / 更新.0 - 完成处理 (I=12, O=11, R=12, W=11, U=0, E=1

2017/10/25 10:54:53 - 1142-jcms26_new2-to-0615-DT_RST - 转换被检测

2017/10/25 10:54:53 - 1142-jcms26_new2-to-0615-DT_RST - 转换正在杀死其他步骤!

2017/10/25 10:54:53 - 表输入jcms_visitcountdetail.0 - Finished reading query, closing connection.

2017/10/25 10:54:53 - 表输入jcms_visitcountdetail.0 - 完成处理 (I=10014, O=0, R=0, W=10013, U=0, E=0

2017/10/25 10:54:53 - 1142-jcms26_new2-to-0615-DT_RST - ERROR (version 6.0-SNAPSHOT, build 1 from 2015-12-30 09.40.21 by huyongliang) : 错误被检测到!

2017/10/25 10:54:53 - Spoon - 转换完成!!

2017/10/25 10:54:53 - 1142-jcms26_new2-to-0615-DT_RST - ERROR (version 6.0-SNAPSHOT, build 1 from 2015-12-30 09.40.21 by huyongliang) : 错误被检测到!

2017/10/25 10:54:53 - 1142-jcms26_new2-to-0615-DT_RST - ERROR (version 6.0-SNAPSHOT, build 1 from 2015-12-30 09.40.21 by huyongliang) : 错误被检测到!

 

通过搜索metalink信息，该问题与bug 17435354非常类似，文档 ID 2293210.1。但是该bug在11.2.0.3之后的版本中已经修复。通过检查节点2的日志信息，并未发现出现类似的错误告警。结合应用程序告警信息，考虑通过重启尝试解决该问题，但是由于是在白天系统正常使用时出现的问题，重启不被允许，数据库资源使用率都比较低，所以就尝试的通过刷新buffer cache解决问题。

SQL> alter system flush buffer_cache;

 

System altered.

刷新完成后，再让开发人员往表里插入数据，反馈已可以正常插入。通过后续跟踪观察，告警日志也未再出现ora-600的错误。