ORA-600 [kddummy_blkchk] [18038] 一例-白红宇

一位客户的Oracle告警日志中出现了ORA-600 [kddummy_blkchk] [18038]故障，alert中的具体信息：

                         
Errors 
in
file
/u01/app/oracle/admin/prdw014a/udump/prdw014a_ora_4377
.trc:
ORA-00600: internal error code, arguments: [kddummy_blkchk], [222], [5792], [18038], [], [], [], []
Mon May 17 15:27:53 2010
Trace dumping is performing 
id
=[cdmp_20100517152753]
Mon May 17 15:27:53 2010
Doing block recovery 
for
file
2 block 504365
Block recovery from logseq 159276, block 166357 to scn 10934615778284
Mon May 17 15:27:53 2010
Recovery of Online Redo Log: Thread 1 Group 4 Seq 159276 Reading mem 0
Mem
# 0: /u01/app/oracle/dataPRDW014/redo04a_1.log
Mem
# 1: /u01/app/oracle/dataPRDW014/redo04a_2.log
Block recovery completed at rba 159276.167277.16, scn 2545.3924010007
Doing block recovery 
for
file
222 block 5792
Block recovery from logseq 159276, block 84741 to scn 10934615778283
Mon May 17 15:27:53 2010
Recovery of Online Redo Log: Thread 1 Group 4 Seq 159276 Reading mem 0
Mem
# 0: /u01/app/oracle/dataPRDW014/redo04a_1.log
Mem
# 1: /u01/app/oracle/dataPRDW014/redo04a_2.log
Block recovery completed at rba 159276.167277.16, scn 2545.3924009964
Mon May 17 15:27:55 2010
Block recovery completed at rba 159276.167277.16, scn 2545.3924009964
Mon May 17 15:27:55 2010
Corrupt Block Found
TSN = 67, TSNAME = OBA_DATA
RFN = 222, BLK = 5792, RDBA = 931141280
OBJN = 1657288, OBJD = 1699775, OBJECT = W_ORG_DS, SUBOBJECT =
SEGMENT OWNER = BMS_OBA_DW, SEGMENT TYPE = Table Segment
Mon May 17 15:32:56 2010
Trace dumping is performing 
id
=[cdmp_20100517153255]
        
      

附600错误产生的trace信息：

                         
/u01/app/oracle/admin/prdw014a/udump/prdw014a_ora_4377
.trc
Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - 64bit Production
With the Partitioning, Real Application Clusters, OLAP, Data Mining Scoring Engine
and Real Application Testing options
ORACLE_HOME = 
/u01/app/oracle/product/102prdw014
System name: SunOS
Node name: v08k405
Release: 5.9
Version: Generic_122300-29
Machine: sun4u
Instance name: prdw014a
Redo thread mounted by this instance: 1
Oracle process number: 109
Unix process pid: 4377, image: oracle@v08k405
*** 2010-05-17 15:23:15.391
*** ACTION NAME:() 2010-05-17 15:23:15.389
*** MODULE NAME:(pmdtm@v04k413 (TNS V1-V3)) 2010-05-17 15:23:15.389
*** SERVICE NAME:(prdw014_taf) 2010-05-17 15:23:15.389
*** SESSION ID:(789.48811) 2010-05-17 15:23:15.389
TYP:0 CLS: 4 AFN:222 DBA:0x378016a0 OBJ:1699775 SCN:0x09f1.e9e3a3eb SEQ: 2 OP:14.4
kteop redo - redo operation on extent map
RESIZE: entry:0 delta:
...
..
..
ksedmp: internal or fatal error
ORA-00600: internal error code, arguments: [kddummy_blkchk], [222], [5792], [18038], [], [], [], []
Current SQL statement 
for
this session:
INSERT /*+ SYS_DL_CURSOR */ INTO bms_oba_dw.W_ORG_DS (
"W_CUSTOMER_CLASS"
,
"NAME"
,
"ST_ADDRESS"
,
"CITY"
,
"STATE"
,
"ZIPCODE"
,
"COUNTRY"
,
"CUST_TYPE_CODE"
,
"CUST_TYPE_NAME"
,
"ACTIVE_FLG"
,
"DOM_ULT_DUNS_NUM"
,
"DUNS_NUM"
,
"EMP_COUNT"
,
"FORMED_DT"
,
"GLBLULT_DUNS_NUM"
,
"ANNUAL_REVENUE"
,
"BRANCH_FLG"
,
"BIRTH_DT"
,
"NO_OF_CHILDREN"
,
"LEGAL_NAME"
,
"FAMILY_NAME"
,
"OTHER_NAME"
,
"PREFERRED_NAME"
,
"INDV_ADDNL_TITLE"
,
"INDV_TITLE"
,
"INDV_MARITAL_STATE"
,
"INDV_GENDER"
,
"EMAIL_ADDRESS"
,
"RELATIONSHIP_STATE"
,
"INDV_EMP_STATUS"
,
"FAX_NUM"
,
"PAGER_NUM"
,
"MOBILE_NUM"
,
"LIFE_CYCLE_STATE"
,
"CUST_CAT_CODE"
,
"CUST_CAT_NAME"
,
"SIC_CODE"
,
"SIC_NAME"
,
"GOVT_ID_TYPE"
,
"GOVT_ID_VALUE"
,
"DUNNS_SITE_NAME"
,
"DUNNS_GLOBAL_NAME"
,
"DUNNS_LEGAL_NAME"
,
"CUSTOMER_NUM"
,
"ALT_CUSTOMER_NUM"
,
"ALT_PHONE_NUM"
,
"INTERNET_HOME_PAGE"
,
"LEGAL_STRUCT_CODE"
,
"LEGAL_STRUCT_NAME"
,
"DIRECT_MKTG_FLG"
,
"SOLICITATION_FLG"
,
"CUSTOMER_HIER1_CODE"
,
"CUSTOMER_HIER1_NAME"
,
"CUSTOMER_HIER2_CODE"
,
"CUSTOMER_HIER2_NAME"
,
"CUSTOMER_HIER3_CODE"
,
"CUSTOMER_HIER3_NAME"
,
"CUSTOMER_HIER4_CODE"
,
"CUSTOMER_HIER4_NAME"
,
"CUSTOMER_HIER5_CODE"
,
"CUSTOMER_HIER5_NAME"
,
"CUSTOMER_HIER6_CODE"
,
"CREATED_BY_ID"
,
"CHANGED_BY_ID"
,
"CREATED_ON_DT"
,
"CHANGED_ON_DT"
,
"AUX1_CHANGED_ON_DT"
,
"AUX2_CHANGED_ON_DT"
,
"AUX3_CHANGED_ON_DT"
,
"AUX4_CHANGED_ON_DT"
,
"SRC_EFF_FROM_DT"
,
"SRC_EFF_TO_DT"
,
"DELETE_FLG"
,
"DATASOURCE_NUM_ID"
,
"INTEGRATION_ID"
,
"TENANT_ID"
,
"X_CUSTOM"
,
"MOT_ATTRIBUTE1"
,
"MOT_ATTRIBUTE2"
,
"MOT_ATTRIBUTE3"
,
"MOT_ATTRIBUTE4"
,
"MOT_ATTRIBUTE5"
,
"MOT_ATTRIBUTE6"
,
"MOT_ATTRIBUTE7"
,
"MOT_ATTRIBUTE8"
,
"MOT_ATTRIBUTE9"
,
"MOT_ATTRIBUTE10"
,
"MOT_ATTRIBUTE11"
,
"MOT_ATTRIBUTE12"
,
"MOT_ATTRIBUTE13"
,
"MOT_ATTRIBUTE14"
,
"MOT_ATTRIBUTE15"
,
"MOT_ATTRIBUTE16"
,
"MOT_ATTRIBUTE17"
,
"MOT_ATTRIBUTE18"
,
"MOT_ATTRIBUTE19"
,
"MOT_ATTRIBUTE20"
,
"MOT_PARTY_TYPE"
,
"MOT_PHONE_AREA_CODE"
,
"MOT_ORIG_SYSTEM_REFERENCE"
,
"MOT_PER_EMAIL_ADDR"
,
"MOT_PERSON_FIRST_NAME"
,
"MOT_PHONE_EXTENSION"
,
"MOT_ALTERNATE_NAME"
,
"MOT_TELEPHONE_TYPE"
,
"MOT_SALES_CHANNEL_CODE"
,
"MOT_ACCOUNT_NAME"
,
"MOT_ATTRIBUTE_CATEGORY"
,
"MOT_INTERCOMPANY_FLAG"
,
"MOT_PARTY_NUMBER"
,
"MOT_PARTY_ID"
,
"MOT_LAST_UPDATE_LOGIN"
,
"MOT_CUST_CLASS_DESC"
,
"MOT_RECEIPT_METHOD_NAME"
,
"MOT_PHONE_NUMBER"
,
"MOT_CONTACT_POINT_PURPOSE"
,
"MOT_SALESREP_NAME"
,
"MOT_PAY_TERMS_CODE"
,
"MOT_PAY_TERMS_NAME"
) VALUES (NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL)
----- Call Stack Trace -----
calling              call     entry                argument values 
in
hex
location             
type
point                (? means dubious value)
-------------------- -------- -------------------- ----------------------------
ksedmp()+744         CALL     ksedst()             000000840 ?
                                                   
FFFFFFFF7FFF620C ?
                                                   
000000000 ?
                                                   
FFFFFFFF7FFF2D00 ?
                                                   
FFFFFFFF7FFF1A68 ?
                                                   
FFFFFFFF7FFF2468 ?
kgerinv()+200        PTR_CALL 0000000000000000     000106800 ? 10681C1C4 ?
                                                   
10681C000 ? 00010681C ?
                                                   
000106800 ? 10681C1C4 ?
kseinpre()+96        CALL     kgerinv()            106816B18 ? 000000000 ?
                                                   
1064564C0 ? 000000003 ?
                                                   
FFFFFFFF7FFF6750 ?
                                                   
000001430 ?
ksesin()+52          CALL     kseinpre()           000106800 ? 000000003 ?
                                                   
00000025F ? 10681C1B8 ?
                                                   
FFFFFFFF7FFF6750 ?
                                                   
1068167D8 ?
kco_blkchk()+2568    CALL     ksesin()             1064564C0 ? 000000003 ?
                                                   
000106800 ? 0000000DE ?
                                                   
000000000 ? 000106800 ?
kcoapl()+1284        CALL     kco_blkchk()         0001900DE ? 0378016A0 ?
                                                   
0000016A0 ? 00000FC00 ?
                                                   
000000000 ?
                                                   
FFFFFFFF7FFF89F8 ?
kcbapl()+412         CALL     kcoapl()             000000002 ? 000002300 ?
                                                   
000105800 ? 583DBC000 ?
                                                   
106816C98 ? 00010598F ?
kcrfw_redo_gen()+16  CALL     kcbapl()             FFFFFFFF7FFF89B8 ?
376                                                583FB7870 ?
                                                   
FFFFFFFF7AF3AA3C ?
                                                   
B6E9FABD0 ? 000000000 ?
                                                   
583DBC000 ?
kcbchg1_main()+1363  CALL     kcrfw_redo_gen()     000000000 ?
2                                                  FFFFFFFF7FFF76C8 ?
                                                   
B693A9998 ? 000000000 ?
                                                   
3800135A0 ?
                                                   
FFFFFFFF7FFF7700 ?
kcbchg1()+1324       CALL     kcbchg1_main()       000100C00 ?
                                                   
FFFFFFFF7FFF7850 ?
                                                   
000000000 ? 583FB7870 ?
                                                   
000000000 ? 00000FFFF ?
ktuchg()+968         CALL     kcbchg1()            000106819 ? 1068195B8 ?
                                                   
1068195C8 ? 106819000 ?
                                                   
000000000 ? 106819000 ?
ktbchg2nt()+104      CALL     ktuchg()             000000002 ? 000000001 ?
                                                   
FFFFFFFF7FFF8928 ?
                                                   
B67A76DD8 ? 000000000 ?
                                                   
000000000 ?
kteopgen()+728       CALL     ktbchg2nt()          FFFFFFFF7FFF89B8 ?
                                                   
FFFFFFFF7FFF87C4 ?
                                                   
000000000 ? 000000000 ?
                                                   
FFFFFFFF7FFF8928 ?
                                                   
FFFFFFFF7FFF9D98 ?
kteopresize()+2276   CALL     kteopgen()           FFFFFFFF7FFF89B8 ?
                                                   
000000006 ? 000106800 ?
                                                   
000000002 ? 10682247C ?
                                                   
106816B18 ?
ktsxbmdelext1()+968  CALL     kteopresize()        FFFFFFFF7FFF9D98 ?
8                                                  FFFFFFFF7FFF9E88 ?
                                                   
000000004 ? 000000002 ?
                                                   
000000000 ? 000000000 ?
ktsstrm_segment()+6  CALL     ktsxbmdelext1()      FFFFFFFF7AD33A78 ?
308                                                0000016A0 ? 0003FFFFF ?
                                                   
FFFFFFFF7AD33A78 ?
                                                   
106822000 ? 000000043 ?
ktsmg_trimf()+1208   CALL     ktsstrm_segment()    000000000 ? 000000003 ?
                                                   
000000001 ? 000100C00 ?
                                                   
106819000 ? 000000000 ?
kdbltrmt()+1916      CALL     ktsmg_trimf()        00010598F ? 0000010E2 ?
                                                   
106822478 ? 000000005 ?
                                                   
10682247C ? 106816B18 ?
kdblfpl()+96         CALL     kdbltrmt()           000000006 ? 000000000 ?
                                                   
FFFFFFFF7AD33918 ?
                                                   
000000180 ? 0000010E4 ?
                                                   
000000008 ?
kdblfl()+1948        CALL     kdblfpl()            FFFFFFFF7FFFB0AC ?
                                                   
FFFFFFFF7AD33918 ?
                                                   
000000000 ?
                                                   
FFFFFFFF7AD33AE0 ?
                                                   
FFFFFFFF7AD33A68 ?
                                                   
000000000 ?
klafin()+160         CALL     kdblfl()             FFFFFFFF7FFFB0AC ?
                                                   
FFFFFFFF7AD33918 ?
                                                   
000000000 ? 000000001 ?
                                                   
000000008 ? 000106800 ?
kpodpfin()+76        CALL     klafin()             FFFFFFFF7AF35C40 ?
                                                   
1059BF2B8 ? 000000321 ?
                                                   
FFFFFFFF7AD33918 ?
                                                   
000000000 ? 000400000 ?
kpodpmop()+320       CALL     kpodpfin()           FFFFFFFF7AF35C40 ?
                                                   
000106816 ? 000106800 ?
                                                   
000000321 ? 000000001 ?
                                                   
FFFFFFFF7AF35BC8 ?
opiodr()+1496        PTR_CALL 0000000000000000     000000301 ? 000000321 ?
        
      

进过与Oracle support确认,定位为Bug 5386204 - Block corruption / OERI[kddummy_blkchk] after direct load of ASSM segment [ID 5386204.8]. "kteop redo - redo operation on extent map" 记录是确定该Bug的一个重要依据。该Bug的Oracle note: Bug 5386204

本文转自maclean_007 51CTO博客，原文链接:http://blog.51cto.com/maclean/1277095