Need urgent help on ASM issue – disk header status problem

If you cannot recover the data by yourself, ask Parnassusdata, the professional ORACLE database recovery team for help.

Parnassusdata Software Database Recovery Team

Service Hotline:  +86 13764045638 E-mail: service@parnassusdata.com

 

 

 
ODA system
+In order to workaround a known issue (startup hang when using Hitachi disks) FE/customer was in the process of replacing Hitachi drives on the system.
+ They pulled 2 disks out simultaneously and new disks put in
+ Diskgroups (DATA and RECO) dismounted  – as diskgroups built in  NORMAL redundancy .
+ Clusterware went down and realizing the problem- customer reinstated the original disks.

Current issue:
Disk groups are not mounting.
ASM disks from slot 0 are not being seen by ASM
ASM disks from slot 1 are being seen, but reported as new disks ( Header status=CANDIDATE)

Mounting the diskgroup with FORCE option has also not helped ( because there is 1 disk from slot 0 missing and 1 disk from slot 1 being reported as candidate)
** Customer has no backup and he needs to find out if it is fixable, or the system needs to be rebuilt from scratch.

 

--------------------------------------------------------------------------------
 Disk          Size Header    Path                                     Disk Group   User     Group   
================================================================================
   1:     491520 Mb CANDIDATE /dev/mapper/HDD_E0_S01_717882548p1       #            grid     asmadmin
   2:      75080 Mb CANDIDATE /dev/mapper/HDD_E0_S01_717882548p2       #            grid     asmadmin
   3:     491520 Mb MEMBER    /dev/mapper/HDD_E0_S04_717894368p1       DATA         grid     asmadmin
   4:      75080 Mb MEMBER    /dev/mapper/HDD_E0_S04_717894368p2       RECO         grid     asmadmin
   5:     491520 Mb MEMBER    /dev/mapper/HDD_E0_S05_717844560p1       DATA         grid     asmadmin
   6:      75080 Mb MEMBER    /dev/mapper/HDD_E0_S05_717844560p2       RECO         grid     asmadmin
   7:     491520 Mb MEMBER    /dev/mapper/HDD_E0_S08_717882264p1       DATA         grid     asmadmin
   8:      75080 Mb MEMBER    /dev/mapper/HDD_E0_S08_717882264p2       RECO         grid     asmadmin
   9:     491520 Mb MEMBER    /dev/mapper/HDD_E0_S09_717844480p1       DATA         grid     asmadmin
  10:      75080 Mb MEMBER    /dev/mapper/HDD_E0_S09_717844480p2       RECO         grid     asmadmin
  11:     491520 Mb MEMBER    /dev/mapper/HDD_E0_S12_717844976p1       DATA         grid     asmadmin
  12:      75080 Mb MEMBER    /dev/mapper/HDD_E0_S12_717844976p2       RECO         grid     asmadmin
  13:     491520 Mb MEMBER    /dev/mapper/HDD_E0_S13_717845048p1       DATA         grid     asmadmin
  14:      75080 Mb MEMBER    /dev/mapper/HDD_E0_S13_717845048p2       RECO         grid     asmadmin
  15:     491520 Mb MEMBER    /dev/mapper/HDD_E0_S16_717895116p1       DATA         grid     asmadmin
  16:      75080 Mb MEMBER    /dev/mapper/HDD_E0_S16_717895116p2       RECO         grid     asmadmin
  17:     491520 Mb MEMBER    /dev/mapper/HDD_E0_S17_717888848p1       DATA         grid     asmadmin
  18:      75080 Mb MEMBER    /dev/mapper/HDD_E0_S17_717888848p2       RECO         grid     asmadmin
  19:     491520 Mb MEMBER    /dev/mapper/HDD_E1_S02_717825396p1       DATA         grid     asmadmin
  20:      75080 Mb MEMBER    /dev/mapper/HDD_E1_S02_717825396p2       RECO         grid     asmadmin
  21:     491520 Mb MEMBER    /dev/mapper/HDD_E1_S03_717894252p1       DATA         grid     asmadmin
  22:      75080 Mb MEMBER    /dev/mapper/HDD_E1_S03_717894252p2       RECO         grid     asmadmin
  23:     491520 Mb MEMBER    /dev/mapper/HDD_E1_S06_717886840p1       DATA         grid     asmadmin
  24:      75080 Mb MEMBER    /dev/mapper/HDD_E1_S06_717886840p2       RECO         grid     asmadmin
  25:     491520 Mb MEMBER    /dev/mapper/HDD_E1_S07_717888592p1       DATA         grid     asmadmin
  26:      75080 Mb MEMBER    /dev/mapper/HDD_E1_S07_717888592p2       RECO         grid     asmadmin
  27:     491520 Mb MEMBER    /dev/mapper/HDD_E1_S10_717843708p1       DATA         grid     asmadmin
  28:      75080 Mb MEMBER    /dev/mapper/HDD_E1_S10_717843708p2       RECO         grid     asmadmin
  29:     491520 Mb MEMBER    /dev/mapper/HDD_E1_S11_717852256p1       DATA         grid     asmadmin
  30:      75080 Mb MEMBER    /dev/mapper/HDD_E1_S11_717852256p2       RECO         grid     asmadmin
  31:     491520 Mb MEMBER    /dev/mapper/HDD_E1_S14_717895376p1       DATA         grid     asmadmin
  32:      75080 Mb MEMBER    /dev/mapper/HDD_E1_S14_717895376p2       RECO         grid     asmadmin
  33:     491520 Mb MEMBER    /dev/mapper/HDD_E1_S15_717843800p1       DATA         grid     asmadmin
  34:      75080 Mb MEMBER    /dev/mapper/HDD_E1_S15_717843800p2       RECO         grid     asmadmin
  35:     491520 Mb MEMBER    /dev/mapper/HDD_E1_S18_717882696p1       DATA         grid     asmadmin
  36:      75080 Mb MEMBER    /dev/mapper/HDD_E1_S18_717882696p2       RECO         grid     asmadmin
  37:     491520 Mb MEMBER    /dev/mapper/HDD_E1_S19_717849420p1       DATA         grid     asmadmin
  38:      75080 Mb MEMBER    /dev/mapper/HDD_E1_S19_717849420p2       RECO         grid     asmadmin
  39:      70005 Mb MEMBER    /dev/mapper/SSD_E0_S20_805725574p1       REDO         grid     asmadmin
  40:      70005 Mb MEMBER    /dev/mapper/SSD_E0_S21_805708282p1       REDO         grid     asmadmin
  41:      70005 Mb MEMBER    /dev/mapper/SSD_E1_S22_805706766p1       REDO         grid     asmadmin
  42:      70005 Mb MEMBER    /dev/mapper/SSD_E1_S23_805706623p1       REDO         grid     asmadmin
--------------------------------------------------------------------------------
ORACLE_SID ORACLE_HOME                                                          
================================================================================
     +ASM1 /u01/app/11.2.0.3/grid                                               
     +ASM2 /u01/app/11.2.0.3/grid                                               

 
What is the backup block status ,

kfed read <device_name> aunum=1 blknum=254

Does it shows proper header ,if so the run kfed repair command.

if other blocks are fine except header this will work … else on next mount while doing COD recovery ,it will crash .

 

ASM log file info
==============

NOTE: cache closing disk 0 of grp 1: (not open) _DROPPED_0000_DATA
ERROR: Disk 1 cannot be offlined, since all the disks [1, 0] with mirrored data would be offline.
ERROR: too many offline disks in PST (grp 1)

Here the disks 0,1 have been put back in respective slots but still same issue.

 

Seems similar to the one described in ORA-15042: ASM disk is missing after add disk took place (Doc ID 1529397.1)

 

Comment

*

沪ICP备14014813号-2

沪公网安备 31010802001379号