SYMPTOMS

On : 10.2.0.4 version, STORAGE

ASM failed to start 

after server reboot, the disk group can not mount,If you try to mount it you may get below error

 SQL> alter diskgroup DATA MOUNT;
alter diskgroup DATA MOUNT
*
ERROR at line 1:
ORA-15032: not all alterations performed
ORA-15024: discovered duplicately numbered ASM disk 4

CHANGES

 There could have been duplication of Major/Minor number at the os level.

CAUSE

Since there is a duplication of  Major/Minor Number when asm is trying to mount a disk group it discovers two disks with same Major/minor number and there is no way asm can determine which one is the one it was using before dismount.

 SQL> alter diskgroup DATA MOUNT;
alter diskgroup DATA MOUNT
*
ERROR at line 1:
ORA-15032: not all alterations performed
ORA-15024: discovered duplicately numbered ASM disk 4

 
After reboot if suddenly you get duplicate numbered ASM disk with serial number of disk it could be because of duplication of Major/Minor number whcih can be confirmed from 

cat /proc/partitions and tallying it with o/p of ls -ltr /dev/sd*

[grid@stgrac1 lib]$ cat /proc/partitions
major minor  #blocks  name

   8     0  156250000 sda
   8     1     200781 sda1
   8     2   30716280 sda2
   8     3    8193150 sda3
   8     4          0 sda4
   8     5    4096543 sda5
   8     6  113041341 sda6
   8    16     262144 sdb
   8    17     262114 sdb1
   8    32    2588672 sdc
   8    33    1270969 sdc1
   8    34    1316756 sdc2
   8    48   20480000 sdd
   8    49    1001456 sdd1
   8    50    1001472 sdd2
   8    51    1001472 sdd3
   8    81          0 sdd4<<<<<<<<<<<<<<<<<<<<
   8    53    2000880 sdd5
   8    54    2000880 sdd6
   8    55    2000880 sdd7
   8    64     262144 sde
   8    65     262114 sde1
   8    80    2785280 sdf
   8    81    1003873 sdf1<<<<<<<<<<<<<<<<<<<<
   8    82          1 sdf2
[grid@stgrac1 lib]$ ls -ltr /dev/sd*
brw-r----- 1 root disk 8,  0 Apr  5 18:35 /dev/sda
brw-r----- 1 root disk 8,  5 Apr  5 18:35 /dev/sda5
brw-r----- 1 root disk 8,  3 Apr  5 18:36 /dev/sda3
brw-r----- 1 root disk 8,  2 Apr  5 18:36 /dev/sda2
brw-r----- 1 root disk 8,  1 Apr  5 18:36 /dev/sda1
brw-r----- 1 root disk 8, 32 Apr  5 19:29 /dev/sdc
brw-r----- 1 root disk 8, 16 Apr  5 19:29 /dev/sdb
brw-r----- 1 root disk 8, 48 Apr  5 19:29 /dev/sdd
brw-r----- 1 root disk 8, 64 Apr  5 19:29 /dev/sde
brw-r----- 1 root disk 8, 80 Apr  5 19:29 /dev/sdf
brw-r----- 1 root disk 8, 49 Apr  5 19:29 /dev/sdd1
brw-r----- 1 root disk 8, 54 Apr  5 19:29 /dev/sdd6
brw-r----- 1 root disk 8, 50 Apr  5 19:29 /dev/sdd2
brw-r----- 1 root disk 8, 51 Apr  5 19:29 /dev/sdd3
brw-r----- 1 root disk 8, 53 Apr  5 19:29 /dev/sdd5
brw-r----- 1 root disk 8,  4 Apr  6 10:02 /dev/sda4
brw-r----- 1 root disk 8,  6 Apr  6 10:02 /dev/sda6
brw-r----- 1 root disk 8, 55 Apr  6 10:02 /dev/sdd7
brw-r----- 1 root disk 8, 81 Apr  6 10:02 /dev/sdd4<<<<<<<<<<<<<<<<<<<<
brw-r----- 1 root disk 8, 82 Apr  6 11:11 /dev/sdf2
brw-r----- 1 root disk 8, 65 Apr  8 00:33 /dev/sde1
brw-r----- 1 root disk 8, 34 Apr  8 00:33 /dev/sdc2
brw-r----- 1 root disk 8, 17 Apr  8 00:33 /dev/sdb1
brw-r----- 1 root disk 8, 33 Apr  8 00:33 /dev/sdc1
brw-r----- 1 root disk 8, 81 Apr  8 00:36 /dev/sdf1<<<<<<<<<<<<<<<<<<<<