Errors in ASM, after upgrade to 11gR2

Alert: Querying v$asm_file Gives ORA-15196 After ASM Was Upgraded From 10gR2 To 11gR2 [ID 1145365.1]




Oracle Server - Enterprise Edition - Version: 11.2.0.1.0 to 11.2.0.1 - Release: 11.2 to 11.2
Information in this document applies to any platform.

This problem has been reported for 10gR2 created disk groups with the AU size greater than 1MB and that have fewer than 3585 user files and after migrating to 11.2 ASM.

SQL> select name,allocation_unit_size from v$asm_diskgroup, will show you the AU size.

Likelihood of Occurrence

1) 11.2 detects no formatted blocks beyond blocks that hold < 3585 user asm file when the au size is > than 1MB (the default)

2) This is identified due to the unpublished Bug 5100163

Possible Symptoms

. After upgrading a 10gR2 ASM instance to 11.2, ASM is able to mount the disk groups but these are dismounted as soon as v$asm_file is queried:

SUCCESS: diskgroup DATA_DG01 was mounted
SUCCESS: ALTER DISKGROUP DATA_DG01 MOUNT /* asm agent */
Thu Jun 24 15:13:05 2010
NOTE: diskgroup resource ora.DATA_DG01.dg is online
Thu Jun 24 15:18:31 2010
WARNNING: cache read a corrupted block group=DATA_DG01 fn=1 blk=512 from disk 0
NOTE: a corrupted block from group DATA_DG01 was dumped to
/u000/app/grid/diag/asm/+asm/+ASM/trace/+ASM_ora_348464.trc
WARNNING: cache read(retry) a corrupted block group=DATA_DG01 fn=1 blk=512
from disk 0
ERROR: cache failed to read group=DATA_DG01 fn=1 blk=512 from disk(s): 0
DATA_DG01_0000
ORA-15196: invalid ASM block header [kfc.c:23925] [hard_kfbh] [1] [512] [0 !=130]

Workaround or Resolution

1) Run the following from 11.2 ASM on each disk group a few times (say 3 times):

SQL> ALTER DISKGROUP <DG_NAME> CHECK ALL REPAIR;

 On the first time they execute this command, the ASM alert log will show entries like:
 "ERROR: file 1 extent 0: blocks XX to XX are unformatted"
 The next runs, the ASM alert logs will show something like:
 ERROR: file 1 extent 0: blocks XX to XX are unformatted"
 "SUCCESS: file 1 extent 0 repaired"

2) Check if ASM in 11.2 still dismounts the disk group when querying v$asm_file.

3) If the disk group continues to be dismounted with the above errors, then restore the 10gR2 environment (like prior to the ASM upgrade) and then apply Patch 5100163 to the 10.2.0.4 ASM home
To obtain the patch go to MyOracleSupport, if the following link does not work:

http://updates.oracle.com/download/5100163.html

4) After the patch was installed, then run the ALTER DISKGROUP...CHECK ALL REPAIR on all the disk groups. Then retry the ASM upgrade.

Patches

1) Before upgrade ASM from release 10.2 to release 11.2 please apply the Patch 5100163 on the 10.2 ASM Oracle Home

2) This patch can be obtained  from MOS:

http://updates.oracle.com/download/5100163.html