ORA-15081: failed to submit an I/O operation to a disk

corruptionoracleoracle-11g-r2recoveryrman

I have a test environment for Oracle single instance on ASM 11.2.0.3.

I have restored database from tape and during the recovery, the system crashed by some I/O error.

I restarted the server, mounted the database and run validate on archivelog(thinking that the blocks in this archivelog may be corrupted)

RMAN> validate archivelog sequence=2994;

RMAN-03009: failure of validate command on ORA_DISK_1 channel at 02/25/2014 12:24:13
ORA-19501: read error on file "+DATA01/crimesys/archivelog/2014_02_24/thread_1_seq_2994.294.8
ORA-15081: failed to submit an I/O operation to a disk
ORA-15081: failed to submit an I/O operation to a disk

And the system crushed again.

Note: Other validates are successful just this archivelog causes the system crash.

Details:

–Asm disk

ls -la /dev/oracleasm/asm_disk1
brw-rw---- 1 grid asmdba 253, 2 Feb 25 13:13 /dev/oracleasm/asm_disk1

–Trace info

more /u01/app/oracle/diag/rdbms/crimesys/crimesys/trace/crimesys_ora_2696.

WARNING: Read Failed. group:1 disk:0 AU:114682 offset:512 size:1048064
path:/dev/oracleasm/asm_disk1
         incarnation:0xe968a075 asynchronous result:'I/O error'
         subsys:System iop:0x7fd93c64b680 bufp:0x7fd93b64e000 osderr:0x0 osderr1:0x0
WARNING: failed to read mirror side 1 of virtual extent 13 logical extent 0 of file 294 in gr
oup [1.353915012] from disk DATA01_0000  allocation unit 114682 reason error; if possible, wi
ll try another mirror side

–Also when I check the disk of ASM it is successful.

SQL> alter diskgroup data01 check 

NOTE: starting check of diskgroup DATA01
Tue Feb 25 13:36:59 2014
GMON checking disk 0 for group 1 at 5 for pid 18, osid 2181
SUCCESS: check of diskgroup DATA01 found no errors
SUCCESS: alter diskgroup data01 check

How to solve? Is the virtual block corrupted? How to check?
Please help.

Best Answer

If your are using multi-path make sure that your O.S. does not point to the file system instead of the disk groups. it happened to me 2 times after box reboot.


This is to answer your question:

SQL> set pagesize 800
SQL> select name, label, path from v$asm_disk;
NAME                           LABEL
------------------------------ -------------------------------
PATH

DGER_0000    /dev/oracleasm/disks/DSK5

DGSTBY_0000 /dev/oracleasm/disks/DSKPATHY


run the follwing command :

 /etc/init.d/oracleasm querydisk -d <your disk here>

post the output !