【Oracle ASM】新增ASM DISK遇到ORA-15080、ORA-15038等错误

SQL> alter diskgroup data add disk '/dev/asmc-disk11';
alter diskgroup data add disk '/dev/asmc-disk11'
*
ERROR at line 1:
ORA-15032: not all alterations performed
ORA-15038: disk '' mismatch on 'Sector Size' with target disk group [4096]
[512]

SQL>  create diskgroup TESTDG1 EXTERNAL REDUNDANCY DISK '/dev/asmc-disk11' ATTRIBUTE  'compatible.asm' = '11.2', 'compatible.rdbms' = '11.2', 'sector_size'='4096';
 create diskgroup TESTDG1 EXTERNAL REDUNDANCY DISK '/dev/asmc-disk11' ATTRIBUTE  'compatible.asm' = '11.2', 'compatible.rdbms' = '11.2', 'sector_size'='4096'
*
ERROR at line 1:
ORA-15018: diskgroup cannot be created
ORA-15080: synchronous I/O operation to a disk failed
ORA-15080: synchronous I/O operation to a disk failed
ORA-15080: synchronous I/O operation to a disk failed
ORA-15080: synchronous I/O operation to a disk failed

SQL> create diskgroup TESTDG1 EXTERNAL REDUNDANCY DISK '/dev/asmc-disk11' ;
create diskgroup TESTDG1 EXTERNAL REDUNDANCY DISK '/dev/asmc-disk11'
*
ERROR at line 1:
ORA-15018: diskgroup cannot be created
ORA-15080: synchronous I/O operation to a disk failed
ORA-15080: synchronous I/O operation to a disk failed
ORA-15080: synchronous I/O operation to a disk failed
ORA-15080: synchronous I/O operation to a disk failed

 

 

新增的Disk , sector size 是4096

 

[root@mlab2 ~]# fdisk -l /dev/sdb
Note: sector size is 4096 (not 512)

WARNING: The size of this disk is 4.0 TB (4000787025920 bytes).
DOS partition table format can not be used on drives for volumes
larger than 2.2 TB (2199023255040 bytes). Use parted(1) and GUID 
partition table format (GPT).

Disk /dev/sdb: 4000.7 GB, 4000787025920 bytes
255 heads, 63 sectors/track, 60800 cylinders
Units = cylinders of 16065 * 4096 = 65802240 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1       60800  3907007748    5  Extended
/dev/sdb5               1        7600   488375496   83  Linux
/dev/sdb6            7601       15200   488375748   83  Linux
/dev/sdb7           15201       21280   390700548   83  Linux
/dev/sdb8           21281       27360   390700548   83  Linux
/dev/sdb9           27361       33440   390700548   83  Linux
/dev/sdb10          33441       39520   390700548   83  Linux
/dev/sdb11          39521       45600   390700548   83  Linux
/dev/sdb12          45601       51680   390700548   83  Linux
/dev/sdb13          51681       57760   390700548   83  Linux
/dev/sdb14          57761       60800   195350148   83  Linux

 

 

相关bug Bug 13941884 : ORA-15080: UNABLE TO CREATE 4K SECTOR SIZE ASM DISK

 

 

Hdr: 13941884 11.2.0.3 RDBMS 11.2.0.3 ASM PRODID-5 PORTID-226 ORA-15080
Abstract: ORA-15080: UNABLE TO CREATE 4K SECTOR SIZE ASM DISK

*** 04/09/12 12:22 pm ***

PROBLEM:
--------

1) Customer configured a 4KB sector size disk:
======================================================
select path, sector_size, HEADER_STATUS from v$asm_disk;

PATH                                               SECTOR_SIZE HEADER_STATU
-------------------------------------------------- ----------- ------------
/dev/fiob                                                 4096 UNKNOWN
======================================================
[root@oracle5 dev]# fdisk -l /dev/fiob
Note: sector size is 4096 (not 512)

Disk /dev/fiob: 160.9 GB, 160937803776 bytes
255 heads, 63 sectors/track, 2445 cylinders
Units = cylinders of 16065 * 4096 = 65802240 bytes

Disk /dev/fiob doesn't contain a valid partition table
======================================================
[oracle@oracle5 dev]$ dd if=/dev/zero of=/dev/fiob bs=8192 count=12800
12800+0 records in
12800+0 records out
104857600 bytes (105 MB) copied, 0.18181 seconds, 577 MB/s
======================================================

2) But the diskgroup cannot be created on ASM release 11.2.0.3 or 11.2.0.2, 
if the try to create the diskgroup using <<'sector_size'='4096'>> and disks 
are “formatted with 4k”, then it fails with the ORA-15080 error:
======================================================
SQL> create diskgroup TESTDG1 EXTERNAL REDUNDANCY DISK '/dev/fiob' ATTRIBUTE 
'compatible.asm' = '11.2', 'compatible.rdbms' = '11.2', 'sector_size'='4096';
create diskgroup TESTDG1 EXTERNAL REDUNDANCY DISK '/dev/fiob' ATTRIBUTE 
'compatible.asm' = '11.2', 'compatible.rdbms' = '11.2', 'sector_size'='4096'
*
ERROR at line 1:
ORA-15018: diskgroup cannot be created
ORA-15080: synchronous I/O operation to a disk failed
======================================================

3) But if they do not specify <<'sector_size'='4096'>>, then the diskgroup 
creation fails with the same error “ORA-15080” as well:
======================================================
SQL> create diskgroup TESTDG1 EXTERNAL REDUNDANCY DISK '/dev/fiob';
create diskgroup TESTDG1 EXTERNAL REDUNDANCY DISK '/dev/fiob'
*
ERROR at line 1:
ORA-15018: diskgroup cannot be created
ORA-15080: synchronous I/O operation to a disk failed
======================================================

4) So the diskgroup creation fails on both cases (using a 4KB sector size 
disk) regardless <<'sector_size'='4096'>> is set or not.

5) But if the disks is formatted with 512 bytes, then the diskgroup can be 
created (of course without the <<'sector_size'='4096'>> attribute:
======================================================

1. Disk formatted with 512b
2. Permissions Set
3. 512b sector size diskgroup - Success
4. 4k sector size diskgroup - Failed with 'Mismatch' error

[root@oracle5 dev]# fdisk -l /dev/fiob

Disk /dev/fiob: 160.9 GB, 160938000384 bytes
255 heads, 63 sectors/track, 19566 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/fiob doesn't contain a valid partition table

[root@oracle5 dev]# ls -l fio*
brw-r----- 1 root   disk 252,  0 Mar 12 14:20 fioa
brwx------ 1 oracle dba-252, 16 Apr  6 16:10 fiob
brwx------ 1 root   disk 252, 32 Apr  5 16:08 fioc
brw-r----- 1 root   disk 252, 48 Apr  4 09:30 fiod
brw-r----- 1 root   disk 252, 64 Mar 12 14:21 fioe

SQL> select path from v$asm_disk;

PATH
------------------------------------------------------------------------------
--
/dev/fiob
ORCL:ASMDISKFIOA
ORCL:ASMDISKFIOB
ORCL:ASMDISKFIOC
ORCL:ASMDISKFIOX

SQL>  create diskgroup TESTDG1 EXTERNAL REDUNDANCY DISK '/dev/fiob';

Diskgroup created.

SQL> drop diskgroup TESTDG1;

Diskgroup dropped.
======================================================

6) This problems looks very similar to the next bug:
======================================================
Bug 9858860 - CREATE ASM DISKGROUP FAILS IF DISK SECTOR SIZE IS 4KB
======================================================

DIAGNOSTIC ANALYSIS:
--------------------
TESTS after Grid Infrastructure upgraded to 11.2.0.3 (11.2.0.2 the error is 
present too)

Test 1:

1. Disk formatted with 4k
2. Permissions Set
3. 4k sector size diskgroup - FAILED

[root@oracle5 dev]# fdisk -l /dev/fiob
Note: sector size is 4096 (not 512)

Disk /dev/fiob: 160.9 GB, 160937803776 bytes
255 heads, 63 sectors/track, 2445 cylinders
Units = cylinders of 16065 * 4096 = 65802240 bytes

Disk /dev/fiob doesn't contain a valid partition table

[root@oracle5 dev]# ls -l fio*
brw-r----- 1 root   disk 252,  0 Mar 12 14:20 fioa
brwx------ 1 oracle dba-252, 16 Apr  6 16:05 fiob
brwx------ 1 root   disk 252, 32 Apr  5 16:08 fioc
brw-r----- 1 root   disk 252, 48 Apr  4 09:30 fiod
brw-r----- 1 root   disk 252, 64 Mar 12 14:21 fioe

Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Automatic Storage Management option

SQL>  alter system set asm_diskstring = 'ORCL:*','/dev/fiob';

System altered.

SQL> select path from v$asm_disk;

PATH
------------------------------------------------------------------------------
--
/dev/fiob
ORCL:ASMDISKFIOA
ORCL:ASMDISKFIOB
ORCL:ASMDISKFIOC
ORCL:ASMDISKFIOX

SQL> create diskgroup TESTDG1 EXTERNAL REDUNDANCY DISK '/dev/fiob' ATTRIBUTE 
'compatible.asm' = '11.2', 'compatible.rdbms' = '11.2', 'sector_size'='4096';
create diskgroup TESTDG1 EXTERNAL REDUNDANCY DISK '/dev/fiob' ATTRIBUTE 
'compatible.asm' = '11.2', 'compatible.rdbms' = '11.2', 'sector_size'='4096'
*
ERROR at line 1:
ORA-15018: diskgroup cannot be created
ORA-15080: synchronous I/O operation to a disk failed

SQL> create diskgroup TESTDG1 EXTERNAL REDUNDANCY DISK '/dev/fiob';
create diskgroup TESTDG1 EXTERNAL REDUNDANCY DISK '/dev/fiob'
*
ERROR at line 1:
ORA-15018: diskgroup cannot be created
ORA-15080: synchronous I/O operation to a disk failed

=======================================================================

Test 2:

1. Disk formatted with 512b
2. Permissions Set
3. 512b sector size diskgroup - Success
4. 4k sector size diskgroup - Failed with 'Mismatch' error

[root@oracle5 dev]# fdisk -l /dev/fiob

Disk /dev/fiob: 160.9 GB, 160938000384 bytes
255 heads, 63 sectors/track, 19566 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/fiob doesn't contain a valid partition table

[root@oracle5 dev]# ls -l fio*
brw-r----- 1 root   disk 252,  0 Mar 12 14:20 fioa
brwx------ 1 oracle dba-252, 16 Apr  6 16:10 fiob
brwx------ 1 root   disk 252, 32 Apr  5 16:08 fioc
brw-r----- 1 root   disk 252, 48 Apr  4 09:30 fiod
brw-r----- 1 root   disk 252, 64 Mar 12 14:21 fioe

SQL> select path from v$asm_disk;

PATH
------------------------------------------------------------------------------
--
/dev/fiob
ORCL:ASMDISKFIOA
ORCL:ASMDISKFIOB
ORCL:ASMDISKFIOC
ORCL:ASMDISKFIOX

SQL>  create diskgroup TESTDG1 EXTERNAL REDUNDANCY DISK '/dev/fiob';

Diskgroup created.

SQL> drop diskgroup TESTDG1;

Diskgroup dropped.

SQL> create diskgroup TESTDG1 EXTERNAL REDUNDANCY DISK '/dev/fiob' ATTRIBUTE 
'compatible.asm' = '11.2', 'compatible.rdbms' = '11.2', 'sector_size'='4096';
create diskgroup TESTDG1 EXTERNAL REDUNDANCY DISK '/dev/fiob' ATTRIBUTE 
'compatible.asm' = '11.2', 'compatible.rdbms' = '11.2', 'sector_size'='4096'
*
ERROR at line 1:
ORA-15018: diskgroup cannot be created
ORA-15038: disk '' mismatch on 'Sector Size' with target disk group [512]
[4096]

该问题有相关的补丁Patch 13941884: UNABLE TO CREATE 4K SECTOR SIZE ASM DISK (ORA-15080),但是并不开发下载,需要密码下载:

asm patch1

即便尝试设置了_disk_sector_size_override参数,仍无法避免ORA-15080

 

alter system set "_disk_sector_size_override" = TRUE scope = both;

SQL> show parameter over

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
_disk_sector_size_override           boolean     TRUE

SQL> create diskgroup TESTDG1 EXTERNAL REDUNDANCY DISK '/dev/asmc-disk11' ATTRIBUTE  'compatible.asm' = '11.2', 'compatible.rdbms' = '11.2', 'sector_size'='4096';
create diskgroup TESTDG1 EXTERNAL REDUNDANCY DISK '/dev/asmc-disk11' ATTRIBUTE  'compatible.asm' = '11.2', 'compatible.rdbms' = '11.2', 'sector_size'='4096'
*
ERROR at line 1:
ORA-15018: diskgroup cannot be created
ORA-15080: synchronous I/O operation to a disk failed
ORA-15080: synchronous I/O operation to a disk failed
ORA-15080: synchronous I/O operation to a disk failed
ORA-15080: synchronous I/O operation to a disk failed

 

 

 

对此的解释包括:

 

 "This explains why the disk group cannot be created since the ASM discovery 
 expects all the disks in the disk group to have the same sector size"
 .
 Point : As you see in the diskgroup creation syntax I use only one disk 
 (which is having 4k sector size). Yes I do have various sector size disks 
 attached to the server.  Again, in this test I used only one disk for a disk 
 group.
 .
 "Not sure why only file /dev/fioe1 is having 4096 sector size and if this is 
 the real cause of this error. Need to review."

 

 

实际观察该问题 可能与OS I/O错误有关,不单纯是ASM的问题:

 

 

ORA-27061: waiting for async I/Os failed
Linux-x86_64 Error: 22: Invalid argument
Additional information: -1
Additional information: 65536
WARNING: Write Failed. group:1 disk:0 AU:0 offset:524288 size:65536
path:/dev/asmc-disk5
         incarnation:0xe968a798 asynchronous result:'I/O error'
         subsys:System iop:0x7f485a145500 bufp:0x7f485a0a4e00 osderr:0x0 osderr1:0x0
ORA-27061: waiting for async I/Os failed
Linux-x86_64 Error: 22: Invalid argument
Additional information: -1
Additional information: 65536
WARNING: Write Failed. group:1 disk:0 AU:0 offset:458752 size:65536
path:/dev/asmc-disk5
         incarnation:0xe968a798 asynchronous result:'I/O error'
         subsys:System iop:0x7f485a1453c0 bufp:0x7f485a084e00 osderr:0x0 osderr1:0x0
ORA-27061: waiting for async I/Os failed
Linux-x86_64 Error: 22: Invalid argument
Additional information: -1
Additional information: 65536
WARNING: Write Failed. group:1 disk:0 AU:0 offset:393216 size:65536
path:/dev/asmc-disk5
         incarnation:0xe968a798 asynchronous result:'I/O error'
         subsys:System iop:0x7f485a145280 bufp:0x7f485a064e00 osderr:0x0 osderr1:0x0
ORA-27061: waiting for async I/Os failed
Linux-x86_64 Error: 22: Invalid argument
Additional information: -1
Additional information: 65536
NOTE: Suppress further IO Write errors on group:1 disk:0
ORA-27061: waiting for async I/Os failed
Linux-x86_64 Error: 22: Invalid argument
Additional information: -1
Additional information: 65536
ORA-27061: waiting for async I/Os failed
Linux-x86_64 Error: 22: Invalid argument
Additional information: -1
Additional information: 65536
ORA-27061: waiting for async I/Os failed
Linux-x86_64 Error: 22: Invalid argument
Additional information: -1
Additional information: 65536
ORA-27061: waiting for async I/Os failed
Linux-x86_64 Error: 22: Invalid argument
Additional information: -1
Additional information: 65536
ORA-27061: waiting for async I/Os failed
Linux-x86_64 Error: 22: Invalid argument
Additional information: -1
Additional information: 65536
ORA-27061: waiting for async I/Os failed
Linux-x86_64 Error: 22: Invalid argument
Additional information: -1
Additional information: 65536
ORA-27061: waiting for async I/Os failed

 

 

用户可以尝试使用ASMLIB来绕过该问题,此处不再鏖述

Comment

*

沪ICP备14014813号-2

沪公网安备 31010802001379号