ASM and FILESYSTEMIO_OPTIONS + DISK_ASYNCH_IO

ASM Inherently Performs Asynchronous I/O Regardless of filesystemio_options Parameter [ID 751463.1]

Purpose

The purpose of this document is to explain the behaviour of I/O calls depending database parameters FILESYSTEMIO_OPTIONS and DISK_ASYNCH_IO when the Oracle RDBMS is located on ASM diskgroups.

Scope and Application

FILESYSTEMIO_OPTIONS controls the behaviour of I/O options given by operating system but it has no affect when ASM is used. FILESYSTEMIO_OPTIONS enables asynchronous I/O (AIO) when the value is set to SETALL/ASYNC. However, ASM bypasses the filesystem layer in this case because ASM I/O is entirely controlled by DISK_ASYNCH_IO parameter. AIO needs to be enabled/disabled by setting disk_asynch_io to parameter values TRUE/FALSE.

In order to test this behaviour database writer process (DBW0) is traced with strace during a tablespace creation. An example strace of dbw0 process when AIO is used shows io_submit/io_getevents calls.

gettimeofday({1227275539, 15789}, NULL) = 0
times(NULL) = 532384532
io_submit(13086720, 2, {...}) = 2
gettimeofday({1227275539, 16181}, NULL) = 0
io_getevents(13086720, 1, 1024, {...}{600, 0}) = 1
times(NULL) = 532384532
io_getevents(13086720, 1, 1024, {...}{600, 0}) = 1

and the  strace of dbw0 process with AIO disabled shows pwrite64:

gettimeofday({1227274991, 462322}, NULL) = 0
times(NULL) = 532329776
pwrite64(22, "\36\242\0\0\3\0@\2\302\33\v\0\0\0\1\4VB\0\0\t\0\0\0\t\0\0\0\0\0\0\0"..., 8192, 6153068544) = 8192
times(NULL) = 532329776
times(NULL) = 532329776
pwrite64(24, "\36\242\0\0\3\0@\2\302\33\v\0\0\0\1\4VB\0\0\t\0\0\0\t\0\0\0\0\0\0\0"..., 8192, 6199205888) = 8192
times(NULL) = 532329776

ASM Inherently Performs Asynchronous I/O Regardless of filesystemio_options Parameter

Test and Results:

100M tablespace is created and oracle dbw0 process of database instance is traced with strace tool.

[root@trlnx4 tmp]# ps -ef|grep dbw
oracle   10131     1  0 15:53 ?        00:00:00 ora_dbw0_DB2

...

strace -af -p 10131 -o dbw.txt &

In a new sqlplus session (database instance), tablespace is created.

sqlplus / as sysdba

sql> create tablespace test1 datafile '+DG1' size 100M;


Same test is repeated for each possible value to enable and disable asynchronous I/O with the parameters and the following results are found:

FILESYSTEMIO_OPTIONS=SETALL
DISK_ASYNCH_IO=TRUE

times(NULL) = 532400079
times(NULL) = 532400079
io_submit(5623808, 2, {...}) = 2
gettimeofday({1227275694, 483102}, NULL) = 0
io_getevents(5623808, 1, 1024, {...}{600, 0}) = 1
times(NULL) = 532400079
io_getevents(5623808, 1, 1024, {...}{600, 0}) = 1
times(NULL) = 532400079

When both parameters are activated to use asynchronous I/O, dbw0 process performs the I/O asynchronously. The result is expected.


FILESYSTEMIO_OPTIONS=NONE
DISK_ASYNCH_IO=TRUE

gettimeofday({1227275539, 15789}, NULL) = 0
times(NULL) = 532384532
io_submit(13086720, 2, {...}) = 2
gettimeofday({1227275539, 16181}, NULL) = 0
io_getevents(13086720, 1, 1024, {...}{600, 0}) = 1
times(NULL) = 532384532
io_getevents(13086720, 1, 1024, {...}{600, 0}) = 1

FILESYSTEMIO_OPTIONS is set to NONE. This should normally disable the DISK_ASYNCH_IO when datafiles are located on a regular filesystem (non-ASM). However, since datafiles are located on ASM, FILESYSTEMIO_OPTIONS has no affect in this case. I/O is asynchronous.


FILESYSTEMIO_OPTIONS=NONE
DISK_ASYNCH_IO=FALSE

gettimeofday({1227274991, 462322}, NULL) = 0
times(NULL) = 532329776
pwrite64(22, "\36\242\0\0\3\0@\2\302\33\v\0\0\0\1\4VB\0\0\t\0\0\0\t\0\0\0\0\0\0\0"..., 8192, 6153068544) = 8192
times(NULL) = 532329776
times(NULL) = 532329776
pwrite64(24, "\36\242\0\0\3\0@\2\302\33\v\0\0\0\1\4VB\0\0\t\0\0\0\t\0\0\0\0\0\0\0"..., 8192, 6199205888) = 8192
times(NULL) = 532329776

FILESYSTEMIO_OPTIONS=SETALL
DISK_ASYNCH_IO=FALSE

gettimeofday({1227275223, 67935}, NULL) = 0
times(NULL) = 532352937
pwrite64(21, "\36\242\0\0\3\0@\1\25k\v\0\0\0\2\4ZA\0\0\5\0\0\0\t\0\0\0\0\0\0\0"..., 8192, 5628780544) = 8192
times(NULL) = 532352937
times(NULL) = 532352937
pwrite64(22, "\36\242\0\0\3\0@\1\25k\v\0\0\0\2\4ZA\0\0\5\0\0\0\t\0\0\0\0\0\0\0"..., 8192, 5565865984) = 8192
times(NULL) = 532352937

In the last two cases, whatever the filesystemio_options is set, if DISK_ASYNCH_IO is false, IO is done synchronously.

ASMLIB behaves similarly but with an exception in strace output. When ASMLIB is used, ASMLIB performs AIO depending on how the I/O is passed to it. It makes no decisions at all. This is entirely up to ASM layer. As the above tests also implies and ASM is entirely controlled by the disk_asynch_io parameter. So, you can also control the ASMLIB by changing the value for disk_asynch_io parameter.

When using ASMlib, the result show read calls that look like this in the strace output:

read(16, "MSA\0\2\0\7\0P\0\0\0\0\0\0\0\30\245\215\16\0\0\0\0\0\360\347\267\0\0\0\0"..., 80) = 80