* Workload Request Size issue
@ 2013-02-16 16:19 Alireza Haghdoost
2013-02-18 8:04 ` Jens Axboe
0 siblings, 1 reply; 4+ messages in thread
From: Alireza Haghdoost @ 2013-02-16 16:19 UTC (permalink / raw)
To: fio
Hello,
I am using fio to generate a simple sequential workload with request
size of 128K with following job file but I can observe the generated
IO request size is 4K, I was wondering who is responsible in block
layer to shrink the fio requests granularity from 128K to 4K ? :
1 ; -- start job file --
2 [global]
3 rw=write
4 size=128M
5
6 [job1]
7 ioengine=libaio
8 iodepth=512
9 bs=128k
10 direct=1
11 numjobs=1
12 filename=/dev/sde
13 ; -- end job file --
The blktrace shows that the request sizes are all 4KB (8 sector), here
is a peace of output trace that collected while fio is running.
21 8,64 5 21 0.000031322 29848 I W 32 + 8 [fio]
22 8,64 5 22 0.000031868 29848 Q WS 40 + 8 [fio]
23 8,64 5 23 0.000032181 29848 G WS 40 + 8 [fio]
24 8,64 5 24 0.000032352 29848 I W 40 + 8 [fio]
25 8,64 5 25 0.000032826 29848 Q WS 48 + 8 [fio]
26 8,64 5 26 0.000033145 29848 G WS 48 + 8 [fio]
27 8,64 5 27 0.000033310 29848 I W 48 + 8 [fio]
28 8,64 5 28 0.000033781 29848 Q WS 56 + 8 [fio]
29 8,64 5 29 0.000034045 29848 G WS 56 + 8 [fio]
30 8,64 5 30 0.000034227 29848 I W 56 + 8 [fio]
31 8,64 5 31 0.000034498 29848 D W 32 + 8 [fio]
32 8,64 5 32 0.000036314 29848 D W 40 + 8 [fio]
33 8,64 5 33 0.000038032 29848 D W 48 + 8 [fio]
^ permalink raw reply [flat|nested] 4+ messages in thread* Re: Workload Request Size issue 2013-02-16 16:19 Workload Request Size issue Alireza Haghdoost @ 2013-02-18 8:04 ` Jens Axboe 2013-02-19 2:49 ` Alireza Haghdoost 0 siblings, 1 reply; 4+ messages in thread From: Jens Axboe @ 2013-02-18 8:04 UTC (permalink / raw) To: Alireza Haghdoost; +Cc: fio On Sat, Feb 16 2013, Alireza Haghdoost wrote: > Hello, > > I am using fio to generate a simple sequential workload with request > size of 128K with following job file but I can observe the generated > IO request size is 4K, I was wondering who is responsible in block > layer to shrink the fio requests granularity from 128K to 4K ? : > > > 1 ; -- start job file -- > 2 [global] > 3 rw=write > 4 size=128M > 5 > 6 [job1] > 7 ioengine=libaio > 8 iodepth=512 > 9 bs=128k > 10 direct=1 > 11 numjobs=1 > 12 filename=/dev/sde > 13 ; -- end job file -- > > > The blktrace shows that the request sizes are all 4KB (8 sector), here > is a peace of output trace that collected while fio is running. > > 21 8,64 5 21 0.000031322 29848 I W 32 + 8 [fio] > 22 8,64 5 22 0.000031868 29848 Q WS 40 + 8 [fio] > 23 8,64 5 23 0.000032181 29848 G WS 40 + 8 [fio] > 24 8,64 5 24 0.000032352 29848 I W 40 + 8 [fio] > 25 8,64 5 25 0.000032826 29848 Q WS 48 + 8 [fio] > 26 8,64 5 26 0.000033145 29848 G WS 48 + 8 [fio] > 27 8,64 5 27 0.000033310 29848 I W 48 + 8 [fio] > 28 8,64 5 28 0.000033781 29848 Q WS 56 + 8 [fio] > 29 8,64 5 29 0.000034045 29848 G WS 56 + 8 [fio] > 30 8,64 5 30 0.000034227 29848 I W 56 + 8 [fio] > 31 8,64 5 31 0.000034498 29848 D W 32 + 8 [fio] > 32 8,64 5 32 0.000036314 29848 D W 40 + 8 [fio] > 33 8,64 5 33 0.000038032 29848 D W 48 + 8 [fio] What kernel are you using? For a proper setup, the above fio job should go down to the hardware as 128KB sized requests. Depending on the kernel and driver, there might be timing issues that cause it to not get properly coalesced in the scheduler, however. -- Jens Axboe ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Workload Request Size issue 2013-02-18 8:04 ` Jens Axboe @ 2013-02-19 2:49 ` Alireza Haghdoost 2013-02-19 12:54 ` Jens Axboe 0 siblings, 1 reply; 4+ messages in thread From: Alireza Haghdoost @ 2013-02-19 2:49 UTC (permalink / raw) To: Jens Axboe; +Cc: fio On Mon, Feb 18, 2013 at 2:04 AM, Jens Axboe <axboe@kernel.dk> wrote: > > On Sat, Feb 16 2013, Alireza Haghdoost wrote: > > Hello, > > > > I am using fio to generate a simple sequential workload with request > > size of 128K with following job file but I can observe the generated > > IO request size is 4K, I was wondering who is responsible in block > > layer to shrink the fio requests granularity from 128K to 4K ? : > > > > > > 1 ; -- start job file -- > > 2 [global] > > 3 rw=write > > 4 size=128M > > 5 > > 6 [job1] > > 7 ioengine=libaio > > 8 iodepth=512 > > 9 bs=128k > > 10 direct=1 > > 11 numjobs=1 > > 12 filename=/dev/sde > > 13 ; -- end job file -- > > > > > > The blktrace shows that the request sizes are all 4KB (8 sector), here > > is a peace of output trace that collected while fio is running. > > > > 21 8,64 5 21 0.000031322 29848 I W 32 + 8 [fio] > > 22 8,64 5 22 0.000031868 29848 Q WS 40 + 8 [fio] > > 23 8,64 5 23 0.000032181 29848 G WS 40 + 8 [fio] > > 24 8,64 5 24 0.000032352 29848 I W 40 + 8 [fio] > > 25 8,64 5 25 0.000032826 29848 Q WS 48 + 8 [fio] > > 26 8,64 5 26 0.000033145 29848 G WS 48 + 8 [fio] > > 27 8,64 5 27 0.000033310 29848 I W 48 + 8 [fio] > > 28 8,64 5 28 0.000033781 29848 Q WS 56 + 8 [fio] > > 29 8,64 5 29 0.000034045 29848 G WS 56 + 8 [fio] > > 30 8,64 5 30 0.000034227 29848 I W 56 + 8 [fio] > > 31 8,64 5 31 0.000034498 29848 D W 32 + 8 [fio] > > 32 8,64 5 32 0.000036314 29848 D W 40 + 8 [fio] > > 33 8,64 5 33 0.000038032 29848 D W 48 + 8 [fio] > > What kernel are you using? For a proper setup, the above fio job should > go down to the hardware as 128KB sized requests. Depending on the kernel > and driver, there might be timing issues that cause it to not get > properly coalesced in the scheduler, however. > > -- > Jens Axboe > Jens, Thanks for your replay. I am using Ubuntu server 10.10 kernel version 2.6.35-22-server. /dev/sde is a NAS drive. NIC card adapter is QLogic Fiber Channel and we are using qla2xxx driver. I believe somewhere in block layer interface 128KB request converts to 8KB requests. I am using noop IO scheduler with queue size (nr_request) of 2048 and nomerge value 2. I really appreciate your further input. Thanks Alireza ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Workload Request Size issue 2013-02-19 2:49 ` Alireza Haghdoost @ 2013-02-19 12:54 ` Jens Axboe 0 siblings, 0 replies; 4+ messages in thread From: Jens Axboe @ 2013-02-19 12:54 UTC (permalink / raw) To: Alireza Haghdoost; +Cc: fio On Mon, Feb 18 2013, Alireza Haghdoost wrote: > On Mon, Feb 18, 2013 at 2:04 AM, Jens Axboe <axboe@kernel.dk> wrote: > > > > On Sat, Feb 16 2013, Alireza Haghdoost wrote: > > > Hello, > > > > > > I am using fio to generate a simple sequential workload with request > > > size of 128K with following job file but I can observe the generated > > > IO request size is 4K, I was wondering who is responsible in block > > > layer to shrink the fio requests granularity from 128K to 4K ? : > > > > > > > > > 1 ; -- start job file -- > > > 2 [global] > > > 3 rw=write > > > 4 size=128M > > > 5 > > > 6 [job1] > > > 7 ioengine=libaio > > > 8 iodepth=512 > > > 9 bs=128k > > > 10 direct=1 > > > 11 numjobs=1 > > > 12 filename=/dev/sde > > > 13 ; -- end job file -- > > > > > > > > > The blktrace shows that the request sizes are all 4KB (8 sector), here > > > is a peace of output trace that collected while fio is running. > > > > > > 21 8,64 5 21 0.000031322 29848 I W 32 + 8 [fio] > > > 22 8,64 5 22 0.000031868 29848 Q WS 40 + 8 [fio] > > > 23 8,64 5 23 0.000032181 29848 G WS 40 + 8 [fio] > > > 24 8,64 5 24 0.000032352 29848 I W 40 + 8 [fio] > > > 25 8,64 5 25 0.000032826 29848 Q WS 48 + 8 [fio] > > > 26 8,64 5 26 0.000033145 29848 G WS 48 + 8 [fio] > > > 27 8,64 5 27 0.000033310 29848 I W 48 + 8 [fio] > > > 28 8,64 5 28 0.000033781 29848 Q WS 56 + 8 [fio] > > > 29 8,64 5 29 0.000034045 29848 G WS 56 + 8 [fio] > > > 30 8,64 5 30 0.000034227 29848 I W 56 + 8 [fio] > > > 31 8,64 5 31 0.000034498 29848 D W 32 + 8 [fio] > > > 32 8,64 5 32 0.000036314 29848 D W 40 + 8 [fio] > > > 33 8,64 5 33 0.000038032 29848 D W 48 + 8 [fio] > > > > What kernel are you using? For a proper setup, the above fio job should > > go down to the hardware as 128KB sized requests. Depending on the kernel > > and driver, there might be timing issues that cause it to not get > > properly coalesced in the scheduler, however. > > > > -- > > Jens Axboe > > > > Jens, > > Thanks for your replay. I am using Ubuntu server 10.10 kernel version > 2.6.35-22-server. /dev/sde is a NAS drive. NIC card adapter is QLogic > Fiber Channel and we are using qla2xxx driver. I believe somewhere in > block layer interface 128KB request converts to 8KB requests. I am > using noop IO scheduler with queue size (nr_request) of 2048 and > nomerge value 2. It's actually the other way around, 4kb pages are submitted and expected to be coalesced in the lower (block) layer. But since you have nomerges set to 2, that will never happen. Unless you have a strictly purely random workload, then leaving nomerges at 0 is the recommended setting. -- Jens Axboe ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2013-02-19 12:56 UTC | newest] Thread overview: 4+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2013-02-16 16:19 Workload Request Size issue Alireza Haghdoost 2013-02-18 8:04 ` Jens Axboe 2013-02-19 2:49 ` Alireza Haghdoost 2013-02-19 12:54 ` Jens Axboe
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox