* Re: output mutating on me
2013-09-16 21:13 ` Jens Axboe
@ 2013-09-16 21:35 ` Brian L.
2013-09-16 22:17 ` Brian L.
0 siblings, 1 reply; 6+ messages in thread
From: Brian L. @ 2013-09-16 21:35 UTC (permalink / raw)
To: Jens Axboe; +Cc: fio@vger.kernel.org
[-- Attachment #1: Type: text/plain, Size: 2080 bytes --]
On Mon, Sep 16, 2013 at 2:13 PM, Jens Axboe <axboe@kernel.dk> wrote:
> On 09/16/2013 02:19 PM, Brian L. wrote:
>> it's running normal directly on the machine.
>>
>> i thought maybe clat and lat might be the same as slat and that is why there is only slat being output but i not sure.
>
> (please don't top post)
>
> I need the full invocations of what you are running. If you are not
> running async IO, then you only get lat. If you are running async IO,
> you get slat and clat as well. I can't completely tell, since you didn't
> not push what each of them ran exactly and the full output.
>
> --
> Jens Axboe
>
Sorry about the top post - this is all new to me.
I will see if I can attach the output log to this email and hopefully
it will go thru and not get rejected.
If you look at the output, you will see that the following lines have
missing 'clat' and 'lat':
(you can grep for the min and max and you should get the line.)
slat (usec): min=78, max=165, avg=119.36, stdev= 8.30
clat percentiles (usec):
write: io=2520.0MB, bw=258928KB/s, iops=252, runt= 9966msec
slat (usec): min=186, max=478, avg=266.46, stdev=17.74
clat percentiles (usec):
For this particular run, it looks like there is only 1 run w/ the
different output so I can just re-run that particular the test to see
if the result comes back correctly.
This is my template file to generate the runs:
(Hopefully, b/w this and the output, it has all of the needed info.)
foundation-air:disk_benchmark blam$ cat template_general.fio
[mytest]
rw=${myrw}
filename=${mydisk}
size=5G
iodepth=${myiodepth}
bs=${myblocksize}
ioengine=libaio
ramp_time=10s
# If value is true, use non-buffered io. This is usually
# O_DIRECT. Note that ZFS on Solaris doesn't support direct io.
# On Windows the synchronous ioengines don't support direct io.
# (1 for non-buffered io)
direct=1
# If true, fsync file contents when a write stage has completed.
end_fsync=0
# Invalidate the buffer/page cache parts for this file prior to
starting io. Defaults to true.
invalidate=1
[-- Attachment #2: latest.log.libaio.5g.securewiped.sep15 --]
[-- Type: application/octet-stream, Size: 327700 bytes --]
start time:
Sun Sep 15 06:58:02 PDT 2013
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdb-randrw-32-512B
===
mytest: (g=0): rw=randrw, bs=512-512/512-512/512-512, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=21132: Sun Sep 15 07:55:01 2013
read : io=5101.2MB, bw=1559.7KB/s, iops=3119, runt=3349128msec
slat (usec): min=3, max=559, avg=10.05, stdev= 3.95
clat (usec): min=0, max=56683, avg=9346.12, stdev=4763.90
lat (usec): min=35, max=56694, avg=9356.44, stdev=4764.09
clat percentiles (usec):
| 1.00th=[ 51], 5.00th=[ 1432], 10.00th=[ 3248], 20.00th=[ 5728],
| 30.00th=[ 6688], 40.00th=[ 7456], 50.00th=[ 8512], 60.00th=[10176],
| 70.00th=[11968], 80.00th=[13760], 90.00th=[15808], 95.00th=[17536],
| 99.00th=[20608], 99.50th=[21632], 99.90th=[24704], 99.95th=[26496],
| 99.99th=[35584]
bw (KB /s): min= 0, max= 2114, per=100.00%, avg=1560.25, stdev=109.53
write: io=5099.1MB, bw=1559.4KB/s, iops=3118, runt=3349128msec
slat (usec): min=3, max=787, avg=12.05, stdev= 4.61
clat (usec): min=0, max=24251, avg=885.41, stdev=1184.80
lat (usec): min=33, max=24258, avg=897.76, stdev=1184.17
clat percentiles (usec):
| 1.00th=[ 31], 5.00th=[ 32], 10.00th=[ 33], 20.00th=[ 37],
| 30.00th=[ 52], 40.00th=[ 137], 50.00th=[ 390], 60.00th=[ 684],
| 70.00th=[ 1112], 80.00th=[ 1640], 90.00th=[ 2352], 95.00th=[ 3312],
| 99.00th=[ 5280], 99.50th=[ 6240], 99.90th=[ 7776], 99.95th=[ 8640],
| 99.99th=[ 9920]
bw (KB /s): min= 595, max= 2008, per=100.00%, avg=1560.15, stdev=97.71
lat (usec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=14.90%
lat (usec) : 100=4.85%, 250=3.37%, 500=4.96%, 750=4.33%, 1000=3.33%
lat (msec) : 2=10.59%, 4=7.84%, 10=25.33%, 20=19.81%, 50=0.70%
lat (msec) : 100=0.01%
cpu : usr=3.30%, sys=8.85%, ctx=13737911, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.4%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=10446856/w=10444627/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=5101.2MB, aggrb=1559KB/s, minb=1559KB/s, maxb=1559KB/s, mint=3349128msec, maxt=3349128msec
WRITE: io=5099.1MB, aggrb=1559KB/s, minb=1559KB/s, maxb=1559KB/s, mint=3349128msec, maxt=3349128msec
Disk stats (read/write):
sdb: ios=10486921/10484656, merge=0/0, ticks=97689998/9353504, in_queue=107032870, util=100.00%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdb-randrw-32-1K
===
mytest: (g=0): rw=randrw, bs=1K-1K/1K-1K/1K-1K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=25263: Sun Sep 15 08:25:22 2013
read : io=5090.1MB, bw=2976.2KB/s, iops=2976, runt=1751657msec
slat (usec): min=4, max=418, avg= 9.80, stdev= 2.91
clat (usec): min=35, max=48586, avg=10161.28, stdev=4177.65
lat (usec): min=45, max=48602, avg=10171.36, stdev=4177.73
clat percentiles (usec):
| 1.00th=[ 3888], 5.00th=[ 4896], 10.00th=[ 5536], 20.00th=[ 6432],
| 30.00th=[ 7136], 40.00th=[ 7968], 50.00th=[ 9152], 60.00th=[10816],
| 70.00th=[12352], 80.00th=[14016], 90.00th=[16064], 95.00th=[17792],
| 99.00th=[21120], 99.50th=[22400], 99.90th=[25216], 99.95th=[26240],
| 99.99th=[28800]
bw (KB /s): min= 0, max= 3524, per=100.00%, avg=2977.25, stdev=180.21
write: io=5089.4MB, bw=2975.2KB/s, iops=2975, runt=1751657msec
slat (usec): min=5, max=1897, avg=11.74, stdev= 3.75
clat (usec): min=0, max=21717, avg=564.63, stdev=777.64
lat (usec): min=40, max=21724, avg=576.65, stdev=777.54
clat percentiles (usec):
| 1.00th=[ 33], 5.00th=[ 36], 10.00th=[ 46], 20.00th=[ 70],
| 30.00th=[ 103], 40.00th=[ 149], 50.00th=[ 215], 60.00th=[ 334],
| 70.00th=[ 564], 80.00th=[ 956], 90.00th=[ 1656], 95.00th=[ 2128],
| 99.00th=[ 3728], 99.50th=[ 4256], 99.90th=[ 5472], 99.95th=[ 6240],
| 99.99th=[ 7136]
bw (KB /s): min= 1868, max= 3368, per=100.00%, avg=2977.17, stdev=147.95
lat (usec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=5.70%
lat (usec) : 100=8.70%, 250=12.40%, 500=7.13%, 750=3.86%, 1000=2.59%
lat (msec) : 2=6.59%, 4=3.24%, 10=27.36%, 20=21.52%, 50=0.89%
cpu : usr=3.25%, sys=8.24%, ctx=7796576, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.6%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=5213128/w=5211497/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=5090.1MB, aggrb=2976KB/s, minb=2976KB/s, maxb=2976KB/s, mint=1751657msec, maxt=1751657msec
WRITE: io=5089.4MB, aggrb=2975KB/s, minb=2975KB/s, maxb=2975KB/s, mint=1751657msec, maxt=1751657msec
Disk stats (read/write):
sdb: ios=5243446/5241696, merge=0/0, ticks=53240338/2978837, in_queue=56216423, util=100.00%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdb-randrw-32-2K
===
mytest: (g=0): rw=randrw, bs=2K-2K/2K-2K/2K-2K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=27678: Sun Sep 15 08:40:47 2013
read : io=5073.3MB, bw=6084.4KB/s, iops=3042, runt=853821msec
slat (usec): min=4, max=234, avg= 9.67, stdev= 2.38
clat (usec): min=36, max=37677, avg=9934.11, stdev=4198.25
lat (usec): min=50, max=37685, avg=9944.05, stdev=4198.20
clat percentiles (usec):
| 1.00th=[ 3888], 5.00th=[ 4832], 10.00th=[ 5472], 20.00th=[ 6304],
| 30.00th=[ 7008], 40.00th=[ 7840], 50.00th=[ 8896], 60.00th=[10304],
| 70.00th=[11712], 80.00th=[13376], 90.00th=[15808], 95.00th=[17792],
| 99.00th=[22144], 99.50th=[23680], 99.90th=[27264], 99.95th=[28544],
| 99.99th=[31360]
bw (KB /s): min= 4268, max= 7548, per=100.00%, avg=6087.99, stdev=775.41
write: io=5070.6MB, bw=6081.3KB/s, iops=3040, runt=853821msec
slat (usec): min=5, max=1354, avg=11.53, stdev= 3.17
clat (usec): min=0, max=13052, avg=558.90, stdev=762.97
lat (usec): min=43, max=13062, avg=570.71, stdev=762.79
clat percentiles (usec):
| 1.00th=[ 37], 5.00th=[ 43], 10.00th=[ 57], 20.00th=[ 78],
| 30.00th=[ 110], 40.00th=[ 155], 50.00th=[ 223], 60.00th=[ 342],
| 70.00th=[ 556], 80.00th=[ 932], 90.00th=[ 1608], 95.00th=[ 2096],
| 99.00th=[ 3664], 99.50th=[ 4256], 99.90th=[ 5728], 99.95th=[ 6368],
| 99.99th=[ 7776]
bw (KB /s): min= 0, max= 7308, per=100.00%, avg=6081.28, stdev=770.69
lat (usec) : 2=0.01%, 10=0.01%, 20=0.01%, 50=3.66%, 100=9.62%
lat (usec) : 250=13.14%, 500=7.55%, 750=4.01%, 1000=2.66%
lat (msec) : 2=6.50%, 4=3.12%, 10=28.64%, 20=19.93%, 50=1.18%
cpu : usr=3.38%, sys=8.19%, ctx=3896962, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.9%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=2597460/w=2596130/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=5073.3MB, aggrb=6084KB/s, minb=6084KB/s, maxb=6084KB/s, mint=853821msec, maxt=853821msec
WRITE: io=5070.6MB, aggrb=6081KB/s, minb=6081KB/s, maxb=6081KB/s, mint=853821msec, maxt=853821msec
Disk stats (read/write):
sdb: ios=2622054/2620571, merge=0/0, ticks=26100044/1485077, in_queue=27584288, util=100.00%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdb-randrw-32-4K
===
mytest: (g=0): rw=randrw, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=29203: Sun Sep 15 08:45:11 2013
read : io=4869.3MB, bw=25674KB/s, iops=6418, runt=194208msec
slat (usec): min=4, max=381, avg= 9.74, stdev= 3.37
clat (usec): min=42, max=17456, avg=4333.70, stdev=2552.79
lat (usec): min=52, max=17465, avg=4343.71, stdev=2552.71
clat percentiles (usec):
| 1.00th=[ 1336], 5.00th=[ 1528], 10.00th=[ 1656], 20.00th=[ 1880],
| 30.00th=[ 2192], 40.00th=[ 2672], 50.00th=[ 3728], 60.00th=[ 4896],
| 70.00th=[ 5856], 80.00th=[ 6688], 90.00th=[ 8096], 95.00th=[ 8768],
| 99.00th=[10816], 99.50th=[11840], 99.90th=[13376], 99.95th=[14016],
| 99.99th=[15808]
bw (KB /s): min= 0, max=27568, per=99.80%, avg=25621.32, stdev=1497.67
write: io=4866.4MB, bw=25659KB/s, iops=6414, runt=194208msec
slat (usec): min=5, max=1481, avg=12.11, stdev= 4.22
clat (usec): min=0, max=14475, avg=625.40, stdev=520.55
lat (usec): min=45, max=14483, avg=637.85, stdev=520.41
clat percentiles (usec):
| 1.00th=[ 40], 5.00th=[ 58], 10.00th=[ 86], 20.00th=[ 141],
| 30.00th=[ 213], 40.00th=[ 294], 50.00th=[ 418], 60.00th=[ 628],
| 70.00th=[ 940], 80.00th=[ 1256], 90.00th=[ 1464], 95.00th=[ 1528],
| 99.00th=[ 1592], 99.50th=[ 1608], 99.90th=[ 2192], 99.95th=[ 2352],
| 99.99th=[ 2512]
bw (KB /s): min=23440, max=27712, per=100.00%, avg=25671.55, stdev=706.99
lat (usec) : 2=0.01%, 10=0.01%, 20=0.01%, 50=2.24%, 100=4.10%
lat (usec) : 250=11.17%, 500=9.77%, 750=4.74%, 1000=3.91%
lat (msec) : 2=26.16%, 4=13.98%, 10=22.72%, 20=1.21%
cpu : usr=7.10%, sys=17.94%, ctx=1653306, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=105.2%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=1246493/w=1245794/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4869.3MB, aggrb=25673KB/s, minb=25673KB/s, maxb=25673KB/s, mint=194208msec, maxt=194208msec
WRITE: io=4866.4MB, aggrb=25658KB/s, minb=25658KB/s, maxb=25658KB/s, mint=194208msec, maxt=194208msec
Disk stats (read/write):
sdb: ios=1310630/1309614, merge=0/0, ticks=5672846/823677, in_queue=6495619, util=100.00%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdb-randrw-32-8K
===
mytest: (g=0): rw=randrw, bs=8K-8K/8K-8K/8K-8K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=29448: Sun Sep 15 08:48:56 2013
read : io=4806.4MB, bw=32044KB/s, iops=4005, runt=153582msec
slat (usec): min=5, max=207, avg=10.32, stdev= 2.61
clat (usec): min=326, max=22936, avg=7315.51, stdev=3479.45
lat (usec): min=341, max=22948, avg=7326.15, stdev=3479.44
clat percentiles (usec):
| 1.00th=[ 2640], 5.00th=[ 3184], 10.00th=[ 3504], 20.00th=[ 3952],
| 30.00th=[ 4448], 40.00th=[ 5088], 50.00th=[ 6752], 60.00th=[ 8256],
| 70.00th=[ 9408], 80.00th=[10688], 90.00th=[12352], 95.00th=[13376],
| 99.00th=[15680], 99.50th=[16768], 99.90th=[18560], 99.95th=[19072],
| 99.99th=[20608]
bw (KB /s): min=29184, max=35088, per=100.00%, avg=32061.26, stdev=1011.95
write: io=4801.7MB, bw=32015KB/s, iops=4001, runt=153582msec
slat (usec): min=6, max=2502, avg=13.33, stdev= 4.97
clat (usec): min=28, max=19130, avg=645.74, stdev=620.19
lat (usec): min=56, max=19140, avg=659.45, stdev=620.03
clat percentiles (usec):
| 1.00th=[ 50], 5.00th=[ 90], 10.00th=[ 129], 20.00th=[ 195],
| 30.00th=[ 258], 40.00th=[ 334], 50.00th=[ 422], 60.00th=[ 532],
| 70.00th=[ 692], 80.00th=[ 996], 90.00th=[ 1656], 95.00th=[ 1960],
| 99.00th=[ 2384], 99.50th=[ 3088], 99.90th=[ 4016], 99.95th=[ 4128],
| 99.99th=[ 5728]
bw (KB /s): min= 0, max=35512, per=99.75%, avg=31935.21, stdev=2107.68
lat (usec) : 50=0.39%, 100=2.73%, 250=11.12%, 500=14.34%, 750=7.99%
lat (usec) : 1000=3.48%
lat (msec) : 2=7.80%, 4=12.47%, 10=27.00%, 20=12.69%, 50=0.01%
cpu : usr=4.86%, sys=11.99%, ctx=929267, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=106.6%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=615145/w=614607/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4806.4MB, aggrb=32043KB/s, minb=32043KB/s, maxb=32043KB/s, mint=153582msec, maxt=153582msec
WRITE: io=4801.7MB, aggrb=32014KB/s, minb=32014KB/s, maxb=32014KB/s, mint=153582msec, maxt=153582msec
Disk stats (read/write):
sdb: ios=655552/654989, merge=0/0, ticks=4795625/427912, in_queue=5223207, util=100.00%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdb-randrw-32-16K
===
mytest: (g=0): rw=randrw, bs=16K-16K/16K-16K/16K-16K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=29682: Sun Sep 15 08:52:09 2013
read : io=4749.3MB, bw=39393KB/s, iops=2461, runt=123452msec
slat (usec): min=6, max=348, avg=11.07, stdev= 3.87
clat (usec): min=498, max=27331, avg=9505.05, stdev=3934.46
lat (usec): min=518, max=27342, avg=9516.42, stdev=3934.69
clat percentiles (usec):
| 1.00th=[ 4128], 5.00th=[ 4704], 10.00th=[ 5024], 20.00th=[ 5536],
| 30.00th=[ 6112], 40.00th=[ 7200], 50.00th=[ 9152], 60.00th=[10688],
| 70.00th=[11968], 80.00th=[13376], 90.00th=[15040], 95.00th=[16192],
| 99.00th=[18816], 99.50th=[19328], 99.90th=[21376], 99.95th=[22144],
| 99.99th=[23680]
bw (KB /s): min=35680, max=43168, per=100.00%, avg=39418.33, stdev=1418.02
write: io=4734.8MB, bw=39273KB/s, iops=2454, runt=123452msec
slat (usec): min=8, max=3805, avg=14.88, stdev= 9.55
clat (usec): min=277, max=24788, avg=3471.82, stdev=1825.80
lat (usec): min=301, max=24798, avg=3487.09, stdev=1826.01
clat percentiles (usec):
| 1.00th=[ 1064], 5.00th=[ 1384], 10.00th=[ 1576], 20.00th=[ 1864],
| 30.00th=[ 2128], 40.00th=[ 2416], 50.00th=[ 2864], 60.00th=[ 3696],
| 70.00th=[ 4448], 80.00th=[ 5024], 90.00th=[ 6304], 95.00th=[ 6752],
| 99.00th=[ 8512], 99.50th=[ 8768], 99.90th=[10432], 99.95th=[10688],
| 99.99th=[12608]
bw (KB /s): min= 1, max=42880, per=99.66%, avg=39140.87, stdev=2841.50
lat (usec) : 500=0.01%, 750=0.04%, 1000=0.27%
lat (msec) : 2=12.43%, 4=20.07%, 10=44.93%, 20=22.12%, 50=0.14%
cpu : usr=4.06%, sys=8.15%, ctx=621512, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=108.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=303926/w=303013/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4749.3MB, aggrb=39393KB/s, minb=39393KB/s, maxb=39393KB/s, mint=123452msec, maxt=123452msec
WRITE: io=4734.8MB, aggrb=39273KB/s, minb=39273KB/s, maxb=39273KB/s, mint=123452msec, maxt=123452msec
Disk stats (read/write):
sdb: ios=327849/326834, merge=0/0, ticks=3118120/1134464, in_queue=4252289, util=100.00%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdb-randrw-32-32K
===
mytest: (g=0): rw=randrw, bs=32K-32K/32K-32K/32K-32K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=29868: Sun Sep 15 08:54:49 2013
read : io=4613.3MB, bw=53195KB/s, iops=1662, runt= 88805msec
slat (usec): min=7, max=319, avg=12.32, stdev= 3.72
clat (usec): min=402, max=26044, avg=11817.30, stdev=4063.36
lat (usec): min=419, max=26057, avg=11829.97, stdev=4063.38
clat percentiles (usec):
| 1.00th=[ 5600], 5.00th=[ 6368], 10.00th=[ 6816], 20.00th=[ 7648],
| 30.00th=[ 8640], 40.00th=[10048], 50.00th=[11456], 60.00th=[12864],
| 70.00th=[14272], 80.00th=[15680], 90.00th=[17536], 95.00th=[18816],
| 99.00th=[20864], 99.50th=[21632], 99.90th=[23168], 99.95th=[23936],
| 99.99th=[24704]
bw (KB /s): min= 3, max=60103, per=99.50%, avg=52929.46, stdev=4525.09
write: io=4613.4MB, bw=53196KB/s, iops=1662, runt= 88805msec
slat (usec): min=10, max=5156, avg=20.20, stdev=18.30
clat (usec): min=424, max=19968, avg=7395.55, stdev=2947.20
lat (usec): min=439, max=19985, avg=7416.45, stdev=2947.14
clat percentiles (usec):
| 1.00th=[ 3184], 5.00th=[ 3696], 10.00th=[ 3984], 20.00th=[ 4448],
| 30.00th=[ 4960], 40.00th=[ 5856], 50.00th=[ 7072], 60.00th=[ 8096],
| 70.00th=[ 9152], 80.00th=[10176], 90.00th=[11584], 95.00th=[12608],
| 99.00th=[14400], 99.50th=[15168], 99.90th=[16512], 99.95th=[17280],
| 99.99th=[18048]
bw (KB /s): min=47296, max=59200, per=100.00%, avg=53199.62, stdev=2085.19
lat (usec) : 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec) : 2=0.01%, 4=5.10%, 10=53.98%, 20=39.88%, 50=1.04%
cpu : usr=3.75%, sys=6.16%, ctx=321191, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=111.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=147609/w=147612/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4613.3MB, aggrb=53194KB/s, minb=53194KB/s, maxb=53194KB/s, mint=88805msec, maxt=88805msec
WRITE: io=4613.4MB, aggrb=53196KB/s, minb=53196KB/s, maxb=53196KB/s, mint=88805msec, maxt=88805msec
Disk stats (read/write):
sdb: ios=163895/163781, merge=0/0, ticks=1940548/1215022, in_queue=3155583, util=100.00%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdb-randrw-32-64K
===
mytest: (g=0): rw=randrw, bs=64K-64K/64K-64K/64K-64K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=30393: Sun Sep 15 08:56:58 2013
read : io=4381.9MB, bw=76723KB/s, iops=1198, runt= 58483msec
slat (usec): min=10, max=75, avg=14.75, stdev= 2.16
clat (msec): min=2, max=34, avg=14.79, stdev= 3.30
lat (msec): min=2, max=34, avg=14.81, stdev= 3.30
clat percentiles (usec):
| 1.00th=[ 8512], 5.00th=[ 9792], 10.00th=[10560], 20.00th=[11840],
| 30.00th=[12736], 40.00th=[13632], 50.00th=[14528], 60.00th=[15552],
| 70.00th=[16512], 80.00th=[17792], 90.00th=[19328], 95.00th=[20352],
| 99.00th=[22656], 99.50th=[23424], 99.90th=[24960], 99.95th=[25472],
| 99.99th=[28032]
bw (KB /s): min=70784, max=83800, per=100.00%, avg=76733.28, stdev=3133.07
write: io=4386.7MB, bw=76797KB/s, iops=1199, runt= 58483msec
slat (usec): min=15, max=9648, avg=25.27, stdev=50.54
clat (usec): min=508, max=30645, avg=11845.02, stdev=3165.30
lat (usec): min=530, max=30673, avg=11870.82, stdev=3164.35
clat percentiles (usec):
| 1.00th=[ 6368], 5.00th=[ 7200], 10.00th=[ 7840], 20.00th=[ 8768],
| 30.00th=[ 9792], 40.00th=[10688], 50.00th=[11584], 60.00th=[12608],
| 70.00th=[13632], 80.00th=[14784], 90.00th=[16192], 95.00th=[17280],
| 99.00th=[19328], 99.50th=[20096], 99.90th=[21632], 99.95th=[22144],
| 99.99th=[24704]
bw (KB /s): min= 6, max=83072, per=99.20%, avg=76185.64, stdev=7664.31
lat (usec) : 750=0.01%, 1000=0.01%
lat (msec) : 4=0.01%, 10=19.35%, 20=77.13%, 50=3.53%
cpu : usr=3.46%, sys=5.18%, ctx=163683, majf=0, minf=28
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=116.8%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=70091/w=70164/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4381.9MB, aggrb=76722KB/s, minb=76722KB/s, maxb=76722KB/s, mint=58483msec, maxt=58483msec
WRITE: io=4386.7MB, aggrb=76797KB/s, minb=76797KB/s, maxb=76797KB/s, mint=58483msec, maxt=58483msec
Disk stats (read/write):
sdb: ios=82000/81923, merge=0/0, ticks=1216120/972847, in_queue=2189035, util=99.95%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdb-randrw-32-128K
===
mytest: (g=0): rw=randrw, bs=128K-128K/128K-128K/128K-128K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=30521: Sun Sep 15 08:58:46 2013
read : io=4065.8MB, bw=109208KB/s, iops=852, runt= 38123msec
slat (usec): min=14, max=192, avg=23.89, stdev= 5.35
clat (msec): min=1, max=46, avg=19.65, stdev= 2.27
lat (msec): min=1, max=46, avg=19.68, stdev= 2.27
clat percentiles (usec):
| 1.00th=[14784], 5.00th=[16192], 10.00th=[17024], 20.00th=[17792],
| 30.00th=[18560], 40.00th=[19072], 50.00th=[19584], 60.00th=[20096],
| 70.00th=[20608], 80.00th=[21376], 90.00th=[22400], 95.00th=[23168],
| 99.00th=[25472], 99.50th=[26496], 99.90th=[32128], 99.95th=[36096],
| 99.99th=[43776]
bw (KB /s): min=98048, max=121600, per=100.00%, avg=109277.36, stdev=4722.41
write: io=4066.7MB, bw=109231KB/s, iops=852, runt= 38123msec
slat (usec): min=22, max=18431, avg=44.49, stdev=151.90
clat (usec): min=914, max=44907, avg=17780.96, stdev=2201.39
lat (usec): min=958, max=44940, avg=17826.24, stdev=2195.23
clat percentiles (usec):
| 1.00th=[12864], 5.00th=[14400], 10.00th=[15168], 20.00th=[16064],
| 30.00th=[16768], 40.00th=[17280], 50.00th=[17792], 60.00th=[18304],
| 70.00th=[18816], 80.00th=[19328], 90.00th=[20352], 95.00th=[21120],
| 99.00th=[23168], 99.50th=[24448], 99.90th=[28288], 99.95th=[31616],
| 99.99th=[41216]
bw (KB /s): min= 12, max=117248, per=98.76%, avg=107876.78, stdev=13064.06
lat (usec) : 1000=0.01%
lat (msec) : 2=0.02%, 4=0.02%, 10=0.05%, 20=72.25%, 50=27.70%
cpu : usr=3.90%, sys=6.38%, ctx=81806, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=125.9%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=32510/w=32518/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4065.8MB, aggrb=109207KB/s, minb=109207KB/s, maxb=109207KB/s, mint=38123msec, maxt=38123msec
WRITE: io=4066.7MB, aggrb=109231KB/s, minb=109231KB/s, maxb=109231KB/s, mint=38123msec, maxt=38123msec
Disk stats (read/write):
sdb: ios=40927/40809, merge=0/0, ticks=804353/726248, in_queue=1530786, util=99.89%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdb-randrw-32-256K
===
mytest: (g=0): rw=randrw, bs=256K-256K/256K-256K/256K-256K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=30650: Sun Sep 15 09:00:25 2013
read : io=3824.0MB, bw=136310KB/s, iops=532, runt= 28727msec
slat (usec): min=24, max=273, avg=39.71, stdev= 7.10
clat (msec): min=3, max=63, avg=30.77, stdev= 2.85
lat (msec): min=3, max=63, avg=30.81, stdev= 2.85
clat percentiles (usec):
| 1.00th=[25216], 5.00th=[26752], 10.00th=[27776], 20.00th=[28800],
| 30.00th=[29568], 40.00th=[30080], 50.00th=[30592], 60.00th=[31360],
| 70.00th=[31872], 80.00th=[32640], 90.00th=[33536], 95.00th=[34560],
| 99.00th=[37120], 99.50th=[38656], 99.90th=[55552], 99.95th=[57600],
| 99.99th=[62720]
bw (KB /s): min= 25, max=147456, per=98.23%, avg=133902.51, stdev=19483.64
write: io=3788.0MB, bw=135027KB/s, iops=526, runt= 28727msec
slat (usec): min=48, max=28153, avg=81.34, stdev=323.79
clat (msec): min=1, max=60, avg=29.50, stdev= 2.78
lat (msec): min=2, max=60, avg=29.59, stdev= 2.75
clat percentiles (usec):
| 1.00th=[23936], 5.00th=[25728], 10.00th=[26496], 20.00th=[27776],
| 30.00th=[28288], 40.00th=[28800], 50.00th=[29568], 60.00th=[30080],
| 70.00th=[30592], 80.00th=[31360], 90.00th=[32384], 95.00th=[33536],
| 99.00th=[36096], 99.50th=[37632], 99.90th=[51456], 99.95th=[55552],
| 99.99th=[59136]
bw (KB /s): min=121370, max=145408, per=100.00%, avg=135173.34, stdev=4747.12
lat (msec) : 2=0.01%, 4=0.03%, 10=0.11%, 20=0.15%, 50=99.65%
lat (msec) : 100=0.16%
cpu : usr=4.02%, sys=6.80%, ctx=40876, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=134.5%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=15283/w=15134/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=3824.0MB, aggrb=136309KB/s, minb=136309KB/s, maxb=136309KB/s, mint=28727msec, maxt=28727msec
WRITE: io=3788.0MB, aggrb=135026KB/s, minb=135026KB/s, maxb=135026KB/s, mint=28727msec, maxt=28727msec
Disk stats (read/write):
sdb: ios=20634/20406, merge=0/0, ticks=633078/601923, in_queue=1235414, util=99.85%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdb-randrw-32-512K
===
mytest: (g=0): rw=randrw, bs=512K-512K/512K-512K/512K-512K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=30803: Sun Sep 15 09:02:01 2013
read : io=3649.0MB, bw=147370KB/s, iops=287, runt= 25355msec
slat (usec): min=42, max=270, avg=67.03, stdev= 6.52
clat (msec): min=3, max=111, avg=55.55, stdev= 5.55
lat (msec): min=3, max=111, avg=55.61, stdev= 5.55
clat percentiles (msec):
| 1.00th=[ 46], 5.00th=[ 50], 10.00th=[ 51], 20.00th=[ 53],
| 30.00th=[ 54], 40.00th=[ 55], 50.00th=[ 56], 60.00th=[ 57],
| 70.00th=[ 58], 80.00th=[ 59], 90.00th=[ 61], 95.00th=[ 62],
| 99.00th=[ 68], 99.50th=[ 84], 99.90th=[ 104], 99.95th=[ 108],
| 99.99th=[ 113]
bw (KB /s): min=130048, max=171688, per=100.00%, avg=147784.40, stdev=9334.94
write: io=3696.0MB, bw=149269KB/s, iops=291, runt= 25355msec
slat (usec): min=93, max=52804, avg=156.73, stdev=854.04
clat (msec): min=3, max=108, avg=54.78, stdev= 5.53
lat (msec): min=3, max=109, avg=54.94, stdev= 5.43
clat percentiles (msec):
| 1.00th=[ 46], 5.00th=[ 49], 10.00th=[ 50], 20.00th=[ 52],
| 30.00th=[ 53], 40.00th=[ 55], 50.00th=[ 55], 60.00th=[ 56],
| 70.00th=[ 57], 80.00th=[ 58], 90.00th=[ 60], 95.00th=[ 61],
| 99.00th=[ 66], 99.50th=[ 82], 99.90th=[ 101], 99.95th=[ 104],
| 99.99th=[ 110]
bw (KB /s): min= 51, max=164864, per=97.95%, avg=146209.55, stdev=21853.84
lat (msec) : 4=0.06%, 10=0.08%, 20=0.19%, 50=7.56%, 100=92.19%
lat (msec) : 250=0.14%
cpu : usr=3.87%, sys=5.91%, ctx=20401, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.2%, 32=139.3%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=7278/w=7381/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=3649.0MB, aggrb=147370KB/s, minb=147370KB/s, maxb=147370KB/s, mint=25355msec, maxt=25355msec
WRITE: io=3696.0MB, aggrb=149268KB/s, minb=149268KB/s, maxb=149268KB/s, mint=25355msec, maxt=25355msec
Disk stats (read/write):
sdb: ios=10271/10195, merge=0/0, ticks=565221/556169, in_queue=1122318, util=99.82%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdb-randrw-32-1024K
===
mytest: (g=0): rw=randrw, bs=1M-1M/1M-1M/1M-1M, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=31218: Sun Sep 15 09:03:34 2013
read : io=3548.0MB, bw=162601KB/s, iops=158, runt= 22344msec
slat (usec): min=77, max=417, avg=122.64, stdev=13.18
clat (msec): min=6, max=261, avg=101.43, stdev=30.72
lat (msec): min=6, max=261, avg=101.56, stdev=30.72
clat percentiles (msec):
| 1.00th=[ 47], 5.00th=[ 57], 10.00th=[ 62], 20.00th=[ 73],
| 30.00th=[ 83], 40.00th=[ 92], 50.00th=[ 100], 60.00th=[ 111],
| 70.00th=[ 120], 80.00th=[ 129], 90.00th=[ 141], 95.00th=[ 153],
| 99.00th=[ 174], 99.50th=[ 184], 99.90th=[ 241], 99.95th=[ 260],
| 99.99th=[ 262]
bw (KB /s): min=100352, max=202752, per=99.27%, avg=161413.55, stdev=21949.72
write: io=3504.0MB, bw=160584KB/s, iops=155, runt= 22344msec
slat (usec): min=188, max=109760, avg=329.19, stdev=2460.60
clat (msec): min=5, max=268, avg=101.29, stdev=31.25
lat (msec): min=7, max=268, avg=101.64, stdev=31.15
clat percentiles (msec):
| 1.00th=[ 46], 5.00th=[ 57], 10.00th=[ 63], 20.00th=[ 72],
| 30.00th=[ 82], 40.00th=[ 91], 50.00th=[ 101], 60.00th=[ 110],
| 70.00th=[ 120], 80.00th=[ 129], 90.00th=[ 141], 95.00th=[ 153],
| 99.00th=[ 172], 99.50th=[ 186], 99.90th=[ 255], 99.95th=[ 262],
| 99.99th=[ 269]
bw (KB /s): min= 102, max=178176, per=97.06%, avg=155869.09, stdev=25952.95
lat (msec) : 10=0.13%, 20=0.14%, 50=1.40%, 100=48.23%, 250=50.45%
lat (msec) : 500=0.10%
cpu : usr=3.97%, sys=6.02%, ctx=10145, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.2%, 16=0.5%, 32=145.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=3536/w=3485/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=3548.0MB, aggrb=162600KB/s, minb=162600KB/s, maxb=162600KB/s, mint=22344msec, maxt=22344msec
WRITE: io=3504.0MB, aggrb=160584KB/s, minb=160584KB/s, maxb=160584KB/s, mint=22344msec, maxt=22344msec
Disk stats (read/write):
sdb: ios=10422/10028, merge=0/0, ticks=1034143/997628, in_queue=2032652, util=99.79%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdc-randrw-32-512B
===
mytest: (g=0): rw=randrw, bs=512-512/512-512/512-512, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=31341: Sun Sep 15 09:15:31 2013
read : io=5082.3MB, bw=8054.1KB/s, iops=16108, runt=646156msec
slat (usec): min=2, max=274, avg= 7.08, stdev= 4.05
clat (usec): min=35, max=44017, avg=1126.35, stdev=1044.34
lat (usec): min=43, max=44034, avg=1133.60, stdev=1044.54
clat percentiles (usec):
| 1.00th=[ 197], 5.00th=[ 290], 10.00th=[ 354], 20.00th=[ 434],
| 30.00th=[ 502], 40.00th=[ 580], 50.00th=[ 724], 60.00th=[ 972],
| 70.00th=[ 1320], 80.00th=[ 1720], 90.00th=[ 2480], 95.00th=[ 2992],
| 99.00th=[ 4384], 99.50th=[ 5408], 99.90th=[ 9920], 99.95th=[12864],
| 99.99th=[20864]
bw (KB /s): min= 609, max=16400, per=100.00%, avg=8066.48, stdev=4585.30
write: io=5081.2MB, bw=8052.5KB/s, iops=16104, runt=646156msec
slat (usec): min=2, max=390, avg= 8.20, stdev= 4.57
clat (usec): min=14, max=41186, avg=841.55, stdev=1008.60
lat (usec): min=38, max=41199, avg=849.93, stdev=1008.62
clat percentiles (usec):
| 1.00th=[ 74], 5.00th=[ 114], 10.00th=[ 163], 20.00th=[ 241],
| 30.00th=[ 294], 40.00th=[ 334], 50.00th=[ 378], 60.00th=[ 502],
| 70.00th=[ 908], 80.00th=[ 1432], 90.00th=[ 2192], 95.00th=[ 2800],
| 99.00th=[ 4048], 99.50th=[ 4960], 99.90th=[ 8640], 99.95th=[11584],
| 99.99th=[19328]
bw (KB /s): min= 0, max=16319, per=100.00%, avg=8058.59, stdev=4587.00
lat (usec) : 20=0.01%, 50=0.06%, 100=1.64%, 250=10.43%, 500=32.69%
lat (usec) : 750=14.16%, 1000=7.39%
lat (msec) : 2=19.97%, 4=12.47%, 10=1.11%, 20=0.07%, 50=0.01%
cpu : usr=7.98%, sys=31.45%, ctx=4364661, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.8%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=10408381/w=10406218/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=5082.3MB, aggrb=8054KB/s, minb=8054KB/s, maxb=8054KB/s, mint=646156msec, maxt=646156msec
WRITE: io=5081.2MB, aggrb=8052KB/s, minb=8052KB/s, maxb=8052KB/s, mint=646156msec, maxt=646156msec
Disk stats (read/write):
sdc: ios=10486713/10484409, merge=0/0, ticks=11687021/8775248, in_queue=20456248, util=100.00%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdc-randrw-32-1K
===
mytest: (g=0): rw=randrw, bs=1K-1K/1K-1K/1K-1K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=32018: Sun Sep 15 09:19:25 2013
read : io=4799.7MB, bw=30037KB/s, iops=30037, runt=163623msec
slat (usec): min=2, max=374, avg= 6.58, stdev= 4.32
clat (usec): min=54, max=25878, avg=741.88, stdev=754.69
lat (usec): min=79, max=25883, avg=748.55, stdev=754.73
clat percentiles (usec):
| 1.00th=[ 175], 5.00th=[ 241], 10.00th=[ 290], 20.00th=[ 358],
| 30.00th=[ 410], 40.00th=[ 454], 50.00th=[ 494], 60.00th=[ 540],
| 70.00th=[ 612], 80.00th=[ 844], 90.00th=[ 1624], 95.00th=[ 2352],
| 99.00th=[ 3216], 99.50th=[ 4576], 99.90th=[ 7328], 99.95th=[ 9280],
| 99.99th=[12608]
bw (KB /s): min= 3694, max=33320, per=100.00%, avg=30141.48, stdev=6736.61
write: io=4798.3MB, bw=30029KB/s, iops=30028, runt=163623msec
slat (usec): min=2, max=367, avg= 7.67, stdev= 4.91
clat (usec): min=21, max=20184, avg=305.98, stdev=405.57
lat (usec): min=40, max=20200, avg=313.77, stdev=405.64
clat percentiles (usec):
| 1.00th=[ 65], 5.00th=[ 96], 10.00th=[ 123], 20.00th=[ 169],
| 30.00th=[ 209], 40.00th=[ 247], 50.00th=[ 282], 60.00th=[ 310],
| 70.00th=[ 334], 80.00th=[ 362], 90.00th=[ 394], 95.00th=[ 426],
| 99.00th=[ 1704], 99.50th=[ 2992], 99.90th=[ 5856], 99.95th=[ 7200],
| 99.99th=[10560]
bw (KB /s): min= 0, max=33224, per=100.00%, avg=30041.03, stdev=6906.34
lat (usec) : 50=0.11%, 100=2.70%, 250=20.58%, 500=51.11%, 750=13.61%
lat (usec) : 1000=2.36%
lat (msec) : 2=5.44%, 4=3.59%, 10=0.47%, 20=0.02%, 50=0.01%
cpu : usr=13.81%, sys=57.88%, ctx=1652160, majf=0, minf=28
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=106.7%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=4914789/w=4913409/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4799.7MB, aggrb=30037KB/s, minb=30037KB/s, maxb=30037KB/s, mint=163623msec, maxt=163623msec
WRITE: io=4798.3MB, aggrb=30028KB/s, minb=30028KB/s, maxb=30028KB/s, mint=163623msec, maxt=163623msec
Disk stats (read/write):
sdc: ios=5243643/5241910, merge=0/0, ticks=3766482/1505578, in_queue=5268646, util=100.00%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdc-randrw-32-2K
===
mytest: (g=0): rw=randrw, bs=2K-2K/2K-2K/2K-2K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=32207: Sun Sep 15 09:21:49 2013
read : io=4473.3MB, bw=62441KB/s, iops=31219, runt= 73359msec
slat (usec): min=2, max=448, avg= 6.96, stdev= 4.44
clat (usec): min=46, max=17081, avg=706.46, stdev=677.21
lat (usec): min=70, max=17088, avg=713.45, stdev=677.19
clat percentiles (usec):
| 1.00th=[ 169], 5.00th=[ 237], 10.00th=[ 286], 20.00th=[ 350],
| 30.00th=[ 402], 40.00th=[ 442], 50.00th=[ 482], 60.00th=[ 524],
| 70.00th=[ 588], 80.00th=[ 772], 90.00th=[ 1528], 95.00th=[ 2288],
| 99.00th=[ 3056], 99.50th=[ 3760], 99.90th=[ 5920], 99.95th=[ 6816],
| 99.99th=[ 9792]
bw (KB /s): min= 0, max=67136, per=99.71%, avg=62258.45, stdev=10543.25
write: io=4470.8MB, bw=62406KB/s, iops=31203, runt= 73359msec
slat (usec): min=2, max=491, avg= 8.14, stdev= 5.01
clat (usec): min=6, max=12215, avg=300.30, stdev=278.44
lat (usec): min=42, max=12222, avg=308.54, stdev=278.43
clat percentiles (usec):
| 1.00th=[ 67], 5.00th=[ 109], 10.00th=[ 143], 20.00th=[ 187],
| 30.00th=[ 225], 40.00th=[ 258], 50.00th=[ 286], 60.00th=[ 314],
| 70.00th=[ 342], 80.00th=[ 370], 90.00th=[ 402], 95.00th=[ 430],
| 99.00th=[ 988], 99.50th=[ 1976], 99.90th=[ 4576], 99.95th=[ 5280],
| 99.99th=[ 7008]
bw (KB /s): min=17132, max=67196, per=100.00%, avg=62650.18, stdev=9229.67
lat (usec) : 10=0.01%, 50=0.11%, 100=1.85%, 250=20.08%, 500=54.39%
lat (usec) : 750=12.69%, 1000=2.11%
lat (msec) : 2=5.21%, 4=3.27%, 10=0.29%, 20=0.01%
cpu : usr=15.34%, sys=66.03%, ctx=923265, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=114.5%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=2290266/w=2289032/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4473.3MB, aggrb=62440KB/s, minb=62440KB/s, maxb=62440KB/s, mint=73359msec, maxt=73359msec
WRITE: io=4470.8MB, aggrb=62406KB/s, minb=62406KB/s, maxb=62406KB/s, mint=73359msec, maxt=73359msec
Disk stats (read/write):
sdc: ios=2621351/2619813, merge=0/0, ticks=1786923/739259, in_queue=2524123, util=99.97%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdc-randrw-32-4K
===
mytest: (g=0): rw=randrw, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=32386: Sun Sep 15 09:23:31 2013
read : io=3905.7MB, bw=124549KB/s, iops=31136, runt= 32111msec
slat (usec): min=2, max=251, avg= 7.03, stdev= 4.15
clat (usec): min=112, max=8927, avg=674.20, stdev=488.94
lat (usec): min=120, max=8938, avg=681.01, stdev=488.85
clat percentiles (usec):
| 1.00th=[ 235], 5.00th=[ 310], 10.00th=[ 358], 20.00th=[ 410],
| 30.00th=[ 454], 40.00th=[ 490], 50.00th=[ 532], 60.00th=[ 580],
| 70.00th=[ 644], 80.00th=[ 732], 90.00th=[ 1112], 95.00th=[ 1816],
| 99.00th=[ 2832], 99.50th=[ 2992], 99.90th=[ 3248], 99.95th=[ 3344],
| 99.99th=[ 3856]
bw (KB /s): min=122776, max=126128, per=100.00%, avg=124582.50, stdev=740.86
write: io=3903.2MB, bw=124470KB/s, iops=31117, runt= 32111msec
slat (usec): min=3, max=231, avg= 8.07, stdev= 4.29
clat (usec): min=41, max=8545, avg=336.19, stdev=95.17
lat (usec): min=52, max=8558, avg=344.01, stdev=94.76
clat percentiles (usec):
| 1.00th=[ 129], 5.00th=[ 187], 10.00th=[ 221], 20.00th=[ 262],
| 30.00th=[ 294], 40.00th=[ 322], 50.00th=[ 346], 60.00th=[ 366],
| 70.00th=[ 386], 80.00th=[ 406], 90.00th=[ 438], 95.00th=[ 462],
| 99.00th=[ 506], 99.50th=[ 524], 99.90th=[ 580], 99.95th=[ 796],
| 99.99th=[ 2704]
bw (KB /s): min= 0, max=126080, per=98.48%, avg=122581.05, stdev=15464.69
lat (usec) : 50=0.01%, 100=0.17%, 250=8.77%, 500=61.53%, 750=20.10%
lat (usec) : 1000=3.88%
lat (msec) : 2=3.42%, 4=2.12%, 10=0.01%
cpu : usr=22.06%, sys=73.91%, ctx=934654, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=131.1%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=999823/w=999208/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=3905.7MB, aggrb=124548KB/s, minb=124548KB/s, maxb=124548KB/s, mint=32111msec, maxt=32111msec
WRITE: io=3903.2MB, aggrb=124470KB/s, minb=124470KB/s, maxb=124470KB/s, mint=32111msec, maxt=32111msec
Disk stats (read/write):
sdc: ios=1305824/1304897, merge=0/0, ticks=881907/432406, in_queue=1313374, util=99.85%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdc-randrw-32-8K
===
mytest: (g=0): rw=randrw, bs=8K-8K/8K-8K/8K-8K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=383: Sun Sep 15 09:25:10 2013
read : io=3774.9MB, bw=137483KB/s, iops=17184, runt= 28116msec
slat (usec): min=2, max=229, avg= 7.99, stdev= 4.07
clat (usec): min=128, max=10506, avg=1240.09, stdev=845.24
lat (usec): min=136, max=10513, avg=1247.85, stdev=845.24
clat percentiles (usec):
| 1.00th=[ 302], 5.00th=[ 478], 10.00th=[ 572], 20.00th=[ 676],
| 30.00th=[ 748], 40.00th=[ 812], 50.00th=[ 892], 60.00th=[ 1032],
| 70.00th=[ 1288], 80.00th=[ 1752], 90.00th=[ 2608], 95.00th=[ 3088],
| 99.00th=[ 4080], 99.50th=[ 4512], 99.90th=[ 5280], 99.95th=[ 5600],
| 99.99th=[ 6560]
bw (KB /s): min= 0, max=144112, per=98.28%, avg=135121.00, stdev=18491.83
write: io=3775.7MB, bw=137511KB/s, iops=17188, runt= 28116msec
slat (usec): min=3, max=458, avg= 9.93, stdev= 4.66
clat (usec): min=44, max=9849, avg=601.20, stdev=363.50
lat (usec): min=57, max=9856, avg=610.89, stdev=363.39
clat percentiles (usec):
| 1.00th=[ 121], 5.00th=[ 235], 10.00th=[ 314], 20.00th=[ 418],
| 30.00th=[ 482], 40.00th=[ 532], 50.00th=[ 572], 60.00th=[ 604],
| 70.00th=[ 644], 80.00th=[ 684], 90.00th=[ 748], 95.00th=[ 1012],
| 99.00th=[ 2544], 99.50th=[ 2736], 99.90th=[ 3088], 99.95th=[ 3440],
| 99.99th=[ 5920]
bw (KB /s): min=129648, max=143024, per=100.00%, avg=137586.66, stdev=2950.29
lat (usec) : 50=0.01%, 100=0.27%, 250=2.85%, 500=16.38%, 750=40.86%
lat (usec) : 1000=16.32%
lat (msec) : 2=14.15%, 4=8.59%, 10=0.58%, 20=0.01%
cpu : usr=21.43%, sys=50.69%, ctx=858276, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=135.6%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=483166/w=483271/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=3774.9MB, aggrb=137482KB/s, minb=137482KB/s, maxb=137482KB/s, mint=28116msec, maxt=28116msec
WRITE: io=3775.7MB, aggrb=137511KB/s, minb=137511KB/s, maxb=137511KB/s, mint=28116msec, maxt=28116msec
Disk stats (read/write):
sdc: ios=652509/651953, merge=0/0, ticks=809251/396458, in_queue=1205398, util=99.83%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdc-randrw-32-16K
===
mytest: (g=0): rw=randrw, bs=16K-16K/16K-16K/16K-16K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=468: Sun Sep 15 09:26:42 2013
read : io=3496.6MB, bw=166557KB/s, iops=10408, runt= 21497msec
slat (usec): min=3, max=207, avg= 8.30, stdev= 2.62
clat (usec): min=261, max=11435, avg=2218.41, stdev=1256.69
lat (usec): min=269, max=11442, avg=2226.28, stdev=1256.70
clat percentiles (usec):
| 1.00th=[ 556], 5.00th=[ 772], 10.00th=[ 924], 20.00th=[ 1112],
| 30.00th=[ 1320], 40.00th=[ 1608], 50.00th=[ 1912], 60.00th=[ 2256],
| 70.00th=[ 2640], 80.00th=[ 3184], 90.00th=[ 4256], 95.00th=[ 4896],
| 99.00th=[ 5536], 99.50th=[ 5728], 99.90th=[ 6048], 99.95th=[ 6176],
| 99.99th=[ 8640]
bw (KB /s): min= 1, max=170784, per=97.71%, avg=162735.65, stdev=25506.29
write: io=3481.6MB, bw=165843KB/s, iops=10364, runt= 21497msec
slat (usec): min=5, max=1161, avg=10.69, stdev= 3.70
clat (usec): min=83, max=10056, avg=837.55, stdev=309.30
lat (usec): min=94, max=10062, avg=847.72, stdev=309.24
clat percentiles (usec):
| 1.00th=[ 326], 5.00th=[ 442], 10.00th=[ 510], 20.00th=[ 604],
| 30.00th=[ 684], 40.00th=[ 756], 50.00th=[ 820], 60.00th=[ 876],
| 70.00th=[ 948], 80.00th=[ 1020], 90.00th=[ 1160], 95.00th=[ 1304],
| 99.00th=[ 1752], 99.50th=[ 1944], 99.90th=[ 2576], 99.95th=[ 4896],
| 99.99th=[ 8032]
bw (KB /s): min=161664, max=171872, per=99.95%, avg=165766.86, stdev=2514.83
lat (usec) : 100=0.01%, 250=0.10%, 500=4.72%, 750=17.16%, 1000=23.59%
lat (msec) : 2=30.48%, 4=18.05%, 10=5.91%, 20=0.01%
cpu : usr=22.12%, sys=34.55%, ctx=644078, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=146.7%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=223756/w=222814/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=3496.6MB, aggrb=166557KB/s, minb=166557KB/s, maxb=166557KB/s, mint=21497msec, maxt=21497msec
WRITE: io=3481.6MB, aggrb=165843KB/s, minb=165843KB/s, maxb=165843KB/s, mint=21497msec, maxt=21497msec
Disk stats (read/write):
sdc: ios=327558/326590, merge=0/0, ticks=724540/276210, in_queue=1000769, util=99.78%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdc-randrw-32-32K
===
mytest: (g=0): rw=randrw, bs=32K-32K/32K-32K/32K-32K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=588: Sun Sep 15 09:28:17 2013
read : io=3675.7MB, bw=154682KB/s, iops=4833, runt= 24333msec
slat (usec): min=7, max=206, avg=11.19, stdev= 1.74
clat (usec): min=332, max=14028, avg=2692.05, stdev=1299.25
lat (usec): min=344, max=14036, avg=2703.27, stdev=1299.16
clat percentiles (usec):
| 1.00th=[ 916], 5.00th=[ 1176], 10.00th=[ 1336], 20.00th=[ 1560],
| 30.00th=[ 1784], 40.00th=[ 2040], 50.00th=[ 2352], 60.00th=[ 2704],
| 70.00th=[ 3216], 80.00th=[ 3824], 90.00th=[ 4576], 95.00th=[ 5152],
| 99.00th=[ 6304], 99.50th=[ 6816], 99.90th=[ 8384], 99.95th=[ 9024],
| 99.99th=[10944]
bw (KB /s): min=141504, max=175808, per=99.92%, avg=154554.94, stdev=8900.99
write: io=3681.3MB, bw=154917KB/s, iops=4840, runt= 24333msec
slat (usec): min=10, max=1301, avg=16.86, stdev= 5.49
clat (usec): min=389, max=14023, avg=3888.75, stdev=1183.72
lat (usec): min=403, max=14036, avg=3906.00, stdev=1183.46
clat percentiles (usec):
| 1.00th=[ 2256], 5.00th=[ 2576], 10.00th=[ 2768], 20.00th=[ 2992],
| 30.00th=[ 3152], 40.00th=[ 3344], 50.00th=[ 3536], 60.00th=[ 3792],
| 70.00th=[ 4192], 80.00th=[ 4768], 90.00th=[ 5536], 95.00th=[ 6176],
| 99.00th=[ 7776], 99.50th=[ 8512], 99.90th=[10048], 99.95th=[10432],
| 99.99th=[11584]
bw (KB /s): min= 3, max=175872, per=97.79%, avg=151490.84, stdev=23901.92
lat (usec) : 500=0.01%, 750=0.12%, 1000=0.80%
lat (msec) : 2=18.35%, 4=55.05%, 10=25.64%, 20=0.06%
cpu : usr=13.40%, sys=20.26%, ctx=325627, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=139.2%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=117611/w=117779/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=3675.7MB, aggrb=154681KB/s, minb=154681KB/s, maxb=154681KB/s, mint=24333msec, maxt=24333msec
WRITE: io=3681.3MB, aggrb=154917KB/s, minb=154917KB/s, maxb=154917KB/s, mint=24333msec, maxt=24333msec
Disk stats (read/write):
sdc: ios=163085/162946, merge=0/1, ticks=444948/644782, in_queue=1089759, util=99.80%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdc-randrw-32-64K
===
mytest: (g=0): rw=randrw, bs=64K-64K/64K-64K/64K-64K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=687: Sun Sep 15 09:29:50 2013
read : io=3553.8MB, bw=159241KB/s, iops=2487, runt= 22852msec
slat (usec): min=9, max=344, avg=13.87, stdev= 4.39
clat (usec): min=605, max=17764, avg=4713.80, stdev=1620.56
lat (usec): min=618, max=17780, avg=4727.60, stdev=1620.43
clat percentiles (usec):
| 1.00th=[ 1592], 5.00th=[ 2064], 10.00th=[ 2416], 20.00th=[ 3184],
| 30.00th=[ 3856], 40.00th=[ 4384], 50.00th=[ 4832], 60.00th=[ 5216],
| 70.00th=[ 5600], 80.00th=[ 6048], 90.00th=[ 6752], 95.00th=[ 7264],
| 99.00th=[ 8512], 99.50th=[ 9152], 99.90th=[10944], 99.95th=[11712],
| 99.99th=[15808]
bw (KB /s): min= 6, max=167168, per=97.80%, avg=155743.78, stdev=24211.97
write: io=3558.2MB, bw=159443KB/s, iops=2490, runt= 22852msec
slat (usec): min=14, max=4019, avg=23.23, stdev=18.27
clat (usec): min=524, max=20653, avg=8096.80, stdev=2417.24
lat (usec): min=545, max=20674, avg=8120.07, stdev=2416.93
clat percentiles (usec):
| 1.00th=[ 4256], 5.00th=[ 4448], 10.00th=[ 4960], 20.00th=[ 6176],
| 30.00th=[ 6880], 40.00th=[ 7328], 50.00th=[ 7840], 60.00th=[ 8384],
| 70.00th=[ 8896], 80.00th=[ 9664], 90.00th=[11456], 95.00th=[12736],
| 99.00th=[15424], 99.50th=[16320], 99.90th=[18304], 99.95th=[18816],
| 99.99th=[19840]
bw (KB /s): min=146322, max=170496, per=100.00%, avg=159498.93, stdev=5796.24
lat (usec) : 750=0.01%, 1000=0.01%
lat (msec) : 2=2.16%, 4=14.24%, 10=74.65%, 20=8.97%, 50=0.01%
cpu : usr=8.85%, sys=12.69%, ctx=163532, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=144.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=56839/w=56920/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=3553.8MB, aggrb=159241KB/s, minb=159241KB/s, maxb=159241KB/s, mint=22852msec, maxt=22852msec
WRITE: io=3558.2MB, aggrb=159442KB/s, minb=159442KB/s, maxb=159442KB/s, mint=22852msec, maxt=22852msec
Disk stats (read/write):
sdc: ios=81610/81512, merge=0/0, ticks=382050/659214, in_queue=1041337, util=99.80%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdc-randrw-32-128K
===
mytest: (g=0): rw=randrw, bs=128K-128K/128K-128K/128K-128K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=816: Sun Sep 15 09:31:18 2013
read : io=3241.6MB, bw=191767KB/s, iops=1497, runt= 17309msec
slat (usec): min=14, max=85, avg=20.93, stdev= 3.23
clat (msec): min=1, max=21, avg= 8.56, stdev= 2.01
lat (msec): min=1, max=21, avg= 8.58, stdev= 2.01
clat percentiles (usec):
| 1.00th=[ 3952], 5.00th=[ 5216], 10.00th=[ 5984], 20.00th=[ 6880],
| 30.00th=[ 7584], 40.00th=[ 8096], 50.00th=[ 8640], 60.00th=[ 9024],
| 70.00th=[ 9536], 80.00th=[10176], 90.00th=[10944], 95.00th=[11712],
| 99.00th=[13632], 99.50th=[14400], 99.90th=[17024], 99.95th=[18048],
| 99.99th=[20352]
bw (KB /s): min= 12, max=205056, per=97.19%, avg=186384.49, stdev=32798.07
write: io=3242.7MB, bw=191834KB/s, iops=1497, runt= 17309msec
slat (usec): min=23, max=264, avg=41.50, stdev= 6.78
clat (msec): min=1, max=29, avg=12.73, stdev= 2.56
lat (msec): min=1, max=29, avg=12.78, stdev= 2.56
clat percentiles (usec):
| 1.00th=[ 8256], 5.00th=[ 8640], 10.00th=[ 9792], 20.00th=[10816],
| 30.00th=[11456], 40.00th=[11968], 50.00th=[12480], 60.00th=[12992],
| 70.00th=[13632], 80.00th=[14400], 90.00th=[15808], 95.00th=[17536],
| 99.00th=[21120], 99.50th=[22400], 99.90th=[25728], 99.95th=[26496],
| 99.99th=[29312]
bw (KB /s): min=177408, max=199536, per=100.00%, avg=191885.68, stdev=5206.26
lat (msec) : 2=0.01%, 4=0.55%, 10=43.93%, 20=54.75%, 50=0.82%
cpu : usr=8.29%, sys=12.09%, ctx=81909, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=157.9%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=25917/w=25925/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=3241.6MB, aggrb=191767KB/s, minb=191767KB/s, maxb=191767KB/s, mint=17309msec, maxt=17309msec
WRITE: io=3242.7MB, aggrb=191833KB/s, minb=191833KB/s, maxb=191833KB/s, mint=17309msec, maxt=17309msec
Disk stats (read/write):
sdc: ios=40801/40757, merge=0/0, ticks=347705/519255, in_queue=867014, util=99.74%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdc-randrw-32-256K
===
mytest: (g=0): rw=randrw, bs=256K-256K/256K-256K/256K-256K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=891: Sun Sep 15 09:32:43 2013
read : io=3090.6MB, bw=209775KB/s, iops=818, runt= 15086msec
slat (usec): min=24, max=260, avg=37.29, stdev= 7.30
clat (msec): min=1, max=37, avg=15.48, stdev= 3.42
lat (msec): min=1, max=37, avg=15.51, stdev= 3.42
clat percentiles (usec):
| 1.00th=[ 7840], 5.00th=[ 9920], 10.00th=[10944], 20.00th=[12480],
| 30.00th=[13632], 40.00th=[14656], 50.00th=[15552], 60.00th=[16512],
| 70.00th=[17536], 80.00th=[18560], 90.00th=[19584], 95.00th=[20608],
| 99.00th=[22912], 99.50th=[24192], 99.90th=[29824], 99.95th=[33024],
| 99.99th=[36608]
bw (KB /s): min= 25, max=221184, per=96.63%, avg=202714.70, stdev=38900.97
write: io=3064.6MB, bw=208011KB/s, iops=811, runt= 15086msec
slat (usec): min=41, max=426, avg=75.47, stdev=11.34
clat (msec): min=3, max=42, avg=23.68, stdev= 4.02
lat (msec): min=3, max=42, avg=23.75, stdev= 4.02
clat percentiles (usec):
| 1.00th=[16768], 5.00th=[18048], 10.00th=[18816], 20.00th=[20096],
| 30.00th=[21120], 40.00th=[22144], 50.00th=[23424], 60.00th=[24448],
| 70.00th=[25472], 80.00th=[27008], 90.00th=[29056], 95.00th=[30848],
| 99.00th=[35072], 99.50th=[36608], 99.90th=[40192], 99.95th=[41216],
| 99.99th=[41728]
bw (KB /s): min=185344, max=220672, per=100.00%, avg=208201.24, stdev=8476.11
lat (msec) : 2=0.01%, 4=0.02%, 10=2.68%, 20=53.63%, 50=43.80%
cpu : usr=7.48%, sys=11.74%, ctx=40959, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=166.3%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=12351/w=12238/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=3090.6MB, aggrb=209775KB/s, minb=209775KB/s, maxb=209775KB/s, mint=15086msec, maxt=15086msec
WRITE: io=3064.6MB, aggrb=208010KB/s, minb=208010KB/s, maxb=208010KB/s, mint=15086msec, maxt=15086msec
Disk stats (read/write):
sdc: ios=20450/20294, merge=0/0, ticks=314669/479303, in_queue=794241, util=99.71%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdc-randrw-32-512K
===
mytest: (g=0): rw=randrw, bs=512K-512K/512K-512K/512K-512K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=1283: Sun Sep 15 09:34:08 2013
read : io=2963.0MB, bw=218360KB/s, iops=425, runt= 13895msec
slat (usec): min=40, max=123, avg=65.68, stdev= 7.21
clat (msec): min=3, max=66, avg=28.60, stdev= 6.78
lat (msec): min=3, max=66, avg=28.67, stdev= 6.78
clat percentiles (usec):
| 1.00th=[12992], 5.00th=[17280], 10.00th=[19584], 20.00th=[22400],
| 30.00th=[24960], 40.00th=[27008], 50.00th=[28800], 60.00th=[30848],
| 70.00th=[33024], 80.00th=[34560], 90.00th=[37120], 95.00th=[38656],
| 99.00th=[41728], 99.50th=[42752], 99.90th=[61184], 99.95th=[64256],
| 99.99th=[66048]
bw (KB /s): min= 51, max=237568, per=96.50%, avg=210724.07, stdev=43640.99
write: io=2974.0MB, bw=219171KB/s, iops=427, runt= 13895msec
slat (usec): min=67, max=357, avg=139.96, stdev=12.44
clat (msec): min=32, max=81, avg=46.13, stdev= 7.91
lat (msec): min=32, max=81, avg=46.27, stdev= 7.91
clat percentiles (usec):
| 1.00th=[33536], 5.00th=[35072], 10.00th=[36608], 20.00th=[38656],
| 30.00th=[41216], 40.00th=[43264], 50.00th=[45312], 60.00th=[47360],
| 70.00th=[49920], 80.00th=[52480], 90.00th=[56576], 95.00th=[59648],
| 99.00th=[69120], 99.50th=[73216], 99.90th=[78336], 99.95th=[79360],
| 99.99th=[81408]
bw (KB /s): min=195584, max=253952, per=99.90%, avg=218945.65, stdev=14391.87
lat (msec) : 4=0.01%, 10=0.11%, 20=5.27%, 50=80.09%, 100=14.79%
cpu : usr=6.87%, sys=10.52%, ctx=20490, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.3%, 32=172.4%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=5909/w=5934/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=2963.0MB, aggrb=218359KB/s, minb=218359KB/s, maxb=218359KB/s, mint=13895msec, maxt=13895msec
WRITE: io=2974.0MB, aggrb=219170KB/s, minb=219170KB/s, maxb=219170KB/s, mint=13895msec, maxt=13895msec
Disk stats (read/write):
sdc: ios=10201/10139, merge=0/0, ticks=287879/466284, in_queue=754642, util=99.67%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdc-randrw-32-1024K
===
mytest: (g=0): rw=randrw, bs=1M-1M/1M-1M/1M-1M, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=1443: Sun Sep 15 09:35:32 2013
read : io=2981.0MB, bw=222489KB/s, iops=216, runt= 13720msec
slat (usec): min=74, max=384, avg=122.10, stdev=12.70
clat (msec): min=6, max=266, avg=67.04, stdev=24.08
lat (msec): min=6, max=266, avg=67.16, stdev=24.08
clat percentiles (msec):
| 1.00th=[ 23], 5.00th=[ 32], 10.00th=[ 38], 20.00th=[ 46],
| 30.00th=[ 53], 40.00th=[ 60], 50.00th=[ 67], 60.00th=[ 73],
| 70.00th=[ 80], 80.00th=[ 87], 90.00th=[ 98], 95.00th=[ 105],
| 99.00th=[ 122], 99.50th=[ 141], 99.90th=[ 241], 99.95th=[ 249],
| 99.99th=[ 269]
bw (KB /s): min= 102, max=249856, per=96.97%, avg=215749.96, stdev=46041.85
write: io=2921.0MB, bw=218010KB/s, iops=211, runt= 13720msec
slat (usec): min=177, max=554, avg=268.94, stdev=25.47
clat (msec): min=10, max=247, avg=81.89, stdev=24.12
lat (msec): min=10, max=248, avg=82.16, stdev=24.12
clat percentiles (msec):
| 1.00th=[ 40], 5.00th=[ 47], 10.00th=[ 52], 20.00th=[ 60],
| 30.00th=[ 68], 40.00th=[ 75], 50.00th=[ 81], 60.00th=[ 88],
| 70.00th=[ 95], 80.00th=[ 103], 90.00th=[ 113], 95.00th=[ 121],
| 99.00th=[ 141], 99.50th=[ 151], 99.90th=[ 237], 99.95th=[ 247],
| 99.99th=[ 249]
bw (KB /s): min=181545, max=251401, per=99.70%, avg=217354.58, stdev=19036.00
lat (msec) : 10=0.05%, 20=0.27%, 50=16.79%, 100=67.82%, 250=15.57%
lat (msec) : 500=0.02%
cpu : usr=6.09%, sys=10.12%, ctx=10240, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.3%, 16=0.5%, 32=173.4%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=2970/w=2901/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=2981.0MB, aggrb=222488KB/s, minb=222488KB/s, maxb=222488KB/s, mint=13720msec, maxt=13720msec
WRITE: io=2921.0MB, aggrb=218010KB/s, minb=218010KB/s, maxb=218010KB/s, mint=13720msec, maxt=13720msec
Disk stats (read/write):
sdc: ios=10338/9940, merge=0/0, ticks=671553/803373, in_queue=1477494, util=99.63%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdd-randrw-32-512B
===
mytest: (g=0): rw=randrw, bs=512-512/512-512/512-512, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=1568: Sun Sep 15 09:50:43 2013
read : io=5013.5MB, bw=6102.9KB/s, iops=12205, runt=841209msec
slat (usec): min=2, max=247, avg= 7.93, stdev= 4.68
clat (usec): min=208, max=667435, avg=1967.07, stdev=4155.58
lat (usec): min=214, max=667445, avg=1975.18, stdev=4155.64
clat percentiles (usec):
| 1.00th=[ 668], 5.00th=[ 892], 10.00th=[ 1004], 20.00th=[ 1128],
| 30.00th=[ 1208], 40.00th=[ 1288], 50.00th=[ 1368], 60.00th=[ 1464],
| 70.00th=[ 1560], 80.00th=[ 1704], 90.00th=[ 2160], 95.00th=[ 7008],
| 99.00th=[10560], 99.50th=[11840], 99.90th=[40704], 99.95th=[43264],
| 99.99th=[46848]
bw (KB /s): min= 366, max=11152, per=100.00%, avg=6136.06, stdev=1763.89
write: io=5012.5MB, bw=6101.7KB/s, iops=12203, runt=841209msec
slat (usec): min=3, max=1690, avg= 9.28, stdev= 5.47
clat (usec): min=0, max=652603, avg=633.83, stdev=2816.80
lat (usec): min=31, max=652614, avg=643.32, stdev=2816.87
clat percentiles (usec):
| 1.00th=[ 29], 5.00th=[ 36], 10.00th=[ 52], 20.00th=[ 91],
| 30.00th=[ 135], 40.00th=[ 183], 50.00th=[ 239], 60.00th=[ 322],
| 70.00th=[ 426], 80.00th=[ 548], 90.00th=[ 764], 95.00th=[ 2192],
| 99.00th=[ 8768], 99.50th=[ 9536], 99.90th=[18048], 99.95th=[40192],
| 99.99th=[44288]
bw (KB /s): min= 0, max=10675, per=100.00%, avg=6131.20, stdev=1767.26
lat (usec) : 2=0.01%, 20=0.01%, 50=4.64%, 100=6.39%, 250=14.73%
lat (usec) : 500=12.62%, 750=7.37%, 1000=5.92%
lat (msec) : 2=40.05%, 4=2.64%, 10=4.58%, 20=0.91%, 50=0.14%
lat (msec) : 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%
cpu : usr=6.59%, sys=25.84%, ctx=5331333, majf=0, minf=28
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=102.1%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=10267437/w=10265439/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=5013.5MB, aggrb=6102KB/s, minb=6102KB/s, maxb=6102KB/s, mint=841209msec, maxt=841209msec
WRITE: io=5012.5MB, aggrb=6101KB/s, minb=6101KB/s, maxb=6101KB/s, mint=841209msec, maxt=841209msec
Disk stats (read/write):
sdd: ios=10486492/10484229, merge=0/0, ticks=20208326/6498659, in_queue=26699558, util=100.00%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdd-randrw-32-1K
===
mytest: (g=0): rw=randrw, bs=1K-1K/1K-1K/1K-1K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=2819: Sun Sep 15 09:59:13 2013
read : io=4986.1MB, bw=11630KB/s, iops=11630, runt=439090msec
slat (usec): min=2, max=389, avg= 7.97, stdev= 5.32
clat (usec): min=381, max=592035, avg=2054.13, stdev=4070.38
lat (usec): min=415, max=592039, avg=2062.30, stdev=4070.41
clat percentiles (usec):
| 1.00th=[ 900], 5.00th=[ 1032], 10.00th=[ 1096], 20.00th=[ 1208],
| 30.00th=[ 1288], 40.00th=[ 1352], 50.00th=[ 1432], 60.00th=[ 1512],
| 70.00th=[ 1624], 80.00th=[ 1768], 90.00th=[ 2384], 95.00th=[ 7072],
| 99.00th=[10688], 99.50th=[11968], 99.90th=[39680], 99.95th=[41728],
| 99.99th=[45824]
bw (KB /s): min= 0, max=14298, per=100.00%, avg=11686.88, stdev=2986.28
write: io=4985.6MB, bw=11627KB/s, iops=11626, runt=439090msec
slat (usec): min=2, max=9361, avg= 9.39, stdev= 7.42
clat (usec): min=0, max=591091, avg=676.08, stdev=2878.13
lat (usec): min=29, max=591096, avg=685.69, stdev=2878.17
clat percentiles (usec):
| 1.00th=[ 29], 5.00th=[ 41], 10.00th=[ 64], 20.00th=[ 117],
| 30.00th=[ 169], 40.00th=[ 219], 50.00th=[ 278], 60.00th=[ 366],
| 70.00th=[ 466], 80.00th=[ 588], 90.00th=[ 804], 95.00th=[ 2480],
| 99.00th=[ 8768], 99.50th=[ 9664], 99.90th=[28032], 99.95th=[38656],
| 99.99th=[43264]
bw (KB /s): min= 580, max=14346, per=100.00%, avg=11697.36, stdev=2953.45
lat (usec) : 2=0.01%, 10=0.01%, 20=0.01%, 50=3.47%, 100=4.88%
lat (usec) : 250=14.43%, 500=13.79%, 750=7.56%, 1000=4.10%
lat (msec) : 2=42.67%, 4=3.20%, 10=4.77%, 20=0.97%, 50=0.14%
lat (msec) : 100=0.01%, 500=0.01%, 750=0.01%
cpu : usr=6.96%, sys=25.50%, ctx=2884700, majf=0, minf=28
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=102.7%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=5106631/w=5105229/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4986.1MB, aggrb=11630KB/s, minb=11630KB/s, maxb=11630KB/s, mint=439090msec, maxt=439090msec
WRITE: io=4985.6MB, aggrb=11626KB/s, minb=11626KB/s, maxb=11626KB/s, mint=439090msec, maxt=439090msec
Disk stats (read/write):
sdd: ios=5243383/5241639, merge=0/0, ticks=10630036/3447950, in_queue=14073534, util=100.00%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdd-randrw-32-2K
===
mytest: (g=0): rw=randrw, bs=2K-2K/2K-2K/2K-2K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=3283: Sun Sep 15 10:03:48 2013
read : io=4853.2MB, bw=24228KB/s, iops=12114, runt=205116msec
slat (usec): min=2, max=365, avg= 8.03, stdev= 5.21
clat (usec): min=342, max=36201, avg=1970.37, stdev=1879.03
lat (usec): min=346, max=36205, avg=1978.56, stdev=1879.03
clat percentiles (usec):
| 1.00th=[ 908], 5.00th=[ 1032], 10.00th=[ 1112], 20.00th=[ 1208],
| 30.00th=[ 1288], 40.00th=[ 1368], 50.00th=[ 1448], 60.00th=[ 1528],
| 70.00th=[ 1640], 80.00th=[ 1784], 90.00th=[ 2320], 95.00th=[ 6752],
| 99.00th=[10560], 99.50th=[11072], 99.90th=[12864], 99.95th=[13376],
| 99.99th=[15680]
bw (KB /s): min= 0, max=28060, per=99.91%, avg=24205.53, stdev=4288.36
write: io=4850.9MB, bw=24217KB/s, iops=12108, runt=205116msec
slat (usec): min=3, max=1817, avg= 9.48, stdev= 6.17
clat (usec): min=0, max=29049, avg=649.97, stdev=1404.89
lat (usec): min=32, max=29059, avg=659.66, stdev=1404.90
clat percentiles (usec):
| 1.00th=[ 33], 5.00th=[ 51], 10.00th=[ 79], 20.00th=[ 139],
| 30.00th=[ 191], 40.00th=[ 245], 50.00th=[ 314], 60.00th=[ 406],
| 70.00th=[ 502], 80.00th=[ 620], 90.00th=[ 828], 95.00th=[ 2288],
| 99.00th=[ 8640], 99.50th=[ 9536], 99.90th=[10944], 99.95th=[11584],
| 99.99th=[12992]
bw (KB /s): min= 6944, max=28608, per=100.00%, avg=24253.05, stdev=4131.67
lat (usec) : 2=0.01%, 10=0.01%, 20=0.01%, 50=2.39%, 100=4.29%
lat (usec) : 250=13.65%, 500=14.63%, 750=8.61%, 1000=4.53%
lat (msec) : 2=42.85%, 4=3.38%, 10=4.69%, 20=0.97%, 50=0.01%
cpu : usr=7.56%, sys=27.42%, ctx=1501521, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=105.5%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=2484782/w=2483625/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4853.2MB, aggrb=24228KB/s, minb=24228KB/s, maxb=24228KB/s, mint=205116msec, maxt=205116msec
WRITE: io=4850.9MB, aggrb=24216KB/s, minb=24216KB/s, maxb=24216KB/s, mint=205116msec, maxt=205116msec
Disk stats (read/write):
sdd: ios=2621514/2619971, merge=0/0, ticks=5091202/1654571, in_queue=6743625, util=100.00%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdd-randrw-32-4K
===
mytest: (g=0): rw=randrw, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=3588: Sun Sep 15 10:05:45 2013
read : io=4196.3MB, bw=93780KB/s, iops=23444, runt= 45820msec
slat (usec): min=2, max=396, avg= 7.74, stdev= 4.24
clat (usec): min=294, max=32919, avg=1125.37, stdev=1377.15
lat (usec): min=298, max=32926, avg=1132.98, stdev=1377.13
clat percentiles (usec):
| 1.00th=[ 486], 5.00th=[ 556], 10.00th=[ 604], 20.00th=[ 652],
| 30.00th=[ 692], 40.00th=[ 724], 50.00th=[ 756], 60.00th=[ 804],
| 70.00th=[ 852], 80.00th=[ 924], 90.00th=[ 1096], 95.00th=[ 5408],
| 99.00th=[ 6816], 99.50th=[ 6944], 99.90th=[ 7328], 99.95th=[ 7520],
| 99.99th=[25472]
bw (KB /s): min= 0, max=96608, per=98.92%, avg=92767.53, stdev=10024.00
write: io=4195.5MB, bw=93752KB/s, iops=23437, runt= 45820msec
slat (usec): min=2, max=693, avg= 9.45, stdev= 5.12
clat (usec): min=1, max=14086, avg=219.22, stdev=130.17
lat (usec): min=40, max=14094, avg=228.69, stdev=130.15
clat percentiles (usec):
| 1.00th=[ 41], 5.00th=[ 67], 10.00th=[ 97], 20.00th=[ 143],
| 30.00th=[ 175], 40.00th=[ 199], 50.00th=[ 223], 60.00th=[ 245],
| 70.00th=[ 266], 80.00th=[ 290], 90.00th=[ 322], 95.00th=[ 350],
| 99.00th=[ 402], 99.50th=[ 430], 99.90th=[ 556], 99.95th=[ 860],
| 99.99th=[ 5536]
bw (KB /s): min=85768, max=95856, per=100.00%, avg=93771.73, stdev=2087.59
lat (usec) : 2=0.01%, 50=1.17%, 100=4.02%, 250=26.18%, 500=19.30%
lat (usec) : 750=23.00%, 1000=19.48%
lat (msec) : 2=3.50%, 4=0.02%, 10=3.33%, 20=0.01%, 50=0.01%
cpu : usr=15.71%, sys=57.64%, ctx=768050, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=122.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=1074230/w=1073921/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4196.3MB, aggrb=93780KB/s, minb=93780KB/s, maxb=93780KB/s, mint=45820msec, maxt=45820msec
WRITE: io=4195.5MB, aggrb=93752KB/s, minb=93752KB/s, maxb=93752KB/s, mint=45820msec, maxt=45820msec
Disk stats (read/write):
sdd: ios=1308507/1307441, merge=0/0, ticks=1449370/267238, in_queue=1715488, util=99.95%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdd-randrw-32-8K
===
mytest: (g=0): rw=randrw, bs=8K-8K/8K-8K/8K-8K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=3717: Sun Sep 15 10:07:41 2013
read : io=4194.3MB, bw=94427KB/s, iops=11802, runt= 45484msec
slat (usec): min=2, max=358, avg= 9.05, stdev= 4.56
clat (usec): min=529, max=33049, avg=2155.78, stdev=1886.04
lat (usec): min=538, max=33058, avg=2164.91, stdev=1885.93
clat percentiles (usec):
| 1.00th=[ 932], 5.00th=[ 1048], 10.00th=[ 1112], 20.00th=[ 1224],
| 30.00th=[ 1304], 40.00th=[ 1384], 50.00th=[ 1464], 60.00th=[ 1560],
| 70.00th=[ 1688], 80.00th=[ 1896], 90.00th=[ 6112], 95.00th=[ 7072],
| 99.00th=[ 7776], 99.50th=[ 7904], 99.90th=[ 8256], 99.95th=[ 8512],
| 99.99th=[32128]
bw (KB /s): min= 0, max=98832, per=98.88%, avg=93367.27, stdev=10051.29
write: io=4191.8MB, bw=94355KB/s, iops=11794, runt= 45484msec
slat (usec): min=4, max=2188, avg=11.57, stdev= 6.66
clat (usec): min=0, max=13721, avg=531.29, stdev=339.37
lat (usec): min=49, max=13726, avg=543.10, stdev=339.18
clat percentiles (usec):
| 1.00th=[ 48], 5.00th=[ 86], 10.00th=[ 137], 20.00th=[ 235],
| 30.00th=[ 326], 40.00th=[ 410], 50.00th=[ 490], 60.00th=[ 580],
| 70.00th=[ 676], 80.00th=[ 796], 90.00th=[ 964], 95.00th=[ 1096],
| 99.00th=[ 1368], 99.50th=[ 1480], 99.90th=[ 1848], 99.95th=[ 4512],
| 99.99th=[ 6048]
bw (KB /s): min=87504, max=98736, per=100.00%, avg=94357.39, stdev=1744.35
lat (usec) : 2=0.01%, 20=0.01%, 50=0.70%, 100=2.45%, 250=7.65%
lat (usec) : 500=14.71%, 750=12.65%, 1000=9.13%
lat (msec) : 2=44.04%, 4=1.98%, 10=6.67%, 20=0.01%, 50=0.01%
cpu : usr=13.50%, sys=34.16%, ctx=772390, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=122.1%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=536840/w=536449/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4194.3MB, aggrb=94426KB/s, minb=94426KB/s, maxb=94426KB/s, mint=45484msec, maxt=45484msec
WRITE: io=4191.8MB, aggrb=94355KB/s, minb=94355KB/s, maxb=94355KB/s, mint=45484msec, maxt=45484msec
Disk stats (read/write):
sdd: ios=655322/654695, merge=0/0, ticks=1412282/350217, in_queue=1761960, util=99.96%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdd-randrw-32-16K
===
mytest: (g=0): rw=randrw, bs=16K-16K/16K-16K/16K-16K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=3845: Sun Sep 15 10:09:31 2013
read : io=4093.2MB, bw=106828KB/s, iops=6676, runt= 39235msec
slat (usec): min=3, max=204, avg= 9.19, stdev= 2.69
clat (usec): min=469, max=34701, avg=3719.40, stdev=2389.78
lat (usec): min=480, max=34710, avg=3728.56, stdev=2389.73
clat percentiles (usec):
| 1.00th=[ 1624], 5.00th=[ 1800], 10.00th=[ 1928], 20.00th=[ 2096],
| 30.00th=[ 2224], 40.00th=[ 2384], 50.00th=[ 2544], 60.00th=[ 2736],
| 70.00th=[ 3152], 80.00th=[ 6752], 90.00th=[ 7712], 95.00th=[ 8160],
| 99.00th=[ 8768], 99.50th=[ 9024], 99.90th=[ 9792], 99.95th=[31104],
| 99.99th=[34048]
bw (KB /s): min=100640, max=111392, per=100.00%, avg=106849.88, stdev=2252.10
write: io=4081.5MB, bw=106522KB/s, iops=6657, runt= 39235msec
slat (usec): min=5, max=3237, avg=12.10, stdev= 9.21
clat (usec): min=27, max=11275, avg=1051.02, stdev=578.15
lat (usec): min=66, max=11292, avg=1063.15, stdev=577.91
clat percentiles (usec):
| 1.00th=[ 71], 5.00th=[ 155], 10.00th=[ 266], 20.00th=[ 498],
| 30.00th=[ 708], 40.00th=[ 900], 50.00th=[ 1064], 60.00th=[ 1224],
| 70.00th=[ 1368], 80.00th=[ 1544], 90.00th=[ 1752], 95.00th=[ 1944],
| 99.00th=[ 2320], 99.50th=[ 2512], 99.90th=[ 4576], 99.95th=[ 5280],
| 99.99th=[ 8032]
bw (KB /s): min= 1, max=111712, per=98.75%, avg=105192.03, stdev=12237.89
lat (usec) : 50=0.01%, 100=1.11%, 250=3.51%, 500=5.41%, 750=6.04%
lat (usec) : 1000=6.77%
lat (msec) : 2=32.29%, 4=31.50%, 10=13.34%, 20=0.01%, 50=0.03%
cpu : usr=12.27%, sys=20.91%, ctx=627995, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=125.3%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=261942/w=261203/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4093.2MB, aggrb=106828KB/s, minb=106828KB/s, maxb=106828KB/s, mint=39235msec, maxt=39235msec
WRITE: io=4081.5MB, aggrb=106522KB/s, minb=106522KB/s, maxb=106522KB/s, mint=39235msec, maxt=39235msec
Disk stats (read/write):
sdd: ios=327952/326901, merge=0/0, ticks=1223960/344879, in_queue=1568956, util=99.93%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdd-randrw-32-32K
===
mytest: (g=0): rw=randrw, bs=32K-32K/32K-32K/32K-32K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=3929: Sun Sep 15 10:11:14 2013
read : io=3946.5MB, bw=121695KB/s, iops=3802, runt= 33207msec
slat (usec): min=7, max=202, avg=11.48, stdev= 1.68
clat (usec): min=669, max=22813, avg=6660.38, stdev=2331.00
lat (usec): min=686, max=22822, avg=6671.95, stdev=2330.92
clat percentiles (usec):
| 1.00th=[ 3152], 5.00th=[ 3536], 10.00th=[ 3792], 20.00th=[ 4192],
| 30.00th=[ 4576], 40.00th=[ 5088], 50.00th=[ 7136], 60.00th=[ 7968],
| 70.00th=[ 8512], 80.00th=[ 9024], 90.00th=[ 9536], 95.00th=[10048],
| 99.00th=[10688], 99.50th=[11072], 99.90th=[11968], 99.95th=[12864],
| 99.99th=[19840]
bw (KB /s): min= 3, max=126912, per=98.56%, avg=119946.33, stdev=15095.48
write: io=3947.4MB, bw=121723KB/s, iops=3803, runt= 33207msec
slat (usec): min=11, max=9863, avg=17.05, stdev=29.01
clat (usec): min=87, max=14063, avg=1721.27, stdev=1063.16
lat (usec): min=109, max=14081, avg=1738.63, stdev=1063.33
clat percentiles (usec):
| 1.00th=[ 115], 5.00th=[ 221], 10.00th=[ 374], 20.00th=[ 668],
| 30.00th=[ 956], 40.00th=[ 1288], 50.00th=[ 1624], 60.00th=[ 1992],
| 70.00th=[ 2384], 80.00th=[ 2736], 90.00th=[ 3152], 95.00th=[ 3472],
| 99.00th=[ 4080], 99.50th=[ 4320], 99.90th=[ 5216], 99.95th=[ 6112],
| 99.99th=[11456]
bw (KB /s): min=114368, max=129600, per=100.00%, avg=121765.86, stdev=3383.93
lat (usec) : 100=0.19%, 250=2.86%, 500=4.04%, 750=4.26%, 1000=4.17%
lat (msec) : 2=14.51%, 4=26.81%, 10=40.67%, 20=2.51%, 50=0.01%
cpu : usr=9.59%, sys=14.80%, ctx=325150, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=129.7%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=126254/w=126314/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=3946.5MB, aggrb=121694KB/s, minb=121694KB/s, maxb=121694KB/s, mint=33207msec, maxt=33207msec
WRITE: io=3947.4MB, aggrb=121722KB/s, minb=121722KB/s, maxb=121722KB/s, mint=33207msec, maxt=33207msec
Disk stats (read/write):
sdd: ios=163890/163796, merge=0/0, ticks=1097624/281285, in_queue=1378931, util=99.91%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdd-randrw-32-64K
===
mytest: (g=0): rw=randrw, bs=64K-64K/64K-64K/64K-64K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=4058: Sun Sep 15 10:12:53 2013
read : io=3813.4MB, bw=140431KB/s, iops=2193, runt= 27806msec
slat (usec): min=9, max=416, avg=14.12, stdev= 4.07
clat (usec): min=762, max=49932, avg=12106.44, stdev=2569.58
lat (usec): min=789, max=49945, avg=12120.65, stdev=2569.44
clat percentiles (usec):
| 1.00th=[ 6816], 5.00th=[ 8160], 10.00th=[ 9152], 20.00th=[10304],
| 30.00th=[10944], 40.00th=[11584], 50.00th=[12096], 60.00th=[12608],
| 70.00th=[13120], 80.00th=[13760], 90.00th=[14784], 95.00th=[15808],
| 99.00th=[17792], 99.50th=[18816], 99.90th=[40704], 99.95th=[42752],
| 99.99th=[44288]
bw (KB /s): min=123136, max=155392, per=99.94%, avg=140350.53, stdev=7827.17
write: io=3814.7MB, bw=140459KB/s, iops=2194, runt= 27806msec
slat (usec): min=14, max=13398, avg=24.40, stdev=71.40
clat (usec): min=127, max=31492, avg=2433.40, stdev=1620.88
lat (usec): min=175, max=31514, avg=2458.39, stdev=1622.31
clat percentiles (usec):
| 1.00th=[ 179], 5.00th=[ 402], 10.00th=[ 588], 20.00th=[ 1020],
| 30.00th=[ 1432], 40.00th=[ 1768], 50.00th=[ 2192], 60.00th=[ 2608],
| 70.00th=[ 3120], 80.00th=[ 3696], 90.00th=[ 4576], 95.00th=[ 5280],
| 99.00th=[ 6496], 99.50th=[ 7008], 99.90th=[12352], 99.95th=[18304],
| 99.99th=[28800]
bw (KB /s): min= 6, max=157952, per=98.30%, avg=138074.05, stdev=20456.41
lat (usec) : 250=0.76%, 500=2.93%, 750=3.12%, 1000=2.52%
lat (msec) : 2=13.35%, 4=19.37%, 10=16.30%, 20=41.48%, 50=0.18%
cpu : usr=7.46%, sys=10.96%, ctx=163537, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=134.2%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=60988/w=61019/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=3813.4MB, aggrb=140431KB/s, minb=140431KB/s, maxb=140431KB/s, mint=27806msec, maxt=27806msec
WRITE: io=3814.7MB, aggrb=140458KB/s, minb=140458KB/s, maxb=140458KB/s, mint=27806msec, maxt=27806msec
Disk stats (read/write):
sdd: ios=81718/81659, merge=0/0, ticks=1003993/196997, in_queue=1201093, util=99.86%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdd-randrw-32-128K
===
mytest: (g=0): rw=randrw, bs=128K-128K/128K-128K/128K-128K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=4182: Sun Sep 15 10:14:27 2013
read : io=3658.3MB, bw=157008KB/s, iops=1225, runt= 23859msec
slat (usec): min=9, max=235, avg=22.53, stdev= 6.87
clat (usec): min=871, max=56887, avg=19745.02, stdev=3051.58
lat (usec): min=903, max=56910, avg=19766.72, stdev=3051.33
clat percentiles (usec):
| 1.00th=[13888], 5.00th=[15424], 10.00th=[16192], 20.00th=[17280],
| 30.00th=[18304], 40.00th=[19072], 50.00th=[19584], 60.00th=[20352],
| 70.00th=[21120], 80.00th=[21888], 90.00th=[23168], 95.00th=[23936],
| 99.00th=[25728], 99.50th=[28032], 99.90th=[48384], 99.95th=[51456],
| 99.99th=[54528]
bw (KB /s): min= 12, max=181248, per=97.81%, avg=153574.38, stdev=24132.61
write: io=3661.4MB, bw=157142KB/s, iops=1227, runt= 23859msec
slat (usec): min=22, max=21121, avg=43.06, stdev=153.75
clat (usec): min=280, max=43596, avg=6273.51, stdev=2596.33
lat (usec): min=317, max=43629, avg=6316.44, stdev=2598.71
clat percentiles (usec):
| 1.00th=[ 892], 5.00th=[ 2320], 10.00th=[ 3344], 20.00th=[ 4384],
| 30.00th=[ 5024], 40.00th=[ 5600], 50.00th=[ 6176], 60.00th=[ 6688],
| 70.00th=[ 7328], 80.00th=[ 8096], 90.00th=[ 9280], 95.00th=[10176],
| 99.00th=[12352], 99.50th=[13376], 99.90th=[32128], 99.95th=[34560],
| 99.99th=[42752]
bw (KB /s): min=136704, max=175616, per=100.00%, avg=157303.72, stdev=9558.12
lat (usec) : 500=0.09%, 750=0.26%, 1000=0.28%
lat (msec) : 2=1.40%, 4=5.67%, 10=39.67%, 20=29.76%, 50=22.90%
lat (msec) : 100=0.03%
cpu : usr=6.07%, sys=9.46%, ctx=81788, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=139.9%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=29242/w=29284/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=3658.3MB, aggrb=157007KB/s, minb=157007KB/s, maxb=157007KB/s, mint=23859msec, maxt=23859msec
WRITE: io=3661.4MB, aggrb=157141KB/s, minb=157141KB/s, maxb=157141KB/s, mint=23859msec, maxt=23859msec
Disk stats (read/write):
sdd: ios=40827/40734, merge=1/1, ticks=816815/256787, in_queue=1073797, util=99.86%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdd-randrw-32-256K
===
mytest: (g=0): rw=randrw, bs=256K-256K/256K-256K/256K-256K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=4602: Sun Sep 15 10:16:00 2013
read : io=3579.0MB, bw=162185KB/s, iops=632, runt= 22597msec
slat (usec): min=24, max=261, avg=38.73, stdev= 7.43
clat (msec): min=2, max=69, avg=33.03, stdev= 3.89
lat (msec): min=2, max=69, avg=33.07, stdev= 3.88
clat percentiles (usec):
| 1.00th=[26496], 5.00th=[28288], 10.00th=[29056], 20.00th=[30336],
| 30.00th=[31104], 40.00th=[32128], 50.00th=[33024], 60.00th=[33536],
| 70.00th=[34560], 80.00th=[35584], 90.00th=[36608], 95.00th=[37632],
| 99.00th=[42752], 99.50th=[57600], 99.90th=[63744], 99.95th=[65280],
| 99.99th=[68096]
bw (KB /s): min= 25, max=177664, per=97.73%, avg=158501.87, stdev=25429.82
write: io=3547.6MB, bw=160758KB/s, iops=627, runt= 22597msec
slat (usec): min=49, max=31670, avg=80.64, stdev=340.21
clat (msec): min=1, max=60, avg=17.52, stdev= 5.08
lat (msec): min=1, max=61, avg=17.60, stdev= 5.08
clat percentiles (usec):
| 1.00th=[ 6240], 5.00th=[ 9408], 10.00th=[11072], 20.00th=[13376],
| 30.00th=[15168], 40.00th=[16512], 50.00th=[17792], 60.00th=[18816],
| 70.00th=[20096], 80.00th=[21376], 90.00th=[22912], 95.00th=[24448],
| 99.00th=[28288], 99.50th=[40704], 99.90th=[51456], 99.95th=[54528],
| 99.99th=[60672]
bw (KB /s): min=131555, max=178688, per=100.00%, avg=161047.75, stdev=9874.70
lat (msec) : 2=0.04%, 4=0.11%, 10=3.06%, 20=31.77%, 50=64.68%
lat (msec) : 100=0.45%
cpu : usr=5.12%, sys=8.25%, ctx=40857, majf=0, minf=28
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=143.6%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=14289/w=14186/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=3579.0MB, aggrb=162185KB/s, minb=162185KB/s, maxb=162185KB/s, mint=22597msec, maxt=22597msec
WRITE: io=3547.6MB, aggrb=160757KB/s, minb=160757KB/s, maxb=160757KB/s, mint=22597msec, maxt=22597msec
Disk stats (read/write):
sdd: ios=20513/20328, merge=0/1, ticks=676637/356412, in_queue=1033357, util=99.82%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdd-randrw-32-512K
===
mytest: (g=0): rw=randrw, bs=512K-512K/512K-512K/512K-512K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=4723: Sun Sep 15 10:17:32 2013
read : io=3467.0MB, bw=167384KB/s, iops=326, runt= 21210msec
slat (usec): min=44, max=194, avg=68.77, stdev= 8.11
clat (msec): min=3, max=106, avg=55.70, stdev= 4.85
lat (msec): min=3, max=106, avg=55.77, stdev= 4.85
clat percentiles (msec):
| 1.00th=[ 49], 5.00th=[ 51], 10.00th=[ 52], 20.00th=[ 53],
| 30.00th=[ 55], 40.00th=[ 55], 50.00th=[ 56], 60.00th=[ 57],
| 70.00th=[ 58], 80.00th=[ 59], 90.00th=[ 60], 95.00th=[ 62],
| 99.00th=[ 79], 99.50th=[ 87], 99.90th=[ 98], 99.95th=[ 100],
| 99.99th=[ 106]
bw (KB /s): min=153600, max=181248, per=100.00%, avg=167666.29, stdev=7276.72
write: io=3511.0MB, bw=169508KB/s, iops=330, runt= 21210msec
slat (usec): min=97, max=389, avg=143.16, stdev=14.25
clat (msec): min=3, max=84, avg=41.52, stdev= 6.26
lat (msec): min=3, max=84, avg=41.67, stdev= 6.26
clat percentiles (usec):
| 1.00th=[25216], 5.00th=[31104], 10.00th=[34048], 20.00th=[37120],
| 30.00th=[39168], 40.00th=[40704], 50.00th=[42240], 60.00th=[43264],
| 70.00th=[44800], 80.00th=[45824], 90.00th=[47872], 95.00th=[49408],
| 99.00th=[58112], 99.50th=[69120], 99.90th=[79360], 99.95th=[81408],
| 99.99th=[84480]
bw (KB /s): min= 51, max=187392, per=97.79%, avg=165764.91, stdev=27855.76
lat (msec) : 4=0.02%, 10=0.04%, 20=0.12%, 50=50.20%, 100=49.81%
lat (msec) : 250=0.02%
cpu : usr=4.47%, sys=7.52%, ctx=20478, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.2%, 32=146.6%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=6918/w=7007/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=3467.0MB, aggrb=167383KB/s, minb=167383KB/s, maxb=167383KB/s, mint=21210msec, maxt=21210msec
WRITE: io=3511.0MB, aggrb=169507KB/s, minb=169507KB/s, maxb=169507KB/s, mint=21210msec, maxt=21210msec
Disk stats (read/write):
sdd: ios=10195/10145, merge=0/0, ticks=564779/420252, in_queue=985786, util=99.81%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdd-randrw-32-1024K
===
mytest: (g=0): rw=randrw, bs=1M-1M/1M-1M/1M-1M, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=5137: Sun Sep 15 10:19:02 2013
read : io=3440.0MB, bw=178060KB/s, iops=173, runt= 19783msec
slat (usec): min=79, max=316, avg=121.02, stdev= 9.04
clat (msec): min=6, max=312, avg=99.79, stdev=28.16
lat (msec): min=6, max=312, avg=99.91, stdev=28.16
clat percentiles (msec):
| 1.00th=[ 55], 5.00th=[ 60], 10.00th=[ 65], 20.00th=[ 74],
| 30.00th=[ 83], 40.00th=[ 90], 50.00th=[ 98], 60.00th=[ 106],
| 70.00th=[ 117], 80.00th=[ 125], 90.00th=[ 137], 95.00th=[ 145],
| 99.00th=[ 161], 99.50th=[ 174], 99.90th=[ 289], 99.95th=[ 306],
| 99.99th=[ 314]
bw (KB /s): min=152988, max=204800, per=100.00%, avg=178550.13, stdev=12628.54
write: io=3378.0MB, bw=174851KB/s, iops=169, runt= 19783msec
slat (usec): min=185, max=448, avg=265.93, stdev=18.38
clat (msec): min=5, max=322, avg=85.76, stdev=29.44
lat (msec): min=6, max=323, avg=86.03, stdev=29.44
clat percentiles (msec):
| 1.00th=[ 33], 5.00th=[ 45], 10.00th=[ 50], 20.00th=[ 60],
| 30.00th=[ 68], 40.00th=[ 76], 50.00th=[ 85], 60.00th=[ 93],
| 70.00th=[ 101], 80.00th=[ 112], 90.00th=[ 124], 95.00th=[ 133],
| 99.00th=[ 157], 99.50th=[ 174], 99.90th=[ 285], 99.95th=[ 306],
| 99.99th=[ 322]
bw (KB /s): min= 102, max=200744, per=97.23%, avg=170002.08, stdev=32973.27
lat (msec) : 10=0.06%, 20=0.16%, 50=4.97%, 100=55.62%, 250=39.49%
lat (msec) : 500=0.16%
cpu : usr=4.42%, sys=6.73%, ctx=10254, majf=0, minf=28
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.2%, 16=0.5%, 32=150.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=3425/w=3362/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=3440.0MB, aggrb=178059KB/s, minb=178059KB/s, maxb=178059KB/s, mint=19783msec, maxt=19783msec
WRITE: io=3378.0MB, aggrb=174850KB/s, minb=174850KB/s, maxb=174850KB/s, mint=19783msec, maxt=19783msec
Disk stats (read/write):
sdd: ios=10420/10040, merge=0/0, ticks=1018538/844254, in_queue=1866813, util=99.77%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sde-randrw-32-512B
===
mytest: (g=0): rw=randrw, bs=512-512/512-512/512-512, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=5265: Sun Sep 15 10:35:38 2013
read : io=5063.5MB, bw=5598.7KB/s, iops=11197, runt=926107msec
slat (usec): min=3, max=359, avg= 6.16, stdev= 4.06
clat (usec): min=150, max=57961, avg=1279.14, stdev=2034.85
lat (usec): min=164, max=57972, avg=1285.46, stdev=2034.86
clat percentiles (usec):
| 1.00th=[ 454], 5.00th=[ 556], 10.00th=[ 620], 20.00th=[ 700],
| 30.00th=[ 828], 40.00th=[ 1128], 50.00th=[ 1192], 60.00th=[ 1240],
| 70.00th=[ 1288], 80.00th=[ 1544], 90.00th=[ 1832], 95.00th=[ 1944],
| 99.00th=[ 2480], 99.50th=[ 2608], 99.90th=[40192], 99.95th=[41216],
| 99.99th=[41728]
bw (KB /s): min= 478, max= 6635, per=100.00%, avg=5607.75, stdev=1414.02
write: io=5062.5MB, bw=5597.6KB/s, iops=11194, runt=926107msec
slat (usec): min=3, max=869, avg= 7.26, stdev= 4.82
clat (usec): min=165, max=69860, avg=1562.72, stdev=2141.48
lat (usec): min=172, max=69865, avg=1570.15, stdev=2141.38
clat percentiles (usec):
| 1.00th=[ 494], 5.00th=[ 628], 10.00th=[ 684], 20.00th=[ 788],
| 30.00th=[ 1112], 40.00th=[ 1192], 50.00th=[ 1256], 60.00th=[ 1400],
| 70.00th=[ 1784], 80.00th=[ 2064], 90.00th=[ 2480], 95.00th=[ 2768],
| 99.00th=[ 3312], 99.50th=[ 3664], 99.90th=[40192], 99.95th=[41216],
| 99.99th=[42752]
bw (KB /s): min= 0, max= 6557, per=100.00%, avg=5603.59, stdev=1417.39
lat (usec) : 250=0.09%, 500=1.64%, 750=19.98%, 1000=7.80%
lat (msec) : 2=57.83%, 4=12.34%, 10=0.03%, 20=0.02%, 50=0.27%
lat (msec) : 100=0.01%
cpu : usr=5.27%, sys=19.35%, ctx=1345504, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=101.1%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=10369929/w=10367757/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=5063.5MB, aggrb=5598KB/s, minb=5598KB/s, maxb=5598KB/s, mint=926107msec, maxt=926107msec
WRITE: io=5062.5MB, aggrb=5597KB/s, minb=5597KB/s, maxb=5597KB/s, mint=926107msec, maxt=926107msec
Disk stats (read/write):
sde: ios=10484230/10481925, merge=0/0, ticks=12191285/15239438, in_queue=27423929, util=99.85%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sde-randrw-32-1K
===
mytest: (g=0): rw=randrw, bs=1K-1K/1K-1K/1K-1K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=6504: Sun Sep 15 10:45:34 2013
read : io=5033.8MB, bw=9808.3KB/s, iops=9808, runt=525532msec
slat (usec): min=2, max=409, avg= 7.46, stdev= 4.64
clat (usec): min=168, max=79946, avg=1404.84, stdev=2024.09
lat (usec): min=178, max=79953, avg=1412.47, stdev=2024.11
clat percentiles (usec):
| 1.00th=[ 466], 5.00th=[ 628], 10.00th=[ 708], 20.00th=[ 804],
| 30.00th=[ 884], 40.00th=[ 980], 50.00th=[ 1128], 60.00th=[ 1320],
| 70.00th=[ 1480], 80.00th=[ 1784], 90.00th=[ 2192], 95.00th=[ 2608],
| 99.00th=[ 3440], 99.50th=[ 3792], 99.90th=[39168], 99.95th=[39680],
| 99.99th=[41216]
bw (KB /s): min= 0, max=11328, per=100.00%, avg=9820.25, stdev=2385.85
write: io=5032.2MB, bw=9805.3KB/s, iops=9805, runt=525532msec
slat (usec): min=2, max=564, avg= 8.85, stdev= 5.37
clat (usec): min=176, max=58305, avg=1838.53, stdev=2235.12
lat (usec): min=194, max=58319, avg=1847.59, stdev=2235.01
clat percentiles (usec):
| 1.00th=[ 644], 5.00th=[ 788], 10.00th=[ 876], 20.00th=[ 988],
| 30.00th=[ 1112], 40.00th=[ 1336], 50.00th=[ 1528], 60.00th=[ 1736],
| 70.00th=[ 2024], 80.00th=[ 2352], 90.00th=[ 2928], 95.00th=[ 3440],
| 99.00th=[ 4576], 99.50th=[ 5216], 99.90th=[39680], 99.95th=[41216],
| 99.99th=[42240]
bw (KB /s): min= 944, max=11304, per=100.00%, avg=9826.54, stdev=2367.27
lat (usec) : 250=0.01%, 500=0.81%, 750=7.74%, 1000=22.90%
lat (msec) : 2=46.10%, 4=21.16%, 10=1.00%, 20=0.01%, 50=0.28%
lat (msec) : 100=0.01%
cpu : usr=5.86%, sys=20.08%, ctx=1918507, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=101.7%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=5154514/w=5152953/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=5033.8MB, aggrb=9808KB/s, minb=9808KB/s, maxb=9808KB/s, mint=525532msec, maxt=525532msec
WRITE: io=5032.2MB, aggrb=9805KB/s, minb=9805KB/s, maxb=9805KB/s, mint=525532msec, maxt=525532msec
Disk stats (read/write):
sde: ios=5243216/5241481, merge=0/0, ticks=7215241/9525900, in_queue=16737345, util=100.00%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sde-randrw-32-2K
===
mytest: (g=0): rw=randrw, bs=2K-2K/2K-2K/2K-2K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=7073: Sun Sep 15 10:51:59 2013
read : io=4958.4MB, bw=16148KB/s, iops=8073, runt=314431msec
slat (usec): min=3, max=256, avg= 8.03, stdev= 3.45
clat (usec): min=135, max=79274, avg=1295.96, stdev=1754.45
lat (usec): min=164, max=79280, avg=1304.21, stdev=1754.45
clat percentiles (usec):
| 1.00th=[ 318], 5.00th=[ 450], 10.00th=[ 540], 20.00th=[ 668],
| 30.00th=[ 764], 40.00th=[ 868], 50.00th=[ 972], 60.00th=[ 1096],
| 70.00th=[ 1320], 80.00th=[ 1736], 90.00th=[ 2480], 95.00th=[ 2960],
| 99.00th=[ 3536], 99.50th=[ 3760], 99.90th=[38144], 99.95th=[39168],
| 99.99th=[40704]
bw (KB /s): min= 1566, max=18340, per=100.00%, avg=16169.39, stdev=3338.98
write: io=4955.8MB, bw=16139KB/s, iops=8069, runt=314431msec
slat (usec): min=4, max=38172, avg= 9.52, stdev=23.90
clat (usec): min=219, max=75958, avg=2647.54, stdev=2374.78
lat (usec): min=228, max=75968, avg=2657.31, stdev=2374.88
clat percentiles (usec):
| 1.00th=[ 724], 5.00th=[ 1020], 10.00th=[ 1240], 20.00th=[ 1688],
| 30.00th=[ 2064], 40.00th=[ 2320], 50.00th=[ 2576], 60.00th=[ 2800],
| 70.00th=[ 3024], 80.00th=[ 3312], 90.00th=[ 3664], 95.00th=[ 3952],
| 99.00th=[ 4512], 99.50th=[ 4960], 99.90th=[41216], 99.95th=[41728],
| 99.99th=[42752]
bw (KB /s): min= 0, max=18492, per=99.97%, avg=16134.24, stdev=3396.66
lat (usec) : 250=0.10%, 500=3.75%, 750=10.84%, 1000=13.85%
lat (msec) : 2=27.65%, 4=41.43%, 10=2.10%, 20=0.01%, 50=0.26%
lat (msec) : 100=0.01%
cpu : usr=5.71%, sys=18.19%, ctx=1229699, majf=0, minf=28
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=103.3%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=2538661/w=2537313/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4958.4MB, aggrb=16147KB/s, minb=16147KB/s, maxb=16147KB/s, mint=314431msec, maxt=314431msec
WRITE: io=4955.8MB, aggrb=16139KB/s, minb=16139KB/s, maxb=16139KB/s, mint=314431msec, maxt=314431msec
Disk stats (read/write):
sde: ios=2620710/2619116, merge=0/0, ticks=3341399/6884994, in_queue=10224765, util=100.00%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sde-randrw-32-4K
===
mytest: (g=0): rw=randrw, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=7813: Sun Sep 15 10:56:20 2013
read : io=4863.8MB, bw=26075KB/s, iops=6518, runt=191008msec
slat (usec): min=4, max=251, avg= 8.50, stdev= 2.50
clat (usec): min=157, max=40916, avg=1580.79, stdev=1406.80
lat (usec): min=167, max=40931, avg=1589.55, stdev=1406.81
clat percentiles (usec):
| 1.00th=[ 334], 5.00th=[ 470], 10.00th=[ 580], 20.00th=[ 796],
| 30.00th=[ 988], 40.00th=[ 1144], 50.00th=[ 1304], 60.00th=[ 1544],
| 70.00th=[ 1896], 80.00th=[ 2416], 90.00th=[ 2896], 95.00th=[ 3152],
| 99.00th=[ 3568], 99.50th=[ 3696], 99.90th=[28544], 99.95th=[36608],
| 99.99th=[39168]
bw (KB /s): min= 0, max=28448, per=99.88%, avg=26042.65, stdev=4484.90
write: io=4860.1MB, bw=26060KB/s, iops=6514, runt=191008msec
slat (usec): min=5, max=759, avg=10.54, stdev= 3.32
clat (usec): min=322, max=81057, avg=3306.88, stdev=2357.68
lat (usec): min=333, max=81065, avg=3317.73, stdev=2357.76
clat percentiles (usec):
| 1.00th=[ 1032], 5.00th=[ 1352], 10.00th=[ 1880], 20.00th=[ 2480],
| 30.00th=[ 2800], 40.00th=[ 3088], 50.00th=[ 3312], 60.00th=[ 3536],
| 70.00th=[ 3728], 80.00th=[ 3952], 90.00th=[ 4256], 95.00th=[ 4448],
| 99.00th=[ 5152], 99.50th=[ 5664], 99.90th=[40704], 99.95th=[41728],
| 99.99th=[75264]
bw (KB /s): min= 2528, max=28472, per=100.00%, avg=26095.35, stdev=4291.39
lat (usec) : 250=0.07%, 500=3.08%, 750=5.78%, 1000=6.86%
lat (msec) : 2=25.99%, 4=49.36%, 10=8.64%, 20=0.02%, 50=0.20%
lat (msec) : 100=0.01%
cpu : usr=6.18%, sys=16.14%, ctx=975068, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=105.3%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=1245103/w=1244387/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4863.8MB, aggrb=26074KB/s, minb=26074KB/s, maxb=26074KB/s, mint=191008msec, maxt=191008msec
WRITE: io=4860.1MB, aggrb=26059KB/s, minb=26059KB/s, maxb=26059KB/s, mint=191008msec, maxt=191008msec
Disk stats (read/write):
sde: ios=1310296/1309206, merge=0/0, ticks=2058217/4310828, in_queue=6368221, util=100.00%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sde-randrw-32-8K
===
mytest: (g=0): rw=randrw, bs=8K-8K/8K-8K/8K-8K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=8055: Sun Sep 15 10:59:42 2013
read : io=4756.2MB, bw=37249KB/s, iops=4656, runt=130746msec
slat (usec): min=4, max=334, avg= 9.29, stdev= 2.81
clat (usec): min=221, max=43891, avg=1664.00, stdev=1170.50
lat (usec): min=232, max=43900, avg=1673.54, stdev=1170.46
clat percentiles (usec):
| 1.00th=[ 410], 5.00th=[ 556], 10.00th=[ 676], 20.00th=[ 860],
| 30.00th=[ 1004], 40.00th=[ 1176], 50.00th=[ 1432], 60.00th=[ 1736],
| 70.00th=[ 2128], 80.00th=[ 2576], 90.00th=[ 2960], 95.00th=[ 3184],
| 99.00th=[ 3536], 99.50th=[ 3664], 99.90th=[ 4448], 99.95th=[20864],
| 99.99th=[40192]
bw (KB /s): min= 0, max=41200, per=99.69%, avg=37134.62, stdev=5429.59
write: io=4753.3MB, bw=37226KB/s, iops=4653, runt=130746msec
slat (usec): min=7, max=973, avg=12.17, stdev= 3.98
clat (usec): min=187, max=122906, avg=5186.02, stdev=3179.46
lat (usec): min=198, max=122916, avg=5198.55, stdev=3179.62
clat percentiles (usec):
| 1.00th=[ 2640], 5.00th=[ 3120], 10.00th=[ 3472], 20.00th=[ 3888],
| 30.00th=[ 4192], 40.00th=[ 4448], 50.00th=[ 4768], 60.00th=[ 5216],
| 70.00th=[ 5856], 80.00th=[ 6432], 90.00th=[ 7072], 95.00th=[ 7456],
| 99.00th=[ 8096], 99.50th=[ 8768], 99.90th=[76288], 99.95th=[81408],
| 99.99th=[89600]
bw (KB /s): min= 3219, max=40800, per=100.00%, avg=37250.14, stdev=4889.07
lat (usec) : 250=0.01%, 500=1.53%, 750=5.23%, 1000=8.01%
lat (msec) : 2=18.92%, 4=28.09%, 10=38.04%, 20=0.06%, 50=0.06%
lat (msec) : 100=0.06%, 250=0.01%
cpu : usr=5.89%, sys=12.84%, ctx=749637, majf=0, minf=28
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=107.7%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=608758/w=608368/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4756.2MB, aggrb=37249KB/s, minb=37249KB/s, maxb=37249KB/s, mint=130746msec, maxt=130746msec
WRITE: io=4753.3MB, aggrb=37225KB/s, minb=37225KB/s, maxb=37225KB/s, mint=130746msec, maxt=130746msec
Disk stats (read/write):
sde: ios=654799/654190, merge=0/0, ticks=1087475/3386675, in_queue=4473799, util=100.00%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sde-randrw-32-16K
===
mytest: (g=0): rw=randrw, bs=16K-16K/16K-16K/16K-16K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=8243: Sun Sep 15 11:02:34 2013
read : io=4670.9MB, bw=46969KB/s, iops=2935, runt=101832msec
slat (usec): min=6, max=208, avg=10.56, stdev= 1.63
clat (usec): min=297, max=58462, avg=1799.12, stdev=1076.08
lat (usec): min=308, max=58474, avg=1809.95, stdev=1076.03
clat percentiles (usec):
| 1.00th=[ 548], 5.00th=[ 708], 10.00th=[ 820], 20.00th=[ 980],
| 30.00th=[ 1144], 40.00th=[ 1368], 50.00th=[ 1624], 60.00th=[ 1896],
| 70.00th=[ 2288], 80.00th=[ 2672], 90.00th=[ 3024], 95.00th=[ 3248],
| 99.00th=[ 3600], 99.50th=[ 3728], 99.90th=[ 5024], 99.95th=[ 9920],
| 99.99th=[37632]
bw (KB /s): min= 4218, max=52416, per=100.00%, avg=46995.55, stdev=4482.59
write: io=4656.4MB, bw=46823KB/s, iops=2926, runt=101832msec
slat (usec): min=8, max=1522, avg=14.21, stdev= 3.84
clat (usec): min=498, max=171799, avg=9100.42, stdev=3966.64
lat (usec): min=514, max=171815, avg=9115.00, stdev=3966.71
clat percentiles (msec):
| 1.00th=[ 6], 5.00th=[ 6], 10.00th=[ 7], 20.00th=[ 8],
| 30.00th=[ 8], 40.00th=[ 9], 50.00th=[ 9], 60.00th=[ 10],
| 70.00th=[ 11], 80.00th=[ 11], 90.00th=[ 12], 95.00th=[ 13],
| 99.00th=[ 15], 99.50th=[ 16], 99.90th=[ 25], 99.95th=[ 119],
| 99.99th=[ 151]
bw (KB /s): min= 1, max=51616, per=99.57%, avg=46620.48, stdev=5551.83
lat (usec) : 500=0.25%, 750=3.01%, 1000=7.49%
lat (msec) : 2=20.85%, 4=18.46%, 10=34.18%, 20=15.68%, 50=0.03%
lat (msec) : 100=0.01%, 250=0.04%
cpu : usr=5.12%, sys=9.13%, ctx=649101, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=109.8%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=298930/w=297979/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4670.9MB, aggrb=46968KB/s, minb=46968KB/s, maxb=46968KB/s, mint=101832msec, maxt=101832msec
WRITE: io=4656.4MB, aggrb=46823KB/s, minb=46823KB/s, maxb=46823KB/s, mint=101832msec, maxt=101832msec
Disk stats (read/write):
sde: ios=328100/327032, merge=0/0, ticks=593111/2978197, in_queue=3571377, util=100.00%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sde-randrw-32-32K
===
mytest: (g=0): rw=randrw, bs=32K-32K/32K-32K/32K-32K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=8444: Sun Sep 15 11:05:08 2013
read : io=4579.3MB, bw=56016KB/s, iops=1750, runt= 83710msec
slat (usec): min=7, max=219, avg=12.27, stdev= 2.27
clat (usec): min=439, max=87166, avg=2126.10, stdev=1105.44
lat (usec): min=452, max=87185, avg=2138.67, stdev=1105.41
clat percentiles (usec):
| 1.00th=[ 772], 5.00th=[ 940], 10.00th=[ 1064], 20.00th=[ 1304],
| 30.00th=[ 1544], 40.00th=[ 1768], 50.00th=[ 1992], 60.00th=[ 2256],
| 70.00th=[ 2608], 80.00th=[ 2928], 90.00th=[ 3280], 95.00th=[ 3504],
| 99.00th=[ 4016], 99.50th=[ 4512], 99.90th=[ 6048], 99.95th=[11072],
| 99.99th=[39680]
bw (KB /s): min= 3, max=61568, per=99.50%, avg=55737.99, stdev=6489.56
write: io=4577.4MB, bw=55994KB/s, iops=1749, runt= 83710msec
slat (usec): min=11, max=3213, avg=18.67, stdev= 9.44
clat (usec): min=450, max=247271, avg=16125.83, stdev=6252.80
lat (usec): min=467, max=247299, avg=16145.00, stdev=6253.04
clat percentiles (msec):
| 1.00th=[ 10], 5.00th=[ 12], 10.00th=[ 12], 20.00th=[ 13],
| 30.00th=[ 14], 40.00th=[ 15], 50.00th=[ 16], 60.00th=[ 17],
| 70.00th=[ 18], 80.00th=[ 20], 90.00th=[ 21], 95.00th=[ 22],
| 99.00th=[ 25], 99.50th=[ 26], 99.90th=[ 36], 99.95th=[ 194],
| 99.99th=[ 231]
bw (KB /s): min= 4834, max=61248, per=100.00%, avg=55996.43, stdev=4736.47
lat (usec) : 500=0.01%, 750=0.39%, 1000=3.19%
lat (msec) : 2=21.62%, 4=24.34%, 10=1.13%, 20=42.61%, 50=6.68%
lat (msec) : 100=0.01%, 250=0.04%
cpu : usr=3.92%, sys=6.32%, ctx=327012, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=111.8%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=146532/w=146448/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4579.3MB, aggrb=56016KB/s, minb=56016KB/s, maxb=56016KB/s, mint=83710msec, maxt=83710msec
WRITE: io=4577.4MB, aggrb=55993KB/s, minb=55993KB/s, maxb=55993KB/s, mint=83710msec, maxt=83710msec
Disk stats (read/write):
sde: ios=163613/163480, merge=0/0, ticks=348611/2639502, in_queue=2988325, util=100.00%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sde-randrw-32-64K
===
mytest: (g=0): rw=randrw, bs=64K-64K/64K-64K/64K-64K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=8626: Sun Sep 15 11:07:30 2013
read : io=4498.8MB, bw=64127KB/s, iops=1001, runt= 71836msec
slat (usec): min=10, max=87, avg=15.17, stdev= 2.64
clat (usec): min=781, max=24681, avg=2668.65, stdev=951.46
lat (usec): min=797, max=24699, avg=2684.14, stdev=951.28
clat percentiles (usec):
| 1.00th=[ 1176], 5.00th=[ 1432], 10.00th=[ 1624], 20.00th=[ 1896],
| 30.00th=[ 2128], 40.00th=[ 2320], 50.00th=[ 2544], 60.00th=[ 2800],
| 70.00th=[ 3088], 80.00th=[ 3408], 90.00th=[ 3760], 95.00th=[ 4080],
| 99.00th=[ 5472], 99.50th=[ 6048], 99.90th=[ 7200], 99.95th=[13120],
| 99.99th=[21120]
bw (KB /s): min= 6, max=72047, per=99.37%, avg=63725.23, stdev=6002.29
write: io=4505.0MB, bw=64217KB/s, iops=1002, runt= 71836msec
slat (usec): min=14, max=5233, avg=25.85, stdev=24.53
clat (usec): min=898, max=51825, avg=29185.84, stdev=5191.96
lat (usec): min=921, max=51847, avg=29212.26, stdev=5191.75
clat percentiles (usec):
| 1.00th=[18560], 5.00th=[21632], 10.00th=[22912], 20.00th=[24704],
| 30.00th=[26240], 40.00th=[27520], 50.00th=[28800], 60.00th=[30080],
| 70.00th=[31616], 80.00th=[33536], 90.00th=[36096], 95.00th=[38144],
| 99.00th=[42240], 99.50th=[43776], 99.90th=[46336], 99.95th=[47872],
| 99.99th=[49920]
bw (KB /s): min=55808, max=71424, per=99.99%, avg=64209.57, stdev=2529.39
lat (usec) : 1000=0.08%
lat (msec) : 2=12.18%, 4=34.73%, 10=3.03%, 20=1.04%, 50=48.94%
lat (msec) : 100=0.01%
cpu : usr=2.80%, sys=4.49%, ctx=163521, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=113.7%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=71978/w=72050/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4498.8MB, aggrb=64127KB/s, minb=64127KB/s, maxb=64127KB/s, mint=71836msec, maxt=71836msec
WRITE: io=4505.0MB, aggrb=64217KB/s, minb=64217KB/s, maxb=64217KB/s, mint=71836msec, maxt=71836msec
Disk stats (read/write):
sde: ios=81942/81805, merge=0/0, ticks=218401/2394064, in_queue=2612761, util=100.00%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sde-randrw-32-128K
===
mytest: (g=0): rw=randrw, bs=128K-128K/128K-128K/128K-128K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=9099: Sun Sep 15 11:09:36 2013
read : io=4374.0MB, bw=81387KB/s, iops=635, runt= 55033msec
slat (usec): min=14, max=214, avg=23.27, stdev= 3.80
clat (msec): min=1, max=37, avg= 3.38, stdev= 1.23
lat (msec): min=1, max=37, avg= 3.40, stdev= 1.23
clat percentiles (usec):
| 1.00th=[ 1560], 5.00th=[ 2024], 10.00th=[ 2192], 20.00th=[ 2512],
| 30.00th=[ 2736], 40.00th=[ 2960], 50.00th=[ 3216], 60.00th=[ 3472],
| 70.00th=[ 3792], 80.00th=[ 4128], 90.00th=[ 4704], 95.00th=[ 5216],
| 99.00th=[ 6688], 99.50th=[ 7136], 99.90th=[11072], 99.95th=[25216],
| 99.99th=[35072]
bw (KB /s): min= 12, max=96256, per=99.25%, avg=80780.21, stdev=9852.39
write: io=4376.8MB, bw=81438KB/s, iops=635, runt= 55033msec
slat (usec): min=24, max=10434, avg=43.41, stdev=54.95
clat (msec): min=2, max=76, avg=46.87, stdev= 7.55
lat (msec): min=2, max=76, avg=46.91, stdev= 7.55
clat percentiles (usec):
| 1.00th=[30080], 5.00th=[35072], 10.00th=[37632], 20.00th=[40704],
| 30.00th=[42752], 40.00th=[44800], 50.00th=[46848], 60.00th=[48384],
| 70.00th=[50944], 80.00th=[52992], 90.00th=[56576], 95.00th=[59136],
| 99.00th=[65280], 99.50th=[67072], 99.90th=[71168], 99.95th=[72192],
| 99.99th=[75264]
bw (KB /s): min=65280, max=92928, per=99.90%, avg=81354.38, stdev=6414.09
lat (msec) : 2=2.33%, 4=35.68%, 10=11.96%, 20=0.13%, 50=33.29%
lat (msec) : 100=16.65%
cpu : usr=2.57%, sys=4.28%, ctx=81804, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=117.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=34991/w=34984/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4374.0MB, aggrb=81387KB/s, minb=81387KB/s, maxb=81387KB/s, mint=55033msec, maxt=55033msec
WRITE: io=4376.8MB, aggrb=81438KB/s, minb=81438KB/s, maxb=81438KB/s, mint=55033msec, maxt=55033msec
Disk stats (read/write):
sde: ios=41061/40906, merge=0/0, ticks=138711/1937174, in_queue=2076384, util=99.97%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sde-randrw-32-256K
===
mytest: (g=0): rw=randrw, bs=256K-256K/256K-256K/256K-256K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=9229: Sun Sep 15 11:11:30 2013
read : io=4193.0MB, bw=98989KB/s, iops=386, runt= 43375msec
slat (usec): min=24, max=118, avg=38.84, stdev= 4.83
clat (msec): min=1, max=60, avg= 5.20, stdev= 2.32
lat (msec): min=1, max=60, avg= 5.24, stdev= 2.32
clat percentiles (usec):
| 1.00th=[ 2480], 5.00th=[ 3024], 10.00th=[ 3504], 20.00th=[ 3888],
| 30.00th=[ 4384], 40.00th=[ 4768], 50.00th=[ 5024], 60.00th=[ 5280],
| 70.00th=[ 5664], 80.00th=[ 6176], 90.00th=[ 6880], 95.00th=[ 7584],
| 99.00th=[ 9152], 99.50th=[10688], 99.90th=[41216], 99.95th=[52992],
| 99.99th=[60160]
bw (KB /s): min= 25, max=114231, per=99.04%, avg=98034.13, stdev=12579.64
write: io=4186.0MB, bw=98823KB/s, iops=385, runt= 43375msec
slat (usec): min=43, max=17763, avg=78.28, stdev=124.91
clat (msec): min=3, max=118, avg=77.61, stdev=10.73
lat (msec): min=3, max=118, avg=77.69, stdev=10.73
clat percentiles (msec):
| 1.00th=[ 53], 5.00th=[ 61], 10.00th=[ 65], 20.00th=[ 70],
| 30.00th=[ 73], 40.00th=[ 76], 50.00th=[ 78], 60.00th=[ 81],
| 70.00th=[ 83], 80.00th=[ 86], 90.00th=[ 91], 95.00th=[ 96],
| 99.00th=[ 105], 99.50th=[ 108], 99.90th=[ 114], 99.95th=[ 116],
| 99.99th=[ 119]
bw (KB /s): min=81408, max=117248, per=99.87%, avg=98694.33, stdev=6971.23
lat (msec) : 2=0.04%, 4=11.37%, 10=38.37%, 20=0.22%, 50=0.38%
lat (msec) : 100=48.45%, 250=1.26%
cpu : usr=2.58%, sys=4.33%, ctx=40853, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=122.1%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=16770/w=16715/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4193.0MB, aggrb=98988KB/s, minb=98988KB/s, maxb=98988KB/s, mint=43375msec, maxt=43375msec
WRITE: io=4186.0MB, aggrb=98823KB/s, minb=98823KB/s, maxb=98823KB/s, mint=43375msec, maxt=43375msec
Disk stats (read/write):
sde: ios=20624/20343, merge=0/0, ticks=105616/1592840, in_queue=1699703, util=99.93%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sde-randrw-32-512K
===
mytest: (g=0): rw=randrw, bs=512K-512K/512K-512K/512K-512K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=9359: Sun Sep 15 11:13:17 2013
read : io=4022.6MB, bw=110870KB/s, iops=216, runt= 37152msec
slat (usec): min=41, max=283, avg=68.90, stdev= 6.20
clat (msec): min=3, max=107, avg= 9.55, stdev= 5.10
lat (msec): min=3, max=107, avg= 9.62, stdev= 5.10
clat percentiles (msec):
| 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 7], 20.00th=[ 7],
| 30.00th=[ 8], 40.00th=[ 9], 50.00th=[ 10], 60.00th=[ 10],
| 70.00th=[ 11], 80.00th=[ 12], 90.00th=[ 13], 95.00th=[ 14],
| 99.00th=[ 18], 99.50th=[ 39], 99.90th=[ 87], 99.95th=[ 98],
| 99.99th=[ 108]
bw (KB /s): min=81756, max=137216, per=100.00%, avg=111537.77, stdev=8668.33
write: io=4098.0MB, bw=112951KB/s, iops=219, runt= 37152msec
slat (usec): min=89, max=26534, avg=146.12, stdev=282.37
clat (msec): min=6, max=204, avg=135.70, stdev=18.19
lat (msec): min=6, max=205, avg=135.84, stdev=18.18
clat percentiles (msec):
| 1.00th=[ 90], 5.00th=[ 112], 10.00th=[ 117], 20.00th=[ 123],
| 30.00th=[ 127], 40.00th=[ 131], 50.00th=[ 135], 60.00th=[ 139],
| 70.00th=[ 143], 80.00th=[ 149], 90.00th=[ 159], 95.00th=[ 167],
| 99.00th=[ 184], 99.50th=[ 190], 99.90th=[ 198], 99.95th=[ 200],
| 99.99th=[ 206]
bw (KB /s): min= 51, max=136942, per=98.21%, avg=110923.26, stdev=15912.57
lat (msec) : 4=0.03%, 10=31.86%, 20=17.32%, 50=0.33%, 100=1.06%
lat (msec) : 250=49.59%
cpu : usr=2.53%, sys=4.27%, ctx=20426, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.2%, 32=126.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=8045/w=8165/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4022.6MB, aggrb=110869KB/s, minb=110869KB/s, maxb=110869KB/s, mint=37152msec, maxt=37152msec
WRITE: io=4098.0MB, aggrb=112950KB/s, minb=112950KB/s, maxb=112950KB/s, mint=37152msec, maxt=37152msec
Disk stats (read/write):
sde: ios=10346/10187, merge=0/0, ticks=95768/1401573, in_queue=1499096, util=99.90%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sde-randrw-32-1024K
===
mytest: (g=0): rw=randrw, bs=1M-1M/1M-1M/1M-1M, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=9487: Sun Sep 15 11:15:03 2013
read : io=4035.0MB, bw=116917KB/s, iops=113, runt= 35340msec
slat (usec): min=72, max=205, avg=120.95, stdev= 8.84
clat (msec): min=6, max=352, avg=79.65, stdev=44.75
lat (msec): min=6, max=352, avg=79.77, stdev=44.75
clat percentiles (msec):
| 1.00th=[ 14], 5.00th=[ 20], 10.00th=[ 26], 20.00th=[ 39],
| 30.00th=[ 52], 40.00th=[ 64], 50.00th=[ 76], 60.00th=[ 88],
| 70.00th=[ 100], 80.00th=[ 116], 90.00th=[ 135], 95.00th=[ 153],
| 99.00th=[ 221], 99.50th=[ 245], 99.90th=[ 297], 99.95th=[ 318],
| 99.99th=[ 355]
bw (KB /s): min=83903, max=146868, per=100.00%, avg=117409.81, stdev=12633.22
write: io=3972.0MB, bw=115091KB/s, iops=111, runt= 35340msec
slat (usec): min=179, max=78248, avg=311.44, stdev=1814.29
clat (msec): min=21, max=490, avg=203.88, stdev=50.77
lat (msec): min=22, max=490, avg=204.20, stdev=50.75
clat percentiles (msec):
| 1.00th=[ 98], 5.00th=[ 131], 10.00th=[ 143], 20.00th=[ 161],
| 30.00th=[ 176], 40.00th=[ 188], 50.00th=[ 200], 60.00th=[ 215],
| 70.00th=[ 229], 80.00th=[ 247], 90.00th=[ 269], 95.00th=[ 289],
| 99.00th=[ 343], 99.50th=[ 363], 99.90th=[ 437], 99.95th=[ 453],
| 99.99th=[ 490]
bw (KB /s): min= 102, max=143073, per=98.30%, avg=113137.16, stdev=17708.24
lat (msec) : 10=0.03%, 20=2.80%, 50=11.97%, 100=21.18%, 250=55.28%
lat (msec) : 500=9.14%
cpu : usr=2.57%, sys=3.70%, ctx=10215, majf=0, minf=28
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.2%, 16=0.4%, 32=127.6%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=4026/w=3950/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4035.0MB, aggrb=116916KB/s, minb=116916KB/s, maxb=116916KB/s, mint=35340msec, maxt=35340msec
WRITE: io=3972.0MB, aggrb=115091KB/s, minb=115091KB/s, maxb=115091KB/s, mint=35340msec, maxt=35340msec
Disk stats (read/write):
sde: ios=10504/10035, merge=0/0, ticks=810954/2028119, in_queue=2844925, util=99.87%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdf-randrw-32-512B
===
mytest: (g=0): rw=randrw, bs=512-512/512-512/512-512, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=9960: Sun Sep 15 11:24:46 2013
read : io=4980.8MB, bw=9957.8KB/s, iops=19915, runt=512188msec
slat (usec): min=2, max=389, avg= 7.66, stdev= 4.95
clat (usec): min=1, max=20035, avg=505.05, stdev=438.00
lat (usec): min=39, max=20041, avg=512.88, stdev=437.96
clat percentiles (usec):
| 1.00th=[ 122], 5.00th=[ 185], 10.00th=[ 219], 20.00th=[ 274],
| 30.00th=[ 318], 40.00th=[ 362], 50.00th=[ 406], 60.00th=[ 454],
| 70.00th=[ 516], 80.00th=[ 596], 90.00th=[ 748], 95.00th=[ 1048],
| 99.00th=[ 2640], 99.50th=[ 2832], 99.90th=[ 3280], 99.95th=[ 3472],
| 99.99th=[ 4128]
bw (KB /s): min= 8740, max=13494, per=99.98%, avg=9954.65, stdev=618.09
write: io=4979.8MB, bw=9955.1KB/s, iops=19911, runt=512188msec
slat (usec): min=2, max=393, avg= 8.97, stdev= 5.61
clat (usec): min=37, max=22863, avg=1081.42, stdev=671.35
lat (usec): min=47, max=22876, avg=1090.59, stdev=671.28
clat percentiles (usec):
| 1.00th=[ 370], 5.00th=[ 498], 10.00th=[ 564], 20.00th=[ 652],
| 30.00th=[ 732], 40.00th=[ 812], 50.00th=[ 892], 60.00th=[ 980],
| 70.00th=[ 1096], 80.00th=[ 1272], 90.00th=[ 1736], 95.00th=[ 2896],
| 99.00th=[ 3408], 99.50th=[ 3536], 99.90th=[ 3824], 99.95th=[ 3984],
| 99.99th=[ 9792]
bw (KB /s): min= 0, max=13809, per=99.88%, avg=9943.13, stdev=691.40
lat (usec) : 2=0.01%, 10=0.01%, 50=0.02%, 100=0.16%, 250=7.51%
lat (usec) : 500=28.92%, 750=24.36%, 1000=17.22%
lat (msec) : 2=15.71%, 4=6.07%, 10=0.02%, 20=0.01%, 50=0.01%
cpu : usr=13.24%, sys=41.47%, ctx=10030992, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=102.8%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=10200464/w=10198561/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4980.8MB, aggrb=9957KB/s, minb=9957KB/s, maxb=9957KB/s, mint=512188msec, maxt=512188msec
WRITE: io=4979.8MB, aggrb=9955KB/s, minb=9955KB/s, maxb=9955KB/s, mint=512188msec, maxt=512188msec
Disk stats (read/write):
sdf: ios=10486973/10483866, merge=386/0, ticks=5202719/11176594, in_queue=16370759, util=100.00%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdf-randrw-32-1K
===
mytest: (g=0): rw=randrw, bs=1K-1K/1K-1K/1K-1K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=10472: Sun Sep 15 11:30:12 2013
read : io=4935.3MB, bw=19739KB/s, iops=19738, runt=256017msec
slat (usec): min=2, max=420, avg= 7.20, stdev= 4.50
clat (usec): min=0, max=7373.8K, avg=553.56, stdev=4683.60
lat (usec): min=46, max=7373.8K, avg=560.86, stdev=4683.59
clat percentiles (usec):
| 1.00th=[ 145], 5.00th=[ 217], 10.00th=[ 262], 20.00th=[ 322],
| 30.00th=[ 366], 40.00th=[ 410], 50.00th=[ 454], 60.00th=[ 502],
| 70.00th=[ 564], 80.00th=[ 636], 90.00th=[ 788], 95.00th=[ 1064],
| 99.00th=[ 2704], 99.50th=[ 2864], 99.90th=[ 3248], 99.95th=[ 3408],
| 99.99th=[ 4320]
bw (KB /s): min= 0, max=25936, per=99.77%, avg=19692.14, stdev=1386.63
write: io=4933.1MB, bw=19734KB/s, iops=19734, runt=256017msec
slat (usec): min=2, max=402, avg= 8.41, stdev= 4.95
clat (usec): min=63, max=250909, avg=1048.89, stdev=836.74
lat (usec): min=79, max=250917, avg=1057.43, stdev=836.70
clat percentiles (usec):
| 1.00th=[ 358], 5.00th=[ 478], 10.00th=[ 540], 20.00th=[ 628],
| 30.00th=[ 708], 40.00th=[ 780], 50.00th=[ 860], 60.00th=[ 948],
| 70.00th=[ 1064], 80.00th=[ 1224], 90.00th=[ 1672], 95.00th=[ 2832],
| 99.00th=[ 3376], 99.50th=[ 3536], 99.90th=[ 3824], 99.95th=[ 3984],
| 99.99th=[ 9280]
bw (KB /s): min= 9628, max=26090, per=99.96%, avg=19725.79, stdev=1090.30
lat (usec) : 2=0.01%, 50=0.01%, 100=0.05%, 250=4.34%, 500=28.60%
lat (usec) : 750=29.10%, 1000=17.58%
lat (msec) : 2=14.31%, 4=5.98%, 10=0.03%, 20=0.01%, 250=0.01%
lat (msec) : 500=0.01%, >=2000=0.01%
cpu : usr=13.07%, sys=40.04%, ctx=3624873, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=103.8%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=5053456/w=5052308/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4935.3MB, aggrb=19738KB/s, minb=19738KB/s, maxb=19738KB/s, mint=256017msec, maxt=256017msec
WRITE: io=4933.1MB, aggrb=19734KB/s, minb=19734KB/s, maxb=19734KB/s, mint=256017msec, maxt=256017msec
Disk stats (read/write):
sdf: ios=5241786/5239057, merge=289/0, ticks=2808259/5428710, in_queue=8233025, util=100.00%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdf-randrw-32-2K
===
mytest: (g=0): rw=randrw, bs=2K-2K/2K-2K/2K-2K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=10822: Sun Sep 15 11:33:26 2013
read : io=4745.7MB, bw=39461KB/s, iops=19730, runt=123147msec
slat (usec): min=2, max=259, avg= 6.45, stdev= 2.85
clat (usec): min=58, max=7378.8K, avg=617.82, stdev=11616.83
lat (usec): min=66, max=7378.8K, avg=624.42, stdev=11616.83
clat percentiles (usec):
| 1.00th=[ 197], 5.00th=[ 274], 10.00th=[ 318], 20.00th=[ 374],
| 30.00th=[ 418], 40.00th=[ 462], 50.00th=[ 506], 60.00th=[ 556],
| 70.00th=[ 612], 80.00th=[ 684], 90.00th=[ 820], 95.00th=[ 1064],
| 99.00th=[ 2736], 99.50th=[ 2896], 99.90th=[ 3184], 99.95th=[ 3312],
| 99.99th=[ 6880]
bw (KB /s): min= 0, max=47800, per=99.61%, avg=39307.14, stdev=3109.58
write: io=4743.7MB, bw=39445KB/s, iops=19722, runt=123147msec
slat (usec): min=2, max=468, avg= 7.58, stdev= 3.41
clat (usec): min=83, max=7378.8K, avg=987.48, stdev=10627.36
lat (usec): min=93, max=7378.8K, avg=995.25, stdev=10627.36
clat percentiles (usec):
| 1.00th=[ 314], 5.00th=[ 422], 10.00th=[ 486], 20.00th=[ 572],
| 30.00th=[ 636], 40.00th=[ 708], 50.00th=[ 788], 60.00th=[ 868],
| 70.00th=[ 980], 80.00th=[ 1144], 90.00th=[ 1560], 95.00th=[ 2736],
| 99.00th=[ 3280], 99.50th=[ 3440], 99.90th=[ 3760], 99.95th=[ 3888],
| 99.99th=[ 9920]
bw (KB /s): min=25613, max=47664, per=100.00%, avg=39451.20, stdev=1843.41
lat (usec) : 100=0.01%, 250=1.78%, 500=28.32%, 750=35.50%, 1000=17.37%
lat (msec) : 2=11.24%, 4=5.75%, 10=0.02%, 20=0.01%, 50=0.01%
lat (msec) : 250=0.01%, 500=0.01%, >=2000=0.01%
cpu : usr=12.79%, sys=39.48%, ctx=992506, majf=0, minf=28
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=107.9%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=2429770/w=2428716/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4745.7MB, aggrb=39461KB/s, minb=39461KB/s, maxb=39461KB/s, mint=123147msec, maxt=123147msec
WRITE: io=4743.7MB, aggrb=39444KB/s, minb=39444KB/s, maxb=39444KB/s, mint=123147msec, maxt=123147msec
Disk stats (read/write):
sdf: ios=2620446/2618017, merge=291/0, ticks=1571078/2553046, in_queue=4122848, util=100.00%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdf-randrw-32-4K
===
mytest: (g=0): rw=randrw, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=11006: Sun Sep 15 11:35:34 2013
read : io=4375.1MB, bw=77045KB/s, iops=19260, runt= 58160msec
slat (usec): min=2, max=229, avg= 6.42, stdev= 2.62
clat (usec): min=73, max=7365.8K, avg=691.04, stdev=13968.92
lat (usec): min=82, max=7365.8K, avg=697.56, stdev=13968.91
clat percentiles (usec):
| 1.00th=[ 221], 5.00th=[ 306], 10.00th=[ 350], 20.00th=[ 418],
| 30.00th=[ 470], 40.00th=[ 516], 50.00th=[ 572], 60.00th=[ 620],
| 70.00th=[ 684], 80.00th=[ 756], 90.00th=[ 892], 95.00th=[ 1160],
| 99.00th=[ 2800], 99.50th=[ 2928], 99.90th=[ 3248], 99.95th=[ 3376],
| 99.99th=[ 7712]
bw (KB /s): min=40848, max=88776, per=100.00%, avg=77049.59, stdev=4389.40
write: io=4373.1MB, bw=77009KB/s, iops=19252, runt= 58160msec
slat (usec): min=2, max=415, avg= 7.72, stdev= 3.20
clat (usec): min=74, max=7365.8K, avg=953.69, stdev=7047.69
lat (usec): min=87, max=7365.8K, avg=961.58, stdev=7047.69
clat percentiles (usec):
| 1.00th=[ 342], 5.00th=[ 450], 10.00th=[ 506], 20.00th=[ 580],
| 30.00th=[ 644], 40.00th=[ 700], 50.00th=[ 764], 60.00th=[ 836],
| 70.00th=[ 932], 80.00th=[ 1080], 90.00th=[ 1464], 95.00th=[ 2640],
| 99.00th=[ 3216], 99.50th=[ 3376], 99.90th=[ 3696], 99.95th=[ 3888],
| 99.99th=[11328]
bw (KB /s): min= 0, max=87784, per=99.15%, avg=76351.61, stdev=8360.24
lat (usec) : 100=0.01%, 250=0.99%, 500=21.57%, 750=40.82%, 1000=20.99%
lat (msec) : 2=9.90%, 4=5.71%, 10=0.02%, 20=0.01%, 250=0.01%
lat (msec) : 500=0.01%, >=2000=0.01%
cpu : usr=14.95%, sys=41.58%, ctx=510947, majf=0, minf=29
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=117.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=1120218/w=1119700/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4375.1MB, aggrb=77044KB/s, minb=77044KB/s, maxb=77044KB/s, mint=58160msec, maxt=58160msec
WRITE: io=4373.1MB, aggrb=77009KB/s, minb=77009KB/s, maxb=77009KB/s, mint=58160msec, maxt=58160msec
Disk stats (read/write):
sdf: ios=1307690/1305825, merge=306/0, ticks=876871/1232003, in_queue=2108214, util=99.97%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdf-randrw-32-8K
===
mytest: (g=0): rw=randrw, bs=8K-8K/8K-8K/8K-8K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=11135: Sun Sep 15 11:37:07 2013
read : io=3543.1MB, bw=162036KB/s, iops=20253, runt= 22396msec
slat (usec): min=2, max=235, avg= 6.84, stdev= 2.59
clat (usec): min=94, max=13833, avg=855.48, stdev=463.44
lat (usec): min=103, max=13839, avg=862.20, stdev=463.34
clat percentiles (usec):
| 1.00th=[ 314], 5.00th=[ 478], 10.00th=[ 604], 20.00th=[ 676],
| 30.00th=[ 716], 40.00th=[ 748], 50.00th=[ 772], 60.00th=[ 796],
| 70.00th=[ 828], 80.00th=[ 868], 90.00th=[ 972], 95.00th=[ 1896],
| 99.00th=[ 2928], 99.50th=[ 3056], 99.90th=[ 3312], 99.95th=[ 3504],
| 99.99th=[11712]
bw (KB /s): min=152256, max=169728, per=100.00%, avg=162035.64, stdev=3460.63
write: io=3542.8MB, bw=161982KB/s, iops=20247, runt= 22396msec
slat (usec): min=3, max=680, avg= 8.35, stdev= 3.09
clat (usec): min=91, max=14959, avg=707.03, stdev=472.88
lat (usec): min=101, max=14973, avg=715.26, stdev=472.93
clat percentiles (usec):
| 1.00th=[ 374], 5.00th=[ 446], 10.00th=[ 482], 20.00th=[ 524],
| 30.00th=[ 548], 40.00th=[ 572], 50.00th=[ 596], 60.00th=[ 612],
| 70.00th=[ 636], 80.00th=[ 668], 90.00th=[ 780], 95.00th=[ 2096],
| 99.00th=[ 2640], 99.50th=[ 2768], 99.90th=[ 3056], 99.95th=[ 3184],
| 99.99th=[11712]
bw (KB /s): min= 0, max=167152, per=97.78%, avg=158382.58, stdev=24359.24
lat (usec) : 100=0.01%, 250=0.20%, 500=9.43%, 750=56.03%, 1000=25.88%
lat (msec) : 2=3.27%, 4=5.17%, 10=0.01%, 20=0.01%
cpu : usr=21.89%, sys=56.45%, ctx=254625, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=144.5%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=453602/w=453455/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=3543.1MB, aggrb=162035KB/s, minb=162035KB/s, maxb=162035KB/s, mint=22396msec, maxt=22396msec
WRITE: io=3542.8MB, aggrb=161982KB/s, minb=161982KB/s, maxb=161982KB/s, mint=22396msec, maxt=22396msec
Disk stats (read/write):
sdf: ios=651932/650507, merge=338/0, ticks=547181/446763, in_queue=993588, util=99.83%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdf-randrw-32-16K
===
mytest: (g=0): rw=randrw, bs=16K-16K/16K-16K/16K-16K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=11263: Sun Sep 15 11:38:36 2013
read : io=3357.8MB, bw=182056KB/s, iops=11377, runt= 18886msec
slat (usec): min=3, max=189, avg= 8.32, stdev= 2.36
clat (usec): min=173, max=13610, avg=1546.87, stdev=693.08
lat (usec): min=184, max=13615, avg=1555.29, stdev=692.94
clat percentiles (usec):
| 1.00th=[ 556], 5.00th=[ 1032], 10.00th=[ 1112], 20.00th=[ 1192],
| 30.00th=[ 1240], 40.00th=[ 1272], 50.00th=[ 1320], 60.00th=[ 1368],
| 70.00th=[ 1432], 80.00th=[ 1576], 90.00th=[ 2928], 95.00th=[ 3344],
| 99.00th=[ 3664], 99.50th=[ 3824], 99.90th=[ 4128], 99.95th=[ 4384],
| 99.99th=[ 7968]
bw (KB /s): min= 1, max=187328, per=97.29%, avg=177119.08, stdev=30102.41
write: io=3341.5MB, bw=181175KB/s, iops=11322, runt= 18886msec
slat (usec): min=4, max=214, avg=10.85, stdev= 2.62
clat (usec): min=212, max=14258, avg=1248.95, stdev=581.76
lat (usec): min=224, max=14263, avg=1259.92, stdev=581.63
clat percentiles (usec):
| 1.00th=[ 692], 5.00th=[ 820], 10.00th=[ 892], 20.00th=[ 964],
| 30.00th=[ 1012], 40.00th=[ 1048], 50.00th=[ 1096], 60.00th=[ 1128],
| 70.00th=[ 1176], 80.00th=[ 1240], 90.00th=[ 1992], 95.00th=[ 2864],
| 99.00th=[ 3280], 99.50th=[ 3344], 99.90th=[ 3504], 99.95th=[ 3696],
| 99.99th=[11456]
bw (KB /s): min=173760, max=186688, per=99.86%, avg=180923.89, stdev=3186.44
lat (usec) : 250=0.02%, 500=0.42%, 750=1.58%, 1000=13.44%
lat (msec) : 2=72.74%, 4=11.69%, 10=0.11%, 20=0.01%
cpu : usr=17.17%, sys=38.25%, ctx=139302, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=152.9%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=214876/w=213842/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=3357.8MB, aggrb=182055KB/s, minb=182055KB/s, maxb=182055KB/s, mint=18886msec, maxt=18886msec
WRITE: io=3341.5MB, aggrb=181175KB/s, minb=181175KB/s, maxb=181175KB/s, mint=18886msec, maxt=18886msec
Disk stats (read/write):
sdf: ios=326255/324592, merge=0/0, ticks=497290/397813, in_queue=894927, util=99.79%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdf-randrw-32-32K
===
mytest: (g=0): rw=randrw, bs=32K-32K/32K-32K/32K-32K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=11333: Sun Sep 15 11:40:03 2013
read : io=3153.7MB, bw=201645KB/s, iops=6300, runt= 16012msec
slat (usec): min=3, max=164, avg=10.01, stdev= 2.27
clat (usec): min=273, max=16676, avg=2736.39, stdev=889.18
lat (usec): min=283, max=16684, avg=2746.42, stdev=889.03
clat percentiles (usec):
| 1.00th=[ 1304], 5.00th=[ 1928], 10.00th=[ 2040], 20.00th=[ 2160],
| 30.00th=[ 2224], 40.00th=[ 2288], 50.00th=[ 2384], 60.00th=[ 2480],
| 70.00th=[ 2672], 80.00th=[ 3792], 90.00th=[ 4256], 95.00th=[ 4448],
| 99.00th=[ 4832], 99.50th=[ 5024], 99.90th=[ 5600], 99.95th=[ 9792],
| 99.99th=[14016]
bw (KB /s): min= 3, max=208640, per=96.88%, avg=195348.84, stdev=35900.50
write: io=3162.3MB, bw=202230KB/s, iops=6318, runt= 16012msec
slat (usec): min=7, max=229, avg=14.73, stdev= 2.69
clat (usec): min=357, max=15592, avg=2306.96, stdev=766.86
lat (usec): min=368, max=15609, avg=2321.71, stdev=766.67
clat percentiles (usec):
| 1.00th=[ 1304], 5.00th=[ 1544], 10.00th=[ 1688], 20.00th=[ 1848],
| 30.00th=[ 1944], 40.00th=[ 2008], 50.00th=[ 2064], 60.00th=[ 2128],
| 70.00th=[ 2224], 80.00th=[ 2512], 90.00th=[ 3760], 95.00th=[ 4048],
| 99.00th=[ 4320], 99.50th=[ 4384], 99.90th=[ 4704], 99.95th=[ 9408],
| 99.99th=[13376]
bw (KB /s): min=193664, max=209856, per=99.94%, avg=202104.61, stdev=3954.64
lat (usec) : 500=0.01%, 750=0.06%, 1000=0.17%
lat (msec) : 2=23.26%, 4=65.42%, 10=11.04%, 20=0.05%
cpu : usr=14.03%, sys=25.76%, ctx=93532, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=162.1%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=100879/w=101179/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=3153.7MB, aggrb=201644KB/s, minb=201644KB/s, maxb=201644KB/s, mint=16012msec, maxt=16012msec
WRITE: io=3162.3MB, aggrb=202230KB/s, minb=202230KB/s, maxb=202230KB/s, mint=16012msec, maxt=16012msec
Disk stats (read/write):
sdf: ios=163882/163226, merge=0/0, ticks=444789/373700, in_queue=818428, util=99.77%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdf-randrw-32-64K
===
mytest: (g=0): rw=randrw, bs=64K-64K/64K-64K/64K-64K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=11457: Sun Sep 15 11:41:27 2013
read : io=2944.2MB, bw=216844KB/s, iops=3386, runt= 13903msec
slat (usec): min=6, max=209, avg=15.11, stdev= 3.89
clat (usec): min=669, max=3646.5K, avg=5071.68, stdev=17616.48
lat (usec): min=686, max=3646.7K, avg=5086.87, stdev=17616.48
clat percentiles (msec):
| 1.00th=[ 4], 5.00th=[ 4], 10.00th=[ 4], 20.00th=[ 5],
| 30.00th=[ 5], 40.00th=[ 5], 50.00th=[ 5], 60.00th=[ 5],
| 70.00th=[ 6], 80.00th=[ 6], 90.00th=[ 7], 95.00th=[ 7],
| 99.00th=[ 8], 99.50th=[ 9], 99.90th=[ 20], 99.95th=[ 121],
| 99.99th=[ 253]
bw (KB /s): min= 6, max=230912, per=96.74%, avg=209772.19, stdev=45676.59
write: io=2946.9MB, bw=217042KB/s, iops=3390, runt= 13903msec
slat (usec): min=18, max=296, avg=27.37, stdev= 5.81
clat (usec): min=656, max=253378, avg=4321.15, stdev=4653.54
lat (usec): min=681, max=253408, avg=4350.13, stdev=4653.56
clat percentiles (msec):
| 1.00th=[ 3], 5.00th=[ 3], 10.00th=[ 4], 20.00th=[ 4],
| 30.00th=[ 4], 40.00th=[ 4], 50.00th=[ 4], 60.00th=[ 5],
| 70.00th=[ 5], 80.00th=[ 6], 90.00th=[ 6], 95.00th=[ 6],
| 99.00th=[ 7], 99.50th=[ 8], 99.90th=[ 15], 99.95th=[ 117],
| 99.99th=[ 251]
bw (KB /s): min=152227, max=230528, per=100.00%, avg=218076.85, stdev=16933.73
lat (usec) : 750=0.01%, 1000=0.01%
lat (msec) : 2=0.24%, 4=36.12%, 10=63.51%, 20=0.08%, 50=0.02%
lat (msec) : 250=0.05%, 500=0.02%, >=2000=0.01%
cpu : usr=12.52%, sys=22.41%, ctx=62614, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=173.8%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=47087/w=47137/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=2944.2MB, aggrb=216844KB/s, minb=216844KB/s, maxb=216844KB/s, mint=13903msec, maxt=13903msec
WRITE: io=2946.9MB, aggrb=217042KB/s, minb=217042KB/s, maxb=217042KB/s, mint=13903msec, maxt=13903msec
Disk stats (read/write):
sdf: ios=81755/81147, merge=0/0, ticks=400270/346700, in_queue=750348, util=99.73%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdf-randrw-32-128K
===
mytest: (g=0): rw=randrw, bs=128K-128K/128K-128K/128K-128K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=11871: Sun Sep 15 11:42:50 2013
read : io=2824.4MB, bw=231169KB/s, iops=1804, runt= 12511msec
slat (usec): min=13, max=223, avg=20.07, stdev= 3.84
clat (msec): min=1, max=3740, avg= 9.28, stdev=26.02
lat (msec): min=1, max=3740, avg= 9.30, stdev=26.02
clat percentiles (msec):
| 1.00th=[ 7], 5.00th=[ 8], 10.00th=[ 8], 20.00th=[ 8],
| 30.00th=[ 8], 40.00th=[ 9], 50.00th=[ 9], 60.00th=[ 9],
| 70.00th=[ 10], 80.00th=[ 11], 90.00th=[ 11], 95.00th=[ 12],
| 99.00th=[ 13], 99.50th=[ 14], 99.90th=[ 126], 99.95th=[ 251],
| 99.99th=[ 255]
bw (KB /s): min=122634, max=258048, per=99.86%, avg=230839.42, stdev=27476.79
write: io=2815.8MB, bw=230463KB/s, iops=1799, runt= 12511msec
slat (usec): min=21, max=254, avg=41.50, stdev= 7.68
clat (usec): min=831, max=3739.9K, avg=8396.46, stdev=25612.21
lat (usec): min=872, max=3739.1K, avg=8439.93, stdev=25612.25
clat percentiles (msec):
| 1.00th=[ 5], 5.00th=[ 7], 10.00th=[ 7], 20.00th=[ 8],
| 30.00th=[ 8], 40.00th=[ 8], 50.00th=[ 8], 60.00th=[ 8],
| 70.00th=[ 9], 80.00th=[ 10], 90.00th=[ 10], 95.00th=[ 11],
| 99.00th=[ 12], 99.50th=[ 13], 99.90th=[ 116], 99.95th=[ 243],
| 99.99th=[ 255]
bw (KB /s): min= 12, max=248576, per=95.77%, avg=220704.32, stdev=52285.92
lat (usec) : 1000=0.01%
lat (msec) : 2=0.01%, 4=0.15%, 10=85.09%, 20=14.62%, 50=0.05%
lat (msec) : 250=0.10%, 500=0.03%, >=2000=0.01%
cpu : usr=11.31%, sys=15.34%, ctx=45345, majf=0, minf=28
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=181.5%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=22575/w=22515/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=2824.4MB, aggrb=231169KB/s, minb=231169KB/s, maxb=231169KB/s, mint=12511msec, maxt=12511msec
WRITE: io=2815.8MB, aggrb=230463KB/s, minb=230463KB/s, maxb=230463KB/s, mint=12511msec, maxt=12511msec
Disk stats (read/write):
sdf: ios=41434/40788, merge=0/0, ticks=370459/335485, in_queue=713263, util=99.71%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdf-randrw-32-256K
===
mytest: (g=0): rw=randrw, bs=256K-256K/256K-256K/256K-256K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=11941: Sun Sep 15 11:44:12 2013
read : io=2694.0MB, bw=242562KB/s, iops=945, runt= 11373msec
slat (usec): min=20, max=393, avg=37.84, stdev= 9.06
clat (msec): min=1, max=2580, avg=17.53, stdev=27.07
lat (msec): min=1, max=2581, avg=17.57, stdev=27.07
clat percentiles (msec):
| 1.00th=[ 14], 5.00th=[ 15], 10.00th=[ 15], 20.00th=[ 16],
| 30.00th=[ 16], 40.00th=[ 17], 50.00th=[ 17], 60.00th=[ 17],
| 70.00th=[ 18], 80.00th=[ 19], 90.00th=[ 19], 95.00th=[ 20],
| 99.00th=[ 22], 99.50th=[ 30], 99.90th=[ 251], 99.95th=[ 260],
| 99.99th=[ 265]
bw (KB /s): min= 25, max=266240, per=95.52%, avg=231683.09, stdev=58148.53
write: io=2677.3MB, bw=241054KB/s, iops=940, runt= 11373msec
slat (usec): min=49, max=399, avg=75.67, stdev=12.19
clat (msec): min=2, max=2581, avg=16.24, stdev=26.57
lat (msec): min=2, max=2582, avg=16.32, stdev=26.57
clat percentiles (msec):
| 1.00th=[ 12], 5.00th=[ 14], 10.00th=[ 14], 20.00th=[ 15],
| 30.00th=[ 15], 40.00th=[ 16], 50.00th=[ 16], 60.00th=[ 16],
| 70.00th=[ 17], 80.00th=[ 17], 90.00th=[ 18], 95.00th=[ 19],
| 99.00th=[ 21], 99.50th=[ 25], 99.90th=[ 241], 99.95th=[ 249],
| 99.99th=[ 262]
bw (KB /s): min=131295, max=263680, per=99.96%, avg=240950.41, stdev=29485.50
lat (msec) : 2=0.01%, 4=0.02%, 10=0.16%, 20=97.64%, 50=2.01%
lat (msec) : 250=0.21%, 500=0.07%, >=2000=0.01%
cpu : usr=9.73%, sys=16.00%, ctx=22530, majf=0, minf=28
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=190.6%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=10758/w=10696/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=2694.0MB, aggrb=242561KB/s, minb=242561KB/s, maxb=242561KB/s, mint=11373msec, maxt=11373msec
WRITE: io=2677.3MB, aggrb=241053KB/s, minb=241053KB/s, maxb=241053KB/s, mint=11373msec, maxt=11373msec
Disk stats (read/write):
sdf: ios=20981/20237, merge=0/1, ticks=346961/320728, in_queue=672605, util=99.69%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdf-randrw-32-512K
===
mytest: (g=0): rw=randrw, bs=512K-512K/512K-512K/512K-512K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=12066: Sun Sep 15 11:45:33 2013
read : io=2581.6MB, bw=250755KB/s, iops=488, runt= 10542msec
slat (usec): min=41, max=127, avg=66.37, stdev= 6.30
clat (msec): min=3, max=277, avg=32.93, stdev=13.47
lat (msec): min=3, max=277, avg=32.99, stdev=13.47
clat percentiles (msec):
| 1.00th=[ 28], 5.00th=[ 30], 10.00th=[ 30], 20.00th=[ 31],
| 30.00th=[ 32], 40.00th=[ 32], 50.00th=[ 32], 60.00th=[ 33],
| 70.00th=[ 34], 80.00th=[ 34], 90.00th=[ 35], 95.00th=[ 36],
| 99.00th=[ 41], 99.50th=[ 68], 99.90th=[ 265], 99.95th=[ 273],
| 99.99th=[ 277]
bw (KB /s): min= 51, max=281600, per=96.11%, avg=241006.95, stdev=62857.00
write: io=2595.0MB, bw=252066KB/s, iops=490, runt= 10542msec
slat (usec): min=90, max=354, avg=147.50, stdev=21.41
clat (msec): min=4, max=2489, avg=32.08, stdev=50.41
lat (msec): min=4, max=2489, avg=32.23, stdev=50.41
clat percentiles (msec):
| 1.00th=[ 24], 5.00th=[ 26], 10.00th=[ 28], 20.00th=[ 29],
| 30.00th=[ 30], 40.00th=[ 30], 50.00th=[ 31], 60.00th=[ 31],
| 70.00th=[ 32], 80.00th=[ 32], 90.00th=[ 33], 95.00th=[ 34],
| 99.00th=[ 52], 99.50th=[ 161], 99.90th=[ 258], 99.95th=[ 281],
| 99.99th=[ 2474]
bw (KB /s): min=183815, max=278528, per=100.00%, avg=252516.45, stdev=21537.36
lat (msec) : 4=0.03%, 10=0.08%, 20=0.21%, 50=99.07%, 100=0.29%
lat (msec) : 250=0.44%, 500=0.16%, >=2000=0.02%
cpu : usr=9.57%, sys=14.63%, ctx=11276, majf=0, minf=28
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.2%, 16=0.3%, 32=197.8%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=5148/w=5174/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=2581.6MB, aggrb=250754KB/s, minb=250754KB/s, maxb=250754KB/s, mint=10542msec, maxt=10542msec
WRITE: io=2595.0MB, aggrb=252066KB/s, minb=252066KB/s, maxb=252066KB/s, mint=10542msec, maxt=10542msec
Disk stats (read/write):
sdf: ios=10838/10178, merge=0/0, ticks=332198/311751, in_queue=649041, util=99.65%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdf-randrw-32-1024K
===
mytest: (g=0): rw=randrw, bs=1M-1M/1M-1M/1M-1M, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=12135: Sun Sep 15 11:46:43 2013
read : io=2600.0MB, bw=267148KB/s, iops=260, runt= 9966msec
slat (usec): min=78, max=165, avg=119.36, stdev= 8.30
clat percentiles (usec):
| 1.00th=[ 0], 5.00th=[ 0], 10.00th=[ 0], 20.00th=[ 0],
| 30.00th=[ 0], 40.00th=[ 0], 50.00th=[ 0], 60.00th=[ 0],
| 70.00th=[ 0], 80.00th=[ 0], 90.00th=[ 0], 95.00th=[ 0],
| 99.00th=[ 0], 99.50th=[ 0], 99.90th=[ 0], 99.95th=[ 0],
| 99.99th=[ 0]
write: io=2520.0MB, bw=258928KB/s, iops=252, runt= 9966msec
slat (usec): min=186, max=478, avg=266.46, stdev=17.74
clat percentiles (usec):
| 1.00th=[ 0], 5.00th=[ 0], 10.00th=[ 0], 20.00th=[ 0],
| 30.00th=[ 0], 40.00th=[ 0], 50.00th=[ 0], 60.00th=[ 0],
| 70.00th=[ 0], 80.00th=[ 0], 90.00th=[ 0], 95.00th=[ 0],
| 99.00th=[ 0], 99.50th=[ 0], 99.90th=[ 0], 99.95th=[ 0],
| 99.99th=[ 0]
cpu : usr=4.11%, sys=6.90%, ctx=4982, majf=0, minf=23
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.2%, 16=0.3%, 32=99.4%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=2600/w=2520/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=2600.0MB, aggrb=267148KB/s, minb=267148KB/s, maxb=267148KB/s, mint=9966msec, maxt=9966msec
WRITE: io=2520.0MB, aggrb=258928KB/s, minb=258928KB/s, maxb=258928KB/s, mint=9966msec, maxt=9966msec
Disk stats (read/write):
sdf: ios=5815/5002, merge=0/0, ticks=326438/296649, in_queue=626052, util=99.11%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdg-randrw-32-512B
===
mytest: (g=0): rw=randrw, bs=512-512/512-512/512-512, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=12211: Sun Sep 15 11:56:09 2013
read : io=4960.5MB, bw=10263KB/s, iops=20526, runt=494928msec
slat (usec): min=3, max=383, avg= 6.51, stdev= 5.39
clat (usec): min=151, max=25282, avg=785.33, stdev=800.69
lat (usec): min=156, max=25288, avg=791.98, stdev=800.27
clat percentiles (usec):
| 1.00th=[ 193], 5.00th=[ 217], 10.00th=[ 241], 20.00th=[ 290],
| 30.00th=[ 342], 40.00th=[ 394], 50.00th=[ 454], 60.00th=[ 572],
| 70.00th=[ 804], 80.00th=[ 1304], 90.00th=[ 1816], 95.00th=[ 2040],
| 99.00th=[ 4448], 99.50th=[ 5472], 99.90th=[ 6624], 99.95th=[ 7968],
| 99.99th=[10944]
bw (KB /s): min= 1333, max=15667, per=100.00%, avg=10276.61, stdev=1791.35
write: io=4959.6MB, bw=10261KB/s, iops=20522, runt=494928msec
slat (usec): min=3, max=381, avg= 7.70, stdev= 6.30
clat (usec): min=152, max=25283, avg=756.63, stdev=784.68
lat (usec): min=158, max=25289, avg=764.51, stdev=784.15
clat percentiles (usec):
| 1.00th=[ 191], 5.00th=[ 213], 10.00th=[ 233], 20.00th=[ 274],
| 30.00th=[ 330], 40.00th=[ 382], 50.00th=[ 438], 60.00th=[ 532],
| 70.00th=[ 756], 80.00th=[ 1224], 90.00th=[ 1784], 95.00th=[ 2024],
| 99.00th=[ 4320], 99.50th=[ 5472], 99.90th=[ 6496], 99.95th=[ 7776],
| 99.99th=[10816]
bw (KB /s): min= 0, max=15466, per=100.00%, avg=10264.27, stdev=1815.41
lat (usec) : 250=13.36%, 500=42.72%, 750=12.90%, 1000=6.85%
lat (msec) : 2=18.52%, 4=4.50%, 10=1.12%, 20=0.02%, 50=0.01%
cpu : usr=6.75%, sys=34.71%, ctx=894428, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=103.2%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=10159034/w=10157073/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4960.5MB, aggrb=10263KB/s, minb=10263KB/s, maxb=10263KB/s, mint=494928msec, maxt=494928msec
WRITE: io=4959.6MB, aggrb=10261KB/s, minb=10261KB/s, maxb=10261KB/s, mint=494928msec, maxt=494928msec
Disk stats (read/write):
sdg: ios=10486503/10484074, merge=0/0, ticks=7264088/6914950, in_queue=14171861, util=100.00%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdg-randrw-32-1K
===
mytest: (g=0): rw=randrw, bs=1K-1K/1K-1K/1K-1K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=13118: Sun Sep 15 12:01:21 2013
read : io=4916.8MB, bw=20823KB/s, iops=20823, runt=241750msec
slat (usec): min=2, max=424, avg= 6.69, stdev= 4.94
clat (usec): min=162, max=18014, avg=773.77, stdev=760.60
lat (usec): min=167, max=18018, avg=780.56, stdev=760.18
clat percentiles (usec):
| 1.00th=[ 209], 5.00th=[ 229], 10.00th=[ 255], 20.00th=[ 302],
| 30.00th=[ 346], 40.00th=[ 390], 50.00th=[ 442], 60.00th=[ 540],
| 70.00th=[ 780], 80.00th=[ 1288], 90.00th=[ 1816], 95.00th=[ 2024],
| 99.00th=[ 4128], 99.50th=[ 5344], 99.90th=[ 6112], 99.95th=[ 6432],
| 99.99th=[ 9152]
bw (KB /s): min= 5972, max=23750, per=100.00%, avg=20867.69, stdev=2377.43
write: io=4914.2MB, bw=20819KB/s, iops=20818, runt=241750msec
slat (usec): min=3, max=366, avg= 7.94, stdev= 5.75
clat (usec): min=92, max=13179, avg=745.42, stdev=745.74
lat (usec): min=97, max=13184, avg=753.48, stdev=745.23
clat percentiles (usec):
| 1.00th=[ 207], 5.00th=[ 225], 10.00th=[ 245], 20.00th=[ 290],
| 30.00th=[ 334], 40.00th=[ 378], 50.00th=[ 430], 60.00th=[ 510],
| 70.00th=[ 732], 80.00th=[ 1208], 90.00th=[ 1784], 95.00th=[ 2024],
| 99.00th=[ 3952], 99.50th=[ 5280], 99.90th=[ 6112], 99.95th=[ 6368],
| 99.99th=[ 9024]
bw (KB /s): min= 0, max=23504, per=100.00%, avg=20819.78, stdev=2552.62
lat (usec) : 100=0.01%, 250=10.19%, 500=47.91%, 750=11.70%, 1000=6.55%
lat (msec) : 2=18.08%, 4=4.55%, 10=1.01%, 20=0.01%
cpu : usr=7.50%, sys=36.33%, ctx=475368, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=104.2%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=5034046/w=5032944/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4916.8MB, aggrb=20823KB/s, minb=20823KB/s, maxb=20823KB/s, mint=241750msec, maxt=241750msec
WRITE: io=4914.2MB, aggrb=20818KB/s, minb=20818KB/s, maxb=20818KB/s, mint=241750msec, maxt=241750msec
Disk stats (read/write):
sdg: ios=5243551/5241637, merge=0/0, ticks=3648287/3470513, in_queue=7114696, util=100.00%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdg-randrw-32-2K
===
mytest: (g=0): rw=randrw, bs=2K-2K/2K-2K/2K-2K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=13435: Sun Sep 15 12:04:29 2013
read : io=4724.4MB, bw=40990KB/s, iops=20495, runt=118013msec
slat (usec): min=2, max=341, avg= 7.20, stdev= 4.54
clat (usec): min=151, max=16229, avg=785.68, stdev=725.53
lat (usec): min=158, max=16234, avg=792.88, stdev=725.19
clat percentiles (usec):
| 1.00th=[ 253], 5.00th=[ 290], 10.00th=[ 314], 20.00th=[ 346],
| 30.00th=[ 378], 40.00th=[ 418], 50.00th=[ 470], 60.00th=[ 548],
| 70.00th=[ 764], 80.00th=[ 1256], 90.00th=[ 1816], 95.00th=[ 2040],
| 99.00th=[ 3760], 99.50th=[ 5280], 99.90th=[ 6048], 99.95th=[ 6176],
| 99.99th=[ 6816]
bw (KB /s): min=28195, max=44964, per=100.00%, avg=41049.92, stdev=2107.99
write: io=4722.1MB, bw=40974KB/s, iops=20486, runt=118013msec
slat (usec): min=2, max=410, avg= 8.68, stdev= 5.34
clat (usec): min=116, max=16246, avg=757.16, stdev=712.79
lat (usec): min=121, max=16252, avg=765.89, stdev=712.34
clat percentiles (usec):
| 1.00th=[ 241], 5.00th=[ 278], 10.00th=[ 298], 20.00th=[ 334],
| 30.00th=[ 370], 40.00th=[ 406], 50.00th=[ 454], 60.00th=[ 524],
| 70.00th=[ 708], 80.00th=[ 1160], 90.00th=[ 1784], 95.00th=[ 2024],
| 99.00th=[ 3568], 99.50th=[ 5216], 99.90th=[ 6048], 99.95th=[ 6112],
| 99.99th=[ 6752]
bw (KB /s): min= 0, max=45276, per=99.72%, avg=40858.20, stdev=3405.41
lat (usec) : 250=1.21%, 500=54.65%, 750=14.50%, 1000=6.78%
lat (msec) : 2=17.11%, 4=4.84%, 10=0.91%, 20=0.01%
cpu : usr=8.25%, sys=39.35%, ctx=292584, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=108.4%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=2418692/w=2417696/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4724.4MB, aggrb=40990KB/s, minb=40990KB/s, maxb=40990KB/s, mint=118013msec, maxt=118013msec
WRITE: io=4722.1MB, aggrb=40973KB/s, minb=40973KB/s, maxb=40973KB/s, mint=118013msec, maxt=118013msec
Disk stats (read/write):
sdg: ios=2619299/2617625, merge=0/0, ticks=1906451/1829526, in_queue=3733593, util=100.00%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdg-randrw-32-4K
===
mytest: (g=0): rw=randrw, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=13620: Sun Sep 15 12:06:42 2013
read : io=4408.5MB, bw=72707KB/s, iops=18176, runt= 62088msec
slat (usec): min=2, max=239, avg= 7.71, stdev= 4.17
clat (usec): min=153, max=14918, avg=885.25, stdev=709.15
lat (usec): min=159, max=14930, avg=892.88, stdev=708.92
clat percentiles (usec):
| 1.00th=[ 386], 5.00th=[ 422], 10.00th=[ 438], 20.00th=[ 462],
| 30.00th=[ 482], 40.00th=[ 516], 50.00th=[ 564], 60.00th=[ 612],
| 70.00th=[ 860], 80.00th=[ 1416], 90.00th=[ 1896], 95.00th=[ 2096],
| 99.00th=[ 3696], 99.50th=[ 5216], 99.90th=[ 6112], 99.95th=[ 6304],
| 99.99th=[ 9664]
bw (KB /s): min=66128, max=78672, per=100.00%, avg=72747.82, stdev=2537.39
write: io=4407.4MB, bw=72689KB/s, iops=18172, runt= 62088msec
slat (usec): min=3, max=1691, avg= 9.45, stdev= 5.17
clat (usec): min=102, max=18945, avg=855.47, stdev=693.97
lat (usec): min=106, max=18950, avg=864.91, stdev=693.68
clat percentiles (usec):
| 1.00th=[ 362], 5.00th=[ 406], 10.00th=[ 430], 20.00th=[ 454],
| 30.00th=[ 474], 40.00th=[ 498], 50.00th=[ 548], 60.00th=[ 596],
| 70.00th=[ 796], 80.00th=[ 1320], 90.00th=[ 1864], 95.00th=[ 2064],
| 99.00th=[ 3376], 99.50th=[ 5152], 99.90th=[ 6112], 99.95th=[ 6304],
| 99.99th=[ 7520]
bw (KB /s): min= 0, max=77912, per=99.25%, avg=72143.30, stdev=6919.09
lat (usec) : 250=0.02%, 500=38.27%, 750=29.22%, 1000=7.11%
lat (msec) : 2=18.47%, 4=6.03%, 10=0.87%, 20=0.01%
cpu : usr=9.88%, sys=40.98%, ctx=550136, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=116.2%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=1128544/w=1128270/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4408.5MB, aggrb=72707KB/s, minb=72707KB/s, maxb=72707KB/s, mint=62088msec, maxt=62088msec
WRITE: io=4407.4MB, aggrb=72689KB/s, minb=72689KB/s, maxb=72689KB/s, mint=62088msec, maxt=62088msec
Disk stats (read/write):
sdg: ios=1309375/1308155, merge=0/0, ticks=1130169/1092303, in_queue=2221213, util=99.96%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdg-randrw-32-8K
===
mytest: (g=0): rw=randrw, bs=8K-8K/8K-8K/8K-8K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=13749: Sun Sep 15 12:08:42 2013
read : io=4249.8MB, bw=88810KB/s, iops=11100, runt= 48993msec
slat (usec): min=2, max=582, avg= 8.81, stdev= 4.49
clat (usec): min=166, max=13661, avg=1456.34, stdev=901.76
lat (usec): min=171, max=13664, avg=1465.29, stdev=901.63
clat percentiles (usec):
| 1.00th=[ 652], 5.00th=[ 684], 10.00th=[ 700], 20.00th=[ 772],
| 30.00th=[ 828], 40.00th=[ 932], 50.00th=[ 1112], 60.00th=[ 1416],
| 70.00th=[ 1896], 80.00th=[ 2160], 90.00th=[ 2384], 95.00th=[ 2512],
| 99.00th=[ 5664], 99.50th=[ 6048], 99.90th=[ 6816], 99.95th=[ 7584],
| 99.99th=[11200]
bw (KB /s): min=80736, max=96896, per=100.00%, avg=88863.66, stdev=3412.41
write: io=4244.8MB, bw=88718KB/s, iops=11089, runt= 48993msec
slat (usec): min=3, max=778, avg=11.64, stdev= 5.57
clat (usec): min=249, max=13626, avg=1403.42, stdev=885.01
lat (usec): min=266, max=13632, avg=1415.38, stdev=884.82
clat percentiles (usec):
| 1.00th=[ 596], 5.00th=[ 668], 10.00th=[ 692], 20.00th=[ 748],
| 30.00th=[ 804], 40.00th=[ 892], 50.00th=[ 1048], 60.00th=[ 1288],
| 70.00th=[ 1800], 80.00th=[ 2128], 90.00th=[ 2352], 95.00th=[ 2480],
| 99.00th=[ 5536], 99.50th=[ 5984], 99.90th=[ 6688], 99.95th=[ 7456],
| 99.99th=[11072]
bw (KB /s): min= 0, max=97392, per=99.04%, avg=87866.49, stdev=9541.95
lat (usec) : 250=0.01%, 500=0.02%, 750=18.61%, 1000=27.29%
lat (msec) : 2=28.66%, 4=23.40%, 10=2.00%, 20=0.02%
cpu : usr=11.19%, sys=30.63%, ctx=658917, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=120.6%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=543864/w=543307/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4249.8MB, aggrb=88809KB/s, minb=88809KB/s, maxb=88809KB/s, mint=48993msec, maxt=48993msec
WRITE: io=4244.8MB, aggrb=88718KB/s, minb=88718KB/s, maxb=88718KB/s, mint=48993msec, maxt=48993msec
Disk stats (read/write):
sdg: ios=655479/654695, merge=0/0, ticks=955092/919878, in_queue=1874487, util=99.93%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdg-randrw-32-16K
===
mytest: (g=0): rw=randrw, bs=16K-16K/16K-16K/16K-16K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=13878: Sun Sep 15 12:10:29 2013
read : io=4047.1MB, bw=110795KB/s, iops=6924, runt= 37412msec
slat (usec): min=3, max=188, avg= 9.00, stdev= 2.86
clat (usec): min=218, max=14681, avg=2334.86, stdev=1111.65
lat (usec): min=226, max=14688, avg=2343.83, stdev=1111.59
clat percentiles (usec):
| 1.00th=[ 1144], 5.00th=[ 1208], 10.00th=[ 1304], 20.00th=[ 1448],
| 30.00th=[ 1592], 40.00th=[ 1752], 50.00th=[ 2128], 60.00th=[ 2544],
| 70.00th=[ 2736], 80.00th=[ 2896], 90.00th=[ 3184], 95.00th=[ 4384],
| 99.00th=[ 6688], 99.50th=[ 7200], 99.90th=[ 8384], 99.95th=[ 8640],
| 99.99th=[11840]
bw (KB /s): min= 1, max=121211, per=98.83%, avg=109492.69, stdev=13665.89
write: io=4035.6MB, bw=110455KB/s, iops=6902, runt= 37412msec
slat (usec): min=5, max=1224, avg=11.95, stdev= 4.86
clat (usec): min=222, max=18592, avg=2268.83, stdev=1094.27
lat (usec): min=230, max=18603, avg=2280.77, stdev=1094.14
clat percentiles (usec):
| 1.00th=[ 1096], 5.00th=[ 1176], 10.00th=[ 1272], 20.00th=[ 1416],
| 30.00th=[ 1544], 40.00th=[ 1688], 50.00th=[ 1992], 60.00th=[ 2448],
| 70.00th=[ 2704], 80.00th=[ 2864], 90.00th=[ 3088], 95.00th=[ 4256],
| 99.00th=[ 6624], 99.50th=[ 7072], 99.90th=[ 8256], 99.95th=[ 8512],
| 99.99th=[11712]
bw (KB /s): min=102176, max=119904, per=100.00%, avg=110610.81, stdev=4431.57
lat (usec) : 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec) : 2=48.80%, 4=44.97%, 10=6.18%, 20=0.03%
cpu : usr=12.20%, sys=21.07%, ctx=603945, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=126.7%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=259052/w=258255/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4047.1MB, aggrb=110794KB/s, minb=110794KB/s, maxb=110794KB/s, mint=37412msec, maxt=37412msec
WRITE: io=4035.6MB, aggrb=110455KB/s, minb=110455KB/s, maxb=110455KB/s, mint=37412msec, maxt=37412msec
Disk stats (read/write):
sdg: ios=327158/325923, merge=0/0, ticks=764703/740345, in_queue=1505019, util=99.89%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdg-randrw-32-32K
===
mytest: (g=0): rw=randrw, bs=32K-32K/32K-32K/32K-32K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=14009: Sun Sep 15 12:12:12 2013
read : io=3897.8MB, bw=125294KB/s, iops=3914, runt= 31855msec
slat (usec): min=7, max=219, avg=11.20, stdev= 1.99
clat (usec): min=447, max=23453, avg=4131.11, stdev=1544.08
lat (usec): min=458, max=23462, avg=4142.39, stdev=1544.07
clat percentiles (usec):
| 1.00th=[ 2128], 5.00th=[ 2448], 10.00th=[ 2608], 20.00th=[ 2832],
| 30.00th=[ 3216], 40.00th=[ 3600], 50.00th=[ 3792], 60.00th=[ 4016],
| 70.00th=[ 4448], 80.00th=[ 5152], 90.00th=[ 6112], 95.00th=[ 7264],
| 99.00th=[ 9152], 99.50th=[ 9792], 99.90th=[12608], 99.95th=[13248],
| 99.99th=[21120]
bw (KB /s): min=114048, max=137900, per=100.00%, avg=125414.59, stdev=5105.66
write: io=3898.7MB, bw=125306KB/s, iops=3915, runt= 31855msec
slat (usec): min=11, max=2594, avg=16.64, stdev=10.22
clat (usec): min=452, max=26065, avg=4008.74, stdev=1498.15
lat (usec): min=466, max=26080, avg=4025.67, stdev=1498.03
clat percentiles (usec):
| 1.00th=[ 2064], 5.00th=[ 2384], 10.00th=[ 2544], 20.00th=[ 2736],
| 30.00th=[ 3088], 40.00th=[ 3504], 50.00th=[ 3760], 60.00th=[ 3920],
| 70.00th=[ 4256], 80.00th=[ 4960], 90.00th=[ 5792], 95.00th=[ 7136],
| 99.00th=[ 9024], 99.50th=[ 9536], 99.90th=[12480], 99.95th=[12864],
| 99.99th=[21632]
bw (KB /s): min= 3, max=136896, per=98.47%, avg=123388.86, stdev=16423.30
lat (usec) : 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec) : 2=0.09%, 4=61.24%, 10=38.28%, 20=0.37%, 50=0.02%
cpu : usr=9.81%, sys=15.10%, ctx=321587, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=131.3%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=124712/w=124721/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=3897.8MB, aggrb=125293KB/s, minb=125293KB/s, maxb=125293KB/s, mint=31855msec, maxt=31855msec
WRITE: io=3898.7MB, aggrb=125305KB/s, minb=125305KB/s, maxb=125305KB/s, mint=31855msec, maxt=31855msec
Disk stats (read/write):
sdg: ios=163527/163236, merge=0/0, ticks=675750/655670, in_queue=1331462, util=99.87%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdg-randrw-32-64K
===
mytest: (g=0): rw=randrw, bs=64K-64K/64K-64K/64K-64K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=14137: Sun Sep 15 12:13:47 2013
read : io=3630.8MB, bw=150708KB/s, iops=2354, runt= 24669msec
slat (usec): min=9, max=202, avg=13.53, stdev= 2.27
clat (usec): min=578, max=21392, avg=6852.91, stdev=1962.81
lat (usec): min=597, max=21404, avg=6866.37, stdev=1962.77
clat percentiles (usec):
| 1.00th=[ 4192], 5.00th=[ 4512], 10.00th=[ 4704], 20.00th=[ 5280],
| 30.00th=[ 5664], 40.00th=[ 5920], 50.00th=[ 6368], 60.00th=[ 6880],
| 70.00th=[ 7456], 80.00th=[ 8384], 90.00th=[ 9664], 95.00th=[10688],
| 99.00th=[12864], 99.50th=[14016], 99.90th=[15936], 99.95th=[17024],
| 99.99th=[19072]
bw (KB /s): min= 6, max=163840, per=98.09%, avg=147828.57, stdev=22577.75
write: io=3633.3MB, bw=150815KB/s, iops=2355, runt= 24669msec
slat (usec): min=15, max=5730, avg=23.49, stdev=29.48
clat (usec): min=580, max=23787, avg=6690.17, stdev=1930.86
lat (usec): min=602, max=23806, avg=6714.01, stdev=1930.30
clat percentiles (usec):
| 1.00th=[ 4048], 5.00th=[ 4448], 10.00th=[ 4640], 20.00th=[ 5088],
| 30.00th=[ 5536], 40.00th=[ 5792], 50.00th=[ 6176], 60.00th=[ 6688],
| 70.00th=[ 7264], 80.00th=[ 8256], 90.00th=[ 9408], 95.00th=[10432],
| 99.00th=[12736], 99.50th=[13888], 99.90th=[15936], 99.95th=[17280],
| 99.99th=[21120]
bw (KB /s): min=139752, max=166272, per=100.00%, avg=151095.12, stdev=6524.72
lat (usec) : 750=0.01%, 1000=0.01%
lat (msec) : 2=0.01%, 4=0.43%, 10=92.22%, 20=7.35%, 50=0.01%
cpu : usr=8.43%, sys=11.26%, ctx=163632, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=141.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=58075/w=58117/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=3630.8MB, aggrb=150708KB/s, minb=150708KB/s, maxb=150708KB/s, mint=24669msec, maxt=24669msec
WRITE: io=3633.3MB, aggrb=150814KB/s, minb=150814KB/s, maxb=150814KB/s, mint=24669msec, maxt=24669msec
Disk stats (read/write):
sdg: ios=81639/81434, merge=0/1, ticks=556622/543302, in_queue=1100085, util=99.82%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdg-randrw-32-128K
===
mytest: (g=0): rw=randrw, bs=128K-128K/128K-128K/128K-128K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=14555: Sun Sep 15 12:15:18 2013
read : io=3412.3MB, bw=174002KB/s, iops=1358, runt= 20081msec
slat (usec): min=14, max=211, avg=20.90, stdev= 3.67
clat (usec): min=964, max=30774, avg=11883.16, stdev=2657.88
lat (usec): min=983, max=30794, avg=11903.73, stdev=2657.87
clat percentiles (usec):
| 1.00th=[ 8160], 5.00th=[ 8640], 10.00th=[ 9024], 20.00th=[ 9664],
| 30.00th=[10176], 40.00th=[10688], 50.00th=[11200], 60.00th=[11968],
| 70.00th=[12864], 80.00th=[14016], 90.00th=[15424], 95.00th=[16768],
| 99.00th=[20096], 99.50th=[21376], 99.90th=[24448], 99.95th=[25472],
| 99.99th=[29568]
bw (KB /s): min=159744, max=189184, per=100.00%, avg=174302.85, stdev=6713.96
write: io=3412.0MB, bw=173990KB/s, iops=1358, runt= 20081msec
slat (usec): min=26, max=254, avg=39.83, stdev= 4.95
clat (msec): min=1, max=30, avg=11.60, stdev= 2.58
lat (msec): min=1, max=30, avg=11.64, stdev= 2.58
clat percentiles (usec):
| 1.00th=[ 7968], 5.00th=[ 8512], 10.00th=[ 8896], 20.00th=[ 9408],
| 30.00th=[ 9920], 40.00th=[10432], 50.00th=[10944], 60.00th=[11584],
| 70.00th=[12608], 80.00th=[13632], 90.00th=[15168], 95.00th=[16512],
| 99.00th=[19328], 99.50th=[20608], 99.90th=[23936], 99.95th=[24448],
| 99.99th=[27520]
bw (KB /s): min= 12, max=187648, per=97.57%, avg=169767.22, stdev=28052.52
lat (usec) : 1000=0.01%
lat (msec) : 2=0.01%, 4=0.01%, 10=29.74%, 20=69.40%, 50=0.90%
cpu : usr=6.84%, sys=10.48%, ctx=81900, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=150.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=27281/w=27282/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=3412.3MB, aggrb=174002KB/s, minb=174002KB/s, maxb=174002KB/s, mint=20081msec, maxt=20081msec
WRITE: io=3412.0MB, aggrb=173989KB/s, minb=173989KB/s, maxb=173989KB/s, mint=20081msec, maxt=20081msec
Disk stats (read/write):
sdg: ios=40851/40753, merge=0/0, ticks=482906/472331, in_queue=955313, util=99.79%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdg-randrw-32-256K
===
mytest: (g=0): rw=randrw, bs=256K-256K/256K-256K/256K-256K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=14681: Sun Sep 15 12:16:45 2013
read : io=3220.0MB, bw=196688KB/s, iops=767, runt= 16764msec
slat (usec): min=24, max=100, avg=35.91, stdev= 4.79
clat (msec): min=1, max=51, avg=21.04, stdev= 3.51
lat (msec): min=1, max=52, avg=21.07, stdev= 3.51
clat percentiles (usec):
| 1.00th=[16064], 5.00th=[16768], 10.00th=[17280], 20.00th=[18048],
| 30.00th=[18560], 40.00th=[19328], 50.00th=[20352], 60.00th=[21376],
| 70.00th=[22656], 80.00th=[23680], 90.00th=[25728], 95.00th=[27520],
| 99.00th=[31616], 99.50th=[34048], 99.90th=[38144], 99.95th=[39680],
| 99.99th=[46336]
bw (KB /s): min= 25, max=216064, per=97.07%, avg=190919.52, stdev=35567.09
write: io=3186.3MB, bw=194627KB/s, iops=759, runt= 16764msec
slat (usec): min=49, max=276, avg=74.21, stdev= 7.77
clat (msec): min=2, max=48, avg=20.74, stdev= 3.45
lat (msec): min=2, max=48, avg=20.81, stdev= 3.45
clat percentiles (usec):
| 1.00th=[15936], 5.00th=[16512], 10.00th=[17024], 20.00th=[17792],
| 30.00th=[18560], 40.00th=[19328], 50.00th=[20096], 60.00th=[21120],
| 70.00th=[22144], 80.00th=[23424], 90.00th=[25216], 95.00th=[27008],
| 99.00th=[31616], 99.50th=[33536], 99.90th=[37120], 99.95th=[38656],
| 99.99th=[45824]
bw (KB /s): min=170836, max=214528, per=100.00%, avg=194863.63, stdev=9515.49
lat (msec) : 2=0.01%, 4=0.01%, 10=0.02%, 20=48.07%, 50=52.01%
lat (msec) : 100=0.01%
cpu : usr=6.24%, sys=10.35%, ctx=40968, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=159.8%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=12863/w=12731/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=3220.0MB, aggrb=196688KB/s, minb=196688KB/s, maxb=196688KB/s, mint=16764msec, maxt=16764msec
WRITE: io=3186.3MB, aggrb=194626KB/s, minb=194626KB/s, maxb=194626KB/s, mint=16764msec, maxt=16764msec
Disk stats (read/write):
sdg: ios=20596/20366, merge=0/0, ticks=429809/421009, in_queue=851023, util=99.74%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdg-randrw-32-512K
===
mytest: (g=0): rw=randrw, bs=512K-512K/512K-512K/512K-512K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=14750: Sun Sep 15 12:18:10 2013
read : io=2994.6MB, bw=216139KB/s, iops=420, runt= 14187msec
slat (usec): min=42, max=128, avg=66.50, stdev= 5.88
clat (msec): min=3, max=73, avg=37.97, stdev= 4.66
lat (msec): min=3, max=73, avg=38.04, stdev= 4.66
clat percentiles (usec):
| 1.00th=[31872], 5.00th=[32640], 10.00th=[33024], 20.00th=[34048],
| 30.00th=[35072], 40.00th=[36096], 50.00th=[37120], 60.00th=[38144],
| 70.00th=[39680], 80.00th=[41728], 90.00th=[43776], 95.00th=[46336],
| 99.00th=[50944], 99.50th=[54528], 99.90th=[69120], 99.95th=[72192],
| 99.99th=[73216]
bw (KB /s): min= 51, max=237568, per=96.80%, avg=209220.00, stdev=42359.80
write: io=3004.0MB, bw=216825KB/s, iops=422, runt= 14187msec
slat (usec): min=90, max=345, avg=141.40, stdev=12.53
clat (msec): min=5, max=69, avg=37.57, stdev= 4.45
lat (msec): min=6, max=69, avg=37.71, stdev= 4.45
clat percentiles (usec):
| 1.00th=[31616], 5.00th=[32384], 10.00th=[33024], 20.00th=[33536],
| 30.00th=[34560], 40.00th=[36096], 50.00th=[36608], 60.00th=[37632],
| 70.00th=[39168], 80.00th=[41216], 90.00th=[43264], 95.00th=[45312],
| 99.00th=[49920], 99.50th=[53504], 99.90th=[63744], 99.95th=[66048],
| 99.99th=[69120]
bw (KB /s): min=201325, max=241664, per=99.85%, avg=216503.32, stdev=10755.95
lat (msec) : 4=0.01%, 10=0.05%, 20=0.08%, 50=98.99%, 100=1.14%
cpu : usr=6.82%, sys=10.45%, ctx=20489, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.3%, 32=170.6%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=5970/w=5996/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=2994.6MB, aggrb=216139KB/s, minb=216139KB/s, maxb=216139KB/s, mint=14187msec, maxt=14187msec
WRITE: io=3004.0MB, aggrb=216824KB/s, minb=216824KB/s, maxb=216824KB/s, mint=14187msec, maxt=14187msec
Disk stats (read/write):
sdg: ios=10253/10132, merge=0/0, ticks=383104/378678, in_queue=762377, util=99.68%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdg-randrw-32-1024K
===
mytest: (g=0): rw=randrw, bs=1M-1M/1M-1M/1M-1M, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=14824: Sun Sep 15 12:19:33 2013
read : io=2904.0MB, bw=231470KB/s, iops=224, runt= 12847msec
slat (usec): min=72, max=298, avg=119.19, stdev= 9.41
clat (msec): min=6, max=250, avg=71.70, stdev=22.56
lat (msec): min=6, max=250, avg=71.82, stdev=22.56
clat percentiles (msec):
| 1.00th=[ 36], 5.00th=[ 40], 10.00th=[ 44], 20.00th=[ 51],
| 30.00th=[ 58], 40.00th=[ 64], 50.00th=[ 71], 60.00th=[ 78],
| 70.00th=[ 84], 80.00th=[ 91], 90.00th=[ 101], 95.00th=[ 110],
| 99.00th=[ 128], 99.50th=[ 141], 99.90th=[ 223], 99.95th=[ 235],
| 99.99th=[ 251]
bw (KB /s): min= 102, max=265179, per=96.72%, avg=223887.40, stdev=50271.74
write: io=2849.0MB, bw=227086KB/s, iops=220, runt= 12847msec
slat (usec): min=186, max=480, avg=265.88, stdev=17.29
clat (msec): min=7, max=244, avg=71.12, stdev=22.41
lat (msec): min=8, max=244, avg=71.39, stdev=22.41
clat percentiles (msec):
| 1.00th=[ 35], 5.00th=[ 39], 10.00th=[ 43], 20.00th=[ 51],
| 30.00th=[ 57], 40.00th=[ 64], 50.00th=[ 71], 60.00th=[ 77],
| 70.00th=[ 83], 80.00th=[ 92], 90.00th=[ 100], 95.00th=[ 109],
| 99.00th=[ 124], 99.50th=[ 135], 99.90th=[ 221], 99.95th=[ 233],
| 99.99th=[ 245]
bw (KB /s): min=201944, max=246330, per=99.24%, avg=225365.75, stdev=13663.24
lat (msec) : 10=0.09%, 20=0.05%, 50=19.19%, 100=70.99%, 250=10.21%
lat (msec) : 500=0.02%
cpu : usr=6.66%, sys=10.15%, ctx=10248, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.3%, 16=0.6%, 32=177.9%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=2889/w=2833/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=2904.0MB, aggrb=231470KB/s, minb=231470KB/s, maxb=231470KB/s, mint=12847msec, maxt=12847msec
WRITE: io=2849.0MB, aggrb=227086KB/s, minb=227086KB/s, maxb=227086KB/s, mint=12847msec, maxt=12847msec
Disk stats (read/write):
sdg: ios=10440/9995, merge=0/0, ticks=730055/700870, in_queue=1433113, util=99.63%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdh-randrw-32-512B
===
mytest: (g=0): rw=randrw, bs=512-512/512-512/512-512, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=15291: Sun Sep 15 12:34:44 2013
read : io=5053.2MB, bw=6157.2KB/s, iops=12314, runt=840371msec
slat (usec): min=2, max=412, avg= 7.20, stdev= 4.54
clat (usec): min=0, max=44246, avg=2474.99, stdev=2342.20
lat (usec): min=36, max=44255, avg=2482.35, stdev=2342.24
clat percentiles (usec):
| 1.00th=[ 183], 5.00th=[ 394], 10.00th=[ 564], 20.00th=[ 820],
| 30.00th=[ 1032], 40.00th=[ 1240], 50.00th=[ 1512], 60.00th=[ 1960],
| 70.00th=[ 3024], 80.00th=[ 4128], 90.00th=[ 5792], 95.00th=[ 7008],
| 99.00th=[ 9408], 99.50th=[10688], 99.90th=[22144], 99.95th=[26240],
| 99.99th=[30080]
bw (KB /s): min= 0, max= 7449, per=100.00%, avg=6157.40, stdev=394.51
write: io=5052.6MB, bw=6155.2KB/s, iops=12311, runt=840371msec
slat (usec): min=2, max=560, avg= 8.51, stdev= 5.22
clat (usec): min=0, max=40565, avg=104.22, stdev=89.82
lat (usec): min=34, max=40571, avg=112.91, stdev=89.19
clat percentiles (usec):
| 1.00th=[ 31], 5.00th=[ 36], 10.00th=[ 37], 20.00th=[ 42],
| 30.00th=[ 50], 40.00th=[ 59], 50.00th=[ 71], 60.00th=[ 90],
| 70.00th=[ 115], 80.00th=[ 153], 90.00th=[ 223], 95.00th=[ 286],
| 99.00th=[ 390], 99.50th=[ 438], 99.90th=[ 548], 99.95th=[ 604],
| 99.99th=[ 1592]
bw (KB /s): min= 3163, max= 7225, per=100.00%, avg=6159.96, stdev=354.48
lat (usec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=14.84%
lat (usec) : 100=17.53%, 250=14.82%, 500=6.67%, 750=4.63%, 1000=5.73%
lat (msec) : 2=16.02%, 4=9.30%, 10=10.09%, 20=0.30%, 50=0.07%
cpu : usr=7.67%, sys=24.61%, ctx=8202807, majf=0, minf=28
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=101.3%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=10348554/w=10346613/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=5053.2MB, aggrb=6157KB/s, minb=6157KB/s, maxb=6157KB/s, mint=840371msec, maxt=840371msec
WRITE: io=5052.6MB, aggrb=6155KB/s, minb=6155KB/s, maxb=6155KB/s, mint=840371msec, maxt=840371msec
Disk stats (read/write):
sdh: ios=10486228/10483919, merge=0/0, ticks=25784617/1024426, in_queue=26801262, util=100.00%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdh-randrw-32-1K
===
mytest: (g=0): rw=randrw, bs=1K-1K/1K-1K/1K-1K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=16472: Sun Sep 15 12:42:28 2013
read : io=4991.5MB, bw=12979KB/s, iops=12979, runt=393791msec
slat (usec): min=2, max=353, avg= 7.11, stdev= 4.77
clat (usec): min=39, max=94705, avg=2358.11, stdev=2427.88
lat (usec): min=48, max=94719, avg=2365.34, stdev=2427.94
clat percentiles (usec):
| 1.00th=[ 223], 5.00th=[ 366], 10.00th=[ 490], 20.00th=[ 684],
| 30.00th=[ 860], 40.00th=[ 1048], 50.00th=[ 1272], 60.00th=[ 1688],
| 70.00th=[ 2960], 80.00th=[ 4128], 90.00th=[ 5728], 95.00th=[ 6880],
| 99.00th=[ 9280], 99.50th=[10688], 99.90th=[24448], 99.95th=[27520],
| 99.99th=[31104]
bw (KB /s): min= 8982, max=14094, per=100.00%, avg=12987.23, stdev=725.70
write: io=4990.2MB, bw=12976KB/s, iops=12975, runt=393791msec
slat (usec): min=2, max=610, avg= 8.35, stdev= 5.43
clat (usec): min=0, max=12604, avg=88.26, stdev=84.55
lat (usec): min=35, max=12610, avg=96.75, stdev=84.07
clat percentiles (usec):
| 1.00th=[ 31], 5.00th=[ 35], 10.00th=[ 37], 20.00th=[ 41],
| 30.00th=[ 47], 40.00th=[ 54], 50.00th=[ 62], 60.00th=[ 73],
| 70.00th=[ 88], 80.00th=[ 114], 90.00th=[ 183], 95.00th=[ 258],
| 99.00th=[ 338], 99.50th=[ 382], 99.90th=[ 486], 99.95th=[ 556],
| 99.99th=[ 2960]
bw (KB /s): min= 0, max=14097, per=99.94%, avg=12967.25, stdev=837.99
lat (usec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=17.16%
lat (usec) : 100=20.36%, 250=10.52%, 500=7.12%, 750=6.67%, 1000=7.00%
lat (msec) : 2=13.02%, 4=7.59%, 10=10.21%, 20=0.26%, 50=0.09%
lat (msec) : 100=0.01%
cpu : usr=7.20%, sys=25.31%, ctx=3182714, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=102.6%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=5111168/w=5109777/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4991.5MB, aggrb=12979KB/s, minb=12979KB/s, maxb=12979KB/s, mint=393791msec, maxt=393791msec
WRITE: io=4990.2MB, aggrb=12975KB/s, minb=12975KB/s, maxb=12975KB/s, mint=393791msec, maxt=393791msec
Disk stats (read/write):
sdh: ios=5241711/5239945, merge=0/0, ticks=12260102/413477, in_queue=12669740, util=100.00%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdh-randrw-32-2K
===
mytest: (g=0): rw=randrw, bs=2K-2K/2K-2K/2K-2K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=16929: Sun Sep 15 12:46:49 2013
read : io=4861.9MB, bw=26057KB/s, iops=13028, runt=191063msec
slat (usec): min=2, max=391, avg= 7.39, stdev= 4.68
clat (usec): min=94, max=161197, avg=2335.77, stdev=2494.78
lat (usec): min=109, max=161201, avg=2343.23, stdev=2494.82
clat percentiles (usec):
| 1.00th=[ 227], 5.00th=[ 366], 10.00th=[ 482], 20.00th=[ 660],
| 30.00th=[ 836], 40.00th=[ 1032], 50.00th=[ 1256], 60.00th=[ 1656],
| 70.00th=[ 2928], 80.00th=[ 4128], 90.00th=[ 5728], 95.00th=[ 6816],
| 99.00th=[ 9152], 99.50th=[10560], 99.90th=[24192], 99.95th=[27520],
| 99.99th=[31360]
bw (KB /s): min= 0, max=28383, per=99.90%, avg=26030.00, stdev=1979.09
write: io=4859.5MB, bw=26044KB/s, iops=13022, runt=191063msec
slat (usec): min=3, max=1191, avg= 8.94, stdev= 5.41
clat (usec): min=0, max=12467, avg=100.58, stdev=99.78
lat (usec): min=37, max=12474, avg=109.64, stdev=99.38
clat percentiles (usec):
| 1.00th=[ 34], 5.00th=[ 39], 10.00th=[ 40], 20.00th=[ 46],
| 30.00th=[ 52], 40.00th=[ 61], 50.00th=[ 72], 60.00th=[ 86],
| 70.00th=[ 106], 80.00th=[ 141], 90.00th=[ 209], 95.00th=[ 262],
| 99.00th=[ 354], 99.50th=[ 398], 99.90th=[ 532], 99.95th=[ 884],
| 99.99th=[ 3536]
bw (KB /s): min=17425, max=28135, per=100.00%, avg=26085.83, stdev=1448.25
lat (usec) : 2=0.01%, 10=0.01%, 20=0.01%, 50=12.91%, 100=20.71%
lat (usec) : 250=14.06%, 500=7.74%, 750=7.09%, 1000=6.82%
lat (msec) : 2=12.68%, 4=7.54%, 10=10.14%, 20=0.24%, 50=0.08%
lat (msec) : 100=0.01%, 250=0.01%
cpu : usr=7.85%, sys=26.75%, ctx=1524980, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=105.3%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=2489220/w=2488048/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4861.9MB, aggrb=26056KB/s, minb=26056KB/s, maxb=26056KB/s, mint=191063msec, maxt=191063msec
WRITE: io=4859.5MB, aggrb=26044KB/s, minb=26044KB/s, maxb=26044KB/s, mint=191063msec, maxt=191063msec
Disk stats (read/write):
sdh: ios=2620320/2618797, merge=0/0, ticks=6066717/239920, in_queue=6304670, util=100.00%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdh-randrw-32-4K
===
mytest: (g=0): rw=randrw, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=17511: Sun Sep 15 12:49:11 2013
read : io=4488.2MB, bw=64426KB/s, iops=16106, runt= 71336msec
slat (usec): min=2, max=240, avg= 7.10, stdev= 3.83
clat (usec): min=117, max=25642, avg=1694.80, stdev=1478.06
lat (usec): min=125, max=25646, avg=1701.78, stdev=1477.98
clat percentiles (usec):
| 1.00th=[ 318], 5.00th=[ 418], 10.00th=[ 474], 20.00th=[ 540],
| 30.00th=[ 596], 40.00th=[ 700], 50.00th=[ 1048], 60.00th=[ 1720],
| 70.00th=[ 2672], 80.00th=[ 3056], 90.00th=[ 3536], 95.00th=[ 3792],
| 99.00th=[ 4320], 99.50th=[ 5088], 99.90th=[18304], 99.95th=[20352],
| 99.99th=[22912]
bw (KB /s): min=59320, max=70176, per=100.00%, avg=64474.15, stdev=1863.31
write: io=4486.6MB, bw=64403KB/s, iops=16100, runt= 71336msec
slat (usec): min=2, max=417, avg= 8.57, stdev= 4.19
clat (usec): min=21, max=10165, avg=273.69, stdev=148.30
lat (usec): min=43, max=10170, avg=282.18, stdev=147.73
clat percentiles (usec):
| 1.00th=[ 43], 5.00th=[ 58], 10.00th=[ 83], 20.00th=[ 145],
| 30.00th=[ 207], 40.00th=[ 262], 50.00th=[ 302], 60.00th=[ 330],
| 70.00th=[ 354], 80.00th=[ 378], 90.00th=[ 406], 95.00th=[ 426],
| 99.00th=[ 462], 99.50th=[ 474], 99.90th=[ 2064], 99.95th=[ 2672],
| 99.99th=[ 3056]
bw (KB /s): min= 0, max=68192, per=99.37%, avg=63998.00, stdev=5653.71
lat (usec) : 50=1.68%, 100=4.72%, 250=12.53%, 500=37.56%, 750=14.61%
lat (usec) : 1000=3.46%
lat (msec) : 2=6.63%, 4=17.57%, 10=1.12%, 20=0.10%, 50=0.03%
cpu : usr=9.56%, sys=34.03%, ctx=914050, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=114.1%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=1148951/w=1148550/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4488.2MB, aggrb=64425KB/s, minb=64425KB/s, maxb=64425KB/s, mint=71336msec, maxt=71336msec
WRITE: io=4486.6MB, aggrb=64402KB/s, minb=64402KB/s, maxb=64402KB/s, mint=71336msec, maxt=71336msec
Disk stats (read/write):
sdh: ios=1309502/1308467, merge=0/0, ticks=2210120/358730, in_queue=2567920, util=99.96%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdh-randrw-32-8K
===
mytest: (g=0): rw=randrw, bs=8K-8K/8K-8K/8K-8K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=17646: Sun Sep 15 12:51:14 2013
read : io=4281.9MB, bw=83293KB/s, iops=10410, runt= 52641msec
slat (usec): min=2, max=392, avg= 8.36, stdev= 5.12
clat (usec): min=134, max=34488, avg=2741.62, stdev=2010.57
lat (usec): min=149, max=34497, avg=2749.95, stdev=2010.42
clat percentiles (usec):
| 1.00th=[ 466], 5.00th=[ 636], 10.00th=[ 716], 20.00th=[ 836],
| 30.00th=[ 1064], 40.00th=[ 1880], 50.00th=[ 2960], 60.00th=[ 3408],
| 70.00th=[ 3760], 80.00th=[ 4128], 90.00th=[ 4576], 95.00th=[ 5088],
| 99.00th=[ 7712], 99.50th=[11200], 99.90th=[22144], 99.95th=[23680],
| 99.99th=[26752]
bw (KB /s): min= 0, max=90320, per=99.29%, avg=82699.87, stdev=9184.97
write: io=4277.9MB, bw=83200KB/s, iops=10400, runt= 52641msec
slat (usec): min=3, max=577, avg=10.41, stdev= 5.78
clat (usec): min=0, max=11459, avg=310.09, stdev=225.07
lat (usec): min=50, max=11469, avg=320.48, stdev=224.48
clat percentiles (usec):
| 1.00th=[ 49], 5.00th=[ 53], 10.00th=[ 66], 20.00th=[ 102],
| 30.00th=[ 153], 40.00th=[ 219], 50.00th=[ 294], 60.00th=[ 370],
| 70.00th=[ 438], 80.00th=[ 502], 90.00th=[ 556], 95.00th=[ 596],
| 99.00th=[ 660], 99.50th=[ 692], 99.90th=[ 2640], 99.95th=[ 2896],
| 99.99th=[ 3984]
bw (KB /s): min=68167, max=90208, per=100.00%, avg=83427.04, stdev=4231.56
lat (usec) : 2=0.01%, 10=0.01%, 20=0.01%, 50=0.82%, 100=8.80%
lat (usec) : 250=12.40%, 500=18.65%, 750=15.37%, 1000=7.91%
lat (msec) : 2=6.37%, 4=18.20%, 10=11.20%, 20=0.18%, 50=0.10%
cpu : usr=11.24%, sys=27.12%, ctx=831524, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=119.6%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=548045/w=547467/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4281.9MB, aggrb=83292KB/s, minb=83292KB/s, maxb=83292KB/s, mint=52641msec, maxt=52641msec
WRITE: io=4277.9MB, aggrb=83200KB/s, minb=83200KB/s, maxb=83200KB/s, mint=52641msec, maxt=52641msec
Disk stats (read/write):
sdh: ios=654174/653620, merge=0/0, ticks=1778992/203613, in_queue=1982071, util=99.95%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdh-randrw-32-16K
===
mytest: (g=0): rw=randrw, bs=16K-16K/16K-16K/16K-16K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=17775: Sun Sep 15 12:53:23 2013
read : io=4308.2MB, bw=75718KB/s, iops=4731, runt= 58262msec
slat (usec): min=5, max=217, avg=10.30, stdev= 2.01
clat (usec): min=233, max=56689, avg=6400.59, stdev=4166.20
lat (usec): min=243, max=56699, avg=6411.12, stdev=4166.24
clat percentiles (usec):
| 1.00th=[ 732], 5.00th=[ 1080], 10.00th=[ 1432], 20.00th=[ 3088],
| 30.00th=[ 4128], 40.00th=[ 4832], 50.00th=[ 5600], 60.00th=[ 6624],
| 70.00th=[ 7904], 80.00th=[ 9536], 90.00th=[11840], 95.00th=[13632],
| 99.00th=[18304], 99.50th=[23936], 99.90th=[33024], 99.95th=[35584],
| 99.99th=[41216]
bw (KB /s): min= 1, max=99136, per=99.25%, avg=75147.62, stdev=9828.76
write: io=4295.7MB, bw=75499KB/s, iops=4718, runt= 58262msec
slat (usec): min=8, max=916, avg=13.60, stdev= 3.16
clat (usec): min=36, max=18046, avg=333.88, stdev=418.67
lat (usec): min=77, max=18057, avg=347.84, stdev=418.29
clat percentiles (usec):
| 1.00th=[ 66], 5.00th=[ 69], 10.00th=[ 79], 20.00th=[ 97],
| 30.00th=[ 125], 40.00th=[ 159], 50.00th=[ 205], 60.00th=[ 274],
| 70.00th=[ 374], 80.00th=[ 516], 90.00th=[ 716], 95.00th=[ 884],
| 99.00th=[ 2160], 99.50th=[ 2704], 99.90th=[ 5024], 99.95th=[ 5920],
| 99.99th=[ 9280]
bw (KB /s): min=62368, max=99136, per=100.00%, avg=75591.89, stdev=6920.10
lat (usec) : 50=0.01%, 100=10.45%, 250=17.95%, 500=11.19%, 750=6.48%
lat (usec) : 1000=4.50%
lat (msec) : 2=6.24%, 4=7.35%, 10=27.07%, 20=8.37%, 50=0.40%
lat (msec) : 100=0.01%
cpu : usr=8.32%, sys=15.61%, ctx=606563, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=119.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=275688/w=274921/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4308.2MB, aggrb=75718KB/s, minb=75718KB/s, maxb=75718KB/s, mint=58262msec, maxt=58262msec
WRITE: io=4295.7MB, aggrb=75499KB/s, minb=75499KB/s, maxb=75499KB/s, mint=58262msec, maxt=58262msec
Disk stats (read/write):
sdh: ios=328067/327074, merge=0/0, ticks=2063513/114109, in_queue=2177764, util=99.96%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdh-randrw-32-32K
===
mytest: (g=0): rw=randrw, bs=32K-32K/32K-32K/32K-32K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=17953: Sun Sep 15 12:55:32 2013
read : io=4372.3MB, bw=76678KB/s, iops=2395, runt= 58389msec
slat (usec): min=8, max=505, avg=12.10, stdev= 3.28
clat (usec): min=384, max=66242, avg=12738.40, stdev=7012.89
lat (usec): min=396, max=66253, avg=12750.82, stdev=7012.83
clat percentiles (usec):
| 1.00th=[ 1512], 5.00th=[ 3152], 10.00th=[ 4704], 20.00th=[ 6624],
| 30.00th=[ 8256], 40.00th=[ 9920], 50.00th=[11712], 60.00th=[13760],
| 70.00th=[15936], 80.00th=[18304], 90.00th=[21888], 95.00th=[24960],
| 99.00th=[32384], 99.50th=[37632], 99.90th=[49920], 99.95th=[54016],
| 99.99th=[60160]
bw (KB /s): min= 3, max=88192, per=99.23%, avg=76086.73, stdev=7822.86
write: io=4372.9MB, bw=76690KB/s, iops=2396, runt= 58389msec
slat (usec): min=11, max=926, avg=19.39, stdev= 6.17
clat (usec): min=15, max=30667, avg=581.89, stdev=1186.45
lat (usec): min=112, max=30683, avg=602.02, stdev=1186.14
clat percentiles (usec):
| 1.00th=[ 95], 5.00th=[ 99], 10.00th=[ 112], 20.00th=[ 135],
| 30.00th=[ 171], 40.00th=[ 211], 50.00th=[ 266], 60.00th=[ 338],
| 70.00th=[ 454], 80.00th=[ 644], 90.00th=[ 1048], 95.00th=[ 1880],
| 99.00th=[ 6432], 99.50th=[ 8384], 99.90th=[13248], 99.95th=[15296],
| 99.99th=[20096]
bw (KB /s): min=71424, max=86592, per=100.00%, avg=76760.45, stdev=2770.99
lat (usec) : 20=0.01%, 50=0.01%, 100=2.68%, 250=20.90%, 500=12.89%
lat (usec) : 750=5.39%, 1000=2.83%
lat (msec) : 2=4.04%, 4=3.77%, 10=17.58%, 20=22.40%, 50=7.47%
lat (msec) : 100=0.05%
cpu : usr=5.76%, sys=9.00%, ctx=320064, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=117.1%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=139880/w=139932/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4372.3MB, aggrb=76678KB/s, minb=76678KB/s, maxb=76678KB/s, mint=58389msec, maxt=58389msec
WRITE: io=4372.9MB, aggrb=76689KB/s, minb=76689KB/s, maxb=76689KB/s, mint=58389msec, maxt=58389msec
Disk stats (read/write):
sdh: ios=163552/163429, merge=0/0, ticks=2080360/97611, in_queue=2178129, util=99.95%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdh-randrw-32-64K
===
mytest: (g=0): rw=randrw, bs=64K-64K/64K-64K/64K-64K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=18082: Sun Sep 15 12:57:36 2013
read : io=4335.0MB, bw=82684KB/s, iops=1291, runt= 53687msec
slat (usec): min=10, max=211, avg=14.65, stdev= 2.52
clat (usec): min=786, max=85171, avg=22261.03, stdev=11224.51
lat (usec): min=799, max=85186, avg=22275.90, stdev=11224.42
clat percentiles (usec):
| 1.00th=[ 3824], 5.00th=[ 7008], 10.00th=[ 9152], 20.00th=[12352],
| 30.00th=[15168], 40.00th=[18048], 50.00th=[20864], 60.00th=[23680],
| 70.00th=[27008], 80.00th=[31104], 90.00th=[37632], 95.00th=[43264],
| 99.00th=[54528], 99.50th=[58112], 99.90th=[68096], 99.95th=[71168],
| 99.99th=[80384]
bw (KB /s): min= 6, max=92231, per=99.08%, avg=81925.02, stdev=8623.43
write: io=4336.4MB, bw=82709KB/s, iops=1292, runt= 53687msec
slat (usec): min=14, max=1419, avg=24.65, stdev= 6.12
clat (usec): min=147, max=54358, avg=2468.94, stdev=4703.24
lat (usec): min=182, max=54381, avg=2494.02, stdev=4703.25
clat percentiles (usec):
| 1.00th=[ 163], 5.00th=[ 183], 10.00th=[ 193], 20.00th=[ 253],
| 30.00th=[ 314], 40.00th=[ 398], 50.00th=[ 524], 60.00th=[ 740],
| 70.00th=[ 1176], 80.00th=[ 3024], 90.00th=[ 8032], 95.00th=[12480],
| 99.00th=[22656], 99.50th=[27008], 99.90th=[35584], 99.95th=[40192],
| 99.99th=[48384]
bw (KB /s): min=76544, max=90570, per=100.00%, avg=82833.44, stdev=2826.78
lat (usec) : 250=9.52%, 500=14.22%, 750=6.46%, 1000=3.36%
lat (msec) : 2=4.38%, 4=4.00%, 10=10.52%, 20=20.40%, 50=26.17%
lat (msec) : 100=0.99%
cpu : usr=3.80%, sys=5.59%, ctx=163752, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=118.1%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=69330/w=69380/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4335.0MB, aggrb=82683KB/s, minb=82683KB/s, maxb=82683KB/s, mint=53687msec, maxt=53687msec
WRITE: io=4336.4MB, aggrb=82708KB/s, minb=82708KB/s, maxb=82708KB/s, mint=53687msec, maxt=53687msec
Disk stats (read/write):
sdh: ios=81746/81657, merge=0/0, ticks=1824874/203680, in_queue=2028687, util=99.95%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdh-randrw-32-128K
===
mytest: (g=0): rw=randrw, bs=128K-128K/128K-128K/128K-128K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=18211: Sun Sep 15 12:59:33 2013
read : io=4279.9MB, bw=93253KB/s, iops=727, runt= 46997msec
slat (usec): min=14, max=360, avg=24.80, stdev= 5.76
clat (usec): min=672, max=141982, avg=35084.73, stdev=17533.82
lat (usec): min=698, max=142005, avg=35109.78, stdev=17533.63
clat percentiles (msec):
| 1.00th=[ 6], 5.00th=[ 11], 10.00th=[ 15], 20.00th=[ 20],
| 30.00th=[ 25], 40.00th=[ 29], 50.00th=[ 34], 60.00th=[ 38],
| 70.00th=[ 43], 80.00th=[ 49], 90.00th=[ 59], 95.00th=[ 68],
| 99.00th=[ 87], 99.50th=[ 95], 99.90th=[ 111], 99.95th=[ 118],
| 99.99th=[ 135]
bw (KB /s): min= 12, max=107682, per=98.98%, avg=92298.83, stdev=11936.60
write: io=4279.8MB, bw=93250KB/s, iops=728, runt= 46997msec
slat (usec): min=22, max=3972, avg=44.18, stdev=21.34
clat (usec): min=284, max=45470, avg=8786.59, stdev=5647.27
lat (usec): min=321, max=45505, avg=8831.31, stdev=5648.21
clat percentiles (usec):
| 1.00th=[ 330], 5.00th=[ 700], 10.00th=[ 1224], 20.00th=[ 3088],
| 30.00th=[ 4960], 40.00th=[ 6816], 50.00th=[ 8512], 60.00th=[10176],
| 70.00th=[11840], 80.00th=[13888], 90.00th=[16512], 95.00th=[18560],
| 99.00th=[21888], 99.50th=[22912], 99.90th=[26496], 99.95th=[30592],
| 99.99th=[40192]
bw (KB /s): min=78080, max=108838, per=100.00%, avg=93405.17, stdev=6796.75
lat (usec) : 500=1.32%, 750=1.61%, 1000=1.21%
lat (msec) : 2=3.14%, 4=5.34%, 10=18.81%, 20=27.59%, 50=31.87%
lat (msec) : 100=9.01%, 250=0.13%
cpu : usr=3.23%, sys=5.36%, ctx=81860, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=119.6%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=34208/w=34238/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4279.9MB, aggrb=93252KB/s, minb=93252KB/s, maxb=93252KB/s, mint=46997msec, maxt=46997msec
WRITE: io=4279.8MB, aggrb=93249KB/s, minb=93249KB/s, maxb=93249KB/s, mint=46997msec, maxt=46997msec
Disk stats (read/write):
sdh: ios=41078/40934, merge=0/0, ticks=1488769/330231, in_queue=1819382, util=99.94%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdh-randrw-32-256K
===
mytest: (g=0): rw=randrw, bs=256K-256K/256K-256K/256K-256K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=18344: Sun Sep 15 13:01:09 2013
read : io=3721.3MB, bw=150342KB/s, iops=586, runt= 25346msec
slat (usec): min=22, max=232, avg=36.93, stdev= 5.16
clat (msec): min=2, max=85, avg=52.40, stdev= 8.12
lat (msec): min=3, max=85, avg=52.44, stdev= 8.12
clat percentiles (usec):
| 1.00th=[35584], 5.00th=[40192], 10.00th=[42240], 20.00th=[45312],
| 30.00th=[47872], 40.00th=[49920], 50.00th=[51968], 60.00th=[54016],
| 70.00th=[56064], 80.00th=[59136], 90.00th=[63232], 95.00th=[66048],
| 99.00th=[73216], 99.50th=[75264], 99.90th=[79360], 99.95th=[80384],
| 99.99th=[83456]
bw (KB /s): min= 25, max=163840, per=97.91%, avg=147203.84, stdev=22155.77
write: io=3683.0MB, bw=148796KB/s, iops=581, runt= 25346msec
slat (usec): min=43, max=6221, avg=75.59, stdev=43.85
clat (usec): min=539, max=39991, avg=2032.28, stdev=1588.34
lat (usec): min=610, max=40053, avg=2108.18, stdev=1588.34
clat percentiles (usec):
| 1.00th=[ 572], 5.00th=[ 604], 10.00th=[ 972], 20.00th=[ 1080],
| 30.00th=[ 1432], 40.00th=[ 1512], 50.00th=[ 1864], 60.00th=[ 1992],
| 70.00th=[ 2384], 80.00th=[ 2544], 90.00th=[ 3024], 95.00th=[ 4048],
| 99.00th=[ 7200], 99.50th=[ 8640], 99.90th=[21888], 99.95th=[29312],
| 99.99th=[36608]
bw (KB /s): min=130810, max=165376, per=100.00%, avg=149094.36, stdev=7132.55
lat (usec) : 750=3.80%, 1000=2.28%
lat (msec) : 2=23.82%, 4=17.30%, 10=2.45%, 20=0.10%, 50=20.13%
lat (msec) : 100=30.23%
cpu : usr=4.42%, sys=6.90%, ctx=40966, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=138.2%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=14854/w=14732/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=3721.3MB, aggrb=150341KB/s, minb=150341KB/s, maxb=150341KB/s, mint=25346msec, maxt=25346msec
WRITE: io=3683.0MB, aggrb=148796KB/s, minb=148796KB/s, maxb=148796KB/s, mint=25346msec, maxt=25346msec
Disk stats (read/write):
sdh: ios=20573/20343, merge=0/0, ticks=1083995/38423, in_queue=1123354, util=99.83%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdh-randrw-32-512K
===
mytest: (g=0): rw=randrw, bs=512K-512K/512K-512K/512K-512K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=18441: Sun Sep 15 13:02:43 2013
read : io=3591.0MB, bw=156810KB/s, iops=305, runt= 23450msec
slat (usec): min=42, max=151, avg=66.59, stdev= 6.84
clat (msec): min=3, max=142, avg=79.93, stdev=15.38
lat (msec): min=3, max=142, avg=80.00, stdev=15.38
clat percentiles (msec):
| 1.00th=[ 51], 5.00th=[ 57], 10.00th=[ 61], 20.00th=[ 68],
| 30.00th=[ 73], 40.00th=[ 77], 50.00th=[ 80], 60.00th=[ 83],
| 70.00th=[ 87], 80.00th=[ 92], 90.00th=[ 99], 95.00th=[ 106],
| 99.00th=[ 124], 99.50th=[ 130], 99.90th=[ 137], 99.95th=[ 139],
| 99.99th=[ 143]
bw (KB /s): min=130048, max=178486, per=100.00%, avg=156892.87, stdev=9332.66
write: io=3634.6MB, bw=158709KB/s, iops=309, runt= 23450msec
slat (usec): min=90, max=46064, avg=147.01, stdev=455.00
clat (msec): min=1, max=87, avg=24.14, stdev=13.45
lat (msec): min=1, max=87, avg=24.29, stdev=13.45
clat percentiles (usec):
| 1.00th=[ 2096], 5.00th=[ 5216], 10.00th=[ 7648], 20.00th=[11200],
| 30.00th=[14528], 40.00th=[18304], 50.00th=[22400], 60.00th=[27264],
| 70.00th=[31872], 80.00th=[37120], 90.00th=[43776], 95.00th=[47872],
| 99.00th=[51968], 99.50th=[54016], 99.90th=[69120], 99.95th=[81408],
| 99.99th=[87552]
bw (KB /s): min= 51, max=184974, per=97.87%, avg=155332.06, stdev=25928.45
lat (msec) : 2=0.28%, 4=1.47%, 10=6.28%, 20=14.33%, 50=27.25%
lat (msec) : 100=46.05%, 250=4.56%
cpu : usr=4.04%, sys=6.50%, ctx=20462, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.2%, 32=141.6%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=7162/w=7258/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=3591.0MB, aggrb=156809KB/s, minb=156809KB/s, maxb=156809KB/s, mint=23450msec, maxt=23450msec
WRITE: io=3634.6MB, aggrb=158709KB/s, minb=158709KB/s, maxb=158709KB/s, mint=23450msec, maxt=23450msec
Disk stats (read/write):
sdh: ios=10283/10163, merge=0/0, ticks=825988/231945, in_queue=1058863, util=99.81%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdh-randrw-32-1024K
===
mytest: (g=0): rw=randrw, bs=1M-1M/1M-1M/1M-1M, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=18561: Sun Sep 15 13:04:15 2013
read : io=3544.0MB, bw=168511KB/s, iops=163, runt= 21536msec
slat (usec): min=76, max=336, avg=120.06, stdev= 8.83
clat (msec): min=6, max=284, avg=124.59, stdev=32.25
lat (msec): min=6, max=285, avg=124.71, stdev=32.25
clat percentiles (msec):
| 1.00th=[ 64], 5.00th=[ 76], 10.00th=[ 84], 20.00th=[ 97],
| 30.00th=[ 105], 40.00th=[ 115], 50.00th=[ 123], 60.00th=[ 133],
| 70.00th=[ 141], 80.00th=[ 153], 90.00th=[ 167], 95.00th=[ 180],
| 99.00th=[ 206], 99.50th=[ 221], 99.90th=[ 277], 99.95th=[ 285],
| 99.99th=[ 285]
bw (KB /s): min= 102, max=190083, per=97.27%, avg=163911.65, stdev=28635.25
write: io=3476.0MB, bw=165278KB/s, iops=160, runt= 21536msec
slat (usec): min=186, max=95124, avg=288.24, stdev=1336.50
clat (msec): min=4, max=223, avg=71.20, stdev=31.78
lat (msec): min=4, max=223, avg=71.50, stdev=31.77
clat percentiles (msec):
| 1.00th=[ 14], 5.00th=[ 23], 10.00th=[ 32], 20.00th=[ 43],
| 30.00th=[ 52], 40.00th=[ 61], 50.00th=[ 70], 60.00th=[ 80],
| 70.00th=[ 89], 80.00th=[ 100], 90.00th=[ 113], 95.00th=[ 125],
| 99.00th=[ 147], 99.50th=[ 161], 99.90th=[ 200], 99.95th=[ 223],
| 99.99th=[ 223]
bw (KB /s): min=139264, max=223232, per=100.00%, avg=165668.57, stdev=15901.91
lat (msec) : 10=0.24%, 20=1.69%, 50=12.31%, 100=37.77%, 250=48.36%
lat (msec) : 500=0.07%
cpu : usr=4.11%, sys=6.08%, ctx=10227, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.2%, 16=0.5%, 32=145.6%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=3523/w=3466/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=3544.0MB, aggrb=168511KB/s, minb=168511KB/s, maxb=168511KB/s, mint=21536msec, maxt=21536msec
WRITE: io=3476.0MB, aggrb=165277KB/s, minb=165277KB/s, maxb=165277KB/s, mint=21536msec, maxt=21536msec
Disk stats (read/write):
sdh: ios=10424/10054, merge=0/0, ticks=1277364/690870, in_queue=1973552, util=99.78%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdi-randrw-32-512B
===
mytest: (g=0): rw=randrw, bs=512-512/512-512/512-512, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=18638: Sun Sep 15 13:18:13 2013
read : io=5043.6MB, bw=6732.9KB/s, iops=13465, runt=767077msec
slat (usec): min=2, max=360, avg= 7.62, stdev= 3.98
clat (usec): min=1, max=35595, avg=2255.76, stdev=2040.18
lat (usec): min=37, max=35603, avg=2263.56, stdev=2040.20
clat percentiles (usec):
| 1.00th=[ 177], 5.00th=[ 382], 10.00th=[ 564], 20.00th=[ 836],
| 30.00th=[ 1064], 40.00th=[ 1304], 50.00th=[ 1560], 60.00th=[ 1928],
| 70.00th=[ 2512], 80.00th=[ 3472], 90.00th=[ 5024], 95.00th=[ 6240],
| 99.00th=[ 8640], 99.50th=[10304], 99.90th=[19328], 99.95th=[21120],
| 99.99th=[24704]
bw (KB /s): min= 0, max= 8495, per=100.00%, avg=6732.37, stdev=375.53
write: io=5042.8MB, bw=6731.8KB/s, iops=13463, runt=767077msec
slat (usec): min=2, max=3122, avg= 8.93, stdev= 4.71
clat (usec): min=0, max=10912, avg=100.29, stdev=84.24
lat (usec): min=35, max=10916, avg=109.41, stdev=83.63
clat percentiles (usec):
| 1.00th=[ 32], 5.00th=[ 36], 10.00th=[ 38], 20.00th=[ 41],
| 30.00th=[ 49], 40.00th=[ 57], 50.00th=[ 67], 60.00th=[ 83],
| 70.00th=[ 108], 80.00th=[ 145], 90.00th=[ 217], 95.00th=[ 278],
| 99.00th=[ 394], 99.50th=[ 438], 99.90th=[ 556], 99.95th=[ 604],
| 99.99th=[ 796]
bw (KB /s): min= 4374, max= 8113, per=100.00%, avg=6735.73, stdev=327.40
lat (usec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=15.38%
lat (usec) : 100=18.40%, 250=13.76%, 500=6.42%, 750=4.31%, 1000=5.15%
lat (msec) : 2=17.35%, 4=11.20%, 10=7.74%, 20=0.23%, 50=0.04%
cpu : usr=9.29%, sys=28.78%, ctx=9395900, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=101.5%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=10329172/w=10327514/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=5043.6MB, aggrb=6732KB/s, minb=6732KB/s, maxb=6732KB/s, mint=767077msec, maxt=767077msec
WRITE: io=5042.8MB, aggrb=6731KB/s, minb=6731KB/s, maxb=6731KB/s, mint=767077msec, maxt=767077msec
Disk stats (read/write):
sdi: ios=10485056/10482751, merge=0/0, ticks=23534428/1049173, in_queue=24577520, util=100.00%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdi-randrw-32-1K
===
mytest: (g=0): rw=randrw, bs=1K-1K/1K-1K/1K-1K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=19766: Sun Sep 15 13:25:21 2013
read : io=4980.5MB, bw=14257KB/s, iops=14256, runt=357724msec
slat (usec): min=2, max=316, avg= 7.53, stdev= 4.33
clat (usec): min=30, max=34056, avg=2141.28, stdev=2040.46
lat (usec): min=43, max=34060, avg=2148.96, stdev=2040.51
clat percentiles (usec):
| 1.00th=[ 217], 5.00th=[ 378], 10.00th=[ 524], 20.00th=[ 748],
| 30.00th=[ 940], 40.00th=[ 1144], 50.00th=[ 1384], 60.00th=[ 1736],
| 70.00th=[ 2352], 80.00th=[ 3376], 90.00th=[ 4960], 95.00th=[ 6112],
| 99.00th=[ 8384], 99.50th=[10304], 99.90th=[19328], 99.95th=[21120],
| 99.99th=[25472]
bw (KB /s): min= 0, max=15310, per=99.93%, avg=14246.11, stdev=757.67
write: io=4979.8MB, bw=14253KB/s, iops=14252, runt=357724msec
slat (usec): min=2, max=780, avg= 8.87, stdev= 5.01
clat (usec): min=0, max=12499, avg=83.18, stdev=73.23
lat (usec): min=36, max=12512, avg=92.20, stdev=72.67
clat percentiles (usec):
| 1.00th=[ 31], 5.00th=[ 36], 10.00th=[ 38], 20.00th=[ 41],
| 30.00th=[ 45], 40.00th=[ 51], 50.00th=[ 58], 60.00th=[ 67],
| 70.00th=[ 82], 80.00th=[ 106], 90.00th=[ 165], 95.00th=[ 245],
| 99.00th=[ 334], 99.50th=[ 370], 99.90th=[ 490], 99.95th=[ 556],
| 99.99th=[ 1320]
bw (KB /s): min=11290, max=15180, per=100.00%, avg=14261.99, stdev=510.80
lat (usec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=19.07%
lat (usec) : 100=19.92%, 250=9.50%, 500=6.05%, 750=5.53%, 1000=6.39%
lat (msec) : 2=15.95%, 4=9.81%, 10=7.52%, 20=0.23%, 50=0.04%
cpu : usr=8.87%, sys=30.26%, ctx=3939990, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=102.8%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=5099938/w=5098572/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4980.5MB, aggrb=14256KB/s, minb=14256KB/s, maxb=14256KB/s, mint=357724msec, maxt=357724msec
WRITE: io=4979.8MB, aggrb=14252KB/s, minb=14252KB/s, maxb=14252KB/s, mint=357724msec, maxt=357724msec
Disk stats (read/write):
sdi: ios=5241499/5239702, merge=0/0, ticks=11163227/415980, in_queue=11575844, util=100.00%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdi-randrw-32-2K
===
mytest: (g=0): rw=randrw, bs=2K-2K/2K-2K/2K-2K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=20169: Sun Sep 15 13:29:24 2013
read : io=4838.3MB, bw=28640KB/s, iops=14319, runt=172988msec
slat (usec): min=2, max=324, avg= 7.61, stdev= 3.93
clat (usec): min=33, max=32974, avg=2125.58, stdev=2042.67
lat (usec): min=48, max=32979, avg=2133.30, stdev=2042.71
clat percentiles (usec):
| 1.00th=[ 221], 5.00th=[ 378], 10.00th=[ 516], 20.00th=[ 732],
| 30.00th=[ 924], 40.00th=[ 1128], 50.00th=[ 1368], 60.00th=[ 1720],
| 70.00th=[ 2320], 80.00th=[ 3344], 90.00th=[ 4960], 95.00th=[ 6048],
| 99.00th=[ 8384], 99.50th=[10304], 99.90th=[19328], 99.95th=[21120],
| 99.99th=[25984]
bw (KB /s): min= 0, max=30632, per=99.78%, avg=28576.89, stdev=1896.21
write: io=4836.8MB, bw=28627KB/s, iops=14313, runt=172988msec
slat (usec): min=2, max=1670, avg= 8.94, stdev= 4.64
clat (usec): min=4, max=12173, avg=88.97, stdev=81.06
lat (usec): min=39, max=12175, avg=98.06, stdev=80.54
clat percentiles (usec):
| 1.00th=[ 35], 5.00th=[ 40], 10.00th=[ 41], 20.00th=[ 45],
| 30.00th=[ 50], 40.00th=[ 56], 50.00th=[ 64], 60.00th=[ 74],
| 70.00th=[ 89], 80.00th=[ 115], 90.00th=[ 177], 95.00th=[ 243],
| 99.00th=[ 330], 99.50th=[ 370], 99.90th=[ 506], 99.95th=[ 732],
| 99.99th=[ 1736]
bw (KB /s): min=23288, max=30620, per=100.00%, avg=28646.95, stdev=1100.15
lat (usec) : 10=0.01%, 20=0.01%, 50=14.86%, 100=22.57%, 250=11.14%
lat (usec) : 500=6.03%, 750=5.74%, 1000=6.52%
lat (msec) : 2=15.76%, 4=9.64%, 10=7.48%, 20=0.22%, 50=0.04%
cpu : usr=9.24%, sys=31.15%, ctx=1981178, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=105.8%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=2477159/w=2476068/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4838.3MB, aggrb=28640KB/s, minb=28640KB/s, maxb=28640KB/s, mint=172988msec, maxt=172988msec
WRITE: io=4836.8MB, aggrb=28627KB/s, minb=28627KB/s, maxb=28627KB/s, mint=172988msec, maxt=172988msec
Disk stats (read/write):
sdi: ios=2620130/2618589, merge=0/0, ticks=5547836/233089, in_queue=5779434, util=100.00%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdi-randrw-32-4K
===
mytest: (g=0): rw=randrw, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=20755: Sun Sep 15 13:31:44 2013
read : io=4473.6MB, bw=65635KB/s, iops=16408, runt= 69786msec
slat (usec): min=2, max=257, avg= 7.05, stdev= 3.80
clat (usec): min=46, max=24337, avg=1711.29, stdev=1348.06
lat (usec): min=56, max=24344, avg=1718.20, stdev=1347.98
clat percentiles (usec):
| 1.00th=[ 322], 5.00th=[ 438], 10.00th=[ 510], 20.00th=[ 660],
| 30.00th=[ 908], 40.00th=[ 1080], 50.00th=[ 1256], 60.00th=[ 1672],
| 70.00th=[ 2256], 80.00th=[ 2768], 90.00th=[ 3312], 95.00th=[ 3728],
| 99.00th=[ 4768], 99.50th=[ 6496], 99.90th=[15296], 99.95th=[16768],
| 99.99th=[22144]
bw (KB /s): min= 0, max=69149, per=99.36%, avg=65215.60, stdev=5888.09
write: io=4471.6MB, bw=65612KB/s, iops=16403, runt= 69786msec
slat (usec): min=3, max=251, avg= 8.09, stdev= 3.98
clat (usec): min=26, max=17322, avg=221.07, stdev=160.41
lat (usec): min=44, max=17330, avg=229.00, stdev=159.89
clat percentiles (usec):
| 1.00th=[ 43], 5.00th=[ 51], 10.00th=[ 63], 20.00th=[ 86],
| 30.00th=[ 123], 40.00th=[ 167], 50.00th=[ 215], 60.00th=[ 262],
| 70.00th=[ 306], 80.00th=[ 342], 90.00th=[ 370], 95.00th=[ 394],
| 99.00th=[ 434], 99.50th=[ 474], 99.90th=[ 1832], 99.95th=[ 2160],
| 99.99th=[ 3408]
bw (KB /s): min=61312, max=69192, per=100.00%, avg=65675.43, stdev=1788.70
lat (usec) : 50=2.30%, 100=9.72%, 250=16.82%, 500=25.59%, 750=7.32%
lat (usec) : 1000=5.62%
lat (msec) : 2=15.44%, 4=15.71%, 10=1.31%, 20=0.17%, 50=0.01%
cpu : usr=11.24%, sys=35.22%, ctx=1081248, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=114.5%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=1145070/w=1144707/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4473.6MB, aggrb=65634KB/s, minb=65634KB/s, maxb=65634KB/s, mint=69786msec, maxt=69786msec
WRITE: io=4471.6MB, aggrb=65612KB/s, minb=65612KB/s, maxb=65612KB/s, mint=69786msec, maxt=69786msec
Disk stats (read/write):
sdi: ios=1310259/1309299, merge=0/0, ticks=2234827/288237, in_queue=2522210, util=99.97%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdi-randrw-32-8K
===
mytest: (g=0): rw=randrw, bs=8K-8K/8K-8K/8K-8K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=20929: Sun Sep 15 13:33:44 2013
read : io=4253.5MB, bw=88831KB/s, iops=11103, runt= 49031msec
slat (usec): min=2, max=367, avg= 8.57, stdev= 4.43
clat (usec): min=155, max=28418, avg=2544.98, stdev=1714.64
lat (usec): min=166, max=28429, avg=2553.62, stdev=1714.50
clat percentiles (usec):
| 1.00th=[ 474], 5.00th=[ 652], 10.00th=[ 748], 20.00th=[ 876],
| 30.00th=[ 1096], 40.00th=[ 1736], 50.00th=[ 2608], 60.00th=[ 3088],
| 70.00th=[ 3504], 80.00th=[ 3888], 90.00th=[ 4320], 95.00th=[ 4704],
| 99.00th=[ 6432], 99.50th=[11328], 99.90th=[16192], 99.95th=[18048],
| 99.99th=[22656]
bw (KB /s): min= 0, max=95984, per=99.14%, avg=88064.76, stdev=9494.59
write: io=4248.6MB, bw=88730KB/s, iops=11091, runt= 49031msec
slat (usec): min=3, max=2072, avg=11.25, stdev= 6.24
clat (usec): min=0, max=9410, avg=313.68, stdev=218.72
lat (usec): min=53, max=9417, avg=325.13, stdev=218.32
clat percentiles (usec):
| 1.00th=[ 50], 5.00th=[ 58], 10.00th=[ 75], 20.00th=[ 108],
| 30.00th=[ 157], 40.00th=[ 223], 50.00th=[ 298], 60.00th=[ 374],
| 70.00th=[ 442], 80.00th=[ 506], 90.00th=[ 564], 95.00th=[ 604],
| 99.00th=[ 668], 99.50th=[ 700], 99.90th=[ 2384], 99.95th=[ 2672],
| 99.99th=[ 3472]
bw (KB /s): min=79365, max=94544, per=100.00%, avg=88881.75, stdev=2871.60
lat (usec) : 2=0.01%, 10=0.01%, 20=0.01%, 50=0.33%, 100=8.54%
lat (usec) : 250=13.05%, 500=18.26%, 750=14.78%, 1000=8.18%
lat (msec) : 2=8.07%, 4=20.24%, 10=8.24%, 20=0.29%, 50=0.02%
cpu : usr=12.70%, sys=30.45%, ctx=808329, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=120.4%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=544406/w=543815/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4253.5MB, aggrb=88831KB/s, minb=88831KB/s, maxb=88831KB/s, mint=49031msec, maxt=49031msec
WRITE: io=4248.6MB, aggrb=88729KB/s, minb=88729KB/s, maxb=88729KB/s, mint=49031msec, maxt=49031msec
Disk stats (read/write):
sdi: ios=654807/654224, merge=0/0, ticks=1666212/208498, in_queue=1874259, util=99.94%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdi-randrw-32-16K
===
mytest: (g=0): rw=randrw, bs=16K-16K/16K-16K/16K-16K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=21058: Sun Sep 15 13:35:34 2013
read : io=4078.1MB, bw=105938KB/s, iops=6620, runt= 39419msec
slat (usec): min=3, max=195, avg= 9.12, stdev= 2.66
clat (usec): min=270, max=36970, avg=4355.40, stdev=2559.71
lat (usec): min=285, max=36975, avg=4364.50, stdev=2559.63
clat percentiles (usec):
| 1.00th=[ 732], 5.00th=[ 1032], 10.00th=[ 1240], 20.00th=[ 1720],
| 30.00th=[ 3152], 40.00th=[ 3952], 50.00th=[ 4448], 60.00th=[ 4896],
| 70.00th=[ 5280], 80.00th=[ 5856], 90.00th=[ 6880], 95.00th=[ 8384],
| 99.00th=[13504], 99.50th=[17024], 99.90th=[21120], 99.95th=[22656],
| 99.99th=[27008]
bw (KB /s): min=89920, max=113277, per=100.00%, avg=106158.97, stdev=4525.65
write: io=4065.7MB, bw=105615KB/s, iops=6600, runt= 39419msec
slat (usec): min=5, max=3548, avg=11.99, stdev= 7.31
clat (usec): min=42, max=10184, avg=454.25, stdev=351.85
lat (usec): min=70, max=10192, avg=466.26, stdev=351.29
clat percentiles (usec):
| 1.00th=[ 68], 5.00th=[ 81], 10.00th=[ 98], 20.00th=[ 141],
| 30.00th=[ 191], 40.00th=[ 270], 50.00th=[ 374], 60.00th=[ 502],
| 70.00th=[ 636], 80.00th=[ 772], 90.00th=[ 916], 95.00th=[ 1004],
| 99.00th=[ 1128], 99.50th=[ 1336], 99.90th=[ 2864], 99.95th=[ 3248],
| 99.99th=[ 8640]
bw (KB /s): min= 1, max=114016, per=98.92%, avg=104477.37, stdev=12740.61
lat (usec) : 50=0.01%, 100=5.17%, 250=13.74%, 500=11.06%, 750=9.83%
lat (usec) : 1000=9.75%
lat (msec) : 2=11.78%, 4=9.04%, 10=28.51%, 20=1.04%, 50=0.08%
cpu : usr=11.93%, sys=20.38%, ctx=627139, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=125.7%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=260971/w=260199/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4078.1MB, aggrb=105937KB/s, minb=105937KB/s, maxb=105937KB/s, mint=39419msec, maxt=39419msec
WRITE: io=4065.7MB, aggrb=105615KB/s, minb=105615KB/s, maxb=105615KB/s, mint=39419msec, maxt=39419msec
Disk stats (read/write):
sdi: ios=327015/325931, merge=0/0, ticks=1419138/150144, in_queue=1569361, util=99.90%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdi-randrw-32-32K
===
mytest: (g=0): rw=randrw, bs=32K-32K/32K-32K/32K-32K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=21187: Sun Sep 15 13:37:16 2013
read : io=3929.7MB, bw=125568KB/s, iops=3923, runt= 32046msec
slat (usec): min=7, max=220, avg=11.31, stdev= 2.41
clat (usec): min=556, max=43742, avg=7309.17, stdev=3726.44
lat (usec): min=569, max=43752, avg=7320.62, stdev=3726.35
clat percentiles (usec):
| 1.00th=[ 1432], 5.00th=[ 1992], 10.00th=[ 2448], 20.00th=[ 4320],
| 30.00th=[ 5536], 40.00th=[ 6304], 50.00th=[ 7008], 60.00th=[ 7712],
| 70.00th=[ 8640], 80.00th=[ 9920], 90.00th=[11840], 95.00th=[13632],
| 99.00th=[19072], 99.50th=[22912], 99.90th=[29056], 99.95th=[31360],
| 99.99th=[37120]
bw (KB /s): min= 3, max=132032, per=98.57%, avg=123770.73, stdev=16053.64
write: io=3931.4MB, bw=125622KB/s, iops=3925, runt= 32046msec
slat (usec): min=11, max=5281, avg=17.46, stdev=17.68
clat (usec): min=69, max=13916, avg=812.70, stdev=606.30
lat (usec): min=113, max=13930, avg=830.74, stdev=606.33
clat percentiles (usec):
| 1.00th=[ 103], 5.00th=[ 133], 10.00th=[ 177], 20.00th=[ 255],
| 30.00th=[ 358], 40.00th=[ 506], 50.00th=[ 684], 60.00th=[ 892],
| 70.00th=[ 1112], 80.00th=[ 1352], 90.00th=[ 1624], 95.00th=[ 1816],
| 99.00th=[ 2096], 99.50th=[ 2608], 99.90th=[ 4832], 99.95th=[ 5984],
| 99.99th=[ 9920]
bw (KB /s): min=118202, max=136000, per=100.00%, avg=125763.22, stdev=4271.74
lat (usec) : 100=0.23%, 250=9.51%, 500=9.99%, 750=6.85%, 1000=5.84%
lat (msec) : 2=19.18%, 4=7.49%, 10=31.32%, 20=9.19%, 50=0.42%
cpu : usr=10.29%, sys=15.51%, ctx=324553, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=130.3%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=125718/w=125803/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=3929.7MB, aggrb=125568KB/s, minb=125568KB/s, maxb=125568KB/s, mint=32046msec, maxt=32046msec
WRITE: io=3931.4MB, aggrb=125622KB/s, minb=125622KB/s, maxb=125622KB/s, mint=32046msec, maxt=32046msec
Disk stats (read/write):
sdi: ios=163573/163433, merge=0/0, ticks=1206918/131041, in_queue=1337953, util=99.88%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdi-randrw-32-64K
===
mytest: (g=0): rw=randrw, bs=64K-64K/64K-64K/64K-64K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=21268: Sun Sep 15 13:38:54 2013
read : io=3775.3MB, bw=141177KB/s, iops=2204, runt= 27383msec
slat (usec): min=10, max=204, avg=13.76, stdev= 2.36
clat (msec): min=1, max=53, avg=12.88, stdev= 5.69
lat (msec): min=1, max=53, avg=12.89, stdev= 5.69
clat percentiles (usec):
| 1.00th=[ 2864], 5.00th=[ 4080], 10.00th=[ 5536], 20.00th=[ 8256],
| 30.00th=[ 9920], 40.00th=[11200], 50.00th=[12352], 60.00th=[13760],
| 70.00th=[15296], 80.00th=[17280], 90.00th=[20096], 95.00th=[22656],
| 99.00th=[29568], 99.50th=[33024], 99.90th=[40704], 99.95th=[43264],
| 99.99th=[47360]
bw (KB /s): min= 6, max=153728, per=98.17%, avg=138595.47, stdev=19558.57
write: io=3777.4MB, bw=141257KB/s, iops=2207, runt= 27383msec
slat (usec): min=14, max=7490, avg=24.03, stdev=36.61
clat (usec): min=105, max=27860, avg=1592.66, stdev=1168.00
lat (usec): min=182, max=27885, avg=1617.17, stdev=1168.18
clat percentiles (usec):
| 1.00th=[ 187], 5.00th=[ 274], 10.00th=[ 362], 20.00th=[ 548],
| 30.00th=[ 796], 40.00th=[ 1080], 50.00th=[ 1384], 60.00th=[ 1736],
| 70.00th=[ 2096], 80.00th=[ 2544], 90.00th=[ 3088], 95.00th=[ 3504],
| 99.00th=[ 4128], 99.50th=[ 5408], 99.90th=[11072], 99.95th=[14144],
| 99.99th=[19584]
bw (KB /s): min=130816, max=152126, per=100.00%, avg=141422.00, stdev=4137.78
lat (usec) : 250=2.01%, 500=6.92%, 750=5.27%, 1000=4.56%
lat (msec) : 2=15.07%, 4=17.79%, 10=13.73%, 20=29.71%, 50=4.97%
lat (msec) : 100=0.01%
cpu : usr=7.99%, sys=10.11%, ctx=163648, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=135.6%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=60373/w=60438/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=3775.3MB, aggrb=141177KB/s, minb=141177KB/s, maxb=141177KB/s, mint=27383msec, maxt=27383msec
WRITE: io=3777.4MB, aggrb=141256KB/s, minb=141256KB/s, maxb=141256KB/s, mint=27383msec, maxt=27383msec
Disk stats (read/write):
sdi: ios=81580/81508, merge=0/0, ticks=1060026/127091, in_queue=1187223, util=99.85%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdi-randrw-32-128K
===
mytest: (g=0): rw=randrw, bs=128K-128K/128K-128K/128K-128K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=21391: Sun Sep 15 13:40:30 2013
read : io=3706.8MB, bw=147647KB/s, iops=1152, runt= 25708msec
slat (usec): min=14, max=85, avg=21.27, stdev= 3.48
clat (msec): min=1, max=79, avg=24.89, stdev=10.48
lat (msec): min=1, max=79, avg=24.92, stdev=10.48
clat percentiles (usec):
| 1.00th=[ 5664], 5.00th=[ 9152], 10.00th=[12352], 20.00th=[15936],
| 30.00th=[18816], 40.00th=[21376], 50.00th=[23936], 60.00th=[26496],
| 70.00th=[29312], 80.00th=[33024], 90.00th=[38656], 95.00th=[43776],
| 99.00th=[54528], 99.50th=[58624], 99.90th=[66048], 99.95th=[70144],
| 99.99th=[75264]
bw (KB /s): min=133795, max=157440, per=99.94%, avg=147554.34, stdev=4668.12
write: io=3714.2MB, bw=147941KB/s, iops=1155, runt= 25708msec
slat (usec): min=22, max=13602, avg=41.36, stdev=96.85
clat (usec): min=288, max=37925, avg=2776.39, stdev=1995.49
lat (usec): min=324, max=37957, avg=2818.17, stdev=1997.22
clat percentiles (usec):
| 1.00th=[ 346], 5.00th=[ 548], 10.00th=[ 668], 20.00th=[ 1048],
| 30.00th=[ 1448], 40.00th=[ 1896], 50.00th=[ 2416], 60.00th=[ 2928],
| 70.00th=[ 3536], 80.00th=[ 4256], 90.00th=[ 5408], 95.00th=[ 6368],
| 99.00th=[ 7776], 99.50th=[ 8768], 99.90th=[16192], 99.95th=[22912],
| 99.99th=[36608]
bw (KB /s): min= 12, max=160382, per=98.26%, avg=145368.65, stdev=21448.13
lat (usec) : 500=1.99%, 750=3.76%, 1000=3.96%
lat (msec) : 2=11.27%, 4=17.44%, 10=14.55%, 20=14.24%, 50=31.82%
lat (msec) : 100=1.03%
cpu : usr=5.48%, sys=8.26%, ctx=81845, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=138.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=29635/w=29701/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=3706.8MB, aggrb=147647KB/s, minb=147647KB/s, maxb=147647KB/s, mint=25708msec, maxt=25708msec
WRITE: io=3714.2MB, aggrb=147940KB/s, minb=147940KB/s, maxb=147940KB/s, mint=25708msec, maxt=25708msec
Disk stats (read/write):
sdi: ios=41077/40962, merge=0/1, ticks=1030592/108573, in_queue=1139456, util=99.84%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdi-randrw-32-256K
===
mytest: (g=0): rw=randrw, bs=256K-256K/256K-256K/256K-256K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=21494: Sun Sep 15 13:42:05 2013
read : io=3678.3MB, bw=153323KB/s, iops=597, runt= 24566msec
slat (usec): min=22, max=262, avg=38.97, stdev= 7.43
clat (msec): min=2, max=80, avg=51.03, stdev= 7.26
lat (msec): min=2, max=80, avg=51.07, stdev= 7.26
clat percentiles (usec):
| 1.00th=[35584], 5.00th=[39680], 10.00th=[42240], 20.00th=[44800],
| 30.00th=[47360], 40.00th=[48896], 50.00th=[50944], 60.00th=[52480],
| 70.00th=[54528], 80.00th=[57088], 90.00th=[60160], 95.00th=[63232],
| 99.00th=[70144], 99.50th=[72192], 99.90th=[76288], 99.95th=[78336],
| 99.99th=[80384]
bw (KB /s): min= 25, max=166400, per=97.80%, avg=149949.88, stdev=23027.23
write: io=3634.6MB, bw=151499KB/s, iops=591, runt= 24566msec
slat (usec): min=35, max=7912, avg=78.11, stdev=72.74
clat (usec): min=546, max=36451, avg=2342.20, stdev=1604.70
lat (usec): min=619, max=36553, avg=2420.95, stdev=1605.64
clat percentiles (usec):
| 1.00th=[ 612], 5.00th=[ 700], 10.00th=[ 1112], 20.00th=[ 1224],
| 30.00th=[ 1608], 40.00th=[ 1784], 50.00th=[ 2128], 60.00th=[ 2384],
| 70.00th=[ 2704], 80.00th=[ 3152], 90.00th=[ 3728], 95.00th=[ 4384],
| 99.00th=[ 6688], 99.50th=[ 8160], 99.90th=[23680], 99.95th=[25728],
| 99.99th=[34560]
bw (KB /s): min=134629, max=172032, per=100.00%, avg=151913.04, stdev=7887.68
lat (usec) : 750=2.94%, 1000=0.36%
lat (msec) : 2=19.56%, 4=23.32%, 10=3.43%, 20=0.06%, 50=23.07%
lat (msec) : 100=27.37%
cpu : usr=4.71%, sys=7.74%, ctx=40946, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=140.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=14682/w=14538/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=3678.3MB, aggrb=153322KB/s, minb=153322KB/s, maxb=153322KB/s, mint=24566msec, maxt=24566msec
WRITE: io=3634.6MB, aggrb=151499KB/s, minb=151499KB/s, maxb=151499KB/s, mint=24566msec, maxt=24566msec
Disk stats (read/write):
sdi: ios=20591/20355, merge=0/1, ticks=1053939/43884, in_queue=1098403, util=99.83%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdi-randrw-32-512K
===
mytest: (g=0): rw=randrw, bs=512K-512K/512K-512K/512K-512K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=21601: Sun Sep 15 13:43:37 2013
read : io=3519.6MB, bw=163453KB/s, iops=318, runt= 22049msec
slat (usec): min=42, max=263, avg=67.08, stdev= 6.71
clat (msec): min=3, max=136, avg=74.83, stdev=14.64
lat (msec): min=3, max=136, avg=74.90, stdev=14.64
clat percentiles (msec):
| 1.00th=[ 50], 5.00th=[ 53], 10.00th=[ 56], 20.00th=[ 62],
| 30.00th=[ 67], 40.00th=[ 71], 50.00th=[ 75], 60.00th=[ 79],
| 70.00th=[ 83], 80.00th=[ 88], 90.00th=[ 95], 95.00th=[ 101],
| 99.00th=[ 111], 99.50th=[ 115], 99.90th=[ 127], 99.95th=[ 131],
| 99.99th=[ 137]
bw (KB /s): min= 51, max=179200, per=97.61%, avg=159546.70, stdev=26343.52
write: io=3561.0MB, bw=165380KB/s, iops=322, runt= 22049msec
slat (usec): min=97, max=45983, avg=147.10, stdev=453.16
clat (msec): min=1, max=96, avg=24.98, stdev=15.30
lat (msec): min=1, max=96, avg=25.13, stdev=15.30
clat percentiles (usec):
| 1.00th=[ 2224], 5.00th=[ 3568], 10.00th=[ 5984], 20.00th=[ 9792],
| 30.00th=[13504], 40.00th=[18048], 50.00th=[22912], 60.00th=[28800],
| 70.00th=[35072], 80.00th=[40704], 90.00th=[46336], 95.00th=[49408],
| 99.00th=[56064], 99.50th=[61184], 99.90th=[70144], 99.95th=[79360],
| 99.99th=[96768]
bw (KB /s): min=138240, max=196844, per=100.00%, avg=165776.30, stdev=13290.58
lat (msec) : 2=0.34%, 4=3.04%, 10=7.17%, 20=11.95%, 50=26.39%
lat (msec) : 100=48.69%, 250=2.64%
cpu : usr=4.33%, sys=6.93%, ctx=20462, majf=0, minf=28
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.2%, 32=144.5%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=7018/w=7112/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=3519.6MB, aggrb=163452KB/s, minb=163452KB/s, maxb=163452KB/s, mint=22049msec, maxt=22049msec
WRITE: io=3561.0MB, aggrb=165380KB/s, minb=165380KB/s, maxb=165380KB/s, mint=22049msec, maxt=22049msec
Disk stats (read/write):
sdi: ios=10291/10215, merge=0/0, ticks=765830/251356, in_queue=1018205, util=99.81%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdi-randrw-32-1024K
===
mytest: (g=0): rw=randrw, bs=1M-1M/1M-1M/1M-1M, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=21738: Sun Sep 15 13:45:10 2013
read : io=3549.0MB, bw=167930KB/s, iops=162, runt= 21641msec
slat (usec): min=73, max=327, avg=120.06, stdev= 9.32
clat (msec): min=6, max=329, avg=122.20, stdev=32.39
lat (msec): min=6, max=329, avg=122.32, stdev=32.39
clat percentiles (msec):
| 1.00th=[ 63], 5.00th=[ 75], 10.00th=[ 82], 20.00th=[ 94],
| 30.00th=[ 102], 40.00th=[ 112], 50.00th=[ 121], 60.00th=[ 131],
| 70.00th=[ 141], 80.00th=[ 151], 90.00th=[ 163], 95.00th=[ 176],
| 99.00th=[ 200], 99.50th=[ 215], 99.90th=[ 289], 99.95th=[ 322],
| 99.99th=[ 330]
bw (KB /s): min= 102, max=189705, per=97.69%, avg=164048.74, stdev=27762.31
write: io=3480.0MB, bw=164665KB/s, iops=160, runt= 21641msec
slat (usec): min=186, max=428, avg=268.43, stdev=21.17
clat (msec): min=4, max=231, avg=74.43, stdev=33.33
lat (msec): min=5, max=232, avg=74.70, stdev=33.33
clat percentiles (msec):
| 1.00th=[ 13], 5.00th=[ 23], 10.00th=[ 32], 20.00th=[ 44],
| 30.00th=[ 55], 40.00th=[ 63], 50.00th=[ 74], 60.00th=[ 83],
| 70.00th=[ 93], 80.00th=[ 103], 90.00th=[ 120], 95.00th=[ 131],
| 99.00th=[ 153], 99.50th=[ 169], 99.90th=[ 200], 99.95th=[ 210],
| 99.99th=[ 233]
bw (KB /s): min=134629, max=199904, per=100.00%, avg=164740.83, stdev=14573.11
lat (msec) : 10=0.21%, 20=1.63%, 50=10.95%, 100=39.47%, 250=48.11%
lat (msec) : 500=0.07%
cpu : usr=3.97%, sys=6.16%, ctx=10258, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.2%, 16=0.5%, 32=145.4%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=3527/w=3471/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=3549.0MB, aggrb=167930KB/s, minb=167930KB/s, maxb=167930KB/s, mint=21641msec, maxt=21641msec
WRITE: io=3480.0MB, aggrb=164665KB/s, minb=164665KB/s, maxb=164665KB/s, mint=21641msec, maxt=21641msec
Disk stats (read/write):
sdi: ios=10399/10008, merge=0/0, ticks=1242281/731292, in_queue=1976919, util=99.77%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdj-randrw-32-512B
===
mytest: (g=0): rw=randrw, bs=512-512/512-512/512-512, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=22142: Sun Sep 15 13:58:09 2013
read : io=5044.4MB, bw=7285.9KB/s, iops=14571, runt=708964msec
slat (usec): min=2, max=366, avg= 7.23, stdev= 4.28
clat (usec): min=17, max=44831, avg=2050.63, stdev=2215.67
lat (usec): min=35, max=44841, avg=2058.01, stdev=2215.71
clat percentiles (usec):
| 1.00th=[ 145], 5.00th=[ 318], 10.00th=[ 442], 20.00th=[ 628],
| 30.00th=[ 788], 40.00th=[ 948], 50.00th=[ 1160], 60.00th=[ 1528],
| 70.00th=[ 2480], 80.00th=[ 3504], 90.00th=[ 4832], 95.00th=[ 5856],
| 99.00th=[ 7904], 99.50th=[ 9152], 99.90th=[28288], 99.95th=[31616],
| 99.99th=[35584]
bw (KB /s): min= 5135, max= 8878, per=100.00%, avg=7289.82, stdev=507.54
write: io=5043.6MB, bw=7284.8KB/s, iops=14569, runt=708964msec
slat (usec): min=3, max=3929, avg= 8.53, stdev= 5.17
clat (usec): min=0, max=27133, avg=126.22, stdev=238.92
lat (usec): min=32, max=27137, avg=134.91, stdev=238.82
clat percentiles (usec):
| 1.00th=[ 30], 5.00th=[ 35], 10.00th=[ 37], 20.00th=[ 45],
| 30.00th=[ 56], 40.00th=[ 69], 50.00th=[ 85], 60.00th=[ 105],
| 70.00th=[ 131], 80.00th=[ 167], 90.00th=[ 231], 95.00th=[ 294],
| 99.00th=[ 548], 99.50th=[ 1080], 99.90th=[ 3888], 99.95th=[ 4512],
| 99.99th=[ 5600]
bw (KB /s): min= 0, max= 8984, per=99.99%, avg=7283.55, stdev=538.94
lat (usec) : 2=0.01%, 10=0.01%, 20=0.01%, 50=12.18%, 100=16.87%
lat (usec) : 250=18.22%, 500=8.53%, 750=7.56%, 1000=7.76%
lat (msec) : 2=11.52%, 4=9.33%, 10=7.83%, 20=0.09%, 50=0.10%
cpu : usr=7.65%, sys=28.28%, ctx=5895316, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=101.5%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=10330819/w=10329202/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=5044.4MB, aggrb=7285KB/s, minb=7285KB/s, maxb=7285KB/s, mint=708964msec, maxt=708964msec
WRITE: io=5043.6MB, aggrb=7284KB/s, minb=7284KB/s, maxb=7284KB/s, mint=708964msec, maxt=708964msec
Disk stats (read/write):
sdj: ios=10485610/10483385, merge=0/0, ticks=21276035/1250124, in_queue=22519295, util=100.00%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdj-randrw-32-1K
===
mytest: (g=0): rw=randrw, bs=1K-1K/1K-1K/1K-1K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=23213: Sun Sep 15 14:04:51 2013
read : io=4966.9MB, bw=15319KB/s, iops=15319, runt=332001msec
slat (usec): min=2, max=349, avg= 7.08, stdev= 4.27
clat (usec): min=39, max=42950, avg=1923.90, stdev=2321.18
lat (usec): min=93, max=42958, avg=1931.09, stdev=2321.26
clat percentiles (usec):
| 1.00th=[ 177], 5.00th=[ 270], 10.00th=[ 330], 20.00th=[ 470],
| 30.00th=[ 612], 40.00th=[ 756], 50.00th=[ 932], 60.00th=[ 1240],
| 70.00th=[ 2416], 80.00th=[ 3504], 90.00th=[ 4832], 95.00th=[ 5728],
| 99.00th=[ 7776], 99.50th=[ 9152], 99.90th=[29312], 99.95th=[32128],
| 99.99th=[36096]
bw (KB /s): min= 0, max=17020, per=99.94%, avg=15310.47, stdev=1294.09
write: io=4965.9MB, bw=15316KB/s, iops=15316, runt=332001msec
slat (usec): min=3, max=4297, avg= 8.38, stdev= 5.45
clat (usec): min=0, max=28232, avg=146.21, stdev=336.72
lat (usec): min=32, max=28236, avg=154.72, stdev=336.63
clat percentiles (usec):
| 1.00th=[ 31], 5.00th=[ 35], 10.00th=[ 37], 20.00th=[ 46],
| 30.00th=[ 57], 40.00th=[ 70], 50.00th=[ 88], 60.00th=[ 110],
| 70.00th=[ 139], 80.00th=[ 179], 90.00th=[ 247], 95.00th=[ 310],
| 99.00th=[ 1064], 99.50th=[ 2704], 99.90th=[ 4576], 99.95th=[ 5024],
| 99.99th=[ 6496]
bw (KB /s): min=10585, max=16822, per=100.00%, avg=15330.03, stdev=1137.89
lat (usec) : 2=0.01%, 10=0.01%, 20=0.01%, 50=11.81%, 100=16.03%
lat (usec) : 250=19.15%, 500=13.03%, 750=9.15%, 1000=6.81%
lat (msec) : 2=7.24%, 4=8.61%, 10=7.96%, 20=0.08%, 50=0.13%
cpu : usr=7.02%, sys=29.08%, ctx=1969249, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=103.1%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=5086052/w=5084991/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4966.9MB, aggrb=15319KB/s, minb=15319KB/s, maxb=15319KB/s, mint=332001msec, maxt=332001msec
WRITE: io=4965.9MB, aggrb=15316KB/s, minb=15316KB/s, maxb=15316KB/s, mint=332001msec, maxt=332001msec
Disk stats (read/write):
sdj: ios=5243128/5241437, merge=2/0, ticks=9933514/694696, in_queue=10624404, util=100.00%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdj-randrw-32-2K
===
mytest: (g=0): rw=randrw, bs=2K-2K/2K-2K/2K-2K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=23975: Sun Sep 15 14:08:45 2013
read : io=4813.6MB, bw=30222KB/s, iops=15110, runt=163093msec
slat (usec): min=2, max=350, avg= 7.28, stdev= 4.55
clat (usec): min=19, max=43231, avg=1886.40, stdev=2307.57
lat (usec): min=97, max=43236, avg=1893.75, stdev=2307.65
clat percentiles (usec):
| 1.00th=[ 183], 5.00th=[ 270], 10.00th=[ 326], 20.00th=[ 430],
| 30.00th=[ 556], 40.00th=[ 724], 50.00th=[ 916], 60.00th=[ 1208],
| 70.00th=[ 2352], 80.00th=[ 3472], 90.00th=[ 4768], 95.00th=[ 5728],
| 99.00th=[ 7712], 99.50th=[ 9024], 99.90th=[29312], 99.95th=[32128],
| 99.99th=[35584]
bw (KB /s): min=21224, max=33888, per=100.00%, avg=30268.02, stdev=2738.74
write: io=4811.9MB, bw=30212KB/s, iops=15105, runt=163093msec
slat (usec): min=3, max=4276, avg= 8.95, stdev= 6.57
clat (usec): min=0, max=27617, avg=211.68, stdev=464.08
lat (usec): min=35, max=27627, avg=220.73, stdev=464.23
clat percentiles (usec):
| 1.00th=[ 32], 5.00th=[ 37], 10.00th=[ 40], 20.00th=[ 53],
| 30.00th=[ 71], 40.00th=[ 94], 50.00th=[ 124], 60.00th=[ 163],
| 70.00th=[ 205], 80.00th=[ 251], 90.00th=[ 326], 95.00th=[ 442],
| 99.00th=[ 2704], 99.50th=[ 3664], 99.90th=[ 4960], 99.95th=[ 5408],
| 99.99th=[ 8384]
bw (KB /s): min= 0, max=33480, per=99.85%, avg=30165.13, stdev=3202.49
lat (usec) : 2=0.01%, 10=0.01%, 20=0.01%, 50=8.76%, 100=12.23%
lat (usec) : 250=20.66%, 500=19.19%, 750=8.44%, 1000=6.45%
lat (msec) : 2=7.39%, 4=8.87%, 10=7.81%, 20=0.07%, 50=0.13%
cpu : usr=7.88%, sys=30.54%, ctx=1068218, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=106.4%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=2464498/w=2463676/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4813.6MB, aggrb=30222KB/s, minb=30222KB/s, maxb=30222KB/s, mint=163093msec, maxt=163093msec
WRITE: io=4811.9MB, aggrb=30211KB/s, minb=30211KB/s, maxb=30211KB/s, mint=163093msec, maxt=163093msec
Disk stats (read/write):
sdj: ios=2621973/2620512, merge=3/5, ticks=4881226/510341, in_queue=5389507, util=100.00%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdj-randrw-32-4K
===
mytest: (g=0): rw=randrw, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=24212: Sun Sep 15 14:11:09 2013
read : io=4471.9MB, bw=62148KB/s, iops=15536, runt= 73681msec
slat (usec): min=2, max=271, avg= 7.33, stdev= 4.02
clat (usec): min=39, max=36628, avg=1636.88, stdev=1732.05
lat (usec): min=66, max=36634, avg=1644.06, stdev=1732.07
clat percentiles (usec):
| 1.00th=[ 245], 5.00th=[ 394], 10.00th=[ 454], 20.00th=[ 532],
| 30.00th=[ 596], 40.00th=[ 836], 50.00th=[ 1096], 60.00th=[ 1688],
| 70.00th=[ 2352], 80.00th=[ 2640], 90.00th=[ 3024], 95.00th=[ 3600],
| 99.00th=[ 5216], 99.50th=[ 6048], 99.90th=[26240], 99.95th=[28800],
| 99.99th=[32128]
bw (KB /s): min= 0, max=70152, per=99.36%, avg=61752.93, stdev=8311.90
write: io=4470.4MB, bw=62127KB/s, iops=15531, runt= 73681msec
slat (usec): min=3, max=1243, avg= 9.14, stdev= 4.72
clat (usec): min=6, max=29317, avg=403.45, stdev=619.05
lat (usec): min=41, max=29332, avg=412.53, stdev=618.88
clat percentiles (usec):
| 1.00th=[ 40], 5.00th=[ 46], 10.00th=[ 73], 20.00th=[ 143],
| 30.00th=[ 215], 40.00th=[ 270], 50.00th=[ 306], 60.00th=[ 342],
| 70.00th=[ 386], 80.00th=[ 426], 90.00th=[ 540], 95.00th=[ 1096],
| 99.00th=[ 3088], 99.50th=[ 3440], 99.90th=[ 5280], 99.95th=[ 5728],
| 99.99th=[20864]
bw (KB /s): min=46896, max=70240, per=100.00%, avg=62158.74, stdev=6577.91
lat (usec) : 10=0.01%, 50=2.85%, 100=4.04%, 250=11.62%, 500=33.92%
lat (usec) : 750=12.35%, 1000=5.68%
lat (msec) : 2=9.50%, 4=17.88%, 10=1.98%, 20=0.05%, 50=0.12%
cpu : usr=9.86%, sys=34.88%, ctx=839625, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=114.5%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=1144756/w=1144402/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4471.9MB, aggrb=62148KB/s, minb=62148KB/s, maxb=62148KB/s, mint=73681msec, maxt=73681msec
WRITE: io=4470.4MB, aggrb=62127KB/s, minb=62127KB/s, maxb=62127KB/s, mint=73681msec, maxt=73681msec
Disk stats (read/write):
sdj: ios=1307973/1306916, merge=10/8, ticks=2135635/502460, in_queue=2637119, util=99.98%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdj-randrw-32-8K
===
mytest: (g=0): rw=randrw, bs=8K-8K/8K-8K/8K-8K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=24347: Sun Sep 15 14:13:20 2013
read : io=4350.4MB, bw=73417KB/s, iops=9176, runt= 60673msec
slat (usec): min=3, max=256, avg= 8.67, stdev= 4.53
clat (usec): min=135, max=47885, avg=2856.35, stdev=2516.36
lat (usec): min=142, max=47890, avg=2865.08, stdev=2516.30
clat percentiles (usec):
| 1.00th=[ 342], 5.00th=[ 580], 10.00th=[ 740], 20.00th=[ 1160],
| 30.00th=[ 1560], 40.00th=[ 2288], 50.00th=[ 2704], 60.00th=[ 2992],
| 70.00th=[ 3312], 80.00th=[ 3856], 90.00th=[ 5088], 95.00th=[ 5728],
| 99.00th=[ 8032], 99.50th=[20864], 99.90th=[33024], 99.95th=[35584],
| 99.99th=[40704]
bw (KB /s): min= 0, max=87280, per=99.21%, avg=72834.88, stdev=11569.42
write: io=4345.2MB, bw=73334KB/s, iops=9166, runt= 60673msec
slat (usec): min=4, max=3402, avg=11.97, stdev= 8.20
clat (usec): min=0, max=26879, avg=606.75, stdev=1023.75
lat (usec): min=49, max=26888, avg=618.99, stdev=1024.44
clat percentiles (usec):
| 1.00th=[ 46], 5.00th=[ 51], 10.00th=[ 72], 20.00th=[ 126],
| 30.00th=[ 195], 40.00th=[ 270], 50.00th=[ 346], 60.00th=[ 418],
| 70.00th=[ 486], 80.00th=[ 564], 90.00th=[ 1304], 95.00th=[ 2960],
| 99.00th=[ 4768], 99.50th=[ 5728], 99.90th=[ 7648], 99.95th=[14016],
| 99.99th=[22656]
bw (KB /s): min=53748, max=87520, per=100.00%, avg=73378.62, stdev=9351.18
lat (usec) : 2=0.01%, 20=0.01%, 50=1.96%, 100=5.72%, 250=11.06%
lat (usec) : 500=18.95%, 750=10.78%, 1000=3.92%
lat (msec) : 2=11.75%, 4=25.85%, 10=9.64%, 20=0.09%, 50=0.27%
cpu : usr=10.27%, sys=25.30%, ctx=779583, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=117.8%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=556776/w=556173/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4350.4MB, aggrb=73417KB/s, minb=73417KB/s, maxb=73417KB/s, mint=60673msec, maxt=60673msec
WRITE: io=4345.2MB, aggrb=73334KB/s, minb=73334KB/s, maxb=73334KB/s, mint=60673msec, maxt=60673msec
Disk stats (read/write):
sdj: ios=653693/653146, merge=7/5, ticks=1886230/369008, in_queue=2254624, util=99.96%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdj-randrw-32-16K
===
mytest: (g=0): rw=randrw, bs=16K-16K/16K-16K/16K-16K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=24477: Sun Sep 15 14:15:28 2013
read : io=4308.8MB, bw=77204KB/s, iops=4824, runt= 57149msec
slat (usec): min=5, max=216, avg=10.08, stdev= 2.20
clat (usec): min=146, max=61551, avg=5616.30, stdev=4050.70
lat (usec): min=195, max=61562, avg=5626.60, stdev=4050.89
clat percentiles (usec):
| 1.00th=[ 548], 5.00th=[ 1112], 10.00th=[ 1752], 20.00th=[ 2928],
| 30.00th=[ 3568], 40.00th=[ 4256], 50.00th=[ 5088], 60.00th=[ 5920],
| 70.00th=[ 6752], 80.00th=[ 7776], 90.00th=[ 9408], 95.00th=[10816],
| 99.00th=[20352], 99.50th=[33024], 99.90th=[43264], 99.95th=[46848],
| 99.99th=[51968]
bw (KB /s): min= 1, max=88512, per=99.25%, avg=76626.69, stdev=9387.32
write: io=4296.2MB, bw=76979KB/s, iops=4811, runt= 57149msec
slat (usec): min=9, max=6879, avg=14.10, stdev=17.92
clat (usec): min=23, max=31763, avg=989.34, stdev=1753.10
lat (usec): min=75, max=31776, avg=1003.93, stdev=1752.74
clat percentiles (usec):
| 1.00th=[ 63], 5.00th=[ 67], 10.00th=[ 80], 20.00th=[ 116],
| 30.00th=[ 169], 40.00th=[ 241], 50.00th=[ 334], 60.00th=[ 454],
| 70.00th=[ 612], 80.00th=[ 932], 90.00th=[ 3408], 95.00th=[ 4704],
| 99.00th=[ 7200], 99.50th=[ 8384], 99.90th=[17792], 99.95th=[23424],
| 99.99th=[27776]
bw (KB /s): min=62208, max=86464, per=100.00%, avg=77076.35, stdev=5678.34
lat (usec) : 50=0.01%, 100=7.92%, 250=12.65%, 500=11.45%, 750=6.96%
lat (usec) : 1000=3.38%
lat (msec) : 2=5.84%, 4=16.22%, 10=31.77%, 20=3.26%, 50=0.53%
lat (msec) : 100=0.01%
cpu : usr=8.80%, sys=16.07%, ctx=596214, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=119.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=275728/w=274955/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4308.8MB, aggrb=77204KB/s, minb=77204KB/s, maxb=77204KB/s, mint=57149msec, maxt=57149msec
WRITE: io=4296.2MB, aggrb=76979KB/s, minb=76979KB/s, maxb=76979KB/s, mint=57149msec, maxt=57149msec
Disk stats (read/write):
sdj: ios=327381/326298, merge=19/19, ticks=1859740/292243, in_queue=2151976, util=99.95%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdj-randrw-32-32K
===
mytest: (g=0): rw=randrw, bs=32K-32K/32K-32K/32K-32K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=24608: Sun Sep 15 14:17:29 2013
read : io=4289.8MB, bw=85851KB/s, iops=2682, runt= 51166msec
slat (usec): min=6, max=209, avg=11.56, stdev= 2.18
clat (usec): min=225, max=84928, avg=10014.57, stdev=6411.41
lat (usec): min=291, max=84940, avg=10026.32, stdev=6411.83
clat percentiles (usec):
| 1.00th=[ 1096], 5.00th=[ 2704], 10.00th=[ 3632], 20.00th=[ 5280],
| 30.00th=[ 6624], 40.00th=[ 7840], 50.00th=[ 9152], 60.00th=[10432],
| 70.00th=[11840], 80.00th=[13760], 90.00th=[16512], 95.00th=[19072],
| 99.00th=[36096], 99.50th=[47872], 99.90th=[59648], 99.95th=[64256],
| 99.99th=[71168]
bw (KB /s): min=75008, max=93696, per=100.00%, avg=85941.47, stdev=3646.91
write: io=4291.9MB, bw=85894KB/s, iops=2684, runt= 51166msec
slat (usec): min=10, max=7589, avg=18.52, stdev=23.18
clat (usec): min=82, max=35759, avg=1877.66, stdev=3048.71
lat (usec): min=110, max=35778, avg=1896.80, stdev=3048.52
clat percentiles (usec):
| 1.00th=[ 95], 5.00th=[ 109], 10.00th=[ 125], 20.00th=[ 163],
| 30.00th=[ 241], 40.00th=[ 346], 50.00th=[ 482], 60.00th=[ 684],
| 70.00th=[ 1080], 80.00th=[ 3536], 90.00th=[ 6496], 95.00th=[ 8256],
| 99.00th=[11712], 99.50th=[13888], 99.90th=[27008], 99.95th=[29568],
| 99.99th=[31872]
bw (KB /s): min= 3, max=92742, per=99.12%, avg=85137.72, stdev=9095.84
lat (usec) : 100=2.06%, 250=13.31%, 500=10.27%, 750=5.79%, 1000=3.35%
lat (msec) : 2=4.48%, 4=7.85%, 10=30.23%, 20=20.50%, 50=1.96%
lat (msec) : 100=0.21%
cpu : usr=6.18%, sys=10.31%, ctx=319710, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=119.3%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=137239/w=137339/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4289.8MB, aggrb=85850KB/s, minb=85850KB/s, maxb=85850KB/s, mint=51166msec, maxt=51166msec
WRITE: io=4291.9MB, aggrb=85893KB/s, minb=85893KB/s, maxb=85893KB/s, mint=51166msec, maxt=51166msec
Disk stats (read/write):
sdj: ios=163406/163296, merge=22/20, ticks=1691542/270820, in_queue=1962494, util=99.93%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdj-randrw-32-64K
===
mytest: (g=0): rw=randrw, bs=64K-64K/64K-64K/64K-64K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=24738: Sun Sep 15 14:19:24 2013
read : io=4247.7MB, bw=97574KB/s, iops=1523, runt= 44571msec
slat (usec): min=8, max=207, avg=13.84, stdev= 2.56
clat (usec): min=666, max=104274, avg=17293.32, stdev=9650.13
lat (usec): min=685, max=104288, avg=17307.27, stdev=9650.97
clat percentiles (usec):
| 1.00th=[ 2544], 5.00th=[ 4704], 10.00th=[ 6560], 20.00th=[ 9280],
| 30.00th=[11712], 40.00th=[13760], 50.00th=[15808], 60.00th=[18048],
| 70.00th=[20864], 80.00th=[24192], 90.00th=[29312], 95.00th=[34048],
| 99.00th=[47360], 99.50th=[58112], 99.90th=[76288], 99.95th=[82432],
| 99.99th=[88576]
bw (KB /s): min= 6, max=114842, per=98.90%, avg=96498.43, stdev=13211.62
write: io=4246.1MB, bw=97572KB/s, iops=1524, runt= 44571msec
slat (usec): min=15, max=14725, avg=25.28, stdev=62.33
clat (usec): min=114, max=62250, avg=3656.53, stdev=4971.11
lat (usec): min=180, max=62274, avg=3682.45, stdev=4971.00
clat percentiles (usec):
| 1.00th=[ 161], 5.00th=[ 181], 10.00th=[ 191], 20.00th=[ 262],
| 30.00th=[ 386], 40.00th=[ 540], 50.00th=[ 828], 60.00th=[ 1448],
| 70.00th=[ 4448], 80.00th=[ 7904], 90.00th=[11712], 95.00th=[14272],
| 99.00th=[18560], 99.50th=[20352], 99.90th=[26496], 99.95th=[32384],
| 99.99th=[44288]
bw (KB /s): min=86912, max=112512, per=100.00%, avg=97762.42, stdev=7691.94
lat (usec) : 250=8.28%, 500=10.08%, 750=5.44%, 1000=3.32%
lat (msec) : 2=4.69%, 4=4.05%, 10=18.47%, 20=29.07%, 50=16.19%
lat (msec) : 100=0.42%, 250=0.01%
cpu : usr=4.80%, sys=6.66%, ctx=163653, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=120.5%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=67922/w=67951/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4247.7MB, aggrb=97574KB/s, minb=97574KB/s, maxb=97574KB/s, mint=44571msec, maxt=44571msec
WRITE: io=4246.1MB, aggrb=97571KB/s, minb=97571KB/s, maxb=97571KB/s, mint=44571msec, maxt=44571msec
Disk stats (read/write):
sdj: ios=81819/81750, merge=27/27, ticks=1492043/261958, in_queue=1754272, util=99.92%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdj-randrw-32-128K
===
mytest: (g=0): rw=randrw, bs=128K-128K/128K-128K/128K-128K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=24868: Sun Sep 15 14:21:15 2013
read : io=4192.2MB, bw=107725KB/s, iops=840, runt= 39849msec
slat (usec): min=13, max=236, avg=23.50, stdev= 6.06
clat (usec): min=990, max=112885, avg=28593.89, stdev=15167.20
lat (msec): min=1, max=112, avg=28.62, stdev=15.17
clat percentiles (msec):
| 1.00th=[ 5], 5.00th=[ 9], 10.00th=[ 12], 20.00th=[ 16],
| 30.00th=[ 20], 40.00th=[ 24], 50.00th=[ 27], 60.00th=[ 31],
| 70.00th=[ 35], 80.00th=[ 40], 90.00th=[ 49], 95.00th=[ 58],
| 99.00th=[ 76], 99.50th=[ 85], 99.90th=[ 98], 99.95th=[ 101],
| 99.99th=[ 112]
bw (KB /s): min=87888, max=131824, per=100.00%, avg=107889.33, stdev=13212.41
write: io=4188.9MB, bw=107642KB/s, iops=840, runt= 39849msec
slat (usec): min=22, max=22416, avg=45.95, stdev=149.38
clat (usec): min=237, max=54701, avg=9377.71, stdev=7729.48
lat (usec): min=320, max=54747, avg=9424.30, stdev=7728.45
clat percentiles (usec):
| 1.00th=[ 314], 5.00th=[ 502], 10.00th=[ 1004], 20.00th=[ 2416],
| 30.00th=[ 4080], 40.00th=[ 5856], 50.00th=[ 7520], 60.00th=[ 9280],
| 70.00th=[11840], 80.00th=[15552], 90.00th=[21376], 95.00th=[25216],
| 99.00th=[31104], 99.50th=[33024], 99.90th=[37632], 99.95th=[39680],
| 99.99th=[43776]
bw (KB /s): min= 12, max=132608, per=98.99%, avg=106558.09, stdev=17832.05
lat (usec) : 250=0.01%, 500=2.45%, 750=1.29%, 1000=1.22%
lat (msec) : 2=3.73%, 4=6.51%, 10=20.45%, 20=23.81%, 50=36.00%
lat (msec) : 100=4.55%, 250=0.03%
cpu : usr=3.90%, sys=6.27%, ctx=81724, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=122.1%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=33506/w=33511/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4192.2MB, aggrb=107725KB/s, minb=107725KB/s, maxb=107725KB/s, mint=39849msec, maxt=39849msec
WRITE: io=4188.9MB, aggrb=107641KB/s, minb=107641KB/s, maxb=107641KB/s, mint=39849msec, maxt=39849msec
Disk stats (read/write):
sdj: ios=40939/40831, merge=43/37, ticks=1259840/339319, in_queue=1599691, util=99.90%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdj-randrw-32-256K
===
mytest: (g=0): rw=randrw, bs=256K-256K/256K-256K/256K-256K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=24996: Sun Sep 15 14:22:51 2013
read : io=3722.6MB, bw=149051KB/s, iops=581, runt= 25574msec
slat (usec): min=20, max=90, avg=36.10, stdev= 5.21
clat (msec): min=1, max=86, avg=39.94, stdev=15.82
lat (msec): min=1, max=86, avg=39.98, stdev=15.82
clat percentiles (usec):
| 1.00th=[ 7072], 5.00th=[12352], 10.00th=[16768], 20.00th=[24192],
| 30.00th=[31104], 40.00th=[37632], 50.00th=[42240], 60.00th=[46336],
| 70.00th=[49920], 80.00th=[54016], 90.00th=[59136], 95.00th=[63232],
| 99.00th=[70144], 99.50th=[73216], 99.90th=[80384], 99.95th=[81408],
| 99.99th=[84480]
bw (KB /s): min=128512, max=164352, per=99.73%, avg=148643.82, stdev=8882.02
write: io=3683.6MB, bw=147490KB/s, iops=576, runt= 25574msec
slat (usec): min=53, max=29846, avg=79.52, stdev=284.41
clat (usec): min=539, max=75407, avg=15091.08, stdev=15013.51
lat (usec): min=607, max=75469, avg=15172.19, stdev=15013.84
clat percentiles (usec):
| 1.00th=[ 564], 5.00th=[ 620], 10.00th=[ 956], 20.00th=[ 1480],
| 30.00th=[ 2096], 40.00th=[ 3664], 50.00th=[ 9280], 60.00th=[16768],
| 70.00th=[23168], 80.00th=[30336], 90.00th=[38144], 95.00th=[43264],
| 99.00th=[51968], 99.50th=[54528], 99.90th=[61696], 99.95th=[65280],
| 99.99th=[75264]
bw (KB /s): min= 25, max=164864, per=98.17%, avg=144790.78, stdev=21864.97
lat (usec) : 750=4.06%, 1000=1.90%
lat (msec) : 2=7.95%, 4=6.66%, 10=6.28%, 20=12.65%, 50=44.78%
lat (msec) : 100=15.83%
cpu : usr=4.34%, sys=7.05%, ctx=40785, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=138.2%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=14860/w=14733/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=3722.6MB, aggrb=149051KB/s, minb=149051KB/s, maxb=149051KB/s, mint=25574msec, maxt=25574msec
WRITE: io=3683.6MB, aggrb=147489KB/s, minb=147489KB/s, maxb=147489KB/s, mint=25574msec, maxt=25574msec
Disk stats (read/write):
sdj: ios=20501/20289, merge=56/61, ticks=911239/230918, in_queue=1143098, util=99.83%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdj-randrw-32-512K
===
mytest: (g=0): rw=randrw, bs=512K-512K/512K-512K/512K-512K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=25121: Sun Sep 15 14:24:22 2013
read : io=3444.6MB, bw=165137KB/s, iops=321, runt= 21359msec
slat (usec): min=39, max=218, avg=65.43, stdev= 6.90
clat (msec): min=3, max=135, avg=56.95, stdev=23.38
lat (msec): min=3, max=135, avg=57.02, stdev=23.39
clat percentiles (msec):
| 1.00th=[ 11], 5.00th=[ 19], 10.00th=[ 25], 20.00th=[ 37],
| 30.00th=[ 46], 40.00th=[ 51], 50.00th=[ 57], 60.00th=[ 63],
| 70.00th=[ 70], 80.00th=[ 78], 90.00th=[ 88], 95.00th=[ 96],
| 99.00th=[ 111], 99.50th=[ 117], 99.90th=[ 131], 99.95th=[ 133],
| 99.99th=[ 135]
bw (KB /s): min= 51, max=191488, per=97.67%, avg=161281.50, stdev=27898.36
write: io=3489.6MB, bw=167295KB/s, iops=326, runt= 21359msec
slat (usec): min=76, max=52778, avg=152.34, stdev=695.69
clat (msec): min=1, max=128, avg=41.58, stdev=22.61
lat (msec): min=1, max=128, avg=41.73, stdev=22.60
clat percentiles (msec):
| 1.00th=[ 3], 5.00th=[ 9], 10.00th=[ 13], 20.00th=[ 22],
| 30.00th=[ 29], 40.00th=[ 35], 50.00th=[ 40], 60.00th=[ 45],
| 70.00th=[ 52], 80.00th=[ 62], 90.00th=[ 75], 95.00th=[ 83],
| 99.00th=[ 100], 99.50th=[ 105], 99.90th=[ 115], 99.95th=[ 126],
| 99.99th=[ 129]
bw (KB /s): min=141312, max=192512, per=99.77%, avg=166908.49, stdev=12404.64
lat (msec) : 2=0.17%, 4=0.61%, 10=2.93%, 20=8.87%, 50=41.11%
lat (msec) : 100=44.32%, 250=2.20%
cpu : usr=4.42%, sys=7.19%, ctx=20427, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.2%, 32=147.6%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=6870/w=6967/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=3444.6MB, aggrb=165137KB/s, minb=165137KB/s, maxb=165137KB/s, mint=21359msec, maxt=21359msec
WRITE: io=3489.6MB, aggrb=167294KB/s, minb=167294KB/s, maxb=167294KB/s, mint=21359msec, maxt=21359msec
Disk stats (read/write):
sdj: ios=10311/10184, merge=0/0, ticks=614540/392810, in_queue=1008379, util=99.79%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdj-randrw-32-1024K
===
mytest: (g=0): rw=randrw, bs=1M-1M/1M-1M/1M-1M, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=25200: Sun Sep 15 14:25:52 2013
read : io=3411.0MB, bw=179462KB/s, iops=174, runt= 19463msec
slat (usec): min=73, max=283, avg=120.12, stdev= 9.13
clat (msec): min=6, max=568, avg=99.23, stdev=53.91
lat (msec): min=6, max=568, avg=99.35, stdev=53.91
clat percentiles (msec):
| 1.00th=[ 13], 5.00th=[ 25], 10.00th=[ 37], 20.00th=[ 57],
| 30.00th=[ 70], 40.00th=[ 82], 50.00th=[ 94], 60.00th=[ 106],
| 70.00th=[ 119], 80.00th=[ 135], 90.00th=[ 163], 95.00th=[ 196],
| 99.00th=[ 281], 99.50th=[ 314], 99.90th=[ 408], 99.95th=[ 433],
| 99.99th=[ 570]
bw (KB /s): min= 102, max=206023, per=97.41%, avg=174812.23, stdev=33282.23
write: io=3340.0MB, bw=175726KB/s, iops=170, runt= 19463msec
slat (usec): min=186, max=93571, avg=319.89, stdev=2087.60
clat (msec): min=3, max=467, avg=85.06, stdev=52.25
lat (msec): min=3, max=467, avg=85.41, stdev=52.19
clat percentiles (msec):
| 1.00th=[ 9], 5.00th=[ 18], 10.00th=[ 27], 20.00th=[ 42],
| 30.00th=[ 55], 40.00th=[ 65], 50.00th=[ 78], 60.00th=[ 90],
| 70.00th=[ 103], 80.00th=[ 120], 90.00th=[ 149], 95.00th=[ 180],
| 99.00th=[ 273], 99.50th=[ 297], 99.90th=[ 322], 99.95th=[ 371],
| 99.99th=[ 469]
bw (KB /s): min=145117, max=206435, per=100.00%, avg=175939.05, stdev=14275.00
lat (msec) : 4=0.01%, 10=0.88%, 20=3.81%, 50=16.98%, 100=39.96%
lat (msec) : 250=37.34%, 500=1.47%, 750=0.01%
cpu : usr=4.48%, sys=6.77%, ctx=10165, majf=0, minf=28
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.2%, 16=0.5%, 32=151.5%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=3398/w=3322/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=3411.0MB, aggrb=179461KB/s, minb=179461KB/s, maxb=179461KB/s, mint=19463msec, maxt=19463msec
WRITE: io=3340.0MB, aggrb=175726KB/s, minb=175726KB/s, maxb=175726KB/s, mint=19463msec, maxt=19463msec
Disk stats (read/write):
sdj: ios=10431/9978, merge=0/0, ticks=1032758/798939, in_queue=1835794, util=99.74%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdk-randrw-32-512B
===
mytest: (g=0): rw=randrw, bs=512-512/512-512/512-512, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=25321: Sun Sep 15 14:38:51 2013
read : io=5044.8MB, bw=7288.1KB/s, iops=14577, runt=708718msec
slat (usec): min=2, max=375, avg= 7.21, stdev= 4.15
clat (usec): min=20, max=82568, avg=2051.49, stdev=2100.50
lat (usec): min=34, max=82579, avg=2058.85, stdev=2100.54
clat percentiles (usec):
| 1.00th=[ 149], 5.00th=[ 318], 10.00th=[ 442], 20.00th=[ 628],
| 30.00th=[ 788], 40.00th=[ 948], 50.00th=[ 1160], 60.00th=[ 1528],
| 70.00th=[ 2480], 80.00th=[ 3536], 90.00th=[ 4896], 95.00th=[ 5920],
| 99.00th=[ 8032], 99.50th=[ 9280], 99.90th=[22400], 99.95th=[25728],
| 99.99th=[29824]
bw (KB /s): min= 0, max= 8891, per=100.00%, avg=7289.39, stdev=572.52
write: io=5043.1MB, bw=7287.8KB/s, iops=14575, runt=708718msec
slat (usec): min=3, max=3794, avg= 8.49, stdev= 5.08
clat (usec): min=0, max=23258, avg=124.56, stdev=236.85
lat (usec): min=32, max=23268, avg=133.21, stdev=236.76
clat percentiles (usec):
| 1.00th=[ 31], 5.00th=[ 35], 10.00th=[ 37], 20.00th=[ 45],
| 30.00th=[ 56], 40.00th=[ 68], 50.00th=[ 83], 60.00th=[ 102],
| 70.00th=[ 127], 80.00th=[ 161], 90.00th=[ 227], 95.00th=[ 290],
| 99.00th=[ 548], 99.50th=[ 1112], 99.90th=[ 3952], 99.95th=[ 4576],
| 99.99th=[ 5728]
bw (KB /s): min= 2871, max= 8991, per=100.00%, avg=7293.43, stdev=534.70
lat (usec) : 2=0.01%, 10=0.01%, 20=0.01%, 50=12.16%, 100=17.51%
lat (usec) : 250=17.80%, 500=8.44%, 750=7.54%, 1000=7.71%
lat (msec) : 2=11.42%, 4=9.21%, 10=8.02%, 20=0.12%, 50=0.07%
lat (msec) : 100=0.01%
cpu : usr=7.58%, sys=28.22%, ctx=5961591, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=101.5%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=10331561/w=10329956/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=5044.8MB, aggrb=7288KB/s, minb=7288KB/s, maxb=7288KB/s, mint=708718msec, maxt=708718msec
WRITE: io=5043.1MB, aggrb=7287KB/s, minb=7287KB/s, maxb=7287KB/s, mint=708718msec, maxt=708718msec
Disk stats (read/write):
sdk: ios=10485115/10482848, merge=0/1, ticks=21308397/1252876, in_queue=22554533, util=100.00%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdk-randrw-32-1K
===
mytest: (g=0): rw=randrw, bs=1K-1K/1K-1K/1K-1K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=26738: Sun Sep 15 14:45:32 2013
read : io=4966.6MB, bw=15417KB/s, iops=15416, runt=329878msec
slat (usec): min=2, max=385, avg= 7.04, stdev= 4.54
clat (usec): min=37, max=36033, avg=1914.09, stdev=2152.34
lat (usec): min=46, max=36037, avg=1921.24, stdev=2152.41
clat percentiles (usec):
| 1.00th=[ 181], 5.00th=[ 270], 10.00th=[ 334], 20.00th=[ 466],
| 30.00th=[ 604], 40.00th=[ 748], 50.00th=[ 924], 60.00th=[ 1240],
| 70.00th=[ 2416], 80.00th=[ 3568], 90.00th=[ 4896], 95.00th=[ 5792],
| 99.00th=[ 7840], 99.50th=[ 9024], 99.90th=[23424], 99.95th=[25984],
| 99.99th=[29568]
bw (KB /s): min= 0, max=16994, per=99.95%, avg=15408.08, stdev=1288.71
write: io=4965.5MB, bw=15414KB/s, iops=15413, runt=329878msec
slat (usec): min=3, max=4296, avg= 8.37, stdev= 5.79
clat (usec): min=0, max=20656, avg=142.68, stdev=321.00
lat (usec): min=32, max=20663, avg=151.19, stdev=321.11
clat percentiles (usec):
| 1.00th=[ 29], 5.00th=[ 34], 10.00th=[ 37], 20.00th=[ 45],
| 30.00th=[ 55], 40.00th=[ 68], 50.00th=[ 84], 60.00th=[ 105],
| 70.00th=[ 135], 80.00th=[ 175], 90.00th=[ 247], 95.00th=[ 314],
| 99.00th=[ 1020], 99.50th=[ 2640], 99.90th=[ 4512], 99.95th=[ 4960],
| 99.99th=[ 6816]
bw (KB /s): min=10672, max=16742, per=100.00%, avg=15427.90, stdev=1130.17
lat (usec) : 2=0.01%, 10=0.01%, 20=0.01%, 50=12.50%, 100=16.39%
lat (usec) : 250=18.04%, 500=13.38%, 750=9.12%, 1000=6.68%
lat (msec) : 2=7.11%, 4=8.43%, 10=8.15%, 20=0.11%, 50=0.08%
cpu : usr=7.32%, sys=29.16%, ctx=1990520, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=103.1%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=5085692/w=5084595/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4966.6MB, aggrb=15416KB/s, minb=15416KB/s, maxb=15416KB/s, mint=329878msec, maxt=329878msec
WRITE: io=4965.5MB, aggrb=15413KB/s, minb=15413KB/s, maxb=15413KB/s, mint=329878msec, maxt=329878msec
Disk stats (read/write):
sdk: ios=5242408/5240689, merge=1/0, ticks=9873862/657580, in_queue=10527437, util=100.00%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdk-randrw-32-2K
===
mytest: (g=0): rw=randrw, bs=2K-2K/2K-2K/2K-2K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=27094: Sun Sep 15 14:49:25 2013
read : io=4813.4MB, bw=30307KB/s, iops=15153, runt=162620msec
slat (usec): min=2, max=232, avg= 7.30, stdev= 4.14
clat (usec): min=38, max=37882, avg=1879.87, stdev=2138.12
lat (usec): min=47, max=37886, avg=1887.24, stdev=2138.23
clat percentiles (usec):
| 1.00th=[ 185], 5.00th=[ 274], 10.00th=[ 326], 20.00th=[ 426],
| 30.00th=[ 548], 40.00th=[ 716], 50.00th=[ 916], 60.00th=[ 1208],
| 70.00th=[ 2384], 80.00th=[ 3504], 90.00th=[ 4832], 95.00th=[ 5792],
| 99.00th=[ 7776], 99.50th=[ 9024], 99.90th=[23168], 99.95th=[25984],
| 99.99th=[29312]
bw (KB /s): min=21148, max=33644, per=100.00%, avg=30337.80, stdev=2791.81
write: io=4811.4MB, bw=30296KB/s, iops=15148, runt=162620msec
slat (usec): min=3, max=4684, avg= 8.65, stdev= 6.07
clat (usec): min=1, max=19831, avg=212.80, stdev=458.26
lat (usec): min=36, max=19839, avg=221.56, stdev=458.12
clat percentiles (usec):
| 1.00th=[ 34], 5.00th=[ 38], 10.00th=[ 41], 20.00th=[ 54],
| 30.00th=[ 70], 40.00th=[ 92], 50.00th=[ 120], 60.00th=[ 155],
| 70.00th=[ 199], 80.00th=[ 249], 90.00th=[ 326], 95.00th=[ 454],
| 99.00th=[ 2800], 99.50th=[ 3728], 99.90th=[ 5024], 99.95th=[ 5536],
| 99.99th=[ 9024]
bw (KB /s): min= 0, max=33504, per=99.80%, avg=30234.78, stdev=3257.38
lat (usec) : 2=0.01%, 20=0.01%, 50=8.41%, 100=13.09%, 250=20.38%
lat (usec) : 500=19.03%, 750=8.36%, 1000=6.40%
lat (msec) : 2=7.38%, 4=8.73%, 10=8.05%, 20=0.11%, 50=0.08%
cpu : usr=7.55%, sys=30.45%, ctx=1119737, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=106.4%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=2464244/w=2463398/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4813.4MB, aggrb=30307KB/s, minb=30307KB/s, maxb=30307KB/s, mint=162620msec, maxt=162620msec
WRITE: io=4811.4MB, aggrb=30296KB/s, minb=30296KB/s, maxb=30296KB/s, mint=162620msec, maxt=162620msec
Disk stats (read/write):
sdk: ios=2621245/2619756, merge=3/4, ticks=4883718/527170, in_queue=5408972, util=100.00%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdk-randrw-32-4K
===
mytest: (g=0): rw=randrw, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=27333: Sun Sep 15 14:51:48 2013
read : io=4466.5MB, bw=62404KB/s, iops=15600, runt= 73284msec
slat (usec): min=2, max=363, avg= 7.17, stdev= 4.75
clat (usec): min=84, max=32410, avg=1630.13, stdev=1519.01
lat (usec): min=111, max=32414, avg=1637.17, stdev=1519.05
clat percentiles (usec):
| 1.00th=[ 247], 5.00th=[ 394], 10.00th=[ 458], 20.00th=[ 532],
| 30.00th=[ 596], 40.00th=[ 844], 50.00th=[ 1112], 60.00th=[ 1688],
| 70.00th=[ 2384], 80.00th=[ 2672], 90.00th=[ 3056], 95.00th=[ 3632],
| 99.00th=[ 5280], 99.50th=[ 5984], 99.90th=[20352], 99.95th=[22656],
| 99.99th=[26496]
bw (KB /s): min= 0, max=70336, per=99.33%, avg=61988.23, stdev=8207.69
write: io=4464.7MB, bw=62385KB/s, iops=15596, runt= 73284msec
slat (usec): min=3, max=3842, avg= 9.00, stdev= 6.85
clat (usec): min=0, max=20448, avg=401.86, stdev=580.98
lat (usec): min=38, max=20458, avg=410.81, stdev=580.85
clat percentiles (usec):
| 1.00th=[ 38], 5.00th=[ 44], 10.00th=[ 71], 20.00th=[ 141],
| 30.00th=[ 211], 40.00th=[ 270], 50.00th=[ 310], 60.00th=[ 346],
| 70.00th=[ 386], 80.00th=[ 426], 90.00th=[ 540], 95.00th=[ 1096],
| 99.00th=[ 3088], 99.50th=[ 3472], 99.90th=[ 5280], 99.95th=[ 5728],
| 99.99th=[15040]
bw (KB /s): min=47160, max=69952, per=100.00%, avg=62402.66, stdev=6435.84
lat (usec) : 2=0.01%, 10=0.01%, 20=0.01%, 50=3.10%, 100=3.95%
lat (usec) : 250=11.58%, 500=33.59%, 750=12.50%, 1000=5.49%
lat (msec) : 2=9.67%, 4=17.93%, 10=2.03%, 20=0.11%, 50=0.05%
cpu : usr=10.42%, sys=34.65%, ctx=921088, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=114.7%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=1143277/w=1142958/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4466.5MB, aggrb=62404KB/s, minb=62404KB/s, maxb=62404KB/s, mint=73284msec, maxt=73284msec
WRITE: io=4464.7MB, aggrb=62385KB/s, minb=62385KB/s, maxb=62385KB/s, mint=73284msec, maxt=73284msec
Disk stats (read/write):
sdk: ios=1310363/1309334, merge=10/7, ticks=2123582/496666, in_queue=2619249, util=99.98%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdk-randrw-32-8K
===
mytest: (g=0): rw=randrw, bs=8K-8K/8K-8K/8K-8K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=27511: Sun Sep 15 14:54:01 2013
read : io=4350.7MB, bw=71385KB/s, iops=8922, runt= 62408msec
slat (usec): min=2, max=403, avg= 8.60, stdev= 4.97
clat (usec): min=142, max=38809, avg=2975.97, stdev=2267.23
lat (usec): min=147, max=38816, avg=2984.61, stdev=2267.17
clat percentiles (usec):
| 1.00th=[ 350], 5.00th=[ 588], 10.00th=[ 740], 20.00th=[ 1192],
| 30.00th=[ 1656], 40.00th=[ 2416], 50.00th=[ 2832], 60.00th=[ 3120],
| 70.00th=[ 3504], 80.00th=[ 4128], 90.00th=[ 5408], 95.00th=[ 6112],
| 99.00th=[ 8512], 99.50th=[15296], 99.90th=[26496], 99.95th=[28800],
| 99.99th=[32640]
bw (KB /s): min= 0, max=82240, per=99.27%, avg=70862.51, stdev=10254.29
write: io=4345.8MB, bw=71305KB/s, iops=8913, runt= 62408msec
slat (usec): min=4, max=2964, avg=11.22, stdev= 7.54
clat (usec): min=0, max=21990, avg=587.20, stdev=989.57
lat (usec): min=47, max=22006, avg=598.60, stdev=989.69
clat percentiles (usec):
| 1.00th=[ 46], 5.00th=[ 49], 10.00th=[ 61], 20.00th=[ 105],
| 30.00th=[ 165], 40.00th=[ 235], 50.00th=[ 310], 60.00th=[ 386],
| 70.00th=[ 462], 80.00th=[ 548], 90.00th=[ 1320], 95.00th=[ 2992],
| 99.00th=[ 4704], 99.50th=[ 5664], 99.90th=[ 7648], 99.95th=[ 9536],
| 99.99th=[17024]
bw (KB /s): min=56192, max=82704, per=100.00%, avg=71369.10, stdev=7854.45
lat (usec) : 2=0.01%, 10=0.01%, 20=0.01%, 50=2.61%, 100=6.83%
lat (usec) : 250=11.69%, 500=17.69%, 750=9.62%, 1000=3.83%
lat (msec) : 2=11.14%, 4=24.89%, 10=11.35%, 20=0.17%, 50=0.18%
cpu : usr=9.66%, sys=24.59%, ctx=826188, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=117.7%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=556845/w=556248/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4350.7MB, aggrb=71385KB/s, minb=71385KB/s, maxb=71385KB/s, mint=62408msec, maxt=62408msec
WRITE: io=4345.8MB, aggrb=71304KB/s, minb=71304KB/s, maxb=71304KB/s, mint=62408msec, maxt=62408msec
Disk stats (read/write):
sdk: ios=654004/653481, merge=8/5, ticks=1953033/355084, in_queue=2307563, util=99.97%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdk-randrw-32-16K
===
mytest: (g=0): rw=randrw, bs=16K-16K/16K-16K/16K-16K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=27644: Sun Sep 15 14:56:11 2013
read : io=4328.6MB, bw=75210KB/s, iops=4700, runt= 58934msec
slat (usec): min=5, max=236, avg=10.10, stdev= 2.18
clat (usec): min=175, max=57682, avg=5809.56, stdev=3778.27
lat (usec): min=197, max=57693, avg=5819.89, stdev=3778.49
clat percentiles (usec):
| 1.00th=[ 580], 5.00th=[ 1176], 10.00th=[ 1832], 20.00th=[ 3056],
| 30.00th=[ 3728], 40.00th=[ 4448], 50.00th=[ 5344], 60.00th=[ 6176],
| 70.00th=[ 7008], 80.00th=[ 8160], 90.00th=[ 9792], 95.00th=[11328],
| 99.00th=[16768], 99.50th=[27776], 99.90th=[37120], 99.95th=[39680],
| 99.99th=[45312]
bw (KB /s): min= 1, max=84224, per=99.39%, avg=74751.44, stdev=8408.94
write: io=4316.0MB, bw=74992KB/s, iops=4686, runt= 58934msec
slat (usec): min=9, max=6294, avg=14.15, stdev=14.40
clat (usec): min=1, max=25374, avg=971.57, stdev=1694.17
lat (usec): min=74, max=25387, avg=986.23, stdev=1693.81
clat percentiles (usec):
| 1.00th=[ 63], 5.00th=[ 66], 10.00th=[ 77], 20.00th=[ 106],
| 30.00th=[ 153], 40.00th=[ 219], 50.00th=[ 306], 60.00th=[ 422],
| 70.00th=[ 588], 80.00th=[ 932], 90.00th=[ 3440], 95.00th=[ 4768],
| 99.00th=[ 7264], 99.50th=[ 8256], 99.90th=[13632], 99.95th=[17536],
| 99.99th=[22144]
bw (KB /s): min=65056, max=83960, per=100.00%, avg=75162.55, stdev=4468.41
lat (usec) : 2=0.01%, 50=0.01%, 100=9.09%, 250=12.91%, 500=10.92%
lat (usec) : 750=6.02%, 1000=3.20%
lat (msec) : 2=5.88%, 4=15.01%, 10=32.20%, 20=4.35%, 50=0.43%
lat (msec) : 100=0.01%
cpu : usr=8.35%, sys=15.85%, ctx=597868, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=118.5%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=277003/w=276218/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4328.6MB, aggrb=75210KB/s, minb=75210KB/s, maxb=75210KB/s, mint=58934msec, maxt=58934msec
WRITE: io=4316.0MB, aggrb=74992KB/s, minb=74992KB/s, maxb=74992KB/s, mint=58934msec, maxt=58934msec
Disk stats (read/write):
sdk: ios=327427/326375, merge=20/16, ticks=1921262/286912, in_queue=2208260, util=99.97%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdk-randrw-32-32K
===
mytest: (g=0): rw=randrw, bs=32K-32K/32K-32K/32K-32K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=27772: Sun Sep 15 14:58:13 2013
read : io=4300.2MB, bw=84435KB/s, iops=2638, runt= 52151msec
slat (usec): min=6, max=349, avg=12.16, stdev= 3.83
clat (usec): min=301, max=76511, avg=10228.72, stdev=5963.16
lat (usec): min=313, max=76523, avg=10241.15, stdev=5963.57
clat percentiles (usec):
| 1.00th=[ 1144], 5.00th=[ 2832], 10.00th=[ 3824], 20.00th=[ 5472],
| 30.00th=[ 6880], 40.00th=[ 8096], 50.00th=[ 9408], 60.00th=[10688],
| 70.00th=[12224], 80.00th=[14272], 90.00th=[17280], 95.00th=[19840],
| 99.00th=[29568], 99.50th=[40192], 99.90th=[52480], 99.95th=[55552],
| 99.99th=[61696]
bw (KB /s): min=76608, max=91776, per=100.00%, avg=84591.43, stdev=3263.04
write: io=4303.5MB, bw=84499KB/s, iops=2640, runt= 52151msec
slat (usec): min=10, max=9977, avg=19.85, stdev=27.15
clat (usec): min=36, max=31416, avg=1859.96, stdev=2934.60
lat (usec): min=108, max=31433, avg=1880.66, stdev=2934.64
clat percentiles (usec):
| 1.00th=[ 94], 5.00th=[ 96], 10.00th=[ 111], 20.00th=[ 149],
| 30.00th=[ 221], 40.00th=[ 334], 50.00th=[ 466], 60.00th=[ 668],
| 70.00th=[ 1080], 80.00th=[ 3600], 90.00th=[ 6560], 95.00th=[ 8384],
| 99.00th=[11584], 99.50th=[13248], 99.90th=[20864], 99.95th=[23168],
| 99.99th=[27520]
bw (KB /s): min= 3, max=92160, per=99.18%, avg=83807.76, stdev=8769.91
lat (usec) : 50=0.01%, 100=3.58%, 250=12.40%, 500=10.13%, 750=5.55%
lat (usec) : 1000=3.12%
lat (msec) : 2=4.39%, 4=7.26%, 10=29.68%, 20=21.41%, 50=2.39%
lat (msec) : 100=0.08%
cpu : usr=6.41%, sys=11.17%, ctx=320167, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=119.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=137578/w=137705/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4300.2MB, aggrb=84434KB/s, minb=84434KB/s, maxb=84434KB/s, mint=52151msec, maxt=52151msec
WRITE: io=4303.5MB, aggrb=84498KB/s, minb=84498KB/s, maxb=84498KB/s, mint=52151msec, maxt=52151msec
Disk stats (read/write):
sdk: ios=163458/163364, merge=22/19, ticks=1724159/266503, in_queue=1990841, util=99.93%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdk-randrw-32-64K
===
mytest: (g=0): rw=randrw, bs=64K-64K/64K-64K/64K-64K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=28244: Sun Sep 15 15:00:09 2013
read : io=4257.4MB, bw=95294KB/s, iops=1488, runt= 45748msec
slat (usec): min=8, max=225, avg=14.28, stdev= 3.26
clat (usec): min=634, max=91163, avg=17919.20, stdev=9903.35
lat (usec): min=646, max=91177, avg=17933.66, stdev=9904.15
clat percentiles (usec):
| 1.00th=[ 2704], 5.00th=[ 4832], 10.00th=[ 6816], 20.00th=[ 9664],
| 30.00th=[11968], 40.00th=[14144], 50.00th=[16320], 60.00th=[18560],
| 70.00th=[21376], 80.00th=[25216], 90.00th=[30848], 95.00th=[36608],
| 99.00th=[48896], 99.50th=[54528], 99.90th=[69120], 99.95th=[72192],
| 99.99th=[77312]
bw (KB /s): min= 6, max=112256, per=98.91%, avg=94258.43, stdev=12954.90
write: io=4257.5MB, bw=95296KB/s, iops=1489, runt= 45748msec
slat (usec): min=15, max=13119, avg=29.31, stdev=64.67
clat (usec): min=113, max=36494, avg=3525.50, stdev=4885.42
lat (usec): min=180, max=36517, avg=3556.24, stdev=4887.02
clat percentiles (usec):
| 1.00th=[ 157], 5.00th=[ 171], 10.00th=[ 187], 20.00th=[ 255],
| 30.00th=[ 374], 40.00th=[ 516], 50.00th=[ 764], 60.00th=[ 1272],
| 70.00th=[ 4080], 80.00th=[ 7584], 90.00th=[11584], 95.00th=[14272],
| 99.00th=[18560], 99.50th=[19840], 99.90th=[23168], 99.95th=[24448],
| 99.99th=[28800]
bw (KB /s): min=83584, max=110278, per=100.00%, avg=95488.90, stdev=7852.35
lat (usec) : 250=9.42%, 500=9.81%, 750=5.55%, 1000=3.27%
lat (msec) : 2=4.52%, 4=3.71%, 10=17.53%, 20=28.53%, 50=17.23%
lat (msec) : 100=0.44%
cpu : usr=5.22%, sys=6.85%, ctx=163575, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=120.2%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=68086/w=68119/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4257.4MB, aggrb=95293KB/s, minb=95293KB/s, maxb=95293KB/s, mint=45748msec, maxt=45748msec
WRITE: io=4257.5MB, aggrb=95296KB/s, minb=95296KB/s, maxb=95296KB/s, mint=45748msec, maxt=45748msec
Disk stats (read/write):
sdk: ios=81934/81851, merge=17/24, ticks=1542593/249851, in_queue=1792596, util=99.92%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdk-randrw-32-128K
===
mytest: (g=0): rw=randrw, bs=128K-128K/128K-128K/128K-128K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=28395: Sun Sep 15 15:01:58 2013
read : io=4207.0MB, bw=111208KB/s, iops=868, runt= 38738msec
slat (usec): min=12, max=221, avg=21.46, stdev= 4.25
clat (usec): min=574, max=119817, avg=26645.38, stdev=14835.64
lat (usec): min=599, max=119841, avg=26666.77, stdev=14836.59
clat percentiles (msec):
| 1.00th=[ 4], 5.00th=[ 7], 10.00th=[ 10], 20.00th=[ 14],
| 30.00th=[ 18], 40.00th=[ 22], 50.00th=[ 25], 60.00th=[ 29],
| 70.00th=[ 32], 80.00th=[ 38], 90.00th=[ 47], 95.00th=[ 56],
| 99.00th=[ 73], 99.50th=[ 79], 99.90th=[ 91], 99.95th=[ 97],
| 99.99th=[ 111]
bw (KB /s): min=86355, max=142592, per=100.00%, avg=111334.04, stdev=15782.87
write: io=4203.4MB, bw=111112KB/s, iops=868, runt= 38738msec
slat (usec): min=21, max=16951, avg=43.45, stdev=132.45
clat (usec): min=286, max=44394, avg=10141.25, stdev=7461.01
lat (usec): min=325, max=44434, avg=10185.43, stdev=7460.77
clat percentiles (usec):
| 1.00th=[ 314], 5.00th=[ 724], 10.00th=[ 1400], 20.00th=[ 3280],
| 30.00th=[ 5152], 40.00th=[ 6944], 50.00th=[ 8768], 60.00th=[10816],
| 70.00th=[13120], 80.00th=[16192], 90.00th=[20864], 95.00th=[24704],
| 99.00th=[31104], 99.50th=[33024], 99.90th=[37120], 99.95th=[38656],
| 99.99th=[42752]
bw (KB /s): min= 12, max=144384, per=99.00%, avg=110004.29, stdev=20252.41
lat (usec) : 500=1.74%, 750=0.97%, 1000=0.94%
lat (msec) : 2=3.10%, 4=5.74%, 10=21.00%, 20=29.10%, 50=33.62%
lat (msec) : 100=3.83%, 250=0.01%
cpu : usr=3.74%, sys=5.80%, ctx=81742, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=121.7%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=33625/w=33627/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4207.0MB, aggrb=111207KB/s, minb=111207KB/s, maxb=111207KB/s, mint=38738msec, maxt=38738msec
WRITE: io=4203.4MB, aggrb=111111KB/s, minb=111111KB/s, maxb=111111KB/s, mint=38738msec, maxt=38738msec
Disk stats (read/write):
sdk: ios=40993/40898, merge=49/39, ticks=1204750/364335, in_queue=1569453, util=99.90%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdk-randrw-32-256K
===
mytest: (g=0): rw=randrw, bs=256K-256K/256K-256K/256K-256K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=28519: Sun Sep 15 15:03:35 2013
read : io=3746.0MB, bw=147927KB/s, iops=576, runt= 25931msec
slat (usec): min=20, max=221, avg=37.97, stdev= 7.46
clat (msec): min=2, max=85, avg=40.38, stdev=15.46
lat (msec): min=2, max=85, avg=40.41, stdev=15.46
clat percentiles (usec):
| 1.00th=[ 7072], 5.00th=[12864], 10.00th=[17536], 20.00th=[25216],
| 30.00th=[32384], 40.00th=[38144], 50.00th=[42752], 60.00th=[46848],
| 70.00th=[49920], 80.00th=[54016], 90.00th=[58624], 95.00th=[62720],
| 99.00th=[71168], 99.50th=[74240], 99.90th=[79360], 99.95th=[81408],
| 99.99th=[84480]
bw (KB /s): min= 25, max=168960, per=97.86%, avg=144766.90, stdev=22430.33
write: io=3704.8MB, bw=146298KB/s, iops=571, runt= 25931msec
slat (usec): min=49, max=24881, avg=79.65, stdev=188.11
clat (usec): min=532, max=86991, avg=15098.71, stdev=15183.91
lat (usec): min=597, max=87076, avg=15179.45, stdev=15182.77
clat percentiles (usec):
| 1.00th=[ 556], 5.00th=[ 612], 10.00th=[ 964], 20.00th=[ 1496],
| 30.00th=[ 2256], 40.00th=[ 3984], 50.00th=[ 8768], 60.00th=[16320],
| 70.00th=[22912], 80.00th=[30080], 90.00th=[38656], 95.00th=[44288],
| 99.00th=[52992], 99.50th=[55040], 99.90th=[62720], 99.95th=[64256],
| 99.99th=[73216]
bw (KB /s): min=133632, max=162816, per=100.00%, avg=146575.70, stdev=7342.30
lat (usec) : 750=4.19%, 1000=1.60%
lat (msec) : 2=7.38%, 4=6.83%, 10=7.10%, 20=12.20%, 50=44.60%
lat (msec) : 100=16.22%
cpu : usr=4.47%, sys=7.35%, ctx=40815, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=137.4%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=14953/w=14819/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=3746.0MB, aggrb=147927KB/s, minb=147927KB/s, maxb=147927KB/s, mint=25931msec, maxt=25931msec
WRITE: io=3704.8MB, aggrb=146298KB/s, minb=146298KB/s, maxb=146298KB/s, mint=25931msec, maxt=25931msec
Disk stats (read/write):
sdk: ios=20432/20243, merge=57/52, ticks=918270/231827, in_queue=1150834, util=99.83%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdk-randrw-32-512K
===
mytest: (g=0): rw=randrw, bs=512K-512K/512K-512K/512K-512K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=28600: Sun Sep 15 15:05:07 2013
read : io=3478.6MB, bw=164268KB/s, iops=320, runt= 21684msec
slat (usec): min=38, max=272, avg=65.97, stdev= 7.20
clat (msec): min=4, max=148, avg=57.45, stdev=23.90
lat (msec): min=4, max=148, avg=57.51, stdev=23.90
clat percentiles (msec):
| 1.00th=[ 11], 5.00th=[ 19], 10.00th=[ 25], 20.00th=[ 37],
| 30.00th=[ 46], 40.00th=[ 52], 50.00th=[ 57], 60.00th=[ 63],
| 70.00th=[ 70], 80.00th=[ 79], 90.00th=[ 89], 95.00th=[ 98],
| 99.00th=[ 115], 99.50th=[ 122], 99.90th=[ 139], 99.95th=[ 141],
| 99.99th=[ 149]
bw (KB /s): min= 51, max=196608, per=97.59%, avg=160306.55, stdev=27144.83
write: io=3523.0MB, bw=166369KB/s, iops=324, runt= 21684msec
slat (usec): min=93, max=51980, avg=151.25, stdev=619.87
clat (msec): min=1, max=129, avg=41.65, stdev=22.72
lat (msec): min=1, max=129, avg=41.80, stdev=22.71
clat percentiles (msec):
| 1.00th=[ 4], 5.00th=[ 9], 10.00th=[ 13], 20.00th=[ 21],
| 30.00th=[ 29], 40.00th=[ 35], 50.00th=[ 40], 60.00th=[ 45],
| 70.00th=[ 51], 80.00th=[ 62], 90.00th=[ 74], 95.00th=[ 84],
| 99.00th=[ 100], 99.50th=[ 108], 99.90th=[ 122], 99.95th=[ 125],
| 99.99th=[ 130]
bw (KB /s): min=135836, max=194560, per=99.92%, avg=166239.84, stdev=11804.03
lat (msec) : 2=0.19%, 4=0.46%, 10=3.24%, 20=8.66%, 50=40.84%
lat (msec) : 100=44.32%, 250=2.51%
cpu : usr=4.50%, sys=7.05%, ctx=20438, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.2%, 32=146.1%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=6941/w=7031/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=3478.6MB, aggrb=164267KB/s, minb=164267KB/s, maxb=164267KB/s, mint=21684msec, maxt=21684msec
WRITE: io=3523.0MB, aggrb=166369KB/s, minb=166369KB/s, maxb=166369KB/s, mint=21684msec, maxt=21684msec
Disk stats (read/write):
sdk: ios=10281/10161, merge=0/0, ticks=623409/392241, in_queue=1016589, util=99.79%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdk-randrw-32-1024K
===
mytest: (g=0): rw=randrw, bs=1M-1M/1M-1M/1M-1M, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=28727: Sun Sep 15 15:06:37 2013
read : io=3409.0MB, bw=181360KB/s, iops=176, runt= 19248msec
slat (usec): min=71, max=338, avg=121.17, stdev= 9.52
clat (msec): min=6, max=468, avg=97.54, stdev=54.82
lat (msec): min=6, max=468, avg=97.66, stdev=54.82
clat percentiles (msec):
| 1.00th=[ 15], 5.00th=[ 25], 10.00th=[ 37], 20.00th=[ 55],
| 30.00th=[ 68], 40.00th=[ 80], 50.00th=[ 91], 60.00th=[ 103],
| 70.00th=[ 116], 80.00th=[ 130], 90.00th=[ 161], 95.00th=[ 200],
| 99.00th=[ 293], 99.50th=[ 326], 99.90th=[ 392], 99.95th=[ 396],
| 99.99th=[ 469]
bw (KB /s): min= 102, max=223232, per=97.61%, avg=177024.08, stdev=33724.55
write: io=3335.0MB, bw=177423KB/s, iops=172, runt= 19248msec
slat (usec): min=186, max=90119, avg=318.05, stdev=2006.48
clat (msec): min=3, max=443, avg=84.95, stdev=55.04
lat (msec): min=3, max=443, avg=85.29, stdev=54.99
clat percentiles (msec):
| 1.00th=[ 8], 5.00th=[ 18], 10.00th=[ 27], 20.00th=[ 42],
| 30.00th=[ 54], 40.00th=[ 65], 50.00th=[ 77], 60.00th=[ 88],
| 70.00th=[ 100], 80.00th=[ 117], 90.00th=[ 151], 95.00th=[ 190],
| 99.00th=[ 285], 99.50th=[ 322], 99.90th=[ 404], 99.95th=[ 412],
| 99.99th=[ 445]
bw (KB /s): min=152988, max=214610, per=100.00%, avg=177656.35, stdev=16363.86
lat (msec) : 4=0.01%, 10=0.88%, 20=3.87%, 50=17.06%, 100=42.08%
lat (msec) : 250=34.60%, 500=1.95%
cpu : usr=4.60%, sys=6.89%, ctx=10171, majf=0, minf=28
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.2%, 16=0.5%, 32=151.6%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=3395/w=3318/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=3409.0MB, aggrb=181359KB/s, minb=181359KB/s, maxb=181359KB/s, mint=19248msec, maxt=19248msec
WRITE: io=3335.0MB, aggrb=177423KB/s, minb=177423KB/s, maxb=177423KB/s, mint=19248msec, maxt=19248msec
Disk stats (read/write):
sdk: ios=10483/10059, merge=0/0, ticks=1020403/808352, in_queue=1834876, util=99.74%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdl-randrw-32-512B
===
mytest: (g=0): rw=randrw, bs=512-512/512-512/512-512, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=28797: Sun Sep 15 15:20:42 2013
read : io=5051.6MB, bw=6676.5KB/s, iops=13352, runt=774772msec
slat (usec): min=3, max=353, avg= 8.38, stdev= 4.56
clat (usec): min=9, max=121273, avg=2284.25, stdev=1918.58
lat (usec): min=136, max=121277, avg=2292.83, stdev=1918.58
clat percentiles (usec):
| 1.00th=[ 161], 5.00th=[ 221], 10.00th=[ 334], 20.00th=[ 692],
| 30.00th=[ 1064], 40.00th=[ 1432], 50.00th=[ 1848], 60.00th=[ 2320],
| 70.00th=[ 2864], 80.00th=[ 3568], 90.00th=[ 4704], 95.00th=[ 5856],
| 99.00th=[ 8640], 99.50th=[ 9920], 99.90th=[12480], 99.95th=[13632],
| 99.99th=[17024]
bw (KB /s): min= 0, max= 7389, per=99.99%, avg=6675.04, stdev=386.45
write: io=5050.6MB, bw=6675.3KB/s, iops=13350, runt=774772msec
slat (usec): min=3, max=5119, avg=10.78, stdev= 5.78
clat (usec): min=0, max=121064, avg=88.89, stdev=298.91
lat (usec): min=64, max=121070, avg=99.86, stdev=298.84
clat percentiles (usec):
| 1.00th=[ 60], 5.00th=[ 64], 10.00th=[ 66], 20.00th=[ 68],
| 30.00th=[ 71], 40.00th=[ 75], 50.00th=[ 80], 60.00th=[ 86],
| 70.00th=[ 93], 80.00th=[ 103], 90.00th=[ 118], 95.00th=[ 133],
| 99.00th=[ 171], 99.50th=[ 191], 99.90th=[ 302], 99.95th=[ 700],
| 99.99th=[ 2448]
bw (KB /s): min= 4032, max= 7465, per=100.00%, avg=6678.15, stdev=347.86
lat (usec) : 2=0.01%, 10=0.01%, 20=0.01%, 50=0.02%, 100=38.56%
lat (usec) : 250=14.66%, 500=4.13%, 750=3.40%, 1000=3.35%
lat (msec) : 2=12.55%, 4=15.58%, 10=7.51%, 20=0.23%, 50=0.01%
lat (msec) : 100=0.01%, 250=0.01%
cpu : usr=11.18%, sys=31.97%, ctx=12101012, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=101.4%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=10345473/w=10343543/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=5051.6MB, aggrb=6676KB/s, minb=6676KB/s, maxb=6676KB/s, mint=774772msec, maxt=774772msec
WRITE: io=5050.6MB, aggrb=6675KB/s, minb=6675KB/s, maxb=6675KB/s, mint=774772msec, maxt=774772msec
Disk stats (read/write):
sdl: ios=10486209/10483896, merge=0/0, ticks=23879628/888442, in_queue=24759092, util=100.00%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdl-randrw-32-1K
===
mytest: (g=0): rw=randrw, bs=1K-1K/1K-1K/1K-1K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=30267: Sun Sep 15 15:28:34 2013
read : io=4997.8MB, bw=12761KB/s, iops=12761, runt=401038msec
slat (usec): min=3, max=360, avg= 8.59, stdev= 4.56
clat (usec): min=39, max=143408, avg=2394.51, stdev=1970.95
lat (usec): min=145, max=143417, avg=2403.29, stdev=1970.95
clat percentiles (usec):
| 1.00th=[ 169], 5.00th=[ 235], 10.00th=[ 378], 20.00th=[ 780],
| 30.00th=[ 1160], 40.00th=[ 1560], 50.00th=[ 1976], 60.00th=[ 2448],
| 70.00th=[ 2992], 80.00th=[ 3696], 90.00th=[ 4832], 95.00th=[ 5984],
| 99.00th=[ 9024], 99.50th=[10176], 99.90th=[12992], 99.95th=[14272],
| 99.99th=[18304]
bw (KB /s): min= 0, max=14056, per=99.93%, avg=12752.37, stdev=742.99
write: io=4996.4MB, bw=12758KB/s, iops=12757, runt=401038msec
slat (usec): min=3, max=5694, avg=11.14, stdev= 6.32
clat (usec): min=0, max=140232, avg=89.21, stdev=376.33
lat (usec): min=65, max=140236, avg=100.53, stdev=376.29
clat percentiles (usec):
| 1.00th=[ 61], 5.00th=[ 65], 10.00th=[ 67], 20.00th=[ 69],
| 30.00th=[ 72], 40.00th=[ 76], 50.00th=[ 80], 60.00th=[ 85],
| 70.00th=[ 92], 80.00th=[ 102], 90.00th=[ 117], 95.00th=[ 131],
| 99.00th=[ 165], 99.50th=[ 185], 99.90th=[ 274], 99.95th=[ 612],
| 99.99th=[ 2704]
bw (KB /s): min= 8224, max=14234, per=100.00%, avg=12764.53, stdev=587.14
lat (usec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.02%
lat (usec) : 100=38.99%, 250=13.77%, 500=3.65%, 750=3.10%, 1000=3.26%
lat (msec) : 2=12.53%, 4=16.35%, 10=8.04%, 20=0.28%, 50=0.01%
lat (msec) : 100=0.01%, 250=0.01%
cpu : usr=10.89%, sys=31.88%, ctx=6127367, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=102.5%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=5117696/w=5116245/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4997.8MB, aggrb=12761KB/s, minb=12761KB/s, maxb=12761KB/s, mint=401038msec, maxt=401038msec
WRITE: io=4996.4MB, aggrb=12757KB/s, minb=12757KB/s, maxb=12757KB/s, mint=401038msec, maxt=401038msec
Disk stats (read/write):
sdl: ios=5242077/5240320, merge=0/0, ticks=12528281/444857, in_queue=12968495, util=100.00%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdl-randrw-32-2K
===
mytest: (g=0): rw=randrw, bs=2K-2K/2K-2K/2K-2K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=30720: Sun Sep 15 15:33:08 2013
read : io=4887.0MB, bw=24599KB/s, iops=12299, runt=203432msec
slat (usec): min=3, max=356, avg= 8.76, stdev= 4.48
clat (usec): min=120, max=142648, avg=2486.21, stdev=1946.93
lat (usec): min=159, max=142653, avg=2495.15, stdev=1946.93
clat percentiles (usec):
| 1.00th=[ 193], 5.00th=[ 282], 10.00th=[ 478], 20.00th=[ 884],
| 30.00th=[ 1272], 40.00th=[ 1656], 50.00th=[ 2064], 60.00th=[ 2512],
| 70.00th=[ 3088], 80.00th=[ 3792], 90.00th=[ 4960], 95.00th=[ 6112],
| 99.00th=[ 9024], 99.50th=[10304], 99.90th=[12864], 99.95th=[14016],
| 99.99th=[17024]
bw (KB /s): min= 0, max=26668, per=99.83%, avg=24557.75, stdev=1655.45
write: io=4884.4MB, bw=24586KB/s, iops=12292, runt=203432msec
slat (usec): min=3, max=6948, avg=11.42, stdev= 7.96
clat (usec): min=0, max=141977, avg=91.12, stdev=363.57
lat (usec): min=68, max=141985, avg=102.71, stdev=363.56
clat percentiles (usec):
| 1.00th=[ 64], 5.00th=[ 68], 10.00th=[ 69], 20.00th=[ 72],
| 30.00th=[ 75], 40.00th=[ 79], 50.00th=[ 83], 60.00th=[ 88],
| 70.00th=[ 95], 80.00th=[ 104], 90.00th=[ 120], 95.00th=[ 135],
| 99.00th=[ 175], 99.50th=[ 195], 99.90th=[ 262], 99.95th=[ 362],
| 99.99th=[ 1368]
bw (KB /s): min=17776, max=26856, per=100.00%, avg=24605.38, stdev=1129.27
lat (usec) : 2=0.01%, 10=0.01%, 20=0.01%, 50=0.02%, 100=37.83%
lat (usec) : 250=13.91%, 500=3.46%, 750=3.05%, 1000=3.24%
lat (msec) : 2=12.69%, 4=16.98%, 10=8.52%, 20=0.29%, 50=0.01%
lat (msec) : 100=0.01%, 250=0.01%
cpu : usr=10.89%, sys=32.26%, ctx=3136090, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=104.8%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=2502114/w=2500785/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4887.0MB, aggrb=24599KB/s, minb=24599KB/s, maxb=24599KB/s, mint=203432msec, maxt=203432msec
WRITE: io=4884.4MB, aggrb=24585KB/s, minb=24585KB/s, maxb=24585KB/s, mint=203432msec, maxt=203432msec
Disk stats (read/write):
sdl: ios=2621704/2620181, merge=0/0, ticks=6514856/226973, in_queue=6739636, util=100.00%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdl-randrw-32-4K
===
mytest: (g=0): rw=randrw, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=31353: Sun Sep 15 15:36:08 2013
read : io=4693.5MB, bw=43832KB/s, iops=10957, runt=109648msec
slat (usec): min=3, max=220, avg= 8.85, stdev= 3.33
clat (usec): min=186, max=140256, avg=2790.33, stdev=2316.65
lat (usec): min=195, max=140265, avg=2799.35, stdev=2316.65
clat percentiles (usec):
| 1.00th=[ 231], 5.00th=[ 314], 10.00th=[ 502], 20.00th=[ 956],
| 30.00th=[ 1384], 40.00th=[ 1816], 50.00th=[ 2288], 60.00th=[ 2832],
| 70.00th=[ 3472], 80.00th=[ 4320], 90.00th=[ 5600], 95.00th=[ 6944],
| 99.00th=[ 9920], 99.50th=[11200], 99.90th=[14016], 99.95th=[15424],
| 99.99th=[20608]
bw (KB /s): min=34872, max=47880, per=100.00%, avg=43904.56, stdev=1913.39
write: io=4689.7MB, bw=43797KB/s, iops=10949, runt=109648msec
slat (usec): min=4, max=6029, avg=11.92, stdev= 7.21
clat (usec): min=1, max=123244, avg=105.14, stdev=630.27
lat (usec): min=75, max=123253, avg=117.18, stdev=630.25
clat percentiles (usec):
| 1.00th=[ 71], 5.00th=[ 74], 10.00th=[ 75], 20.00th=[ 78],
| 30.00th=[ 82], 40.00th=[ 87], 50.00th=[ 92], 60.00th=[ 98],
| 70.00th=[ 107], 80.00th=[ 118], 90.00th=[ 135], 95.00th=[ 155],
| 99.00th=[ 203], 99.50th=[ 227], 99.90th=[ 326], 99.95th=[ 596],
| 99.99th=[ 2192]
bw (KB /s): min= 0, max=47632, per=99.71%, avg=43668.71, stdev=3509.93
lat (usec) : 2=0.01%, 20=0.01%, 50=0.01%, 100=30.73%, 250=20.08%
lat (usec) : 500=4.12%, 750=2.68%, 1000=2.85%
lat (msec) : 2=11.47%, 4=16.43%, 10=11.15%, 20=0.48%, 50=0.01%
lat (msec) : 100=0.01%, 250=0.01%
cpu : usr=11.27%, sys=30.33%, ctx=1704971, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=109.1%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=1201496/w=1200553/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4693.5MB, aggrb=43832KB/s, minb=43832KB/s, maxb=43832KB/s, mint=109648msec, maxt=109648msec
WRITE: io=4689.7MB, aggrb=43796KB/s, minb=43796KB/s, maxb=43796KB/s, mint=109648msec, maxt=109648msec
Disk stats (read/write):
sdl: ios=1310320/1309262, merge=0/0, ticks=3659796/142972, in_queue=3802191, util=100.00%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdl-randrw-32-8K
===
mytest: (g=0): rw=randrw, bs=8K-8K/8K-8K/8K-8K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=31534: Sun Sep 15 15:38:24 2013
read : io=4448.9MB, bw=69120KB/s, iops=8639, runt= 65908msec
slat (usec): min=3, max=228, avg= 9.53, stdev= 2.90
clat (usec): min=228, max=147907, avg=3550.84, stdev=2762.85
lat (usec): min=238, max=147916, avg=3560.53, stdev=2762.85
clat percentiles (usec):
| 1.00th=[ 322], 5.00th=[ 434], 10.00th=[ 628], 20.00th=[ 1128],
| 30.00th=[ 1752], 40.00th=[ 2384], 50.00th=[ 3024], 60.00th=[ 3760],
| 70.00th=[ 4576], 80.00th=[ 5536], 90.00th=[ 7072], 95.00th=[ 8512],
| 99.00th=[11840], 99.50th=[13248], 99.90th=[16512], 99.95th=[18048],
| 99.99th=[24960]
bw (KB /s): min=50432, max=73680, per=100.00%, avg=69183.11, stdev=2925.55
write: io=4444.1MB, bw=69060KB/s, iops=8632, runt= 65908msec
slat (usec): min=4, max=5867, avg=13.24, stdev= 8.45
clat (usec): min=18, max=141192, avg=125.97, stdev=742.99
lat (usec): min=84, max=141210, avg=139.31, stdev=742.99
clat percentiles (usec):
| 1.00th=[ 80], 5.00th=[ 82], 10.00th=[ 83], 20.00th=[ 87],
| 30.00th=[ 94], 40.00th=[ 102], 50.00th=[ 109], 60.00th=[ 119],
| 70.00th=[ 131], 80.00th=[ 143], 90.00th=[ 169], 95.00th=[ 193],
| 99.00th=[ 258], 99.50th=[ 290], 99.90th=[ 466], 99.95th=[ 1288],
| 99.99th=[10560]
bw (KB /s): min= 0, max=73776, per=99.35%, avg=68613.53, stdev=6741.19
lat (usec) : 20=0.01%, 50=0.01%, 100=18.17%, 250=31.23%, 500=3.93%
lat (usec) : 750=3.04%, 1000=2.44%
lat (msec) : 2=8.20%, 4=14.66%, 10=17.12%, 20=1.21%, 50=0.01%
lat (msec) : 100=0.01%, 250=0.01%
cpu : usr=10.94%, sys=27.17%, ctx=964258, majf=0, minf=28
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=115.1%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=569416/w=568947/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4448.9MB, aggrb=69120KB/s, minb=69120KB/s, maxb=69120KB/s, mint=65908msec, maxt=65908msec
WRITE: io=4444.1MB, aggrb=69059KB/s, minb=69059KB/s, maxb=69059KB/s, mint=65908msec, maxt=65908msec
Disk stats (read/write):
sdl: ios=654208/653671, merge=0/0, ticks=2326960/84042, in_queue=2410855, util=99.95%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdl-randrw-32-16K
===
mytest: (g=0): rw=randrw, bs=16K-16K/16K-16K/16K-16K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=31660: Sun Sep 15 15:40:17 2013
read : io=4137.1MB, bw=100775KB/s, iops=6297, runt= 42046msec
slat (usec): min=4, max=212, avg=10.29, stdev= 2.37
clat (usec): min=327, max=146023, avg=4875.71, stdev=3658.56
lat (usec): min=338, max=146027, avg=4886.19, stdev=3658.55
clat percentiles (usec):
| 1.00th=[ 494], 5.00th=[ 700], 10.00th=[ 972], 20.00th=[ 1656],
| 30.00th=[ 2480], 40.00th=[ 3312], 50.00th=[ 4192], 60.00th=[ 5152],
| 70.00th=[ 6176], 80.00th=[ 7520], 90.00th=[ 9664], 95.00th=[11584],
| 99.00th=[15808], 99.50th=[17536], 99.90th=[21632], 99.95th=[23424],
| 99.99th=[28800]
bw (KB /s): min=74240, max=106592, per=100.00%, avg=100846.14, stdev=3918.19
write: io=4126.7MB, bw=100501KB/s, iops=6281, runt= 42046msec
slat (usec): min=6, max=5434, avg=15.33, stdev=13.70
clat (usec): min=71, max=145775, avg=175.45, stdev=1142.49
lat (usec): min=100, max=145782, avg=190.91, stdev=1142.53
clat percentiles (usec):
| 1.00th=[ 95], 5.00th=[ 97], 10.00th=[ 99], 20.00th=[ 115],
| 30.00th=[ 126], 40.00th=[ 139], 50.00th=[ 149], 60.00th=[ 165],
| 70.00th=[ 183], 80.00th=[ 207], 90.00th=[ 247], 95.00th=[ 290],
| 99.00th=[ 402], 99.50th=[ 466], 99.90th=[ 660], 99.95th=[ 852],
| 99.99th=[ 2040]
bw (KB /s): min= 1, max=107104, per=98.87%, avg=99366.65, stdev=11820.21
lat (usec) : 100=5.50%, 250=39.69%, 500=5.12%, 750=2.54%, 1000=2.33%
lat (msec) : 2=6.85%, 4=11.80%, 10=21.76%, 20=4.31%, 50=0.09%
lat (msec) : 100=0.01%, 250=0.01%
cpu : usr=10.40%, sys=22.78%, ctx=507159, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=123.9%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=264794/w=264104/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4137.1MB, aggrb=100775KB/s, minb=100775KB/s, maxb=100775KB/s, mint=42046msec, maxt=42046msec
WRITE: io=4126.7MB, aggrb=100500KB/s, minb=100500KB/s, maxb=100500KB/s, mint=42046msec, maxt=42046msec
Disk stats (read/write):
sdl: ios=327692/326652, merge=0/0, ticks=1597482/57701, in_queue=1655167, util=99.89%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdl-randrw-32-32K
===
mytest: (g=0): rw=randrw, bs=32K-32K/32K-32K/32K-32K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=31788: Sun Sep 15 15:41:58 2013
read : io=3860.5MB, bw=127549KB/s, iops=3984, runt= 30993msec
slat (usec): min=7, max=290, avg=12.31, stdev= 3.18
clat (usec): min=562, max=148968, avg=7697.93, stdev=5442.26
lat (usec): min=574, max=148985, avg=7710.56, stdev=5442.25
clat percentiles (usec):
| 1.00th=[ 748], 5.00th=[ 1192], 10.00th=[ 1752], 20.00th=[ 2992],
| 30.00th=[ 4256], 40.00th=[ 5536], 50.00th=[ 6752], 60.00th=[ 8096],
| 70.00th=[ 9664], 80.00th=[11584], 90.00th=[14784], 95.00th=[17536],
| 99.00th=[23936], 99.50th=[26496], 99.90th=[33024], 99.95th=[37632],
| 99.99th=[134144]
bw (KB /s): min=97152, max=135040, per=100.00%, avg=127735.52, stdev=5026.09
write: io=3859.1MB, bw=127532KB/s, iops=3985, runt= 30993msec
slat (usec): min=13, max=5227, avg=23.20, stdev=17.68
clat (usec): min=27, max=144536, avg=289.67, stdev=1621.28
lat (usec): min=140, max=144550, avg=313.77, stdev=1621.27
clat percentiles (usec):
| 1.00th=[ 123], 5.00th=[ 125], 10.00th=[ 129], 20.00th=[ 153],
| 30.00th=[ 177], 40.00th=[ 199], 50.00th=[ 225], 60.00th=[ 253],
| 70.00th=[ 298], 80.00th=[ 346], 90.00th=[ 442], 95.00th=[ 548],
| 99.00th=[ 924], 99.50th=[ 1304], 99.90th=[ 2448], 99.95th=[ 2832],
| 99.99th=[136192]
bw (KB /s): min= 3, max=136000, per=98.51%, avg=125626.89, stdev=17115.62
lat (usec) : 50=0.01%, 100=0.01%, 250=29.58%, 500=17.10%, 750=2.89%
lat (usec) : 1000=1.58%
lat (msec) : 2=4.81%, 4=7.97%, 10=22.03%, 20=12.68%, 50=1.34%
lat (msec) : 100=0.01%, 250=0.01%
cpu : usr=9.93%, sys=18.61%, ctx=261784, majf=0, minf=28
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=132.6%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=123505/w=123518/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=3860.5MB, aggrb=127548KB/s, minb=127548KB/s, maxb=127548KB/s, mint=30993msec, maxt=30993msec
WRITE: io=3859.1MB, aggrb=127532KB/s, minb=127532KB/s, maxb=127532KB/s, mint=30993msec, maxt=30993msec
Disk stats (read/write):
sdl: ios=163765/163667, merge=0/0, ticks=1256894/45902, in_queue=1302761, util=99.85%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdl-randrw-32-64K
===
mytest: (g=0): rw=randrw, bs=64K-64K/64K-64K/64K-64K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=31910: Sun Sep 15 15:43:35 2013
read : io=3656.4MB, bw=143881KB/s, iops=2246, runt= 26022msec
slat (usec): min=11, max=70, avg=14.68, stdev= 1.53
clat (usec): min=871, max=160815, avg=13319.38, stdev=9509.92
lat (usec): min=886, max=160835, avg=13334.36, stdev=9509.93
clat percentiles (usec):
| 1.00th=[ 1320], 5.00th=[ 2352], 10.00th=[ 3376], 20.00th=[ 5280],
| 30.00th=[ 7136], 40.00th=[ 9024], 50.00th=[11200], 60.00th=[13760],
| 70.00th=[16512], 80.00th=[20352], 90.00th=[25728], 95.00th=[31360],
| 99.00th=[42240], 99.50th=[46848], 99.90th=[58624], 99.95th=[65280],
| 99.99th=[150528]
bw (KB /s): min= 6, max=153728, per=98.18%, avg=141265.35, stdev=21194.26
write: io=3657.6MB, bw=143927KB/s, iops=2248, runt= 26022msec
slat (usec): min=18, max=4165, avg=27.98, stdev=21.55
clat (usec): min=177, max=142909, avg=869.35, stdev=2371.82
lat (usec): min=207, max=142933, avg=897.63, stdev=2371.83
clat percentiles (usec):
| 1.00th=[ 187], 5.00th=[ 205], 10.00th=[ 211], 20.00th=[ 286],
| 30.00th=[ 338], 40.00th=[ 410], 50.00th=[ 516], 60.00th=[ 628],
| 70.00th=[ 828], 80.00th=[ 1176], 90.00th=[ 1960], 95.00th=[ 2704],
| 99.00th=[ 4256], 99.50th=[ 4960], 99.90th=[ 7008], 99.95th=[ 8384],
| 99.99th=[142336]
bw (KB /s): min=106496, max=157312, per=100.00%, avg=144381.49, stdev=7186.70
lat (usec) : 250=7.01%, 500=17.60%, 750=9.00%, 1000=4.50%
lat (msec) : 2=8.84%, 4=8.98%, 10=16.34%, 20=17.44%, 50=10.15%
lat (msec) : 100=0.14%, 250=0.03%
cpu : usr=7.13%, sys=12.17%, ctx=130434, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=140.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=58470/w=58520/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=3656.4MB, aggrb=143880KB/s, minb=143880KB/s, maxb=143880KB/s, mint=26022msec, maxt=26022msec
WRITE: io=3657.6MB, aggrb=143927KB/s, minb=143927KB/s, maxb=143927KB/s, mint=26022msec, maxt=26022msec
Disk stats (read/write):
sdl: ios=81867/81772, merge=0/0, ticks=1079828/66756, in_queue=1146719, util=99.82%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdl-randrw-32-128K
===
mytest: (g=0): rw=randrw, bs=128K-128K/128K-128K/128K-128K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=31986: Sun Sep 15 15:45:10 2013
read : io=3623.4MB, bw=149888KB/s, iops=1169, runt= 24754msec
slat (usec): min=16, max=80, avg=22.76, stdev= 3.13
clat (msec): min=1, max=165, avg=21.46, stdev=11.82
lat (msec): min=1, max=165, avg=21.48, stdev=11.82
clat percentiles (msec):
| 1.00th=[ 6], 5.00th=[ 8], 10.00th=[ 10], 20.00th=[ 13],
| 30.00th=[ 15], 40.00th=[ 17], 50.00th=[ 20], 60.00th=[ 22],
| 70.00th=[ 25], 80.00th=[ 29], 90.00th=[ 37], 95.00th=[ 44],
| 99.00th=[ 62], 99.50th=[ 69], 99.90th=[ 88], 99.95th=[ 139],
| 99.99th=[ 159]
bw (KB /s): min= 12, max=167680, per=98.00%, avg=146892.04, stdev=22535.95
write: io=3627.3MB, bw=150049KB/s, iops=1172, runt= 24754msec
slat (usec): min=26, max=7185, avg=46.73, stdev=55.70
clat (usec): min=303, max=143601, avg=5798.34, stdev=4175.78
lat (usec): min=345, max=143642, avg=5845.27, stdev=4175.50
clat percentiles (usec):
| 1.00th=[ 338], 5.00th=[ 1032], 10.00th=[ 1896], 20.00th=[ 3152],
| 30.00th=[ 4080], 40.00th=[ 4896], 50.00th=[ 5664], 60.00th=[ 6368],
| 70.00th=[ 7200], 80.00th=[ 8160], 90.00th=[ 9536], 95.00th=[10560],
| 99.00th=[13120], 99.50th=[13888], 99.90th=[19840], 99.95th=[100864],
| 99.99th=[142336]
bw (KB /s): min=115200, max=159488, per=100.00%, avg=150307.65, stdev=6347.58
lat (usec) : 500=0.81%, 750=0.97%, 1000=0.66%
lat (msec) : 2=2.96%, 4=9.36%, 10=37.38%, 20=24.86%, 50=21.64%
lat (msec) : 100=1.37%, 250=0.05%
cpu : usr=5.24%, sys=9.60%, ctx=65302, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=141.2%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=28961/w=29013/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=3623.4MB, aggrb=149888KB/s, minb=149888KB/s, maxb=149888KB/s, mint=24754msec, maxt=24754msec
WRITE: io=3627.3MB, aggrb=150048KB/s, minb=150048KB/s, maxb=150048KB/s, mint=24754msec, maxt=24754msec
Disk stats (read/write):
sdl: ios=41044/40915, merge=0/0, ticks=873322/233587, in_queue=1107232, util=99.85%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdl-randrw-32-256K
===
mytest: (g=0): rw=randrw, bs=256K-256K/256K-256K/256K-256K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=32114: Sun Sep 15 15:46:48 2013
read : io=3749.3MB, bw=142479KB/s, iops=556, runt= 26946msec
slat (usec): min=26, max=237, avg=38.91, stdev= 4.91
clat (msec): min=8, max=159, avg=28.10, stdev= 9.45
lat (msec): min=8, max=159, avg=28.14, stdev= 9.45
clat percentiles (msec):
| 1.00th=[ 15], 5.00th=[ 18], 10.00th=[ 19], 20.00th=[ 22],
| 30.00th=[ 23], 40.00th=[ 25], 50.00th=[ 27], 60.00th=[ 29],
| 70.00th=[ 31], 80.00th=[ 35], 90.00th=[ 40], 95.00th=[ 44],
| 99.00th=[ 56], 99.50th=[ 63], 99.90th=[ 149], 99.95th=[ 159],
| 99.99th=[ 159]
bw (KB /s): min= 25, max=160958, per=98.16%, avg=139854.02, stdev=21473.18
write: io=3714.3MB, bw=141149KB/s, iops=550, runt= 26946msec
slat (usec): min=55, max=12334, avg=84.19, stdev=163.82
clat (msec): min=9, max=161, avg=29.56, stdev= 8.85
lat (msec): min=9, max=161, avg=29.64, stdev= 8.85
clat percentiles (msec):
| 1.00th=[ 18], 5.00th=[ 20], 10.00th=[ 21], 20.00th=[ 23],
| 30.00th=[ 25], 40.00th=[ 27], 50.00th=[ 29], 60.00th=[ 30],
| 70.00th=[ 33], 80.00th=[ 36], 90.00th=[ 41], 95.00th=[ 45],
| 99.00th=[ 56], 99.50th=[ 61], 99.90th=[ 88], 99.95th=[ 151],
| 99.99th=[ 159]
bw (KB /s): min=102400, max=158208, per=100.00%, avg=141176.33, stdev=8094.41
lat (msec) : 10=0.01%, 20=10.85%, 50=86.90%, 100=2.24%, 250=0.10%
cpu : usr=3.97%, sys=7.08%, ctx=36298, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=137.1%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=14983/w=14840/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=3749.3MB, aggrb=142478KB/s, minb=142478KB/s, maxb=142478KB/s, mint=26946msec, maxt=26946msec
WRITE: io=3714.3MB, aggrb=141148KB/s, minb=141148KB/s, maxb=141148KB/s, mint=26946msec, maxt=26946msec
Disk stats (read/write):
sdl: ios=20534/20283, merge=0/0, ticks=573530/598577, in_queue=1172518, util=99.85%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdl-randrw-32-512K
===
mytest: (g=0): rw=randrw, bs=512K-512K/512K-512K/512K-512K, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=32186: Sun Sep 15 15:48:24 2013
read : io=3667.6MB, bw=144116KB/s, iops=280, runt= 26059msec
slat (usec): min=47, max=126, avg=69.66, stdev= 5.77
clat (msec): min=11, max=208, avg=55.29, stdev=14.35
lat (msec): min=11, max=208, avg=55.36, stdev=14.35
clat percentiles (msec):
| 1.00th=[ 32], 5.00th=[ 37], 10.00th=[ 40], 20.00th=[ 44],
| 30.00th=[ 48], 40.00th=[ 51], 50.00th=[ 53], 60.00th=[ 57],
| 70.00th=[ 61], 80.00th=[ 66], 90.00th=[ 74], 95.00th=[ 81],
| 99.00th=[ 100], 99.50th=[ 112], 99.90th=[ 135], 99.95th=[ 155],
| 99.99th=[ 208]
bw (KB /s): min=112986, max=171688, per=100.00%, avg=144286.96, stdev=13006.69
write: io=3717.6MB, bw=146081KB/s, iops=284, runt= 26059msec
slat (usec): min=98, max=32753, avg=153.90, stdev=421.98
clat (msec): min=8, max=182, avg=57.46, stdev=14.22
lat (msec): min=8, max=182, avg=57.62, stdev=14.20
clat percentiles (msec):
| 1.00th=[ 35], 5.00th=[ 40], 10.00th=[ 43], 20.00th=[ 47],
| 30.00th=[ 50], 40.00th=[ 53], 50.00th=[ 56], 60.00th=[ 59],
| 70.00th=[ 63], 80.00th=[ 68], 90.00th=[ 76], 95.00th=[ 82],
| 99.00th=[ 103], 99.50th=[ 121], 99.90th=[ 149], 99.95th=[ 165],
| 99.99th=[ 184]
bw (KB /s): min= 51, max=164864, per=97.93%, avg=143054.48, stdev=22364.34
lat (msec) : 10=0.01%, 20=0.04%, 50=35.59%, 100=63.53%, 250=1.04%
cpu : usr=3.61%, sys=6.11%, ctx=18749, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.2%, 32=138.5%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=7321/w=7418/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=3667.6MB, aggrb=144116KB/s, minb=144116KB/s, maxb=144116KB/s, mint=26059msec, maxt=26059msec
WRITE: io=3717.6MB, aggrb=146080KB/s, minb=146080KB/s, maxb=146080KB/s, mint=26059msec, maxt=26059msec
Disk stats (read/write):
sdl: ios=10367/10207, merge=0/0, ticks=565192/583466, in_queue=1149632, util=99.84%
===
{mydisk}- {myrw}- {myiodepth}- {myblocksize}
/dev/sdl-randrw-32-1024K
===
mytest: (g=0): rw=randrw, bs=1M-1M/1M-1M/1M-1M, ioengine=libaio, iodepth=32
fio-2.1
Starting 1 process
mytest: (groupid=0, jobs=1): err= 0: pid=32312: Sun Sep 15 15:50:00 2013
read : io=3695.0MB, bw=147939KB/s, iops=143, runt= 25576msec
slat (usec): min=102, max=328, avg=133.21, stdev=10.88
clat (msec): min=15, max=252, avg=109.31, stdev=17.97
lat (msec): min=15, max=252, avg=109.44, stdev=17.97
clat percentiles (msec):
| 1.00th=[ 77], 5.00th=[ 86], 10.00th=[ 90], 20.00th=[ 96],
| 30.00th=[ 101], 40.00th=[ 104], 50.00th=[ 109], 60.00th=[ 112],
| 70.00th=[ 116], 80.00th=[ 121], 90.00th=[ 129], 95.00th=[ 137],
| 99.00th=[ 161], 99.50th=[ 188], 99.90th=[ 239], 99.95th=[ 243],
| 99.99th=[ 253]
bw (KB /s): min= 102, max=194560, per=97.86%, avg=144775.53, stdev=27656.51
write: io=3639.0MB, bw=145697KB/s, iops=141, runt= 25576msec
slat (usec): min=183, max=71969, avg=317.06, stdev=1474.46
clat (msec): min=18, max=257, avg=113.76, stdev=17.87
lat (msec): min=18, max=257, avg=114.09, stdev=17.79
clat percentiles (msec):
| 1.00th=[ 84], 5.00th=[ 92], 10.00th=[ 96], 20.00th=[ 101],
| 30.00th=[ 105], 40.00th=[ 109], 50.00th=[ 113], 60.00th=[ 117],
| 70.00th=[ 120], 80.00th=[ 126], 90.00th=[ 135], 95.00th=[ 143],
| 99.00th=[ 169], 99.50th=[ 200], 99.90th=[ 241], 99.95th=[ 243],
| 99.99th=[ 258]
bw (KB /s): min=112415, max=167968, per=100.00%, avg=145884.10, stdev=10619.77
lat (msec) : 20=0.03%, 50=0.15%, 100=23.63%, 250=76.58%, 500=0.03%
cpu : usr=3.24%, sys=5.68%, ctx=9615, majf=0, minf=27
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.2%, 16=0.4%, 32=139.4%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=3677/w=3626/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=3695.0MB, aggrb=147938KB/s, minb=147938KB/s, maxb=147938KB/s, mint=25576msec, maxt=25576msec
WRITE: io=3639.0MB, aggrb=145696KB/s, minb=145696KB/s, maxb=145696KB/s, mint=25576msec, maxt=25576msec
Disk stats (read/write):
sdl: ios=10513/10036, merge=0/0, ticks=1062339/1068900, in_queue=2134678, util=99.83%
end time:
Sun Sep 15 15:50:01 PDT 2013
^ permalink raw reply [flat|nested] 6+ messages in thread