Does fio write only 0x00s?

Does fio write only 0x00s?

[Troels Arvin]

Hello,

I'm benchmarking a new FC-attached storage system which we are bringing 
into production, in order to see if it yields acceptable performance - 
and to have a baseline for potential performance trouble in the future.

One of the tools I'm using is fio, which is certainly great. Especially 
the fact that it includes data regarding latencies, and that it makes it 
easy to have a mix of reads and writes.

Now, compared to an another FC-attached storage system, and compared to 
a local RAID10, the new system yields fio numbers which are extremely 
much better.
This made me have a quick and superficial look into the files which fio 
works with. I seems that the files contain only 0-bytes.

Does fio only write bytes with a single value?
If so: I would be afraid that some kind of 
compression/de-duplication/thin provisioning feature in the storage 
system invalidates the fio results I'm seeing. Is this a valid fear?

-- 
Regards,
Troels Arvin <troels@arvin.dk>
http://troels.arvin.dk/

Re: Does fio write only 0x00s?

[Jens Axboe]

On Mon, Mar 15 2010, Troels Arvin wrote:
> Hello,
>
> I'm benchmarking a new FC-attached storage system which we are bringing  
> into production, in order to see if it yields acceptable performance -  
> and to have a baseline for potential performance trouble in the future.
>
> One of the tools I'm using is fio, which is certainly great. Especially  
> the fact that it includes data regarding latencies, and that it makes it  
> easy to have a mix of reads and writes.
>
> Now, compared to an another FC-attached storage system, and compared to  
> a local RAID10, the new system yields fio numbers which are extremely  
> much better.
> This made me have a quick and superficial look into the files which fio  
> works with. I seems that the files contain only 0-bytes.
>
> Does fio only write bytes with a single value?
> If so: I would be afraid that some kind of  
> compression/de-duplication/thin provisioning feature in the storage  
> system invalidates the fio results I'm seeing. Is this a valid fear?

By default, fio will at init time randomly fill the buffer of the
allocated IO units. If you are using the sync io engine, then only one
buffer will be allocated and that will be repeatedly written. So yes,
that'll compress very nicely. You can enable refill_buffers=1 and
that'll cause fio to randomly fill it everytime it's submitted instead.
That should effectively disable compression at the storage end.

But you should not be seeing zeroes only. Are you perhaps sparsely
filling a file with random writes, and you are observing the holes?

-- 
Jens Axboe

Re: Does fio write only 0x00s?

[Troels Arvin]

Hello,

On Mar 15, Jens Axboe wrote:
> By default, fio will at init time randomly fill the buffer of the
> allocated IO units. If you are using the sync io engine, then only one
> buffer will be allocated and that will be repeatedly written. So yes,
> that'll compress very nicely.

I'm using libaio. So the data from that shouldn't compress all that much?


> You can enable refill_buffers=1 and
> that'll cause fio to randomly fill it everytime it's submitted instead.
> That should effectively disable compression at the storage end.

Turning refill_buffers on or off doesn't seem to make much of a 
difference when compressing fil's work-file with "gzip -2" (201MB vs 
198MB). But perhaps, "gzip -2" still compreses more than one can expect 
from a storage system?

I'm probably missing something. Here's the job description file I'm 
using (in this case only testing a small amount of I/O; when I was 
testing the storage systems, I used size=10g and six numjobs=6):
===========================================================
[global]
description=Emulation of Intel IOmeter File Server Access Pattern

[iometer]
bssplit=512/10:1k/5:2k/5:4k/60:8k/2:16k/4:32k/4:64k/10
rw=randrw
rwmixread=70
direct=1
size=1g
ioengine=libaio
iodepth=256
refill_buffers=0
===========================================================

-- 
Regards,
Troels Arvin <troels@arvin.dk>
http://troels.arvin.dk/

Re: Does fio write only 0x00s?

[Jens Axboe]

On Tue, Mar 16 2010, Troels Arvin wrote:
> Hello,
>
> On Mar 15, Jens Axboe wrote:
>> By default, fio will at init time randomly fill the buffer of the
>> allocated IO units. If you are using the sync io engine, then only one
>> buffer will be allocated and that will be repeatedly written. So yes,
>> that'll compress very nicely.
>
> I'm using libaio. So the data from that shouldn't compress all that much?

Each buffer used will have different contents, so it'll depend on how
large the file is. If you end up writing the same 256 buffers (in your
case) thousands of time, the output will still be compressible.

>> You can enable refill_buffers=1 and
>> that'll cause fio to randomly fill it everytime it's submitted instead.
>> That should effectively disable compression at the storage end.
>
> Turning refill_buffers on or off doesn't seem to make much of a  
> difference when compressing fil's work-file with "gzip -2" (201MB vs  
> 198MB). But perhaps, "gzip -2" still compreses more than one can expect  
> from a storage system?

Hard to say really, if it's inline compression then perhaps not as
strong, background data-dedup could be a lot stronger.

> I'm probably missing something. Here's the job description file I'm  
> using (in this case only testing a small amount of I/O; when I was  
> testing the storage systems, I used size=10g and six numjobs=6):
> ===========================================================
> [global]
> description=Emulation of Intel IOmeter File Server Access Pattern
>
> [iometer]
> bssplit=512/10:1k/5:2k/5:4k/60:8k/2:16k/4:32k/4:64k/10
> rw=randrw
> rwmixread=70
> direct=1
> size=1g
> ioengine=libaio
> iodepth=256
> refill_buffers=0
> ===========================================================

I think what you are missing is that the random writes will create a
large sparse file. The Output is 10G, and you are doing a lot of reads.
So you could end up writing only 30% of the 10G, the rest would be
sparse holes in the file.

-- 
Jens Axboe

Re: Does fio write only 0x00s?

[Troels Arvin]

Hello,

Den 16-03-2010 09:05, skrev Jens Axboe:
> I think what you are missing is that the random writes will create a
> large sparse file. The Output is 10G, and you are doing a lot of reads.
> So you could end up writing only 30% of the 10G, the rest would be
> sparse holes in the file.

Yes, that part about the file sparseness was I had to grasp.

Expressed in another way: If I filter out the null-bytes from fio's 
work-file (cat iometer.1.0 | tr -d '\0'), then the remainder doesn't 
compress at all using gzip with default options.

So I'll stop worrying about storage-side compression/de-duplication when 
interpreting fio's results.

Thanks.

-- 
Regards,
Troels Arvin <troels@arvin.dk>
http://troels.arvin.dk/