verify with norandommap

verify with norandommap

[Gary Foster]

Hi all.

I've been running some random IO tests where I want to verify the data
was written correctly.

I have set norandommap=1 as I was running out of memory, but the man
page indicates
the data will not be verified

--- cut ---
       norandommap
              Normally  fio  will  cover every block of the file when
doing random I/O. If this parameter is given, a new offset will be
              chosen without looking at past I/O history.  This
parameter is mutually exclusive with verify.
-- cut ----

However, the output to stdio seems to indicate it is verifying (see
the 'V' below) - but is it really verifying
anything ? and if so, whats the difference between the verify with
norandommap being set to 0 ?

-- cut ---
fio 2.0.7
Starting 13 processes
bs: 13 (f=13): [wVwwwwwwwwwww] [0.0% done] [3695K/3679K /s] [902 /898
iops] [eta 84d:23h:10m:19s]
-- cut --

Here's my job file

[global]
blocksize=4K
readwrite=randwr
ioengine=libaio
verify=crc32c
verify_backlog=10
iodepth=1
direct=1
norandommap=1

Re: verify with norandommap

[Gary Foster]

I realise most people are using Fio to do performance testing, ie
replicating Windows IOMeter.

But surely there are some guys using Fio for data integrity testing,
could you guys give me some help with this ?

On 10 August 2012 15:16, Gary Foster <gpfwestie@gmail.com> wrote:
>
> Hi all.
>
> I've been running some random IO tests where I want to verify the data
> was written correctly.
>
> I have set norandommap=1 as I was running out of memory, but the man
> page indicates
> the data will not be verified
>
> --- cut ---
>        norandommap
>               Normally  fio  will  cover every block of the file when
> doing random I/O. If this parameter is given, a new offset will be
>               chosen without looking at past I/O history.  This
> parameter is mutually exclusive with verify.
> -- cut ----
>
> However, the output to stdio seems to indicate it is verifying (see
> the 'V' below) - but is it really verifying
> anything ? and if so, whats the difference between the verify with
> norandommap being set to 0 ?
>
> -- cut ---
> fio 2.0.7
> Starting 13 processes
> bs: 13 (f=13): [wVwwwwwwwwwww] [0.0% done] [3695K/3679K /s] [902 /898
> iops] [eta 84d:23h:10m:19s]
> -- cut --
>
> Here's my job file
>
> [global]
> blocksize=4K
> readwrite=randwr
> ioengine=libaio
> verify=crc32c
> verify_backlog=10
> iodepth=1
> direct=1
> norandommap=1

Re: verify with norandommap

[Jens Axboe]

On 2012-09-02 18:14, Gary Foster wrote:
> I realise most people are using Fio to do performance testing, ie
> replicating Windows IOMeter.
> 
> But surely there are some guys using Fio for data integrity testing,
> could you guys give me some help with this ?

It will work OK with norandommap for your workload. If you have mixed
block sizes, then you could run into verification failures if smaller
blocks overwrite parts of a bigger verification piece. With pattern
verification and a pattern size set to the smallest blocksize that
should work too, though.

If you see the 'V', it's definitely verifying. The comment on
norandommap refers to seeing spurious verification failures, not that it
wont verify at all. Your job file should be fine.

-- 
Jens Axboe