Using offsets in fio

Using offsets in fio

[Michael O'Sullivan]

Hi everyone,

We are using fio to emulate the SPC-1 benchmark (we hope to get this 
included as a --profile in fio in the not too distant future). However, 
we have noticed that when running our benchmark the disk we are testing 
does not seem to be overly busy seeking to different locations on the 
disk. When we run tests using Iometer we can hear the disk seeking to 
different sections of the disk pretty much the whole time. We are using 
io_u->offset to define the position in the file we wish to read/write 
at, but it is not clear to us whether this is having the desired effect. 
It doesn't seem so at this point... We have been using the sync and 
libaio engines.

Can someone please give us some advice to help us troubleshoot this 
problem? Or point us to a tool that will let us see where fio is 
reading/writing on the disk? Any help is greatly appreciated.

Kind regards, Mike

Re: Using offsets in fio

[Jens Axboe]

On Tue, Apr 27 2010, Michael O'Sullivan wrote:
> Hi everyone,
>
> We are using fio to emulate the SPC-1 benchmark (we hope to get this  
> included as a --profile in fio in the not too distant future). However,  
> we have noticed that when running our benchmark the disk we are testing  
> does not seem to be overly busy seeking to different locations on the  
> disk. When we run tests using Iometer we can hear the disk seeking to  
> different sections of the disk pretty much the whole time. We are using  
> io_u->offset to define the position in the file we wish to read/write  
> at, but it is not clear to us whether this is having the desired effect.  
> It doesn't seem so at this point... We have been using the sync and  
> libaio engines.

io_u->offset is the byte offset that the engine will seek to before
performing IO, so whatever you put in there will be the real offset. If
the disk doesn't sound as busy as it should, perhaps it's because
iometer drives a higher queue depth as well?

> Can someone please give us some advice to help us troubleshoot this  
> problem? Or point us to a tool that will let us see where fio is  
> reading/writing on the disk? Any help is greatly appreciated.

For shorter samples, you could always just strace fio and see what it's
doing. For a better view, you could use blktrace.

-- 
Jens Axboe

Re: Using offsets in fio

[DongJin Lee]

[-- Attachment #1: Type: text/plain, Size: 1663 bytes --]

So for example, you'd expect, for example,
at the end of 1TB position (1073741824 bytes) the offset is to be 1073741824
?
and similarly for end of 2TB position, the offset would be the twice?

Regards


On Tue, Apr 27, 2010 at 6:42 PM, Jens Axboe <jens.axboe@oracle.com> wrote:

> On Tue, Apr 27 2010, Michael O'Sullivan wrote:
> > Hi everyone,
> >
> > We are using fio to emulate the SPC-1 benchmark (we hope to get this
> > included as a --profile in fio in the not too distant future). However,
> > we have noticed that when running our benchmark the disk we are testing
> > does not seem to be overly busy seeking to different locations on the
> > disk. When we run tests using Iometer we can hear the disk seeking to
> > different sections of the disk pretty much the whole time. We are using
> > io_u->offset to define the position in the file we wish to read/write
> > at, but it is not clear to us whether this is having the desired effect.
> > It doesn't seem so at this point... We have been using the sync and
> > libaio engines.
>
> io_u->offset is the byte offset that the engine will seek to before
> performing IO, so whatever you put in there will be the real offset. If
> the disk doesn't sound as busy as it should, perhaps it's because
> iometer drives a higher queue depth as well?
>
> > Can someone please give us some advice to help us troubleshoot this
> > problem? Or point us to a tool that will let us see where fio is
> > reading/writing on the disk? Any help is greatly appreciated.
>
> For shorter samples, you could always just strace fio and see what it's
> doing. For a better view, you could use blktrace.
>
> --
> Jens Axboe
>
>

[-- Attachment #2: Type: text/html, Size: 2180 bytes --]

Re: Using offsets in fio

[Jens Axboe]

On Tue, Apr 27 2010, DongJin Lee wrote:
> So for example, you'd expect, for example,
> at the end of 1TB position (1073741824 bytes) the offset is to be 1073741824
> ?

1073741824 byte offset would be 1GB.

> and similarly for end of 2TB position, the offset would be the twice?

Yeah, there's no magic there, it's simply a byte offset where the IO
will take place.

-- 
Jens Axboe