multiple --name parameters with non-file engine?

multiple --name parameters with non-file engine?

[Mark Nelson]

Hi Guys,

I'm using the librbd engine and trying to determine what exactly happens 
when multiple --name parameters are passed.  Primarily, I'm wondering if 
each fio process ends up writing to the same blocks on the rbd volume 
during sequential writes.  Looking at the code, I see a note that we 
pretend to deal with files even if the engine doesn't understand the 
concept of files:

https://github.com/axboe/fio/blob/master/engines/rbd.c#L405
https://github.com/axboe/fio/blob/master/filesetup.c#L1239

I'm not seeing anything (might be missing it) in the rbd engine or there 
that would partition the volume based on the process though.  Am I 
missing anything?

Thanks,
Mark

Re: multiple --name parameters with non-file engine?

[Jens Axboe]

On 2014-08-18 19:42, Mark Nelson wrote:
> Hi Guys,
>
> I'm using the librbd engine and trying to determine what exactly happens
> when multiple --name parameters are passed.  Primarily, I'm wondering if
> each fio process ends up writing to the same blocks on the rbd volume
> during sequential writes.  Looking at the code, I see a note that we
> pretend to deal with files even if the engine doesn't understand the
> concept of files:
>
> https://github.com/axboe/fio/blob/master/engines/rbd.c#L405
> https://github.com/axboe/fio/blob/master/filesetup.c#L1239
>
> I'm not seeing anything (might be missing it) in the rbd engine or there
> that would partition the volume based on the process though.  Am I
> missing anything?

The same thing will happen as if you ran two instances of fio with the 
same options. For rbd, each job will create/connect/open and rbd 
instance. For each of these, you can set the rbd pool and clientname.

-- 
Jens Axboe

Re: multiple --name parameters with non-file engine?

[Mark Nelson]

On 08/22/2014 05:23 PM, Jens Axboe wrote:
> On 2014-08-18 19:42, Mark Nelson wrote:
>> Hi Guys,
>>
>> I'm using the librbd engine and trying to determine what exactly happens
>> when multiple --name parameters are passed.  Primarily, I'm wondering if
>> each fio process ends up writing to the same blocks on the rbd volume
>> during sequential writes.  Looking at the code, I see a note that we
>> pretend to deal with files even if the engine doesn't understand the
>> concept of files:
>>
>> https://github.com/axboe/fio/blob/master/engines/rbd.c#L405
>> https://github.com/axboe/fio/blob/master/filesetup.c#L1239
>>
>> I'm not seeing anything (might be missing it) in the rbd engine or there
>> that would partition the volume based on the process though.  Am I
>> missing anything?
>
> The same thing will happen as if you ran two instances of fio with the
> same options. For rbd, each job will create/connect/open and rbd
> instance. For each of these, you can set the rbd pool and clientname.
>

Thanks Jens.  What I'd like to be able to do is start multiple fio 
processes but have each one only write to a non-overlapping portion of 
the block device.  Preferably I wouldn't have to manually figure out the 
ranges myself.  Any way to do something like that?

For now if we are interested in 4MB IOs to RBD we can just do 4MB random 
as that more or less translates into the same thing behind the scenes in 
Ceph anyway.

Mark

Re: multiple --name parameters with non-file engine?

[Sitsofe Wheeler]

On 23 August 2014 02:45, Mark Nelson <mark.a.nelson@gmail.com> wrote:
> Thanks Jens.  What I'd like to be able to do is start multiple fio processes
> but have each one only write to a non-overlapping portion of the block
> device.  Preferably I wouldn't have to manually figure out the ranges
> myself.  Any way to do something like that?

A quick Google for "fio zones jobs" tuns up
http://www.spinics.net/lists/fio/msg01791.html where the fourth answer
on that page seems relevant to your case.

For each job (essentially name section) you can set an offset and a
size and I/O will be done between offset and offset+size. If you want
to move the starting offset of each extra job automatically you could
use offset_increment
(http://git.kernel.dk/?p=fio.git;a=blob;f=HOWTO;h=73e58ff695c3384df30094d6c15ebe9ed615460f;hb=HEAD#l770
). You can also use numjobs
(http://git.kernel.dk/?p=fio.git;a=blob;f=HOWTO;h=73e58ff695c3384df30094d6c15ebe9ed615460f;hb=HEAD#l1275
) to automatically create job clones.

Perhaps this sort of "parallel non-overlapping zoned I/O" case would
be a useful example fio job file (for example I never realised size
was relative to offset...)?

-- 
Sitsofe | http://sucs.org/~sits/