Re: uneven IOPS among HD

[Jens Axboe <jaxboe@fusionio.com>]

On 2011-02-26 15:31, Josh Aune wrote:
> On Sunday, February 20, 2011, Jens Axboe <jaxboe@fusionio.com> wrote:
>> On 2011-02-17 03:14, john smith wrote:
>>> I have a couple of questions:
>>>
>>> 1) if it is normal (and if so, why) that while running a number of identical jobs (e.g 4), each bound to a different CPU (cpu_allowed=i) and each performing sequential (bs=512) reads on (e.g. 4) different HD, fio reports IOPS numbers that are highly unbalanced among HD with one HD reporting more than 7x of the IOPS of the minimum IOPS HD.
>>> Different runs may show most of the IOPS on a different HD, so it's not that one HD is faster.
>>>
>>> Here are some possibly interesting parameters:
>>>
>>> ioengine=libaio
>>> iodepth=32
>>> direct=1
>>
>> There should be very little variance between runs, and between jobs.
>> Each are separate threads/processes, so most of the variance you will
>> see will be due to system "artifacts". So I'd suggest you look there. If
>> you don't pin to specific CPUs, do you see smaller or larger
>> differences?
>>
>> Are you doing random or sequential IO? Could also be timing differences
>> in submission and dispatch from the IO scheduler.
> 
> Is hyper threading enabled?  Could be running some of the jobs on the
> same CPU core.
> 
> Does it track to different sockets?

No, the replay doesn't factor in CPU placement of the apps submitting
IO.

-- 
Jens Axboe