Re: uneven IOPS among HD

[Jens Axboe <jaxboe@fusionio.com>]

On 2011-02-17 03:14, john smith wrote:
> I have a couple of questions:
> 
> 1) if it is normal (and if so, why) that while running a number of identical jobs (e.g 4), each bound to a different CPU (cpu_allowed=i) and each performing sequential (bs=512) reads on (e.g. 4) different HD, fio reports IOPS numbers that are highly unbalanced among HD with one HD reporting more than 7x of the IOPS of the minimum IOPS HD.
> Different runs may show most of the IOPS on a different HD, so it's not that one HD is faster.
> 
> Here are some possibly interesting parameters:
> 
> ioengine=libaio
> iodepth=32
> direct=1

There should be very little variance between runs, and between jobs.
Each are separate threads/processes, so most of the variance you will
see will be due to system "artifacts". So I'd suggest you look there. If
you don't pin to specific CPUs, do you see smaller or larger
differences?

Are you doing random or sequential IO? Could also be timing differences
in submission and dispatch from the IO scheduler.

> 2) why increasing "iodepth' (eg to 512) decreses the IOPS results significantly (by 10x)?

You probably don't want a larger depth in fio than the system supports.

-- 
Jens Axboe