Re: New driver mtipx2xx submission

[Jeff Moyer <jmoyer@redhat.com>]

"Asai Thambi Samymuthu Pattrayasamy (asamymuthupa) [CONTRACTOR]"
<asamymuthupa@micron.com> writes:

>> 2.6.39 introduced the on-stack plugging work from Jens, the intent of
>> which is to reduce queue lock contention.  It would be great if you
>> could run with that kernel and noop to see if we make up some of the
>> performance gap (which looks to be just north of 10%).
>> 
>> Asai, thanks for running these tests and providing all of this data!
>> 
>> -Jeff
>
> At least with noop and deadline for ahci driver, now queue lock is not
> the top offender. With the new change in plugging, seems noop and
> deadline are spending more time in processing I/O similar to Micron
> block driver.
>
> With the current test results with 2.6.39.1 (new optimization for
> plugging) and application queue depth of 32, 
>      * Micron block driver exhibits 43% better IOPS than ahci driver
> with noop
>      * Micron block driver slightly better in CPU utilization.
>
> With application queue depth of 256, Micron block driver is able to
> leverage the device capability, and hence performance increases more
> than 225%.
>

>From the perf report, I would have guessed that the CPU utilization for
the ahci test case would have been lower than the Micron block driver.
Odd, I wonder what I'm missing.  Asai, did you notice if any of the CPUs
was completely pegged during testing with ahci?  You're using a NUMA
box, right?  I also wonder what the irq distribution looked like, and
whether rq_affinity is hurting performance for the ahci case.  Also,
does the Micron driver do any sort of interrupt coalescing that maybe
the ahci driver isn't doing?

Anywho, a 40% difference is pretty significant (though NUMA can have
that sort of impact).  Alan, what do you think?  I was never clear on
how exactly the ahci driver would handle a queue depth larger than 32
(if it can't, then clearly we'd need a block driver for this hardware).

Cheers,
Jeff