Re: rq_affinity doesn't seem to work?

[Jens Axboe <axboe@kernel.dk>]

On 2011-07-13 19:10, Matthew Wilcox wrote:
> On Tue, Jul 12, 2011 at 10:30:35PM +0200, Jens Axboe wrote:
>> It's probably the grouping, we need to do something about that. Does the
>> below patch make it behave as you expect?
> 
> "something", absolutely.  But there is benefit from doing some aggregation
> (we tried disabling it entirely with the "well-known OLTP benchmark" and
> performance went down).

Yep, that's why the current solution is somewhat middle of the road...

> Ideally we'd do something like "if the softirq is taking up more than 10%
> of a core, split the grouping".  Do we have enough stats to do that kind
> of monitoring?

I don't think we have those stats, though it could/should be pulled from
the ksoftirqX threads. We could have some metric, ala

        dest_cpu = get_group_completion_cpu(rq->cpu);
        if (ksoftirqd_of(dest_cpu) >= 90% busy)
                dest_cpu = rq->cpu;

to send things completely local to the submitter of the IO, IFF the
current CPU is close to running at full tilt.

-- 
Jens Axboe