Re: Approaches to making io_submit not block

[Dave Chinner <david@fromorbit.com>]

On Thu, Sep 01, 2011 at 06:39:47AM +0200, Andi Kleen wrote:
> > Allocations are already serialised by a single resource - the AGF
> > lock - so whether they block on the workqueue queue or on the AGF
> > lock is irrelevant to scheduling. And a single thread can only have
> 
> It's not about blocking, but about who gets the work accounted
> when it is done.

Don't trim the context away and respond with a completely different
argument that is irrelevant to the original context!  Accounting who
did the work is irrelevant to the discussion context of the fairness
of queuing and dispatching synchronous allocations via a FIFO wq
implementation....

> > If we get lots of allocations queued on the one per-CPU wq, they
> > will have all had to come from different contexts. In which case,
> > FIFO processing of the work queued up is *exactly* the fairness we
> > want, because that is exactly what doing them from process context
> > would end up with.
> 
> You want the work accounted to the originator so that it can be
> slowed down when it does too much (e.g. hit its cgroups or CFQ limits)

So fix the workqueue infrastructure to track it properly. Don't keep
bringing this up as a reason for saying moving work to workqueues is
bad.

> Networking learned these lessons a long time ago, it's much 
> better for overload behavior when as much as possible is done in
> process context.

Apples to oranges - there's orders of magnitude of difference in the
number of operations that the different stacks do. Allocation in XFS
when it does not block can still take milliseconds of CPU time; in
comparison, the networking stack is expected to process thousands of
packets in that same time frame.  IOWs, the scale of processing per
item of work is -vastly- different - that's why working in process
context matters a great deal to the networking stack but not to
allocation in XFS.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com