Re: [PATCH] RT: Add priority-queuing and priority-inheritance to workqueue infrastructure

[Peter Zijlstra <peterz@infradead.org>]

On Mon, 2007-08-06 at 15:29 +0200, Peter Zijlstra wrote:
> On Mon, 2007-08-06 at 17:18 +0400, Oleg Nesterov wrote:
> 
> > Yes, I still disagree with the whole idea because I hope we can make
> > something more simpler to solve the problem, but I must admit I don't
> > quite understand what the problem is.
> > 
> > So, please consider a noise from my side as my attempt to help. And
> > in fact, I am very curious about -rt tree, just I never had a time
> > to study it :)
> 
> 
> Well, the thing is, suppose we have 2 drivers both using keventd say a
> NIC and some USB thingy.
> 
> Now the NIC is deemed important hand gets irq thread prio 90, and the
> USB could not be cared less about and gets 10 (note that on -rt irq
> handlers are threaded and run SCHED_FIFO).
> 
> So now you can get priority inversion in keventd. Say the USB thingy
> schedules a work item which will be executed. Then during the execution
> of this work the NIC will also schedule a work item. Now the NIC (fifo
> 90) will have to wait for the USB work (fifo 10) to complete.

/me hits himself.

of course today everything will run on whatever prio keventd ends up,
regardless of the prio of the submitter.

still this does not change the fundamental issue of a high prio piece of
work waiting on a lower prio task.

> The typical solution is priority inheritance, where the highest prio of
> any waiter is propagated to the currently running work, so that it can
> finish and get on with the more important work.
> 
> 
> So these patches aimed to provide proper PI in the workqueue structure
> to avoid this problem.
> 
> However as you rightly noted, this horribly breaks the barrier/flush
> semantics.
> 
> I suspect most of the barrier/flush semantics could be replaced with
> completions from specific work items. Doing this will be a lot of work
> though.
> 
> I hope this rambling is not confusing you any further :-)