Re: [PATCH] RT: Add priority-queuing and priority-inheritance to workqueue infrastructure

[Peter Zijlstra <peterz@infradead.org>]

On Mon, 2007-08-06 at 17:18 +0400, Oleg Nesterov wrote:

> Yes, I still disagree with the whole idea because I hope we can make
> something more simpler to solve the problem, but I must admit I don't
> quite understand what the problem is.
> 
> So, please consider a noise from my side as my attempt to help. And
> in fact, I am very curious about -rt tree, just I never had a time
> to study it :)


Well, the thing is, suppose we have 2 drivers both using keventd say a
NIC and some USB thingy.

Now the NIC is deemed important hand gets irq thread prio 90, and the
USB could not be cared less about and gets 10 (note that on -rt irq
handlers are threaded and run SCHED_FIFO).

So now you can get priority inversion in keventd. Say the USB thingy
schedules a work item which will be executed. Then during the execution
of this work the NIC will also schedule a work item. Now the NIC (fifo
90) will have to wait for the USB work (fifo 10) to complete.

The typical solution is priority inheritance, where the highest prio of
any waiter is propagated to the currently running work, so that it can
finish and get on with the more important work.


So these patches aimed to provide proper PI in the workqueue structure
to avoid this problem.

However as you rightly noted, this horribly breaks the barrier/flush
semantics.

I suspect most of the barrier/flush semantics could be replaced with
completions from specific work items. Doing this will be a lot of work
though.

I hope this rambling is not confusing you any further :-)