Re: [PATCH] kthread: run kthreadd with max priority SCHED_FIFO

[Andrew Morton <akpm@linux-foundation.org>]

On Mon, 17 Dec 2007 23:43:14 +0100 Michal Schmidt <mschmidt@redhat.com> wrote:

> kthreadd, the creator of other kernel threads, runs as a normal
> priority task. This is a potential for priority inversion when a task
> wants to spawn a high-priority kernel thread. A middle priority
> SCHED_FIFO task can block kthreadd's execution indefinitely and thus
> prevent the timely creation of the high-priority kernel thread.
>     
> This causes a practical problem. When a runaway real-time task is
> eating 100% CPU and we attempt to put the CPU offline, sometimes we
> block while waiting for the creation of the highest-priority
> "kstopmachine" thread. 
> 
> The fix is to run kthreadd with the highest possible SCHED_FIFO
> priority. Its children must still run as slightly negatively reniced
> SCHED_NORMAL tasks.

Did you hit this problem with the stock kernel, or have you been working on
other stuff?

A locked-up SCHED_FIFO process will cause kernel threads all sorts of
problems.  You've hit one instance, but there will be others.  (pdflush
stops working, for one).

The general approach we've taken to this is "don't do that".  Yes, we could
boost lots of kernel threads in the way which this patch does but this
actually takes control *away* from userspace.  Userspace no longer has the
ability to guarantee itself minimum possible latency without getting
preempted by kernel threads.

And yes, giving userspace this minimum-latency capability does imply that
userspace has a responsibility to not 100% starve kernel threads.  It's a
reasonable compromise, I think?