Re: [PATCH] kthread: run kthreadd with max priority SCHED_FIFO

[Jon Masters <jcm@redhat.com>]

On Sat, 2007-12-22 at 01:30 -0800, Andrew Morton wrote:
> On Mon, 17 Dec 2007 23:43:14 +0100 Michal Schmidt <mschmidt@redhat.com> wrote:
> 
> > kthreadd, the creator of other kernel threads, runs as a normal
> > priority task. This is a potential for priority inversion when a task
> > wants to spawn a high-priority kernel thread. A middle priority
> > SCHED_FIFO task can block kthreadd's execution indefinitely and thus
> > prevent the timely creation of the high-priority kernel thread.
> >     
> > This causes a practical problem. When a runaway real-time task is
> > eating 100% CPU and we attempt to put the CPU offline, sometimes we
> > block while waiting for the creation of the highest-priority
> > "kstopmachine" thread. 
> > 
> > The fix is to run kthreadd with the highest possible SCHED_FIFO
> > priority. Its children must still run as slightly negatively reniced
> > SCHED_NORMAL tasks.
> 
> Did you hit this problem with the stock kernel, or have you been working on
> other stuff?

This kind of problem is *far* more likely to happen on the -RT kernel
(more example cases), but it's also a general problem too.

> A locked-up SCHED_FIFO process will cause kernel threads all sorts of
> problems.  You've hit one instance, but there will be others.  (pdflush
> stops working, for one).

Right. Agreed that this is just one "fix" out of many possibly needed,
if upstream wants to address this kind of problem.

> The general approach we've taken to this is "don't do that".  Yes, we could
> boost lots of kernel threads in the way which this patch does but this
> actually takes control *away* from userspace.  Userspace no longer has the
> ability to guarantee itself minimum possible latency without getting
> preempted by kernel threads.
> 
> And yes, giving userspace this minimum-latency capability does imply that
> userspace has a responsibility to not 100% starve kernel threads.  It's a
> reasonable compromise, I think?

So, user tasks running with SCHED_FIFO should be able to lock a system?
I guess I see both sides of this argument - yes, it's userspace at
fault, but in other cases when userspace is at fault, we take action
(OOM, segfault, others). Isn't this situation just another case where
the kernel needs to avoid the evils of userland going awry?

Jon.