Re: [PATCH] soft lockup: kill realtime threads before panic

[Mike Galbraith <umgwanakikbuti@gmail.com>]

On Tue, 2015-07-21 at 22:18 -0700, Jörn Engel wrote:
> On Wed, Jul 22, 2015 at 06:36:30AM +0200, Mike Galbraith wrote:
> > On Tue, 2015-07-21 at 15:07 -0700, Spencer Baugh wrote:
> > 
> > > We have observed cases where the soft lockup detector triggered, but no
> > > kernel bug existed.  Instead we had a buggy realtime thread that
> > > monopolized a cpu.  So let's kill the responsible party and not panic
> > > the entire system.
> > 
> > If you don't tell the kernel to panic, it won't, and if you don't remove
> > its leash (the throttle), your not so tame rt beast won't maul you.
> 
> Not sure if this patch is something for mainline, but those two
> alternatives have problems of their own.  Not panicking on lockups can
> leave a system disabled until some human come around.  In many cases
> that human will do no better than power-cycle.  A panic reduces the
> downtime.

If a realtime task goes bonkers, the realtime game is over, you're down.
 
> And the realtime throttling gives non-realtime threads some minimum
> runtime, but does nothing to help low-priority realtime threads.  It
> also introduces latencies, often when workloads are high and you would
> like any available cpu to get through that rough spot.

You can use group scheduling as a debug crutch until the little beasts
are housebroken.

> I don't think we have a good answer to this problem in the mainline
> kernel yet.

IMHO, there's no point in trying to make rt warm/fuzzy/cuddly.  Just
don't stuff a Hells Angel into a super-suit, that gets real ugly ;-)

	-Mike