Re: [PATCH] watchdog: Make sure the watchdog thread gets CPU on loaded system

[Mandeep Singh Baines <msb@chromium.org>]

Don Zickus (dzickus@redhat.com) wrote:
> On Thu, Mar 15, 2012 at 03:00:51PM +0100, Peter Zijlstra wrote:
> > On Thu, 2012-03-15 at 13:42 +0100, Ingo Molnar wrote:
> > > So unless there's concensus to remove everything but the hard 
> > > lockup detection facilities, lets solve the technical problem at 
> > > hand, ok? 
> > 
> > Well, at least make it possible to disable the silly soft thing.
> > 
> > And I really wouldn't know how the soft thing could possible help,
> > except when not actually having a NMI watchdog. What case does it
> > trigger where the NMI one doesn't?
> 
> I think softlockup really boils down to a pre-emption disabled detector
> much like how the hardlockup really is a interrupts disabled detector.
> 
> The amount of code preventing the scheduler from running is most likely a
> lot lower than the code the prevents interrrupts from happening.
> 

Its a good tool for catching problems of scale. As we move to more and
more cores you'll uncover bugs where data structures start to blow up.
Hash tables get huge, when you have 100000s of processes or millions of
TCP flows, or cgroups or namespace. That critical section (spinlock,
spinlock_bh, or preempt_disable) that used to be OK might no longer be.

There are some labs that are already there:

http://www.wine-reviews.net/wine-reviews/news/megatux-to-run-1-million-copies-on-wine-to-simulate-the-internet.html

With the softlockup detector, they'll get a useful stack trace in dmesg
that they can then send to lkml so that we can fix a scalability issue
that we hadn't previously known about.

Regards,
Mandeep

> Cheers,
> Don
>