Re: [resend][PATCH v2] mlock() doesn't wait to finish lru_add_drain_all()

[Andrew Morton <akpm@linux-foundation.org>]

On Fri,  9 Oct 2009 11:21:55 +0900 (JST)
KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> wrote:

> Recently, Mike Galbraith reported mlock() makes hang-up very long time in
> his system. Peter Zijlstra explainted the reason.
> 
>   Suppose you have 2 cpus, cpu1 is busy doing a SCHED_FIFO-99 while(1),
>   cpu0 does mlock()->lru_add_drain_all(), which does
>   schedule_on_each_cpu(), which then waits for all cpus to complete the
>   work. Except that cpu1, which is busy with the RT task, will never run
>   keventd until the RT load goes away.
> 
>   This is not so much an actual deadlock as a serious starvation case.
> 
> His system has two partions using cpusets and RT-task partion cpu doesn't
> have any PCP cache. thus, this result was pretty unexpected.
> 
> The fact is, mlock() doesn't need to wait to finish lru_add_drain_all().
> if mlock() can't turn on PG_mlock, vmscan turn it on later.
> 
> Thus, this patch replace it with lru_add_drain_all_async().

So why don't we just remove the lru_add_drain_all() call from sys_mlock()?

How did you work out why the lru_add_drain_all() is present in
sys_mlock() anyway?  Neither the code nor the original changelog tell
us.  Who do I thwap for that?  Nick and his reviewers.  Sigh.

There are many callers of lru_add_drain_all() all over the place.  Each
of those is vulnerable to the same starvation issue, is it not?

If so, it would be better to just fix up lru_add_drain_all().  Afaict
all of its functions can be performed in hard IRQ context, so we can
use smp_call_function()?