Re: [PATCH] thp: reduce khugepaged freezing latency

[Andrea Arcangeli <aarcange@redhat.com>]

On Wed, Nov 09, 2011 at 07:53:42AM -0800, Tejun Heo wrote:
> Hello, Andrea, Srivatsa.
> 
> On Wed, Nov 09, 2011 at 06:15:38PM +0530, Srivatsa S. Bhat wrote:
> > >> diff --git a/mm/huge_memory.c b/mm/huge_memory.c
> > >> index 4298aba..67311d1 100644
> > >> --- a/mm/huge_memory.c
> > >> +++ b/mm/huge_memory.c
> > >> @@ -2277,6 +2277,7 @@ static struct page *khugepaged_alloc_hugepage(void)
> > >>  		if (!hpage) {
> > >>  			count_vm_event(THP_COLLAPSE_ALLOC_FAILED);
> > >>  			khugepaged_alloc_sleep();
> > >> +			try_to_freeze();
> > >>  		} else
> > >>  			count_vm_event(THP_COLLAPSE_ALLOC);
> > >>  	} while (unlikely(!hpage) &&
> > >> @@ -2331,7 +2332,7 @@ static int khugepaged(void *none)
> > >>  {
> > >>  	struct mm_slot *mm_slot;
> > >>
> > >> -	set_freezable();
> > >> +	set_freezable_with_signal();
> > >>  	set_user_nice(current, 19);
> 
> Oooh, please don't do that.  It's already gone in the pm tree.  It
> would be best if wait_event_freezable_timeout() can be used
> (ie. wakeup condition should be set somewhere) but, if not, something
> like the following sould work.
> 
> static void khugepaged_alloc_sleep(void)
> {
> 	DEFINE_WAIT(wait);
> 	add_wait_queue(&khugepaged_wait, &wait);
> 	try_to_freeze();

XXXXX

> 	schedule_timeout_interruptible(
> 			msecs_to_jiffies(
> 				khugepaged_alloc_sleep_millisecs));
> 	try_to_freeze();
> 	remove_wait_queue(&khugepaged_wait, &wait);
> }

I thought about that but isn't there a race condition if TIF_FREEZE is
set just in the point I marked above? I thought the
set_freezable_with_signal by forcing the task runnable would fix it.

How exactly wait_event_freezable_timeout() would avoid the same race
as above? I mean the freezer won't have visibility on the
khugepaged_wait waitqueue head so it surely cannot wake it up. And if
the freezing() check happens before TIF_FREEZE get set but before
schedule() is called, we're still screwed even if I use
wait_event_freezable_timeout()... Or is the signal_pending check
fixing that? But without set_freezable_with_signal() we don't set
TIF_SIGPENDING... so it's not immediately care how this whole logic is
race free. If you use stop_machine that could avoid the races though,
but it doesn't look like the freezer uses that.