Re: [RFC][PATCH] mm: couple rcu and memory reclaim

[Peter Zijlstra <peterz@infradead.org>]

On Mon, 24 Sep 2007 16:52:15 +0530 Balbir Singh
<balbir@linux.vnet.ibm.com> wrote:

> Peter Zijlstra wrote:
> > On Mon, 24 Sep 2007 16:12:19 +0530 Balbir Singh
> > <balbir@linux.vnet.ibm.com> wrote:
> > 
> >> Peter Zijlstra wrote:
> >>> Just an idea I had, it seems like a good idea to wait for RCU callbacks
> >>> in reclaim so that we won't get all of memory stuck there.
> >>>
> >>> If this location is too aggressive we might stick it next to
> >>> disable_swap_token().
> >>>
> >>> ---
> >>> Couple RCU and reclaim.
> >>>
> >>> There could be a lot of memory stuck in RCU callbacks. Wait for RCU to
> >>> finish before giving it another go.
> >>>
> >>> Placed in kswapd and not direct reclaim path because kswapd never holds
> >>> rcu_read_lock() at this point and can thus not deadlock. Direct reclaim
> >>> callers might hold rcu_read_lock() and would suffer from deadlocks if
> >>> sync_rcu() were to be called.
> >>>
> >>> Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
> >>> ---

---
 mm/vmscan.c |    4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

Index: linux-2.6/mm/vmscan.c
===================================================================
--- linux-2.6.orig/mm/vmscan.c
+++ linux-2.6/mm/vmscan.c
@@ -1527,8 +1527,10 @@ loop_again:
 		 * OK, kswapd is getting into trouble.  Take a nap, then take
 		 * another pass across the zones.
 		 */
-		if (total_scanned && priority < DEF_PRIORITY - 2)
+		if (total_scanned && priority < DEF_PRIORITY - 2) {
+			synchronize_rcu();
 			congestion_wait(WRITE, HZ/10);
+		}
 
 		/*
 		 * We do this so kswapd doesn't build up large priorities for

> > Only kswapd can do this, direct reclaim has deadlock potential.
> 
> Yes, but not in all cases, do you want to add any gfp_mask
> based smartness for direct reclaim?

gfp_mask doesn't carry the needed information. It depends on whether
the current context holds a rcu_read_lock().

so something like:

   rcu_read_lock()
   foo = kmalloc(sizeof(foo))
      new_slab()
        __alloc_pages()
          try_to_free_pages()
            synchronise_rcu() <-- deadlock
   rcu_read_unlock()