Re: [PATCH] Deadlock during heavy write activity to userspace NFS server on local NFS mount

[Marcelo Tosatti <marcelo.tosatti@cyclades.com>]

On Thu, Jul 29, 2004 at 06:29:12PM +1000, Nick Piggin wrote:
> Avi Kivity wrote:
> >Nick Piggin wrote:
> >
> >>Avi Kivity wrote:
> >>
> >>>Nick Piggin wrote:
> >>
> >>
> >>
> >>>>>What's stopping the NFS server from ooming the machine then? Every 
> >>>>>time some bit of memory becomes free, the server will consume it 
> >>>>>instantly. Eventually ext3 will not be able to write anything out 
> >>>>>because it is out of memory.
> >>>>>
> >>>>The NFS server should do the writeout a page at a time.
> >>>
> >>>
> >>>
> >>>
> >>>The NFS server writes not only in response to page reclaim (as a 
> >>>local NFS client), but also in response to pressure from non-local 
> >>>clients. If both ext3 and NFS have the same allocation limits, NFS 
> >>>may starve out ext3.
> >>>
> >>
> >>What do you mean starve out ext3? ext3 gets written to *by the NFS 
> >>server*
> >>which is PF_MEMALLOC. 
> >
> >
> >When the NFS server writes, it allocates pagecache and temporary 
> >objects. When ext3 writes, it allocates temporary objects. If the NFS 
> >server writes too much, ext3 can't allocate memory, and will never be 
> >able to allocate memory.
> >
> 
> That is because your NFS server shouldn't hog as much memory as
> it likes when it is PF_MEMALLOC. The entire writeout path should
> do a page at a time if it is PF_MEMALLOC. Ie, the server should
> be doing write, fsync.
> 
> But now that I think about it, I guess you may not be able to
> distinguish that from regular writeout, so doing a page at a time
> would hurt performance too much.
> 
> Hmm so I guess the idea of a per task reserve limit may be the way
> to do it, yes. Thanks for bearing with me!

Hi, 

By reading the discussion I also agree that we need levels of "allowed deepness"
into the reservations.

We could have a global limit for normal allocators, and per-task 
limit for special "kswapd helpers" (which run with PF_MEMALLOC). 
And with kswapd being the most "deep" eater, as you guys said.

The thing is, those deadlocks are quite uncommon special cases and 
we will need to change core VM logic to handle them. Well...

We need to come up with a generic way of doing it, as Avi says, 
otherwise people will have to keep doing "hacks" to make it work.

Someone needs to sit down and come up with a design. It shouldnt 
be that hard.

Just my two pennies...