Re: [PATCH] Deadlock during heavy write activity to userspace NFS server on local NFS mount

[Nick Piggin <nickpiggin@yahoo.com.au>]

Avi Kivity wrote:
> Pavel Machek wrote:
> 
>> I'd hope that kswapd was carefully to make sure that it always has
>> enough pages...
>>
>> ...it is harder to do the same auditing with userland program.
>>
>>  
>>
> Very true. But is a kernel thread like kswapd depends on a userspace 
> program, then that program better be well behaved.
> 
>>> A more complete solution would be to assign memory reserve levels 
>>> below which a process starts allocating synchronously. For example, 
>>> normal processes must have >20MB to make forward progress, kswapd 
>>> wants >15MB and the NFS server needs >10MB. Some way would be needed 
>>> to express the dependencies.
>>>   
>>
>>
>> Yes, something like that would be neccessary. I believe it would be
>> slightly more complicated, like
>>
>> "NFS server needs > 10MB *and working kswapd*", so you'd need 25MB in
>> fact... and this info should be stored in some readable form so that
>> it can be checked.
>>
>>  
>>
> If the NFS server needed kswapd, we'd deadlock pretty soon, as kswapd 
> *really* needs the NFS server. In our case, all block I/O is done using 
> unbuffered I/O, and all memory is preallocated, so we don't need kswapd 
> at all, just that small bit of memory that syscalls consume.
> 
> If the NFS server really needs kswapd, then there'd better be two of 
> them. Regular processes would depend on one kswapd, which depends on the 
> NFS server, which depends on the second kswapd, which depends on the 
> hardware alone. It should be fun trying to describe that topology to the 
> kernel through some API.
> 
> Our filesystem actually does something like that internally, except the 
> dependency chain length is seven, not two.
> 

There is some need arising for a call to set the PF_MEMALLOC flag for
userspace tasks, so you could probably get a patch accepted. Don't
call it KSWAPD_HELPER though, maybe MEMFREE or RECLAIM or RECLAIM_HELPER.

But why is your NFS server needed to reclaim memory? Do you have the
filesystem mounted locally?