Re: [PATCH] Deadlock during heavy write activity to userspace NFS server on local NFS mount

[Avi Kivity <avi@exanet.com>]

Pavel Machek wrote:

>I'd hope that kswapd was carefully to make sure that it always has
>enough pages...
>
>...it is harder to do the same auditing with userland program.
>
>  
>
Very true. But is a kernel thread like kswapd depends on a userspace 
program, then that program better be well behaved.

>>A more complete solution would be to assign memory reserve levels below 
>>which a process starts allocating synchronously. For example, normal 
>>processes must have >20MB to make forward progress, kswapd wants >15MB 
>>and the NFS server needs >10MB. Some way would be needed to express the 
>>dependencies.
>>    
>>
>
>Yes, something like that would be neccessary. I believe it would be
>slightly more complicated, like
>
>"NFS server needs > 10MB *and working kswapd*", so you'd need 25MB in
>fact... and this info should be stored in some readable form so that
>it can be checked.
>
>  
>
If the NFS server needed kswapd, we'd deadlock pretty soon, as kswapd 
*really* needs the NFS server. In our case, all block I/O is done using 
unbuffered I/O, and all memory is preallocated, so we don't need kswapd 
at all, just that small bit of memory that syscalls consume.

If the NFS server really needs kswapd, then there'd better be two of 
them. Regular processes would depend on one kswapd, which depends on the 
NFS server, which depends on the second kswapd, which depends on the 
hardware alone. It should be fun trying to describe that topology to the 
kernel through some API.

Our filesystem actually does something like that internally, except the 
dependency chain length is seven, not two.

Avi