Re: rpc.mountd can be blocked by a bad client

[NeilBrown <neilb@suse.de>]

[-- Attachment #1: Type: text/plain, Size: 2183 bytes --]

On Wed, 24 Sep 2014 12:57:09 +0200 "Strösser, Bodo"
<bodo.stroesser@ts.fujitsu.com> wrote:

> Hello,
> 
> a few days ago we had some trouble with a NFS server. The clients most of the time no longer
> could mount any shares, but in rare cases they had success.
> 
> We found out, that during the times when mounts failed, rpc.mountd hung on a write() to a TCP
> socket. netstat showed, that Send-Q was full and Recv-Q counted up slowly. After a long time
> the write ended with an error ("TCP timeout" IIRC) and rpc.mountd worked normally for a short
> while until it again hung on write() for the same reason. The problem was caused by a MTU size
> configured wrong. So, one single bad client (or as much clients as the number of threads used
> by rpc.mountd) can block rpc.mountd entirely.
> 
> But what will happen, if someone intentionally sends RPC requests, but doesn't read() the
> answers? I wrote a small tool to test this situation. It fires DUMP requests to rpc.mountd as
> fast as possible, but does not read from the socket. The result is the same as with the
> problem above: rpc.mountd hangs in write() and no longer responds to other requests while no
> TCP timeout breaks up this situation.
> 
> So it's quite easy to intentionally block rpc.mountd from remote.

That's rather nasty.
We could possibly set the socket to be non-blocking, or we could set an alarm
just before handling a request.
Probably rpc_dispatch() in support/nfs/rpcdispatch.c would be the best place
to put the timeout.
 catch SIGALRM (don't set SA_RESTART)
 alarm(10);
 call svc_sendreply
 alarm(0);

if the alarm fires while svc_sendreply is writing to the socket it should get
an error and close the connection.

This would only fix mountd (as it is the only process to use rpc_dispatch).
Is a similar thing needed for statd I wonder??  It isn't so important.

NeilBrown

> 
> Please CC me, I'm not on the list.
> 
> Best regards,
> Bodo
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]