RPC: multiple fragments per record not supported

RPC: multiple fragments per record not supported

[Simon Kirby]

Hello!

With 2.6.28.10 and 2.6.29.4 knfsd after a few hours of heavy concurrent
NFSv3 write loads (migration of Maildirs with rsync), we are seeing this
error:

	kernel: RPC: multiple fragments per record not supported

...followed by knfsd immediately ignoring all further requests. 
Restarting knfsd (/etc/init.d/nfs-kernel-server restart) seems to make
it start responding and work as normal again.

"ps -eo pid,stat,args,wchan" during the event shows all nfsd processes
sleeping in "svc_recv":

22714 S<   [nfsd]                      svc_recv

This client with the high write load is an old 2.6.17 kernel, mounting
the server with rw,nolock,hard,intr,timeo=9,retrans=3,rsize=8192,wsize=8192
(jumbo frames are enabled).

Is this any sort of known issue?  I'm trying to reproduce it in a testing
environment.

Cheers,

Simon-

Re: RPC: multiple fragments per record not supported

[NeilBrown]

On Tue, June 9, 2009 10:20 am, Simon Kirby wrote:
> Hello!
>
> With 2.6.28.10 and 2.6.29.4 knfsd after a few hours of heavy concurrent
> NFSv3 write loads (migration of Maildirs with rsync), we are seeing this
> error:
>
> 	kernel: RPC: multiple fragments per record not supported
>
> ...followed by knfsd immediately ignoring all further requests.
> Restarting knfsd (/etc/init.d/nfs-kernel-server restart) seems to make
> it start responding and work as normal again.
>
> "ps -eo pid,stat,args,wchan" during the event shows all nfsd processes
> sleeping in "svc_recv":
>
> 22714 S<   [nfsd]                      svc_recv
>
> This client with the high write load is an old 2.6.17 kernel, mounting
> the server with
> rw,nolock,hard,intr,timeo=9,retrans=3,rsize=8192,wsize=8192
> (jumbo frames are enabled).
>
> Is this any sort of known issue?  I'm trying to reproduce it in a testing
> environment.

Yes.  Known issue.  The data stream from the client gets corrupted.
Fix is

http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commitdiff;h=e0ab53deaa91293a7958d63d5a2cf4c5645ad6f0

which is in 2.6.18

NeilBrown