Re: Random IO errors on nfs clients running linux > 4.20

[Alberto Gonzalez Iniesta <alberto.gonzalez@udima.es>]

On Mon, Oct 26, 2020 at 09:58:17AM -0400, Chuck Lever wrote:
> >> So all I notice from this one is the readdir EIO came from call_decode.
> >> I suspect that means it failed in the xdr decoding.  Looks like xdr
> >> decoding of the actual directory data (which is the complicated part) is
> >> done later, so this means it failed decoding the header or verifier,
> >> which is a little odd:
> >> 
> >>> Sep  8 16:03:23 portatil264 kernel: [15033.016276] RPC:  3284 call_decode result -5
> >>> Sep  8 16:03:23 portatil264 kernel: [15033.016281] nfs41_sequence_process: Error 1 free the slot 
> >>> Sep  8 16:03:23 portatil264 kernel: [15033.016286] RPC:       wake_up_first(00000000d3f50f4d "ForeChannel Slot table")
> >>> Sep  8 16:03:23 portatil264 kernel: [15033.016288] nfs4_free_slot: slotid 0 highest_used_slotid 4294967295
> >>> Sep  8 16:03:23 portatil264 kernel: [15033.016290] RPC:  3284 return 0, status -5
> >>> Sep  8 16:03:23 portatil264 kernel: [15033.016291] RPC:  3284 release task
> >>> Sep  8 16:03:23 portatil264 kernel: [15033.016295] RPC:       freeing buffer of size 4144 at 00000000a3649daf
> >>> Sep  8 16:03:23 portatil264 kernel: [15033.016298] RPC:  3284 release request 0000000079df89b2
> >>> Sep  8 16:03:23 portatil264 kernel: [15033.016300] RPC:       wake_up_first(00000000c5ee49ee "xprt_backlog")
> >>> Sep  8 16:03:23 portatil264 kernel: [15033.016302] RPC:       rpc_release_client(00000000b930c343)
> >>> Sep  8 16:03:23 portatil264 kernel: [15033.016304] RPC:  3284 freeing task
> >>> Sep  8 16:03:23 portatil264 kernel: [15033.016309] _nfs4_proc_readdir: returns -5
> >>> Sep  8 16:03:23 portatil264 kernel: [15033.016318] NFS: readdir(departamentos/innovacion) returns -5
> > 
> > Hi, Bruce et al.
> > 
> > Is there anything we can do to help debugging/fixing this? It's still
> > biting our users with a +4.20.x kernel.
> 
> Alberto, can you share a snippet of a raw network capture that shows
> the READDIR Reply that fails to decode?

Hi, Chuck.

Thanks for your reply. We're using "sec=krb5p", which makes the network
capture useless :-(
I'm afraid it's quite difficult to disable that, since we have +200 NFS
clients on production.

-- 
Alberto González Iniesta             | Universidad a Distancia
alberto.gonzalez@udima.es            | de Madrid