Re: nfsd becomes a zombie

Harald Dunkel
2024-07-02T17:32:13Z

Hi folks,

my NAS ran into this problem again. NFS got stuck somehow, and
the nfsd couldn't be killed :-(.

dmesg:
[Tue Jul 2 17:20:19 2024] receive_cb_reply: Got unrecognized reply: calldir 0x1 xpt_bc_xprt 00000000749c823f xid 5bf8d3d0
[Tue Jul 2 17:20:19 2024] receive_cb_reply: Got unrecognized reply: calldir 0x1 xpt_bc_xprt 00000000ce307050 xid 3b4fbd9f
[Tue Jul 2 17:20:19 2024] receive_cb_reply: Got unrecognized reply: calldir 0x1 xpt_bc_xprt 00000000f7f9161e xid 0a26635c
[Tue Jul 2 17:20:19 2024] receive_cb_reply: Got unrecognized reply: calldir 0x1 xpt_bc_xprt 000000007c978512 xid 384cbf0c
[Tue Jul 2 17:20:19 2024] receive_cb_reply: Got unrecognized reply: calldir 0x1 xpt_bc_xprt 00000000dc3c09f6 xid 53cc0e3e
[Tue Jul 2 17:20:19 2024] receive_cb_reply: Got unrecognized reply: calldir 0x1 xpt_bc_xprt 00000000d1675728 xid 129006af
[Tue Jul 2 17:20:19 2024] receive_cb_reply: Got unrecognized reply: calldir 0x1 xpt_bc_xprt 0000000047159b90 xid 0c06b6e0
[Tue Jul 2 17:20:19 2024] receive_cb_reply: Got unrecognized reply: calldir 0x1 xpt_bc_xprt 0000000008b3b3ac xid 641bb0da
[Tue Jul 2 17:20:19 2024] receive_cb_reply: Got unrecognized reply: calldir 0x1 xpt_bc_xprt 000000009eb832dc xid 005fcc99
[Tue Jul 2 17:20:19 2024] receive_cb_reply: Got unrecognized reply: calldir 0x1 xpt_bc_xprt 0000000042dcce88 xid b3cf5de4
[Tue Jul 2 17:20:19 2024] receive_cb_reply: Got unrecognized reply: calldir 0x1 xpt_bc_xprt 00000000b66bbd6f xid d4f06b56
[Tue Jul 2 17:20:19 2024] receive_cb_reply: Got unrecognized reply: calldir 0x1 xpt_bc_xprt 00000000b5e5e5a3 xid c032dbba
[Tue Jul 2 17:20:19 2024] receive_cb_reply: Got unrecognized reply: calldir 0x1 xpt_bc_xprt 00000000e123efc9 xid 99fa75d9
[Tue Jul 2 17:20:19 2024] receive_cb_reply: Got unrecognized reply: calldir 0x1 xpt_bc_xprt 00000000ca43f6f0 xid e38d5b74
[Tue Jul 2 17:20:19 2024] receive_cb_reply: Got unrecognized reply: calldir 0x1 xpt_bc_xprt 00000000ad683927 xid 277cde8c
[Tue Jul 2 17:20:19 2024] receive_cb_reply: Got unrecognized reply: calldir 0x1 xpt_bc_xprt 00000000e8e01f09 xid 641df4a4
[Tue Jul 2 17:20:19 2024] receive_cb_reply: Got unrecognized reply: calldir 0x1 xpt_bc_xprt 000000006223d195 xid 3dba2d2a
[Tue Jul 2 17:20:19 2024] receive_cb_reply: Got unrecognized reply: calldir 0x1 xpt_bc_xprt 00000000b73943aa xid a688e47f
[Tue Jul 2 17:20:19 2024] receive_cb_reply: Got unrecognized reply: calldir 0x1 xpt_bc_xprt 000000004cd80e49 xid 64e688ca
[Tue Jul 2 17:20:19 2024] receive_cb_reply: Got unrecognized reply: calldir 0x1 xpt_bc_xprt 00000000ef92587f xid 70bf2e44
[Tue Jul 2 17:20:19 2024] receive_cb_reply: Got unrecognized reply: calldir 0x1 xpt_bc_xprt 00000000a5ff94a6 xid c0f7a668
[Tue Jul 2 17:20:19 2024] receive_cb_reply: Got unrecognized reply: calldir 0x1 xpt_bc_xprt 00000000fd9a0890 xid 0df7d2c7
[Tue Jul 2 17:20:19 2024] receive_cb_reply: Got unrecognized reply: calldir 0x1 xpt_bc_xprt 00000000c42ddaac xid 800e710e
[Tue Jul 2 17:20:19 2024] receive_cb_reply: Got unrecognized reply: calldir 0x1 xpt_bc_xprt 00000000f43275cf xid 8b05e704
[Tue Jul 2 17:20:19 2024] receive_cb_reply: Got unrecognized reply: calldir 0x1 xpt_bc_xprt 000000009a1d5dcf xid 3c2ba924
[Tue Jul 2 17:20:19 2024] receive_cb_reply: Got unrecognized reply: calldir 0x1 xpt_bc_xprt 000000007cad732d xid e73a0429
[Tue Jul 2 17:20:19 2024] receive_cb_reply: Got unrecognized reply: calldir 0x1 xpt_bc_xprt 000000008e7d297f xid 075a98e5
[Tue Jul 2 17:20:19 2024] receive_cb_reply: Got unrecognized reply: calldir 0x1 xpt_bc_xprt 00000000ed964446 xid 8bb8e568
[Tue Jul 2 17:20:19 2024] receive_cb_reply: Got unrecognized reply: calldir 0x1 xpt_bc_xprt 00000000b14782f0 xid 4c4ae7c5
[Tue Jul 2 17:23:28 2024] INFO: task nfsd:3037 blocked for more than 120 seconds.
[Tue Jul 2 17:23:28 2024] Not tainted 6.1.0-21-amd64 #1 Debian 6.1.90-1
[Tue Jul 2 17:23:28 2024] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[Tue Jul 2 17:23:28 2024] task:nfsd state:D stack:0 pid:3037 ppid:2 flags:0x00004000
[Tue Jul 2 17:23:28 2024] Call Trace:
[Tue Jul 2 17:23:28 2024] <TASK>
[Tue Jul 2 17:23:28 2024] __schedule+0x34d/0x9e0
[Tue Jul 2 17:23:28 2024] schedule+0x5a/0xd0
[Tue Jul 2 17:23:28 2024] schedule_timeout+0x118/0x150
[Tue Jul 2 17:23:28 2024] wait_for_completion+0x86/0x160
[Tue Jul 2 17:23:28 2024] __flush_workqueue+0x152/0x420
[Tue Jul 2 17:23:28 2024] nfsd4_destroy_session+0x1b6/0x250 [nfsd]
[Tue Jul 2 17:23:28 2024] nfsd4_proc_compound+0x355/0x660 [nfsd]
[Tue Jul 2 17:23:28 2024] nfsd_dispatch+0x1a1/0x2b0 [nfsd]
[Tue Jul 2 17:23:28 2024] svc_process_common+0x289/0x5e0 [sunrpc]
[Tue Jul 2 17:23:28 2024] ? svc_recv+0x4e5/0x890 [sunrpc]
[Tue Jul 2 17:23:28 2024] ? nfsd_svc+0x360/0x360 [nfsd]
[Tue Jul 2 17:23:28 2024] ? nfsd_shutdown_threads+0x90/0x90 [nfsd]
[Tue Jul 2 17:23:28 2024] svc_process+0xad/0x100 [sunrpc]
[Tue Jul 2 17:23:28 2024] nfsd+0xd5/0x190 [nfsd]
[Tue Jul 2 17:23:28 2024] kthread+0xda/0x100
[Tue Jul 2 17:23:28 2024] ? kthread_complete_and_exit+0x20/0x20
[Tue Jul 2 17:23:28 2024] ret_from_fork+0x22/0x30
[Tue Jul 2 17:23:28 2024] </TASK>
[Tue Jul 2 17:23:28 2024] INFO: task nfsd:3038 blocked for more than 120 seconds.
[Tue Jul 2 17:23:28 2024] Not tainted 6.1.0-21-amd64 #1 Debian 6.1.90-1
[Tue Jul 2 17:23:28 2024] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.

/var/log/kern.log:
2024-06-28T10:40:40.273493+02:00 nasl006b kernel: [959982.169372] receive_cb_reply: Got unrecognized reply: calldir 0x1 xpt_bc_xprt 00000000d1675728 xid 372e06af
2024-06-28T10:40:40.273507+02:00 nasl006b kernel: [959982.169374] receive_cb_reply: Got unrecognized reply: calldir 0x1 xpt_bc_xprt 00000000506887ca xid 5be3c4d4
2024-06-28T10:40:40.273508+02:00 nasl006b kernel: [959982.169379] receive_cb_reply: Got unrecognized reply: calldir 0x1 xpt_bc_xprt 00000000b5e5e5a3 xid e5d0daba
2024-06-28T10:40:40.273509+02:00 nasl006b kernel: [959982.169423] receive_cb_reply: Got unrecognized reply: calldir 0x1 xpt_bc_xprt 00000000b66bbd6f xid 69696b56
2024-06-28T10:40:40.273509+02:00 nasl006b kernel: [959982.169498] receive_cb_reply: Got unrecognized reply: calldir 0x1 xpt_bc_xprt 0000000008b3b3ac xid 89b9afda
2024-06-28T10:40:40.273510+02:00 nasl006b kernel: [959982.169504] receive_cb_reply: Got unrecognized reply: calldir 0x1 xpt_bc_xprt 00000000a5ff94a6 xid e595a668
2024-06-28T10:40:40.273512+02:00 nasl006b kernel: [959982.169529] receive_cb_reply: Got unrecognized reply: calldir 0x1 xpt_bc_xprt 00000000f7f9161e xid 2fc4625c
2024-06-28T10:40:40.273513+02:00 nasl006b kernel: [959982.169659] receive_cb_reply: Got unrecognized reply: calldir 0x1 xpt_bc_xprt 00000000b73943aa xid cb26e47f
2024-06-28T10:40:40.273514+02:00 nasl006b kernel: [959982.169691] receive_cb_reply: Got unrecognized reply: calldir 0x1 xpt_bc_xprt 000000009a1d5dcf xid 61c9a824
2024-06-28T10:40:40.273514+02:00 nasl006b kernel: [959982.169697] receive_cb_reply: Got unrecognized reply: calldir 0x1 xpt_bc_xprt 00000000749c823f xid 8096d3d0
2024-06-28T10:40:40.944609+02:00 nasl006b kernel: [959983.506736] receive_cb_reply: Got unrecognized reply: calldir 0x1 xpt_bc_xprt 00000000190b801c xid bdd1dcd0
2024-06-28T10:40:40.948612+02:00 nasl006b kernel: [959983.512235] receive_cb_reply: Got unrecognized reply: calldir 0x1 xpt_bc_xprt 0000000042dcce88 xid d76d5de4
2024-06-28T10:40:40.952617+02:00 nasl006b kernel: [959983.514349] receive_cb_reply: Got unrecognized reply: calldir 0x1 xpt_bc_xprt 000000007cad732d xid 0bd90329
2024-06-28T10:40:40.952623+02:00 nasl006b kernel: [959983.514564] receive_cb_reply: Got unrecognized reply: calldir 0x1 xpt_bc_xprt 000000004cd80e49 xid 898488ca
2024-06-28T10:40:40.952624+02:00 nasl006b kernel: [959983.514951] receive_cb_reply: Got unrecognized reply: calldir 0x1 xpt_bc_xprt 00000000abccd646 xid d0d28401
2024-06-28T10:40:40.952624+02:00 nasl006b kernel: [959983.515009] receive_cb_reply: Got unrecognized reply: calldir 0x1 xpt_bc_xprt 00000000ef92587f xid 955d2e44
2024-06-28T10:40:40.952625+02:00 nasl006b kernel: [959983.515060] receive_cb_reply: Got unrecognized reply: calldir 0x1 xpt_bc_xprt 00000000ed964446 xid b056e568
2024-07-02T17:20:23.113792+02:00 nasl006b kernel: [1329564.790305] receive_cb_reply: Got unrecognized reply: calldir 0x1 xpt_bc_xprt 00000000b14782f0 xid 4c4ae7c5
2024-07-02T17:23:32.268700+02:00 nasl006b kernel: [1329753.944957] nfsd_dispatch+0x1a1/0x2b0 [nfsd]
2024-07-02T17:23:32.300740+02:00 nasl006b kernel: [1329753.969482] svc_process_common+0x289/0x5e0 [sunrpc]
2024-07-02T17:23:32.300757+02:00 nasl006b kernel: [1329753.969919] nfsd+0xd5/0x190 [nfsd]
2024-07-02T17:23:32.364636+02:00 nasl006b kernel: [1329754.041100] ? nfsd_svc+0x360/0x360 [nfsd]
2024-07-02T17:23:32.419012+02:00 nasl006b kernel: [1329754.088290] svc_process+0xad/0x100 [sunrpc]
2024-07-02T17:23:32.419020+02:00 nasl006b kernel: [1329754.088337] ret_from_fork+0x22/0x30
2024-07-02T17:23:32.443744+02:00 nasl006b kernel: [1329754.111842] svc_process+0xad/0x100 [sunrpc]
2024-07-02T17:23:32.443749+02:00 nasl006b kernel: [1329754.111882] ? kthread_complete_and_exit+0x20/0x20
2024-07-02T17:23:32.488628+02:00 nasl006b kernel: [1329754.161331] ? kthread_complete_and_exit+0x20/0x20

Regards
Harri