All of lore.kernel.org
 help / color / mirror / Atom feed
* kernel panic when starting nfsd on OpenWrt with kernel 6.12.19
@ 2025-03-22 10:05 John
  2025-03-22 14:49 ` Chuck Lever
  0 siblings, 1 reply; 6+ messages in thread
From: John @ 2025-03-22 10:05 UTC (permalink / raw)
  To: linux-nfs@vger.kernel.org

Starting nfsd with kernel 6.12.19 triggers a kernel panic. Using the 6.6.83 kernel is rock solid. I confirmed this on two x86/64 machines (Intel-N150 based and AMD 8845H-based). Posting here seeking debug advice. It is not clear to me if this is something in the kernel code or in something else within the distro.

I get the following from dmesg:
Kernel BUG at nfsd4_probe_callback_sync+0x1094/0x25b0 [nfsd] [verbose debug info unavailable]
Oops: invalid opcode: 0000 [#1] SMP NOPTI
CPU: 2 UID: 0 PID: 17091 Comm: rpc.nfsd Not tainted 6.12.19 #0
Hardware name: iKOOLCORE R2Max/R2Max, BIOS v1.1 12/17/2024
RIP: 0010:nfsd4_probe_callback_sync+0x1094/0x25b0 [nfsd]
Code: 89 de 48 c7 c7 e0 20 95 a0 e8 08 fc ff ff 41 89 c4 85 c0 0f 85 ce 2d 00 00 48 c7 c7 58 dd 95 a0 45 31 e4 e8 8e 7a 0b e1 eb 08 <0f> 0b 41 bc f4 ff ff ff 48 83 c4 08 44 89 e0 5b 41 5c 41 5d 41 5e
RSP: 0018:ffffc9000613bc98 EFLAGS: 00010282
RAX: 0000000000000049 RBX: ffff888105e06000 RCX: 0000000000000000
RDX: ffff88846fb21920 RSI: ffff88846fb1d980 RDI: ffff88846fb1d980
RBP: ffffc9000613bcc0 R08: 0000000000000000 R09: ffffffff824b56c8
R10: ffffffff8249d688 R11: 0000000000000003 R12: ffffffff82736400
R13: ffff888105e06000 R14: ffff888105e06000 R15: ffffc9000613bcd0
FS:  00007f1a404e5b28(0000) GS:ffff88846fb00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f1a40411000 CR3: 000000010d47a006 CR4: 0000000000370ef0
Call Trace:
 <TASK>
 ? show_regs.part.0+0x1d/0x20
 ? __die+0x52/0x91
 ? die+0x2a/0x50
 ? do_trap+0x103/0x110
 ? do_error_trap+0x6c/0x90
 ? nfsd4_probe_callback_sync+0x1094/0x25b0 [nfsd]
 ? exc_invalid_op+0x4f/0x70
 ? nfsd4_probe_callback_sync+0x1094/0x25b0 [nfsd]
 ? asm_exc_invalid_op+0x1b/0x20
 ? nfsd4_probe_callback_sync+0x1094/0x25b0 [nfsd]
 ? nfsd4_probe_callback_sync+0xff6/0x25b0 [nfsd]
 nfsd4_client_tracking_init+0x39/0x150 [nfsd]
 nfs4_state_start_net+0x2ce/0x370 [nfsd]
 nfsd_svc+0x1a0/0x2d0 [nfsd]
 nfssvc_encode_voidres+0x19a1/0x1be0 [nfsd]
 ? simple_transaction_get+0xb7/0xe0
 ? nfssvc_encode_voidres+0x18f0/0x1be0 [nfsd]
 nfssvc_encode_voidres+0x1a3/0x1be0 [nfsd]
 vfs_write+0xcb/0x390
 ? putname+0x4c/0x60
 ksys_write+0x57/0xd0
 __x64_sys_write+0x14/0x20
 x64_sys_call+0x79/0x1780
 do_syscall_64+0x7b/0x190
 entry_SYSCALL_64_after_hwframe+0x67/0x6f
RIP: 0033:0x7f1a404c9659
Code: c3 8b 07 85 c0 75 24 49 89 fb 48 89 f0 48 89 d7 48 89 ce 4c 89 c2 4d 89 ca 4c 8b 44 24 08 4c 8b 4c 24 10 4c 89 5c 24 08 0f 05 <c3> e9 82 d8 ff ff 41 54 b8 02 00 00 00 55 48 89 f5 be 00 88 08 00
RSP: 002b:00007fff99aeb2c8 EFLAGS: 00000246 ORIG_RAX: 0000000000000001
RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f1a404c9659
RDX: 0000000000000003 RSI: 000000000040d360 RDI: 0000000000000003
RBP: 00007f1a404e5b28 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000001
R13: 0000000000020000 R14: 0000000000409112 R15: 00007f1a40430dd0
 </TASK>
Modules linked in: xt_connlimit pppoe ppp_async nf_conncount i915 xt_state xt_helper xt_conntrack xt_connmark xt_connlabel xt_connbytes xt_CT wireguard video pppox ppp_mppe ppp_generic nft_redir nft_nat nft_masq nft_flow_offload nft_fib_inet nft_ct nft_chain_nat nf_nat nf_flow_table_inet nf_flow_table nf_conntrack_netlink nf_conntrack libchacha20poly1305 ipt_REJECT iavf i40e curve25519_x86_64 chacha_x86_64 cdc_ncm cdc_ether aqc111 zstd xt_time xt_tcpudp xt_tcpmss xt_statistic xt_recent xt_quota xt_pkttype xt_owner xt_multiport xt_mark xt_mac xt_limit xt_length xt_hl xt_ecn xt_dscp xt_comment xt_cgroup xt_addrtype xt_TCPMSS xt_LOG xt_HL xt_DSCP xt_CLASSIFY xt_CHECKSUM wmi usbnet unix_diag ts_kmp ts_fsm ts_bm slhc sch_cake poly1305_x86_64 nft_reject_ipv6 nft_reject_ipv4 nft_reject_inet nft_reject nft_quota nft_queue nft_numgen nft_log nft_limit nft_hash nft_fib_ipv6 nft_fib_ipv4 nft_fib nft_compat nfnetlink_queue nf_tables nf_reject_ipv4 nf_log_syslog nf_defrag_ipv6 nf_defrag_ipv4 macvlan lzo_rle lzo libie
 libeth libcurve25519_generic libchacha iptable_mangle iptable_filter ipt_ECN ip_tables forcedeth e1000e drm_display_helper drm_buddy crc_ccitt cdc_acm bnx2 atlantic af_packet_diag sch_teql sch_sfq sch_multiq sch_gred sch_fq sch_ets sch_codel em_text em_nbyte em_meta em_cmp act_skbmod act_simple act_pedit act_csum sch_tbf sch_ingress sch_htb sch_hfsc em_u32 cls_u32 cls_route cls_matchall cls_fw cls_flow cls_basic act_skbedit act_mirred act_gact configs evdev drivetemp i2c_piix4 i2c_smbus i2c_dev xt_set ip_set_list_set ip_set_hash_netportnet ip_set_hash_netport ip_set_hash_netnet ip_set_hash_netiface ip_set_hash_net ip_set_hash_mac ip_set_hash_ipportnet ip_set_hash_ipportip ip_set_hash_ipport ip_set_hash_ipmark ip_set_hash_ipmac ip_set_hash_ip ip_set_bitmap_port ip_set_bitmap_ipmac ip_set_bitmap_ip ip_set nfnetlink dwmac_intel ip6t_rt ip6t_mh ip6t_ipv6header ip6t_hbh ip6t_frag ip6t_eui64 ip6t_ah ip6table_mangle ip6table_filter ip6_tables ip6t_REJECT x_tables nf_reject_ipv6 nfsv4 dwmac_generic
 stmmac_platform stmmac nfsd nfs bonding ixgbe igbvf e1000 amd_xgbe ifb sctp udp_tunnel ip6_udp_tunnel mdio netlink_diag udp_diag tcp_diag raw_diag inet_diag rpcsec_gss_krb5 auth_rpcgss oid_registry veth lockd sunrpc grace dns_resolver nls_utf8 pcs_xpcs macsec ena ecdh_generic ecc xxhash_generic crypto_user algif_skcipher algif_rng algif_hash algif_aead af_alg sha512_ssse3 sha512_generic sha1_ssse3 sha1_generic seqiv sha3_generic jitterentropy_rng drbg md5 kpp hmac geniv rng des_generic libdes cts chacha20poly1305 blake2b_generic authencesn authenc arc4 crypto_acompress nls_iso8859_1 nls_cp437 r8169 mdio_devres xhci_plat_hcd iTCO_wdt iTCO_vendor_support fsl_mph_dr_of ehci_platform ehci_fsl igb igc vfat fat btrfs zstd_decompress zstd_compress zstd_common xxhash xor raid6_pq lzo_decompress lzo_compress libcrc32c dm_mirror dm_region_hash dm_log dm_crypt dm_mod dax tg3 realtek phylink mii libphy tpm cbc sha256_ssse3 sha256_generic encrypted_keys trusted
---[ end trace 0000000000000000 ]---
RIP: 0010:nfsd4_probe_callback_sync+0x1094/0x25b0 [nfsd]
Code: 89 de 48 c7 c7 e0 20 95 a0 e8 08 fc ff ff 41 89 c4 85 c0 0f 85 ce 2d 00 00 48 c7 c7 58 dd 95 a0 45 31 e4 e8 8e 7a 0b e1 eb 08 <0f> 0b 41 bc f4 ff ff ff 48 83 c4 08 44 89 e0 5b 41 5c 41 5d 41 5e
RSP: 0018:ffffc9000613bc98 EFLAGS: 00010282
RAX: 0000000000000049 RBX: ffff888105e06000 RCX: 0000000000000000
RDX: ffff88846fb21920 RSI: ffff88846fb1d980 RDI: ffff88846fb1d980
RBP: ffffc9000613bcc0 R08: 0000000000000000 R09: ffffffff824b56c8
R10: ffffffff8249d688 R11: 0000000000000003 R12: ffffffff82736400
R13: ffff888105e06000 R14: ffff888105e06000 R15: ffffc9000613bcd0
FS:  00007f1a404e5b28(0000) GS:ffff88846fb00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f1a40411000 CR3: 000000010d47a006 CR4: 0000000000370ef0

Here is /etc/exports:
/mnt/data/nfs        10.9.8.0/24(ro,fsid=root,no_subtree_check,insecure,async,all_squash,pnfs,anonuid=99,anongid=99)
/mnt/data/nfs/misc   10.9.8.0/24(ro,no_subtree_check,insecure,async,all_squash,pnfs,anonuid=99,anongid=99)


^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: kernel panic when starting nfsd on OpenWrt with kernel 6.12.19
  2025-03-22 10:05 kernel panic when starting nfsd on OpenWrt with kernel 6.12.19 John
@ 2025-03-22 14:49 ` Chuck Lever
  2025-03-22 17:20   ` John
  0 siblings, 1 reply; 6+ messages in thread
From: Chuck Lever @ 2025-03-22 14:49 UTC (permalink / raw)
  To: John; +Cc: linux-nfs@vger.kernel.org

On 3/22/25 6:05 AM, John wrote:
> Starting nfsd with kernel 6.12.19 triggers a kernel panic. Using the 6.6.83 kernel is rock solid. I confirmed this on two x86/64 machines (Intel-N150 based and AMD 8845H-based). Posting here seeking debug advice. It is not clear to me if this is something in the kernel code or in something else within the distro.
> 
> I get the following from dmesg:
> Kernel BUG at nfsd4_probe_callback_sync+0x1094/0x25b0 [nfsd] [verbose debug info unavailable]
> Oops: invalid opcode: 0000 [#1] SMP NOPTI
> CPU: 2 UID: 0 PID: 17091 Comm: rpc.nfsd Not tainted 6.12.19 #0
> Hardware name: iKOOLCORE R2Max/R2Max, BIOS v1.1 12/17/2024
> RIP: 0010:nfsd4_probe_callback_sync+0x1094/0x25b0 [nfsd]
> Code: 89 de 48 c7 c7 e0 20 95 a0 e8 08 fc ff ff 41 89 c4 85 c0 0f 85 ce 2d 00 00 48 c7 c7 58 dd 95 a0 45 31 e4 e8 8e 7a 0b e1 eb 08 <0f> 0b 41 bc f4 ff ff ff 48 83 c4 08 44 89 e0 5b 41 5c 41 5d 41 5e
> RSP: 0018:ffffc9000613bc98 EFLAGS: 00010282
> RAX: 0000000000000049 RBX: ffff888105e06000 RCX: 0000000000000000
> RDX: ffff88846fb21920 RSI: ffff88846fb1d980 RDI: ffff88846fb1d980
> RBP: ffffc9000613bcc0 R08: 0000000000000000 R09: ffffffff824b56c8
> R10: ffffffff8249d688 R11: 0000000000000003 R12: ffffffff82736400
> R13: ffff888105e06000 R14: ffff888105e06000 R15: ffffc9000613bcd0
> FS:  00007f1a404e5b28(0000) GS:ffff88846fb00000(0000) knlGS:0000000000000000
> CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> CR2: 00007f1a40411000 CR3: 000000010d47a006 CR4: 0000000000370ef0
> Call Trace:
>  <TASK>
>  ? show_regs.part.0+0x1d/0x20
>  ? __die+0x52/0x91
>  ? die+0x2a/0x50
>  ? do_trap+0x103/0x110
>  ? do_error_trap+0x6c/0x90
>  ? nfsd4_probe_callback_sync+0x1094/0x25b0 [nfsd]
>  ? exc_invalid_op+0x4f/0x70
>  ? nfsd4_probe_callback_sync+0x1094/0x25b0 [nfsd]
>  ? asm_exc_invalid_op+0x1b/0x20
>  ? nfsd4_probe_callback_sync+0x1094/0x25b0 [nfsd]
>  ? nfsd4_probe_callback_sync+0xff6/0x25b0 [nfsd]
>  nfsd4_client_tracking_init+0x39/0x150 [nfsd]
>  nfs4_state_start_net+0x2ce/0x370 [nfsd]
>  nfsd_svc+0x1a0/0x2d0 [nfsd]
>  nfssvc_encode_voidres+0x19a1/0x1be0 [nfsd]
>  ? simple_transaction_get+0xb7/0xe0
>  ? nfssvc_encode_voidres+0x18f0/0x1be0 [nfsd]
>  nfssvc_encode_voidres+0x1a3/0x1be0 [nfsd]
>  vfs_write+0xcb/0x390
>  ? putname+0x4c/0x60
>  ksys_write+0x57/0xd0
>  __x64_sys_write+0x14/0x20
>  x64_sys_call+0x79/0x1780
>  do_syscall_64+0x7b/0x190
>  entry_SYSCALL_64_after_hwframe+0x67/0x6f
> RIP: 0033:0x7f1a404c9659
> Code: c3 8b 07 85 c0 75 24 49 89 fb 48 89 f0 48 89 d7 48 89 ce 4c 89 c2 4d 89 ca 4c 8b 44 24 08 4c 8b 4c 24 10 4c 89 5c 24 08 0f 05 <c3> e9 82 d8 ff ff 41 54 b8 02 00 00 00 55 48 89 f5 be 00 88 08 00
> RSP: 002b:00007fff99aeb2c8 EFLAGS: 00000246 ORIG_RAX: 0000000000000001
> RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f1a404c9659
> RDX: 0000000000000003 RSI: 000000000040d360 RDI: 0000000000000003
> RBP: 00007f1a404e5b28 R08: 0000000000000000 R09: 0000000000000000
> R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000001
> R13: 0000000000020000 R14: 0000000000409112 R15: 00007f1a40430dd0
>  </TASK>
> Modules linked in: xt_connlimit pppoe ppp_async nf_conncount i915 xt_state xt_helper xt_conntrack xt_connmark xt_connlabel xt_connbytes xt_CT wireguard video pppox ppp_mppe ppp_generic nft_redir nft_nat nft_masq nft_flow_offload nft_fib_inet nft_ct nft_chain_nat nf_nat nf_flow_table_inet nf_flow_table nf_conntrack_netlink nf_conntrack libchacha20poly1305 ipt_REJECT iavf i40e curve25519_x86_64 chacha_x86_64 cdc_ncm cdc_ether aqc111 zstd xt_time xt_tcpudp xt_tcpmss xt_statistic xt_recent xt_quota xt_pkttype xt_owner xt_multiport xt_mark xt_mac xt_limit xt_length xt_hl xt_ecn xt_dscp xt_comment xt_cgroup xt_addrtype xt_TCPMSS xt_LOG xt_HL xt_DSCP xt_CLASSIFY xt_CHECKSUM wmi usbnet unix_diag ts_kmp ts_fsm ts_bm slhc sch_cake poly1305_x86_64 nft_reject_ipv6 nft_reject_ipv4 nft_reject_inet nft_reject nft_quota nft_queue nft_numgen nft_log nft_limit nft_hash nft_fib_ipv6 nft_fib_ipv4 nft_fib nft_compat nfnetlink_queue nf_tables nf_reject_ipv4 nf_log_syslog nf_defrag_ipv6 nf_defrag_ipv4 macvlan lzo_rle lzo libie
>  libeth libcurve25519_generic libchacha iptable_mangle iptable_filter ipt_ECN ip_tables forcedeth e1000e drm_display_helper drm_buddy crc_ccitt cdc_acm bnx2 atlantic af_packet_diag sch_teql sch_sfq sch_multiq sch_gred sch_fq sch_ets sch_codel em_text em_nbyte em_meta em_cmp act_skbmod act_simple act_pedit act_csum sch_tbf sch_ingress sch_htb sch_hfsc em_u32 cls_u32 cls_route cls_matchall cls_fw cls_flow cls_basic act_skbedit act_mirred act_gact configs evdev drivetemp i2c_piix4 i2c_smbus i2c_dev xt_set ip_set_list_set ip_set_hash_netportnet ip_set_hash_netport ip_set_hash_netnet ip_set_hash_netiface ip_set_hash_net ip_set_hash_mac ip_set_hash_ipportnet ip_set_hash_ipportip ip_set_hash_ipport ip_set_hash_ipmark ip_set_hash_ipmac ip_set_hash_ip ip_set_bitmap_port ip_set_bitmap_ipmac ip_set_bitmap_ip ip_set nfnetlink dwmac_intel ip6t_rt ip6t_mh ip6t_ipv6header ip6t_hbh ip6t_frag ip6t_eui64 ip6t_ah ip6table_mangle ip6table_filter ip6_tables ip6t_REJECT x_tables nf_reject_ipv6 nfsv4 dwmac_generic
>  stmmac_platform stmmac nfsd nfs bonding ixgbe igbvf e1000 amd_xgbe ifb sctp udp_tunnel ip6_udp_tunnel mdio netlink_diag udp_diag tcp_diag raw_diag inet_diag rpcsec_gss_krb5 auth_rpcgss oid_registry veth lockd sunrpc grace dns_resolver nls_utf8 pcs_xpcs macsec ena ecdh_generic ecc xxhash_generic crypto_user algif_skcipher algif_rng algif_hash algif_aead af_alg sha512_ssse3 sha512_generic sha1_ssse3 sha1_generic seqiv sha3_generic jitterentropy_rng drbg md5 kpp hmac geniv rng des_generic libdes cts chacha20poly1305 blake2b_generic authencesn authenc arc4 crypto_acompress nls_iso8859_1 nls_cp437 r8169 mdio_devres xhci_plat_hcd iTCO_wdt iTCO_vendor_support fsl_mph_dr_of ehci_platform ehci_fsl igb igc vfat fat btrfs zstd_decompress zstd_compress zstd_common xxhash xor raid6_pq lzo_decompress lzo_compress libcrc32c dm_mirror dm_region_hash dm_log dm_crypt dm_mod dax tg3 realtek phylink mii libphy tpm cbc sha256_ssse3 sha256_generic encrypted_keys trusted
> ---[ end trace 0000000000000000 ]---
> RIP: 0010:nfsd4_probe_callback_sync+0x1094/0x25b0 [nfsd]
> Code: 89 de 48 c7 c7 e0 20 95 a0 e8 08 fc ff ff 41 89 c4 85 c0 0f 85 ce 2d 00 00 48 c7 c7 58 dd 95 a0 45 31 e4 e8 8e 7a 0b e1 eb 08 <0f> 0b 41 bc f4 ff ff ff 48 83 c4 08 44 89 e0 5b 41 5c 41 5d 41 5e
> RSP: 0018:ffffc9000613bc98 EFLAGS: 00010282
> RAX: 0000000000000049 RBX: ffff888105e06000 RCX: 0000000000000000
> RDX: ffff88846fb21920 RSI: ffff88846fb1d980 RDI: ffff88846fb1d980
> RBP: ffffc9000613bcc0 R08: 0000000000000000 R09: ffffffff824b56c8
> R10: ffffffff8249d688 R11: 0000000000000003 R12: ffffffff82736400
> R13: ffff888105e06000 R14: ffff888105e06000 R15: ffffc9000613bcd0
> FS:  00007f1a404e5b28(0000) GS:ffff88846fb00000(0000) knlGS:0000000000000000
> CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> CR2: 00007f1a40411000 CR3: 000000010d47a006 CR4: 0000000000370ef0
> 
> Here is /etc/exports:
> /mnt/data/nfs        10.9.8.0/24(ro,fsid=root,no_subtree_check,insecure,async,all_squash,pnfs,anonuid=99,anongid=99)
> /mnt/data/nfs/misc   10.9.8.0/24(ro,no_subtree_check,insecure,async,all_squash,pnfs,anonuid=99,anongid=99)
> 
> 

I recommend bisecting between 6.6 and 6.12.19 to locate the culprit.

When you have that information, open a bug on bugzilla.kernel.org under
Filesystems/NFSD and attach your information there.


-- 
Chuck Lever

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: kernel panic when starting nfsd on OpenWrt with kernel 6.12.19
  2025-03-22 14:49 ` Chuck Lever
@ 2025-03-22 17:20   ` John
  2025-03-22 17:33     ` Chuck Lever
  2025-03-22 18:06     ` Chuck Lever
  0 siblings, 2 replies; 6+ messages in thread
From: John @ 2025-03-22 17:20 UTC (permalink / raw)
  To: Chuck Lever; +Cc: linux-nfs@vger.kernel.org



On Saturday, March 22nd, 2025 at 10:49 AM, Chuck Lever <chuck.lever@oracle.com> wrote:
> 
> I recommend bisecting between 6.6 and 6.12.19 to locate the culprit.
> 
> When you have that information, open a bug on bugzilla.kernel.org under
> Filesystems/NFSD and attach your information there.

Wish I could, but the way OpenWrt is designed, moving from one kernel branch to another is major undertaking often requiring months of tweaks 1,000s of patches, Makefiles, config files, etc.

I have been reading up on NFSv4 and am wondering if there is a way to simply disable the callback function. Could that work-around this crash?

Unfortunately, OpenWrt's nfs-kernel-server package does not ship nor use something like /etc/nfs.conf. At the link below you can see their init script to start nfsd. Many thank for any ideas.

https://github.com/openwrt/packages/blob/master/net/nfs-kernel-server/files/nfsd.init

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: kernel panic when starting nfsd on OpenWrt with kernel 6.12.19
  2025-03-22 17:20   ` John
@ 2025-03-22 17:33     ` Chuck Lever
  2025-03-22 18:06     ` Chuck Lever
  1 sibling, 0 replies; 6+ messages in thread
From: Chuck Lever @ 2025-03-22 17:33 UTC (permalink / raw)
  To: John; +Cc: linux-nfs@vger.kernel.org

On 3/22/25 1:20 PM, John wrote:
> 
> 
> On Saturday, March 22nd, 2025 at 10:49 AM, Chuck Lever <chuck.lever@oracle.com> wrote:
>>
>> I recommend bisecting between 6.6 and 6.12.19 to locate the culprit.
>>
>> When you have that information, open a bug on bugzilla.kernel.org under
>> Filesystems/NFSD and attach your information there.
> 
> Wish I could, but the way OpenWrt is designed, moving from one kernel branch to another is major undertaking often requiring months of tweaks 1,000s of patches, Makefiles, config files, etc.
> 
> I have been reading up on NFSv4 and am wondering if there is a way to simply disable the callback function. Could that work-around this crash?
> 
> Unfortunately, OpenWrt's nfs-kernel-server package does not ship nor use something like /etc/nfs.conf. At the link below you can see their init script to start nfsd. Many thank for any ideas.
> 
> https://github.com/openwrt/packages/blob/master/net/nfs-kernel-server/files/nfsd.init

Open a bug anyway, please. I have a nightly CI for NFSD that has not
crashed on v6.12 LTS, so this is a new and unseen issue.

Thanks for the report!


-- 
Chuck Lever

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: kernel panic when starting nfsd on OpenWrt with kernel 6.12.19
  2025-03-22 17:20   ` John
  2025-03-22 17:33     ` Chuck Lever
@ 2025-03-22 18:06     ` Chuck Lever
  2025-03-22 18:52       ` John
  1 sibling, 1 reply; 6+ messages in thread
From: Chuck Lever @ 2025-03-22 18:06 UTC (permalink / raw)
  To: John; +Cc: linux-nfs@vger.kernel.org

On 3/22/25 1:20 PM, John wrote:
> 
> 
> On Saturday, March 22nd, 2025 at 10:49 AM, Chuck Lever <chuck.lever@oracle.com> wrote:
>>
>> I recommend bisecting between 6.6 and 6.12.19 to locate the culprit.
>>
>> When you have that information, open a bug on bugzilla.kernel.org under
>> Filesystems/NFSD and attach your information there.
> 
> Wish I could, but the way OpenWrt is designed, moving from one kernel branch to another is major undertaking often requiring months of tweaks 1,000s of patches, Makefiles, config files, etc.
> 
> I have been reading up on NFSv4 and am wondering if there is a way to simply disable the callback function. Could that work-around this crash?
> 
> Unfortunately, OpenWrt's nfs-kernel-server package does not ship nor use something like /etc/nfs.conf. At the link below you can see their init script to start nfsd. Many thank for any ideas.
> 
> https://github.com/openwrt/packages/blob/master/net/nfs-kernel-server/files/nfsd.init
> 

If this is a "product" kernel then you should approach the distributor
first. I do not have the hardware, the binaries, the .config, or a
cross compiler for whatever "Hardware name: iKOOLCORE R2Max/R2Max" is
(guessing RPi ?)

It would be helpful to get

$ scripts/faddr2line fs/nfsd/nfs4callback.c \
nfsd4_probe_callback_sync+0x1094

although, IME, +1094 means the crash happened far outside of the
nfs4callback.c source file.

This:

  nfsd4_client_tracking_init+0x39/0x150 [nfsd]

suggests that your distributor (or LTS) might be missing a fix
for recent crashes that are due to incorrectly setting
CONFIG_NFSD_LEGACY_CLIENT_TRACKING. I'm thinking of perhaps

  de71d4e211ed ("nfsd: fix legacy client tracking initialization")

in particular, which does not appear to have been applied to
v6.12.19.


-- 
Chuck Lever

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: kernel panic when starting nfsd on OpenWrt with kernel 6.12.19
  2025-03-22 18:06     ` Chuck Lever
@ 2025-03-22 18:52       ` John
  0 siblings, 0 replies; 6+ messages in thread
From: John @ 2025-03-22 18:52 UTC (permalink / raw)
  To: Chuck Lever; +Cc: linux-nfs@vger.kernel.org


On Saturday, March 22nd, 2025 at 2:06 PM, Chuck Lever <chuck.lever@oracle.com> wrote:

> If this is a "product" kernel then you should approach the distributor
> first. I do not have the hardware, the binaries, the .config, or a
> cross compiler for whatever "Hardware name: iKOOLCORE R2Max/R2Max" is
> (guessing RPi ?)

No, this is the vanilla kernel + OpenWrt project patches. No vendor involved. Also, both of the machines that are experiencing this bug are x86/64, not ARM.

> This:
> 
> nfsd4_client_tracking_init+0x39/0x150 [nfsd]
> 
> suggests that your distributor (or LTS) might be missing a fix
> for recent crashes that are due to incorrectly setting
> CONFIG_NFSD_LEGACY_CLIENT_TRACKING. I'm thinking of perhaps
> 
> de71d4e211ed ("nfsd: fix legacy client tracking initialization")
> 
> in particular, which does not appear to have been applied to
> v6.12.19.

I applied this commit to my image build and found no bug upon starting nfsd with 6.12.19. I will comment the bug report[1] with this info. Many thanks for pointing this out, Chuck!

1. https://bugzilla.kernel.org/show_bug.cgi?id=219911

^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2025-03-22 18:58 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-03-22 10:05 kernel panic when starting nfsd on OpenWrt with kernel 6.12.19 John
2025-03-22 14:49 ` Chuck Lever
2025-03-22 17:20   ` John
2025-03-22 17:33     ` Chuck Lever
2025-03-22 18:06     ` Chuck Lever
2025-03-22 18:52       ` John

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.