* [Bug report] xfstests cause nfs crash, kernel BUG at kernel/cred.c:104!
@ 2025-05-08 5:18 Zorro Lang
2025-06-04 17:18 ` Mike Snitzer
0 siblings, 1 reply; 7+ messages in thread
From: Zorro Lang @ 2025-05-08 5:18 UTC (permalink / raw)
To: linux-nfs
Hi,
Recently I always hit nfs crash[1] on latest mainline upstream linux 6.15.0-rc5+,
which HEAD=707df3375124b51048233625a7e1c801e8c8a7fd . I've hit this bug on aarch64
and x86_64 several times, by running xfstests on nfsv4.2 [2].
Thanks,
Zorro
[1]
[ 1004.936621] run fstests generic/023 at 2025-05-07 11:06:07
[ 1005.840825] ------------[ cut here ]------------
[ 1005.845547] kernel BUG at kernel/cred.c:104!
[ 1005.849848] Oops: invalid opcode: 0000 [#1] SMP KASAN NOPTI
[ 1005.855440] CPU: 12 UID: 0 PID: 78471 Comm: kworker/u128:20 Not tainted 6.15.0-rc5+ #1 PREEMPT(voluntary)
[ 1005.865104] Hardware name: Dell Inc. PowerEdge R660/0R5JJC, BIOS 2.1.5 03/14/2024
[ 1005.872583] Workqueue: nfslocaliod nfs_local_call_write [nfs]
[ 1005.878416] RIP: 0010:__put_cred+0xea/0x120
[ 1005.882618] Code: 2d 8b 83 a8 00 00 00 85 c0 74 0b 48 83 c4 08 5b 5d e9 fa fb ff ff 48 83 c4 08 48 c7 c6 c0 af da 83 5b 5d e9 28 54 1d 00 0f 0b <0f> 0b 0f 0b 48 89 3c 24 e8 09 dc 99 00 48 8b 3c 24 eb c4 48 89 df
[ 1005.901383] RSP: 0018:ffa00000247a78d8 EFLAGS: 00010246
[ 1005.906626] RAX: dffffc0000000000 RBX: ff110001428ea000 RCX: ffffffff83dab30c
[ 1005.913757] RDX: 1fe220004203d19a RSI: 0000000000000008 RDI: ff110002101e8cd0
[ 1005.920892] RBP: ff110002101e8000 R08: 0000000000000000 R09: ffe21c002851d400
[ 1005.928024] R10: ff110001428ea007 R11: 0000000000000001 R12: 0000000000000000
[ 1005.935157] R13: ff110001f6c1ce88 R14: ff110001f6c1ce68 R15: ff110001f6c1ce00
[ 1005.942289] FS: 0000000000000000(0000) GS:ff110008938c7000(0000) knlGS:0000000000000000
[ 1005.950376] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 1005.956122] CR2: 00007f619475fdc0 CR3: 00000003f6c76004 CR4: 0000000000f73ef0
[ 1005.963256] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 1005.970388] DR3: 0000000000000000 DR6: 00000000fffe07f0 DR7: 0000000000000400
[ 1005.977520] PKRU: 55555554
[ 1005.980234] Call Trace:
[ 1005.982685] <TASK>
[ 1005.984794] __fput+0x61e/0xaa0
[ 1005.987958] nfsd_file_free+0x1a1/0x3e0 [nfsd]
[ 1005.992505] nfsd_file_put_local+0x35/0x50 [nfsd]
[ 1005.997290] nfs_close_local_fh+0x362/0x6f0 [nfs_localio]
[ 1006.002709] __put_nfs_open_context+0x380/0x500 [nfs]
[ 1006.007813] ? nfs_write_completion+0x111/0x8a0 [nfs]
[ 1006.012926] nfs_pgio_header_free+0x41/0xe0 [nfs]
[ 1006.017693] nfs_write_completion+0x139/0x8a0 [nfs]
[ 1006.022631] ? __pfx_nfs_writeback_done.part.0+0x10/0x10 [nfs]
[ 1006.028517] ? __pfx_nfs_write_completion+0x10/0x10 [nfs]
[ 1006.033981] nfs_local_call_write+0x4fc/0xa50 [nfs]
[ 1006.038918] ? __pfx_nfs_local_call_write+0x10/0x10 [nfs]
[ 1006.044376] ? rcu_is_watching+0x15/0xb0
[ 1006.048320] ? rcu_is_watching+0x15/0xb0
[ 1006.052246] process_one_work+0xd6e/0x1330
[ 1006.056349] ? __pfx_process_one_work+0x10/0x10
[ 1006.060897] ? assign_work+0x16c/0x240
[ 1006.064668] worker_thread+0x5f3/0xfe0
[ 1006.068437] ? __kthread_parkme+0xb4/0x200
[ 1006.072553] ? __pfx_worker_thread+0x10/0x10
[ 1006.076842] kthread+0x3b1/0x770
[ 1006.080077] ? local_clock_noinstr+0xd/0xe0
[ 1006.084278] ? __pfx_kthread+0x10/0x10
[ 1006.088032] ? __lock_release.isra.0+0x1a4/0x2c0
[ 1006.092667] ? rcu_is_watching+0x15/0xb0
[ 1006.096611] ? __pfx_kthread+0x10/0x10
[ 1006.100382] ret_from_fork+0x31/0x70
[ 1006.103978] ? __pfx_kthread+0x10/0x10
[ 1006.107738] ret_from_fork_asm+0x1a/0x30
[ 1006.111687] </TASK>
[ 1006.113894] Modules linked in: rpcsec_gss_krb5 nfsv4 dns_resolver nfs netfs rpcrdma rdma_cm iw_cm ib_cm ib_core nfsd auth_rpcgss nfs_acl lockd grace nfs_localio rfkill intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_ifs sunrpc i10nm_edac skx_edac_common nfit x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm dax_hmem irqbypass cxl_acpi iTCO_wdt igb mgag200 cxl_port rapl dell_pc pmt_telemetry iTCO_vendor_support pmt_class cxl_core intel_sdsi dell_smbios platform_profile intel_cstate ipmi_ssif ses isst_if_mmio dcdbas wmi_bmof dell_wmi_descriptor pcspkr intel_uncore enclosure einj isst_if_mbox_pci mei_me isst_if_common intel_vsec i2c_i801 i2c_algo_bit tg3 dca mei i2c_smbus i2c_ismt acpi_power_meter ipmi_si acpi_ipmi ipmi_devintf ipmi_msghandler sg fuse loop dm_mod nfnetlink xfs sd_mod iaa_crypto mpt3sas ahci raid_class ghash_clmulni_intel libahci scsi_transport_sas idxd libata idxd_bus wmi pinctrl_emmitsburg
[ 1006.198764] ---[ end trace 0000000000000000 ]---
[ 1006.223531] RIP: 0010:__put_cred+0xea/0x120
[ 1006.227744] Code: 2d 8b 83 a8 00 00 00 85 c0 74 0b 48 83 c4 08 5b 5d e9 fa fb ff ff 48 83 c4 08 48 c7 c6 c0 af da 83 5b 5d e9 28 54 1d 00 0f 0b <0f> 0b 0f 0b 48 89 3c 24 e8 09 dc 99 00 48 8b 3c 24 eb c4 48 89 df
[ 1006.246509] RSP: 0018:ffa00000247a78d8 EFLAGS: 00010246
[ 1006.251762] RAX: dffffc0000000000 RBX: ff110001428ea000 RCX: ffffffff83dab30c
[ 1006.258911] RDX: 1fe220004203d19a RSI: 0000000000000008 RDI: ff110002101e8cd0
[ 1006.266065] RBP: ff110002101e8000 R08: 0000000000000000 R09: ffe21c002851d400
[ 1006.273218] R10: ff110001428ea007 R11: 0000000000000001 R12: 0000000000000000
[ 1006.280368] R13: ff110001f6c1ce88 R14: ff110001f6c1ce68 R15: ff110001f6c1ce00
[ 1006.287521] FS: 0000000000000000(0000) GS:ff110008938c7000(0000) knlGS:0000000000000000
[ 1006.295623] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 1006.301387] CR2: 00007f619475fdc0 CR3: 00000003f6c76004 CR4: 0000000000f73ef0
[ 1006.308535] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 1006.315687] DR3: 0000000000000000 DR6: 00000000fffe07f0 DR7: 0000000000000400
[ 1006.322836] PKRU: 55555554
[ 1006.325569] Kernel panic - not syncing: Fatal exception
[ 1006.330842] Kernel Offset: 0x2800000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)
[ 1006.361714] ---[ end Kernel panic - not syncing: Fatal exception ]---
[-- MARK -- Wed May 7 15:10:00 2025]
[-- MARK -- Wed May 7 15:15:00 2025]
[2]
export FSTYP=nfs
export TEST_DEV=xxxxxxx.xxxxxxxxxx.xxxxxxxxxxxx:/mnt/xfstests/test/nfs-server
export TEST_DIR=/mnt/xfstests/test/nfs-client
export SCRATCH_DEV=xxxxxxx.xxxxxxxxxx.xxxxxxxxxxxx:/mnt/xfstests/scratch/nfs-server
export SCRATCH_MNT=/mnt/xfstests/scratch/nfs-client
export MOUNT_OPTIONS="-o vers=4.2"
export TEST_FS_MOUNT_OPTS="-o vers=4.2"
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [Bug report] xfstests cause nfs crash, kernel BUG at kernel/cred.c:104!
2025-05-08 5:18 [Bug report] xfstests cause nfs crash, kernel BUG at kernel/cred.c:104! Zorro Lang
@ 2025-06-04 17:18 ` Mike Snitzer
2025-06-06 2:38 ` Zorro Lang
2025-08-05 22:34 ` Scott Mayhew
0 siblings, 2 replies; 7+ messages in thread
From: Mike Snitzer @ 2025-06-04 17:18 UTC (permalink / raw)
To: Zorro Lang; +Cc: linux-nfs
First I'm seeing this, I haven't hit it myself.
Can you elaborate on how you've configured xfstest? I'll take a look
at generic/023 to understand how NFS is being used (but for it to be
hitting this with LOCALIO it must be doing loopback mount testing?)
neilbrown's localio patchset landed upstream yesterday via anna's 6.16
pull request, so it'd be useful to know if linus/master still has
this. v6.15 should get neilb's localio patchset because it was marked
for stable@ backport.
Thanks,
Mike
On Thu, May 08, 2025 at 01:18:29PM +0800, Zorro Lang wrote:
> Hi,
>
> Recently I always hit nfs crash[1] on latest mainline upstream linux 6.15.0-rc5+,
> which HEAD=707df3375124b51048233625a7e1c801e8c8a7fd . I've hit this bug on aarch64
> and x86_64 several times, by running xfstests on nfsv4.2 [2].
>
> Thanks,
> Zorro
>
> [1]
> [ 1004.936621] run fstests generic/023 at 2025-05-07 11:06:07
> [ 1005.840825] ------------[ cut here ]------------
> [ 1005.845547] kernel BUG at kernel/cred.c:104!
> [ 1005.849848] Oops: invalid opcode: 0000 [#1] SMP KASAN NOPTI
> [ 1005.855440] CPU: 12 UID: 0 PID: 78471 Comm: kworker/u128:20 Not tainted 6.15.0-rc5+ #1 PREEMPT(voluntary)
> [ 1005.865104] Hardware name: Dell Inc. PowerEdge R660/0R5JJC, BIOS 2.1.5 03/14/2024
> [ 1005.872583] Workqueue: nfslocaliod nfs_local_call_write [nfs]
> [ 1005.878416] RIP: 0010:__put_cred+0xea/0x120
> [ 1005.882618] Code: 2d 8b 83 a8 00 00 00 85 c0 74 0b 48 83 c4 08 5b 5d e9 fa fb ff ff 48 83 c4 08 48 c7 c6 c0 af da 83 5b 5d e9 28 54 1d 00 0f 0b <0f> 0b 0f 0b 48 89 3c 24 e8 09 dc 99 00 48 8b 3c 24 eb c4 48 89 df
> [ 1005.901383] RSP: 0018:ffa00000247a78d8 EFLAGS: 00010246
> [ 1005.906626] RAX: dffffc0000000000 RBX: ff110001428ea000 RCX: ffffffff83dab30c
> [ 1005.913757] RDX: 1fe220004203d19a RSI: 0000000000000008 RDI: ff110002101e8cd0
> [ 1005.920892] RBP: ff110002101e8000 R08: 0000000000000000 R09: ffe21c002851d400
> [ 1005.928024] R10: ff110001428ea007 R11: 0000000000000001 R12: 0000000000000000
> [ 1005.935157] R13: ff110001f6c1ce88 R14: ff110001f6c1ce68 R15: ff110001f6c1ce00
> [ 1005.942289] FS: 0000000000000000(0000) GS:ff110008938c7000(0000) knlGS:0000000000000000
> [ 1005.950376] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 1005.956122] CR2: 00007f619475fdc0 CR3: 00000003f6c76004 CR4: 0000000000f73ef0
> [ 1005.963256] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 1005.970388] DR3: 0000000000000000 DR6: 00000000fffe07f0 DR7: 0000000000000400
> [ 1005.977520] PKRU: 55555554
> [ 1005.980234] Call Trace:
> [ 1005.982685] <TASK>
> [ 1005.984794] __fput+0x61e/0xaa0
> [ 1005.987958] nfsd_file_free+0x1a1/0x3e0 [nfsd]
> [ 1005.992505] nfsd_file_put_local+0x35/0x50 [nfsd]
> [ 1005.997290] nfs_close_local_fh+0x362/0x6f0 [nfs_localio]
> [ 1006.002709] __put_nfs_open_context+0x380/0x500 [nfs]
> [ 1006.007813] ? nfs_write_completion+0x111/0x8a0 [nfs]
> [ 1006.012926] nfs_pgio_header_free+0x41/0xe0 [nfs]
> [ 1006.017693] nfs_write_completion+0x139/0x8a0 [nfs]
> [ 1006.022631] ? __pfx_nfs_writeback_done.part.0+0x10/0x10 [nfs]
> [ 1006.028517] ? __pfx_nfs_write_completion+0x10/0x10 [nfs]
> [ 1006.033981] nfs_local_call_write+0x4fc/0xa50 [nfs]
> [ 1006.038918] ? __pfx_nfs_local_call_write+0x10/0x10 [nfs]
> [ 1006.044376] ? rcu_is_watching+0x15/0xb0
> [ 1006.048320] ? rcu_is_watching+0x15/0xb0
> [ 1006.052246] process_one_work+0xd6e/0x1330
> [ 1006.056349] ? __pfx_process_one_work+0x10/0x10
> [ 1006.060897] ? assign_work+0x16c/0x240
> [ 1006.064668] worker_thread+0x5f3/0xfe0
> [ 1006.068437] ? __kthread_parkme+0xb4/0x200
> [ 1006.072553] ? __pfx_worker_thread+0x10/0x10
> [ 1006.076842] kthread+0x3b1/0x770
> [ 1006.080077] ? local_clock_noinstr+0xd/0xe0
> [ 1006.084278] ? __pfx_kthread+0x10/0x10
> [ 1006.088032] ? __lock_release.isra.0+0x1a4/0x2c0
> [ 1006.092667] ? rcu_is_watching+0x15/0xb0
> [ 1006.096611] ? __pfx_kthread+0x10/0x10
> [ 1006.100382] ret_from_fork+0x31/0x70
> [ 1006.103978] ? __pfx_kthread+0x10/0x10
> [ 1006.107738] ret_from_fork_asm+0x1a/0x30
> [ 1006.111687] </TASK>
> [ 1006.113894] Modules linked in: rpcsec_gss_krb5 nfsv4 dns_resolver nfs netfs rpcrdma rdma_cm iw_cm ib_cm ib_core nfsd auth_rpcgss nfs_acl lockd grace nfs_localio rfkill intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_ifs sunrpc i10nm_edac skx_edac_common nfit x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm dax_hmem irqbypass cxl_acpi iTCO_wdt igb mgag200 cxl_port rapl dell_pc pmt_telemetry iTCO_vendor_support pmt_class cxl_core intel_sdsi dell_smbios platform_profile intel_cstate ipmi_ssif ses isst_if_mmio dcdbas wmi_bmof dell_wmi_descriptor pcspkr intel_uncore enclosure einj isst_if_mbox_pci mei_me isst_if_common intel_vsec i2c_i801 i2c_algo_bit tg3 dca mei i2c_smbus i2c_ismt acpi_power_meter ipmi_si acpi_ipmi ipmi_devintf ipmi_msghandler sg fuse loop dm_mod nfnetlink xfs sd_mod iaa_crypto mpt3sas ahci raid_class ghash_clmulni_intel libahci scsi_transport_sas idxd libata idxd_bus wmi pinctrl_emmitsburg
> [ 1006.198764] ---[ end trace 0000000000000000 ]---
> [ 1006.223531] RIP: 0010:__put_cred+0xea/0x120
> [ 1006.227744] Code: 2d 8b 83 a8 00 00 00 85 c0 74 0b 48 83 c4 08 5b 5d e9 fa fb ff ff 48 83 c4 08 48 c7 c6 c0 af da 83 5b 5d e9 28 54 1d 00 0f 0b <0f> 0b 0f 0b 48 89 3c 24 e8 09 dc 99 00 48 8b 3c 24 eb c4 48 89 df
> [ 1006.246509] RSP: 0018:ffa00000247a78d8 EFLAGS: 00010246
> [ 1006.251762] RAX: dffffc0000000000 RBX: ff110001428ea000 RCX: ffffffff83dab30c
> [ 1006.258911] RDX: 1fe220004203d19a RSI: 0000000000000008 RDI: ff110002101e8cd0
> [ 1006.266065] RBP: ff110002101e8000 R08: 0000000000000000 R09: ffe21c002851d400
> [ 1006.273218] R10: ff110001428ea007 R11: 0000000000000001 R12: 0000000000000000
> [ 1006.280368] R13: ff110001f6c1ce88 R14: ff110001f6c1ce68 R15: ff110001f6c1ce00
> [ 1006.287521] FS: 0000000000000000(0000) GS:ff110008938c7000(0000) knlGS:0000000000000000
> [ 1006.295623] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 1006.301387] CR2: 00007f619475fdc0 CR3: 00000003f6c76004 CR4: 0000000000f73ef0
> [ 1006.308535] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 1006.315687] DR3: 0000000000000000 DR6: 00000000fffe07f0 DR7: 0000000000000400
> [ 1006.322836] PKRU: 55555554
> [ 1006.325569] Kernel panic - not syncing: Fatal exception
> [ 1006.330842] Kernel Offset: 0x2800000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)
> [ 1006.361714] ---[ end Kernel panic - not syncing: Fatal exception ]---
> [-- MARK -- Wed May 7 15:10:00 2025]
> [-- MARK -- Wed May 7 15:15:00 2025]
>
> [2]
> export FSTYP=nfs
> export TEST_DEV=xxxxxxx.xxxxxxxxxx.xxxxxxxxxxxx:/mnt/xfstests/test/nfs-server
> export TEST_DIR=/mnt/xfstests/test/nfs-client
> export SCRATCH_DEV=xxxxxxx.xxxxxxxxxx.xxxxxxxxxxxx:/mnt/xfstests/scratch/nfs-server
> export SCRATCH_MNT=/mnt/xfstests/scratch/nfs-client
> export MOUNT_OPTIONS="-o vers=4.2"
> export TEST_FS_MOUNT_OPTS="-o vers=4.2"
>
>
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [Bug report] xfstests cause nfs crash, kernel BUG at kernel/cred.c:104!
2025-06-04 17:18 ` Mike Snitzer
@ 2025-06-06 2:38 ` Zorro Lang
2025-08-05 22:34 ` Scott Mayhew
1 sibling, 0 replies; 7+ messages in thread
From: Zorro Lang @ 2025-06-06 2:38 UTC (permalink / raw)
To: Mike Snitzer; +Cc: linux-nfs
On Wed, Jun 04, 2025 at 01:18:56PM -0400, Mike Snitzer wrote:
> First I'm seeing this, I haven't hit it myself.
>
> Can you elaborate on how you've configured xfstest? I'll take a look
> at generic/023 to understand how NFS is being used (but for it to be
> hitting this with LOCALIO it must be doing loopback mount testing?)
>
> neilbrown's localio patchset landed upstream yesterday via anna's 6.16
> pull request, so it'd be useful to know if linus/master still has
> this. v6.15 should get neilb's localio patchset because it was marked
> for stable@ backport.
I just hit this bug[1] on latest upstream linux again
(HEAD=e271ed52b344ac02d4581286961d0c40acc54c03), so the bug is still there.
My underlying test device isn't loopback device, it's general disk, and make
xfs on it:
meta-data=/dev/sda5 isize=512 agcount=4, agsize=983040 blks
= sectsz=4096 attr=2, projid32bit=1
= crc=1 finobt=1, sparse=1, rmapbt=1
= reflink=1 bigtime=1 inobtcount=1 nrext64=1
= exchange=0 metadir=0
data = bsize=4096 blocks=3932160, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0, ftype=1, parent=0
log =internal log bsize=4096 blocks=54324, version=2
= sectsz=4096 sunit=1 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
= rgcount=0 rgsize=0 extents
= zoned=0 start=0 reserved=0
Two xfs are mounted on /mnt/xfstests/test and /mnt/xfstests/scratch seperately,
then export as:
# cat /etc/exports
/mnt/xfstests/test/nfs-server *(rw,insecure,no_root_squash)
/mnt/xfstests/scratch/nfs-server *(rw,insecure,no_root_squash)
The nfs mount option is only "-o vers=4.2". BTW, nfs server and client are
in same machine/system.
# cat local.config
export FSTYP=nfs
export TEST_DEV=xxxx-xxx-xxxx.xxxx.xxx.xxx:/mnt/xfstests/test/nfs-server
export TEST_DIR=/mnt/xfstests/test/nfs-client
export SCRATCH_DEV=xxxx-xxx-xxxx.xxxx.xxx.xxx:/mnt/xfstests/scratch/nfs-server
export SCRATCH_MNT=/mnt/xfstests/scratch/nfs-client
export MOUNT_OPTIONS="-o vers=4.2"
export TEST_FS_MOUNT_OPTS="-o vers=4.2"
Thanks,
Zorro
[1]
[ 964.504249] run fstests generic/023 at 2025-06-05 21:35:27
[ 965.376924] ------------[ cut here ]------------
[ 965.381641] kernel BUG at kernel/cred.c:104!
[ 965.385945] Oops: invalid opcode: 0000 [#1] SMP KASAN NOPTI
[ 965.391534] CPU: 24 UID: 0 PID: 222 Comm: kworker/u128:14 Tainted: G S 6.15.0+ #1 PREEMPT(voluntary)
[ 965.402237] Tainted: [S]=CPU_OUT_OF_SPEC
[ 965.406169] Hardware name: Dell Inc. PowerEdge R660/0R5JJC, BIOS 2.4.4 09/27/2024
[ 965.413648] Workqueue: nfslocaliod nfs_local_call_write [nfs]
[ 965.419480] RIP: 0010:__put_cred+0xea/0x120
[ 965.423684] Code: 2d 8b 83 a8 00 00 00 85 c0 74 0b 48 83 c4 08 5b 5d e9 fa fb ff ff 48 83 c4 08 48 c7 c6 80 b8 78 92 5b 5d e9 f8 b0 1d 00 0f 0b <0f> 0b 0f 0b 48 89 3c 24 e8 09 17 9b 00 48 8b 3c 24 eb c4 48 89 df
[ 965.442448] RSP: 0018:ffa000000156f790 EFLAGS: 00010246
[ 965.447693] RAX: dffffc0000000000 RBX: ff110001cb191b00 RCX: ffffffff9278bbcc
[ 965.454825] RDX: 1fe2200020620972 RSI: 0000000000000008 RDI: ff11000103104b90
[ 965.461958] RBP: ff11000103103ec0 R08: 0000000000000000 R09: ffe21c0039632360
[ 965.469089] R10: ff110001cb191b07 R11: 0000000000000001 R12: 0000000000000000
[ 965.476224] R13: ff11000170651208 R14: ff110001706511e8 R15: ff11000170651180
[ 965.483357] FS: 0000000000000000(0000) GS:ff11000887ef3000(0000) knlGS:0000000000000000
[ 965.491458] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 965.497205] CR2: 000056321a5700b8 CR3: 00000004a5c7a006 CR4: 0000000000f73ef0
[ 965.504339] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 965.511469] DR3: 0000000000000000 DR6: 00000000fffe07f0 DR7: 0000000000000400
[ 965.518603] PKRU: 55555554
[ 965.521315] Call Trace:
[ 965.523770] <TASK>
[ 965.525876] __fput+0x61e/0xaa0
[ 965.529042] nfsd_file_free+0x1a1/0x3e0 [nfsd]
[ 965.533580] nfsd_file_put_local+0x4c/0x60 [nfsd]
[ 965.538357] nfs_close_local_fh+0x278/0x9a0 [nfs_localio]
[ 965.543773] ? __pfx_nfs_close_local_fh+0x10/0x10 [nfs_localio]
[ 965.549708] ? rcu_is_watching+0x15/0xb0
[ 965.553651] ? kfree+0x286/0x4b0
[ 965.556902] __put_nfs_open_context+0x380/0x500 [nfs]
[ 965.562016] ? nfs_write_completion+0x111/0x8a0 [nfs]
[ 965.567131] nfs_pgio_header_free+0x41/0xe0 [nfs]
[ 965.571897] nfs_write_completion+0x139/0x8a0 [nfs]
[ 965.576826] ? rcu_is_watching+0x15/0xb0
[ 965.580770] ? nfs_writeback_done.part.0+0x3dc/0x5e0 [nfs]
[ 965.586308] ? __pfx_nfs_write_completion+0x10/0x10 [nfs]
[ 965.591759] ? __pfx_nfs_writeback_done.part.0+0x10/0x10 [nfs]
[ 965.597645] ? lockdep_hardirqs_on+0x78/0x100
[ 965.602023] ? __pfx_nfs_writeback_result+0x10/0x10 [nfs]
[ 965.607472] ? nfs_writeback_done+0xec/0x130 [nfs]
[ 965.612320] nfs_local_pgio_release+0x261/0x3b0 [nfs]
[ 965.617421] ? __pfx_nfs_local_pgio_release+0x10/0x10 [nfs]
[ 965.623048] nfs_local_call_write+0x4fc/0xa50 [nfs]
[ 965.627981] ? __pfx_nfs_local_call_write+0x10/0x10 [nfs]
[ 965.633430] ? rcu_is_watching+0x15/0xb0
[ 965.637373] ? rcu_is_watching+0x15/0xb0
[ 965.641299] process_one_work+0xd88/0x1320
[ 965.645419] ? __pfx_process_one_work+0x10/0x10
[ 965.649967] ? assign_work+0x16c/0x240
[ 965.653736] worker_thread+0x5f3/0xfe0
[ 965.657510] ? __pfx_worker_thread+0x10/0x10
[ 965.661796] kthread+0x3b1/0x770
[ 965.665028] ? local_clock_noinstr+0x45/0xe0
[ 965.669320] ? __pfx_kthread+0x10/0x10
[ 965.673071] ? __lock_release.isra.0+0x1a4/0x2c0
[ 965.677707] ? rcu_is_watching+0x15/0xb0
[ 965.681652] ? __pfx_kthread+0x10/0x10
[ 965.685404] ret_from_fork+0x390/0x480
[ 965.689173] ? __pfx_kthread+0x10/0x10
[ 965.692927] ? __pfx_kthread+0x10/0x10
[ 965.696681] ret_from_fork_asm+0x1a/0x30
[ 965.700627] </TASK>
[ 965.702833] Modules linked in: rpcsec_gss_krb5 nfsv4 dns_resolver nfs netfs rpcrdma rdma_cm iw_cm ib_cm ib_core nfsd auth_rpcgss nfs_acl lockd grace nfs_localio rfkill intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_ifs i10nm_edac skx_edac_common nfit x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm sunrpc dax_hmem irqbypass iTCO_wdt pmt_telemetry rapl mgag200 iTCO_vendor_support pmt_class intel_sdsi intel_cstate platform_profile mei_me dell_smbios ses ipmi_ssif isst_if_mmio isst_if_mbox_pci dcdbas dell_wmi_descriptor wmi_bmof intel_uncore enclosure pcspkr isst_if_common intel_vsec i2c_algo_bit tg3 acpi_power_meter mei i2c_i801 i2c_ismt i2c_smbus ipmi_si acpi_ipmi ipmi_devintf ipmi_msghandler sg loop fuse dm_mod nfnetlink xfs sd_mod iaa_crypto mpt3sas ahci raid_class ghash_clmulni_intel idxd libahci scsi_transport_sas libata idxd_bus wmi pinctrl_emmitsburg
[ 965.783610] ---[ end trace 0000000000000000 ]---
[ 965.810265] RIP: 0010:__put_cred+0xea/0x120
[ 965.814484] Code: 2d 8b 83 a8 00 00 00 85 c0 74 0b 48 83 c4 08 5b 5d e9 fa fb ff ff 48 83 c4 08 48 c7 c6 80 b8 78 92 5b 5d e9 f8 b0 1d 00 0f 0b <0f> 0b 0f 0b 48 89 3c 24 e8 09 17 9b 00 48 8b 3c 24 eb c4 48 89 df
[ 965.833245] RSP: 0018:ffa000000156f790 EFLAGS: 00010246
[ 965.838490] RAX: dffffc0000000000 RBX: ff110001cb191b00 RCX: ffffffff9278bbcc
[ 965.845644] RDX: 1fe2200020620972 RSI: 0000000000000008 RDI: ff11000103104b90
[ 965.852802] RBP: ff11000103103ec0 R08: 0000000000000000 R09: ffe21c0039632360
[ 965.859949] R10: ff110001cb191b07 R11: 0000000000000001 R12: 0000000000000000
[ 965.867089] R13: ff11000170651208 R14: ff110001706511e8 R15: ff11000170651180
[ 965.874239] FS: 0000000000000000(0000) GS:ff11000887ef3000(0000) knlGS:0000000000000000
[ 965.882344] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 965.888096] CR2: 000056321a5700b8 CR3: 00000004a5c7a006 CR4: 0000000000f73ef0
[ 965.895248] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 965.902396] DR3: 0000000000000000 DR6: 00000000fffe07f0 DR7: 0000000000000400
[ 965.909549] PKRU: 55555554
[ 965.912278] Kernel panic - not syncing: Fatal exception
[ 965.917552] Kernel Offset: 0x11200000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)
[ 965.950419] ---[ end Kernel panic - not syncing: Fatal exception ]---
[-- MARK -- Fri Jun 6 01:40:00 2025]
>
> Thanks,
> Mike
>
> On Thu, May 08, 2025 at 01:18:29PM +0800, Zorro Lang wrote:
> > Hi,
> >
> > Recently I always hit nfs crash[1] on latest mainline upstream linux 6.15.0-rc5+,
> > which HEAD=707df3375124b51048233625a7e1c801e8c8a7fd . I've hit this bug on aarch64
> > and x86_64 several times, by running xfstests on nfsv4.2 [2].
> >
> > Thanks,
> > Zorro
> >
> > [1]
> > [ 1004.936621] run fstests generic/023 at 2025-05-07 11:06:07
> > [ 1005.840825] ------------[ cut here ]------------
> > [ 1005.845547] kernel BUG at kernel/cred.c:104!
> > [ 1005.849848] Oops: invalid opcode: 0000 [#1] SMP KASAN NOPTI
> > [ 1005.855440] CPU: 12 UID: 0 PID: 78471 Comm: kworker/u128:20 Not tainted 6.15.0-rc5+ #1 PREEMPT(voluntary)
> > [ 1005.865104] Hardware name: Dell Inc. PowerEdge R660/0R5JJC, BIOS 2.1.5 03/14/2024
> > [ 1005.872583] Workqueue: nfslocaliod nfs_local_call_write [nfs]
> > [ 1005.878416] RIP: 0010:__put_cred+0xea/0x120
> > [ 1005.882618] Code: 2d 8b 83 a8 00 00 00 85 c0 74 0b 48 83 c4 08 5b 5d e9 fa fb ff ff 48 83 c4 08 48 c7 c6 c0 af da 83 5b 5d e9 28 54 1d 00 0f 0b <0f> 0b 0f 0b 48 89 3c 24 e8 09 dc 99 00 48 8b 3c 24 eb c4 48 89 df
> > [ 1005.901383] RSP: 0018:ffa00000247a78d8 EFLAGS: 00010246
> > [ 1005.906626] RAX: dffffc0000000000 RBX: ff110001428ea000 RCX: ffffffff83dab30c
> > [ 1005.913757] RDX: 1fe220004203d19a RSI: 0000000000000008 RDI: ff110002101e8cd0
> > [ 1005.920892] RBP: ff110002101e8000 R08: 0000000000000000 R09: ffe21c002851d400
> > [ 1005.928024] R10: ff110001428ea007 R11: 0000000000000001 R12: 0000000000000000
> > [ 1005.935157] R13: ff110001f6c1ce88 R14: ff110001f6c1ce68 R15: ff110001f6c1ce00
> > [ 1005.942289] FS: 0000000000000000(0000) GS:ff110008938c7000(0000) knlGS:0000000000000000
> > [ 1005.950376] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> > [ 1005.956122] CR2: 00007f619475fdc0 CR3: 00000003f6c76004 CR4: 0000000000f73ef0
> > [ 1005.963256] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> > [ 1005.970388] DR3: 0000000000000000 DR6: 00000000fffe07f0 DR7: 0000000000000400
> > [ 1005.977520] PKRU: 55555554
> > [ 1005.980234] Call Trace:
> > [ 1005.982685] <TASK>
> > [ 1005.984794] __fput+0x61e/0xaa0
> > [ 1005.987958] nfsd_file_free+0x1a1/0x3e0 [nfsd]
> > [ 1005.992505] nfsd_file_put_local+0x35/0x50 [nfsd]
> > [ 1005.997290] nfs_close_local_fh+0x362/0x6f0 [nfs_localio]
> > [ 1006.002709] __put_nfs_open_context+0x380/0x500 [nfs]
> > [ 1006.007813] ? nfs_write_completion+0x111/0x8a0 [nfs]
> > [ 1006.012926] nfs_pgio_header_free+0x41/0xe0 [nfs]
> > [ 1006.017693] nfs_write_completion+0x139/0x8a0 [nfs]
> > [ 1006.022631] ? __pfx_nfs_writeback_done.part.0+0x10/0x10 [nfs]
> > [ 1006.028517] ? __pfx_nfs_write_completion+0x10/0x10 [nfs]
> > [ 1006.033981] nfs_local_call_write+0x4fc/0xa50 [nfs]
> > [ 1006.038918] ? __pfx_nfs_local_call_write+0x10/0x10 [nfs]
> > [ 1006.044376] ? rcu_is_watching+0x15/0xb0
> > [ 1006.048320] ? rcu_is_watching+0x15/0xb0
> > [ 1006.052246] process_one_work+0xd6e/0x1330
> > [ 1006.056349] ? __pfx_process_one_work+0x10/0x10
> > [ 1006.060897] ? assign_work+0x16c/0x240
> > [ 1006.064668] worker_thread+0x5f3/0xfe0
> > [ 1006.068437] ? __kthread_parkme+0xb4/0x200
> > [ 1006.072553] ? __pfx_worker_thread+0x10/0x10
> > [ 1006.076842] kthread+0x3b1/0x770
> > [ 1006.080077] ? local_clock_noinstr+0xd/0xe0
> > [ 1006.084278] ? __pfx_kthread+0x10/0x10
> > [ 1006.088032] ? __lock_release.isra.0+0x1a4/0x2c0
> > [ 1006.092667] ? rcu_is_watching+0x15/0xb0
> > [ 1006.096611] ? __pfx_kthread+0x10/0x10
> > [ 1006.100382] ret_from_fork+0x31/0x70
> > [ 1006.103978] ? __pfx_kthread+0x10/0x10
> > [ 1006.107738] ret_from_fork_asm+0x1a/0x30
> > [ 1006.111687] </TASK>
> > [ 1006.113894] Modules linked in: rpcsec_gss_krb5 nfsv4 dns_resolver nfs netfs rpcrdma rdma_cm iw_cm ib_cm ib_core nfsd auth_rpcgss nfs_acl lockd grace nfs_localio rfkill intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_ifs sunrpc i10nm_edac skx_edac_common nfit x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm dax_hmem irqbypass cxl_acpi iTCO_wdt igb mgag200 cxl_port rapl dell_pc pmt_telemetry iTCO_vendor_support pmt_class cxl_core intel_sdsi dell_smbios platform_profile intel_cstate ipmi_ssif ses isst_if_mmio dcdbas wmi_bmof dell_wmi_descriptor pcspkr intel_uncore enclosure einj isst_if_mbox_pci mei_me isst_if_common intel_vsec i2c_i801 i2c_algo_bit tg3 dca mei i2c_smbus i2c_ismt acpi_power_meter ipmi_si acpi_ipmi ipmi_devintf ipmi_msghandler sg fuse loop dm_mod nfnetlink xfs sd_mod iaa_crypto mpt3sas ahci raid_class ghash_clmulni_intel libahci scsi_transport_sas idxd libata idxd_bus wmi pinctrl_emmitsburg
> > [ 1006.198764] ---[ end trace 0000000000000000 ]---
> > [ 1006.223531] RIP: 0010:__put_cred+0xea/0x120
> > [ 1006.227744] Code: 2d 8b 83 a8 00 00 00 85 c0 74 0b 48 83 c4 08 5b 5d e9 fa fb ff ff 48 83 c4 08 48 c7 c6 c0 af da 83 5b 5d e9 28 54 1d 00 0f 0b <0f> 0b 0f 0b 48 89 3c 24 e8 09 dc 99 00 48 8b 3c 24 eb c4 48 89 df
> > [ 1006.246509] RSP: 0018:ffa00000247a78d8 EFLAGS: 00010246
> > [ 1006.251762] RAX: dffffc0000000000 RBX: ff110001428ea000 RCX: ffffffff83dab30c
> > [ 1006.258911] RDX: 1fe220004203d19a RSI: 0000000000000008 RDI: ff110002101e8cd0
> > [ 1006.266065] RBP: ff110002101e8000 R08: 0000000000000000 R09: ffe21c002851d400
> > [ 1006.273218] R10: ff110001428ea007 R11: 0000000000000001 R12: 0000000000000000
> > [ 1006.280368] R13: ff110001f6c1ce88 R14: ff110001f6c1ce68 R15: ff110001f6c1ce00
> > [ 1006.287521] FS: 0000000000000000(0000) GS:ff110008938c7000(0000) knlGS:0000000000000000
> > [ 1006.295623] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> > [ 1006.301387] CR2: 00007f619475fdc0 CR3: 00000003f6c76004 CR4: 0000000000f73ef0
> > [ 1006.308535] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> > [ 1006.315687] DR3: 0000000000000000 DR6: 00000000fffe07f0 DR7: 0000000000000400
> > [ 1006.322836] PKRU: 55555554
> > [ 1006.325569] Kernel panic - not syncing: Fatal exception
> > [ 1006.330842] Kernel Offset: 0x2800000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)
> > [ 1006.361714] ---[ end Kernel panic - not syncing: Fatal exception ]---
> > [-- MARK -- Wed May 7 15:10:00 2025]
> > [-- MARK -- Wed May 7 15:15:00 2025]
> >
> > [2]
> > export FSTYP=nfs
> > export TEST_DEV=xxxxxxx.xxxxxxxxxx.xxxxxxxxxxxx:/mnt/xfstests/test/nfs-server
> > export TEST_DIR=/mnt/xfstests/test/nfs-client
> > export SCRATCH_DEV=xxxxxxx.xxxxxxxxxx.xxxxxxxxxxxx:/mnt/xfstests/scratch/nfs-server
> > export SCRATCH_MNT=/mnt/xfstests/scratch/nfs-client
> > export MOUNT_OPTIONS="-o vers=4.2"
> > export TEST_FS_MOUNT_OPTS="-o vers=4.2"
> >
> >
>
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [Bug report] xfstests cause nfs crash, kernel BUG at kernel/cred.c:104!
2025-06-04 17:18 ` Mike Snitzer
2025-06-06 2:38 ` Zorro Lang
@ 2025-08-05 22:34 ` Scott Mayhew
2025-08-05 23:30 ` Mike Snitzer
1 sibling, 1 reply; 7+ messages in thread
From: Scott Mayhew @ 2025-08-05 22:34 UTC (permalink / raw)
To: Mike Snitzer; +Cc: Zorro Lang, linux-nfs
On Wed, 04 Jun 2025, Mike Snitzer wrote:
> First I'm seeing this, I haven't hit it myself.
>
> Can you elaborate on how you've configured xfstest? I'll take a look
> at generic/023 to understand how NFS is being used (but for it to be
> hitting this with LOCALIO it must be doing loopback mount testing?)
>
> neilbrown's localio patchset landed upstream yesterday via anna's 6.16
> pull request, so it'd be useful to know if linus/master still has
> this. v6.15 should get neilb's localio patchset because it was marked
> for stable@ backport.
>
> Thanks,
> Mike
I just hit a similar oops running cthon. The first time I hit it I was
testing the xprtsec patch I was working on, but then I hit the panic on
a stock kernel on Fedora rawhide
(6.17.0-0.rc0.250804gd2eedaa3909b.10.fc43.x86_64).
crash> bt
PID: 1909 TASK: ffff8b156f120000 CPU: 5 COMMAND: "kworker/u38:3"
#0 [ffffcf5bc439fa18] machine_kexec at ffffffff91d6b3fb
#1 [ffffcf5bc439fa38] __crash_kexec at ffffffff91ef40f0
#2 [ffffcf5bc439faf8] crash_kexec at ffffffff91ef4604
#3 [ffffcf5bc439fb00] oops_end at ffffffff91d1b999
#4 [ffffcf5bc439fb20] do_trap at ffffffff91d15fea
#5 [ffffcf5bc439fb60] do_error_trap at ffffffff91d1634a
#6 [ffffcf5bc439fba0] exc_invalid_op at ffffffff92f55be2
#7 [ffffcf5bc439fbc0] asm_exc_invalid_op at ffffffff91a0123a
[exception RIP: __put_cred+0x56]
RIP: ffffffff91df4306 RSP: ffffcf5bc439fc78 RFLAGS: 00010246
RAX: ffff8b156f120000 RBX: ffff8b1566671900 RCX: ffffffff951eb530
RDX: 0000000000000002 RSI: 0000000000000002 RDI: ffff8b1548ea5cc0
RBP: 0000000000000000 R8: 0000000000000002 R9: 0000000000000020
R10: ffffffffffffffff R11: 0000000000000003 R12: ffff8b1550579538
R13: ffff8b1548047520 R14: ffff8b15405b0480 R15: ffff8b1545956300
ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018
#8 [ffffcf5bc439fc78] __fput at ffffffff921dfb37
#9 [ffffcf5bc439fca8] nfsd_file_free at ffffffffc0c6f95c [nfsd]
#10 [ffffcf5bc439fcc0] nfsd_file_put_local at ffffffffc0c70e77 [nfsd]
#11 [ffffcf5bc439fcd0] nfs_close_local_fh at ffffffffc0c2aa15 [nfs_localio]
#12 [ffffcf5bc439fd40] __put_nfs_open_context at ffffffffc1323afc [nfs]
#13 [ffffcf5bc439fd78] nfs_pgio_header_free at ffffffffc132d14e [nfs]
#14 [ffffcf5bc439fd88] nfs_write_completion at ffffffffc1334584 [nfs]
#15 [ffffcf5bc439fde0] nfs_local_call_write at ffffffffc134808f [nfs]
#16 [ffffcf5bc439fe58] process_one_work at ffffffff91ddeb52
#17 [ffffcf5bc439fe98] worker_thread at ffffffff91ddf66a
#18 [ffffcf5bc439fee0] kthread at ffffffff91de9d29
#19 [ffffcf5bc439ff30] ret_from_fork at ffffffff91d2796f
#20 [ffffcf5bc439ff50] ret_from_fork_asm at ffffffff91cd670a
We hit this bug in __put_cred():
BUG_ON(cred == current->cred);
The address of the nfsd_file is on the stack in __fput():
crash> bt -F
...
#8 [ffffcf5bc439fc78] __fput at ffffffff921dfb37
ffffcf5bc439fc80: [nfsd_file] 0000000000000000
ffffcf5bc439fc90: [kmalloc-rnd-08-2k] [nfs_write_data]
ffffcf5bc439fca0: 0000000000000000 nfsd_file_free+0x8c
#9 [ffffcf5bc439fca8] nfsd_file_free at ffffffffc0c6f95c [nfsd]
...
crash> rd ffffcf5bc439fc80
ffffcf5bc439fc80: ffff8b1544477280 .rGD....
crash> nfsd_file.nf_file ffff8b1544477280
nf_file = 0xffff8b1566671900,
crash> file.f_cred 0xffff8b1566671900
f_cred = 0xffff8b1548ea5cc0,
crash> task -R cred | grep ffff8b1548ea5cc0
cred = 0xffff8b1548ea5cc0,
-Scott
>
> On Thu, May 08, 2025 at 01:18:29PM +0800, Zorro Lang wrote:
> > Hi,
> >
> > Recently I always hit nfs crash[1] on latest mainline upstream linux 6.15.0-rc5+,
> > which HEAD=707df3375124b51048233625a7e1c801e8c8a7fd . I've hit this bug on aarch64
> > and x86_64 several times, by running xfstests on nfsv4.2 [2].
> >
> > Thanks,
> > Zorro
> >
> > [1]
> > [ 1004.936621] run fstests generic/023 at 2025-05-07 11:06:07
> > [ 1005.840825] ------------[ cut here ]------------
> > [ 1005.845547] kernel BUG at kernel/cred.c:104!
> > [ 1005.849848] Oops: invalid opcode: 0000 [#1] SMP KASAN NOPTI
> > [ 1005.855440] CPU: 12 UID: 0 PID: 78471 Comm: kworker/u128:20 Not tainted 6.15.0-rc5+ #1 PREEMPT(voluntary)
> > [ 1005.865104] Hardware name: Dell Inc. PowerEdge R660/0R5JJC, BIOS 2.1.5 03/14/2024
> > [ 1005.872583] Workqueue: nfslocaliod nfs_local_call_write [nfs]
> > [ 1005.878416] RIP: 0010:__put_cred+0xea/0x120
> > [ 1005.882618] Code: 2d 8b 83 a8 00 00 00 85 c0 74 0b 48 83 c4 08 5b 5d e9 fa fb ff ff 48 83 c4 08 48 c7 c6 c0 af da 83 5b 5d e9 28 54 1d 00 0f 0b <0f> 0b 0f 0b 48 89 3c 24 e8 09 dc 99 00 48 8b 3c 24 eb c4 48 89 df
> > [ 1005.901383] RSP: 0018:ffa00000247a78d8 EFLAGS: 00010246
> > [ 1005.906626] RAX: dffffc0000000000 RBX: ff110001428ea000 RCX: ffffffff83dab30c
> > [ 1005.913757] RDX: 1fe220004203d19a RSI: 0000000000000008 RDI: ff110002101e8cd0
> > [ 1005.920892] RBP: ff110002101e8000 R08: 0000000000000000 R09: ffe21c002851d400
> > [ 1005.928024] R10: ff110001428ea007 R11: 0000000000000001 R12: 0000000000000000
> > [ 1005.935157] R13: ff110001f6c1ce88 R14: ff110001f6c1ce68 R15: ff110001f6c1ce00
> > [ 1005.942289] FS: 0000000000000000(0000) GS:ff110008938c7000(0000) knlGS:0000000000000000
> > [ 1005.950376] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> > [ 1005.956122] CR2: 00007f619475fdc0 CR3: 00000003f6c76004 CR4: 0000000000f73ef0
> > [ 1005.963256] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> > [ 1005.970388] DR3: 0000000000000000 DR6: 00000000fffe07f0 DR7: 0000000000000400
> > [ 1005.977520] PKRU: 55555554
> > [ 1005.980234] Call Trace:
> > [ 1005.982685] <TASK>
> > [ 1005.984794] __fput+0x61e/0xaa0
> > [ 1005.987958] nfsd_file_free+0x1a1/0x3e0 [nfsd]
> > [ 1005.992505] nfsd_file_put_local+0x35/0x50 [nfsd]
> > [ 1005.997290] nfs_close_local_fh+0x362/0x6f0 [nfs_localio]
> > [ 1006.002709] __put_nfs_open_context+0x380/0x500 [nfs]
> > [ 1006.007813] ? nfs_write_completion+0x111/0x8a0 [nfs]
> > [ 1006.012926] nfs_pgio_header_free+0x41/0xe0 [nfs]
> > [ 1006.017693] nfs_write_completion+0x139/0x8a0 [nfs]
> > [ 1006.022631] ? __pfx_nfs_writeback_done.part.0+0x10/0x10 [nfs]
> > [ 1006.028517] ? __pfx_nfs_write_completion+0x10/0x10 [nfs]
> > [ 1006.033981] nfs_local_call_write+0x4fc/0xa50 [nfs]
> > [ 1006.038918] ? __pfx_nfs_local_call_write+0x10/0x10 [nfs]
> > [ 1006.044376] ? rcu_is_watching+0x15/0xb0
> > [ 1006.048320] ? rcu_is_watching+0x15/0xb0
> > [ 1006.052246] process_one_work+0xd6e/0x1330
> > [ 1006.056349] ? __pfx_process_one_work+0x10/0x10
> > [ 1006.060897] ? assign_work+0x16c/0x240
> > [ 1006.064668] worker_thread+0x5f3/0xfe0
> > [ 1006.068437] ? __kthread_parkme+0xb4/0x200
> > [ 1006.072553] ? __pfx_worker_thread+0x10/0x10
> > [ 1006.076842] kthread+0x3b1/0x770
> > [ 1006.080077] ? local_clock_noinstr+0xd/0xe0
> > [ 1006.084278] ? __pfx_kthread+0x10/0x10
> > [ 1006.088032] ? __lock_release.isra.0+0x1a4/0x2c0
> > [ 1006.092667] ? rcu_is_watching+0x15/0xb0
> > [ 1006.096611] ? __pfx_kthread+0x10/0x10
> > [ 1006.100382] ret_from_fork+0x31/0x70
> > [ 1006.103978] ? __pfx_kthread+0x10/0x10
> > [ 1006.107738] ret_from_fork_asm+0x1a/0x30
> > [ 1006.111687] </TASK>
> > [ 1006.113894] Modules linked in: rpcsec_gss_krb5 nfsv4 dns_resolver nfs netfs rpcrdma rdma_cm iw_cm ib_cm ib_core nfsd auth_rpcgss nfs_acl lockd grace nfs_localio rfkill intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_ifs sunrpc i10nm_edac skx_edac_common nfit x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm dax_hmem irqbypass cxl_acpi iTCO_wdt igb mgag200 cxl_port rapl dell_pc pmt_telemetry iTCO_vendor_support pmt_class cxl_core intel_sdsi dell_smbios platform_profile intel_cstate ipmi_ssif ses isst_if_mmio dcdbas wmi_bmof dell_wmi_descriptor pcspkr intel_uncore enclosure einj isst_if_mbox_pci mei_me isst_if_common intel_vsec i2c_i801 i2c_algo_bit tg3 dca mei i2c_smbus i2c_ismt acpi_power_meter ipmi_si acpi_ipmi ipmi_devintf ipmi_msghandler sg fuse loop dm_mod nfnetlink xfs sd_mod iaa_crypto mpt3sas ahci raid_class ghash_clmulni_intel libahci scsi_transport_sas idxd libata idxd_bus wmi pinctrl_emmitsburg
> > [ 1006.198764] ---[ end trace 0000000000000000 ]---
> > [ 1006.223531] RIP: 0010:__put_cred+0xea/0x120
> > [ 1006.227744] Code: 2d 8b 83 a8 00 00 00 85 c0 74 0b 48 83 c4 08 5b 5d e9 fa fb ff ff 48 83 c4 08 48 c7 c6 c0 af da 83 5b 5d e9 28 54 1d 00 0f 0b <0f> 0b 0f 0b 48 89 3c 24 e8 09 dc 99 00 48 8b 3c 24 eb c4 48 89 df
> > [ 1006.246509] RSP: 0018:ffa00000247a78d8 EFLAGS: 00010246
> > [ 1006.251762] RAX: dffffc0000000000 RBX: ff110001428ea000 RCX: ffffffff83dab30c
> > [ 1006.258911] RDX: 1fe220004203d19a RSI: 0000000000000008 RDI: ff110002101e8cd0
> > [ 1006.266065] RBP: ff110002101e8000 R08: 0000000000000000 R09: ffe21c002851d400
> > [ 1006.273218] R10: ff110001428ea007 R11: 0000000000000001 R12: 0000000000000000
> > [ 1006.280368] R13: ff110001f6c1ce88 R14: ff110001f6c1ce68 R15: ff110001f6c1ce00
> > [ 1006.287521] FS: 0000000000000000(0000) GS:ff110008938c7000(0000) knlGS:0000000000000000
> > [ 1006.295623] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> > [ 1006.301387] CR2: 00007f619475fdc0 CR3: 00000003f6c76004 CR4: 0000000000f73ef0
> > [ 1006.308535] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> > [ 1006.315687] DR3: 0000000000000000 DR6: 00000000fffe07f0 DR7: 0000000000000400
> > [ 1006.322836] PKRU: 55555554
> > [ 1006.325569] Kernel panic - not syncing: Fatal exception
> > [ 1006.330842] Kernel Offset: 0x2800000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)
> > [ 1006.361714] ---[ end Kernel panic - not syncing: Fatal exception ]---
> > [-- MARK -- Wed May 7 15:10:00 2025]
> > [-- MARK -- Wed May 7 15:15:00 2025]
> >
> > [2]
> > export FSTYP=nfs
> > export TEST_DEV=xxxxxxx.xxxxxxxxxx.xxxxxxxxxxxx:/mnt/xfstests/test/nfs-server
> > export TEST_DIR=/mnt/xfstests/test/nfs-client
> > export SCRATCH_DEV=xxxxxxx.xxxxxxxxxx.xxxxxxxxxxxx:/mnt/xfstests/scratch/nfs-server
> > export SCRATCH_MNT=/mnt/xfstests/scratch/nfs-client
> > export MOUNT_OPTIONS="-o vers=4.2"
> > export TEST_FS_MOUNT_OPTS="-o vers=4.2"
> >
> >
>
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [Bug report] xfstests cause nfs crash, kernel BUG at kernel/cred.c:104!
2025-08-05 22:34 ` Scott Mayhew
@ 2025-08-05 23:30 ` Mike Snitzer
2025-08-07 16:49 ` [PATCH] nfs/localio: restore creds before releasing pageio data Scott Mayhew
0 siblings, 1 reply; 7+ messages in thread
From: Mike Snitzer @ 2025-08-05 23:30 UTC (permalink / raw)
To: Scott Mayhew, trondmy; +Cc: Zorro Lang, linux-nfs
On Tue, Aug 05, 2025 at 06:34:26PM -0400, Scott Mayhew wrote:
> On Wed, 04 Jun 2025, Mike Snitzer wrote:
>
> > First I'm seeing this, I haven't hit it myself.
> >
> > Can you elaborate on how you've configured xfstest? I'll take a look
> > at generic/023 to understand how NFS is being used (but for it to be
> > hitting this with LOCALIO it must be doing loopback mount testing?)
> >
> > neilbrown's localio patchset landed upstream yesterday via anna's 6.16
> > pull request, so it'd be useful to know if linus/master still has
> > this. v6.15 should get neilb's localio patchset because it was marked
> > for stable@ backport.
> >
> > Thanks,
> > Mike
>
> I just hit a similar oops running cthon. The first time I hit it I was
> testing the xprtsec patch I was working on, but then I hit the panic on
> a stock kernel on Fedora rawhide
> (6.17.0-0.rc0.250804gd2eedaa3909b.10.fc43.x86_64).
There have been various fixes to LOCALIO to fix issues introduced
during thr 6.16 merge window. Chuck's nfsd-6.17 pull request seems to
have _not_ included NeilBrown's fix that is now staged in Chuck's
nfsd-testing branch:
854328c725e6d ("nfsd: avoid ref leak in nfsd_open_local_fh()")
There are also 3 fixes that AFAIK are needed on the NFS side of the
house, which I included in v7 of this patchset (first 3 patches), see:
https://lore.kernel.org/linux-nfs/20250805232106.8656-1-snitzer@kernel.org/
Trond, please pick up your 3 LOCALIO fixes, or did you drop them for a
reason?
Thanks,
Mike
> crash> bt
> PID: 1909 TASK: ffff8b156f120000 CPU: 5 COMMAND: "kworker/u38:3"
> #0 [ffffcf5bc439fa18] machine_kexec at ffffffff91d6b3fb
> #1 [ffffcf5bc439fa38] __crash_kexec at ffffffff91ef40f0
> #2 [ffffcf5bc439faf8] crash_kexec at ffffffff91ef4604
> #3 [ffffcf5bc439fb00] oops_end at ffffffff91d1b999
> #4 [ffffcf5bc439fb20] do_trap at ffffffff91d15fea
> #5 [ffffcf5bc439fb60] do_error_trap at ffffffff91d1634a
> #6 [ffffcf5bc439fba0] exc_invalid_op at ffffffff92f55be2
> #7 [ffffcf5bc439fbc0] asm_exc_invalid_op at ffffffff91a0123a
> [exception RIP: __put_cred+0x56]
> RIP: ffffffff91df4306 RSP: ffffcf5bc439fc78 RFLAGS: 00010246
> RAX: ffff8b156f120000 RBX: ffff8b1566671900 RCX: ffffffff951eb530
> RDX: 0000000000000002 RSI: 0000000000000002 RDI: ffff8b1548ea5cc0
> RBP: 0000000000000000 R8: 0000000000000002 R9: 0000000000000020
> R10: ffffffffffffffff R11: 0000000000000003 R12: ffff8b1550579538
> R13: ffff8b1548047520 R14: ffff8b15405b0480 R15: ffff8b1545956300
> ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018
> #8 [ffffcf5bc439fc78] __fput at ffffffff921dfb37
> #9 [ffffcf5bc439fca8] nfsd_file_free at ffffffffc0c6f95c [nfsd]
> #10 [ffffcf5bc439fcc0] nfsd_file_put_local at ffffffffc0c70e77 [nfsd]
> #11 [ffffcf5bc439fcd0] nfs_close_local_fh at ffffffffc0c2aa15 [nfs_localio]
> #12 [ffffcf5bc439fd40] __put_nfs_open_context at ffffffffc1323afc [nfs]
> #13 [ffffcf5bc439fd78] nfs_pgio_header_free at ffffffffc132d14e [nfs]
> #14 [ffffcf5bc439fd88] nfs_write_completion at ffffffffc1334584 [nfs]
> #15 [ffffcf5bc439fde0] nfs_local_call_write at ffffffffc134808f [nfs]
> #16 [ffffcf5bc439fe58] process_one_work at ffffffff91ddeb52
> #17 [ffffcf5bc439fe98] worker_thread at ffffffff91ddf66a
> #18 [ffffcf5bc439fee0] kthread at ffffffff91de9d29
> #19 [ffffcf5bc439ff30] ret_from_fork at ffffffff91d2796f
> #20 [ffffcf5bc439ff50] ret_from_fork_asm at ffffffff91cd670a
>
> We hit this bug in __put_cred():
>
> BUG_ON(cred == current->cred);
>
> The address of the nfsd_file is on the stack in __fput():
>
> crash> bt -F
> ...
> #8 [ffffcf5bc439fc78] __fput at ffffffff921dfb37
> ffffcf5bc439fc80: [nfsd_file] 0000000000000000
> ffffcf5bc439fc90: [kmalloc-rnd-08-2k] [nfs_write_data]
> ffffcf5bc439fca0: 0000000000000000 nfsd_file_free+0x8c
> #9 [ffffcf5bc439fca8] nfsd_file_free at ffffffffc0c6f95c [nfsd]
> ...
>
> crash> rd ffffcf5bc439fc80
> ffffcf5bc439fc80: ffff8b1544477280 .rGD....
>
> crash> nfsd_file.nf_file ffff8b1544477280
> nf_file = 0xffff8b1566671900,
>
> crash> file.f_cred 0xffff8b1566671900
> f_cred = 0xffff8b1548ea5cc0,
>
> crash> task -R cred | grep ffff8b1548ea5cc0
> cred = 0xffff8b1548ea5cc0,
>
> -Scott
>
> >
> > On Thu, May 08, 2025 at 01:18:29PM +0800, Zorro Lang wrote:
> > > Hi,
> > >
> > > Recently I always hit nfs crash[1] on latest mainline upstream linux 6.15.0-rc5+,
> > > which HEAD=707df3375124b51048233625a7e1c801e8c8a7fd . I've hit this bug on aarch64
> > > and x86_64 several times, by running xfstests on nfsv4.2 [2].
> > >
> > > Thanks,
> > > Zorro
> > >
> > > [1]
> > > [ 1004.936621] run fstests generic/023 at 2025-05-07 11:06:07
> > > [ 1005.840825] ------------[ cut here ]------------
> > > [ 1005.845547] kernel BUG at kernel/cred.c:104!
> > > [ 1005.849848] Oops: invalid opcode: 0000 [#1] SMP KASAN NOPTI
> > > [ 1005.855440] CPU: 12 UID: 0 PID: 78471 Comm: kworker/u128:20 Not tainted 6.15.0-rc5+ #1 PREEMPT(voluntary)
> > > [ 1005.865104] Hardware name: Dell Inc. PowerEdge R660/0R5JJC, BIOS 2.1.5 03/14/2024
> > > [ 1005.872583] Workqueue: nfslocaliod nfs_local_call_write [nfs]
> > > [ 1005.878416] RIP: 0010:__put_cred+0xea/0x120
> > > [ 1005.882618] Code: 2d 8b 83 a8 00 00 00 85 c0 74 0b 48 83 c4 08 5b 5d e9 fa fb ff ff 48 83 c4 08 48 c7 c6 c0 af da 83 5b 5d e9 28 54 1d 00 0f 0b <0f> 0b 0f 0b 48 89 3c 24 e8 09 dc 99 00 48 8b 3c 24 eb c4 48 89 df
> > > [ 1005.901383] RSP: 0018:ffa00000247a78d8 EFLAGS: 00010246
> > > [ 1005.906626] RAX: dffffc0000000000 RBX: ff110001428ea000 RCX: ffffffff83dab30c
> > > [ 1005.913757] RDX: 1fe220004203d19a RSI: 0000000000000008 RDI: ff110002101e8cd0
> > > [ 1005.920892] RBP: ff110002101e8000 R08: 0000000000000000 R09: ffe21c002851d400
> > > [ 1005.928024] R10: ff110001428ea007 R11: 0000000000000001 R12: 0000000000000000
> > > [ 1005.935157] R13: ff110001f6c1ce88 R14: ff110001f6c1ce68 R15: ff110001f6c1ce00
> > > [ 1005.942289] FS: 0000000000000000(0000) GS:ff110008938c7000(0000) knlGS:0000000000000000
> > > [ 1005.950376] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> > > [ 1005.956122] CR2: 00007f619475fdc0 CR3: 00000003f6c76004 CR4: 0000000000f73ef0
> > > [ 1005.963256] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> > > [ 1005.970388] DR3: 0000000000000000 DR6: 00000000fffe07f0 DR7: 0000000000000400
> > > [ 1005.977520] PKRU: 55555554
> > > [ 1005.980234] Call Trace:
> > > [ 1005.982685] <TASK>
> > > [ 1005.984794] __fput+0x61e/0xaa0
> > > [ 1005.987958] nfsd_file_free+0x1a1/0x3e0 [nfsd]
> > > [ 1005.992505] nfsd_file_put_local+0x35/0x50 [nfsd]
> > > [ 1005.997290] nfs_close_local_fh+0x362/0x6f0 [nfs_localio]
> > > [ 1006.002709] __put_nfs_open_context+0x380/0x500 [nfs]
> > > [ 1006.007813] ? nfs_write_completion+0x111/0x8a0 [nfs]
> > > [ 1006.012926] nfs_pgio_header_free+0x41/0xe0 [nfs]
> > > [ 1006.017693] nfs_write_completion+0x139/0x8a0 [nfs]
> > > [ 1006.022631] ? __pfx_nfs_writeback_done.part.0+0x10/0x10 [nfs]
> > > [ 1006.028517] ? __pfx_nfs_write_completion+0x10/0x10 [nfs]
> > > [ 1006.033981] nfs_local_call_write+0x4fc/0xa50 [nfs]
> > > [ 1006.038918] ? __pfx_nfs_local_call_write+0x10/0x10 [nfs]
> > > [ 1006.044376] ? rcu_is_watching+0x15/0xb0
> > > [ 1006.048320] ? rcu_is_watching+0x15/0xb0
> > > [ 1006.052246] process_one_work+0xd6e/0x1330
> > > [ 1006.056349] ? __pfx_process_one_work+0x10/0x10
> > > [ 1006.060897] ? assign_work+0x16c/0x240
> > > [ 1006.064668] worker_thread+0x5f3/0xfe0
> > > [ 1006.068437] ? __kthread_parkme+0xb4/0x200
> > > [ 1006.072553] ? __pfx_worker_thread+0x10/0x10
> > > [ 1006.076842] kthread+0x3b1/0x770
> > > [ 1006.080077] ? local_clock_noinstr+0xd/0xe0
> > > [ 1006.084278] ? __pfx_kthread+0x10/0x10
> > > [ 1006.088032] ? __lock_release.isra.0+0x1a4/0x2c0
> > > [ 1006.092667] ? rcu_is_watching+0x15/0xb0
> > > [ 1006.096611] ? __pfx_kthread+0x10/0x10
> > > [ 1006.100382] ret_from_fork+0x31/0x70
> > > [ 1006.103978] ? __pfx_kthread+0x10/0x10
> > > [ 1006.107738] ret_from_fork_asm+0x1a/0x30
> > > [ 1006.111687] </TASK>
> > > [ 1006.113894] Modules linked in: rpcsec_gss_krb5 nfsv4 dns_resolver nfs netfs rpcrdma rdma_cm iw_cm ib_cm ib_core nfsd auth_rpcgss nfs_acl lockd grace nfs_localio rfkill intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_ifs sunrpc i10nm_edac skx_edac_common nfit x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm dax_hmem irqbypass cxl_acpi iTCO_wdt igb mgag200 cxl_port rapl dell_pc pmt_telemetry iTCO_vendor_support pmt_class cxl_core intel_sdsi dell_smbios platform_profile intel_cstate ipmi_ssif ses isst_if_mmio dcdbas wmi_bmof dell_wmi_descriptor pcspkr intel_uncore enclosure einj isst_if_mbox_pci mei_me isst_if_common intel_vsec i2c_i801 i2c_algo_bit tg3 dca mei i2c_smbus i2c_ismt acpi_power_meter ipmi_si acpi_ipmi ipmi_devintf ipmi_msghandler sg fuse loop dm_mod nfnetlink xfs sd_mod iaa_crypto mpt3sas ahci raid_class ghash_clmulni_intel libahci scsi_transport_sas idxd libata idxd_bus wmi pinctrl_emmitsburg
> > > [ 1006.198764] ---[ end trace 0000000000000000 ]---
> > > [ 1006.223531] RIP: 0010:__put_cred+0xea/0x120
> > > [ 1006.227744] Code: 2d 8b 83 a8 00 00 00 85 c0 74 0b 48 83 c4 08 5b 5d e9 fa fb ff ff 48 83 c4 08 48 c7 c6 c0 af da 83 5b 5d e9 28 54 1d 00 0f 0b <0f> 0b 0f 0b 48 89 3c 24 e8 09 dc 99 00 48 8b 3c 24 eb c4 48 89 df
> > > [ 1006.246509] RSP: 0018:ffa00000247a78d8 EFLAGS: 00010246
> > > [ 1006.251762] RAX: dffffc0000000000 RBX: ff110001428ea000 RCX: ffffffff83dab30c
> > > [ 1006.258911] RDX: 1fe220004203d19a RSI: 0000000000000008 RDI: ff110002101e8cd0
> > > [ 1006.266065] RBP: ff110002101e8000 R08: 0000000000000000 R09: ffe21c002851d400
> > > [ 1006.273218] R10: ff110001428ea007 R11: 0000000000000001 R12: 0000000000000000
> > > [ 1006.280368] R13: ff110001f6c1ce88 R14: ff110001f6c1ce68 R15: ff110001f6c1ce00
> > > [ 1006.287521] FS: 0000000000000000(0000) GS:ff110008938c7000(0000) knlGS:0000000000000000
> > > [ 1006.295623] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> > > [ 1006.301387] CR2: 00007f619475fdc0 CR3: 00000003f6c76004 CR4: 0000000000f73ef0
> > > [ 1006.308535] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> > > [ 1006.315687] DR3: 0000000000000000 DR6: 00000000fffe07f0 DR7: 0000000000000400
> > > [ 1006.322836] PKRU: 55555554
> > > [ 1006.325569] Kernel panic - not syncing: Fatal exception
> > > [ 1006.330842] Kernel Offset: 0x2800000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)
> > > [ 1006.361714] ---[ end Kernel panic - not syncing: Fatal exception ]---
> > > [-- MARK -- Wed May 7 15:10:00 2025]
> > > [-- MARK -- Wed May 7 15:15:00 2025]
> > >
> > > [2]
> > > export FSTYP=nfs
> > > export TEST_DEV=xxxxxxx.xxxxxxxxxx.xxxxxxxxxxxx:/mnt/xfstests/test/nfs-server
> > > export TEST_DIR=/mnt/xfstests/test/nfs-client
> > > export SCRATCH_DEV=xxxxxxx.xxxxxxxxxx.xxxxxxxxxxxx:/mnt/xfstests/scratch/nfs-server
> > > export SCRATCH_MNT=/mnt/xfstests/scratch/nfs-client
> > > export MOUNT_OPTIONS="-o vers=4.2"
> > > export TEST_FS_MOUNT_OPTS="-o vers=4.2"
> > >
> > >
> >
>
^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH] nfs/localio: restore creds before releasing pageio data
2025-08-05 23:30 ` Mike Snitzer
@ 2025-08-07 16:49 ` Scott Mayhew
2025-08-08 2:34 ` Mike Snitzer
0 siblings, 1 reply; 7+ messages in thread
From: Scott Mayhew @ 2025-08-07 16:49 UTC (permalink / raw)
To: snitzer, trondmy, anna; +Cc: zlang, linux-nfs
Otherwise if the nfsd filecache code releases the nfsd_file
immediately, it can trigger the BUG_ON(cred == current->cred) in
__put_cred() when it puts the nfsd_file->nf_file->f-cred.
Fixes: b9f5dd57f4a5 ("nfs/localio: use dedicated workqueues for filesystem read and write")
Signed-off-by: Scott Mayhew <smayhew@redhat.com>
---
Mike - I applied the four patches you mentioned and I still see the oops
quite frequently. This patch fixes it for me.
fs/nfs/localio.c | 12 +++++++-----
1 file changed, 7 insertions(+), 5 deletions(-)
diff --git a/fs/nfs/localio.c b/fs/nfs/localio.c
index 510d0a16cfe9..e2213ef18bae 100644
--- a/fs/nfs/localio.c
+++ b/fs/nfs/localio.c
@@ -453,12 +453,13 @@ static void nfs_local_call_read(struct work_struct *work)
nfs_local_iter_init(&iter, iocb, READ);
status = filp->f_op->read_iter(&iocb->kiocb, &iter);
+
+ revert_creds(save_cred);
+
if (status != -EIOCBQUEUED) {
nfs_local_read_done(iocb, status);
nfs_local_pgio_release(iocb);
}
-
- revert_creds(save_cred);
}
static int
@@ -649,14 +650,15 @@ static void nfs_local_call_write(struct work_struct *work)
file_start_write(filp);
status = filp->f_op->write_iter(&iocb->kiocb, &iter);
file_end_write(filp);
+
+ revert_creds(save_cred);
+ current->flags = old_flags;
+
if (status != -EIOCBQUEUED) {
nfs_local_write_done(iocb, status);
nfs_local_vfs_getattr(iocb);
nfs_local_pgio_release(iocb);
}
-
- revert_creds(save_cred);
- current->flags = old_flags;
}
static int
--
2.50.1
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: nfs/localio: restore creds before releasing pageio data
2025-08-07 16:49 ` [PATCH] nfs/localio: restore creds before releasing pageio data Scott Mayhew
@ 2025-08-08 2:34 ` Mike Snitzer
0 siblings, 0 replies; 7+ messages in thread
From: Mike Snitzer @ 2025-08-08 2:34 UTC (permalink / raw)
To: Scott Mayhew; +Cc: trondmy, anna, zlang, linux-nfs
On Thu, Aug 07, 2025 at 12:49:38PM -0400, Scott Mayhew wrote:
> Otherwise if the nfsd filecache code releases the nfsd_file
> immediately, it can trigger the BUG_ON(cred == current->cred) in
> __put_cred() when it puts the nfsd_file->nf_file->f-cred.
>
> Fixes: b9f5dd57f4a5 ("nfs/localio: use dedicated workqueues for filesystem read and write")
> Signed-off-by: Scott Mayhew <smayhew@redhat.com>
> ---
> Mike - I applied the four patches you mentioned and I still see the oops
> quite frequently. This patch fixes it for me.
Looks good. Interesting that I've never experienced this BUG_ON.
Reviewed-by: Mike Snitzer <snitzer@kernel.org>
Thanks!
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2025-08-08 2:34 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-05-08 5:18 [Bug report] xfstests cause nfs crash, kernel BUG at kernel/cred.c:104! Zorro Lang
2025-06-04 17:18 ` Mike Snitzer
2025-06-06 2:38 ` Zorro Lang
2025-08-05 22:34 ` Scott Mayhew
2025-08-05 23:30 ` Mike Snitzer
2025-08-07 16:49 ` [PATCH] nfs/localio: restore creds before releasing pageio data Scott Mayhew
2025-08-08 2:34 ` Mike Snitzer
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).