Re: [REGRESSION] xfs kernel panic

[Greg KH <gregkh@linuxfoundation.org>]

On Mon, Feb 17, 2025 at 09:29:57AM -0800, Darrick J. Wong wrote:
> On Mon, Feb 17, 2025 at 05:27:33PM +0100, Lorenz Brun wrote:
> > Am Mo., 17. Feb. 2025 um 16:00 Uhr schrieb Lorenz Brun <lorenz@monogon.tech>:
> > >
> > > Hi everyone,
> > >
> > > Linux 6.12.14 (released today) contains a regression for XFS, causing
> > > a kernel panic after just a few seconds of working with a
> > > freshly-created (xfsprogs 6.9) XFS filesystem. I have not yet bisected
> > > this because I wanted to get this report out ASAP but I'm going to do
> > > that now. There are multiple associated stack traces, but all of them
> > > have xfs_buf_offset as the faulting function.
> > >
> > > Example backtrace:
> > > [   31.745932] BUG: kernel NULL pointer dereference, address: 0000000000000098
> > > [   31.746590] #PF: supervisor read access in kernel mode
> > > [   31.747072] #PF: error_code(0x0000) - not-present page
> > > [   31.747537] PGD 5bee067 P4D 5bee067 PUD 5bef067 PMD 0
> > > [   31.748016] Oops: Oops: 0000 [#1] PREEMPT SMP NOPTI
> > > [   31.748459] CPU: 0 UID: 0 PID: 116 Comm: xfsaild/vda4 Not tainted
> > > 6.12.14-metropolis #1 9b2470be3d7713b818a3236e4a2804dd9cbef735
> > > [   31.749490] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009),
> > > BIOS 0.0.0 02/06/2015
> > > [   31.750340] RIP: 0010:xfs_buf_offset+0x9/0x50
> > > [   31.750823] Code: 08 5b e9 8a 2c c4 00 66 2e 0f 1f 84 00 00 00 00
> > > 00 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 66 0f 1f 00 0f 1f
> > > 44 00 00 <48> 8b 87 98 00 00 00 48 85 c0 75 2e 48 8b 87 00 01 00 00 48
> > > 89 f2
> > > [   31.752775] RSP: 0018:ffffbf50c07abdb8 EFLAGS: 00010246
> > > [   31.753343] RAX: 0000000000000002 RBX: ffff9c0985817d58 RCX: 0000000000000016
> > > [   31.754103] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000
> > > [   31.754734] RBP: 0000000000000000 R08: ffff9c09fb704000 R09: 00000000e0be9fc4
> > > [   31.755396] R10: 0000000000000000 R11: ffff9c0985827df8 R12: ffff9c09fb57ff58
> > > [   31.756078] R13: ffff9c0985817eb0 R14: ffff9c09fb704000 R15: ffff9c0985817f00
> > > [   31.756764] FS:  0000000000000000(0000) GS:ffff9c09fc000000(0000)
> > > knlGS:0000000000000000
> > > [   31.757529] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> > > [   31.758041] CR2: 0000000000000098 CR3: 0000000005b70000 CR4: 0000000000350ef0
> > > [   31.758696] Call Trace:
> > > [   31.758940]  <TASK>
> > > [   31.759172]  ? __die+0x56/0x97
> > > [   31.759473]  ? page_fault_oops+0x15c/0x2d0
> > > [   31.759853]  ? exc_page_fault+0x4c5/0x790
> > > [   31.760237]  ? asm_exc_page_fault+0x26/0x30
> > > [   31.760637]  ? xfs_buf_offset+0x9/0x50
> > > [   31.761002]  ? srso_return_thunk+0x5/0x5f
> > > [   31.761409]  xfs_qm_dqflush+0xd0/0x350
> > > [   31.761799]  xfs_qm_dquot_logitem_push+0xe9/0x140
> > > [   31.762253]  xfsaild+0x347/0xa10
> > > [   31.762567]  ? srso_return_thunk+0x5/0x5f
> > > [   31.762952]  ? srso_return_thunk+0x5/0x5f
> > > [   31.763325]  ? __pfx_xfsaild+0x10/0x10
> > > [   31.763665]  kthread+0xd2/0x100
> > > [   31.763985]  ? __pfx_kthread+0x10/0x10
> > > [   31.764342]  ret_from_fork+0x34/0x50
> > > [   31.764675]  ? __pfx_kthread+0x10/0x10
> > > [   31.765029]  ret_from_fork_asm+0x1a/0x30
> > > [   31.765408]  </TASK>
> > > [   31.765618] Modules linked in: kvm_amd
> > > [   31.765978] CR2: 0000000000000098
> > > [   31.766297] ---[ end trace 0000000000000000 ]---
> > > [   32.371004] RIP: 0010:xfs_buf_offset+0x9/0x50
> > > [   32.371453] Code: 08 5b e9 8a 2c c4 00 66 2e 0f 1f 84 00 00 00 00
> > > 00 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 66 0f 1f 00 0f 1f
> > > 44 00 00 <48> 8b 87 98 00 00 00 48 85 c0 75 2e 48 8b 87 00 01 00 00 48
> > > 89 f2
> > > [   32.373133] RSP: 0018:ffffbf50c07abdb8 EFLAGS: 00010246
> > > [   32.373611] RAX: 0000000000000002 RBX: ffff9c0985817d58 RCX: 0000000000000016
> > > [   32.374275] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000
> > > [   32.374921] RBP: 0000000000000000 R08: ffff9c09fb704000 R09: 00000000e0be9fc4
> > > [   32.375720] R10: 0000000000000000 R11: ffff9c0985827df8 R12: ffff9c09fb57ff58
> > > [   32.376376] R13: ffff9c0985817eb0 R14: ffff9c09fb704000 R15: ffff9c0985817f00
> > > [   32.377027] FS:  0000000000000000(0000) GS:ffff9c09fc000000(0000)
> > > knlGS:0000000000000000
> > > [   32.377761] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> > > [   32.378292] CR2: 0000000000000098 CR3: 0000000005b70000 CR4: 0000000000350ef0
> > > [   32.378940] Kernel panic - not syncing: Fatal exception
> > > [   32.379492] Kernel Offset: 0x2a600000 from 0xffffffff81000000
> > > (relocation range: 0xffffffff80000000-0xffffffffbfffffff)
> > >
> > > #regzbot introduced: v6.12.13..v6.12.14
> > >
> > > Regards,
> > > Lorenz
> > 
> > Hi everyone,
> > 
> > I root-caused this to 5808d420 ("xfs: attach dquot buffer to dquot log
> > item buffer"), but needs reverting of the 3 follow-up commits
> > (d331fc15, ee6984a2 and 84307caf) as well as they depend on the broken
> > one. With that 6.12.14 passes our test suite again. Reproduction
> > should be rather easy by just creating a fresh filesystem, mounting
> > with "prjquota" and performing I/O.
> 
> Known bug, will patch soon.

Great, thanks, I'll go push out a new 6.12 release with this fix in it
now.

greg k-h