From: "Theodore Ts'o" <tytso@mit.edu>
To: "Darrick J. Wong" <djwong@kernel.org>
Cc: linux-xfs@vger.kernel.org, leah.rumancik@gmail.com
Subject: soft lockup in xfs/170 on a file system formatted with -m crc=0
Date: Fri, 5 Nov 2021 13:25:05 -0400 [thread overview]
Message-ID: <YYVo8ZyKpy4Di0pK@mit.edu> (raw)
[-- Attachment #1: Type: text/plain, Size: 5582 bytes --]
Is this a known failure? I can reliably reproduce this soft lockup
running xfs/170 using "gce-xfstests -c xfs/v4 xfs/170" using
v5.15-rc4. The xfs/v4 test config formats the file system using -m
crc=0 with no special mount options.
I've attached the kernel config that I used; it's the standard one
obtained via "gce-xfstests install-kconfig"[1].
[1] https://github.com/tytso/xfstests-bld/blob/master/kvm-xfstests/util/install-kconfig
Thanks,
- Ted
commit adac31869b098d5f85d0930874dcf6a524d128d3
Author: Theodore Ts'o <tytso@mit.edu>
Date: Fri Nov 5 13:10:35 2021 -0400
test-appliance: add xfs/170 to xfs/v4's exclude file
The xfs/170 test is reliably causing a soft lockup when run on a file
system formatted with mkfs.xfs -m crc=0.
CONFIG: xfs/v4
ZONE: us-east1-b
VM STATUS: timeout on one test (xfs/170)
SINCE LAST UPDATE: 3h6m10s
TEST STATUS: hang
run fstests xfs/170 at 2021-11-05 02:27:05
...
[11024.269799] XFS (dm-1): Unmounting Filesystem
[11024.695394] XFS (dm-1): Mounting V4 Filesystem
[11024.731406] XFS (dm-1): Ending clean mount
[11024.731552] xfs filesystem being mounted at /xt-vdc supports timestamps until 2038 (0x7fffffff)
watchdog: BUG: soft lockup - CPU#1 stuck for 26s! [dd:9671]
irq event stamp: 22146924
hardirqs last enabled at (22146923): [<ffffffff903cebad>] _raw_spin_unlock_irqrestore+0x2d/0x40
hardirqs last disabled at (22146924): [<ffffffff903bd4fb>] sysvec_apic_timer_interrupt+0xb/0x90
softirqs last enabled at (16886422): [<ffffffff906002ce>] __do_softirq+0x2ce/0x3fd
softirqs last disabled at (16886399): [<ffffffff8f6bc1e8>] __irq_exit_rcu+0x88/0xb0
CPU: 1 PID: 9671 Comm: dd Not tainted 5.15.0-rc4-xfstests-00018-g124e7c61deb2 #382
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
RIP: 0010:_raw_spin_unlock_irqrestore+0x35/0x40
Code: c7 18 53 48 89 f3 48 8b 74 24 10 e8 e5 3b 35 ff 48 89 ef e8 dd 66 35 ff 80 e7 02 74 06 e8 f3 77 3e ff fb 65 ff 0d 4b 82 c4 6f <5b> 5d c3 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 41 54 65 ff 05 32
RSP: 0018:ffffa055c2913598 EFLAGS: 00000246
RAX: 000000000151ef6b RBX: 0000000000000282 RCX: 0000000000000040
RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffffff903cebad
RBP: ffff945d3aff4be0 R08: 0000000000000001 R09: 0000000000000001
R10: 0000000000000000 R11: 0000000000080000 R12: 0000000000000000
R13: ffffa055c2913640 R14: ffff945d13892a10 R15: ffff945d13892a58
FS: 00007f62407d8580(0000) GS:ffff945dd9400000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f6240613000 CR3: 000000014bc04004 CR4: 00000000003706e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
down_trylock+0x25/0x30
xfs_buf_trylock+0x17/0x190
xfs_buf_find+0x1b3/0x4a0
xfs_buf_get_map+0x44/0x3a0
xfs_buf_read_map+0x52/0x2e0
? xfs_read_agf+0xa3/0x180
xfs_trans_read_buf_map+0x144/0x430
? xfs_read_agf+0xa3/0x180
? __lock_acquire+0x3a7/0x6c0
xfs_read_agf+0xa3/0x180
xfs_alloc_read_agf+0x4c/0x110
xfs_alloc_pagf_init+0x27/0x60
xfs_filestream_pick_ag+0x280/0x530
xfs_filestream_new_ag+0x87/0x100
xfs_bmap_btalloc_filestreams.constprop.0+0xe0/0x120
xfs_bmap_btalloc+0x3e6/0x700
xfs_bmapi_allocate+0xe4/0x310
xfs_bmapi_write+0x42a/0x580
xfs_iomap_write_direct+0x18d/0x210
xfs_direct_write_iomap_begin+0x3ab/0x690
? lock_is_held_type+0x98/0x100
iomap_iter+0x12b/0x240
__iomap_dio_rw+0x1ff/0x630
iomap_dio_rw+0xa/0x30
xfs_file_dio_write_aligned+0xae/0x1c0
xfs_file_write_iter+0xd8/0x130
new_sync_write+0x122/0x1b0
? mod_objcg_state+0x180/0x2a0
vfs_write+0x25d/0x370
ksys_write+0x68/0xe0
do_syscall_64+0x3b/0x90
entry_SYSCALL_64_after_hwframe+0x44/0xae
RIP: 0033:0x7f6240700504
Code: 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b3 0f 1f 80 00 00 00 00 48 8d 05 f9 61 0d 00 8b 00 85 c0 75 13 b8 01 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 54 c3 0f 1f 00 41 54 49 89 d4 55 48 89 f5 53
RSP: 002b:00007ffdac48f148 EFLAGS: 00000246 ORIG_RAX: 0000000000000001
RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f6240700504
RDX: 0000000000100000 RSI: 00007f6240514000 RDI: 0000000000000001
RBP: 0000000000100000 R08: 00000000ffffffff R09: 0000000000000000
R10: ffffffffffffff3b R11: 0000000000000246 R12: 00007f6240514000
R13: 0000000000000000 R14: 0000000000000000 R15: 00007f6240514000
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
diff --git a/kvm-xfstests/test-appliance/files/root/fs/xfs/cfg/v4.exclude b/kvm-xfstests/test-appliance/files/root/fs/xfs/cfg/v4.exclude
index a9acba9c..83ccfd79 100644
--- a/kvm-xfstests/test-appliance/files/root/fs/xfs/cfg/v4.exclude
+++ b/kvm-xfstests/test-appliance/files/root/fs/xfs/cfg/v4.exclude
@@ -1,2 +1,16 @@
# Normal configurations don't support dax
-g dax
+
+# On a 5.15-rc4 kernel, xfs/170 reliably causes a soft lockup in
+# xfs_alloc_read_agf()+0x41/0x110. Call stack:
+#
+# xfs_alloc_pagf_init+0x27/0x60
+# xfs_filestream_pick_ag+0x280/0x530
+# xfs_filestream_new_ag+0x87/0x100
+# xfs_bmap_btalloc_filestreams.constprop.0+0xe0/0x120
+# xfs_bmap_btalloc+0x3e6/0x700
+# xfs_bmapi_allocate+0xe4/0x310
+# xfs_bmapi_convert_delalloc+0x26c/0x480
+# xfs_map_blocks+0x1b5/0x510
+# ...
+xfs/170
[-- Attachment #2: config.gz --]
[-- Type: application/gzip, Size: 18534 bytes --]
next reply other threads:[~2021-11-05 17:25 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-11-05 17:25 Theodore Ts'o [this message]
2021-11-06 1:58 ` xfs/076 takes a long long time testing with a realtime volume Theodore Ts'o
2021-11-06 2:08 ` Darrick J. Wong
2021-11-06 16:43 ` Theodore Ts'o
2021-11-06 2:10 ` soft lockup in xfs/170 on a file system formatted with -m crc=0 Darrick J. Wong
2021-11-06 14:52 ` Theodore Ts'o
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=YYVo8ZyKpy4Di0pK@mit.edu \
--to=tytso@mit.edu \
--cc=djwong@kernel.org \
--cc=leah.rumancik@gmail.com \
--cc=linux-xfs@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.