* Re: btrfs crash in kobject_del while running xfstest
[not found] <200e5b49-5c51-bbe5-de93-c6bd6339bb7f@nvidia.com>
@ 2020-08-11 11:19 ` John Hubbard
2020-08-11 14:02 ` David Sterba
0 siblings, 1 reply; 2+ messages in thread
From: John Hubbard @ 2020-08-11 11:19 UTC (permalink / raw)
To: Chris Mason, Josef Bacik, David Sterba, linux-btrfs, LKML; +Cc: linux-fsdevel
Somehow the copy-paste of Chris Mason's name failed (user error
on my end), sorry about that Chris!
On 8/11/20 4:17 AM, John Hubbard wrote:
> Hi,
>
> Here's an early warning of a possible problem.
>
> I'm seeing a new btrfs crash when running xfstests, as of
> 00e4db51259a5f936fec1424b884f029479d3981 ("Merge tag
> 'perf-tools-2020-08-10' of
> git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux") in linux.git.
>
> This doesn't crash in v5.8, so I attempted to bisect, but ended up with
> the net-next merge commit as the offending one: commit
> 47ec5303d73ea344e84f46660fff693c57641386 ("Merge
> git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next"), which
> doesn't really help because it's 2088 files changed, of course.
>
> I'm attaching the .config that I used.
>
> This is easily reproducible via something like (change to match your setup,
> of course):
>
> sudo TEST_DEV=/dev/nvme0n1p8 TEST_DIR=/xfstest_btrfs \
> SCRATCH_DEV=/dev/nvme0n1p9 SCRATCH_MNT=/xfstest_scratch ./check \
> btrfs/002
>
> which leads to:
>
> [ 586.097360] BTRFS info (device nvme0n1p8): disk space caching is enabled
> [ 586.103232] BTRFS info (device nvme0n1p8): has skinny extents
> [ 586.115169] BTRFS info (device nvme0n1p8): enabling ssd optimizations
> [ 586.308264] BTRFS: device fsid 5dfff89d-8f8d-42ac-8538-acb95164d0be devid 1 transid 5
> /dev/nvme0n1p9 scanned by mkfs.btrfs (6374)
> [ 586.342776] BTRFS info (device nvme0n1p9): disk space caching is enabled
> [ 586.348585] BTRFS info (device nvme0n1p9): has skinny extents
> [ 586.353413] BTRFS info (device nvme0n1p9): flagging fs with big metadata feature
> [ 586.368129] BTRFS info (device nvme0n1p9): enabling ssd optimizations
> [ 586.373996] BTRFS info (device nvme0n1p9): checking UUID tree
> [ 586.387449] BUG: kernel NULL pointer dereference, address: 0000000000000018
> [ 586.393485] #PF: supervisor read access in kernel mode
> [ 586.397623] #PF: error_code(0x0000) - not-present page
> [ 586.401763] PGD 0 P4D 0
> [ 586.403219] Oops: 0000 [#1] SMP PTI
> [ 586.405650] CPU: 1 PID: 6405 Comm: umount Not tainted 5.8.0-hubbard-github+ #171
> [ 586.412118] Hardware name: Gigabyte Technology Co., Ltd. To be filled by O.E.M./X99-UD3P-CF, BIOS
> F1 02/10/2015
> [ 586.421360] RIP: 0010:kobject_del+0x1/0x20
> [ 586.424427] Code: 48 c7 43 18 00 00 00 00 5b 5d c3 c3 be 01 00 00 00 48 89 df e8 60 1b 00 00 eb
> c9 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 55 <48> 8b 6f 18 e8 86 ff ff ff 48 89 ef 5d e9 cd fe ff
> ff 66 66 2e 0f
> [ 586.442644] RSP: 0018:ffffc90009ef7e08 EFLAGS: 00010246
> [ 586.446914] RAX: 0000000000000000 RBX: ffff888896080000 RCX: 0000000000000006
> [ 586.453149] RDX: ffff88888ee4b000 RSI: ffffffff82669a00 RDI: 0000000000000000
> [ 586.459390] RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000001
> [ 586.465631] R10: 0000000000000001 R11: 0000000000000000 R12: ffff888896080000
> [ 586.471866] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [ 586.478106] FS: 00007f5595739c80(0000) GS:ffff88889fc40000(0000) knlGS:0000000000000000
> [ 586.485325] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 586.490129] CR2: 0000000000000018 CR3: 0000000896d5a006 CR4: 00000000001706e0
> [ 586.496372] Call Trace:
> [ 586.497807] btrfs_sysfs_del_qgroups+0xa5/0xe0 [btrfs]
> [ 586.502017] close_ctree+0x1c5/0x2b6 [btrfs]
> [ 586.505307] ? fsnotify_destroy_marks+0x24/0x124
> [ 586.508948] generic_shutdown_super+0x67/0x100
> [ 586.512408] kill_anon_super+0x14/0x30
> [ 586.515159] btrfs_kill_super+0x12/0x20 [btrfs]
> [ 586.518704] deactivate_locked_super+0x36/0x90
> [ 586.522159] cleanup_mnt+0x12d/0x190
> [ 586.524720] task_work_run+0x5c/0xa0
> [ 586.527285] exit_to_user_mode_loop+0xb9/0xc0
> [ 586.530648] exit_to_user_mode_prepare+0xab/0xe0
> [ 586.534276] syscall_exit_to_user_mode+0x17/0x50
> [ 586.537908] entry_SYSCALL_64_after_hwframe+0x44/0xa9
> [ 586.541984] RIP: 0033:0x7f55959896fb
> [ 586.544531] Code: 07 0c 00 f7 d8 64 89 01 48 83 c8 ff c3 66 90 f3 0f 1e fa 31 f6 e9 05 00 00 00
> 0f 1f 44 00 00 f3 0f 1e fa b8 a6 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 5d 07 0c 00 f7
> d8 64 89 01 48
> [ 586.562775] RSP: 002b:00007fffcc431228 EFLAGS: 00000246 ORIG_RAX: 00000000000000a6
> [ 586.569485] RAX: 0000000000000000 RBX: 00007f5595ab31e4 RCX: 00007f55959896fb
> [ 586.575753] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 00005601fb16bb80
> [ 586.582020] RBP: 00005601fb16b970 R08: 0000000000000000 R09: 00007fffcc42ffa0
> [ 586.588278] R10: 00005601fb16c930 R11: 0000000000000246 R12: 00005601fb16bb80
> [ 586.594534] R13: 0000000000000000 R14: 00005601fb16ba68 R15: 0000000000000000
> [ 586.600805] Modules linked in: xfs rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace
> fscache bpfilter dm_mirror dm_region_hash dm_log dm_mod iTCO_wdt iTCO_vendor_support
> x86_pkg_temp_thermal coretemp crct10dif_pclmul crc32_pclmul btrfs ghash_clmulni_intel aesni_intel
> blake2b_generic crypto_simd xor cryptd zstd_compress glue_helper input_leds raid6_pq libcrc32c
> lpc_ich i2c_i801 mfd_core mei_me i2c_smbus mei rpcrdma sunrpc ib_isert iscsi_target_mod ib_iser
> libiscsi ib_srpt target_core_mod ib_srp ib_ipoib rdma_ucm ib_uverbs ib_umad sr_mod cdrom sd_mod
> nouveau ahci libahci nvme crc32c_intel video e1000e led_class nvme_core libata t10_pi ttm mxm_wmi
> wmi fuse
> [ 586.661098] CR2: 0000000000000018
> [ 586.663455] ---[ end trace 158f42d646f4715d ]---
>
> A quick peek shows that this is crashing here:
>
> void kobject_del(struct kobject *kobj)
> {
> struct kobject *parent = kobj->parent; <---- CRASHES HERE with NULL kobj
>
> __kobject_del(kobj);
> kobject_put(parent);
> }
> EXPORT_SYMBOL(kobject_del);
>
> The crash at 0x18 matches passes in a null, because that's the right offset for
> ->parent, and the disassembly confirms that 0x18 gets offset right at kobject_del+0x1:
>
> Dump of assembler code for function kobject_del:
> 0xffffffff81534ec0 <+0>: push %rbp
> 0xffffffff81534ec1 <+1>: mov 0x18(%rdi),%rbp
> 0xffffffff81534ec5 <+5>: callq 0xffffffff81534e50 <__kobject_del>
> 0xffffffff81534eca <+10>: mov %rbp,%rdi
> 0xffffffff81534ecd <+13>: pop %rbp
> 0xffffffff81534ece <+14>: jmpq 0xffffffff81534da0 <kobject_put>
> End of assembler dump.
>
> But as for how we ended up with a null kobj here, that's actually hard to see, at least
> for a non-btrfs person, which is why I hoped git bisect would help more than it did here.
>
>
> thanks,
thanks,
--
John Hubbard
NVIDIA
^ permalink raw reply [flat|nested] 2+ messages in thread