From: "Darrick J. Wong" <djwong@kernel.org>
To: xfs <linux-xfs@vger.kernel.org>
Subject: Re: KASAN report while running generic/475?
Date: Thu, 30 Jun 2022 13:18:10 -0700 [thread overview]
Message-ID: <Yr4FAt6oJfrdsMBQ@magnolia> (raw)
In-Reply-To: <YrzyfMZg4UcIACC8@magnolia>
On Wed, Jun 29, 2022 at 05:46:52PM -0700, Darrick J. Wong wrote:
> Hi all,
>
> I turned on KASAN and my test rig tripped over generic/475. I've not
> added Dave's log scalability patchset to my test systems yet, so I know
> that's /not/ to blame. I'll take a further look in the morning, but if
> this looks familiar to anyone, let me know?
>
> (Yes, this is djwong-dev, but I don't have any log changes in my
> development tree right now...)
Aha, I think there's a race between the ioend worker shutting down the
log, and log unmount freeing the CIL. Patches soon.
--D
> --D
>
> XFS (dm-0): Mounting V5 Filesystem
> XFS (dm-0): Starting recovery (logdev: internal)
> XFS (dm-0): Ending recovery (logdev: internal)
> Buffer I/O error on dev dm-0, logical block 20639616, async page read
> Buffer I/O error on dev dm-0, logical block 20639617, async page read
> XFS (dm-0): log I/O error -5
> XFS (dm-0): Filesystem has been shut down due to log error (0x2).
> XFS (dm-0): Unmounting Filesystem
> XFS (dm-0): Please unmount the filesystem and rectify the problem(s).
> ==================================================================
> BUG: KASAN: use-after-free in do_raw_spin_lock+0x246/0x270
> Read of size 4 at addr ffff888109dd84c4 by task 3:1H/136
>
> CPU: 3 PID: 136 Comm: 3:1H Not tainted 5.19.0-rc4-xfsx #rc4 8e53ab5ad0fddeb31cee5e7063ff9c361915a9c4
> Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.15.0-1 04/01/2014
> Workqueue: xfs-log/dm-0 xlog_ioend_work [xfs]
> Call Trace:
> <TASK>
> dump_stack_lvl+0x34/0x44
> print_report.cold+0x2b8/0x661
> ? do_raw_spin_lock+0x246/0x270
> kasan_report+0xab/0x120
> ? do_raw_spin_lock+0x246/0x270
> do_raw_spin_lock+0x246/0x270
> ? rwlock_bug.part.0+0x90/0x90
> xlog_force_shutdown+0xf6/0x370 [xfs 4ad76ae0d6add7e8183a553e624c31e9ed567318]
> xlog_ioend_work+0x100/0x190 [xfs 4ad76ae0d6add7e8183a553e624c31e9ed567318]
> process_one_work+0x672/0x1040
> worker_thread+0x59b/0xec0
> ? __kthread_parkme+0xc6/0x1f0
> ? process_one_work+0x1040/0x1040
> ? process_one_work+0x1040/0x1040
> kthread+0x29e/0x340
> ? kthread_complete_and_exit+0x20/0x20
> ret_from_fork+0x1f/0x30
> </TASK>
>
> Allocated by task 154099:
> kasan_save_stack+0x1e/0x40
> __kasan_kmalloc+0x81/0xa0
> kmem_alloc+0x8d/0x2e0 [xfs]
> xlog_cil_init+0x1f/0x540 [xfs]
> xlog_alloc_log+0xd1e/0x1260 [xfs]
> xfs_log_mount+0xba/0x640 [xfs]
> xfs_mountfs+0xf2b/0x1d00 [xfs]
> xfs_fs_fill_super+0x10af/0x1910 [xfs]
> get_tree_bdev+0x383/0x670
> vfs_get_tree+0x7d/0x240
> path_mount+0xdb7/0x1890
> __x64_sys_mount+0x1fa/0x270
> do_syscall_64+0x2b/0x80
> entry_SYSCALL_64_after_hwframe+0x46/0xb0
>
> Freed by task 154151:
> kasan_save_stack+0x1e/0x40
> kasan_set_track+0x21/0x30
> kasan_set_free_info+0x20/0x30
> ____kasan_slab_free+0x110/0x190
> slab_free_freelist_hook+0xab/0x180
> kfree+0xbc/0x310
> xlog_dealloc_log+0x1b/0x2b0 [xfs]
> xfs_unmountfs+0x119/0x200 [xfs]
> xfs_fs_put_super+0x6e/0x2e0 [xfs]
> generic_shutdown_super+0x12b/0x3a0
> kill_block_super+0x95/0xd0
> deactivate_locked_super+0x80/0x130
> cleanup_mnt+0x329/0x4d0
> task_work_run+0xc5/0x160
> exit_to_user_mode_prepare+0xd4/0xe0
> syscall_exit_to_user_mode+0x1d/0x40
> entry_SYSCALL_64_after_hwframe+0x46/0xb0
>
> Last potentially related work creation:
> kasan_save_stack+0x1e/0x40
> __kasan_record_aux_stack+0x9f/0xb0
> call_rcu+0x6f/0x700
> xfs_destroy_mount_workqueues+0x37/0x150 [xfs]
> xfs_fs_put_super+0x201/0x2e0 [xfs]
> generic_shutdown_super+0x12b/0x3a0
> kill_block_super+0x95/0xd0
> deactivate_locked_super+0x80/0x130
> cleanup_mnt+0x329/0x4d0
> task_work_run+0xc5/0x160
> exit_to_user_mode_prepare+0xd4/0xe0
> syscall_exit_to_user_mode+0x1d/0x40
> entry_SYSCALL_64_after_hwframe+0x46/0xb0
>
> Second to last potentially related work creation:
> kasan_save_stack+0x1e/0x40
> __kasan_record_aux_stack+0x9f/0xb0
> call_rcu+0x6f/0x700
> xfs_destroy_mount_workqueues+0x93/0x150 [xfs]
> xfs_fs_put_super+0x201/0x2e0 [xfs]
> generic_shutdown_super+0x12b/0x3a0
> kill_block_super+0x95/0xd0
> deactivate_locked_super+0x80/0x130
> cleanup_mnt+0x329/0x4d0
> task_work_run+0xc5/0x160
> exit_to_user_mode_prepare+0xd4/0xe0
> syscall_exit_to_user_mode+0x1d/0x40
> entry_SYSCALL_64_after_hwframe+0x46/0xb0
>
> The buggy address belongs to the object at ffff888109dd8400
> which belongs to the cache kmalloc-512 of size 512
> The buggy address is located 196 bytes inside of
> 512-byte region [ffff888109dd8400, ffff888109dd8600)
>
> The buggy address belongs to the physical page:
> page:ffffea0004277600 refcount:1 mapcount:0 mapping:0000000000000000 index:0x0 pfn:0x109dd8
> head:ffffea0004277600 order:2 compound_mapcount:0 compound_pincount:0
> flags: 0x17ff80000010200(slab|head|node=0|zone=2|lastcpupid=0xfff)
> raw: 017ff80000010200 ffffea00043cba00 dead000000000002 ffff888100042c80
> raw: 0000000000000000 0000000000100010 00000001ffffffff 0000000000000000
> page dumped because: kasan: bad access detected
>
> Memory state around the buggy address:
> ffff888109dd8380: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
> ffff888109dd8400: fa fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
> >ffff888109dd8480: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
> ^
> ffff888109dd8500: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
> ffff888109dd8580: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
> ==================================================================
> Disabling lock debugging due to kernel taint
prev parent reply other threads:[~2022-06-30 20:18 UTC|newest]
Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-06-30 0:46 KASAN report while running generic/475? Darrick J. Wong
2022-06-30 20:18 ` Darrick J. Wong [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=Yr4FAt6oJfrdsMBQ@magnolia \
--to=djwong@kernel.org \
--cc=linux-xfs@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.