From: Ritesh Harjani <riteshh@linux.ibm.com>
To: Qu Wenruo <quwenruo.btrfs@gmx.com>
Cc: Qu Wenruo <wqu@suse.com>, linux-btrfs@vger.kernel.org
Subject: Re: [Patch v2 41/42] btrfs: fix the use-after-free bug in writeback subpage helper
Date: Mon, 10 May 2021 17:59:33 +0530 [thread overview]
Message-ID: <20210510122933.mcg2sac2ugdennbs@riteshh-domain> (raw)
In-Reply-To: <e7e6ebdd-a220-e4ec-64e4-d031d7a9b181@gmx.com>
On 21/05/10 04:38PM, Qu Wenruo wrote:
> Hi Ritesh,
>
> I guess no error report so far is a good thing?
Sorry about the delay in starting of my testing. Was not keeping well since
Friday onwards, hence could not start the testing. (Feeling much better now).
So -g quick passed w/o any fatal issues. But with -g auto I got a kernel bug
with btrfs/28. Below is the report.
>
> Just to report what my result is, I ran my latest github branch for the
> full weekend, over 50 hours, and around 20 runs of full generic/auto
> without defrag groups.
>
> And I see no crash at all.
>
> But there is a special note, there is a new patch, introduced just
> before the weekend (Fri May 7 09:31:43 2021 +0800), titled "btrfs: fix a
> possible use-after-free race in metadata read path", is a new fix for a
> bug I reproduced once locally.
Yes, I already have this in my tree. This is the latest patch in my tree which
I am testing.
"btrfs: remove io_failure_record::in_validation"
>
> The bug should only happen when read is slow and only happens for
> metadata read path.
>
> The details can be found in the commit message, although it's rare to
> hit, I have hit such problem around 3 times in total.
>
> Hopes you didn't hit any crash during your test.
I am hitting below bug_on(). Since I saw your email just now, so I am directly
reporting this failure, w/o analyzing. Please let me know if you need anything
else from my end for this.
I will halt the testing of "-g auto" for now. Once we have some conclusion on
this one, then will resume the testing.
btrfs/028 32s ... [10:41:18][ 780.104573] run fstests btrfs/028 at 2021-05-10 10:41:18
[ 780.732073] BTRFS: device fsid be9b827d-28ee-4a5e-80a0-e19971061a58 devid 1 transid 5 /dev/vdc scanned by mkfs.btrfs (21129)
[ 780.759754] BTRFS info (device vdc): disk space caching is enabled
[ 780.759848] BTRFS info (device vdc): has skinny extents
[ 780.759888] BTRFS warning (device vdc): read-write for sector size 4096 with page size 65536 is experimental
<...>
[ 784.580404] BTRFS info (device vdc): found 21 extents, stage: move data extents
[ 784.878376] BTRFS info (device vdc): found 13 extents, stage: update data pointers
[ 785.175349] BTRFS info (device vdc): balance: ended with status: 0
[ 785.367729] BTRFS info (device vdc): balance: start -d
[ 785.400884] BTRFS info (device vdc): relocating block group 2446327808 flags data
[ 785.527858] btrfs_print_data_csum_error: 18 callbacks suppressed
[ 785.527865] BTRFS warning (device vdc): csum failed root -9 ino 262 off 393216 csum 0x8941f998 expected csum 0x9439dda4 mirror 1
[ 785.528406] btrfs_dev_stat_print_on_error: 18 callbacks suppressed
[ 785.528409] BTRFS error (device vdc): bdev /dev/vdc errs: wr 0, rd 0, flush 0, corrupt 1, gen 0
[ 785.528857] BTRFS warning (device vdc): csum failed root -9 ino 262 off 397312 csum 0x8941f998 expected csum 0x9439dda4 mirror 1
[ 785.529166] BTRFS error (device vdc): bdev /dev/vdc errs: wr 0, rd 0, flush 0, corrupt 2, gen 0
[ 785.529412] BTRFS warning (device vdc): csum failed root -9 ino 262 off 401408 csum 0x8941f998 expected csum 0x667b7e1e mirror 1
[ 785.529714] BTRFS error (device vdc): bdev /dev/vdc errs: wr 0, rd 0, flush 0, corrupt 3, gen 0
[ 785.530321] BTRFS warning (device vdc): csum failed root -9 ino 262 off 393216 csum 0x8941f998 expected csum 0x9439dda4 mirror 1
[ 785.530637] BTRFS error (device vdc): bdev /dev/vdc errs: wr 0, rd 0, flush 0, corrupt 4, gen 0
[ 785.530882] BTRFS warning (device vdc): csum failed root -9 ino 262 off 397312 csum 0x8941f998 expected csum 0x9439dda4 mirror 1
[ 785.531185] BTRFS error (device vdc): bdev /dev/vdc errs: wr 0, rd 0, flush 0, corrupt 5, gen 0
[ 785.531428] BTRFS warning (device vdc): csum failed root -9 ino 262 off 401408 csum 0x8941f998 expected csum 0x667b7e1e mirror 1
[ 785.531719] BTRFS error (device vdc): bdev /dev/vdc errs: wr 0, rd 0, flush 0, corrupt 6, gen 0
<...>
[ 803.459877] BTRFS info (device vdc): relocating block group 10499391488 flags data
[ 803.776810] BTRFS info (device vdc): found 29 extents, stage: move data extents
[ 803.979572] BTRFS info (device vdc): found 18 extents, stage: update data pointers
[ 804.276370] BTRFS info (device vdc): balance: ended with status: 0
[ 804.427621] BTRFS info (device vdc): balance: start -d
[ 804.454527] BTRFS info (device vdc): relocating block group 11036262400 flags data
[ 804.623962] BTRFS warning (device vdc): csum failed root -9 ino 282 off 684032 csum 0x8941f998 expected csum 0x605aaa22 mirror 1
[ 804.624147] BTRFS error (device vdc): bdev /dev/vdc errs: wr 0, rd 0, flush 0, corrupt 15, gen 0
[ 804.624277] BTRFS warning (device vdc): csum failed root -9 ino 282 off 688128 csum 0x8941f998 expected csum 0xe90a7889 mirror 1
[ 804.624435] BTRFS error (device vdc): bdev /dev/vdc errs: wr 0, rd 0, flush 0, corrupt 16, gen 0
[ 804.624682] assertion failed: atomic_read(&subpage->readers) >= nbits, in fs/btrfs/subpage.c:203
[ 804.624902] ------------[ cut here ]------------
[ 804.624989] kernel BUG at fs/btrfs/ctree.h:3415!
cpu 0x1: Vector: 700 (Program Check) at [c000000007b47640]
pc: c000000000af297c: assertfail.constprop.11+0x34/0x38
lr: c000000000af2978: assertfail.constprop.11+0x30/0x38
sp: c000000007b478e0
msr: 800000000282b033
current = 0xc000000007999800
paca = 0xc00000003fffee00 irqmask: 0x03 irq_happened: 0x01
pid = 23, comm = kworker/u4:1
kernel BUG at fs/btrfs/ctree.h:3415!
Linux version 5.12.0-rc8-00160-gcd0da6627caa (root@ltctulc6a-p1) (gcc (Ubuntu 8.4.0-1ubuntu1~18.04) 8.4.0, GNU ld (GNU Binutils for Ubuntu) 2.30) #25 SMP Mon May 10 01:31:44 CDT 2021
enter ? for help
[c000000007b47940] c000000000aefdac btrfs_subpage_end_reader+0x5c/0xb0
[c000000007b47980] c000000000a379f0 end_page_read+0x1d0/0x200
[c000000007b479c0] c000000000a41554 end_bio_extent_readpage+0x784/0x9b0
[c000000007b47b30] c000000000b4a234 bio_endio+0x254/0x270
[c000000007b47b70] c0000000009f6178 end_workqueue_fn+0x48/0x80
[c000000007b47ba0] c000000000a5c960 btrfs_work_helper+0x260/0x8e0
[c000000007b47c40] c00000000020a7f4 process_one_work+0x434/0x7d0
[c000000007b47d10] c00000000020ae94 worker_thread+0x304/0x570
[c000000007b47da0] c0000000002173cc kthread+0x1bc/0x1d0
[c000000007b47e10] c00000000000d6ec ret_from_kernel_thread+0x5c/0x70
-ritesh
next prev parent reply other threads:[~2021-05-10 13:40 UTC|newest]
Thread overview: 117+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-04-27 23:03 [Patch v2 00/42] btrfs: add data write support for subpage Qu Wenruo
2021-04-27 23:03 ` [Patch v2 01/42] btrfs: scrub: fix subpage scrub repair error caused by hardcoded PAGE_SIZE Qu Wenruo
2021-05-13 22:57 ` David Sterba
2021-05-13 23:32 ` Qu Wenruo
2021-04-27 23:03 ` [Patch v2 02/42] btrfs: make free space cache size consistent across different PAGE_SIZE Qu Wenruo
2021-04-27 23:03 ` [Patch v2 03/42] btrfs: remove the unused parameter @len for btrfs_bio_fits_in_stripe() Qu Wenruo
2021-05-13 22:58 ` David Sterba
2021-05-13 23:07 ` David Sterba
2021-04-27 23:03 ` [Patch v2 04/42] btrfs: allow btrfs_bio_fits_in_stripe() to accept bio without any page Qu Wenruo
2021-04-27 23:03 ` [Patch v2 05/42] btrfs: refactor submit_extent_page() to make bio and its flag tracing easier Qu Wenruo
2021-05-13 23:03 ` David Sterba
2021-05-21 11:06 ` Johannes Thumshirn
2021-05-21 11:26 ` Qu Wenruo
2021-05-21 13:30 ` David Sterba
2021-04-27 23:03 ` [Patch v2 06/42] btrfs: make subpage metadata write path to call its own endio functions Qu Wenruo
2021-04-27 23:03 ` [Patch v2 07/42] btrfs: pass btrfs_inode into btrfs_writepage_endio_finish_ordered() Qu Wenruo
2021-05-13 23:06 ` David Sterba
2021-05-13 23:35 ` Qu Wenruo
2021-05-21 14:27 ` Josef Bacik
2021-05-21 20:22 ` David Sterba
2021-05-22 0:24 ` Qu Wenruo
2021-05-23 7:40 ` Qu Wenruo
2021-05-23 13:43 ` Josef Bacik
2021-05-23 13:50 ` Qu Wenruo
2021-05-23 14:08 ` Josef Bacik
2021-04-27 23:03 ` [Patch v2 08/42] btrfs: make Private2 lifespan more consistent Qu Wenruo
2021-04-27 23:03 ` [Patch v2 09/42] btrfs: refactor how we finish ordered extent io for endio functions Qu Wenruo
2021-05-13 23:11 ` David Sterba
2021-04-27 23:03 ` [Patch v2 10/42] btrfs: update the comments in btrfs_invalidatepage() Qu Wenruo
2021-04-27 23:03 ` [Patch v2 11/42] btrfs: introduce btrfs_lookup_first_ordered_range() Qu Wenruo
2021-05-13 23:13 ` David Sterba
2021-04-27 23:03 ` [Patch v2 12/42] btrfs: refactor btrfs_invalidatepage() Qu Wenruo
2021-04-27 23:03 ` [Patch v2 13/42] btrfs: rename PagePrivate2 to PageOrdered inside btrfs Qu Wenruo
2021-04-27 23:03 ` [Patch v2 14/42] btrfs: pass bytenr directly to __process_pages_contig() Qu Wenruo
2021-04-27 23:03 ` [Patch v2 15/42] btrfs: refactor the page status update into process_one_page() Qu Wenruo
2021-04-27 23:03 ` [Patch v2 16/42] btrfs: provide btrfs_page_clamp_*() helpers Qu Wenruo
2021-04-27 23:03 ` [Patch v2 17/42] btrfs: only require sector size alignment for end_bio_extent_writepage() Qu Wenruo
2021-04-27 23:03 ` [Patch v2 18/42] btrfs: make btrfs_dirty_pages() to be subpage compatible Qu Wenruo
2021-04-27 23:03 ` [Patch v2 19/42] btrfs: make __process_pages_contig() to handle subpage dirty/error/writeback status Qu Wenruo
2021-04-27 23:03 ` [Patch v2 20/42] btrfs: make end_bio_extent_writepage() to be subpage compatible Qu Wenruo
2021-04-27 23:03 ` [Patch v2 21/42] btrfs: make process_one_page() to handle subpage locking Qu Wenruo
2021-04-27 23:03 ` [Patch v2 22/42] btrfs: introduce helpers for subpage ordered status Qu Wenruo
2021-04-27 23:03 ` [Patch v2 23/42] btrfs: make page Ordered bit to be subpage compatible Qu Wenruo
2021-04-27 23:03 ` [Patch v2 24/42] btrfs: update locked page dirty/writeback/error bits in __process_pages_contig Qu Wenruo
2021-04-27 23:03 ` [Patch v2 25/42] btrfs: prevent extent_clear_unlock_delalloc() to unlock page not locked by __process_pages_contig() Qu Wenruo
2021-04-27 23:03 ` [Patch v2 26/42] btrfs: make btrfs_set_range_writeback() subpage compatible Qu Wenruo
2021-04-27 23:03 ` [Patch v2 27/42] btrfs: make __extent_writepage_io() only submit dirty range for subpage Qu Wenruo
2021-04-27 23:03 ` [Patch v2 28/42] btrfs: make btrfs_truncate_block() to be subpage compatible Qu Wenruo
2021-04-27 23:03 ` [Patch v2 29/42] btrfs: make btrfs_page_mkwrite() " Qu Wenruo
2021-04-27 23:03 ` [Patch v2 30/42] btrfs: reflink: make copy_inline_to_page() " Qu Wenruo
2021-04-27 23:03 ` [Patch v2 31/42] btrfs: fix the filemap_range_has_page() call in btrfs_punch_hole_lock_range() Qu Wenruo
2021-04-27 23:03 ` [Patch v2 32/42] btrfs: don't clear page extent mapped if we're not invalidating the full page Qu Wenruo
2021-04-27 23:03 ` [Patch v2 33/42] btrfs: extract relocation page read and dirty part into its own function Qu Wenruo
2021-04-27 23:03 ` [Patch v2 34/42] btrfs: make relocate_one_page() to handle subpage case Qu Wenruo
2021-04-27 23:03 ` [Patch v2 35/42] btrfs: fix wild subpage writeback which does not have ordered extent Qu Wenruo
2021-04-27 23:03 ` [Patch v2 36/42] btrfs: disable inline extent creation for subpage Qu Wenruo
2021-05-04 4:28 ` Qu Wenruo
2021-04-27 23:03 ` [Patch v2 37/42] btrfs: skip validation for subpage read repair Qu Wenruo
2021-04-27 23:03 ` [Patch v2 38/42] btrfs: allow submit_extent_page() to do bio split for subpage Qu Wenruo
2021-04-27 23:03 ` [Patch v2 39/42] btrfs: reject raid5/6 fs " Qu Wenruo
2021-04-28 14:22 ` Neal Gompa
2021-04-28 23:11 ` Qu Wenruo
2021-05-12 22:04 ` David Sterba
2021-04-27 23:03 ` [Patch v2 40/42] btrfs: fix a crash caused by race between prepare_pages() and btrfs_releasepage() Qu Wenruo
2021-04-28 10:56 ` Filipe Manana
2021-04-27 23:03 ` [Patch v2 41/42] btrfs: fix the use-after-free bug in writeback subpage helper Qu Wenruo
2021-05-06 23:46 ` Qu Wenruo
2021-05-07 4:57 ` Ritesh Harjani
2021-05-07 5:14 ` Qu Wenruo
2021-05-10 8:38 ` Qu Wenruo
2021-05-10 12:29 ` Ritesh Harjani [this message]
2021-05-10 13:10 ` Qu Wenruo
2021-05-11 10:48 ` Ritesh Harjani
2021-05-11 11:15 ` Qu Wenruo
2021-05-12 1:49 ` Qu Wenruo
2021-05-12 7:09 ` Ritesh Harjani
2021-05-13 16:33 ` Ritesh Harjani
2021-05-13 21:36 ` Ritesh Harjani
2021-05-13 23:41 ` Qu Wenruo
2021-05-14 15:08 ` Ritesh Harjani
2021-05-14 17:53 ` Ritesh Harjani
2021-05-14 22:22 ` Qu Wenruo
2021-05-15 9:59 ` Ritesh Harjani
2021-05-15 10:15 ` Qu Wenruo
2021-05-25 4:43 ` Ritesh Harjani
2021-05-25 5:52 ` Qu Wenruo
2021-05-25 6:14 ` Qu Wenruo
2021-05-25 9:23 ` Ritesh Harjani
2021-05-25 9:45 ` Qu Wenruo
2021-05-25 9:49 ` Qu Wenruo
2021-05-25 10:20 ` Ritesh Harjani
2021-05-25 11:41 ` Qu Wenruo
2021-05-25 13:02 ` Ritesh Harjani
2021-05-26 5:29 ` Ritesh Harjani
2021-05-26 5:58 ` Qu Wenruo
2021-05-26 13:45 ` Ritesh Harjani
2021-05-28 8:26 ` Qu Wenruo
2021-05-28 8:59 ` Ritesh Harjani
2021-05-28 10:25 ` Qu Wenruo
2021-05-30 1:50 ` Qu Wenruo
2021-04-27 23:03 ` [Patch v2 42/42] btrfs: allow read-write for 4K sectorsize on 64K page size systems Qu Wenruo
2021-05-12 22:18 ` [Patch v2 00/42] btrfs: add data write support for subpage David Sterba
2021-05-12 23:48 ` Qu Wenruo
2021-05-13 2:21 ` Qu Wenruo
2021-05-13 22:54 ` David Sterba
2021-05-14 1:41 ` Qu Wenruo
2021-05-14 2:26 ` riteshh
2021-05-14 10:28 ` riteshh
2021-05-14 11:28 ` David Sterba
2021-05-14 14:38 ` riteshh
2021-05-14 11:30 ` David Sterba
2021-05-14 22:25 ` David Sterba
2021-05-14 22:45 ` Qu Wenruo
2021-05-14 23:05 ` David Sterba
2021-05-14 23:17 ` Qu Wenruo
2021-05-17 13:22 ` David Sterba
2021-05-17 23:20 ` Qu Wenruo
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210510122933.mcg2sac2ugdennbs@riteshh-domain \
--to=riteshh@linux.ibm.com \
--cc=linux-btrfs@vger.kernel.org \
--cc=quwenruo.btrfs@gmx.com \
--cc=wqu@suse.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).