linux-block.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Yu Kuai <yukuai1@huaweicloud.com>
To: Yu Kuai <yukuai1@huaweicloud.com>,
	Bart Van Assche <bvanassche@acm.org>,
	hch@infradead.org, colyli@kernel.org, hare@suse.de,
	dlemoal@kernel.org, tieren@fnnas.com, axboe@kernel.dk,
	tj@kernel.org, josef@toxicpanda.com, song@kernel.org,
	kmo@daterainc.com, satyat@google.com, ebiggers@google.com,
	neil@brown.name, akpm@linux-foundation.org
Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org,
	cgroups@vger.kernel.org, linux-raid@vger.kernel.org,
	yi.zhang@huawei.com, yangerkun@huawei.com,
	johnny.chenyi@huawei.com, "yukuai (C)" <yukuai3@huawei.com>
Subject: Re: [PATCH RFC v3 00/15] block: fix disordered IO in the case recursive split
Date: Tue, 2 Sep 2025 16:04:37 +0800	[thread overview]
Message-ID: <850263b4-fbc7-379e-2b9f-80f602ee0e72@huaweicloud.com> (raw)
In-Reply-To: <130482e9-8363-6051-5fc6-549cf9aad57b@huaweicloud.com>

Hi,

在 2025/09/02 9:50, Yu Kuai 写道:
> Hi,
> 
> 在 2025/09/01 22:09, Bart Van Assche 写道:
>> On 8/31/25 8:32 PM, Yu Kuai wrote:
>>> This set is just test for raid5 for now, see details in patch 9;
>>
>> Does this mean that this patch series doesn't fix reordering caused by
>> recursive splitting for zoned block devices? A test case that triggers
>> an I/O error is available here:
>> https://lore.kernel.org/linux-block/a8a714c7-de3d-4cc9-8c23-38b8dc06f5bb@acm.org/ 
>>
> I'll try this test.
> 

This test can't run directly in my VM, then I debug a bit and modify the
test a bit, following is the result by the block trace event of
block_io_start:

Before this set:

           dd-3014    [000] .N...  1918.939253: block_io_start: 252,2 WS 
524288 () 0 + 1024 be,0,4 [dd]
     kworker/0:1H-37      [000] .....  1918.952434: block_io_start: 
252,2 WS 524288 () 1024 + 1024 be,0,4 [kworker/0:1H]
               dd-3014    [000] .....  1918.973499: block_io_start: 
252,2 WS 524288 () 8192 + 1024 be,0,4 [dd]
     kworker/0:1H-37      [000] .....  1918.984805: block_io_start: 
252,2 WS 524288 () 9216 + 1024 be,0,4 [kworker/0:1H]
               dd-3014    [000] .N...  1919.010224: block_io_start: 
252,2 WS 524288 () 16384 + 1024 be,0,4 [dd]
     kworker/0:1H-37      [000] .....  1919.021667: block_io_start: 
252,2 WS 524288 () 17408 + 1024 be,0,4 [kworker/0:1H]
               dd-3014    [000] .....  1919.053072: block_io_start: 
252,2 WS 524288 () 24576 + 1024 be,0,4 [dd]
     kworker/0:1H-37      [000] .....  1919.064781: block_io_start: 
252,2 WS 524288 () 25600 + 1024 be,0,4 [kworker/0:1H]
               dd-3014    [000] .N...  1919.100657: block_io_start: 
252,2 WS 524288 () 32768 + 1024 be,0,4 [dd]
     kworker/0:1H-37      [000] .....  1919.112999: block_io_start: 
252,2 WS 524288 () 33792 + 1024 be,0,4 [kworker/0:1H]
               dd-3014    [000] .....  1919.145032: block_io_start: 
252,2 WS 524288 () 40960 + 1024 be,0,4 [dd]
     kworker/0:1H-37      [000] .....  1919.156677: block_io_start: 
252,2 WS 524288 () 41984 + 1024 be,0,4 [kworker/0:1H]
               dd-3014    [000] .N...  1919.188287: block_io_start: 
252,2 WS 524288 () 49152 + 1024 be,0,4 [dd]
     kworker/0:1H-37      [000] .....  1919.199869: block_io_start: 
252,2 WS 524288 () 50176 + 1024 be,0,4 [kworker/0:1H]
               dd-3014    [000] .N...  1919.233467: block_io_start: 
252,2 WS 524288 () 57344 + 1024 be,0,4 [dd]
     kworker/0:1H-37      [000] .....  1919.245487: block_io_start: 
252,2 WS 524288 () 58368 + 1024 be,0,4 [kworker/0:1H]
               dd-3014    [000] .N...  1919.281146: block_io_start: 
252,2 WS 524288 () 65536 + 1024 be,0,4 [dd]
     kworker/0:1H-37      [000] .....  1919.292812: block_io_start: 
252,2 WS 524288 () 66560 + 1024 be,0,4 [kworker/0:1H]
               dd-3014    [000] .N...  1919.326543: block_io_start: 
252,2 WS 524288 () 73728 + 1024 be,0,4 [dd]
     kworker/0:1H-37      [000] .....  1919.338412: block_io_start: 
252,2 WS 524288 () 74752 + 1024 be,0,4 [kworker/0:1H]
               dd-3014    [000] .N...  1919.374312: block_io_start: 
252,2 WS 524288 () 81920 + 1024 be,0,4 [dd]
     kworker/0:1H-37      [000] .....  1919.386481: block_io_start: 
252,2 WS 524288 () 82944 + 1024 be,0,4 [kworker/0:1H]
               dd-3014    [000] .....  1919.419795: block_io_start: 
252,2 WS 524288 () 90112 + 1024 be,0,4 [dd]
     kworker/0:1H-37      [000] .....  1919.431454: block_io_start: 
252,2 WS 524288 () 91136 + 1024 be,0,4 [kworker/0:1H]
               dd-3014    [000] .N...  1919.466208: block_io_start: 
252,2 WS 524288 () 98304 + 1024 be,0,4 [dd]

We can see block_io_start is not sequential, and test will report out of
space failure.

With this set and zone device checking removed:

diff:
diff --git a/block/blk-core.c b/block/blk-core.c
index 6ca3c45f421c..37b5dd396e22 100644
--- a/block/blk-core.c
+++ b/block/blk-core.c
@@ -746,7 +746,7 @@ void submit_bio_noacct_nocheck(struct bio *bio, bool 
split)
          * it is active, and then process them after it returned.
          */
         if (current->bio_list) {
-               if (split && !bdev_is_zoned(bio->bi_bdev))
+               if (split)
                         bio_list_add_head(&current->bio_list[0], bio);
                 else
                         bio_list_add(&current->bio_list[0], bio);

result:
              dd-612     [000] .N...    52.856395: block_io_start: 252,2 
WS 524288 () 0 + 1024 be,0,4 [dd]
     kworker/0:1H-37      [000] .....    52.869947: block_io_start: 
252,2 WS 524288 () 1024 + 1024 be,0,4 [kworker/0:1H]
     kworker/0:1H-37      [000] .....    52.880295: block_io_start: 
252,2 WS 524288 () 2048 + 1024 be,0,4 [kworker/0:1H]
     kworker/0:1H-37      [000] .....    52.890541: block_io_start: 
252,2 WS 524288 () 3072 + 1024 be,0,4 [kworker/0:1H]
     kworker/0:1H-37      [000] .....    52.900951: block_io_start: 
252,2 WS 524288 () 4096 + 1024 be,0,4 [kworker/0:1H]
     kworker/0:1H-37      [000] .....    52.911370: block_io_start: 
252,2 WS 524288 () 5120 + 1024 be,0,4 [kworker/0:1H]
     kworker/0:1H-37      [000] .....    52.922160: block_io_start: 
252,2 WS 524288 () 6144 + 1024 be,0,4 [kworker/0:1H]
     kworker/0:1H-37      [000] .....    52.932823: block_io_start: 
252,2 WS 524288 () 7168 + 1024 be,0,4 [kworker/0:1H]
               dd-612     [000] .N...    52.968469: block_io_start: 
252,2 WS 524288 () 8192 + 1024 be,0,4 [dd]
     kworker/0:1H-37      [000] .....    52.980892: block_io_start: 
252,2 WS 524288 () 9216 + 1024 be,0,4 [kworker/0:1H]
     kworker/0:1H-37      [000] .....    52.991500: block_io_start: 
252,2 WS 524288 () 10240 + 1024 be,0,4 [kworker/0:1H]
     kworker/0:1H-37      [000] .....    53.002088: block_io_start: 
252,2 WS 524288 () 11264 + 1024 be,0,4 [kworker/0:1H]
     kworker/0:1H-37      [000] .....    53.012879: block_io_start: 
252,2 WS 524288 () 12288 + 1024 be,0,4 [kworker/0:1H]
     kworker/0:1H-37      [000] .....    53.023518: block_io_start: 
252,2 WS 524288 () 13312 + 1024 be,0,4 [kworker/0:1H]
     kworker/0:1H-37      [000] .....    53.034365: block_io_start: 
252,2 WS 524288 () 14336 + 1024 be,0,4 [kworker/0:1H]
     kworker/0:1H-37      [000] .....    53.045077: block_io_start: 
252,2 WS 524288 () 15360 + 1024 be,0,4 [kworker/0:1H]
               dd-612     [000] .N...    53.082148: block_io_start: 
252,2 WS 524288 () 16384 + 1024 be,0,4 [dd]

We can see that block_io_start is sequential now.

Thanks,
Kuai

> zoned block device is bypassed in patch 14 by:
> 
> +        if (split && !bdev_is_zoned(bio->bi_bdev))
> +            bio_list_add_head(&current->bio_list[0], bio);
> 
> If I can find a reporducer for zoned block, and verify that recursive
> split can be fixed as well, I can remove the checking for zoned devices
> in the next verison.
> 
> Thanks,
> Kuai
> 
>>
>> I have not yet had the time to review this patch series but plan to take
>> a look soon.
>>
>> Thanks,
>>
>> Bart.
>> .
>>
> 
> .
> 


      reply	other threads:[~2025-09-02  8:04 UTC|newest]

Thread overview: 56+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2025-09-01  3:32 [PATCH RFC v3 00/15] block: fix disordered IO in the case recursive split Yu Kuai
2025-09-01  3:32 ` [PATCH RFC v3 01/15] block: cleanup bio_issue Yu Kuai
2025-09-01  3:43   ` Damien Le Moal
2025-09-01  6:22     ` Yu Kuai
2025-09-03 13:23   ` Christoph Hellwig
2025-09-01  3:32 ` [PATCH RFC v3 02/15] block: add QUEUE_FLAG_BIO_ISSUE Yu Kuai
2025-09-02 17:05   ` Bart Van Assche
2025-09-03  0:54     ` Yu Kuai
2025-09-03 13:24   ` Christoph Hellwig
2025-09-03 16:54     ` Yu Kuai
2025-09-04  2:44       ` Yu Kuai
2025-09-04  5:09         ` Christoph Hellwig
2025-09-01  3:32 ` [PATCH RFC v3 03/15] md: fix mssing blktrace bio split events Yu Kuai
2025-09-01  6:30   ` Damien Le Moal
2025-09-01  7:53     ` Yu Kuai
2025-09-03 13:25   ` Christoph Hellwig
2025-09-04  0:58     ` Yu Kuai
2025-09-01  3:32 ` [PATCH RFC v3 04/15] blk-crypto: fix missing processing for split bio Yu Kuai
2025-09-01  6:31   ` Damien Le Moal
2025-09-03 13:26   ` Christoph Hellwig
2025-09-01  3:32 ` [PATCH RFC v3 05/15] block: factor out a helper bio_submit_split_bioset() Yu Kuai
2025-09-01  6:34   ` Damien Le Moal
2025-09-02 17:12   ` Bart Van Assche
2025-09-03 13:28     ` Christoph Hellwig
2025-09-03 13:28   ` Christoph Hellwig
2025-09-01  3:32 ` [PATCH RFC v3 06/15] md/raid0: convert raid0_handle_discard() to use bio_submit_split_bioset() Yu Kuai
2025-09-01  6:37   ` Damien Le Moal
2025-09-01  7:57     ` Yu Kuai
2025-09-03 13:29   ` Christoph Hellwig
2025-09-01  3:32 ` [PATCH RFC v3 07/15] md/raid1: convert " Yu Kuai
2025-09-01  6:43   ` Damien Le Moal
2025-09-01  8:03     ` Yu Kuai
2025-09-03 13:30       ` Christoph Hellwig
2025-09-01  3:32 ` [PATCH RFC v3 08/15] md/raid10: add a new r10bio flag R10BIO_Returned Yu Kuai
2025-09-01  3:32 ` [PATCH RFC v3 09/15] md/raid10: convert read/write to use bio_submit_split_bioset() Yu Kuai
2025-09-01  3:32 ` [PATCH RFC v3 10/15] md/raid5: convert " Yu Kuai
2025-09-01  3:32 ` [PATCH RFC v3 11/15] md/md-linear: " Yu Kuai
2025-09-03 17:43   ` Bart Van Assche
2025-09-04  0:50     ` Yu Kuai
2025-09-01  3:32 ` [PATCH RFC v3 12/15] blk-crypto: " Yu Kuai
2025-09-03 13:31   ` Christoph Hellwig
2025-09-01  3:32 ` [PATCH RFC v3 13/15] block: skip unnecessary checks for split bio Yu Kuai
2025-09-03 13:33   ` Christoph Hellwig
2025-09-03 17:00     ` Yu Kuai
2025-09-01  3:32 ` [PATCH RFC v3 14/15] block: fix disordered IO in the case recursive split Yu Kuai
2025-09-02 17:20   ` Bart Van Assche
2025-09-03  1:00     ` Yu Kuai
2025-09-03  1:12       ` Bart Van Assche
2025-09-03  1:41         ` Yu Kuai
2025-09-03 13:34   ` Christoph Hellwig
2025-09-03 16:59     ` Yu Kuai
2025-09-03 17:52       ` Bart Van Assche
2025-09-01  3:32 ` [PATCH RFC v3 15/15] md/raid0: convert raid0_make_request() to use bio_submit_split_bioset() Yu Kuai
2025-09-01 14:09 ` [PATCH RFC v3 00/15] block: fix disordered IO in the case recursive split Bart Van Assche
2025-09-02  1:50   ` Yu Kuai
2025-09-02  8:04     ` Yu Kuai [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=850263b4-fbc7-379e-2b9f-80f602ee0e72@huaweicloud.com \
    --to=yukuai1@huaweicloud.com \
    --cc=akpm@linux-foundation.org \
    --cc=axboe@kernel.dk \
    --cc=bvanassche@acm.org \
    --cc=cgroups@vger.kernel.org \
    --cc=colyli@kernel.org \
    --cc=dlemoal@kernel.org \
    --cc=ebiggers@google.com \
    --cc=hare@suse.de \
    --cc=hch@infradead.org \
    --cc=johnny.chenyi@huawei.com \
    --cc=josef@toxicpanda.com \
    --cc=kmo@daterainc.com \
    --cc=linux-block@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-raid@vger.kernel.org \
    --cc=neil@brown.name \
    --cc=satyat@google.com \
    --cc=song@kernel.org \
    --cc=tieren@fnnas.com \
    --cc=tj@kernel.org \
    --cc=yangerkun@huawei.com \
    --cc=yi.zhang@huawei.com \
    --cc=yukuai3@huawei.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).