public inbox for linux-raid@vger.kernel.org
 help / color / mirror / Atom feed
From: "Yu Kuai" <yukuai@fnnas.com>
To: "Josh Hunt" <johunt@akamai.com>, <song@kernel.org>,
	 <linan122@huawei.com>, <linux-raid@vger.kernel.org>,
	<yukuai@fnnas.com>
Cc: <ncroxon@redhat.com>, <stable@vger.kernel.org>
Subject: Re: [PATCH v3] md/raid10: fix deadlock with check operation and nowait requests
Date: Wed, 4 Mar 2026 10:18:40 +0800	[thread overview]
Message-ID: <54af459e-2671-417b-bc9f-2b13f111f749@fnnas.com> (raw)
In-Reply-To: <20260303005619.1352958-1-johunt@akamai.com>

在 2026/3/3 8:56, Josh Hunt 写道:

> When an array check is running it will raise the barrier at which point
> normal requests will become blocked and increment the nr_pending value to
> signal there is work pending inside of wait_barrier(). NOWAIT requests
> do not block and so will return immediately with an error, and additionally
> do not increment nr_pending in wait_barrier(). Upstream change
> 43806c3d5b9b ("raid10: cleanup memleak at raid10_make_request") added a
> call to raid_end_bio_io() to fix a memory leak when NOWAIT requests hit
> this condition. raid_end_bio_io() eventually calls allow_barrier() and
> it will unconditionally do an atomic_dec_and_test(&conf->nr_pending) even
> though the corresponding increment on nr_pending didn't happen in the
> NOWAIT case.
>
> This can be easily seen by starting a check operation while an application is
> doing nowait IO on the same array. This results in a deadlocked state due to
> nr_pending value underflowing and so the md resync thread gets stuck waiting
> for nr_pending to == 0.
>
> Output of r10conf state of the array when we hit this condition:
>
> crash> struct r10conf.barrier,nr_pending,nr_waiting,nr_queued <addr of r10conf>
> 	barrier = 1,
>          nr_pending = {
>            counter = -41
>          },
>          nr_waiting = 15,
>          nr_queued = 0,
>
> Example of md_sync thread stuck waiting on raise_barrier() and other requests
> stuck in wait_barrier():
>
> md1_resync
> [<0>] raise_barrier+0xce/0x1c0
> [<0>] raid10_sync_request+0x1ca/0x1ed0
> [<0>] md_do_sync+0x779/0x1110
> [<0>] md_thread+0x90/0x160
> [<0>] kthread+0xbe/0xf0
> [<0>] ret_from_fork+0x34/0x50
> [<0>] ret_from_fork_asm+0x1a/0x30
>
> kworker/u1040:2+flush-253:4
> [<0>] wait_barrier+0x1de/0x220
> [<0>] regular_request_wait+0x30/0x180
> [<0>] raid10_make_request+0x261/0x1000
> [<0>] md_handle_request+0x13b/0x230
> [<0>] __submit_bio+0x107/0x1f0
> [<0>] submit_bio_noacct_nocheck+0x16f/0x390
> [<0>] ext4_io_submit+0x24/0x40
> [<0>] ext4_do_writepages+0x254/0xc80
> [<0>] ext4_writepages+0x84/0x120
> [<0>] do_writepages+0x7a/0x260
> [<0>] __writeback_single_inode+0x3d/0x300
> [<0>] writeback_sb_inodes+0x1dd/0x470
> [<0>] __writeback_inodes_wb+0x4c/0xe0
> [<0>] wb_writeback+0x18b/0x2d0
> [<0>] wb_workfn+0x2a1/0x400
> [<0>] process_one_work+0x149/0x330
> [<0>] worker_thread+0x2d2/0x410
> [<0>] kthread+0xbe/0xf0
> [<0>] ret_from_fork+0x34/0x50
> [<0>] ret_from_fork_asm+0x1a/0x30
>
> Fixes: 43806c3d5b9b ("raid10: cleanup memleak at raid10_make_request")
> Cc: stable@vger.kernel.org
> Signed-off-by: Josh Hunt <johunt@akamai.com>
> ---
> v3:
>    * Call free_r10bio() as per Yu Kuai's suggestion
> ---
>   drivers/md/raid10.c | 4 ++--
>   1 file changed, 2 insertions(+), 2 deletions(-)

There are some checkpatch errors and warnings, applied to md-7.0 with some
changes to commit message.

> diff --git a/drivers/md/raid10.c b/drivers/md/raid10.c
> index 9debb20cf129..b4892c5d571c 100644
> --- a/drivers/md/raid10.c
> +++ b/drivers/md/raid10.c
> @@ -1184,7 +1184,7 @@ static void raid10_read_request(struct mddev *mddev, struct bio *bio,
>   	}
>   
>   	if (!regular_request_wait(mddev, conf, bio, r10_bio->sectors)) {
> -		raid_end_bio_io(r10_bio);
> +		free_r10bio(r10_bio);
>   		return;
>   	}
>   
> @@ -1372,7 +1372,7 @@ static void raid10_write_request(struct mddev *mddev, struct bio *bio,
>   
>   	sectors = r10_bio->sectors;
>   	if (!regular_request_wait(mddev, conf, bio, sectors)) {
> -		raid_end_bio_io(r10_bio);
> +		free_r10bio(r10_bio);
>   		return;
>   	}
>   

-- 
Thansk,
Kuai

      reply	other threads:[~2026-03-04  2:18 UTC|newest]

Thread overview: 2+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-03-03  0:56 [PATCH v3] md/raid10: fix deadlock with check operation and nowait requests Josh Hunt
2026-03-04  2:18 ` Yu Kuai [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=54af459e-2671-417b-bc9f-2b13f111f749@fnnas.com \
    --to=yukuai@fnnas.com \
    --cc=johunt@akamai.com \
    --cc=linan122@huawei.com \
    --cc=linux-raid@vger.kernel.org \
    --cc=ncroxon@redhat.com \
    --cc=song@kernel.org \
    --cc=stable@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox