From: Kevin Wolf <kwolf@redhat.com>
To: luzhipeng <luzhipeng@cestc.cn>
Cc: qemu-block@nongnu.org, Alberto Garcia <berto@igalia.com>,
Hanna Reitz <hreitz@redhat.com>,
qemu-devel@nongnu.org
Subject: Re: [PATCH resend] block: add single-check guard in throttle_group_restart_queue to address race with schedule_next_request
Date: Tue, 2 Dec 2025 14:16:22 +0100 [thread overview]
Message-ID: <aS7mpt3GHFkNfWZT@redhat.com> (raw)
In-Reply-To: <20251117011045.1232-1-luzhipeng@cestc.cn>
Am 17.11.2025 um 02:10 hat luzhipeng geschrieben:
> A race condition exists between throttle_group_restart_queue() and
> schedule_next_request(): when multiple ThrottleGroupMembers in the same
> throttle group are assigned to different IOThreads, concurrent execution
> can cause schedule_next_request() to re-arm a throttle timer while
> throttle_group_restart_queue() is being called (e.g., from a timer
> callback or external restart). This violates the assumption that no
> timer is pending upon entry to throttle_group_restart_queue(), triggering
> an assertion failure and causing QEMU to abort.
>
> This patch replaces the assert with a single early-return check:
> if the timer for the given direction is already pending, the function
> returns immediately. This prevents duplicate coroutine scheduling and
> avoids crashes under race conditions, without altering the core
> (non-thread-safe) throttle group logic.
>
> For details, see: https://gitlab.com/qemu-project/qemu/-/issues/3194
>
> Signed-off-by: luzhipeng <luzhipeng@cestc.cn>
> ---
> block/throttle-groups.c | 7 +++----
> 1 file changed, 3 insertions(+), 4 deletions(-)
>
> diff --git a/block/throttle-groups.c b/block/throttle-groups.c
> index 66fdce9a90..9dcc6b4923 100644
> --- a/block/throttle-groups.c
> +++ b/block/throttle-groups.c
> @@ -430,15 +430,14 @@ static void throttle_group_restart_queue(ThrottleGroupMember *tgm,
> ThrottleDirection direction)
> {
> Coroutine *co;
> + if (timer_pending(tgm->throttle_timers.timers[direction])) {
> + return;
> + }
> RestartData *rd = g_new0(RestartData, 1);
>
> rd->tgm = tgm;
> rd->direction = direction;
>
> - /* This function is called when a timer is fired or when
> - * throttle_group_restart_tgm() is called. Either way, there can
> - * be no timer pending on this tgm at this point */
> - assert(!timer_pending(tgm->throttle_timers.timers[direction]));
>
> qatomic_inc(&tgm->restart_pending);
I'm not really familiar with the finer details of this throttling code,
but this doesn't look like a proper fix for a race to me. If it was
possible for a timer to be scheduled between the caller and here, then
it can certainly also be scheduled between your new check and whatever
relies on the condition after it.
If I understand the commit message of 25b8e4db7f3 correctly, the idea is
that you never have a timer pending that wouldn't actually restart a
request. If the timer is rescheduled from a different thread right after
this check, we'll already resume all of the requests now and the timer
will still be pending without actually having any work to do.
So it looks to me like we'll need some extended locking (tg->lock is
already held by callers of throttle_group_schedule_timer(), but only for
a short duration in throttle_group_restart_queue_entry() instead of
during the whole process), or just give up on the invariant, but then we
don't have to return early either.
Berto, any thoughts?
Kevin
prev parent reply other threads:[~2025-12-02 13:17 UTC|newest]
Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-11-17 1:10 [PATCH resend] block: add single-check guard in throttle_group_restart_queue to address race with schedule_next_request luzhipeng
2025-12-02 13:16 ` Kevin Wolf [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=aS7mpt3GHFkNfWZT@redhat.com \
--to=kwolf@redhat.com \
--cc=berto@igalia.com \
--cc=hreitz@redhat.com \
--cc=luzhipeng@cestc.cn \
--cc=qemu-block@nongnu.org \
--cc=qemu-devel@nongnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).