From: Ming Lei <ming.lei@redhat.com>
To: Jens Axboe <axboe@kernel.dk>
Cc: Yi Zhang <yi.zhang@redhat.com>,
fengnanchang@gmail.com, linux-block <linux-block@vger.kernel.org>,
Shinichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Subject: Re: [bug report][bisected] kernel BUG at lib/list_debug.c:32! triggered by blktests nvme/049
Date: Wed, 14 Jan 2026 23:20:42 +0800 [thread overview]
Message-ID: <aWe0SjcDRQZM2t2G@fedora> (raw)
In-Reply-To: <9ae067ba-d0b6-49ac-9e96-01d23348261f@kernel.dk>
On Wed, Jan 14, 2026 at 07:58:54AM -0700, Jens Axboe wrote:
> On 1/14/26 7:43 AM, Jens Axboe wrote:
> > On 1/14/26 7:11 AM, Ming Lei wrote:
> >> On Wed, Jan 14, 2026 at 01:58:03PM +0800, Yi Zhang wrote:
> >>> On Thu, Jan 8, 2026 at 2:39?PM Yi Zhang <yi.zhang@redhat.com> wrote:
> >>>>
> >>>> On Thu, Jan 8, 2026 at 12:48?AM Jens Axboe <axboe@kernel.dk> wrote:
> >>>>>
> >>>>> On 1/7/26 9:39 AM, Yi Zhang wrote:
> >>>>>> Hi
> >>>>>> The following issue[2] was triggered by blktests nvme/059 and it's
> >>>>>
> >>>>> nvme/049 presumably?
> >>>>>
> >>>> Yes.
> >>>>
> >>>>>> 100% reproduced with commit[1]. Please help check it and let me know
> >>>>>> if you need any info/test for it.
> >>>>>> Seems it's one regression, I will try to test with the latest
> >>>>>> linux-block/for-next and also bisect it tomorrow.
> >>>>>
> >>>>> Doesn't reproduce for me on the current tree, but nothing since:
> >>>>>
> >>>>>> commit 5ee81d4ae52ec4e9206efb4c1b06e269407aba11
> >>>>>> Merge: 29cefd61e0c6 fcf463b92a08
> >>>>>> Author: Jens Axboe <axboe@kernel.dk>
> >>>>>> Date: Tue Jan 6 05:48:07 2026 -0700
> >>>>>>
> >>>>>> Merge branch 'for-7.0/blk-pvec' into for-next
> >>>>>
> >>>>> should have impacted that. So please do bisect.
> >>>>
> >>>> Hi Jens
> >>>> The issue seems was introduced from below commit.
> >>>> and the issue cannot be reproduced after reverting this commit.
> >>>
> >>> The issue still can be reproduced on the latest linux-block/for-next
> >>
> >> Hi Yi,
> >>
> >> Can you try the following patch?
> >>
> >>
> >> diff --git a/drivers/nvme/host/ioctl.c b/drivers/nvme/host/ioctl.c
> >> index a9c097dacad6..7b0e62b8322b 100644
> >> --- a/drivers/nvme/host/ioctl.c
> >> +++ b/drivers/nvme/host/ioctl.c
> >> @@ -425,14 +425,23 @@ static enum rq_end_io_ret nvme_uring_cmd_end_io(struct request *req,
> >> pdu->result = le64_to_cpu(nvme_req(req)->result.u64);
> >>
> >> /*
> >> - * IOPOLL could potentially complete this request directly, but
> >> - * if multiple rings are polling on the same queue, then it's possible
> >> - * for one ring to find completions for another ring. Punting the
> >> - * completion via task_work will always direct it to the right
> >> - * location, rather than potentially complete requests for ringA
> >> - * under iopoll invocations from ringB.
> >> + * For IOPOLL, complete the request inline. The request's io_kiocb
> >> + * uses a union for io_task_work and iopoll_node, so scheduling
> >> + * task_work would corrupt the iopoll_list while the request is
> >> + * still on it. io_uring_cmd_done() handles IOPOLL by setting
> >> + * iopoll_completed rather than scheduling task_work.
> >> + *
> >> + * For non-IOPOLL, complete via task_work to ensure we run in the
> >> + * submitter's context and handling multiple rings is safe.
> >> */
> >> - io_uring_cmd_do_in_task_lazy(ioucmd, nvme_uring_task_cb);
> >> + if (blk_rq_is_poll(req)) {
> >> + if (pdu->bio)
> >> + blk_rq_unmap_user(pdu->bio);
> >> + io_uring_cmd_done32(ioucmd, pdu->status, pdu->result, 0);
> >> + } else {
> >> + io_uring_cmd_do_in_task_lazy(ioucmd, nvme_uring_task_cb);
> >> + }
> >> +
> >> return RQ_END_IO_FREE;
> >> }
> >>
> >
> > Ah yes that should fix it, the task_work addition will conflict with
> > the list addition. Don't think it's safe though, which is why I made
> > them all use task_work previously. Let me fix it in the IOPOLL patch
> > instead.
>
> This should be better:
>
> diff --git a/include/linux/io_uring_types.h b/include/linux/io_uring_types.h
> index dd084a55bed8..1fa8d829cbac 100644
> --- a/include/linux/io_uring_types.h
> +++ b/include/linux/io_uring_types.h
> @@ -719,13 +719,10 @@ struct io_kiocb {
> atomic_t refs;
> bool cancel_seq_set;
>
> - /*
> - * IOPOLL doesn't use task_work, so use the ->iopoll_node list
> - * entry to manage pending iopoll requests.
> - */
> union {
> struct io_task_work io_task_work;
> - struct list_head iopoll_node;
> + /* For IOPOLL setup queues, with hybrid polling */
> + u64 iopoll_start;
> };
>
> union {
> @@ -734,8 +731,8 @@ struct io_kiocb {
> * poll
> */
> struct hlist_node hash_node;
> - /* For IOPOLL setup queues, with hybrid polling */
> - u64 iopoll_start;
> + /* IOPOLL completion handling */
> + struct list_head iopoll_node;
> /* for private io_kiocb freeing */
> struct rcu_head rcu_head;
> };
This way looks better, just `req->iopoll_start` needs to read to local
variable first in io_uring_hybrid_poll().
Thanks,
Ming
next prev parent reply other threads:[~2026-01-14 15:20 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-01-07 16:39 [bug report] kernel BUG at lib/list_debug.c:32! triggered by blktests nvme/049 Yi Zhang
2026-01-07 16:48 ` Jens Axboe
2026-01-08 6:39 ` Yi Zhang
2026-01-14 5:58 ` [bug report][bisected] " Yi Zhang
2026-01-14 9:40 ` Alexander Atanasov
2026-01-14 12:43 ` Christoph Hellwig
2026-01-14 14:11 ` Ming Lei
2026-01-14 14:43 ` Jens Axboe
2026-01-14 14:58 ` Jens Axboe
2026-01-14 15:20 ` Ming Lei [this message]
2026-01-14 15:26 ` Jens Axboe
2026-01-16 11:54 ` Alexander Atanasov
2026-01-16 12:41 ` Ming Lei
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=aWe0SjcDRQZM2t2G@fedora \
--to=ming.lei@redhat.com \
--cc=axboe@kernel.dk \
--cc=fengnanchang@gmail.com \
--cc=linux-block@vger.kernel.org \
--cc=shinichiro.kawasaki@wdc.com \
--cc=yi.zhang@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.