public inbox for linux-block@vger.kernel.org
 help / color / mirror / Atom feed
From: Ming Lei <ming.lei@redhat.com>
To: Jens Axboe <axboe@kernel.dk>
Cc: Yi Zhang <yi.zhang@redhat.com>,
	fengnanchang@gmail.com, linux-block <linux-block@vger.kernel.org>,
	Shinichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Subject: Re: [bug report][bisected] kernel BUG at lib/list_debug.c:32! triggered by blktests nvme/049
Date: Wed, 14 Jan 2026 23:20:42 +0800	[thread overview]
Message-ID: <aWe0SjcDRQZM2t2G@fedora> (raw)
In-Reply-To: <9ae067ba-d0b6-49ac-9e96-01d23348261f@kernel.dk>

On Wed, Jan 14, 2026 at 07:58:54AM -0700, Jens Axboe wrote:
> On 1/14/26 7:43 AM, Jens Axboe wrote:
> > On 1/14/26 7:11 AM, Ming Lei wrote:
> >> On Wed, Jan 14, 2026 at 01:58:03PM +0800, Yi Zhang wrote:
> >>> On Thu, Jan 8, 2026 at 2:39?PM Yi Zhang <yi.zhang@redhat.com> wrote:
> >>>>
> >>>> On Thu, Jan 8, 2026 at 12:48?AM Jens Axboe <axboe@kernel.dk> wrote:
> >>>>>
> >>>>> On 1/7/26 9:39 AM, Yi Zhang wrote:
> >>>>>> Hi
> >>>>>> The following issue[2] was triggered by blktests nvme/059 and it's
> >>>>>
> >>>>> nvme/049 presumably?
> >>>>>
> >>>> Yes.
> >>>>
> >>>>>> 100% reproduced with commit[1]. Please help check it and let me know
> >>>>>> if you need any info/test for it.
> >>>>>> Seems it's one regression, I will try to test with the latest
> >>>>>> linux-block/for-next and also bisect it tomorrow.
> >>>>>
> >>>>> Doesn't reproduce for me on the current tree, but nothing since:
> >>>>>
> >>>>>> commit 5ee81d4ae52ec4e9206efb4c1b06e269407aba11
> >>>>>> Merge: 29cefd61e0c6 fcf463b92a08
> >>>>>> Author: Jens Axboe <axboe@kernel.dk>
> >>>>>> Date:   Tue Jan 6 05:48:07 2026 -0700
> >>>>>>
> >>>>>>     Merge branch 'for-7.0/blk-pvec' into for-next
> >>>>>
> >>>>> should have impacted that. So please do bisect.
> >>>>
> >>>> Hi Jens
> >>>> The issue seems was introduced from below commit.
> >>>> and the issue cannot be reproduced after reverting this commit.
> >>>
> >>> The issue still can be reproduced on the latest linux-block/for-next
> >>
> >> Hi Yi,
> >>
> >> Can you try the following patch?
> >>
> >>
> >> diff --git a/drivers/nvme/host/ioctl.c b/drivers/nvme/host/ioctl.c
> >> index a9c097dacad6..7b0e62b8322b 100644
> >> --- a/drivers/nvme/host/ioctl.c
> >> +++ b/drivers/nvme/host/ioctl.c
> >> @@ -425,14 +425,23 @@ static enum rq_end_io_ret nvme_uring_cmd_end_io(struct request *req,
> >>  	pdu->result = le64_to_cpu(nvme_req(req)->result.u64);
> >>  
> >>  	/*
> >> -	 * IOPOLL could potentially complete this request directly, but
> >> -	 * if multiple rings are polling on the same queue, then it's possible
> >> -	 * for one ring to find completions for another ring. Punting the
> >> -	 * completion via task_work will always direct it to the right
> >> -	 * location, rather than potentially complete requests for ringA
> >> -	 * under iopoll invocations from ringB.
> >> +	 * For IOPOLL, complete the request inline. The request's io_kiocb
> >> +	 * uses a union for io_task_work and iopoll_node, so scheduling
> >> +	 * task_work would corrupt the iopoll_list while the request is
> >> +	 * still on it. io_uring_cmd_done() handles IOPOLL by setting
> >> +	 * iopoll_completed rather than scheduling task_work.
> >> +	 *
> >> +	 * For non-IOPOLL, complete via task_work to ensure we run in the
> >> +	 * submitter's context and handling multiple rings is safe.
> >>  	 */
> >> -	io_uring_cmd_do_in_task_lazy(ioucmd, nvme_uring_task_cb);
> >> +	if (blk_rq_is_poll(req)) {
> >> +		if (pdu->bio)
> >> +			blk_rq_unmap_user(pdu->bio);
> >> +		io_uring_cmd_done32(ioucmd, pdu->status, pdu->result, 0);
> >> +	} else {
> >> +		io_uring_cmd_do_in_task_lazy(ioucmd, nvme_uring_task_cb);
> >> +	}
> >> +
> >>  	return RQ_END_IO_FREE;
> >>  }
> >>  
> > 
> > Ah yes that should fix it, the task_work addition will conflict with
> > the list addition. Don't think it's safe though, which is why I made
> > them all use task_work previously. Let me fix it in the IOPOLL patch
> > instead.
> 
> This should be better:
> 
> diff --git a/include/linux/io_uring_types.h b/include/linux/io_uring_types.h
> index dd084a55bed8..1fa8d829cbac 100644
> --- a/include/linux/io_uring_types.h
> +++ b/include/linux/io_uring_types.h
> @@ -719,13 +719,10 @@ struct io_kiocb {
>  	atomic_t			refs;
>  	bool				cancel_seq_set;
>  
> -	/*
> -	 * IOPOLL doesn't use task_work, so use the ->iopoll_node list
> -	 * entry to manage pending iopoll requests.
> -	 */
>  	union {
>  		struct io_task_work	io_task_work;
> -		struct list_head	iopoll_node;
> +		/* For IOPOLL setup queues, with hybrid polling */
> +		u64                     iopoll_start;
>  	};
>  
>  	union {
> @@ -734,8 +731,8 @@ struct io_kiocb {
>  		 * poll
>  		 */
>  		struct hlist_node	hash_node;
> -		/* For IOPOLL setup queues, with hybrid polling */
> -		u64                     iopoll_start;
> +		/* IOPOLL completion handling */
> +		struct list_head	iopoll_node;
>  		/* for private io_kiocb freeing */
>  		struct rcu_head		rcu_head;
>  	};

This way looks better, just `req->iopoll_start` needs to read to local
variable first in io_uring_hybrid_poll().


Thanks,
Ming


  reply	other threads:[~2026-01-14 15:20 UTC|newest]

Thread overview: 13+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-01-07 16:39 [bug report] kernel BUG at lib/list_debug.c:32! triggered by blktests nvme/049 Yi Zhang
2026-01-07 16:48 ` Jens Axboe
2026-01-08  6:39   ` Yi Zhang
2026-01-14  5:58     ` [bug report][bisected] " Yi Zhang
2026-01-14  9:40       ` Alexander Atanasov
2026-01-14 12:43         ` Christoph Hellwig
2026-01-14 14:11       ` Ming Lei
2026-01-14 14:43         ` Jens Axboe
2026-01-14 14:58           ` Jens Axboe
2026-01-14 15:20             ` Ming Lei [this message]
2026-01-14 15:26               ` Jens Axboe
2026-01-16 11:54         ` Alexander Atanasov
2026-01-16 12:41           ` Ming Lei

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=aWe0SjcDRQZM2t2G@fedora \
    --to=ming.lei@redhat.com \
    --cc=axboe@kernel.dk \
    --cc=fengnanchang@gmail.com \
    --cc=linux-block@vger.kernel.org \
    --cc=shinichiro.kawasaki@wdc.com \
    --cc=yi.zhang@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox