[PATCH v6] loop: Wait for I/O completion in __loop_clr_fd()

[PATCH v6] loop: Wait for I/O completion in __loop_clr_fd()

[Bart Van Assche]

From: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>

Syzbot reports a NULL pointer dereference in lo_rw_aio(). This crash
probably happens because commit b15e4310633f ("jfs: Set the lbmDONE flag at
the end of lbmIODone"), merged in the 7.1 merge window, lets the JFS log
code report a write as finished while it is still in flight. JFS then tears
down and releases the loop device with that write still sitting in the loop
workqueue.

Although the root cause is in JFS, improve the loop driver by waiting in
__loop_clr_fd() for ongoing I/O. Release &disk->open_mutex temporarily
because otherwise a deadlock would be triggered.

This patch introduces a behavior change: openers racing with teardown now
get -ENXIO instead of waiting.

Link: https://lkml.kernel.org/r/fbb3edda-f108-4e5b-acf2-266f043f8125@I-love.SAKURA.ne.jp
Link: https://syzkaller.appspot.com/bug?extid=cd8a9a308e879a4e2c28
Link: https://syzkaller.appspot.com/bug?extid=bc273027d5643e48e5b3
Cc: Christoph Hellwig <hch@lst.de>
Cc: Edward Adam Davis <eadavis@qq.com>
Cc: Dave Kleikamp <shaggy@kernel.org>
Cc: jfs-discussion@lists.sourceforge.net
Assisted-by: Gemini Pro
Assisted-by: Claude Opus
Signed-off-by: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
[ bvanassche: rewrote description, eliminated .post_release() callback,
  changed a synchronize_rcu() call into quiescing + unquiescing the
  request queue, reordered code in __loop_clr_fd() and rewrote several
  comments ]
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
---

Changes from v5 to v6:
- Drop the invasive core block-layer changes (adding .post_release to
  struct block_device_operations and calling it from block/bdev.c). Keep
  all changes completely self-contained within drivers/block/loop.c.
- Instead of deferring teardown to a core post_release callback, temporarily
  unlock and re-acquire &disk->open_mutex inside __loop_clr_fd().
- Add WARN_ON_ONCE(lo->lo_state != Lo_rundown) checks around the unlock/lock
  window to ensure racing openers get -ENXIO via lo_open() rather than seeing
  a partially torn-down device.
- Replace drain_workqueue() with flush_workqueue() to prevent warnings about
  queueing work on a draining workqueue (__WQ_DRAINING).
- Replace synchronize_rcu() with blk_mq_quiesce_queue() + blk_mq_unquiesce_queue()
  to properly synchronize against in-flight loop_queue_rq() instances using
  standard block-layer primitives.
- Add an extensive comment documenting the exact 3-step circular locking
  scenario caught by syzbot (disk->open_mutex -> wq_completion -> VFS/subsystem
  locks -> disk->open_mutex).
- Identify and document the probable trigger in JFS (commit b15e4310633f,
  "jfs: Set the lbmDONE flag at the end of lbmIODone") in the commit message.
- Add Cc tags for JFS maintainers (Dave Kleikamp, jfs-discussion) and Christoph
  Hellwig. Update AI assistance tags to conform to Linux kernel attribution guidelines.

Changes from v4 to v5:
- Introduce a new .post_release() callback in struct block_device_operations
  called from bdev_release() after dropping disk->open_mutex.
- Invoke __loop_clr_fd() from the new loop_post_release() callback instead of
  calling it synchronously under disk->open_mutex in loop_release().

Changes from v3 to v4:
- Add blk_mq_freeze_queue() / blk_mq_unfreeze_queue() in __loop_clr_fd() to
  wait for pending asynchronous direct I/O (lo_rw_aio returning -EIOCBQUEUED)
  which drain_workqueue() alone does not wait for.

Changes from v2 to v3:
- Address the syzbot circular locking report (bug extid=2f62807dc3239b8f584e)
  caused by calling drain_workqueue() while holding disk->open_mutex.

Changes from v1 to v2:
- Remove the custom SRCU domain (loop_io_srcu).
- Replace srcu_read_lock() / synchronize_srcu() with rcu_read_lock() and
  synchronize_rcu().

 drivers/block/loop.c | 72 ++++++++++++++++++++++++++++++++++++++------
 1 file changed, 63 insertions(+), 9 deletions(-)

diff --git a/drivers/block/loop.c b/drivers/block/loop.c
index 758c20678bf6..94a62b92b5a1 100644
--- a/drivers/block/loop.c
+++ b/drivers/block/loop.c
@@ -1138,11 +1138,65 @@ static int loop_configure(struct loop_device *lo, blk_mode_t mode,
 
 static void __loop_clr_fd(struct loop_device *lo)
 {
+	struct gendisk *disk = lo->lo_disk;
 	struct queue_limits lim;
 	struct file *filp;
 	gfp_t gfp = lo->old_gfp_mask;
 	int err;
 
+	/*
+	 * lo->lo_state must be Lo_rundown before we release disk->open_mutex
+	 * to ensure that lo_open() fails while disk->open_mutex is unlocked.
+	 */
+	WARN_ON_ONCE(lo->lo_state != Lo_rundown);
+
+	/*
+	 * Unlock disk->open_mutex to prevent circular locking. Without the
+	 * mutex_unlock() call below, circular locking would be triggered as
+	 * follows when an autoclearing loop device is closed by its last opener:
+	 * 1. open_mutex → Workqueue Completion / Queue Freeze: Waiting for
+	 *    pending I/O requires draining the loop device's execution
+	 *    workqueue (flush_workqueue(lo->workqueue)) and/or freezing the
+	 *    request queue (blk_mq_freeze_queue()). Draining the workqueue
+	 *    acquires its completion lockdep tracking context
+	 *    ((wq_completion)loopX). If called while disk->open_mutex is held,
+	 *    lockdep records the lock acquisition ordering: disk->open_mutex ⟶
+	 *    (wq_completion)loopX
+	 * 2. Workqueue Completion → Subsystem/VFS Locks: In-flight work items
+	 *    executed by worker threads on lo->workqueue (loop_process_work()→
+	 *    lo_rw_aio()) perform file operations on the underlying backing
+	 *    file. These operations acquire VFS and filesystem locks (such as
+	 *    inode locks, file-specific locks, kernfs mutexes like &of->mutex,
+	 *    or seq_file locks).
+	 * 3. Subsystem/VFS Locks → open_mutex: Other code paths in the kernel
+	 *    (for instance, partition scanners, power management/hibernation
+	 *    resume routines such as resume_store() → swsusp_check(), or sysfs
+	 *    operations) acquire subsystem/filesystem locks first and then call
+	 *    bdev_open(), which acquires disk->open_mutex.
+	 *
+	 * This creates a circular locking dependency: disk->open_mutex ⟶
+	 * (wq_completion)loopX ⟶ VFS / Subsystem Locks ⟶ disk->open_mutex
+	 *
+	 * Syzbot caught this exact circular locking scenario. See also
+	 * https://syzkaller.appspot.com/bug?extid=2f62807dc3239b8f584e.
+	 */
+	mutex_unlock(&disk->open_mutex);
+
+	/* Wait for all outstanding I/O requests to finish. */
+	blk_mq_unfreeze_queue(lo->lo_queue, blk_mq_freeze_queue(lo->lo_queue));
+	/*
+	 * Wait until all ongoing loop_queue_rq() calls have finished. Freezing
+	 * the request queue may finish before loop_queue_rq() has finished.
+	 */
+	blk_mq_quiesce_queue(lo->lo_queue);
+	blk_mq_unquiesce_queue(lo->lo_queue);
+	/* Wait for ongoing loop_handle_cmd() calls to finish. */
+	flush_workqueue(lo->workqueue);
+
+	/* Reacquire disk->open_mutex and clear *lo. */
+	mutex_lock(&disk->open_mutex);
+	WARN_ON_ONCE(lo->lo_state != Lo_rundown);
+
 	spin_lock_irq(&lo->lo_lock);
 	filp = lo->lo_backing_file;
 	lo->lo_backing_file = NULL;
@@ -1153,16 +1207,16 @@ static void __loop_clr_fd(struct loop_device *lo)
 	lo->lo_sizelimit = 0;
 	memset(lo->lo_file_name, 0, LO_NAME_SIZE);
 
-	/*
-	 * Reset the block size to the default.
-	 *
-	 * No queue freezing needed because this is called from the final
-	 * ->release call only, so there can't be any outstanding I/O.
-	 */
+	/* Reset the block size to the default. */
 	lim = queue_limits_start_update(lo->lo_queue);
 	lim.logical_block_size = SECTOR_SIZE;
 	lim.physical_block_size = SECTOR_SIZE;
 	lim.io_min = SECTOR_SIZE;
+	/*
+	 * No queue freeze around the queue limits update below because any I/O
+	 * that races with this update fails in loop_queue_rq(), so a
+	 * transiently inconsistent view of the limits is harmless.
+	 */
 	queue_limits_commit_update(lo->lo_queue, &lim);
 
 	invalidate_disk(lo->lo_disk);
@@ -1202,9 +1256,9 @@ static void __loop_clr_fd(struct loop_device *lo)
 	mutex_unlock(&lo->lo_mutex);
 
 	/*
-	 * Need not hold lo_mutex to fput backing file. Calling fput holding
-	 * lo_mutex triggers a circular lock dependency possibility warning as
-	 * fput can take open_mutex which is usually taken before lo_mutex.
+	 * fput() calls file->f_op->release() asynchronously. Hence, calling
+	 * fput() while holding disk->open_mutex won't trigger recursive locking
+	 * for disk->open_mutex in case of stacked loop devices.
 	 */
 	fput(filp);
 }

Re: [PATCH v6] loop: Wait for I/O completion in __loop_clr_fd()

[Tetsuo Handa]

On 2026/10/06 6:06, Bart Van Assche wrote:
> From: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
> 
> Syzbot reports a NULL pointer dereference in lo_rw_aio(). This crash
> probably happens because commit b15e4310633f ("jfs: Set the lbmDONE flag at
> the end of lbmIODone"), merged in the 7.1 merge window, lets the JFS log
> code report a write as finished while it is still in flight. JFS then tears
> down and releases the loop device with that write still sitting in the loop
> workqueue.
> 
> Although the root cause is in JFS, improve the loop driver by waiting in
> __loop_clr_fd() for ongoing I/O. Release &disk->open_mutex temporarily
> because otherwise a deadlock would be triggered.
> 
> This patch introduces a behavior change: openers racing with teardown now
> get -ENXIO instead of waiting.
> 
Are you sure that commit b15e4310633f ("jfs: Set the lbmDONE flag at the end of
lbmIODone") is the root cause for JFS case? If yes, it is a step forward fixing
this regression in JFS side.

Was your concern "releasing and reacquiring disk->open_mutex from __loop_clr_fd()
seems risky to me"
( https://lkml.kernel.org/r/e5518546-701e-4d3c-a077-18e34aa8c188@I-love.SAKURA.ne.jp )
dissolved?

sashiko commented at
https://sashiko.dev/#/patchset/fb1e0f16346994e32b479e4be596436b2a614b03.1791234123.git.bvanassche%40acm.org
that blk_mq_quiesce_queue() / flush_workqueue() are redundant if we call
blk_mq_freeze_queue() first.

Even without releasing and reacquiring disk->open_mutex, I think we can silence
lockdep by removing synchronize_rcu() / drain_workqueue() because these are
optional as long as we call blk_mq_freeze_queue(). But is calling
blk_mq_freeze_queue() with disk->open_mutex held guaranteed to be safe?

Re: [PATCH v6] loop: Wait for I/O completion in __loop_clr_fd()

[Bart Van Assche]

On 10/6/26 5:01 AM, Tetsuo Handa wrote:
> On 2026/10/06 6:06, Bart Van Assche wrote:
>> From: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
>>
>> Syzbot reports a NULL pointer dereference in lo_rw_aio(). This crash
>> probably happens because commit b15e4310633f ("jfs: Set the lbmDONE flag at
>> the end of lbmIODone"), merged in the 7.1 merge window, lets the JFS log
>> code report a write as finished while it is still in flight. JFS then tears
>> down and releases the loop device with that write still sitting in the loop
>> workqueue.
>>
>> Although the root cause is in JFS, improve the loop driver by waiting in
>> __loop_clr_fd() for ongoing I/O. Release &disk->open_mutex temporarily
>> because otherwise a deadlock would be triggered.
>>
>> This patch introduces a behavior change: openers racing with teardown now
>> get -ENXIO instead of waiting.
>>
> Are you sure that commit b15e4310633f ("jfs: Set the lbmDONE flag at the end of
> lbmIODone") is the root cause for JFS case? If yes, it is a step forward fixing
> this regression in JFS side.

I'm not completely sure. But all the console logs that I checked on the
Syzbot dashboard include JFS error messages. I think that's a strong
indication.

> Was your concern "releasing and reacquiring disk->open_mutex from __loop_clr_fd()
> seems risky to me"
> ( https://lkml.kernel.org/r/e5518546-701e-4d3c-a077-18e34aa8c188@I-love.SAKURA.ne.jp )
> dissolved?

After I wrote that comment I noticed that lo->lo_state == Lo_rundown
while __loop_clr_fd() releases disk->open_mutex. lo_open() fails if
lo->lo_state == Lo_rundown. Hence, bdev_open() will fail while
__loop_clr_fd() releases disk->open_mutex. This is a behavior change
but I hope that this is acceptable.

> sashiko commented at
> https://sashiko.dev/#/patchset/fb1e0f16346994e32b479e4be596436b2a614b03.1791234123.git.bvanassche%40acm.org
> that blk_mq_quiesce_queue() / flush_workqueue() are redundant if we call
> blk_mq_freeze_queue() first.

Sashiko is a great and very useful tool but in this case Sashiko is
wrong. From queue_zoned_qd1_writes_store():

	memflags = blk_mq_freeze_queue(q);
	blk_mq_quiesce_queue(q);
	[ ... ]
	blk_mq_unquiesce_queue(q);
	blk_mq_unfreeze_queue(q, memflags);

If calling blk_mq_quiesce_queue() after blk_mq_freeze_queue() would be
redundant, the blk_mq_quiesce_queue() call would never have been
introduced in that code path. There are more occurrences of the above
pattern in the block layer, e.g. in blk_throtl_init().

Calling flush_workqueue() is necessary because calling
blk_mq_quiesce_queue() doesn't guarantee that all queued work has
finished.

> Even without releasing and reacquiring disk->open_mutex, I think we can silence
> lockdep by removing synchronize_rcu() / drain_workqueue() because these are
> optional as long as we call blk_mq_freeze_queue(). But is calling
> blk_mq_freeze_queue() with disk->open_mutex held guaranteed to be safe?

My patch unlocks disk->open_mutex before blk_mq_freeze_queue() is
called, isn't it?

Thanks,

Bart.

Re: [PATCH v6] loop: Wait for I/O completion in __loop_clr_fd()

[Christoph Hellwig]

On Mon, Oct 05, 2026 at 02:06:02PM -0700, Bart Van Assche wrote:
> From: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
> 
> Syzbot reports a NULL pointer dereference in lo_rw_aio(). This crash
> probably happens because commit b15e4310633f ("jfs: Set the lbmDONE flag at
> the end of lbmIODone"), merged in the 7.1 merge window, lets the JFS log
> code report a write as finished while it is still in flight. JFS then tears
> down and releases the loop device with that write still sitting in the loop
> workqueue.

Why do we go offtrack messing with loop instead of investigating the
file system misbehavior?

> Although the root cause is in JFS, improve the loop driver by waiting in
> __loop_clr_fd() for ongoing I/O. Release &disk->open_mutex temporarily
> because otherwise a deadlock would be triggered.

We can't just drop the held lock.

Re: [PATCH v6] loop: Wait for I/O completion in __loop_clr_fd()

[Tetsuo Handa]

On 2026/10/07 22:08, Christoph Hellwig wrote:
> On Mon, Oct 05, 2026 at 02:06:02PM -0700, Bart Van Assche wrote:
>> From: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
>>
>> Syzbot reports a NULL pointer dereference in lo_rw_aio(). This crash
>> probably happens because commit b15e4310633f ("jfs: Set the lbmDONE flag at
>> the end of lbmIODone"), merged in the 7.1 merge window, lets the JFS log
>> code report a write as finished while it is still in flight. JFS then tears
>> down and releases the loop device with that write still sitting in the loop
>> workqueue.
> 
> Why do we go offtrack messing with loop instead of investigating the
> file system misbehavior?

How can we investigate the file system misbehavior?
Earlier versions of this patch
( https://lkml.kernel.org/r/30ccdd3c-6353-4a0f-bdc7-230ec3bf4765@I-love.SAKURA.ne.jp )
included debug code for emitting BUG: message, which is intended for helping filesystem
developers identify the caller of an I/O request that failed to wait for completion,
but I got no response. Please show us viable approaches.

> 
>> Although the root cause is in JFS, improve the loop driver by waiting in
>> __loop_clr_fd() for ongoing I/O. Release &disk->open_mutex temporarily
>> because otherwise a deadlock would be triggered.
> 
> We can't just drop the held lock.
> 

That's why I proposed a new callback which you are refusing to add.

[PATCH] loop: Detect write requests that failed to wait for completion.

[Tetsuo Handa]

This is a debug patch which will be reverted after we observed what is
happening. This patch does not attempt to fix the problem itself.

syzbot is reporting NULL pointer dereference in lo_rw_aio() [1][2].
An analysis by the Gemini AI collaborator [3] considers that this problem
is caused by a timing shift primarily exposed by commit 65565ca5f99b
("block: unify the synchronous bi_end_io callbacks"), along with helper
refactorings like commit 92c3737a2473 ("block: add a bio_submit_or_kill
helper").

But due to difficulty of reproducing this race, discussion about what is
happening and how to fix this problem is stalling. Also, we haven't
identified how many filesystems are subjected to this problem. Therefore,
we need to proceed to debug printk() approach, for syzbot cannot find a
reproducer for this problem.

Another analysis by the Gemini AI collaborator found possibility of
sync_blockdev() not waiting for write requests due to skipping
__filemap_fdatawait_range() when filemap_fdatawrite_range() returned -EIO.

  Step 1: A test process is created and calls unshare().
  Step 2: That process extracts a crafted JFS image to a memfd storage.
  Step 3: That process binds that image to a loop device.
  Step 4: That process mounts that loop device as read-write.
  Step 5: That process issues some buffered write requests.
  Step 6: That process is killed before writeback for Step 5 starts.
  Step 7: Automatic unmount operation starts due to teardown of a mount
          namespace.
  Step 8: jfs_umount() calls filemap_write_and_wait() (directly or via
          sync_blockdev()).
  Step 9: filemap_fdatawrite_range() returns -EIO due to filesystem errors
          after several submit_bio() have been already issued.
  Step 10: jfs_umount() completes without calling
           __filemap_fdatawait_range() which should wait for completion of
           I/O requests issued by Step 5 due to Step 9.
  Step 11: fput() is called from __loop_clr_fd() from lo_release(), and
           destruction of "struct file" starts after returning from
           lo_release().
  Step 12: The in-flight I/O request issued by Step 9 is finally processed
           by the loop backend kworker. However, since the backing "struct
           file" (and its associated structures) is already under
           destruction, the kworker attempts to dereference now-invalid
           fields, leading to WARN_ON_ONCE().

In order to check whether the steps described above is actually happening,
add debug code to the loop module for reporting I/O requests that are
processed after __loop_clr_fd() started. Note that this debug code is
enabled only if CONFIG_KCOV=y, for this check is a waste of computation
resources for almost all users.

Link: https://syzkaller.appspot.com/bug?extid=cd8a9a308e879a4e2c28 [1]
Link: https://syzkaller.appspot.com/bug?extid=bc273027d5643e48e5b3 [2]
Link: https://lkml.kernel.org/r/fbb3edda-f108-4e5b-acf2-266f043f8125@I-love.SAKURA.ne.jp [3]
Assisted-by: Gemini-Pro
Signed-off-by: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
---
Sashiko, does this analysis look correct for you?

 drivers/block/loop.c | 30 +++++++++++++++++++++++++++++-
 1 file changed, 29 insertions(+), 1 deletion(-)

diff --git a/drivers/block/loop.c b/drivers/block/loop.c
index 758c20678bf6..09b72138543d 100644
--- a/drivers/block/loop.c
+++ b/drivers/block/loop.c
@@ -86,8 +86,27 @@ struct loop_cmd {
 	struct bio_vec *bvec;
 	struct cgroup_subsys_state *blkcg_css;
 	struct cgroup_subsys_state *memcg_css;
+#ifdef CONFIG_KCOV
+	unsigned long stack_entries[30];
+	int stack_nr;
+	pid_t pid;
+	char comm[TASK_COMM_LEN];
+#endif
 };
 
+static void loop_check_io_race(struct loop_device *lo, struct loop_cmd *cmd)
+{
+#ifdef CONFIG_KCOV
+	if (unlikely(READ_ONCE(lo->lo_state) == Lo_rundown &&
+		     disk_openers(lo->lo_disk) == 0)) {
+		pr_err("BUG: %s/%u is doing I/O request on loop%d in Lo_rundown state.\n",
+		       cmd->comm, cmd->pid, lo->lo_number);
+		printk("Call Trace:\n");
+		stack_trace_print(cmd->stack_entries, cmd->stack_nr, 4);
+	}
+#endif
+}
+
 #define LOOP_IDLE_WORKER_TIMEOUT (60 * HZ)
 #define LOOP_DEFAULT_HW_Q_DEPTH 128
 
@@ -1879,10 +1898,18 @@ static blk_status_t loop_queue_rq(struct blk_mq_hw_ctx *hctx,
 	struct loop_cmd *cmd = blk_mq_rq_to_pdu(rq);
 	struct loop_device *lo = rq->q->queuedata;
 
+#ifdef CONFIG_KCOV
+	cmd->stack_nr = stack_trace_save(cmd->stack_entries, ARRAY_SIZE(cmd->stack_entries), 0);
+	cmd->pid = current->pid;
+	get_task_comm(cmd->comm, current);
+#endif
+
 	blk_mq_start_request(rq);
 
-	if (data_race(READ_ONCE(lo->lo_state)) != Lo_bound)
+	if (data_race(READ_ONCE(lo->lo_state)) != Lo_bound) {
+		loop_check_io_race(lo, cmd);
 		return BLK_STS_IOERR;
+	}
 
 	switch (req_op(rq)) {
 	case REQ_OP_FLUSH:
@@ -1925,6 +1952,7 @@ static void loop_handle_cmd(struct loop_cmd *cmd)
 	int ret = 0;
 	struct mem_cgroup *old_memcg = NULL;
 
+	loop_check_io_race(lo, cmd);
 	if (write && (lo->lo_flags & LO_FLAGS_READ_ONLY)) {
 		ret = -EIO;
 		goto failed;
-- 
2.56.0

[PATCH v2] loop: Detect write requests that failed to wait for completion.

[Tetsuo Handa]

This is a debug patch which will be reverted after we observed what is
happening. This patch does not attempt to fix the problem itself.

syzbot is reporting NULL pointer dereference in lo_rw_aio() [1][2].
An analysis by the Gemini AI collaborator [3] considers that this problem
is caused by a timing shift primarily exposed by commit 65565ca5f99b
("block: unify the synchronous bi_end_io callbacks"), along with helper
refactorings like commit 92c3737a2473 ("block: add a bio_submit_or_kill
helper").

But due to difficulty of reproducing this race, discussion about what is
happening and how to fix this problem is stalling. Also, we haven't
identified how many filesystems are subjected to this problem. Therefore,
we need to proceed to debug printk() approach, for syzbot cannot find a
reproducer for this problem.

Another analysis by the Gemini AI collaborator found possibility of
sync_blockdev() not waiting for write requests due to skipping
__filemap_fdatawait_range() when filemap_fdatawrite_range() returned -EIO.

  Step 1: A test process is created and calls unshare().
  Step 2: That process extracts a crafted JFS image to a memfd storage.
  Step 3: That process binds that image to a loop device.
  Step 4: That process mounts that loop device as read-write.
  Step 5: That process issues some buffered write requests.
  Step 6: That process is killed before writeback for Step 5 starts.
  Step 7: Automatic unmount operation starts due to teardown of a mount
          namespace.
  Step 8: jfs_umount() calls filemap_write_and_wait() (directly or via
          sync_blockdev()).
  Step 9: filemap_fdatawrite_range() returns -EIO due to filesystem errors
          after several submit_bio() have been already issued.
  Step 10: jfs_umount() completes without calling
           __filemap_fdatawait_range() which should wait for completion of
           I/O requests issued by Step 5 due to Step 9.
  Step 11: fput() is called from __loop_clr_fd() from lo_release(), and
           destruction of "struct file" starts after returning from
           lo_release().
  Step 12: The in-flight I/O request issued by Step 9 is finally processed
           by the loop backend kworker. However, since the backing "struct
           file" (and its associated structures) is already under
           destruction, the kworker attempts to dereference now-invalid
           fields, leading to WARN_ON_ONCE().

In order to check whether the steps described above is actually happening,
add debug code to the loop module for reporting I/O requests that are
processed after __loop_clr_fd() started. Note that this debug code is
enabled only if CONFIG_KCOV=y, for this check is a waste of computation
resources for almost all users.

Link: https://syzkaller.appspot.com/bug?extid=cd8a9a308e879a4e2c28 [1]
Link: https://syzkaller.appspot.com/bug?extid=bc273027d5643e48e5b3 [2]
Link: https://lkml.kernel.org/r/fbb3edda-f108-4e5b-acf2-266f043f8125@I-love.SAKURA.ne.jp [3]
Assisted-by: Gemini-Pro
Signed-off-by: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
---
Changes in v2:
  - Include linux/stacktrace.h file.
  - Sashiko's review spotted that lo->lo_state is already Lo_unbound
    by the time fput() is called. Therefore, don't check lo->lo_state
    value, and check from only loop_handle_cmd() path. Since syzbot is
    reporting that kworker is finding inflight I/O, this change should
    still be able to catch [1] and [2] above.
  - I would like to introduce a kernel config option like
    CONFIG_DEBUG_AID_FOR_SYZBOT for gating debug code used by syzbot.
    But that will be a different discussion.
  - Not using ratelimit for reporting, for there should be only few
    if any when loop_handle_cmd() finds one. Not checking from 
    loop_queue_rq() path should reduce the amount of output a lot.

 drivers/block/loop.c | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/drivers/block/loop.c b/drivers/block/loop.c
index 758c20678bf6..11fbe7691367 100644
--- a/drivers/block/loop.c
+++ b/drivers/block/loop.c
@@ -36,6 +36,7 @@
 #include <linux/blk-mq.h>
 #include <linux/spinlock.h>
 #include <uapi/linux/loop.h>
+#include <linux/stacktrace.h>
 
 /* Possible states of device */
 enum {
@@ -86,6 +87,12 @@ struct loop_cmd {
 	struct bio_vec *bvec;
 	struct cgroup_subsys_state *blkcg_css;
 	struct cgroup_subsys_state *memcg_css;
+#ifdef CONFIG_KCOV
+	unsigned long stack_entries[30];
+	int stack_nr;
+	pid_t pid;
+	char comm[TASK_COMM_LEN];
+#endif
 };
 
 #define LOOP_IDLE_WORKER_TIMEOUT (60 * HZ)
@@ -1879,6 +1886,12 @@ static blk_status_t loop_queue_rq(struct blk_mq_hw_ctx *hctx,
 	struct loop_cmd *cmd = blk_mq_rq_to_pdu(rq);
 	struct loop_device *lo = rq->q->queuedata;
 
+#ifdef CONFIG_KCOV
+	cmd->stack_nr = stack_trace_save(cmd->stack_entries, ARRAY_SIZE(cmd->stack_entries), 0);
+	cmd->pid = current->pid;
+	get_task_comm(cmd->comm, current);
+#endif
+
 	blk_mq_start_request(rq);
 
 	if (data_race(READ_ONCE(lo->lo_state)) != Lo_bound)
@@ -1925,6 +1938,14 @@ static void loop_handle_cmd(struct loop_cmd *cmd)
 	int ret = 0;
 	struct mem_cgroup *old_memcg = NULL;
 
+#ifdef CONFIG_KCOV
+	if (unlikely(disk_openers(lo->lo_disk) == 0)) {
+		pr_err("BUG: loop%d has no users, but backend found I/O issued by %s/%u.\n",
+		       lo->lo_number, cmd->comm, cmd->pid);
+		printk("Call Trace:\n");
+		stack_trace_print(cmd->stack_entries, cmd->stack_nr, 4);
+	}
+#endif
 	if (write && (lo->lo_flags & LO_FLAGS_READ_ONLY)) {
 		ret = -EIO;
 		goto failed;
-- 
2.56.0

[PATCH v3] loop: Detect write requests that failed to wait for completion.

[Tetsuo Handa]

This is a debug patch which will be reverted after we observed what is
happening. This patch does not attempt to fix the problem itself.

syzbot is reporting NULL pointer dereference in lo_rw_aio() [1][2].
An analysis by the Gemini AI collaborator [3] considers that this problem
is caused by a timing shift primarily exposed by commit 65565ca5f99b
("block: unify the synchronous bi_end_io callbacks"), along with helper
refactorings like commit 92c3737a2473 ("block: add a bio_submit_or_kill
helper").

But due to difficulty of reproducing this race, discussion about what is
happening and how to fix this problem is stalling. Also, we haven't
identified how many filesystems are subjected to this problem. Therefore,
we need to proceed to debug printk() approach, for syzbot cannot find a
reproducer for this problem.

Another analysis by the Gemini AI collaborator found possibility of
sync_blockdev() not waiting for write requests due to skipping
__filemap_fdatawait_range() when filemap_fdatawrite_range() returned -EIO.

  Step 1: A test process is created and calls unshare().
  Step 2: That process extracts a crafted JFS image to a memfd storage.
  Step 3: That process binds that image to a loop device.
  Step 4: That process mounts that loop device as read-write.
  Step 5: That process issues some buffered write requests.
  Step 6: That process is killed before writeback for Step 5 starts.
  Step 7: Automatic unmount operation starts due to teardown of a mount
          namespace.
  Step 8: jfs_umount() calls filemap_write_and_wait() (directly or via
          sync_blockdev()).
  Step 9: filemap_fdatawrite_range() returns -EIO due to filesystem errors
          after several submit_bio() have been already issued.
  Step 10: jfs_umount() completes without calling
           __filemap_fdatawait_range() which should wait for completion of
           I/O requests issued by Step 5 due to Step 9.
  Step 11: fput() is called from __loop_clr_fd() from lo_release(), and
           destruction of "struct file" starts after returning from
           lo_release().
  Step 12: The in-flight I/O request issued by Step 9 is finally processed
           by the loop backend kworker. However, since the backing "struct
           file" (and its associated structures) is already under
           destruction, the kworker attempts to dereference now-invalid
           fields, leading to WARN_ON_ONCE().

In order to check whether the steps described above is actually happening,
add debug code to the loop module for reporting I/O requests that are
processed after __loop_clr_fd() started. Note that this debug code is
enabled only if CONFIG_KCOV=y, for this check is a waste of computation
resources for almost all users.

Link: https://syzkaller.appspot.com/bug?extid=cd8a9a308e879a4e2c28 [1]
Link: https://syzkaller.appspot.com/bug?extid=bc273027d5643e48e5b3 [2]
Link: https://lkml.kernel.org/r/fbb3edda-f108-4e5b-acf2-266f043f8125@I-love.SAKURA.ne.jp [3]
Assisted-by: Gemini-Pro
Signed-off-by: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
---
Changes in v3:
  - Use "loop device is no longer bound" and "there is no users" as
    the conditions to report.

Changes in v2:
  - Include linux/stacktrace.h file.
  - Sashiko's review spotted that lo->lo_state is already Lo_unbound
    by the time fput() is called. Therefore, don't check lo->lo_state
    value, and check from only loop_handle_cmd() path. Since syzbot is
    reporting that kworker is finding inflight I/O, this change should
    still be able to catch [1] and [2] above.
  - I would like to introduce a kernel config option like
    CONFIG_DEBUG_AID_FOR_SYZBOT for gating debug code used by syzbot.
    But that will be a different discussion.
  - Not using ratelimit for reporting, for there should be only few
    if any when loop_handle_cmd() finds one. Not checking from 
    loop_queue_rq() path should reduce the amount of  output a lot.

 drivers/block/loop.c | 29 +++++++++++++++++++++++++++++
 1 file changed, 29 insertions(+)

diff --git a/drivers/block/loop.c b/drivers/block/loop.c
index 758c20678bf6..191807599bf7 100644
--- a/drivers/block/loop.c
+++ b/drivers/block/loop.c
@@ -36,6 +36,7 @@
 #include <linux/blk-mq.h>
 #include <linux/spinlock.h>
 #include <uapi/linux/loop.h>
+#include <linux/stacktrace.h>
 
 /* Possible states of device */
 enum {
@@ -86,6 +87,12 @@ struct loop_cmd {
 	struct bio_vec *bvec;
 	struct cgroup_subsys_state *blkcg_css;
 	struct cgroup_subsys_state *memcg_css;
+#if defined(CONFIG_KCOV) && defined(CONFIG_STACKTRACE)
+	unsigned long stack_entries[30];
+	int stack_nr;
+	pid_t pid;
+	char comm[TASK_COMM_LEN];
+#endif
 };
 
 #define LOOP_IDLE_WORKER_TIMEOUT (60 * HZ)
@@ -1910,6 +1917,13 @@ static blk_status_t loop_queue_rq(struct blk_mq_hw_ctx *hctx,
 #endif
 	}
 #endif
+
+#if defined(CONFIG_KCOV) && defined(CONFIG_STACKTRACE)
+	cmd->stack_nr = stack_trace_save(cmd->stack_entries, ARRAY_SIZE(cmd->stack_entries), 0);
+	cmd->pid = current->pid;
+	get_task_comm(cmd->comm, current);
+#endif
+
 	loop_queue_work(lo, cmd);
 
 	return BLK_STS_OK;
@@ -1925,6 +1939,21 @@ static void loop_handle_cmd(struct loop_cmd *cmd)
 	int ret = 0;
 	struct mem_cgroup *old_memcg = NULL;
 
+#if defined(CONFIG_KCOV) && defined(CONFIG_STACKTRACE)
+	/*
+	 * This request was queued after checking lo->lo_state == Lo_bound
+	 * in loop_queue_rq(). The state should remain unchanged unless
+	 * __loop_clr_fd() by error starts before blk_mq_complete_request()
+	 * is called.
+	 */
+	if (unlikely(data_race(READ_ONCE(lo->lo_state)) != Lo_bound &&
+		     disk_openers(lo->lo_disk) == 0)) {
+		pr_err("BUG: found I/O issued by %s/%u but loop%d is no longer bound.\n",
+		       cmd->comm, cmd->pid, lo->lo_number);
+		printk("Call Trace:\n");
+		stack_trace_print(cmd->stack_entries, cmd->stack_nr, 4);
+	}
+#endif
 	if (write && (lo->lo_flags & LO_FLAGS_READ_ONLY)) {
 		ret = -EIO;
 		goto failed;
-- 
2.56.0

[syzbot ci] Re: loop: Detect write requests that failed to wait for completion.

[syzbot ci]

syzbot ci has tested the following series

[v3] loop: Detect write requests that failed to wait for completion.
https://lore.kernel.org/all/cc2910d3-3113-44d4-94ef-0c8d365fac9c@I-love.SAKURA.ne.jp
* [PATCH v3] loop: Detect write requests that failed to wait for completion.

and found the following issues:
* BUG: found I/O issued by syz.NUM.NUM/NUM but loop0 is no longer bound.
* BUG: found I/O issued by syz.NUM.NUM/NUM but loop2 is no longer bound.

Full report is available here:
https://ci.syzbot.org/series/d3a0b036-fd1e-44c6-8869-386ea1642385

***

BUG: found I/O issued by syz.NUM.NUM/NUM but loop0 is no longer bound.

tree:      axboe
URL:       https://kernel.googlesource.com/pub/scm/linux/kernel/git/axboe/linux.git
base:      4da93acb4f2ed06640255ad98801d0f0c734ff07
arch:      amd64
compiler:  Debian clang version 22.1.8 (++20260613092233+e80beda6e255-1~exp1~20260613092250.77), Debian LLD 22.1.8
config:    https://ci.syzbot.org/builds/a23afb97-88bb-4fba-8885-d3062b5890e2/config
syz repro: https://ci.syzbot.org/findings/c9ad5ff9-784f-4528-805f-7971e0359600/syz_repro

BUG: found I/O issued by syz.0.760/7647 but loop0 is no longer bound.
Call Trace:
     loop_queue_rq+0x232/0xc90 drivers/block/loop.c:1922
     __blk_mq_issue_directly block/blk-mq.c:2708 [inline]
     blk_mq_try_issue_directly+0x3f4/0x880 block/blk-mq.c:2768
     blk_mq_submit_bio+0x21de/0x2bb0 block/blk-mq.c:3220
     __submit_bio_noacct_mq block/blk-core.c:756 [inline]
     submit_bio_noacct_nocheck+0x2f4/0xa40 block/blk-core.c:790
     bh_submit fs/buffer.c:1146 [inline]
     __bh_read fs/buffer.c:2943 [inline]
     bh_readahead include/linux/buffer_head.h:437 [inline]
     __breadahead+0x15c/0x1f0 fs/buffer.c:1359
     nilfs_search_super_root+0x2be/0x1530 fs/nilfs2/recovery.c:891
     load_nilfs+0x363/0xfb0 fs/nilfs2/the_nilfs.c:259
     nilfs_fill_super+0x1a0/0x4e0 fs/nilfs2/super.c:1073
     nilfs_get_tree+0x491/0x850 fs/nilfs2/super.c:1226
     vfs_get_tree+0x92/0x2a0 fs/super.c:1932
     fc_mount fs/namespace.c:1198 [inline]
     do_new_mount_fc fs/namespace.c:3772 [inline]
     do_new_mount+0x319/0xdc0 fs/namespace.c:3848
     do_mount fs/namespace.c:4181 [inline]
     __do_sys_mount fs/namespace.c:4397 [inline]
     __se_sys_mount+0x31d/0x420 fs/namespace.c:4374
     do_syscall_x64 arch/x86/entry/syscall_64.c:61 [inline]
     do_syscall_64+0x166/0x520 arch/x86/entry/syscall_64.c:84
     entry_SYSCALL_64_after_hwframe+0x77/0x7f
BUG: found I/O issued by syz.0.760/7647 but loop0 is no longer bound.
Call Trace:
     loop_queue_rq+0x232/0xc90 drivers/block/loop.c:1922
     __blk_mq_issue_directly block/blk-mq.c:2708 [inline]
     blk_mq_try_issue_directly+0x3f4/0x880 block/blk-mq.c:2768
     blk_mq_submit_bio+0x21de/0x2bb0 block/blk-mq.c:3220
     __submit_bio_noacct_mq block/blk-core.c:756 [inline]
     submit_bio_noacct_nocheck+0x2f4/0xa40 block/blk-core.c:790
     bh_submit fs/buffer.c:1146 [inline]
     __bh_read fs/buffer.c:2943 [inline]
     bh_readahead include/linux/buffer_head.h:437 [inline]
     __breadahead+0x15c/0x1f0 fs/buffer.c:1359
     nilfs_search_super_root+0x2be/0x1530 fs/nilfs2/recovery.c:891
     load_nilfs+0x363/0xfb0 fs/nilfs2/the_nilfs.c:259
     nilfs_fill_super+0x1a0/0x4e0 fs/nilfs2/super.c:1073
     nilfs_get_tree+0x491/0x850 fs/nilfs2/super.c:1226
     vfs_get_tree+0x92/0x2a0 fs/super.c:1932
     fc_mount fs/namespace.c:1198 [inline]
     do_new_mount_fc fs/namespace.c:3772 [inline]
     do_new_mount+0x319/0xdc0 fs/namespace.c:3848
     do_mount fs/namespace.c:4181 [inline]
     __do_sys_mount fs/namespace.c:4397 [inline]
     __se_sys_mount+0x31d/0x420 fs/namespace.c:4374
     do_syscall_x64 arch/x86/entry/syscall_64.c:61 [inline]
     do_syscall_64+0x166/0x520 arch/x86/entry/syscall_64.c:84
     entry_SYSCALL_64_after_hwframe+0x77/0x7f


***

BUG: found I/O issued by syz.NUM.NUM/NUM but loop2 is no longer bound.

tree:      axboe
URL:       https://kernel.googlesource.com/pub/scm/linux/kernel/git/axboe/linux.git
base:      4da93acb4f2ed06640255ad98801d0f0c734ff07
arch:      amd64
compiler:  Debian clang version 22.1.8 (++20260613092233+e80beda6e255-1~exp1~20260613092250.77), Debian LLD 22.1.8
config:    https://ci.syzbot.org/builds/a23afb97-88bb-4fba-8885-d3062b5890e2/config
syz repro: https://ci.syzbot.org/findings/bad9ae98-7887-4209-8226-7e84bb070faa/syz_repro

BUG: found I/O issued by syz.2.866/7955 but loop2 is no longer bound.
Call Trace:
     loop_queue_rq+0x232/0xc90 drivers/block/loop.c:1922
     __blk_mq_issue_directly block/blk-mq.c:2708 [inline]
     blk_mq_try_issue_directly+0x3f4/0x880 block/blk-mq.c:2768
     blk_mq_submit_bio+0x21de/0x2bb0 block/blk-mq.c:3220
     __submit_bio_noacct_mq block/blk-core.c:756 [inline]
     submit_bio_noacct_nocheck+0x2f4/0xa40 block/blk-core.c:790
     bh_submit fs/buffer.c:1146 [inline]
     __bh_read fs/buffer.c:2943 [inline]
     bh_readahead include/linux/buffer_head.h:437 [inline]
     __breadahead+0x15c/0x1f0 fs/buffer.c:1359
     nilfs_search_super_root+0x2be/0x1530 fs/nilfs2/recovery.c:891
     load_nilfs+0x363/0xfb0 fs/nilfs2/the_nilfs.c:259
     nilfs_fill_super+0x1a0/0x4e0 fs/nilfs2/super.c:1073
     nilfs_get_tree+0x491/0x850 fs/nilfs2/super.c:1226
     vfs_get_tree+0x92/0x2a0 fs/super.c:1932
     fc_mount fs/namespace.c:1198 [inline]
     do_new_mount_fc fs/namespace.c:3772 [inline]
     do_new_mount+0x319/0xdc0 fs/namespace.c:3848
     do_mount fs/namespace.c:4181 [inline]
     __do_sys_mount fs/namespace.c:4397 [inline]
     __se_sys_mount+0x31d/0x420 fs/namespace.c:4374
     do_syscall_x64 arch/x86/entry/syscall_64.c:61 [inline]
     do_syscall_64+0x166/0x520 arch/x86/entry/syscall_64.c:84
     entry_SYSCALL_64_after_hwframe+0x77/0x7f
BUG: found I/O issued by syz.2.866/7955 but loop2 is no longer bound.
Call Trace:
     loop_queue_rq+0x232/0xc90 drivers/block/loop.c:1922
     __blk_mq_issue_directly block/blk-mq.c:2708 [inline]
     blk_mq_try_issue_directly+0x3f4/0x880 block/blk-mq.c:2768
     blk_mq_submit_bio+0x21de/0x2bb0 block/blk-mq.c:3220
     __submit_bio_noacct_mq block/blk-core.c:756 [inline]
     submit_bio_noacct_nocheck+0x2f4/0xa40 block/blk-core.c:790
     bh_submit fs/buffer.c:1146 [inline]
     __bh_read fs/buffer.c:2943 [inline]
     bh_readahead include/linux/buffer_head.h:437 [inline]
     __breadahead+0x15c/0x1f0 fs/buffer.c:1359
     nilfs_search_super_root+0x2be/0x1530 fs/nilfs2/recovery.c:891
     load_nilfs+0x363/0xfb0 fs/nilfs2/the_nilfs.c:259
     nilfs_fill_super+0x1a0/0x4e0 fs/nilfs2/super.c:1073
     nilfs_get_tree+0x491/0x850 fs/nilfs2/super.c:1226
     vfs_get_tree+0x92/0x2a0 fs/super.c:1932
     fc_mount fs/namespace.c:1198 [inline]
     do_new_mount_fc fs/namespace.c:3772 [inline]
     do_new_mount+0x319/0xdc0 fs/namespace.c:3848
     do_mount fs/namespace.c:4181 [inline]
     __do_sys_mount fs/namespace.c:4397 [inline]
     __se_sys_mount+0x31d/0x420 fs/namespace.c:4374
     do_syscall_x64 arch/x86/entry/syscall_64.c:61 [inline]
     do_syscall_64+0x166/0x520 arch/x86/entry/syscall_64.c:84
     entry_SYSCALL_64_after_hwframe+0x77/0x7f
BUG: found I/O issued by syz.2.866/7955 but loop2 is no longer bound.
Call Trace:
     loop_queue_rq+0x232/0xc90 drivers/block/loop.c:1922
     __blk_mq_issue_directly block/blk-mq.c:2708 [inline]
     blk_mq_try_issue_directly+0x3f4/0x880 block/blk-mq.c:2768
     blk_mq_submit_bio+0x21de/0x2bb0 block/blk-mq.c:3220
     __submit_bio_noacct_mq block/blk-core.c:756 [inline]
     submit_bio_noacct_nocheck+0x2f4/0xa40 block/blk-core.c:790
     bh_submit fs/buffer.c:1146 [inline]
     __bh_read fs/buffer.c:2943 [inline]
     bh_readahead include/linux/buffer_head.h:437 [inline]
     __breadahead+0x15c/0x1f0 fs/buffer.c:1359
     nilfs_search_super_root+0x2be/0x1530 fs/nilfs2/recovery.c:891
     load_nilfs+0x363/0xfb0 fs/nilfs2/the_nilfs.c:259
     nilfs_fill_super+0x1a0/0x4e0 fs/nilfs2/super.c:1073
     nilfs_get_tree+0x491/0x850 fs/nilfs2/super.c:1226
     vfs_get_tree+0x92/0x2a0 fs/super.c:1932
     fc_mount fs/namespace.c:1198 [inline]
     do_new_mount_fc fs/namespace.c:3772 [inline]
     do_new_mount+0x319/0xdc0 fs/namespace.c:3848
     do_mount fs/namespace.c:4181 [inline]
     __do_sys_mount fs/namespace.c:4397 [inline]
     __se_sys_mount+0x31d/0x420 fs/namespace.c:4374
     do_syscall_x64 arch/x86/entry/syscall_64.c:61 [inline]
     do_syscall_64+0x166/0x520 arch/x86/entry/syscall_64.c:84
     entry_SYSCALL_64_after_hwframe+0x77/0x7f
BUG: found I/O issued by syz.2.866/7955 but loop2 is no longer bound.
Call Trace:
     loop_queue_rq+0x232/0xc90 drivers/block/loop.c:1922
     __blk_mq_issue_directly block/blk-mq.c:2708 [inline]
     blk_mq_try_issue_directly+0x3f4/0x880 block/blk-mq.c:2768
     blk_mq_submit_bio+0x21de/0x2bb0 block/blk-mq.c:3220
     __submit_bio_noacct_mq block/blk-core.c:756 [inline]
     submit_bio_noacct_nocheck+0x2f4/0xa40 block/blk-core.c:790
     bh_submit fs/buffer.c:1146 [inline]
     __bh_read fs/buffer.c:2943 [inline]
     bh_readahead include/linux/buffer_head.h:437 [inline]
     __breadahead+0x15c/0x1f0 fs/buffer.c:1359
     nilfs_search_super_root+0x2be/0x1530 fs/nilfs2/recovery.c:891
     load_nilfs+0x363/0xfb0 fs/nilfs2/the_nilfs.c:259
     nilfs_fill_super+0x1a0/0x4e0 fs/nilfs2/super.c:1073
     nilfs_get_tree+0x491/0x850 fs/nilfs2/super.c:1226
     vfs_get_tree+0x92/0x2a0 fs/super.c:1932
     fc_mount fs/namespace.c:1198 [inline]
     do_new_mount_fc fs/namespace.c:3772 [inline]
     do_new_mount+0x319/0xdc0 fs/namespace.c:3848
     do_mount fs/namespace.c:4181 [inline]
     __do_sys_mount fs/namespace.c:4397 [inline]
     __se_sys_mount+0x31d/0x420 fs/namespace.c:4374
     do_syscall_x64 arch/x86/entry/syscall_64.c:61 [inline]
     do_syscall_64+0x166/0x520 arch/x86/entry/syscall_64.c:84
     entry_SYSCALL_64_after_hwframe+0x77/0x7f
BUG: found I/O issued by syz.2.866/7955 but loop2 is no longer bound.
Call Trace:
     loop_queue_rq+0x232/0xc90 drivers/block/loop.c:1922
     __blk_mq_issue_directly block/blk-mq.c:2708 [inline]
     blk_mq_try_issue_directly+0x3f4/0x880 block/blk-mq.c:2768
     blk_mq_submit_bio+0x21de/0x2bb0 block/blk-mq.c:3220
     __submit_bio_noacct_mq block/blk-core.c:756 [inline]
     submit_bio_noacct_nocheck+0x2f4/0xa40 block/blk-core.c:790
     bh_submit fs/buffer.c:1146 [inline]
     __bh_read fs/buffer.c:2943 [inline]
     bh_readahead include/linux/buffer_head.h:437 [inline]
     __breadahead+0x15c/0x1f0 fs/buffer.c:1359
     nilfs_search_super_root+0x2be/0x1530 fs/nilfs2/recovery.c:891
     load_nilfs+0x363/0xfb0 fs/nilfs2/the_nilfs.c:259
     nilfs_fill_super+0x1a0/0x4e0 fs/nilfs2/super.c:1073
     nilfs_get_tree+0x491/0x850 fs/nilfs2/super.c:1226
     vfs_get_tree+0x92/0x2a0 fs/super.c:1932
     fc_mount fs/namespace.c:1198 [inline]
     do_new_mount_fc fs/namespace.c:3772 [inline]
     do_new_mount+0x319/0xdc0 fs/namespace.c:3848
     do_mount fs/namespace.c:4181 [inline]
     __do_sys_mount fs/namespace.c:4397 [inline]
     __se_sys_mount+0x31d/0x420 fs/namespace.c:4374
     do_syscall_x64 arch/x86/entry/syscall_64.c:61 [inline]
     do_syscall_64+0x166/0x520 arch/x86/entry/syscall_64.c:84
     entry_SYSCALL_64_after_hwframe+0x77/0x7f
BUG: found I/O issued by syz.2.866/7955 but loop2 is no longer bound.
Call Trace:
     loop_queue_rq+0x232/0xc90 drivers/block/loop.c:1922
     __blk_mq_issue_directly block/blk-mq.c:2708 [inline]
     blk_mq_try_issue_directly+0x3f4/0x880 block/blk-mq.c:2768
     blk_mq_submit_bio+0x21de/0x2bb0 block/blk-mq.c:3220
     __submit_bio_noacct_mq block/blk-core.c:756 [inline]
     submit_bio_noacct_nocheck+0x2f4/0xa40 block/blk-core.c:790
     bh_submit fs/buffer.c:1146 [inline]
     __bh_read fs/buffer.c:2943 [inline]
     bh_readahead include/linux/buffer_head.h:437 [inline]
     __breadahead+0x15c/0x1f0 fs/buffer.c:1359
     nilfs_search_super_root+0x2be/0x1530 fs/nilfs2/recovery.c:891
     load_nilfs+0x363/0xfb0 fs/nilfs2/the_nilfs.c:259
     nilfs_fill_super+0x1a0/0x4e0 fs/nilfs2/super.c:1073
     nilfs_get_tree+0x491/0x850 fs/nilfs2/super.c:1226
     vfs_get_tree+0x92/0x2a0 fs/super.c:1932
     fc_mount fs/namespace.c:1198 [inline]
     do_new_mount_fc fs/namespace.c:3772 [inline]
     do_new_mount+0x319/0xdc0 fs/namespace.c:3848
     do_mount fs/namespace.c:4181 [inline]
     __do_sys_mount fs/namespace.c:4397 [inline]
     __se_sys_mount+0x31d/0x420 fs/namespace.c:4374
     do_syscall_x64 arch/x86/entry/syscall_64.c:61 [inline]
     do_syscall_64+0x166/0x520 arch/x86/entry/syscall_64.c:84
     entry_SYSCALL_64_after_hwframe+0x77/0x7f
BUG: found I/O issued by syz.2.866/7955 but loop2 is no longer bound.
Call Trace:
     loop_queue_rq+0x232/0xc90 drivers/block/loop.c:1922
     __blk_mq_issue_directly block/blk-mq.c:2708 [inline]
     blk_mq_try_issue_directly+0x3f4/0x880 block/blk-mq.c:2768
     blk_mq_submit_bio+0x21de/0x2bb0 block/blk-mq.c:3220
     __submit_bio_noacct_mq block/blk-core.c:756 [inline]
     submit_bio_noacct_nocheck+0x2f4/0xa40 block/blk-core.c:790
     bh_submit fs/buffer.c:1146 [inline]
     __bh_read fs/buffer.c:2943 [inline]
     bh_readahead include/linux/buffer_head.h:437 [inline]
     __breadahead+0x15c/0x1f0 fs/buffer.c:1359
     nilfs_search_super_root+0x2be/0x1530 fs/nilfs2/recovery.c:891
     load_nilfs+0x363/0xfb0 fs/nilfs2/the_nilfs.c:259
     nilfs_fill_super+0x1a0/0x4e0 fs/nilfs2/super.c:1073
     nilfs_get_tree+0x491/0x850 fs/nilfs2/super.c:1226
     vfs_get_tree+0x92/0x2a0 fs/super.c:1932
     fc_mount fs/namespace.c:1198 [inline]
     do_new_mount_fc fs/namespace.c:3772 [inline]
     do_new_mount+0x319/0xdc0 fs/namespace.c:3848
     do_mount fs/namespace.c:4181 [inline]
     __do_sys_mount fs/namespace.c:4397 [inline]
     __se_sys_mount+0x31d/0x420 fs/namespace.c:4374
     do_syscall_x64 arch/x86/entry/syscall_64.c:61 [inline]
     do_syscall_64+0x166/0x520 arch/x86/entry/syscall_64.c:84
     entry_SYSCALL_64_after_hwframe+0x77/0x7f
BUG: found I/O issued by syz.2.866/7955 but loop2 is no longer bound.
Call Trace:
     loop_queue_rq+0x232/0xc90 drivers/block/loop.c:1922
     __blk_mq_issue_directly block/blk-mq.c:2708 [inline]
     blk_mq_try_issue_directly+0x3f4/0x880 block/blk-mq.c:2768
     blk_mq_submit_bio+0x21de/0x2bb0 block/blk-mq.c:3220
     __submit_bio_noacct_mq block/blk-core.c:756 [inline]
     submit_bio_noacct_nocheck+0x2f4/0xa40 block/blk-core.c:790
     bh_submit fs/buffer.c:1146 [inline]
     __bh_read fs/buffer.c:2943 [inline]
     bh_readahead include/linux/buffer_head.h:437 [inline]
     __breadahead+0x15c/0x1f0 fs/buffer.c:1359
     nilfs_search_super_root+0x2be/0x1530 fs/nilfs2/recovery.c:891
     load_nilfs+0x363/0xfb0 fs/nilfs2/the_nilfs.c:259
     nilfs_fill_super+0x1a0/0x4e0 fs/nilfs2/super.c:1073
     nilfs_get_tree+0x491/0x850 fs/nilfs2/super.c:1226
     vfs_get_tree+0x92/0x2a0 fs/super.c:1932
     fc_mount fs/namespace.c:1198 [inline]
     do_new_mount_fc fs/namespace.c:3772 [inline]
     do_new_mount+0x319/0xdc0 fs/namespace.c:3848
     do_mount fs/namespace.c:4181 [inline]
     __do_sys_mount fs/namespace.c:4397 [inline]
     __se_sys_mount+0x31d/0x420 fs/namespace.c:4374
     do_syscall_x64 arch/x86/entry/syscall_64.c:61 [inline]
     do_syscall_64+0x166/0x520 arch/x86/entry/syscall_64.c:84
     entry_SYSCALL_64_after_hwframe+0x77/0x7f
BUG: found I/O issued by syz.2.866/7955 but loop2 is no longer bound.
Call Trace:
     loop_queue_rq+0x232/0xc90 drivers/block/loop.c:1922
     __blk_mq_issue_directly block/blk-mq.c:2708 [inline]
     blk_mq_try_issue_directly+0x3f4/0x880 block/blk-mq.c:2768
     blk_mq_submit_bio+0x21de/0x2bb0 block/blk-mq.c:3220
     __submit_bio_noacct_mq block/blk-core.c:756 [inline]
     submit_bio_noacct_nocheck+0x2f4/0xa40 block/blk-core.c:790
     bh_submit fs/buffer.c:1146 [inline]
     __bh_read fs/buffer.c:2943 [inline]
     bh_readahead include/linux/buffer_head.h:437 [inline]
     __breadahead+0x15c/0x1f0 fs/buffer.c:1359
     nilfs_search_super_root+0x2be/0x1530 fs/nilfs2/recovery.c:891
     load_nilfs+0x363/0xfb0 fs/nilfs2/the_nilfs.c:259
     nilfs_fill_super+0x1a0/0x4e0 fs/nilfs2/super.c:1073
     nilfs_get_tree+0x491/0x850 fs/nilfs2/super.c:1226
     vfs_get_tree+0x92/0x2a0 fs/super.c:1932
     fc_mount fs/namespace.c:1198 [inline]
     do_new_mount_fc fs/namespace.c:3772 [inline]
     do_new_mount+0x319/0xdc0 fs/namespace.c:3848
     do_mount fs/namespace.c:4181 [inline]
     __do_sys_mount fs/namespace.c:4397 [inline]
     __se_sys_mount+0x31d/0x420 fs/namespace.c:4374
     do_syscall_x64 arch/x86/entry/syscall_64.c:61 [inline]
     do_syscall_64+0x166/0x520 arch/x86/entry/syscall_64.c:84
     entry_SYSCALL_64_after_hwframe+0x77/0x7f
BUG: found I/O issued by syz.2.866/7955 but loop2 is no longer bound.
Call Trace:
     loop_queue_rq+0x232/0xc90 drivers/block/loop.c:1922
     __blk_mq_issue_directly block/blk-mq.c:2708 [inline]
     blk_mq_try_issue_directly+0x3f4/0x880 block/blk-mq.c:2768
     blk_mq_submit_bio+0x21de/0x2bb0 block/blk-mq.c:3220
     __submit_bio_noacct_mq block/blk-core.c:756 [inline]
     submit_bio_noacct_nocheck+0x2f4/0xa40 block/blk-core.c:790
     bh_submit fs/buffer.c:1146 [inline]
     __bh_read fs/buffer.c:2943 [inline]
     bh_readahead include/linux/buffer_head.h:437 [inline]
     __breadahead+0x15c/0x1f0 fs/buffer.c:1359
     nilfs_search_super_root+0x2be/0x1530 fs/nilfs2/recovery.c:891
     load_nilfs+0x363/0xfb0 fs/nilfs2/the_nilfs.c:259
     nilfs_fill_super+0x1a0/0x4e0 fs/nilfs2/super.c:1073
     nilfs_get_tree+0x491/0x850 fs/nilfs2/super.c:1226
     vfs_get_tree+0x92/0x2a0 fs/super.c:1932
     fc_mount fs/namespace.c:1198 [inline]
     do_new_mount_fc fs/namespace.c:3772 [inline]
     do_new_mount+0x319/0xdc0 fs/namespace.c:3848
     do_mount fs/namespace.c:4181 [inline]
     __do_sys_mount fs/namespace.c:4397 [inline]
     __se_sys_mount+0x31d/0x420 fs/namespace.c:4374
     do_syscall_x64 arch/x86/entry/syscall_64.c:61 [inline]
     do_syscall_64+0x166/0x520 arch/x86/entry/syscall_64.c:84
     entry_SYSCALL_64_after_hwframe+0x77/0x7f
BUG: found I/O issued by syz.2.866/7955 but loop2 is no longer bound.
Call Trace:
     loop_queue_rq+0x232/0xc90 drivers/block/loop.c:1922
     __blk_mq_issue_directly block/blk-mq.c:2708 [inline]
     blk_mq_try_issue_directly+0x3f4/0x880 block/blk-mq.c:2768
     blk_mq_submit_bio+0x21de/0x2bb0 block/blk-mq.c:3220
     __submit_bio_noacct_mq block/blk-core.c:756 [inline]
     submit_bio_noacct_nocheck+0x2f4/0xa40 block/blk-core.c:790
     bh_submit fs/buffer.c:1146 [inline]
     __bh_read fs/buffer.c:2943 [inline]
     bh_readahead include/linux/buffer_head.h:437 [inline]
     __breadahead+0x15c/0x1f0 fs/buffer.c:1359
     nilfs_search_super_root+0x2be/0x1530 fs/nilfs2/recovery.c:891
     load_nilfs+0x363/0xfb0 fs/nilfs2/the_nilfs.c:259
     nilfs_fill_super+0x1a0/0x4e0 fs/nilfs2/super.c:1073
     nilfs_get_tree+0x491/0x850 fs/nilfs2/super.c:1226
     vfs_get_tree+0x92/0x2a0 fs/super.c:1932
     fc_mount fs/namespace.c:1198 [inline]
     do_new_mount_fc fs/namespace.c:3772 [inline]
     do_new_mount+0x319/0xdc0 fs/namespace.c:3848
     do_mount fs/namespace.c:4181 [inline]
     __do_sys_mount fs/namespace.c:4397 [inline]
     __se_sys_mount+0x31d/0x420 fs/namespace.c:4374
     do_syscall_x64 arch/x86/entry/syscall_64.c:61 [inline]
     do_syscall_64+0x166/0x520 arch/x86/entry/syscall_64.c:84
     entry_SYSCALL_64_after_hwframe+0x77/0x7f


***

If these findings have caused you to resend the series or submit a
separate fix, please add the following tag to your commit message:
  Tested-by: syzbot@syzkaller.appspotmail.com

---
This report is generated by a bot. It may contain errors.
syzbot ci engineers can be reached at syzkaller@googlegroups.com.

To test a fix for this bug, please reply with `#syz test`
(on a separate line) and attach the patch to the email.

Notes:
- The patch will be applied on top of the tested series (as an
  incremental fix).
- To test a new version of the whole series, please send it directly
  to syzbot@lists.linux.dev.
- Arguments like custom git repos and branches are not supported.

[PATCH] block: Wait for inflight requests in bdev_release()

[Tetsuo Handa]

syzbot is reporting a NULL pointer dereference [1] and invalid reading of
destroyed "struct file" [2] in lo_rw_aio() within the loop driver. Also,
syzbot-ci is reporting read attempt on a no-longer bound loop device [3].
These reports are caused by current implementation that a block device's
backing storage and associated structures (such as "struct file") are
invalidated before all outstanding asynchronous or file-system-driven I/O
requests have fully completed.

Investigation revealed that filesystems can leak outstanding non-blocking
Read/Write requests during error-rollbacks in the mount failure path or
during standard unmount sequences:

1. Write Leaks via -EIO: When filemap_fdatawrite_range() returns an error
   (such as -EIO due to filesystem errors), filemap_write_and_wait_range()
   skips the subsequent __filemap_fdatawait_range() call. As a result,
   sync_blockdev() returns prematurely without waiting for the completion
   of write requests already sitting in flight.
2. Read Leaks via Readahead: Asynchronous metadata readahead requests are
   entirely ignored by sync_blockdev(), allowing ongoing Read I/O to bypass
   data synchronization entirely.

When these leaked requests later complete or attempt backend processing via
loop's kworker (loop_handle_cmd()), they dereference now-invalid or
nullified fields.

While earlier attempts tried to address this regression inside the loop
driver by temporarily releasing disk->open_mutex [4] or proposing a core
layer post_release() callback [5], these approaches were rejected because
they either introduced invasive driver-specific hooks into the block layer
or broke the core mutual exclusion semantics that the block layer relies
upon.

To fix this comprehensively without introducing deadlocks or altering
driver-specific semantics, enforce an explicit queue freeze in the core
VFS close path inside bdev_release(), right before disk->open_mutex is
acquired:

1. Lock-Free Congestion Clearing: Calling blk_mq_freeze_queue() before
   acquiring disk->open_mutex avoids any circular locking dependencies
   (disk->open_mutex => wq_completion => VFS locks => disk->open_mutex),
   guaranteeing that the lockdep hierarchy remains perfectly undisturbed.
2. Guaranteed Quenching: Freezing the queue ensures that all outstanding
   asynchronous Direct I/O (cmd->use_aio != 0) and page-cache readahead
   requests are physically completed and drained before the loop device
   proceeds to teardown and unbind the backing file via fput().

This architecture-compliant fix should be able to ensure data integrity
and lifecycle synchronization across all virtual block devices without
muddying block layer locks.

Reported-by: syzbot+cd8a9a308e879a4e2c28@syzkaller.appspotmail.com
Closes: https://syzkaller.appspot.com/bug?extid=cd8a9a308e879a4e2c28 [1]
Reported-by: syzbot+bc273027d5643e48e5b3@syzkaller.appspotmail.com
Closes: https://syzkaller.appspot.com/bug?extid=bc273027d5643e48e5b3 [2]
Link: https://lkml.kernel.org/r/6acbe268.8cdf6287.1eb53.00ed.GAE@google.com [3]
Link: https://lkml.kernel.org/r/8dedfc40-9cae-44ff-9960-e0eb1825e963@I-love.SAKURA.ne.jp [4]
Link: https://lkml.kernel.org/r/69960302-1535-441a-be4e-d652766d65c2@I-love.SAKURA.ne.jp [5]
Assisted-by: Gemini-Pro
Signed-off-by: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
---
 block/bdev.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/block/bdev.c b/block/bdev.c
index cd8323083740..2ddffc65f9da 100644
--- a/block/bdev.c
+++ b/block/bdev.c
@@ -1170,6 +1170,19 @@ void bdev_release(struct file *bdev_file)
 	if (atomic_read(&bdev->bd_openers) == 1)
 		sync_blockdev(bdev);
 
+	/*
+	 * Always wait for all inflight read/write requests to complete.
+	 *
+	 * We can't omit this wait when it does not look like we're the last
+	 * one, for it is possible that two processes concurrently call
+	 * bdev_release() and see atomic_read(&bdev->bd_openers) > 1 and then
+	 * sequentially call blkdev_put_whole() with disk->open_mutex held.
+	 *
+	 * Also, we can't wait with disk->open_mutex held because there is
+	 * possibility of deadlock.
+	 */
+	blk_mq_unfreeze_queue(bdev->bd_queue, blk_mq_freeze_queue(bdev->bd_queue));
+
 	mutex_lock(&disk->open_mutex);
 	bdev_yield_write_access(bdev_file);
 
-- 
2.56.0

Re: [PATCH] block: Wait for inflight requests in bdev_release()

[Tetsuo Handa]

sashiko's review on this patch
( https://sashiko.dev/#/patchset/53c3ccf7-1a5e-4861-afc0-7cbfd3284655%40I-love.SAKURA.ne.jp )
pointed out that attempting to freeze the queue or wait for in-flight requests
generically inside the block core's bdev_release() is highly dangerous.

To resolve this safely, we must move the synchronization out of bdev_release()
and back into the loop driver context. However, any viable loop-specific fix
must reconcile three architectural constraints explained below.

*** The Three Verified Constraints

1. __loop_clr_fd() must complete before close() returns to userspace

  We cannot defer the teardown asynchronously to a background workqueue or task_work
  context after close() returns. Past attempts (e.g., commit 322c4293ecc5) proved
  that breaking synchronous autoclear semantics causes failures due to delayed fput()
  on the backing file.

2. Outstanding loop I/O cannot be safely drained while holding disk->open_mutex

  Draining the loop workqueue or freezing the request queue while disk->open_mutex is
  held introduces heavy lock-inversion risks. As verified by syzbot, this creates a
  circular locking dependency: disk->open_mutex => (wq_completion)loopX => VFS /
  Subsystem Locks => disk->open_mutex.

3. Leaked in-flight requests can execute after device unbinding

  As demonstrated by recent debug trace logs, filesystems can leak outstanding
  reads/writes during mount/unmount. If the loop device unbinds the backing file
  pointer via __loop_clr_fd() before these leaked requests have been physically
  drained, the loop backend kworker will later attempt to process them against
  invalid or nullified structures, resulting in oops reported by syzbot.

*** The Two Viable Architectural Paths

  Because we cannot hold disk->open_mutex while flushing loop requests, yet we must
  finish flushing before lo_release() concludes, the block layer must choose one of
  the following two infrastructure designs:

Path A: Temporarily dropping disk->open_mutex inside loop.c

  We explicitly drop and re-acquire disk->open_mutex inside __loop_clr_fd() to
  safely execute synchronize_rcu(), drain_workqueue(), and queue freezing.

  * Safety Verification: This is entirely safe against parallel openers because
    the loop device state has already transitioned to Lo_rundown. Any concurrent
    lo_open() entering during the temporary unlock window will immediately and
    reliably fail with -ENXIO.

Path B: Introducing a new post_release block core callback

  We introduce a void (*post_release)(struct gendisk *disk) callback in struct
  block_device_operations. The block core will invoke this callback in bdev_release()
  immediately after it has dropped disk->open_mutex, but before calling module_put().

  * Safety Verification: This cleanly isolates loop-specific synchronization from the
    open_mutex lock context. Since it executes within the built-in block core lifecycle,
    it guarantees module safety and completely eliminates the lock-inversion risk.

*** Conclusion

  We cannot fix this loop regression without either temporarily releasing open_mutex
  (Path A) or introducing an interface to perform the cleanup after open_mutex is
  dropped (Path B).

Jens and Christoph, which architectural trade-off does the block layer team prefer to accept?