[PATCH] loop: move vfs_fsync() out of loop_update_dio()

[PATCH] loop: move vfs_fsync() out of loop_update_dio()

[Ming Lei]

If vfs_flush() is called with queue frozen, the queue freeze lock may be
connected with FS internal lock, and lockdep warning can be triggered
because the queue freeze lock is connected with too many global or
sub-system locks.

Fix the warning by moving vfs_fsync() out of loop_update_dio():

- vfs_fsync() is only needed when switching to dio

- only loop_change_fd() and loop_configure() may switch from buffered
IO to direct IO, so call vfs_fsync() directly here. This way is safe
because either loop is in unbound, or new file isn't attached

- for the other two cases of set_status and set_block_size, direct IO
can only become off, so no need to call vfs_fsync()

Cc: Christoph Hellwig <hch@infradead.org>
Reported-by: Kun Hu <huk23@m.fudan.edu.cn>
Reported-by: Jiaji Qin <jjtan24@m.fudan.edu.cn>
Closes: https://lore.kernel.org/linux-block/359BC288-B0B1-4815-9F01-3A349B12E816@m.fudan.edu.cn/T/#u
Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
 drivers/block/loop.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/drivers/block/loop.c b/drivers/block/loop.c
index 1ec7417c7f00..be7e20064427 100644
--- a/drivers/block/loop.c
+++ b/drivers/block/loop.c
@@ -205,8 +205,6 @@ static bool lo_can_use_dio(struct loop_device *lo)
  */
 static inline void loop_update_dio(struct loop_device *lo)
 {
-	bool dio_in_use = lo->lo_flags & LO_FLAGS_DIRECT_IO;
-
 	lockdep_assert_held(&lo->lo_mutex);
 	WARN_ON_ONCE(lo->lo_state == Lo_bound &&
 		     lo->lo_queue->mq_freeze_depth == 0);
@@ -215,10 +213,6 @@ static inline void loop_update_dio(struct loop_device *lo)
 		lo->lo_flags |= LO_FLAGS_DIRECT_IO;
 	if ((lo->lo_flags & LO_FLAGS_DIRECT_IO) && !lo_can_use_dio(lo))
 		lo->lo_flags &= ~LO_FLAGS_DIRECT_IO;
-
-	/* flush dirty pages before starting to issue direct I/O */
-	if ((lo->lo_flags & LO_FLAGS_DIRECT_IO) && !dio_in_use)
-		vfs_fsync(lo->lo_backing_file, 0);
 }
 
 /**
@@ -621,6 +615,9 @@ static int loop_change_fd(struct loop_device *lo, struct block_device *bdev,
 	if (get_loop_size(lo, file) != get_loop_size(lo, old_file))
 		goto out_err;
 
+	/* may work in dio, so flush page cache for avoiding race */
+	vfs_fsync(file, 0);
+
 	/* and ... switch */
 	disk_force_media_change(lo->lo_disk);
 	blk_mq_freeze_queue(lo->lo_queue);
@@ -1098,6 +1095,9 @@ static int loop_configure(struct loop_device *lo, blk_mode_t mode,
 	if (error)
 		goto out_unlock;
 
+	/* may work in dio, so flush page cache for avoiding race */
+	vfs_fsync(file, 0);
+
 	loop_update_dio(lo);
 	loop_sysfs_init(lo);
 
-- 
2.44.0

Re: [PATCH] loop: move vfs_fsync() out of loop_update_dio()

[Christoph Hellwig]

NAK for the same reason as v1.  It's not needed because there is no
deadlock, just a false positive due to missing annotations of lock
instances, and we need to fsync with a frozen queue to ensure there
is no outstanding I/O.

Re: [PATCH] loop: move vfs_fsync() out of loop_update_dio()

[Ming Lei]

On Mon, Jan 13, 2025 at 05:50:34AM -0800, Christoph Hellwig wrote:
> NAK for the same reason as v1.  It's not needed because there is no
> deadlock, just a false positive due to missing annotations of lock

Yes, but annotation should be the last straw.

> instances, and we need to fsync with a frozen queue to ensure there
> is no outstanding I/O.

loop_configure() is on unbound loop, so there isn't outstanding I/O.

loop_change_fd() is switching to this new file, so no outstanding I/O
on this new file before unfreeze.

The other two can only switch to buffered IO, which needn't the fsync.

So can you point out anything is wrong?

And this way is sort of simplification.


Thanks,
Ming

Re: [PATCH] loop: move vfs_fsync() out of loop_update_dio()

[Christoph Hellwig]

On Mon, Jan 13, 2025 at 11:04:51PM +0800, Ming Lei wrote:
> > instances, and we need to fsync with a frozen queue to ensure there
> > is no outstanding I/O.
> 
> loop_configure() is on unbound loop, so there isn't outstanding I/O.

Yeah.

> loop_change_fd() is switching to this new file, so no outstanding I/O
> on this new file before unfreeze.

True.

> The other two can only switch to buffered IO, which needn't the fsync.
> 
> So can you point out anything is wrong?
> 
> And this way is sort of simplification.

Yeah, I think this is fine in this version.  But please update the
comment away from the avoiding races to spell out that this is
writing out the page cache before starting to use direct I/O.

Re:[PATCH] loop: move vfs_fsync() out of loop_update_dio()

[胡焜]

> drivers/block/loop.c | 12 ++++++------
> 1 file changed, 6 insertions(+), 6 deletions(-)

> diff --git a/drivers/block/loop.c b/drivers/block/loop.c
> index 1ec7417c7f00..be7e20064427 100644
> --- a/drivers/block/loop.c
> +++ b/drivers/block/loop.c
> @@ -205,8 +205,6 @@ static bool lo_can_use_dio(struct loop_device *lo)
> */
>  static inline void loop_update_dio(struct loop_device *lo)
>  {
> - bool dio_in_use = lo->lo_flags & LO_FLAGS_DIRECT_IO;
> -
>   lockdep_assert_held(&lo->lo_mutex);
>   WARN_ON_ONCE(lo->lo_state == Lo_bound &&
>        lo->lo_queue->mq_freeze_depth == 0);
> @@ -215,10 +213,6 @@ static inline void loop_update_dio(struct loop_device *lo)
>   lo->lo_flags |= LO_FLAGS_DIRECT_IO;
>   if ((lo->lo_flags & LO_FLAGS_DIRECT_IO) && !lo_can_use_dio(lo))
>   lo->lo_flags &= ~LO_FLAGS_DIRECT_IO;
> -
> - /* flush dirty pages before starting to issue direct I/O */
> - if ((lo->lo_flags & LO_FLAGS_DIRECT_IO) && !dio_in_use)
> - vfs_fsync(lo->lo_backing_file, 0);
>  }
>  
>  /**
> @@ -621,6 +615,9 @@ static int loop_change_fd(struct loop_device *lo, struct block_device *bdev,
>   if (get_loop_size(lo, file) != get_loop_size(lo, old_file))
>  goto out_err;
 
> + /* may work in dio, so flush page cache for avoiding race */
> + vfs_fsync(file, 0);
> +
>   /* and ... switch */
>   disk_force_media_change(lo->lo_disk);
>   blk_mq_freeze_queue(lo->lo_queue);
> @@ -1098,6 +1095,9 @@ static int loop_configure(struct loop_device *lo, blk_mode_t mode,
>   if (error)
>   goto out_unlock;
 
> + /* may work in dio, so flush page cache for avoiding race */
> + vfs_fsync(file, 0);
> +
>   loop_update_dio(lo);
>   loop_sysfs_init(lo);
>  
> --
> 2.44.0


Hello Ming,

I would like to double check that this fix doesn't seem to have been merged into the main thread, will this version still be merged into mainline kernel tree?

Thanks,
Kun

Re: [PATCH] loop: move vfs_fsync() out of loop_update_dio()

[Ming Lei]

On Mon, Mar 17, 2025 at 04:52:16PM +0800, 胡焜 wrote:
> > drivers/block/loop.c | 12 ++++++------
> > 1 file changed, 6 insertions(+), 6 deletions(-)
> 
> > diff --git a/drivers/block/loop.c b/drivers/block/loop.c
> > index 1ec7417c7f00..be7e20064427 100644
> > --- a/drivers/block/loop.c
> > +++ b/drivers/block/loop.c
> > @@ -205,8 +205,6 @@ static bool lo_can_use_dio(struct loop_device *lo)
> > */
> >  static inline void loop_update_dio(struct loop_device *lo)
> >  {
> > - bool dio_in_use = lo->lo_flags & LO_FLAGS_DIRECT_IO;
> > -
> >   lockdep_assert_held(&lo->lo_mutex);
> >   WARN_ON_ONCE(lo->lo_state == Lo_bound &&
> >        lo->lo_queue->mq_freeze_depth == 0);
> > @@ -215,10 +213,6 @@ static inline void loop_update_dio(struct loop_device *lo)
> >   lo->lo_flags |= LO_FLAGS_DIRECT_IO;
> >   if ((lo->lo_flags & LO_FLAGS_DIRECT_IO) && !lo_can_use_dio(lo))
> >   lo->lo_flags &= ~LO_FLAGS_DIRECT_IO;
> > -
> > - /* flush dirty pages before starting to issue direct I/O */
> > - if ((lo->lo_flags & LO_FLAGS_DIRECT_IO) && !dio_in_use)
> > - vfs_fsync(lo->lo_backing_file, 0);
> >  }
> >  
> >  /**
> > @@ -621,6 +615,9 @@ static int loop_change_fd(struct loop_device *lo, struct block_device *bdev,
> >   if (get_loop_size(lo, file) != get_loop_size(lo, old_file))
> >  goto out_err;
>  
> > + /* may work in dio, so flush page cache for avoiding race */
> > + vfs_fsync(file, 0);
> > +
> >   /* and ... switch */
> >   disk_force_media_change(lo->lo_disk);
> >   blk_mq_freeze_queue(lo->lo_queue);
> > @@ -1098,6 +1095,9 @@ static int loop_configure(struct loop_device *lo, blk_mode_t mode,
> >   if (error)
> >   goto out_unlock;
>  
> > + /* may work in dio, so flush page cache for avoiding race */
> > + vfs_fsync(file, 0);
> > +
> >   loop_update_dio(lo);
> >   loop_sysfs_init(lo);
> >  
> > --
> > 2.44.0
> 
> 
> Hello Ming,
> 
> I would like to double check that this fix doesn't seem to have been merged into the main thread, will this version still be merged into mainline kernel tree?

The V2 has been sent out after updating comment, please verify if it fixes your issue:

https://lore.kernel.org/linux-block/20250318010318.3861682-1-ming.lei@redhat.com/

If yes, feel free to provide one tested-by for moving on.


Thanks, 
Ming

Re: [PATCH] loop: move vfs_fsync() out of loop_update_dio()

[胡焜]

> 2025年3月18日 09:07，Ming Lei <ming.lei@redhat.com> 写道：
> 
> On Mon, Mar 17, 2025 at 04:52:16PM +0800, 胡焜 wrote:
>>> drivers/block/loop.c | 12 ++++++------
>>> 1 file changed, 6 insertions(+), 6 deletions(-)
>> 
>>> diff --git a/drivers/block/loop.c b/drivers/block/loop.c
>>> index 1ec7417c7f00..be7e20064427 100644
>>> --- a/drivers/block/loop.c
>>> +++ b/drivers/block/loop.c
>>> @@ -205,8 +205,6 @@ static bool lo_can_use_dio(struct loop_device *lo)
>>> */
>>> static inline void loop_update_dio(struct loop_device *lo)
>>> {
>>> - bool dio_in_use = lo->lo_flags & LO_FLAGS_DIRECT_IO;
>>> -
>>>  lockdep_assert_held(&lo->lo_mutex);
>>>  WARN_ON_ONCE(lo->lo_state == Lo_bound &&
>>>       lo->lo_queue->mq_freeze_depth == 0);
>>> @@ -215,10 +213,6 @@ static inline void loop_update_dio(struct loop_device *lo)
>>>  lo->lo_flags |= LO_FLAGS_DIRECT_IO;
>>>  if ((lo->lo_flags & LO_FLAGS_DIRECT_IO) && !lo_can_use_dio(lo))
>>>  lo->lo_flags &= ~LO_FLAGS_DIRECT_IO;
>>> -
>>> - /* flush dirty pages before starting to issue direct I/O */
>>> - if ((lo->lo_flags & LO_FLAGS_DIRECT_IO) && !dio_in_use)
>>> - vfs_fsync(lo->lo_backing_file, 0);
>>> }
>>> 
>>> /**
>>> @@ -621,6 +615,9 @@ static int loop_change_fd(struct loop_device *lo, struct block_device *bdev,
>>>  if (get_loop_size(lo, file) != get_loop_size(lo, old_file))
>>> goto out_err;
>> 
>>> + /* may work in dio, so flush page cache for avoiding race */
>>> + vfs_fsync(file, 0);
>>> +
>>>  /* and ... switch */
>>>  disk_force_media_change(lo->lo_disk);
>>>  blk_mq_freeze_queue(lo->lo_queue);
>>> @@ -1098,6 +1095,9 @@ static int loop_configure(struct loop_device *lo, blk_mode_t mode,
>>>  if (error)
>>>  goto out_unlock;
>> 
>>> + /* may work in dio, so flush page cache for avoiding race */
>>> + vfs_fsync(file, 0);
>>> +
>>>  loop_update_dio(lo);
>>>  loop_sysfs_init(lo);
>>> 
>>> --
>>> 2.44.0
>> 
>> 
>> Hello Ming,
>> 
>> I would like to double check that this fix doesn't seem to have been merged into the main thread, will this version still be merged into mainline kernel tree?
> 
> The V2 has been sent out after updating comment, please verify if it fixes your issue:
> 
> https://lore.kernel.org/linux-block/20250318010318.3861682-1-ming.lei@redhat.com/
> 
> If yes, feel free to provide one tested-by for moving on.
> 
> 
> Thanks, 
> Ming


Hi Ming,

I have tested the V2 patch you sent, and it has successfully fixed the issue. Thank you for the update!

Tested-by: Kun Hu <huk23@m.fudan.edu.cn>, Jiaji Qin <jjtan24@m.fudan.edu.cn>.


—————
Thanks,
Kun