lazy bounce buffering for checksummed reads

lazy bounce buffering for checksummed reads

[Christoph Hellwig]

Hi all,

this series improves performance and resource usage for reads from
devices that require stable pages due to checksumming on XFS.

Currently XFS unconditionally bounce buffers reads on such devices to
prevent user modifications to the buffer from corrupting the data,
leading to checksum failures.

This uses DRAM bandwidth and CPU cycles for copies that are not needed
most of the time, and due to the use of a bio_vec for the bounce
buffer to smaller than wanted and unaligned I/O sizes when using 4k
user pages (i.e. 1MB-4k I/O).

This series addresses this by reading without the bounce buffer first,
and then only allocating a buffer and reading into that again on an
initial checksum failure.  To accommodate for rare (or hypothetical?)
applications that have legitimate needs to frequently modify in-flight
buffers, a sysfs know is provided to revert to the old behavior.

The series is a bit of a mess.  Patch 1 is queued up in the VFS tree
for 7.3, although it a fix that really should go into 7.2, but the
request and ping for that were ignored.  Patches 2-6 were submitted
to the block layer independently and will hopefully go in soon in in
some form soon.  Patches 7 and 8 need a resubmission from Tal.

The rest is split over 3 subsystems, and for two of them will
conflict with large in-flight patch series (iomap iter next and
xfs fsverity).  So for now I'm mostly looking for review and will
worry about a merge strategy later if/when we agree on the overall
approach.

A git tree is available to help with the review here:

    git://git.infradead.org/users/hch/misc.git lazy-bounce

Gitweb:

    https://git.infradead.org/?p=users/hch/misc.git;a=shortlog;h=refs/heads/lazy-bounce

[PATCH 01/22] iomap: add a separate bio_set for iomap_split_ioend

[Christoph Hellwig]

iomap_split_ioend can split bios that already come from
iomap_ioend_bioset and thus deadlock when the bioset is exhausted.

Add a separate bio_set to avoid this deadlock.

Fixes: 5fcbd555d483 ("iomap: split bios to zone append limits in the submission handlers")
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 fs/iomap/ioend.c | 21 +++++++++++++++++++--
 1 file changed, 19 insertions(+), 2 deletions(-)

diff --git a/fs/iomap/ioend.c b/fs/iomap/ioend.c
index 0565328764c1..2ec755a89228 100644
--- a/fs/iomap/ioend.c
+++ b/fs/iomap/ioend.c
@@ -13,6 +13,7 @@
 
 struct bio_set iomap_ioend_bioset;
 EXPORT_SYMBOL_GPL(iomap_ioend_bioset);
+static struct bio_set iomap_ioend_split_bioset;
 
 struct iomap_ioend *iomap_init_ioend(struct inode *inode,
 		struct bio *bio, loff_t file_offset, u16 ioend_flags)
@@ -486,7 +487,8 @@ struct iomap_ioend *iomap_split_ioend(struct iomap_ioend *ioend,
 	sector_offset = ALIGN_DOWN(sector_offset << SECTOR_SHIFT,
 			i_blocksize(ioend->io_inode)) >> SECTOR_SHIFT;
 
-	split = bio_split(bio, sector_offset, GFP_NOFS, &iomap_ioend_bioset);
+	split = bio_split(bio, sector_offset, GFP_NOFS,
+			&iomap_ioend_split_bioset);
 	if (IS_ERR(split))
 		return ERR_CAST(split);
 	split->bi_private = bio->bi_private;
@@ -509,8 +511,23 @@ EXPORT_SYMBOL_GPL(iomap_split_ioend);
 
 static int __init iomap_ioend_init(void)
 {
-	return bioset_init(&iomap_ioend_bioset, 4 * (PAGE_SIZE / SECTOR_SIZE),
+	const unsigned int nr_mempool_entries = 4 * (PAGE_SIZE / SECTOR_SIZE);
+	int error;
+
+	error = bioset_init(&iomap_ioend_bioset, nr_mempool_entries,
 			   offsetof(struct iomap_ioend, io_bio),
 			   BIOSET_NEED_BVECS);
+	if (error)
+		return error;
+	error = bioset_init(&iomap_ioend_split_bioset, nr_mempool_entries,
+			   offsetof(struct iomap_ioend, io_bio),
+			   BIOSET_NEED_BVECS);
+	if (error)
+		goto out_exit_ioend_bioset;
+	return 0;
+
+out_exit_ioend_bioset:
+	bioset_exit(&iomap_ioend_bioset);
+	return error;
 }
 fs_initcall(iomap_ioend_init);
-- 
2.53.0

Re: [PATCH 01/22] iomap: add a separate bio_set for iomap_split_ioend

[Darrick J. Wong]

On Thu, Jul 23, 2026 at 04:49:26PM +0200, Christoph Hellwig wrote:
> iomap_split_ioend can split bios that already come from
> iomap_ioend_bioset and thus deadlock when the bioset is exhausted.
> 
> Add a separate bio_set to avoid this deadlock.
> 
> Fixes: 5fcbd555d483 ("iomap: split bios to zone append limits in the submission handlers")
> Signed-off-by: Christoph Hellwig <hch@lst.de>

Cc: <stable@vger.kernel.org> # v6.15
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>

--D

> ---
>  fs/iomap/ioend.c | 21 +++++++++++++++++++--
>  1 file changed, 19 insertions(+), 2 deletions(-)
> 
> diff --git a/fs/iomap/ioend.c b/fs/iomap/ioend.c
> index 0565328764c1..2ec755a89228 100644
> --- a/fs/iomap/ioend.c
> +++ b/fs/iomap/ioend.c
> @@ -13,6 +13,7 @@
>  
>  struct bio_set iomap_ioend_bioset;
>  EXPORT_SYMBOL_GPL(iomap_ioend_bioset);
> +static struct bio_set iomap_ioend_split_bioset;
>  
>  struct iomap_ioend *iomap_init_ioend(struct inode *inode,
>  		struct bio *bio, loff_t file_offset, u16 ioend_flags)
> @@ -486,7 +487,8 @@ struct iomap_ioend *iomap_split_ioend(struct iomap_ioend *ioend,
>  	sector_offset = ALIGN_DOWN(sector_offset << SECTOR_SHIFT,
>  			i_blocksize(ioend->io_inode)) >> SECTOR_SHIFT;
>  
> -	split = bio_split(bio, sector_offset, GFP_NOFS, &iomap_ioend_bioset);
> +	split = bio_split(bio, sector_offset, GFP_NOFS,
> +			&iomap_ioend_split_bioset);
>  	if (IS_ERR(split))
>  		return ERR_CAST(split);
>  	split->bi_private = bio->bi_private;
> @@ -509,8 +511,23 @@ EXPORT_SYMBOL_GPL(iomap_split_ioend);
>  
>  static int __init iomap_ioend_init(void)
>  {
> -	return bioset_init(&iomap_ioend_bioset, 4 * (PAGE_SIZE / SECTOR_SIZE),
> +	const unsigned int nr_mempool_entries = 4 * (PAGE_SIZE / SECTOR_SIZE);
> +	int error;
> +
> +	error = bioset_init(&iomap_ioend_bioset, nr_mempool_entries,
>  			   offsetof(struct iomap_ioend, io_bio),
>  			   BIOSET_NEED_BVECS);
> +	if (error)
> +		return error;
> +	error = bioset_init(&iomap_ioend_split_bioset, nr_mempool_entries,
> +			   offsetof(struct iomap_ioend, io_bio),
> +			   BIOSET_NEED_BVECS);
> +	if (error)
> +		goto out_exit_ioend_bioset;
> +	return 0;
> +
> +out_exit_ioend_bioset:
> +	bioset_exit(&iomap_ioend_bioset);
> +	return error;
>  }
>  fs_initcall(iomap_ioend_init);
> -- 
> 2.53.0
> 
> 

Re: [PATCH 01/22] iomap: add a separate bio_set for iomap_split_ioend

[Christoph Hellwig]

On Thu, Jul 23, 2026 at 09:49:07AM -0700, Darrick J. Wong wrote:
> On Thu, Jul 23, 2026 at 04:49:26PM +0200, Christoph Hellwig wrote:
> > iomap_split_ioend can split bios that already come from
> > iomap_ioend_bioset and thus deadlock when the bioset is exhausted.
> > 
> > Add a separate bio_set to avoid this deadlock.
> > 
> > Fixes: 5fcbd555d483 ("iomap: split bios to zone append limits in the submission handlers")
> > Signed-off-by: Christoph Hellwig <hch@lst.de>
> 
> Cc: <stable@vger.kernel.org> # v6.15
> Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>

Note that this already is queued up in the vfs iomap branch for 7.3.
I've been begging to get іt queued up for 7.2, but so far I've not
even managed to a reply to that.

[PATCH 02/22] block: remove bip_should_check

[Christoph Hellwig]

There is no benefit in using this helper over the simple flags check.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Anuj Gupta <anuj20.g@samsung.com>
---
 block/bio-integrity-auto.c | 8 ++------
 1 file changed, 2 insertions(+), 6 deletions(-)

diff --git a/block/bio-integrity-auto.c b/block/bio-integrity-auto.c
index b1c733ecfd2e..43ac9f338183 100644
--- a/block/bio-integrity-auto.c
+++ b/block/bio-integrity-auto.c
@@ -45,10 +45,6 @@ static void bio_integrity_verify_fn(struct work_struct *work)
 }
 
 #define BIP_CHECK_FLAGS (BIP_CHECK_GUARD | BIP_CHECK_REFTAG | BIP_CHECK_APPTAG)
-static bool bip_should_check(struct bio_integrity_payload *bip)
-{
-	return bip->bip_flags & BIP_CHECK_FLAGS;
-}
 
 /**
  * __bio_integrity_endio - Integrity I/O completion function
@@ -66,7 +62,7 @@ bool __bio_integrity_endio(struct bio *bio)
 		container_of(bip, struct bio_integrity_data, bip);
 
 	if (bio_op(bio) == REQ_OP_READ && !bio->bi_status &&
-	    bip_should_check(bip)) {
+	    (bip->bip_flags & BIP_CHECK_FLAGS)) {
 		INIT_WORK(&bid->work, bio_integrity_verify_fn);
 		queue_work(kintegrityd_wq, &bid->work);
 		return false;
@@ -99,7 +95,7 @@ void bio_integrity_prep(struct bio *bio, unsigned int action)
 		bio_integrity_setup_default(bio);
 
 	/* Auto-generate integrity metadata if this is a write */
-	if (bio_data_dir(bio) == WRITE && bip_should_check(&bid->bip))
+	if (bio_data_dir(bio) == WRITE && (bid->bip.bip_flags & BIP_CHECK_FLAGS))
 		bio_integrity_generate(bio);
 	else
 		bid->saved_bio_iter = bio->bi_iter;
-- 
2.53.0

[PATCH 03/22] block: lift BIP_CHECK_FLAGS to include/linux/bio-integrity.h

[Christoph Hellwig]

To allow for users outside of bio-integrity-auto.c.  Also add a little
comment explaining it.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Anuj Gupta <anuj20.g@samsung.com>
---
 block/bio-integrity-auto.c    | 2 --
 include/linux/bio-integrity.h | 3 +++
 2 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/block/bio-integrity-auto.c b/block/bio-integrity-auto.c
index 43ac9f338183..9456dcffd17a 100644
--- a/block/bio-integrity-auto.c
+++ b/block/bio-integrity-auto.c
@@ -44,8 +44,6 @@ static void bio_integrity_verify_fn(struct work_struct *work)
 	bio_endio(bio);
 }
 
-#define BIP_CHECK_FLAGS (BIP_CHECK_GUARD | BIP_CHECK_REFTAG | BIP_CHECK_APPTAG)
-
 /**
  * __bio_integrity_endio - Integrity I/O completion function
  * @bio:	Protected bio
diff --git a/include/linux/bio-integrity.h b/include/linux/bio-integrity.h
index c3dda32fd803..0ea2a8bf7efb 100644
--- a/include/linux/bio-integrity.h
+++ b/include/linux/bio-integrity.h
@@ -17,6 +17,9 @@ enum bip_flags {
 	BIP_MEMPOOL		= 1 << 15, /* buffer backed by mempool */
 };
 
+/* flags that require generate/verify action. */
+#define BIP_CHECK_FLAGS (BIP_CHECK_GUARD | BIP_CHECK_REFTAG | BIP_CHECK_APPTAG)
+
 struct bio_integrity_payload {
 	struct bvec_iter	bip_iter;
 
-- 
2.53.0

[PATCH 04/22] block: handle nogenerate/noverify properly in fs-integrity

[Christoph Hellwig]

Check the BIP_CHECK flags before generating or verifying PI information,
otherwise this can be incorrectly called for non-PI metadata and
cause generation of incorrect metadata and crashed in the verification
handler.

The new behavior matches that of the block layer auto-generated
metadata.

Fixes: 0bde8a12b554 ("block: add fs_bio_integrity helpers")
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 block/bio-integrity-fs.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/block/bio-integrity-fs.c b/block/bio-integrity-fs.c
index 9c5fe5fa8f0d..692403dfa047 100644
--- a/block/bio-integrity-fs.c
+++ b/block/bio-integrity-fs.c
@@ -46,7 +46,8 @@ void fs_bio_integrity_free(struct bio *bio)
 
 void fs_bio_integrity_generate(struct bio *bio)
 {
-	if (fs_bio_integrity_alloc(bio))
+	if (fs_bio_integrity_alloc(bio) &&
+	    (bio_integrity(bio)->bip_flags & BIP_CHECK_FLAGS))
 		bio_integrity_generate(bio);
 }
 EXPORT_SYMBOL_GPL(fs_bio_integrity_generate);
@@ -60,6 +61,9 @@ int fs_bio_integrity_verify(struct bio *bio, sector_t sector, unsigned int size)
 		.bi_size	= size,
 	};
 
+	if (!bip || !(bip->bip_flags & BIP_CHECK_FLAGS))
+		return 0;
+
 	/*
 	 * Reinitialize bip->bip_iter.
 	 *
-- 
2.53.0

Re: [PATCH 04/22] block: handle nogenerate/noverify properly in fs-integrity

[Anuj gupta]

Reviewed-by: Anuj Gupta <anuj20.g@samsung.com>

[PATCH 05/22] iomap: don't free integrity payload that doesn't exist

[Christoph Hellwig]

fs_bio_integrity_alloc might not allocate a bio integrity payload if PI
verification is disabled on the block device.  Check for that case before
calling fs_bio_integirty_free in iomap_bio_read_folio_range_sync to
avoid a NULL pointer dereferences.

Make the branch cover the PI verification as well - while
fs_bio_integrity_verify works without an integrity payload, it requires
one to actually do useful work.

Fixes: 0b10a370529c ("iomap: support T10 protection information")
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 fs/iomap/bio.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/fs/iomap/bio.c b/fs/iomap/bio.c
index dc8ac7e370a5..30ef78a66b4f 100644
--- a/fs/iomap/bio.c
+++ b/fs/iomap/bio.c
@@ -179,7 +179,7 @@ int iomap_bio_read_folio_range_sync(const struct iomap_iter *iter,
 	if (srcmap->flags & IOMAP_F_INTEGRITY)
 		fs_bio_integrity_alloc(&bio);
 	error = submit_bio_wait(&bio);
-	if (srcmap->flags & IOMAP_F_INTEGRITY) {
+	if (bio_integrity(&bio)) {
 		if (!error)
 			error = fs_bio_integrity_verify(&bio, sector, len);
 		fs_bio_integrity_free(&bio);
-- 
2.53.0

Re: [PATCH 05/22] iomap: don't free integrity payload that doesn't exist

[Darrick J. Wong]

On Thu, Jul 23, 2026 at 04:49:30PM +0200, Christoph Hellwig wrote:
> fs_bio_integrity_alloc might not allocate a bio integrity payload if PI
> verification is disabled on the block device.  Check for that case before
> calling fs_bio_integirty_free in iomap_bio_read_folio_range_sync to
> avoid a NULL pointer dereferences.
> 
> Make the branch cover the PI verification as well - while
> fs_bio_integrity_verify works without an integrity payload, it requires
> one to actually do useful work.
> 
> Fixes: 0b10a370529c ("iomap: support T10 protection information")
> Signed-off-by: Christoph Hellwig <hch@lst.de>

That makes sense.

Cc: <stable@vger.kernel.org> # v7.1
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>

--D

> ---
>  fs/iomap/bio.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/fs/iomap/bio.c b/fs/iomap/bio.c
> index dc8ac7e370a5..30ef78a66b4f 100644
> --- a/fs/iomap/bio.c
> +++ b/fs/iomap/bio.c
> @@ -179,7 +179,7 @@ int iomap_bio_read_folio_range_sync(const struct iomap_iter *iter,
>  	if (srcmap->flags & IOMAP_F_INTEGRITY)
>  		fs_bio_integrity_alloc(&bio);
>  	error = submit_bio_wait(&bio);
> -	if (srcmap->flags & IOMAP_F_INTEGRITY) {
> +	if (bio_integrity(&bio)) {
>  		if (!error)
>  			error = fs_bio_integrity_verify(&bio, sector, len);
>  		fs_bio_integrity_free(&bio);
> -- 
> 2.53.0
> 
> 

[PATCH 06/22] block,iomap: fix protection information verification with initial bvec offset

[Christoph Hellwig]

When reconstructing a bvec_iter from an ioend for protection information
verification, iomap currently ignores the offset into the initial
bio_vec.

This can't happen for buffered I/O an direct I/O to user addresses, but
is exercised by split on O_DIRECT file descriptors or when using the loop
driver.

Fortunately the only file system PI user (XFS) currently always bounce
buffers, so this can't actually be triggered yet.  But we'll want to make
the bounce buffering conditional soon, for which this needs to be fixed.

Store the initial offset in struct iomap_ioend, and pass a
pre-constructed bvec_iter to fs_bio_integrity_verify.  For the
synchronous read case the fix is even simpler as this path can
simply stash away the original bvec_iter.

Fixes: 0bde8a12b554 ("block: add fs_bio_integrity helpers")
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 block/bio-integrity-fs.c      | 13 +++++--------
 fs/iomap/bio.c                |  4 +++-
 fs/iomap/ioend.c              | 14 ++++++++++----
 include/linux/bio-integrity.h |  3 +--
 include/linux/iomap.h         |  8 ++++++++
 5 files changed, 27 insertions(+), 15 deletions(-)

diff --git a/block/bio-integrity-fs.c b/block/bio-integrity-fs.c
index 692403dfa047..dd6c85530913 100644
--- a/block/bio-integrity-fs.c
+++ b/block/bio-integrity-fs.c
@@ -52,14 +52,10 @@ void fs_bio_integrity_generate(struct bio *bio)
 }
 EXPORT_SYMBOL_GPL(fs_bio_integrity_generate);
 
-int fs_bio_integrity_verify(struct bio *bio, sector_t sector, unsigned int size)
+int fs_bio_integrity_verify(struct bio *bio, struct bvec_iter *data_iter)
 {
 	struct blk_integrity *bi = blk_get_integrity(bio->bi_bdev->bd_disk);
 	struct bio_integrity_payload *bip = bio_integrity(bio);
-	struct bvec_iter data_iter = {
-		.bi_sector	= sector,
-		.bi_size	= size,
-	};
 
 	if (!bip || !(bip->bip_flags & BIP_CHECK_FLAGS))
 		return 0;
@@ -71,9 +67,10 @@ int fs_bio_integrity_verify(struct bio *bio, sector_t sector, unsigned int size)
 	 * bio.  Requires the submitter to remember the sector and the size.
 	 */
 	memset(&bip->bip_iter, 0, sizeof(bip->bip_iter));
-	bip->bip_iter.bi_sector = sector;
-	bip->bip_iter.bi_size = bio_integrity_bytes(bi, size >> SECTOR_SHIFT);
-	return blk_status_to_errno(bio_integrity_verify(bio, &data_iter));
+	bip->bip_iter.bi_sector = data_iter->bi_sector;
+	bip->bip_iter.bi_size =
+		bio_integrity_bytes(bi, data_iter->bi_size >> SECTOR_SHIFT);
+	return blk_status_to_errno(bio_integrity_verify(bio, data_iter));
 }
 
 static int __init fs_bio_integrity_init(void)
diff --git a/fs/iomap/bio.c b/fs/iomap/bio.c
index 30ef78a66b4f..813002b2299d 100644
--- a/fs/iomap/bio.c
+++ b/fs/iomap/bio.c
@@ -169,6 +169,7 @@ int iomap_bio_read_folio_range_sync(const struct iomap_iter *iter,
 {
 	const struct iomap *srcmap = iomap_iter_srcmap(iter);
 	sector_t sector = iomap_sector(srcmap, pos);
+	struct bvec_iter saved_iter;
 	struct bio_vec bvec;
 	struct bio bio;
 	int error;
@@ -178,10 +179,11 @@ int iomap_bio_read_folio_range_sync(const struct iomap_iter *iter,
 	bio_add_folio_nofail(&bio, folio, len, offset_in_folio(folio, pos));
 	if (srcmap->flags & IOMAP_F_INTEGRITY)
 		fs_bio_integrity_alloc(&bio);
+	saved_iter = bio.bi_iter;
 	error = submit_bio_wait(&bio);
 	if (bio_integrity(&bio)) {
 		if (!error)
-			error = fs_bio_integrity_verify(&bio, sector, len);
+			error = fs_bio_integrity_verify(&bio, &saved_iter);
 		fs_bio_integrity_free(&bio);
 	}
 	return error;
diff --git a/fs/iomap/ioend.c b/fs/iomap/ioend.c
index 2ec755a89228..d43e6229900c 100644
--- a/fs/iomap/ioend.c
+++ b/fs/iomap/ioend.c
@@ -25,6 +25,7 @@ struct iomap_ioend *iomap_init_ioend(struct inode *inode,
 	ioend->io_parent = NULL;
 	INIT_LIST_HEAD(&ioend->io_list);
 	ioend->io_flags = ioend_flags;
+	ioend->io_bvec_offset = bio->bi_iter.bi_bvec_done;
 	ioend->io_inode = inode;
 	ioend->io_offset = file_offset;
 	ioend->io_size = bio->bi_iter.bi_size;
@@ -311,6 +312,13 @@ ssize_t iomap_add_to_ioend(struct iomap_writepage_ctx *wpc, struct folio *folio,
 }
 EXPORT_SYMBOL_GPL(iomap_add_to_ioend);
 
+static int iomap_ioend_integrity_verify(struct iomap_ioend *ioend)
+{
+	struct bvec_iter data_iter = BVEC_ITER_IOEND(ioend);
+
+	return fs_bio_integrity_verify(&ioend->io_bio, &data_iter);
+}
+
 static u32 iomap_finish_ioend(struct iomap_ioend *ioend, int error)
 {
 	if (ioend->io_parent) {
@@ -328,10 +336,8 @@ static u32 iomap_finish_ioend(struct iomap_ioend *ioend, int error)
 
 	if (!ioend->io_error &&
 	    bio_integrity(&ioend->io_bio) &&
-	    bio_op(&ioend->io_bio) == REQ_OP_READ) {
-		ioend->io_error = fs_bio_integrity_verify(&ioend->io_bio,
-			ioend->io_sector, ioend->io_size);
-	}
+	    bio_op(&ioend->io_bio) == REQ_OP_READ)
+		ioend->io_error = iomap_ioend_integrity_verify(ioend);
 
 	if (ioend->io_flags & IOMAP_IOEND_DIRECT)
 		return iomap_finish_ioend_direct(ioend);
diff --git a/include/linux/bio-integrity.h b/include/linux/bio-integrity.h
index 0ea2a8bf7efb..a954c97be0b3 100644
--- a/include/linux/bio-integrity.h
+++ b/include/linux/bio-integrity.h
@@ -151,7 +151,6 @@ void bio_integrity_setup_default(struct bio *bio);
 unsigned int fs_bio_integrity_alloc(struct bio *bio);
 void fs_bio_integrity_free(struct bio *bio);
 void fs_bio_integrity_generate(struct bio *bio);
-int fs_bio_integrity_verify(struct bio *bio, sector_t sector,
-		unsigned int size);
+int fs_bio_integrity_verify(struct bio *bio, struct bvec_iter *data_iter);
 
 #endif /* _LINUX_BIO_INTEGRITY_H */
diff --git a/include/linux/iomap.h b/include/linux/iomap.h
index 56b43d594e6e..e7bee96379df 100644
--- a/include/linux/iomap.h
+++ b/include/linux/iomap.h
@@ -424,6 +424,7 @@ sector_t iomap_bmap(struct address_space *mapping, sector_t bno,
 struct iomap_ioend {
 	struct list_head	io_list;	/* next ioend in chain */
 	u16			io_flags;	/* IOMAP_IOEND_* */
+	u32			io_bvec_offset;	/* offset into first bvec */
 	struct inode		*io_inode;	/* file being written to */
 	size_t			io_size;	/* size of the extent */
 	atomic_t		io_remaining;	/* completetion defer count */
@@ -441,6 +442,13 @@ static inline struct iomap_ioend *iomap_ioend_from_bio(struct bio *bio)
 	return container_of(bio, struct iomap_ioend, io_bio);
 }
 
+#define BVEC_ITER_IOEND(_ioend)				\
+{							\
+	.bi_sector	= (_ioend)->io_sector,		\
+	.bi_size	= (_ioend)->io_size,		\
+	.bi_bvec_done	= (_ioend)->io_bvec_offset,	\
+}
+
 struct iomap_writeback_ops {
 	/*
 	 * Performs writeback on the passed in range
-- 
2.53.0

[PATCH 07/22] block: add task-context bio completion infrastructure

[Christoph Hellwig]

From: Tal Zussman <tz2294@columbia.edu>

Some bio completion handlers need to run from preemptible task context,
but bio_endio() may be called from IRQ context (e.g., buffer_head
writeback). Callers need a way to ensure their callback eventually runs
from a sleepable context. Add infrastructure for that, in two forms:

  1. BIO_COMPLETE_IN_TASK, a bio flag the submitter sets when it knows
     in advance that its callback needs task context (e.g., dropbehind
     writeback). bio_endio() sees the flag and offloads completion to a
     worker automatically.

  2. bio_complete_in_task(), a helper that completion callbacks can
     invoke from within bi_end_io() when the deferral decision is
     dynamic (e.g., fserror reporting).

Both share a per-CPU batch list drained by a delayed work item on a
WQ_PERCPU workqueue. Producers push the bio onto the local CPU's batch
and schedule the work item, which then dispatches each bio's bi_end_io()
from task context. The delayed work item uses a 1-jiffie delay to allow
batches of completions to accumulate before processing.

Both methods are gated on bio_in_atomic(), which returns true in any
context where a sleeping bi_end_io() is unsafe, including
non-preemptible task context. This logic is copied from commit
c99fab6e80b7 ("erofs: fix atomic context detection when
!CONFIG_DEBUG_LOCK_ALLOC").

Two CPU hotplug callbacks are used to drain remaining bios from the
departing CPU's batch, while maintaining the per-CPU behavior. The
CPUHP_AP_ONLINE_DYN callback disables the per-CPU delayed work while the
CPU is still online, preventing it from running on an unbound worker
later. CPUHP_BP_PREPARE_DYN then drains any bios added between disabling
the work item and CPU offline.

Link: https://lore.kernel.org/all/20260409160243.1008358-1-hch@lst.de/
Suggested-by: Matthew Wilcox <willy@infradead.org>
Suggested-by: Christoph Hellwig <hch@infradead.org>
Signed-off-by: Tal Zussman <tz2294@columbia.edu>
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 block/bio.c               | 147 +++++++++++++++++++++++++++++++++++++-
 include/linux/bio.h       |  32 +++++++++
 include/linux/blk_types.h |   1 +
 3 files changed, 179 insertions(+), 1 deletion(-)

diff --git a/block/bio.c b/block/bio.c
index 6a2f6fc3413e..f228940d6b42 100644
--- a/block/bio.c
+++ b/block/bio.c
@@ -19,6 +19,7 @@
 #include <linux/blk-crypto.h>
 #include <linux/xarray.h>
 #include <linux/kmemleak.h>
+#include <linux/local_lock.h>
 
 #include <trace/events/block.h>
 #include "blk.h"
@@ -1741,6 +1742,79 @@ void bio_check_pages_dirty(struct bio *bio)
 	schedule_work(&bio_dirty_work);
 }
 
+/*
+ * Infrastructure for deferring bio completions to task-context via a per-CPU
+ * workqueue. Triggered either by the BIO_COMPLETE_IN_TASK bio flag (static
+ * decision at submit time) or by calling bio_complete_in_task() from
+ * bi_end_io() (dynamic decision at completion time).
+ */
+
+struct bio_complete_batch {
+	local_lock_t lock;
+	struct bio_list list;
+	struct delayed_work work;
+	int cpu;
+};
+
+static DEFINE_PER_CPU(struct bio_complete_batch, bio_complete_batch) = {
+	.lock = INIT_LOCAL_LOCK(lock),
+};
+static struct workqueue_struct *bio_complete_wq;
+
+static void bio_complete_work_fn(struct work_struct *w)
+{
+	struct delayed_work *dw = to_delayed_work(w);
+	struct bio_complete_batch *batch =
+		container_of(dw, struct bio_complete_batch, work);
+
+	while (1) {
+		struct bio_list list;
+		struct bio *bio;
+
+		local_lock_irq(&bio_complete_batch.lock);
+		list = batch->list;
+		bio_list_init(&batch->list);
+		local_unlock_irq(&bio_complete_batch.lock);
+
+		if (bio_list_empty(&list))
+			break;
+
+		while ((bio = bio_list_pop(&list)))
+			bio->bi_end_io(bio);
+
+		if (need_resched()) {
+			bool is_empty;
+
+			local_lock_irq(&bio_complete_batch.lock);
+			is_empty = bio_list_empty(&batch->list);
+			local_unlock_irq(&bio_complete_batch.lock);
+			if (!is_empty)
+				mod_delayed_work_on(batch->cpu,
+						    bio_complete_wq,
+						    &batch->work, 0);
+			break;
+		}
+	}
+}
+
+void __bio_complete_in_task(struct bio *bio)
+{
+	struct bio_complete_batch *batch;
+	unsigned long flags;
+	bool was_empty;
+
+	local_lock_irqsave(&bio_complete_batch.lock, flags);
+	batch = this_cpu_ptr(&bio_complete_batch);
+	was_empty = bio_list_empty(&batch->list);
+	bio_list_add(&batch->list, bio);
+	local_unlock_irqrestore(&bio_complete_batch.lock, flags);
+
+	if (was_empty)
+		mod_delayed_work_on(batch->cpu, bio_complete_wq,
+				    &batch->work, 1);
+}
+EXPORT_SYMBOL_GPL(__bio_complete_in_task);
+
 static inline bool bio_remaining_done(struct bio *bio)
 {
 	/*
@@ -1815,7 +1889,9 @@ void bio_endio(struct bio *bio)
 	}
 #endif
 
-	if (bio->bi_end_io)
+	if (bio_flagged(bio, BIO_COMPLETE_IN_TASK) && bio_in_atomic())
+		__bio_complete_in_task(bio);
+	else if (bio->bi_end_io)
 		bio->bi_end_io(bio);
 }
 EXPORT_SYMBOL(bio_endio);
@@ -2001,6 +2077,51 @@ int bioset_init(struct bio_set *bs,
 }
 EXPORT_SYMBOL(bioset_init);
 
+static int bio_complete_batch_cpu_online(unsigned int cpu)
+{
+	enable_delayed_work(&per_cpu(bio_complete_batch, cpu).work);
+	return 0;
+}
+
+/*
+ * Disable this CPU's delayed work so that it cannot run on an unbound worker
+ * after the CPU is offlined.
+ */
+static int bio_complete_batch_cpu_down_prep(unsigned int cpu)
+{
+	disable_delayed_work_sync(&per_cpu(bio_complete_batch, cpu).work);
+	return 0;
+}
+
+/*
+ * Drain a dead CPU's deferred bio completions. The CPU is dead and the worker
+ * is canceled so no locking is needed.
+ */
+static int bio_complete_batch_cpu_dead(unsigned int cpu)
+{
+	struct bio_complete_batch *batch =
+		per_cpu_ptr(&bio_complete_batch, cpu);
+	struct bio *bio;
+
+	while ((bio = bio_list_pop(&batch->list)))
+		bio->bi_end_io(bio);
+
+	return 0;
+}
+
+static void __init bio_complete_batch_init(int cpu)
+{
+	struct bio_complete_batch *batch =
+		per_cpu_ptr(&bio_complete_batch, cpu);
+
+	bio_list_init(&batch->list);
+	INIT_DELAYED_WORK(&batch->work, bio_complete_work_fn);
+	batch->cpu = cpu;
+
+	if (!cpu_online(cpu))
+		disable_delayed_work_sync(&batch->work);
+}
+
 static int __init init_bio(void)
 {
 	int i;
@@ -2015,6 +2136,30 @@ static int __init init_bio(void)
 				SLAB_HWCACHE_ALIGN | SLAB_PANIC, NULL);
 	}
 
+	for_each_possible_cpu(i)
+		bio_complete_batch_init(i);
+
+	bio_complete_wq = alloc_workqueue("bio_complete",
+					   WQ_MEM_RECLAIM | WQ_PERCPU, 0);
+	if (!bio_complete_wq)
+		panic("bio: can't allocate bio_complete workqueue\n");
+
+	/*
+	 * bio task-context completion draining on hot-unplugged CPUs:
+	 *
+	 *   1. Stop the per-CPU delayed work while the CPU is still online, so
+	 *      that it cannot run on an unbound worker later.
+	 *   2. Drain leftover bios added between worker disabling and CPU
+	 *      offlining.
+	 */
+	cpuhp_setup_state_nocalls(CPUHP_AP_ONLINE_DYN,
+				  "block/bio:complete:online",
+				  bio_complete_batch_cpu_online,
+				  bio_complete_batch_cpu_down_prep);
+	cpuhp_setup_state_nocalls(CPUHP_BP_PREPARE_DYN,
+				  "block/bio:complete:dead",
+				  NULL, bio_complete_batch_cpu_dead);
+
 	cpuhp_setup_state_multi(CPUHP_BIO_DEAD, "block/bio:dead", NULL,
 					bio_cpu_dead);
 
diff --git a/include/linux/bio.h b/include/linux/bio.h
index 8f33f717b14f..501847105aa8 100644
--- a/include/linux/bio.h
+++ b/include/linux/bio.h
@@ -368,6 +368,38 @@ static inline struct bio *bio_alloc(struct block_device *bdev,
 
 void submit_bio(struct bio *bio);
 
+/**
+ * bio_in_atomic - check if the current context is unsafe for bio completion
+ *
+ * Return: %true in atomic contexts (e.g. hard/soft IRQ, preempt-disabled);
+ * %false when a bio can be safely completed in the current context.
+ */
+static inline bool bio_in_atomic(void)
+{
+	if (IS_ENABLED(CONFIG_PREEMPTION) && rcu_preempt_depth())
+		return true;
+	if (!IS_ENABLED(CONFIG_PREEMPT_COUNT))
+		return true;
+	return !preemptible();
+}
+
+void __bio_complete_in_task(struct bio *bio);
+
+/**
+ * bio_complete_in_task - ensure a bio is completed in preemptible task context
+ * @bio: bio to complete
+ *
+ * If called from non-task context, offload the bio completion to a worker
+ * thread and return %true. Else return %false and do nothing.
+ */
+static inline bool bio_complete_in_task(struct bio *bio)
+{
+	if (!bio_in_atomic())
+		return false;
+	__bio_complete_in_task(bio);
+	return true;
+}
+
 extern void bio_endio(struct bio *);
 
 /**
diff --git a/include/linux/blk_types.h b/include/linux/blk_types.h
index 8808ee76e73c..d49d97a050d0 100644
--- a/include/linux/blk_types.h
+++ b/include/linux/blk_types.h
@@ -322,6 +322,7 @@ enum {
 	BIO_REMAPPED,
 	BIO_ZONE_WRITE_PLUGGING, /* bio handled through zone write plugging */
 	BIO_EMULATES_ZONE_APPEND, /* bio emulates a zone append operation */
+	BIO_COMPLETE_IN_TASK, /* complete bi_end_io() in task context */
 	BIO_FLAG_LAST
 };
 
-- 
2.53.0

[PATCH 08/22] block: don't delay bio task completions

[Christoph Hellwig]

The delay causes a more than an order of magnitude slow down when using
it to defer the T10 PI processing.

Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 block/bio.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/block/bio.c b/block/bio.c
index f228940d6b42..79fb94f85a5f 100644
--- a/block/bio.c
+++ b/block/bio.c
@@ -1811,7 +1811,7 @@ void __bio_complete_in_task(struct bio *bio)
 
 	if (was_empty)
 		mod_delayed_work_on(batch->cpu, bio_complete_wq,
-				    &batch->work, 1);
+				    &batch->work, 0);
 }
 EXPORT_SYMBOL_GPL(__bio_complete_in_task);
 
-- 
2.53.0

[PATCH 09/22] block: split bio_iov_iter_bounce_write

[Christoph Hellwig]

Factor out a bio_alloc_bounce_folios helper that we'll use for a
different take on read-side bounce buffering soon.

For that make it and also bio_free_folios available to callers outside of
bio.c.

Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 block/bio.c         | 58 ++++++++++++++++++++++++++-------------------
 include/linux/bio.h |  2 ++
 2 files changed, 35 insertions(+), 25 deletions(-)

diff --git a/block/bio.c b/block/bio.c
index 79fb94f85a5f..545cd8622ebd 100644
--- a/block/bio.c
+++ b/block/bio.c
@@ -1298,7 +1298,7 @@ static struct folio *folio_alloc_greedy(gfp_t gfp, size_t *size,
 	return folio_alloc(gfp, get_order(*size));
 }
 
-static void bio_free_folios(struct bio *bio)
+void bio_free_folios(struct bio *bio)
 {
 	struct bio_vec *bv;
 	int i;
@@ -1311,11 +1311,8 @@ static void bio_free_folios(struct bio *bio)
 	}
 }
 
-static int bio_iov_iter_bounce_write(struct bio *bio, struct iov_iter *iter,
-		size_t maxlen, size_t minsize)
+int bio_alloc_bounce_folios(struct bio *bio, size_t total_len, size_t minsize)
 {
-	size_t total_len = min(maxlen, iov_iter_count(iter));
-
 	if (WARN_ON_ONCE(bio_flagged(bio, BIO_CLONED)))
 		return -EINVAL;
 	if (WARN_ON_ONCE(bio->bi_iter.bi_size))
@@ -1325,7 +1322,6 @@ static int bio_iov_iter_bounce_write(struct bio *bio, struct iov_iter *iter,
 
 	do {
 		size_t this_len = min(total_len, SZ_1M);
-		size_t copied;
 		struct folio *folio;
 
 		if (this_len > minsize * 2)
@@ -1338,32 +1334,44 @@ static int bio_iov_iter_bounce_write(struct bio *bio, struct iov_iter *iter,
 		if (!folio)
 			break;
 		bio_add_folio_nofail(bio, folio, this_len, 0);
+		total_len -= this_len;
+	} while (total_len && bio->bi_vcnt < bio->bi_max_vecs);
+
+	if (!bio->bi_iter.bi_size)
+		return -ENOMEM;
+	return 0;
+}
+
+static int bio_iov_iter_bounce_write(struct bio *bio, struct iov_iter *iter,
+		size_t maxlen, size_t minsize)
+{
+	size_t total_len = min(maxlen, iov_iter_count(iter));
+	size_t total_copied = 0;
+	struct bio_vec *bv;
+	int i, error;
+
+	error = bio_alloc_bounce_folios(bio, total_len, minsize);
+	if (error)
+		return error;
+
+	bio_for_each_bvec_all(bv, bio, i) {
+		struct folio *folio = page_folio(bv->bv_page);
+		size_t copied;
 
 		if (iter->nofault)
-			copied = copy_folio_from_iter_atomic(folio, 0, this_len,
-							     iter);
+			copied = copy_folio_from_iter_atomic(folio, 0,
+					bv->bv_len, iter);
 		else
-			copied = copy_folio_from_iter(folio, 0, this_len, iter);
-		if (copied < this_len) {
-			/*
-			 * Need to revert the iov iter for all bytes we have
-			 * copied.
-			 *
-			 * However the bio size differs from the real copied
-			 * bytes as @this_len is queued but only advanced
-			 * less than that.
-			 * Need to compensate that for the revert.
-			 */
-			iov_iter_revert(iter, bio->bi_iter.bi_size - this_len +
-					copied);
+			copied = copy_folio_from_iter(folio, 0, bv->bv_len,
+					iter);
+		total_copied += copied;
+		if (copied < bv->bv_len) {
+			iov_iter_revert(iter, total_copied);
 			bio_free_folios(bio);
 			return -EFAULT;
 		}
-		total_len -= this_len;
-	} while (total_len && bio->bi_vcnt < bio->bi_max_vecs);
+	}
 
-	if (!bio->bi_iter.bi_size)
-		return -ENOMEM;
 	return bio_iov_iter_align_down(bio, iter,
 			&bio->bi_io_vec[bio->bi_vcnt - 1], minsize - 1);
 }
diff --git a/include/linux/bio.h b/include/linux/bio.h
index 501847105aa8..7db87ea0e1b0 100644
--- a/include/linux/bio.h
+++ b/include/linux/bio.h
@@ -519,6 +519,8 @@ extern void bio_check_pages_dirty(struct bio *bio);
 int bio_iov_iter_bounce(struct bio *bio, struct iov_iter *iter, size_t maxlen,
 		size_t minsize);
 void bio_iov_iter_unbounce(struct bio *bio, bool is_error, bool mark_dirty);
+int bio_alloc_bounce_folios(struct bio *bio, size_t total_len, size_t minsize);
+void bio_free_folios(struct bio *bio);
 
 extern void bio_copy_data(struct bio *dst, struct bio *src);
 extern void bio_free_pages(struct bio *bio);
-- 
2.53.0

[PATCH 10/22] block: export fs_bio_integrity_{alloc,free}

[Christoph Hellwig]

We'll move integrity generation and verification into the iomap
submission helpers, which means they will be needed in modular file
system code.

Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 block/bio-integrity-fs.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/block/bio-integrity-fs.c b/block/bio-integrity-fs.c
index dd6c85530913..c8e91ada8ca6 100644
--- a/block/bio-integrity-fs.c
+++ b/block/bio-integrity-fs.c
@@ -31,6 +31,7 @@ unsigned int fs_bio_integrity_alloc(struct bio *bio)
 		bio_integrity_setup_default(bio);
 	return action;
 }
+EXPORT_SYMBOL_GPL(fs_bio_integrity_alloc);
 
 void fs_bio_integrity_free(struct bio *bio)
 {
@@ -43,6 +44,7 @@ void fs_bio_integrity_free(struct bio *bio)
 	bio->bi_integrity = NULL;
 	bio->bi_opf &= ~REQ_INTEGRITY;
 }
+EXPORT_SYMBOL_GPL(fs_bio_integrity_free);
 
 void fs_bio_integrity_generate(struct bio *bio)
 {
-- 
2.53.0

[PATCH 11/22] block: don't include blk-integrity.h in bdev.c

[Christoph Hellwig]

Not needed, probably since this file was split out.

Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 block/bdev.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/block/bdev.c b/block/bdev.c
index 85ce57bd2ae4..90baef0bd174 100644
--- a/block/bdev.c
+++ b/block/bdev.c
@@ -12,7 +12,6 @@
 #include <linux/major.h>
 #include <linux/device_cgroup.h>
 #include <linux/blkdev.h>
-#include <linux/blk-integrity.h>
 #include <linux/backing-dev.h>
 #include <linux/module.h>
 #include <linux/blkpg.h>
-- 
2.53.0

[PATCH 12/22] iomap: better read bounce buffering support

[Christoph Hellwig]

Add helpers to bounce buffer an upper bio into one or more lower bios
using bounce buffers, and to copy the data back on completion.

Compared to the existing IOMAP_DIO_BOUNCE support for read bios, this
has two advantages:  by removing the special bounce bio_vec it allows
to the full and "round" size of a single bio, i.e., 1MiB when using
4k pages.  This is important for good performance on HDD.  Additionally
it allows to bounce buffer a bio from completion conext, and thus
implement a "lazy" bounce buffering scheme, where the data is only
read into a bounce buffer after an initial checksum validation failure,
thus avoiding the bounce buffering I/O for most I/O.

Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 fs/iomap/ioend.c      | 87 +++++++++++++++++++++++++++++++++++++++++++
 include/linux/iomap.h |  5 +++
 2 files changed, 92 insertions(+)

diff --git a/fs/iomap/ioend.c b/fs/iomap/ioend.c
index d43e6229900c..34f71c8b5ebd 100644
--- a/fs/iomap/ioend.c
+++ b/fs/iomap/ioend.c
@@ -515,6 +515,93 @@ struct iomap_ioend *iomap_split_ioend(struct iomap_ioend *ioend,
 }
 EXPORT_SYMBOL_GPL(iomap_split_ioend);
 
+void iomap_bounce_read(struct iomap_ioend *orig_ioend, unsigned int minsize,
+		void (*submit_ioend)(struct iomap_ioend *ioend))
+{
+	struct inode *inode = orig_ioend->io_inode;
+	struct bio *orig_bio = &orig_ioend->io_bio;
+	loff_t file_offset = orig_ioend->io_offset;
+	sector_t sector = orig_ioend->io_sector;
+	size_t total_len = round_up(orig_ioend->io_size, minsize);
+
+	WARN_ON_ONCE(!(orig_ioend->io_flags & IOMAP_IOEND_DIRECT));
+
+	do {
+		struct iomap_ioend *ioend;
+		struct bio *bio;
+		int error;
+
+		bio = bio_alloc_bioset(orig_bio->bi_bdev,
+				min(total_len / minsize, BIO_MAX_VECS),
+				orig_bio->bi_opf, GFP_KERNEL,
+				&iomap_ioend_split_bioset);
+		error = bio_alloc_bounce_folios(bio, total_len, minsize);
+		if (error) {
+			bio_put(bio);
+			orig_bio->bi_status = errno_to_blk_status(error);
+			break;
+		}
+		bio->bi_ioprio = orig_bio->bi_ioprio;
+		bio->bi_write_hint = orig_bio->bi_write_hint;
+		bio->bi_write_stream = orig_bio->bi_write_stream;
+		bio->bi_iter.bi_sector = sector;
+
+		ioend = iomap_init_ioend(inode, bio, file_offset,
+				orig_ioend->io_flags);
+
+		total_len -= bio->bi_iter.bi_size;
+		file_offset += bio->bi_iter.bi_size;
+		sector += (bio->bi_iter.bi_size >> SECTOR_SHIFT);
+
+		bio->bi_private = orig_bio;
+		bio_inc_remaining(orig_bio);
+		submit_ioend(ioend);
+	} while (total_len > 0);
+
+	bio_endio(&orig_ioend->io_bio);
+}
+EXPORT_SYMBOL_GPL(iomap_bounce_read);
+
+static void iomap_ioend_unbounce(struct iomap_ioend *orig_ioend,
+		struct iomap_ioend *ioend)
+{
+	struct bio *orig_bio = &orig_ioend->io_bio;
+	struct iov_iter to;
+	struct bio_vec *bv;
+	int i;
+
+	iov_iter_bvec(&to, ITER_DEST, orig_bio->bi_io_vec, orig_bio->bi_vcnt,
+			orig_ioend->io_size);
+	to.iov_offset = orig_ioend->io_bvec_offset;
+
+	if (ioend->io_offset != orig_ioend->io_offset) {
+		WARN_ON_ONCE(ioend->io_offset < orig_ioend->io_offset);
+		iov_iter_advance(&to, ioend->io_offset - orig_ioend->io_offset);
+	}
+
+	/* copying to pinned pages should always work */
+	bio_for_each_bvec_all(bv, &ioend->io_bio, i)
+		WARN_ON_ONCE(copy_to_iter(bvec_virt(bv), bv->bv_len, &to) !=
+				bv->bv_len);
+}
+
+void iomap_bounce_read_end_io(struct iomap_ioend *ioend, struct bio *orig_bio,
+		int error)
+{
+	if (error)
+		orig_bio->bi_status = errno_to_blk_status(error);
+	else
+		iomap_ioend_unbounce(iomap_ioend_from_bio(orig_bio), ioend);
+
+	bio_free_folios(&ioend->io_bio);
+	if (bio_integrity(&ioend->io_bio))
+		fs_bio_integrity_free(&ioend->io_bio);
+	bio_put(&ioend->io_bio);
+
+	bio_endio(orig_bio);
+}
+EXPORT_SYMBOL_GPL(iomap_bounce_read_end_io);
+
 static int __init iomap_ioend_init(void)
 {
 	const unsigned int nr_mempool_entries = 4 * (PAGE_SIZE / SECTOR_SIZE);
diff --git a/include/linux/iomap.h b/include/linux/iomap.h
index e7bee96379df..ff9afdd39682 100644
--- a/include/linux/iomap.h
+++ b/include/linux/iomap.h
@@ -509,6 +509,11 @@ void iomap_finish_folio_write(struct inode *inode, struct folio *folio,
 int iomap_writeback_folio(struct iomap_writepage_ctx *wpc, struct folio *folio);
 int iomap_writepages(struct iomap_writepage_ctx *wpc);
 
+void iomap_bounce_read(struct iomap_ioend *orig_ioend, unsigned int minsize,
+		void (*submit_ioend)(struct iomap_ioend *ioend));
+void iomap_bounce_read_end_io(struct iomap_ioend *ioend, struct bio *orig_bio,
+		int error);
+
 struct iomap_read_folio_ctx {
 	const struct iomap_read_ops *ops;
 	struct folio		*cur_folio;
-- 
2.53.0

Re: [PATCH 12/22] iomap: better read bounce buffering support

[Darrick J. Wong]

On Thu, Jul 23, 2026 at 04:49:37PM +0200, Christoph Hellwig wrote:
> Add helpers to bounce buffer an upper bio into one or more lower bios
> using bounce buffers, and to copy the data back on completion.
> 
> Compared to the existing IOMAP_DIO_BOUNCE support for read bios, this
> has two advantages:  by removing the special bounce bio_vec it allows
> to the full and "round" size of a single bio, i.e., 1MiB when using
> 4k pages.  This is important for good performance on HDD.  Additionally
> it allows to bounce buffer a bio from completion conext, and thus
> implement a "lazy" bounce buffering scheme, where the data is only
> read into a bounce buffer after an initial checksum validation failure,
> thus avoiding the bounce buffering I/O for most I/O.
> 
> Signed-off-by: Christoph Hellwig <hch@lst.de>

Now that I can see how this is used ("xfs: add support for lazy direct
read bounce buffering") I think I'm ok with this.  Should there be some
Documentation/ about how to use PI with iomap?

In the mean time,
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>

--D

> ---
>  fs/iomap/ioend.c      | 87 +++++++++++++++++++++++++++++++++++++++++++
>  include/linux/iomap.h |  5 +++
>  2 files changed, 92 insertions(+)
> 
> diff --git a/fs/iomap/ioend.c b/fs/iomap/ioend.c
> index d43e6229900c..34f71c8b5ebd 100644
> --- a/fs/iomap/ioend.c
> +++ b/fs/iomap/ioend.c
> @@ -515,6 +515,93 @@ struct iomap_ioend *iomap_split_ioend(struct iomap_ioend *ioend,
>  }
>  EXPORT_SYMBOL_GPL(iomap_split_ioend);
>  
> +void iomap_bounce_read(struct iomap_ioend *orig_ioend, unsigned int minsize,
> +		void (*submit_ioend)(struct iomap_ioend *ioend))
> +{
> +	struct inode *inode = orig_ioend->io_inode;
> +	struct bio *orig_bio = &orig_ioend->io_bio;
> +	loff_t file_offset = orig_ioend->io_offset;
> +	sector_t sector = orig_ioend->io_sector;
> +	size_t total_len = round_up(orig_ioend->io_size, minsize);
> +
> +	WARN_ON_ONCE(!(orig_ioend->io_flags & IOMAP_IOEND_DIRECT));
> +
> +	do {
> +		struct iomap_ioend *ioend;
> +		struct bio *bio;
> +		int error;
> +
> +		bio = bio_alloc_bioset(orig_bio->bi_bdev,
> +				min(total_len / minsize, BIO_MAX_VECS),
> +				orig_bio->bi_opf, GFP_KERNEL,
> +				&iomap_ioend_split_bioset);
> +		error = bio_alloc_bounce_folios(bio, total_len, minsize);
> +		if (error) {
> +			bio_put(bio);
> +			orig_bio->bi_status = errno_to_blk_status(error);
> +			break;
> +		}
> +		bio->bi_ioprio = orig_bio->bi_ioprio;
> +		bio->bi_write_hint = orig_bio->bi_write_hint;
> +		bio->bi_write_stream = orig_bio->bi_write_stream;
> +		bio->bi_iter.bi_sector = sector;
> +
> +		ioend = iomap_init_ioend(inode, bio, file_offset,
> +				orig_ioend->io_flags);
> +
> +		total_len -= bio->bi_iter.bi_size;
> +		file_offset += bio->bi_iter.bi_size;
> +		sector += (bio->bi_iter.bi_size >> SECTOR_SHIFT);
> +
> +		bio->bi_private = orig_bio;
> +		bio_inc_remaining(orig_bio);
> +		submit_ioend(ioend);
> +	} while (total_len > 0);
> +
> +	bio_endio(&orig_ioend->io_bio);
> +}
> +EXPORT_SYMBOL_GPL(iomap_bounce_read);
> +
> +static void iomap_ioend_unbounce(struct iomap_ioend *orig_ioend,
> +		struct iomap_ioend *ioend)
> +{
> +	struct bio *orig_bio = &orig_ioend->io_bio;
> +	struct iov_iter to;
> +	struct bio_vec *bv;
> +	int i;
> +
> +	iov_iter_bvec(&to, ITER_DEST, orig_bio->bi_io_vec, orig_bio->bi_vcnt,
> +			orig_ioend->io_size);
> +	to.iov_offset = orig_ioend->io_bvec_offset;
> +
> +	if (ioend->io_offset != orig_ioend->io_offset) {
> +		WARN_ON_ONCE(ioend->io_offset < orig_ioend->io_offset);
> +		iov_iter_advance(&to, ioend->io_offset - orig_ioend->io_offset);
> +	}
> +
> +	/* copying to pinned pages should always work */
> +	bio_for_each_bvec_all(bv, &ioend->io_bio, i)
> +		WARN_ON_ONCE(copy_to_iter(bvec_virt(bv), bv->bv_len, &to) !=
> +				bv->bv_len);
> +}
> +
> +void iomap_bounce_read_end_io(struct iomap_ioend *ioend, struct bio *orig_bio,
> +		int error)
> +{
> +	if (error)
> +		orig_bio->bi_status = errno_to_blk_status(error);
> +	else
> +		iomap_ioend_unbounce(iomap_ioend_from_bio(orig_bio), ioend);
> +
> +	bio_free_folios(&ioend->io_bio);
> +	if (bio_integrity(&ioend->io_bio))
> +		fs_bio_integrity_free(&ioend->io_bio);
> +	bio_put(&ioend->io_bio);
> +
> +	bio_endio(orig_bio);
> +}
> +EXPORT_SYMBOL_GPL(iomap_bounce_read_end_io);
> +
>  static int __init iomap_ioend_init(void)
>  {
>  	const unsigned int nr_mempool_entries = 4 * (PAGE_SIZE / SECTOR_SIZE);
> diff --git a/include/linux/iomap.h b/include/linux/iomap.h
> index e7bee96379df..ff9afdd39682 100644
> --- a/include/linux/iomap.h
> +++ b/include/linux/iomap.h
> @@ -509,6 +509,11 @@ void iomap_finish_folio_write(struct inode *inode, struct folio *folio,
>  int iomap_writeback_folio(struct iomap_writepage_ctx *wpc, struct folio *folio);
>  int iomap_writepages(struct iomap_writepage_ctx *wpc);
>  
> +void iomap_bounce_read(struct iomap_ioend *orig_ioend, unsigned int minsize,
> +		void (*submit_ioend)(struct iomap_ioend *ioend));
> +void iomap_bounce_read_end_io(struct iomap_ioend *ioend, struct bio *orig_bio,
> +		int error);
> +
>  struct iomap_read_folio_ctx {
>  	const struct iomap_read_ops *ops;
>  	struct folio		*cur_folio;
> -- 
> 2.53.0
> 
> 

Re: [PATCH 12/22] iomap: better read bounce buffering support

[Christoph Hellwig]

On Thu, Jul 23, 2026 at 02:10:04PM -0700, Darrick J. Wong wrote:
> On Thu, Jul 23, 2026 at 04:49:37PM +0200, Christoph Hellwig wrote:
> > Add helpers to bounce buffer an upper bio into one or more lower bios
> > using bounce buffers, and to copy the data back on completion.
> > 
> > Compared to the existing IOMAP_DIO_BOUNCE support for read bios, this
> > has two advantages:  by removing the special bounce bio_vec it allows
> > to the full and "round" size of a single bio, i.e., 1MiB when using
> > 4k pages.  This is important for good performance on HDD.  Additionally
> > it allows to bounce buffer a bio from completion conext, and thus
> > implement a "lazy" bounce buffering scheme, where the data is only
> > read into a bounce buffer after an initial checksum validation failure,
> > thus avoiding the bounce buffering I/O for most I/O.
> > 
> > Signed-off-by: Christoph Hellwig <hch@lst.de>
> 
> Now that I can see how this is used ("xfs: add support for lazy direct
> read bounce buffering") I think I'm ok with this.  Should there be some
> Documentation/ about how to use PI with iomap?

Maybe.  Although I'd rather have a second example first before coming
up with definitively guidlines.  Note that even without fs pi they
should use bounce buffering for devices that require stable pages to
avoid corruption, but no one has been particularly interested (well,
minus btrfs where the native checksums make it more common).

[PATCH 13/22] iomap: add a iomap_ioend_flags helper

[Christoph Hellwig]

Add a helper to initialize the ioend flags to the values that can be
directly derived from the iomap.

Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 fs/iomap/ioend.c      | 10 +++-------
 include/linux/iomap.h | 12 ++++++++++++
 2 files changed, 15 insertions(+), 7 deletions(-)

diff --git a/fs/iomap/ioend.c b/fs/iomap/ioend.c
index 34f71c8b5ebd..b1bca36de430 100644
--- a/fs/iomap/ioend.c
+++ b/fs/iomap/ioend.c
@@ -216,7 +216,7 @@ ssize_t iomap_add_to_ioend(struct iomap_writepage_ctx *wpc, struct folio *folio,
 {
 	struct iomap_ioend *ioend = wpc->wb_ctx;
 	size_t poff = offset_in_folio(folio, pos);
-	unsigned int ioend_flags = 0;
+	unsigned int ioend_flags = iomap_ioend_flags(&wpc->iomap);
 	unsigned int map_len = min_t(u64, dirty_len,
 		wpc->iomap.offset + wpc->iomap.length - pos);
 	int error;
@@ -226,20 +226,16 @@ ssize_t iomap_add_to_ioend(struct iomap_writepage_ctx *wpc, struct folio *folio,
 	WARN_ON_ONCE(!folio->private && map_len < dirty_len);
 
 	switch (wpc->iomap.type) {
+	case IOMAP_HOLE:
+		return map_len;
 	case IOMAP_UNWRITTEN:
-		ioend_flags |= IOMAP_IOEND_UNWRITTEN;
-		break;
 	case IOMAP_MAPPED:
 		break;
-	case IOMAP_HOLE:
-		return map_len;
 	default:
 		WARN_ON_ONCE(1);
 		return -EIO;
 	}
 
-	if (wpc->iomap.flags & IOMAP_F_SHARED)
-		ioend_flags |= IOMAP_IOEND_SHARED;
 	if (folio_test_dropbehind(folio))
 		ioend_flags |= IOMAP_IOEND_DONTCACHE;
 	if (pos == wpc->iomap.offset && (wpc->iomap.flags & IOMAP_F_BOUNDARY))
diff --git a/include/linux/iomap.h b/include/linux/iomap.h
index ff9afdd39682..2541ebfabca0 100644
--- a/include/linux/iomap.h
+++ b/include/linux/iomap.h
@@ -415,6 +415,18 @@ sector_t iomap_bmap(struct address_space *mapping, sector_t bno,
 	(IOMAP_IOEND_SHARED | IOMAP_IOEND_UNWRITTEN | IOMAP_IOEND_DIRECT | \
 	 IOMAP_IOEND_DONTCACHE)
 
+/* ioend flags directly implied by iomap flags */
+static inline u16 iomap_ioend_flags(const struct iomap *iomap)
+{
+	unsigned int flags = 0;
+
+	if (iomap->type == IOMAP_UNWRITTEN)
+		flags |= IOMAP_IOEND_UNWRITTEN;
+	if (iomap->flags & IOMAP_F_SHARED)
+		flags |= IOMAP_IOEND_SHARED;
+	return flags;
+}
+
 /*
  * Structure for writeback I/O completions.
  *
-- 
2.53.0

Re: [PATCH 13/22] iomap: add a iomap_ioend_flags helper

[Darrick J. Wong]

On Thu, Jul 23, 2026 at 04:49:38PM +0200, Christoph Hellwig wrote:
> Add a helper to initialize the ioend flags to the values that can be
> directly derived from the iomap.
> 
> Signed-off-by: Christoph Hellwig <hch@lst.de>

Pretty straightforward hoist,
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>

--D

> ---
>  fs/iomap/ioend.c      | 10 +++-------
>  include/linux/iomap.h | 12 ++++++++++++
>  2 files changed, 15 insertions(+), 7 deletions(-)
> 
> diff --git a/fs/iomap/ioend.c b/fs/iomap/ioend.c
> index 34f71c8b5ebd..b1bca36de430 100644
> --- a/fs/iomap/ioend.c
> +++ b/fs/iomap/ioend.c
> @@ -216,7 +216,7 @@ ssize_t iomap_add_to_ioend(struct iomap_writepage_ctx *wpc, struct folio *folio,
>  {
>  	struct iomap_ioend *ioend = wpc->wb_ctx;
>  	size_t poff = offset_in_folio(folio, pos);
> -	unsigned int ioend_flags = 0;
> +	unsigned int ioend_flags = iomap_ioend_flags(&wpc->iomap);
>  	unsigned int map_len = min_t(u64, dirty_len,
>  		wpc->iomap.offset + wpc->iomap.length - pos);
>  	int error;
> @@ -226,20 +226,16 @@ ssize_t iomap_add_to_ioend(struct iomap_writepage_ctx *wpc, struct folio *folio,
>  	WARN_ON_ONCE(!folio->private && map_len < dirty_len);
>  
>  	switch (wpc->iomap.type) {
> +	case IOMAP_HOLE:
> +		return map_len;
>  	case IOMAP_UNWRITTEN:
> -		ioend_flags |= IOMAP_IOEND_UNWRITTEN;
> -		break;
>  	case IOMAP_MAPPED:
>  		break;
> -	case IOMAP_HOLE:
> -		return map_len;
>  	default:
>  		WARN_ON_ONCE(1);
>  		return -EIO;
>  	}
>  
> -	if (wpc->iomap.flags & IOMAP_F_SHARED)
> -		ioend_flags |= IOMAP_IOEND_SHARED;
>  	if (folio_test_dropbehind(folio))
>  		ioend_flags |= IOMAP_IOEND_DONTCACHE;
>  	if (pos == wpc->iomap.offset && (wpc->iomap.flags & IOMAP_F_BOUNDARY))
> diff --git a/include/linux/iomap.h b/include/linux/iomap.h
> index ff9afdd39682..2541ebfabca0 100644
> --- a/include/linux/iomap.h
> +++ b/include/linux/iomap.h
> @@ -415,6 +415,18 @@ sector_t iomap_bmap(struct address_space *mapping, sector_t bno,
>  	(IOMAP_IOEND_SHARED | IOMAP_IOEND_UNWRITTEN | IOMAP_IOEND_DIRECT | \
>  	 IOMAP_IOEND_DONTCACHE)
>  
> +/* ioend flags directly implied by iomap flags */
> +static inline u16 iomap_ioend_flags(const struct iomap *iomap)
> +{
> +	unsigned int flags = 0;
> +
> +	if (iomap->type == IOMAP_UNWRITTEN)
> +		flags |= IOMAP_IOEND_UNWRITTEN;
> +	if (iomap->flags & IOMAP_F_SHARED)
> +		flags |= IOMAP_IOEND_SHARED;
> +	return flags;
> +}
> +
>  /*
>   * Structure for writeback I/O completions.
>   *
> -- 
> 2.53.0
> 
> 

[PATCH 14/22] iomap: add a IOMAP_IOEND_INTEGRITY flag

[Christoph Hellwig]

This will make it easier to share code for the various submit callback
provided by the file system to iomap.

Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 fs/iomap/ioend.c      |  2 +-
 fs/xfs/xfs_aops.c     |  3 ++-
 fs/xfs/xfs_file.c     |  5 +++--
 include/linux/iomap.h | 11 ++++++++++-
 4 files changed, 16 insertions(+), 5 deletions(-)

diff --git a/fs/iomap/ioend.c b/fs/iomap/ioend.c
index b1bca36de430..438be92e131b 100644
--- a/fs/iomap/ioend.c
+++ b/fs/iomap/ioend.c
@@ -150,7 +150,7 @@ int iomap_ioend_writeback_submit(struct iomap_writepage_ctx *wpc, int error)
 		return error;
 	}
 
-	if (wpc->iomap.flags & IOMAP_F_INTEGRITY)
+	if (ioend->io_flags & IOMAP_IOEND_INTEGRITY)
 		fs_bio_integrity_generate(&ioend->io_bio);
 	submit_bio(&ioend->io_bio);
 	return 0;
diff --git a/fs/xfs/xfs_aops.c b/fs/xfs/xfs_aops.c
index 51293b6f331f..b95d74e40b06 100644
--- a/fs/xfs/xfs_aops.c
+++ b/fs/xfs/xfs_aops.c
@@ -763,7 +763,8 @@ xfs_bio_submit_read(
 	struct bio			*bio = ctx->read_ctx;
 
 	/* defer read completions to the ioend workqueue */
-	iomap_init_ioend(iter->inode, bio, ctx->read_ctx_file_offset, 0);
+	iomap_init_ioend(iter->inode, bio, ctx->read_ctx_file_offset,
+		iomap_ioend_flags(&iter->iomap));
 	iomap_bio_submit_read_endio(iter, ctx, xfs_end_bio);
 }
 
diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c
index 845a97c9b063..e94a26f32e65 100644
--- a/fs/xfs/xfs_file.c
+++ b/fs/xfs/xfs_file.c
@@ -235,7 +235,8 @@ xfs_dio_read_bounce_submit_io(
 	struct bio		*bio,
 	loff_t			file_offset)
 {
-	iomap_init_ioend(iter->inode, bio, file_offset, IOMAP_IOEND_DIRECT);
+	iomap_init_ioend(iter->inode, bio, file_offset,
+		iomap_ioend_flags(&iter->iomap) | IOMAP_IOEND_DIRECT);
 	bio->bi_end_io = xfs_end_bio;
 	submit_bio(bio);
 }
@@ -724,7 +725,7 @@ xfs_dio_zoned_submit_io(
 
 	bio->bi_end_io = xfs_end_bio;
 	ioend = iomap_init_ioend(iter->inode, bio, file_offset,
-			IOMAP_IOEND_DIRECT);
+		iomap_ioend_flags(&iter->iomap) | IOMAP_IOEND_DIRECT);
 	xfs_zone_alloc_and_submit(ioend, &ac->open_zone);
 }
 
diff --git a/include/linux/iomap.h b/include/linux/iomap.h
index 2541ebfabca0..1916a4f28002 100644
--- a/include/linux/iomap.h
+++ b/include/linux/iomap.h
@@ -406,6 +406,12 @@ sector_t iomap_bmap(struct address_space *mapping, sector_t bno,
 #define IOMAP_IOEND_DIRECT		(1U << 3)
 /* is DONTCACHE I/O */
 #define IOMAP_IOEND_DONTCACHE		(1U << 4)
+/* generate integrity (PI) information */
+#ifdef CONFIG_BLK_DEV_INTEGRITY
+#define IOMAP_IOEND_INTEGRITY		(1U << 5)
+#else
+#define IOMAP_IOEND_INTEGRITY		0
+#endif /* CONFIG_BLK_DEV_INTEGRITY */
 
 /*
  * Flags that if set on either ioend prevent the merge of two ioends.
@@ -413,7 +419,7 @@ sector_t iomap_bmap(struct address_space *mapping, sector_t bno,
  */
 #define IOMAP_IOEND_NOMERGE_FLAGS \
 	(IOMAP_IOEND_SHARED | IOMAP_IOEND_UNWRITTEN | IOMAP_IOEND_DIRECT | \
-	 IOMAP_IOEND_DONTCACHE)
+	 IOMAP_IOEND_DONTCACHE | IOMAP_IOEND_INTEGRITY)
 
 /* ioend flags directly implied by iomap flags */
 static inline u16 iomap_ioend_flags(const struct iomap *iomap)
@@ -424,6 +430,9 @@ static inline u16 iomap_ioend_flags(const struct iomap *iomap)
 		flags |= IOMAP_IOEND_UNWRITTEN;
 	if (iomap->flags & IOMAP_F_SHARED)
 		flags |= IOMAP_IOEND_SHARED;
+	if (iomap->flags & IOMAP_F_INTEGRITY)
+		flags |= IOMAP_IOEND_INTEGRITY;
+
 	return flags;
 }
 
-- 
2.53.0

Re: [PATCH 14/22] iomap: add a IOMAP_IOEND_INTEGRITY flag

[Darrick J. Wong]

On Thu, Jul 23, 2026 at 04:49:39PM +0200, Christoph Hellwig wrote:
> This will make it easier to share code for the various submit callback
> provided by the file system to iomap.
> 
> Signed-off-by: Christoph Hellwig <hch@lst.de>

This also looks pretty straightforward, so
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>

--D

> ---
>  fs/iomap/ioend.c      |  2 +-
>  fs/xfs/xfs_aops.c     |  3 ++-
>  fs/xfs/xfs_file.c     |  5 +++--
>  include/linux/iomap.h | 11 ++++++++++-
>  4 files changed, 16 insertions(+), 5 deletions(-)
> 
> diff --git a/fs/iomap/ioend.c b/fs/iomap/ioend.c
> index b1bca36de430..438be92e131b 100644
> --- a/fs/iomap/ioend.c
> +++ b/fs/iomap/ioend.c
> @@ -150,7 +150,7 @@ int iomap_ioend_writeback_submit(struct iomap_writepage_ctx *wpc, int error)
>  		return error;
>  	}
>  
> -	if (wpc->iomap.flags & IOMAP_F_INTEGRITY)
> +	if (ioend->io_flags & IOMAP_IOEND_INTEGRITY)
>  		fs_bio_integrity_generate(&ioend->io_bio);
>  	submit_bio(&ioend->io_bio);
>  	return 0;
> diff --git a/fs/xfs/xfs_aops.c b/fs/xfs/xfs_aops.c
> index 51293b6f331f..b95d74e40b06 100644
> --- a/fs/xfs/xfs_aops.c
> +++ b/fs/xfs/xfs_aops.c
> @@ -763,7 +763,8 @@ xfs_bio_submit_read(
>  	struct bio			*bio = ctx->read_ctx;
>  
>  	/* defer read completions to the ioend workqueue */
> -	iomap_init_ioend(iter->inode, bio, ctx->read_ctx_file_offset, 0);
> +	iomap_init_ioend(iter->inode, bio, ctx->read_ctx_file_offset,
> +		iomap_ioend_flags(&iter->iomap));
>  	iomap_bio_submit_read_endio(iter, ctx, xfs_end_bio);
>  }
>  
> diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c
> index 845a97c9b063..e94a26f32e65 100644
> --- a/fs/xfs/xfs_file.c
> +++ b/fs/xfs/xfs_file.c
> @@ -235,7 +235,8 @@ xfs_dio_read_bounce_submit_io(
>  	struct bio		*bio,
>  	loff_t			file_offset)
>  {
> -	iomap_init_ioend(iter->inode, bio, file_offset, IOMAP_IOEND_DIRECT);
> +	iomap_init_ioend(iter->inode, bio, file_offset,
> +		iomap_ioend_flags(&iter->iomap) | IOMAP_IOEND_DIRECT);
>  	bio->bi_end_io = xfs_end_bio;
>  	submit_bio(bio);
>  }
> @@ -724,7 +725,7 @@ xfs_dio_zoned_submit_io(
>  
>  	bio->bi_end_io = xfs_end_bio;
>  	ioend = iomap_init_ioend(iter->inode, bio, file_offset,
> -			IOMAP_IOEND_DIRECT);
> +		iomap_ioend_flags(&iter->iomap) | IOMAP_IOEND_DIRECT);
>  	xfs_zone_alloc_and_submit(ioend, &ac->open_zone);
>  }
>  
> diff --git a/include/linux/iomap.h b/include/linux/iomap.h
> index 2541ebfabca0..1916a4f28002 100644
> --- a/include/linux/iomap.h
> +++ b/include/linux/iomap.h
> @@ -406,6 +406,12 @@ sector_t iomap_bmap(struct address_space *mapping, sector_t bno,
>  #define IOMAP_IOEND_DIRECT		(1U << 3)
>  /* is DONTCACHE I/O */
>  #define IOMAP_IOEND_DONTCACHE		(1U << 4)
> +/* generate integrity (PI) information */
> +#ifdef CONFIG_BLK_DEV_INTEGRITY
> +#define IOMAP_IOEND_INTEGRITY		(1U << 5)
> +#else
> +#define IOMAP_IOEND_INTEGRITY		0
> +#endif /* CONFIG_BLK_DEV_INTEGRITY */
>  
>  /*
>   * Flags that if set on either ioend prevent the merge of two ioends.
> @@ -413,7 +419,7 @@ sector_t iomap_bmap(struct address_space *mapping, sector_t bno,
>   */
>  #define IOMAP_IOEND_NOMERGE_FLAGS \
>  	(IOMAP_IOEND_SHARED | IOMAP_IOEND_UNWRITTEN | IOMAP_IOEND_DIRECT | \
> -	 IOMAP_IOEND_DONTCACHE)
> +	 IOMAP_IOEND_DONTCACHE | IOMAP_IOEND_INTEGRITY)
>  
>  /* ioend flags directly implied by iomap flags */
>  static inline u16 iomap_ioend_flags(const struct iomap *iomap)
> @@ -424,6 +430,9 @@ static inline u16 iomap_ioend_flags(const struct iomap *iomap)
>  		flags |= IOMAP_IOEND_UNWRITTEN;
>  	if (iomap->flags & IOMAP_F_SHARED)
>  		flags |= IOMAP_IOEND_SHARED;
> +	if (iomap->flags & IOMAP_F_INTEGRITY)
> +		flags |= IOMAP_IOEND_INTEGRITY;
> +
>  	return flags;
>  }
>  
> -- 
> 2.53.0
> 
> 

[PATCH 15/22] iomap,xfs: move T10 PI handling for direct I/O into ->submit_io

[Christoph Hellwig]

Currently the buffered I/O code defers PI handling into the submit
handlers, while direct I/O does it in common code.  Move it all
into the file system callbacks or their generic implementations
to be consistent, and to allow file systems to shared helpers
for submission of buffered and direct bios.

Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 fs/iomap/direct-io.c | 22 ++++++++++++----------
 fs/xfs/xfs_file.c    | 13 ++++++++++---
 2 files changed, 22 insertions(+), 13 deletions(-)

diff --git a/fs/iomap/direct-io.c b/fs/iomap/direct-io.c
index e2cd5f92babe..8d476f40a3bb 100644
--- a/fs/iomap/direct-io.c
+++ b/fs/iomap/direct-io.c
@@ -75,10 +75,19 @@ static void iomap_dio_submit_bio(const struct iomap_iter *iter,
 
 	if (dio->dops && dio->dops->submit_io) {
 		dio->dops->submit_io(iter, bio, pos);
-	} else {
-		WARN_ON_ONCE(iter->iomap.flags & IOMAP_F_ANON_WRITE);
-		blk_crypto_submit_bio(bio);
+		return;
+	}
+
+	WARN_ON_ONCE(iter->iomap.flags & IOMAP_F_ANON_WRITE);
+
+	if (iter->iomap.flags & IOMAP_F_INTEGRITY) {
+		if (dio->flags & IOMAP_DIO_WRITE)
+			fs_bio_integrity_generate(bio);
+		else
+			fs_bio_integrity_alloc(bio);
 	}
+
+	blk_crypto_submit_bio(bio);
 }
 
 static inline enum fserror_type iomap_dio_err_type(const struct iomap_dio *dio)
@@ -372,13 +381,6 @@ static ssize_t iomap_dio_bio_iter_one(struct iomap_iter *iter,
 		goto out_bio_release_pages;
 	}
 
-	if (iter->iomap.flags & IOMAP_F_INTEGRITY) {
-		if (dio->flags & IOMAP_DIO_WRITE)
-			fs_bio_integrity_generate(bio);
-		else
-			fs_bio_integrity_alloc(bio);
-	}
-
 	if (dio->flags & IOMAP_DIO_WRITE)
 		task_io_account_write(ret);
 	else if ((dio->flags & IOMAP_DIO_USER_BACKED) &&
diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c
index e94a26f32e65..80dd278e023f 100644
--- a/fs/xfs/xfs_file.c
+++ b/fs/xfs/xfs_file.c
@@ -37,6 +37,7 @@
 #include <linux/fadvise.h>
 #include <linux/mount.h>
 #include <linux/filelock.h>
+#include <linux/bio-integrity.h>
 
 static const struct vm_operations_struct xfs_file_vm_ops;
 
@@ -235,8 +236,12 @@ xfs_dio_read_bounce_submit_io(
 	struct bio		*bio,
 	loff_t			file_offset)
 {
-	iomap_init_ioend(iter->inode, bio, file_offset,
-		iomap_ioend_flags(&iter->iomap) | IOMAP_IOEND_DIRECT);
+	struct iomap_ioend	*ioend;
+
+	ioend = iomap_init_ioend(iter->inode, bio, file_offset,
+			iomap_ioend_flags(&iter->iomap) | IOMAP_IOEND_DIRECT);
+	if (ioend->io_flags & IOMAP_IOEND_INTEGRITY)
+		fs_bio_integrity_alloc(bio);
 	bio->bi_end_io = xfs_end_bio;
 	submit_bio(bio);
 }
@@ -725,7 +730,9 @@ xfs_dio_zoned_submit_io(
 
 	bio->bi_end_io = xfs_end_bio;
 	ioend = iomap_init_ioend(iter->inode, bio, file_offset,
-		iomap_ioend_flags(&iter->iomap) | IOMAP_IOEND_DIRECT);
+			iomap_ioend_flags(&iter->iomap) | IOMAP_IOEND_DIRECT);
+	if (ioend->io_flags & IOMAP_IOEND_INTEGRITY)
+		fs_bio_integrity_generate(bio);
 	xfs_zone_alloc_and_submit(ioend, &ac->open_zone);
 }
 
-- 
2.53.0

Re: [PATCH 15/22] iomap,xfs: move T10 PI handling for direct I/O into ->submit_io

[Darrick J. Wong]

On Thu, Jul 23, 2026 at 04:49:40PM +0200, Christoph Hellwig wrote:
> Currently the buffered I/O code defers PI handling into the submit
> handlers, while direct I/O does it in common code.  Move it all
> into the file system callbacks or their generic implementations
> to be consistent, and to allow file systems to shared helpers
> for submission of buffered and direct bios.
> 
> Signed-off-by: Christoph Hellwig <hch@lst.de>

I think I like this cleanup, the bio integrity setup code was scattered
around a lot of places.

Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>

--D

> ---
>  fs/iomap/direct-io.c | 22 ++++++++++++----------
>  fs/xfs/xfs_file.c    | 13 ++++++++++---
>  2 files changed, 22 insertions(+), 13 deletions(-)
> 
> diff --git a/fs/iomap/direct-io.c b/fs/iomap/direct-io.c
> index e2cd5f92babe..8d476f40a3bb 100644
> --- a/fs/iomap/direct-io.c
> +++ b/fs/iomap/direct-io.c
> @@ -75,10 +75,19 @@ static void iomap_dio_submit_bio(const struct iomap_iter *iter,
>  
>  	if (dio->dops && dio->dops->submit_io) {
>  		dio->dops->submit_io(iter, bio, pos);
> -	} else {
> -		WARN_ON_ONCE(iter->iomap.flags & IOMAP_F_ANON_WRITE);
> -		blk_crypto_submit_bio(bio);
> +		return;
> +	}
> +
> +	WARN_ON_ONCE(iter->iomap.flags & IOMAP_F_ANON_WRITE);
> +
> +	if (iter->iomap.flags & IOMAP_F_INTEGRITY) {
> +		if (dio->flags & IOMAP_DIO_WRITE)
> +			fs_bio_integrity_generate(bio);
> +		else
> +			fs_bio_integrity_alloc(bio);
>  	}
> +
> +	blk_crypto_submit_bio(bio);
>  }
>  
>  static inline enum fserror_type iomap_dio_err_type(const struct iomap_dio *dio)
> @@ -372,13 +381,6 @@ static ssize_t iomap_dio_bio_iter_one(struct iomap_iter *iter,
>  		goto out_bio_release_pages;
>  	}
>  
> -	if (iter->iomap.flags & IOMAP_F_INTEGRITY) {
> -		if (dio->flags & IOMAP_DIO_WRITE)
> -			fs_bio_integrity_generate(bio);
> -		else
> -			fs_bio_integrity_alloc(bio);
> -	}
> -
>  	if (dio->flags & IOMAP_DIO_WRITE)
>  		task_io_account_write(ret);
>  	else if ((dio->flags & IOMAP_DIO_USER_BACKED) &&
> diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c
> index e94a26f32e65..80dd278e023f 100644
> --- a/fs/xfs/xfs_file.c
> +++ b/fs/xfs/xfs_file.c
> @@ -37,6 +37,7 @@
>  #include <linux/fadvise.h>
>  #include <linux/mount.h>
>  #include <linux/filelock.h>
> +#include <linux/bio-integrity.h>
>  
>  static const struct vm_operations_struct xfs_file_vm_ops;
>  
> @@ -235,8 +236,12 @@ xfs_dio_read_bounce_submit_io(
>  	struct bio		*bio,
>  	loff_t			file_offset)
>  {
> -	iomap_init_ioend(iter->inode, bio, file_offset,
> -		iomap_ioend_flags(&iter->iomap) | IOMAP_IOEND_DIRECT);
> +	struct iomap_ioend	*ioend;
> +
> +	ioend = iomap_init_ioend(iter->inode, bio, file_offset,
> +			iomap_ioend_flags(&iter->iomap) | IOMAP_IOEND_DIRECT);
> +	if (ioend->io_flags & IOMAP_IOEND_INTEGRITY)
> +		fs_bio_integrity_alloc(bio);
>  	bio->bi_end_io = xfs_end_bio;
>  	submit_bio(bio);
>  }
> @@ -725,7 +730,9 @@ xfs_dio_zoned_submit_io(
>  
>  	bio->bi_end_io = xfs_end_bio;
>  	ioend = iomap_init_ioend(iter->inode, bio, file_offset,
> -		iomap_ioend_flags(&iter->iomap) | IOMAP_IOEND_DIRECT);
> +			iomap_ioend_flags(&iter->iomap) | IOMAP_IOEND_DIRECT);
> +	if (ioend->io_flags & IOMAP_IOEND_INTEGRITY)
> +		fs_bio_integrity_generate(bio);
>  	xfs_zone_alloc_and_submit(ioend, &ac->open_zone);
>  }
>  
> -- 
> 2.53.0
> 
> 

[PATCH 16/22] xfs: move PI generation into xfs_zone_alloc_and_submit

[Christoph Hellwig]

Keep it in one place for all the zoned write bio submissions.

Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 fs/xfs/xfs_aops.c       | 3 ---
 fs/xfs/xfs_file.c       | 2 --
 fs/xfs/xfs_zone_alloc.c | 4 ++++
 3 files changed, 4 insertions(+), 5 deletions(-)

diff --git a/fs/xfs/xfs_aops.c b/fs/xfs/xfs_aops.c
index b95d74e40b06..0b757b7f35c0 100644
--- a/fs/xfs/xfs_aops.c
+++ b/fs/xfs/xfs_aops.c
@@ -22,7 +22,6 @@
 #include "xfs_icache.h"
 #include "xfs_zone_alloc.h"
 #include "xfs_rtgroup.h"
-#include <linux/bio-integrity.h>
 
 struct xfs_writepage_ctx {
 	struct iomap_writepage_ctx ctx;
@@ -673,8 +672,6 @@ xfs_zoned_writeback_submit(
 		bio_endio(&ioend->io_bio);
 		return error;
 	}
-	if (wpc->iomap.flags & IOMAP_F_INTEGRITY)
-		fs_bio_integrity_generate(&ioend->io_bio);
 	xfs_zone_alloc_and_submit(ioend, &XFS_ZWPC(wpc)->open_zone);
 	return 0;
 }
diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c
index 80dd278e023f..2c19e69b93cd 100644
--- a/fs/xfs/xfs_file.c
+++ b/fs/xfs/xfs_file.c
@@ -731,8 +731,6 @@ xfs_dio_zoned_submit_io(
 	bio->bi_end_io = xfs_end_bio;
 	ioend = iomap_init_ioend(iter->inode, bio, file_offset,
 			iomap_ioend_flags(&iter->iomap) | IOMAP_IOEND_DIRECT);
-	if (ioend->io_flags & IOMAP_IOEND_INTEGRITY)
-		fs_bio_integrity_generate(bio);
 	xfs_zone_alloc_and_submit(ioend, &ac->open_zone);
 }
 
diff --git a/fs/xfs/xfs_zone_alloc.c b/fs/xfs/xfs_zone_alloc.c
index 7d13fa7ab30a..3fd1069beea0 100644
--- a/fs/xfs/xfs_zone_alloc.c
+++ b/fs/xfs/xfs_zone_alloc.c
@@ -26,6 +26,7 @@
 #include "xfs_zones.h"
 #include "xfs_trace.h"
 #include "xfs_mru_cache.h"
+#include <linux/bio-integrity.h>
 
 static void
 xfs_open_zone_free_rcu(
@@ -868,6 +869,9 @@ xfs_zone_alloc_and_submit(
 	if (xfs_is_shutdown(mp))
 		goto out_error;
 
+	if (ioend->io_flags & IOMAP_IOEND_INTEGRITY)
+		fs_bio_integrity_generate(&ioend->io_bio);
+
 	/*
 	 * If we don't have a locally cached zone in this write context, see if
 	 * the inode is still associated with a zone and use that if so.
-- 
2.53.0

Re: [PATCH 16/22] xfs: move PI generation into xfs_zone_alloc_and_submit

[Darrick J. Wong]

On Thu, Jul 23, 2026 at 04:49:41PM +0200, Christoph Hellwig wrote:
> Keep it in one place for all the zoned write bio submissions.
> 
> Signed-off-by: Christoph Hellwig <hch@lst.de>

Looks ok to me,
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>

--D

> ---
>  fs/xfs/xfs_aops.c       | 3 ---
>  fs/xfs/xfs_file.c       | 2 --
>  fs/xfs/xfs_zone_alloc.c | 4 ++++
>  3 files changed, 4 insertions(+), 5 deletions(-)
> 
> diff --git a/fs/xfs/xfs_aops.c b/fs/xfs/xfs_aops.c
> index b95d74e40b06..0b757b7f35c0 100644
> --- a/fs/xfs/xfs_aops.c
> +++ b/fs/xfs/xfs_aops.c
> @@ -22,7 +22,6 @@
>  #include "xfs_icache.h"
>  #include "xfs_zone_alloc.h"
>  #include "xfs_rtgroup.h"
> -#include <linux/bio-integrity.h>
>  
>  struct xfs_writepage_ctx {
>  	struct iomap_writepage_ctx ctx;
> @@ -673,8 +672,6 @@ xfs_zoned_writeback_submit(
>  		bio_endio(&ioend->io_bio);
>  		return error;
>  	}
> -	if (wpc->iomap.flags & IOMAP_F_INTEGRITY)
> -		fs_bio_integrity_generate(&ioend->io_bio);
>  	xfs_zone_alloc_and_submit(ioend, &XFS_ZWPC(wpc)->open_zone);
>  	return 0;
>  }
> diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c
> index 80dd278e023f..2c19e69b93cd 100644
> --- a/fs/xfs/xfs_file.c
> +++ b/fs/xfs/xfs_file.c
> @@ -731,8 +731,6 @@ xfs_dio_zoned_submit_io(
>  	bio->bi_end_io = xfs_end_bio;
>  	ioend = iomap_init_ioend(iter->inode, bio, file_offset,
>  			iomap_ioend_flags(&iter->iomap) | IOMAP_IOEND_DIRECT);
> -	if (ioend->io_flags & IOMAP_IOEND_INTEGRITY)
> -		fs_bio_integrity_generate(bio);
>  	xfs_zone_alloc_and_submit(ioend, &ac->open_zone);
>  }
>  
> diff --git a/fs/xfs/xfs_zone_alloc.c b/fs/xfs/xfs_zone_alloc.c
> index 7d13fa7ab30a..3fd1069beea0 100644
> --- a/fs/xfs/xfs_zone_alloc.c
> +++ b/fs/xfs/xfs_zone_alloc.c
> @@ -26,6 +26,7 @@
>  #include "xfs_zones.h"
>  #include "xfs_trace.h"
>  #include "xfs_mru_cache.h"
> +#include <linux/bio-integrity.h>
>  
>  static void
>  xfs_open_zone_free_rcu(
> @@ -868,6 +869,9 @@ xfs_zone_alloc_and_submit(
>  	if (xfs_is_shutdown(mp))
>  		goto out_error;
>  
> +	if (ioend->io_flags & IOMAP_IOEND_INTEGRITY)
> +		fs_bio_integrity_generate(&ioend->io_bio);
> +
>  	/*
>  	 * If we don't have a locally cached zone in this write context, see if
>  	 * the inode is still associated with a zone and use that if so.
> -- 
> 2.53.0
> 
> 

[PATCH 17/22] xfs: split ioend handling into a separate source file

[Christoph Hellwig]

The ioend handling used to be only for buffered writeback, but has been
extended to direct I/O and reads.  Split it into a new source file.

Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 fs/xfs/Makefile    |   1 +
 fs/xfs/xfs_aops.c  | 179 +------------------------------------------
 fs/xfs/xfs_aops.h  |   1 -
 fs/xfs/xfs_file.c  |   2 +-
 fs/xfs/xfs_ioend.c | 184 +++++++++++++++++++++++++++++++++++++++++++++
 fs/xfs/xfs_ioend.h |  16 ++++
 6 files changed, 203 insertions(+), 180 deletions(-)
 create mode 100644 fs/xfs/xfs_ioend.c
 create mode 100644 fs/xfs/xfs_ioend.h

diff --git a/fs/xfs/Makefile b/fs/xfs/Makefile
index 9f7133e02576..399a207f2d0e 100644
--- a/fs/xfs/Makefile
+++ b/fs/xfs/Makefile
@@ -91,6 +91,7 @@ xfs-y				+= xfs_aops.o \
 				   xfs_healthmon.o \
 				   xfs_icache.o \
 				   xfs_ioctl.o \
+				   xfs_ioend.o \
 				   xfs_iomap.o \
 				   xfs_iops.o \
 				   xfs_inode.o \
diff --git a/fs/xfs/xfs_aops.c b/fs/xfs/xfs_aops.c
index 0b757b7f35c0..49d21d905cc3 100644
--- a/fs/xfs/xfs_aops.c
+++ b/fs/xfs/xfs_aops.c
@@ -20,6 +20,7 @@
 #include "xfs_errortag.h"
 #include "xfs_error.h"
 #include "xfs_icache.h"
+#include "xfs_ioend.h"
 #include "xfs_zone_alloc.h"
 #include "xfs_rtgroup.h"
 
@@ -35,15 +36,6 @@ XFS_WPC(struct iomap_writepage_ctx *ctx)
 	return container_of(ctx, struct xfs_writepage_ctx, ctx);
 }
 
-/*
- * Fast and loose check if this write could update the on-disk inode size.
- */
-static inline bool xfs_ioend_is_append(struct iomap_ioend *ioend)
-{
-	return ioend->io_offset + ioend->io_size >
-		XFS_I(ioend->io_inode)->i_disk_size;
-}
-
 /*
  * Update on-disk file size now that data has been written to disk.
  */
@@ -79,175 +71,6 @@ xfs_setfilesize(
 	return xfs_trans_commit(tp);
 }
 
-static void
-xfs_ioend_put_open_zones(
-	struct iomap_ioend	*ioend)
-{
-	struct iomap_ioend *tmp;
-
-	/*
-	 * Put the open zone for all ioends merged into this one (if any).
-	 */
-	list_for_each_entry(tmp, &ioend->io_list, io_list)
-		xfs_open_zone_put(tmp->io_private);
-
-	/*
-	 * The main ioend might not have an open zone if the submission failed
-	 * before xfs_zone_alloc_and_submit got called.
-	 */
-	if (ioend->io_private)
-		xfs_open_zone_put(ioend->io_private);
-}
-
-/*
- * IO write completion.
- */
-STATIC void
-xfs_end_ioend_write(
-	struct iomap_ioend	*ioend)
-{
-	struct xfs_inode	*ip = XFS_I(ioend->io_inode);
-	struct xfs_mount	*mp = ip->i_mount;
-	bool			is_zoned = xfs_is_zoned_inode(ip);
-	xfs_off_t		offset = ioend->io_offset;
-	size_t			size = ioend->io_size;
-	unsigned int		nofs_flag;
-	int			error;
-
-	/*
-	 * We can allocate memory here while doing writeback on behalf of
-	 * memory reclaim.  To avoid memory allocation deadlocks set the
-	 * task-wide nofs context for the following operations.
-	 */
-	nofs_flag = memalloc_nofs_save();
-
-	/*
-	 * Just clean up the in-memory structures if the fs has been shut down.
-	 */
-	if (xfs_is_shutdown(mp)) {
-		error = -EIO;
-		goto done;
-	}
-
-	/*
-	 * Clean up all COW blocks and underlying data fork delalloc blocks on
-	 * I/O error. The delalloc punch is required because this ioend was
-	 * mapped to blocks in the COW fork and the associated pages are no
-	 * longer dirty. If we don't remove delalloc blocks here, they become
-	 * stale and can corrupt free space accounting on unmount.
-	 */
-	error = blk_status_to_errno(ioend->io_bio.bi_status);
-	if (unlikely(error)) {
-		/*
-		 * Zoned writes update the in-core open zone accounting before
-		 * I/O submission.  A failed write leaves that state
-		 * inconsistent, so shut down the filesystem instead of letting
-		 * later writers wait forever for open zone space to become
-		 * available.
-		 */
-		if (is_zoned) {
-			xfs_force_shutdown(mp, SHUTDOWN_META_IO_ERROR);
-			goto done;
-		}
-		if (ioend->io_flags & IOMAP_IOEND_SHARED) {
-			ASSERT(!is_zoned);
-			xfs_reflink_cancel_cow_range(ip, offset, size, true);
-			xfs_bmap_punch_delalloc_range(ip, XFS_DATA_FORK, offset,
-					offset + size, NULL);
-		}
-		goto done;
-	}
-
-	/*
-	 * Success: commit the COW or unwritten blocks if needed.
-	 */
-	if (is_zoned)
-		error = xfs_zoned_end_io(ip, offset, size, ioend->io_sector,
-				ioend->io_private, NULLFSBLOCK);
-	else if (ioend->io_flags & IOMAP_IOEND_SHARED)
-		error = xfs_reflink_end_cow(ip, offset, size);
-	else if (ioend->io_flags & IOMAP_IOEND_UNWRITTEN)
-		error = xfs_iomap_write_unwritten(ip, offset, size, false);
-
-	if (!error &&
-	    !(ioend->io_flags & IOMAP_IOEND_DIRECT) &&
-	    xfs_ioend_is_append(ioend))
-		error = xfs_setfilesize(ip, offset, size);
-done:
-	if (is_zoned)
-		xfs_ioend_put_open_zones(ioend);
-	iomap_finish_ioends(ioend, error);
-	memalloc_nofs_restore(nofs_flag);
-}
-
-/*
- * Finish all pending IO completions that require transactional modifications.
- *
- * We try to merge physical and logically contiguous ioends before completion to
- * minimise the number of transactions we need to perform during IO completion.
- * Both unwritten extent conversion and COW remapping need to iterate and modify
- * one physical extent at a time, so we gain nothing by merging physically
- * discontiguous extents here.
- *
- * The ioend chain length that we can be processing here is largely unbound in
- * length and we may have to perform significant amounts of work on each ioend
- * to complete it. Hence we have to be careful about holding the CPU for too
- * long in this loop.
- */
-void
-xfs_end_io(
-	struct work_struct	*work)
-{
-	struct xfs_inode	*ip =
-		container_of(work, struct xfs_inode, i_ioend_work);
-	struct iomap_ioend	*ioend;
-	struct list_head	tmp;
-	unsigned long		flags;
-
-	spin_lock_irqsave(&ip->i_ioend_lock, flags);
-	list_replace_init(&ip->i_ioend_list, &tmp);
-	spin_unlock_irqrestore(&ip->i_ioend_lock, flags);
-
-	iomap_sort_ioends(&tmp);
-	while ((ioend = list_first_entry_or_null(&tmp, struct iomap_ioend,
-			io_list))) {
-		list_del_init(&ioend->io_list);
-		iomap_ioend_try_merge(ioend, &tmp);
-		if (bio_op(&ioend->io_bio) == REQ_OP_READ)
-			iomap_finish_ioends(ioend,
-				blk_status_to_errno(ioend->io_bio.bi_status));
-		else
-			xfs_end_ioend_write(ioend);
-		cond_resched();
-	}
-}
-
-void
-xfs_end_bio(
-	struct bio		*bio)
-{
-	struct iomap_ioend	*ioend = iomap_ioend_from_bio(bio);
-	struct xfs_inode	*ip = XFS_I(ioend->io_inode);
-	struct xfs_mount	*mp = ip->i_mount;
-	unsigned long		flags;
-
-	/*
-	 * For Appends record the actually written block number and set the
-	 * boundary flag if needed.
-	 */
-	if (IS_ENABLED(CONFIG_XFS_RT) && bio_is_zone_append(bio)) {
-		ioend->io_sector = bio->bi_iter.bi_sector;
-		xfs_mark_rtg_boundary(ioend);
-	}
-
-	spin_lock_irqsave(&ip->i_ioend_lock, flags);
-	if (list_empty(&ip->i_ioend_list))
-		WARN_ON_ONCE(!queue_work(mp->m_unwritten_workqueue,
-					 &ip->i_ioend_work));
-	list_add_tail(&ioend->io_list, &ip->i_ioend_list);
-	spin_unlock_irqrestore(&ip->i_ioend_lock, flags);
-}
-
 /*
  * We cannot cancel the ioend directly on error.  We may have already set other
  * pages under writeback and hence we have to run I/O completion to mark the
diff --git a/fs/xfs/xfs_aops.h b/fs/xfs/xfs_aops.h
index 5a7a0f1a0b49..d5ae5c9d4c26 100644
--- a/fs/xfs/xfs_aops.h
+++ b/fs/xfs/xfs_aops.h
@@ -10,6 +10,5 @@ extern const struct address_space_operations xfs_address_space_operations;
 extern const struct address_space_operations xfs_dax_aops;
 
 int xfs_setfilesize(struct xfs_inode *ip, xfs_off_t offset, size_t size);
-void xfs_end_bio(struct bio *bio);
 
 #endif /* __XFS_AOPS_H__ */
diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c
index 2c19e69b93cd..c0c3a11e7ff2 100644
--- a/fs/xfs/xfs_file.c
+++ b/fs/xfs/xfs_file.c
@@ -25,7 +25,7 @@
 #include "xfs_iomap.h"
 #include "xfs_reflink.h"
 #include "xfs_file.h"
-#include "xfs_aops.h"
+#include "xfs_ioend.h"
 #include "xfs_zone_alloc.h"
 #include "xfs_error.h"
 #include "xfs_errortag.h"
diff --git a/fs/xfs/xfs_ioend.c b/fs/xfs/xfs_ioend.c
new file mode 100644
index 000000000000..40695d18dac0
--- /dev/null
+++ b/fs/xfs/xfs_ioend.c
@@ -0,0 +1,184 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2016-2025 Christoph Hellwig.
+ * All Rights Reserved.
+ */
+#include "xfs_platform.h"
+#include "xfs_shared.h"
+#include "xfs_format.h"
+#include "xfs_log_format.h"
+#include "xfs_trans_resv.h"
+#include "xfs_mount.h"
+#include "xfs_inode.h"
+#include "xfs_iomap.h"
+#include "xfs_trace.h"
+#include "xfs_bmap_util.h"
+#include "xfs_reflink.h"
+#include "xfs_zone_alloc.h"
+#include "xfs_ioend.h"
+
+static void
+xfs_ioend_put_open_zones(
+	struct iomap_ioend	*ioend)
+{
+	struct iomap_ioend *tmp;
+
+	/*
+	 * Put the open zone for all ioends merged into this one (if any).
+	 */
+	list_for_each_entry(tmp, &ioend->io_list, io_list)
+		xfs_open_zone_put(tmp->io_private);
+
+	/*
+	 * The main ioend might not have an open zone if the submission failed
+	 * before xfs_zone_alloc_and_submit got called.
+	 */
+	if (ioend->io_private)
+		xfs_open_zone_put(ioend->io_private);
+}
+
+static void
+xfs_end_ioend_write(
+	struct iomap_ioend	*ioend)
+{
+	struct xfs_inode	*ip = XFS_I(ioend->io_inode);
+	struct xfs_mount	*mp = ip->i_mount;
+	bool			is_zoned = xfs_is_zoned_inode(ip);
+	xfs_off_t		offset = ioend->io_offset;
+	size_t			size = ioend->io_size;
+	unsigned int		nofs_flag;
+	int			error;
+
+	/*
+	 * We can allocate memory here while doing writeback on behalf of
+	 * memory reclaim.  To avoid memory allocation deadlocks set the
+	 * task-wide nofs context for the following operations.
+	 */
+	nofs_flag = memalloc_nofs_save();
+
+	/*
+	 * Just clean up the in-memory structures if the fs has been shut down.
+	 */
+	if (xfs_is_shutdown(mp)) {
+		error = -EIO;
+		goto done;
+	}
+
+	/*
+	 * Clean up all COW blocks and underlying data fork delalloc blocks on
+	 * I/O error. The delalloc punch is required because this ioend was
+	 * mapped to blocks in the COW fork and the associated pages are no
+	 * longer dirty. If we don't remove delalloc blocks here, they become
+	 * stale and can corrupt free space accounting on unmount.
+	 */
+	error = blk_status_to_errno(ioend->io_bio.bi_status);
+	if (unlikely(error)) {
+		/*
+		 * Zoned writes update the in-core open zone accounting before
+		 * I/O submission.  A failed write leaves that state
+		 * inconsistent, so shut down the filesystem instead of letting
+		 * later writers wait forever for open zone space to become
+		 * available.
+		 */
+		if (is_zoned) {
+			xfs_force_shutdown(mp, SHUTDOWN_META_IO_ERROR);
+			goto done;
+		}
+		if (ioend->io_flags & IOMAP_IOEND_SHARED) {
+			ASSERT(!is_zoned);
+			xfs_reflink_cancel_cow_range(ip, offset, size, true);
+			xfs_bmap_punch_delalloc_range(ip, XFS_DATA_FORK, offset,
+					offset + size, NULL);
+		}
+		goto done;
+	}
+
+	/*
+	 * Success: commit the COW or unwritten blocks if needed.
+	 */
+	if (is_zoned)
+		error = xfs_zoned_end_io(ip, offset, size, ioend->io_sector,
+				ioend->io_private, NULLFSBLOCK);
+	else if (ioend->io_flags & IOMAP_IOEND_SHARED)
+		error = xfs_reflink_end_cow(ip, offset, size);
+	else if (ioend->io_flags & IOMAP_IOEND_UNWRITTEN)
+		error = xfs_iomap_write_unwritten(ip, offset, size, false);
+
+	if (!error &&
+	    !(ioend->io_flags & IOMAP_IOEND_DIRECT) &&
+	    xfs_ioend_is_append(ioend))
+		error = xfs_setfilesize(ip, offset, size);
+done:
+	if (is_zoned)
+		xfs_ioend_put_open_zones(ioend);
+	iomap_finish_ioends(ioend, error);
+	memalloc_nofs_restore(nofs_flag);
+}
+
+/*
+ * Finish all pending IO completions that require transactional modifications.
+ *
+ * We try to merge physical and logically contiguous ioends before completion to
+ * minimise the number of transactions we need to perform during IO completion.
+ * Both unwritten extent conversion and COW remapping need to iterate and modify
+ * one physical extent at a time, so we gain nothing by merging physically
+ * discontiguous extents here.
+ *
+ * The ioend chain length that we can be processing here is largely unbound in
+ * length and we may have to perform significant amounts of work on each ioend
+ * to complete it. Hence we have to be careful about holding the CPU for too
+ * long in this loop.
+ */
+void
+xfs_end_io(
+	struct work_struct	*work)
+{
+	struct xfs_inode	*ip =
+		container_of(work, struct xfs_inode, i_ioend_work);
+	struct iomap_ioend	*ioend;
+	struct list_head	tmp;
+	unsigned long		flags;
+
+	spin_lock_irqsave(&ip->i_ioend_lock, flags);
+	list_replace_init(&ip->i_ioend_list, &tmp);
+	spin_unlock_irqrestore(&ip->i_ioend_lock, flags);
+
+	iomap_sort_ioends(&tmp);
+	while ((ioend = list_first_entry_or_null(&tmp, struct iomap_ioend,
+			io_list))) {
+		list_del_init(&ioend->io_list);
+		iomap_ioend_try_merge(ioend, &tmp);
+		if (bio_op(&ioend->io_bio) == REQ_OP_READ)
+			iomap_finish_ioends(ioend,
+				blk_status_to_errno(ioend->io_bio.bi_status));
+		else
+			xfs_end_ioend_write(ioend);
+		cond_resched();
+	}
+}
+
+void
+xfs_end_bio(
+	struct bio		*bio)
+{
+	struct iomap_ioend	*ioend = iomap_ioend_from_bio(bio);
+	struct xfs_inode	*ip = XFS_I(ioend->io_inode);
+	struct xfs_mount	*mp = ip->i_mount;
+	unsigned long		flags;
+
+	/*
+	 * For Appends record the actually written block number and set the
+	 * boundary flag if needed.
+	 */
+	if (IS_ENABLED(CONFIG_XFS_RT) && bio_is_zone_append(bio)) {
+		ioend->io_sector = bio->bi_iter.bi_sector;
+		xfs_mark_rtg_boundary(ioend);
+	}
+
+	spin_lock_irqsave(&ip->i_ioend_lock, flags);
+	if (list_empty(&ip->i_ioend_list))
+		WARN_ON_ONCE(!queue_work(mp->m_unwritten_workqueue,
+					 &ip->i_ioend_work));
+	list_add_tail(&ioend->io_list, &ip->i_ioend_list);
+	spin_unlock_irqrestore(&ip->i_ioend_lock, flags);
+}
diff --git a/fs/xfs/xfs_ioend.h b/fs/xfs/xfs_ioend.h
new file mode 100644
index 000000000000..525865767fca
--- /dev/null
+++ b/fs/xfs/xfs_ioend.h
@@ -0,0 +1,16 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __XFS_IOEND_H
+#define __XFS_IOEND_H
+
+/*
+ * Fast and loose check if this write could update the on-disk inode size.
+ */
+static inline bool xfs_ioend_is_append(struct iomap_ioend *ioend)
+{
+	return ioend->io_offset + ioend->io_size >
+		XFS_I(ioend->io_inode)->i_disk_size;
+}
+
+void xfs_end_bio(struct bio *bio);
+
+#endif /* __XFS_IOEND_H */
-- 
2.53.0

Re: [PATCH 17/22] xfs: split ioend handling into a separate source file

[Darrick J. Wong]

On Thu, Jul 23, 2026 at 04:49:42PM +0200, Christoph Hellwig wrote:
> The ioend handling used to be only for buffered writeback, but has been
> extended to direct I/O and reads.  Split it into a new source file.
> 
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> ---
>  fs/xfs/Makefile    |   1 +
>  fs/xfs/xfs_aops.c  | 179 +------------------------------------------
>  fs/xfs/xfs_aops.h  |   1 -
>  fs/xfs/xfs_file.c  |   2 +-
>  fs/xfs/xfs_ioend.c | 184 +++++++++++++++++++++++++++++++++++++++++++++
>  fs/xfs/xfs_ioend.h |  16 ++++

Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>

--D

>  6 files changed, 203 insertions(+), 180 deletions(-)
>  create mode 100644 fs/xfs/xfs_ioend.c
>  create mode 100644 fs/xfs/xfs_ioend.h
> 
> diff --git a/fs/xfs/Makefile b/fs/xfs/Makefile
> index 9f7133e02576..399a207f2d0e 100644
> --- a/fs/xfs/Makefile
> +++ b/fs/xfs/Makefile
> @@ -91,6 +91,7 @@ xfs-y				+= xfs_aops.o \
>  				   xfs_healthmon.o \
>  				   xfs_icache.o \
>  				   xfs_ioctl.o \
> +				   xfs_ioend.o \
>  				   xfs_iomap.o \
>  				   xfs_iops.o \
>  				   xfs_inode.o \
> diff --git a/fs/xfs/xfs_aops.c b/fs/xfs/xfs_aops.c
> index 0b757b7f35c0..49d21d905cc3 100644
> --- a/fs/xfs/xfs_aops.c
> +++ b/fs/xfs/xfs_aops.c
> @@ -20,6 +20,7 @@
>  #include "xfs_errortag.h"
>  #include "xfs_error.h"
>  #include "xfs_icache.h"
> +#include "xfs_ioend.h"
>  #include "xfs_zone_alloc.h"
>  #include "xfs_rtgroup.h"
>  
> @@ -35,15 +36,6 @@ XFS_WPC(struct iomap_writepage_ctx *ctx)
>  	return container_of(ctx, struct xfs_writepage_ctx, ctx);
>  }
>  
> -/*
> - * Fast and loose check if this write could update the on-disk inode size.
> - */
> -static inline bool xfs_ioend_is_append(struct iomap_ioend *ioend)
> -{
> -	return ioend->io_offset + ioend->io_size >
> -		XFS_I(ioend->io_inode)->i_disk_size;
> -}
> -
>  /*
>   * Update on-disk file size now that data has been written to disk.
>   */
> @@ -79,175 +71,6 @@ xfs_setfilesize(
>  	return xfs_trans_commit(tp);
>  }
>  
> -static void
> -xfs_ioend_put_open_zones(
> -	struct iomap_ioend	*ioend)
> -{
> -	struct iomap_ioend *tmp;
> -
> -	/*
> -	 * Put the open zone for all ioends merged into this one (if any).
> -	 */
> -	list_for_each_entry(tmp, &ioend->io_list, io_list)
> -		xfs_open_zone_put(tmp->io_private);
> -
> -	/*
> -	 * The main ioend might not have an open zone if the submission failed
> -	 * before xfs_zone_alloc_and_submit got called.
> -	 */
> -	if (ioend->io_private)
> -		xfs_open_zone_put(ioend->io_private);
> -}
> -
> -/*
> - * IO write completion.
> - */
> -STATIC void
> -xfs_end_ioend_write(
> -	struct iomap_ioend	*ioend)
> -{
> -	struct xfs_inode	*ip = XFS_I(ioend->io_inode);
> -	struct xfs_mount	*mp = ip->i_mount;
> -	bool			is_zoned = xfs_is_zoned_inode(ip);
> -	xfs_off_t		offset = ioend->io_offset;
> -	size_t			size = ioend->io_size;
> -	unsigned int		nofs_flag;
> -	int			error;
> -
> -	/*
> -	 * We can allocate memory here while doing writeback on behalf of
> -	 * memory reclaim.  To avoid memory allocation deadlocks set the
> -	 * task-wide nofs context for the following operations.
> -	 */
> -	nofs_flag = memalloc_nofs_save();
> -
> -	/*
> -	 * Just clean up the in-memory structures if the fs has been shut down.
> -	 */
> -	if (xfs_is_shutdown(mp)) {
> -		error = -EIO;
> -		goto done;
> -	}
> -
> -	/*
> -	 * Clean up all COW blocks and underlying data fork delalloc blocks on
> -	 * I/O error. The delalloc punch is required because this ioend was
> -	 * mapped to blocks in the COW fork and the associated pages are no
> -	 * longer dirty. If we don't remove delalloc blocks here, they become
> -	 * stale and can corrupt free space accounting on unmount.
> -	 */
> -	error = blk_status_to_errno(ioend->io_bio.bi_status);
> -	if (unlikely(error)) {
> -		/*
> -		 * Zoned writes update the in-core open zone accounting before
> -		 * I/O submission.  A failed write leaves that state
> -		 * inconsistent, so shut down the filesystem instead of letting
> -		 * later writers wait forever for open zone space to become
> -		 * available.
> -		 */
> -		if (is_zoned) {
> -			xfs_force_shutdown(mp, SHUTDOWN_META_IO_ERROR);
> -			goto done;
> -		}
> -		if (ioend->io_flags & IOMAP_IOEND_SHARED) {
> -			ASSERT(!is_zoned);
> -			xfs_reflink_cancel_cow_range(ip, offset, size, true);
> -			xfs_bmap_punch_delalloc_range(ip, XFS_DATA_FORK, offset,
> -					offset + size, NULL);
> -		}
> -		goto done;
> -	}
> -
> -	/*
> -	 * Success: commit the COW or unwritten blocks if needed.
> -	 */
> -	if (is_zoned)
> -		error = xfs_zoned_end_io(ip, offset, size, ioend->io_sector,
> -				ioend->io_private, NULLFSBLOCK);
> -	else if (ioend->io_flags & IOMAP_IOEND_SHARED)
> -		error = xfs_reflink_end_cow(ip, offset, size);
> -	else if (ioend->io_flags & IOMAP_IOEND_UNWRITTEN)
> -		error = xfs_iomap_write_unwritten(ip, offset, size, false);
> -
> -	if (!error &&
> -	    !(ioend->io_flags & IOMAP_IOEND_DIRECT) &&
> -	    xfs_ioend_is_append(ioend))
> -		error = xfs_setfilesize(ip, offset, size);
> -done:
> -	if (is_zoned)
> -		xfs_ioend_put_open_zones(ioend);
> -	iomap_finish_ioends(ioend, error);
> -	memalloc_nofs_restore(nofs_flag);
> -}
> -
> -/*
> - * Finish all pending IO completions that require transactional modifications.
> - *
> - * We try to merge physical and logically contiguous ioends before completion to
> - * minimise the number of transactions we need to perform during IO completion.
> - * Both unwritten extent conversion and COW remapping need to iterate and modify
> - * one physical extent at a time, so we gain nothing by merging physically
> - * discontiguous extents here.
> - *
> - * The ioend chain length that we can be processing here is largely unbound in
> - * length and we may have to perform significant amounts of work on each ioend
> - * to complete it. Hence we have to be careful about holding the CPU for too
> - * long in this loop.
> - */
> -void
> -xfs_end_io(
> -	struct work_struct	*work)
> -{
> -	struct xfs_inode	*ip =
> -		container_of(work, struct xfs_inode, i_ioend_work);
> -	struct iomap_ioend	*ioend;
> -	struct list_head	tmp;
> -	unsigned long		flags;
> -
> -	spin_lock_irqsave(&ip->i_ioend_lock, flags);
> -	list_replace_init(&ip->i_ioend_list, &tmp);
> -	spin_unlock_irqrestore(&ip->i_ioend_lock, flags);
> -
> -	iomap_sort_ioends(&tmp);
> -	while ((ioend = list_first_entry_or_null(&tmp, struct iomap_ioend,
> -			io_list))) {
> -		list_del_init(&ioend->io_list);
> -		iomap_ioend_try_merge(ioend, &tmp);
> -		if (bio_op(&ioend->io_bio) == REQ_OP_READ)
> -			iomap_finish_ioends(ioend,
> -				blk_status_to_errno(ioend->io_bio.bi_status));
> -		else
> -			xfs_end_ioend_write(ioend);
> -		cond_resched();
> -	}
> -}
> -
> -void
> -xfs_end_bio(
> -	struct bio		*bio)
> -{
> -	struct iomap_ioend	*ioend = iomap_ioend_from_bio(bio);
> -	struct xfs_inode	*ip = XFS_I(ioend->io_inode);
> -	struct xfs_mount	*mp = ip->i_mount;
> -	unsigned long		flags;
> -
> -	/*
> -	 * For Appends record the actually written block number and set the
> -	 * boundary flag if needed.
> -	 */
> -	if (IS_ENABLED(CONFIG_XFS_RT) && bio_is_zone_append(bio)) {
> -		ioend->io_sector = bio->bi_iter.bi_sector;
> -		xfs_mark_rtg_boundary(ioend);
> -	}
> -
> -	spin_lock_irqsave(&ip->i_ioend_lock, flags);
> -	if (list_empty(&ip->i_ioend_list))
> -		WARN_ON_ONCE(!queue_work(mp->m_unwritten_workqueue,
> -					 &ip->i_ioend_work));
> -	list_add_tail(&ioend->io_list, &ip->i_ioend_list);
> -	spin_unlock_irqrestore(&ip->i_ioend_lock, flags);
> -}
> -
>  /*
>   * We cannot cancel the ioend directly on error.  We may have already set other
>   * pages under writeback and hence we have to run I/O completion to mark the
> diff --git a/fs/xfs/xfs_aops.h b/fs/xfs/xfs_aops.h
> index 5a7a0f1a0b49..d5ae5c9d4c26 100644
> --- a/fs/xfs/xfs_aops.h
> +++ b/fs/xfs/xfs_aops.h
> @@ -10,6 +10,5 @@ extern const struct address_space_operations xfs_address_space_operations;
>  extern const struct address_space_operations xfs_dax_aops;
>  
>  int xfs_setfilesize(struct xfs_inode *ip, xfs_off_t offset, size_t size);
> -void xfs_end_bio(struct bio *bio);
>  
>  #endif /* __XFS_AOPS_H__ */
> diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c
> index 2c19e69b93cd..c0c3a11e7ff2 100644
> --- a/fs/xfs/xfs_file.c
> +++ b/fs/xfs/xfs_file.c
> @@ -25,7 +25,7 @@
>  #include "xfs_iomap.h"
>  #include "xfs_reflink.h"
>  #include "xfs_file.h"
> -#include "xfs_aops.h"
> +#include "xfs_ioend.h"
>  #include "xfs_zone_alloc.h"
>  #include "xfs_error.h"
>  #include "xfs_errortag.h"
> diff --git a/fs/xfs/xfs_ioend.c b/fs/xfs/xfs_ioend.c
> new file mode 100644
> index 000000000000..40695d18dac0
> --- /dev/null
> +++ b/fs/xfs/xfs_ioend.c
> @@ -0,0 +1,184 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (c) 2016-2025 Christoph Hellwig.
> + * All Rights Reserved.
> + */
> +#include "xfs_platform.h"
> +#include "xfs_shared.h"
> +#include "xfs_format.h"
> +#include "xfs_log_format.h"
> +#include "xfs_trans_resv.h"
> +#include "xfs_mount.h"
> +#include "xfs_inode.h"
> +#include "xfs_iomap.h"
> +#include "xfs_trace.h"
> +#include "xfs_bmap_util.h"
> +#include "xfs_reflink.h"
> +#include "xfs_zone_alloc.h"
> +#include "xfs_ioend.h"
> +
> +static void
> +xfs_ioend_put_open_zones(
> +	struct iomap_ioend	*ioend)
> +{
> +	struct iomap_ioend *tmp;
> +
> +	/*
> +	 * Put the open zone for all ioends merged into this one (if any).
> +	 */
> +	list_for_each_entry(tmp, &ioend->io_list, io_list)
> +		xfs_open_zone_put(tmp->io_private);
> +
> +	/*
> +	 * The main ioend might not have an open zone if the submission failed
> +	 * before xfs_zone_alloc_and_submit got called.
> +	 */
> +	if (ioend->io_private)
> +		xfs_open_zone_put(ioend->io_private);
> +}
> +
> +static void
> +xfs_end_ioend_write(
> +	struct iomap_ioend	*ioend)
> +{
> +	struct xfs_inode	*ip = XFS_I(ioend->io_inode);
> +	struct xfs_mount	*mp = ip->i_mount;
> +	bool			is_zoned = xfs_is_zoned_inode(ip);
> +	xfs_off_t		offset = ioend->io_offset;
> +	size_t			size = ioend->io_size;
> +	unsigned int		nofs_flag;
> +	int			error;
> +
> +	/*
> +	 * We can allocate memory here while doing writeback on behalf of
> +	 * memory reclaim.  To avoid memory allocation deadlocks set the
> +	 * task-wide nofs context for the following operations.
> +	 */
> +	nofs_flag = memalloc_nofs_save();
> +
> +	/*
> +	 * Just clean up the in-memory structures if the fs has been shut down.
> +	 */
> +	if (xfs_is_shutdown(mp)) {
> +		error = -EIO;
> +		goto done;
> +	}
> +
> +	/*
> +	 * Clean up all COW blocks and underlying data fork delalloc blocks on
> +	 * I/O error. The delalloc punch is required because this ioend was
> +	 * mapped to blocks in the COW fork and the associated pages are no
> +	 * longer dirty. If we don't remove delalloc blocks here, they become
> +	 * stale and can corrupt free space accounting on unmount.
> +	 */
> +	error = blk_status_to_errno(ioend->io_bio.bi_status);
> +	if (unlikely(error)) {
> +		/*
> +		 * Zoned writes update the in-core open zone accounting before
> +		 * I/O submission.  A failed write leaves that state
> +		 * inconsistent, so shut down the filesystem instead of letting
> +		 * later writers wait forever for open zone space to become
> +		 * available.
> +		 */
> +		if (is_zoned) {
> +			xfs_force_shutdown(mp, SHUTDOWN_META_IO_ERROR);
> +			goto done;
> +		}
> +		if (ioend->io_flags & IOMAP_IOEND_SHARED) {
> +			ASSERT(!is_zoned);
> +			xfs_reflink_cancel_cow_range(ip, offset, size, true);
> +			xfs_bmap_punch_delalloc_range(ip, XFS_DATA_FORK, offset,
> +					offset + size, NULL);
> +		}
> +		goto done;
> +	}
> +
> +	/*
> +	 * Success: commit the COW or unwritten blocks if needed.
> +	 */
> +	if (is_zoned)
> +		error = xfs_zoned_end_io(ip, offset, size, ioend->io_sector,
> +				ioend->io_private, NULLFSBLOCK);
> +	else if (ioend->io_flags & IOMAP_IOEND_SHARED)
> +		error = xfs_reflink_end_cow(ip, offset, size);
> +	else if (ioend->io_flags & IOMAP_IOEND_UNWRITTEN)
> +		error = xfs_iomap_write_unwritten(ip, offset, size, false);
> +
> +	if (!error &&
> +	    !(ioend->io_flags & IOMAP_IOEND_DIRECT) &&
> +	    xfs_ioend_is_append(ioend))
> +		error = xfs_setfilesize(ip, offset, size);
> +done:
> +	if (is_zoned)
> +		xfs_ioend_put_open_zones(ioend);
> +	iomap_finish_ioends(ioend, error);
> +	memalloc_nofs_restore(nofs_flag);
> +}
> +
> +/*
> + * Finish all pending IO completions that require transactional modifications.
> + *
> + * We try to merge physical and logically contiguous ioends before completion to
> + * minimise the number of transactions we need to perform during IO completion.
> + * Both unwritten extent conversion and COW remapping need to iterate and modify
> + * one physical extent at a time, so we gain nothing by merging physically
> + * discontiguous extents here.
> + *
> + * The ioend chain length that we can be processing here is largely unbound in
> + * length and we may have to perform significant amounts of work on each ioend
> + * to complete it. Hence we have to be careful about holding the CPU for too
> + * long in this loop.
> + */
> +void
> +xfs_end_io(
> +	struct work_struct	*work)
> +{
> +	struct xfs_inode	*ip =
> +		container_of(work, struct xfs_inode, i_ioend_work);
> +	struct iomap_ioend	*ioend;
> +	struct list_head	tmp;
> +	unsigned long		flags;
> +
> +	spin_lock_irqsave(&ip->i_ioend_lock, flags);
> +	list_replace_init(&ip->i_ioend_list, &tmp);
> +	spin_unlock_irqrestore(&ip->i_ioend_lock, flags);
> +
> +	iomap_sort_ioends(&tmp);
> +	while ((ioend = list_first_entry_or_null(&tmp, struct iomap_ioend,
> +			io_list))) {
> +		list_del_init(&ioend->io_list);
> +		iomap_ioend_try_merge(ioend, &tmp);
> +		if (bio_op(&ioend->io_bio) == REQ_OP_READ)
> +			iomap_finish_ioends(ioend,
> +				blk_status_to_errno(ioend->io_bio.bi_status));
> +		else
> +			xfs_end_ioend_write(ioend);
> +		cond_resched();
> +	}
> +}
> +
> +void
> +xfs_end_bio(
> +	struct bio		*bio)
> +{
> +	struct iomap_ioend	*ioend = iomap_ioend_from_bio(bio);
> +	struct xfs_inode	*ip = XFS_I(ioend->io_inode);
> +	struct xfs_mount	*mp = ip->i_mount;
> +	unsigned long		flags;
> +
> +	/*
> +	 * For Appends record the actually written block number and set the
> +	 * boundary flag if needed.
> +	 */
> +	if (IS_ENABLED(CONFIG_XFS_RT) && bio_is_zone_append(bio)) {
> +		ioend->io_sector = bio->bi_iter.bi_sector;
> +		xfs_mark_rtg_boundary(ioend);
> +	}
> +
> +	spin_lock_irqsave(&ip->i_ioend_lock, flags);
> +	if (list_empty(&ip->i_ioend_list))
> +		WARN_ON_ONCE(!queue_work(mp->m_unwritten_workqueue,
> +					 &ip->i_ioend_work));
> +	list_add_tail(&ioend->io_list, &ip->i_ioend_list);
> +	spin_unlock_irqrestore(&ip->i_ioend_lock, flags);
> +}
> diff --git a/fs/xfs/xfs_ioend.h b/fs/xfs/xfs_ioend.h
> new file mode 100644
> index 000000000000..525865767fca
> --- /dev/null
> +++ b/fs/xfs/xfs_ioend.h
> @@ -0,0 +1,16 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +#ifndef __XFS_IOEND_H
> +#define __XFS_IOEND_H
> +
> +/*
> + * Fast and loose check if this write could update the on-disk inode size.
> + */
> +static inline bool xfs_ioend_is_append(struct iomap_ioend *ioend)
> +{
> +	return ioend->io_offset + ioend->io_size >
> +		XFS_I(ioend->io_inode)->i_disk_size;
> +}
> +
> +void xfs_end_bio(struct bio *bio);
> +
> +#endif /* __XFS_IOEND_H */
> -- 
> 2.53.0
> 
> 

[PATCH 18/22] xfs: use BIO_COMPLETE_IN_TASK for bounce buffered read I/Os

[Christoph Hellwig]

Stop using the xfs per-inode work struct for completing read bios, as
unlike writes we don't want to serialize reads on a single inode as
there is no exclusive resource contention for them.

Factor the code for kicking off a read that needs and ioend and the
task context completion into a single helper so that it is split off
the xfs_end_bio machinery, which is not only used for writes.

Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 fs/xfs/xfs_aops.c  | 10 ++++------
 fs/xfs/xfs_file.c  |  9 +--------
 fs/xfs/xfs_ioend.c | 32 +++++++++++++++++++++++++++-----
 fs/xfs/xfs_ioend.h |  2 ++
 4 files changed, 34 insertions(+), 19 deletions(-)

diff --git a/fs/xfs/xfs_aops.c b/fs/xfs/xfs_aops.c
index 49d21d905cc3..76918bd15ca8 100644
--- a/fs/xfs/xfs_aops.c
+++ b/fs/xfs/xfs_aops.c
@@ -580,12 +580,10 @@ xfs_bio_submit_read(
 	const struct iomap_iter		*iter,
 	struct iomap_read_folio_ctx	*ctx)
 {
-	struct bio			*bio = ctx->read_ctx;
-
-	/* defer read completions to the ioend workqueue */
-	iomap_init_ioend(iter->inode, bio, ctx->read_ctx_file_offset,
-		iomap_ioend_flags(&iter->iomap));
-	iomap_bio_submit_read_endio(iter, ctx, xfs_end_bio);
+	xfs_ioend_submit_read(iter->inode, ctx->read_ctx,
+			ctx->read_ctx_file_offset,
+			iomap_ioend_flags(&iter->iomap));
+	ctx->read_ctx = NULL;
 }
 
 static const struct iomap_read_ops xfs_iomap_read_ops = {
diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c
index c0c3a11e7ff2..d31a1dddcdc3 100644
--- a/fs/xfs/xfs_file.c
+++ b/fs/xfs/xfs_file.c
@@ -37,7 +37,6 @@
 #include <linux/fadvise.h>
 #include <linux/mount.h>
 #include <linux/filelock.h>
-#include <linux/bio-integrity.h>
 
 static const struct vm_operations_struct xfs_file_vm_ops;
 
@@ -236,14 +235,8 @@ xfs_dio_read_bounce_submit_io(
 	struct bio		*bio,
 	loff_t			file_offset)
 {
-	struct iomap_ioend	*ioend;
-
-	ioend = iomap_init_ioend(iter->inode, bio, file_offset,
+	xfs_ioend_submit_read(iter->inode, bio, file_offset,
 			iomap_ioend_flags(&iter->iomap) | IOMAP_IOEND_DIRECT);
-	if (ioend->io_flags & IOMAP_IOEND_INTEGRITY)
-		fs_bio_integrity_alloc(bio);
-	bio->bi_end_io = xfs_end_bio;
-	submit_bio(bio);
 }
 
 static const struct iomap_dio_ops xfs_dio_read_bounce_ops = {
diff --git a/fs/xfs/xfs_ioend.c b/fs/xfs/xfs_ioend.c
index 40695d18dac0..37a3ae8066e9 100644
--- a/fs/xfs/xfs_ioend.c
+++ b/fs/xfs/xfs_ioend.c
@@ -16,6 +16,32 @@
 #include "xfs_reflink.h"
 #include "xfs_zone_alloc.h"
 #include "xfs_ioend.h"
+#include <linux/bio-integrity.h>
+
+static void
+xfs_end_io_read(
+	struct bio		*bio)
+{
+	struct iomap_ioend	*ioend = iomap_ioend_from_bio(bio);
+	int			error = blk_status_to_errno(bio->bi_status);
+
+	iomap_finish_ioends(ioend, error);
+}
+
+void
+xfs_ioend_submit_read(
+	struct inode		*inode,
+	struct bio		*bio,
+	loff_t			file_offset,
+	u16			ioend_flags)
+{
+	iomap_init_ioend(inode, bio, file_offset, ioend_flags);
+	if (ioend_flags & IOMAP_IOEND_INTEGRITY)
+		fs_bio_integrity_alloc(bio);
+	bio->bi_end_io = xfs_end_io_read;
+	bio_set_flag(bio, BIO_COMPLETE_IN_TASK);
+	submit_bio(bio);
+}
 
 static void
 xfs_ioend_put_open_zones(
@@ -148,11 +174,7 @@ xfs_end_io(
 			io_list))) {
 		list_del_init(&ioend->io_list);
 		iomap_ioend_try_merge(ioend, &tmp);
-		if (bio_op(&ioend->io_bio) == REQ_OP_READ)
-			iomap_finish_ioends(ioend,
-				blk_status_to_errno(ioend->io_bio.bi_status));
-		else
-			xfs_end_ioend_write(ioend);
+		xfs_end_ioend_write(ioend);
 		cond_resched();
 	}
 }
diff --git a/fs/xfs/xfs_ioend.h b/fs/xfs/xfs_ioend.h
index 525865767fca..7c2a1ea3e6ed 100644
--- a/fs/xfs/xfs_ioend.h
+++ b/fs/xfs/xfs_ioend.h
@@ -12,5 +12,7 @@ static inline bool xfs_ioend_is_append(struct iomap_ioend *ioend)
 }
 
 void xfs_end_bio(struct bio *bio);
+void xfs_ioend_submit_read(struct inode *inode, struct bio *bio,
+		loff_t file_offset, u16 ioend_flags);
 
 #endif /* __XFS_IOEND_H */
-- 
2.53.0

Re: [PATCH 18/22] xfs: use BIO_COMPLETE_IN_TASK for bounce buffered read I/Os

[Andrey Albershteyn]

On 2026-07-23 16:49:43, Christoph Hellwig wrote:
> Stop using the xfs per-inode work struct for completing read bios, as
> unlike writes we don't want to serialize reads on a single inode as
> there is no exclusive resource contention for them.
> 
> Factor the code for kicking off a read that needs and ioend and the
> task context completion into a single helper so that it is split off
> the xfs_end_bio machinery, which is not only used for writes.
> 
> Signed-off-by: Christoph Hellwig <hch@lst.de>

I would probably need something like this for fsverity.

When using integrity checksums with fsverity together, both file data
ioends and fsverity metadata ioends get onto the inode queue
(without checksums there's no metadata ioends). This way worker could
self-deadlock if data ioend processed first in xfs_end_io(). It will
call verify_bio to read merkle pages which could be already waiting
in queue.

I initially considered changing xfs_end_io, for read ioends, to just
schedule them instead of adding to the queue, but decided just sort
metadata ioends first in the queue [1] as a bit simpler fix.

The difference is that work won't be scheduled on the high-priority
fsverity's workqueue. Not sure how critical this is as with
checksums all reads would be the same priority.

1: https://lore.kernel.org/fsverity/20260721184346.416657-15-aalbersh@kernel.org/T/#u

-- 
- Andrey

> ---
>  fs/xfs/xfs_aops.c  | 10 ++++------
>  fs/xfs/xfs_file.c  |  9 +--------
>  fs/xfs/xfs_ioend.c | 32 +++++++++++++++++++++++++++-----
>  fs/xfs/xfs_ioend.h |  2 ++
>  4 files changed, 34 insertions(+), 19 deletions(-)
> 
> diff --git a/fs/xfs/xfs_aops.c b/fs/xfs/xfs_aops.c
> index 49d21d905cc3..76918bd15ca8 100644
> --- a/fs/xfs/xfs_aops.c
> +++ b/fs/xfs/xfs_aops.c
> @@ -580,12 +580,10 @@ xfs_bio_submit_read(
>  	const struct iomap_iter		*iter,
>  	struct iomap_read_folio_ctx	*ctx)
>  {
> -	struct bio			*bio = ctx->read_ctx;
> -
> -	/* defer read completions to the ioend workqueue */
> -	iomap_init_ioend(iter->inode, bio, ctx->read_ctx_file_offset,
> -		iomap_ioend_flags(&iter->iomap));
> -	iomap_bio_submit_read_endio(iter, ctx, xfs_end_bio);
> +	xfs_ioend_submit_read(iter->inode, ctx->read_ctx,
> +			ctx->read_ctx_file_offset,
> +			iomap_ioend_flags(&iter->iomap));
> +	ctx->read_ctx = NULL;
>  }
>  
>  static const struct iomap_read_ops xfs_iomap_read_ops = {
> diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c
> index c0c3a11e7ff2..d31a1dddcdc3 100644
> --- a/fs/xfs/xfs_file.c
> +++ b/fs/xfs/xfs_file.c
> @@ -37,7 +37,6 @@
>  #include <linux/fadvise.h>
>  #include <linux/mount.h>
>  #include <linux/filelock.h>
> -#include <linux/bio-integrity.h>
>  
>  static const struct vm_operations_struct xfs_file_vm_ops;
>  
> @@ -236,14 +235,8 @@ xfs_dio_read_bounce_submit_io(
>  	struct bio		*bio,
>  	loff_t			file_offset)
>  {
> -	struct iomap_ioend	*ioend;
> -
> -	ioend = iomap_init_ioend(iter->inode, bio, file_offset,
> +	xfs_ioend_submit_read(iter->inode, bio, file_offset,
>  			iomap_ioend_flags(&iter->iomap) | IOMAP_IOEND_DIRECT);
> -	if (ioend->io_flags & IOMAP_IOEND_INTEGRITY)
> -		fs_bio_integrity_alloc(bio);
> -	bio->bi_end_io = xfs_end_bio;
> -	submit_bio(bio);
>  }
>  
>  static const struct iomap_dio_ops xfs_dio_read_bounce_ops = {
> diff --git a/fs/xfs/xfs_ioend.c b/fs/xfs/xfs_ioend.c
> index 40695d18dac0..37a3ae8066e9 100644
> --- a/fs/xfs/xfs_ioend.c
> +++ b/fs/xfs/xfs_ioend.c
> @@ -16,6 +16,32 @@
>  #include "xfs_reflink.h"
>  #include "xfs_zone_alloc.h"
>  #include "xfs_ioend.h"
> +#include <linux/bio-integrity.h>
> +
> +static void
> +xfs_end_io_read(
> +	struct bio		*bio)
> +{
> +	struct iomap_ioend	*ioend = iomap_ioend_from_bio(bio);
> +	int			error = blk_status_to_errno(bio->bi_status);
> +
> +	iomap_finish_ioends(ioend, error);
> +}
> +
> +void
> +xfs_ioend_submit_read(
> +	struct inode		*inode,
> +	struct bio		*bio,
> +	loff_t			file_offset,
> +	u16			ioend_flags)
> +{
> +	iomap_init_ioend(inode, bio, file_offset, ioend_flags);
> +	if (ioend_flags & IOMAP_IOEND_INTEGRITY)
> +		fs_bio_integrity_alloc(bio);
> +	bio->bi_end_io = xfs_end_io_read;
> +	bio_set_flag(bio, BIO_COMPLETE_IN_TASK);
> +	submit_bio(bio);
> +}
>  
>  static void
>  xfs_ioend_put_open_zones(
> @@ -148,11 +174,7 @@ xfs_end_io(
>  			io_list))) {
>  		list_del_init(&ioend->io_list);
>  		iomap_ioend_try_merge(ioend, &tmp);
> -		if (bio_op(&ioend->io_bio) == REQ_OP_READ)
> -			iomap_finish_ioends(ioend,
> -				blk_status_to_errno(ioend->io_bio.bi_status));
> -		else
> -			xfs_end_ioend_write(ioend);
> +		xfs_end_ioend_write(ioend);
>  		cond_resched();
>  	}
>  }
> diff --git a/fs/xfs/xfs_ioend.h b/fs/xfs/xfs_ioend.h
> index 525865767fca..7c2a1ea3e6ed 100644
> --- a/fs/xfs/xfs_ioend.h
> +++ b/fs/xfs/xfs_ioend.h
> @@ -12,5 +12,7 @@ static inline bool xfs_ioend_is_append(struct iomap_ioend *ioend)
>  }
>  
>  void xfs_end_bio(struct bio *bio);
> +void xfs_ioend_submit_read(struct inode *inode, struct bio *bio,
> +		loff_t file_offset, u16 ioend_flags);
>  
>  #endif /* __XFS_IOEND_H */
> -- 
> 2.53.0
> 
> 

Re: [PATCH 18/22] xfs: use BIO_COMPLETE_IN_TASK for bounce buffered read I/Os

[Christoph Hellwig]

On Thu, Jul 23, 2026 at 05:58:49PM +0200, Andrey Albershteyn wrote:
> I would probably need something like this for fsverity.
> 
> When using integrity checksums with fsverity together, both file data
> ioends and fsverity metadata ioends get onto the inode queue
> (without checksums there's no metadata ioends). This way worker could
> self-deadlock if data ioend processed first in xfs_end_io(). It will
> call verify_bio to read merkle pages which could be already waiting
> in queue.

Yes.  Same as the situation we have (or had as this is changing
for 7.3) with fsverity and fscrypt.

> I initially considered changing xfs_end_io, for read ioends, to just
> schedule them instead of adding to the queue, but decided just sort
> metadata ioends first in the queue [1] as a bit simpler fix.
> 
> The difference is that work won't be scheduled on the high-priority
> fsverity's workqueue. Not sure how critical this is as with
> checksums all reads would be the same priority.

I'll need to go back to your series, but just sorting isn't going
to fix the problem, as the reading of the verity metadata could
be kicked off from the completion workqueue and you could be blocking
the rescuer thread.  So either verity needs it's own WQ, or we'd need
some way for a currently running I/O completion to exit and get
restarted.

With BIO_COMPLETE_IN_TASK we could probably do that restart thing.
Or add a second set of workers for "metadata".

Re: [PATCH 18/22] xfs: use BIO_COMPLETE_IN_TASK for bounce buffered read I/Os

[Andrey Albershteyn]

On 2026-07-24 08:21:20, Christoph Hellwig wrote:
> On Thu, Jul 23, 2026 at 05:58:49PM +0200, Andrey Albershteyn wrote:
> > I would probably need something like this for fsverity.
> > 
> > When using integrity checksums with fsverity together, both file data
> > ioends and fsverity metadata ioends get onto the inode queue
> > (without checksums there's no metadata ioends). This way worker could
> > self-deadlock if data ioend processed first in xfs_end_io(). It will
> > call verify_bio to read merkle pages which could be already waiting
> > in queue.
> 
> Yes.  Same as the situation we have (or had as this is changing
> for 7.3) with fsverity and fscrypt.
> 
> > I initially considered changing xfs_end_io, for read ioends, to just
> > schedule them instead of adding to the queue, but decided just sort
> > metadata ioends first in the queue [1] as a bit simpler fix.
> > 
> > The difference is that work won't be scheduled on the high-priority
> > fsverity's workqueue. Not sure how critical this is as with
> > checksums all reads would be the same priority.
> 
> I'll need to go back to your series, but just sorting isn't going
> to fix the problem, as the reading of the verity metadata could
> be kicked off from the completion workqueue and you could be blocking
> the rescuer thread.  So either verity needs it's own WQ, or we'd need
> some way for a currently running I/O completion to exit and get
> restarted.

Oh I see, then, the data reads could be queue further with
fsverity_enqueue_verify_work(ioend). This way bio completion queue
can continue process incoming metadata reads while data ioend is
waiting for verification.

> 
> With BIO_COMPLETE_IN_TASK we could probably do that restart thing.
> Or add a second set of workers for "metadata".
> 

-- 
- Andrey

Re: [PATCH 18/22] xfs: use BIO_COMPLETE_IN_TASK for bounce buffered read I/Os

[Darrick J. Wong]

On Thu, Jul 23, 2026 at 04:49:43PM +0200, Christoph Hellwig wrote:
> Stop using the xfs per-inode work struct for completing read bios, as
> unlike writes we don't want to serialize reads on a single inode as
> there is no exclusive resource contention for them.
> 
> Factor the code for kicking off a read that needs and ioend and the
> task context completion into a single helper so that it is split off
> the xfs_end_bio machinery, which is not only used for writes.
> 
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> ---
>  fs/xfs/xfs_aops.c  | 10 ++++------
>  fs/xfs/xfs_file.c  |  9 +--------
>  fs/xfs/xfs_ioend.c | 32 +++++++++++++++++++++++++++-----
>  fs/xfs/xfs_ioend.h |  2 ++
>  4 files changed, 34 insertions(+), 19 deletions(-)
> 
> diff --git a/fs/xfs/xfs_aops.c b/fs/xfs/xfs_aops.c
> index 49d21d905cc3..76918bd15ca8 100644
> --- a/fs/xfs/xfs_aops.c
> +++ b/fs/xfs/xfs_aops.c
> @@ -580,12 +580,10 @@ xfs_bio_submit_read(
>  	const struct iomap_iter		*iter,
>  	struct iomap_read_folio_ctx	*ctx)
>  {
> -	struct bio			*bio = ctx->read_ctx;
> -
> -	/* defer read completions to the ioend workqueue */
> -	iomap_init_ioend(iter->inode, bio, ctx->read_ctx_file_offset,
> -		iomap_ioend_flags(&iter->iomap));
> -	iomap_bio_submit_read_endio(iter, ctx, xfs_end_bio);
> +	xfs_ioend_submit_read(iter->inode, ctx->read_ctx,
> +			ctx->read_ctx_file_offset,
> +			iomap_ioend_flags(&iter->iomap));
> +	ctx->read_ctx = NULL;

Hmm, so I guess the advantage here is that instead of chaining together
a lot of ioends to do all the read completion stuff serially, we can
instead process them all in parallel(ish) since we don't really need to
grab ILOCKs and stuff like that, right?

If so then I think this it's appropriate not to use the ioend coalescing
anymore:

Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>

--D

>  }
>  
>  static const struct iomap_read_ops xfs_iomap_read_ops = {
> diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c
> index c0c3a11e7ff2..d31a1dddcdc3 100644
> --- a/fs/xfs/xfs_file.c
> +++ b/fs/xfs/xfs_file.c
> @@ -37,7 +37,6 @@
>  #include <linux/fadvise.h>
>  #include <linux/mount.h>
>  #include <linux/filelock.h>
> -#include <linux/bio-integrity.h>
>  
>  static const struct vm_operations_struct xfs_file_vm_ops;
>  
> @@ -236,14 +235,8 @@ xfs_dio_read_bounce_submit_io(
>  	struct bio		*bio,
>  	loff_t			file_offset)
>  {
> -	struct iomap_ioend	*ioend;
> -
> -	ioend = iomap_init_ioend(iter->inode, bio, file_offset,
> +	xfs_ioend_submit_read(iter->inode, bio, file_offset,
>  			iomap_ioend_flags(&iter->iomap) | IOMAP_IOEND_DIRECT);
> -	if (ioend->io_flags & IOMAP_IOEND_INTEGRITY)
> -		fs_bio_integrity_alloc(bio);
> -	bio->bi_end_io = xfs_end_bio;
> -	submit_bio(bio);
>  }
>  
>  static const struct iomap_dio_ops xfs_dio_read_bounce_ops = {
> diff --git a/fs/xfs/xfs_ioend.c b/fs/xfs/xfs_ioend.c
> index 40695d18dac0..37a3ae8066e9 100644
> --- a/fs/xfs/xfs_ioend.c
> +++ b/fs/xfs/xfs_ioend.c
> @@ -16,6 +16,32 @@
>  #include "xfs_reflink.h"
>  #include "xfs_zone_alloc.h"
>  #include "xfs_ioend.h"
> +#include <linux/bio-integrity.h>
> +
> +static void
> +xfs_end_io_read(
> +	struct bio		*bio)
> +{
> +	struct iomap_ioend	*ioend = iomap_ioend_from_bio(bio);
> +	int			error = blk_status_to_errno(bio->bi_status);
> +
> +	iomap_finish_ioends(ioend, error);
> +}
> +
> +void
> +xfs_ioend_submit_read(
> +	struct inode		*inode,
> +	struct bio		*bio,
> +	loff_t			file_offset,
> +	u16			ioend_flags)
> +{
> +	iomap_init_ioend(inode, bio, file_offset, ioend_flags);
> +	if (ioend_flags & IOMAP_IOEND_INTEGRITY)
> +		fs_bio_integrity_alloc(bio);
> +	bio->bi_end_io = xfs_end_io_read;
> +	bio_set_flag(bio, BIO_COMPLETE_IN_TASK);
> +	submit_bio(bio);
> +}
>  
>  static void
>  xfs_ioend_put_open_zones(
> @@ -148,11 +174,7 @@ xfs_end_io(
>  			io_list))) {
>  		list_del_init(&ioend->io_list);
>  		iomap_ioend_try_merge(ioend, &tmp);
> -		if (bio_op(&ioend->io_bio) == REQ_OP_READ)
> -			iomap_finish_ioends(ioend,
> -				blk_status_to_errno(ioend->io_bio.bi_status));
> -		else
> -			xfs_end_ioend_write(ioend);
> +		xfs_end_ioend_write(ioend);
>  		cond_resched();
>  	}
>  }
> diff --git a/fs/xfs/xfs_ioend.h b/fs/xfs/xfs_ioend.h
> index 525865767fca..7c2a1ea3e6ed 100644
> --- a/fs/xfs/xfs_ioend.h
> +++ b/fs/xfs/xfs_ioend.h
> @@ -12,5 +12,7 @@ static inline bool xfs_ioend_is_append(struct iomap_ioend *ioend)
>  }
>  
>  void xfs_end_bio(struct bio *bio);
> +void xfs_ioend_submit_read(struct inode *inode, struct bio *bio,
> +		loff_t file_offset, u16 ioend_flags);
>  
>  #endif /* __XFS_IOEND_H */
> -- 
> 2.53.0
> 
> 

Re: [PATCH 18/22] xfs: use BIO_COMPLETE_IN_TASK for bounce buffered read I/Os

[Christoph Hellwig]

On Thu, Jul 23, 2026 at 01:58:49PM -0700, Darrick J. Wong wrote:
> Hmm, so I guess the advantage here is that instead of chaining together
> a lot of ioends to do all the read completion stuff serially, we can
> instead process them all in parallel(ish) since we don't really need to
> grab ILOCKs and stuff like that, right?

Yes.

[PATCH 19/22] iomap,xfs: move integrity verification to the file system

[Christoph Hellwig]

Integrity support in file systems already requires file system-specific
completion handling because it must be run in process context.

Move the actual verification to the file system so that it can better
handle errors in file system specific ways, and to support lazy bounce
buffering.

Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 fs/iomap/ioend.c      | 8 ++------
 fs/xfs/xfs_ioend.c    | 3 +++
 include/linux/iomap.h | 1 +
 3 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/fs/iomap/ioend.c b/fs/iomap/ioend.c
index 438be92e131b..b1078f7c08c6 100644
--- a/fs/iomap/ioend.c
+++ b/fs/iomap/ioend.c
@@ -308,12 +308,13 @@ ssize_t iomap_add_to_ioend(struct iomap_writepage_ctx *wpc, struct folio *folio,
 }
 EXPORT_SYMBOL_GPL(iomap_add_to_ioend);
 
-static int iomap_ioend_integrity_verify(struct iomap_ioend *ioend)
+int iomap_ioend_integrity_verify(struct iomap_ioend *ioend)
 {
 	struct bvec_iter data_iter = BVEC_ITER_IOEND(ioend);
 
 	return fs_bio_integrity_verify(&ioend->io_bio, &data_iter);
 }
+EXPORT_SYMBOL_GPL(iomap_ioend_integrity_verify);
 
 static u32 iomap_finish_ioend(struct iomap_ioend *ioend, int error)
 {
@@ -330,11 +331,6 @@ static u32 iomap_finish_ioend(struct iomap_ioend *ioend, int error)
 	if (!atomic_dec_and_test(&ioend->io_remaining))
 		return 0;
 
-	if (!ioend->io_error &&
-	    bio_integrity(&ioend->io_bio) &&
-	    bio_op(&ioend->io_bio) == REQ_OP_READ)
-		ioend->io_error = iomap_ioend_integrity_verify(ioend);
-
 	if (ioend->io_flags & IOMAP_IOEND_DIRECT)
 		return iomap_finish_ioend_direct(ioend);
 	if (bio_op(&ioend->io_bio) == REQ_OP_READ)
diff --git a/fs/xfs/xfs_ioend.c b/fs/xfs/xfs_ioend.c
index 37a3ae8066e9..a095cf217863 100644
--- a/fs/xfs/xfs_ioend.c
+++ b/fs/xfs/xfs_ioend.c
@@ -25,6 +25,9 @@ xfs_end_io_read(
 	struct iomap_ioend	*ioend = iomap_ioend_from_bio(bio);
 	int			error = blk_status_to_errno(bio->bi_status);
 
+	if (!error && (ioend->io_flags & IOMAP_IOEND_INTEGRITY))
+		error = iomap_ioend_integrity_verify(ioend);
+
 	iomap_finish_ioends(ioend, error);
 }
 
diff --git a/include/linux/iomap.h b/include/linux/iomap.h
index 1916a4f28002..f9e2fce21be0 100644
--- a/include/linux/iomap.h
+++ b/include/linux/iomap.h
@@ -518,6 +518,7 @@ void iomap_finish_ioends(struct iomap_ioend *ioend, int error);
 void iomap_ioend_try_merge(struct iomap_ioend *ioend,
 		struct list_head *more_ioends);
 void iomap_sort_ioends(struct list_head *ioend_list);
+int iomap_ioend_integrity_verify(struct iomap_ioend *ioend);
 ssize_t iomap_add_to_ioend(struct iomap_writepage_ctx *wpc, struct folio *folio,
 		loff_t pos, loff_t end_pos, unsigned int dirty_len);
 int iomap_ioend_writeback_submit(struct iomap_writepage_ctx *wpc, int error);
-- 
2.53.0

Re: [PATCH 19/22] iomap,xfs: move integrity verification to the file system

[Darrick J. Wong]

On Thu, Jul 23, 2026 at 04:49:44PM +0200, Christoph Hellwig wrote:
> Integrity support in file systems already requires file system-specific
> completion handling because it must be run in process context.
> 
> Move the actual verification to the file system so that it can better
> handle errors in file system specific ways, and to support lazy bounce
> buffering.
> 
> Signed-off-by: Christoph Hellwig <hch@lst.de>

Sounds good to me.  I gather the idea here is that direct reads go
straight into the user's buffer, and if they're evil and mess with the
buffer contents during the read to make the PI verification fail, we can
then retry the read with a bounce buffer out of spite for userspace?

If so, then
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>

--D

> ---
>  fs/iomap/ioend.c      | 8 ++------
>  fs/xfs/xfs_ioend.c    | 3 +++
>  include/linux/iomap.h | 1 +
>  3 files changed, 6 insertions(+), 6 deletions(-)
> 
> diff --git a/fs/iomap/ioend.c b/fs/iomap/ioend.c
> index 438be92e131b..b1078f7c08c6 100644
> --- a/fs/iomap/ioend.c
> +++ b/fs/iomap/ioend.c
> @@ -308,12 +308,13 @@ ssize_t iomap_add_to_ioend(struct iomap_writepage_ctx *wpc, struct folio *folio,
>  }
>  EXPORT_SYMBOL_GPL(iomap_add_to_ioend);
>  
> -static int iomap_ioend_integrity_verify(struct iomap_ioend *ioend)
> +int iomap_ioend_integrity_verify(struct iomap_ioend *ioend)
>  {
>  	struct bvec_iter data_iter = BVEC_ITER_IOEND(ioend);
>  
>  	return fs_bio_integrity_verify(&ioend->io_bio, &data_iter);
>  }
> +EXPORT_SYMBOL_GPL(iomap_ioend_integrity_verify);
>  
>  static u32 iomap_finish_ioend(struct iomap_ioend *ioend, int error)
>  {
> @@ -330,11 +331,6 @@ static u32 iomap_finish_ioend(struct iomap_ioend *ioend, int error)
>  	if (!atomic_dec_and_test(&ioend->io_remaining))
>  		return 0;
>  
> -	if (!ioend->io_error &&
> -	    bio_integrity(&ioend->io_bio) &&
> -	    bio_op(&ioend->io_bio) == REQ_OP_READ)
> -		ioend->io_error = iomap_ioend_integrity_verify(ioend);
> -
>  	if (ioend->io_flags & IOMAP_IOEND_DIRECT)
>  		return iomap_finish_ioend_direct(ioend);
>  	if (bio_op(&ioend->io_bio) == REQ_OP_READ)
> diff --git a/fs/xfs/xfs_ioend.c b/fs/xfs/xfs_ioend.c
> index 37a3ae8066e9..a095cf217863 100644
> --- a/fs/xfs/xfs_ioend.c
> +++ b/fs/xfs/xfs_ioend.c
> @@ -25,6 +25,9 @@ xfs_end_io_read(
>  	struct iomap_ioend	*ioend = iomap_ioend_from_bio(bio);
>  	int			error = blk_status_to_errno(bio->bi_status);
>  
> +	if (!error && (ioend->io_flags & IOMAP_IOEND_INTEGRITY))
> +		error = iomap_ioend_integrity_verify(ioend);
> +
>  	iomap_finish_ioends(ioend, error);
>  }
>  
> diff --git a/include/linux/iomap.h b/include/linux/iomap.h
> index 1916a4f28002..f9e2fce21be0 100644
> --- a/include/linux/iomap.h
> +++ b/include/linux/iomap.h
> @@ -518,6 +518,7 @@ void iomap_finish_ioends(struct iomap_ioend *ioend, int error);
>  void iomap_ioend_try_merge(struct iomap_ioend *ioend,
>  		struct list_head *more_ioends);
>  void iomap_sort_ioends(struct list_head *ioend_list);
> +int iomap_ioend_integrity_verify(struct iomap_ioend *ioend);
>  ssize_t iomap_add_to_ioend(struct iomap_writepage_ctx *wpc, struct folio *folio,
>  		loff_t pos, loff_t end_pos, unsigned int dirty_len);
>  int iomap_ioend_writeback_submit(struct iomap_writepage_ctx *wpc, int error);
> -- 
> 2.53.0
> 
> 

Re: [PATCH 19/22] iomap,xfs: move integrity verification to the file system

[Christoph Hellwig]

On Thu, Jul 23, 2026 at 02:02:16PM -0700, Darrick J. Wong wrote:
> Sounds good to me.  I gather the idea here is that direct reads go
> straight into the user's buffer, and if they're evil and mess with the
> buffer contents during the read to make the PI verification fail, we can
> then retry the read with a bounce buffer out of spite for userspace?

Yes.

[PATCH 20/22] xfs: add support for lazy direct read bounce buffering

[Christoph Hellwig]

Currently direct I/O reads always bounce buffer the I/O to deal with
the case where userspace is modifying the buffer in-flight while
reading data into it.

This is a very expensive countermeasure for something no sane application
should do, so try to avoid it by reading without a bounce buffer first,
and retrying the read on a checksum failure.  This avoids the cost of
bounce buffering for sanely behave applications.  For the rare case of
an application regularly modifying in-flight buffers, allow forcing the
always bounce buffer behavior through sysfs.  And now that we have that
knob, allow disabling read-side bounce buffering entirely for those who
live fast and dangerous.

Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 fs/xfs/xfs_file.c  |  4 +-
 fs/xfs/xfs_ioend.c | 99 ++++++++++++++++++++++++++++++++++++++++++++--
 fs/xfs/xfs_mount.h |  8 ++++
 fs/xfs/xfs_super.c |  1 +
 fs/xfs/xfs_sysfs.c | 65 ++++++++++++++++++++++++++++++
 fs/xfs/xfs_trace.h |  1 +
 6 files changed, 172 insertions(+), 6 deletions(-)

diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c
index d31a1dddcdc3..04301ab977a3 100644
--- a/fs/xfs/xfs_file.c
+++ b/fs/xfs/xfs_file.c
@@ -264,10 +264,8 @@ xfs_file_dio_read(
 	ret = xfs_ilock_iocb(iocb, XFS_IOLOCK_SHARED);
 	if (ret)
 		return ret;
-	if (mapping_stable_writes(iocb->ki_filp->f_mapping)) {
+	if (mapping_stable_writes(iocb->ki_filp->f_mapping))
 		dio_ops = &xfs_dio_read_bounce_ops;
-		dio_flags |= IOMAP_DIO_BOUNCE;
-	}
 	ret = iomap_dio_rw(iocb, to, &xfs_read_iomap_ops, dio_ops, dio_flags,
 			NULL, 0);
 	xfs_iunlock(ip, XFS_IOLOCK_SHARED);
diff --git a/fs/xfs/xfs_ioend.c b/fs/xfs/xfs_ioend.c
index a095cf217863..94a81fb679ed 100644
--- a/fs/xfs/xfs_ioend.c
+++ b/fs/xfs/xfs_ioend.c
@@ -1,6 +1,6 @@
 // SPDX-License-Identifier: GPL-2.0
 /*
- * Copyright (c) 2016-2025 Christoph Hellwig.
+ * Copyright (c) 2016-2026 Christoph Hellwig.
  * All Rights Reserved.
  */
 #include "xfs_platform.h"
@@ -18,15 +18,97 @@
 #include "xfs_ioend.h"
 #include <linux/bio-integrity.h>
 
+static void
+xfs_end_bio_bounced(
+	struct bio		*bio)
+{
+	iomap_finish_ioends(iomap_ioend_from_bio(bio),
+			blk_status_to_errno(bio->bi_status));
+}
+
+static void
+xfs_dio_bounce_end_io(
+	struct bio		*bio)
+{
+	struct iomap_ioend	*ioend = iomap_ioend_from_bio(bio);
+	int			error = blk_status_to_errno(bio->bi_status);
+	struct bio		*orig_bio = bio->bi_private;
+
+	if ((ioend->io_flags & IOMAP_IOEND_INTEGRITY) && !bio->bi_status)
+		error = iomap_ioend_integrity_verify(ioend);
+	iomap_bounce_read_end_io(ioend, orig_bio, error);
+}
+
+static void
+xfs_bounce_submit_ioend(
+	struct iomap_ioend	*ioend)
+{
+	if (ioend->io_flags & IOMAP_IOEND_INTEGRITY)
+		fs_bio_integrity_alloc(&ioend->io_bio);
+	ioend->io_bio.bi_end_io = xfs_dio_bounce_end_io;
+	bio_set_flag(&ioend->io_bio, BIO_COMPLETE_IN_TASK);
+	submit_bio(&ioend->io_bio);
+}
+
+static void
+xfs_read_bounce_and_resubmit(
+	struct iomap_ioend	*ioend)
+{
+	struct bio		*bio = &ioend->io_bio;
+
+	trace_xfs_bounce_reread(XFS_I(ioend->io_inode), ioend->io_offset,
+			ioend->io_size);
+
+	/*
+	 * Free the bio integrity data for the original bio, as we'll allocate
+	 * ons for each sub-I/O, which could deadlock if we keep the original
+	 * one around.
+	 */
+	if (ioend->io_flags & IOMAP_IOEND_INTEGRITY)
+		fs_bio_integrity_free(bio);
+
+	/*
+	 * Reset the remaining count, iter and bdev as we submit the bio to the
+	 * block layer again and we need a clean slate.  Switch to and end_io
+	 * handler that simply complets the ioend, as all the verification is
+	 * done by the end_I/O handlers for the clone bio(s).
+	 */
+	atomic_set(&bio->__bi_remaining, 1);
+	bio->bi_iter = (struct bvec_iter) {
+		.bi_sector	= ioend->io_sector,
+		.bi_size	= ioend->io_size,
+		.bi_bvec_done	= ioend->io_bvec_offset,
+	};
+	bio->bi_bdev = xfs_inode_buftarg(XFS_I(ioend->io_inode))->bt_bdev;
+	bio->bi_end_io = xfs_end_bio_bounced;
+	iomap_bounce_read(ioend, bdev_logical_block_size(bio->bi_bdev),
+			xfs_bounce_submit_ioend);
+}
+
 static void
 xfs_end_io_read(
 	struct bio		*bio)
 {
 	struct iomap_ioend	*ioend = iomap_ioend_from_bio(bio);
+	struct xfs_inode	*ip = XFS_I(ioend->io_inode);
+	struct xfs_mount	*mp = ip->i_mount;
 	int			error = blk_status_to_errno(bio->bi_status);
 
-	if (!error && (ioend->io_flags & IOMAP_IOEND_INTEGRITY))
+	if (!error && (ioend->io_flags & IOMAP_IOEND_INTEGRITY)) {
 		error = iomap_ioend_integrity_verify(ioend);
+		if ((ioend->io_flags & IOMAP_IOEND_DIRECT) &&
+		    READ_ONCE(mp->m_read_bounce) == XFS_READ_BOUNCE_LAZY) {
+			/*
+			 * We only really need to retry for guard tag errors,
+			 * but right now we can't distinguish them from other
+			 * (i.e, reftag) errors.
+			 */
+			if (error) {
+				xfs_read_bounce_and_resubmit(ioend);
+				return;
+			}
+		}
+	}
 
 	iomap_finish_ioends(ioend, error);
 }
@@ -38,7 +120,18 @@ xfs_ioend_submit_read(
 	loff_t			file_offset,
 	u16			ioend_flags)
 {
-	iomap_init_ioend(inode, bio, file_offset, ioend_flags);
+	struct xfs_inode	*ip = XFS_I(inode);
+	struct xfs_mount	*mp = ip->i_mount;
+	struct iomap_ioend	*ioend;
+
+	ioend = iomap_init_ioend(inode, bio, file_offset, ioend_flags);
+	if ((ioend_flags & IOMAP_IOEND_DIRECT) &&
+	    READ_ONCE(mp->m_read_bounce) == XFS_READ_BOUNCE_ALWAYS) {
+		iomap_bounce_read(ioend, bdev_logical_block_size(bio->bi_bdev),
+				xfs_bounce_submit_ioend);
+		return;
+	}
+
 	if (ioend_flags & IOMAP_IOEND_INTEGRITY)
 		fs_bio_integrity_alloc(bio);
 	bio->bi_end_io = xfs_end_io_read;
diff --git a/fs/xfs/xfs_mount.h b/fs/xfs/xfs_mount.h
index 66a02d1b9ad7..6f8119bd959c 100644
--- a/fs/xfs/xfs_mount.h
+++ b/fs/xfs/xfs_mount.h
@@ -142,6 +142,12 @@ struct xfs_freecounter {
 	uint64_t		res_saved;
 };
 
+enum xfs_read_bounce {
+	XFS_READ_BOUNCE_NEVER,
+	XFS_READ_BOUNCE_ALWAYS,
+	XFS_READ_BOUNCE_LAZY,
+};
+
 /*
  * The struct xfsmount layout is optimised to separate read-mostly variables
  * from variables that are frequently modified. We put the read-mostly variables
@@ -177,6 +183,7 @@ typedef struct xfs_mount {
 	struct workqueue_struct	*m_sync_workqueue;
 	struct workqueue_struct *m_blockgc_wq;
 	struct workqueue_struct *m_inodegc_wq;
+	enum xfs_read_bounce	m_read_bounce;
 
 	int			m_bsize;	/* fs logical block size */
 	uint8_t			m_blkbit_log;	/* blocklog + NBBY */
@@ -291,6 +298,7 @@ typedef struct xfs_mount {
 	struct xfs_zone_info	*m_zone_info;	/* zone allocator information */
 	struct dentry		*m_debugfs;	/* debugfs parent */
 	struct xfs_kobj		m_kobj;
+	struct xfs_kobj		m_csum_kobj;
 	struct xfs_kobj		m_error_kobj;
 	struct xfs_kobj		m_error_meta_kobj;
 	struct xfs_error_cfg	m_error_cfg[XFS_ERR_CLASS_MAX][XFS_ERR_ERRNO_MAX];
diff --git a/fs/xfs/xfs_super.c b/fs/xfs/xfs_super.c
index 8531d526fc44..7c8424185e06 100644
--- a/fs/xfs/xfs_super.c
+++ b/fs/xfs/xfs_super.c
@@ -2270,6 +2270,7 @@ xfs_init_fs_context(
 	mp->m_logbufs = -1;
 	mp->m_logbsize = -1;
 	mp->m_allocsize_log = 16; /* 64k */
+	mp->m_read_bounce = XFS_READ_BOUNCE_LAZY;
 
 	xfs_hooks_init(&mp->m_dir_update_hooks);
 
diff --git a/fs/xfs/xfs_sysfs.c b/fs/xfs/xfs_sysfs.c
index b62712187324..1e44bb8b30e8 100644
--- a/fs/xfs/xfs_sysfs.c
+++ b/fs/xfs/xfs_sysfs.c
@@ -392,6 +392,63 @@ const struct kobj_type xfs_stats_ktype = {
 	.default_groups = xfs_stats_groups,
 };
 
+static inline struct xfs_mount *csum_to_mp(struct kobject *kobj)
+{
+	return container_of(to_kobj(kobj), struct xfs_mount, m_csum_kobj);
+}
+
+static ssize_t
+read_bounce_show(
+	struct kobject		*kobj,
+	char			*buf)
+{
+	struct xfs_mount	*mp = csum_to_mp(kobj);
+
+	switch (READ_ONCE(mp->m_read_bounce)) {
+	case XFS_READ_BOUNCE_NEVER:
+		return sysfs_emit(buf, "never\n");
+	case XFS_READ_BOUNCE_ALWAYS:
+		return sysfs_emit(buf, "always\n");
+	case XFS_READ_BOUNCE_LAZY:
+		return sysfs_emit(buf, "lazy\n");
+	default:
+		return sysfs_emit(buf, "invalid\n");
+	}
+}
+
+static ssize_t
+read_bounce_store(
+	struct kobject	*kobj,
+	const char	*buf,
+	size_t		count)
+{
+	struct xfs_mount	*mp = csum_to_mp(kobj);
+
+	if (!strcmp(buf, "never"))
+		WRITE_ONCE(mp->m_read_bounce, XFS_READ_BOUNCE_NEVER);
+	else if (!strcmp(buf, "always"))
+		WRITE_ONCE(mp->m_read_bounce, XFS_READ_BOUNCE_ALWAYS);
+	else if (!strcmp(buf, "lazy"))
+		WRITE_ONCE(mp->m_read_bounce, XFS_READ_BOUNCE_LAZY);
+	else
+		return -EINVAL;
+
+	return count;
+}
+XFS_SYSFS_ATTR_RW(read_bounce);
+
+static struct attribute *xfs_csum_attrs[] = {
+	ATTR_LIST(read_bounce),
+	NULL,
+};
+ATTRIBUTE_GROUPS(xfs_csum);
+
+static const struct kobj_type xfs_csum_ktype = {
+	.release = xfs_sysfs_release,
+	.sysfs_ops = &xfs_sysfs_ops,
+	.default_groups = xfs_csum_groups,
+};
+
 /* xlog */
 
 static inline struct xlog *
@@ -837,6 +894,12 @@ xfs_mount_sysfs_init(
 	if (error)
 		goto out_remove_error_dir;
 
+	/* .../xfs/<dev>/csum/ */
+	error = xfs_sysfs_init(&mp->m_csum_kobj, &xfs_csum_ktype,
+			       &mp->m_kobj, "csum");
+	if (error)
+		goto out_remove_error_dir;
+
 	return 0;
 
 out_remove_error_dir:
@@ -855,6 +918,8 @@ xfs_mount_sysfs_del(
 	struct xfs_error_cfg	*cfg;
 	int			i, j;
 
+	xfs_sysfs_del(&mp->m_csum_kobj);
+
 	for (i = 0; i < XFS_ERR_CLASS_MAX; i++) {
 		for (j = 0; j < XFS_ERR_ERRNO_MAX; j++) {
 			cfg = &mp->m_error_cfg[i][j];
diff --git a/fs/xfs/xfs_trace.h b/fs/xfs/xfs_trace.h
index aeb89ac53bf1..af44551fd305 100644
--- a/fs/xfs/xfs_trace.h
+++ b/fs/xfs/xfs_trace.h
@@ -1896,6 +1896,7 @@ DEFINE_SIMPLE_IO_EVENT(xfs_zero_eof);
 DEFINE_SIMPLE_IO_EVENT(xfs_end_io_direct_write);
 DEFINE_SIMPLE_IO_EVENT(xfs_file_splice_read);
 DEFINE_SIMPLE_IO_EVENT(xfs_zoned_map_blocks);
+DEFINE_SIMPLE_IO_EVENT(xfs_bounce_reread);
 
 DECLARE_EVENT_CLASS(xfs_itrunc_class,
 	TP_PROTO(struct xfs_inode *ip, xfs_fsize_t new_size),
-- 
2.53.0

Re: [PATCH 20/22] xfs: add support for lazy direct read bounce buffering

[Darrick J. Wong]

On Thu, Jul 23, 2026 at 04:49:45PM +0200, Christoph Hellwig wrote:
> Currently direct I/O reads always bounce buffer the I/O to deal with
> the case where userspace is modifying the buffer in-flight while
> reading data into it.
> 
> This is a very expensive countermeasure for something no sane application
> should do, so try to avoid it by reading without a bounce buffer first,
> and retrying the read on a checksum failure.  This avoids the cost of
> bounce buffering for sanely behave applications.  For the rare case of
> an application regularly modifying in-flight buffers, allow forcing the
> always bounce buffer behavior through sysfs.  And now that we have that
> knob, allow disabling read-side bounce buffering entirely for those who
> live fast and dangerous.
> 
> Signed-off-by: Christoph Hellwig <hch@lst.de>

Mostly looks fine, with only a couple of questions...

> ---
>  fs/xfs/xfs_file.c  |  4 +-
>  fs/xfs/xfs_ioend.c | 99 ++++++++++++++++++++++++++++++++++++++++++++--
>  fs/xfs/xfs_mount.h |  8 ++++
>  fs/xfs/xfs_super.c |  1 +
>  fs/xfs/xfs_sysfs.c | 65 ++++++++++++++++++++++++++++++
>  fs/xfs/xfs_trace.h |  1 +
>  6 files changed, 172 insertions(+), 6 deletions(-)
> 
> diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c
> index d31a1dddcdc3..04301ab977a3 100644
> --- a/fs/xfs/xfs_file.c
> +++ b/fs/xfs/xfs_file.c
> @@ -264,10 +264,8 @@ xfs_file_dio_read(
>  	ret = xfs_ilock_iocb(iocb, XFS_IOLOCK_SHARED);
>  	if (ret)
>  		return ret;
> -	if (mapping_stable_writes(iocb->ki_filp->f_mapping)) {
> +	if (mapping_stable_writes(iocb->ki_filp->f_mapping))
>  		dio_ops = &xfs_dio_read_bounce_ops;
> -		dio_flags |= IOMAP_DIO_BOUNCE;
> -	}
>  	ret = iomap_dio_rw(iocb, to, &xfs_read_iomap_ops, dio_ops, dio_flags,
>  			NULL, 0);
>  	xfs_iunlock(ip, XFS_IOLOCK_SHARED);
> diff --git a/fs/xfs/xfs_ioend.c b/fs/xfs/xfs_ioend.c
> index a095cf217863..94a81fb679ed 100644
> --- a/fs/xfs/xfs_ioend.c
> +++ b/fs/xfs/xfs_ioend.c
> @@ -1,6 +1,6 @@
>  // SPDX-License-Identifier: GPL-2.0
>  /*
> - * Copyright (c) 2016-2025 Christoph Hellwig.
> + * Copyright (c) 2016-2026 Christoph Hellwig.
>   * All Rights Reserved.
>   */
>  #include "xfs_platform.h"
> @@ -18,15 +18,97 @@
>  #include "xfs_ioend.h"
>  #include <linux/bio-integrity.h>
>  
> +static void
> +xfs_end_bio_bounced(
> +	struct bio		*bio)
> +{
> +	iomap_finish_ioends(iomap_ioend_from_bio(bio),
> +			blk_status_to_errno(bio->bi_status));
> +}
> +
> +static void
> +xfs_dio_bounce_end_io(
> +	struct bio		*bio)
> +{
> +	struct iomap_ioend	*ioend = iomap_ioend_from_bio(bio);
> +	int			error = blk_status_to_errno(bio->bi_status);
> +	struct bio		*orig_bio = bio->bi_private;
> +
> +	if ((ioend->io_flags & IOMAP_IOEND_INTEGRITY) && !bio->bi_status)
> +		error = iomap_ioend_integrity_verify(ioend);
> +	iomap_bounce_read_end_io(ioend, orig_bio, error);
> +}
> +
> +static void
> +xfs_bounce_submit_ioend(
> +	struct iomap_ioend	*ioend)
> +{
> +	if (ioend->io_flags & IOMAP_IOEND_INTEGRITY)
> +		fs_bio_integrity_alloc(&ioend->io_bio);
> +	ioend->io_bio.bi_end_io = xfs_dio_bounce_end_io;
> +	bio_set_flag(&ioend->io_bio, BIO_COMPLETE_IN_TASK);
> +	submit_bio(&ioend->io_bio);
> +}
> +
> +static void
> +xfs_read_bounce_and_resubmit(
> +	struct iomap_ioend	*ioend)
> +{
> +	struct bio		*bio = &ioend->io_bio;
> +
> +	trace_xfs_bounce_reread(XFS_I(ioend->io_inode), ioend->io_offset,
> +			ioend->io_size);
> +
> +	/*
> +	 * Free the bio integrity data for the original bio, as we'll allocate
> +	 * ons for each sub-I/O, which could deadlock if we keep the original
> +	 * one around.
> +	 */
> +	if (ioend->io_flags & IOMAP_IOEND_INTEGRITY)
> +		fs_bio_integrity_free(bio);
> +
> +	/*
> +	 * Reset the remaining count, iter and bdev as we submit the bio to the
> +	 * block layer again and we need a clean slate.  Switch to and end_io
> +	 * handler that simply complets the ioend, as all the verification is
> +	 * done by the end_I/O handlers for the clone bio(s).
> +	 */
> +	atomic_set(&bio->__bi_remaining, 1);
> +	bio->bi_iter = (struct bvec_iter) {
> +		.bi_sector	= ioend->io_sector,
> +		.bi_size	= ioend->io_size,
> +		.bi_bvec_done	= ioend->io_bvec_offset,
> +	};
> +	bio->bi_bdev = xfs_inode_buftarg(XFS_I(ioend->io_inode))->bt_bdev;
> +	bio->bi_end_io = xfs_end_bio_bounced;
> +	iomap_bounce_read(ioend, bdev_logical_block_size(bio->bi_bdev),
> +			xfs_bounce_submit_ioend);
> +}
> +
>  static void
>  xfs_end_io_read(
>  	struct bio		*bio)
>  {
>  	struct iomap_ioend	*ioend = iomap_ioend_from_bio(bio);
> +	struct xfs_inode	*ip = XFS_I(ioend->io_inode);
> +	struct xfs_mount	*mp = ip->i_mount;
>  	int			error = blk_status_to_errno(bio->bi_status);
>  
> -	if (!error && (ioend->io_flags & IOMAP_IOEND_INTEGRITY))
> +	if (!error && (ioend->io_flags & IOMAP_IOEND_INTEGRITY)) {
>  		error = iomap_ioend_integrity_verify(ioend);
> +		if ((ioend->io_flags & IOMAP_IOEND_DIRECT) &&
> +		    READ_ONCE(mp->m_read_bounce) == XFS_READ_BOUNCE_LAZY) {
> +			/*
> +			 * We only really need to retry for guard tag errors,
> +			 * but right now we can't distinguish them from other
> +			 * (i.e, reftag) errors.
> +			 */
> +			if (error) {
> +				xfs_read_bounce_and_resubmit(ioend);

Ahah, yes we are being mean and making userspace wait for a slow bounce
buffer workaround if they mess with us.

> +				return;
> +			}
> +		}
> +	}
>  
>  	iomap_finish_ioends(ioend, error);
>  }
> @@ -38,7 +120,18 @@ xfs_ioend_submit_read(
>  	loff_t			file_offset,
>  	u16			ioend_flags)
>  {
> -	iomap_init_ioend(inode, bio, file_offset, ioend_flags);
> +	struct xfs_inode	*ip = XFS_I(inode);
> +	struct xfs_mount	*mp = ip->i_mount;
> +	struct iomap_ioend	*ioend;
> +
> +	ioend = iomap_init_ioend(inode, bio, file_offset, ioend_flags);
> +	if ((ioend_flags & IOMAP_IOEND_DIRECT) &&
> +	    READ_ONCE(mp->m_read_bounce) == XFS_READ_BOUNCE_ALWAYS) {
> +		iomap_bounce_read(ioend, bdev_logical_block_size(bio->bi_bdev),
> +				xfs_bounce_submit_ioend);
> +		return;
> +	}
> +
>  	if (ioend_flags & IOMAP_IOEND_INTEGRITY)
>  		fs_bio_integrity_alloc(bio);
>  	bio->bi_end_io = xfs_end_io_read;
> diff --git a/fs/xfs/xfs_mount.h b/fs/xfs/xfs_mount.h
> index 66a02d1b9ad7..6f8119bd959c 100644
> --- a/fs/xfs/xfs_mount.h
> +++ b/fs/xfs/xfs_mount.h
> @@ -142,6 +142,12 @@ struct xfs_freecounter {
>  	uint64_t		res_saved;
>  };
>  
> +enum xfs_read_bounce {
> +	XFS_READ_BOUNCE_NEVER,
> +	XFS_READ_BOUNCE_ALWAYS,
> +	XFS_READ_BOUNCE_LAZY,
> +};
> +
>  /*
>   * The struct xfsmount layout is optimised to separate read-mostly variables
>   * from variables that are frequently modified. We put the read-mostly variables
> @@ -177,6 +183,7 @@ typedef struct xfs_mount {
>  	struct workqueue_struct	*m_sync_workqueue;
>  	struct workqueue_struct *m_blockgc_wq;
>  	struct workqueue_struct *m_inodegc_wq;
> +	enum xfs_read_bounce	m_read_bounce;
>  
>  	int			m_bsize;	/* fs logical block size */
>  	uint8_t			m_blkbit_log;	/* blocklog + NBBY */
> @@ -291,6 +298,7 @@ typedef struct xfs_mount {
>  	struct xfs_zone_info	*m_zone_info;	/* zone allocator information */
>  	struct dentry		*m_debugfs;	/* debugfs parent */
>  	struct xfs_kobj		m_kobj;
> +	struct xfs_kobj		m_csum_kobj;
>  	struct xfs_kobj		m_error_kobj;
>  	struct xfs_kobj		m_error_meta_kobj;
>  	struct xfs_error_cfg	m_error_cfg[XFS_ERR_CLASS_MAX][XFS_ERR_ERRNO_MAX];
> diff --git a/fs/xfs/xfs_super.c b/fs/xfs/xfs_super.c
> index 8531d526fc44..7c8424185e06 100644
> --- a/fs/xfs/xfs_super.c
> +++ b/fs/xfs/xfs_super.c
> @@ -2270,6 +2270,7 @@ xfs_init_fs_context(
>  	mp->m_logbufs = -1;
>  	mp->m_logbsize = -1;
>  	mp->m_allocsize_log = 16; /* 64k */
> +	mp->m_read_bounce = XFS_READ_BOUNCE_LAZY;
>  
>  	xfs_hooks_init(&mp->m_dir_update_hooks);
>  
> diff --git a/fs/xfs/xfs_sysfs.c b/fs/xfs/xfs_sysfs.c
> index b62712187324..1e44bb8b30e8 100644
> --- a/fs/xfs/xfs_sysfs.c
> +++ b/fs/xfs/xfs_sysfs.c
> @@ -392,6 +392,63 @@ const struct kobj_type xfs_stats_ktype = {
>  	.default_groups = xfs_stats_groups,
>  };
>  
> +static inline struct xfs_mount *csum_to_mp(struct kobject *kobj)
> +{
> +	return container_of(to_kobj(kobj), struct xfs_mount, m_csum_kobj);
> +}
> +
> +static ssize_t
> +read_bounce_show(
> +	struct kobject		*kobj,
> +	char			*buf)
> +{
> +	struct xfs_mount	*mp = csum_to_mp(kobj);
> +
> +	switch (READ_ONCE(mp->m_read_bounce)) {
> +	case XFS_READ_BOUNCE_NEVER:
> +		return sysfs_emit(buf, "never\n");
> +	case XFS_READ_BOUNCE_ALWAYS:
> +		return sysfs_emit(buf, "always\n");
> +	case XFS_READ_BOUNCE_LAZY:
> +		return sysfs_emit(buf, "lazy\n");
> +	default:
> +		return sysfs_emit(buf, "invalid\n");
> +	}
> +}
> +
> +static ssize_t
> +read_bounce_store(
> +	struct kobject	*kobj,
> +	const char	*buf,
> +	size_t		count)
> +{
> +	struct xfs_mount	*mp = csum_to_mp(kobj);
> +
> +	if (!strcmp(buf, "never"))
> +		WRITE_ONCE(mp->m_read_bounce, XFS_READ_BOUNCE_NEVER);
> +	else if (!strcmp(buf, "always"))
> +		WRITE_ONCE(mp->m_read_bounce, XFS_READ_BOUNCE_ALWAYS);
> +	else if (!strcmp(buf, "lazy"))
> +		WRITE_ONCE(mp->m_read_bounce, XFS_READ_BOUNCE_LAZY);
> +	else
> +		return -EINVAL;
> +
> +	return count;
> +}
> +XFS_SYSFS_ATTR_RW(read_bounce);
> +
> +static struct attribute *xfs_csum_attrs[] = {
> +	ATTR_LIST(read_bounce),
> +	NULL,
> +};
> +ATTRIBUTE_GROUPS(xfs_csum);
> +
> +static const struct kobj_type xfs_csum_ktype = {
> +	.release = xfs_sysfs_release,
> +	.sysfs_ops = &xfs_sysfs_ops,
> +	.default_groups = xfs_csum_groups,
> +};
> +
>  /* xlog */
>  
>  static inline struct xlog *
> @@ -837,6 +894,12 @@ xfs_mount_sysfs_init(
>  	if (error)
>  		goto out_remove_error_dir;
>  
> +	/* .../xfs/<dev>/csum/ */
> +	error = xfs_sysfs_init(&mp->m_csum_kobj, &xfs_csum_ktype,
> +			       &mp->m_kobj, "csum");
> +	if (error)
> +		goto out_remove_error_dir;

/me wonders if this is a debugging knob and therefore should go in
debugfs?  Or is there a solid usecase for normal sysadmins to be able to
control this?

--D

> +
>  	return 0;
>  
>  out_remove_error_dir:
> @@ -855,6 +918,8 @@ xfs_mount_sysfs_del(
>  	struct xfs_error_cfg	*cfg;
>  	int			i, j;
>  
> +	xfs_sysfs_del(&mp->m_csum_kobj);
> +
>  	for (i = 0; i < XFS_ERR_CLASS_MAX; i++) {
>  		for (j = 0; j < XFS_ERR_ERRNO_MAX; j++) {
>  			cfg = &mp->m_error_cfg[i][j];
> diff --git a/fs/xfs/xfs_trace.h b/fs/xfs/xfs_trace.h
> index aeb89ac53bf1..af44551fd305 100644
> --- a/fs/xfs/xfs_trace.h
> +++ b/fs/xfs/xfs_trace.h
> @@ -1896,6 +1896,7 @@ DEFINE_SIMPLE_IO_EVENT(xfs_zero_eof);
>  DEFINE_SIMPLE_IO_EVENT(xfs_end_io_direct_write);
>  DEFINE_SIMPLE_IO_EVENT(xfs_file_splice_read);
>  DEFINE_SIMPLE_IO_EVENT(xfs_zoned_map_blocks);
> +DEFINE_SIMPLE_IO_EVENT(xfs_bounce_reread);
>  
>  DECLARE_EVENT_CLASS(xfs_itrunc_class,
>  	TP_PROTO(struct xfs_inode *ip, xfs_fsize_t new_size),
> -- 
> 2.53.0
> 
> 

Re: [PATCH 20/22] xfs: add support for lazy direct read bounce buffering

[Christoph Hellwig]

On Thu, Jul 23, 2026 at 02:05:50PM -0700, Darrick J. Wong wrote:
> > +			 * We only really need to retry for guard tag errors,
> > +			 * but right now we can't distinguish them from other
> > +			 * (i.e, reftag) errors.
> > +			 */
> > +			if (error) {
> > +				xfs_read_bounce_and_resubmit(ioend);
> 
> Ahah, yes we are being mean and making userspace wait for a slow bounce
> buffer workaround if they mess with us.

Yes.

> > +	/* .../xfs/<dev>/csum/ */
> > +	error = xfs_sysfs_init(&mp->m_csum_kobj, &xfs_csum_ktype,
> > +			       &mp->m_kobj, "csum");
> > +	if (error)
> > +		goto out_remove_error_dir;
> 
> /me wonders if this is a debugging knob and therefore should go in
> debugfs?  Or is there a solid usecase for normal sysadmins to be able to
> control this?

I consider it an insurance again mispredicting user workloads.  debugfs
also is really painful in general, and especially for trivial files
likes this where it requires tons of boilerplate for no reason.

Re: [PATCH 20/22] xfs: add support for lazy direct read bounce buffering

[Christoph Hellwig]

On Fri, Jul 24, 2026 at 08:24:43AM +0200, Christoph Hellwig wrote:
> > > +	/* .../xfs/<dev>/csum/ */
> > > +	error = xfs_sysfs_init(&mp->m_csum_kobj, &xfs_csum_ktype,
> > > +			       &mp->m_kobj, "csum");
> > > +	if (error)
> > > +		goto out_remove_error_dir;
> > 
> > /me wonders if this is a debugging knob and therefore should go in
> > debugfs?  Or is there a solid usecase for normal sysadmins to be able to
> > control this?
> 
> I consider it an insurance again mispredicting user workloads.  debugfs
> also is really painful in general, and especially for trivial files
> likes this where it requires tons of boilerplate for no reason.

Also: I'm not particular happy about the placement of this file
in sysfs.  Do we want a misc group, or rw?  The current one feels
a bit too specific, which means we'd keep on growing way too many
groups in the future.

[PATCH 21/22] xfs: add error injection for lazy bounce buffering

[Christoph Hellwig]

Add an error injection know to exercise the lazy bounce buffering
code path, i.e. to inject direct I/O re-read using the bounce buffer.

Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 fs/xfs/libxfs/xfs_errortag.h | 6 ++++--
 fs/xfs/xfs_ioend.c           | 5 ++++-
 2 files changed, 8 insertions(+), 3 deletions(-)

diff --git a/fs/xfs/libxfs/xfs_errortag.h b/fs/xfs/libxfs/xfs_errortag.h
index 6de207fed2d8..2dc441da0333 100644
--- a/fs/xfs/libxfs/xfs_errortag.h
+++ b/fs/xfs/libxfs/xfs_errortag.h
@@ -75,7 +75,8 @@
 #define XFS_ERRTAG_METAFILE_RESV_CRITICAL		45
 #define XFS_ERRTAG_FORCE_ZERO_RANGE			46
 #define XFS_ERRTAG_ZONE_RESET				47
-#define XFS_ERRTAG_MAX					48
+#define XFS_ERRTAG_BOUNCE_REREAD			48
+#define XFS_ERRTAG_MAX					49
 
 /*
  * Random factors for above tags, 1 means always, 2 means 1/2 time, etc.
@@ -137,7 +138,8 @@ XFS_ERRTAG(WRITE_DELAY_MS,	write_delay_ms,		3000) \
 XFS_ERRTAG(EXCHMAPS_FINISH_ONE,	exchmaps_finish_one,	1) \
 XFS_ERRTAG(METAFILE_RESV_CRITICAL, metafile_resv_crit,	4) \
 XFS_ERRTAG(FORCE_ZERO_RANGE,	force_zero_range,	4) \
-XFS_ERRTAG(ZONE_RESET,		zone_reset,		1)
+XFS_ERRTAG(ZONE_RESET,		zone_reset,		1)  \
+XFS_ERRTAG(BOUNCE_REREAD,	bounce_reread,		XFS_RANDOM_DEFAULT)
 #endif /* XFS_ERRTAG */
 
 #endif /* __XFS_ERRORTAG_H_ */
diff --git a/fs/xfs/xfs_ioend.c b/fs/xfs/xfs_ioend.c
index 94a81fb679ed..641f0d881b07 100644
--- a/fs/xfs/xfs_ioend.c
+++ b/fs/xfs/xfs_ioend.c
@@ -16,6 +16,8 @@
 #include "xfs_reflink.h"
 #include "xfs_zone_alloc.h"
 #include "xfs_ioend.h"
+#include "xfs_error.h"
+#include "xfs_errortag.h"
 #include <linux/bio-integrity.h>
 
 static void
@@ -103,7 +105,8 @@ xfs_end_io_read(
 			 * but right now we can't distinguish them from other
 			 * (i.e, reftag) errors.
 			 */
-			if (error) {
+			if (error ||
+			    XFS_TEST_ERROR(mp, XFS_ERRTAG_BOUNCE_REREAD)) {
 				xfs_read_bounce_and_resubmit(ioend);
 				return;
 			}
-- 
2.53.0

Re: [PATCH 21/22] xfs: add error injection for lazy bounce buffering

[Darrick J. Wong]

On Thu, Jul 23, 2026 at 04:49:46PM +0200, Christoph Hellwig wrote:
> Add an error injection know to exercise the lazy bounce buffering
> code path, i.e. to inject direct I/O re-read using the bounce buffer.
> 
> Signed-off-by: Christoph Hellwig <hch@lst.de>

LGTM
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>

--D

> ---
>  fs/xfs/libxfs/xfs_errortag.h | 6 ++++--
>  fs/xfs/xfs_ioend.c           | 5 ++++-
>  2 files changed, 8 insertions(+), 3 deletions(-)
> 
> diff --git a/fs/xfs/libxfs/xfs_errortag.h b/fs/xfs/libxfs/xfs_errortag.h
> index 6de207fed2d8..2dc441da0333 100644
> --- a/fs/xfs/libxfs/xfs_errortag.h
> +++ b/fs/xfs/libxfs/xfs_errortag.h
> @@ -75,7 +75,8 @@
>  #define XFS_ERRTAG_METAFILE_RESV_CRITICAL		45
>  #define XFS_ERRTAG_FORCE_ZERO_RANGE			46
>  #define XFS_ERRTAG_ZONE_RESET				47
> -#define XFS_ERRTAG_MAX					48
> +#define XFS_ERRTAG_BOUNCE_REREAD			48
> +#define XFS_ERRTAG_MAX					49
>  
>  /*
>   * Random factors for above tags, 1 means always, 2 means 1/2 time, etc.
> @@ -137,7 +138,8 @@ XFS_ERRTAG(WRITE_DELAY_MS,	write_delay_ms,		3000) \
>  XFS_ERRTAG(EXCHMAPS_FINISH_ONE,	exchmaps_finish_one,	1) \
>  XFS_ERRTAG(METAFILE_RESV_CRITICAL, metafile_resv_crit,	4) \
>  XFS_ERRTAG(FORCE_ZERO_RANGE,	force_zero_range,	4) \
> -XFS_ERRTAG(ZONE_RESET,		zone_reset,		1)
> +XFS_ERRTAG(ZONE_RESET,		zone_reset,		1)  \
> +XFS_ERRTAG(BOUNCE_REREAD,	bounce_reread,		XFS_RANDOM_DEFAULT)
>  #endif /* XFS_ERRTAG */
>  
>  #endif /* __XFS_ERRORTAG_H_ */
> diff --git a/fs/xfs/xfs_ioend.c b/fs/xfs/xfs_ioend.c
> index 94a81fb679ed..641f0d881b07 100644
> --- a/fs/xfs/xfs_ioend.c
> +++ b/fs/xfs/xfs_ioend.c
> @@ -16,6 +16,8 @@
>  #include "xfs_reflink.h"
>  #include "xfs_zone_alloc.h"
>  #include "xfs_ioend.h"
> +#include "xfs_error.h"
> +#include "xfs_errortag.h"
>  #include <linux/bio-integrity.h>
>  
>  static void
> @@ -103,7 +105,8 @@ xfs_end_io_read(
>  			 * but right now we can't distinguish them from other
>  			 * (i.e, reftag) errors.
>  			 */
> -			if (error) {
> +			if (error ||
> +			    XFS_TEST_ERROR(mp, XFS_ERRTAG_BOUNCE_REREAD)) {
>  				xfs_read_bounce_and_resubmit(ioend);
>  				return;
>  			}
> -- 
> 2.53.0
> 
> 

[PATCH 22/22] xfs: log a message at mount time when using integrity protection

[Christoph Hellwig]

Log a message in the kernel when using T10 protection information.

Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 fs/xfs/xfs_buf.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/fs/xfs/xfs_buf.c b/fs/xfs/xfs_buf.c
index e1465e950acc..744e4a94a879 100644
--- a/fs/xfs/xfs_buf.c
+++ b/fs/xfs/xfs_buf.c
@@ -5,6 +5,7 @@
  */
 #include "xfs_platform.h"
 #include <linux/backing-dev.h>
+#include <linux/blk-integrity.h>
 #include <linux/dax.h>
 
 #include "xfs_shared.h"
@@ -1674,6 +1675,7 @@ xfs_configure_buftarg(
 	struct xfs_mount	*mp = btp->bt_mount;
 
 	if (btp->bt_bdev) {
+		struct blk_integrity *bi = bdev_get_integrity(btp->bt_bdev);
 		int		error;
 
 		error = bdev_validate_blocksize(btp->bt_bdev, sectorsize);
@@ -1686,6 +1688,15 @@ xfs_configure_buftarg(
 
 		if (bdev_can_atomic_write(btp->bt_bdev))
 			xfs_configure_buftarg_atomic_writes(btp);
+
+		if (!bi)
+			;
+		else if (btp->bt_bdev == btp->bt_mount->m_super->s_bdev)
+			xfs_info(mp, "using %s integrity profile",
+				blk_integrity_profile_name(bi));
+		else
+			xfs_info(mp, "using %s integrity profile for %pg",
+				blk_integrity_profile_name(bi), btp->bt_bdev);
 	}
 
 	btp->bt_meta_sectorsize = sectorsize;
-- 
2.53.0

Re: [PATCH 22/22] xfs: log a message at mount time when using integrity protection

[Darrick J. Wong]

On Thu, Jul 23, 2026 at 04:49:47PM +0200, Christoph Hellwig wrote:
> Log a message in the kernel when using T10 protection information.
> 
> Signed-off-by: Christoph Hellwig <hch@lst.de>

Do these new log messages show up for any PI-capable device?  So if I
put the log on a PI blockdev, it'll say "Using arglebargle integrity
profile for /dev/sda" ?

The code looks ok to me so

Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>

--D

> ---
>  fs/xfs/xfs_buf.c | 11 +++++++++++
>  1 file changed, 11 insertions(+)
> 
> diff --git a/fs/xfs/xfs_buf.c b/fs/xfs/xfs_buf.c
> index e1465e950acc..744e4a94a879 100644
> --- a/fs/xfs/xfs_buf.c
> +++ b/fs/xfs/xfs_buf.c
> @@ -5,6 +5,7 @@
>   */
>  #include "xfs_platform.h"
>  #include <linux/backing-dev.h>
> +#include <linux/blk-integrity.h>
>  #include <linux/dax.h>
>  
>  #include "xfs_shared.h"
> @@ -1674,6 +1675,7 @@ xfs_configure_buftarg(
>  	struct xfs_mount	*mp = btp->bt_mount;
>  
>  	if (btp->bt_bdev) {
> +		struct blk_integrity *bi = bdev_get_integrity(btp->bt_bdev);
>  		int		error;
>  
>  		error = bdev_validate_blocksize(btp->bt_bdev, sectorsize);
> @@ -1686,6 +1688,15 @@ xfs_configure_buftarg(
>  
>  		if (bdev_can_atomic_write(btp->bt_bdev))
>  			xfs_configure_buftarg_atomic_writes(btp);
> +
> +		if (!bi)
> +			;
> +		else if (btp->bt_bdev == btp->bt_mount->m_super->s_bdev)
> +			xfs_info(mp, "using %s integrity profile",
> +				blk_integrity_profile_name(bi));
> +		else
> +			xfs_info(mp, "using %s integrity profile for %pg",
> +				blk_integrity_profile_name(bi), btp->bt_bdev);
>  	}
>  
>  	btp->bt_meta_sectorsize = sectorsize;
> -- 
> 2.53.0
> 
> 

Re: [PATCH 22/22] xfs: log a message at mount time when using integrity protection

[Christoph Hellwig]

On Thu, Jul 23, 2026 at 02:07:58PM -0700, Darrick J. Wong wrote:
> On Thu, Jul 23, 2026 at 04:49:47PM +0200, Christoph Hellwig wrote:
> > Log a message in the kernel when using T10 protection information.
> > 
> > Signed-off-by: Christoph Hellwig <hch@lst.de>
> 
> Do these new log messages show up for any PI-capable device?  So if I
> put the log on a PI blockdev, it'll say "Using arglebargle integrity
> profile for /dev/sda" ?

Yes.