public inbox for linux-fsdevel@vger.kernel.org
 help / color / mirror / Atom feed
* support file system generated / verified integrity information
@ 2026-01-21  6:43 Christoph Hellwig
  2026-01-21  6:43 ` [PATCH 01/15] block: factor out a bio_integrity_action helper Christoph Hellwig
                   ` (15 more replies)
  0 siblings, 16 replies; 70+ messages in thread
From: Christoph Hellwig @ 2026-01-21  6:43 UTC (permalink / raw)
  To: Jens Axboe, Christian Brauner
  Cc: Darrick J. Wong, Carlos Maiolino, Martin K. Petersen, Anuj Gupta,
	Kanchan Joshi, linux-block, nvdimm, linux-fsdevel, linux-xfs

Hi all,

this series adds support to generate and verify integrity information
(aka T10 PI) in the file system, instead of the automatic below the
covers support that is currently used.

There two reasons for this:

  a) to increase the protection enveloped.  Right now this is just a
     minor step from the bottom of the block layer to the file system,
     but it is required to support io_uring integrity data passthrough in
     the file system similar to the currently existing support for block
     devices, which will follow next.  It also allows the file system to
     directly see the integrity error and act upon in, e.g. when using
     RAID either integrated (as in btrfs) or by supporting reading
     redundant copies through the block layer.
  b) to make the PI processing more efficient.  This is primarily a
     concern for reads, where the block layer auto PI has to schedule a
     work item for each bio, and the file system them has to do it again
     for bounce buffering.  Additionally the current iomap post-I/O
     workqueue handling is a lot more efficient by supporting merging and
     avoiding workqueue scheduling storms.

The implementation is based on refactoring the existing block layer PI
code to be reusable for this use case, and then adding relatively small
wrappers for the file system use case.  These are then used in iomap
to implement the semantics, and wired up in XFS with a small amount of
glue code.

Compared to the baseline (iomap-bounce branch), this does not change
performance for writes, but increases read performance up to 15% for 4k
I/O, with the benefit decreasing with larger I/O sizes as even the
baseline maxes out the device quickly.  But even for larger I/O sizes,
the number of context switches decreases and cpu usage sees a minor
decrease.

The patches both require the recent integrity changes in the block tree
and the direct I/O bounce buffering series I previously sent out, so
you're probably better off just using the git tree instead of applying
the patches.

Git tree:

    git://git.infradead.org/users/hch/misc.git iomap-pi

Gitweb:

    https://git.infradead.org/?p=users/hch/misc.git;a=shortlog;h=refs/heads/iomap-pi

Diffstat:
 block/Makefile                |    2 
 block/bio-integrity-auto.c    |   80 +++---------------------
 block/bio-integrity-fs.c      |   81 +++++++++++++++++++++++++
 block/bio-integrity.c         |   64 +++++++++++++++++++
 block/bio.c                   |   17 +++--
 block/blk-mq.c                |    6 +
 block/blk-settings.c          |   13 ----
 block/blk.h                   |    6 +
 block/t10-pi.c                |   12 +--
 drivers/nvdimm/btt.c          |    6 +
 fs/fuse/file.c                |    5 -
 fs/iomap/bio.c                |  135 ++++++++++++++++++++++++++++--------------
 fs/iomap/buffered-io.c        |    8 +-
 fs/iomap/direct-io.c          |   15 ++++
 fs/iomap/internal.h           |   14 ++++
 fs/iomap/ioend.c              |   28 +++++++-
 fs/xfs/xfs_aops.c             |   47 +++++++++++++-
 fs/xfs/xfs_iomap.c            |    9 +-
 include/linux/bio-integrity.h |   12 ++-
 include/linux/bio.h           |    2 
 include/linux/blk-integrity.h |   21 ++++--
 include/linux/blkdev.h        |   34 ++++++++--
 include/linux/iomap.h         |   20 +++++-
 23 files changed, 456 insertions(+), 181 deletions(-)

^ permalink raw reply	[flat|nested] 70+ messages in thread
* support file system generated / verified integrity information v2
@ 2026-01-28 16:14 Christoph Hellwig
  2026-01-28 16:15 ` [PATCH 11/15] iomap: allow file systems to hook into buffered read bio submission Christoph Hellwig
  0 siblings, 1 reply; 70+ messages in thread
From: Christoph Hellwig @ 2026-01-28 16:14 UTC (permalink / raw)
  To: Jens Axboe, Christian Brauner
  Cc: Darrick J. Wong, Carlos Maiolino, Martin K. Petersen, Anuj Gupta,
	Kanchan Joshi, linux-block, nvdimm, linux-fsdevel, linux-xfs

Hi all,

this series adds support to generate and verify integrity information
(aka T10 PI) in the file system, instead of the automatic below the
covers support that is currently used.

There two reasons for this:

  a) to increase the protection enveloped.  Right now this is just a
     minor step from the bottom of the block layer to the file system,
     but it is required to support io_uring integrity data passthrough in
     the file system similar to the currently existing support for block
     devices, which will follow next.  It also allows the file system to
     directly see the integrity error and act upon in, e.g. when using
     RAID either integrated (as in btrfs) or by supporting reading
     redundant copies through the block layer.
  b) to make the PI processing more efficient.  This is primarily a
     concern for reads, where the block layer auto PI has to schedule a
     work item for each bio, and the file system them has to do it again
     for bounce buffering.  Additionally the current iomap post-I/O
     workqueue handling is a lot more efficient by supporting merging and
     avoiding workqueue scheduling storms.

The implementation is based on refactoring the existing block layer PI
code to be reusable for this use case, and then adding relatively small
wrappers for the file system use case.  These are then used in iomap
to implement the semantics, and wired up in XFS with a small amount of
glue code.

Compared to the baseline (iomap-bounce branch), this does not change
performance for writes, but increases read performance up to 15% for 4k
I/O, with the benefit decreasing with larger I/O sizes as even the
baseline maxes out the device quickly on my older enterprise SSD.

Anuj gupta also measured a large decrease in QD1 latency on an
Intel optane device for small I/O sizes, but also an increase for
very large ones.

The series is based on the for-7.0/block-stable-pages in Jens's tee.

Git tree:

    git://git.infradead.org/users/hch/misc.git iomap-pi

Gitweb:

    https://git.infradead.org/?p=users/hch/misc.git;a=shortlog;h=refs/heads/iomap-pi

Changes since v1:
 - document usage of BI_ACT_*
 - return the action from fs_bio_integrity_alloc and use it in
   fs_bio_integrity_generation to be safe from misuse
 - use the newly added BIO_MAX_SIZE
 - rename bio_read_ops to xfs_bio_read_ops
 - fix commit message and comment typos

Diffstat:
 block/Makefile                |    2 
 block/bio-integrity-auto.c    |   80 +++---------------------
 block/bio-integrity-fs.c      |   81 +++++++++++++++++++++++++
 block/bio-integrity.c         |   64 +++++++++++++++++++
 block/bio.c                   |   17 +++--
 block/blk-mq.c                |    6 +
 block/blk-settings.c          |   13 ----
 block/blk.h                   |    6 +
 block/t10-pi.c                |   12 +--
 drivers/nvdimm/btt.c          |    6 +
 fs/fuse/file.c                |    5 -
 fs/iomap/bio.c                |  135 ++++++++++++++++++++++++++++--------------
 fs/iomap/buffered-io.c        |    8 +-
 fs/iomap/direct-io.c          |   15 ++++
 fs/iomap/internal.h           |   14 ++++
 fs/iomap/ioend.c              |   28 +++++++-
 fs/xfs/xfs_aops.c             |   49 ++++++++++++++-
 fs/xfs/xfs_iomap.c            |    9 +-
 include/linux/bio-integrity.h |   12 ++-
 include/linux/bio.h           |    2 
 include/linux/blk-integrity.h |   28 +++++++-
 include/linux/blkdev.h        |   34 ++++++++--
 include/linux/iomap.h         |   20 +++++-
 23 files changed, 465 insertions(+), 181 deletions(-)

^ permalink raw reply	[flat|nested] 70+ messages in thread
* support file system generated / verified integrity information v3
@ 2026-02-18  6:11 Christoph Hellwig
  2026-02-18  6:12 ` [PATCH 11/15] iomap: allow file systems to hook into buffered read bio submission Christoph Hellwig
  0 siblings, 1 reply; 70+ messages in thread
From: Christoph Hellwig @ 2026-02-18  6:11 UTC (permalink / raw)
  To: Jens Axboe, Christian Brauner
  Cc: Darrick J. Wong, Carlos Maiolino, Martin K. Petersen, Anuj Gupta,
	Kanchan Joshi, linux-block, nvdimm, linux-fsdevel, linux-xfs

Hi all,

this series adds support to generate and verify integrity information
(aka T10 PI) in the file system, instead of the automatic below the
covers support that is currently used.

There two reasons for this:

  a) to increase the protection enveloped.  Right now this is just a
     minor step from the bottom of the block layer to the file system,
     but it is required to support io_uring integrity data passthrough in
     the file system similar to the currently existing support for block
     devices, which will follow next.  It also allows the file system to
     directly see the integrity error and act upon in, e.g. when using
     RAID either integrated (as in btrfs) or by supporting reading
     redundant copies through the block layer.
  b) to make the PI processing more efficient.  This is primarily a
     concern for reads, where the block layer auto PI has to schedule a
     work item for each bio, and the file system them has to do it again
     for bounce buffering.  Additionally the current iomap post-I/O
     workqueue handling is a lot more efficient by supporting merging and
     avoiding workqueue scheduling storms.

The implementation is based on refactoring the existing block layer PI
code to be reusable for this use case, and then adding relatively small
wrappers for the file system use case.  These are then used in iomap
to implement the semantics, and wired up in XFS with a small amount of
glue code.

Compared to the baseline (iomap-bounce branch), this does not change
performance for writes, but increases read performance up to 15% for 4k
I/O, with the benefit decreasing with larger I/O sizes as even the
baseline maxes out the device quickly on my older enterprise SSD.

Anuj Gupta also measured a large decrease in QD1 latency on an Intel
Optane device for small I/O sizes, but also an increase for very large
ones.

Note that the upcoming XFS fsverity support also depends on some
infrastructure in this series.

Git tree:

    git://git.infradead.org/users/hch/misc.git iomap-pi

Gitweb:

    https://git.infradead.org/?p=users/hch/misc.git;a=shortlog;h=refs/heads/iomap-pi

Changes since v2:
 - rebased to current linus tree
 - use folio->mapping instead of file->f_mapping to support the upcoming
   fsverity use case
 - rename and slightly refactor the xfs iomap read ops to better addres
   that this will be used by fsverity as well.

Changes since v1:
 - document usage of BI_ACT_*
 - return the action from fs_bio_integrity_alloc and use it in
   fs_bio_integrity_generation to be safe from misuse
 - use the newly added BIO_MAX_SIZE
 - rename bio_read_ops to xfs_bio_read_ops
 - fix commit message and comment typos

Diffstat:
 block/Makefile                |    2 
 block/bio-integrity-auto.c    |   80 +++---------------------
 block/bio-integrity-fs.c      |   81 +++++++++++++++++++++++++
 block/bio-integrity.c         |   64 +++++++++++++++++++
 block/bio.c                   |   17 +++--
 block/blk-mq.c                |    6 +
 block/blk-settings.c          |   13 ----
 block/blk.h                   |    6 +
 block/t10-pi.c                |   12 +--
 drivers/nvdimm/btt.c          |    6 +
 fs/fuse/file.c                |    5 -
 fs/iomap/bio.c                |  135 ++++++++++++++++++++++++++++--------------
 fs/iomap/buffered-io.c        |    8 +-
 fs/iomap/direct-io.c          |   15 ++++
 fs/iomap/internal.h           |   14 ++++
 fs/iomap/ioend.c              |   28 +++++++-
 fs/xfs/xfs_aops.c             |   47 +++++++++++++-
 fs/xfs/xfs_iomap.c            |    9 +-
 include/linux/bio-integrity.h |   12 ++-
 include/linux/bio.h           |    2 
 include/linux/blk-integrity.h |   28 +++++++-
 include/linux/blkdev.h        |   34 ++++++++--
 include/linux/iomap.h         |   20 +++++-
 23 files changed, 463 insertions(+), 181 deletions(-)

^ permalink raw reply	[flat|nested] 70+ messages in thread

end of thread, other threads:[~2026-02-18  6:13 UTC | newest]

Thread overview: 70+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-01-21  6:43 support file system generated / verified integrity information Christoph Hellwig
2026-01-21  6:43 ` [PATCH 01/15] block: factor out a bio_integrity_action helper Christoph Hellwig
2026-01-23  0:01   ` Darrick J. Wong
2026-01-23  6:03     ` Christoph Hellwig
2026-01-23  7:13       ` Darrick J. Wong
2026-01-26  5:03         ` Christoph Hellwig
2026-01-25 19:46   ` Kanchan Joshi
2026-01-27 14:07   ` Martin K. Petersen
2026-01-27 14:55   ` Anuj gupta
2026-01-21  6:43 ` [PATCH 02/15] block: factor out a bio_integrity_setup_default helper Christoph Hellwig
2026-01-23  0:05   ` Darrick J. Wong
2026-01-23  6:08     ` Christoph Hellwig
2026-01-23  7:14       ` Darrick J. Wong
2026-01-25 20:14   ` Kanchan Joshi
2026-01-27 14:08   ` Martin K. Petersen
2026-01-27 14:55   ` Anuj gupta
2026-01-21  6:43 ` [PATCH 03/15] block: add a bdev_has_integrity_csum helper Christoph Hellwig
2026-01-23  0:07   ` Darrick J. Wong
2026-01-26 18:03   ` Kanchan Joshi
2026-01-27 14:08   ` Martin K. Petersen
2026-01-27 14:55   ` Anuj gupta
2026-01-21  6:43 ` [PATCH 04/15] block: prepare generation / verification helpers for fs usage Christoph Hellwig
2026-01-23  0:07   ` Darrick J. Wong
2026-01-26 18:04   ` Kanchan Joshi
2026-01-27 14:09   ` Martin K. Petersen
2026-01-27 14:56   ` Anuj gupta
2026-01-21  6:43 ` [PATCH 05/15] block: make max_integrity_io_size public Christoph Hellwig
2026-01-23  0:08   ` Darrick J. Wong
2026-01-26 18:04   ` Kanchan Joshi
2026-01-27 14:10   ` Martin K. Petersen
2026-01-27 14:56   ` Anuj gupta
2026-01-21  6:43 ` [PATCH 06/15] block: add fs_bio_integrity helpers Christoph Hellwig
2026-01-23  0:11   ` Darrick J. Wong
2026-01-26 18:12   ` Kanchan Joshi
2026-01-27  5:15     ` Christoph Hellwig
2026-01-27 14:12   ` Martin K. Petersen
2026-01-27 14:57   ` Anuj gupta
2026-01-21  6:43 ` [PATCH 07/15] block: pass a maxlen argument to bio_iov_iter_bounce Christoph Hellwig
2026-01-22  1:04   ` Darrick J. Wong
2026-01-22  6:04     ` Christoph Hellwig
2026-01-22 18:02       ` Darrick J. Wong
2026-01-27 14:12   ` Martin K. Petersen
2026-01-27 14:57   ` Anuj gupta
2026-01-21  6:43 ` [PATCH 08/15] iomap: refactor iomap_bio_read_folio_range Christoph Hellwig
2026-01-22  0:42   ` Darrick J. Wong
2026-01-21  6:43 ` [PATCH 09/15] iomap: pass the iomap_iter to ->submit_read Christoph Hellwig
2026-01-22  0:43   ` Darrick J. Wong
2026-01-21  6:43 ` [PATCH 10/15] iomap: only call into ->submit_read when there is a read_ctx Christoph Hellwig
2026-01-22  0:44   ` Darrick J. Wong
2026-01-22  2:44     ` Joanne Koong
2026-01-22  5:59       ` Christoph Hellwig
2026-01-22 18:02         ` Darrick J. Wong
2026-01-21  6:43 ` [PATCH 11/15] iomap: allow file systems to hook into buffered read bio submission Christoph Hellwig
2026-01-22  0:49   ` Darrick J. Wong
2026-01-22  6:01     ` Christoph Hellwig
2026-01-22 18:04       ` Darrick J. Wong
2026-01-21  6:43 ` [PATCH 12/15] iomap: add a bioset pointer to iomap_read_folio_ops Christoph Hellwig
2026-01-22  0:49   ` Darrick J. Wong
2026-01-21  6:43 ` [PATCH 13/15] iomap: support ioends for buffered reads Christoph Hellwig
2026-01-22  0:50   ` Darrick J. Wong
2026-01-21  6:43 ` [PATCH 14/15] iomap: support T10 protection information Christoph Hellwig
2026-01-22  0:59   ` Darrick J. Wong
2026-01-22  6:03     ` Christoph Hellwig
2026-01-21  6:43 ` [PATCH 15/15] xfs: " Christoph Hellwig
2026-01-22  1:02   ` Darrick J. Wong
2026-01-27 14:54 ` support file system generated / verified integrity information Anuj gupta
2026-01-27 15:16   ` Christoph Hellwig
2026-01-29  9:23     ` Anuj Gupta
  -- strict thread matches above, loose matches on Subject: below --
2026-01-28 16:14 support file system generated / verified integrity information v2 Christoph Hellwig
2026-01-28 16:15 ` [PATCH 11/15] iomap: allow file systems to hook into buffered read bio submission Christoph Hellwig
2026-02-18  6:11 support file system generated / verified integrity information v3 Christoph Hellwig
2026-02-18  6:12 ` [PATCH 11/15] iomap: allow file systems to hook into buffered read bio submission Christoph Hellwig

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox