[PATCH 00/13] block, ext4: allow encryption with a block size larger than the page size

[Baokun Li <libaokun@linux.alibaba.com>]

Motivation
==========

ext4 supports a block size larger than the page size (LBS), e.g. a 32k
block size on a machine with 4k pages [1].  Encryption, however, is
still refused on such a filesystem: file contents used to be encrypted
page by page in the fs-layer path, which cannot handle a folio that
covers several data units, so the checks that refuse encryption with
such a block size stayed in place.

Since commit 5acc6f649a68 ("fscrypt: Always use blk-crypto for
contents on block-based filesystems") file contents are always
encrypted through blk-crypto.  Inline encryption hardware can already
en/decrypt a data unit that spans several pages, but the fallback and
the block layer cannot carry such a data unit:

  1. blk-crypto-fallback en/decrypts a data unit with the bio_vecs of
     a single page-clamped segment, so a larger data unit fails with
     BLK_STS_INVAL;
  2. the block layer requires every bio_vec of an encrypted bio to be
     data unit aligned, and splits such a bio without rounding the
     split size down to the data unit size;
  3. iomap cuts the direct I/O bios of an encrypted file only at the
     logical block size of the device, so a bio can end in the middle
     of a data unit.

This series adds the missing pieces on all three layers, removes the
refusals, and enables large folios for encrypted files on the way.

[1]: https://lore.kernel.org/all/20251121090654.631996-1-libaokun@huaweicloud.com

Patch Overview
==============

  * Patches 1-2 (ext4, from Alberto Garcia) [2]: Refuse to set an
    encryption policy when the block size is larger than the page size,
    and reject encrypted inodes of such a filesystem.  They are the
    fix for the stable branches; patch 11 removes the refusals again
    for mainline.  They are carried along unchanged so that the series
    applies cleanly on its own and avoids adaptation conflicts with
    the stable fix.

  * Patch 3 (blk-crypto): Reject the data unit sizes that
    blk-crypto-fallback can't handle when the key is set up, instead of
    failing every I/O with BLK_STS_INVAL.

  * Patches 4-6 (blk-crypto-fallback): En/decrypt one data unit per
    crypto request, so that a data unit can span several bio_vecs, and
    support the data units that are larger than a page, whose bio_vecs
    and scatterlists come from a mempool.

  * Patch 7 (iomap): Cut the direct I/O bios of encrypted files at the
    block size, so that they cover whole data units.

  * Patches 8-10 (block): Round the split size of an encrypted bio
    down to the data unit size, warn when a bio that must be split
    cannot be split into a valid request, and drop the per-bio_vec data
    unit alignment requirement, which a data unit larger than a page
    cannot satisfy.  The contract is documented: an encrypted bio
    starts at a data unit boundary and covers whole data units.

  * Patches 11-13 (ext4): Remove the refusals, route the folios that
    cover a data unit number wraparound to the buffer_head read path,
    and enable large folios for encrypted files.

[2]: https://lore.kernel.org/r/cover.1790343958.git.berto@igalia.com

Benchmark
=========

Hardware: 96-core machine, 3.84T NVMe, 512G memory
Filesystem: ext4, encryption on the fallback path

Buffered sequential 1M writes and reads (1G, fsync) in MiB/s; medians
of 3 runs.

Base tree, 4k-block filesystem:

  job         4k plain    4k enc
  --------    --------    ------
  seqwrite        1966      1134
  seqread         1878       996

With LBS enabled, encrypted 4k-block vs 32k-block filesystem:

  job         4k enc    32k enc    32k plain    4k->32k
  --------    ------    -------    ---------    -------
  seqwrite      1044       1399         2062       +34%
  seqread        931       1793         1791       +93%

Buffered sequential writes (1G, fsync) of the encrypted 4k-block
filesystem across request sizes, base tree vs this series:

  bs      Before    After    gain
  ----    ------    -----    ----
  4k        1118     1162     +4%
  16k       1260     1756    +39%
  64k       1272     1977    +55%
  256k      1308     2081    +59%
  1M        1260     1455    +15%

Testing
=======

  * kvm-xfstests -c ext4/all -g encrypt (which includes ext4/32k) has
    been executed with no new failures.

  * Manually verified read/write, fallocate -z and punch hole on 4k,
    8k, 16k and 32k block sizes, and direct I/O with a properly
    aligned user buffer on 4k, 8k and 32k block sizes.

  * Manually verified fsx and fsstress on a 64k block size, and direct
    I/O to an encrypted file on a 64k block size with a user buffer
    backed by large folios.

Comments and questions are, as always, welcome.

Thanks,
Baokun

Alberto Garcia (2):
  ext4: refuse encryption when block size > page size
  ext4: reject encrypted inodes when block size > page size

Baokun Li (11):
  blk-crypto: reject fallback data unit sizes it can't handle
  blk-crypto: decrypt one data unit per crypto request
  blk-crypto: encrypt one data unit per crypto request
  blk-crypto: support data units larger than a page
  iomap: align the direct I/O bios of encrypted files to the block size
  block: split encrypted bios at data unit boundaries
  block: warn about bios that can't be split
  block: don't require data unit aligned bio_vecs for encrypted bios
  ext4: allow encryption with a block size larger than the page size
  ext4: send folios that cover a wraparound to the buffer_head path
  ext4: allow large folios for encrypted files

 Documentation/block/inline-encryption.rst |  13 +
 block/blk-crypto-fallback.c               | 446 +++++++++++++++++-----
 block/blk-crypto-internal.h               |   4 +-
 block/blk-crypto.c                        |  40 +-
 block/blk-merge.c                         |  48 ++-
 fs/crypto/block.c                         |   3 +
 fs/ext4/readpage.c                        |  11 +
 fs/ext4/super.c                           |  22 +-
 fs/iomap/direct-io.c                      |  10 +
 9 files changed, 462 insertions(+), 135 deletions(-)


base-commit: fd73f4a6659897191fa0d40695fe370925dd3780
-- 
2.43.7