Linux EXT4 FS development
 help / color / mirror / Atom feed
* [PATCH v14 00/12] Data in direntry (dirdata) feature
@ 2026-11-03 13:49 Artem Blagodarenko
  2026-11-03 13:49 ` [PATCH v14 01/12] ext4: validate count against limit in ext4_dx_csum_verify/_set Artem Blagodarenko
                   ` (11 more replies)
  0 siblings, 12 replies; 25+ messages in thread
From: Artem Blagodarenko @ 2026-11-03 13:49 UTC (permalink / raw)
  To: linux-ext4; +Cc: tytso, jack, adilger.kernel, ablagodarenko

EXT4 currently stores an 8-byte hash in the directory entry immediately
after the file name to support simultaneous fscrypt and casefold
functionality.

The dirdata feature provides a general mechanism to store multiple
metadata records in each directory entry after the NUL filename
terminator. The unused high 4 bits of 'file_type' indicate which
records are present; each record starts with a 1-byte length field,
providing forward compatibility.

The first user of this feature is LUFID (Locally Unique File ID), an
identifier that must survive rename and be accessible from directory
readdir without an inode lookup.

e2fsprogs support is provided in a separate patch series:
  [PATCH v4 00/13] e2fsprogs: Add support for dirdata feature
and the fstests coverage in:
  [RFC PATCH v6 0/8] ext4: tests for the dirdata feature

As Andreas noted in the v12 thread, the first patches are bugfixes,
code cleanups and reorganisation that could be reviewed and merged ahead
of the main feature.  In v14 these are patches 1-8, and the feature is
patches 9-12.  Patch 02 is new: it fixes a bug in current upstream for
casefolded+encrypted directories and does not depend on the rest.

Changes in v14 (addresses the Sashiko review of v13 and two kernel test
robot reports; no human review yet).  Patch numbers below are v14's;
v13 patch N is v14 patch N+1 from patch 02 on.

- New patch 02, a fix for an upstream bug that the Sashiko comment on
  v13 patch 05 led to: on casefolded+encrypted directories the kernel
  computes the index node limit one smaller than e2fsck does.  e2fsck
  reports index nodes written by the kernel as corrupt leaf blocks, and
  the kernel refuses index nodes written by e2fsck -D ("dx entry: limit
  510 != node limit 509").  Reproduced on ext4.git dev without this
  series.  The patch uses the e2fsck value and still accepts the old
  one.  It does not depend on the rest of the series.

- Patch 05: declare the hash words of struct ext4_dirent_hash as plain
  __le32 fields, which clang -Wunaligned-access reported for the nested
  struct inside the packed one (lkp, 202610301711.NaWjQ2jw).

- Patch 06:
  * dx_get_dx_info() finds ".." through the rec_len of ".", as
    make_indexed_dir(), get_dx_countlimit() and ext4_htree_fill_tree()
    do, so a "." entry with slack can no longer put dx_root_info on top
    of "..".  It also checks that the root has room for dx_root_info,
    two dx_entry slots and the dx_tail.
  * make_indexed_dir() looks up dx_root_info before it appends a block
    or wipes the old entries, so a failure leaves the directory as it
    was.
  * v13 changed dx_node_limit() here, which made the kernel reject the
    index nodes it had written itself; that is now patch 02.

- Patch 07: ext4_dir_rec_len() keeps reserving the casefold+fscrypt hash
  on dirdata filesystems.  ext4_dir_entry_len() returns the larger of
  the length with the dirdata records and the length with the hash at
  its fixed offset, so an entry without a valid CFHASH record still
  covers the bytes the fallback readers use (Sashiko on v13 06, 08, 10).
  dx_root_limit() cannot underflow.

- Patch 08: add_dirent_to_buf() checks that the entry do_split() hands
  over has room for the new entry before anything is written.

- Patch 09: the dirdata size of a new entry is computed in one helper
  used by ext4_find_dest_de(), add_dirent_to_buf() and the inline-data
  path.  get_dx_countlimit() includes struct dx_countlimit in its bound.
  The fake "." and ".." entries in ext4_inlinedir_to_tree() are zeroed.

- Patch 10: ext4_setent() removes a LUFID record by moving the records
  after it down, and adds one in place when the entry has room.  On a
  failed RENAME_WHITEOUT rename, ext4_resetent() puts the LUFID
  snapshot back instead of only setting the flag, so the entry is not
  left with CFHASH bytes where LUFID is expected.  ext4_lufid_snapshot()
  reads ddh_length once.

- Patch 11: ext4_setent_compact_exts() is gone (patch 10 covers it, and
  INO64 records are now moved too).  The legacy hash read in
  ext4_dirdata_get() is bounds-checked.

- Patch 12:
  * the LUFID payload is limited to 48 bytes, so a split 1k block always
    has room for the largest entry;
  * ext4_dirdata_set_lufid() calls dquot_initialize(), honours DIRSYNC,
    checks strict casefold names before deleting the entry, re-adds the
    stored name (decrypted in encrypted casefolded directories) and
    sets up both names before deleting, so the rollback does not
    allocate;
  * it no longer locks the target inode, which protected nothing and
    could invert the lock order on a corrupted directory;
  * its stack frame dropped from 1104 to 560 bytes on x86_64 (lkp,
    202610302057.dZ9WDGS4).

Not changed: ext4_dir_entry_len() still parses dirdata without checking
the superblock feature, because callers pass a NULL inode for "." and
"..".  The parse stays inside rec_len, and the overflow it could lead to
is closed by the check in patch 08.

Tested on ext4.git dev in a VM: every patch builds with W=1, and with
DX_DEBUG (with -Wno-missing-prototypes, which DX_DEBUG already needs on
ext4.git dev); fstests
ext4/064-069 pass with and without -O dirdata; 1k-block directories with
255-byte names and 48-byte LUFIDs split cleanly; a RENAME_WHITEOUT that
fails with ENOSPC keeps the old LUFID; a multi-level casefolded+encrypted
directory (12000 255-byte names, with and without dirdata) passes
e2fsck v4 before and after e2fsck -fD and after more kernel inserts.

v13: https://lore.kernel.org/linux-ext4/20261029122413.230409-1-ablagodarenko@thelustrecollective.com/

Changes in v13 (v13 patch numbers):

- Rebased onto ext4.git dev (8c4f6e337634 "ext4: introduce a mount
  option for iomap buffered I/O path").

- Patch 07: also rename the ext4_dir_rec_len() callers added upstream
  since v12 by:
    bc4b7b0414c3 ("ext4: validate readdir offset before accessing dirent")
    9333cc809f0a ("ext4: fix out-of-bounds read in ext4_read_inline_dir()")
    c7e6b863d298 ("ext4: check dir entry fits before reading the hash
                   trailer in ext4_search_dir()")

- Patch 10: keep the hash-trailer bounds check added by c7e6b863d298 in
  ext4_search_dir(), together with the new buf_size argument to
  ext4_match().

- Every patch now builds on its own. In v12 patches 05-10 only built
  with the whole series applied (Sashiko's build-failure reports on
  parts 5 and 7 were correct). The final code is unchanged; only hunks
  moved between patches:
    05: ext4_dir_entry_len() moved here from 06, since dx_get_dx_info()
        is its first user; dx_root_limit() no longer calls
        dx_get_dx_info() before 06 changes its signature;
        dx_node_limit() keeps ext4_dir_rec_len() until the rename in 07.
    07: add_dirent_to_buf() passes dlen = 0 to ext4_find_dest_de(), as
        the inline-data caller already does, until 08 computes it;
        dx_show_leaf() (DX_DEBUG) is renamed here too.
    08: the (struct ext4_dirent_fid *) cast in ext4_dentry_get_fid() is
        added here instead of in 11.
  Each patch was built with W=1 and with DX_DEBUG.

v12: https://lore.kernel.org/linux-ext4/20260801181227.5198-1-ablagodarenko@thelustrecollective.com/

Changes in v12:
- Patch 09: allocate edp in ext4_rename() before ext4_setent() so that
  an -ENOMEM cannot leave a RENAME_WHITEOUT source entry with the LUFID
  flag restored but its bytes already overwritten.


Artem Blagodarenko (12):
  ext4: validate count against limit in ext4_dx_csum_verify/_set
  ext4: use the e2fsck limit for htree index nodes
  ext4: replace ext4_dir_entry with ext4_dir_entry_2
  ext4: add ext4_dir_entry_is_tail()
  ext4: add dirdata format definitions and access helpers
  ext4: refactor dx_root to support variable dirent sizes
  ext4: add ext4_dir_entry_len()
  ext4: rename ext4_dir_rec_len() and clarify dirdata usage
  ext4: dirdata feature
  ext4: add dirdata LUFID support for directory entry rename
  ext4: add dirdata set/get helpers
  ext4: Add EXT4_IOC_SET_LUFID ioctl for setting LUFID on directory
    entries

 fs/ext4/dir.c             |   53 +-
 fs/ext4/ext4.h            |  295 ++++++++-
 fs/ext4/fast_commit.c     |    1 +
 fs/ext4/fast_commit.h     |    1 +
 fs/ext4/inline.c          |   66 +-
 fs/ext4/ioctl.c           |   87 +++
 fs/ext4/namei.c           | 1326 ++++++++++++++++++++++++++++++-------
 fs/ext4/sysfs.c           |    2 +
 include/uapi/linux/ext4.h |   14 +
 9 files changed, 1570 insertions(+), 275 deletions(-)


base-commit: 8c4f6e3376348fc3162eed86a05b933921c8f9ed
-- 
2.52.0


^ permalink raw reply	[flat|nested] 25+ messages in thread

end of thread, other threads:[~2026-11-03 14:29 UTC | newest]

Thread overview: 25+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-03 13:49 [PATCH v14 00/12] Data in direntry (dirdata) feature Artem Blagodarenko
2026-11-03 13:49 ` [PATCH v14 01/12] ext4: validate count against limit in ext4_dx_csum_verify/_set Artem Blagodarenko
2026-11-03 13:56   ` sashiko-bot
2026-11-03 13:49 ` [PATCH v14 02/12] ext4: use the e2fsck limit for htree index nodes Artem Blagodarenko
2026-11-03 13:56   ` sashiko-bot
2026-11-03 13:49 ` [PATCH v14 03/12] ext4: replace ext4_dir_entry with ext4_dir_entry_2 Artem Blagodarenko
2026-11-03 14:05   ` sashiko-bot
2026-11-03 13:49 ` [PATCH v14 04/12] ext4: add ext4_dir_entry_is_tail() Artem Blagodarenko
2026-11-03 13:58   ` sashiko-bot
2026-11-03 13:49 ` [PATCH v14 05/12] ext4: add dirdata format definitions and access helpers Artem Blagodarenko
2026-11-03 13:56   ` sashiko-bot
2026-11-03 13:49 ` [PATCH v14 06/12] ext4: refactor dx_root to support variable dirent sizes Artem Blagodarenko
2026-11-03 14:18   ` sashiko-bot
2026-11-03 13:49 ` [PATCH v14 07/12] ext4: add ext4_dir_entry_len() Artem Blagodarenko
2026-11-03 14:08   ` sashiko-bot
2026-11-03 13:49 ` [PATCH v14 08/12] ext4: rename ext4_dir_rec_len() and clarify dirdata usage Artem Blagodarenko
2026-11-03 14:20   ` sashiko-bot
2026-11-03 13:49 ` [PATCH v14 09/12] ext4: dirdata feature Artem Blagodarenko
2026-11-03 14:27   ` sashiko-bot
2026-11-03 13:49 ` [PATCH v14 10/12] ext4: add dirdata LUFID support for directory entry rename Artem Blagodarenko
2026-11-03 14:17   ` sashiko-bot
2026-11-03 13:49 ` [PATCH v14 11/12] ext4: add dirdata set/get helpers Artem Blagodarenko
2026-11-03 14:18   ` sashiko-bot
2026-11-03 13:49 ` [PATCH v14 12/12] ext4: Add EXT4_IOC_SET_LUFID ioctl for setting LUFID on directory entries Artem Blagodarenko
2026-11-03 14:29   ` sashiko-bot

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox