[RFC 0/7] erofs-utils: add manifest input support

[Wenyuan Xu <sereno.xu@deepseek.com>]

Hi,

This series adds manifest input support to mkfs.erofs. It lets callers
specify the image tree and metadata without first creating a matching
staging directory, while reading regular file contents from a backing
directory. Only entries listed in the manifest are imported; source
directories are not scanned recursively.

This series is based on origin/experimental-tests.

Usage:

  mkfs.erofs --manifest=manifest.txt out.erofs basedir/

The manifest accepts two record formats, which can be mixed:

  PATH PAYLOAD SIZE MODE UID GID RDEV MTIME [KEY=VALUE ...]
  PATH PAYLOAD

The complete record syntax is inspired by composefs's
"mkcomposefs --from-file" format, documented in composefs-dump(5) [1].
This series uses eight fixed fields with PAYLOAD immediately after PATH,
omitting the NLINK, CONTENT and DIGEST fields from the composefs format.

Complete records specify metadata and xattrs explicitly. Regular file
payloads identify backing files relative to basedir, symlink payloads
provide link contents, and a MODE beginning with @ creates a hard link
to an earlier entry. Fields unused for a file type are not parsed and
can be written as '-'.

Two-field records import metadata, contents and xattrs from the backing
file using local import policies. Symlinks are imported without
following their targets, and source hard links are preserved unless
--hard-dereference is specified.

PATH must start with '/', with the root directory defined first and
parents preceding their children. Duplicate paths and xattr keys are
rejected. String fields support \xXY byte escapes, including binary
xattr values. MTIME uses seconds.nanoseconds, where 1.1 means one second
and one nanosecond.

Both formats support regular file compression, xattr name prefixes and
shared xattrs. They also work with --incremental=data: the manifest
provides the update tree, matching directories are merged, and directory
metadata comes from the new records. No on-disk format changes are
required.

Patch 1 moves local root inode initialization to the local source path.
Patches 2 and 3 add helpers for duplicate xattr detection and building
shared xattrs from already loaded inodes. Patches 4 and 5 implement the
complete and two-field formats. Patches 6 and 7 add documentation and
regression coverage.

Test 033 covers file types, metadata, hard links, escaping, binary and
shared xattrs, name prefixes, malformed records, compression and
incremental updates. It checks images using dump.erofs and fsck.erofs.

Please apply the fix in [2] before running test 033.

[1] https://man.archlinux.org/man/composefs-dump.5.en
[2] https://lore.kernel.org/all/20261104035208.2471090-1-sereno.xu@deepseek.com/

Feedback on the record formats and importer integration is welcome.

Assisted-by: LLM
Signed-off-by: Wenyuan Xu <sereno.xu@deepseek.com>

Wenyuan Xu (7):
  erofs-utils: mkfs: init root inode in local source mode
  erofs-utils: lib: add erofs_inode_has_xattr helper
  erofs-utils: lib: add erofs_build_shared_xattrs() helper
  erofs-utils: mkfs: add manifest input mode
  erofs-utils: mkfs: support two-field manifest records
  erofs-utils: man: document manifest input formats
  erofs-utils: tests: cover manifest input

 include/erofs/inode.h    |   2 +
 include/erofs/manifest.h |  17 ++
 include/erofs/xattr.h    |   3 +
 lib/Makefile.am          |   3 +-
 lib/inode.c              |  30 +-
 lib/manifest.c           | 527 ++++++++++++++++++++++++++++++++
 lib/xattr.c              |  62 +++-
 man/mkfs.erofs.1         | 114 +++++++
 mkfs/main.c              |  43 ++-
 tests/Makefile.am        |   3 +
 tests/erofs/033          | 637 +++++++++++++++++++++++++++++++++++++++
 tests/erofs/033.out      |   2 +
 12 files changed, 1403 insertions(+), 40 deletions(-)
 create mode 100644 include/erofs/manifest.h
 create mode 100644 lib/manifest.c
 create mode 100755 tests/erofs/033
 create mode 100644 tests/erofs/033.out

-- 
2.43.0