* [PATCH v2 0/5] erofs-utils: implement the FULLDATA rebuild mode
@ 2026-03-09 16:38 Lucas Karpinski
2026-03-09 16:38 ` [PATCH v2 1/5] erofs-utils: lib: pass uniaddr_offset to erofs_rebuild_load_tree Lucas Karpinski
` (4 more replies)
0 siblings, 5 replies; 7+ messages in thread
From: Lucas Karpinski @ 2026-03-09 16:38 UTC (permalink / raw)
To: linux-erofs; +Cc: jcalmels, Lucas Karpinski
Currently, erofs-utils supports backing blobs for multi-image setups.
This implements the FULLDATA import which allows for the merging of
multiple source images into a single self-contained erofs image.
To optimize the rebuild process, erofs_copy_file_range() is used to
leverage the copy_file_range(2) if available. This bypasses userspace
buffering and enables kernel side data transfers.
Verification:
1. Created a source directory containing flat inodes, inline inodes,
symlinks and absolute symlinks. Verified data integrity by comparing
checksums of files within the mounted image.
2. Built same image with default rebuild and rebuild with FULLDATA. Then
ran F-i-f/tdiff comparing the two.
changes in v2:
- reworked erofs_rebuild_load_trees_full into
erofs_mkfs_rebuild_load_trees.
- removed --merge option (use --clean=data instead).
- updated man.
Signed-off-by: Lucas Karpinski <lkarpinski@nvidia.com>
---
Lucas Karpinski (5):
erofs-utils: lib: pass uniaddr_offset to erofs_rebuild_load_tree
erofs-utils: lib: add helper function erofs_uuid_unparse_as_tag
erofs-utils: lib: preserve primarydevice_blocks if already larger
erofs-utils: mfks: add rebuild FULLDATA for combined EROFS images
erofs-utils: manpages: update to reflect fulldata support
lib/cache.c | 6 +++
lib/importer.c | 5 ++-
lib/liberofs_cache.h | 1 +
lib/liberofs_rebuild.h | 7 ++-
lib/liberofs_uuid.h | 1 +
lib/rebuild.c | 117 +++++++++++++++++++++++++++++++++++++++++++++++--
lib/uuid_unparse.c | 16 ++++++-
man/mkfs.erofs.1 | 7 ++-
mkfs/main.c | 53 ++++++++++++++--------
9 files changed, 185 insertions(+), 28 deletions(-)
---
^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH v2 1/5] erofs-utils: lib: pass uniaddr_offset to erofs_rebuild_load_tree
2026-03-09 16:38 [PATCH v2 0/5] erofs-utils: implement the FULLDATA rebuild mode Lucas Karpinski
@ 2026-03-09 16:38 ` Lucas Karpinski
2026-03-09 16:38 ` [PATCH v2 2/5] erofs-utils: lib: add helper function erofs_uuid_unparse_as_tag Lucas Karpinski
` (3 subsequent siblings)
4 siblings, 0 replies; 7+ messages in thread
From: Lucas Karpinski @ 2026-03-09 16:38 UTC (permalink / raw)
To: linux-erofs; +Cc: jcalmels, Lucas Karpinski
uniaddr_offset will be needed for rebuild fulldata mode. Add a parameter
to erofs_rebuild_load_tree and include it in the erofs_rebuild_dir_context
struct as well.
Signed-off-by: Lucas Karpinski <lkarpinski@nvidia.com>
---
lib/liberofs_rebuild.h | 3 ++-
lib/rebuild.c | 5 ++++-
mkfs/main.c | 2 +-
3 files changed, 7 insertions(+), 3 deletions(-)
diff --git a/lib/liberofs_rebuild.h b/lib/liberofs_rebuild.h
index 69802fb..d8c4c8a 100644
--- a/lib/liberofs_rebuild.h
+++ b/lib/liberofs_rebuild.h
@@ -14,7 +14,8 @@ struct erofs_dentry *erofs_rebuild_get_dentry(struct erofs_inode *pwd,
char *path, bool aufs, bool *whout, bool *opq, bool to_head);
int erofs_rebuild_load_tree(struct erofs_inode *root, struct erofs_sb_info *sbi,
- enum erofs_rebuild_datamode mode);
+ enum erofs_rebuild_datamode mode,
+ erofs_blk_t uniaddr_offset);
int erofs_rebuild_load_basedir(struct erofs_inode *dir, u64 *nr_subdirs,
unsigned int *i_nlink);
diff --git a/lib/rebuild.c b/lib/rebuild.c
index f89a17c..7e62bc9 100644
--- a/lib/rebuild.c
+++ b/lib/rebuild.c
@@ -286,6 +286,7 @@ struct erofs_rebuild_dir_context {
unsigned int *i_nlink;
};
};
+ erofs_blk_t uniaddr_offset; /* unified addressing offset for FULL mode */
};
static int erofs_rebuild_dirent_iter(struct erofs_dir_context *ctx)
@@ -419,7 +420,8 @@ out:
}
int erofs_rebuild_load_tree(struct erofs_inode *root, struct erofs_sb_info *sbi,
- enum erofs_rebuild_datamode mode)
+ enum erofs_rebuild_datamode mode,
+ erofs_blk_t uniaddr_offset)
{
struct erofs_inode inode = {};
struct erofs_rebuild_dir_context ctx;
@@ -452,6 +454,7 @@ int erofs_rebuild_load_tree(struct erofs_inode *root, struct erofs_sb_info *sbi,
.ctx.cb = erofs_rebuild_dirent_iter,
.mergedir = root,
.datamode = mode,
+ .uniaddr_offset = uniaddr_offset,
};
ret = erofs_iterate_dir(&ctx.ctx, false);
free(inode.i_srcpath);
diff --git a/mkfs/main.c b/mkfs/main.c
index 58c18f9..c2c0a1d 100644
--- a/mkfs/main.c
+++ b/mkfs/main.c
@@ -1736,7 +1736,7 @@ static int erofs_mkfs_rebuild_load_trees(struct erofs_inode *root)
list_for_each_entry(src, &rebuild_src_list, list) {
src->xamgr = g_sbi.xamgr;
- ret = erofs_rebuild_load_tree(root, src, datamode);
+ ret = erofs_rebuild_load_tree(root, src, datamode, 0);
src->xamgr = NULL;
if (ret) {
erofs_err("failed to load %s", src->devname);
--
Git-155)
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH v2 2/5] erofs-utils: lib: add helper function erofs_uuid_unparse_as_tag
2026-03-09 16:38 [PATCH v2 0/5] erofs-utils: implement the FULLDATA rebuild mode Lucas Karpinski
2026-03-09 16:38 ` [PATCH v2 1/5] erofs-utils: lib: pass uniaddr_offset to erofs_rebuild_load_tree Lucas Karpinski
@ 2026-03-09 16:38 ` Lucas Karpinski
2026-03-09 16:38 ` [PATCH v2 3/5] erofs-utils: lib: preserve primarydevice_blocks if already larger Lucas Karpinski
` (2 subsequent siblings)
4 siblings, 0 replies; 7+ messages in thread
From: Lucas Karpinski @ 2026-03-09 16:38 UTC (permalink / raw)
To: linux-erofs; +Cc: jcalmels, Lucas Karpinski
Add helper function for converting uuid to tag. This will also be used in
rebuild FULLDATA mode.
Signed-off-by: Lucas Karpinski <lkarpinski@nvidia.com>
---
lib/liberofs_uuid.h | 1 +
lib/uuid_unparse.c | 16 +++++++++++++++-
mkfs/main.c | 11 +----------
3 files changed, 17 insertions(+), 11 deletions(-)
diff --git a/lib/liberofs_uuid.h b/lib/liberofs_uuid.h
index 63b358a..c35762a 100644
--- a/lib/liberofs_uuid.h
+++ b/lib/liberofs_uuid.h
@@ -4,6 +4,7 @@
void erofs_uuid_generate(unsigned char *out);
void erofs_uuid_unparse_lower(const unsigned char *buf, char *out);
+void erofs_uuid_unparse_as_tag(const unsigned char *buf, char *out);
int erofs_uuid_parse(const char *in, unsigned char *uu);
#endif
diff --git a/lib/uuid_unparse.c b/lib/uuid_unparse.c
index 3255c4b..18e87d1 100644
--- a/lib/uuid_unparse.c
+++ b/lib/uuid_unparse.c
@@ -8,7 +8,8 @@
#include "erofs/config.h"
#include "liberofs_uuid.h"
-void erofs_uuid_unparse_lower(const unsigned char *buf, char *out) {
+void erofs_uuid_unparse_lower(const unsigned char *buf, char *out)
+{
sprintf(out, "%04x%04x-%04x-%04x-%04x-%04x%04x%04x",
(buf[0] << 8) | buf[1],
(buf[2] << 8) | buf[3],
@@ -19,3 +20,16 @@ void erofs_uuid_unparse_lower(const unsigned char *buf, char *out) {
(buf[12] << 8) | buf[13],
(buf[14] << 8) | buf[15]);
}
+
+void erofs_uuid_unparse_as_tag(const unsigned char *buf, char *out)
+{
+ sprintf(out, "%04x%04x%04x%04x%04x%04x%04x%04x",
+ (buf[0] << 8) | buf[1],
+ (buf[2] << 8) | buf[3],
+ (buf[4] << 8) | buf[5],
+ (buf[6] << 8) | buf[7],
+ (buf[8] << 8) | buf[9],
+ (buf[10] << 8) | buf[11],
+ (buf[12] << 8) | buf[13],
+ (buf[14] << 8) | buf[15]);
+}
diff --git a/mkfs/main.c b/mkfs/main.c
index c2c0a1d..48da20f 100644
--- a/mkfs/main.c
+++ b/mkfs/main.c
@@ -1782,16 +1782,7 @@ static int erofs_mkfs_rebuild_load_trees(struct erofs_inode *root)
memcpy(devs[idx].tag, tag, sizeof(devs[0].tag));
else
/* convert UUID of the source image to a hex string */
- sprintf((char *)g_sbi.devs[idx].tag,
- "%04x%04x%04x%04x%04x%04x%04x%04x",
- (src->uuid[0] << 8) | src->uuid[1],
- (src->uuid[2] << 8) | src->uuid[3],
- (src->uuid[4] << 8) | src->uuid[5],
- (src->uuid[6] << 8) | src->uuid[7],
- (src->uuid[8] << 8) | src->uuid[9],
- (src->uuid[10] << 8) | src->uuid[11],
- (src->uuid[12] << 8) | src->uuid[13],
- (src->uuid[14] << 8) | src->uuid[15]);
+ erofs_uuid_unparse_as_tag(src->uuid, (char *)g_sbi.devs[idx].tag);
}
return 0;
}
--
Git-155)
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH v2 3/5] erofs-utils: lib: preserve primarydevice_blocks if already larger
2026-03-09 16:38 [PATCH v2 0/5] erofs-utils: implement the FULLDATA rebuild mode Lucas Karpinski
2026-03-09 16:38 ` [PATCH v2 1/5] erofs-utils: lib: pass uniaddr_offset to erofs_rebuild_load_tree Lucas Karpinski
2026-03-09 16:38 ` [PATCH v2 2/5] erofs-utils: lib: add helper function erofs_uuid_unparse_as_tag Lucas Karpinski
@ 2026-03-09 16:38 ` Lucas Karpinski
2026-03-09 16:38 ` [PATCH v2 4/5] erofs-utils: mfks: add rebuild FULLDATA for combined EROFS images Lucas Karpinski
2026-03-09 16:38 ` [PATCH v2 5/5] erofs-utils: manpages: update to reflect fulldata support Lucas Karpinski
4 siblings, 0 replies; 7+ messages in thread
From: Lucas Karpinski @ 2026-03-09 16:38 UTC (permalink / raw)
To: linux-erofs; +Cc: jcalmels, Lucas Karpinski
Change erofs_importer_flush_all() to use max(). This allows callers to
pre-allocate space for data beyond the metadata region.
Signed-off-by: Lucas Karpinski <lkarpinski@nvidia.com>
---
lib/importer.c | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/lib/importer.c b/lib/importer.c
index 26c86a0..56c8ea8 100644
--- a/lib/importer.c
+++ b/lib/importer.c
@@ -126,8 +126,9 @@ int erofs_importer_flush_all(struct erofs_importer *im)
fsalignblks = im->params->fsalignblks ?
roundup_pow_of_two(im->params->fsalignblks) : 1;
- sbi->primarydevice_blocks = roundup(erofs_mapbh(sbi->bmgr, NULL),
- fsalignblks);
+ sbi->primarydevice_blocks = max_t(erofs_blk_t, sbi->primarydevice_blocks,
+ roundup(erofs_mapbh(sbi->bmgr, NULL),
+ fsalignblks));
err = erofs_write_device_table(sbi);
if (err)
return err;
--
Git-155)
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH v2 4/5] erofs-utils: mfks: add rebuild FULLDATA for combined EROFS images
2026-03-09 16:38 [PATCH v2 0/5] erofs-utils: implement the FULLDATA rebuild mode Lucas Karpinski
` (2 preceding siblings ...)
2026-03-09 16:38 ` [PATCH v2 3/5] erofs-utils: lib: preserve primarydevice_blocks if already larger Lucas Karpinski
@ 2026-03-09 16:38 ` Lucas Karpinski
2026-03-10 10:24 ` Gao Xiang
2026-03-09 16:38 ` [PATCH v2 5/5] erofs-utils: manpages: update to reflect fulldata support Lucas Karpinski
4 siblings, 1 reply; 7+ messages in thread
From: Lucas Karpinski @ 2026-03-09 16:38 UTC (permalink / raw)
To: linux-erofs; +Cc: jcalmels, Lucas Karpinski
This patch introduces experimental support for merging multiple source
images in mkfs. Each source image becomes a directory directly under root
and keeps its UUID stored as a device table tag. The raw block data from
each source is copied using erofs_copy_file_range. We preserve the file
metadata and layout (FLAT_PLAIN and FLAT_INLINE). Symlink paths are handled
by reading and copy link targets.
This does not yet support chunk-based files at this time or compressed
images.
Signed-off-by: Lucas Karpinski <lkarpinski@nvidia.com>
---
lib/cache.c | 6 +++
lib/liberofs_cache.h | 1 +
lib/liberofs_rebuild.h | 4 ++
lib/rebuild.c | 112 +++++++++++++++++++++++++++++++++++++++++++++++--
mkfs/main.c | 44 ++++++++++++++-----
5 files changed, 154 insertions(+), 13 deletions(-)
diff --git a/lib/cache.c b/lib/cache.c
index 4c7c386..49742bc 100644
--- a/lib/cache.c
+++ b/lib/cache.c
@@ -544,6 +544,12 @@ erofs_blk_t erofs_total_metablocks(struct erofs_bufmgr *bmgr)
return bmgr->metablkcnt;
}
+void erofs_bset_tail(struct erofs_bufmgr *bmgr, erofs_blk_t blkaddr)
+{
+ if (blkaddr > bmgr->tail_blkaddr)
+ bmgr->tail_blkaddr = blkaddr;
+}
+
void erofs_buffer_exit(struct erofs_bufmgr *bmgr)
{
DBG_BUGON(__erofs_bflush(bmgr, NULL, true));
diff --git a/lib/liberofs_cache.h b/lib/liberofs_cache.h
index baac609..55e8f25 100644
--- a/lib/liberofs_cache.h
+++ b/lib/liberofs_cache.h
@@ -138,6 +138,7 @@ int erofs_bflush(struct erofs_bufmgr *bmgr,
struct erofs_buffer_block *bb);
void erofs_bdrop(struct erofs_buffer_head *bh, bool tryrevoke);
+void erofs_bset_tail(struct erofs_bufmgr *bmgr, erofs_blk_t blkaddr);
void erofs_buffer_exit(struct erofs_bufmgr *bmgr);
#ifdef __cplusplus
diff --git a/lib/liberofs_rebuild.h b/lib/liberofs_rebuild.h
index d8c4c8a..fba7f39 100644
--- a/lib/liberofs_rebuild.h
+++ b/lib/liberofs_rebuild.h
@@ -17,6 +17,10 @@ int erofs_rebuild_load_tree(struct erofs_inode *root, struct erofs_sb_info *sbi,
enum erofs_rebuild_datamode mode,
erofs_blk_t uniaddr_offset);
+int erofs_rebuild_copy_src(struct erofs_sb_info *sbi,
+ struct erofs_sb_info *src,
+ struct erofs_device_info *dev);
+
int erofs_rebuild_load_basedir(struct erofs_inode *dir, u64 *nr_subdirs,
unsigned int *i_nlink);
#endif
diff --git a/lib/rebuild.c b/lib/rebuild.c
index 7e62bc9..451307a 100644
--- a/lib/rebuild.c
+++ b/lib/rebuild.c
@@ -14,8 +14,10 @@
#include "erofs/xattr.h"
#include "erofs/blobchunk.h"
#include "erofs/internal.h"
+#include "erofs/io.h"
#include "liberofs_rebuild.h"
#include "liberofs_uuid.h"
+#include "liberofs_cache.h"
#ifdef HAVE_LINUX_AUFS_TYPE_H
#include <linux/aufs_type.h>
@@ -221,9 +223,60 @@ err:
return ret;
}
+static int erofs_rebuild_write_full_data(struct erofs_inode *inode,
+ erofs_blk_t uniaddr_offset)
+{
+ struct erofs_sb_info *src_sbi = inode->sbi;
+ int err = 0;
+
+ if (inode->datalayout == EROFS_INODE_FLAT_PLAIN) {
+ if (inode->u.i_blkaddr != EROFS_NULL_ADDR)
+ inode->u.i_blkaddr += uniaddr_offset;
+ } else if (inode->datalayout == EROFS_INODE_FLAT_INLINE) {
+ erofs_blk_t nblocks = erofs_blknr(src_sbi, inode->i_size);
+ unsigned int inline_size = inode->i_size % erofs_blksiz(src_sbi);
+
+ if (nblocks > 0 && inode->u.i_blkaddr != EROFS_NULL_ADDR)
+ inode->u.i_blkaddr += uniaddr_offset;
+
+ inode->idata_size = inline_size;
+ if (inline_size > 0) {
+ struct erofs_vfile vf;
+ erofs_off_t tail_offset = erofs_pos(src_sbi, nblocks);
+
+ inode->idata = malloc(inline_size);
+ if (!inode->idata)
+ return -ENOMEM;
+ err = erofs_iopen(&vf, inode);
+ if (err) {
+ free(inode->idata);
+ inode->idata = NULL;
+ return err;
+ }
+ err = erofs_pread(&vf, inode->idata, inline_size,
+ tail_offset);
+ if (err) {
+ free(inode->idata);
+ inode->idata = NULL;
+ return err;
+ }
+ }
+ } else if (inode->datalayout == EROFS_INODE_CHUNK_BASED) {
+ erofs_err("chunk-based files not yet supported: %s",
+ inode->i_srcpath);
+ err = -EOPNOTSUPP;
+ } else if (is_inode_layout_compression(inode)) {
+ erofs_err("compressed files not yet supported: %s",
+ inode->i_srcpath);
+ err = -EOPNOTSUPP;
+ }
+ return err;
+}
+
static int erofs_rebuild_update_inode(struct erofs_sb_info *dst_sb,
struct erofs_inode *inode,
- enum erofs_rebuild_datamode datamode)
+ enum erofs_rebuild_datamode datamode,
+ erofs_blk_t uniaddr_offset)
{
int err = 0;
@@ -265,6 +318,8 @@ static int erofs_rebuild_update_inode(struct erofs_sb_info *dst_sb,
err = erofs_rebuild_write_blob_index(dst_sb, inode);
else if (datamode == EROFS_REBUILD_DATA_RESVSP)
inode->datasource = EROFS_INODE_DATA_SOURCE_RESVSP;
+ else if (datamode == EROFS_REBUILD_DATA_FULL)
+ err = erofs_rebuild_write_full_data(inode, uniaddr_offset);
else
err = -EOPNOTSUPP;
break;
@@ -387,7 +442,8 @@ static int erofs_rebuild_dirent_iter(struct erofs_dir_context *ctx)
inode->i_nlink = 1;
ret = erofs_rebuild_update_inode(&g_sbi, inode,
- rctx->datamode);
+ rctx->datamode,
+ rctx->uniaddr_offset);
if (ret) {
erofs_iput(inode);
goto out;
@@ -425,6 +481,7 @@ int erofs_rebuild_load_tree(struct erofs_inode *root, struct erofs_sb_info *sbi,
{
struct erofs_inode inode = {};
struct erofs_rebuild_dir_context ctx;
+ struct erofs_inode *mergedir;
char uuid_str[37];
char *fsid = sbi->devname;
int ret;
@@ -447,16 +504,19 @@ int erofs_rebuild_load_tree(struct erofs_inode *root, struct erofs_sb_info *sbi,
erofs_err("failed to read root inode of %s", fsid);
return ret;
}
+
+ mergedir = root;
inode.i_srcpath = strdup("/");
ctx = (struct erofs_rebuild_dir_context) {
.ctx.dir = &inode,
.ctx.cb = erofs_rebuild_dirent_iter,
- .mergedir = root,
+ .mergedir = mergedir,
.datamode = mode,
.uniaddr_offset = uniaddr_offset,
};
ret = erofs_iterate_dir(&ctx.ctx, false);
+
free(inode.i_srcpath);
return ret;
}
@@ -556,3 +616,49 @@ int erofs_rebuild_load_basedir(struct erofs_inode *dir, u64 *nr_subdirs,
};
return erofs_iterate_dir(&ctx.ctx, false);
}
+
+int erofs_rebuild_copy_src(struct erofs_sb_info *sbi,
+ struct erofs_sb_info *src,
+ struct erofs_device_info *dev)
+{
+ erofs_blk_t cur = sbi->primarydevice_blocks;
+ u64 src_off = 0, dst_off, len;
+ int src_fd, dst_fd;
+ int ret;
+
+ ret = erofs_read_superblock(src);
+ if (ret) {
+ erofs_err("failed to read superblock of %s: %s",
+ src->devname, erofs_strerror(ret));
+ return ret;
+ }
+
+ dev->blocks = src->primarydevice_blocks;
+ dev->uniaddr = cur;
+
+ erofs_info("Copying %s: %u blocks at unified address %u",
+ src->devname, dev->blocks, cur);
+
+ src_fd = src->bdev.fd;
+ dst_fd = sbi->bdev.fd;
+ if (src_fd < 0 || dst_fd < 0) {
+ erofs_err("failed to get file descriptors");
+ return -EINVAL;
+ }
+ dst_off = erofs_pos(sbi, cur);
+ len = erofs_pos(src, dev->blocks);
+ while (len > 0) {
+ ssize_t copied = erofs_copy_file_range(src_fd, &src_off,
+ dst_fd, &dst_off, len);
+ if (copied < 0) {
+ erofs_err("failed to copy data from %s: %s",
+ src->devname, erofs_strerror(-copied));
+ return copied;
+ }
+ if (copied == 0)
+ break;
+ len -= copied;
+ }
+ sbi->primarydevice_blocks += dev->blocks;
+ return 0;
+}
diff --git a/mkfs/main.c b/mkfs/main.c
index 48da20f..4ac835f 100644
--- a/mkfs/main.c
+++ b/mkfs/main.c
@@ -15,9 +15,11 @@
#include <getopt.h>
#include "erofs/config.h"
#include "erofs/print.h"
+#include "erofs/io.h"
#include "erofs/importer.h"
#include "erofs/diskbuf.h"
#include "erofs/inode.h"
+#include "erofs/dir.h"
#include "erofs/tar.h"
#include "erofs/dedupe.h"
#include "erofs/xattr.h"
@@ -30,6 +32,7 @@
#include "../lib/liberofs_metabox.h"
#include "../lib/liberofs_oci.h"
#include "../lib/liberofs_private.h"
+#include "../lib/liberofs_cache.h"
#include "../lib/liberofs_rebuild.h"
#include "../lib/liberofs_s3.h"
#include "../lib/liberofs_uuid.h"
@@ -1717,7 +1720,7 @@ static int erofs_mkfs_rebuild_load_trees(struct erofs_inode *root)
struct erofs_sb_info *src;
unsigned int extra_devices = 0;
erofs_blk_t nblocks;
- int ret, idx;
+ int ret, idx = 0;
enum erofs_rebuild_datamode datamode;
switch (dataimport_mode) {
@@ -1734,9 +1737,33 @@ static int erofs_mkfs_rebuild_load_trees(struct erofs_inode *root)
return -EINVAL;
}
+ if (datamode != EROFS_REBUILD_DATA_RESVSP) {
+ ret = erofs_mkfs_init_devices(&g_sbi, rebuild_src_count);
+ if (ret) {
+ erofs_err("failed to initialize devices: %s",
+ erofs_strerror(ret));
+ return ret;
+ }
+ devs = g_sbi.devs;
+ }
+
list_for_each_entry(src, &rebuild_src_list, list) {
+ erofs_blk_t uniaddr = 0;
+
+ if (datamode == EROFS_REBUILD_DATA_FULL) {
+ /* Copy source data blocks */
+ ret = erofs_rebuild_copy_src(&g_sbi, src, &devs[idx]);
+ if (ret)
+ return ret;
+
+ uniaddr = devs[idx].uniaddr;
+
+ /* Advance buffer manager past copied data */
+ erofs_bset_tail(g_sbi.bmgr, g_sbi.primarydevice_blocks);
+ }
+
src->xamgr = g_sbi.xamgr;
- ret = erofs_rebuild_load_tree(root, src, datamode, 0);
+ ret = erofs_rebuild_load_tree(root, src, datamode, uniaddr);
src->xamgr = NULL;
if (ret) {
erofs_err("failed to load %s", src->devname);
@@ -1748,9 +1775,10 @@ static int erofs_mkfs_rebuild_load_trees(struct erofs_inode *root)
return -EOPNOTSUPP;
}
extra_devices += src->extra_devices;
+ idx++;
}
- if (datamode != EROFS_REBUILD_DATA_BLOB_INDEX)
+ if (datamode == EROFS_REBUILD_DATA_RESVSP)
return 0;
/* Each blob has either no extra device or only one device for TarFS */
@@ -1760,11 +1788,6 @@ static int erofs_mkfs_rebuild_load_trees(struct erofs_inode *root)
return -EOPNOTSUPP;
}
- ret = erofs_mkfs_init_devices(&g_sbi, rebuild_src_count);
- if (ret)
- return ret;
-
- devs = g_sbi.devs;
list_for_each_entry(src, &rebuild_src_list, list) {
u8 *tag = NULL;
@@ -1775,14 +1798,15 @@ static int erofs_mkfs_rebuild_load_trees(struct erofs_inode *root)
tag = src->devs[0].tag;
} else {
nblocks = src->primarydevice_blocks;
- devs[idx].src_path = strdup(src->devname);
+ if (datamode == EROFS_REBUILD_DATA_BLOB_INDEX)
+ devs[idx].src_path = strdup(src->devname);
}
devs[idx].blocks = nblocks;
if (tag && *tag)
memcpy(devs[idx].tag, tag, sizeof(devs[0].tag));
else
/* convert UUID of the source image to a hex string */
- erofs_uuid_unparse_as_tag(src->uuid, (char *)g_sbi.devs[idx].tag);
+ erofs_uuid_unparse_as_tag(src->uuid, (char *)devs[idx].tag);
}
return 0;
}
--
Git-155)
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH v2 5/5] erofs-utils: manpages: update to reflect fulldata support
2026-03-09 16:38 [PATCH v2 0/5] erofs-utils: implement the FULLDATA rebuild mode Lucas Karpinski
` (3 preceding siblings ...)
2026-03-09 16:38 ` [PATCH v2 4/5] erofs-utils: mfks: add rebuild FULLDATA for combined EROFS images Lucas Karpinski
@ 2026-03-09 16:38 ` Lucas Karpinski
4 siblings, 0 replies; 7+ messages in thread
From: Lucas Karpinski @ 2026-03-09 16:38 UTC (permalink / raw)
To: linux-erofs; +Cc: jcalmels, Lucas Karpinski
Specify that data (fulldata) mode is now supporting alongside rsvp when
using --clean={data|rsvp}.
Signed-off-by: Lucas Karpinski <lkarpinski@nvidia.com>
---
man/mkfs.erofs.1 | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/man/mkfs.erofs.1 b/man/mkfs.erofs.1
index a102e65..65ec807 100644
--- a/man/mkfs.erofs.1
+++ b/man/mkfs.erofs.1
@@ -229,7 +229,7 @@ Only \fBdata\fR is supported. \fBrvsp\fR and \fB0\fR will be ignored.
Note that \fBrvsp\fR takes precedence over \fB--tar=i\fR or \fB--tar=headerball\fR.
.TP
.I Rebuild mode
-Only \fBrvsp\fR is supported.
+\fBdata\fR and \fBrvsp\fR are supported.
.TP
.I S3 source (\fB\-\-s3\fR)
\fBdata\fR and \fB0\fR are supported.
@@ -521,6 +521,11 @@ source images, which act as external blob devices. This creates a compact
metadata layer suitable for layered filesystem scenarios, similar to container
image layers.
.TP
+.I data mode
+\fB\-\-clean=data\fR: Import complete file data from all source images into
+the destination image, producing a fully self-contained EROFS image that does
+not depend on external blob devices.
+.TP
.I rvsp mode
\fB\-\-clean=rvsp\fR or \fB\-\-incremental=rvsp\fR: Reserve space for file
data without copying actual content, useful for creating sparse images.
--
Git-155)
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH v2 4/5] erofs-utils: mfks: add rebuild FULLDATA for combined EROFS images
2026-03-09 16:38 ` [PATCH v2 4/5] erofs-utils: mfks: add rebuild FULLDATA for combined EROFS images Lucas Karpinski
@ 2026-03-10 10:24 ` Gao Xiang
0 siblings, 0 replies; 7+ messages in thread
From: Gao Xiang @ 2026-03-10 10:24 UTC (permalink / raw)
To: Lucas Karpinski; +Cc: linux-erofs, jcalmels
Hi Lucas,
On Mon, Mar 09, 2026 at 12:38:20PM -0400, Lucas Karpinski wrote:
> This patch introduces experimental support for merging multiple source
> images in mkfs. Each source image becomes a directory directly under root
> and keeps its UUID stored as a device table tag. The raw block data from
> each source is copied using erofs_copy_file_range. We preserve the file
> metadata and layout (FLAT_PLAIN and FLAT_INLINE). Symlink paths are handled
> by reading and copy link targets.
>
> This does not yet support chunk-based files at this time or compressed
> images.
>
> Signed-off-by: Lucas Karpinski <lkarpinski@nvidia.com>
Thanks for your effort, I finally get the time to look into this
new feature.
So you'd like to cleanly rebuild a new filesystem with given sub
filesystems?
I think first, yes, that is what `--clean=data` is used instead.
But I think uniaddr shouldn't be used like this (uniaddr is used to
give a flat mapping for multiple blobs, in addition to the device id +
offset ones, which is mainly used for compressed data multi blob
support), instead, I think we need to find a way to wrap up the data
source into a valid vfile (so that erofs_pread can be called to get the
source data) , and make erofs_mkfs_job_write_file() to write
(un)compressed data from its source instead.
Thanks,
Gao Xiang
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2026-03-10 10:24 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-03-09 16:38 [PATCH v2 0/5] erofs-utils: implement the FULLDATA rebuild mode Lucas Karpinski
2026-03-09 16:38 ` [PATCH v2 1/5] erofs-utils: lib: pass uniaddr_offset to erofs_rebuild_load_tree Lucas Karpinski
2026-03-09 16:38 ` [PATCH v2 2/5] erofs-utils: lib: add helper function erofs_uuid_unparse_as_tag Lucas Karpinski
2026-03-09 16:38 ` [PATCH v2 3/5] erofs-utils: lib: preserve primarydevice_blocks if already larger Lucas Karpinski
2026-03-09 16:38 ` [PATCH v2 4/5] erofs-utils: mfks: add rebuild FULLDATA for combined EROFS images Lucas Karpinski
2026-03-10 10:24 ` Gao Xiang
2026-03-09 16:38 ` [PATCH v2 5/5] erofs-utils: manpages: update to reflect fulldata support Lucas Karpinski
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox