[RFC] erofs-utils: addressing the 16 GiB xattr prefix offset limit in incremental builds

[RFC] erofs-utils: addressing the 16 GiB xattr prefix offset limit in incremental builds

[lanbingying]

[-- Attachment #1: Type: text/plain, Size: 2857 bytes --]

Hi,
xattr_prefix_start is a 32-bit field storing an offset in four-byte units. When an incremental build places a new long xattr name prefix table at or beyond 16 GiB, the offset cannot be represented and mkfs.erofs rejects the build.
Before working on patches, I would like to discuss several possible solutions:

 * 
Store the prefix table in the packed inode.
Use the existing format support so that the table is addressed by a logical offset within the packed inode rather than a physical offset in the image. This avoids a format extension, although the table’s logical offset remains subject to the same limit, and existing references to fragment data must be preserved.

 * 
Reserve space for the prefix table near the beginning of the image.
Reserve sufficient space at a low offset when creating the image, with a feature flag identifying this layout. Subsequent incremental builds would use the reserved region, while images without the flag would retain the existing behavior. Existing images without reserved space would require migration or rebuilding to adopt this layout.

 * 
Extend the offset using reserved superblock space.
Add an upper field for xattr_prefix_start in the superblock’s reserved space and combine it with the existing field. A new incompat feature would enable the extended interpretation; without it, readers would retain the current interpretation. This would allow the table to remain appended to the image, but require reader updates.

 * 
Reserve space for the prefix table within the packed inode.
Building on option 1, reserve a dedicated region at a low logical offset within the packed inode when creating the image. Subsequent incremental builds could use this region instead of appending a new table after the growing fragment data. A feature flag could identify the reservation; existing images without such a region would need a migration or rebuild path.
All options should preserve the interpretation of existing images. Where a new interpretation is introduced, readers that do not support it should reject the feature rather than read the table from an incorrect location.
Separately, we could reuse the existing prefix table when the required prefix entries and their indices match those in the base image, provided the original table remains valid and accessible. For example, newly added files may use existing long prefixes even if their xattr values differ, so no table update is needed. This would avoid writing another identical table at the end of the image. However, it is an optimization rather than a complete solution: the addressing limit still needs to be handled when new prefixes are added or the table must be relocated.
Which approach would best fit EROFS’s long-term format and maintenance goals? Are there other options or constraints that should be considered?
Thanks,
Bingying

[-- Attachment #2: Type: text/html, Size: 3960 bytes --]

Re: [RFC] erofs-utils: addressing the 16 GiB xattr prefix offset limit in incremental builds

[Gao Xiang]

Hi,

On Fri, Oct 30, 2026 at 04:32:48PM +0800, lanbingying wrote:
> Hi,
> xattr_prefix_start is a 32-bit field storing an offset in four-byte units. When an incremental build places a new long xattr name prefix table at or beyond 16 GiB, the offset cannot be represented and mkfs.erofs rejects the build.
> Before working on patches, I would like to discuss several possible solutions:

Thanks for discussing this.

I wonder in which case the long xattr name prefix table needs to be
extended?

> 
>  * 
> Store the prefix table in the packed inode.
> Use the existing format support so that the table is addressed by a logical offset within the packed inode rather than a physical offset in the image. This avoids a format extension, although the table’s logical offset remains subject to the same limit, and existing references to fragment data must be preserved.

I think we can this one.  Currently, it seems that mkfs.erofs should
support keeping long xattr name prefix table in the packed inode,
but I'm not sure if it supports incremental builds.

> 
>  * 
> Reserve space for the prefix table near the beginning of the image.
> Reserve sufficient space at a low offset when creating the image, with a feature flag identifying this layout. Subsequent incremental builds would use the reserved region, while images without the flag would retain the existing behavior. Existing images without reserved space would require migration or rebuilding to adopt this layout.

I think this one is okay too, and it's doable, maybe even without a new
feature flag. 

> 
>  * 
> Extend the offset using reserved superblock space.
> Add an upper field for xattr_prefix_start in the superblock’s reserved space and combine it with the existing field. A new incompat feature would enable the extended interpretation; without it, readers would retain the current interpretation. This would allow the table to remain appended to the image, but require reader updates.

I don't think it needs to be done, because long xattr name prefix
table has 128 entries at most. we could just reserve at the beginning.

and long xattr name prefix table table should not be vulnerable to
change a lot when doing incremental builds.

> 
>  * 
> Reserve space for the prefix table within the packed inode.

This one is doable too.

Thanks,
Gao Xiang

> Building on option 1, reserve a dedicated region at a low logical offset within the packed inode when creating the image. Subsequent incremental builds could use this region instead of appending a new table after the growing fragment data. A feature flag could identify the reservation; existing images without such a region would need a migration or rebuild path.
> All options should preserve the interpretation of existing images. Where a new interpretation is introduced, readers that do not support it should reject the feature rather than read the table from an incorrect location.
> Separately, we could reuse the existing prefix table when the required prefix entries and their indices match those in the base image, provided the original table remains valid and accessible. For example, newly added files may use existing long prefixes even if their xattr values differ, so no table update is needed. This would avoid writing another identical table at the end of the image. However, it is an optimization rather than a complete solution: the addressing limit still needs to be handled when new prefixes are added or the table must be relocated.
> Which approach would best fit EROFS’s long-term format and maintenance goals? Are there other options or constraints that should be considered?
> Thanks,
> Bingying

Re: Re: [RFC] erofs-utils: addressing the 16 GiB xattr prefix offset limit in incremental builds

[lanbingying]

[-- Attachment #1: Type: text/plain, Size: 7066 bytes --]

Hi,
Thanks for your feedback.
Regarding when the long xattr name prefix table needs to be extended, my earlier description was imprecise. The failure I encountered comes from rewriting the prefix table at the end of the image during an incremental build. Even if no prefix entries are added, the new location can exceed the range representable by `xattr_prefix_start`, causing the build to fail. Reusing the existing table when its entries and indices remain unchanged would address this case.
To also accommodate cases where new prefix entries are needed, I would like to discuss the following reservation approach:
1. Reserve 64 KiB during a full build.
 Reserve a fixed 64 KiB region near the beginning of the image, after the superblock and any required configuration data, and point `xattr_prefix_start` to it. Subsequent incremental builds could update the table within this region while preserving the meaning of existing prefix indices.
 The rationale for 64 KiB is that the table currently supports at most 128 entries. Using a conservative bound of 255 bytes of prefix content, a 2-byte length field, and a 1-byte `base_index`, each entry occupies at most 260 bytes after 4-byte alignment. The entire table therefore requires at most 33,280 bytes, or 32.5 KiB. A fixed 64 KiB reservation covers this limit with some headroom.
2. Record a reservation marker in a reserved superblock field.
 I agree that locating the table does not require a new feature flag: the kernel can continue using `xattr_prefix_start` as before. However, subsequent mkfs runs need to distinguish an existing table from a table backed by a full 64 KiB reservation, to avoid overwriting adjacent data.
 I therefore propose a marker for the build tool to recognize the reservation. Its location would be given by `xattr_prefix_start`, and its capacity would be fixed at 64 KiB, so no separate reservation address or size would be needed.
3. Reuse the existing table when an older image has no reservation marker.
 If the existing entries and their indices satisfy the current build, keep referencing the old table without rewriting it.
4. Try to establish a reservation at the end of the image.
 If a complete 64 KiB region fits between the aligned append position and the 16 GiB boundary, reserve it there and update the offset and marker.
 Otherwise, return an error asking the user to perform a full rebuild with a reservation-capable tool, producing a new base image before continuing incremental builds. This would be an explicit migration step so that users can schedule its cost.
This would preserve the interpretation of existing images without changing the kernel’s table lookup logic. Older images could adopt the approach through table reuse, an appended reservation, or a one-time full rebuild.
Does this approach seem reasonable? In particular, would a marker in the superblock’s reserved space be appropriate for identifying the reservation, or is there a better existing mechanism?
Thanks, 
Bingying
------------------------------------------------------------------
发件人：Gao Xiang <xiang@kernel.org>
发送时间：2026年10月30日(周五) 21:11
收件人：lanbingying<lanbingying.lby@linux.alibaba.com>
抄　送："linux-erofs"<linux-erofs@lists.ozlabs.org>; Gao Xiang<xiang@kernel.org>; Chao Yu<chao@kernel.org>
主　题：Re: [RFC] erofs-utils: addressing the 16 GiB xattr prefix offset limit in incremental builds
Hi,
On Fri, Oct 30, 2026 at 04:32:48PM +0800, lanbingying wrote:
> Hi,
> xattr_prefix_start is a 32-bit field storing an offset in four-byte units. When an incremental build places a new long xattr name prefix table at or beyond 16 GiB, the offset cannot be represented and mkfs.erofs rejects the build.
> Before working on patches, I would like to discuss several possible solutions:
Thanks for discussing this.
I wonder in which case the long xattr name prefix table needs to be
extended?
> 
> * 
> Store the prefix table in the packed inode.
> Use the existing format support so that the table is addressed by a logical offset within the packed inode rather than a physical offset in the image. This avoids a format extension, although the table’s logical offset remains subject to the same limit, and existing references to fragment data must be preserved.
I think we can this one. Currently, it seems that mkfs.erofs should
support keeping long xattr name prefix table in the packed inode,
but I'm not sure if it supports incremental builds.
> 
> * 
> Reserve space for the prefix table near the beginning of the image.
> Reserve sufficient space at a low offset when creating the image, with a feature flag identifying this layout. Subsequent incremental builds would use the reserved region, while images without the flag would retain the existing behavior. Existing images without reserved space would require migration or rebuilding to adopt this layout.
I think this one is okay too, and it's doable, maybe even without a new
feature flag. 
> 
> * 
> Extend the offset using reserved superblock space.
> Add an upper field for xattr_prefix_start in the superblock’s reserved space and combine it with the existing field. A new incompat feature would enable the extended interpretation; without it, readers would retain the current interpretation. This would allow the table to remain appended to the image, but require reader updates.
I don't think it needs to be done, because long xattr name prefix
table has 128 entries at most. we could just reserve at the beginning.
and long xattr name prefix table table should not be vulnerable to
change a lot when doing incremental builds.
> 
> * 
> Reserve space for the prefix table within the packed inode.
This one is doable too.
Thanks,
Gao Xiang
> Building on option 1, reserve a dedicated region at a low logical offset within the packed inode when creating the image. Subsequent incremental builds could use this region instead of appending a new table after the growing fragment data. A feature flag could identify the reservation; existing images without such a region would need a migration or rebuild path.
> All options should preserve the interpretation of existing images. Where a new interpretation is introduced, readers that do not support it should reject the feature rather than read the table from an incorrect location.
> Separately, we could reuse the existing prefix table when the required prefix entries and their indices match those in the base image, provided the original table remains valid and accessible. For example, newly added files may use existing long prefixes even if their xattr values differ, so no table update is needed. This would avoid writing another identical table at the end of the image. However, it is an optimization rather than a complete solution: the addressing limit still needs to be handled when new prefixes are added or the table must be relocated.
> Which approach would best fit EROFS’s long-term format and maintenance goals? Are there other options or constraints that should be considered?
> Thanks,
> Bingying

[-- Attachment #2: Type: text/html, Size: 9273 bytes --]