public inbox for linux-erofs@ozlabs.org
 help / color / mirror / Atom feed
* [PATCH] blobchunk: fix 48-bit format detection to use final remapped block addresses
@ 2026-02-24  5:57 puneeth_aditya_5656
  2026-02-24  6:44 ` Gao Xiang
                   ` (4 more replies)
  0 siblings, 5 replies; 9+ messages in thread
From: puneeth_aditya_5656 @ 2026-02-24  5:57 UTC (permalink / raw)
  To: linux-erofs; +Cc: hsiangkao, puneeth_aditya_5656

---
 lib/blobchunk.c | 17 +++++++++++++----
 1 file changed, 13 insertions(+), 4 deletions(-)

diff --git a/lib/blobchunk.c b/lib/blobchunk.c
index a051904..9b8112b 100644
--- a/lib/blobchunk.c
+++ b/lib/blobchunk.c
@@ -154,6 +154,19 @@ int erofs_write_chunk_indexes(struct erofs_inode *inode, struct erofs_vfile *vf,
 		unit = EROFS_BLOCK_MAP_ENTRY_SIZE;
 
 	chunkblks = 1ULL << (inode->u.chunkformat & EROFS_CHUNK_FORMAT_BLKBITS_MASK);
+
+	/* check if any chunk lands above 32-bit range once remapped_base is applied */
+	for (src = 0; src < inode->extent_isize / unit * sizeof(void *);
+	     src += sizeof(void *)) {
+		struct erofs_blobchunk *chunk = *(void **)(inode->chunkindexes + src);
+
+		if (chunk->blkaddr != EROFS_NULL_ADDR && !chunk->device_id &&
+		    remapped_base + chunk->blkaddr > UINT32_MAX) {
+			inode->u.chunkformat |= EROFS_CHUNK_FORMAT_48BIT;
+			break;
+		}
+	}
+
 	_48bit = inode->u.chunkformat & EROFS_CHUNK_FORMAT_48BIT;
 	for (dst = src = 0; dst < inode->extent_isize;
 	     src += sizeof(void *), dst += unit) {
@@ -380,10 +393,6 @@ int erofs_blob_write_chunked_file(struct erofs_inode *inode, int fd,
 			goto err;
 		}
 
-		/* FIXME! `chunk->blkaddr` is not the final blkaddr here */
-		if (chunk->blkaddr != EROFS_NULL_ADDR &&
-		    chunk->blkaddr >= UINT32_MAX)
-			inode->u.chunkformat |= EROFS_CHUNK_FORMAT_48BIT;
 		if (!erofs_blob_can_merge(sbi, lastch, chunk)) {
 			erofs_update_minextblks(sbi, interval_start, pos,
 						&minextblks);
-- 
2.52.0



^ permalink raw reply related	[flat|nested] 9+ messages in thread

* Re: [PATCH] blobchunk: fix 48-bit format detection to use final remapped block addresses
  2026-02-24  5:57 [PATCH] blobchunk: fix 48-bit format detection to use final remapped block addresses puneeth_aditya_5656
@ 2026-02-24  6:44 ` Gao Xiang
  2026-02-24 16:37 ` [PATCH v2] erofs-utils: lib: fix 48bit addressing detection for chunk-based format puneeth_aditya_5656
                   ` (3 subsequent siblings)
  4 siblings, 0 replies; 9+ messages in thread
From: Gao Xiang @ 2026-02-24  6:44 UTC (permalink / raw)
  To: puneeth_aditya_5656, linux-erofs

Hi puneeth_aditya_5656,

Could you format the commit message instead of leaving
the commit message empty (maximum 72 chars per line).

The subject needs to be fixed as:

erofs-utils: lib: fix 48bit addressing detection for chunk-based format

On 2026/2/24 13:57, puneeth_aditya_5656 wrote:
> ---
>   lib/blobchunk.c | 17 +++++++++++++----
>   1 file changed, 13 insertions(+), 4 deletions(-)
> 
> diff --git a/lib/blobchunk.c b/lib/blobchunk.c
> index a051904..9b8112b 100644
> --- a/lib/blobchunk.c
> +++ b/lib/blobchunk.c
> @@ -154,6 +154,19 @@ int erofs_write_chunk_indexes(struct erofs_inode *inode, struct erofs_vfile *vf,
>   		unit = EROFS_BLOCK_MAP_ENTRY_SIZE;
>   
>   	chunkblks = 1ULL << (inode->u.chunkformat & EROFS_CHUNK_FORMAT_BLKBITS_MASK);
> +
> +	/* check if any chunk lands above 32-bit range once remapped_base is applied */
> +	for (src = 0; src < inode->extent_isize / unit * sizeof(void *);


I think it's too late to adjust inode->u.chunkformat,
see erofs_iflush():

I think you currently just add a new function like:

erofs_inode_fixup_chunkformat() {

	u64 extent_count = inode->extent_isize / unit;

	_48bit = inode->u.chunkformat & EROFS_CHUNK_FORMAT_48BIT;
	if (_48bit)
		return;

	for (src = 0; src < extent_count; ++src) {
		if (chunk->blkaddr == EROFS_NULL_ADDR)
			continue;
		if (chunk->device_id) {
			if (chunk->blkaddr > UINT32_MAX) {
				_48bit = true;
				break;
			}
		} else if (remapped_base + chunk->blkaddr > UINT32_MAX) {
			_48bit = true;
			break;
		}
	}
	if (_48bit)
		inode->u.chunkformat |= EROFS_CHUNK_FORMAT_48BIT;
}

Also if we really would like to fix this, we need considering add
a testcase for this, possibly use `--offset` to make the image
exceeds 32-bit, see experimental-tests.

Thanks,
Gao Xiang


^ permalink raw reply	[flat|nested] 9+ messages in thread

* [PATCH v2] erofs-utils: lib: fix 48bit addressing detection for chunk-based format
  2026-02-24  5:57 [PATCH] blobchunk: fix 48-bit format detection to use final remapped block addresses puneeth_aditya_5656
  2026-02-24  6:44 ` Gao Xiang
@ 2026-02-24 16:37 ` puneeth_aditya_5656
  2026-02-24 17:44   ` Gao Xiang
  2026-02-24 19:10 ` puneeth_aditya_5656
                   ` (2 subsequent siblings)
  4 siblings, 1 reply; 9+ messages in thread
From: puneeth_aditya_5656 @ 2026-02-24 16:37 UTC (permalink / raw)
  To: linux-erofs; +Cc: hsiangkao, puneeth_aditya_5656

The 48-bit chunk format flag was being set inside
erofs_blob_write_chunked_file right after erofs_blob_getchunk returns.
At that point chunk->blkaddr is the chunk's offset in the temporary
blob buffer, not the final image address. The real address is only
known after erofs_mkfs_dump_blobs applies remapped_base.

This means the detection was unreliable in both directions: a chunk
whose blob offset looks large but fits in 32-bits after remapping gets
flagged unnecessarily, and worse, a chunk that lands above UINT32_MAX
after remapping may not get flagged at all, producing a corrupt image.

Fix this by introducing erofs_inode_fixup_chunkformat() which walks
the chunk array after remapped_base is finalized and sets the 48-bit
flag if any chunk address exceeds UINT32_MAX. The fixup is called from
erofs_bh_flush_write_inode before erofs_iflush so that the correct
chunkformat is written into the on-disk inode header. Both blob chunks
(remapped_base + chunk->blkaddr) and device chunks (chunk->blkaddr
directly) are handled.
---
 include/erofs/blobchunk.h |  1 +
 lib/blobchunk.c           | 40 +++++++++++++++++++++++++++++++++++----
 lib/inode.c               |  3 +++
 3 files changed, 40 insertions(+), 4 deletions(-)

diff --git a/include/erofs/blobchunk.h b/include/erofs/blobchunk.h
index ef06773..48fca63 100644
--- a/include/erofs/blobchunk.h
+++ b/include/erofs/blobchunk.h
@@ -16,6 +16,7 @@ extern "C"
 
 struct erofs_blobchunk *erofs_get_unhashed_chunk(unsigned int device_id,
 		erofs_blk_t blkaddr, erofs_off_t sourceoffset);
+void erofs_inode_fixup_chunkformat(struct erofs_inode *inode);
 int erofs_write_chunk_indexes(struct erofs_inode *inode, struct erofs_vfile *vf,
 			      erofs_off_t off);
 int erofs_blob_write_chunked_file(struct erofs_inode *inode, int fd,
diff --git a/lib/blobchunk.c b/lib/blobchunk.c
index a051904..96c161b 100644
--- a/lib/blobchunk.c
+++ b/lib/blobchunk.c
@@ -136,6 +136,42 @@ static int erofs_blob_hashmap_cmp(const void *a, const void *b,
 		      sizeof(ec1->sha256));
 }
 
+void erofs_inode_fixup_chunkformat(struct erofs_inode *inode)
+{
+	unsigned int unit, src;
+	u64 extent_count;
+	bool _48bit;
+
+	if (inode->u.chunkformat & EROFS_CHUNK_FORMAT_INDEXES)
+		unit = sizeof(struct erofs_inode_chunk_index);
+	else
+		unit = EROFS_BLOCK_MAP_ENTRY_SIZE;
+
+	_48bit = inode->u.chunkformat & EROFS_CHUNK_FORMAT_48BIT;
+	if (_48bit)
+		return;
+
+	extent_count = inode->extent_isize / unit;
+	for (src = 0; src < extent_count; ++src) {
+		struct erofs_blobchunk *chunk =
+			*(void **)(inode->chunkindexes + src * sizeof(void *));
+
+		if (chunk->blkaddr == EROFS_NULL_ADDR)
+			continue;
+		if (chunk->device_id) {
+			if (chunk->blkaddr > UINT32_MAX) {
+				_48bit = true;
+				break;
+			}
+		} else if (remapped_base + chunk->blkaddr > UINT32_MAX) {
+			_48bit = true;
+			break;
+		}
+	}
+	if (_48bit)
+		inode->u.chunkformat |= EROFS_CHUNK_FORMAT_48BIT;
+}
+
 int erofs_write_chunk_indexes(struct erofs_inode *inode, struct erofs_vfile *vf,
 			      erofs_off_t off)
 {
@@ -380,10 +416,6 @@ int erofs_blob_write_chunked_file(struct erofs_inode *inode, int fd,
 			goto err;
 		}
 
-		/* FIXME! `chunk->blkaddr` is not the final blkaddr here */
-		if (chunk->blkaddr != EROFS_NULL_ADDR &&
-		    chunk->blkaddr >= UINT32_MAX)
-			inode->u.chunkformat |= EROFS_CHUNK_FORMAT_48BIT;
 		if (!erofs_blob_can_merge(sbi, lastch, chunk)) {
 			erofs_update_minextblks(sbi, interval_start, pos,
 						&minextblks);
diff --git a/lib/inode.c b/lib/inode.c
index 4a214f9..7a1d982 100644
--- a/lib/inode.c
+++ b/lib/inode.c
@@ -908,6 +908,9 @@ static int erofs_bh_flush_write_inode(struct erofs_buffer_head *bh, bool abort)
 
 	DBG_BUGON(inode->bh != bh);
 	if (!abort) {
+		if (inode->datalayout == EROFS_INODE_CHUNK_BASED &&
+		    inode->chunkindexes)
+			erofs_inode_fixup_chunkformat(inode);
 		ret = erofs_iflush(inode);
 		if (ret)
 			return ret;
-- 
2.52.0



^ permalink raw reply related	[flat|nested] 9+ messages in thread

* Re: [PATCH v2] erofs-utils: lib: fix 48bit addressing detection for chunk-based format
  2026-02-24 16:37 ` [PATCH v2] erofs-utils: lib: fix 48bit addressing detection for chunk-based format puneeth_aditya_5656
@ 2026-02-24 17:44   ` Gao Xiang
  0 siblings, 0 replies; 9+ messages in thread
From: Gao Xiang @ 2026-02-24 17:44 UTC (permalink / raw)
  To: puneeth_aditya_5656, linux-erofs



On 2026/2/25 00:37, puneeth_aditya_5656 wrote:
> The 48-bit chunk format flag was being set inside
> erofs_blob_write_chunked_file right after erofs_blob_getchunk returns.
> At that point chunk->blkaddr is the chunk's offset in the temporary
> blob buffer, not the final image address. The real address is only
> known after erofs_mkfs_dump_blobs applies remapped_base.
> 
> This means the detection was unreliable in both directions: a chunk
> whose blob offset looks large but fits in 32-bits after remapping gets
> flagged unnecessarily, and worse, a chunk that lands above UINT32_MAX
> after remapping may not get flagged at all, producing a corrupt image.
> 
> Fix this by introducing erofs_inode_fixup_chunkformat() which walks
> the chunk array after remapped_base is finalized and sets the 48-bit
> flag if any chunk address exceeds UINT32_MAX. The fixup is called from
> erofs_bh_flush_write_inode before erofs_iflush so that the correct
> chunkformat is written into the on-disk inode header. Both blob chunks
> (remapped_base + chunk->blkaddr) and device chunks (chunk->blkaddr
> directly) are handled.

Closes: https://github.com/erofs/erofs-utils/issues/39

Also, a proper Signed-off-by: is needed.

> ---
>   include/erofs/blobchunk.h |  1 +
>   lib/blobchunk.c           | 40 +++++++++++++++++++++++++++++++++++----
>   lib/inode.c               |  3 +++
>   3 files changed, 40 insertions(+), 4 deletions(-)
> 
> diff --git a/include/erofs/blobchunk.h b/include/erofs/blobchunk.h
> index ef06773..48fca63 100644
> --- a/include/erofs/blobchunk.h
> +++ b/include/erofs/blobchunk.h
> @@ -16,6 +16,7 @@ extern "C"
>   
>   struct erofs_blobchunk *erofs_get_unhashed_chunk(unsigned int device_id,
>   		erofs_blk_t blkaddr, erofs_off_t sourceoffset);
> +void erofs_inode_fixup_chunkformat(struct erofs_inode *inode);
>   int erofs_write_chunk_indexes(struct erofs_inode *inode, struct erofs_vfile *vf,
>   			      erofs_off_t off);
>   int erofs_blob_write_chunked_file(struct erofs_inode *inode, int fd,
> diff --git a/lib/blobchunk.c b/lib/blobchunk.c
> index a051904..96c161b 100644
> --- a/lib/blobchunk.c
> +++ b/lib/blobchunk.c
> @@ -136,6 +136,42 @@ static int erofs_blob_hashmap_cmp(const void *a, const void *b,
>   		      sizeof(ec1->sha256));
>   }
>   

...


>   int erofs_write_chunk_indexes(struct erofs_inode *inode, struct erofs_vfile *vf,
>   			      erofs_off_t off)
>   {
> @@ -380,10 +416,6 @@ int erofs_blob_write_chunked_file(struct erofs_inode *inode, int fd,
>   			goto err;
>   		}
>   
> -		/* FIXME! `chunk->blkaddr` is not the final blkaddr here */
> -		if (chunk->blkaddr != EROFS_NULL_ADDR &&
> -		    chunk->blkaddr >= UINT32_MAX)
> -			inode->u.chunkformat |= EROFS_CHUNK_FORMAT_48BIT;
>   		if (!erofs_blob_can_merge(sbi, lastch, chunk)) {
>   			erofs_update_minextblks(sbi, interval_start, pos,
>   						&minextblks);
> diff --git a/lib/inode.c b/lib/inode.c
> index 4a214f9..7a1d982 100644
> --- a/lib/inode.c
> +++ b/lib/inode.c
> @@ -908,6 +908,9 @@ static int erofs_bh_flush_write_inode(struct erofs_buffer_head *bh, bool abort)
>   
>   	DBG_BUGON(inode->bh != bh);
>   	if (!abort) {
> +		if (inode->datalayout == EROFS_INODE_CHUNK_BASED &&
> +		    inode->chunkindexes)
> +			erofs_inode_fixup_chunkformat(inode);

How about moving this into erofs_iflush() rather than here?

Thanks,
Gao Xiang

>   		ret = erofs_iflush(inode);
>   		if (ret)
>   			return ret;



^ permalink raw reply	[flat|nested] 9+ messages in thread

* [PATCH v2] erofs-utils: lib: fix 48bit addressing detection for chunk-based format
  2026-02-24  5:57 [PATCH] blobchunk: fix 48-bit format detection to use final remapped block addresses puneeth_aditya_5656
  2026-02-24  6:44 ` Gao Xiang
  2026-02-24 16:37 ` [PATCH v2] erofs-utils: lib: fix 48bit addressing detection for chunk-based format puneeth_aditya_5656
@ 2026-02-24 19:10 ` puneeth_aditya_5656
  2026-02-24 19:13 ` [PATCH v3] " puneeth_aditya_5656
  2026-02-25  7:39 ` [PATCH v4] " puneeth_aditya_5656
  4 siblings, 0 replies; 9+ messages in thread
From: puneeth_aditya_5656 @ 2026-02-24 19:10 UTC (permalink / raw)
  To: linux-erofs; +Cc: puneeth_aditya_5656

The 48-bit chunk format flag was being set inside
erofs_blob_write_chunked_file right after erofs_blob_getchunk returns.
At that point chunk->blkaddr is the chunk's offset in the temporary
blob buffer, not the final image address. The real address is only
known after erofs_mkfs_dump_blobs applies remapped_base.

This means the detection was unreliable in both directions: a chunk
whose blob offset looks large but fits in 32-bits after remapping gets
flagged unnecessarily, and worse, a chunk that lands above UINT32_MAX
after remapping may not get flagged at all, producing a corrupt image.

Fix this by introducing erofs_inode_fixup_chunkformat() which walks
the chunk array after remapped_base is finalized and sets the 48-bit
flag if any chunk address exceeds UINT32_MAX. The fixup is called from
erofs_bh_flush_write_inode before erofs_iflush so that the correct
chunkformat is written into the on-disk inode header. Both blob chunks
(remapped_base + chunk->blkaddr) and device chunks (chunk->blkaddr
directly) are handled.
---
 include/erofs/blobchunk.h |  1 +
 lib/blobchunk.c           | 40 +++++++++++++++++++++++++++++++++++----
 lib/inode.c               |  3 +++
 3 files changed, 40 insertions(+), 4 deletions(-)

diff --git a/include/erofs/blobchunk.h b/include/erofs/blobchunk.h
index ef06773..48fca63 100644
--- a/include/erofs/blobchunk.h
+++ b/include/erofs/blobchunk.h
@@ -16,6 +16,7 @@ extern "C"
 
 struct erofs_blobchunk *erofs_get_unhashed_chunk(unsigned int device_id,
 		erofs_blk_t blkaddr, erofs_off_t sourceoffset);
+void erofs_inode_fixup_chunkformat(struct erofs_inode *inode);
 int erofs_write_chunk_indexes(struct erofs_inode *inode, struct erofs_vfile *vf,
 			      erofs_off_t off);
 int erofs_blob_write_chunked_file(struct erofs_inode *inode, int fd,
diff --git a/lib/blobchunk.c b/lib/blobchunk.c
index a051904..96c161b 100644
--- a/lib/blobchunk.c
+++ b/lib/blobchunk.c
@@ -136,6 +136,42 @@ static int erofs_blob_hashmap_cmp(const void *a, const void *b,
 		      sizeof(ec1->sha256));
 }
 
+void erofs_inode_fixup_chunkformat(struct erofs_inode *inode)
+{
+	unsigned int unit, src;
+	u64 extent_count;
+	bool _48bit;
+
+	if (inode->u.chunkformat & EROFS_CHUNK_FORMAT_INDEXES)
+		unit = sizeof(struct erofs_inode_chunk_index);
+	else
+		unit = EROFS_BLOCK_MAP_ENTRY_SIZE;
+
+	_48bit = inode->u.chunkformat & EROFS_CHUNK_FORMAT_48BIT;
+	if (_48bit)
+		return;
+
+	extent_count = inode->extent_isize / unit;
+	for (src = 0; src < extent_count; ++src) {
+		struct erofs_blobchunk *chunk =
+			*(void **)(inode->chunkindexes + src * sizeof(void *));
+
+		if (chunk->blkaddr == EROFS_NULL_ADDR)
+			continue;
+		if (chunk->device_id) {
+			if (chunk->blkaddr > UINT32_MAX) {
+				_48bit = true;
+				break;
+			}
+		} else if (remapped_base + chunk->blkaddr > UINT32_MAX) {
+			_48bit = true;
+			break;
+		}
+	}
+	if (_48bit)
+		inode->u.chunkformat |= EROFS_CHUNK_FORMAT_48BIT;
+}
+
 int erofs_write_chunk_indexes(struct erofs_inode *inode, struct erofs_vfile *vf,
 			      erofs_off_t off)
 {
@@ -380,10 +416,6 @@ int erofs_blob_write_chunked_file(struct erofs_inode *inode, int fd,
 			goto err;
 		}
 
-		/* FIXME! `chunk->blkaddr` is not the final blkaddr here */
-		if (chunk->blkaddr != EROFS_NULL_ADDR &&
-		    chunk->blkaddr >= UINT32_MAX)
-			inode->u.chunkformat |= EROFS_CHUNK_FORMAT_48BIT;
 		if (!erofs_blob_can_merge(sbi, lastch, chunk)) {
 			erofs_update_minextblks(sbi, interval_start, pos,
 						&minextblks);
diff --git a/lib/inode.c b/lib/inode.c
index 4a214f9..7a1d982 100644
--- a/lib/inode.c
+++ b/lib/inode.c
@@ -908,6 +908,9 @@ static int erofs_bh_flush_write_inode(struct erofs_buffer_head *bh, bool abort)
 
 	DBG_BUGON(inode->bh != bh);
 	if (!abort) {
+		if (inode->datalayout == EROFS_INODE_CHUNK_BASED &&
+		    inode->chunkindexes)
+			erofs_inode_fixup_chunkformat(inode);
 		ret = erofs_iflush(inode);
 		if (ret)
 			return ret;
-- 
2.52.0



^ permalink raw reply related	[flat|nested] 9+ messages in thread

* [PATCH v3] erofs-utils: lib: fix 48bit addressing detection for chunk-based format
  2026-02-24  5:57 [PATCH] blobchunk: fix 48-bit format detection to use final remapped block addresses puneeth_aditya_5656
                   ` (2 preceding siblings ...)
  2026-02-24 19:10 ` puneeth_aditya_5656
@ 2026-02-24 19:13 ` puneeth_aditya_5656
  2026-02-25  3:55   ` Gao Xiang
  2026-02-25  7:39 ` [PATCH v4] " puneeth_aditya_5656
  4 siblings, 1 reply; 9+ messages in thread
From: puneeth_aditya_5656 @ 2026-02-24 19:13 UTC (permalink / raw)
  To: linux-erofs; +Cc: puneeth_aditya_5656

The 48-bit chunk format flag was being set inside
erofs_blob_write_chunked_file right after erofs_blob_getchunk returns.
At that point chunk->blkaddr is the chunk's offset in the temporary
blob buffer, not the final image address. The real address is only
known after erofs_mkfs_dump_blobs applies remapped_base.

This means the detection was unreliable in both directions: a chunk
whose blob offset looks large but fits in 32-bits after remapping gets
flagged unnecessarily, and worse, a chunk that lands above UINT32_MAX
after remapping may not get flagged at all, producing a corrupt image.

Fix this by introducing erofs_inode_fixup_chunkformat() which walks
the chunk array after remapped_base is finalized and sets the 48-bit
flag if any chunk address exceeds UINT32_MAX. The fixup is called from
erofs_iflush so that the correct chunkformat is written into the
on-disk inode header. Both blob chunks (remapped_base + chunk->blkaddr)
and device chunks (chunk->blkaddr directly) are handled.

Signed-off-by: Puneeth Aditya <myakampuneeth@gmail.com>
---
 include/erofs/blobchunk.h |  1 +
 lib/blobchunk.c           | 40 +++++++++++++++++++++++++++++++++++----
 lib/inode.c               |  2 ++
 3 files changed, 39 insertions(+), 4 deletions(-)

diff --git a/include/erofs/blobchunk.h b/include/erofs/blobchunk.h
index ef06773..48fca63 100644
--- a/include/erofs/blobchunk.h
+++ b/include/erofs/blobchunk.h
@@ -16,6 +16,7 @@ extern "C"
 
 struct erofs_blobchunk *erofs_get_unhashed_chunk(unsigned int device_id,
 		erofs_blk_t blkaddr, erofs_off_t sourceoffset);
+void erofs_inode_fixup_chunkformat(struct erofs_inode *inode);
 int erofs_write_chunk_indexes(struct erofs_inode *inode, struct erofs_vfile *vf,
 			      erofs_off_t off);
 int erofs_blob_write_chunked_file(struct erofs_inode *inode, int fd,
diff --git a/lib/blobchunk.c b/lib/blobchunk.c
index a051904..96c161b 100644
--- a/lib/blobchunk.c
+++ b/lib/blobchunk.c
@@ -136,6 +136,42 @@ static int erofs_blob_hashmap_cmp(const void *a, const void *b,
 		      sizeof(ec1->sha256));
 }
 
+void erofs_inode_fixup_chunkformat(struct erofs_inode *inode)
+{
+	unsigned int unit, src;
+	u64 extent_count;
+	bool _48bit;
+
+	if (inode->u.chunkformat & EROFS_CHUNK_FORMAT_INDEXES)
+		unit = sizeof(struct erofs_inode_chunk_index);
+	else
+		unit = EROFS_BLOCK_MAP_ENTRY_SIZE;
+
+	_48bit = inode->u.chunkformat & EROFS_CHUNK_FORMAT_48BIT;
+	if (_48bit)
+		return;
+
+	extent_count = inode->extent_isize / unit;
+	for (src = 0; src < extent_count; ++src) {
+		struct erofs_blobchunk *chunk =
+			*(void **)(inode->chunkindexes + src * sizeof(void *));
+
+		if (chunk->blkaddr == EROFS_NULL_ADDR)
+			continue;
+		if (chunk->device_id) {
+			if (chunk->blkaddr > UINT32_MAX) {
+				_48bit = true;
+				break;
+			}
+		} else if (remapped_base + chunk->blkaddr > UINT32_MAX) {
+			_48bit = true;
+			break;
+		}
+	}
+	if (_48bit)
+		inode->u.chunkformat |= EROFS_CHUNK_FORMAT_48BIT;
+}
+
 int erofs_write_chunk_indexes(struct erofs_inode *inode, struct erofs_vfile *vf,
 			      erofs_off_t off)
 {
@@ -380,10 +416,6 @@ int erofs_blob_write_chunked_file(struct erofs_inode *inode, int fd,
 			goto err;
 		}
 
-		/* FIXME! `chunk->blkaddr` is not the final blkaddr here */
-		if (chunk->blkaddr != EROFS_NULL_ADDR &&
-		    chunk->blkaddr >= UINT32_MAX)
-			inode->u.chunkformat |= EROFS_CHUNK_FORMAT_48BIT;
 		if (!erofs_blob_can_merge(sbi, lastch, chunk)) {
 			erofs_update_minextblks(sbi, interval_start, pos,
 						&minextblks);
diff --git a/lib/inode.c b/lib/inode.c
index 4a214f9..25087ca 100644
--- a/lib/inode.c
+++ b/lib/inode.c
@@ -794,6 +794,8 @@ int erofs_iflush(struct erofs_inode *inode)
 	} else if (is_inode_layout_compression(inode)) {
 		u1.blocks_lo = cpu_to_le32(inode->u.i_blocks);
 	} else if (inode->datalayout == EROFS_INODE_CHUNK_BASED) {
+		if (inode->chunkindexes)
+			erofs_inode_fixup_chunkformat(inode);
 		u1.c.format = cpu_to_le16(inode->u.chunkformat);
 	} else {
 		ret = erofs_inode_map_flat_blkaddr(inode);
-- 
2.52.0



^ permalink raw reply related	[flat|nested] 9+ messages in thread

* Re: [PATCH v3] erofs-utils: lib: fix 48bit addressing detection for chunk-based format
  2026-02-24 19:13 ` [PATCH v3] " puneeth_aditya_5656
@ 2026-02-25  3:55   ` Gao Xiang
  0 siblings, 0 replies; 9+ messages in thread
From: Gao Xiang @ 2026-02-25  3:55 UTC (permalink / raw)
  To: puneeth_aditya_5656, linux-erofs



On 2026/2/25 03:13, puneeth_aditya_5656 wrote:
> The 48-bit chunk format flag was being set inside
> erofs_blob_write_chunked_file right after erofs_blob_getchunk returns.
> At that point chunk->blkaddr is the chunk's offset in the temporary
> blob buffer, not the final image address. The real address is only
> known after erofs_mkfs_dump_blobs applies remapped_base.
> 
> This means the detection was unreliable in both directions: a chunk
> whose blob offset looks large but fits in 32-bits after remapping gets
> flagged unnecessarily, and worse, a chunk that lands above UINT32_MAX

I think the first case is impossible for the current remapping
mechanism.

> after remapping may not get flagged at all, producing a corrupt image.
> 
> Fix this by introducing erofs_inode_fixup_chunkformat() which walks
> the chunk array after remapped_base is finalized and sets the 48-bit
> flag if any chunk address exceeds UINT32_MAX. The fixup is called from
> erofs_iflush so that the correct chunkformat is written into the
> on-disk inode header. Both blob chunks (remapped_base + chunk->blkaddr)
> and device chunks (chunk->blkaddr directly) are handled.
> 
> Signed-off-by: Puneeth Aditya <myakampuneeth@gmail.com>
> ---
>   include/erofs/blobchunk.h |  1 +
>   lib/blobchunk.c           | 40 +++++++++++++++++++++++++++++++++++----
>   lib/inode.c               |  2 ++
>   3 files changed, 39 insertions(+), 4 deletions(-)
> 
> diff --git a/include/erofs/blobchunk.h b/include/erofs/blobchunk.h
> index ef06773..48fca63 100644
> --- a/include/erofs/blobchunk.h
> +++ b/include/erofs/blobchunk.h
> @@ -16,6 +16,7 @@ extern "C"
>   
>   struct erofs_blobchunk *erofs_get_unhashed_chunk(unsigned int device_id,
>   		erofs_blk_t blkaddr, erofs_off_t sourceoffset);
> +void erofs_inode_fixup_chunkformat(struct erofs_inode *inode);
>   int erofs_write_chunk_indexes(struct erofs_inode *inode, struct erofs_vfile *vf,
>   			      erofs_off_t off);
>   int erofs_blob_write_chunked_file(struct erofs_inode *inode, int fd,
> diff --git a/lib/blobchunk.c b/lib/blobchunk.c
> index a051904..96c161b 100644
> --- a/lib/blobchunk.c
> +++ b/lib/blobchunk.c
> @@ -136,6 +136,42 @@ static int erofs_blob_hashmap_cmp(const void *a, const void *b,
>   		      sizeof(ec1->sha256));
>   }
>   

...

> +
>   int erofs_write_chunk_indexes(struct erofs_inode *inode, struct erofs_vfile *vf,
>   			      erofs_off_t off)
>   {
> @@ -380,10 +416,6 @@ int erofs_blob_write_chunked_file(struct erofs_inode *inode, int fd,
>   			goto err;
>   		}
>   
> -		/* FIXME! `chunk->blkaddr` is not the final blkaddr here */
> -		if (chunk->blkaddr != EROFS_NULL_ADDR &&
> -		    chunk->blkaddr >= UINT32_MAX)
> -			inode->u.chunkformat |= EROFS_CHUNK_FORMAT_48BIT;
>   		if (!erofs_blob_can_merge(sbi, lastch, chunk)) {
>   			erofs_update_minextblks(sbi, interval_start, pos,
>   						&minextblks);
> diff --git a/lib/inode.c b/lib/inode.c
> index 4a214f9..25087ca 100644
> --- a/lib/inode.c
> +++ b/lib/inode.c
> @@ -794,6 +794,8 @@ int erofs_iflush(struct erofs_inode *inode)
>   	} else if (is_inode_layout_compression(inode)) {
>   		u1.blocks_lo = cpu_to_le32(inode->u.i_blocks);
>   	} else if (inode->datalayout == EROFS_INODE_CHUNK_BASED) {
> +		if (inode->chunkindexes)

It's a useless check, just remove this.

Thanks,
Gao Xiang


^ permalink raw reply	[flat|nested] 9+ messages in thread

* [PATCH v4] erofs-utils: lib: fix 48bit addressing detection for chunk-based format
  2026-02-24  5:57 [PATCH] blobchunk: fix 48-bit format detection to use final remapped block addresses puneeth_aditya_5656
                   ` (3 preceding siblings ...)
  2026-02-24 19:13 ` [PATCH v3] " puneeth_aditya_5656
@ 2026-02-25  7:39 ` puneeth_aditya_5656
  2026-02-25  8:19   ` Gao Xiang
  4 siblings, 1 reply; 9+ messages in thread
From: puneeth_aditya_5656 @ 2026-02-25  7:39 UTC (permalink / raw)
  To: linux-erofs; +Cc: puneeth_aditya_5656

The 48-bit chunk format flag was being set inside
erofs_blob_write_chunked_file right after erofs_blob_getchunk returns.
At that point chunk->blkaddr is the chunk's offset in the temporary
blob buffer, not the final image address. The real address is only
known after erofs_mkfs_dump_blobs applies remapped_base, so a chunk
that lands above UINT32_MAX after remapping may not get flagged at all,
producing a corrupt image.

Fix this by introducing erofs_inode_fixup_chunkformat() which walks
the chunk array after remapped_base is finalized and sets the 48-bit
flag if any chunk address exceeds UINT32_MAX. The fixup is called from
erofs_iflush so that the correct chunkformat is written into the
on-disk inode header. Both blob chunks (remapped_base + chunk->blkaddr)
and device chunks (chunk->blkaddr directly) are handled.

Signed-off-by: Puneeth Aditya <myakampuneeth@gmail.com>
---
 include/erofs/blobchunk.h |  1 +
 lib/blobchunk.c           | 40 +++++++++++++++++++++++++++++++++++----
 lib/inode.c               |  1 +
 3 files changed, 38 insertions(+), 4 deletions(-)

diff --git a/include/erofs/blobchunk.h b/include/erofs/blobchunk.h
index ef06773..48fca63 100644
--- a/include/erofs/blobchunk.h
+++ b/include/erofs/blobchunk.h
@@ -16,6 +16,7 @@ extern "C"
 
 struct erofs_blobchunk *erofs_get_unhashed_chunk(unsigned int device_id,
 		erofs_blk_t blkaddr, erofs_off_t sourceoffset);
+void erofs_inode_fixup_chunkformat(struct erofs_inode *inode);
 int erofs_write_chunk_indexes(struct erofs_inode *inode, struct erofs_vfile *vf,
 			      erofs_off_t off);
 int erofs_blob_write_chunked_file(struct erofs_inode *inode, int fd,
diff --git a/lib/blobchunk.c b/lib/blobchunk.c
index a051904..96c161b 100644
--- a/lib/blobchunk.c
+++ b/lib/blobchunk.c
@@ -136,6 +136,42 @@ static int erofs_blob_hashmap_cmp(const void *a, const void *b,
 		      sizeof(ec1->sha256));
 }
 
+void erofs_inode_fixup_chunkformat(struct erofs_inode *inode)
+{
+	unsigned int unit, src;
+	u64 extent_count;
+	bool _48bit;
+
+	if (inode->u.chunkformat & EROFS_CHUNK_FORMAT_INDEXES)
+		unit = sizeof(struct erofs_inode_chunk_index);
+	else
+		unit = EROFS_BLOCK_MAP_ENTRY_SIZE;
+
+	_48bit = inode->u.chunkformat & EROFS_CHUNK_FORMAT_48BIT;
+	if (_48bit)
+		return;
+
+	extent_count = inode->extent_isize / unit;
+	for (src = 0; src < extent_count; ++src) {
+		struct erofs_blobchunk *chunk =
+			*(void **)(inode->chunkindexes + src * sizeof(void *));
+
+		if (chunk->blkaddr == EROFS_NULL_ADDR)
+			continue;
+		if (chunk->device_id) {
+			if (chunk->blkaddr > UINT32_MAX) {
+				_48bit = true;
+				break;
+			}
+		} else if (remapped_base + chunk->blkaddr > UINT32_MAX) {
+			_48bit = true;
+			break;
+		}
+	}
+	if (_48bit)
+		inode->u.chunkformat |= EROFS_CHUNK_FORMAT_48BIT;
+}
+
 int erofs_write_chunk_indexes(struct erofs_inode *inode, struct erofs_vfile *vf,
 			      erofs_off_t off)
 {
@@ -380,10 +416,6 @@ int erofs_blob_write_chunked_file(struct erofs_inode *inode, int fd,
 			goto err;
 		}
 
-		/* FIXME! `chunk->blkaddr` is not the final blkaddr here */
-		if (chunk->blkaddr != EROFS_NULL_ADDR &&
-		    chunk->blkaddr >= UINT32_MAX)
-			inode->u.chunkformat |= EROFS_CHUNK_FORMAT_48BIT;
 		if (!erofs_blob_can_merge(sbi, lastch, chunk)) {
 			erofs_update_minextblks(sbi, interval_start, pos,
 						&minextblks);
diff --git a/lib/inode.c b/lib/inode.c
index 4a214f9..2cfc6c5 100644
--- a/lib/inode.c
+++ b/lib/inode.c
@@ -794,6 +794,7 @@ int erofs_iflush(struct erofs_inode *inode)
 	} else if (is_inode_layout_compression(inode)) {
 		u1.blocks_lo = cpu_to_le32(inode->u.i_blocks);
 	} else if (inode->datalayout == EROFS_INODE_CHUNK_BASED) {
+		erofs_inode_fixup_chunkformat(inode);
 		u1.c.format = cpu_to_le16(inode->u.chunkformat);
 	} else {
 		ret = erofs_inode_map_flat_blkaddr(inode);
-- 
2.52.0



^ permalink raw reply related	[flat|nested] 9+ messages in thread

* Re: [PATCH v4] erofs-utils: lib: fix 48bit addressing detection for chunk-based format
  2026-02-25  7:39 ` [PATCH v4] " puneeth_aditya_5656
@ 2026-02-25  8:19   ` Gao Xiang
  0 siblings, 0 replies; 9+ messages in thread
From: Gao Xiang @ 2026-02-25  8:19 UTC (permalink / raw)
  To: puneeth_aditya_5656, linux-erofs



On 2026/2/25 15:39, puneeth_aditya_5656 wrote:
> The 48-bit chunk format flag was being set inside
> erofs_blob_write_chunked_file right after erofs_blob_getchunk returns.
> At that point chunk->blkaddr is the chunk's offset in the temporary
> blob buffer, not the final image address. The real address is only
> known after erofs_mkfs_dump_blobs applies remapped_base, so a chunk
> that lands above UINT32_MAX after remapping may not get flagged at all,
> producing a corrupt image.
> 
> Fix this by introducing erofs_inode_fixup_chunkformat() which walks
> the chunk array after remapped_base is finalized and sets the 48-bit
> flag if any chunk address exceeds UINT32_MAX. The fixup is called from
> erofs_iflush so that the correct chunkformat is written into the
> on-disk inode header. Both blob chunks (remapped_base + chunk->blkaddr)
> and device chunks (chunk->blkaddr directly) are handled.
> 
> Signed-off-by: Puneeth Aditya <myakampuneeth@gmail.com>

LGTM, will apply, although I still wonder how to add a reasonable
testcase since it needs to generate a huge image.)

Thanks,
Gao Xiang


^ permalink raw reply	[flat|nested] 9+ messages in thread

end of thread, other threads:[~2026-02-25  8:19 UTC | newest]

Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-02-24  5:57 [PATCH] blobchunk: fix 48-bit format detection to use final remapped block addresses puneeth_aditya_5656
2026-02-24  6:44 ` Gao Xiang
2026-02-24 16:37 ` [PATCH v2] erofs-utils: lib: fix 48bit addressing detection for chunk-based format puneeth_aditya_5656
2026-02-24 17:44   ` Gao Xiang
2026-02-24 19:10 ` puneeth_aditya_5656
2026-02-24 19:13 ` [PATCH v3] " puneeth_aditya_5656
2026-02-25  3:55   ` Gao Xiang
2026-02-25  7:39 ` [PATCH v4] " puneeth_aditya_5656
2026-02-25  8:19   ` Gao Xiang

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox