[PATCH 0/3] bootm: size the noload buffer from the compressor header

[PATCH 0/3] bootm: size the noload buffer from the compressor header

[Aristo Chen via U-Boot]

For a compressed kernel_noload image, bootm_load_os() currently sizes
the decompression buffer as ALIGN(image_len * 8, SZ_1M). The 8x
heuristic works for typical kernels, but any well-compressed payload
(e.g. a long run of zeros) can exceed it, and no fixed multiplier is
safe against arbitrarily compressible input.

This series reads the real uncompressed size from the compressor
header instead. A new helper image_decomp_get_uncompressed_size()
returns the size from gzip ISIZE, lzma's fixed 8-byte header field,
lz4's Content_Size (when the FLG bit is set), or zstd's
Frame_Content_Size. Where the format lacks a size (bzip2, lzo, xz) or
the specific stream omits it (lzma "unknown", lz4 without
--content-size), bootm falls back to the existing 8x heuristic. The
header-derived value is attacker-controlled, so it is capped at
CONFIG_SYS_BOOTM_LEN before use.

Patch 1 adds the helper and wires it into bootm_load_os().

Patch 2 covers gzip, lz4 (with --content-size), and zstd end-to-end
through bootm on sandbox. Every noload_decomp test now carries the
compressor in its name (test_fit_kernel_noload_decomp_<comp>_*); the
lz4 and zstd cases are guarded with requiredtool markers so they skip
cleanly on hosts that don't ship the corresponding compressor. The
lying-header case is exercised for gzip only, because the
CONFIG_SYS_BOOTM_LEN cap lives in one format-agnostic branch of
bootm_load_os() that every parser feeds into.

Patch 3 covers the lzma branch of the helper via a C-level unit test
with a hand-crafted static blob, because standard Ubuntu's xz-utils
lzma shim and Python's lzma.FORMAT_ALONE both write the header size
as "unknown".

Tested on sandbox; the five kernel_noload_decomp pytests pass, the
new compression_test_image_decomp_lzma unit test passes alongside the
14 existing compression unit tests, and each commit builds in
isolation.

Aristo Chen (3):
  bootm: size the noload decompression buffer from the compressor header
  test: fit: cover the kernel_noload header-size and lying-header paths
  test: lib: cover image_decomp_get_uncompressed_size() for lzma streams

 boot/bootm.c              |  20 +++--
 boot/image.c              |  79 +++++++++++++++++
 include/image.h           |  25 ++++++
 test/lib/compression.c    |  66 ++++++++++++++
 test/py/tests/test_fit.py | 182 +++++++++++++++++++++++++++++++++-----
 5 files changed, 346 insertions(+), 26 deletions(-)

-- 
2.43.0

[PATCH 1/3] bootm: size the noload decompression buffer from the compressor header

[Aristo Chen via U-Boot]

For a compressed kernel_noload image, bootm_load_os() allocates a
per-image decompression buffer of ALIGN(image_len * 8, SZ_1M). The 8x
multiplier is a heuristic: it comfortably covers what zstd and xz
achieve on real kernels, but any well-compressed payload (say, a big
run of zeros) can exceed it and fail decompression, and no fixed
multiplier is safe against arbitrarily compressible input.

Read the real uncompressed size from the compressor header instead.
Add a small helper image_decomp_get_uncompressed_size() that returns
the uncompressed size when the format carries one: gzip ISIZE, lzma
header uncompressed size, lz4 frame Content_Size when the FLG bit is
set, and zstd Frame_Content_Size. Other formats return -EOPNOTSUPP.
Bootm uses it to size the buffer to ALIGN(hdr_size, SZ_1M), capped at
CONFIG_SYS_BOOTM_LEN because the value is attacker-controlled, and
falls back to the 8x heuristic for formats without a size field
(bzip2, lzo, xz) or when the header lacks the size (some lzma or lz4
streams).

Suggested-by: Simon Glass <sjg@chromium.org>
Signed-off-by: Aristo Chen <aristo.chen@canonical.com>
---
 boot/bootm.c    | 20 +++++++++----
 boot/image.c    | 79 +++++++++++++++++++++++++++++++++++++++++++++++++
 include/image.h | 25 ++++++++++++++++
 3 files changed, 119 insertions(+), 5 deletions(-)

diff --git a/boot/bootm.c b/boot/bootm.c
index 3bce8586834..6ce98485889 100644
--- a/boot/bootm.c
+++ b/boot/bootm.c
@@ -654,17 +654,28 @@ static int bootm_load_os(struct bootm_headers *images, int boot_progress)
 	void *load_buf, *image_buf;
 	int err;
 
+	image_buf = map_sysmem(os.image_start, image_len);
+
 	/*
 	 * For a "noload" compressed kernel we need to allocate a buffer large
 	 * enough to decompress in to and use that as the load address now.
-	 * Allow up to 8x compression: this comfortably covers what zstd and xz
-	 * achieve on real kernels, with headroom for well-compressed payloads.
-	 * Use an alignment of 2MB since this might help arm64
+	 * Prefer the uncompressed size the compressor header carries (gzip,
+	 * lzma, lz4-with-content-size, zstd); the value is attacker-controlled
+	 * so cap it at CONFIG_SYS_BOOTM_LEN. Otherwise fall back to an 8x
+	 * multiplier, which comfortably covers what zstd and xz achieve on
+	 * real kernels with headroom for well-compressed payloads. Align to
+	 * 2MB since this might help arm64.
 	 */
 	if (os.type == IH_TYPE_KERNEL_NOLOAD && os.comp != IH_COMP_NONE) {
 		phys_addr_t addr;
+		ulong hdr_size = 0;
 
-		decomp_len = ALIGN(image_len * 8, SZ_1M);
+		if (!image_decomp_get_uncompressed_size(os.comp, image_buf,
+							image_len, &hdr_size) &&
+		    hdr_size && hdr_size <= CONFIG_SYS_BOOTM_LEN)
+			decomp_len = ALIGN(hdr_size, SZ_1M);
+		else
+			decomp_len = ALIGN(image_len * 8, SZ_1M);
 		decomp_limit = BOOTM_DECOMP_LIMIT_PER_IMAGE;
 		err = lmb_alloc_mem(LMB_MEM_ALLOC_ANY, SZ_2M, &addr,
 				    decomp_len, LMB_NONE);
@@ -679,7 +690,6 @@ static int bootm_load_os(struct bootm_headers *images, int boot_progress)
 	}
 
 	load_buf = map_sysmem(load, 0);
-	image_buf = map_sysmem(os.image_start, image_len);
 	err = image_decomp(os.comp, load, os.image_start, os.type,
 			   load_buf, image_buf, image_len,
 			   decomp_len, &load_end);
diff --git a/boot/image.c b/boot/image.c
index 185d52ba492..5476c81bb6f 100644
--- a/boot/image.c
+++ b/boot/image.c
@@ -22,6 +22,7 @@
 
 #include <linux/errno.h>
 #include <asm/io.h>
+#include <asm/unaligned.h>
 
 /* Set this if we have less than 4 MB of malloc() space */
 #if CONFIG_SYS_MALLOC_LEN < (4096 * 1024)
@@ -442,6 +443,84 @@ int image_decomp_type(const unsigned char *buf, ulong len)
 	return cmagic->comp_id;
 }
 
+#ifndef USE_HOSTCC
+int image_decomp_get_uncompressed_size(int comp, const void *src, ulong len,
+				       ulong *sizep)
+{
+	const u8 *bytes = src;
+
+	switch (comp) {
+	case IH_COMP_GZIP: {
+		u32 isize;
+
+		/* Minimum gzip: 10-byte header + 2-byte deflate + 8-byte trailer */
+		if (len < 20)
+			return -EINVAL;
+		if (bytes[0] != 0x1f || bytes[1] != 0x8b)
+			return -EINVAL;
+		isize = get_unaligned_le32(bytes + len - 4);
+		*sizep = isize;
+		return 0;
+	}
+	case IH_COMP_LZMA:
+		if (CONFIG_IS_ENABLED(LZMA)) {
+			u64 usize;
+
+			/* LZMA header: 5-byte props + 8-byte uncompressed size */
+			if (len < LZMA_PROPS_SIZE + 8)
+				return -EINVAL;
+			usize = get_unaligned_le64(bytes + LZMA_PROPS_SIZE);
+			/* All-ones means "unknown", per the LZMA reference */
+			if (usize == U64_MAX)
+				return -EOPNOTSUPP;
+			if (usize > ULONG_MAX)
+				return -EINVAL;
+			*sizep = (ulong)usize;
+			return 0;
+		}
+		return -EOPNOTSUPP;
+	case IH_COMP_LZ4:
+		if (CONFIG_IS_ENABLED(LZ4)) {
+			u8 flg;
+
+			/* LZ4 frame: 4-byte magic + FLG + BD + optional 8-byte size */
+			if (len < 6)
+				return -EINVAL;
+			if (get_unaligned_le32(bytes) != LZ4F_MAGIC)
+				return -EINVAL;
+			flg = bytes[4];
+			/* Content-size flag (FLG bit 3): 8 bytes follow BD */
+			if (!(flg & 0x08))
+				return -EOPNOTSUPP;
+			if (len < 14)
+				return -EINVAL;
+			*sizep = get_unaligned_le64(bytes + 6);
+			return 0;
+		}
+		return -EOPNOTSUPP;
+	case IH_COMP_ZSTD:
+		if (CONFIG_IS_ENABLED(ZSTD)) {
+			zstd_frame_header hdr;
+			size_t ret;
+
+			ret = zstd_get_frame_header(&hdr, src, len);
+			if (zstd_is_error(ret) || ret > 0)
+				return -EINVAL;
+			if (hdr.frameContentSize == ZSTD_CONTENTSIZE_UNKNOWN)
+				return -EOPNOTSUPP;
+			if (hdr.frameContentSize == ZSTD_CONTENTSIZE_ERROR ||
+			    hdr.frameContentSize > ULONG_MAX)
+				return -EINVAL;
+			*sizep = (ulong)hdr.frameContentSize;
+			return 0;
+		}
+		return -EOPNOTSUPP;
+	default:
+		return -EOPNOTSUPP;
+	}
+}
+#endif /* !USE_HOSTCC */
+
 int image_decomp(int comp, ulong load, ulong image_start, int type,
 		 void *load_buf, void *image_buf, ulong image_len,
 		 uint unc_len, ulong *load_end)
diff --git a/include/image.h b/include/image.h
index 4149ebbcce9..5d590916208 100644
--- a/include/image.h
+++ b/include/image.h
@@ -1092,6 +1092,31 @@ int image_decomp(int comp, ulong load, ulong image_start, int type,
 		 void *load_buf, void *image_buf, ulong image_len,
 		 uint unc_len, ulong *load_end);
 
+/**
+ * image_decomp_get_uncompressed_size() - Read the uncompressed size from a
+ *                                        compressed stream's header
+ *
+ * Peeks at a compressed image and returns the uncompressed size where the
+ * format carries one: gzip ISIZE, lzma header uncompressed size, lz4 frame
+ * Content_Size (only when the FLG bit is set), zstd Frame_Content_Size. The
+ * value is attacker-controlled, so callers must sanity-check against an
+ * upper bound before using it as an allocation size.
+ *
+ * gzip's ISIZE is the original size modulo 2^32, so this API is only useful
+ * for images up to 4 GiB. That is more than enough for a kernel_noload
+ * decompression hint.
+ *
+ * @comp:	Compression type (IH_COMP_...)
+ * @src:	Compressed data
+ * @len:	Length of @src
+ * @sizep:	Set to the uncompressed size on success
+ * Return: 0 on success, -EOPNOTSUPP if @comp does not carry an uncompressed
+ *         size (or is not enabled in this build), -EINVAL on a malformed or
+ *         truncated header
+ */
+int image_decomp_get_uncompressed_size(int comp, const void *src, ulong len,
+				       ulong *sizep);
+
 /**
  * Set up properties in the FDT
  *
-- 
2.43.0

Re: [PATCH 1/3] bootm: size the noload decompression buffer from the compressor header

[Tom Rini]

[-- Attachment #1: Type: text/plain, Size: 1266 bytes --]

On Sun, Aug 09, 2026 at 04:23:27AM +0000, Aristo Chen wrote:

> For a compressed kernel_noload image, bootm_load_os() allocates a
> per-image decompression buffer of ALIGN(image_len * 8, SZ_1M). The 8x
> multiplier is a heuristic: it comfortably covers what zstd and xz
> achieve on real kernels, but any well-compressed payload (say, a big
> run of zeros) can exceed it and fail decompression, and no fixed
> multiplier is safe against arbitrarily compressible input.
> 
> Read the real uncompressed size from the compressor header instead.
> Add a small helper image_decomp_get_uncompressed_size() that returns
> the uncompressed size when the format carries one: gzip ISIZE, lzma
> header uncompressed size, lz4 frame Content_Size when the FLG bit is
> set, and zstd Frame_Content_Size. Other formats return -EOPNOTSUPP.
> Bootm uses it to size the buffer to ALIGN(hdr_size, SZ_1M), capped at
> CONFIG_SYS_BOOTM_LEN because the value is attacker-controlled, and
> falls back to the 8x heuristic for formats without a size field
> (bzip2, lzo, xz) or when the header lacks the size (some lzma or lz4
> streams).

Have we gotten actual problem reports? This is a good bit of growth for
a problem I'm not sure we're seeing. Thanks.

-- 
Tom

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: [PATCH 1/3] bootm: size the noload decompression buffer from the compressor header

[Aristo Chen via U-Boot]

On Sun, Aug 9, 2026 at 11:27 PM Tom Rini <trini@konsulko.com> wrote:
>
> On Sun, Aug 09, 2026 at 04:23:27AM +0000, Aristo Chen wrote:
>
> > For a compressed kernel_noload image, bootm_load_os() allocates a
> > per-image decompression buffer of ALIGN(image_len * 8, SZ_1M). The 8x
> > multiplier is a heuristic: it comfortably covers what zstd and xz
> > achieve on real kernels, but any well-compressed payload (say, a big
> > run of zeros) can exceed it and fail decompression, and no fixed
> > multiplier is safe against arbitrarily compressible input.
> >
> > Read the real uncompressed size from the compressor header instead.
> > Add a small helper image_decomp_get_uncompressed_size() that returns
> > the uncompressed size when the format carries one: gzip ISIZE, lzma
> > header uncompressed size, lz4 frame Content_Size when the FLG bit is
> > set, and zstd Frame_Content_Size. Other formats return -EOPNOTSUPP.
> > Bootm uses it to size the buffer to ALIGN(hdr_size, SZ_1M), capped at
> > CONFIG_SYS_BOOTM_LEN because the value is attacker-controlled, and
> > falls back to the 8x heuristic for formats without a size field
> > (bzip2, lzo, xz) or when the header lacks the size (some lzma or lz4
> > streams).
>
> Have we gotten actual problem reports? This is a good bit of growth for
> a problem I'm not sure we're seeing. Thanks.

Thanks for the review! Honest answer: no bug report against the
current 8x multiplier has crossed the list. This is preventive rather
than reactive, and I should have made that clearer in the cover
letter.

The reasons for this patch set are:
  * The multiplier is fundamentally a heuristic. Nora raised the same
concern in the v1 round of the earlier
series(<https://lists.denx.de/pipermail/u-boot/2026-June/621575.html>):
"Deriving a buffer size from the compressed size is not possible, as
the compression ratio may be arbitrarily high for data with many
repetitions (for example ranges of 0x00 or 0xff)."She dropped her
replacement patch when we bumped 4x to 8x, but the underlying point
stands: any fixed factor can be defeated by a highly    compressible
payload, and further bumps are just moving the ceiling.

  * Simon suggested the header-size approach as the principled fix in
the same round (<https://lists.denx.de/pipermail/u-boot/2026-May/620121.html>):
"It might be worth updating image_decomp() to take a ulong size... I
believe in each case it is also possible to find out the decomp size
by looking at the header."

So the growth buys correctness: the buffer size is derived from the
compressed stream itself, not from a multiplier guess that can be
defeated by any payload with a high enough compression ratio.

That said, I understand the "we're not seeing it" concern. If you
would rather wait for a concrete report, I am happy to drop the series
and re-send when one lands, or to shrink patch 1 to gzip only, which
cuts about 40 lines of parser code. Let me know which you prefer.
>
> --
> Tom

Best Regards,
Aristo

Re: [PATCH 1/3] bootm: size the noload decompression buffer from the compressor header

[Tom Rini]

[-- Attachment #1: Type: text/plain, Size: 3502 bytes --]

On Mon, Aug 10, 2026 at 10:32:12AM +0800, Aristo Chen wrote:
> On Sun, Aug 9, 2026 at 11:27 PM Tom Rini <trini@konsulko.com> wrote:
> >
> > On Sun, Aug 09, 2026 at 04:23:27AM +0000, Aristo Chen wrote:
> >
> > > For a compressed kernel_noload image, bootm_load_os() allocates a
> > > per-image decompression buffer of ALIGN(image_len * 8, SZ_1M). The 8x
> > > multiplier is a heuristic: it comfortably covers what zstd and xz
> > > achieve on real kernels, but any well-compressed payload (say, a big
> > > run of zeros) can exceed it and fail decompression, and no fixed
> > > multiplier is safe against arbitrarily compressible input.
> > >
> > > Read the real uncompressed size from the compressor header instead.
> > > Add a small helper image_decomp_get_uncompressed_size() that returns
> > > the uncompressed size when the format carries one: gzip ISIZE, lzma
> > > header uncompressed size, lz4 frame Content_Size when the FLG bit is
> > > set, and zstd Frame_Content_Size. Other formats return -EOPNOTSUPP.
> > > Bootm uses it to size the buffer to ALIGN(hdr_size, SZ_1M), capped at
> > > CONFIG_SYS_BOOTM_LEN because the value is attacker-controlled, and
> > > falls back to the 8x heuristic for formats without a size field
> > > (bzip2, lzo, xz) or when the header lacks the size (some lzma or lz4
> > > streams).
> >
> > Have we gotten actual problem reports? This is a good bit of growth for
> > a problem I'm not sure we're seeing. Thanks.
> 
> Thanks for the review! Honest answer: no bug report against the
> current 8x multiplier has crossed the list. This is preventive rather
> than reactive, and I should have made that clearer in the cover
> letter.
> 
> The reasons for this patch set are:
>   * The multiplier is fundamentally a heuristic. Nora raised the same
> concern in the v1 round of the earlier
> series(<https://lists.denx.de/pipermail/u-boot/2026-June/621575.html>):
> "Deriving a buffer size from the compressed size is not possible, as
> the compression ratio may be arbitrarily high for data with many
> repetitions (for example ranges of 0x00 or 0xff)."She dropped her
> replacement patch when we bumped 4x to 8x, but the underlying point
> stands: any fixed factor can be defeated by a highly    compressible
> payload, and further bumps are just moving the ceiling.

Yeah, I recall this. But we aren't really handling arbitrary data here,
so it's not as much of a valid concern I think, without real examples.

>   * Simon suggested the header-size approach as the principled fix in
> the same round (<https://lists.denx.de/pipermail/u-boot/2026-May/620121.html>):
> "It might be worth updating image_decomp() to take a ulong size... I
> believe in each case it is also possible to find out the decomp size
> by looking at the header."

Which would make sense for a more general problem, or a less constrained
system.

> So the growth buys correctness: the buffer size is derived from the
> compressed stream itself, not from a multiplier guess that can be
> defeated by any payload with a high enough compression ratio.
> 
> That said, I understand the "we're not seeing it" concern. If you
> would rather wait for a concrete report, I am happy to drop the series
> and re-send when one lands, or to shrink patch 1 to gzip only, which
> cuts about 40 lines of parser code. Let me know which you prefer.

Yes, I'd like to wait and see what problem reports we get at this point,
thanks.

-- 
Tom

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: [PATCH 1/3] bootm: size the noload decompression buffer from the compressor header

[Nora Schiffer]

On Mon, 2026-08-10 at 10:37 -0600, Tom Rini wrote:
> On Mon, Aug 10, 2026 at 10:32:12AM +0800, Aristo Chen wrote:
> > On Sun, Aug 9, 2026 at 11:27 PM Tom Rini <trini@konsulko.com> wrote:
> > > 
> > > On Sun, Aug 09, 2026 at 04:23:27AM +0000, Aristo Chen wrote:
> > > 
> > > > For a compressed kernel_noload image, bootm_load_os() allocates a
> > > > per-image decompression buffer of ALIGN(image_len * 8, SZ_1M). The 8x
> > > > multiplier is a heuristic: it comfortably covers what zstd and xz
> > > > achieve on real kernels, but any well-compressed payload (say, a big
> > > > run of zeros) can exceed it and fail decompression, and no fixed
> > > > multiplier is safe against arbitrarily compressible input.
> > > > 
> > > > Read the real uncompressed size from the compressor header instead.
> > > > Add a small helper image_decomp_get_uncompressed_size() that returns
> > > > the uncompressed size when the format carries one: gzip ISIZE, lzma
> > > > header uncompressed size, lz4 frame Content_Size when the FLG bit is
> > > > set, and zstd Frame_Content_Size. Other formats return -EOPNOTSUPP.
> > > > Bootm uses it to size the buffer to ALIGN(hdr_size, SZ_1M), capped at
> > > > CONFIG_SYS_BOOTM_LEN because the value is attacker-controlled, and
> > > > falls back to the 8x heuristic for formats without a size field
> > > > (bzip2, lzo, xz) or when the header lacks the size (some lzma or lz4
> > > > streams).
> > > 
> > > Have we gotten actual problem reports? This is a good bit of growth for
> > > a problem I'm not sure we're seeing. Thanks.
> > 
> > Thanks for the review! Honest answer: no bug report against the
> > current 8x multiplier has crossed the list. This is preventive rather
> > than reactive, and I should have made that clearer in the cover
> > letter.
> > 
> > The reasons for this patch set are:
> >   * The multiplier is fundamentally a heuristic. Nora raised the same
> > concern in the v1 round of the earlier
> > series(<https://lists.denx.de/pipermail/u-boot/2026-June/621575.html>):
> > "Deriving a buffer size from the compressed size is not possible, as
> > the compression ratio may be arbitrarily high for data with many
> > repetitions (for example ranges of 0x00 or 0xff)."She dropped her
> > replacement patch when we bumped 4x to 8x, but the underlying point
> > stands: any fixed factor can be defeated by a highly    compressible
> > payload, and further bumps are just moving the ceiling.
> 
> Yeah, I recall this. But we aren't really handling arbitrary data here,
> so it's not as much of a valid concern I think, without real examples.

It's probably not a problem when the OS image is a proper kernel, but if the
next image is a tiny loader itself, even a small amount of padding (either
inside the .data section or at the end of the image) might result in high
compression ratios.

While irrelevant for current U-Boot, one example would be OpenWrt's lzma-loader:
it has a build mode where the <100KiB binary is padded to 1MiB (I may be
remembering the exact numbers wrong) before compression to force a cache
writeback during decompression (to work around ancient U-Boot versions that did
not implement cache handling correctly.)

Specifically the case of kernel_noload would usually be used with EFI
applications, for which additional loaders (shim, systemd-boot, ...) are quite
common. The combination with FIT and compression is probably less common...

Nonetheless, I think a principled fix is preferable - I like the EFI-in-FIT
approach a lot (we may make that the default setup in our TQ-Systems standard
BSPs in the future), thus I would like the feature to be well-supported and
without known bugs.

Best,
Nora


> 
> >   * Simon suggested the header-size approach as the principled fix in
> > the same round (<https://lists.denx.de/pipermail/u-boot/2026-May/620121.html>):
> > "It might be worth updating image_decomp() to take a ulong size... I
> > believe in each case it is also possible to find out the decomp size
> > by looking at the header."
> 
> Which would make sense for a more general problem, or a less constrained
> system.
> 
> > So the growth buys correctness: the buffer size is derived from the
> > compressed stream itself, not from a multiplier guess that can be
> > defeated by any payload with a high enough compression ratio.
> > 
> > That said, I understand the "we're not seeing it" concern. If you
> > would rather wait for a concrete report, I am happy to drop the series
> > and re-send when one lands, or to shrink patch 1 to gzip only, which
> > cuts about 40 lines of parser code. Let me know which you prefer.
> 
> Yes, I'd like to wait and see what problem reports we get at this point,
> thanks.


-- 
TQ-Systems GmbH | Mühlstraße 2, Gut Delling | 82229 Seefeld, Germany
Amtsgericht München, HRB 105018
Geschäftsführer: Detlef Schneider, Rüdiger Stahl, Stefan Schneider
https://www.tq-group.com/

Re: [PATCH 1/3] bootm: size the noload decompression buffer from the compressor header

[Tom Rini]

[-- Attachment #1: Type: text/plain, Size: 4572 bytes --]

On Wed, Aug 12, 2026 at 09:45:52AM +0200, Nora Schiffer wrote:
> On Mon, 2026-08-10 at 10:37 -0600, Tom Rini wrote:
> > On Mon, Aug 10, 2026 at 10:32:12AM +0800, Aristo Chen wrote:
> > > On Sun, Aug 9, 2026 at 11:27 PM Tom Rini <trini@konsulko.com> wrote:
> > > > 
> > > > On Sun, Aug 09, 2026 at 04:23:27AM +0000, Aristo Chen wrote:
> > > > 
> > > > > For a compressed kernel_noload image, bootm_load_os() allocates a
> > > > > per-image decompression buffer of ALIGN(image_len * 8, SZ_1M). The 8x
> > > > > multiplier is a heuristic: it comfortably covers what zstd and xz
> > > > > achieve on real kernels, but any well-compressed payload (say, a big
> > > > > run of zeros) can exceed it and fail decompression, and no fixed
> > > > > multiplier is safe against arbitrarily compressible input.
> > > > > 
> > > > > Read the real uncompressed size from the compressor header instead.
> > > > > Add a small helper image_decomp_get_uncompressed_size() that returns
> > > > > the uncompressed size when the format carries one: gzip ISIZE, lzma
> > > > > header uncompressed size, lz4 frame Content_Size when the FLG bit is
> > > > > set, and zstd Frame_Content_Size. Other formats return -EOPNOTSUPP.
> > > > > Bootm uses it to size the buffer to ALIGN(hdr_size, SZ_1M), capped at
> > > > > CONFIG_SYS_BOOTM_LEN because the value is attacker-controlled, and
> > > > > falls back to the 8x heuristic for formats without a size field
> > > > > (bzip2, lzo, xz) or when the header lacks the size (some lzma or lz4
> > > > > streams).
> > > > 
> > > > Have we gotten actual problem reports? This is a good bit of growth for
> > > > a problem I'm not sure we're seeing. Thanks.
> > > 
> > > Thanks for the review! Honest answer: no bug report against the
> > > current 8x multiplier has crossed the list. This is preventive rather
> > > than reactive, and I should have made that clearer in the cover
> > > letter.
> > > 
> > > The reasons for this patch set are:
> > >   * The multiplier is fundamentally a heuristic. Nora raised the same
> > > concern in the v1 round of the earlier
> > > series(<https://lists.denx.de/pipermail/u-boot/2026-June/621575.html>):
> > > "Deriving a buffer size from the compressed size is not possible, as
> > > the compression ratio may be arbitrarily high for data with many
> > > repetitions (for example ranges of 0x00 or 0xff)."She dropped her
> > > replacement patch when we bumped 4x to 8x, but the underlying point
> > > stands: any fixed factor can be defeated by a highly    compressible
> > > payload, and further bumps are just moving the ceiling.
> > 
> > Yeah, I recall this. But we aren't really handling arbitrary data here,
> > so it's not as much of a valid concern I think, without real examples.
> 
> It's probably not a problem when the OS image is a proper kernel, but if the
> next image is a tiny loader itself, even a small amount of padding (either
> inside the .data section or at the end of the image) might result in high
> compression ratios.
> 
> While irrelevant for current U-Boot, one example would be OpenWrt's lzma-loader:
> it has a build mode where the <100KiB binary is padded to 1MiB (I may be
> remembering the exact numbers wrong) before compression to force a cache
> writeback during decompression (to work around ancient U-Boot versions that did
> not implement cache handling correctly.)
> 
> Specifically the case of kernel_noload would usually be used with EFI
> applications, for which additional loaders (shim, systemd-boot, ...) are quite
> common. The combination with FIT and compression is probably less common...
> 
> Nonetheless, I think a principled fix is preferable - I like the EFI-in-FIT
> approach a lot (we may make that the default setup in our TQ-Systems standard
> BSPs in the future), thus I would like the feature to be well-supported and
> without known bugs.

Thanks for explaining. My concern, now that I've put it through a wider
test, is that of about 1550 platforms, 1297 grow under this as-is. Of
those, ~375 grow by around 400 bytes (380 is average, a few go higher).
The rest are around 170 bytes. This is all presumably the difference
between gzip only and gzip+others (with the few much high growth being
all algorithms).

Maybe a question here is, haven't we already validated the compression
header, and so don't need to do it a second time? If we really can't
live with a good enough heuristic, we need to work the size growth as
this is very much not an opt-in feature.

-- 
Tom

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: [PATCH 1/3] bootm: size the noload decompression buffer from the compressor header

[Simon Glass]

Hi,

On Wed, 12 Aug 2026 at 09:57, Tom Rini <trini@konsulko.com> wrote:
>
> On Wed, Aug 12, 2026 at 09:45:52AM +0200, Nora Schiffer wrote:
> > On Mon, 2026-08-10 at 10:37 -0600, Tom Rini wrote:
> > > On Mon, Aug 10, 2026 at 10:32:12AM +0800, Aristo Chen wrote:
> > > > On Sun, Aug 9, 2026 at 11:27 PM Tom Rini <trini@konsulko.com> wrote:
> > > > >
> > > > > On Sun, Aug 09, 2026 at 04:23:27AM +0000, Aristo Chen wrote:
> > > > >
> > > > > > For a compressed kernel_noload image, bootm_load_os() allocates a
> > > > > > per-image decompression buffer of ALIGN(image_len * 8, SZ_1M). The 8x
> > > > > > multiplier is a heuristic: it comfortably covers what zstd and xz
> > > > > > achieve on real kernels, but any well-compressed payload (say, a big
> > > > > > run of zeros) can exceed it and fail decompression, and no fixed
> > > > > > multiplier is safe against arbitrarily compressible input.
> > > > > >
> > > > > > Read the real uncompressed size from the compressor header instead.
> > > > > > Add a small helper image_decomp_get_uncompressed_size() that returns
> > > > > > the uncompressed size when the format carries one: gzip ISIZE, lzma
> > > > > > header uncompressed size, lz4 frame Content_Size when the FLG bit is
> > > > > > set, and zstd Frame_Content_Size. Other formats return -EOPNOTSUPP.
> > > > > > Bootm uses it to size the buffer to ALIGN(hdr_size, SZ_1M), capped at
> > > > > > CONFIG_SYS_BOOTM_LEN because the value is attacker-controlled, and
> > > > > > falls back to the 8x heuristic for formats without a size field
> > > > > > (bzip2, lzo, xz) or when the header lacks the size (some lzma or lz4
> > > > > > streams).
> > > > >
> > > > > Have we gotten actual problem reports? This is a good bit of growth for
> > > > > a problem I'm not sure we're seeing. Thanks.
> > > >
> > > > Thanks for the review! Honest answer: no bug report against the
> > > > current 8x multiplier has crossed the list. This is preventive rather
> > > > than reactive, and I should have made that clearer in the cover
> > > > letter.
> > > >
> > > > The reasons for this patch set are:
> > > >   * The multiplier is fundamentally a heuristic. Nora raised the same
> > > > concern in the v1 round of the earlier
> > > > series(<https://lists.denx.de/pipermail/u-boot/2026-June/621575.html>):
> > > > "Deriving a buffer size from the compressed size is not possible, as
> > > > the compression ratio may be arbitrarily high for data with many
> > > > repetitions (for example ranges of 0x00 or 0xff)."She dropped her
> > > > replacement patch when we bumped 4x to 8x, but the underlying point
> > > > stands: any fixed factor can be defeated by a highly    compressible
> > > > payload, and further bumps are just moving the ceiling.
> > >
> > > Yeah, I recall this. But we aren't really handling arbitrary data here,
> > > so it's not as much of a valid concern I think, without real examples.
> >
> > It's probably not a problem when the OS image is a proper kernel, but if the
> > next image is a tiny loader itself, even a small amount of padding (either
> > inside the .data section or at the end of the image) might result in high
> > compression ratios.
> >
> > While irrelevant for current U-Boot, one example would be OpenWrt's lzma-loader:
> > it has a build mode where the <100KiB binary is padded to 1MiB (I may be
> > remembering the exact numbers wrong) before compression to force a cache
> > writeback during decompression (to work around ancient U-Boot versions that did
> > not implement cache handling correctly.)
> >
> > Specifically the case of kernel_noload would usually be used with EFI
> > applications, for which additional loaders (shim, systemd-boot, ...) are quite
> > common. The combination with FIT and compression is probably less common...
> >
> > Nonetheless, I think a principled fix is preferable - I like the EFI-in-FIT
> > approach a lot (we may make that the default setup in our TQ-Systems standard
> > BSPs in the future), thus I would like the feature to be well-supported and
> > without known bugs.
>
> Thanks for explaining. My concern, now that I've put it through a wider
> test, is that of about 1550 platforms, 1297 grow under this as-is. Of
> those, ~375 grow by around 400 bytes (380 is average, a few go higher).
> The rest are around 170 bytes. This is all presumably the difference
> between gzip only and gzip+others (with the few much high growth being
> all algorithms).
>
> Maybe a question here is, haven't we already validated the compression
> header, and so don't need to do it a second time? If we really can't
> live with a good enough heuristic, we need to work the size growth as
> this is very much not an opt-in feature.

Given these comments I'm going to hold off reviewing this series. I
agree that getting the real uncompressed size is a nice idea, but if
it is too expensive in terms of code size, then we might be better to
stick with what we have. Another options is to write the uncompressed
size as a property in the FIT image.

Regards,
Simon

Re: [PATCH 1/3] bootm: size the noload decompression buffer from the compressor header

[Aristo Chen via U-Boot]

Hi Simon, Tom,

On Sun, Aug 16, 2026 at 2:33 AM Simon Glass <sjg@chromium.org> wrote:
>
> Hi,
>
> On Wed, 12 Aug 2026 at 09:57, Tom Rini <trini@konsulko.com> wrote:
> >
> > On Wed, Aug 12, 2026 at 09:45:52AM +0200, Nora Schiffer wrote:
> > > On Mon, 2026-08-10 at 10:37 -0600, Tom Rini wrote:
> > > > On Mon, Aug 10, 2026 at 10:32:12AM +0800, Aristo Chen wrote:
> > > > > On Sun, Aug 9, 2026 at 11:27 PM Tom Rini <trini@konsulko.com> wrote:
> > > > > >
> > > > > > On Sun, Aug 09, 2026 at 04:23:27AM +0000, Aristo Chen wrote:
> > > > > >
> > > > > > > For a compressed kernel_noload image, bootm_load_os() allocates a
> > > > > > > per-image decompression buffer of ALIGN(image_len * 8, SZ_1M). The 8x
> > > > > > > multiplier is a heuristic: it comfortably covers what zstd and xz
> > > > > > > achieve on real kernels, but any well-compressed payload (say, a big
> > > > > > > run of zeros) can exceed it and fail decompression, and no fixed
> > > > > > > multiplier is safe against arbitrarily compressible input.
> > > > > > >
> > > > > > > Read the real uncompressed size from the compressor header instead.
> > > > > > > Add a small helper image_decomp_get_uncompressed_size() that returns
> > > > > > > the uncompressed size when the format carries one: gzip ISIZE, lzma
> > > > > > > header uncompressed size, lz4 frame Content_Size when the FLG bit is
> > > > > > > set, and zstd Frame_Content_Size. Other formats return -EOPNOTSUPP.
> > > > > > > Bootm uses it to size the buffer to ALIGN(hdr_size, SZ_1M), capped at
> > > > > > > CONFIG_SYS_BOOTM_LEN because the value is attacker-controlled, and
> > > > > > > falls back to the 8x heuristic for formats without a size field
> > > > > > > (bzip2, lzo, xz) or when the header lacks the size (some lzma or lz4
> > > > > > > streams).
> > > > > >
> > > > > > Have we gotten actual problem reports? This is a good bit of growth for
> > > > > > a problem I'm not sure we're seeing. Thanks.
> > > > >
> > > > > Thanks for the review! Honest answer: no bug report against the
> > > > > current 8x multiplier has crossed the list. This is preventive rather
> > > > > than reactive, and I should have made that clearer in the cover
> > > > > letter.
> > > > >
> > > > > The reasons for this patch set are:
> > > > >   * The multiplier is fundamentally a heuristic. Nora raised the same
> > > > > concern in the v1 round of the earlier
> > > > > series(<https://lists.denx.de/pipermail/u-boot/2026-June/621575.html>):
> > > > > "Deriving a buffer size from the compressed size is not possible, as
> > > > > the compression ratio may be arbitrarily high for data with many
> > > > > repetitions (for example ranges of 0x00 or 0xff)."She dropped her
> > > > > replacement patch when we bumped 4x to 8x, but the underlying point
> > > > > stands: any fixed factor can be defeated by a highly    compressible
> > > > > payload, and further bumps are just moving the ceiling.
> > > >
> > > > Yeah, I recall this. But we aren't really handling arbitrary data here,
> > > > so it's not as much of a valid concern I think, without real examples.
> > >
> > > It's probably not a problem when the OS image is a proper kernel, but if the
> > > next image is a tiny loader itself, even a small amount of padding (either
> > > inside the .data section or at the end of the image) might result in high
> > > compression ratios.
> > >
> > > While irrelevant for current U-Boot, one example would be OpenWrt's lzma-loader:
> > > it has a build mode where the <100KiB binary is padded to 1MiB (I may be
> > > remembering the exact numbers wrong) before compression to force a cache
> > > writeback during decompression (to work around ancient U-Boot versions that did
> > > not implement cache handling correctly.)
> > >
> > > Specifically the case of kernel_noload would usually be used with EFI
> > > applications, for which additional loaders (shim, systemd-boot, ...) are quite
> > > common. The combination with FIT and compression is probably less common...
> > >
> > > Nonetheless, I think a principled fix is preferable - I like the EFI-in-FIT
> > > approach a lot (we may make that the default setup in our TQ-Systems standard
> > > BSPs in the future), thus I would like the feature to be well-supported and
> > > without known bugs.
> >
> > Thanks for explaining. My concern, now that I've put it through a wider
> > test, is that of about 1550 platforms, 1297 grow under this as-is. Of
> > those, ~375 grow by around 400 bytes (380 is average, a few go higher).
> > The rest are around 170 bytes. This is all presumably the difference
> > between gzip only and gzip+others (with the few much high growth being
> > all algorithms).
> >
> > Maybe a question here is, haven't we already validated the compression
> > header, and so don't need to do it a second time? If we really can't
> > live with a good enough heuristic, we need to work the size growth as
> > this is very much not an opt-in feature.
>
> Given these comments I'm going to hold off reviewing this series. I
> agree that getting the real uncompressed size is a nice idea, but if
> it is too expensive in terms of code size, then we might be better to
> stick with what we have. Another options is to write the uncompressed
> size as a property in the FIT image.

Thanks Simon, I think a FIT property is an attractive option,
especially for the EFI-in-FIT case that motivated this: the boot-side
cost becomes a single property read, the ITS author or build system
already knows the uncompressed size so nothing needs to parse the
stream anywhere, it works even for formats whose streams carry no
size field, and images without the property simply keep the current
8x fallback. The trade-offs are that it needs a binding addition plus
image-generation support, only images that carry the property
benefit, and the legacy uImage form of kernel_noload stays on the
heuristic (which is probably acceptable). The property value would
still need the CONFIG_SYS_BOOTM_LEN cap before allocating, same as a
header value.

To Tom's earlier question about validating the header twice: the
value is used only as an allocation hint. bootm performs only the
format-specific parsing needed to obtain the size, caps it at
CONFIG_SYS_BOOTM_LEN, and the decompressor remains authoritative for
validating and decoding the stream. But I agree the property answers
that concern even more directly, since bootm then reads nothing from
the stream at all.

On the size growth, since that was the blocker: I reworked the v1
implementation into per-format helpers that are only compiled when
the matching decompressor is enabled, and re-ran the world build
(all 1550 defconfigs, v1 and the rework applied to the same base
commit). Median growth on changed boards drops from +160 to +96
bytes, the 856 gzip-only boards go from +112 to +80, boards without
any of the formats go from +108 to zero, and 1222 of 1496 comparable
boards end up smaller than with v1. Tom, the "few go higher"
outliers in your run should be the binutils Cortex-A53 erratum
843419 workaround: each triggered veneer is padded to a full 4 KiB
page, and any few-hundred-byte change re-rolls which arm64 boards
gain or lose one, so those jumps are not code from the patch itself.

So from my side both directions are workable:

  - if the reworked cost is acceptable, I can post it as v2, split
    per format so any individual decompressor can be dropped;
  - if the property route is preferred, I am happy to prototype that
    instead (binding plus the bootm side, with the 8x fallback for
    images without the property) and hold the rework.

The two also compose rather than conflict: with both in place bootm
would prefer the property, then the stream header, then the 8x
fallback, so picking one now does not rule out adding the other
later. I have not listed the combination as a third option only
because its boot-side cost is the sum of the two, so it becomes
interesting once the header cost itself is judged acceptable.

Which would you prefer?

>
> Regards,
> Simon

Regards,
Aristo

Re: [PATCH 1/3] bootm: size the noload decompression buffer from the compressor header

[Tom Rini]

[-- Attachment #1: Type: text/plain, Size: 8674 bytes --]

On Tue, Aug 18, 2026 at 12:01:31AM +0800, Aristo Chen wrote:
> Hi Simon, Tom,
> 
> On Sun, Aug 16, 2026 at 2:33 AM Simon Glass <sjg@chromium.org> wrote:
> >
> > Hi,
> >
> > On Wed, 12 Aug 2026 at 09:57, Tom Rini <trini@konsulko.com> wrote:
> > >
> > > On Wed, Aug 12, 2026 at 09:45:52AM +0200, Nora Schiffer wrote:
> > > > On Mon, 2026-08-10 at 10:37 -0600, Tom Rini wrote:
> > > > > On Mon, Aug 10, 2026 at 10:32:12AM +0800, Aristo Chen wrote:
> > > > > > On Sun, Aug 9, 2026 at 11:27 PM Tom Rini <trini@konsulko.com> wrote:
> > > > > > >
> > > > > > > On Sun, Aug 09, 2026 at 04:23:27AM +0000, Aristo Chen wrote:
> > > > > > >
> > > > > > > > For a compressed kernel_noload image, bootm_load_os() allocates a
> > > > > > > > per-image decompression buffer of ALIGN(image_len * 8, SZ_1M). The 8x
> > > > > > > > multiplier is a heuristic: it comfortably covers what zstd and xz
> > > > > > > > achieve on real kernels, but any well-compressed payload (say, a big
> > > > > > > > run of zeros) can exceed it and fail decompression, and no fixed
> > > > > > > > multiplier is safe against arbitrarily compressible input.
> > > > > > > >
> > > > > > > > Read the real uncompressed size from the compressor header instead.
> > > > > > > > Add a small helper image_decomp_get_uncompressed_size() that returns
> > > > > > > > the uncompressed size when the format carries one: gzip ISIZE, lzma
> > > > > > > > header uncompressed size, lz4 frame Content_Size when the FLG bit is
> > > > > > > > set, and zstd Frame_Content_Size. Other formats return -EOPNOTSUPP.
> > > > > > > > Bootm uses it to size the buffer to ALIGN(hdr_size, SZ_1M), capped at
> > > > > > > > CONFIG_SYS_BOOTM_LEN because the value is attacker-controlled, and
> > > > > > > > falls back to the 8x heuristic for formats without a size field
> > > > > > > > (bzip2, lzo, xz) or when the header lacks the size (some lzma or lz4
> > > > > > > > streams).
> > > > > > >
> > > > > > > Have we gotten actual problem reports? This is a good bit of growth for
> > > > > > > a problem I'm not sure we're seeing. Thanks.
> > > > > >
> > > > > > Thanks for the review! Honest answer: no bug report against the
> > > > > > current 8x multiplier has crossed the list. This is preventive rather
> > > > > > than reactive, and I should have made that clearer in the cover
> > > > > > letter.
> > > > > >
> > > > > > The reasons for this patch set are:
> > > > > >   * The multiplier is fundamentally a heuristic. Nora raised the same
> > > > > > concern in the v1 round of the earlier
> > > > > > series(<https://lists.denx.de/pipermail/u-boot/2026-June/621575.html>):
> > > > > > "Deriving a buffer size from the compressed size is not possible, as
> > > > > > the compression ratio may be arbitrarily high for data with many
> > > > > > repetitions (for example ranges of 0x00 or 0xff)."She dropped her
> > > > > > replacement patch when we bumped 4x to 8x, but the underlying point
> > > > > > stands: any fixed factor can be defeated by a highly    compressible
> > > > > > payload, and further bumps are just moving the ceiling.
> > > > >
> > > > > Yeah, I recall this. But we aren't really handling arbitrary data here,
> > > > > so it's not as much of a valid concern I think, without real examples.
> > > >
> > > > It's probably not a problem when the OS image is a proper kernel, but if the
> > > > next image is a tiny loader itself, even a small amount of padding (either
> > > > inside the .data section or at the end of the image) might result in high
> > > > compression ratios.
> > > >
> > > > While irrelevant for current U-Boot, one example would be OpenWrt's lzma-loader:
> > > > it has a build mode where the <100KiB binary is padded to 1MiB (I may be
> > > > remembering the exact numbers wrong) before compression to force a cache
> > > > writeback during decompression (to work around ancient U-Boot versions that did
> > > > not implement cache handling correctly.)
> > > >
> > > > Specifically the case of kernel_noload would usually be used with EFI
> > > > applications, for which additional loaders (shim, systemd-boot, ...) are quite
> > > > common. The combination with FIT and compression is probably less common...
> > > >
> > > > Nonetheless, I think a principled fix is preferable - I like the EFI-in-FIT
> > > > approach a lot (we may make that the default setup in our TQ-Systems standard
> > > > BSPs in the future), thus I would like the feature to be well-supported and
> > > > without known bugs.
> > >
> > > Thanks for explaining. My concern, now that I've put it through a wider
> > > test, is that of about 1550 platforms, 1297 grow under this as-is. Of
> > > those, ~375 grow by around 400 bytes (380 is average, a few go higher).
> > > The rest are around 170 bytes. This is all presumably the difference
> > > between gzip only and gzip+others (with the few much high growth being
> > > all algorithms).
> > >
> > > Maybe a question here is, haven't we already validated the compression
> > > header, and so don't need to do it a second time? If we really can't
> > > live with a good enough heuristic, we need to work the size growth as
> > > this is very much not an opt-in feature.
> >
> > Given these comments I'm going to hold off reviewing this series. I
> > agree that getting the real uncompressed size is a nice idea, but if
> > it is too expensive in terms of code size, then we might be better to
> > stick with what we have. Another options is to write the uncompressed
> > size as a property in the FIT image.
> 
> Thanks Simon, I think a FIT property is an attractive option,
> especially for the EFI-in-FIT case that motivated this: the boot-side
> cost becomes a single property read, the ITS author or build system
> already knows the uncompressed size so nothing needs to parse the
> stream anywhere, it works even for formats whose streams carry no
> size field, and images without the property simply keep the current
> 8x fallback. The trade-offs are that it needs a binding addition plus
> image-generation support, only images that carry the property
> benefit, and the legacy uImage form of kernel_noload stays on the
> heuristic (which is probably acceptable). The property value would
> still need the CONFIG_SYS_BOOTM_LEN cap before allocating, same as a
> header value.
> 
> To Tom's earlier question about validating the header twice: the
> value is used only as an allocation hint. bootm performs only the
> format-specific parsing needed to obtain the size, caps it at
> CONFIG_SYS_BOOTM_LEN, and the decompressor remains authoritative for
> validating and decoding the stream. But I agree the property answers
> that concern even more directly, since bootm then reads nothing from
> the stream at all.
> 
> On the size growth, since that was the blocker: I reworked the v1
> implementation into per-format helpers that are only compiled when
> the matching decompressor is enabled, and re-ran the world build
> (all 1550 defconfigs, v1 and the rework applied to the same base
> commit). Median growth on changed boards drops from +160 to +96
> bytes, the 856 gzip-only boards go from +112 to +80, boards without
> any of the formats go from +108 to zero, and 1222 of 1496 comparable
> boards end up smaller than with v1. Tom, the "few go higher"
> outliers in your run should be the binutils Cortex-A53 erratum
> 843419 workaround: each triggered veneer is padded to a full 4 KiB
> page, and any few-hundred-byte change re-rolls which arm64 boards
> gain or lose one, so those jumps are not code from the patch itself.
> 
> So from my side both directions are workable:
> 
>   - if the reworked cost is acceptable, I can post it as v2, split
>     per format so any individual decompressor can be dropped;
>   - if the property route is preferred, I am happy to prototype that
>     instead (binding plus the bootm side, with the 8x fallback for
>     images without the property) and hold the rework.
> 
> The two also compose rather than conflict: with both in place bootm
> would prefer the property, then the stream header, then the 8x
> fallback, so picking one now does not rule out adding the other
> later. I have not listed the combination as a third option only
> because its boot-side cost is the sum of the two, so it becomes
> interesting once the header cost itself is judged acceptable.
> 
> Which would you prefer?

Lets see a v2 of the rework. I think some of the higher size growth I
was talking about was on the platforms which enabled multiple
algorithms, fwiw.

-- 
Tom

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

[PATCH 2/3] test: fit: cover the kernel_noload header-size and lying-header paths

[Aristo Chen via U-Boot]

Reshape and extend the kernel_noload decompression pytests to match the
new bootm behaviour that reads the uncompressed size from the compressor
header:

  - Rename test_fit_kernel_noload_decomp_overflow to
    test_fit_kernel_noload_decomp_gzip_lying_hdr. Its setup (a 4 MiB
    payload of zeros gzipped) used to force the failure via the 8x
    heuristic starving the buffer; now that bootm reads gzip ISIZE, the
    honest trailer sizes the buffer correctly, so overwrite ISIZE with a
    tiny value instead and verify the resulting decompression is still
    stopped at the buffer boundary. This is the direct test of the
    CONFIG_SYS_BOOTM_LEN cap on the attacker-controlled header value.

  - Add test_fit_kernel_noload_decomp_gzip_hdr_sized: a 6 MiB gzipped
    payload whose compression ratio is past the 8x heuristic
    decompresses cleanly because ISIZE is consulted.

  - Add test_fit_kernel_noload_decomp_lz4_hdr_sized: the same, for lz4
    with --content-size so the frame's FLG bit is set.

  - Add test_fit_kernel_noload_decomp_zstd_hdr_sized: the same, for
    zstd whose default encoder embeds Frame_Content_Size in a
    single-segment frame.

  - Rename the pre-existing test_fit_kernel_noload_decomp_boundary to
    test_fit_kernel_noload_decomp_gzip_boundary so every noload_decomp
    test carries the compressor in its name.

  - Parametrise NOLOAD_ITS on compression so lz4, zstd, and future
    formats can share the template.

The lying-header case is covered for gzip only because the
CONFIG_SYS_BOOTM_LEN cap and buffer allocation live in a single
format-agnostic branch of bootm_load_os(): every parser feeds the same
code path, so one test is enough to exercise the security invariant
end-to-end. Per-parser correctness is covered by the hdr_sized tests
above.

The lzma branch of the helper is exercised separately in
test/lib/compression.c because standard Ubuntu ships xz-utils' lzma
shim which always writes the header size as "unknown".

Signed-off-by: Aristo Chen <aristo.chen@canonical.com>
---
 test/py/tests/test_fit.py | 182 +++++++++++++++++++++++++++++++++-----
 1 file changed, 161 insertions(+), 21 deletions(-)

diff --git a/test/py/tests/test_fit.py b/test/py/tests/test_fit.py
index 76adb98e2c5..0e1175fbea6 100755
--- a/test/py/tests/test_fit.py
+++ b/test/py/tests/test_fit.py
@@ -118,8 +118,9 @@ host save hostfs 0 %(loadables2_addr)x %(loadables2_out)s %(loadables2_size)x
 '''
 
 # A minimal ITS for a compressed 'kernel_noload' kernel. bootm allocates a
-# per-image decompression buffer for this image type, sized as a multiple of
-# the compressed length; see the test_fit_kernel_noload_decomp_* tests.
+# per-image decompression buffer for this image type, sized either from the
+# compressor header or as a multiple of the compressed length; see the
+# test_fit_kernel_noload_decomp_* tests.
 NOLOAD_ITS = '''
 /dts-v1/;
 
@@ -133,7 +134,7 @@ NOLOAD_ITS = '''
                         type = "kernel_noload";
                         arch = "sandbox";
                         os = "linux";
-                        compression = "gzip";
+                        compression = "%(compression)s";
                         load = <0>;
                         entry = <0>;
                 };
@@ -511,14 +512,13 @@ class TestFitImage:
             + output)
 
     @pytest.mark.buildconfigspec('gzip')
-    def test_fit_kernel_noload_decomp_overflow(self, ubman, fsetup):
-        """Test that an over-large compressed kernel_noload image is rejected
+    def test_fit_kernel_noload_decomp_gzip_lying_hdr(self, ubman, fsetup):
+        """A tampered gzip ISIZE cannot shrink the buffer past the payload
 
-        For a compressed 'kernel_noload' kernel, bootm_load_os() allocates a
-        decompression buffer of ALIGN(image_len * 8, SZ_1M) and must bound the
-        decompressor by that buffer. A kernel that decompresses to far more
-        than eight times its compressed size must therefore fail with a
-        decompression error instead of overflowing the buffer.
+        bootm_load_os() sizes the kernel_noload decompression buffer from the
+        compressor header (gzip ISIZE). That value is attacker-controlled;
+        rewriting ISIZE to understate the real size must not let decompression
+        overflow the resulting buffer.
         """
         sz_1m = 1 << 20
 
@@ -527,23 +527,24 @@ class TestFitImage:
         # per-image kernel_noload buffer rather than by that global limit.
         bootm_len = int(ubman.config.buildconfig['config_sys_bootm_len'], 0)
 
-        # 4MB of zeros compresses to a few KB, so the decompression buffer
-        # (ALIGN(image_len * 8, SZ_1M), i.e. 1MB here) ends up far smaller
-        # than the uncompressed image.
         decomp_size = 4 * sz_1m
+        assert decomp_size <= bootm_len, (
+            'Test setup error: uncompressed size (%#x) must be <= '
+            'CONFIG_SYS_BOOTM_LEN (%#x)' % (decomp_size, bootm_len))
         kernel = fit_util.make_fname(ubman, 'test-noload-kernel.bin')
         with open(kernel, 'wb') as fd:
             fd.write(b'\0' * decomp_size)
         kernel_gz = self.make_compressed(ubman, kernel)
 
-        image_len = self.filesize(kernel_gz)
-        req_size = (image_len * 8 + sz_1m - 1) // sz_1m * sz_1m
-        assert req_size < decomp_size <= bootm_len, (
-            'Test setup error: need decomp buffer (%#x) < image (%#x) <= '
-            'CONFIG_SYS_BOOTM_LEN (%#x)' % (req_size, decomp_size, bootm_len))
+        # Rewrite gzip ISIZE (the last 4 bytes) to claim a tiny image, so
+        # bootm allocates ALIGN(<lie>, SZ_1M) = 1 MiB and the real 4 MiB
+        # decompression has to overrun that buffer.
+        with open(kernel_gz, 'r+b') as fd:
+            fd.seek(-4, os.SEEK_END)
+            fd.write((256).to_bytes(4, 'little'))
 
         fit = fit_util.make_fit(ubman, fsetup['mkimage'], NOLOAD_ITS,
-                                {'kernel': kernel_gz})
+                                {'kernel': kernel_gz, 'compression': 'gzip'})
         fit_addr = fsetup['fit_addr']
 
         ubman.run_command_list([
@@ -563,7 +564,146 @@ class TestFitImage:
             ubman.restart_uboot()
 
     @pytest.mark.buildconfigspec('gzip')
-    def test_fit_kernel_noload_decomp_boundary(self, ubman, fsetup):
+    def test_fit_kernel_noload_decomp_gzip_hdr_sized(self, ubman, fsetup):
+        """A well-compressed kernel_noload image fits when ISIZE is honest
+
+        bootm_load_os() reads gzip ISIZE to size the decompression buffer.
+        For a well-compressed image whose ratio exceeds the 8x fallback
+        heuristic (e.g. 6 MiB of zeros gzipping to a few KiB), an ISIZE-sized
+        buffer is the only way the decompression fits.
+        """
+        sz_1m = 1 << 20
+        bootm_len = int(ubman.config.buildconfig['config_sys_bootm_len'], 0)
+
+        # Stay under CONFIG_SYS_BOOTM_LEN so the ISIZE hint isn't rejected as
+        # bogus; still large enough that image_len * 8 falls well short.
+        decomp_size = 6 * sz_1m
+        assert decomp_size <= bootm_len, (
+            'Test setup error: decomp_size (%#x) must be <= '
+            'CONFIG_SYS_BOOTM_LEN (%#x)' % (decomp_size, bootm_len))
+        kernel = fit_util.make_fname(ubman, 'test-noload-kernel-hdrsized.bin')
+        with open(kernel, 'wb') as fd:
+            fd.write(b'\0' * decomp_size)
+        kernel_gz = self.make_compressed(ubman, kernel)
+
+        image_len = self.filesize(kernel_gz)
+        heuristic_bound = (image_len * 8 + sz_1m - 1) // sz_1m * sz_1m
+        assert heuristic_bound < decomp_size, (
+            'Test setup error: 8x heuristic bound (%#x) must be < uncompressed '
+            'size (%#x); if this fires, the compressor got less effective and '
+            'the test needs a bigger payload' % (heuristic_bound, decomp_size))
+
+        fit = fit_util.make_fit(ubman, fsetup['mkimage'], NOLOAD_ITS,
+                                {'kernel': kernel_gz, 'compression': 'gzip'},
+                                basename='test-noload-hdrsized.fit')
+        fit_addr = fsetup['fit_addr']
+
+        # Decompression must succeed: bootm read ISIZE and allocated a big
+        # enough buffer despite the ratio being past the fallback heuristic.
+        output = ubman.run_command_list([
+            'host load hostfs 0 %x %s' % (fit_addr, fit),
+            'bootm start %x' % fit_addr,
+            'bootm loados',
+        ])
+        text = '\n'.join(output)
+        assert 'Image too large' not in text, (
+            'bootm rejected a well-compressed kernel_noload image whose '
+            'ISIZE trailer records the real uncompressed size: %s' % text)
+
+    @pytest.mark.buildconfigspec('lz4')
+    @pytest.mark.requiredtool('lz4')
+    def test_fit_kernel_noload_decomp_lz4_hdr_sized(self, ubman, fsetup):
+        """A well-compressed lz4 kernel_noload image fits when the frame
+        header carries the content size.
+
+        Same as test_fit_kernel_noload_decomp_gzip_hdr_sized but for lz4: the tool
+        must be invoked with --content-size so the frame's FLG bit is set and
+        bootm can read the size instead of falling back to the 8x heuristic.
+        """
+        sz_1m = 1 << 20
+        bootm_len = int(ubman.config.buildconfig['config_sys_bootm_len'], 0)
+
+        decomp_size = 6 * sz_1m
+        assert decomp_size <= bootm_len, (
+            'Test setup error: decomp_size (%#x) must be <= '
+            'CONFIG_SYS_BOOTM_LEN (%#x)' % (decomp_size, bootm_len))
+        kernel = fit_util.make_fname(ubman, 'test-noload-kernel-lz4.bin')
+        with open(kernel, 'wb') as fd:
+            fd.write(b'\0' * decomp_size)
+        kernel_lz4 = kernel + '.lz4'
+        utils.run_and_log(
+            ubman, ['lz4', '--content-size', '-f', kernel, kernel_lz4])
+
+        image_len = self.filesize(kernel_lz4)
+        heuristic_bound = (image_len * 8 + sz_1m - 1) // sz_1m * sz_1m
+        assert heuristic_bound < decomp_size, (
+            'Test setup error: 8x heuristic bound (%#x) must be < uncompressed '
+            'size (%#x); if this fires, lz4 got less effective and the test '
+            'needs a bigger payload' % (heuristic_bound, decomp_size))
+
+        fit = fit_util.make_fit(ubman, fsetup['mkimage'], NOLOAD_ITS,
+                                {'kernel': kernel_lz4, 'compression': 'lz4'},
+                                basename='test-noload-lz4-hdrsized.fit')
+        fit_addr = fsetup['fit_addr']
+
+        output = ubman.run_command_list([
+            'host load hostfs 0 %x %s' % (fit_addr, fit),
+            'bootm start %x' % fit_addr,
+            'bootm loados',
+        ])
+        text = '\n'.join(output)
+        assert 'Image too large' not in text, (
+            'bootm rejected a well-compressed lz4 kernel_noload image whose '
+            'frame header records the real content size: %s' % text)
+
+    @pytest.mark.buildconfigspec('zstd')
+    @pytest.mark.requiredtool('zstd')
+    def test_fit_kernel_noload_decomp_zstd_hdr_sized(self, ubman, fsetup):
+        """A well-compressed zstd kernel_noload image fits when the frame
+        header carries Frame_Content_Size.
+
+        Same as test_fit_kernel_noload_decomp_gzip_hdr_sized but for zstd. The
+        default zstd encoder embeds Frame_Content_Size for a single-segment
+        frame, so no extra flag is needed; bootm reads it and sizes the
+        buffer accordingly.
+        """
+        sz_1m = 1 << 20
+        bootm_len = int(ubman.config.buildconfig['config_sys_bootm_len'], 0)
+
+        decomp_size = 6 * sz_1m
+        assert decomp_size <= bootm_len, (
+            'Test setup error: decomp_size (%#x) must be <= '
+            'CONFIG_SYS_BOOTM_LEN (%#x)' % (decomp_size, bootm_len))
+        kernel = fit_util.make_fname(ubman, 'test-noload-kernel-zstd.bin')
+        with open(kernel, 'wb') as fd:
+            fd.write(b'\0' * decomp_size)
+        kernel_zstd = kernel + '.zst'
+        utils.run_and_log(ubman, ['zstd', '-f', kernel, '-o', kernel_zstd])
+
+        image_len = self.filesize(kernel_zstd)
+        heuristic_bound = (image_len * 8 + sz_1m - 1) // sz_1m * sz_1m
+        assert heuristic_bound < decomp_size, (
+            'Test setup error: 8x heuristic bound (%#x) must be < uncompressed '
+            'size (%#x); if this fires, zstd got less effective and the test '
+            'needs a bigger payload' % (heuristic_bound, decomp_size))
+
+        fit = fit_util.make_fit(ubman, fsetup['mkimage'], NOLOAD_ITS,
+                                {'kernel': kernel_zstd, 'compression': 'zstd'},
+                                basename='test-noload-zstd-hdrsized.fit')
+        fit_addr = fsetup['fit_addr']
+
+        output = ubman.run_command_list([
+            'host load hostfs 0 %x %s' % (fit_addr, fit),
+            'bootm start %x' % fit_addr,
+            'bootm loados',
+        ])
+        text = '\n'.join(output)
+        assert 'Image too large' not in text, (
+            'bootm rejected a well-compressed zstd kernel_noload image whose '
+            'frame header records the real content size: %s' % text)
+
+    @pytest.mark.buildconfigspec('gzip')
+    def test_fit_kernel_noload_decomp_gzip_boundary(self, ubman, fsetup):
         """Test that decompression succeeds exactly at the buffer limit
 
         For a compressed 'kernel_noload' kernel, bootm_load_os() allocates a
@@ -589,7 +729,7 @@ class TestFitImage:
             % (decomp_size, req_size))
 
         fit = fit_util.make_fit(ubman, fsetup['mkimage'], NOLOAD_ITS,
-                                {'kernel': kernel_gz},
+                                {'kernel': kernel_gz, 'compression': 'gzip'},
                                 basename='test-noload-boundary.fit')
         fit_addr = fsetup['fit_addr']
 
-- 
2.43.0

[PATCH 3/3] test: lib: cover image_decomp_get_uncompressed_size() for lzma streams

[Aristo Chen via U-Boot]

The gzip, lz4, and zstd branches of image_decomp_get_uncompressed_size()
are exercised end-to-end by test/py/tests/test_fit.py's
kernel_noload_decomp_*_hdr_sized cases. The lzma branch cannot be
reached that way: standard Ubuntu ships xz-utils' lzma shim (as does
Python's lzma.FORMAT_ALONE), and both always write the header's 8-byte
uncompressed-size field as the "unknown" marker 0xff..ff, so a
runtime-generated stream would only ever exercise the -EOPNOTSUPP
fallback path.

Add a hand-crafted lzma_with_size_compressed blob built with the
standalone lzma-alone tool (whose header carries the real size) and a
new unit test compression_test_image_decomp_lzma that asserts:

  - the existing lzma_compressed blob (unknown-size marker) returns
    -EOPNOTSUPP so bootm falls back to its 8x heuristic;
  - the new lzma_with_size_compressed blob returns strlen(plain) so
    bootm can pre-size the noload decompression buffer.

Signed-off-by: Aristo Chen <aristo.chen@canonical.com>
---
 test/lib/compression.c | 66 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 66 insertions(+)

diff --git a/test/lib/compression.c b/test/lib/compression.c
index 31b6e5b1eb4..930cd542294 100644
--- a/test/lib/compression.c
+++ b/test/lib/compression.c
@@ -74,6 +74,31 @@ static const char lzma_compressed[] =
 	"\xfd\xf5\x50\x8d\xca";
 static const unsigned long lzma_compressed_size = sizeof(lzma_compressed) - 1;
 
+/*
+ * lzma -kf plain.txt (standalone lzma-alone tool, not the xz-utils shim)
+ * The 8-byte uncompressed-size field at offset 5 holds strlen(plain) rather
+ * than the 0xff..ff "unknown" marker written by xz-utils and Python's
+ * lzma.FORMAT_ALONE. Used to exercise the size-parsing branch of
+ * image_decomp_get_uncompressed_size() for lzma streams.
+ */
+static const char lzma_with_size_compressed[] =
+	"\x5d\x00\x00\x80\x00\x5e\x01\x00\x00\x00\x00\x00\x00\x00\x24\x88"
+	"\x08\x26\xd8\x41\xff\x99\xc8\xcf\x66\x3d\x80\xac\xba\x17\xf1\xc8"
+	"\xb9\xdf\x49\x37\xb1\x68\xa0\x2a\xdd\x63\xd1\xa7\xa3\x66\xf8\x15"
+	"\xef\xa6\x67\x8a\x14\x18\x80\xcb\xc7\xb1\xcb\x84\x6a\xb2\x51\x16"
+	"\xa1\x45\xa0\xd6\x3e\x55\x44\x8a\x5c\xa0\x7c\xe5\xa8\xbd\x04\x57"
+	"\x8f\x24\xfd\xb9\x34\x50\x83\x2f\xf3\x46\x3e\xb9\xb0\x00\x1a\xf5"
+	"\xd3\x86\x7e\x8f\x77\xd1\x5d\x0e\x7c\xe1\xac\xde\xf8\x65\x1f\x4d"
+	"\xce\x7f\xa7\x3d\xaa\xcf\x26\xa7\x58\x69\x1e\x4c\xea\x68\x8a\xe5"
+	"\x89\xd1\xdc\x4d\xc7\xe0\x07\x42\xbf\x0c\x9d\x06\xd7\x51\xa2\x0b"
+	"\x7c\x83\x35\xe1\x85\xdf\xee\xfb\xa3\xee\x2f\x47\x5f\x8b\x70\x2b"
+	"\xe1\x37\xf3\x16\xf6\x27\x54\x8a\x33\x72\x49\xea\x53\x7d\x60\x0b"
+	"\x21\x90\x66\xe7\x9e\x56\x61\x5d\xd8\xdc\x59\xf0\xac\x2f\xd6\x49"
+	"\x6b\x85\x40\x08\x1f\xdf\x26\x25\x3b\x72\x44\xb0\xb8\x21\x2f\xb3"
+	"\xd7\x9b\x24\x30\x78\x26\x44\x07\xc3\x33\xd1\x4c\xe1\x05\x55\x6d";
+static const unsigned long lzma_with_size_compressed_size =
+	sizeof(lzma_with_size_compressed) - 1;
+
 /* lzop -c /tmp/plain.txt > /tmp/plain.lzo */
 static const char lzo_compressed[] =
 	"\x89\x4c\x5a\x4f\x00\x0d\x0a\x1a\x0a\x10\x30\x20\x60\x09\x40\x01"
@@ -606,3 +631,44 @@ static int compression_test_bootm_none(struct unit_test_state *uts)
 	return run_bootm_test(uts, IH_COMP_NONE, compress_using_none);
 }
 LIB_TEST(compression_test_bootm_none, 0);
+
+/*
+ * image_decomp_get_uncompressed_size() has a dedicated code path per
+ * format. gzip, lz4 with --content-size, and zstd are covered end-to-end
+ * by test/py/tests/test_fit.py's kernel_noload_decomp_*_hdr_sized cases.
+ * The lzma path is exercised here instead: standard Ubuntu ships xz-utils'
+ * lzma shim (as does Python's lzma.FORMAT_ALONE), and both always write
+ * the header size as "unknown" (0xff..ff), so a runtime-generated stream
+ * cannot reach the size-parsing branch. A hand-crafted static blob is the
+ * only portable way to cover it.
+ */
+static int compression_test_image_decomp_lzma(struct unit_test_state *uts)
+{
+	ulong sz;
+
+	/*
+	 * The existing lzma_compressed blob was made with the xz-utils
+	 * shim and carries the 0xff..ff "unknown" size marker. Bootm must
+	 * decline to size the buffer from that.
+	 */
+	ut_asserteq(-EOPNOTSUPP,
+		    image_decomp_get_uncompressed_size(IH_COMP_LZMA,
+						       lzma_compressed,
+						       lzma_compressed_size,
+						       &sz));
+
+	/*
+	 * The lzma_with_size_compressed blob was made with the standalone
+	 * lzma-alone tool and carries the real size in the header. Bootm
+	 * must return that size so the noload path can pre-size its buffer.
+	 */
+	sz = 0;
+	ut_assertok(image_decomp_get_uncompressed_size(IH_COMP_LZMA,
+						       lzma_with_size_compressed,
+						       lzma_with_size_compressed_size,
+						       &sz));
+	ut_asserteq(strlen(plain), sz);
+
+	return 0;
+}
+LIB_TEST(compression_test_image_decomp_lzma, 0);
-- 
2.43.0

[PATCH v2 0/8] bootm: size the noload buffer from the compressor header

[Aristo Chen]

This is v2 of "bootm: size the noload decompression buffer from the
compressor header". Tom pushed back on v1
(https://patchwork.ozlabs.org/project/uboot/patch/20260809042338.63397-2-aristo.chen@canonical.com/)
on two grounds:

  1. No concrete problem report driving the change.
  2. ~1297 platforms grew by ~170-400 bytes; the change is not
     opt-in, so the size cost falls on everyone.

On the first point, Nora Schiffer replied with a concrete use case
(EFI-in-FIT plus padded loaders such as shim, systemd-boot, and
OpenWrt's lzma-loader can produce compression ratios that outrun the
8x heuristic), and mentioned this is on the road map for TQ-Systems
standard BSPs.

On the second point, v2 reworks the implementation to cut the size
cost, measures it across the format and architecture buckets, and
splits the work per format so each decompressor's support can be
taken or dropped on its own.

Background: for a compressed kernel_noload image, bootm_load_os()
sizes the decompression buffer as ALIGN(image_len * 8, SZ_1M). The
8x heuristic works for typical kernels, but any well-compressed
payload can exceed it, and no fixed multiplier is safe against
arbitrarily compressible input.

Each implementation patch adds a small static header-parse helper in
bootm.c (no new public API) and wires it into a size-hint switch;
helper and switch case are only compiled when the matching
decompressor is enabled, so boards that do not build a format pay no
code for it. gzip's ISIZE is a fixed trailer read, lzma's size a
fixed header read, lz4 mirrors ulz4fn()'s frame-header validation,
and zstd asks zstd_get_frame_header(), whose frame-parsing code
already ships with the zstd decompressor. The header-recorded value
is attacker-controlled, so it is capped at CONFIG_SYS_BOOTM_LEN, and
it is only an allocation hint: the decoder stays authoritative
during the actual decompression.

Text size deltas of the u-boot ELF (size(1), distro gcc 13.3 cross
toolchains); data/bss are unchanged everywhere. To make the columns
directly comparable, the v1 column is v1's implementation commit
cherry-picked onto this series' base, so both columns share one
baseline:

  board          arch     decompressors        v1        v2
  qemu_arm       arm      gzip                +160      +104
  qemu-ppce500   powerpc  gzip                +176      +112
  mt7623n_bpir2  arm      gzip+lzma           +184      +128
  qemu_arm64     arm64    gzip+lzma+lz4       +384      +368
  qemu-riscv64   riscv64  gzip+lzma+lz4       +332      +352
  th1520_lpi4a   riscv64  all four            +412      +404
  am62x_evm_a53  arm64    all four, LTO       +0 *    +8192 *
  qemu-x86       x86      none                +108        -2

  * am62x_evm_a53's number is dominated by the Cortex-A53 erratum
    843419 linker workaround (default-enabled in distro binutils for
    aarch64): symbol-level code growth (nm -S) is +392 for v1 and
    +452 for v2, but those bytes shift which ADRP instructions land
    at the erratum's page offsets, and ld pads each inserted veneer
    to a full 4 KiB page. v2 happens to trigger two such pages here;
    v1 triggered the same two on its own original base and none on
    this one. See the world-build note below.

To see how much each bucket weighs, I configured all 1550 defconfigs
and sorted them by which decompressors they enable next to bootm:

  858  gzip only (722 of them arm, essentially the 32-bit boards)
  530  gzip+lzma+lz4 (494 arm, mostly arm64, plus 36 riscv)
   38  gzip+lzma
   31  bootm with no decompressor at all
   27  gzip+lzma+lz4+zstd
   19  gzip+lz4
   15  gzip+zstd
    6  other combinations
   26  do not link bootm at all

To measure at the same scale as the original objection, I also ran a
full world build (buildman, all 1550 defconfigs, distro plus
kernel.org toolchains, gcc 13.3/14.2) over one branch holding the
base, the v1 implementation, its revert, and this series. 1496
boards built on all four commits with the revert reproducing the
base sizes exactly (44 boards did not build on every commit, and 10
built nondeterministically; both sets were excluded). Of those 1496,
the same 1375 change under either version and the rest are
untouched, including every board without bootm or without a
decompressor:

                                        v1            v2
  mean delta over all boards          +194 B        +166 B
  median delta (changed boards)       +160 B         +96 B
  median, 856 gzip-only boards        +112 B         +80 B
  median, 517 gzip+lzma+lz4 boards    +392 B        +376 B
  boards cheaper with v2                   -          1222
  boards costlier with v2                  -            74

The world build also puts the am62x footnote in proportion: 53
boards under v1 and 49 under v2 (28 in both sets), all arm64, show
size(1) jumps of one or two 4 KiB pages in either direction (min
-8192, max +8192). The mechanism is the Cortex-A53 erratum 843419
linker workaround: when a code change shifts which ADRP instructions
land at page offsets 0xff8/0xffc, ld materialises a 16-byte veneer
and pads it to a full 4 KiB page so the page offsets of all
downstream code stay unchanged. Any few-hundred-byte change re-rolls
which boards are affected, in both directions; symbol-level growth
on every such board I checked matches the byte ranges above.

Since Tom noted the higher growth in his run was on multi-algorithm
platforms: building each patch in sequence on a gzip+lzma+lz4 board
(qemu_arm64) and an all-four board (th1520_lpi4a) gives the
per-format cost directly, in bytes:

               qemu_arm64    th1520_lpi4a
  gzip             +112             +86
  zstd               +0             +62
  lz4              +176            +176
  lzma              +80             +80

(zstd is +0 on qemu_arm64 because that board does not enable it, so
the guard really does compile the helper out.) lz4 is the most
expensive parser because it mirrors ulz4fn()'s frame validation;
zstd is the cheapest because zstd_get_frame_header() already ships
with the decompressor. Since the series is split per format, if the
multi-algorithm cost still looks too high, dropping the lz4 patch
alone would cut the 517-board gzip+lzma+lz4 bucket from a median of
+376 to roughly +200; lz4 images then simply keep the 8x fallback.

In the v1 thread Simon suggested recording the uncompressed size as
a FIT property instead. As discussed there, the two compose: a FIT
property could be layered on top later, with bootm preferring the
property, then the stream header, then the 8x fallback. This series
provides the part that works for every existing image and for the
legacy uImage form of kernel_noload.

Series layout, one decompressor at a time:

  1. gzip helper + wiring
  2. gzip pytests (lying-header overflow, header-sized, boundary)
  3. zstd helper
  4. zstd pytest (guarded by requiredtool zstd)
  5. lz4 helper
  6. lz4 pytest (guarded by requiredtool lz4)
  7. lzma helper
  8. lzma pytests (real size patched into the header field, plus the
     "unknown" size marker fallback; needs no external tool since
     Python's lzma module is in the standard library)

Every patch builds in isolation on sandbox_defconfig and
qemu_arm_defconfig; the seven kernel_noload_decomp pytests and the
full test_fit class pass on sandbox.

Changes in v2:
- split the single implementation patch into per-format patches
  (gzip, zstd, lz4, lzma), each acceptable or droppable on its own
- make the helpers static in bootm.c, compiled only when the
  matching decompressor is enabled, instead of one always-built
  public image_decomp_get_uncompressed_size() in boot/image.c; this
  removes the cost from boards without the formats entirely
- regroup the pytests per format, next to the patch they exercise
- add runtime lzma coverage (header-recorded size and the "unknown"
  marker fallback) in place of v1's parser-only C unit test, which
  cannot reach a static helper
- measure the size cost on a common base across all 1550 boards and
  document the per-bucket numbers and the arm64 erratum-843419 page
  effect in this cover letter

Aristo Chen (8):
  bootm: size the noload gzip decompression buffer from ISIZE
  test: fit: cover the kernel_noload gzip header-size and lying-header
    paths
  bootm: size the noload zstd decompression buffer from
    Frame_Content_Size
  test: fit: cover the kernel_noload zstd header-size path
  bootm: size the noload lz4 decompression buffer from Content_Size
  test: fit: cover the kernel_noload lz4 header-size path
  bootm: size the noload lzma decompression buffer from the header
  test: fit: cover the kernel_noload lzma header-size and unknown-size
    paths

 boot/bootm.c              | 144 ++++++++++++++++++++++-
 test/py/tests/test_fit.py | 282 ++++++++++++++++++++++++++++++++++++++++++----
 2 files changed, 400 insertions(+), 26 deletions(-)

--
2.43.0

[PATCH v2 1/8] bootm: size the noload gzip decompression buffer from ISIZE

[Aristo Chen]

For a compressed kernel_noload image, bootm_load_os() allocates a
per-image decompression buffer of ALIGN(image_len * 8, SZ_1M). The 8x
multiplier is a heuristic that comfortably covers what zstd and xz
achieve on real kernels, but a highly compressible payload (say, a
run of zeros) can exceed it and fail decompression.

gzip carries the original size in the last 4 bytes of the stream
(ISIZE, modulo 2^32). Where the compressed image is a gzip stream,
read ISIZE and use ALIGN(hdr_size, SZ_1M) as the buffer, capped at
CONFIG_SYS_BOOTM_LEN because the value is attacker-controlled. For
non-gzip streams or when ISIZE cannot be trusted, fall back to the
existing 8x multiplier.

The size read is done via a small static helper in bootm.c, wired up
via a switch on os.comp so the same pattern can be extended to other
formats without adding a new public interface. The other formats
U-Boot supports (lzma, lz4, zstd) also carry a size hint and are
added in follow-up patches.

Suggested-by: Simon Glass <sjg@chromium.org>
Signed-off-by: Aristo Chen <aristo.chen@canonical.com>
---
 boot/bootm.c | 47 ++++++++++++++++++++++++++++++++++++++++++-----
 1 file changed, 42 insertions(+), 5 deletions(-)

diff --git a/boot/bootm.c b/boot/bootm.c
index 3bce8586834..ab787979f2e 100644
--- a/boot/bootm.c
+++ b/boot/bootm.c
@@ -23,6 +23,7 @@
 #include <asm/cache.h>
 #include <asm/global_data.h>
 #include <asm/io.h>
+#include <asm/unaligned.h>
 #include <linux/sizes.h>
 #include <tpm-v2.h>
 #include <tpm_tcg2.h>
@@ -638,6 +639,24 @@ static int handle_decomp_error(int comp_type, size_t uncomp_size,
 #endif
 
 #ifndef USE_HOSTCC
+#if CONFIG_IS_ENABLED(GZIP)
+/*
+ * Return the gzip stream's uncompressed size from its ISIZE trailer, or
+ * 0 if the buffer is not a gzip stream. Only the two magic bytes are
+ * checked, since a fuller validation happens inside gunzip() during
+ * decompression; the caller uses the return value as a size hint only.
+ */
+static ulong bootm_gzip_uncompressed_size(const void *src, ulong len)
+{
+	const u8 *b = src;
+
+	/* Minimum gzip: 10-byte header + 2-byte deflate + 8-byte trailer */
+	if (len < 20 || b[0] != 0x1f || b[1] != 0x8b)
+		return 0;
+	return get_unaligned_le32(b + len - 4);
+}
+#endif
+
 static int bootm_load_os(struct bootm_headers *images, int boot_progress)
 {
 	const struct image_info os = images->os;
@@ -654,17 +673,36 @@ static int bootm_load_os(struct bootm_headers *images, int boot_progress)
 	void *load_buf, *image_buf;
 	int err;
 
+	image_buf = map_sysmem(os.image_start, image_len);
+
 	/*
 	 * For a "noload" compressed kernel we need to allocate a buffer large
 	 * enough to decompress in to and use that as the load address now.
-	 * Allow up to 8x compression: this comfortably covers what zstd and xz
-	 * achieve on real kernels, with headroom for well-compressed payloads.
-	 * Use an alignment of 2MB since this might help arm64
+	 * For a gzip stream the trailing 4-byte ISIZE field holds the
+	 * original size modulo 2^32; when it is present and within
+	 * CONFIG_SYS_BOOTM_LEN, allocate exactly that. Otherwise fall back
+	 * to an 8x multiplier, which comfortably covers what zstd and xz
+	 * achieve on real kernels with headroom for well-compressed
+	 * payloads. Use an alignment of 2MB since this might help arm64.
 	 */
 	if (os.type == IH_TYPE_KERNEL_NOLOAD && os.comp != IH_COMP_NONE) {
 		phys_addr_t addr;
+		ulong hdr_size = 0;
 
-		decomp_len = ALIGN(image_len * 8, SZ_1M);
+		switch (os.comp) {
+#if CONFIG_IS_ENABLED(GZIP)
+		case IH_COMP_GZIP:
+			hdr_size = bootm_gzip_uncompressed_size(image_buf,
+								image_len);
+			break;
+#endif
+		default:
+			break;
+		}
+		if (hdr_size && hdr_size <= CONFIG_SYS_BOOTM_LEN)
+			decomp_len = ALIGN(hdr_size, SZ_1M);
+		else
+			decomp_len = ALIGN(image_len * 8, SZ_1M);
 		decomp_limit = BOOTM_DECOMP_LIMIT_PER_IMAGE;
 		err = lmb_alloc_mem(LMB_MEM_ALLOC_ANY, SZ_2M, &addr,
 				    decomp_len, LMB_NONE);
@@ -679,7 +717,6 @@ static int bootm_load_os(struct bootm_headers *images, int boot_progress)
 	}
 
 	load_buf = map_sysmem(load, 0);
-	image_buf = map_sysmem(os.image_start, image_len);
 	err = image_decomp(os.comp, load, os.image_start, os.type,
 			   load_buf, image_buf, image_len,
 			   decomp_len, &load_end);
-- 
2.43.0

[PATCH v2 2/8] test: fit: cover the kernel_noload gzip header-size and lying-header paths

[Aristo Chen]

Reshape and extend the kernel_noload decompression pytests to match
the new bootm behaviour that reads ISIZE from the gzip trailer:

  - Rename test_fit_kernel_noload_decomp_overflow to
    test_fit_kernel_noload_decomp_gzip_lying_hdr. Its setup (a 4 MiB
    payload of zeros gzipped) used to force the failure via the 8x
    heuristic starving the buffer; now that bootm reads ISIZE, the
    honest trailer sizes the buffer correctly, so overwrite ISIZE
    with a tiny value instead and verify the resulting decompression
    is still stopped at the buffer boundary. This is the direct test
    of the CONFIG_SYS_BOOTM_LEN cap on the attacker-controlled
    header value.

  - Add test_fit_kernel_noload_decomp_gzip_hdr_sized: a 6 MiB gzipped
    payload whose compression ratio is past the 8x heuristic
    decompresses cleanly because ISIZE is consulted.

  - Rename the pre-existing test_fit_kernel_noload_decomp_boundary to
    test_fit_kernel_noload_decomp_gzip_boundary so every
    noload_decomp test carries the compressor in its name.

Signed-off-by: Aristo Chen <aristo.chen@canonical.com>
---
 test/py/tests/test_fit.py | 84 ++++++++++++++++++++++++++++++---------
 1 file changed, 66 insertions(+), 18 deletions(-)

diff --git a/test/py/tests/test_fit.py b/test/py/tests/test_fit.py
index 76adb98e2c5..81df84f54c9 100755
--- a/test/py/tests/test_fit.py
+++ b/test/py/tests/test_fit.py
@@ -118,8 +118,9 @@ host save hostfs 0 %(loadables2_addr)x %(loadables2_out)s %(loadables2_size)x
 '''
 
 # A minimal ITS for a compressed 'kernel_noload' kernel. bootm allocates a
-# per-image decompression buffer for this image type, sized as a multiple of
-# the compressed length; see the test_fit_kernel_noload_decomp_* tests.
+# per-image decompression buffer for this image type, sized either from the
+# gzip ISIZE trailer or as a multiple of the compressed length; see the
+# test_fit_kernel_noload_decomp_* tests.
 NOLOAD_ITS = '''
 /dts-v1/;
 
@@ -511,14 +512,13 @@ class TestFitImage:
             + output)
 
     @pytest.mark.buildconfigspec('gzip')
-    def test_fit_kernel_noload_decomp_overflow(self, ubman, fsetup):
-        """Test that an over-large compressed kernel_noload image is rejected
+    def test_fit_kernel_noload_decomp_gzip_lying_hdr(self, ubman, fsetup):
+        """A tampered gzip ISIZE cannot shrink the buffer past the payload
 
-        For a compressed 'kernel_noload' kernel, bootm_load_os() allocates a
-        decompression buffer of ALIGN(image_len * 8, SZ_1M) and must bound the
-        decompressor by that buffer. A kernel that decompresses to far more
-        than eight times its compressed size must therefore fail with a
-        decompression error instead of overflowing the buffer.
+        bootm_load_os() sizes the kernel_noload decompression buffer from the
+        gzip ISIZE trailer. That value is attacker-controlled; rewriting
+        ISIZE to understate the real size must not let decompression overflow
+        the resulting buffer.
         """
         sz_1m = 1 << 20
 
@@ -527,20 +527,21 @@ class TestFitImage:
         # per-image kernel_noload buffer rather than by that global limit.
         bootm_len = int(ubman.config.buildconfig['config_sys_bootm_len'], 0)
 
-        # 4MB of zeros compresses to a few KB, so the decompression buffer
-        # (ALIGN(image_len * 8, SZ_1M), i.e. 1MB here) ends up far smaller
-        # than the uncompressed image.
         decomp_size = 4 * sz_1m
+        assert decomp_size <= bootm_len, (
+            'Test setup error: uncompressed size (%#x) must be <= '
+            'CONFIG_SYS_BOOTM_LEN (%#x)' % (decomp_size, bootm_len))
         kernel = fit_util.make_fname(ubman, 'test-noload-kernel.bin')
         with open(kernel, 'wb') as fd:
             fd.write(b'\0' * decomp_size)
         kernel_gz = self.make_compressed(ubman, kernel)
 
-        image_len = self.filesize(kernel_gz)
-        req_size = (image_len * 8 + sz_1m - 1) // sz_1m * sz_1m
-        assert req_size < decomp_size <= bootm_len, (
-            'Test setup error: need decomp buffer (%#x) < image (%#x) <= '
-            'CONFIG_SYS_BOOTM_LEN (%#x)' % (req_size, decomp_size, bootm_len))
+        # Rewrite gzip ISIZE (the last 4 bytes) to claim a tiny image, so
+        # bootm allocates ALIGN(<lie>, SZ_1M) = 1 MiB and the real 4 MiB
+        # decompression has to overrun that buffer.
+        with open(kernel_gz, 'r+b') as fd:
+            fd.seek(-4, os.SEEK_END)
+            fd.write((256).to_bytes(4, 'little'))
 
         fit = fit_util.make_fit(ubman, fsetup['mkimage'], NOLOAD_ITS,
                                 {'kernel': kernel_gz})
@@ -563,7 +564,54 @@ class TestFitImage:
             ubman.restart_uboot()
 
     @pytest.mark.buildconfigspec('gzip')
-    def test_fit_kernel_noload_decomp_boundary(self, ubman, fsetup):
+    def test_fit_kernel_noload_decomp_gzip_hdr_sized(self, ubman, fsetup):
+        """A well-compressed kernel_noload image fits when ISIZE is honest
+
+        bootm_load_os() reads gzip ISIZE to size the decompression buffer.
+        For a well-compressed image whose ratio exceeds the 8x fallback
+        heuristic (e.g. 6 MiB of zeros gzipping to a few KiB), an ISIZE-sized
+        buffer is the only way the decompression fits.
+        """
+        sz_1m = 1 << 20
+        bootm_len = int(ubman.config.buildconfig['config_sys_bootm_len'], 0)
+
+        # Stay under CONFIG_SYS_BOOTM_LEN so the ISIZE hint isn't rejected as
+        # bogus; still large enough that image_len * 8 falls well short.
+        decomp_size = 6 * sz_1m
+        assert decomp_size <= bootm_len, (
+            'Test setup error: decomp_size (%#x) must be <= '
+            'CONFIG_SYS_BOOTM_LEN (%#x)' % (decomp_size, bootm_len))
+        kernel = fit_util.make_fname(ubman, 'test-noload-kernel-hdrsized.bin')
+        with open(kernel, 'wb') as fd:
+            fd.write(b'\0' * decomp_size)
+        kernel_gz = self.make_compressed(ubman, kernel)
+
+        image_len = self.filesize(kernel_gz)
+        heuristic_bound = (image_len * 8 + sz_1m - 1) // sz_1m * sz_1m
+        assert heuristic_bound < decomp_size, (
+            'Test setup error: 8x heuristic bound (%#x) must be < uncompressed '
+            'size (%#x); if this fires, the compressor got less effective and '
+            'the test needs a bigger payload' % (heuristic_bound, decomp_size))
+
+        fit = fit_util.make_fit(ubman, fsetup['mkimage'], NOLOAD_ITS,
+                                {'kernel': kernel_gz},
+                                basename='test-noload-hdrsized.fit')
+        fit_addr = fsetup['fit_addr']
+
+        # Decompression must succeed: bootm read ISIZE and allocated a big
+        # enough buffer despite the ratio being past the fallback heuristic.
+        output = ubman.run_command_list([
+            'host load hostfs 0 %x %s' % (fit_addr, fit),
+            'bootm start %x' % fit_addr,
+            'bootm loados',
+        ])
+        text = '\n'.join(output)
+        assert 'Image too large' not in text, (
+            'bootm rejected a well-compressed kernel_noload image whose '
+            'ISIZE trailer records the real uncompressed size: %s' % text)
+
+    @pytest.mark.buildconfigspec('gzip')
+    def test_fit_kernel_noload_decomp_gzip_boundary(self, ubman, fsetup):
         """Test that decompression succeeds exactly at the buffer limit
 
         For a compressed 'kernel_noload' kernel, bootm_load_os() allocates a
-- 
2.43.0

[PATCH v2 3/8] bootm: size the noload zstd decompression buffer from Frame_Content_Size

[Aristo Chen]

Add a small static helper bootm_zstd_uncompressed_size() that returns
the frame's Frame_Content_Size via zstd_get_frame_header(), and wire
it into bootm_load_os() as a new case in the size-hint switch
alongside the existing gzip case.

zstd_get_frame_header() and the frame-parsing code behind it ship
with the zstd decompressor, which is already linked into any board
that enables ZSTD, so calling it here adds no new zstd code to the
image. The returned value is used as an allocation hint only and is
capped by the caller; full validation still runs inside
zstd_decompress() during the actual decompression.

Signed-off-by: Aristo Chen <aristo.chen@canonical.com>
---
 boot/bootm.c | 44 ++++++++++++++++++++++++++++++++++++++------
 1 file changed, 38 insertions(+), 6 deletions(-)

diff --git a/boot/bootm.c b/boot/bootm.c
index ab787979f2e..e4284fe9844 100644
--- a/boot/bootm.c
+++ b/boot/bootm.c
@@ -25,6 +25,7 @@
 #include <asm/io.h>
 #include <asm/unaligned.h>
 #include <linux/sizes.h>
+#include <linux/zstd.h>
 #include <tpm-v2.h>
 #include <tpm_tcg2.h>
 #if defined(CONFIG_CMD_USB)
@@ -657,6 +658,31 @@ static ulong bootm_gzip_uncompressed_size(const void *src, ulong len)
 }
 #endif
 
+#if CONFIG_IS_ENABLED(ZSTD)
+/*
+ * Return the zstd frame's Frame_Content_Size, or 0 if the header does
+ * not parse or the size is absent. zstd_get_frame_header() and the
+ * frame-parsing code behind it are part of the zstd decompressor that
+ * is already linked into any board with ZSTD enabled, so the call adds
+ * only the call site. The value is an allocation hint; the decoder
+ * stays authoritative during the actual decompression.
+ */
+static ulong bootm_zstd_uncompressed_size(const void *src, ulong len)
+{
+	zstd_frame_header hdr;
+	size_t ret;
+
+	ret = zstd_get_frame_header(&hdr, src, len);
+	if (zstd_is_error(ret) || ret > 0)
+		return 0;
+	if (hdr.frameContentSize == ZSTD_CONTENTSIZE_UNKNOWN ||
+	    hdr.frameContentSize == ZSTD_CONTENTSIZE_ERROR ||
+	    hdr.frameContentSize > ULONG_MAX)
+		return 0;
+	return (ulong)hdr.frameContentSize;
+}
+#endif
+
 static int bootm_load_os(struct bootm_headers *images, int boot_progress)
 {
 	const struct image_info os = images->os;
@@ -678,12 +704,12 @@ static int bootm_load_os(struct bootm_headers *images, int boot_progress)
 	/*
 	 * For a "noload" compressed kernel we need to allocate a buffer large
 	 * enough to decompress in to and use that as the load address now.
-	 * For a gzip stream the trailing 4-byte ISIZE field holds the
-	 * original size modulo 2^32; when it is present and within
-	 * CONFIG_SYS_BOOTM_LEN, allocate exactly that. Otherwise fall back
-	 * to an 8x multiplier, which comfortably covers what zstd and xz
-	 * achieve on real kernels with headroom for well-compressed
-	 * payloads. Use an alignment of 2MB since this might help arm64.
+	 * When the compressed stream records its uncompressed size and that
+	 * value is within CONFIG_SYS_BOOTM_LEN, allocate exactly that.
+	 * Otherwise fall back to an 8x multiplier, which comfortably covers
+	 * what zstd and xz achieve on real kernels with headroom for
+	 * well-compressed payloads. Use an alignment of 2MB since this
+	 * might help arm64.
 	 */
 	if (os.type == IH_TYPE_KERNEL_NOLOAD && os.comp != IH_COMP_NONE) {
 		phys_addr_t addr;
@@ -695,6 +721,12 @@ static int bootm_load_os(struct bootm_headers *images, int boot_progress)
 			hdr_size = bootm_gzip_uncompressed_size(image_buf,
 								image_len);
 			break;
+#endif
+#if CONFIG_IS_ENABLED(ZSTD)
+		case IH_COMP_ZSTD:
+			hdr_size = bootm_zstd_uncompressed_size(image_buf,
+								image_len);
+			break;
 #endif
 		default:
 			break;
-- 
2.43.0

[PATCH v2 4/8] test: fit: cover the kernel_noload zstd header-size path

[Aristo Chen]

Parametrise NOLOAD_ITS on the compression field so a single template
can drive gzip, zstd, and future format tests, and update the
existing gzip callers to pass compression='gzip'.

Add test_fit_kernel_noload_decomp_zstd_hdr_sized: a 6 MiB payload
whose zstd compression ratio is past the 8x heuristic decompresses
cleanly because Frame_Content_Size is consulted. The test is guarded
by @pytest.mark.requiredtool('zstd') so it skips on hosts that do
not ship the zstd command.

Signed-off-by: Aristo Chen <aristo.chen@canonical.com>
---
 test/py/tests/test_fit.py | 59 +++++++++++++++++++++++++++++++++++----
 1 file changed, 53 insertions(+), 6 deletions(-)

diff --git a/test/py/tests/test_fit.py b/test/py/tests/test_fit.py
index 81df84f54c9..f59010c8c35 100755
--- a/test/py/tests/test_fit.py
+++ b/test/py/tests/test_fit.py
@@ -119,8 +119,9 @@ host save hostfs 0 %(loadables2_addr)x %(loadables2_out)s %(loadables2_size)x
 
 # A minimal ITS for a compressed 'kernel_noload' kernel. bootm allocates a
 # per-image decompression buffer for this image type, sized either from the
-# gzip ISIZE trailer or as a multiple of the compressed length; see the
-# test_fit_kernel_noload_decomp_* tests.
+# compressor header (gzip ISIZE, zstd Frame_Content_Size, ...) or as a
+# multiple of the compressed length; see the test_fit_kernel_noload_decomp_*
+# tests.
 NOLOAD_ITS = '''
 /dts-v1/;
 
@@ -134,7 +135,7 @@ NOLOAD_ITS = '''
                         type = "kernel_noload";
                         arch = "sandbox";
                         os = "linux";
-                        compression = "gzip";
+                        compression = "%(compression)s";
                         load = <0>;
                         entry = <0>;
                 };
@@ -544,7 +545,7 @@ class TestFitImage:
             fd.write((256).to_bytes(4, 'little'))
 
         fit = fit_util.make_fit(ubman, fsetup['mkimage'], NOLOAD_ITS,
-                                {'kernel': kernel_gz})
+                                {'kernel': kernel_gz, 'compression': 'gzip'})
         fit_addr = fsetup['fit_addr']
 
         ubman.run_command_list([
@@ -594,7 +595,7 @@ class TestFitImage:
             'the test needs a bigger payload' % (heuristic_bound, decomp_size))
 
         fit = fit_util.make_fit(ubman, fsetup['mkimage'], NOLOAD_ITS,
-                                {'kernel': kernel_gz},
+                                {'kernel': kernel_gz, 'compression': 'gzip'},
                                 basename='test-noload-hdrsized.fit')
         fit_addr = fsetup['fit_addr']
 
@@ -610,6 +611,52 @@ class TestFitImage:
             'bootm rejected a well-compressed kernel_noload image whose '
             'ISIZE trailer records the real uncompressed size: %s' % text)
 
+    @pytest.mark.buildconfigspec('zstd')
+    @pytest.mark.requiredtool('zstd')
+    def test_fit_kernel_noload_decomp_zstd_hdr_sized(self, ubman, fsetup):
+        """A well-compressed zstd kernel_noload image fits when the frame
+        header carries Frame_Content_Size.
+
+        Same as test_fit_kernel_noload_decomp_gzip_hdr_sized but for zstd.
+        The default zstd encoder embeds Frame_Content_Size for a
+        single-segment frame, so bootm can read it and size the buffer
+        accordingly.
+        """
+        sz_1m = 1 << 20
+        bootm_len = int(ubman.config.buildconfig['config_sys_bootm_len'], 0)
+
+        decomp_size = 6 * sz_1m
+        assert decomp_size <= bootm_len, (
+            'Test setup error: decomp_size (%#x) must be <= '
+            'CONFIG_SYS_BOOTM_LEN (%#x)' % (decomp_size, bootm_len))
+        kernel = fit_util.make_fname(ubman, 'test-noload-kernel-zstd.bin')
+        with open(kernel, 'wb') as fd:
+            fd.write(b'\0' * decomp_size)
+        kernel_zstd = kernel + '.zst'
+        utils.run_and_log(ubman, ['zstd', '-f', kernel, '-o', kernel_zstd])
+
+        image_len = self.filesize(kernel_zstd)
+        heuristic_bound = (image_len * 8 + sz_1m - 1) // sz_1m * sz_1m
+        assert heuristic_bound < decomp_size, (
+            'Test setup error: 8x heuristic bound (%#x) must be < uncompressed '
+            'size (%#x); if this fires, zstd got less effective and the test '
+            'needs a bigger payload' % (heuristic_bound, decomp_size))
+
+        fit = fit_util.make_fit(ubman, fsetup['mkimage'], NOLOAD_ITS,
+                                {'kernel': kernel_zstd, 'compression': 'zstd'},
+                                basename='test-noload-zstd-hdrsized.fit')
+        fit_addr = fsetup['fit_addr']
+
+        output = ubman.run_command_list([
+            'host load hostfs 0 %x %s' % (fit_addr, fit),
+            'bootm start %x' % fit_addr,
+            'bootm loados',
+        ])
+        text = '\n'.join(output)
+        assert 'Image too large' not in text, (
+            'bootm rejected a well-compressed zstd kernel_noload image whose '
+            'frame header records the real content size: %s' % text)
+
     @pytest.mark.buildconfigspec('gzip')
     def test_fit_kernel_noload_decomp_gzip_boundary(self, ubman, fsetup):
         """Test that decompression succeeds exactly at the buffer limit
@@ -637,7 +684,7 @@ class TestFitImage:
             % (decomp_size, req_size))
 
         fit = fit_util.make_fit(ubman, fsetup['mkimage'], NOLOAD_ITS,
-                                {'kernel': kernel_gz},
+                                {'kernel': kernel_gz, 'compression': 'gzip'},
                                 basename='test-noload-boundary.fit')
         fit_addr = fsetup['fit_addr']
 
-- 
2.43.0

[PATCH v2 5/8] bootm: size the noload lz4 decompression buffer from Content_Size

[Aristo Chen]

Add a small static helper bootm_lz4_uncompressed_size() that parses
the lz4 frame header and returns Content_Size when the FLG bit is
set, and wire it into bootm_load_os() alongside gzip and zstd.

The header parse mirrors ulz4fn()'s validation (magic, version==1,
reserved bits, independent-block flag) so the helper does not accept
a frame the decoder itself would reject. Only Content_Size is
extracted; the full validation still runs inside ulz4fn() during the
actual decompression call.

The lz4 command needs the --content-size option to set the FLG bit
that carries the size; frames produced without it fall back to the
existing 8x heuristic.

Signed-off-by: Aristo Chen <aristo.chen@canonical.com>
---
 boot/bootm.c | 36 ++++++++++++++++++++++++++++++++++++
 1 file changed, 36 insertions(+)

diff --git a/boot/bootm.c b/boot/bootm.c
index e4284fe9844..c5bdc909053 100644
--- a/boot/bootm.c
+++ b/boot/bootm.c
@@ -658,6 +658,36 @@ static ulong bootm_gzip_uncompressed_size(const void *src, ulong len)
 }
 #endif
 
+#if CONFIG_IS_ENABLED(LZ4)
+/*
+ * Return the lz4 frame's Content_Size, or 0 if the buffer is not an
+ * lz4 frame or the frame does not carry the size. The header parse
+ * mirrors ulz4fn()'s validation so we do not accept a stream the
+ * decoder itself would refuse.
+ */
+static ulong bootm_lz4_uncompressed_size(const void *src, ulong len)
+{
+	const u8 *b = src;
+	u8 flg, version, indep_blocks, has_content_size, bd;
+	u64 cs;
+
+	if (len < 4 + 2 || get_unaligned_le32(b) != LZ4F_MAGIC)
+		return 0;
+	flg = b[4];
+	bd = b[5];
+	version = (flg >> 6) & 3;
+	indep_blocks = (flg >> 5) & 1;
+	has_content_size = (flg >> 3) & 1;
+	if (version != 1 || !indep_blocks || (flg & 3) || (bd & 0x8f) ||
+	    !has_content_size)
+		return 0;
+	if (len < 4 + 2 + 8)
+		return 0;
+	cs = get_unaligned_le64(b + 6);
+	return cs > ULONG_MAX ? 0 : (ulong)cs;
+}
+#endif
+
 #if CONFIG_IS_ENABLED(ZSTD)
 /*
  * Return the zstd frame's Frame_Content_Size, or 0 if the header does
@@ -722,6 +752,12 @@ static int bootm_load_os(struct bootm_headers *images, int boot_progress)
 								image_len);
 			break;
 #endif
+#if CONFIG_IS_ENABLED(LZ4)
+		case IH_COMP_LZ4:
+			hdr_size = bootm_lz4_uncompressed_size(image_buf,
+							       image_len);
+			break;
+#endif
 #if CONFIG_IS_ENABLED(ZSTD)
 		case IH_COMP_ZSTD:
 			hdr_size = bootm_zstd_uncompressed_size(image_buf,
-- 
2.43.0

[PATCH v2 6/8] test: fit: cover the kernel_noload lz4 header-size path

[Aristo Chen]

Add test_fit_kernel_noload_decomp_lz4_hdr_sized: a 6 MiB payload
whose lz4 compression ratio is past the 8x heuristic decompresses
cleanly because Content_Size is consulted. The tool must be invoked
with --content-size so the frame's FLG bit is set. The test is
guarded by @pytest.mark.requiredtool('lz4') so it skips on hosts
that do not ship the lz4 command.

Signed-off-by: Aristo Chen <aristo.chen@canonical.com>
---
 test/py/tests/test_fit.py | 47 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 47 insertions(+)

diff --git a/test/py/tests/test_fit.py b/test/py/tests/test_fit.py
index f59010c8c35..42edddb0600 100755
--- a/test/py/tests/test_fit.py
+++ b/test/py/tests/test_fit.py
@@ -611,6 +611,53 @@ class TestFitImage:
             'bootm rejected a well-compressed kernel_noload image whose '
             'ISIZE trailer records the real uncompressed size: %s' % text)
 
+    @pytest.mark.buildconfigspec('lz4')
+    @pytest.mark.requiredtool('lz4')
+    def test_fit_kernel_noload_decomp_lz4_hdr_sized(self, ubman, fsetup):
+        """A well-compressed lz4 kernel_noload image fits when the frame
+        header carries the content size.
+
+        Same as test_fit_kernel_noload_decomp_gzip_hdr_sized but for lz4:
+        the tool must be invoked with --content-size so the frame's FLG
+        bit is set and bootm can read the size instead of falling back to
+        the 8x heuristic.
+        """
+        sz_1m = 1 << 20
+        bootm_len = int(ubman.config.buildconfig['config_sys_bootm_len'], 0)
+
+        decomp_size = 6 * sz_1m
+        assert decomp_size <= bootm_len, (
+            'Test setup error: decomp_size (%#x) must be <= '
+            'CONFIG_SYS_BOOTM_LEN (%#x)' % (decomp_size, bootm_len))
+        kernel = fit_util.make_fname(ubman, 'test-noload-kernel-lz4.bin')
+        with open(kernel, 'wb') as fd:
+            fd.write(b'\0' * decomp_size)
+        kernel_lz4 = kernel + '.lz4'
+        utils.run_and_log(
+            ubman, ['lz4', '--content-size', '-f', kernel, kernel_lz4])
+
+        image_len = self.filesize(kernel_lz4)
+        heuristic_bound = (image_len * 8 + sz_1m - 1) // sz_1m * sz_1m
+        assert heuristic_bound < decomp_size, (
+            'Test setup error: 8x heuristic bound (%#x) must be < uncompressed '
+            'size (%#x); if this fires, lz4 got less effective and the test '
+            'needs a bigger payload' % (heuristic_bound, decomp_size))
+
+        fit = fit_util.make_fit(ubman, fsetup['mkimage'], NOLOAD_ITS,
+                                {'kernel': kernel_lz4, 'compression': 'lz4'},
+                                basename='test-noload-lz4-hdrsized.fit')
+        fit_addr = fsetup['fit_addr']
+
+        output = ubman.run_command_list([
+            'host load hostfs 0 %x %s' % (fit_addr, fit),
+            'bootm start %x' % fit_addr,
+            'bootm loados',
+        ])
+        text = '\n'.join(output)
+        assert 'Image too large' not in text, (
+            'bootm rejected a well-compressed lz4 kernel_noload image whose '
+            'frame header records the real content size: %s' % text)
+
     @pytest.mark.buildconfigspec('zstd')
     @pytest.mark.requiredtool('zstd')
     def test_fit_kernel_noload_decomp_zstd_hdr_sized(self, ubman, fsetup):
-- 
2.43.0

[PATCH v2 7/8] bootm: size the noload lzma decompression buffer from the header

[Aristo Chen]

Add a small static helper bootm_lzma_uncompressed_size() that reads
the uncompressed size out of the .lzma-alone header, and wire it
into bootm_load_os() alongside gzip, lz4, and zstd.

The .lzma-alone format keeps the uncompressed size in a fixed 8-byte
field right after the 5-byte properties block; a marker of all ones
means the size is unknown, and the caller falls back to the 8x
heuristic in that case. Streaming encoders (xz-utils' 'lzma' shim,
Python's lzma.FORMAT_ALONE) write the unknown marker, while LZMA SDK
style encoders record the real size.

Signed-off-by: Aristo Chen <aristo.chen@canonical.com>
---
 boot/bootm.c | 29 +++++++++++++++++++++++++++++
 1 file changed, 29 insertions(+)

diff --git a/boot/bootm.c b/boot/bootm.c
index c5bdc909053..758edeb964c 100644
--- a/boot/bootm.c
+++ b/boot/bootm.c
@@ -26,6 +26,7 @@
 #include <asm/unaligned.h>
 #include <linux/sizes.h>
 #include <linux/zstd.h>
+#include <lzma/LzmaDec.h>
 #include <tpm-v2.h>
 #include <tpm_tcg2.h>
 #if defined(CONFIG_CMD_USB)
@@ -658,6 +659,28 @@ static ulong bootm_gzip_uncompressed_size(const void *src, ulong len)
 }
 #endif
 
+#if CONFIG_IS_ENABLED(LZMA)
+/*
+ * Return the uncompressed size recorded in the lzma stream header, or
+ * 0 if the buffer is too short or the size field carries the "unknown"
+ * marker (0xff..ff). The .lzma-alone format keeps the size in a fixed
+ * 8-byte field right after the 5-byte properties block; nothing else
+ * is validated since the value is only an allocation hint.
+ */
+static ulong bootm_lzma_uncompressed_size(const void *src, ulong len)
+{
+	const u8 *b = src;
+	u64 usize;
+
+	if (len < LZMA_PROPS_SIZE + 8)
+		return 0;
+	usize = get_unaligned_le64(b + LZMA_PROPS_SIZE);
+	if (usize == U64_MAX || usize > ULONG_MAX)
+		return 0;
+	return (ulong)usize;
+}
+#endif
+
 #if CONFIG_IS_ENABLED(LZ4)
 /*
  * Return the lz4 frame's Content_Size, or 0 if the buffer is not an
@@ -752,6 +775,12 @@ static int bootm_load_os(struct bootm_headers *images, int boot_progress)
 								image_len);
 			break;
 #endif
+#if CONFIG_IS_ENABLED(LZMA)
+		case IH_COMP_LZMA:
+			hdr_size = bootm_lzma_uncompressed_size(image_buf,
+								image_len);
+			break;
+#endif
 #if CONFIG_IS_ENABLED(LZ4)
 		case IH_COMP_LZ4:
 			hdr_size = bootm_lz4_uncompressed_size(image_buf,
-- 
2.43.0

[PATCH v2 8/8] test: fit: cover the kernel_noload lzma header-size and unknown-size paths

[Aristo Chen]

Exercise bootm_lzma_uncompressed_size() end-to-end on sandbox:

- test_fit_kernel_noload_decomp_lzma_hdr_sized boots a 6 MiB
  kernel_noload payload that lzma compresses far past the 8x fallback
  heuristic, so the boot only succeeds when bootm sizes the buffer
  from the header's uncompressed-size field. Streaming encoders write
  the "unknown" marker into that field, so the test compresses with
  Python's lzma module and patches the real size into the fixed
  8-byte field, matching what LZMA SDK style encoders record.

- test_fit_kernel_noload_decomp_lzma_unknown_size leaves the marker in
  place and checks that bootm falls back to the 8x heuristic buffer
  and still boots the image.

No external tool is required: Python's lzma module is part of the
standard library.

Signed-off-by: Aristo Chen <aristo.chen@canonical.com>
---
 test/py/tests/test_fit.py | 98 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 98 insertions(+)

diff --git a/test/py/tests/test_fit.py b/test/py/tests/test_fit.py
index 42edddb0600..0edf875a9e1 100755
--- a/test/py/tests/test_fit.py
+++ b/test/py/tests/test_fit.py
@@ -704,6 +704,104 @@ class TestFitImage:
             'bootm rejected a well-compressed zstd kernel_noload image whose '
             'frame header records the real content size: %s' % text)
 
+    @pytest.mark.buildconfigspec('lzma')
+    def test_fit_kernel_noload_decomp_lzma_hdr_sized(self, ubman, fsetup):
+        """A well-compressed lzma kernel_noload image fits when the header
+        records the real uncompressed size.
+
+        Same as test_fit_kernel_noload_decomp_gzip_hdr_sized but for lzma.
+        Streaming encoders write the "unknown" marker into the .lzma-alone
+        size field, so compress with Python's lzma module and patch the
+        real size into the fixed 8-byte header field, the way LZMA SDK
+        style encoders record it.
+        """
+        lzma = pytest.importorskip('lzma')
+        sz_1m = 1 << 20
+        bootm_len = int(ubman.config.buildconfig['config_sys_bootm_len'], 0)
+
+        decomp_size = 6 * sz_1m
+        assert decomp_size <= bootm_len, (
+            'Test setup error: decomp_size (%#x) must be <= '
+            'CONFIG_SYS_BOOTM_LEN (%#x)' % (decomp_size, bootm_len))
+        kernel = fit_util.make_fname(ubman, 'test-noload-kernel-lzma.bin')
+        with open(kernel, 'wb') as fd:
+            fd.write(b'\0' * decomp_size)
+        filters = [{'id': lzma.FILTER_LZMA1, 'preset': 6,
+                    'dict_size': 1 << 20}]
+        blob = lzma.compress(self.read_file(kernel),
+                             format=lzma.FORMAT_ALONE, filters=filters)
+        assert blob[5:13] == b'\xff' * 8, (
+            'Test setup error: expected the streaming encoder to write the '
+            '"unknown" size marker')
+        blob = blob[:5] + decomp_size.to_bytes(8, 'little') + blob[13:]
+        kernel_lzma = kernel + '.lzma'
+        with open(kernel_lzma, 'wb') as fd:
+            fd.write(blob)
+
+        image_len = self.filesize(kernel_lzma)
+        heuristic_bound = (image_len * 8 + sz_1m - 1) // sz_1m * sz_1m
+        assert heuristic_bound < decomp_size, (
+            'Test setup error: 8x heuristic bound (%#x) must be < uncompressed '
+            'size (%#x); if this fires, lzma got less effective and the test '
+            'needs a bigger payload' % (heuristic_bound, decomp_size))
+
+        fit = fit_util.make_fit(ubman, fsetup['mkimage'], NOLOAD_ITS,
+                                {'kernel': kernel_lzma, 'compression': 'lzma'},
+                                basename='test-noload-lzma-hdrsized.fit')
+        fit_addr = fsetup['fit_addr']
+
+        output = ubman.run_command_list([
+            'host load hostfs 0 %x %s' % (fit_addr, fit),
+            'bootm start %x' % fit_addr,
+            'bootm loados',
+        ])
+        text = '\n'.join(output)
+        assert 'Image too large' not in text, (
+            'bootm rejected a well-compressed lzma kernel_noload image whose '
+            'header records the real uncompressed size: %s' % text)
+
+    @pytest.mark.buildconfigspec('lzma')
+    def test_fit_kernel_noload_decomp_lzma_unknown_size(self, ubman, fsetup):
+        """An lzma stream with the "unknown" size marker falls back cleanly
+
+        Streaming encoders write 0xff..ff into the .lzma-alone size field.
+        bootm must fall back to the 8x heuristic buffer and still boot the
+        image.
+        """
+        lzma = pytest.importorskip('lzma')
+        sz_1m = 1 << 20
+
+        # Incompressible data keeps the real size well inside the 8x
+        # fallback buffer.
+        payload = os.urandom(sz_1m)
+        kernel = fit_util.make_fname(ubman, 'test-noload-kernel-lzma-unk.bin')
+        filters = [{'id': lzma.FILTER_LZMA1, 'preset': 6,
+                    'dict_size': 1 << 20}]
+        blob = lzma.compress(payload, format=lzma.FORMAT_ALONE,
+                             filters=filters)
+        assert blob[5:13] == b'\xff' * 8, (
+            'Test setup error: expected the streaming encoder to write the '
+            '"unknown" size marker')
+        kernel_lzma = kernel + '.lzma'
+        with open(kernel_lzma, 'wb') as fd:
+            fd.write(blob)
+
+        fit = fit_util.make_fit(ubman, fsetup['mkimage'], NOLOAD_ITS,
+                                {'kernel': kernel_lzma, 'compression': 'lzma'},
+                                basename='test-noload-lzma-unk.fit')
+        fit_addr = fsetup['fit_addr']
+
+        output = ubman.run_command_list([
+            'host load hostfs 0 %x %s' % (fit_addr, fit),
+            'bootm start %x' % fit_addr,
+            'bootm loados',
+        ])
+        text = '\n'.join(output)
+        assert 'Image too large' not in text, (
+            'bootm rejected an lzma kernel_noload image carrying the '
+            '"unknown" size marker; the 8x fallback should have covered '
+            'it: %s' % text)
+
     @pytest.mark.buildconfigspec('gzip')
     def test_fit_kernel_noload_decomp_gzip_boundary(self, ubman, fsetup):
         """Test that decompression succeeds exactly at the buffer limit
-- 
2.43.0

Re: [PATCH v2 0/8] bootm: size the noload buffer from the compressor header

[Tom Rini]

[-- Attachment #1: Type: text/plain, Size: 5483 bytes --]

On Tue, Aug 18, 2026 at 01:23:14PM +0000, Aristo Chen wrote:

> This is v2 of "bootm: size the noload decompression buffer from the
> compressor header". Tom pushed back on v1
> (https://patchwork.ozlabs.org/project/uboot/patch/20260809042338.63397-2-aristo.chen@canonical.com/)
> on two grounds:
> 
>   1. No concrete problem report driving the change.
>   2. ~1297 platforms grew by ~170-400 bytes; the change is not
>      opt-in, so the size cost falls on everyone.

So, the first example that pops up in my builds is imx8mn_beacon_2g.
And for v1 of the series:
   aarch64: (for 1/1 boards) all +16384.0 data +14336.0 text +2048.0
            imx8mn_beacon_2g: all +16384 data +14336 text +2048
               u-boot: add: 0/0, grow: 1/0 bytes: 328/0 (328)
                 function                                   old     new   delta
                 bootm_run_states                          3316    3644    +328

And now for v2:
   aarch64: (for 1/1 boards) all +16384.0 data +14336.0 text +2048.0
            imx8mn_beacon_2g: all +16384 data +14336 text +2048
               u-boot: add: 0/0, grow: 1/0 bytes: 368/0 (368)
                 function                                   old     new   delta
                 bootm_run_states                          3316    3684    +368

Next, picking turris_mox as it enables ZSTD:
v1:
   aarch64: (for 1/1 boards) all +472.0 text +472.0
            turris_mox     : all +472 text +472
               u-boot: add: 2/0, grow: 1/0 bytes: 472/0 (472)
                 function                                   old     new   delta
                 image_decomp_get_uncompressed_size           -     416    +416
                 bootm_run_states                          2360    2412     +52
                 zstd_get_frame_header                        -       4      +4
v2:
   aarch64: (for 1/1 boards) all +428.0 text +428.0
            turris_mox     : all +428 text +428
               u-boot: add: 1/0, grow: 1/0 bytes: 428/0 (428)
                 function                                   old     new   delta
                 bootm_run_states                          2360    2784    +424
                 zstd_get_frame_header                        -       4      +4

So, that is better. Looking at smartweb, both iterations are the same:
       arm: (for 1/1 boards) all +96.0 text +96.0
            smartweb       : all +96 text +96
               u-boot: add: 0/0, grow: 1/0 bytes: 76/0 (76)
                 function                                   old     new   delta
                 bootm_run_states                          3592    3668     +76

What's honestly concerning is chromebook_coral where v2 *shrinks*:
               u-boot: add: 0/0, grow: 0/-1 bytes: 0/-2 (-2)
                 function                                   old     new   delta
                 bootm_load_os                              520     518      -2
but v1 grows:
               u-boot: add: 1/0, grow: 1/0 bytes: 110/0 (110)
                 function                                   old     new   delta
                 bootm_load_os                              520     587     +67
                 image_decomp_get_uncompressed_size           -      43     +43

> On the first point, Nora Schiffer replied with a concrete use case
> (EFI-in-FIT plus padded loaders such as shim, systemd-boot, and
> OpenWrt's lzma-loader can produce compression ratios that outrun the
> 8x heuristic), and mentioned this is on the road map for TQ-Systems
> standard BSPs.
> 
> On the second point, v2 reworks the implementation to cut the size
> cost, measures it across the format and architecture buckets, and
> splits the work per format so each decompressor's support can be
> taken or dropped on its own.
> 
> Background: for a compressed kernel_noload image, bootm_load_os()
> sizes the decompression buffer as ALIGN(image_len * 8, SZ_1M). The
> 8x heuristic works for typical kernels, but any well-compressed
> payload can exceed it, and no fixed multiplier is safe against
> arbitrarily compressible input.
> 
> Each implementation patch adds a small static header-parse helper in
> bootm.c (no new public API) and wires it into a size-hint switch;
> helper and switch case are only compiled when the matching
> decompressor is enabled, so boards that do not build a format pay no
> code for it. gzip's ISIZE is a fixed trailer read, lzma's size a
> fixed header read, lz4 mirrors ulz4fn()'s frame-header validation,
> and zstd asks zstd_get_frame_header(), whose frame-parsing code
> already ships with the zstd decompressor. The header-recorded value
> is attacker-controlled, so it is capped at CONFIG_SYS_BOOTM_LEN, and
> it is only an allocation hint: the decoder stays authoritative
> during the actual decompression.
> 
> Text size deltas of the u-boot ELF (size(1), distro gcc 13.3 cross
> toolchains); data/bss are unchanged everywhere. To make the columns
> directly comparable, the v1 column is v1's implementation commit
> cherry-picked onto this series' base, so both columns share one
> baseline:

Please use binman to look at the size changes, as it gives much more
useful information. I've noted
https://git.u-boot-project.org/u-boot/u-boot-extras/-/blob/master/contrib/trini/u-boot-size-test.sh?ref_type=heads
for others before as a wrapper around the options to get the most useful
information out.

-- 
Tom

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: [PATCH v2 0/8] bootm: size the noload buffer from the compressor header

[Aristo Chen]

Hi Tom,

On Wed, Aug 19, 2026 at 6:10 AM Tom Rini <trini@konsulko.com> wrote:
>
> On Tue, Aug 18, 2026 at 01:23:14PM +0000, Aristo Chen wrote:
>
> > This is v2 of "bootm: size the noload decompression buffer from the
> > compressor header". Tom pushed back on v1
> > (https://patchwork.ozlabs.org/project/uboot/patch/20260809042338.63397-2-aristo.chen@canonical.com/)
> > on two grounds:
> >
> >   1. No concrete problem report driving the change.
> >   2. ~1297 platforms grew by ~170-400 bytes; the change is not
> >      opt-in, so the size cost falls on everyone.
>
> So, the first example that pops up in my builds is imx8mn_beacon_2g.
> And for v1 of the series:
>    aarch64: (for 1/1 boards) all +16384.0 data +14336.0 text +2048.0
>             imx8mn_beacon_2g: all +16384 data +14336 text +2048
>                u-boot: add: 0/0, grow: 1/0 bytes: 328/0 (328)
>                  function                                   old     new   delta
>                  bootm_run_states                          3316    3644    +328
>
> And now for v2:
>    aarch64: (for 1/1 boards) all +16384.0 data +14336.0 text +2048.0
>             imx8mn_beacon_2g: all +16384 data +14336 text +2048
>                u-boot: add: 0/0, grow: 1/0 bytes: 368/0 (368)
>                  function                                   old     new   delta
>                  bootm_run_states                          3316    3684    +368
>
> Next, picking turris_mox as it enables ZSTD:
> v1:
>    aarch64: (for 1/1 boards) all +472.0 text +472.0
>             turris_mox     : all +472 text +472
>                u-boot: add: 2/0, grow: 1/0 bytes: 472/0 (472)
>                  function                                   old     new   delta
>                  image_decomp_get_uncompressed_size           -     416    +416
>                  bootm_run_states                          2360    2412     +52
>                  zstd_get_frame_header                        -       4      +4
> v2:
>    aarch64: (for 1/1 boards) all +428.0 text +428.0
>             turris_mox     : all +428 text +428
>                u-boot: add: 1/0, grow: 1/0 bytes: 428/0 (428)
>                  function                                   old     new   delta
>                  bootm_run_states                          2360    2784    +424
>                  zstd_get_frame_header                        -       4      +4
>
> So, that is better. Looking at smartweb, both iterations are the same:
>        arm: (for 1/1 boards) all +96.0 text +96.0
>             smartweb       : all +96 text +96
>                u-boot: add: 0/0, grow: 1/0 bytes: 76/0 (76)
>                  function                                   old     new   delta
>                  bootm_run_states                          3592    3668     +76
>
> What's honestly concerning is chromebook_coral where v2 *shrinks*:
>                u-boot: add: 0/0, grow: 0/-1 bytes: 0/-2 (-2)
>                  function                                   old     new   delta
>                  bootm_load_os                              520     518      -2
> but v1 grows:
>                u-boot: add: 1/0, grow: 1/0 bytes: 110/0 (110)
>                  function                                   old     new   delta
>                  bootm_load_os                              520     587     +67
>                  image_decomp_get_uncompressed_size           -      43     +43
>

That one is working as intended: chromebook_coral enables none of
GZIP/LZMA/LZ4/ZSTD, so it is one of the "bootm with no decompressor
at all" boards from the cover letter. In v2 every helper and its
switch case sit behind CONFIG_IS_ENABLED(<format>), so on that board
they all compile away and only the unchanged 8x fallback remains.
The -2 bytes is codegen noise from the restructure; I diffed the
disassembly and the function is otherwise unchanged. v1 grew there
because its helper in image.c was built unconditionally. Behaviour is
unchanged either way: with no decompressor enabled, a compressed
kernel_noload image already fails in image_decomp().

The imx8mn_beacon_2g result has a similar shape to what I measured
on am62x_evm_a53: both enable LTO, and on these LTO configurations
compiler inlining and layout make v2 a few tens of bytes larger
than v1 (+368 vs +328 here, with the growth landing inside
bootm_run_states either way). That is the trade of the per-format
split, which is what makes the no-decompressor boards free, trims
the gzip-only majority, and keeps each format individually
droppable.

turris_mox is the non-LTO counterpart and shows the intended
direction for the multi-algorithm case: v2 comes in 44 bytes below
v1 there (+428 vs +472 in your run).

smartweb is the expected gzip-only LTO case: both versions cost
essentially the same (+96 in your run).

Thanks for the u-boot-size-test.sh pointer. I re-ran your four
boards with it against this series' base and reproduce your numbers
to within a few bytes of toolchain difference, including coral's -2
(here: v1 +108 with bootm_load_os +65 plus the unconditional helper
+43, v2 -2). I will use the script for the size numbers from now on;
if you would like the cover letter regenerated with those numbers, I
am happy to respin as v3 with the code unchanged.

> > On the first point, Nora Schiffer replied with a concrete use case
> > (EFI-in-FIT plus padded loaders such as shim, systemd-boot, and
> > OpenWrt's lzma-loader can produce compression ratios that outrun the
> > 8x heuristic), and mentioned this is on the road map for TQ-Systems
> > standard BSPs.
> >
> > On the second point, v2 reworks the implementation to cut the size
> > cost, measures it across the format and architecture buckets, and
> > splits the work per format so each decompressor's support can be
> > taken or dropped on its own.
> >
> > Background: for a compressed kernel_noload image, bootm_load_os()
> > sizes the decompression buffer as ALIGN(image_len * 8, SZ_1M). The
> > 8x heuristic works for typical kernels, but any well-compressed
> > payload can exceed it, and no fixed multiplier is safe against
> > arbitrarily compressible input.
> >
> > Each implementation patch adds a small static header-parse helper in
> > bootm.c (no new public API) and wires it into a size-hint switch;
> > helper and switch case are only compiled when the matching
> > decompressor is enabled, so boards that do not build a format pay no
> > code for it. gzip's ISIZE is a fixed trailer read, lzma's size a
> > fixed header read, lz4 mirrors ulz4fn()'s frame-header validation,
> > and zstd asks zstd_get_frame_header(), whose frame-parsing code
> > already ships with the zstd decompressor. The header-recorded value
> > is attacker-controlled, so it is capped at CONFIG_SYS_BOOTM_LEN, and
> > it is only an allocation hint: the decoder stays authoritative
> > during the actual decompression.
> >
> > Text size deltas of the u-boot ELF (size(1), distro gcc 13.3 cross
> > toolchains); data/bss are unchanged everywhere. To make the columns
> > directly comparable, the v1 column is v1's implementation commit
> > cherry-picked onto this series' base, so both columns share one
> > baseline:
>
> Please use binman to look at the size changes, as it gives much more
> useful information. I've noted
> https://git.u-boot-project.org/u-boot/u-boot-extras/-/blob/master/contrib/trini/u-boot-size-test.sh?ref_type=heads
> for others before as a wrapper around the options to get the most useful
> information out.
>
> --
> Tom

Regards,
Aristo

Re: [PATCH v2 0/8] bootm: size the noload buffer from the compressor header

[Tom Rini]

[-- Attachment #1: Type: text/plain, Size: 5883 bytes --]

On Wed, Aug 19, 2026 at 10:53:57PM +0800, Aristo Chen wrote:
> Hi Tom,
> 
> On Wed, Aug 19, 2026 at 6:10 AM Tom Rini <trini@konsulko.com> wrote:
> >
> > On Tue, Aug 18, 2026 at 01:23:14PM +0000, Aristo Chen wrote:
> >
> > > This is v2 of "bootm: size the noload decompression buffer from the
> > > compressor header". Tom pushed back on v1
> > > (https://patchwork.ozlabs.org/project/uboot/patch/20260809042338.63397-2-aristo.chen@canonical.com/)
> > > on two grounds:
> > >
> > >   1. No concrete problem report driving the change.
> > >   2. ~1297 platforms grew by ~170-400 bytes; the change is not
> > >      opt-in, so the size cost falls on everyone.
> >
> > So, the first example that pops up in my builds is imx8mn_beacon_2g.
> > And for v1 of the series:
> >    aarch64: (for 1/1 boards) all +16384.0 data +14336.0 text +2048.0
> >             imx8mn_beacon_2g: all +16384 data +14336 text +2048
> >                u-boot: add: 0/0, grow: 1/0 bytes: 328/0 (328)
> >                  function                                   old     new   delta
> >                  bootm_run_states                          3316    3644    +328
> >
> > And now for v2:
> >    aarch64: (for 1/1 boards) all +16384.0 data +14336.0 text +2048.0
> >             imx8mn_beacon_2g: all +16384 data +14336 text +2048
> >                u-boot: add: 0/0, grow: 1/0 bytes: 368/0 (368)
> >                  function                                   old     new   delta
> >                  bootm_run_states                          3316    3684    +368
> >
> > Next, picking turris_mox as it enables ZSTD:
> > v1:
> >    aarch64: (for 1/1 boards) all +472.0 text +472.0
> >             turris_mox     : all +472 text +472
> >                u-boot: add: 2/0, grow: 1/0 bytes: 472/0 (472)
> >                  function                                   old     new   delta
> >                  image_decomp_get_uncompressed_size           -     416    +416
> >                  bootm_run_states                          2360    2412     +52
> >                  zstd_get_frame_header                        -       4      +4
> > v2:
> >    aarch64: (for 1/1 boards) all +428.0 text +428.0
> >             turris_mox     : all +428 text +428
> >                u-boot: add: 1/0, grow: 1/0 bytes: 428/0 (428)
> >                  function                                   old     new   delta
> >                  bootm_run_states                          2360    2784    +424
> >                  zstd_get_frame_header                        -       4      +4
> >
> > So, that is better. Looking at smartweb, both iterations are the same:
> >        arm: (for 1/1 boards) all +96.0 text +96.0
> >             smartweb       : all +96 text +96
> >                u-boot: add: 0/0, grow: 1/0 bytes: 76/0 (76)
> >                  function                                   old     new   delta
> >                  bootm_run_states                          3592    3668     +76
> >
> > What's honestly concerning is chromebook_coral where v2 *shrinks*:
> >                u-boot: add: 0/0, grow: 0/-1 bytes: 0/-2 (-2)
> >                  function                                   old     new   delta
> >                  bootm_load_os                              520     518      -2
> > but v1 grows:
> >                u-boot: add: 1/0, grow: 1/0 bytes: 110/0 (110)
> >                  function                                   old     new   delta
> >                  bootm_load_os                              520     587     +67
> >                  image_decomp_get_uncompressed_size           -      43     +43
> >
> 
> That one is working as intended: chromebook_coral enables none of
> GZIP/LZMA/LZ4/ZSTD, so it is one of the "bootm with no decompressor
> at all" boards from the cover letter. In v2 every helper and its
> switch case sit behind CONFIG_IS_ENABLED(<format>), so on that board
> they all compile away and only the unchanged 8x fallback remains.
> The -2 bytes is codegen noise from the restructure; I diffed the
> disassembly and the function is otherwise unchanged. v1 grew there
> because its helper in image.c was built unconditionally. Behaviour is
> unchanged either way: with no decompressor enabled, a compressed
> kernel_noload image already fails in image_decomp().
> 
> The imx8mn_beacon_2g result has a similar shape to what I measured
> on am62x_evm_a53: both enable LTO, and on these LTO configurations
> compiler inlining and layout make v2 a few tens of bytes larger
> than v1 (+368 vs +328 here, with the growth landing inside
> bootm_run_states either way). That is the trade of the per-format
> split, which is what makes the no-decompressor boards free, trims
> the gzip-only majority, and keeps each format individually
> droppable.
> 
> turris_mox is the non-LTO counterpart and shows the intended
> direction for the multi-algorithm case: v2 comes in 44 bytes below
> v1 there (+428 vs +472 in your run).
> 
> smartweb is the expected gzip-only LTO case: both versions cost
> essentially the same (+96 in your run).
> 
> Thanks for the u-boot-size-test.sh pointer. I re-ran your four
> boards with it against this series' base and reproduce your numbers
> to within a few bytes of toolchain difference, including coral's -2
> (here: v1 +108 with bootm_load_os +65 plus the unconditional helper
> +43, v2 -2). I will use the script for the size numbers from now on;
> if you would like the cover letter regenerated with those numbers, I
> am happy to respin as v3 with the code unchanged.

Thanks for explaining and digging a bit more. At the end of the day, I
wish we could solve this problem, but have smaller growth, but I don't
see it. So v2 is fine as-is from my point of view, no need to spin a v3
unless there's other feedback.

-- 
Tom

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]