[PATCH v2 0/2] Enable compressed files in EXTRA_FIRMWARE

[PATCH v2 0/2] Enable compressed files in EXTRA_FIRMWARE

[Kevin Martin]

The linux-firmware packages on Gentoo, Fedora, Arch, and others compress
the firmware files. This works well with CONFIG_FW_LOADER_COMPRESS, but
does not work with CONFIG_EXTRA_FIRMWARE. This patch allows the build
system to decompress firmware files specified by CONFIG_EXTRA_FIRMWARE.

PATCH 1/2 adds decompression routines next to the compression routines
in scripts/Makefile.lib. That patch is then used by PATCH 2/2 to
decompress files before compiling them into the kernel.

The patch works by copying or decompressing the specified firmware files
into the build directory, then compiling them in from there. I would
prefer to not copy any uncompressed files, but I have not found a clean
way to do that.

 drivers/base/firmware_loader/Kconfig            |  5 ++++-
 drivers/base/firmware_loader/builtin/.gitignore |  5 ++++-
 drivers/base/firmware_loader/builtin/Makefile   | 16 ++++++++++++----
 scripts/Makefile.lib                            |  6 ++++++
 4 files changed, 26 insertions(+), 6 deletions(-)

-- 
2.41.0

[PATCH v2 1/2] kbuild: Enable decompression for use by EXTRA_FIRMWARE

[Kevin Martin]

The build system can currently only compress files. This patch adds the
functionality to decompress files. Decompression is needed for building
firmware files into the kernel if those files are compressed on the
filesystem. Compressed firmware files are in use by Gentoo, Fedora, Arch,
and others.

Signed-off-by: Kevin Martin <kevinmbecause@gmail.com>
---
Changes in v2:
- Skipped running 'cat' and now just pass the file names directly.
- Added '--quiet' since 'zstd' started printing the status of each file
now that it knows the file names.

 scripts/Makefile.lib | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
index 1a965fe68..d043be3dc 100644
--- a/scripts/Makefile.lib
+++ b/scripts/Makefile.lib
@@ -523,6 +523,9 @@ quiet_cmd_xzkern_with_size = XZKERN  $@
 quiet_cmd_xzmisc = XZMISC  $@
       cmd_xzmisc = cat $(real-prereqs) | $(XZ) --check=crc32 --lzma2=dict=1MiB > $@
 
+quiet_cmd_xzdec = XZDEC   $@
+      cmd_xzdec = $(XZ) --decompress --stdout $< > $@
+
 # ZSTD
 # ---------------------------------------------------------------------------
 # Appends the uncompressed size of the data using size_append. The .zst
@@ -548,6 +551,9 @@ quiet_cmd_zstd22 = ZSTD22  $@
 quiet_cmd_zstd22_with_size = ZSTD22  $@
       cmd_zstd22_with_size = { cat $(real-prereqs) | $(ZSTD) -22 --ultra; $(size_append); } > $@
 
+quiet_cmd_zstddec = ZSTDDEC $@
+      cmd_zstddec = $(ZSTD) --decompress --force --quiet -o $@ $<
+
 # ASM offsets
 # ---------------------------------------------------------------------------
 
-- 
2.41.0

Re: [PATCH v2 1/2] kbuild: Enable decompression for use by EXTRA_FIRMWARE

[Masahiro Yamada]

On Fri, Jan 5, 2024 at 3:11 PM Kevin Martin <kevinmbecause@gmail.com> wrote:
>
> The build system can currently only compress files. This patch adds the
> functionality to decompress files. Decompression is needed for building
> firmware files into the kernel if those files are compressed on the
> filesystem. Compressed firmware files are in use by Gentoo, Fedora, Arch,
> and others.
>
> Signed-off-by: Kevin Martin <kevinmbecause@gmail.com>
> ---
> Changes in v2:
> - Skipped running 'cat' and now just pass the file names directly.
> - Added '--quiet' since 'zstd' started printing the status of each file
> now that it knows the file names.
>


Acked-by: Masahiro Yamada <masahiroy@kernel.org>



>  scripts/Makefile.lib | 6 ++++++
>  1 file changed, 6 insertions(+)
>
> diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
> index 1a965fe68..d043be3dc 100644
> --- a/scripts/Makefile.lib
> +++ b/scripts/Makefile.lib
> @@ -523,6 +523,9 @@ quiet_cmd_xzkern_with_size = XZKERN  $@
>  quiet_cmd_xzmisc = XZMISC  $@
>        cmd_xzmisc = cat $(real-prereqs) | $(XZ) --check=crc32 --lzma2=dict=1MiB > $@
>
> +quiet_cmd_xzdec = XZDEC   $@
> +      cmd_xzdec = $(XZ) --decompress --stdout $< > $@
> +
>  # ZSTD
>  # ---------------------------------------------------------------------------
>  # Appends the uncompressed size of the data using size_append. The .zst
> @@ -548,6 +551,9 @@ quiet_cmd_zstd22 = ZSTD22  $@
>  quiet_cmd_zstd22_with_size = ZSTD22  $@
>        cmd_zstd22_with_size = { cat $(real-prereqs) | $(ZSTD) -22 --ultra; $(size_append); } > $@
>
> +quiet_cmd_zstddec = ZSTDDEC $@
> +      cmd_zstddec = $(ZSTD) --decompress --force --quiet -o $@ $<
> +
>  # ASM offsets
>  # ---------------------------------------------------------------------------
>
> --
> 2.41.0
>


-- 
Best Regards
Masahiro Yamada