[Buildroot] [PATCH] linux: Fix initramfs compression

[Buildroot] [PATCH] linux: Fix initramfs compression

[Valentine Barshak]

Initramfs compression does not make much sense for the architectures
that support compressed kernel images because in this case the data
would be compressed twice. This will eventually result in a bigger
kernel image and time overhead when uncompressing it.
The only reason to use initramfs compression is to reduce memory
usage when the kernel prepares initramfs, and both the unpacked
filesystem and initramfs.cpio are present in the memory.

Buildroot attempts to force GZIP compression for initramfs,
however it doesn't always work because initramfs compression mode
depends on RAM disk compression supported by the kernel.
Thus, CONFIG_INITRAMFS_COMPRESSION_GZIP depends on CONFIG_RD_GZIP.
If CONFIG_RD_GZIP is not set, setting GZIP initramfs compression
will have no effect.

Besides, the kernel also supports other compression methods,
like BZIP2, LZMA, XZ and LZO. Forcing the good old GZIP does not
really make much sense any more.

This attempts to find the first RAM disk compression mode
supported by the kernel, parsing CONFIG_RD_... variables,
and sets CONFIG_INITRAMFS_COMPRESSION_ accordingly.
If no RAM disk compression is supported, no initramfs
compression is set, and the kernel will use
CONFIG_INITRAMFS_COMPRESSION_NONE.

I've slightly changed the KCONFIG_[SET|ENABLE|DISABLE]_OPT
functions to be able to call them from shell "if" statement.

Signed-off-by: Valentine Barshak <gvaxon@gmail.com>
---
 fs/initramfs/Config.in | 5 +++--
 linux/linux.mk         | 9 +++++++--
 package/pkg-utils.mk   | 6 +++---
 3 files changed, 13 insertions(+), 7 deletions(-)

diff --git a/fs/initramfs/Config.in b/fs/initramfs/Config.in
index bbc2ab0..774a575 100644
--- a/fs/initramfs/Config.in
+++ b/fs/initramfs/Config.in
@@ -9,8 +9,9 @@ config BR2_TARGET_ROOTFS_INITRAMFS
 
 	  A rootfs.cpio file will be generated in the images/ directory.
 	  This is the archive that will be included in the kernel image.
-	  The rootfs in the kernel will always be gzip'ed, regardless
-	  of how buildroot's cpio archive is configured.
+	  The rootfs compression in the kernel will be set according to
+	  the initial ram disk compression mode (CONFIG_RD_...) enabled,
+	  regardless of how buildroot's cpio archive is configured.
 
 	  Note that enabling initramfs together with another filesystem
 	  formats doesn't make sense: you would end up having two
diff --git a/linux/linux.mk b/linux/linux.mk
index c4bdf90..3521dcd 100644
--- a/linux/linux.mk
+++ b/linux/linux.mk
@@ -170,8 +170,13 @@ define LINUX_CONFIGURE_CMDS
 		$(call KCONFIG_SET_OPT,CONFIG_INITRAMFS_SOURCE,\"$(BINARIES_DIR)/rootfs.cpio\",$(@D)/.config)
 		$(call KCONFIG_SET_OPT,CONFIG_INITRAMFS_ROOT_UID,0,$(@D)/.config)
 		$(call KCONFIG_SET_OPT,CONFIG_INITRAMFS_ROOT_GID,0,$(@D)/.config)
-		$(call KCONFIG_DISABLE_OPT,CONFIG_INITRAMFS_COMPRESSION_NONE,$(@D)/.config)
-		$(call KCONFIG_ENABLE_OPT,CONFIG_INITRAMFS_COMPRESSION_GZIP,$(@D)/.config))
+		for c in GZIP BZIP2 LZMA XZ LZO; do \
+			if grep -qm1 "CONFIG_RD_$$c=y" $(@D)/.config; then \
+				$(call KCONFIG_DISABLE_OPT,CONFIG_INITRAMFS_COMPRESSION_NONE,$(@D)/.config); \
+				$(call KCONFIG_ENABLE_OPT,CONFIG_INITRAMFS_COMPRESSION_$$c,$(@D)/.config); \
+				break; \
+			fi; \
+		done)
 	$(if $(BR2_ROOTFS_DEVICE_CREATION_STATIC),,
 		$(call KCONFIG_ENABLE_OPT,CONFIG_DEVTMPFS,$(@D)/.config)
 		$(call KCONFIG_ENABLE_OPT,CONFIG_DEVTMPFS_MOUNT,$(@D)/.config))
diff --git a/package/pkg-utils.mk b/package/pkg-utils.mk
index 293bf4f..5568943 100644
--- a/package/pkg-utils.mk
+++ b/package/pkg-utils.mk
@@ -30,17 +30,17 @@ UPPERCASE = $(strip $(eval __tmp := $1) \
 #
 
 define KCONFIG_ENABLE_OPT
-	$(SED) "/\\<$(1)\\>/d" $(2)
+	$(SED) "/\\<$(1)\\>/d" $(2); \
 	echo "$(1)=y" >> $(2)
 endef
 
 define KCONFIG_SET_OPT
-	$(SED) "/\\<$(1)\\>/d" $(3)
+	$(SED) "/\\<$(1)\\>/d" $(3); \
 	echo "$(1)=$(2)" >> $(3)
 endef
 
 define KCONFIG_DISABLE_OPT
-	$(SED) "/\\<$(1)\\>/d" $(2)
+	$(SED) "/\\<$(1)\\>/d" $(2); \
 	echo "# $(1) is not set" >> $(2)
 endef
 
-- 
1.7.11.4

[Buildroot] [PATCH] linux: Fix initramfs compression

[Arnout Vandecappelle]

On 06/10/12 21:22, Valentine Barshak wrote:
> -		$(call KCONFIG_DISABLE_OPT,CONFIG_INITRAMFS_COMPRESSION_NONE,$(@D)/.config)
> -		$(call KCONFIG_ENABLE_OPT,CONFIG_INITRAMFS_COMPRESSION_GZIP,$(@D)/.config))
> +		for c in GZIP BZIP2 LZMA XZ LZO; do \
> +			if grep -qm1 "CONFIG_RD_$$c=y" $(@D)/.config; then \
> +				$(call KCONFIG_DISABLE_OPT,CONFIG_INITRAMFS_COMPRESSION_NONE,$(@D)/.config); \
> +				$(call KCONFIG_ENABLE_OPT,CONFIG_INITRAMFS_COMPRESSION_$$c,$(@D)/.config); \
> +				break; \
> +			fi; \
> +		done)

  I think this is a bit too complex for something that won't really be used by
most people.  Why not just remove the COMPRESSION lines from linux.mk?  Then
we fall back on the default if the user hasn't configured it explicitly.  Gives
the user the possibility to choose something else than the default, without adding
complexity to buildroot.

  Regards,
  Arnout
-- 
Arnout Vandecappelle                               arnout at mind be
Senior Embedded Software Architect                 +32-16-286540
Essensium/Mind                                     http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium                BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] [PATCH] linux: Fix initramfs compression

[Valentine Barshak]

On 10/07/2012 03:31 PM, Arnout Vandecappelle wrote:
> On 06/10/12 21:22, Valentine Barshak wrote:
>> -        $(call
>> KCONFIG_DISABLE_OPT,CONFIG_INITRAMFS_COMPRESSION_NONE,$(@D)/.config)
>> -        $(call
>> KCONFIG_ENABLE_OPT,CONFIG_INITRAMFS_COMPRESSION_GZIP,$(@D)/.config))
>> +        for c in GZIP BZIP2 LZMA XZ LZO; do \
>> +            if grep -qm1 "CONFIG_RD_$$c=y" $(@D)/.config; then \
>> +                $(call
>> KCONFIG_DISABLE_OPT,CONFIG_INITRAMFS_COMPRESSION_NONE,$(@D)/.config); \
>> +                $(call
>> KCONFIG_ENABLE_OPT,CONFIG_INITRAMFS_COMPRESSION_$$c,$(@D)/.config); \
>> +                break; \
>> +            fi; \
>> +        done)
>
>   I think this is a bit too complex for something that won't really be
> used by
> most people.  Why not just remove the COMPRESSION lines from linux.mk?
> Then
> we fall back on the default if the user hasn't configured it
> explicitly.  Gives
> the user the possibility to choose something else than the default,
> without adding
> complexity to buildroot.

The problem here is that we can't set initramfs compression mode unless 
the initramfs source (CONFIG_INITRAMFS_SOURCE) is set. We can only set 
RAM disk compression modes that should be supported by the kernel 
(CONFIR_RD_...).

The source is set by buildroot. So the user has no capability to 
configure compression explicitly, unless he (she) sets a fake initramfs 
source file, which will be overridden by buildroot. IMHO, this is a bit 
hackish way to set anything other than default, which is COMPRESSION_NONE.

Yes, that is what most people want. I just didn't want to drop other 
options in case somebody wants a compressed initramfs image.
This could be needed for systems with low memory, for example.

>
>   Regards,
>   Arnout

Thanks,
Val.

[Buildroot] [PATCH] linux: Fix initramfs compression

[Arnout Vandecappelle]

On 07/10/12 22:22, Valentine Barshak wrote:
> The problem here is that we can't set initramfs compression mode unless the initramfs source (CONFIG_INITRAMFS_SOURCE)
> is set. We can only set RAM disk compression modes that should be supported by the kernel (CONFIR_RD_...).
>
> The source is set by buildroot. So the user has no capability to configure compression explicitly, unless he (she) sets
> a fake initramfs source file, which will be overridden by buildroot. IMHO, this is a bit hackish way to set anything
> other than default, which is COMPRESSION_NONE.
>
> Yes, that is what most people want. I just didn't want to drop other options in case somebody wants a compressed
> initramfs image.
> This could be needed for systems with low memory, for example.

  True.  Still, I don't think that this corner case warrants the additional
complexity of grepping etc. in the buildroot makefile.  Especially because it
still doesn't give the user complete configuration freedom: if both GZIP and
BZIP2 initrd compression options are enabled (like is the case in some of the
kernel's defconfigs), then still GZIP will be chosen.

  If we really want to support this in buildroot, then the choice should be added
to fs/initramfs/Config.in.  But also here I doubt that the (maintenance) work of
having this option is worth it, because it's likely that it will _never_ be used.

  As you say yourself, if you really do need the option, there is a possibility
to do it.  And it's not as bad as you say, because when you run
'make linux-menuconfig', buildroot will first run a 'linux-configure' so the
INITRAMFS_SOURCE will already be set.  (At least, I think so :-)


  Regards,
  Arnout


-- 
Arnout Vandecappelle                               arnout at mind be
Senior Embedded Software Architect                 +32-16-286540
Essensium/Mind                                     http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium                BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] [PATCH] linux: Fix initramfs compression

[Valentine Barshak]

On 10/08/2012 11:43 AM, Arnout Vandecappelle wrote:
> On 07/10/12 22:22, Valentine Barshak wrote:
>> The problem here is that we can't set initramfs compression mode
>> unless the initramfs source (CONFIG_INITRAMFS_SOURCE)
>> is set. We can only set RAM disk compression modes that should be
>> supported by the kernel (CONFIR_RD_...).
>>
>> The source is set by buildroot. So the user has no capability to
>> configure compression explicitly, unless he (she) sets
>> a fake initramfs source file, which will be overridden by buildroot.
>> IMHO, this is a bit hackish way to set anything
>> other than default, which is COMPRESSION_NONE.
>>
>> Yes, that is what most people want. I just didn't want to drop other
>> options in case somebody wants a compressed
>> initramfs image.
>> This could be needed for systems with low memory, for example.
>
>   True.  Still, I don't think that this corner case warrants the additional
> complexity of grepping etc. in the buildroot makefile.  Especially
> because it
> still doesn't give the user complete configuration freedom: if both GZIP
> and
> BZIP2 initrd compression options are enabled (like is the case in some
> of the
> kernel's defconfigs), then still GZIP will be chosen.

Right.

>
>   If we really want to support this in buildroot, then the choice should
> be added
> to fs/initramfs/Config.in.  But also here I doubt that the (maintenance)
> work of
> having this option is worth it, because it's likely that it will _never_
> be used.
>
>   As you say yourself, if you really do need the option, there is a
> possibility
> to do it.  And it's not as bad as you say, because when you run
> 'make linux-menuconfig', buildroot will first run a 'linux-configure' so
> the
> INITRAMFS_SOURCE will already be set.  (At least, I think so :-)

Indeed, so it's probably better to let the kernel config handle it, 
instead of attempting to force any compression mode in BR.

>
>
>   Regards,
>   Arnout
>
Thanks,
Val.