[Buildroot] [PATCH 1/1 v2] build host lz4 to support lz4 compression in the kernel.

[Buildroot] [PATCH 1/1 v2] build host lz4 to support lz4 compression in the kernel.

[Sagaert Johan]

Selecting lz4 compression in the kernel configuration yields a
significant improvement in boot time. Therefore we need lz4 as
dependency for the kernel.
Note that LZ4 is only supported on ARM kernel >=3.11.x
Signed-off-by: Sagaert Johan <sagaert.johan@proximus.be>
---
V1->V2
	Only add the host-lz4 dependency when ARM is selected.
	Don't touch host-lz4 package.
---
 linux/linux.mk | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/linux/linux.mk b/linux/linux.mk
index 22fce35..c9b8fc5 100644
--- a/linux/linux.mk
+++ b/linux/linux.mk
@@ -71,6 +71,13 @@ LINUX_MAKE_ENV = \
 # going to be installed in the target filesystem.
 LINUX_VERSION_PROBED = $(shell $(MAKE) $(LINUX_MAKE_FLAGS) -C $(LINUX_DIR) --no-print-directory -s kernelrelease)
 
+# lz4 kernel compression is only available on ARM and kernel >= 3.11
+# we always build host-lz4 when the target is ARM.
+# (how could i use LINUX_VERSION_PROBED to check for >=3.11 ?)
+ifeq ($(BR2_arm),y)
+LINUX_DEPENDENCIES += host-lz4
+endif
+
 ifeq ($(BR2_LINUX_KERNEL_USE_INTREE_DTS),y)
 KERNEL_DTS_NAME = $(call qstrip,$(BR2_LINUX_KERNEL_INTREE_DTS_NAME))
 else ifeq ($(BR2_LINUX_KERNEL_USE_CUSTOM_DTS),y)
 
 
-- 
2.3.3

[Buildroot] [PATCH 1/1 v2] build host lz4 to support lz4 compression in the kernel.

[Thomas Petazzoni]

Dear Sagaert Johan,

On Wed,  1 Apr 2015 22:12:33 +0200, Sagaert Johan wrote:

> +# lz4 kernel compression is only available on ARM and kernel >= 3.11
> +# we always build host-lz4 when the target is ARM.
> +# (how could i use LINUX_VERSION_PROBED to check for >=3.11 ?)
> +ifeq ($(BR2_arm),y)
> +LINUX_DEPENDENCIES += host-lz4
> +endif

No that still isn't good: LZ4 compressed kernel is available on ARM,
MIPS, x86 and S390 (we don't care about the latter in Buildroot).

No, I really think we need to add new options in linux/Config.in.

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCH 1/1 v2] build host lz4 to support lz4 compression in the kernel.

[Arnout Vandecappelle]

On 01/04/15 22:37, Thomas Petazzoni wrote:
> Dear Sagaert Johan,
> 
> On Wed,  1 Apr 2015 22:12:33 +0200, Sagaert Johan wrote:
> 
>> +# lz4 kernel compression is only available on ARM and kernel >= 3.11
>> +# we always build host-lz4 when the target is ARM.
>> +# (how could i use LINUX_VERSION_PROBED to check for >=3.11 ?)
>> +ifeq ($(BR2_arm),y)
>> +LINUX_DEPENDENCIES += host-lz4
>> +endif
> 
> No that still isn't good: LZ4 compressed kernel is available on ARM,
> MIPS, x86 and S390 (we don't care about the latter in Buildroot).
> 
> No, I really think we need to add new options in linux/Config.in.

 Well, I'm in agreement with Johan's earlier statement that the overhead of the
host-lz4 build is negligible. It takes 20 seconds on my laptop, of which 6 are
buildroot overhead from parsing package/*/*.mk.

 Peter said before that all this hacking of linux (or other) config files should
be avoided, and I agree.


 Regards,
 Arnout


-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] [PATCH 1/1 v2] build host lz4 to support lz4 compression in the kernel.

[Thomas Petazzoni]

Dear Arnout Vandecappelle,

On Wed, 01 Apr 2015 23:55:01 +0200, Arnout Vandecappelle wrote:

>  Well, I'm in agreement with Johan's earlier statement that the overhead of the
> host-lz4 build is negligible. It takes 20 seconds on my laptop, of which 6 are
> buildroot overhead from parsing package/*/*.mk.

20 seconds, plus the host-lzop build time that we already build
unconditionally.

>  Peter said before that all this hacking of linux (or other) config files should
> be avoided, and I agree.

Correct, except when it's related to needed dependencies in Buildroot.

But if that's fine for everyone, I'm OK with merging an unconditional
host-lz4 dependency. I'll just follow what the consensus is on this
matter, since I don't feel like arguing too much on this
not-so-important issue.

Thanks for the feedback, definitely very useful to have the opinion of
others to help take a decision.

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCH 1/1 v2] build host lz4 to support lz4 compression in the kernel.

[Yann E. MORIN]

Thomas, All,

On 2015-04-02 00:01 +0200, Thomas Petazzoni spake thusly:
> On Wed, 01 Apr 2015 23:55:01 +0200, Arnout Vandecappelle wrote:
> 
> >  Well, I'm in agreement with Johan's earlier statement that the overhead of the
> > host-lz4 build is negligible. It takes 20 seconds on my laptop, of which 6 are
> > buildroot overhead from parsing package/*/*.mk.
> 
> 20 seconds, plus the host-lzop build time that we already build
> unconditionally.
> 
> >  Peter said before that all this hacking of linux (or other) config files should
> > be avoided, and I agree.
> 
> Correct, except when it's related to needed dependencies in Buildroot.
> 
> But if that's fine for everyone, I'm OK with merging an unconditional
> host-lz4 dependency. I'll just follow what the consensus is on this
> matter, since I don't feel like arguing too much on this
> not-so-important issue.
> 
> Thanks for the feedback, definitely very useful to have the opinion of
> others to help take a decision.

Well, I always find it very frustrating to build something that I do not
use. 20 seconds is an eternity (even the 15-or-so it takes here).

Also, I see a lot of people doing this use-case, and being quite
frustrated:

    make foo_defconfig
    make linux-menuconfig
    [Bam, 20s overhead, 35s on slower machines]

Otherwise, 15s for a lzo, yet another 10s for lz4. What next?

I would side with Thomas: add an entry in the Buildroot menuconfig to
configure what compression to use.

My 2-cent advice... ;-)

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 1/1 v2] build host lz4 to support lz4 compression in the kernel.

[Johan Sagaert]

Dear ;

Ok, added a selection menu in my v4 patch.

Regards  Johan

Yann E. MORIN schreef op 3/04/2015 om 0:06:
> Thomas, All,
>
> On 2015-04-02 00:01 +0200, Thomas Petazzoni spake thusly:
>> On Wed, 01 Apr 2015 23:55:01 +0200, Arnout Vandecappelle wrote:
>>
>>>   Well, I'm in agreement with Johan's earlier statement that the overhead of the
>>> host-lz4 build is negligible. It takes 20 seconds on my laptop, of which 6 are
>>> buildroot overhead from parsing package/*/*.mk.
>> 20 seconds, plus the host-lzop build time that we already build
>> unconditionally.
>>
>>>   Peter said before that all this hacking of linux (or other) config files should
>>> be avoided, and I agree.
>> Correct, except when it's related to needed dependencies in Buildroot.
>>
>> But if that's fine for everyone, I'm OK with merging an unconditional
>> host-lz4 dependency. I'll just follow what the consensus is on this
>> matter, since I don't feel like arguing too much on this
>> not-so-important issue.
>>
>> Thanks for the feedback, definitely very useful to have the opinion of
>> others to help take a decision.
> Well, I always find it very frustrating to build something that I do not
> use. 20 seconds is an eternity (even the 15-or-so it takes here).
>
> Also, I see a lot of people doing this use-case, and being quite
> frustrated:
>
>      make foo_defconfig
>      make linux-menuconfig
>      [Bam, 20s overhead, 35s on slower machines]
>
> Otherwise, 15s for a lzo, yet another 10s for lz4. What next?
>
> I would side with Thomas: add an entry in the Buildroot menuconfig to
> configure what compression to use.
>
> My 2-cent advice... ;-)
>
> Regards,
> Yann E. MORIN.
>

[Buildroot] [PATCH 1/1 v2] build host lz4 to support lz4 compression in the kernel.

[Johan Sagaert]

Yann E. MORIN schreef op 3/04/2015 om 0:06:
> Thomas, All,
>
> On 2015-04-02 00:01 +0200, Thomas Petazzoni spake thusly:
>> On Wed, 01 Apr 2015 23:55:01 +0200, Arnout Vandecappelle wrote:
>>
>>>   Well, I'm in agreement with Johan's earlier statement that the overhead of the
>>> host-lz4 build is negligible. It takes 20 seconds on my laptop, of which 6 are
>>> buildroot overhead from parsing package/*/*.mk.
>> 20 seconds, plus the host-lzop build time that we already build
>> unconditionally.
>>
>>>   Peter said before that all this hacking of linux (or other) config files should
>>> be avoided, and I agree.
>> Correct, except when it's related to needed dependencies in Buildroot.
>>
>> But if that's fine for everyone, I'm OK with merging an unconditional
>> host-lz4 dependency. I'll just follow what the consensus is on this
>> matter, since I don't feel like arguing too much on this
>> not-so-important issue.
>>
>> Thanks for the feedback, definitely very useful to have the opinion of
>> others to help take a decision.
> Well, I always find it very frustrating to build something that I do not
> use. 20 seconds is an eternity (even the 15-or-so it takes here).
>
> Also, I see a lot of people doing this use-case, and being quite
> frustrated:
>
>      make foo_defconfig
>      make linux-menuconfig
>      [Bam, 20s overhead, 35s on slower machines]
>
> Otherwise, 15s for a lzo, yet another 10s for lz4. What next?
>
> I would side with Thomas: add an entry in the Buildroot menuconfig to
> configure what compression to use.
>
> My 2-cent advice... ;-)
>
> Regards,
> Yann E. MORIN.
>
What about overhead for checking hashes ?
Meanwhile i submitted a new patch with a menu entry in the kernel section.
Best Regards Johan

[Buildroot] [PATCH 1/1 v2] build host lz4 to support lz4 compression in the kernel.

[Yann E. MORIN]

Johan, All,

On 2015-04-05 00:51 +0200, Johan Sagaert spake thusly:
> Yann E. MORIN schreef op 3/04/2015 om 0:06:
[--SNIP--]
> >Well, I always find it very frustrating to build something that I do not
> >use. 20 seconds is an eternity (even the 15-or-so it takes here).
[--SNIP--]
> What about overhead for checking hashes ?

That's not exactly the same purpose.

Hashes are an integral part of the infrastructure that ensure the
integrity (and authenticity) of the downloaded archives. That's the
reason they are mandatory. Also, hashing is not that slow: sha256 on
~256MiB takes about 1s here, where lzo takes ~20s.

lzo/lz4 on the other hand are a dependency of some optional kernel
features.

> Meanwhile i submitted a new patch with a menu entry in the kernel section.

Thanks! :-) I'll have a look at it.

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'