[PATCH] bitbake.conf: add lz4c to HOSTTOOLS_NONFATAL

[PATCH] bitbake.conf: add lz4c to HOSTTOOLS_NONFATAL

[Justin Bronder]

Commit fe167e082cbde1c6d186ecdda531abef610ac2ac switched to requiring
lz4 instead of lz4c which allows us to support distros dropping lz4c.
However, it's only in the 6.13 kernel that CONFIG_KERNEL_LZ4 makes the
switch from lz4c to lz4.  So we should continue to link lz4c if it's
available to support older kernels.

Signed-off-by: Justin Bronder <jsbronder@cold-front.org>
---
 meta/conf/bitbake.conf | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index 1d2c2e0022..c7927d19a0 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -553,6 +553,9 @@ HOSTTOOLS_NONFATAL += "gsutil"
 # Link to git-lfs if present
 HOSTTOOLS_NONFATAL += "git-lfs"
 
+# Link to lz4c if present, used by linux <6.13 with CONFIG_KERNEL_LZ4
+HOSTTOOLS_NONFATAL += "lz4c"
+
 CCACHE ??= ""
 
 TOOLCHAIN_OPTIONS = " --sysroot=${STAGING_DIR_TARGET}"
-- 
2.47.0

Re: [OE-core] [PATCH] bitbake.conf: add lz4c to HOSTTOOLS_NONFATAL

[Richard Purdie]

On Tue, 2024-12-10 at 10:10 -0500, Justin Bronder via lists.openembedded.org wrote:
> Commit fe167e082cbde1c6d186ecdda531abef610ac2ac switched to requiring
> lz4 instead of lz4c which allows us to support distros dropping lz4c.
> However, it's only in the 6.13 kernel that CONFIG_KERNEL_LZ4 makes the
> switch from lz4c to lz4.  So we should continue to link lz4c if it's
> available to support older kernels.
> 
> Signed-off-by: Justin Bronder <jsbronder@cold-front.org>
> ---
>  meta/conf/bitbake.conf | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
> index 1d2c2e0022..c7927d19a0 100644
> --- a/meta/conf/bitbake.conf
> +++ b/meta/conf/bitbake.conf
> @@ -553,6 +553,9 @@ HOSTTOOLS_NONFATAL += "gsutil"
>  # Link to git-lfs if present
>  HOSTTOOLS_NONFATAL += "git-lfs"
>  
> +# Link to lz4c if present, used by linux <6.13 with CONFIG_KERNEL_LZ4
> +HOSTTOOLS_NONFATAL += "lz4c"
> +
>  CCACHE ??= ""
>  
>  TOOLCHAIN_OPTIONS = " --sysroot=${STAGING_DIR_TARGET}"
> 

The YP TSC chatted a bit about this. Personally, I'm a bit nervous as
this could make the builds a bit non-deterministic and allow the builds
to fail well into the build. We try to do that "up front" if and where
we can to make developers lives easier. This becomes more likely as
distros drop lz4c and releases age, i.e. we'll hit problems years down
the line.

I did have an idea for another potential solution which would be to put
a lz4c wrapper in the scripts/native-intercept directory which tweaks
the parameters and calls lz4. Could I convince someone to see if that
would work?

Cheers,

Richard

Re: bitbake.conf: add lz4c to HOSTTOOLS_NONFATAL

[Justin Bronder]

On 10/12/24 21:53 +0000, Richard Purdie wrote:
> On Tue, 2024-12-10 at 10:10 -0500, Justin Bronder via lists.openembedded.org wrote:
> > Commit fe167e082cbde1c6d186ecdda531abef610ac2ac switched to requiring
> > lz4 instead of lz4c which allows us to support distros dropping lz4c.
> > However, it's only in the 6.13 kernel that CONFIG_KERNEL_LZ4 makes the
> > switch from lz4c to lz4.� So we should continue to link lz4c if it's
> > available to support older kernels.
> > 
> > Signed-off-by: Justin Bronder <jsbronder@cold-front.org>
> > ---
> > �meta/conf/bitbake.conf | 3 +++
> > �1 file changed, 3 insertions(+)
> > 
> > diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
> > index 1d2c2e0022..c7927d19a0 100644
> > --- a/meta/conf/bitbake.conf
> > +++ b/meta/conf/bitbake.conf
> > @@ -553,6 +553,9 @@ HOSTTOOLS_NONFATAL += "gsutil"
> > �# Link to git-lfs if present
> > �HOSTTOOLS_NONFATAL += "git-lfs"
> > �
> > +# Link to lz4c if present, used by linux <6.13 with CONFIG_KERNEL_LZ4
> > +HOSTTOOLS_NONFATAL += "lz4c"
> > +
> > �CCACHE ??= ""
> > �
> > �TOOLCHAIN_OPTIONS = " --sysroot=${STAGING_DIR_TARGET}"
> > 
> 
> The YP TSC chatted a bit about this. Personally, I'm a bit nervous as
> this could make the builds a bit non-deterministic and allow the builds
> to fail well into the build. We try to do that "up front" if and where
> we can to make developers lives easier. This becomes more likely as
> distros drop lz4c and releases age, i.e. we'll hit problems years down
> the line.
> 
> I did have an idea for another potential solution which would be to put
> a lz4c wrapper in the scripts/native-intercept directory which tweaks
> the parameters and calls lz4.�Could I convince someone to see if that
> would work?

That works for the kernel build if I put the wrapper in scripts/.  But it
doesn't if the wrapper is in scripts/native-intercept as that's only added to
the PATH if native.bbclass is inherited.

Is adding a wrapper in script/ acceptable?  I see we already have one for git.

-- 
Justin Bronder

Re: bitbake.conf: add lz4c to HOSTTOOLS_NONFATAL

[Richard Purdie]

On Mon, 2024-12-16 at 12:06 -0500, Justin Bronder wrote:
> On 10/12/24 21:53 +0000, Richard Purdie wrote:
> > On Tue, 2024-12-10 at 10:10 -0500, Justin Bronder via lists.openembedded.org wrote:
> > > Commit fe167e082cbde1c6d186ecdda531abef610ac2ac switched to requiring
> > > lz4 instead of lz4c which allows us to support distros dropping lz4c.
> > > However, it's only in the 6.13 kernel that CONFIG_KERNEL_LZ4 makes the
> > > switch from lz4c to lz4.  So we should continue to link lz4c if it's
> > > available to support older kernels.
> > > 
> > > Signed-off-by: Justin Bronder <jsbronder@cold-front.org>
> > > ---
> > >  meta/conf/bitbake.conf | 3 +++
> > >  1 file changed, 3 insertions(+)
> > > 
> > > diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
> > > index 1d2c2e0022..c7927d19a0 100644
> > > --- a/meta/conf/bitbake.conf
> > > +++ b/meta/conf/bitbake.conf
> > > @@ -553,6 +553,9 @@ HOSTTOOLS_NONFATAL += "gsutil"
> > >  # Link to git-lfs if present
> > >  HOSTTOOLS_NONFATAL += "git-lfs"
> > >  
> > > +# Link to lz4c if present, used by linux <6.13 with CONFIG_KERNEL_LZ4
> > > +HOSTTOOLS_NONFATAL += "lz4c"
> > > +
> > >  CCACHE ??= ""
> > >  
> > >  TOOLCHAIN_OPTIONS = " --sysroot=${STAGING_DIR_TARGET}"
> > > 
> > 
> > The YP TSC chatted a bit about this. Personally, I'm a bit nervous as
> > this could make the builds a bit non-deterministic and allow the builds
> > to fail well into the build. We try to do that "up front" if and where
> > we can to make developers lives easier. This becomes more likely as
> > distros drop lz4c and releases age, i.e. we'll hit problems years down
> > the line.
> > 
> > I did have an idea for another potential solution which would be to put
> > a lz4c wrapper in the scripts/native-intercept directory which tweaks
> > the parameters and calls lz4. Could I convince someone to see if that
> > would work?
> 
> That works for the kernel build if I put the wrapper in scripts/.  But it
> doesn't if the wrapper is in scripts/native-intercept as that's only added to
> the PATH if native.bbclass is inherited.
> 
> Is adding a wrapper in script/ acceptable?  I see we already have one for git.

It isn't ideal but we can probably do that. As you say, there is precedent...

Cheers,

Richard