Any recommendation to make software layer Yocto Compatible?

Any recommendation to make software layer Yocto Compatible?

[Duy]

[-- Attachment #1: Type: text/plain, Size: 529 bytes --]

Hi,

I'm currently trying to make my software layer Yocto compatible. Running yocto-check-layer script is a convenient way.
But in practice, I found that for software layer, it is difficult to make it pass test_signatures case. It requires to put the condition check to every variable or function appending in bbappend file. With a deep customizing software layer, it reduces code readability.
Can anyone share your experience in making a software layer Yocto compatible while keeping it easy to read?

Best regards,
Duy

[-- Attachment #2: Type: text/html, Size: 557 bytes --]

Re: [yocto] Any recommendation to make software layer Yocto Compatible?

[Richard Purdie]

On Mon, 2024-04-08 at 02:25 -0700, Duy wrote:
> I'm currently trying to make my software layer Yocto compatible.
> Running yocto-check-layer script is a convenient way.
> But in practice, I found that for software layer, it is difficult to
> make it pass test_signatures case. It requires to put the condition
> check to every variable or function appending in bbappend file. With
> a deep customizing software layer, it reduces code readability.
> Can anyone share your experience in making a software layer Yocto
> compatible while keeping it easy to read?

The question to ask is probably why are you needing to bbappend and
change the behaviour of so many recipes?

Software layers adding software wouldn't need to do that, so it sounds
like you're changing configuration. If you're doing that, it is a
distro configuration change and that isn't a software layer and the
config changes should be conditional on the distro.

Can you share some example bbappends you're needing?

Cheers,

Richard

Re: [yocto] Any recommendation to make software layer Yocto Compatible?

[Duy]

[-- Attachment #1: Type: text/plain, Size: 934 bytes --]

Hi Richard,

Thanks for your response.
Here is one of the recipe bbappend files I'm working on:
meta-agl-ic-container/recipes-extended/dlt-daemon/dlt-daemon_%.bbappend · master · Automotive Grade Linux / AGL / meta-agl-devel · GitLab ( https://gitlab.com/automotivegradelinux/AGL/meta-agl-devel/-/blob/master/meta-agl-ic-container/recipes-extended/dlt-daemon/dlt-daemon_%25.bbappend?ref_type=heads )
meta-agl-ic-container/recipes-extended/dlt-daemon/dlt-daemon_%.bbappend · master · Automotive Grade Linux / AGL / meta-agl-devel · GitLab ( https://gitlab.com/automotivegradelinux/AGL/meta-agl-devel/-/blob/master/meta-agl-ic-container/recipes-extended/dlt-daemon/dlt-daemon_%25.bbappend?ref_type=heads )

I think some configurations are better to be upstreaming, e.g: Adding new PACKAGES, adding new PACKAGECONFIG. It avoids adding too much to bbappend. If you have any idea, please share it.

Best Regards,
Duy Dang

[-- Attachment #2: Type: text/html, Size: 1030 bytes --]

Re: [yocto] Any recommendation to make software layer Yocto Compatible?

[Paul Barker]

[-- Attachment #1.1.1: Type: text/plain, Size: 1811 bytes --]

On 08/04/2024 13:36, Duy wrote:
> Hi Richard,
> 
> Thanks for your response.
> Here is one of the recipe bbappend files I'm working on:
> meta-agl-ic-container/recipes-extended/dlt-daemon/dlt-daemon_%.bbappend · master · Automotive Grade Linux / AGL / meta-agl-devel · GitLab <https://gitlab.com/automotivegradelinux/AGL/meta-agl-devel/-/blob/master/meta-agl-ic-container/recipes-extended/dlt-daemon/dlt-daemon_%25.bbappend?ref_type=heads>
> meta-agl-ic-container/recipes-extended/dlt-daemon/dlt-daemon_%.bbappend · master · Automotive Grade Linux / AGL / meta-agl-devel · GitLab <https://gitlab.com/automotivegradelinux/AGL/meta-agl-devel/-/blob/master/meta-agl-ic-container/recipes-extended/dlt-daemon/dlt-daemon_%25.bbappend?ref_type=heads>
> 
> I think some configurations are better to be upstreaming, e.g: Adding new PACKAGES, adding new PACKAGECONFIG. It avoids adding too much to bbappend. If you have any idea, please share it.

When in doubt, steal from meta-virtualization.

In https://git.yoctoproject.org/meta-virtualization/tree/recipes-kernel/linux/linux-%25.bbappend,
there is a conditional include based on DISTRO_FEATURES. You can do the
same based on DISTRO.

Then you have a .inc file which is only included when the appropriate
configuration is enabled, like
https://git.yoctoproject.org/meta-virtualization/tree/recipes-kernel/linux/linux-yocto_virtualization.inc.
The statements in this file don't each need to check DISTRO, etc.

There is more useful information in my Creating Friendly Layers
presentation:
  * https://pub.pbarker.dev/presentations/2022-12-01%20Yocto%20Project%20Summit%202022.11%20-%20Creating%20Friendly%20Layers%202022%20Edition/Friendly%20Layers%202022.pdf
  * https://www.youtube.com/watch?v=6iGuKViITjg

Thanks,

-- 
Paul Barker

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 3577 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

Re: [yocto] Any recommendation to make software layer Yocto Compatible?

[Richard Purdie]

On Mon, 2024-04-08 at 05:36 -0700, Duy wrote:
> Hi Richard,
> 
> Thanks for your response.
> Here is one of the recipe bbappend files I'm working on:
> meta-agl-ic-container/recipes-extended/dlt-daemon/dlt-
> daemon_%.bbappend · master · Automotive Grade Linux / AGL / meta-agl-
> devel · GitLab
> meta-agl-ic-container/recipes-extended/dlt-daemon/dlt-
> daemon_%.bbappend · master · Automotive Grade Linux / AGL / meta-agl-
> devel · GitLab
> 
> I think some configurations are better to be upstreaming, e.g: Adding
> new PACKAGES, adding new PACKAGECONFIG. It avoids adding too much to
> bbappend. If you have any idea, please share it.

Where possible please do share changes with upstream recipes,
particularly for things like PACKAGECONFIG entries. There is no good
reason to have them separate in most cases.

Other parts of this look to be adding configuration files. Are these
generally useful? That might help to know if they should be upstream.
If not, should they be in a separate recipe?

This isn't just a software layer, it has quite a bit of
configuration/distro policy in there so the layer checks are right to
flag it as problematic in my view.

Cheers,

Richard

Re: [yocto] Any recommendation to make software layer Yocto Compatible?

[Jan-Simon Möller]

Hi Duy,

The recipe in question was done by the Instrument Cluster EG, we can work on 
this with the group (main contact Yamaguchi-san). They do meet every other 
monday. See: https://lists.automotivelinux.org/g/agl-dev-community/calendar 

We do exacly what Paul describes all the time in meta-agl to pass the yocto-
check-layer:

e.g.
https://git.automotivelinux.org/AGL/meta-agl/tree/meta-agl-core/recipes-core/
systemd/systemd_%25.bbappend
and
https://git.automotivelinux.org/AGL/meta-agl/tree/meta-agl-core/recipes-core/
systemd/systemd_aglcore.inc

The condition can be on DISTRO_FEATURES or other variables. We try not to 
overload the usage of DISTRO_FEATURES as a change there will trigger a reparse 
in a lot of locations and hence possibly rebuilds. Thus the use of 
AGL_FEATURES in above example for anything that is directly related to AGL and 
is not a DISTRO_FEATURE in yocto already. 

For dlt-daemon in your example, we have these options and recommendations:
a) use such a conditional include as shown above ... simple but it will not fix 
all the issues
b) use :append:<some_other_conditional>  only in .bbappend files
     (name says it all ;) )
    e.g. :append:aglcontainerguest
    or
    e.g.  :append:aglcontainerhost
    The 2nd condition / override is important here to pass the check.
    A simple :append  would triger
c) distill the recipe down to just the required changes & if possible upstream 
these (e.g. PACKAGECONFIG options) ... config file changes can also be added in 
a "dlt-daemon-conf" package and replace or amend the original files. This helps 
most.

a) will have immediate effect but only c) and b) will lower the workload mid-
term. 

A path forward would then be:
Upstream the PACKAGECONFIG bits . It might make sense to rework the package so 
there is a configuration sub-package that is replaceable by the user. Then most 
conf file changes are in a separate package and can be easily replaced by the 
user w/o even touching the main (binary) package. That is of importance for 
reproducible builds and binary feeds. 



Am Montag, 8. April 2024, 14:36:38 CEST schrieb Duy via 
lists.yoctoproject.org:
> Hi Richard,
> 
> Thanks for your response.
> Here is one of the recipe bbappend files I'm working on:
> meta-agl-ic-container/recipes-extended/dlt-daemon/dlt-daemon_%.bbappend ·
> master · Automotive Grade Linux / AGL / meta-agl-devel · GitLab (
> https://gitlab.com/automotivegradelinux/AGL/meta-agl-devel/-/blob/master/me
> ta-agl-ic-container/recipes-extended/dlt-daemon/dlt-daemon_%25.bbappend?ref_
> type=heads )
> meta-agl-ic-container/recipes-extended/dlt-daemon/dlt-daemon_%.bbappend ·
> master · Automotive Grade Linux / AGL / meta-agl-devel · GitLab (
> https://gitlab.com/automotivegradelinux/AGL/meta-agl-devel/-/blob/master/me
> ta-agl-ic-container/recipes-extended/dlt-daemon/dlt-daemon_%25.bbappend?ref_
> type=heads )
> 
> I think some configurations are better to be upstreaming, e.g: Adding new
> PACKAGES, adding new PACKAGECONFIG. It avoids adding too much to bbappend.
> If you have any idea, please share it.
> 
> Best Regards,
> Duy Dang

Re: [yocto] Any recommendation to make software layer Yocto Compatible?

[Duy]

[-- Attachment #1: Type: text/plain, Size: 121 bytes --]

Hi Paul, Richard, Jan-Simon,

Thank you for your very useful sharing. I understand the concept.

Best Regards,
Duy

[-- Attachment #2: Type: text/html, Size: 141 bytes --]