Dependencies ?

Dependencies ?

[Michael Lynch]

[-- Attachment #1: Type: text/plain, Size: 1154 bytes --]

Hi all,

I hope this is the correct place to ask this.  I'm in the process of updating from yocto kirkstone branches to yocto scarthgap branches and ran into an issue that brought up a question about yocto/bitbake dependencies.  My normal work flow for updating is to start with an empty directory tree, clone all of the layers, checkout the specific branch (scarthgap)  for each and do a fresh build of core-image-minimal.  That all went smoothly except for a couple of recipes in the meta-intel layer.  After messing around with it a bit and not finding a solution on my own I posted a message in the meta-intel forum.  The response was that the recipes I was trying to build are not compatible with the version of clang from the scarthgap branch of meta-clang and that I needed to use the mickeldore branch of meta-clang.  So here are my questions, first, is there some way to find out if a recipe depends on specific versions of items?  Second, is there a way for yocto/bitbake to inform the user when there is a specific version dependency issue instead of the recipe simply failing to build?  Thanks and mostly just curious.

-- Mike

[-- Attachment #2: Type: text/html, Size: 1206 bytes --]

Re: [bitbake-devel] Dependencies ?

[Quentin Schulz]

Hi Michael,

On 5/24/24 3:57 PM, Michael Lynch via lists.openembedded.org wrote:
> You don't often get email from mlynch=hi-techniques.com@lists.openembedded.org. Learn why this is important<https://aka.ms/LearnAboutSenderIdentification>
> Hi all,
> 
> I hope this is the correct place to ask this.  I'm in the process of updating from yocto kirkstone branches to yocto scarthgap branches and ran into an issue that brought up a question about yocto/bitbake dependencies.  My normal work flow for updating is to start with an empty directory tree, clone all of the layers, checkout the specific branch (scarthgap)  for each and do a fresh build of core-image-minimal.  That all went smoothly except for a couple of recipes in the meta-intel layer.  After messing around with it a bit and not finding a solution on my own I posted a message in the meta-intel forum.  The response was that the recipes I was trying to build are not compatible with the version of clang from the scarthgap branch of meta-clang and that I needed to use the mickeldore branch of meta-clang.  So here are my questions, first, is there some way to find out if a recipe depends on specific versions of items?  Second, is there a way for yocto/bitbake to inform the user when there is a specific version dependency issue instead of the recipe simply failing to build?  Thanks and mostly just curious.
> 

On a layer level we do have this mechanism yes. Maintainers of the layer 
are supposed to provide with which release of Yocto/OE-Core they are 
compatible with in LAYERSERIES_COMPAT_<name in BBFILE_COLLECTIONS> 
variable in conf/layer.conf.

https://git.yoctoproject.org/meta-intel/tree/conf/layer.conf?h=scarthgap#n22 
specifies it is compatible with scarthgap. However it doesn't mean it's 
compatible with all layers that support scarthgap.

As for a recipe-level mechanism, I don't think we have this. One selects 
versions of recipes to build via PREFERRED_VERSION if there are multiple 
versions, but that is set from the global context, not from within 
another recipe.

As for package-level mechanism, we have something, see "(operator 
version)" in RDEPENDS variable, but that would be too late already for 
you as the issue is the build failing and not an incompatible version 
being pulled for runtime.

I don't know which recipes from meta-intel aren't compatible with clang 
from meta-clang, but if most, maybe they should think about 1) 
documenting those limitations 2) remove the clang-layer parts in 
BBFILES_DYNAMIC 
(https://git.yoctoproject.org/meta-intel/tree/conf/layer.conf?h=scarthgap#n25) 
until someone fixes the recipes?

I'm sure meta-intel would be happy to see patches to fix those issues as 
well, so feel free to send them some bug fixes :)

Cheers,
Quentin

Re: [bitbake-devel] Dependencies ?

[Alexander Kanavin]

On Fri, 24 May 2024 at 16:14, Quentin Schulz via
lists.openembedded.org
<quentin.schulz=cherry.de@lists.openembedded.org> wrote:

> I'm sure meta-intel would be happy to see patches to fix those issues as
> well, so feel free to send them some bug fixes :)

Not even meta-intel. This needs to be fixed upstream first. The
correct course of action is:

-  check upstream repositories of failing components if they contain
needed fixes
- if they do, then update the recipes in meta-intel to take the
versions that contain the fixes, or if no such versions are yet
released, backport them as recipe patches
- if they don't then make the fixes, and submit them upstream first,
and obtain upstream acceptance, then backport the fixes as in point 2

What isn't a correct course of action:
- roll back some of the layers to versions/branches that mismatch
everything else
- mark the recipes as somehow 'incompatible', if the fixes can be
trivially developed (and in case of clang fails, they almost always
are).

Oh, and: bitbake-devel is for bitbake discussions and development.
This is a general yocto matter, so yocto list is better. meta-intel
also has its own list: https://lists.yoctoproject.org/g/meta-intel

Alex

Re: Dependencies ?

[Michael Lynch]

[-- Attachment #1: Type: text/plain, Size: 925 bytes --]

For informational purposes the two recipes in question here are intel-compute-runtime and intel-graphics-compiler.  Both are items needed to use OpenCL with INTEL GPUs.

Alex, it sounds like the "wrong" thing to do is to "rollback" the meta-clang layer back to the "mickledore" branch as was suggested?  Here's the weird thing, the layer compatibility for meta-clang in the "mickledore" branch actually lists "scarthgap" in addition to "mickeldore" and one other.  Where as layer compatibility of meta-clang in the "scarthgap" branch is set to "scarthgap" only.  Did someone maybe do the wrong thing with layer compatibility in meta-clang on the mickledore branch?  Just curious because it seemed that mixing layer branches to make things build seemed a bit off of the whole philosophy of how yocto/bitbake was intended to work but I thought maybe that was just me not understanding everything completely.

-- Mike

[-- Attachment #2: Type: text/html, Size: 979 bytes --]

Re: [bitbake-devel] Dependencies ?

[Alexander Kanavin]

On Fri, 24 May 2024 at 17:33, Michael Lynch via lists.openembedded.org
<mlynch=hi-techniques.com@lists.openembedded.org> wrote:
>
> For informational purposes the two recipes in question here are intel-compute-runtime and intel-graphics-compiler.  Both are items needed to use OpenCL with INTEL GPUs.
>
> Alex, it sounds like the "wrong" thing to do is to "rollback" the meta-clang layer back to the "mickledore" branch as was suggested?  Here's the weird thing, the layer compatibility for meta-clang in the "mickledore" branch actually lists "scarthgap" in addition to "mickeldore" and one other.  Where as layer compatibility of meta-clang in the "scarthgap" branch is set to "scarthgap" only.  Did someone maybe do the wrong thing with layer compatibility in meta-clang on the mickledore branch?  Just curious because it seemed that mixing layer branches to make things build seemed a bit off of the whole philosophy of how yocto/bitbake was intended to work but I thought maybe that was just me not understanding everything completely.

Let's continue in the proper list.

This is the commit that added scarthgap compatibility to mickledore:
https://github.com/kraj/meta-clang/commit/8fbc4b443893ab3842063e65d8fe27a5d166bd3e

So this was intentional and not a mistake, but it's still problematic:
what if something else in your setup would require meta-clang to be at
scarthgap branch? You might get cryptic errors when that happens and
no easy way to resolve the conflict.

Sadly llvm 16 support in the intel items is a deeper issue than I
thought, and certainly not a trivial fix. Upstream will get to it, at
some point:
https://github.com/intel/intel-graphics-compiler/issues/289


Alex

Re: [bitbake-devel] Dependencies ?

[Mittal, Anuj]

On Fri, 2024-05-24 at 16:59 +0200, Alexander Kanavin wrote:
> On Fri, 24 May 2024 at 16:14, Quentin Schulz via
> lists.openembedded.org
> <quentin.schulz=cherry.de@lists.openembedded.org> wrote:
> 
> > I'm sure meta-intel would be happy to see patches to fix those
> > issues as
> > well, so feel free to send them some bug fixes :)
> 
> Not even meta-intel. This needs to be fixed upstream first. The
> correct course of action is:
> 
> -  check upstream repositories of failing components if they contain
> needed fixes
> - if they do, then update the recipes in meta-intel to take the
> versions that contain the fixes, or if no such versions are yet
> released, backport them as recipe patches
> - if they don't then make the fixes, and submit them upstream first,
> and obtain upstream acceptance, then backport the fixes as in point 2

Unfortunately in this case, fixes are not trivial and upstream needs
time to make changes to support LLVM versions >= 16:

https://github.com/intel/intel-graphics-compiler/issues/289

Thanks,

Anuj