Package names in IMAGE_MANIFEST and PACKAGES

Package names in IMAGE_MANIFEST and PACKAGES

[mikko.murto]

Hello,

I'm developing a meta layer to save details about an image created by Yocto to an SPDX document (https://github.com/doubleopen-project/meta-doubleopen).

I've encountered two issues regarding package names in IMAGE_MANIFEST and PACKAGES variables, https://github.com/doubleopen-project/meta-doubleopen/issues/2 and https://github.com/doubleopen-project/meta-doubleopen/issues/3.

The crux of the matter is that I need to find packages created by recipes and to link the packages listed in image's manifest files to these packages. 

First, the PACKAGES variable of all recipes doesn't seem to include all packages created. For example util-linux's PACKAGES doesn't include util-linux-sulogin, but util-linux-sulogin may be included in an image's manifest and it has a directory in the packages-split directory of util-linux. What would be the correct way to get information about all packages?

Second, the package names in the image manifest may differ from those in the PACKAGES variable or in packages-split directory. As an example, a manifest file may include libkmod2, but recipe for kmod creates package named libkmod. How to make the link from libkmod2 to libkmod?

Best regards,

Mikko Murto
Associate

HH Partners, Attorneys-at-law Ltd
Eteläesplanadi 22 a, 00130 Helsinki, Finland
P.O. Box 232, 00101 Helsinki, Finland
Tel: +358 9 177 613, Fax: +358 9 653 873
mikko.murto@hhpartners.fi 
www.hhpartners.fi

HH Partners shines in international rankings. See details at hhpartners.fi.

Privileged and confidential information may be contained in this message. If you are not the addressee indicated in this message (or responsible for the delivery of the message to such person), you may not copy or deliver this message to anyone. In such case, kindly notify us by reply e-mail and delete this message immediately. Thank you.

Re: [yocto] Package names in IMAGE_MANIFEST and PACKAGES

[Jérôme Carretero]

[-- Attachment #1: Type: text/plain, Size: 701 bytes --]

Hi Mikko,


On Fri, 26 Feb 2021 14:18:59 +0000
"Mikko Murto" <mikko.murto@hhpartners.fi> wrote:

> The crux of the matter is that I need to find packages created by recipes and to link the packages listed in image's manifest files to these packages. 

The way I've been doing it, which is probably not optimal and specific
to one package format, is to lookup .ipk packages from the image
manifest, then use the .ipk control info "OE" key to find the
originating recipe.

Here's a link of this method in use in order to try and identify
license-related obligations from an image:
 https://gitlab.com/exmakhina/xm_oe/-/blob/47183b74/licenses.py#L24


Hoping to help,

-- 
Jérôme

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 849 bytes --]

Re: [licensing] [yocto] Package names in IMAGE_MANIFEST and PACKAGES

[Mikko Rapeli]

Hi,

On Fri, Feb 26, 2021 at 10:08:47AM -0500, Jérôme Carretero wrote:
> On Fri, 26 Feb 2021 14:18:59 +0000
> "Mikko Murto" <mikko.murto@hhpartners.fi> wrote:
> 
> > The crux of the matter is that I need to find packages created by recipes and to link the packages listed in image's manifest files to these packages. 
> 
> The way I've been doing it, which is probably not optimal and specific
> to one package format, is to lookup .ipk packages from the image
> manifest, then use the .ipk control info "OE" key to find the
> originating recipe.

FWIW, the mapping from binary package names recipes and recipe metadata like
LICENSE is available from buildhistory. Also binary package content of images
is available from buildhistory. With some scripting it is possible to list
recipes which produce binaries to images, except for static linking and
header-only recipes but I hope these cought via some other way.

Cheers,

-Mikko

Re: [licensing] [yocto] Package names in IMAGE_MANIFEST and PACKAGES

[Jérôme Carretero]

[-- Attachment #1: Type: text/plain, Size: 1034 bytes --]

On Fri, 26 Feb 2021 15:21:10 +0000
"Mikko Rapeli" <mikko.rapeli@bmw.de> wrote:

> FWIW, the mapping from binary package names recipes and recipe metadata like
> LICENSE is available from buildhistory. Also binary package content of images
> is available from buildhistory.

Definitely more powerful than just image manifest + packages.

> With some scripting it is possible to list
> recipes which produce binaries to images, except for static linking and
> header-only recipes but I hope these cought via some other way.

Yeah, that "except" part is something that should be accounted for.

The thing is, I don't see how recipes make any difference between a
build-time dependency such as a build tool, or a statically linked
library, so that some additional information should be input somewhere;
failing to provide that, since I don't feel like finding static library
symbols in binaries to perform a discovery, I just tell my clients to
distribute everything except their proprietary bits...

-- 
Jérôme

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 849 bytes --]

Re: [yocto] Package names in IMAGE_MANIFEST and PACKAGES

[Richard Purdie]

On Fri, 2021-02-26 at 14:18 +0000, Mikko Murto wrote:
> Hello,
> 
> I'm developing a meta layer to save details about an image created by Yocto to an 
> SPDX document ( https://github.com/doubleopen-project/meta-doubleopen).
> 
> I've encountered two issues regarding package names in IMAGE_MANIFEST 
> and PACKAGES variables,   
> 
> https://github.com/doubleopen-project/meta-doubleopen/issues/2 and     
> https://github.com/doubleopen-project/meta-doubleopen/issues/3.
> 
> The crux of the matter is that I need to find packages created by recipes 
> and to link the packages listed in image's manifest files to these packages. 
> 
> First, the PACKAGES variable of all recipes doesn't seem to include all 
> packages created. For example util-linux's PACKAGES doesn't include 
> util-linux-sulogin, but util-linux-sulogin may be included in an image's 
> manifest and it has a directory in the packages-split directory of util-linux. 
> What would be the correct way to get information about all packages?
> 
> Second, the package names in the image manifest may differ from those in 
> the PACKAGES variable or in packages-split directory. As an example, a
>  manifest file may include libkmod2, but recipe for kmod creates package 
> named libkmod. How to make the link from libkmod2 to libkmod?

I'd suggesting looking at packagedata which is how do_package and friends 
internally looks up things like which recipe provides which package and 
what the final package name is.

An example of this in action is scripts/oe-pkgdata-util.

There is also meta/lib/oe/packagedata.py. Sadly the API is horrible, it 
was never really designed for "public" use but I would love to see a better
API around that (maybe one oe-pkgdata-util could use too).

Cheers,

Richard