How to get a list of packages (with versions) used to create an image?

How to get a list of packages (with versions) used to create an image?

[Brian Hutchinson]

[-- Attachment #1: Type: text/plain, Size: 532 bytes --]

Hi,

I've seen the bitbake -g command that lists the packages in an image:

bitbake -g <*image>* && cat pn-depends.dot | grep -v -e '-native' | grep -v
digraph | grep -v -e '-image' | awk '{print $1}' | sort | uniq

... but for documentation I need to generate this list with package version
information as well.

I've been just running opkg list-installed on the target but I'd like to
know how to do this kind of thing without the target and I'm striking out
in all the searches I've been doing.

Regards,

Brian

[-- Attachment #2: Type: text/html, Size: 672 bytes --]

Re: How to get a list of packages (with versions) used to create an image?

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 735 bytes --]

On Thu, Jan 23, 2014 at 02:35:06PM -0500, Brian Hutchinson wrote:
> Hi,
> 
> I've seen the bitbake -g command that lists the packages in an image:
> 
> bitbake -g <*image>* && cat pn-depends.dot | grep -v -e '-native' | grep -v
> digraph | grep -v -e '-image' | awk '{print $1}' | sort | uniq
> 
> ... but for documentation I need to generate this list with package version
> information as well.
> 
> I've been just running opkg list-installed on the target but I'd like to
> know how to do this kind of thing without the target and I'm striking out
> in all the searches I've been doing.

I would recommend using buildhistory.bbclass it's really useful.

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 205 bytes --]

Re: How to get a list of packages (with versions) used to create an image?

[Brian Hutchinson]

[-- Attachment #1: Type: text/plain, Size: 1388 bytes --]

On Thu, Jan 23, 2014 at 3:31 PM, Christian Ege <k4230r6@gmail.com> wrote:

> Hi,
> >
> > I've seen the bitbake -g command that lists the packages in an image:
> >
> > bitbake -g <image> && cat pn-depends.dot | grep -v -e '-native' | grep
> -v digraph | grep -v -e '-image' | awk '{print $1}' | sort | uniq
> >
> > ... but for documentation I need to generate this list with package
> version information as well.
>
> Have you considered to use the information in Packages files which are
> generated by
>
> *bitbake* package-index
>
> Another solution could be the build history:
> http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html#maintaining-build-output-quality
>
Yes, but:

1.  I don't think package-index is image specific ... its just a list of
everything that has been bitbaked on the box (which doesn't equate to one
particular image ... it's the net of all the packages from all the images
that have been built thus far if I'm not mistaken ... it main use is so the
feeds work and the the target know what packages are available).
2.  I felt like this was probably something common and a way already
existed so I thought I should ask in case I missed something before I
wasted time re-inventing the wheel.

Based on the responses so far ... it looks like this might not be as common
a request as I thought!

Regards,

Brian

[-- Attachment #2: Type: text/html, Size: 2027 bytes --]

Re: How to get a list of packages (with versions) used to create an image?

[Nicolas Dechesne]

On Thu, Jan 23, 2014 at 9:42 PM, Brian Hutchinson <b.hutchman@gmail.com> wrote:
> Yes, but:
>
> 1.  I don't think package-index is image specific ... its just a list of
> everything that has been bitbaked on the box (which doesn't equate to one
> particular image ... it's the net of all the packages from all the images
> that have been built thus far if I'm not mistaken ... it main use is so the
> feeds work and the the target know what packages are available).
> 2.  I felt like this was probably something common and a way already existed
> so I thought I should ask in case I missed something before I wasted time
> re-inventing the wheel.
>
> Based on the responses so far ... it looks like this might not be as common
> a request as I thought!
>

no. i think it is very common, and it's the purpose of 'buildhistory',
as mentioned already. for each image generated, you will get (among
other things)

installed-package-names.txt
installed-package-sizes.txt
installed-packages.txt

list of all packages with version, sizes, ..

Re: How to get a list of packages (with versions) used to create an image?

[Barros Pena, Belen]

On 23/01/2014 21:09, "Nicolas Dechesne" <nicolas.dechesne@linaro.org>
wrote:

>On Thu, Jan 23, 2014 at 9:42 PM, Brian Hutchinson <b.hutchman@gmail.com>
>wrote:
>> Yes, but:
>>
>> 1.  I don't think package-index is image specific ... its just a list of
>> everything that has been bitbaked on the box (which doesn't equate to
>>one
>> particular image ... it's the net of all the packages from all the
>>images
>> that have been built thus far if I'm not mistaken ... it main use is so
>>the
>> feeds work and the the target know what packages are available).
>> 2.  I felt like this was probably something common and a way already
>>existed
>> so I thought I should ask in case I missed something before I wasted
>>time
>> re-inventing the wheel.
>>
>> Based on the responses so far ... it looks like this might not be as
>>common
>> a request as I thought!
>>
>
>no. i think it is very common, and it's the purpose of 'buildhistory',
>as mentioned already. for each image generated, you will get (among
>other things)
>
>installed-package-names.txt
>installed-package-sizes.txt
>installed-packages.txt
>
>list of all packages with version, sizes, ..

I know this will not help, since we are still building Toaster. But is it
something like this

http://www.yoctoproject.org/toaster/image-information-packages.html

you are thinking of? I would be interested in hearing if this would give
you what you need. 

Cheers

Belén


>_______________________________________________
>yocto mailing list
>yocto@yoctoproject.org
>https://lists.yoctoproject.org/listinfo/yocto

Re: How to get a list of packages (with versions) used to create an image?

[Brian Hutchinson]

[-- Attachment #1: Type: text/plain, Size: 2126 bytes --]

On Fri, Jan 24, 2014 at 6:10 AM, Barros Pena, Belen <
belen.barros.pena@intel.com> wrote:

>
>
> On 23/01/2014 21:09, "Nicolas Dechesne" <nicolas.dechesne@linaro.org>
> wrote:
>
> >On Thu, Jan 23, 2014 at 9:42 PM, Brian Hutchinson <b.hutchman@gmail.com>
> >wrote:
> >> Yes, but:
> >>
> >> 1.  I don't think package-index is image specific ... its just a list of
> >> everything that has been bitbaked on the box (which doesn't equate to
> >>one
> >> particular image ... it's the net of all the packages from all the
> >>images
> >> that have been built thus far if I'm not mistaken ... it main use is so
> >>the
> >> feeds work and the the target know what packages are available).
> >> 2.  I felt like this was probably something common and a way already
> >>existed
> >> so I thought I should ask in case I missed something before I wasted
> >>time
> >> re-inventing the wheel.
> >>
> >> Based on the responses so far ... it looks like this might not be as
> >>common
> >> a request as I thought!
> >>
> >
> >no. i think it is very common, and it's the purpose of 'buildhistory',
> >as mentioned already. for each image generated, you will get (among
> >other things)
> >
> >installed-package-names.txt
> >installed-package-sizes.txt
> >installed-packages.txt
> >
> >list of all packages with version, sizes, ..
>
> I know this will not help, since we are still building Toaster. But is it
> something like this
>
> http://www.yoctoproject.org/toaster/image-information-packages.html
>
> you are thinking of? I would be interested in hearing if this would give
> you what you need.
>
>
Hi Belén,

That is VERY nice!  I like it.  I'm looking for something a bit simpler
though ... just want to be able to have a text file with the package names
and their versions that we can either include as a file or copy paste in
our release notes since we have to keep up with this (and license info.)
for every release we do.

How did you generate the html for that?  Looks like buildhistory is what I
need to be looking into from the responses so far.

Regards,

Brian

[-- Attachment #2: Type: text/html, Size: 3149 bytes --]

Re: How to get a list of packages (with versions) used to create an image?

[Paul Eggleton]

On Friday 24 January 2014 08:46:02 Brian Hutchinson wrote:
> That is VERY nice!  I like it.  I'm looking for something a bit simpler
> though ... just want to be able to have a text file with the package names
> and their versions that we can either include as a file or copy paste in
> our release notes since we have to keep up with this (and license info.)
> for every release we do.
> 
> How did you generate the html for that?  Looks like buildhistory is what I
> need to be looking into from the responses so far.

If you want license and version information, probably the best place to
get that is the automatically produced license manifest for the image:

http://www.yoctoproject.org/docs/current/dev-manual/dev-manual.html#maintaining-open-source-license-compliance-during-your-products-lifecycle

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: How to get a list of packages (with versions) used to create an image?

[Brian Hutchinson]

[-- Attachment #1: Type: text/plain, Size: 966 bytes --]

On Fri, Jan 24, 2014 at 8:53 AM, Paul Eggleton <
paul.eggleton@linux.intel.com> wrote:

>
> If you want license and version information, probably the best place to
> get that is the automatically produced license manifest for the image:
>
>
> http://www.yoctoproject.org/docs/current/dev-manual/dev-manual.html#maintaining-open-source-license-compliance-during-your-products-lifecycle
>
> Cheers,
> Paul
>
> For license compliance, I've just been taring up the deploy/licenses dir.
I know it includes way more than just the licenses for the packages in the
images but I've not investigated yet if there's a fancy way of managing
those better.

I've seen that section of the document you point out before ... I just need
to spend more time in it.  Our release procedure in this area needs to be
automated/streamlined and improved too!  I'll have to search the archives
and see if I can find prior discussions on that topic.

Regards,

Brian

[-- Attachment #2: Type: text/html, Size: 1595 bytes --]

Re: How to get a list of packages (with versions) used to create an image?

[Barros Pena, Belen]

On 24/01/2014 13:46, "Brian Hutchinson" <b.hutchman@gmail.com> wrote:

>
>On Fri, Jan 24, 2014 at 6:10 AM, Barros Pena, Belen
><belen.barros.pena@intel.com> wrote:
>
>
>
>On 23/01/2014 21:09, "Nicolas Dechesne" <nicolas.dechesne@linaro.org>
>wrote:
>
>>On Thu, Jan 23, 2014 at 9:42 PM, Brian Hutchinson <b.hutchman@gmail.com>
>>wrote:
>>> Yes, but:
>>>
>>> 1.  I don't think package-index is image specific ... its just a list
>>>of
>>> everything that has been bitbaked on the box (which doesn't equate to
>>>one
>>> particular image ... it's the net of all the packages from all the
>>>images
>>> that have been built thus far if I'm not mistaken ... it main use is so
>>>the
>>> feeds work and the the target know what packages are available).
>>> 2.  I felt like this was probably something common and a way already
>>>existed
>>> so I thought I should ask in case I missed something before I wasted
>>>time
>>> re-inventing the wheel.
>>>
>>> Based on the responses so far ... it looks like this might not be as
>>>common
>>> a request as I thought!
>>>
>>
>>no. i think it is very common, and it's the purpose of 'buildhistory',
>>as mentioned already. for each image generated, you will get (among
>>other things)
>>
>>installed-package-names.txt
>>installed-package-sizes.txt
>>installed-packages.txt
>>
>>list of all packages with version, sizes, ..
>
>
>
>I know this will not help, since we are still building Toaster. But is it
>something like this
>
>http://www.yoctoproject.org/toaster/image-information-packages.html
>
>you are thinking of? I would be interested in hearing if this would give
>you what you need.
>
>
>
>
>
>
>Hi Belén,
>
>
>That is VERY nice!  I like it.

Thanks, Brian. That is very encouraging :)

>I'm looking for something a bit simpler though ... just want to be able
>to have a text file with the package names and their versions that we can
>either include as a file or copy paste in our release
> notes since we have to keep up with this (and license info.) for every
>release we do.
>
>How did you generate the html for that?

What you looked at is only a prototype of the Toaster interface. Toaster
is an utility you can run alongside the build system. It collects
information about the build process and the artefacts generated, stores it
in a database, and presents it in a web interface. We are currently
building the web interface, but the back end of Toaster is already in
place. The back end comes with a rest API which in theory you could use to
collect the package, package version and license for a certain image, as
long as you had Toaster running during the build.

The API specification is here:

https://wiki.yoctoproject.org/wiki/Analysis_REST_API_Contracts

Instructions on how to Toaster are here:


>Looks like buildhistory is what I need to be looking into from the
>responses so far.
>
>
>
>Regards,
>
>Brian
>
>
>

Re: How to get a list of packages (with versions) used to create an image?

[Barros Pena, Belen]

On 24/01/2014 14:10, "Barros Pena, Belen" <belen.barros.pena@intel.com>
wrote:

>
>
>On 24/01/2014 13:46, "Brian Hutchinson" <b.hutchman@gmail.com> wrote:
>
>>
>>On Fri, Jan 24, 2014 at 6:10 AM, Barros Pena, Belen
>><belen.barros.pena@intel.com> wrote:
>>
>>
>>
>>On 23/01/2014 21:09, "Nicolas Dechesne" <nicolas.dechesne@linaro.org>
>>wrote:
>>
>>>On Thu, Jan 23, 2014 at 9:42 PM, Brian Hutchinson <b.hutchman@gmail.com>
>>>wrote:
>>>> Yes, but:
>>>>
>>>> 1.  I don't think package-index is image specific ... its just a list
>>>>of
>>>> everything that has been bitbaked on the box (which doesn't equate to
>>>>one
>>>> particular image ... it's the net of all the packages from all the
>>>>images
>>>> that have been built thus far if I'm not mistaken ... it main use is
>>>>so
>>>>the
>>>> feeds work and the the target know what packages are available).
>>>> 2.  I felt like this was probably something common and a way already
>>>>existed
>>>> so I thought I should ask in case I missed something before I wasted
>>>>time
>>>> re-inventing the wheel.
>>>>
>>>> Based on the responses so far ... it looks like this might not be as
>>>>common
>>>> a request as I thought!
>>>>
>>>
>>>no. i think it is very common, and it's the purpose of 'buildhistory',
>>>as mentioned already. for each image generated, you will get (among
>>>other things)
>>>
>>>installed-package-names.txt
>>>installed-package-sizes.txt
>>>installed-packages.txt
>>>
>>>list of all packages with version, sizes, ..
>>
>>
>>
>>I know this will not help, since we are still building Toaster. But is it
>>something like this
>>
>>http://www.yoctoproject.org/toaster/image-information-packages.html
>>
>>you are thinking of? I would be interested in hearing if this would give
>>you what you need.
>>
>>
>>
>>
>>
>>
>>Hi Belén,
>>
>>
>>That is VERY nice!  I like it.
>
>Thanks, Brian. That is very encouraging :)
>
>>I'm looking for something a bit simpler though ... just want to be able
>>to have a text file with the package names and their versions that we can
>>either include as a file or copy paste in our release
>> notes since we have to keep up with this (and license info.) for every
>>release we do.
>>
>>How did you generate the html for that?
>
>What you looked at is only a prototype of the Toaster interface. Toaster
>is an utility you can run alongside the build system. It collects
>information about the build process and the artefacts generated, stores it
>in a database, and presents it in a web interface. We are currently
>building the web interface, but the back end of Toaster is already in
>place. The back end comes with a rest API which in theory you could use to
>collect the package, package version and license for a certain image, as
>long as you had Toaster running during the build.
>
>The API specification is here:
>
>https://wiki.yoctoproject.org/wiki/Analysis_REST_API_Contracts
>
>Instructions on how to Toaster are here:

Oops: I meant here

https://wiki.yoctoproject.org/wiki/Toaster#Installation_and_Running


>
>
>>Looks like buildhistory is what I need to be looking into from the
>>responses so far.
>>
>>
>>
>>Regards,
>>
>>Brian
>>
>>
>>
>

Re: How to get a list of packages (with versions) used to create an image?

[Paul Eggleton]

On Friday 24 January 2014 09:07:46 Brian Hutchinson wrote:
> On Fri, Jan 24, 2014 at 8:53 AM, Paul Eggleton <
> 
> paul.eggleton@linux.intel.com> wrote:
> > If you want license and version information, probably the best place to
> > get that is the automatically produced license manifest for the image:
> > 
> > 
> > http://www.yoctoproject.org/docs/current/dev-manual/dev-manual.html#mainta
> > ining-open-source-license-compliance-during-your-products-lifecycle
> > 
> > Cheers,
> > Paul
> 
> For license compliance, I've just been taring up the deploy/licenses dir.
> 
> I know it includes way more than just the licenses for the packages in the
> images but I've not investigated yet if there's a fancy way of managing
> those better.
> 
> I've seen that section of the document you point out before ... I just need
> to spend more time in it.  Our release procedure in this area needs to be
> automated/streamlined and improved too!  I'll have to search the archives
> and see if I can find prior discussions on that topic.

The point is, there is a file 
tmp/deploy/licenses/<image_name>-<datestamp>/license.manifest which contains
a list of all installed packages, their version, their license and the recipe 
they were produced by. Isn't that what you were after?

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: How to get a list of packages (with versions) used to create an image?

[Brian Hutchinson]

[-- Attachment #1: Type: text/plain, Size: 1077 bytes --]

On Fri, Jan 24, 2014 at 9:20 AM, Paul Eggleton <
paul.eggleton@linux.intel.com> wrote:

>
>
> The point is, there is a file
> tmp/deploy/licenses/<image_name>-<datestamp>/license.manifest which
> contains
> a list of all installed packages, their version, their license and the
> recipe
> they were produced by. Isn't that what you were after?
>
> Cheers,
> Paul
>
> --
>

You are the man Paul!  My directory is so huge (multiple images for many
boards) that I never paid attention to those.  There is a license.manifest
and package.manifest.  The package file isn't too interesting as it is
missing the version info. but the license.manifest is gold!

Now if I could automate collecting the license info for just the packages
in that license.manifest I'd really be set.  I think that's possible too I
just need to spend more time in that section you pointed out in the manual.

But yes, that will do for now!  I wonder now how long these have been being
generated and sitting there all along and I just didn't realise it!  :(

Regards,

Brian

[-- Attachment #2: Type: text/html, Size: 1600 bytes --]

Re: How to get a list of packages (with versions) used to create an image?

[Trevor Woerner]

On 01/23/14 14:35, Brian Hutchinson wrote:
> but for documentation I need to generate this list with package
> version information as well.

I realize this has already been answered a couple different ways, but
(for completeness' sake) if you're using ipk as your package manager you
can also use the following.

Starting from your build directory, assuming you're building on an
x86_64 host, you're building "core-image-minimal", the machine you're
building for is "qemuarm", and your tmp directory (as defined in
conf/local.conf) is "tmp":

$ tmp/sysroots/x86_64-linux/usr/bin/opkg-cl \
    -o tmp/work/qemuarm-oe-linux-gnueabi/core-image-minimal/1.0-r0/rootfs \
    list-installed