of recipes and packages

of recipes and packages

[Trevor Woerner]

Okay, if the confusion over "yocto" versus "yocto project" versus
"poky" and trying to decide whether poky or yocto is a distribution or
a build system or sometimes both or neither of either wasn't bad
enough... there now comes a new confusion:

recipes versus packages

an image is composed of packages
a recipe for a given piece of software generates a set of packages

to discover the list of packages generated by a recipe one can use:
$ bitbake -e <recipe> | grep "^PACKAGES="

to discover the list of available recipes (for a given set of layers)
one can use:
$ bitbake -s

However:

$ bitbake --help | grep -- -s
...
  -s, --show-versions   show current and preferred versions of all packages
...

"...versions of all _packages_"!? Shouldn't that be "recipes"?

Re: of recipes and packages

[Jeff Osier-Mixon]

I tend to think of packages as "ingredients", in keeping with the
cooking metaphor. They have version numbers, just as ingredients in
the refrigerator have expiration dates. The recipes, however, have
their own version numbers, but they are less critical to the success
of the build - maybe my Aunt Agnes has a version of Betty Crocker's
yellow cake recipe, but improved slightly. If I make that recipe with
old eggs, though, it will fail.

In this case, bitbake is looking for the expiration dates on your
ingredients - the version of the package that each recipe builds. With
the list that comes from bitbake -s you can determine whether any of
your packages is the right one.

As for the versions of the recipes themselves, the Yocto Project's QA
and testing process makes sure that each of the included recipes
works, in the sense that it builds the package reliably. You can spice
them up if you like.

Chris Hallinan wrote up a good description of many of the terms used
in the project:
http://blogs.mentor.com/chrishallinan/blog/2012/04/13/yocto-versus-poky-versus-angstrom-etc/

Hope that helps

On Wed, Aug 22, 2012 at 10:05 AM, Trevor Woerner <twoerner@gmail.com> wrote:
> Okay, if the confusion over "yocto" versus "yocto project" versus
> "poky" and trying to decide whether poky or yocto is a distribution or
> a build system or sometimes both or neither of either wasn't bad
> enough... there now comes a new confusion:
>
> recipes versus packages
>
> an image is composed of packages
> a recipe for a given piece of software generates a set of packages
>
> to discover the list of packages generated by a recipe one can use:
> $ bitbake -e <recipe> | grep "^PACKAGES="
>
> to discover the list of available recipes (for a given set of layers)
> one can use:
> $ bitbake -s
>
> However:
>
> $ bitbake --help | grep -- -s
> ...
>   -s, --show-versions   show current and preferred versions of all packages
> ...
>
> "...versions of all _packages_"!? Shouldn't that be "recipes"?
> _______________________________________________
> yocto mailing list
> yocto@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto



-- 
Jeff Osier-Mixon http://jefro.net/blog
Yocto Project Community Manager @Intel http://yoctoproject.org

Re: of recipes and packages

[Trevor Woerner]

Hi Jeff,

Thanks for your tasty metaphors!

On Wed, Aug 22, 2012 at 1:15 PM, Jeff Osier-Mixon <jefro@jefro.net> wrote:
> In this case, bitbake is looking for the expiration dates on your
> ingredients - the version of the package that each recipe builds.

But your wording would seem to imply _a_ recipe builds _a_ package.
But a recipe doesn't build a package, a recipe potentially builds lots
of packages. For example the "net-snmp" recipe builds the following
packages: "net-snmp-dbg net-snmp-doc net-snmp-dev net-snmp-staticdev
net-snmp-static net-snmp-libs net-snmp-mibs net-snmp-server
net-snmp-client"

"bitbake -s" doesn't list "net-snmp-dbg net-snmp-doc net-snmp-dev..."
it lists "net-snmp". Therefore couldn't the wording of the bitbake
help be improved to say:

-s, --show-versions   show current and preferred versions of all __recipes__

instead of:

>>   -s, --show-versions   show current and preferred versions of all __packages__

?

> Chris Hallinan wrote up a good description of many of the terms used

Actually I wrote this email after reading his very next blog post:

http://blogs.mentor.com/chrishallinan/blog/2012/04/27/more-on-yocto-terminology-recipes-and-packages/

:-)

Re: of recipes and packages

[Chris Larson]

On Wed, Aug 22, 2012 at 12:23 PM, Trevor Woerner <twoerner@gmail.com> wrote:
> On Wed, Aug 22, 2012 at 1:15 PM, Jeff Osier-Mixon <jefro@jefro.net> wrote:
>> In this case, bitbake is looking for the expiration dates on your
>> ingredients - the version of the package that each recipe builds.
>
> But your wording would seem to imply _a_ recipe builds _a_ package.
> But a recipe doesn't build a package, a recipe potentially builds lots
> of packages. For example the "net-snmp" recipe builds the following
> packages: "net-snmp-dbg net-snmp-doc net-snmp-dev net-snmp-staticdev
> net-snmp-static net-snmp-libs net-snmp-mibs net-snmp-server
> net-snmp-client"
>
> "bitbake -s" doesn't list "net-snmp-dbg net-snmp-doc net-snmp-dev..."
> it lists "net-snmp". Therefore couldn't the wording of the bitbake
> help be improved to say:
>
> -s, --show-versions   show current and preferred versions of all __recipes__
>
> instead of:


It could, and should, be changed in that way, yes.
-- 
Christopher Larson

Re: of recipes and packages

[Trevor Woerner]

On Wed, Aug 22, 2012 at 3:24 PM, Chris Larson <clarson@kergoth.com> wrote:
> On Wed, Aug 22, 2012 at 12:23 PM, Trevor Woerner <twoerner@gmail.com> wrote:
>> "bitbake -s" doesn't list "net-snmp-dbg net-snmp-doc net-snmp-dev..."
>> it lists "net-snmp". Therefore couldn't the wording of the bitbake
>> help be improved to say:
>>
>> -s, --show-versions   show current and preferred versions of all __recipes__
>>
>> instead of:
>
>
> It could, and should, be changed in that way, yes.

Do the OE people accept this wording too, or is this a
yocto-project-only thing? In other words, should a potential patch be
sent to OE or here?

Re: of recipes and packages

[Chris Larson]

On Wed, Aug 22, 2012 at 1:15 PM, Trevor Woerner <twoerner@gmail.com> wrote:
> On Wed, Aug 22, 2012 at 3:24 PM, Chris Larson <clarson@kergoth.com> wrote:
>> On Wed, Aug 22, 2012 at 12:23 PM, Trevor Woerner <twoerner@gmail.com> wrote:
>>> "bitbake -s" doesn't list "net-snmp-dbg net-snmp-doc net-snmp-dev..."
>>> it lists "net-snmp". Therefore couldn't the wording of the bitbake
>>> help be improved to say:
>>>
>>> -s, --show-versions   show current and preferred versions of all __recipes__
>>>
>>> instead of:
>>
>>
>> It could, and should, be changed in that way, yes.
>
> Do the OE people accept this wording too, or is this a
> yocto-project-only thing? In other words, should a potential patch be
> sent to OE or here?

The change would be to a core component, which is part of OE and which
yocto pulls in — bitbake. So it would make no sense to send the patch
here. No changes to bitbake are going into poky without going into the
main bitbake repository. The bitbake-devel mailing list is the correct
place for it.
-- 
Christopher Larson

Re: of recipes and packages

[Robert P. J. Day]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 1829 bytes --]

On Wed, 22 Aug 2012, Chris Larson wrote:

> On Wed, Aug 22, 2012 at 1:15 PM, Trevor Woerner <twoerner@gmail.com> wrote:
> > On Wed, Aug 22, 2012 at 3:24 PM, Chris Larson <clarson@kergoth.com> wrote:
> >> On Wed, Aug 22, 2012 at 12:23 PM, Trevor Woerner <twoerner@gmail.com> wrote:
> >>> "bitbake -s" doesn't list "net-snmp-dbg net-snmp-doc net-snmp-dev..."
> >>> it lists "net-snmp". Therefore couldn't the wording of the bitbake
> >>> help be improved to say:
> >>>
> >>> -s, --show-versions   show current and preferred versions of all __recipes__
> >>>
> >>> instead of:
> >>
> >>
> >> It could, and should, be changed in that way, yes.
> >
> > Do the OE people accept this wording too, or is this a
> > yocto-project-only thing? In other words, should a potential patch be
> > sent to OE or here?
>
> The change would be to a core component, which is part of OE and which
> yocto pulls in — bitbake. So it would make no sense to send the patch
> here. No changes to bitbake are going into poky without going into the
> main bitbake repository. The bitbake-devel mailing list is the correct
> place for it.

  there really should be an official glossary somewhere, and it should
be backed up with *actual* *examples* from the source as much as
possible.  that is, don't use "foo" if there's an existing recipe or
package whose use would be more informative.

rday

-- 

========================================================================
Robert P. J. Day                                 Ottawa, Ontario, CANADA
                        http://crashcourse.ca

Twitter:                                       http://twitter.com/rpjday
LinkedIn:                               http://ca.linkedin.com/in/rpjday
========================================================================

Re: of recipes and packages

[Tim Bird]

On 08/22/2012 01:27 PM, Robert P. J. Day wrote:
> On Wed, 22 Aug 2012, Chris Larson wrote:
> 
>> On Wed, Aug 22, 2012 at 1:15 PM, Trevor Woerner <twoerner@gmail.com> wrote:
>>> On Wed, Aug 22, 2012 at 3:24 PM, Chris Larson <clarson@kergoth.com> wrote:
>>>> On Wed, Aug 22, 2012 at 12:23 PM, Trevor Woerner <twoerner@gmail.com> wrote:
>>>>> "bitbake -s" doesn't list "net-snmp-dbg net-snmp-doc net-snmp-dev..."
>>>>> it lists "net-snmp". Therefore couldn't the wording of the bitbake
>>>>> help be improved to say:
>>>>>
>>>>> -s, --show-versions   show current and preferred versions of all __recipes__
>>>>>
>>>>> instead of:
>>>>
>>>>
>>>> It could, and should, be changed in that way, yes.
>>>
>>> Do the OE people accept this wording too, or is this a
>>> yocto-project-only thing? In other words, should a potential patch be
>>> sent to OE or here?
>>
>> The change would be to a core component, which is part of OE and which
>> yocto pulls in — bitbake. So it would make no sense to send the patch
>> here. No changes to bitbake are going into poky without going into the
>> main bitbake repository. The bitbake-devel mailing list is the correct
>> place for it.
> 
>   there really should be an official glossary somewhere, and it should
> be backed up with *actual* *examples* from the source as much as
> possible.  that is, don't use "foo" if there's an existing recipe or
> package whose use would be more informative.

From Jeff's description, it sounded like the package (especially 'package version')
comes from the stuff that is the recipe's *input*, and not the recipe's output
 -- if you've selected to build packages and not just a straight image.

Is 'package' also used in that sense, to describe, say, the tarball for busybox
before it's processed by bitbake and made into an busybox binary ipkg or rpm?

Or am I just muddying the waters further?

BTW, on denzil, I get the following:
$ bitbake -s | grep busybox
busybox                         :1.19.4-r2

Note that this includes the version of busybox (the input source version), as
well as (I think) the recipe revision number.

And yes - a definitive glossary would be great.
 -- Tim

=============================
Tim Bird
Architecture Group Chair, CE Workgroup of the Linux Foundation
Senior Staff Engineer, Sony Network Entertainment
=============================

Re: of recipes and packages

[Khem Raj]

On Wed, Aug 22, 2012 at 4:24 PM, Tim Bird <tim.bird@am.sony.com> wrote:
> On 08/22/2012 01:27 PM, Robert P. J. Day wrote:
>> On Wed, 22 Aug 2012, Chris Larson wrote:
>>
>>> On Wed, Aug 22, 2012 at 1:15 PM, Trevor Woerner <twoerner@gmail.com> wrote:
>>>> On Wed, Aug 22, 2012 at 3:24 PM, Chris Larson <clarson@kergoth.com> wrote:
>>>>> On Wed, Aug 22, 2012 at 12:23 PM, Trevor Woerner <twoerner@gmail.com> wrote:
>>>>>> "bitbake -s" doesn't list "net-snmp-dbg net-snmp-doc net-snmp-dev..."
>>>>>> it lists "net-snmp". Therefore couldn't the wording of the bitbake
>>>>>> help be improved to say:
>>>>>>
>>>>>> -s, --show-versions   show current and preferred versions of all __recipes__
>>>>>>
>>>>>> instead of:
>>>>>
>>>>>
>>>>> It could, and should, be changed in that way, yes.
>>>>
>>>> Do the OE people accept this wording too, or is this a
>>>> yocto-project-only thing? In other words, should a potential patch be
>>>> sent to OE or here?
>>>
>>> The change would be to a core component, which is part of OE and which
>>> yocto pulls in — bitbake. So it would make no sense to send the patch
>>> here. No changes to bitbake are going into poky without going into the
>>> main bitbake repository. The bitbake-devel mailing list is the correct
>>> place for it.
>>
>>   there really should be an official glossary somewhere, and it should
>> be backed up with *actual* *examples* from the source as much as
>> possible.  that is, don't use "foo" if there's an existing recipe or
>> package whose use would be more informative.
>
> From Jeff's description, it sounded like the package (especially 'package version')
> comes from the stuff that is the recipe's *input*, and not the recipe's output
>  -- if you've selected to build packages and not just a straight image.
>
> Is 'package' also used in that sense, to describe, say, the tarball for busybox
> before it's processed by bitbake and made into an busybox binary ipkg or rpm?
>
> Or am I just muddying the waters further?
>
> BTW, on denzil, I get the following:
> $ bitbake -s | grep busybox
> busybox                         :1.19.4-r2
>


anything you bake using bitbake is a recipe (input rules) which then
generates packages (output) and there can be many packages generated
from single recipes and one of the name of output package can be same
as recipe name.


> Note that this includes the version of busybox (the input source version), as
> well as (I think) the recipe revision number.
>
> And yes - a definitive glossary would be great.
>  -- Tim
>
> =============================
> Tim Bird
> Architecture Group Chair, CE Workgroup of the Linux Foundation
> Senior Staff Engineer, Sony Network Entertainment
> =============================
>
> _______________________________________________
> yocto mailing list
> yocto@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto

Re: of recipes and packages

[Paul Eggleton]

On Wednesday 22 August 2012 13:05:15 Trevor Woerner wrote:
> to discover the list of available recipes (for a given set of layers)
> one can use:
> $ bitbake -s
> 
> However:
> 
> $ bitbake --help | grep -- -s
> ...
>   -s, --show-versions   show current and preferred versions of all packages
> ...
> 
> "...versions of all _packages_"!? Shouldn't that be "recipes"?

It should, yes. FYI a long time ago we just used the word "package" to 
describe both the recipe and the output package - eventually it was realised 
that this was confusing and it was changed; however there are still odd 
vestiges of it in a few places, one of which you just pointed out. In this 
instance we can change it very easily in the BitBake help text and we should 
do so.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: of recipes and packages

[Trevor Woerner]

On Thu, Aug 23, 2012 at 3:14 AM, Paul Eggleton
<paul.eggleton@linux.intel.com> wrote:
> In this
> instance we can change it very easily in the BitBake help text and we should
> do so.

So should I send in a patch against the Yocto Project's
poky/bitbake/bin/bitbake or try to have it accepted upstream?

Re: of recipes and packages

[Chris Larson]

On Thu, Aug 23, 2012 at 11:02 AM, Trevor Woerner <twoerner@gmail.com> wrote:
> On Thu, Aug 23, 2012 at 3:14 AM, Paul Eggleton
> <paul.eggleton@linux.intel.com> wrote:
>> In this
>> instance we can change it very easily in the BitBake help text and we should
>> do so.
>
> So should I send in a patch against the Yocto Project's
> poky/bitbake/bin/bitbake or try to have it accepted upstream?

As I already said once in this thread, bitbake-devel is the correct
place for it to go.
-- 
Christopher Larson