Using "latest git" versions of recipes

Using "latest git" versions of recipes

[Joel A Cohen]

[-- Attachment #1: Type: text/plain, Size: 1572 bytes --]

Hi all,

What I've done for quite a while when creating recipes for code that is
under heavy development is create 2 version of the recipe.

1) recipe_1.0.0.bb (a known-stable recipe)

2) recipe_git.bb (which contains something like:
SRCREV="${AUTOREV}"
SRCPV="1.0.1+git${SRCPV}"
DEFAULT_PREFERENCE = "-1"


The major problem I have with this, is that it defeats BB_NO_NETWORK, even
if I'm not using the git version of the recipe (because bitbake wants to go
check what the latest version is of the recipe during parsing).

I'm currently experimenting with a different approapch, which seems to have
several advantages. I wanted to share it and get commentary.

1) Add a new class, "use-git.bbclass":
# This allows using the latest version of the source without requiring a
separate
# "recipe_git.bb". To use:
# 1) Add: INHERIT += "use-git"
# 2) Add: GIT_PNS = "recipe1 recipe2 etc"

GIT_PNS ?= ""
GIT_PNS[type] = "list"

python () {
    pn = d.getVar('PN')
    git_pns = d.getVar('GIT_PNS', [])

    if pn in git_pns:
        d.setVar('SRCREV', d.getVar('AUTOREV'))
        srcpv = d.getVar('SRCPV')
        d.appendVar('PV', '+git' + srcpv)
        bb.note('Using latest git source code for %s, as recipe %s' % (pn,
d.getVar('BP')))
}



2) In local.conf:
INHERIT += "use-git"
GIT_PNS = "recipe1 recipe2 etc"



This is nice, because I no longer have to create those recipe_git.bb
recipes, and it's a central place to go configure a "use latest source
code" setting.

Is this interesting to anyone else?

--Aaron

[-- Attachment #2: Type: text/html, Size: 2370 bytes --]

Re: [yocto] Using "latest git" versions of recipes

[Denys Dmytriyenko]

On Tue, Feb 25, 2020 at 01:21:53PM -0500, Joel A Cohen wrote:
> Hi all,
> 
> What I've done for quite a while when creating recipes for code that is
> under heavy development is create 2 version of the recipe.
> 
> 1) recipe_1.0.0.bb (a known-stable recipe)
> 
> 2) recipe_git.bb (which contains something like:
> SRCREV="${AUTOREV}"
> SRCPV="1.0.1+git${SRCPV}"
> DEFAULT_PREFERENCE = "-1"
> 
> 
> The major problem I have with this, is that it defeats BB_NO_NETWORK, even
> if I'm not using the git version of the recipe (because bitbake wants to go
> check what the latest version is of the recipe during parsing).
> 
> I'm currently experimenting with a different approapch, which seems to have
> several advantages. I wanted to share it and get commentary.
> 
> 1) Add a new class, "use-git.bbclass":
> # This allows using the latest version of the source without requiring a
> separate
> # "recipe_git.bb". To use:
> # 1) Add: INHERIT += "use-git"
> # 2) Add: GIT_PNS = "recipe1 recipe2 etc"
> 
> GIT_PNS ?= ""
> GIT_PNS[type] = "list"
> 
> python () {
>     pn = d.getVar('PN')
>     git_pns = d.getVar('GIT_PNS', [])
> 
>     if pn in git_pns:
>         d.setVar('SRCREV', d.getVar('AUTOREV'))
>         srcpv = d.getVar('SRCPV')
>         d.appendVar('PV', '+git' + srcpv)
>         bb.note('Using latest git source code for %s, as recipe %s' % (pn,
> d.getVar('BP')))
> }
> 
> 
> 
> 2) In local.conf:
> INHERIT += "use-git"
> GIT_PNS = "recipe1 recipe2 etc"
> 
> 
> 
> This is nice, because I no longer have to create those recipe_git.bb
> recipes, and it's a central place to go configure a "use latest source
> code" setting.
> 
> Is this interesting to anyone else?
> 
> --Aaron

You know that you can poke SRCREV and other vars from a config file, 
such as local.conf?

SRCREV_pn-recipe1 = "${AUTOREV}"
SRCREV_pn-recipe2 = "${AUTOREV}"

-- 
Denys

Re: [yocto] Using "latest git" versions of recipes

[Joel A Cohen]

[-- Attachment #1: Type: text/plain, Size: 2653 bytes --]

Hmm, that is a very good suggestion that I hadn't thought of. I guess the
only disadvantage is that PV doesn't get changed so you end up having to
keep track of which "recipe_1.0.0.rpm" is the test one and which is the
good version, but even that you could probably do:

SRCREV_pn-recipe1 = "${AUTOREV}"
PV_pn-recipe1 = "1.0.1+git${SRCPV}"  # I'm not sure if this would actually
work?

My bbclass thing is a little easier if you have to do a bunch of those, but
maybe not worthwhile then.

Thanks!
Aaron


On Tue, Feb 25, 2020 at 1:30 PM Denys Dmytriyenko <denis@denix.org> wrote:

> On Tue, Feb 25, 2020 at 01:21:53PM -0500, Joel A Cohen wrote:
> > Hi all,
> >
> > What I've done for quite a while when creating recipes for code that is
> > under heavy development is create 2 version of the recipe.
> >
> > 1) recipe_1.0.0.bb (a known-stable recipe)
> >
> > 2) recipe_git.bb (which contains something like:
> > SRCREV="${AUTOREV}"
> > SRCPV="1.0.1+git${SRCPV}"
> > DEFAULT_PREFERENCE = "-1"
> >
> >
> > The major problem I have with this, is that it defeats BB_NO_NETWORK,
> even
> > if I'm not using the git version of the recipe (because bitbake wants to
> go
> > check what the latest version is of the recipe during parsing).
> >
> > I'm currently experimenting with a different approapch, which seems to
> have
> > several advantages. I wanted to share it and get commentary.
> >
> > 1) Add a new class, "use-git.bbclass":
> > # This allows using the latest version of the source without requiring a
> > separate
> > # "recipe_git.bb". To use:
> > # 1) Add: INHERIT += "use-git"
> > # 2) Add: GIT_PNS = "recipe1 recipe2 etc"
> >
> > GIT_PNS ?= ""
> > GIT_PNS[type] = "list"
> >
> > python () {
> >     pn = d.getVar('PN')
> >     git_pns = d.getVar('GIT_PNS', [])
> >
> >     if pn in git_pns:
> >         d.setVar('SRCREV', d.getVar('AUTOREV'))
> >         srcpv = d.getVar('SRCPV')
> >         d.appendVar('PV', '+git' + srcpv)
> >         bb.note('Using latest git source code for %s, as recipe %s' %
> (pn,
> > d.getVar('BP')))
> > }
> >
> >
> >
> > 2) In local.conf:
> > INHERIT += "use-git"
> > GIT_PNS = "recipe1 recipe2 etc"
> >
> >
> >
> > This is nice, because I no longer have to create those recipe_git.bb
> > recipes, and it's a central place to go configure a "use latest source
> > code" setting.
> >
> > Is this interesting to anyone else?
> >
> > --Aaron
>
> You know that you can poke SRCREV and other vars from a config file,
> such as local.conf?
>
> SRCREV_pn-recipe1 = "${AUTOREV}"
> SRCREV_pn-recipe2 = "${AUTOREV}"
>
> --
> Denys
>

[-- Attachment #2: Type: text/html, Size: 3903 bytes --]

Re: [yocto] Using "latest git" versions of recipes

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 3324 bytes --]

On Tue, Feb 25, 2020 at 02:10:15PM -0500, Joel A Cohen wrote:
> Hmm, that is a very good suggestion that I hadn't thought of. I guess the
> only disadvantage is that PV doesn't get changed so you end up having to
> keep track of which "recipe_1.0.0.rpm" is the test one and which is the
> good version, but even that you could probably do:
> 
> SRCREV_pn-recipe1 = "${AUTOREV}"

You can put these in .inc file so that other people can enable it for
the same group of recipes in one "require" line in local.conf.

> PV_pn-recipe1 = "1.0.1+git${SRCPV}"  # I'm not sure if this would actually
> work?

Yes, this is how SRCPV is supposed to be used, but you can have this in
the normal recipe without the override.

You can also use one of the bbclasses like:
meta-openembedded/meta-oe/classes/gitpkgv.bbclass
to have nicer versions, if the components have some usable git tags

> My bbclass thing is a little easier if you have to do a bunch of those, but
> maybe not worthwhile then.
> 
> Thanks!
> Aaron
> 
> 
> On Tue, Feb 25, 2020 at 1:30 PM Denys Dmytriyenko <denis@denix.org> wrote:
> 
> > On Tue, Feb 25, 2020 at 01:21:53PM -0500, Joel A Cohen wrote:
> > > Hi all,
> > >
> > > What I've done for quite a while when creating recipes for code that is
> > > under heavy development is create 2 version of the recipe.
> > >
> > > 1) recipe_1.0.0.bb (a known-stable recipe)
> > >
> > > 2) recipe_git.bb (which contains something like:
> > > SRCREV="${AUTOREV}"
> > > SRCPV="1.0.1+git${SRCPV}"
> > > DEFAULT_PREFERENCE = "-1"
> > >
> > >
> > > The major problem I have with this, is that it defeats BB_NO_NETWORK,
> > even
> > > if I'm not using the git version of the recipe (because bitbake wants to
> > go
> > > check what the latest version is of the recipe during parsing).
> > >
> > > I'm currently experimenting with a different approapch, which seems to
> > have
> > > several advantages. I wanted to share it and get commentary.
> > >
> > > 1) Add a new class, "use-git.bbclass":
> > > # This allows using the latest version of the source without requiring a
> > > separate
> > > # "recipe_git.bb". To use:
> > > # 1) Add: INHERIT += "use-git"
> > > # 2) Add: GIT_PNS = "recipe1 recipe2 etc"
> > >
> > > GIT_PNS ?= ""
> > > GIT_PNS[type] = "list"
> > >
> > > python () {
> > >     pn = d.getVar('PN')
> > >     git_pns = d.getVar('GIT_PNS', [])
> > >
> > >     if pn in git_pns:
> > >         d.setVar('SRCREV', d.getVar('AUTOREV'))
> > >         srcpv = d.getVar('SRCPV')
> > >         d.appendVar('PV', '+git' + srcpv)
> > >         bb.note('Using latest git source code for %s, as recipe %s' %
> > (pn,
> > > d.getVar('BP')))
> > > }
> > >
> > >
> > >
> > > 2) In local.conf:
> > > INHERIT += "use-git"
> > > GIT_PNS = "recipe1 recipe2 etc"
> > >
> > >
> > >
> > > This is nice, because I no longer have to create those recipe_git.bb
> > > recipes, and it's a central place to go configure a "use latest source
> > > code" setting.
> > >
> > > Is this interesting to anyone else?
> > >
> > > --Aaron
> >
> > You know that you can poke SRCREV and other vars from a config file,
> > such as local.conf?
> >
> > SRCREV_pn-recipe1 = "${AUTOREV}"
> > SRCREV_pn-recipe2 = "${AUTOREV}"
> >
> > --
> > Denys
> >

> 


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 201 bytes --]

Re: [yocto] Using "latest git" versions of recipes

[Joel A Cohen]

[-- Attachment #1: Type: text/plain, Size: 646 bytes --]

On Tue, Feb 25, 2020 at 2:28 PM Martin Jansa <martin.jansa@gmail.com> wrote:

> On Tue, Feb 25, 2020 at 02:10:15PM -0500, Joel A Cohen wrote:
>
>
> > PV_pn-recipe1 = "1.0.1+git${SRCPV}"  # I'm not sure if this would
> actually
> > work?
>
> Yes, this is how SRCPV is supposed to be used, but you can have this in
> the normal recipe without the override.
>

Everything else you said makes sense to me, but why would I want to append
"+git${SRCPV}" to the actual recipe? Surely I only want to append the git
suffix stuff if I'm using a "latest" version, and I want the PV to be set
by the filename in the default case?

--Aaron

[-- Attachment #2: Type: text/html, Size: 1061 bytes --]

Re: [yocto] Using "latest git" versions of recipes

[Joshua Watt]

On Tue, Feb 25, 2020 at 1:32 PM Joel A Cohen <aaron@assonance.org> wrote:
>
>
>
> On Tue, Feb 25, 2020 at 2:28 PM Martin Jansa <martin.jansa@gmail.com> wrote:
>>
>> On Tue, Feb 25, 2020 at 02:10:15PM -0500, Joel A Cohen wrote:
>>
>>
>> > PV_pn-recipe1 = "1.0.1+git${SRCPV}"  # I'm not sure if this would actually
>> > work?
>>
>> Yes, this is how SRCPV is supposed to be used, but you can have this in
>> the normal recipe without the override.
>
>
> Everything else you said makes sense to me, but why would I want to append "+git${SRCPV}" to the actual recipe? Surely I only want to append the git suffix stuff if I'm using a "latest" version, and I want the PV to be set by the filename in the default case?

You should checkout devupstream.bbclass
(https://www.yoctoproject.org/docs/current/mega-manual/mega-manual.html#ref-classes-devupstream)
which allows you to do this inside a single recipe (or bbappend and
existing recipe). You can also combine this with ${AUTOREV} in a
common include file. I recently fixed this exact same problem with
this combination in meta-webkit; see
https://github.com/Igalia/meta-webkit/pull/136

>
> --Aaron
>
> 

Re: [yocto] Using "latest git" versions of recipes

[Denys Dmytriyenko]

On Tue, Feb 25, 2020 at 02:39:28PM -0600, Joshua Watt wrote:
> On Tue, Feb 25, 2020 at 1:32 PM Joel A Cohen <aaron@assonance.org> wrote:
> >
> >
> >
> > On Tue, Feb 25, 2020 at 2:28 PM Martin Jansa <martin.jansa@gmail.com> wrote:
> >>
> >> On Tue, Feb 25, 2020 at 02:10:15PM -0500, Joel A Cohen wrote:
> >>
> >>
> >> > PV_pn-recipe1 = "1.0.1+git${SRCPV}"  # I'm not sure if this would actually
> >> > work?
> >>
> >> Yes, this is how SRCPV is supposed to be used, but you can have this in
> >> the normal recipe without the override.
> >
> >
> > Everything else you said makes sense to me, but why would I want to append "+git${SRCPV}" to the actual recipe? Surely I only want to append the git suffix stuff if I'm using a "latest" version, and I want the PV to be set by the filename in the default case?
> 
> You should checkout devupstream.bbclass
> (https://www.yoctoproject.org/docs/current/mega-manual/mega-manual.html#ref-classes-devupstream)
> which allows you to do this inside a single recipe (or bbappend and
> existing recipe). You can also combine this with ${AUTOREV} in a
> common include file. I recently fixed this exact same problem with
> this combination in meta-webkit; see
> https://github.com/Igalia/meta-webkit/pull/136

Ah, nice touch with devupstream class and I see bleeding.inc concept lives on! :)

-- 
Denys