Coordinating inter-layer dependencies

Coordinating inter-layer dependencies

[Koen Kooi]

[-- Attachment #1: Type: text/plain, Size: 1675 bytes --]

Hi,

During the past month there have been a number of updates to OE-core recipes that triggered parsing errors due to bbappend in other layers. A small seleciton:

* netbase
* libdrm
* xserver-xorg
* clutter

My view is that layer maintainers need to keep an eye on potential breakage and have updates ready when patches land into OE-core. Looking back I can see that while the situation is improving a bit, it's still not working. The problem with slow updates to layers is that (with my angstrom hat on) users (and with my TI hat on) customers and coworkers can't do builds without rm'ing the bbappends or disabling the layer.

This is bad for a number of reasons:

1) It creates unmanaged local diffs
2) it can cause PR to fluctuate back and forth if you rm is a bit overzealous or if you disable the complete layer.

My proposal is that OE-core recipe upgrades with known bbappends look like this:

1) add new version with a warning about bbappends
2) wait a N days (2 < N < 7)
3) delete old version

To avoid stressing out RP and Sau! I would strongly urge layer maintainers to respond to recipe update patches with "I have a bbappend, but my review process needs time, please use the above proposal" if you need time to test and update  your bbappends.

We're all human in the end and timezone differences aren't always helping with these kinds of problems, so let's work together to minimize parsing breakage.

Related to that: Please include information on how to send patches upstream in your layer/repo README. The meta-intel/emenlow lacks that and the top level README doesn't mention that there are sublevel READMEs.

regards,

Koen

[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 169 bytes --]

Re: Coordinating inter-layer dependencies

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 2625 bytes --]

On Thu, Dec 01, 2011 at 12:23:08PM +0100, Koen Kooi wrote:
> Hi,
> 
> During the past month there have been a number of updates to OE-core recipes that triggered parsing errors due to bbappend in other layers. A small seleciton:
> 
> * netbase
> * libdrm
> * xserver-xorg
> * clutter
> 
> My view is that layer maintainers need to keep an eye on potential breakage and have updates ready when patches land into OE-core. Looking back I can see that while the situation is improving a bit, it's still not working. The problem with slow updates to layers is that (with my angstrom hat on) users (and with my TI hat on) customers and coworkers can't do builds without rm'ing the bbappends or disabling the layer.
> 
> This is bad for a number of reasons:
> 
> 1) It creates unmanaged local diffs
> 2) it can cause PR to fluctuate back and forth if you rm is a bit overzealous or if you disable the complete layer.
> 
> My proposal is that OE-core recipe upgrades with known bbappends look like this:
> 
> 1) add new version with a warning about bbappends
> 2) wait a N days (2 < N < 7)
> 3) delete old version
> 
> To avoid stressing out RP and Sau! I would strongly urge layer maintainers to respond to recipe update patches with "I have a bbappend, but my review process needs time, please use the above proposal" if you need time to test and update  your bbappends.
> 
> We're all human in the end and timezone differences aren't always helping with these kinds of problems, so let's work together to minimize parsing breakage.

That's exactly the reason why I'm using contrib/shr branches and my
setup scripts are updating only from contrib/shr, so I can push changes
to all related branches almost at once and when they are ready.

Sometimes it means extra patches in contrib/shr which are not yet in
master (and sometimes they are on ML already) sometimes it means that
I'm waiting with pulling newer master to shr branch before I prepare
corresponding changes in meta-smartphone (this was the case ie for last
libdrm upgrade). But to test corresponding changes sometimes takes many
hours to even build it...

A while back I've proposed to make .bbappend without corresponding .bb
only big fat warning, but not fatal to parse. Now you cannot even build
eglibc if there is libdrm bbappend you don't care at all about..

Of course if someone ignores big fat warning and build libdrm without
bbappend and then his mesa build is failing, he has to go back and
cleansstate libdrm to build it again properly.

Regards,

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 205 bytes --]

Re: Coordinating inter-layer dependencies

[Richard Purdie]

On Thu, 2011-12-01 at 13:24 +0100, Martin Jansa wrote:
> A while back I've proposed to make .bbappend without corresponding .bb
> only big fat warning, but not fatal to parse. Now you cannot even build
> eglibc if there is libdrm bbappend you don't care at all about..

You can do this by setting:

BB_DANGLINGAPPENDS_WARNONLY

Cheers,

Richard

Re: Coordinating inter-layer dependencies

[Otavio Salvador]

[-- Attachment #1: Type: text/plain, Size: 828 bytes --]

On Thu, Dec 1, 2011 at 10:37, Richard Purdie <
richard.purdie@linuxfoundation.org> wrote:

> On Thu, 2011-12-01 at 13:24 +0100, Martin Jansa wrote:
> > A while back I've proposed to make .bbappend without corresponding .bb
> > only big fat warning, but not fatal to parse. Now you cannot even build
> > eglibc if there is libdrm bbappend you don't care at all about..
>
> You can do this by setting:
>
> BB_DANGLINGAPPENDS_WARNONLY
>

This is even worse; you end up with a package without the changes done on
the bbappend and as most bbappend files do not change PR, adding it later
won't force a package update.

-- 
Otavio Salvador                             O.S. Systems
E-mail: otavio@ossystems.com.br  http://www.ossystems.com.br
Mobile: +55 53 9981-7854              http://projetos.ossystems.com.br

[-- Attachment #2: Type: text/html, Size: 1346 bytes --]

Re: Coordinating inter-layer dependencies

[Richard Purdie]

On Thu, 2011-12-01 at 10:59 -0200, Otavio Salvador wrote:
> On Thu, Dec 1, 2011 at 10:37, Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
>         On Thu, 2011-12-01 at 13:24 +0100, Martin Jansa wrote:
>         > A while back I've proposed to make .bbappend without
>         corresponding .bb
>         > only big fat warning, but not fatal to parse. Now you cannot
>         even build
>         > eglibc if there is libdrm bbappend you don't care at all
>         about..
>         
>         
>         You can do this by setting:
>         
>         BB_DANGLINGAPPENDS_WARNONLY
> 
> 
> This is even worse; you end up with a package without the changes done
> on the bbappend and as most bbappend files do not change PR, adding it
> later won't force a package update.

Which is why its off by default. My point is you can do with Martin is
suggesting, its just not without its drawbacks.

Cheers,

Richard

Re: Coordinating inter-layer dependencies

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 1559 bytes --]

On Thu, Dec 01, 2011 at 01:02:38PM +0000, Richard Purdie wrote:
> On Thu, 2011-12-01 at 10:59 -0200, Otavio Salvador wrote:
> > On Thu, Dec 1, 2011 at 10:37, Richard Purdie
> > <richard.purdie@linuxfoundation.org> wrote:
> >         On Thu, 2011-12-01 at 13:24 +0100, Martin Jansa wrote:
> >         > A while back I've proposed to make .bbappend without
> >         corresponding .bb
> >         > only big fat warning, but not fatal to parse. Now you cannot
> >         even build
> >         > eglibc if there is libdrm bbappend you don't care at all
> >         about..
> >         
> >         
> >         You can do this by setting:
> >         
> >         BB_DANGLINGAPPENDS_WARNONLY

Good to know, thanks.

> > This is even worse; you end up with a package without the changes done
> > on the bbappend and as most bbappend files do not change PR, adding it
> > later won't force a package update.
> 
> Which is why its off by default. My point is you can do with Martin is
> suggesting, its just not without its drawbacks.

I think the main advantage of this is that you're allowed to build stuff
which doesn't use those dangling appends. Ie start build of eglibc if
you know that nothing is bbappending to eglibc and to its dependency
tree. And when .bbaappends are fixed you can disable
BB_DANGLINGAPPENDS_WARNONLY and build the rest.

But waiting for _all_ recipes in _all_ layers to get their .bbappends
right can sometimes a bit long..

Cheers,

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 205 bytes --]

Re: Coordinating inter-layer dependencies

[Chris Larson]

On Thu, Dec 1, 2011 at 6:13 AM, Martin Jansa <martin.jansa@gmail.com> wrote:
> On Thu, Dec 01, 2011 at 01:02:38PM +0000, Richard Purdie wrote:
>> On Thu, 2011-12-01 at 10:59 -0200, Otavio Salvador wrote:
>> > On Thu, Dec 1, 2011 at 10:37, Richard Purdie
>> > <richard.purdie@linuxfoundation.org> wrote:
>> >         On Thu, 2011-12-01 at 13:24 +0100, Martin Jansa wrote:
>> >         > A while back I've proposed to make .bbappend without
>> >         corresponding .bb
>> >         > only big fat warning, but not fatal to parse. Now you cannot
>> >         even build
>> >         > eglibc if there is libdrm bbappend you don't care at all
>> >         about..
>> >
>> >
>> >         You can do this by setting:
>> >
>> >         BB_DANGLINGAPPENDS_WARNONLY
>
> Good to know, thanks.
>
>> > This is even worse; you end up with a package without the changes done
>> > on the bbappend and as most bbappend files do not change PR, adding it
>> > later won't force a package update.

People use bbappends without changing PR? I've been doing a PR .= ".1"
or whatever in every bbappend I've created since they were added to
bitbake, and used to do the same with amend.inc. It can result in some
long PR values, but I'd rather that than the alternative..
-- 
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics

Re: Coordinating inter-layer dependencies

[McClintock Matthew-B29882]

On Thu, Dec 1, 2011 at 8:33 AM, Chris Larson <clarson@kergoth.com> wrote:
>>> > This is even worse; you end up with a package without the changes done
>>> > on the bbappend and as most bbappend files do not change PR, adding it
>>> > later won't force a package update.
>
> People use bbappends without changing PR? I've been doing a PR .= ".1"
> or whatever in every bbappend I've created since they were added to
> bitbake, and used to do the same with amend.inc. It can result in some
> long PR values, but I'd rather that than the alternative..

It would be nice to have something that WARNS or FATALS if there is no
PR modification in the bbappend. This should be required IMO but I
guess there could be some other issues that this would cause.

-M

Re: Coordinating inter-layer dependencies

[Koen Kooi]

[-- Attachment #1: Type: text/plain, Size: 2058 bytes --]


Op 1 dec. 2011, om 14:13 heeft Martin Jansa het volgende geschreven:

> On Thu, Dec 01, 2011 at 01:02:38PM +0000, Richard Purdie wrote:
>> On Thu, 2011-12-01 at 10:59 -0200, Otavio Salvador wrote:
>>> On Thu, Dec 1, 2011 at 10:37, Richard Purdie
>>> <richard.purdie@linuxfoundation.org> wrote:
>>>        On Thu, 2011-12-01 at 13:24 +0100, Martin Jansa wrote:
>>>> A while back I've proposed to make .bbappend without
>>>        corresponding .bb
>>>> only big fat warning, but not fatal to parse. Now you cannot
>>>        even build
>>>> eglibc if there is libdrm bbappend you don't care at all
>>>        about..
>>> 
>>> 
>>>        You can do this by setting:
>>> 
>>>        BB_DANGLINGAPPENDS_WARNONLY
> 
> Good to know, thanks.
> 
>>> This is even worse; you end up with a package without the changes done
>>> on the bbappend and as most bbappend files do not change PR, adding it
>>> later won't force a package update.
>> 
>> Which is why its off by default. My point is you can do with Martin is
>> suggesting, its just not without its drawbacks.
> 
> I think the main advantage of this is that you're allowed to build stuff
> which doesn't use those dangling appends. Ie start build of eglibc if
> you know that nothing is bbappending to eglibc and to its dependency
> tree. And when .bbaappends are fixed you can disable
> BB_DANGLINGAPPENDS_WARNONLY and build the rest.
> 
> But waiting for _all_ recipes in _all_ layers to get their .bbappends
> right can sometimes a bit long..

Which is why I sent this proposal, to give slow layers like meta-intel time to fix their stuff without breaking everyones build for 2 days till RP gets fed up and fixes it himself.
I don't have the time to maintain forks of every layer like you do with SHR and frankly speaking, it shouldn't be needed. I understand that things like review cycles take some time which is why the proposal tries to workaround the delays in layers in OE-core itself instead of angrily demanding maintainers to act quicker.

regards,

Koen

[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 169 bytes --]

Re: Coordinating inter-layer dependencies

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 3228 bytes --]

On Thu, Dec 01, 2011 at 04:36:40PM +0100, Koen Kooi wrote:
> 
> Op 1 dec. 2011, om 14:13 heeft Martin Jansa het volgende geschreven:
> 
> > On Thu, Dec 01, 2011 at 01:02:38PM +0000, Richard Purdie wrote:
> >> On Thu, 2011-12-01 at 10:59 -0200, Otavio Salvador wrote:
> >>> On Thu, Dec 1, 2011 at 10:37, Richard Purdie
> >>> <richard.purdie@linuxfoundation.org> wrote:
> >>>        On Thu, 2011-12-01 at 13:24 +0100, Martin Jansa wrote:
> >>>> A while back I've proposed to make .bbappend without
> >>>        corresponding .bb
> >>>> only big fat warning, but not fatal to parse. Now you cannot
> >>>        even build
> >>>> eglibc if there is libdrm bbappend you don't care at all
> >>>        about..
> >>> 
> >>> 
> >>>        You can do this by setting:
> >>> 
> >>>        BB_DANGLINGAPPENDS_WARNONLY
> > 
> > Good to know, thanks.
> > 
> >>> This is even worse; you end up with a package without the changes done
> >>> on the bbappend and as most bbappend files do not change PR, adding it
> >>> later won't force a package update.
> >> 
> >> Which is why its off by default. My point is you can do with Martin is
> >> suggesting, its just not without its drawbacks.
> > 
> > I think the main advantage of this is that you're allowed to build stuff
> > which doesn't use those dangling appends. Ie start build of eglibc if
> > you know that nothing is bbappending to eglibc and to its dependency
> > tree. And when .bbaappends are fixed you can disable
> > BB_DANGLINGAPPENDS_WARNONLY and build the rest.
> > 
> > But waiting for _all_ recipes in _all_ layers to get their .bbappends
> > right can sometimes a bit long..
> 
> Which is why I sent this proposal, to give slow layers like meta-intel time to fix their stuff without breaking everyones build for 2 days till RP gets fed up and fixes it himself.
> I don't have the time to maintain forks of every layer like you do with SHR and frankly speaking, it shouldn't be needed. I understand that things like review cycles take some time which is why the proposal tries to workaround the delays in layers in OE-core itself instead of angrily demanding maintainers to act quicker.

But the problem is that we cannot even push newer .bbappend in advance,
I would be happy to push libdrm-2.4.27.bbappend to master branch if it
doesn't break my builds which were still on 2.4.26.

Would be nice to be able to push danglings bbappends for stuff which is
only sitting on ML for review just in case I'll be at daywork or on
holidays or whatever when it gets applied to ie oe-core and someone just
hits update button..

I think the problem is not with *big* layers like oe-core and meta-oe
where is only 1 main maintainer but at least having full time job
related to maintaining it. But to maintain some hobbyist or community
layer in general in free time is sometimes pretty demanding just to stay
compatible with the rest of world (not breaking the rest of world if
they just want some BSP layer available from it). I wouldn't be
surprised if meta-smartphone BSP layers get disabled in layerman next
time I leave for month long holiday...

Just my 2c

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 205 bytes --]

Re: Coordinating inter-layer dependencies

[Koen Kooi]

[-- Attachment #1: Type: text/plain, Size: 3941 bytes --]


Op 1 dec. 2011, om 16:56 heeft Martin Jansa het volgende geschreven:

> On Thu, Dec 01, 2011 at 04:36:40PM +0100, Koen Kooi wrote:
>> 
>> Op 1 dec. 2011, om 14:13 heeft Martin Jansa het volgende geschreven:
>> 
>>> On Thu, Dec 01, 2011 at 01:02:38PM +0000, Richard Purdie wrote:
>>>> On Thu, 2011-12-01 at 10:59 -0200, Otavio Salvador wrote:
>>>>> On Thu, Dec 1, 2011 at 10:37, Richard Purdie
>>>>> <richard.purdie@linuxfoundation.org> wrote:
>>>>>       On Thu, 2011-12-01 at 13:24 +0100, Martin Jansa wrote:
>>>>>> A while back I've proposed to make .bbappend without
>>>>>       corresponding .bb
>>>>>> only big fat warning, but not fatal to parse. Now you cannot
>>>>>       even build
>>>>>> eglibc if there is libdrm bbappend you don't care at all
>>>>>       about..
>>>>> 
>>>>> 
>>>>>       You can do this by setting:
>>>>> 
>>>>>       BB_DANGLINGAPPENDS_WARNONLY
>>> 
>>> Good to know, thanks.
>>> 
>>>>> This is even worse; you end up with a package without the changes done
>>>>> on the bbappend and as most bbappend files do not change PR, adding it
>>>>> later won't force a package update.
>>>> 
>>>> Which is why its off by default. My point is you can do with Martin is
>>>> suggesting, its just not without its drawbacks.
>>> 
>>> I think the main advantage of this is that you're allowed to build stuff
>>> which doesn't use those dangling appends. Ie start build of eglibc if
>>> you know that nothing is bbappending to eglibc and to its dependency
>>> tree. And when .bbaappends are fixed you can disable
>>> BB_DANGLINGAPPENDS_WARNONLY and build the rest.
>>> 
>>> But waiting for _all_ recipes in _all_ layers to get their .bbappends
>>> right can sometimes a bit long..
>> 
>> Which is why I sent this proposal, to give slow layers like meta-intel time to fix their stuff without breaking everyones build for 2 days till RP gets fed up and fixes it himself.
>> I don't have the time to maintain forks of every layer like you do with SHR and frankly speaking, it shouldn't be needed. I understand that things like review cycles take some time which is why the proposal tries to workaround the delays in layers in OE-core itself instead of angrily demanding maintainers to act quicker.
> 
> But the problem is that we cannot even push newer .bbappend in advance,
> I would be happy to push libdrm-2.4.27.bbappend to master branch if it
> doesn't break my builds which were still on 2.4.26.

I thnk that either

a) becomes a matter of PREFERRED_VERSION of your distro
b) DEFAULT_PREFERENCE = -1 for the update in the grace period.

> Would be nice to be able to push danglings bbappends for stuff which is
> only sitting on ML for review just in case I'll be at daywork or on
> holidays or whatever when it gets applied to ie oe-core and someone just
> hits update button..

Which is where the grace period comes in :)

> I think the problem is not with *big* layers like oe-core and meta-oe
> where is only 1 main maintainer but at least having full time job
> related to maintaining it. But to maintain some hobbyist or community
> layer in general in free time is sometimes pretty demanding just to stay
> compatible with the rest of world (not breaking the rest of world if
> they just want some BSP layer available from it). I wouldn't be
> surprised if meta-smartphone BSP layers get disabled in layerman next
> time I leave for month long holiday...

That's an unfortunate side effect of having a low busfactor. I'll put that in the "shit happens" bin. The proposal won't fix every breakage, but it would make all our lives a lot easier for most upgrades.

Maybe after a while we'll realize that tracking OE-core is too painfull with a full set of slow layers, but I'd like to try this proposal first before giving up[1].

regards,

Koen

[1] Translation: I'm too lazy to add an automated 'lock layer' button to layerman right now 

[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 169 bytes --]

Re: Coordinating inter-layer dependencies

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 2706 bytes --]

On Thu, Dec 01, 2011 at 10:59:03AM -0200, Otavio Salvador wrote:
> On Thu, Dec 1, 2011 at 10:37, Richard Purdie <
> richard.purdie@linuxfoundation.org> wrote:
> 
> > On Thu, 2011-12-01 at 13:24 +0100, Martin Jansa wrote:
> > > A while back I've proposed to make .bbappend without corresponding .bb
> > > only big fat warning, but not fatal to parse. Now you cannot even build
> > > eglibc if there is libdrm bbappend you don't care at all about..
> >
> > You can do this by setting:
> >
> > BB_DANGLINGAPPENDS_WARNONLY
> >
> 
> This is even worse; you end up with a package without the changes done on
> the bbappend and as most bbappend files do not change PR, adding it later
> won't force a package update.

If we find a way to allow PRINC in multiple bbappends for same .bb then
we can say that every .bbappend should use PRINC.

For record I'll include my discussion about PRINC with RP and kergoth:
10:47 < JaMa> RP__: is there any way to improve PRINC concept to allow multiple increments for same recipe while parsing multiple layers?
10:48 < RP__> JaMa: PRINC_append = ".1" ?
10:49 < JaMa> RP__: ie when meta-openmoko sets PRINC = "1" and meta-shr sets PRINC = "2" then if you're unlucky meta-openmoko is parsed later and bumping PRINC in meta-shr won't help
10:49 < RP__> JaMa: I wonder if you could do PRINC := "${PRINC + 1}"
10:50 < JaMa> and do we have default PRINC = 0 somewhere?
10:50 < RP__> JaMa: you might need to add that
10:50 < JaMa> ok, I'll try this, thanks
10:51 < JaMa> currently I'm moving PRINC only to meta-shr layer.. but that breaks stuff if someone is using any BSP layer from meta-smartphone..

14:53 < JaMa> RP__: btw that PRINC trick didn't work (int type didn't like expresion :/)
15:13 < RP__> JaMa: ah, try PRINC := "${int(PRINC) + 1}"
15:21 < JaMa> RP__: still ValueError: invalid literal for int() with base 10: '${int(PRINC) + 1}'
15:21 < JaMa> with added PRINC := 0 to bitbake.conf
15:22 < RP__> PRINC := "${int(d.getVar(PRINC)) + 1}" ? :/
15:22 < JaMa> whole log http://paste.pocoo.org/show/514437/
15:22  * RP__ was trying to be too clever I suspect
15:23 < JaMa> ValueError: invalid literal for int() with base 10: '${int(d.getVar(PRINC)) + 1}'
15:41 < kergoth> PRINC is unquoted there, so it tries to get a value for a key of None
16:24 < RP__> kergoth: right, trying to do too many things at once :/
16:24 < RP__> kergoth: any thoughts on that knotty change to add the footer?
17:05 < JaMa> kergoth: something like this? ValueError: invalid literal for int() with base 10: "${int(d.getVar('PRINC')) + 1}"

Maybe someone else has better idea?

Cheers,
-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 205 bytes --]

Re: Coordinating inter-layer dependencies

[Richard Purdie]

On Thu, 2011-12-01 at 14:07 +0100, Martin Jansa wrote:
> On Thu, Dec 01, 2011 at 10:59:03AM -0200, Otavio Salvador wrote:
> > On Thu, Dec 1, 2011 at 10:37, Richard Purdie <
> > richard.purdie@linuxfoundation.org> wrote:
> > 
> > > On Thu, 2011-12-01 at 13:24 +0100, Martin Jansa wrote:
> > > > A while back I've proposed to make .bbappend without corresponding .bb
> > > > only big fat warning, but not fatal to parse. Now you cannot even build
> > > > eglibc if there is libdrm bbappend you don't care at all about..
> > >
> > > You can do this by setting:
> > >
> > > BB_DANGLINGAPPENDS_WARNONLY
> > >
> > 
> > This is even worse; you end up with a package without the changes done on
> > the bbappend and as most bbappend files do not change PR, adding it later
> > won't force a package update.
> 
> If we find a way to allow PRINC in multiple bbappends for same .bb then
> we can say that every .bbappend should use PRINC.
> 
> For record I'll include my discussion about PRINC with RP and kergoth:
> 10:47 < JaMa> RP__: is there any way to improve PRINC concept to allow multiple increments for same recipe while parsing multiple layers?
> 10:48 < RP__> JaMa: PRINC_append = ".1" ?
> 10:49 < JaMa> RP__: ie when meta-openmoko sets PRINC = "1" and meta-shr sets PRINC = "2" then if you're unlucky meta-openmoko is parsed later and bumping PRINC in meta-shr won't help
> 10:49 < RP__> JaMa: I wonder if you could do PRINC := "${PRINC + 1}"
> 10:50 < JaMa> and do we have default PRINC = 0 somewhere?
> 10:50 < RP__> JaMa: you might need to add that
> 10:50 < JaMa> ok, I'll try this, thanks
> 10:51 < JaMa> currently I'm moving PRINC only to meta-shr layer.. but that breaks stuff if someone is using any BSP layer from meta-smartphone..
> 
> 14:53 < JaMa> RP__: btw that PRINC trick didn't work (int type didn't like expresion :/)
> 15:13 < RP__> JaMa: ah, try PRINC := "${int(PRINC) + 1}"
> 15:21 < JaMa> RP__: still ValueError: invalid literal for int() with base 10: '${int(PRINC) + 1}'
> 15:21 < JaMa> with added PRINC := 0 to bitbake.conf
> 15:22 < RP__> PRINC := "${int(d.getVar(PRINC)) + 1}" ? :/
> 15:22 < JaMa> whole log http://paste.pocoo.org/show/514437/
> 15:22  * RP__ was trying to be too clever I suspect
> 15:23 < JaMa> ValueError: invalid literal for int() with base 10: '${int(d.getVar(PRINC)) + 1}'
> 15:41 < kergoth> PRINC is unquoted there, so it tries to get a value for a key of None
> 16:24 < RP__> kergoth: right, trying to do too many things at once :/
> 16:24 < RP__> kergoth: any thoughts on that knotty change to add the footer?
> 17:05 < JaMa> kergoth: something like this? ValueError: invalid literal for int() with base 10: "${int(d.getVar('PRINC')) + 1}"
> 
> Maybe someone else has better idea?

Looking at that I was missing something obvious. Try:

PRINC := "${@int(PRINC) + 1}"

Cheers,

Richard

Re: Coordinating inter-layer dependencies

[Mats Kärrman]

> From: openembedded-core-bounces@lists.openembedded.org [openembedded-core-bounces@lists.openembedded.org] on behalf of Richard Purdie [richard.purdie@linuxfoundation.org]
> Sent: Thursday, December 01, 2011 9:33 PM
> To: Patches and discussions about the oe-core layer
> Subject: Re: [OE-core] Coordinating inter-layer dependencies
> 
> On Thu, 2011-12-01 at 14:07 +0100, Martin Jansa wrote:
> > On Thu, Dec 01, 2011 at 10:59:03AM -0200, Otavio Salvador wrote:
> > > On Thu, Dec 1, 2011 at 10:37, Richard Purdie <
> > > richard.purdie@linuxfoundation.org> wrote:
> > >
> > > > On Thu, 2011-12-01 at 13:24 +0100, Martin Jansa wrote:
> > > > > A while back I've proposed to make .bbappend without corresponding .bb
> > > > > only big fat warning, but not fatal to parse. Now you cannot even build
> > > > > eglibc if there is libdrm bbappend you don't care at all about..
> > > >
> > > > You can do this by setting:
> > > >
> > > > BB_DANGLINGAPPENDS_WARNONLY
> > > >
> > >
> > > This is even worse; you end up with a package without the changes done on
> > > the bbappend and as most bbappend files do not change PR, adding it later
> > > won't force a package update.
> >
> > If we find a way to allow PRINC in multiple bbappends for same .bb then
> > we can say that every .bbappend should use PRINC.
> >
> > For record I'll include my discussion about PRINC with RP and kergoth:
> > 10:47 < JaMa> RP__: is there any way to improve PRINC concept to allow multiple increments for same recipe while parsing multiple layers?
> > 10:48 < RP__> JaMa: PRINC_append = ".1" ?
> > 10:49 < JaMa> RP__: ie when meta-openmoko sets PRINC = "1" and meta-shr sets PRINC = "2" then if you're unlucky meta-openmoko is parsed later and bumping PRINC in meta-shr won't help
> > 10:49 < RP__> JaMa: I wonder if you could do PRINC := "${PRINC + 1}"
> > 10:50 < JaMa> and do we have default PRINC = 0 somewhere?
> > 10:50 < RP__> JaMa: you might need to add that
> > 10:50 < JaMa> ok, I'll try this, thanks
> > 10:51 < JaMa> currently I'm moving PRINC only to meta-shr layer.. but that breaks stuff if someone is using any BSP layer from meta-smartphone..
> >
> > 14:53 < JaMa> RP__: btw that PRINC trick didn't work (int type didn't like expresion :/)
> > 15:13 < RP__> JaMa: ah, try PRINC := "${int(PRINC) + 1}"
> > 15:21 < JaMa> RP__: still ValueError: invalid literal for int() with base 10: '${int(PRINC) + 1}'
> > 15:21 < JaMa> with added PRINC := 0 to bitbake.conf
> > 15:22 < RP__> PRINC := "${int(d.getVar(PRINC)) + 1}" ? :/
> > 15:22 < JaMa> whole log http://paste.pocoo.org/show/514437/
> > 15:22  * RP__ was trying to be too clever I suspect
> > 15:23 < JaMa> ValueError: invalid literal for int() with base 10: '${int(d.getVar(PRINC)) + 1}'
> > 15:41 < kergoth> PRINC is unquoted there, so it tries to get a value for a key of None
> > 16:24 < RP__> kergoth: right, trying to do too many things at once :/
> > 16:24 < RP__> kergoth: any thoughts on that knotty change to add the footer?
> > 17:05 < JaMa> kergoth: something like this? ValueError: invalid literal for int() with base 10: "${int(d.getVar('PRINC')) + 1}"
> >
> > Maybe someone else has better idea?
> 
> Looking at that I was missing something obvious. Try:
> 
> PRINC := "${@int(PRINC) + 1}"
> 
> Cheers,
> 
> Richard
> 

As I understand it, the PRINC increases the PR of the appended recipe. Then the resulting package will have a version that is the same as "the next" version of the base recipe. Since many people (me included) hardly remember what they did a month ago, isn't there a big risk of confusion in your deployed systems if packages are upgraded there?

I do something like  PR .= ".local0"  where "local" is the name of the layer containing the .bbappend.
Of course this will create extensive version strings if multiple .bbappends are applied but the alternative is that you have no idea what layers that appended and what the the base version of the recipe really was.

// Mats

Re: Coordinating inter-layer dependencies

[Frans Meulenbroeks]

[-- Attachment #1: Type: text/plain, Size: 1085 bytes --]

Ideally a specific revision of a layer should specify on which revision it
depends on/was tested with, so people can reproduce that same setup.

Randomly mixing revisions is a recipe for problems.
Expecting that a layer informs its users that something is changing is
probably not workable. How would a layer maintainer know what layers are
actually depending on it (especially for things like oe-core)?

If one is using the head of a layer one is living on the bleeding edge (and
hence sometimes one bleeds) (and doing production work based on that seems
unwise).

Then again, I seem to recall that somewhere in the spring we agreed on a
deprecation policy where "high impact changes" like for toolchain would be
done like:
- commit new version
- wait a while (2 weeks?)
- remove old version

thus giving people a chance to migrate.

Frans.

(PS: of course it is also possible to proceed in a perfectly synchronised
lockstep, but that probably requires some central coordination and some
staging trees or so to test against before the lockstep is performed)

[-- Attachment #2: Type: text/html, Size: 1127 bytes --]

Re: Coordinating inter-layer dependencies

[Otavio Salvador]

[-- Attachment #1: Type: text/plain, Size: 821 bytes --]

On Fri, Dec 2, 2011 at 11:34, Frans Meulenbroeks <
fransmeulenbroeks@gmail.com> wrote:

> Ideally a specific revision of a layer should specify on which revision it
> depends on/was tested with, so people can reproduce that same setup.
>

I've been using combo-layer to this to work here at O.S. Systems and this
has been very easy to handle.

I made an internal repository that has oe-core and the layers I need so
co-workers can get it and trust it will work; the person doing the update
on the layers is expected to do a build before pushing it onto our internal
repository and thus this works very well.

-- 
Otavio Salvador                             O.S. Systems
E-mail: otavio@ossystems.com.br  http://www.ossystems.com.br
Mobile: +55 53 9981-7854              http://projetos.ossystems.com.br

[-- Attachment #2: Type: text/html, Size: 1296 bytes --]

Re: Coordinating inter-layer dependencies

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 5457 bytes --]

On Fri, Dec 02, 2011 at 08:46:43AM +0000, Mats Kärrman wrote:
> > From: openembedded-core-bounces@lists.openembedded.org [openembedded-core-bounces@lists.openembedded.org] on behalf of Richard Purdie [richard.purdie@linuxfoundation.org]
> > Sent: Thursday, December 01, 2011 9:33 PM
> > To: Patches and discussions about the oe-core layer
> > Subject: Re: [OE-core] Coordinating inter-layer dependencies
> > 
> > On Thu, 2011-12-01 at 14:07 +0100, Martin Jansa wrote:
> > > On Thu, Dec 01, 2011 at 10:59:03AM -0200, Otavio Salvador wrote:
> > > > On Thu, Dec 1, 2011 at 10:37, Richard Purdie <
> > > > richard.purdie@linuxfoundation.org> wrote:
> > > >
> > > > > On Thu, 2011-12-01 at 13:24 +0100, Martin Jansa wrote:
> > > > > > A while back I've proposed to make .bbappend without corresponding .bb
> > > > > > only big fat warning, but not fatal to parse. Now you cannot even build
> > > > > > eglibc if there is libdrm bbappend you don't care at all about..
> > > > >
> > > > > You can do this by setting:
> > > > >
> > > > > BB_DANGLINGAPPENDS_WARNONLY
> > > > >
> > > >
> > > > This is even worse; you end up with a package without the changes done on
> > > > the bbappend and as most bbappend files do not change PR, adding it later
> > > > won't force a package update.
> > >
> > > If we find a way to allow PRINC in multiple bbappends for same .bb then
> > > we can say that every .bbappend should use PRINC.
> > >
> > > For record I'll include my discussion about PRINC with RP and kergoth:
> > > 10:47 < JaMa> RP__: is there any way to improve PRINC concept to allow multiple increments for same recipe while parsing multiple layers?
> > > 10:48 < RP__> JaMa: PRINC_append = ".1" ?
> > > 10:49 < JaMa> RP__: ie when meta-openmoko sets PRINC = "1" and meta-shr sets PRINC = "2" then if you're unlucky meta-openmoko is parsed later and bumping PRINC in meta-shr won't help
> > > 10:49 < RP__> JaMa: I wonder if you could do PRINC := "${PRINC + 1}"
> > > 10:50 < JaMa> and do we have default PRINC = 0 somewhere?
> > > 10:50 < RP__> JaMa: you might need to add that
> > > 10:50 < JaMa> ok, I'll try this, thanks
> > > 10:51 < JaMa> currently I'm moving PRINC only to meta-shr layer.. but that breaks stuff if someone is using any BSP layer from meta-smartphone..
> > >
> > > 14:53 < JaMa> RP__: btw that PRINC trick didn't work (int type didn't like expresion :/)
> > > 15:13 < RP__> JaMa: ah, try PRINC := "${int(PRINC) + 1}"
> > > 15:21 < JaMa> RP__: still ValueError: invalid literal for int() with base 10: '${int(PRINC) + 1}'
> > > 15:21 < JaMa> with added PRINC := 0 to bitbake.conf
> > > 15:22 < RP__> PRINC := "${int(d.getVar(PRINC)) + 1}" ? :/
> > > 15:22 < JaMa> whole log http://paste.pocoo.org/show/514437/
> > > 15:22  * RP__ was trying to be too clever I suspect
> > > 15:23 < JaMa> ValueError: invalid literal for int() with base 10: '${int(d.getVar(PRINC)) + 1}'
> > > 15:41 < kergoth> PRINC is unquoted there, so it tries to get a value for a key of None
> > > 16:24 < RP__> kergoth: right, trying to do too many things at once :/
> > > 16:24 < RP__> kergoth: any thoughts on that knotty change to add the footer?
> > > 17:05 < JaMa> kergoth: something like this? ValueError: invalid literal for int() with base 10: "${int(d.getVar('PRINC')) + 1}"
> > >
> > > Maybe someone else has better idea?
> > 
> > Looking at that I was missing something obvious. Try:
> > 
> > PRINC := "${@int(PRINC) + 1}"
> > 
> > Cheers,
> > 
> > Richard
> > 
> 
> As I understand it, the PRINC increases the PR of the appended recipe. Then the resulting package will have a version that is the same as "the next" version of the base recipe. Since many people (me included) hardly remember what they did a month ago, isn't there a big risk of confusion in your deployed systems if packages are upgraded there?
> 
> I do something like  PR .= ".local0"  where "local" is the name of the layer containing the .bbappend.
> Of course this will create extensive version strings if multiple .bbappends are applied but the alternative is that you have no idea what layers that appended and what the the base version of the recipe really was.

So if you have some layer adding
layerB: PR .= ".b0"
and other
layerA: PR .= ".a0"

and PR appends are evaluated in this order, then you will break upgrade
patch forever if you have to remove layerB for some reason and you
cannot even fix it by bumping numbers in your layerA unless you rename
all PR appends as ".c0" right? That's why I prefer to use PRINC.

Yes it increases PR of desired number, but every time.. so if upper
layer increments PR your .bbappend will again increment it so you still
get always increasing sequence.

Only problem was multiple .bbappends overwritting PRINC value in each
other, but RP's latest proposal:
PRINC := "${@int(PRINC) + 1}"
works and I'm going to push meta-smartphone migration to this PRINC
usage instead of more common 'PRINC = "1"'

for bb in `git grep "^PRINC" . | sed 's/:.*//g'`; do sed -i 's/PRINC *= *"\(.*\)"/PRINC := "${@int(PRINC) + \1}"/g' $bb; done

And I'll send patch here to add default PRINC = "0" to bitbake.conf,
because otherwise it fails with:
ERROR: Failure expanding variable PRINC[:=], expression was
${@int(PRINC) + 1} which triggered exception NameError: name 'PRINC' is
not defined

Cheers,

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 205 bytes --]

[PATCH] bitbake.conf: add default PRINC 0 to be able to increment it

[Martin Jansa]

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
---
 meta/conf/bitbake.conf |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index 87efd8e..552942b 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -166,6 +166,7 @@ ASSUME_PROVIDED = "\
 PN = "${@bb.parse.BBHandler.vars_from_file(d.getVar('FILE'),d)[0] or 'defaultpkgname'}"
 PV = "${@bb.parse.BBHandler.vars_from_file(d.getVar('FILE'),d)[1] or '1.0'}"
 PR = "${@bb.parse.BBHandler.vars_from_file(d.getVar('FILE'),d)[2] or 'r0'}"
+PRINC := "0"
 PF = "${PN}-${EXTENDPE}${PV}-${PR}"
 EXTENDPE = "${@['','${PE\x7d_'][d.getVar('PE',1) > 0]}"
 P = "${PN}-${PV}"
-- 
1.7.8.rc4

Re: [PATCH] bitbake.conf: add default PRINC 0 to be able to increment it

[Richard Purdie]

On Fri, 2011-12-02 at 17:22 +0100, Martin Jansa wrote:
> Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
> ---
>  meta/conf/bitbake.conf |    1 +
>  1 files changed, 1 insertions(+), 0 deletions(-)
> 
> diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
> index 87efd8e..552942b 100644
> --- a/meta/conf/bitbake.conf
> +++ b/meta/conf/bitbake.conf
> @@ -166,6 +166,7 @@ ASSUME_PROVIDED = "\
>  PN = "${@bb.parse.BBHandler.vars_from_file(d.getVar('FILE'),d)[0] or 'defaultpkgname'}"
>  PV = "${@bb.parse.BBHandler.vars_from_file(d.getVar('FILE'),d)[1] or '1.0'}"
>  PR = "${@bb.parse.BBHandler.vars_from_file(d.getVar('FILE'),d)[2] or 'r0'}"
> +PRINC := "0"
>  PF = "${PN}-${EXTENDPE}${PV}-${PR}"
>  EXTENDPE = "${@['','${PE\x7d_'][d.getVar('PE',1) > 0]}"
>  P = "${PN}-${PV}"

Why does this need := ? (it doesn't)

Also, can you change the expression in base.bbclass from:

    princ = d.getVar('PRINC', True)
    if princ:

to:
    if princ and princ != "0":

since otherwise I think this will have rather a negative effect on
parsing speed.

Cheers,

Richard

[PATCHv2] bitbake.conf: add default PRINC 0 to be able to increment it

[Martin Jansa]

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
---
 meta/classes/base.bbclass |    2 +-
 meta/conf/bitbake.conf    |    1 +
 2 files changed, 2 insertions(+), 1 deletions(-)

diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
index ea53498..fbcaefb 100644
--- a/meta/classes/base.bbclass
+++ b/meta/classes/base.bbclass
@@ -335,7 +335,7 @@ python () {
 
     # If PRINC is set, try and increase the PR value by the amount specified
     princ = d.getVar('PRINC', True)
-    if princ:
+    if princ and princ != "0":
         pr = d.getVar('PR', True)
         pr_prefix = re.search("\D+",pr)
         prval = re.search("\d+",pr)
diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index acba388..e80cc32 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -167,6 +167,7 @@ ASSUME_PROVIDED = "\
 PN = "${@bb.parse.BBHandler.vars_from_file(d.getVar('FILE'),d)[0] or 'defaultpkgname'}"
 PV = "${@bb.parse.BBHandler.vars_from_file(d.getVar('FILE'),d)[1] or '1.0'}"
 PR = "${@bb.parse.BBHandler.vars_from_file(d.getVar('FILE'),d)[2] or 'r0'}"
+PRINC ?= "0"
 PF = "${PN}-${EXTENDPE}${PV}-${PR}"
 EXTENDPE = "${@['','${PE\x7d_'][d.getVar('PE',1) > 0]}"
 P = "${PN}-${PV}"
-- 
1.7.8.rc4

Re: [PATCHv2] bitbake.conf: add default PRINC 0 to be able to increment it

[Chris Larson]

On Fri, Dec 2, 2011 at 11:39 AM, Martin Jansa <martin.jansa@gmail.com> wrote:
> Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
> ---
>  meta/classes/base.bbclass |    2 +-
>  meta/conf/bitbake.conf    |    1 +
>  2 files changed, 2 insertions(+), 1 deletions(-)
>
> diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
> index ea53498..fbcaefb 100644
> --- a/meta/classes/base.bbclass
> +++ b/meta/classes/base.bbclass
> @@ -335,7 +335,7 @@ python () {
>
>     # If PRINC is set, try and increase the PR value by the amount specified
>     princ = d.getVar('PRINC', True)
> -    if princ:
> +    if princ and princ != "0":

Could also do:

  if int(princ):

Or:

  PRINC[type] = "integer" # and:

  if oe.data.typed_value('PRINC', d):
-- 
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics

Re: [PATCHv2] bitbake.conf: add default PRINC 0 to be able to increment it

[Richard Purdie]

On Fri, 2011-12-02 at 19:39 +0100, Martin Jansa wrote:
> Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
> ---
>  meta/classes/base.bbclass |    2 +-
>  meta/conf/bitbake.conf    |    1 +
>  2 files changed, 2 insertions(+), 1 deletions(-)

Merged to master, thanks.

Richard

Re: Coordinating inter-layer dependencies

[Chris Larson]

On Fri, Dec 2, 2011 at 9:18 AM, Martin Jansa <martin.jansa@gmail.com> wrote:
> So if you have some layer adding
> layerB: PR .= ".b0"
> and other
> layerA: PR .= ".a0"
>
> and PR appends are evaluated in this order, then you will break upgrade
> patch forever if you have to remove layerB for some reason and you
> cannot even fix it by bumping numbers in your layerA unless you rename
> all PR appends as ".c0" right? That's why I prefer to use PRINC.

And if you *remove* a layer you used to use? Will this not result in
it decrementing and breaking the upgrade path in that way? Or does
this persist the way localcount does?
-- 
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics

Re: Coordinating inter-layer dependencies

[Mats Kärrman]

> From: openembedded-core-bounces@lists.openembedded.org [openembedded-core-bounces@lists.openembedded.org] on behalf of Martin Jansa [martin.jansa@gmail.com]
> Sent: Friday, December 02, 2011 5:18 PM
> To: Patches and discussions about the oe-core layer
> Subject: Re: [OE-core] Coordinating inter-layer dependencies
> 

...cut...

> > As I understand it, the PRINC increases the PR of the appended recipe. Then the resulting package will have a version that is the same as "the next" version of the base recipe. Since many people (me included) hardly remember what they did a month ago, isn't there a big risk of confusion in your deployed systems if packages are upgraded there?
> >
> > I do something like  PR .= ".local0"  where "local" is the name of the layer containing the .bbappend.
> > Of course this will create extensive version strings if multiple .bbappends are applied but the alternative is that you have no idea what layers that appended and what the the base version of the recipe really was.
> 
> So if you have some layer adding
> layerB: PR .= ".b0"
> and other
> layerA: PR .= ".a0"
> 
> and PR appends are evaluated in this order, then you will break upgrade
> patch forever if you have to remove layerB for some reason and you
> cannot even fix it by bumping numbers in your layerA unless you rename
> all PR appends as ".c0" right? That's why I prefer to use PRINC.

I don't really see what is breaking but I'm guessing that the problem you describe has to do with feed based upgrades and automatic evaluation of which version is latest (?). I don't do things that way but instead explicitly download and replace packages and then I prefer to be able to see exactly what was built. Of course you can achieve that also by some external procedure to make sure that you always no matter what produce increasing PR numbers on your packages but if you fail there is no telling... 

> 
> Yes it increases PR of desired number, but every time.. so if upper
> layer increments PR your .bbappend will again increment it so you still
> get always increasing sequence.

Unless, again, I remove a layer, then I'll have to extra-bump another "innocent" layer to keep the numbers up.


BR // Mats

Re: Coordinating inter-layer dependencies

[McClintock Matthew-B29882]

On Thu, Dec 1, 2011 at 5:23 AM, Koen Kooi <koen@dominion.thruhere.net> wrote:
> Hi,
>
> During the past month there have been a number of updates to OE-core recipes that triggered parsing errors due to bbappend in other layers. A small seleciton:
>
> * netbase
> * libdrm
> * xserver-xorg
> * clutter
>
> My view is that layer maintainers need to keep an eye on potential breakage and have updates ready when patches land into OE-core. Looking back I can see that while the situation is improving a bit, it's still not working. The problem with slow updates to layers is that (with my angstrom hat on) users (and with my TI hat on) customers and coworkers can't do builds without rm'ing the bbappends or disabling the layer.
>
> This is bad for a number of reasons:
>
> 1) It creates unmanaged local diffs
> 2) it can cause PR to fluctuate back and forth if you rm is a bit overzealous or if you disable the complete layer.
>
> My proposal is that OE-core recipe upgrades with known bbappends look like this:
>
> 1) add new version with a warning about bbappends
> 2) wait a N days (2 < N < 7)
> 3) delete old version
>
> To avoid stressing out RP and Sau! I would strongly urge layer maintainers to respond to recipe update patches with "I have a bbappend, but my review process needs time, please use the above proposal" if you need time to test and update  your bbappends.
>
> We're all human in the end and timezone differences aren't always helping with these kinds of problems, so let's work together to minimize parsing breakage.
>
> Related to that: Please include information on how to send patches upstream in your layer/repo README. The meta-intel/emenlow lacks that and the top level README doesn't mention that there are sublevel READMEs.

Why don't we make a version of bbappend that applies to all packages
with the same name? This would fix the constant changing
netbase_vXYZ.bbappend issue.

-M

Re: Coordinating inter-layer dependencies

[Chris Larson]

On Thu, Dec 1, 2011 at 12:55 PM, McClintock Matthew-B29882
<B29882@freescale.com> wrote:
>> To avoid stressing out RP and Sau! I would strongly urge layer maintainers to respond to recipe update patches with "I have a bbappend, but my review process needs time, please use the above proposal" if you need time to test and update  your bbappends.
>>
>> We're all human in the end and timezone differences aren't always helping with these kinds of problems, so let's work together to minimize parsing breakage.
>>
>> Related to that: Please include information on how to send patches upstream in your layer/repo README. The meta-intel/emenlow lacks that and the top level README doesn't mention that there are sublevel READMEs.
>
> Why don't we make a version of bbappend that applies to all packages
> with the same name? This would fix the constant changing
> netbase_vXYZ.bbappend issue.

I'm pretty sure going by filename was a conscious design decision in
bitbake, to keep things simple and avoid the need to parse up front
before being able to determine the appends to use. RP would know the
details.
-- 
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics

Re: Coordinating inter-layer dependencies

[Richard Purdie]

On Thu, 2011-12-01 at 12:23 +0100, Koen Kooi wrote:
> During the past month there have been a number of updates to OE-core
> recipes that triggered parsing errors due to bbappend in other layers.
> A small seleciton:
> 
> * netbase
> * libdrm
> * xserver-xorg
> * clutter
> 
> My view is that layer maintainers need to keep an eye on potential
> breakage and have updates ready when patches land into OE-core.
> Looking back I can see that while the situation is improving a bit,
> it's still not working. The problem with slow updates to layers is
> that (with my angstrom hat on) users (and with my TI hat on) customers
> and coworkers can't do builds without rm'ing the bbappends or
> disabling the layer.


I've been thinking about this. I wondered if the notion of wildcards
for .bbappend might help, e.g. a file called:

netbase_%.bbappend

would match all versions of the netbase recipe.

The % is consistent with PREFERRED_VERSION, I'm a little reluctant to
use * since its a shell quoting mess.

It wouldn't be hard to allow bitbake to support this and would solve
some of the issues we've been having...

Cheers,

Richard

Re: Coordinating inter-layer dependencies

[McClintock Matthew-B29882]

On Wed, Dec 7, 2011 at 6:17 PM, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> On Thu, 2011-12-01 at 12:23 +0100, Koen Kooi wrote:
>> During the past month there have been a number of updates to OE-core
>> recipes that triggered parsing errors due to bbappend in other layers.
>> A small seleciton:
>>
>> * netbase
>> * libdrm
>> * xserver-xorg
>> * clutter
>>
>> My view is that layer maintainers need to keep an eye on potential
>> breakage and have updates ready when patches land into OE-core.
>> Looking back I can see that while the situation is improving a bit,
>> it's still not working. The problem with slow updates to layers is
>> that (with my angstrom hat on) users (and with my TI hat on) customers
>> and coworkers can't do builds without rm'ing the bbappends or
>> disabling the layer.
>
>
> I've been thinking about this. I wondered if the notion of wildcards
> for .bbappend might help, e.g. a file called:
>
> netbase_%.bbappend
>
> would match all versions of the netbase recipe.
>
> The % is consistent with PREFERRED_VERSION, I'm a little reluctant to
> use * since its a shell quoting mess.
>
> It wouldn't be hard to allow bitbake to support this and would solve
> some of the issues we've been having...

For what it's worth, I like the wild card idea.

-M

Re: Coordinating inter-layer dependencies

[Andreas Müller]

On Thursday, December 08, 2011 01:17:37 AM Richard Purdie wrote:
> On Thu, 2011-12-01 at 12:23 +0100, Koen Kooi wrote:
> > During the past month there have been a number of updates to OE-core
> > recipes that triggered parsing errors due to bbappend in other layers.
> > A small seleciton:
> > 
> > * netbase
> > * libdrm
> > * xserver-xorg
> > * clutter
> > 
> > My view is that layer maintainers need to keep an eye on potential
> > breakage and have updates ready when patches land into OE-core.
> > Looking back I can see that while the situation is improving a bit,
> > it's still not working. The problem with slow updates to layers is
> > that (with my angstrom hat on) users (and with my TI hat on) customers
> > and coworkers can't do builds without rm'ing the bbappends or
> > disabling the layer.
> 
> I've been thinking about this. I wondered if the notion of wildcards
> for .bbappend might help, e.g. a file called:
> 
> netbase_%.bbappend
> 
> would match all versions of the netbase recipe.
> 
> The % is consistent with PREFERRED_VERSION, I'm a little reluctant to
> use * since its a shell quoting mess.
> 
> It wouldn't be hard to allow bitbake to support this and would solve
> some of the issues we've been having...
> 
If being used in sub layers parse errors (weekend breakers) would turn into

* no issue
* build error

which are both enhancing current situation.

Andreas