complex versioning scenario

complex versioning scenario

[Steffen Sledz]

We've a complex versioning scenario here which leads me to my limits. :(

There are two recipes. One for a shared library and one for an application using this library.

Both use GNU autotools (so they have internal version information). For continuous integration purposes both use AUTOREV.

At the moment the recipes look like this:


------------ libfoo_git.bb -------------
PR = "r7"
PE = "2"
SRCREV="${AUTOREV}"
PV = "gitr${SRCPV}"
...


------------ app_git.bb ----------------
DEPENDS = "... libfoo ..."
PR = "r10"
PE = "1"
SRCREV="${AUTOREV}"
PV = "gitr${SRCPV}"
...


Now we have the following problem. libfoo has some incompatible changes in its interface (a new internal major version).

In my opinion this should find its represenation in the package versioning in a way that the dependency checker can guarantee that the library and the application package match each other.

But how?

Regards,
Steffen

-- 
DResearch Fahrzeugelektronik GmbH
Otto-Schmirgal-Str. 3, 10319 Berlin, Germany
Tel: +49 30 515932-237 mailto:sledz@dresearch-fe.de
Fax: +49 30 515932-299
Geschäftsführer: Dr. Michael Weber, Werner Mögle;
Amtsgericht Berlin Charlottenburg; HRB 130120 B;
Ust.-IDNr. DE273952058

Re: complex versioning scenario

[Richard Purdie]

On Mon, 2014-03-24 at 13:16 +0100, Steffen Sledz wrote:
> We've a complex versioning scenario here which leads me to my limits. :(
> 
> There are two recipes. One for a shared library and one for an application using this library.
> 
> Both use GNU autotools (so they have internal version information). For continuous integration purposes both use AUTOREV.
> 
> At the moment the recipes look like this:
> 
> 
> ------------ libfoo_git.bb -------------
> PR = "r7"
> PE = "2"
> SRCREV="${AUTOREV}"
> PV = "gitr${SRCPV}"
> ...
> 
> 
> ------------ app_git.bb ----------------
> DEPENDS = "... libfoo ..."
> PR = "r10"
> PE = "1"
> SRCREV="${AUTOREV}"
> PV = "gitr${SRCPV}"
> ...
> 
> 
> Now we have the following problem. libfoo has some incompatible
> changes in its interface (a new internal major version).
> 
> In my opinion this should find its represenation in the package
> versioning in a way that the dependency checker can guarantee that the
> library and the application package match each other.

It is generally impossible to directly compare two git hashes and decide
whether one is "greater" than the other. This is why most git recipes
have PV = "0.0+git${SRCPV}" so that you can change 0.0 when something
major changes. That way you can put a constraint in the second recipe.

This is a fundamental problem with git versioning and not something we
can fix generically.

Cheers,

Richard

Re: complex versioning scenario

[Steffen Sledz]

On 24.03.2014 13:35, Richard Purdie wrote:
> On Mon, 2014-03-24 at 13:16 +0100, Steffen Sledz wrote:
>> We've a complex versioning scenario here which leads me to my limits. :(
>>
>> There are two recipes. One for a shared library and one for an application using this library.
>>
>> Both use GNU autotools (so they have internal version information). For continuous integration purposes both use AUTOREV.
>>
>> At the moment the recipes look like this:
>>
>>
>> ------------ libfoo_git.bb -------------
>> PR = "r7"
>> PE = "2"
>> SRCREV="${AUTOREV}"
>> PV = "gitr${SRCPV}"
>> ...
>>
>>
>> ------------ app_git.bb ----------------
>> DEPENDS = "... libfoo ..."
>> PR = "r10"
>> PE = "1"
>> SRCREV="${AUTOREV}"
>> PV = "gitr${SRCPV}"
>> ...
>>
>>
>> Now we have the following problem. libfoo has some incompatible
>> changes in its interface (a new internal major version).
>>
>> In my opinion this should find its represenation in the package
>> versioning in a way that the dependency checker can guarantee that the
>> library and the application package match each other.
> 
> It is generally impossible to directly compare two git hashes and decide
> whether one is "greater" than the other. This is why most git recipes
> have PV = "0.0+git${SRCPV}" so that you can change 0.0 when something
> major changes. That way you can put a constraint in the second recipe.
> 
> This is a fundamental problem with git versioning and not something we
> can fix generically.

To have an order in the git based versions we use the PRSERV method. This works well.

But this does not help here. The change in the library interface leads directly to a new version of the library package itself (e.g. from libfoo0_gitr100+somehash to libfoo0_gitr101+someotherhash). But i need something i can write into the DEPENDS list of the application. :(

Steffen

BTW: Where comes the 0 in libfoo0 from?

-- 
DResearch Fahrzeugelektronik GmbH
Otto-Schmirgal-Str. 3, 10319 Berlin, Germany
Tel: +49 30 515932-237 mailto:sledz@dresearch-fe.de
Fax: +49 30 515932-299
Geschäftsführer: Dr. Michael Weber, Werner Mögle;
Amtsgericht Berlin Charlottenburg; HRB 130120 B;
Ust.-IDNr. DE273952058

Re: complex versioning scenario

[Richard Purdie]

On Mon, 2014-03-24 at 13:49 +0100, Steffen Sledz wrote:
> On 24.03.2014 13:35, Richard Purdie wrote:
> > On Mon, 2014-03-24 at 13:16 +0100, Steffen Sledz wrote:
> >> We've a complex versioning scenario here which leads me to my limits. :(
> >>
> >> There are two recipes. One for a shared library and one for an application using this library.
> >>
> >> Both use GNU autotools (so they have internal version information). For continuous integration purposes both use AUTOREV.
> >>
> >> At the moment the recipes look like this:
> >>
> >>
> >> ------------ libfoo_git.bb -------------
> >> PR = "r7"
> >> PE = "2"
> >> SRCREV="${AUTOREV}"
> >> PV = "gitr${SRCPV}"
> >> ...
> >>
> >>
> >> ------------ app_git.bb ----------------
> >> DEPENDS = "... libfoo ..."
> >> PR = "r10"
> >> PE = "1"
> >> SRCREV="${AUTOREV}"
> >> PV = "gitr${SRCPV}"
> >> ...
> >>
> >>
> >> Now we have the following problem. libfoo has some incompatible
> >> changes in its interface (a new internal major version).
> >>
> >> In my opinion this should find its represenation in the package
> >> versioning in a way that the dependency checker can guarantee that the
> >> library and the application package match each other.
> > 
> > It is generally impossible to directly compare two git hashes and decide
> > whether one is "greater" than the other. This is why most git recipes
> > have PV = "0.0+git${SRCPV}" so that you can change 0.0 when something
> > major changes. That way you can put a constraint in the second recipe.
> > 
> > This is a fundamental problem with git versioning and not something we
> > can fix generically.
> 
> To have an order in the git based versions we use the PRSERV method. This works well.
> 
> But this does not help here. The change in the library interface leads
> directly to a new version of the library package itself (e.g. from
> libfoo0_gitr100+somehash to libfoo0_gitr101+someotherhash). But i need
> something i can write into the DEPENDS list of the application. :(
> 
> Steffen
> 
> BTW: Where comes the 0 in libfoo0 from?

debian.bbclass (debian package naming) which I believe in turn is
derived from the actual library version.

Its a class specific implementation so you can't depend on it in version
information though.

I still think your only solution here is to inject a real version into
PV...

Cheers,

Richard

Re: complex versioning scenario

[Steffen Sledz]

On 24.03.2014 13:53, Richard Purdie wrote:
> On Mon, 2014-03-24 at 13:49 +0100, Steffen Sledz wrote:
>> On 24.03.2014 13:35, Richard Purdie wrote:
>>> On Mon, 2014-03-24 at 13:16 +0100, Steffen Sledz wrote:
>>>> We've a complex versioning scenario here which leads me to my limits. :(
>>>>
>>>> There are two recipes. One for a shared library and one for an application using this library.
>>>>
>>>> Both use GNU autotools (so they have internal version information). For continuous integration purposes both use AUTOREV.
>>>>
>>>> At the moment the recipes look like this:
>>>>
>>>>
>>>> ------------ libfoo_git.bb -------------
>>>> PR = "r7"
>>>> PE = "2"
>>>> SRCREV="${AUTOREV}"
>>>> PV = "gitr${SRCPV}"
>>>> ...
>>>>
>>>>
>>>> ------------ app_git.bb ----------------
>>>> DEPENDS = "... libfoo ..."
>>>> PR = "r10"
>>>> PE = "1"
>>>> SRCREV="${AUTOREV}"
>>>> PV = "gitr${SRCPV}"
>>>> ...
>>>>
>>>>
>>>> Now we have the following problem. libfoo has some incompatible
>>>> changes in its interface (a new internal major version).
>>>>
>>>> In my opinion this should find its represenation in the package
>>>> versioning in a way that the dependency checker can guarantee that the
>>>> library and the application package match each other.
>>>
>>> It is generally impossible to directly compare two git hashes and decide
>>> whether one is "greater" than the other. This is why most git recipes
>>> have PV = "0.0+git${SRCPV}" so that you can change 0.0 when something
>>> major changes. That way you can put a constraint in the second recipe.
>>>
>>> This is a fundamental problem with git versioning and not something we
>>> can fix generically.
>>
>> To have an order in the git based versions we use the PRSERV method. This works well.
>>
>> But this does not help here. The change in the library interface leads
>> directly to a new version of the library package itself (e.g. from
>> libfoo0_gitr100+somehash to libfoo0_gitr101+someotherhash). But i need
>> something i can write into the DEPENDS list of the application. :(
>>
>> Steffen
>>
>> BTW: Where comes the 0 in libfoo0 from?
> 
> debian.bbclass (debian package naming) which I believe in turn is
> derived from the actual library version.
> 
> Its a class specific implementation so you can't depend on it in version
> information though.

But where does it come from? A bb variable?

> I still think your only solution here is to inject a real version into
> PV...

I tried this:

change libfoo recipe to PV = "1.0+gitr${SRCPV}"
leads to libfoo0_1.0+gitr102+someotherhash....ipk

then

change app recipe to DEPENDS = "... libfoo-1.0 ..."
leads to: "ERROR: Nothing PROVIDES 'libfoo-1.0'" :(

change app recipe to DEPENDS = "... libfoo0-1.0 ..."
leads to: "ERROR: Nothing PROVIDES 'libfoo0-1.0'" :(

change app recipe to DEPENDS = "... libfoo0_1.0 ..."
leads to: "ERROR: Nothing PROVIDES 'libfoo0_1.0'" :(

Where's the mistake?

-- 
DResearch Fahrzeugelektronik GmbH
Otto-Schmirgal-Str. 3, 10319 Berlin, Germany
Tel: +49 30 515932-237 mailto:sledz@dresearch-fe.de
Fax: +49 30 515932-299
Geschäftsführer: Dr. Michael Weber, Werner Mögle;
Amtsgericht Berlin Charlottenburg; HRB 130120 B;
Ust.-IDNr. DE273952058

Re: complex versioning scenario

[Richard Purdie]

On Mon, 2014-03-24 at 15:22 +0100, Steffen Sledz wrote:
> On 24.03.2014 13:53, Richard Purdie wrote:
> > On Mon, 2014-03-24 at 13:49 +0100, Steffen Sledz wrote:
> >> On 24.03.2014 13:35, Richard Purdie wrote:
> >>> On Mon, 2014-03-24 at 13:16 +0100, Steffen Sledz wrote:
> > debian.bbclass (debian package naming) which I believe in turn is
> > derived from the actual library version.
> > 
> > Its a class specific implementation so you can't depend on it in version
> > information though.
> 
> But where does it come from? A bb variable?

Read debian.bbclass. It does objdump -p and looks at the SONAME in the
output.

> 
> > I still think your only solution here is to inject a real version into
> > PV...
> 
> I tried this:
> 
> change libfoo recipe to PV = "1.0+gitr${SRCPV}"
> leads to libfoo0_1.0+gitr102+someotherhash....ipk
> 
> then
> 
> change app recipe to DEPENDS = "... libfoo-1.0 ..."
> leads to: "ERROR: Nothing PROVIDES 'libfoo-1.0'" :(
> 
> change app recipe to DEPENDS = "... libfoo0-1.0 ..."
> leads to: "ERROR: Nothing PROVIDES 'libfoo0-1.0'" :(
> 
> change app recipe to DEPENDS = "... libfoo0_1.0 ..."
> leads to: "ERROR: Nothing PROVIDES 'libfoo0_1.0'" :(
> 
> Where's the mistake?

You'd need to do RDEPENDS_xxxx = "libfoo (>=1.0)"

Cheers,

Richard

Re: complex versioning scenario

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 3617 bytes --]

On Mon, Mar 24, 2014 at 03:22:35PM +0100, Steffen Sledz wrote:
> On 24.03.2014 13:53, Richard Purdie wrote:
> > On Mon, 2014-03-24 at 13:49 +0100, Steffen Sledz wrote:
> >> On 24.03.2014 13:35, Richard Purdie wrote:
> >>> On Mon, 2014-03-24 at 13:16 +0100, Steffen Sledz wrote:
> >>>> We've a complex versioning scenario here which leads me to my limits. :(
> >>>>
> >>>> There are two recipes. One for a shared library and one for an application using this library.
> >>>>
> >>>> Both use GNU autotools (so they have internal version information). For continuous integration purposes both use AUTOREV.
> >>>>
> >>>> At the moment the recipes look like this:
> >>>>
> >>>>
> >>>> ------------ libfoo_git.bb -------------
> >>>> PR = "r7"
> >>>> PE = "2"
> >>>> SRCREV="${AUTOREV}"
> >>>> PV = "gitr${SRCPV}"
> >>>> ...
> >>>>
> >>>>
> >>>> ------------ app_git.bb ----------------
> >>>> DEPENDS = "... libfoo ..."
> >>>> PR = "r10"
> >>>> PE = "1"
> >>>> SRCREV="${AUTOREV}"
> >>>> PV = "gitr${SRCPV}"
> >>>> ...
> >>>>
> >>>>
> >>>> Now we have the following problem. libfoo has some incompatible
> >>>> changes in its interface (a new internal major version).
> >>>>
> >>>> In my opinion this should find its represenation in the package
> >>>> versioning in a way that the dependency checker can guarantee that the
> >>>> library and the application package match each other.
> >>>
> >>> It is generally impossible to directly compare two git hashes and decide
> >>> whether one is "greater" than the other. This is why most git recipes
> >>> have PV = "0.0+git${SRCPV}" so that you can change 0.0 when something
> >>> major changes. That way you can put a constraint in the second recipe.
> >>>
> >>> This is a fundamental problem with git versioning and not something we
> >>> can fix generically.
> >>
> >> To have an order in the git based versions we use the PRSERV method. This works well.
> >>
> >> But this does not help here. The change in the library interface leads
> >> directly to a new version of the library package itself (e.g. from
> >> libfoo0_gitr100+somehash to libfoo0_gitr101+someotherhash). But i need
> >> something i can write into the DEPENDS list of the application. :(
> >>
> >> Steffen
> >>
> >> BTW: Where comes the 0 in libfoo0 from?
> > 
> > debian.bbclass (debian package naming) which I believe in turn is
> > derived from the actual library version.
> > 
> > Its a class specific implementation so you can't depend on it in version
> > information though.
> 
> But where does it come from? A bb variable?

SONAME header in library

so if you're using debian.bbclass and change ABI then you should just
increase major version in SONAME (that way your foo will rdepend on
libfoo0 until it's rebuilt against newer libfoo1).

> > I still think your only solution here is to inject a real version into
> > PV...
> 
> I tried this:
> 
> change libfoo recipe to PV = "1.0+gitr${SRCPV}"
> leads to libfoo0_1.0+gitr102+someotherhash....ipk
> 
> then
> 
> change app recipe to DEPENDS = "... libfoo-1.0 ..."
> leads to: "ERROR: Nothing PROVIDES 'libfoo-1.0'" :(
> 
> change app recipe to DEPENDS = "... libfoo0-1.0 ..."
> leads to: "ERROR: Nothing PROVIDES 'libfoo0-1.0'" :(
> 
> change app recipe to DEPENDS = "... libfoo0_1.0 ..."
> leads to: "ERROR: Nothing PROVIDES 'libfoo0_1.0'" :(
> 
> Where's the mistake?

it provides only the full P libfoo-1.0+gitr102+someotherhash so you
would need to add own PROVIDES value in libfoo.

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 205 bytes --]

Re: complex versioning scenario

[Steffen Sledz]

On 24.03.2014 16:15, Martin Jansa wrote:
> On Mon, Mar 24, 2014 at 03:22:35PM +0100, Steffen Sledz wrote:
>> On 24.03.2014 13:53, Richard Purdie wrote:
>>> On Mon, 2014-03-24 at 13:49 +0100, Steffen Sledz wrote:
>>>> On 24.03.2014 13:35, Richard Purdie wrote:
>>>>> On Mon, 2014-03-24 at 13:16 +0100, Steffen Sledz wrote:
>>>>>> We've a complex versioning scenario here which leads me to my limits. :(
>>>>>> 
>>>>>> There are two recipes. One for a shared library and one for an application using this library.
>>>>>> 
>>>>>> Both use GNU autotools (so they have internal version information). For continuous integration purposes both use AUTOREV.
>>>>>> 
>>>>>> At the moment the recipes look like this:
>>>>>> 
>>>>>> 
>>>>>> ------------ libfoo_git.bb ------------- PR = "r7" PE = "2" SRCREV="${AUTOREV}" PV = "gitr${SRCPV}" ...
>>>>>> 
>>>>>> 
>>>>>> ------------ app_git.bb ---------------- DEPENDS = "... libfoo ..." PR = "r10" PE = "1" SRCREV="${AUTOREV}" PV = "gitr${SRCPV}" ...
>>>>>> 
>>>>>> 
>>>>>> Now we have the following problem. libfoo has some incompatible changes in its interface (a new internal major version).
>>>>>> 
>>>>>> In my opinion this should find its represenation in the package versioning in a way that the dependency checker can guarantee that the library and the application package match each other.
>>>>> 
>>>>> It is generally impossible to directly compare two git hashes and decide whether one is "greater" than the other. This is why most git recipes have PV = "0.0+git${SRCPV}" so that you can change 0.0 when something major changes. That way you can put a constraint in the second recipe.
>>>>> 
>>>>> This is a fundamental problem with git versioning and not something we can fix generically.
>>>> 
>>>> To have an order in the git based versions we use the PRSERV method. This works well.
>>>> 
>>>> But this does not help here. The change in the library interface leads directly to a new version of the library package itself (e.g. from libfoo0_gitr100+somehash to libfoo0_gitr101+someotherhash). But i need something i can write into the DEPENDS list of the application. :(
>>>> 
>>>> Steffen
>>>> 
>>>> BTW: Where comes the 0 in libfoo0 from?
>>> 
>>> debian.bbclass (debian package naming) which I believe in turn is derived from the actual library version.
>>> 
>>> Its a class specific implementation so you can't depend on it in version information though.
>> 
>> But where does it come from? A bb variable?
> 
> SONAME header in library
> 
> so if you're using debian.bbclass and change ABI then you should just increase major version in SONAME (that way your foo will rdepend on libfoo0 until it's rebuilt against newer libfoo1).

Thanx, this was the decisive hint.

I've increased the version in the SONAME header of the library and the result is a libfoo1 package. :)

But now i hit the next problem. The following rootfs stage results in this error:

---------------> snip <-----------------
| Collected errors:
|  * satisfy_dependencies_for: Cannot satisfy the following dependencies for app:
|  *    libfoo0 (>= gitr101+somehash) *
---------------> snap <-----------------

Should the new build of libfoo1 trigger a new compile of all packages with DEPENDS containing libfoo?

-- 
DResearch Fahrzeugelektronik GmbH
Otto-Schmirgal-Str. 3, 10319 Berlin, Germany
Tel: +49 30 515932-237 mailto:sledz@dresearch-fe.de
Fax: +49 30 515932-299
Geschäftsführer: Dr. Michael Weber, Werner Mögle;
Amtsgericht Berlin Charlottenburg; HRB 130120 B;
Ust.-IDNr. DE273952058

Re: complex versioning scenario

[Richard Purdie]

On Tue, 2014-03-25 at 11:31 +0100, Steffen Sledz wrote:
> On 24.03.2014 16:15, Martin Jansa wrote:
> Thanx, this was the decisive hint.
> 
> I've increased the version in the SONAME header of the library and the result is a libfoo1 package. :)
> 
> But now i hit the next problem. The following rootfs stage results in this error:
> 
> ---------------> snip <-----------------
> | Collected errors:
> |  * satisfy_dependencies_for: Cannot satisfy the following dependencies for app:
> |  *    libfoo0 (>= gitr101+somehash) *
> ---------------> snap <-----------------
> 
> Should the new build of libfoo1 trigger a new compile of all packages with DEPENDS containing libfoo?

In theory, yes and in practise, bitbake does actually do this.
Unfortunately autotools sometimes do not rebuild things when it should.
This is why I've been promoting the B != S patch set which forces things
to rebuild in scenarios like this and removes the issue from autotools
and lets us handle it ourselves.

So my recommendation would be to opt in to that include file and this
should help errors like this.

The change I'd love to commit is:

http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=rpurdie/t2&id=aedf7d207190f18466e82cbc9b3d49b19b3f376f

but I think its now too late to do this by default in 1.6.

Cheers,

Richard

Re: complex versioning scenario

[Mark Hatle]

On 3/25/14, 5:31 AM, Steffen Sledz wrote:
> On 24.03.2014 16:15, Martin Jansa wrote:
>> On Mon, Mar 24, 2014 at 03:22:35PM +0100, Steffen Sledz wrote:
>>> On 24.03.2014 13:53, Richard Purdie wrote:
>>>> On Mon, 2014-03-24 at 13:49 +0100, Steffen Sledz wrote:
>>>>> On 24.03.2014 13:35, Richard Purdie wrote:
>>>>>> On Mon, 2014-03-24 at 13:16 +0100, Steffen Sledz wrote:
>>>>>>> We've a complex versioning scenario here which leads me to my limits. :(
>>>>>>>
>>>>>>> There are two recipes. One for a shared library and one for an application using this library.
>>>>>>>
>>>>>>> Both use GNU autotools (so they have internal version information). For continuous integration purposes both use AUTOREV.
>>>>>>>
>>>>>>> At the moment the recipes look like this:
>>>>>>>
>>>>>>>
>>>>>>> ------------ libfoo_git.bb ------------- PR = "r7" PE = "2" SRCREV="${AUTOREV}" PV = "gitr${SRCPV}" ...
>>>>>>>
>>>>>>>
>>>>>>> ------------ app_git.bb ---------------- DEPENDS = "... libfoo ..." PR = "r10" PE = "1" SRCREV="${AUTOREV}" PV = "gitr${SRCPV}" ...
>>>>>>>
>>>>>>>
>>>>>>> Now we have the following problem. libfoo has some incompatible changes in its interface (a new internal major version).
>>>>>>>
>>>>>>> In my opinion this should find its represenation in the package versioning in a way that the dependency checker can guarantee that the library and the application package match each other.
>>>>>>
>>>>>> It is generally impossible to directly compare two git hashes and decide whether one is "greater" than the other. This is why most git recipes have PV = "0.0+git${SRCPV}" so that you can change 0.0 when something major changes. That way you can put a constraint in the second recipe.
>>>>>>
>>>>>> This is a fundamental problem with git versioning and not something we can fix generically.
>>>>>
>>>>> To have an order in the git based versions we use the PRSERV method. This works well.
>>>>>
>>>>> But this does not help here. The change in the library interface leads directly to a new version of the library package itself (e.g. from libfoo0_gitr100+somehash to libfoo0_gitr101+someotherhash). But i need something i can write into the DEPENDS list of the application. :(
>>>>>
>>>>> Steffen
>>>>>
>>>>> BTW: Where comes the 0 in libfoo0 from?
>>>>
>>>> debian.bbclass (debian package naming) which I believe in turn is derived from the actual library version.
>>>>
>>>> Its a class specific implementation so you can't depend on it in version information though.
>>>
>>> But where does it come from? A bb variable?
>>
>> SONAME header in library
>>
>> so if you're using debian.bbclass and change ABI then you should just increase major version in SONAME (that way your foo will rdepend on libfoo0 until it's rebuilt against newer libfoo1).
>
> Thanx, this was the decisive hint.
>
> I've increased the version in the SONAME header of the library and the result is a libfoo1 package. :)
>
> But now i hit the next problem. The following rootfs stage results in this error:
>
> ---------------> snip <-----------------
> | Collected errors:
> |  * satisfy_dependencies_for: Cannot satisfy the following dependencies for app:
> |  *    libfoo0 (>= gitr101+somehash) *
> ---------------> snap <-----------------
>
> Should the new build of libfoo1 trigger a new compile of all packages with DEPENDS containing libfoo?
>

If the package 'requiring libfoo' has a DEPENDS += ... in it.. then yes, it 
should have been rebuilt when the libfoo was rebuilt.

--Mark

Re: complex versioning scenario

[Steffen Sledz]

On 25.03.2014 16:03, Mark Hatle wrote:
> On 3/25/14, 5:31 AM, Steffen Sledz wrote:
>> On 24.03.2014 16:15, Martin Jansa wrote:
>>> On Mon, Mar 24, 2014 at 03:22:35PM +0100, Steffen Sledz wrote:
>>>> On 24.03.2014 13:53, Richard Purdie wrote:
>>>>> On Mon, 2014-03-24 at 13:49 +0100, Steffen Sledz wrote:
>>>>>> On 24.03.2014 13:35, Richard Purdie wrote:
>>>>>>> On Mon, 2014-03-24 at 13:16 +0100, Steffen Sledz wrote:
>>>>>>>> We've a complex versioning scenario here which leads me to my limits. :(
>>>>>>>>
>>>>>>>> There are two recipes. One for a shared library and one for an application using this library.
>>>>>>>>
>>>>>>>> Both use GNU autotools (so they have internal version information). For continuous integration purposes both use AUTOREV.
>>>>>>>>
>>>>>>>> At the moment the recipes look like this:
>>>>>>>>
>>>>>>>>
>>>>>>>> ------------ libfoo_git.bb ------------- PR = "r7" PE = "2" SRCREV="${AUTOREV}" PV = "gitr${SRCPV}" ...
>>>>>>>>
>>>>>>>>
>>>>>>>> ------------ app_git.bb ---------------- DEPENDS = "... libfoo ..." PR = "r10" PE = "1" SRCREV="${AUTOREV}" PV = "gitr${SRCPV}" ...
>>>>>>>>
>>>>>>>>
>>>>>>>> Now we have the following problem. libfoo has some incompatible changes in its interface (a new internal major version).
>>>>>>>>
>>>>>>>> In my opinion this should find its represenation in the package versioning in a way that the dependency checker can guarantee that the library and the application package match each other.
>>>>>>>
>>>>>>> It is generally impossible to directly compare two git hashes and decide whether one is "greater" than the other. This is why most git recipes have PV = "0.0+git${SRCPV}" so that you can change 0.0 when something major changes. That way you can put a constraint in the second recipe.
>>>>>>>
>>>>>>> This is a fundamental problem with git versioning and not something we can fix generically.
>>>>>>
>>>>>> To have an order in the git based versions we use the PRSERV method. This works well.
>>>>>>
>>>>>> But this does not help here. The change in the library interface leads directly to a new version of the library package itself (e.g. from libfoo0_gitr100+somehash to libfoo0_gitr101+someotherhash). But i need something i can write into the DEPENDS list of the application. :(
>>>>>>
>>>>>> Steffen
>>>>>>
>>>>>> BTW: Where comes the 0 in libfoo0 from?
>>>>>
>>>>> debian.bbclass (debian package naming) which I believe in turn is derived from the actual library version.
>>>>>
>>>>> Its a class specific implementation so you can't depend on it in version information though.
>>>>
>>>> But where does it come from? A bb variable?
>>>
>>> SONAME header in library
>>>
>>> so if you're using debian.bbclass and change ABI then you should just increase major version in SONAME (that way your foo will rdepend on libfoo0 until it's rebuilt against newer libfoo1).
>>
>> Thanx, this was the decisive hint.
>>
>> I've increased the version in the SONAME header of the library and the result is a libfoo1 package. :)
>>
>> But now i hit the next problem. The following rootfs stage results in this error:
>>
>> ---------------> snip <-----------------
>> | Collected errors:
>> |  * satisfy_dependencies_for: Cannot satisfy the following dependencies for app:
>> |  *    libfoo0 (>= gitr101+somehash) *
>> ---------------> snap <-----------------
>>
>> Should the new build of libfoo1 trigger a new compile of all packages with DEPENDS containing libfoo?
>>
> 
> If the package 'requiring libfoo' has a DEPENDS += ... in it.. then yes, it should have been rebuilt when the libfoo was rebuilt.

Unfortunately i can't confirm that.  :(

part of the real app recipe:
------------> snip <-------------
DEPENDS = "vala-native libdrtrace libdrhip libdrbcc jansson"
RDEPENDS_${PN} = "dropmodes"
------------> snap <-------------

part of the real resulting opkg control file for this app:
------------> snip <-------------
Depends: dropmodes, libglib-2.0-0 (>= 2.36.4), libdrhip1 (>= gitr27+42af787eb2), libjansson4 (>= 2.4), libc6 (>= 2.18)
------------> snap <-------------

I miss the runtime dependencies for libdrtrace and libdrbcc. Where are they gone?

-- 
DResearch Fahrzeugelektronik GmbH
Otto-Schmirgal-Str. 3, 10319 Berlin, Germany
Tel: +49 30 515932-237 mailto:sledz@dresearch-fe.de
Fax: +49 30 515932-299
Geschäftsführer: Dr. Michael Weber, Werner Mögle;
Amtsgericht Berlin Charlottenburg; HRB 130120 B;
Ust.-IDNr. DE273952058

Re: complex versioning scenario

[Steffen Sledz]

On 07.04.2014 14:37, Steffen Sledz wrote:
> On 25.03.2014 16:03, Mark Hatle wrote:
>> ...
>> If the package 'requiring libfoo' has a DEPENDS += ... in it.. then yes, it should have been rebuilt when the libfoo was rebuilt.
> 
> Unfortunately i can't confirm that.  :(
> 
> part of the real app recipe:
> ------------> snip <-------------
> DEPENDS = "vala-native libdrtrace libdrhip libdrbcc jansson"
> RDEPENDS_${PN} = "dropmodes"
> ------------> snap <-------------
> 
> part of the real resulting opkg control file for this app:
> ------------> snip <-------------
> Depends: dropmodes, libglib-2.0-0 (>= 2.36.4), libdrhip1 (>= gitr27+42af787eb2), libjansson4 (>= 2.4), libc6 (>= 2.18)
> ------------> snap <-------------
> 
> I miss the runtime dependencies for libdrtrace and libdrbcc. Where are they gone?

Some additional info:
------------> snip <-------------
# objdump -p ./package/usr/lib/libdrhip.so.1.0.0

./package/usr/lib/libdrhip.so.1.0.0:     file format elf32-littlearm
...
Dynamic Section:
  NEEDED               libc.so.6
  SONAME               libdrhip.so.1
...

# objdump -p ./package/usr/lib/libdrbcc.so.1.0.0

./package/usr/lib/libdrbcc.so.1.0.0:     file format elf32-littlearm
...
Dynamic Section:
  NEEDED               libdrtrace.so.0
  NEEDED               libm.so.6
  NEEDED               libreadline.so.6
  NEEDED               libpthread.so.0
  NEEDED               libc.so.6
  SONAME               libdrbcc.so.1
...

# objdump -p ./package/usr/bin/drbccproxy

./package/usr/bin/drbccproxy:     file format elf32-littlearm
...
Dynamic Section:
  NEEDED               libdrhip.so.1
  NEEDED               libdrbcc.so.0
  NEEDED               libdrtrace.so.0
  NEEDED               libgio-2.0.so.0
  NEEDED               libgobject-2.0.so.0
  NEEDED               libglib-2.0.so.0
  NEEDED               libjansson.so.4
  NEEDED               librt.so.1
  NEEDED               libpthread.so.0
  NEEDED               libc.so.6
...
------------> snap <-------------

So it seems the data objdump shows are OK.

E.g. the app drbccproxy really has a dependency to a libdrbcc. But this is not refelected in the control file.

-- 
DResearch Fahrzeugelektronik GmbH
Otto-Schmirgal-Str. 3, 10319 Berlin, Germany
Tel: +49 30 515932-237 mailto:sledz@dresearch-fe.de
Fax: +49 30 515932-299
Geschäftsführer: Dr. Michael Weber, Werner Mögle;
Amtsgericht Berlin Charlottenburg; HRB 130120 B;
Ust.-IDNr. DE273952058

Re: complex versioning scenario

[Richard Purdie]

On Mon, 2014-04-07 at 15:22 +0200, Steffen Sledz wrote:
> On 07.04.2014 14:37, Steffen Sledz wrote:
> > On 25.03.2014 16:03, Mark Hatle wrote:
> >> ...
> >> If the package 'requiring libfoo' has a DEPENDS += ... in it.. then yes, it should have been rebuilt when the libfoo was rebuilt.
> > 
> > Unfortunately i can't confirm that.  :(
> > 
> > part of the real app recipe:
> > ------------> snip <-------------
> > DEPENDS = "vala-native libdrtrace libdrhip libdrbcc jansson"
> > RDEPENDS_${PN} = "dropmodes"
> > ------------> snap <-------------
> > 
> > part of the real resulting opkg control file for this app:
> > ------------> snip <-------------
> > Depends: dropmodes, libglib-2.0-0 (>= 2.36.4), libdrhip1 (>= gitr27+42af787eb2), libjansson4 (>= 2.4), libc6 (>= 2.18)
> > ------------> snap <-------------
> > 
> > I miss the runtime dependencies for libdrtrace and libdrbcc. Where are they gone?
> 
> Some additional info:
> ------------> snip <-------------
> # objdump -p ./package/usr/lib/libdrhip.so.1.0.0
> 
> ./package/usr/lib/libdrhip.so.1.0.0:     file format elf32-littlearm
> ...
> Dynamic Section:
>   NEEDED               libc.so.6
>   SONAME               libdrhip.so.1
> ...
> 
> # objdump -p ./package/usr/lib/libdrbcc.so.1.0.0
> 
> ./package/usr/lib/libdrbcc.so.1.0.0:     file format elf32-littlearm
> ...
> Dynamic Section:
>   NEEDED               libdrtrace.so.0
>   NEEDED               libm.so.6
>   NEEDED               libreadline.so.6
>   NEEDED               libpthread.so.0
>   NEEDED               libc.so.6
>   SONAME               libdrbcc.so.1
> ...
> 
> # objdump -p ./package/usr/bin/drbccproxy
> 
> ./package/usr/bin/drbccproxy:     file format elf32-littlearm
> ...
> Dynamic Section:
>   NEEDED               libdrhip.so.1
>   NEEDED               libdrbcc.so.0
>   NEEDED               libdrtrace.so.0
>   NEEDED               libgio-2.0.so.0
>   NEEDED               libgobject-2.0.so.0
>   NEEDED               libglib-2.0.so.0
>   NEEDED               libjansson.so.4
>   NEEDED               librt.so.1
>   NEEDED               libpthread.so.0
>   NEEDED               libc.so.6
> ...
> ------------> snap <-------------
> 
> So it seems the data objdump shows are OK.
> 
> E.g. the app drbccproxy really has a dependency to a libdrbcc. But this is not refelected in the control file.

At this point you'll probably have to look at the shlibs code in
package.bbclass and see why its not picking up the shlib dependencies
correctly for the packages...

It does appear there is some problem there but its hard to saw what
without a test case to reproduce.

Cheers,

Richard

Re: complex versioning scenario

[Steffen Sledz]

On 07.04.2014 16:49, Richard Purdie wrote:
> On Mon, 2014-04-07 at 15:22 +0200, Steffen Sledz wrote:
>> On 07.04.2014 14:37, Steffen Sledz wrote:
>>> On 25.03.2014 16:03, Mark Hatle wrote:
>>>> ...
>>>> If the package 'requiring libfoo' has a DEPENDS += ... in it.. then yes, it should have been rebuilt when the libfoo was rebuilt.
>>>
>>> Unfortunately i can't confirm that.  :(
>>>
>>> part of the real app recipe:
>>> ------------> snip <-------------
>>> DEPENDS = "vala-native libdrtrace libdrhip libdrbcc jansson"
>>> RDEPENDS_${PN} = "dropmodes"
>>> ------------> snap <-------------
>>>
>>> part of the real resulting opkg control file for this app:
>>> ------------> snip <-------------
>>> Depends: dropmodes, libglib-2.0-0 (>= 2.36.4), libdrhip1 (>= gitr27+42af787eb2), libjansson4 (>= 2.4), libc6 (>= 2.18)
>>> ------------> snap <-------------
>>>
>>> I miss the runtime dependencies for libdrtrace and libdrbcc. Where are they gone?
>>
>> Some additional info:
>> ------------> snip <-------------
>> # objdump -p ./package/usr/lib/libdrhip.so.1.0.0
>>
>> ./package/usr/lib/libdrhip.so.1.0.0:     file format elf32-littlearm
>> ...
>> Dynamic Section:
>>   NEEDED               libc.so.6
>>   SONAME               libdrhip.so.1
>> ...
>>
>> # objdump -p ./package/usr/lib/libdrbcc.so.1.0.0
>>
>> ./package/usr/lib/libdrbcc.so.1.0.0:     file format elf32-littlearm
>> ...
>> Dynamic Section:
>>   NEEDED               libdrtrace.so.0
>>   NEEDED               libm.so.6
>>   NEEDED               libreadline.so.6
>>   NEEDED               libpthread.so.0
>>   NEEDED               libc.so.6
>>   SONAME               libdrbcc.so.1
>> ...
>>
>> # objdump -p ./package/usr/bin/drbccproxy
>>
>> ./package/usr/bin/drbccproxy:     file format elf32-littlearm
>> ...
>> Dynamic Section:
>>   NEEDED               libdrhip.so.1
>>   NEEDED               libdrbcc.so.0
>>   NEEDED               libdrtrace.so.0
>>   NEEDED               libgio-2.0.so.0
>>   NEEDED               libgobject-2.0.so.0
>>   NEEDED               libglib-2.0.so.0
>>   NEEDED               libjansson.so.4
>>   NEEDED               librt.so.1
>>   NEEDED               libpthread.so.0
>>   NEEDED               libc.so.6
>> ...
>> ------------> snap <-------------
>>
>> So it seems the data objdump shows are OK.
>>
>> E.g. the app drbccproxy really has a dependency to a libdrbcc. But this is not refelected in the control file.
> 
> At this point you'll probably have to look at the shlibs code in
> package.bbclass and see why its not picking up the shlib dependencies
> correctly for the packages...
> 
> It does appear there is some problem there but its hard to saw what
> without a test case to reproduce.

I could isolate the problem. :)

If a package contains a shared library *and* a binary (e.g. a related command line tool) then the soname major version *is not appended* to the package name (see debian_package_name_hook in debian.bbclass). All other problems are aftereffects.

Is this an intended behaviour? There should be a big fat warning if this is the case.

Or is it a bug?

-- 
DResearch Fahrzeugelektronik GmbH
Otto-Schmirgal-Str. 3, 10319 Berlin, Germany
Tel: +49 30 515932-237 mailto:sledz@dresearch-fe.de
Fax: +49 30 515932-299
Geschäftsführer: Dr. Michael Weber, Werner Mögle;
Amtsgericht Berlin Charlottenburg; HRB 130120 B;
Ust.-IDNr. DE273952058

Re: complex versioning scenario

[Khem Raj]

On Tue, Apr 8, 2014 at 5:33 AM, Steffen Sledz <sledz@dresearch-fe.de> wrote:
> If a package contains a shared library *and* a binary (e.g. a related command line tool) then the soname major version *is not appended* to the package name (see debian_package_name_hook in debian.bbclass). All other problems are aftereffects.

shared objects which are to be used by other independent packages (
public .so) should be packaged separately into runtime shared object
package which is separate from packages with binaries so that multiple
versions of same librararies
can be packaged on single system, your usecase seems to indicate that
the .so is just used by this given binary you bundle together is that
the case ?

Re: complex versioning scenario

[Steffen Sledz]

Am 08.04.2014 19:20, schrieb Khem Raj:
> On Tue, Apr 8, 2014 at 5:33 AM, Steffen Sledz <sledz@dresearch-fe.de> wrote:
>> If a package contains a shared library *and* a binary (e.g. a related command line tool) then the soname major version *is not appended* to the package name (see debian_package_name_hook in debian.bbclass). All other problems are aftereffects.
> 
> shared objects which are to be used by other independent packages (
> public .so) should be packaged separately into runtime shared object
> package which is separate from packages with binaries so that multiple
> versions of same librararies
> can be packaged on single system, your usecase seems to indicate that
> the .so is just used by this given binary you bundle together is that
> the case ?

No it is used by the binary which is part of the package *and* some other binaries.

So i've to split the package into two packages of which one contains only the shared object. Right?

-- 
DResearch Fahrzeugelektronik GmbH
Otto-Schmirgal-Str. 3, 10319 Berlin, Germany
Tel: +49 30 515932-237 mailto:sledz@dresearch-fe.de
Fax: +49 30 515932-299
Geschäftsführer: Dr. Michael Weber, Werner Mögle;
Amtsgericht Berlin Charlottenburg; HRB 130120 B;
Ust.-IDNr. DE273952058

Re: complex versioning scenario

[Khem Raj]

[-- Attachment #1: Type: text/plain, Size: 1393 bytes --]

On Apr 8, 2014 8:58 AM, "Steffen Sledz" <sledz@dresearch-fe.de> wrote:
>
> Am 08.04.2014 19:20, schrieb Khem Raj:
> > On Tue, Apr 8, 2014 at 5:33 AM, Steffen Sledz <sledz@dresearch-fe.de>
wrote:
> >> If a package contains a shared library *and* a binary (e.g. a related
command line tool) then the soname major version *is not appended* to the
package name (see debian_package_name_hook in debian.bbclass). All other
problems are aftereffects.
> >
> > shared objects which are to be used by other independent packages (
> > public .so) should be packaged separately into runtime shared object
> > package which is separate from packages with binaries so that multiple
> > versions of same librararies
> > can be packaged on single system, your usecase seems to indicate that
> > the .so is just used by this given binary you bundle together is that
> > the case ?
>
> No it is used by the binary which is part of the package *and* some other
binaries.
>
> So i've to split the package into two packages of which one contains only
the shared object. Right?
yes
>
> --
> DResearch Fahrzeugelektronik GmbH
> Otto-Schmirgal-Str. 3, 10319 Berlin, Germany
> Tel: +49 30 515932-237 mailto:sledz@dresearch-fe.de
> Fax: +49 30 515932-299
> Geschäftsführer: Dr. Michael Weber, Werner Mögle;
> Amtsgericht Berlin Charlottenburg; HRB 130120 B;
> Ust.-IDNr. DE273952058

[-- Attachment #2: Type: text/html, Size: 1785 bytes --]

Re: complex versioning scenario

[Khem Raj]

On Mon, Mar 24, 2014 at 5:16 AM, Steffen Sledz <sledz@dresearch-fe.de> wrote:
> We've a complex versioning scenario here which leads me to my limits. :(
>
> There are two recipes. One for a shared library and one for an application using this library.
>
> Both use GNU autotools (so they have internal version information). For continuous integration purposes both use AUTOREV.
>
> At the moment the recipes look like this:
>
>
> ------------ libfoo_git.bb -------------
> PR = "r7"
> PE = "2"
> SRCREV="${AUTOREV}"
> PV = "gitr${SRCPV}"
> ...
>
>
> ------------ app_git.bb ----------------
> DEPENDS = "... libfoo ..."
> PR = "r10"
> PE = "1"
> SRCREV="${AUTOREV}"
> PV = "gitr${SRCPV}"
> ...
>
>
> Now we have the following problem. libfoo has some incompatible changes in its interface (a new internal major version).
>

This is the key question here. what is this new interface doing, is it
breaking any API ?
if not then why bother if it does then use gnu versioning scheme and
create versioned symbols
for this interface and you can simplify the problem outside the build
system and your library
remains compatible with older app too.



> In my opinion this should find its represenation in the package versioning in a way that the dependency checker can guarantee that the library and the application package match each other.
>
> But how?
>
> Regards,
> Steffen
>
> --
> DResearch Fahrzeugelektronik GmbH
> Otto-Schmirgal-Str. 3, 10319 Berlin, Germany
> Tel: +49 30 515932-237 mailto:sledz@dresearch-fe.de
> Fax: +49 30 515932-299
> Geschäftsführer: Dr. Michael Weber, Werner Mögle;
> Amtsgericht Berlin Charlottenburg; HRB 130120 B;
> Ust.-IDNr. DE273952058
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core