[RFC] Allarch and packagegroup improvement proposal

[RFC] Allarch and packagegroup improvement proposal

[Richard Purdie]

As some people are already painfully aware, the current way packagegroup
and allarch interact with the sstate signatures is painful.

Some example problem cases:

* An allarch package of scripts with an interpretor dependency (e.g perl)

* A packagegroup with dependencies on something which is debian renamed 
  e.g. build-essentials on libgmp 


Currently, packagegroups default to allarch and have their sstate
dependencies truncated. This means that when something like gmp
rebuilds, it can change package name thanks to debian renaming but the
packagegroup referencing the old name remains, causing image creation
failures.

So the "obvious" fix is to stop truncating the dependencies? Well, that
means that the allarch package is written out every time machine changes
to a different arch.

So we need to make packagegroups PACKAGE_ARCH specific? Well, this still
doesn't solve the problem of allarch packages having ever greater
problems with things like dependencies on perl.

An alternative solution I'm wondering about to address both issues is:

* the sstatesig packagegroup code for dependencies handling should only
trigger for allarch 
* we add a sanity test that checks if renaming has happened for
packagegroup dependencies and warn if it has.

The trouble is the sstatesig code doesn't have the full data store, only
a limited cross section so it can't check if PACKAGE_ARCH == all, only
if the class is inherited. We therefore probably need to change
packagegroup to do conditional class inclusion (which is possible with
the modern code base at least). The use of the class can be tested from
the sstatesig code and hence indicate allarch.


For the second part, we could add something like:

+    if bb.data.inherits_class('packagegroup', d) and bb.data.inherits_class('allarch', d):    
+        if set(deps) != set(new_depends):
+            bb.warn("allarch packagegroup %s has renamed dependency %s" % (pkg, str(set(deps) ^ set(new_depends)))) 

to runtime_mapping_rename in package.bbclass which tells us which
packagegroups need to move to be PACKAGE_ARCH. Unfortunately, a quick
run of OE-Core shows:

WARNING: allarch packagegroup packagegroup-core-sdk has renamed dependency set(['libgomp', 'libgomp1'])
WARNING: allarch packagegroup packagegroup-core-x11-base has renamed dependency set(['dbus-1', 'dbus'])
WARNING: allarch packagegroup packagegroup-core-standalone-sdk-target has renamed dependency set(['libgcc1', 'libgcc-s-dev', 'libstdc++', 'libc6-dev', 'libgcc-dev', 'libgcc', 'libc6', 'eglibc-dbg', 'libc6-dbg', 'eglibc-dev', 'eglibc-thread-db', 'eglibc', 'libthread-db1', 'libstdc++6'])
WARNING: allarch packagegroup packagegroup-core-standalone-sdk-target-dbg has renamed dependency set(['libgcc-s-dbg', 'eglibc-dbg', 'libc6-dbg', 'libgcc-dbg'])
WARNING: allarch packagegroup packagegroup-core-standalone-sdk-target-dev has renamed dependency set(['libgcc-s-dev', 'libc6-dev', 'libgcc-dev', 'eglibc-dev'])
WARNING: allarch packagegroup packagegroup-core-x11-sato-base has renamed dependency set(['libsdl-1.2-0', 'libsdl'])
WARNING: allarch packagegroup packagegroup-core-x11-sato-base-dbg has renamed dependency set(['libsdl-dbg', 'libsdl-1.2-dbg'])
WARNING: allarch packagegroup packagegroup-core-x11-sato-base-dev has renamed dependency set(['libsdl-dev', 'libsdl-1.2-dev'])
WARNING: allarch packagegroup packagegroup-core-sdk has renamed dependency set(['libgomp', 'libgomp1'])
WARNING: allarch packagegroup packagegroup-core-x11-base has renamed dependency set(['dbus-1', 'dbus'])
WARNING: allarch packagegroup packagegroup-core-standalone-sdk-target has renamed dependency set(['libgcc1', 'libgcc-s-dev', 'libstdc++', 'libc6-dev', 'libgcc-dev', 'libgcc', 'libc6', 'eglibc-dbg', 'libc6-dbg', 'eglibc-dev', 'eglibc-thread-db', 'eglibc', 'libthread-db1', 'libstdc++6'])
WARNING: allarch packagegroup packagegroup-core-x11-sato-base has renamed dependency set(['libsdl-1.2-0', 'libsdl'])
WARNING: allarch packagegroup packagegroup-core-standalone-sdk-target-dbg has renamed dependency set(['libgcc-s-dbg', 'eglibc-dbg', 'libc6-dbg', 'libgcc-dbg'])
WARNING: allarch packagegroup packagegroup-core-standalone-sdk-target-dev has renamed dependency set(['libgcc-s-dev', 'libc6-dev', 'libgcc-dev', 'eglibc-dev'])
WARNING: allarch packagegroup packagegroup-core-buildessential has renamed dependency set(['libstdc++', 'libstdc++6'])
WARNING: allarch packagegroup packagegroup-core-buildessential has renamed dependency set(['libstdc++', 'libstdc++6'])

so we do have the problem in a number of places.

Does anyone have any thoughts?

Cheers,

Richard

Re: [RFC] Allarch and packagegroup improvement proposal

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 5377 bytes --]

On Mon, Aug 18, 2014 at 01:46:50PM +0100, Richard Purdie wrote:
> As some people are already painfully aware, the current way packagegroup
> and allarch interact with the sstate signatures is painful.
> 
> Some example problem cases:
> 
> * An allarch package of scripts with an interpretor dependency (e.g perl)
> 
> * A packagegroup with dependencies on something which is debian renamed 
>   e.g. build-essentials on libgmp 
> 
> 
> Currently, packagegroups default to allarch and have their sstate
> dependencies truncated. This means that when something like gmp
> rebuilds, it can change package name thanks to debian renaming but the
> packagegroup referencing the old name remains, causing image creation
> failures.
> 
> So the "obvious" fix is to stop truncating the dependencies? Well, that
> means that the allarch package is written out every time machine changes
> to a different arch.
> 
> So we need to make packagegroups PACKAGE_ARCH specific? Well, this still
> doesn't solve the problem of allarch packages having ever greater
> problems with things like dependencies on perl.

Well then maybe that allarch package with perl dependency shouldn't be
marked as allarch.

> An alternative solution I'm wondering about to address both issues is:
> 
> * the sstatesig packagegroup code for dependencies handling should only
> trigger for allarch 
> * we add a sanity test that checks if renaming has happened for
> packagegroup dependencies and warn if it has.
> 
> The trouble is the sstatesig code doesn't have the full data store, only
> a limited cross section so it can't check if PACKAGE_ARCH == all, only
> if the class is inherited. We therefore probably need to change
> packagegroup to do conditional class inclusion (which is possible with
> the modern code base at least). The use of the class can be tested from
> the sstatesig code and hence indicate allarch.
> 
> 
> For the second part, we could add something like:
> 
> +    if bb.data.inherits_class('packagegroup', d) and bb.data.inherits_class('allarch', d):    
> +        if set(deps) != set(new_depends):
> +            bb.warn("allarch packagegroup %s has renamed dependency %s" % (pkg, str(set(deps) ^ set(new_depends)))) 
> 
> to runtime_mapping_rename in package.bbclass which tells us which
> packagegroups need to move to be PACKAGE_ARCH. Unfortunately, a quick
> run of OE-Core shows:
> 
> WARNING: allarch packagegroup packagegroup-core-sdk has renamed dependency set(['libgomp', 'libgomp1'])
> WARNING: allarch packagegroup packagegroup-core-x11-base has renamed dependency set(['dbus-1', 'dbus'])
> WARNING: allarch packagegroup packagegroup-core-standalone-sdk-target has renamed dependency set(['libgcc1', 'libgcc-s-dev', 'libstdc++', 'libc6-dev', 'libgcc-dev', 'libgcc', 'libc6', 'eglibc-dbg', 'libc6-dbg', 'eglibc-dev', 'eglibc-thread-db', 'eglibc', 'libthread-db1', 'libstdc++6'])
> WARNING: allarch packagegroup packagegroup-core-standalone-sdk-target-dbg has renamed dependency set(['libgcc-s-dbg', 'eglibc-dbg', 'libc6-dbg', 'libgcc-dbg'])
> WARNING: allarch packagegroup packagegroup-core-standalone-sdk-target-dev has renamed dependency set(['libgcc-s-dev', 'libc6-dev', 'libgcc-dev', 'eglibc-dev'])
> WARNING: allarch packagegroup packagegroup-core-x11-sato-base has renamed dependency set(['libsdl-1.2-0', 'libsdl'])
> WARNING: allarch packagegroup packagegroup-core-x11-sato-base-dbg has renamed dependency set(['libsdl-dbg', 'libsdl-1.2-dbg'])
> WARNING: allarch packagegroup packagegroup-core-x11-sato-base-dev has renamed dependency set(['libsdl-dev', 'libsdl-1.2-dev'])
> WARNING: allarch packagegroup packagegroup-core-sdk has renamed dependency set(['libgomp', 'libgomp1'])
> WARNING: allarch packagegroup packagegroup-core-x11-base has renamed dependency set(['dbus-1', 'dbus'])
> WARNING: allarch packagegroup packagegroup-core-standalone-sdk-target has renamed dependency set(['libgcc1', 'libgcc-s-dev', 'libstdc++', 'libc6-dev', 'libgcc-dev', 'libgcc', 'libc6', 'eglibc-dbg', 'libc6-dbg', 'eglibc-dev', 'eglibc-thread-db', 'eglibc', 'libthread-db1', 'libstdc++6'])
> WARNING: allarch packagegroup packagegroup-core-x11-sato-base has renamed dependency set(['libsdl-1.2-0', 'libsdl'])
> WARNING: allarch packagegroup packagegroup-core-standalone-sdk-target-dbg has renamed dependency set(['libgcc-s-dbg', 'eglibc-dbg', 'libc6-dbg', 'libgcc-dbg'])
> WARNING: allarch packagegroup packagegroup-core-standalone-sdk-target-dev has renamed dependency set(['libgcc-s-dev', 'libc6-dev', 'libgcc-dev', 'eglibc-dev'])
> WARNING: allarch packagegroup packagegroup-core-buildessential has renamed dependency set(['libstdc++', 'libstdc++6'])
> WARNING: allarch packagegroup packagegroup-core-buildessential has renamed dependency set(['libstdc++', 'libstdc++6'])
> 
> so we do have the problem in a number of places.
> 
> Does anyone have any thoughts?

I'm in favor of removing default allarch and setting correct
PACKAGE_ARCH manually in the packagegroup recipes like we do elsewhere.

packagegroups are small and "rebuilt" quickly, so I don't mind
"building" them once per TUNE_PKGARCH or even once per MACHINE_ARCH like
we do for couple of them already.

I can even find few changes from me on ML which do exactly that.

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 188 bytes --]

Re: [RFC] Allarch and packagegroup improvement proposal

[Robert Yang]

On 08/18/2014 09:14 PM, Martin Jansa wrote:
> On Mon, Aug 18, 2014 at 01:46:50PM +0100, Richard Purdie wrote:
>> As some people are already painfully aware, the current way packagegroup
>> and allarch interact with the sstate signatures is painful.
>>
>> Some example problem cases:
>>
>> * An allarch package of scripts with an interpretor dependency (e.g perl)
>>
>> * A packagegroup with dependencies on something which is debian renamed
>>    e.g. build-essentials on libgmp
>>
>>
>> Currently, packagegroups default to allarch and have their sstate
>> dependencies truncated. This means that when something like gmp
>> rebuilds, it can change package name thanks to debian renaming but the
>> packagegroup referencing the old name remains, causing image creation
>> failures.
>>
>> So the "obvious" fix is to stop truncating the dependencies? Well, that
>> means that the allarch package is written out every time machine changes
>> to a different arch.
>>
>> So we need to make packagegroups PACKAGE_ARCH specific? Well, this still
>> doesn't solve the problem of allarch packages having ever greater
>> problems with things like dependencies on perl.
>
> Well then maybe that allarch package with perl dependency shouldn't be
> marked as allarch.
>
>> An alternative solution I'm wondering about to address both issues is:
>>
>> * the sstatesig packagegroup code for dependencies handling should only
>> trigger for allarch
>> * we add a sanity test that checks if renaming has happened for
>> packagegroup dependencies and warn if it has.
>>
>> The trouble is the sstatesig code doesn't have the full data store, only
>> a limited cross section so it can't check if PACKAGE_ARCH == all, only
>> if the class is inherited. We therefore probably need to change
>> packagegroup to do conditional class inclusion (which is possible with
>> the modern code base at least). The use of the class can be tested from
>> the sstatesig code and hence indicate allarch.
>>
>>
>> For the second part, we could add something like:
>>
>> +    if bb.data.inherits_class('packagegroup', d) and bb.data.inherits_class('allarch', d):
>> +        if set(deps) != set(new_depends):
>> +            bb.warn("allarch packagegroup %s has renamed dependency %s" % (pkg, str(set(deps) ^ set(new_depends))))
>>
>> to runtime_mapping_rename in package.bbclass which tells us which
>> packagegroups need to move to be PACKAGE_ARCH. Unfortunately, a quick
>> run of OE-Core shows:
>>
>> WARNING: allarch packagegroup packagegroup-core-sdk has renamed dependency set(['libgomp', 'libgomp1'])
>> WARNING: allarch packagegroup packagegroup-core-x11-base has renamed dependency set(['dbus-1', 'dbus'])
>> WARNING: allarch packagegroup packagegroup-core-standalone-sdk-target has renamed dependency set(['libgcc1', 'libgcc-s-dev', 'libstdc++', 'libc6-dev', 'libgcc-dev', 'libgcc', 'libc6', 'eglibc-dbg', 'libc6-dbg', 'eglibc-dev', 'eglibc-thread-db', 'eglibc', 'libthread-db1', 'libstdc++6'])
>> WARNING: allarch packagegroup packagegroup-core-standalone-sdk-target-dbg has renamed dependency set(['libgcc-s-dbg', 'eglibc-dbg', 'libc6-dbg', 'libgcc-dbg'])
>> WARNING: allarch packagegroup packagegroup-core-standalone-sdk-target-dev has renamed dependency set(['libgcc-s-dev', 'libc6-dev', 'libgcc-dev', 'eglibc-dev'])
>> WARNING: allarch packagegroup packagegroup-core-x11-sato-base has renamed dependency set(['libsdl-1.2-0', 'libsdl'])
>> WARNING: allarch packagegroup packagegroup-core-x11-sato-base-dbg has renamed dependency set(['libsdl-dbg', 'libsdl-1.2-dbg'])
>> WARNING: allarch packagegroup packagegroup-core-x11-sato-base-dev has renamed dependency set(['libsdl-dev', 'libsdl-1.2-dev'])
>> WARNING: allarch packagegroup packagegroup-core-sdk has renamed dependency set(['libgomp', 'libgomp1'])
>> WARNING: allarch packagegroup packagegroup-core-x11-base has renamed dependency set(['dbus-1', 'dbus'])
>> WARNING: allarch packagegroup packagegroup-core-standalone-sdk-target has renamed dependency set(['libgcc1', 'libgcc-s-dev', 'libstdc++', 'libc6-dev', 'libgcc-dev', 'libgcc', 'libc6', 'eglibc-dbg', 'libc6-dbg', 'eglibc-dev', 'eglibc-thread-db', 'eglibc', 'libthread-db1', 'libstdc++6'])
>> WARNING: allarch packagegroup packagegroup-core-x11-sato-base has renamed dependency set(['libsdl-1.2-0', 'libsdl'])
>> WARNING: allarch packagegroup packagegroup-core-standalone-sdk-target-dbg has renamed dependency set(['libgcc-s-dbg', 'eglibc-dbg', 'libc6-dbg', 'libgcc-dbg'])
>> WARNING: allarch packagegroup packagegroup-core-standalone-sdk-target-dev has renamed dependency set(['libgcc-s-dev', 'libc6-dev', 'libgcc-dev', 'eglibc-dev'])
>> WARNING: allarch packagegroup packagegroup-core-buildessential has renamed dependency set(['libstdc++', 'libstdc++6'])
>> WARNING: allarch packagegroup packagegroup-core-buildessential has renamed dependency set(['libstdc++', 'libstdc++6'])
>>
>> so we do have the problem in a number of places.
>>
>> Does anyone have any thoughts?
>
> I'm in favor of removing default allarch and setting correct
> PACKAGE_ARCH manually in the packagegroup recipes like we do elsewhere.

I did a rough find and grep in oe-core, we have 27 packagegroup recipes:
$ find meta -name 'packagegroup-*.bb' | wc -l
27

And 7 ones which set the PACKAGE_ARCH:
$ grep PACKAGE_ARCH meta -r | grep 'packagegroup-.*.bb:' | wc -l
7

It seems that allarch still wins.

I think that we really need such a WARNING not matter whether remove the
default allarch or not. (if removed, we have to set the allarch for a few
packagegroup recipes, then we still need such a check.)

// Robert

>
> packagegroups are small and "rebuilt" quickly, so I don't mind
> "building" them once per TUNE_PKGARCH or even once per MACHINE_ARCH like
> we do for couple of them already.
>
> I can even find few changes from me on ML which do exactly that.
>
>
>

Re: [RFC] Allarch and packagegroup improvement proposal

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 6212 bytes --]

On Mon, Aug 18, 2014 at 09:42:50PM +0800, Robert Yang wrote:
> 
> 
> On 08/18/2014 09:14 PM, Martin Jansa wrote:
> > On Mon, Aug 18, 2014 at 01:46:50PM +0100, Richard Purdie wrote:
> >> As some people are already painfully aware, the current way packagegroup
> >> and allarch interact with the sstate signatures is painful.
> >>
> >> Some example problem cases:
> >>
> >> * An allarch package of scripts with an interpretor dependency (e.g perl)
> >>
> >> * A packagegroup with dependencies on something which is debian renamed
> >>    e.g. build-essentials on libgmp
> >>
> >>
> >> Currently, packagegroups default to allarch and have their sstate
> >> dependencies truncated. This means that when something like gmp
> >> rebuilds, it can change package name thanks to debian renaming but the
> >> packagegroup referencing the old name remains, causing image creation
> >> failures.
> >>
> >> So the "obvious" fix is to stop truncating the dependencies? Well, that
> >> means that the allarch package is written out every time machine changes
> >> to a different arch.
> >>
> >> So we need to make packagegroups PACKAGE_ARCH specific? Well, this still
> >> doesn't solve the problem of allarch packages having ever greater
> >> problems with things like dependencies on perl.
> >
> > Well then maybe that allarch package with perl dependency shouldn't be
> > marked as allarch.
> >
> >> An alternative solution I'm wondering about to address both issues is:
> >>
> >> * the sstatesig packagegroup code for dependencies handling should only
> >> trigger for allarch
> >> * we add a sanity test that checks if renaming has happened for
> >> packagegroup dependencies and warn if it has.
> >>
> >> The trouble is the sstatesig code doesn't have the full data store, only
> >> a limited cross section so it can't check if PACKAGE_ARCH == all, only
> >> if the class is inherited. We therefore probably need to change
> >> packagegroup to do conditional class inclusion (which is possible with
> >> the modern code base at least). The use of the class can be tested from
> >> the sstatesig code and hence indicate allarch.
> >>
> >>
> >> For the second part, we could add something like:
> >>
> >> +    if bb.data.inherits_class('packagegroup', d) and bb.data.inherits_class('allarch', d):
> >> +        if set(deps) != set(new_depends):
> >> +            bb.warn("allarch packagegroup %s has renamed dependency %s" % (pkg, str(set(deps) ^ set(new_depends))))
> >>
> >> to runtime_mapping_rename in package.bbclass which tells us which
> >> packagegroups need to move to be PACKAGE_ARCH. Unfortunately, a quick
> >> run of OE-Core shows:
> >>
> >> WARNING: allarch packagegroup packagegroup-core-sdk has renamed dependency set(['libgomp', 'libgomp1'])
> >> WARNING: allarch packagegroup packagegroup-core-x11-base has renamed dependency set(['dbus-1', 'dbus'])
> >> WARNING: allarch packagegroup packagegroup-core-standalone-sdk-target has renamed dependency set(['libgcc1', 'libgcc-s-dev', 'libstdc++', 'libc6-dev', 'libgcc-dev', 'libgcc', 'libc6', 'eglibc-dbg', 'libc6-dbg', 'eglibc-dev', 'eglibc-thread-db', 'eglibc', 'libthread-db1', 'libstdc++6'])
> >> WARNING: allarch packagegroup packagegroup-core-standalone-sdk-target-dbg has renamed dependency set(['libgcc-s-dbg', 'eglibc-dbg', 'libc6-dbg', 'libgcc-dbg'])
> >> WARNING: allarch packagegroup packagegroup-core-standalone-sdk-target-dev has renamed dependency set(['libgcc-s-dev', 'libc6-dev', 'libgcc-dev', 'eglibc-dev'])
> >> WARNING: allarch packagegroup packagegroup-core-x11-sato-base has renamed dependency set(['libsdl-1.2-0', 'libsdl'])
> >> WARNING: allarch packagegroup packagegroup-core-x11-sato-base-dbg has renamed dependency set(['libsdl-dbg', 'libsdl-1.2-dbg'])
> >> WARNING: allarch packagegroup packagegroup-core-x11-sato-base-dev has renamed dependency set(['libsdl-dev', 'libsdl-1.2-dev'])
> >> WARNING: allarch packagegroup packagegroup-core-sdk has renamed dependency set(['libgomp', 'libgomp1'])
> >> WARNING: allarch packagegroup packagegroup-core-x11-base has renamed dependency set(['dbus-1', 'dbus'])
> >> WARNING: allarch packagegroup packagegroup-core-standalone-sdk-target has renamed dependency set(['libgcc1', 'libgcc-s-dev', 'libstdc++', 'libc6-dev', 'libgcc-dev', 'libgcc', 'libc6', 'eglibc-dbg', 'libc6-dbg', 'eglibc-dev', 'eglibc-thread-db', 'eglibc', 'libthread-db1', 'libstdc++6'])
> >> WARNING: allarch packagegroup packagegroup-core-x11-sato-base has renamed dependency set(['libsdl-1.2-0', 'libsdl'])
> >> WARNING: allarch packagegroup packagegroup-core-standalone-sdk-target-dbg has renamed dependency set(['libgcc-s-dbg', 'eglibc-dbg', 'libc6-dbg', 'libgcc-dbg'])
> >> WARNING: allarch packagegroup packagegroup-core-standalone-sdk-target-dev has renamed dependency set(['libgcc-s-dev', 'libc6-dev', 'libgcc-dev', 'eglibc-dev'])
> >> WARNING: allarch packagegroup packagegroup-core-buildessential has renamed dependency set(['libstdc++', 'libstdc++6'])
> >> WARNING: allarch packagegroup packagegroup-core-buildessential has renamed dependency set(['libstdc++', 'libstdc++6'])
> >>
> >> so we do have the problem in a number of places.
> >>
> >> Does anyone have any thoughts?
> >
> > I'm in favor of removing default allarch and setting correct
> > PACKAGE_ARCH manually in the packagegroup recipes like we do elsewhere.
> 
> I did a rough find and grep in oe-core, we have 27 packagegroup recipes:
> $ find meta -name 'packagegroup-*.bb' | wc -l
> 27
> 
> And 7 ones which set the PACKAGE_ARCH:
> $ grep PACKAGE_ARCH meta -r | grep 'packagegroup-.*.bb:' | wc -l
> 7
> 
> It seems that allarch still wins.
> 
> I think that we really need such a WARNING not matter whether remove the
> default allarch or not. (if removed, we have to set the allarch for a few
> packagegroup recipes, then we still need such a check.)

FWIW: there is also scripts/sstate-diff-machines.sh which can easily
show when some packagegroup recipe has incorrect PACKAGE_ARCH

Example output:
http://lists.openembedded.org/pipermail/openembedded-core/2014-August/095195.html

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 188 bytes --]

Re: [RFC] Allarch and packagegroup improvement proposal

[Richard Purdie]

On Mon, 2014-08-18 at 15:14 +0200, Martin Jansa wrote:
> On Mon, Aug 18, 2014 at 01:46:50PM +0100, Richard Purdie wrote:
> > As some people are already painfully aware, the current way packagegroup
> > and allarch interact with the sstate signatures is painful.
> > 
> > Some example problem cases:
> > 
> > * An allarch package of scripts with an interpretor dependency (e.g perl)
> > 
> > * A packagegroup with dependencies on something which is debian renamed 
> >   e.g. build-essentials on libgmp 
> > 
> > 
> > Currently, packagegroups default to allarch and have their sstate
> > dependencies truncated. This means that when something like gmp
> > rebuilds, it can change package name thanks to debian renaming but the
> > packagegroup referencing the old name remains, causing image creation
> > failures.
> > 
> > So the "obvious" fix is to stop truncating the dependencies? Well, that
> > means that the allarch package is written out every time machine changes
> > to a different arch.
> > 
> > So we need to make packagegroups PACKAGE_ARCH specific? Well, this still
> > doesn't solve the problem of allarch packages having ever greater
> > problems with things like dependencies on perl.
> 
> Well then maybe that allarch package with perl dependency shouldn't be
> marked as allarch.

Take a step back and think about this from the end user system
perspective. The packages are all identical for each architecture,
"perl" doesn't change name. 

To me that means the correct end result is such a package should be
"allarch" in the package feeds.

The question then becomes, how do we generate such things in a sane way.

> I'm in favor of removing default allarch and setting correct
> PACKAGE_ARCH manually in the packagegroup recipes like we do elsewhere.
> 
> packagegroups are small and "rebuilt" quickly, so I don't mind
> "building" them once per TUNE_PKGARCH or even once per MACHINE_ARCH like
> we do for couple of them already.
> 
> I can even find few changes from me on ML which do exactly that.

It does seem a bit of a cop out to do this on the grounds that its
small/fast :/.

I agree there is good reason why some should be PKGARCH but we can
probably do better than just marking them all that way IMO. I think we
should try and mark them correctly too, i.e. think about whether the
packages really are identical and/or make sense as allarch and try and
avoid duplication if so.

Cheers,

Richard

Re: [RFC] Allarch and packagegroup improvement proposal

[Richard Purdie]

On Mon, 2014-08-18 at 15:39 +0100, Richard Purdie wrote:
> On Mon, 2014-08-18 at 15:14 +0200, Martin Jansa wrote:
> > Well then maybe that allarch package with perl dependency shouldn't be
> > marked as allarch.
> 
> Take a step back and think about this from the end user system
> perspective. The packages are all identical for each architecture,
> "perl" doesn't change name. 
> 
> To me that means the correct end result is such a package should be
> "allarch" in the package feeds.
> 
> The question then becomes, how do we generate such things in a sane way.
> 
> > I'm in favor of removing default allarch and setting correct
> > PACKAGE_ARCH manually in the packagegroup recipes like we do elsewhere.
> > 
> > packagegroups are small and "rebuilt" quickly, so I don't mind
> > "building" them once per TUNE_PKGARCH or even once per MACHINE_ARCH like
> > we do for couple of them already.
> > 
> > I can even find few changes from me on ML which do exactly that.
> 
> It does seem a bit of a cop out to do this on the grounds that its
> small/fast :/.
> 
> I agree there is good reason why some should be PKGARCH but we can
> probably do better than just marking them all that way IMO. I think we
> should try and mark them correctly too, i.e. think about whether the
> packages really are identical and/or make sense as allarch and try and
> avoid duplication if so.

I also have one other idea. We could adjust debian.bbclass so that it
adds an RPROVIDES for the original package name. We could then instruct
packagegroups specifically not to adjust the naming for debian renaming.

This would mean that the packagegroups would know the dependencies by
their non-debian, unversioned name.

Would that work for people?

I'm torn on the idea right now but thought I'd share it.

I did look into and have code for making the allarch inherit
conditional, the only issue is that you need to set PACKAGE_ARCH before
the inherit packagegroup and no recipes currently do this, its exactly
the opposite situation.

Cheers,

Richard

Re: [RFC] Allarch and packagegroup improvement proposal

[Koen Kooi]

Op 18 aug. 2014, om 17:35 heeft Richard Purdie <richard.purdie@linuxfoundation.org> het volgende geschreven:

> On Mon, 2014-08-18 at 15:39 +0100, Richard Purdie wrote:
>> On Mon, 2014-08-18 at 15:14 +0200, Martin Jansa wrote:
>>> Well then maybe that allarch package with perl dependency shouldn't be
>>> marked as allarch.
>> 
>> Take a step back and think about this from the end user system
>> perspective. The packages are all identical for each architecture,
>> "perl" doesn't change name. 
>> 
>> To me that means the correct end result is such a package should be
>> "allarch" in the package feeds.
>> 
>> The question then becomes, how do we generate such things in a sane way.
>> 
>>> I'm in favor of removing default allarch and setting correct
>>> PACKAGE_ARCH manually in the packagegroup recipes like we do elsewhere.
>>> 
>>> packagegroups are small and "rebuilt" quickly, so I don't mind
>>> "building" them once per TUNE_PKGARCH or even once per MACHINE_ARCH like
>>> we do for couple of them already.
>>> 
>>> I can even find few changes from me on ML which do exactly that.
>> 
>> It does seem a bit of a cop out to do this on the grounds that its
>> small/fast :/.
>> 
>> I agree there is good reason why some should be PKGARCH but we can
>> probably do better than just marking them all that way IMO. I think we
>> should try and mark them correctly too, i.e. think about whether the
>> packages really are identical and/or make sense as allarch and try and
>> avoid duplication if so.
> 
> I also have one other idea. We could adjust debian.bbclass so that it
> adds an RPROVIDES for the original package name. We could then instruct
> packagegroups specifically not to adjust the naming for debian renaming.
> 
> This would mean that the packagegroups would know the dependencies by
> their non-debian, unversioned name.
> 
> Would that work for people?
> 
> I'm torn on the idea right now but thought I'd share it.

What happens in the following situation:

foo.bb generates foo.ipk (/usr/bin/foo) and libfoo5.ipk (/usr/lib/libfoo.so.5), will libfoo5 RPROVIDE 'foo', 'libfoo' or something else?

regards,

Koen

Re: [RFC] Allarch and packagegroup improvement proposal

[Richard Purdie]

On Tue, 2014-08-19 at 12:41 +0200, Koen Kooi wrote:
> Op 18 aug. 2014, om 17:35 heeft Richard Purdie <richard.purdie@linuxfoundation.org> het volgende geschreven:
> 
> > On Mon, 2014-08-18 at 15:39 +0100, Richard Purdie wrote:
> >> On Mon, 2014-08-18 at 15:14 +0200, Martin Jansa wrote:
> >>> Well then maybe that allarch package with perl dependency shouldn't be
> >>> marked as allarch.
> >> 
> >> Take a step back and think about this from the end user system
> >> perspective. The packages are all identical for each architecture,
> >> "perl" doesn't change name. 
> >> 
> >> To me that means the correct end result is such a package should be
> >> "allarch" in the package feeds.
> >> 
> >> The question then becomes, how do we generate such things in a sane way.
> >> 
> >>> I'm in favor of removing default allarch and setting correct
> >>> PACKAGE_ARCH manually in the packagegroup recipes like we do elsewhere.
> >>> 
> >>> packagegroups are small and "rebuilt" quickly, so I don't mind
> >>> "building" them once per TUNE_PKGARCH or even once per MACHINE_ARCH like
> >>> we do for couple of them already.
> >>> 
> >>> I can even find few changes from me on ML which do exactly that.
> >> 
> >> It does seem a bit of a cop out to do this on the grounds that its
> >> small/fast :/.
> >> 
> >> I agree there is good reason why some should be PKGARCH but we can
> >> probably do better than just marking them all that way IMO. I think we
> >> should try and mark them correctly too, i.e. think about whether the
> >> packages really are identical and/or make sense as allarch and try and
> >> avoid duplication if so.
> > 
> > I also have one other idea. We could adjust debian.bbclass so that it
> > adds an RPROVIDES for the original package name. We could then instruct
> > packagegroups specifically not to adjust the naming for debian renaming.
> > 
> > This would mean that the packagegroups would know the dependencies by
> > their non-debian, unversioned name.
> > 
> > Would that work for people?
> > 
> > I'm torn on the idea right now but thought I'd share it.
> 
> What happens in the following situation:
> 
> foo.bb generates foo.ipk (/usr/bin/foo) and libfoo5.ipk
> (/usr/lib/libfoo.so.5), will libfoo5 RPROVIDE 'foo', 'libfoo' or
> something else?

libfoo

Its very unlikely someone would write a package which the namespaces
conflicted in the way we use things currently.

The other related question is whether we should be doing automatic
remapping in RPROVIDES/RREPLACES/RCONFLICTS ?

Since at the moment if we put libfoo into RPROVIDES, it gets mapped to
libfoo5 by the remapping code...

I have patches which make various things work, I'll probably post them
and people can see what they think. 

Cheers,

Richard