[RFC] Handling of virtual/* namespace in packages

[RFC] Handling of virtual/* namespace in packages

[Marcin Juszkiewicz]

In OpenedHand we got hit by virtual/update-alternatives and packaging. 
Poky contains two providers for virtual/u-a:

1. update-alternatives-cworth
2. update-alternatives-dpkg

First one comes from IPKG source and is written in pure shell. Second one 
is in Perl so it fetch too much to be usable.

The problem comes when you build both of them (it happens on our 
autobuilder due to 'world' build) - package manager (ipkg) selects random 
one - in our case it is always 'update-alternatives-dpkg' so generated 
rootfs is too big to be usable.

Solution which came to my mind is quite simple. Our distro (Poky) already 
prefers 'u-a-cworth' as provider for 'virtual/update-alternatives'. So if 
package depends on 'virtual/update-alternatives' then let package.bbclass 
mangle its dependencies in such way that 'u-a-cworth' (preferred by 
distro) would be writen in package.

What do you think about it?

-- 
JID: hrw-jabber.org
OpenEmbedded developer/consultant

         Computers are not intelligent. They only think they are.

Re: [RFC] Handling of virtual/* namespace in packages

[pHilipp Zabel]

On 10/15/07, Marcin Juszkiewicz <openembedded@haerwu.biz> wrote:
>
> In OpenedHand we got hit by virtual/update-alternatives and packaging.
> Poky contains two providers for virtual/u-a:
>
> 1. update-alternatives-cworth
> 2. update-alternatives-dpkg
>
> First one comes from IPKG source and is written in pure shell. Second one
> is in Perl so it fetch too much to be usable.
>
> The problem comes when you build both of them (it happens on our
> autobuilder due to 'world' build) - package manager (ipkg) selects random
> one - in our case it is always 'update-alternatives-dpkg' so generated
> rootfs is too big to be usable.
>
> Solution which came to my mind is quite simple. Our distro (Poky) already
> prefers 'u-a-cworth' as provider for 'virtual/update-alternatives'. So if
> package depends on 'virtual/update-alternatives' then let package.bbclass
> mangle its dependencies in such way that 'u-a-cworth' (preferred by
> distro) would be writen in package.
>
> What do you think about it?

You propose to add mangle
"Depends: virtual/u-a" into "Depends: u-a-cworth"
in the package control file in package.bbclass if there is
PREFERRED_PROVIDER_virtual/u-a = "u-a-cworth" set?

I don't think that's such a good idea, generally, as suddenly a package that
should work with any virtual/u-a can only be installed with u-a-cworth.

Maybe this mangling could be done for distro specific task packages only?

regards
Philipp

Re: [RFC] Handling of virtual/* namespace in packages

[Koen Kooi]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Marcin Juszkiewicz schreef:
> In OpenedHand we got hit by virtual/update-alternatives and packaging. 
> Poky contains two providers for virtual/u-a:
> 
> 1. update-alternatives-cworth
> 2. update-alternatives-dpkg
> 
> First one comes from IPKG source and is written in pure shell. Second one 
> is in Perl so it fetch too much to be usable.
> 
> The problem comes when you build both of them (it happens on our 
> autobuilder due to 'world' build) - package manager (ipkg) selects random 
> one - in our case it is always 'update-alternatives-dpkg' so generated 
> rootfs is too big to be usable.

Heh, I asked about that a while back and was told "everything works in poky". So much for
'working' ;)

> Solution which came to my mind is quite simple. Our distro (Poky) already 
> prefers 'u-a-cworth' as provider for 'virtual/update-alternatives'. So if 
> package depends on 'virtual/update-alternatives' then let package.bbclass 
> mangle its dependencies in such way that 'u-a-cworth' (preferred by 
> distro) would be writen in package.
> 
> What do you think about it?

Sounds like a good idea.

regards,

KOen


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iD8DBQFHE37dMkyGM64RGpERAtjNAJ48UusMliCz+nxv2sEV+RWoM3YX2wCZAahB
HnrSbCUKssdolCEWAN0W2Ng=
=I+0s
-----END PGP SIGNATURE-----

Re: [RFC] Handling of virtual/* namespace in packages

[Richard Purdie]

On Mon, 2007-10-15 at 14:19 +0200, Marcin Juszkiewicz wrote:
> In OpenedHand we got hit by virtual/update-alternatives and packaging. 
> Poky contains two providers for virtual/u-a:
> 
> 1. update-alternatives-cworth
> 2. update-alternatives-dpkg
> 
> First one comes from IPKG source and is written in pure shell. Second one 
> is in Perl so it fetch too much to be usable.
> 
> The problem comes when you build both of them (it happens on our 
> autobuilder due to 'world' build) - package manager (ipkg) selects random 
> one - in our case it is always 'update-alternatives-dpkg' so generated 
> rootfs is too big to be usable.
> 
> Solution which came to my mind is quite simple. Our distro (Poky) already 
> prefers 'u-a-cworth' as provider for 'virtual/update-alternatives'. So if 
> package depends on 'virtual/update-alternatives' then let package.bbclass 
> mangle its dependencies in such way that 'u-a-cworth' (preferred by 
> distro) would be writen in package.
> 
> What do you think about it?

I really liked this idea and I did some experiments but they had mixed
results. The problem is if we have:

RDEPENDS_${PN} = "virtual/xyz" 

then bitbake gets upset as it can't work out what to build to fulfil
that. 

We also need to avoid cross contaminating the DEPENDS and RDEPENDS
namespaces, they're separate and people *need* to think of them
separately.

I've started thinking of lots of perverse edge cases with the above too,
the problem is PREFERRED_PROVIDER is just a preference, not an absolute
and bitbake can ignore it to make a build work, especially in -k
(continue on failure) mode.

I was thinking we could have bitbake eject a set of variables:

VRPROVIDER_xyz = "abc"
VRPROVIDER_wxy = "bcd"

etc. which would tell us which provider bitbake was planning on using.
We could also adjust bitbake internals to cope with "virtual-runtime/"
variables in the RDEPENDS space but even that isn't going to be enough.
Consider:

PREFERRED_PROVIDER_virtual/xserver = "xserver-kdrive"

and bitbake setting:

VRPROVIDER_xserver = "xserver-kdrive"

None of this fixes the existing problem of how to get
"xserver-kdrive-fbdev" back and bitbake is always going to need help to
work that one out...


So perhaps we should just standardise the existing xserver behaviour
into a dedicated namespace:

DEPENDS = "virtual/xserver"
RDEPENDS_xyz = "${VIRTUAL-RUNTIME_xserver}"

and the machine.conf in this case would say

PREFERRED_PROVIDER_virtual/xserver = "xserver-kdrive"
VIRTUAL-RUNTIME_xserver = "xserver-kdrive-fbdev"

I'd love to find a nicer solution but I just can't come up with one that
will work...

Cheers,

Richard

Re: [RFC] Handling of virtual/* namespace in packages

[Koen Kooi]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Richard Purdie schreef:

> Consider:
> 
> PREFERRED_PROVIDER_virtual/xserver = "xserver-kdrive"
> 
> and bitbake setting:
> 
> VRPROVIDER_xserver = "xserver-kdrive"
> 
> None of this fixes the existing problem of how to get
> "xserver-kdrive-fbdev" back and bitbake is always going to need help to
> work that one out...
> 
> 
> So perhaps we should just standardise the existing xserver behaviour
> into a dedicated namespace:
> 
> DEPENDS = "virtual/xserver"
> RDEPENDS_xyz = "${VIRTUAL-RUNTIME_xserver}"
> 
> and the machine.conf in this case would say
> 
> PREFERRED_PROVIDER_virtual/xserver = "xserver-kdrive"
> VIRTUAL-RUNTIME_xserver = "xserver-kdrive-fbdev"

We have the ${XSERVER} var for runtime selecting it:

amsdelta.conf:XSERVER = "xserver-kdrive-fbdev"
asus730.conf:XSERVER = "xserver-kdrive" <- wrong!
bd-neon.conf:XSERVER = "xserver-kdrive-fbdev"
c7x0.conf:XSERVER = "xserver-kdrive-imageon"
er0100.conf:XSERVER = "xserver-kdrive-fbdev"
htcuniversal.conf:XSERVER = "xserver-kdrive-fbdev"
hx4700.conf:XSERVER = "xserver-kdrive-w100"
magician.conf:XSERVER = "xserver-kdrive-fbdev"
netbook-pro.conf:XSERVER = "xserver-kdrive-epson"
netvista.conf:XSERVER = "xserver-kdrive-vesa"
nokia770.conf:XSERVER = "xserver-kdrive-xomap"
nokia800.conf:XSERVER = "xserver-kdrive-xomap"

gpe-image.bb:XSERVER ?= "xserver-kdrive-fbdev"
gpephone-image-vm.bb:XSERVER ?= "xserver-kdrive-fbdev"
gpephone-image.bb:XSERVER ?= "xserver-kdrive-fbdev"
maemo-image.bb:XSERVER ?= "xserver-kdrive-omap"
sectest-gpe-image.bb:XSERVER ?= "xserver-kdrive-fbdev"
x11-gpe-image.bb:XSERVER ?= "xserver-kdrive-fbdev"
x11-image.bb:XSERVER ?= "xserver-kdrive-fbdev"
x11-office-image.bb:XSERVER ?= "xserver-kdrive-fbdev"
x11-pimlico-image.bb:XSERVER ?= "xserver-kdrive-fbdev"
xfce-image.bb:XSERVER ?= "xserver-kdrive-fbdev"
xfce-image.bb:X_RDEPENDS = "${XSERVER}"



regards,

Koen
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iD8DBQFHE61fMkyGM64RGpERAj+LAJ9D/X1UU1hdZXWCvIVskvpkIPXktgCgu5fF
rzbDzmZhE9w3Wm4L0Uad6IA=
=hBIx
-----END PGP SIGNATURE-----

Re: [RFC] Handling of virtual/* namespace in packages

[Richard Purdie]

On Mon, 2007-10-15 at 16:53 +0200, Koen Kooi wrote:
> Marcin Juszkiewicz schreef:
> > In OpenedHand we got hit by virtual/update-alternatives and packaging. 
> > Poky contains two providers for virtual/u-a:
> > 
> > 1. update-alternatives-cworth
> > 2. update-alternatives-dpkg
> > 
> > First one comes from IPKG source and is written in pure shell. Second one 
> > is in Perl so it fetch too much to be usable.
> > 
> > The problem comes when you build both of them (it happens on our 
> > autobuilder due to 'world' build) - package manager (ipkg) selects random 
> > one - in our case it is always 'update-alternatives-dpkg' so generated 
> > rootfs is too big to be usable.
> 
> Heh, I asked about that a while back and was told "everything works in poky". So much for
> 'working' ;)

Well, it did. Nothing showed any errors and it all seemed to work until
I changed some things... ;-)

Cheers,

Richard

Re: [RFC] Handling of virtual/* namespace in packages

[Richard Purdie]

On Mon, 2007-10-15 at 16:35 +0200, pHilipp Zabel wrote:
> You propose to add mangle
> "Depends: virtual/u-a" into "Depends: u-a-cworth"
> in the package control file in package.bbclass if there is
> PREFERRED_PROVIDER_virtual/u-a = "u-a-cworth" set?
> 
> I don't think that's such a good idea, generally, as suddenly a package that
> should work with any virtual/u-a can only be installed with u-a-cworth.

We have a problem already here with ipkg and friends as I don't know how
you tell them to prefer one provider over another when you have multiple
providers. At the very least we don't currently do this and we should if
its possible. Having "virtual/u-a" in the Depends will break since the
virtual/ namespace is supposed to be for DEPENDS only, not RDEPENDS as
RDEPENDS can't cope with the choice :-( (or the '/' character in
debian's case).

There is also a bug in ipkg since it will error out if you have a arm5te
package which provides update-alternatives and depends perl, an armv4t
package which also provides it but doesn't depend on perl and no perl
armv4t package exists (armv5te isn't listed as a valid arch).

> Maybe this mangling could be done for distro specific task packages only?

PREFERRED_PROVIDER is usually distro specific config so the choices
would be elected by the distro.

Cheers,

Richard

Re: [RFC] Handling of virtual/* namespace in packages

[Richard Purdie]

On Mon, 2007-10-15 at 20:11 +0200, Koen Kooi wrote:
> Richard Purdie schreef:
> > So perhaps we should just standardise the existing xserver behaviour
> > into a dedicated namespace:
> > 
> > DEPENDS = "virtual/xserver"
> > RDEPENDS_xyz = "${VIRTUAL-RUNTIME_xserver}"
> > 
> > and the machine.conf in this case would say
> > 
> > PREFERRED_PROVIDER_virtual/xserver = "xserver-kdrive"
> > VIRTUAL-RUNTIME_xserver = "xserver-kdrive-fbdev"
> 
> We have the ${XSERVER} var for runtime selecting it:

Indeed. So what should we use for update-alternatives,
${UPDATE-ALTERNATIVES} ? My point was that this is going to keep biting
us and a dedicated namespace might make sense?

Cheers,

Richard

Re: [RFC] Handling of virtual/* namespace in packages

[Koen Kooi]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Richard Purdie schreef:
> On Mon, 2007-10-15 at 20:11 +0200, Koen Kooi wrote:
>> Richard Purdie schreef:
>>> So perhaps we should just standardise the existing xserver behaviour
>>> into a dedicated namespace:
>>>
>>> DEPENDS = "virtual/xserver"
>>> RDEPENDS_xyz = "${VIRTUAL-RUNTIME_xserver}"
>>>
>>> and the machine.conf in this case would say
>>>
>>> PREFERRED_PROVIDER_virtual/xserver = "xserver-kdrive"
>>> VIRTUAL-RUNTIME_xserver = "xserver-kdrive-fbdev"
>> We have the ${XSERVER} var for runtime selecting it:
> 
> Indeed. So what should we use for update-alternatives,
> ${UPDATE-ALTERNATIVES} ? My point was that this is going to keep biting
> us and a dedicated namespace might make sense?

I'd say we copy the VIRTUAL-RUNTIME (note the OEDEM hyphen :) ) from poky for kdrive and
u-a. Any objections?

regards,

Koen
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iD8DBQFHFdCzMkyGM64RGpERAjzzAJ9NO/OysPMsAvQwoTShIhPawLm1sgCginm5
qL4BSJa9aw9FMgt7p3vzChk=
=pzAm
-----END PGP SIGNATURE-----

Re: [RFC] Handling of virtual/* namespace in packages

[Dr. Michael Lauer]

Koen Kooi wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1

> Richard Purdie schreef:
>> On Mon, 2007-10-15 at 20:11 +0200, Koen Kooi wrote:
>>> Richard Purdie schreef:
>>>> So perhaps we should just standardise the existing xserver behaviour
>>>> into a dedicated namespace:
>>>>
>>>> DEPENDS = "virtual/xserver"
>>>> RDEPENDS_xyz = "${VIRTUAL-RUNTIME_xserver}"
>>>>
>>>> and the machine.conf in this case would say
>>>>
>>>> PREFERRED_PROVIDER_virtual/xserver = "xserver-kdrive"
>>>> VIRTUAL-RUNTIME_xserver = "xserver-kdrive-fbdev"
>>> We have the ${XSERVER} var for runtime selecting it:
>> 
>> Indeed. So what should we use for update-alternatives,
>> ${UPDATE-ALTERNATIVES} ? My point was that this is going to keep biting
>> us and a dedicated namespace might make sense?

> I'd say we copy the VIRTUAL-RUNTIME (note the OEDEM hyphen   ) from poky for kdrive and
> u-a. Any objections?

Go for it.

Regards,

:M:
-- 
Michael 'Mickey' Lauer | IT-Freelancer | http://www.vanille-media.de

Re: [RFC] Handling of virtual/* namespace in packages

[Koen Kooi]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Dr. Michael Lauer schreef:
> Koen Kooi wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
> 
>> Richard Purdie schreef:
>>> On Mon, 2007-10-15 at 20:11 +0200, Koen Kooi wrote:
>>>> Richard Purdie schreef:
>>>>> So perhaps we should just standardise the existing xserver behaviour
>>>>> into a dedicated namespace:
>>>>>
>>>>> DEPENDS = "virtual/xserver"
>>>>> RDEPENDS_xyz = "${VIRTUAL-RUNTIME_xserver}"
>>>>>
>>>>> and the machine.conf in this case would say
>>>>>
>>>>> PREFERRED_PROVIDER_virtual/xserver = "xserver-kdrive"
>>>>> VIRTUAL-RUNTIME_xserver = "xserver-kdrive-fbdev"
>>>> We have the ${XSERVER} var for runtime selecting it:
>>> Indeed. So what should we use for update-alternatives,
>>> ${UPDATE-ALTERNATIVES} ? My point was that this is going to keep biting
>>> us and a dedicated namespace might make sense?
> 
>> I'd say we copy the VIRTUAL-RUNTIME (note the OEDEM hyphen   ) from poky for kdrive and
>> u-a. Any objections?

Question: glibc, eglibc and uclibc map to 'libc6' at package time, how do we get that into
meta-toolchain? I added this:

LIBC_linux = "glibc"
LIBC_linux-uclibc = "uclibc"
LIBC_linux-gnueabi = "glibc"
LIBC_linux-uclibcgnueabi = "uclibc"

which works fine for glibc and uclibc, but not for eglibc. To make it worse, we also need
(e)(g)(uc)libc-dev in the toolchain....

regards,

Koen


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iD8DBQFHFnFjMkyGM64RGpERAtYcAJ47IBLERVKRgBIVRsZXoPD7G34sOQCgprRK
8swM0hbLxajzSANUPWNBTRs=
=/NNm
-----END PGP SIGNATURE-----