python skills and bug 2412

python skills and bug 2412

[Robert Schuster]

[-- Attachment #1: Type: text/plain, Size: 468 bytes --]

Hi
Rodigo and me are suffering from bug 2412[0]. We talked a bit about the
issue and I provided an idea on how to fix it. Unfortunately I lack the
python experience to do it. I do not understand how the name conversion
in debian.bbclass is actually done and where I could add code to replace
slashes with hyphens.

Can anyone who has a better understanding of this help us out?

Regards
Robert

[0] - http://bugs.openembedded.org/show_bug.cgi?id=2412



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

Re: python skills and bug 2412

[Richard Purdie]

On Mon, 2007-11-26 at 17:55 +0100, Robert Schuster wrote:
> Hi
> Rodigo and me are suffering from bug 2412[0]. We talked a bit about the
> issue and I provided an idea on how to fix it. Unfortunately I lack the
> python experience to do it. I do not understand how the name conversion
> in debian.bbclass is actually done and where I could add code to replace
> slashes with hyphens.
> 
> Can anyone who has a better understanding of this help us out?
> 
> Regards
> Robert
> 
> [0] - http://bugs.openembedded.org/show_bug.cgi?id=2412

I commented on the bug but I will repeat here for wider exposure.

"The real problem here is that virtual/libx11 should not be appearing in
the package manager namespace. RPROVIDES = "virtual/libx11" is plain
wrong and is the real bug."

I've discussed this on the mailing list in the past, virtual/* shouldn't
ever be ending up in the runtime namespaces :/.

Cheers,

Richard

Re: python skills and bug 2412

[Paul Sokolovsky]

Hello Richard,

Tuesday, November 27, 2007, 2:22:35 PM, you wrote:

> On Mon, 2007-11-26 at 17:55 +0100, Robert Schuster wrote:
>> Hi
>> Rodigo and me are suffering from bug 2412[0]. We talked a bit about the
>> issue and I provided an idea on how to fix it. Unfortunately I lack the
>> python experience to do it. I do not understand how the name conversion
>> in debian.bbclass is actually done and where I could add code to replace
>> slashes with hyphens.
>> 
>> Can anyone who has a better understanding of this help us out?
>> 
>> Regards
>> Robert
>> 
>> [0] - http://bugs.openembedded.org/show_bug.cgi?id=2412

> I commented on the bug but I will repeat here for wider exposure.

> "The real problem here is that virtual/libx11 should not be appearing in
> the package manager namespace. RPROVIDES = "virtual/libx11" is plain
> wrong and is the real bug."

  What exactly is wrong? RPROVIDES or "virtual/" or maybe "/" after
all?

> I've discussed this on the mailing list in the past, virtual/* shouldn't
> ever be ending up in the runtime namespaces :/.

  And why is this? Virtual packages aka services aka facilities are
core feature of flexible package management. IMHO, they are heavily
underused in the current OE, which disallows creation of flexible,
adaptable, and at the same time, consistent systems.

> Cheers,

> Richard


-- 
Best regards,
 Paul                            mailto:pmiscml@gmail.com

Re: python skills and bug 2412

[Richard Purdie]

On Tue, 2007-11-27 at 15:43 +0200, Paul Sokolovsky wrote:
> Tuesday, November 27, 2007, 2:22:35 PM, you wrote:
> > On Mon, 2007-11-26 at 17:55 +0100, Robert Schuster wrote:
> > I commented on the bug but I will repeat here for wider exposure.
> 
> > "The real problem here is that virtual/libx11 should not be appearing in
> > the package manager namespace. RPROVIDES = "virtual/libx11" is plain
> > wrong and is the real bug."
> 
>   What exactly is wrong? RPROVIDES or "virtual/" or maybe "/" after
> all?

The whole idea. Using the virtual namespace in a runtime Provides is
meaningless.

This is not to be confused with PROVIDES = "virtual/*" which are
perfectly fine.

> > I've discussed this on the mailing list in the past, virtual/* shouldn't
> > ever be ending up in the runtime namespaces :/.
> 
>   And why is this? Virtual packages aka services aka facilities are
> core feature of flexible package management. IMHO, they are heavily
> underused in the current OE, which disallows creation of flexible,
> adaptable, and at the same time, consistent systems.

It doesn't work in OE as things stand which is why its underused. There
are at least two problems I can think of offhand:

a) When you compile a package against libx11 does it end up with a
Dependencies: libx11 or virtual/libx11? The answer is the former. This
means you can't switch between different providers "on device". This
means that a given distro must use one provider or the other and that
virtual/* in the runtime package management namespace is just plain
dangerous. If a distro doesn't do this, they will end up breaking their
feeds.

b) There is no way for bitbake's choices of providers to be passed to
the package manager. When faced with two virtual/libx11 packages, which
one should the package manager choose? The only possible solution at the
moment is not to get into that position or you end up with randomness in
builds (very bad). Packaged Staging could potentially help with this
FWIW. The same problem also exists in feeds which devices are accessing
so its not just isolated to the bitbake/package manager interface.

Yes, there may be ways OE/bitbake could be enhanced to do clever things
with virtual runtime namespaces but we have no current support for it
and shouldn't pretend otherwise. If someone wants to implement that, I'm
more than happy to look at, discuss and review a proposal but currently
it won't work.

A proper use for RPROVIDES is for something like gtk+-directfb to
RPROVIDE gtk+. If an app linked against one, it should be able to run
against the other (I'm assuming thats the case, if its not there
shouldn't be an RPROVIDES).

Does that help explain things?

Cheers,

Richard

Re: python skills and bug 2412

[Paul Sokolovsky]

Hello Richard,

Tuesday, November 27, 2007, 4:03:23 PM, you wrote:

> On Tue, 2007-11-27 at 15:43 +0200, Paul Sokolovsky wrote:
>> Tuesday, November 27, 2007, 2:22:35 PM, you wrote:
>> > On Mon, 2007-11-26 at 17:55 +0100, Robert Schuster wrote:
>> > I commented on the bug but I will repeat here for wider exposure.
>> 
>> > "The real problem here is that virtual/libx11 should not be appearing in
>> > the package manager namespace. RPROVIDES = "virtual/libx11" is plain
>> > wrong and is the real bug."
>> 
>>   What exactly is wrong? RPROVIDES or "virtual/" or maybe "/" after
>> all?

> The whole idea. Using the virtual namespace in a runtime Provides is
> meaningless.

[]

> A proper use for RPROVIDES is for something like gtk+-directfb to
> RPROVIDE gtk+. If an app linked against one, it should be able to run
> against the other (I'm assuming thats the case, if its not there
> shouldn't be an RPROVIDES).

> Does that help explain things?

  Ok, in other words, you say that OE's, build-time, virtuals, are
completely different thing than package manager's Provides facilities,
also common called virtuals? And that then the easiest solution is to
make sure they are never mixed at all, instead of trying to do
mind-boggling and still very limited logic of converting OE's virtuals
to package-manager virtuals? Instead, package-manager virtuals should
be managed manually, based on the actual properties of packages?

  Makes sense. But with this ambiguity of "virtuals" such questions
would come up again and again, I guess. So I also wished we cut off
OE-virtual bleeding to package manager realm soon ;-).

> Cheers,

> Richard




-- 
Best regards,
 Paul                            mailto:pmiscml@gmail.com

Re: python skills and bug 2412

[Richard Purdie]

On Tue, 2007-11-27 at 16:17 +0200, Paul Sokolovsky wrote:
> > On Tue, 2007-11-27 at 15:43 +0200, Paul Sokolovsky wrote:
> > A proper use for RPROVIDES is for something like gtk+-directfb to
> > RPROVIDE gtk+. If an app linked against one, it should be able to run
> > against the other (I'm assuming thats the case, if its not there
> > shouldn't be an RPROVIDES).
> 
> > Does that help explain things?
> 
>   Ok, in other words, you say that OE's, build-time, virtuals, are
> completely different thing than package manager's Provides facilities,
> also common called virtuals? 

Yes.

> And that then the easiest solution is to
> make sure they are never mixed at all, instead of trying to do
> mind-boggling and still very limited logic of converting OE's virtuals
> to package-manager virtuals? 

Yes.

> Instead, package-manager virtuals should
> be managed manually, based on the actual properties of packages?

Yes.

>   Makes sense. But with this ambiguity of "virtuals" such questions
> would come up again and again, I guess. So I also wished we cut off
> OE-virtual bleeding to package manager realm soon ;-).

Indeed. Thinking about this stuff makes my head hurt :).

PROVIDES + virtual/* = good
RPROVIDES + virtual/* = nightmare + bug

My point was that any of the latter will never do what people want/need
and should be removed. If we ever do that, we should use a namespace
other than "virtual" too.

Cheers,

Richard

Build-time vs run-time virtuals, was: Re: python skills and bug 2412

[Paul Sokolovsky]

Hello Richard,

Tuesday, November 27, 2007, 5:05:42 PM, you wrote:

> On Tue, 2007-11-27 at 16:17 +0200, Paul Sokolovsky wrote:
>> > On Tue, 2007-11-27 at 15:43 +0200, Paul Sokolovsky wrote:
>> > A proper use for RPROVIDES is for something like gtk+-directfb to
>> > RPROVIDE gtk+. If an app linked against one, it should be able to run
>> > against the other (I'm assuming thats the case, if its not there
>> > shouldn't be an RPROVIDES).
>> 
>> > Does that help explain things?
>> 
>>   Ok, in other words, you say that OE's, build-time, virtuals, are
>> completely different thing than package manager's Provides facilities,
>> also common called virtuals? 

> Yes.

[]

> Indeed. Thinking about this stuff makes my head hurt :).

> PROVIDES + virtual/* = good
> RPROVIDES + virtual/* = nightmare + bug

> My point was that any of the latter will never do what people want/need
> and should be removed. If we ever do that, we should use a namespace
> other than "virtual" too.

  Yes, thought about namespace came to my mind too. I wonder, if
there're any ideas regarding that. Indeed, reusing "virtual-" prefix
while being familiar, would be confusing ;-(. Any ideas about another
standard prefix, or there would be adhoc naming conventions? From the
examples given, at least sometimes it makes sense to use prefix-less
Provides names:

gtk+ - real package, also implicitly provides gtk+
gtk+-directfb - real packages, Provides gtk+

Or infamous libxine. I envision it should be:

libxine-x11, Provides libxine
libxine-qpe, Provides libxine

(The problem here is that actually built package will be linked
against real package, and thus would apparently pull its name via
shlib dependencies).

> Cheers,

> Richard



-- 
Best regards,
 Paul                            mailto:pmiscml@gmail.com

Re: python skills and bug 2412

[Matt Hoosier]

On Nov 27, 2007 8:03 AM, Richard Purdie <rpurdie@rpsys.net> wrote:
> A proper use for RPROVIDES is for something like gtk+-directfb to
> RPROVIDE gtk+. If an app linked against one, it should be able to run
> against the other (I'm assuming thats the case, if its not there
> shouldn't be an RPROVIDES).

I've tried this, and you have problems with the apps already being
linked not only against libgtk-2.0.so.x.y.z, but also the closure of
the dependencies which it had at link-time. So if the X11 version of
Gtk was staged at the time your app linked, the references to
libx11.so will exist its its binary.

Is there some way to suppress this eager resolution of transitive
shared library dependencies?

Re: python skills and bug 2412

[Richard Purdie]

On Tue, 2007-11-27 at 13:53 -0600, Matt Hoosier wrote:
> On Nov 27, 2007 8:03 AM, Richard Purdie <rpurdie@rpsys.net> wrote:
> > A proper use for RPROVIDES is for something like gtk+-directfb to
> > RPROVIDE gtk+. If an app linked against one, it should be able to run
> > against the other (I'm assuming thats the case, if its not there
> > shouldn't be an RPROVIDES).
> 
> I've tried this, and you have problems with the apps already being
> linked not only against libgtk-2.0.so.x.y.z, but also the closure of
> the dependencies which it had at link-time. So if the X11 version of
> Gtk was staged at the time your app linked, the references to
> libx11.so will exist its its binary.
> 
> Is there some way to suppress this eager resolution of transitive
> shared library dependencies?

To be honest, I don't know, its not something I've looked at before.

As Paul mentions, the shlibs code will also cause the package's
Dependencies field to become hardcoded. If the binaries have references
like you mention, that sounds like its currently doing the right thing
really...

Cheers,

Richard

Re: Build-time vs run-time virtuals, was: Re: python skills and bug 2412

[Richard Purdie]

On Tue, 2007-11-27 at 17:19 +0200, Paul Sokolovsky wrote:
>   Yes, thought about namespace came to my mind too. I wonder, if
> there're any ideas regarding that. Indeed, reusing "virtual-" prefix
> while being familiar, would be confusing ;-(. Any ideas about another
> standard prefix, or there would be adhoc naming conventions? From the
> examples given, at least sometimes it makes sense to use prefix-less
> Provides names:
> 
> gtk+ - real package, also implicitly provides gtk+
> gtk+-directfb - real packages, Provides gtk+
> 
> Or infamous libxine. I envision it should be:
> 
> libxine-x11, Provides libxine
> libxine-qpe, Provides libxine

Yes, various solutions like that are possible and probably the best way
to proceed.

> (The problem here is that actually built package will be linked
> against real package, and thus would apparently pull its name via
> shlib dependencies).

This is the real problem. If what Matt mentioned is correct about the
library names being embedded in the binaries at link time it could well
be doing the right thing though...

I guess someone interested in resolving this will have to experiment :)

Cheers,

Richard