Re: [yocto] opengl / libgl / libgles

Re: [yocto] opengl / libgl / libgles

[Burton, Ross]

(CCing oe-core, discussion should happen there really)

On 6 September 2012 11:00, Andrei Gherzan <andrei@gherzan.ro> wrote:
> In DISTRO_FEATURES we have opengl. That is pretty vague and generally we
> don't want to have mesa on machines where there is no libgl but only gles +
> egl. For example if we want to compile something that adds a DEPENDS based
> on DISTRO_FEATURE opengl, this dependency will be added even if there is no
> libgl implemented for that specific machine.
>
> First idea would be that opengl (gl / gles) are machine related. On the
> other hand opengl can be a DISTRO_FEATURE as we have a software
> implementation - mesa.
>
> How would you guys see a solution for this? The idea that came into my mind
> was to map opengl to libgl or libgles. Or to both if there is the case.

(First I must apologise for the uneven flow of this mail - my mornings
been pretty choppy but hopefully people can see what I'm thinking)

I've been thinking about this quite a lot recently, as the current
setup is broken in some way on most hardware.  There is a bigger
problem than just distro or machine features - the packaging isn't
obvious or correct either.

First, we need a concrete definition of the "opengl" distro feature.
I propose just "some form of OpenGL functionality" without any
implications about what specification is present.

Then, machine features for the portions of OpenGL that the hardware
supports - gl, gles1, gles2.  I'd love to say "gles" means GLESv2 but
I suspect we're not quite safe from the horrors of hardware that only
supports GLESv1 yet.

We then need more virtual providers for the GL libraries.  Mesa is an
obvious provider - it provides libgl, libglesv1, libglesv2 and libegl.
 On Raspberry Pi the binary drivers provide libglesv1, glesv2, and
egl.  As the performance of the software renderer will be terrible,
unsetting the libgl provider makes a lot of sense.

Because there are multiple providers of the GL libraries, and they are
not all specific to a single machine, every package that provides GL
packages need to be namespaced.  The Debian-renaming makes this
trickier but luckily it can be overridden in the recipe, so Mesa
should be building libgl-mesa, libegl-mesa, and so on.  Every other
provider of the GL libraries should do this too.

Library dependencies are also broken, of course.  A package that links
to libGL.so.1 shouldn't get a dependency on mesa-dri or whatever was
installed to provide that library, but a dependency on a virtual
"libgl" package.  As far as I'm aware this isn't something that the
automatic shlibs dependency generation handles, but there is plenty of
prior art for this.

Finally we can talk about packages.  Take xorg-xserver, this can
optionally support the GLX extension.  If the opengl distro feature is
set then it can pass --enable-glx (I've a patch for this[1] already).
GLX requires GL so on Raspberry Pi this will mean building mesa and
using the software renderer which really isn't a good idea, so that is
a strong argument for making the xserver machine-specific and checking
both the opengl distro feature *and* the gl machine feature.
(side-note: GLX is deprecated, use EGL!)

Maybe I need to spend tomorrow morning writing my scribbles down into
something more organised on the wiki, after the collective wisdom of
oe-core has pointed out why this plan is stupid.

Cheers,
Ross

[1] http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commit/?h=ross/xorg&id=e7b034379026457f291356e181c6a9ecd7fdb23b

Re: [yocto] opengl / libgl / libgles

[Koen Kooi]

Op 6 sep. 2012, om 15:04 heeft "Burton, Ross" <ross.burton@intel.com> het volgende geschreven:

> (CCing oe-core, discussion should happen there really)
> 
> On 6 September 2012 11:00, Andrei Gherzan <andrei@gherzan.ro> wrote:
>> In DISTRO_FEATURES we have opengl. That is pretty vague and generally we
>> don't want to have mesa on machines where there is no libgl but only gles +
>> egl. For example if we want to compile something that adds a DEPENDS based
>> on DISTRO_FEATURE opengl, this dependency will be added even if there is no
>> libgl implemented for that specific machine.
>> 
>> First idea would be that opengl (gl / gles) are machine related. On the
>> other hand opengl can be a DISTRO_FEATURE as we have a software
>> implementation - mesa.
>> 
>> How would you guys see a solution for this? The idea that came into my mind
>> was to map opengl to libgl or libgles. Or to both if there is the case.
> 
> (First I must apologise for the uneven flow of this mail - my mornings
> been pretty choppy but hopefully people can see what I'm thinking)
> 
> I've been thinking about this quite a lot recently, as the current
> setup is broken in some way on most hardware.  There is a bigger
> problem than just distro or machine features - the packaging isn't
> obvious or correct either.
> 
> First, we need a concrete definition of the "opengl" distro feature.
> I propose just "some form of OpenGL functionality" without any
> implications about what specification is present.
> 
> Then, machine features for the portions of OpenGL that the hardware
> supports - gl, gles1, gles2.  I'd love to say "gles" means GLESv2 but
> I suspect we're not quite safe from the horrors of hardware that only
> supports GLESv1 yet.
> 
> We then need more virtual providers for the GL libraries.  Mesa is an
> obvious provider - it provides libgl, libglesv1, libglesv2 and libegl.
> On Raspberry Pi the binary drivers provide libglesv1, glesv2, and
> egl.  As the performance of the software renderer will be terrible,
> unsetting the libgl provider makes a lot of sense.
> 
> Because there are multiple providers of the GL libraries, and they are
> not all specific to a single machine, every package that provides GL
> packages need to be namespaced.  The Debian-renaming makes this
> trickier but luckily it can be overridden in the recipe, so Mesa
> should be building libgl-mesa, libegl-mesa, and so on.  Every other
> provider of the GL libraries should do this too.
> 
> Library dependencies are also broken, of course.  A package that links
> to libGL.so.1 shouldn't get a dependency on mesa-dri or whatever was
> installed to provide that library, but a dependency on a virtual
> "libgl" package.  As far as I'm aware this isn't something that the
> automatic shlibs dependency generation handles, but there is plenty of
> prior art for this.

It's not automatic, but you can do something like this:

	http://git.yoctoproject.org/cgit/cgit.cgi/meta-ti/commit/?id=23b02134

That recipe ships multiple copies of the libraries to account for different silicon revisions (the shader compiler will offload different things to the CPU), but we don't want the shlibs code to pickup the copies. So for mesa you can do:

	PRIVATE_LIBS_${PN}-dri = "libGL.so"
	RPROVIDES_${PN}-dri = $magic_python_function(PACKAGE_CONFIG, sometthing, something, libgl)

And in the recipes that link to a GL lib you do:

	RDEPENDS_${PN} = "libGL"

Which only leaves the DISTRO to sort out the runtime preferred provider, which is likely a conf file with a bunch of PACKAGE_CONFIG entries for all the machines the DISTRO wants to support.

regards,

Koen

Re: [yocto] opengl / libgl / libgles

[Burton, Ross]

On 6 September 2012 14:25, Koen Kooi <koen@dominion.thruhere.net> wrote:
> It's not automatic, but you can do something like this:
>
>         http://git.yoctoproject.org/cgit/cgit.cgi/meta-ti/commit/?id=23b02134
>
> That recipe ships multiple copies of the libraries to account for different silicon revisions (the shader compiler will offload different things to the CPU), but we don't want the shlibs code to pickup the copies. So for mesa you can do:
>
>         PRIVATE_LIBS_${PN}-dri = "libGL.so"
>         RPROVIDES_${PN}-dri = $magic_python_function(PACKAGE_CONFIG, sometthing, something, libgl)
>
> And in the recipes that link to a GL lib you do:
>
>         RDEPENDS_${PN} = "libGL"

Ah, interesting.  I'm not entirely keen on the manual dependencies but
that's certainly a step in the right direction.

If automatic replacement dependencies were added to bitbake, how would
you implement them?  In Debian, libgl1-mesa installs a
libgl1-mesa-glx.shlibs file with this contents:

libGL 1 libgl1-mesa-glx | libgl1

"when linking to libGL.so, use the dependency 'libgl1-mesa-glx |
libg1".  All packages which contain libgl.so.1 have Provides: libgl1,
so they are interchangable.

Some way of expressing that in the bitbake recipe would be great.

Ross

Re: [yocto] opengl / libgl / libgles

[Tomas Frydrych]

On 06/09/12 14:51, Burton, Ross wrote:
> On 6 September 2012 14:25, Koen Kooi <koen@dominion.thruhere.net> wrote:
>> It's not automatic, but you can do something like this:
>>
>>         http://git.yoctoproject.org/cgit/cgit.cgi/meta-ti/commit/?id=23b02134
>>
>>         RDEPENDS_${PN} = "libGL"
> 
> Ah, interesting.  I'm not entirely keen on the manual dependencies but
> that's certainly a step in the right direction.

Is there much that actually links against libGL / libgles? Things like
cogl dlopen() these these, so you have to manually specify the RDEPENDS
anyway. Just thinking this could be less of an issue than it might appear.

Tomas

Re: [yocto] opengl / libgl / libgles

[Koen Kooi]

Op 6 sep. 2012, om 16:37 heeft Tomas Frydrych <tf+lists.yocto@r-finger.com> het volgende geschreven:

> On 06/09/12 14:51, Burton, Ross wrote:
>> On 6 September 2012 14:25, Koen Kooi <koen@dominion.thruhere.net> wrote:
>>> It's not automatic, but you can do something like this:
>>> 
>>>        http://git.yoctoproject.org/cgit/cgit.cgi/meta-ti/commit/?id=23b02134
>>> 
>>>        RDEPENDS_${PN} = "libGL"
>> 
>> Ah, interesting.  I'm not entirely keen on the manual dependencies but
>> that's certainly a step in the right direction.
> 
> Is there much that actually links against libGL / libgles? Things like
> cogl dlopen() these these, so you have to manually specify the RDEPENDS
> anyway. Just thinking this could be less of an issue than it might appear.


xbmc is one

Re: [yocto] opengl / libgl / libgles

[Burton, Ross]

On 6 September 2012 15:37, Tomas Frydrych <tf+lists.yocto@r-finger.com> wrote:
> Is there much that actually links against libGL / libgles? Things like
> cogl dlopen() these these, so you have to manually specify the RDEPENDS
> anyway. Just thinking this could be less of an issue than it might appear.

That is certainly true.

Ross

Re: [yocto] opengl / libgl / libgles

[Richard Purdie]

On Thu, 2012-09-06 at 14:51 +0100, Burton, Ross wrote:
> On 6 September 2012 14:25, Koen Kooi <koen@dominion.thruhere.net> wrote:
> > It's not automatic, but you can do something like this:
> >
> >         http://git.yoctoproject.org/cgit/cgit.cgi/meta-ti/commit/?id=23b02134
> >
> > That recipe ships multiple copies of the libraries to account for different silicon revisions (the shader compiler will offload different things to the CPU), but we don't want the shlibs code to pickup the copies. So for mesa you can do:
> >
> >         PRIVATE_LIBS_${PN}-dri = "libGL.so"
> >         RPROVIDES_${PN}-dri = $magic_python_function(PACKAGE_CONFIG, sometthing, something, libgl)
> >
> > And in the recipes that link to a GL lib you do:
> >
> >         RDEPENDS_${PN} = "libGL"
> 
> Ah, interesting.  I'm not entirely keen on the manual dependencies but
> that's certainly a step in the right direction.
> 
> If automatic replacement dependencies were added to bitbake, how would
> you implement them?  In Debian, libgl1-mesa installs a
> libgl1-mesa-glx.shlibs file with this contents:
> 
> libGL 1 libgl1-mesa-glx | libgl1
> 
> "when linking to libGL.so, use the dependency 'libgl1-mesa-glx |
> libg1".  All packages which contain libgl.so.1 have Provides: libgl1,
> so they are interchangable.
> 
> Some way of expressing that in the bitbake recipe would be great.

Note that this needs to be expressed at the core level, not recipe level
since it needs to be injected into several places and we don't want to
teach each one this. At best each would have to inherit
opengllib.bbclass.

We might just want to add a table of these and have package.bbclass add
dependencies using the table?

Yes, its suboptimal and not pretty but it would probably be very
effective.

Cheers,

Richard

Re: [yocto] opengl / libgl / libgles

[Tomas Frydrych]

On 06/09/12 15:50, Koen Kooi wrote:
> 
> Op 6 sep. 2012, om 16:37 heeft Tomas Frydrych <tf+lists.yocto@r-finger.com> het volgende geschreven:
> 
>> On 06/09/12 14:51, Burton, Ross wrote:
>>> On 6 September 2012 14:25, Koen Kooi <koen@dominion.thruhere.net> wrote:
>>>> It's not automatic, but you can do something like this:
>>>>
>>>>        http://git.yoctoproject.org/cgit/cgit.cgi/meta-ti/commit/?id=23b02134
>>>>
>>>>        RDEPENDS_${PN} = "libGL"
>>>
>>> Ah, interesting.  I'm not entirely keen on the manual dependencies but
>>> that's certainly a step in the right direction.
>>
>> Is there much that actually links against libGL / libgles? Things like
>> cogl dlopen() these these, so you have to manually specify the RDEPENDS
>> anyway. Just thinking this could be less of an issue than it might appear.
> 
> 
> xbmc is one

The problem is basically the mesa package, right? If the mesa package is
split so that the mesa libGL parts are provided separately from the more
generic parts, and any GL bits, mesa or otherwise, are only built as a
specific machine feature, then we will never have any GL bits in the
non-machine part of the sysroot, and will always be dealing with the
correct, and only, libGL. Packages that link directly to libGL would
also be machine specific, but if we are talking about a hand full
things, this is not a major issue. Or I am missing something?

Ross, I think it gets unnecessarily complicated if you start thinking of
it in the terms of hot-swapping libGL at runtime the way desktop distros do.

Tomas

Re: [yocto] opengl / libgl / libgles

[Koen Kooi]

Op 6 sep. 2012, om 18:10 heeft Tomas Frydrych <tf+lists.yocto@r-finger.com> het volgende geschreven:

> On 06/09/12 15:50, Koen Kooi wrote:
>> 
>> Op 6 sep. 2012, om 16:37 heeft Tomas Frydrych <tf+lists.yocto@r-finger.com> het volgende geschreven:
>> 
>>> On 06/09/12 14:51, Burton, Ross wrote:
>>>> On 6 September 2012 14:25, Koen Kooi <koen@dominion.thruhere.net> wrote:
>>>>> It's not automatic, but you can do something like this:
>>>>> 
>>>>>       http://git.yoctoproject.org/cgit/cgit.cgi/meta-ti/commit/?id=23b02134
>>>>> 
>>>>>       RDEPENDS_${PN} = "libGL"
>>>> 
>>>> Ah, interesting.  I'm not entirely keen on the manual dependencies but
>>>> that's certainly a step in the right direction.
>>> 
>>> Is there much that actually links against libGL / libgles? Things like
>>> cogl dlopen() these these, so you have to manually specify the RDEPENDS
>>> anyway. Just thinking this could be less of an issue than it might appear.
>> 
>> 
>> xbmc is one
> 
> The problem is basically the mesa package, right? If the mesa package is
> split so that the mesa libGL parts are provided separately from the more
> generic parts, and any GL bits, mesa or otherwise, are only built as a
> specific machine feature, then we will never have any GL bits in the
> non-machine part of the sysroot, and will always be dealing with the
> correct, and only, libGL. Packages that link directly to libGL would
> also be machine specific, but if we are talking about a hand full
> things, this is not a major issue. Or I am missing something?

No, that is pretty much exactly what I proposed earlier for mesa. 

> Ross, I think it gets unnecessarily complicated if you start thinking of
> it in the terms of hot-swapping libGL at runtime the way desktop distros do.

In theory it should work :)

regards,

Koen

Re: [yocto] opengl / libgl / libgles

[Andrei Gherzan]

[-- Attachment #1: Type: text/plain, Size: 4099 bytes --]

On Thu, Sep 6, 2012 at 4:25 PM, Koen Kooi <koen@dominion.thruhere.net>wrote:

>
> Op 6 sep. 2012, om 15:04 heeft "Burton, Ross" <ross.burton@intel.com> het
> volgende geschreven:
>
> > (CCing oe-core, discussion should happen there really)
> >
> > On 6 September 2012 11:00, Andrei Gherzan <andrei@gherzan.ro> wrote:
> >> In DISTRO_FEATURES we have opengl. That is pretty vague and generally we
> >> don't want to have mesa on machines where there is no libgl but only
> gles +
> >> egl. For example if we want to compile something that adds a DEPENDS
> based
> >> on DISTRO_FEATURE opengl, this dependency will be added even if there
> is no
> >> libgl implemented for that specific machine.
> >>
> >> First idea would be that opengl (gl / gles) are machine related. On the
> >> other hand opengl can be a DISTRO_FEATURE as we have a software
> >> implementation - mesa.
> >>
> >> How would you guys see a solution for this? The idea that came into my
> mind
> >> was to map opengl to libgl or libgles. Or to both if there is the case.
> >
> > (First I must apologise for the uneven flow of this mail - my mornings
> > been pretty choppy but hopefully people can see what I'm thinking)
> >
> > I've been thinking about this quite a lot recently, as the current
> > setup is broken in some way on most hardware.  There is a bigger
> > problem than just distro or machine features - the packaging isn't
> > obvious or correct either.
> >
> > First, we need a concrete definition of the "opengl" distro feature.
> > I propose just "some form of OpenGL functionality" without any
> > implications about what specification is present.
> >
> > Then, machine features for the portions of OpenGL that the hardware
> > supports - gl, gles1, gles2.  I'd love to say "gles" means GLESv2 but
> > I suspect we're not quite safe from the horrors of hardware that only
> > supports GLESv1 yet.
> >
> > We then need more virtual providers for the GL libraries.  Mesa is an
> > obvious provider - it provides libgl, libglesv1, libglesv2 and libegl.
> > On Raspberry Pi the binary drivers provide libglesv1, glesv2, and
> > egl.  As the performance of the software renderer will be terrible,
> > unsetting the libgl provider makes a lot of sense.
> >
> > Because there are multiple providers of the GL libraries, and they are
> > not all specific to a single machine, every package that provides GL
> > packages need to be namespaced.  The Debian-renaming makes this
> > trickier but luckily it can be overridden in the recipe, so Mesa
> > should be building libgl-mesa, libegl-mesa, and so on.  Every other
> > provider of the GL libraries should do this too.
> >
> > Library dependencies are also broken, of course.  A package that links
> > to libGL.so.1 shouldn't get a dependency on mesa-dri or whatever was
> > installed to provide that library, but a dependency on a virtual
> > "libgl" package.  As far as I'm aware this isn't something that the
> > automatic shlibs dependency generation handles, but there is plenty of
> > prior art for this.
>
> It's not automatic, but you can do something like this:
>
>
> http://git.yoctoproject.org/cgit/cgit.cgi/meta-ti/commit/?id=23b02134
>
> That recipe ships multiple copies of the libraries to account for
> different silicon revisions (the shader compiler will offload different
> things to the CPU), but we don't want the shlibs code to pickup the copies.
> So for mesa you can do:
>
>         PRIVATE_LIBS_${PN}-dri = "libGL.so"
>         RPROVIDES_${PN}-dri = $magic_python_function(PACKAGE_CONFIG,
> sometthing, something, libgl)
>
> And in the recipes that link to a GL lib you do:
>
>         RDEPENDS_${PN} = "libGL"
>
> Which only leaves the DISTRO to sort out the runtime preferred provider,
> which is likely a conf file with a bunch of PACKAGE_CONFIG entries for all
> the machines the DISTRO wants to support.
>
> I'm not sure how this would work as there are recipes like libsdl with:
${@base_contains('DISTRO_FEATURES', 'opengl', 'virtual/libgl', '', d)}

ag

[-- Attachment #2: Type: text/html, Size: 5082 bytes --]