[PATCH] linux-libc-headers: exclude drm headers from sysroot

[PATCH] linux-libc-headers: exclude drm headers from sysroot

[Paul Gortmaker]

While diagnosing a problem with xf86-video-intel I noticed we had two
copies of drm headers in the sysroot; one from here and one from
the libdrm package.   The xf86-video-intel turned out to be another
thing, but that doesn't mean we want two copies in the sysroot with
different content and luck of include path indicating which one we
get.

This one landed in usr/include/drm and the libdrm one put its files
at usr/include/libdrm, so there was no obvious over-write conflict.

The obvious risk here would be unearthing implicit dependencies on
the libdrm; things trying to build before it has populated the sysroot
but two full highly parallel builds containing a full desktop graphics
suite did not show any issues.

Cc: Bruce Ashfield <bruce.ashfield@windriver.com>
Cc: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
---
 meta/recipes-kernel/linux-libc-headers/linux-libc-headers.inc | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/meta/recipes-kernel/linux-libc-headers/linux-libc-headers.inc b/meta/recipes-kernel/linux-libc-headers/linux-libc-headers.inc
index 566bc1e6154b..bda9d3f765ed 100644
--- a/meta/recipes-kernel/linux-libc-headers/linux-libc-headers.inc
+++ b/meta/recipes-kernel/linux-libc-headers/linux-libc-headers.inc
@@ -61,6 +61,10 @@ do_install() {
 	# Kernel should not be exporting this header
 	rm -f ${D}${exec_prefix}/include/scsi/scsi.h
 
+	# if we take drm from the kernel, and a different set from libdrm
+	# at different paths in the sysroot, then mayhem can result.
+	rm -rf ${D}${exec_prefix}/include/drm
+
 	# The ..install.cmd conflicts between various configure runs
 	find ${D}${includedir} -name ..install.cmd | xargs rm -f
 }
-- 
1.8.2.3

Re: [PATCH] linux-libc-headers: exclude drm headers from sysroot

[Richard Purdie]

On Tue, 2015-08-04 at 09:58 -0400, Paul Gortmaker wrote:
> While diagnosing a problem with xf86-video-intel I noticed we had two
> copies of drm headers in the sysroot; one from here and one from
> the libdrm package.   The xf86-video-intel turned out to be another
> thing, but that doesn't mean we want two copies in the sysroot with
> different content and luck of include path indicating which one we
> get.
> 
> This one landed in usr/include/drm and the libdrm one put its files
> at usr/include/libdrm, so there was no obvious over-write conflict.
> 
> The obvious risk here would be unearthing implicit dependencies on
> the libdrm; things trying to build before it has populated the sysroot
> but two full highly parallel builds containing a full desktop graphics
> suite did not show any issues.
> 
> Cc: Bruce Ashfield <bruce.ashfield@windriver.com>
> Cc: Richard Purdie <richard.purdie@linuxfoundation.org>
> Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>

Is this something which should get addressed in the upstream kernel?

I agree we likely don't want two sets of those.

Cheers,

Richard

Re: [PATCH] linux-libc-headers: exclude drm headers from sysroot

[Paul Gortmaker]

On 2015-08-04 10:52 AM, Richard Purdie wrote:
> On Tue, 2015-08-04 at 09:58 -0400, Paul Gortmaker wrote:
>> While diagnosing a problem with xf86-video-intel I noticed we had two
>> copies of drm headers in the sysroot; one from here and one from
>> the libdrm package.   The xf86-video-intel turned out to be another
>> thing, but that doesn't mean we want two copies in the sysroot with
>> different content and luck of include path indicating which one we
>> get.
>>
>> This one landed in usr/include/drm and the libdrm one put its files
>> at usr/include/libdrm, so there was no obvious over-write conflict.
>>
>> The obvious risk here would be unearthing implicit dependencies on
>> the libdrm; things trying to build before it has populated the sysroot
>> but two full highly parallel builds containing a full desktop graphics
>> suite did not show any issues.
>>
>> Cc: Bruce Ashfield <bruce.ashfield@windriver.com>
>> Cc: Richard Purdie <richard.purdie@linuxfoundation.org>
>> Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
> 
> Is this something which should get addressed in the upstream kernel?

I don't think so ; my (fun!) investigation into libdrm and the commits
there seem to indicate they tend to treat the kernel as the master
repository for header content and fold changes from the uapi dir in
the kernel back into libdrm content/repository.

That said, since we (yocto) advocate people to not get all twitchy about
having the latest and greatest kernel headers, for wider compatibility,
it seemed most sensible to clobber the kernel ones and ensure the ones
we used matched the functionality of the libdrm that we are building and
actually installing.

Maybe there are arguments for going the other way, but say if we were
using the 3.19 headers still, then we'd definitely be out of sync with
the libdrm binaries we generate and deploy.

Paul.
--

> 
> I agree we likely don't want two sets of those.
> 
> Cheers,
> 
> Richard
> 
> 

Re: [PATCH] linux-libc-headers: exclude drm headers from sysroot

[Bruce Ashfield]

On Tue, Aug 4, 2015 at 11:01 AM, Paul Gortmaker
<paul.gortmaker@windriver.com> wrote:
> On 2015-08-04 10:52 AM, Richard Purdie wrote:
>> On Tue, 2015-08-04 at 09:58 -0400, Paul Gortmaker wrote:
>>> While diagnosing a problem with xf86-video-intel I noticed we had two
>>> copies of drm headers in the sysroot; one from here and one from
>>> the libdrm package.   The xf86-video-intel turned out to be another
>>> thing, but that doesn't mean we want two copies in the sysroot with
>>> different content and luck of include path indicating which one we
>>> get.
>>>
>>> This one landed in usr/include/drm and the libdrm one put its files
>>> at usr/include/libdrm, so there was no obvious over-write conflict.
>>>
>>> The obvious risk here would be unearthing implicit dependencies on
>>> the libdrm; things trying to build before it has populated the sysroot
>>> but two full highly parallel builds containing a full desktop graphics
>>> suite did not show any issues.
>>>
>>> Cc: Bruce Ashfield <bruce.ashfield@windriver.com>
>>> Cc: Richard Purdie <richard.purdie@linuxfoundation.org>
>>> Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
>>
>> Is this something which should get addressed in the upstream kernel?
>
> I don't think so ; my (fun!) investigation into libdrm and the commits
> there seem to indicate they tend to treat the kernel as the master
> repository for header content and fold changes from the uapi dir in
> the kernel back into libdrm content/repository.
>
> That said, since we (yocto) advocate people to not get all twitchy about
> having the latest and greatest kernel headers, for wider compatibility,
> it seemed most sensible to clobber the kernel ones and ensure the ones
> we used matched the functionality of the libdrm that we are building and
> actually installing.
>
> Maybe there are arguments for going the other way, but say if we were
> using the 3.19 headers still, then we'd definitely be out of sync with
> the libdrm binaries we generate and deploy.

What about a scenario where libdrm isn't being built ? I agree that we do want
the latest drm headers, and we don't want to encourage people to patch the
kernel and then expect those headers to be in linux-libc-headers ... the latest
headers can come from some sort of application specific package, like libdrm.

But if someone isn't ever building libdrm, do we want to not have those headers
in the sysroot ? We could say that any sane user of those headers has the
right dependency, but there is a chance of some silent and unexpected behaviour.

That being said, aren't there some more combinations to consider here ?

  - get libdrm to install to a different location and not clobber libc
headers ? You
    said two copies in your commit message, are they to different
places now ? or
    is libdrm going right over top of the kernel versions ?

   If we did this, applications would have to be modified to look in
the right place ..
   which really isn't a good thing either. But I mention the option anyway.

   What is the delta in libdrm ? Do they leave a half and half sort of
situation ?

 - That combination where libdrm isn't being built .. do we want
incomplete headers ?
   We could get the kernel to depend on libdrm I suppose :)

 - What about SDK builds ? We need to make sure that libdrm is in the depenency
   chain and pulled in before they are packaged up, or the headers
will be missing.

Bruce

>
> Paul.
> --
>
>>
>> I agree we likely don't want two sets of those.
>>
>> Cheers,
>>
>> Richard
>>
>>
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core



-- 
"Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end"

Re: [PATCH] linux-libc-headers: exclude drm headers from sysroot

[Khem Raj]

On Wed, Aug 5, 2015 at 7:00 AM, Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
> On Tue, Aug 4, 2015 at 11:01 AM, Paul Gortmaker
> <paul.gortmaker@windriver.com> wrote:
>> On 2015-08-04 10:52 AM, Richard Purdie wrote:
>>> On Tue, 2015-08-04 at 09:58 -0400, Paul Gortmaker wrote:
>>>> While diagnosing a problem with xf86-video-intel I noticed we had two
>>>> copies of drm headers in the sysroot; one from here and one from
>>>> the libdrm package.   The xf86-video-intel turned out to be another
>>>> thing, but that doesn't mean we want two copies in the sysroot with
>>>> different content and luck of include path indicating which one we
>>>> get.
>>>>
>>>> This one landed in usr/include/drm and the libdrm one put its files
>>>> at usr/include/libdrm, so there was no obvious over-write conflict.
>>>>
>>>> The obvious risk here would be unearthing implicit dependencies on
>>>> the libdrm; things trying to build before it has populated the sysroot
>>>> but two full highly parallel builds containing a full desktop graphics
>>>> suite did not show any issues.
>>>>
>>>> Cc: Bruce Ashfield <bruce.ashfield@windriver.com>
>>>> Cc: Richard Purdie <richard.purdie@linuxfoundation.org>
>>>> Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
>>>
>>> Is this something which should get addressed in the upstream kernel?
>>
>> I don't think so ; my (fun!) investigation into libdrm and the commits
>> there seem to indicate they tend to treat the kernel as the master
>> repository for header content and fold changes from the uapi dir in
>> the kernel back into libdrm content/repository.
>>
>> That said, since we (yocto) advocate people to not get all twitchy about
>> having the latest and greatest kernel headers, for wider compatibility,
>> it seemed most sensible to clobber the kernel ones and ensure the ones
>> we used matched the functionality of the libdrm that we are building and
>> actually installing.
>>
>> Maybe there are arguments for going the other way, but say if we were
>> using the 3.19 headers still, then we'd definitely be out of sync with
>> the libdrm binaries we generate and deploy.
>
> What about a scenario where libdrm isn't being built ? I agree that we do want
> the latest drm headers, and we don't want to encourage people to patch the
> kernel and then expect those headers to be in linux-libc-headers ... the latest
> headers can come from some sort of application specific package, like libdrm.
>
> But if someone isn't ever building libdrm, do we want to not have those headers
> in the sysroot ? We could say that any sane user of those headers has the
> right dependency, but there is a chance of some silent and unexpected behaviour.
>
> That being said, aren't there some more combinations to consider here ?
>
>   - get libdrm to install to a different location and not clobber libc
> headers ? You
>     said two copies in your commit message, are they to different
> places now ? or
>     is libdrm going right over top of the kernel versions ?
>
>    If we did this, applications would have to be modified to look in
> the right place ..
>    which really isn't a good thing either. But I mention the option anyway.
>
>    What is the delta in libdrm ? Do they leave a half and half sort of
> situation ?

I dont think you can ever make delta to be nil becauase a given X
version of libdrm might copy the headers from Y version of kernel
and our linux-libc-headers my be driven out of Z version of linux
kernel. So it will be a perpetual pain for linux-libc-headers to take
delta
based upon libdrm and slap it into kernel sources. If you want to sign
up for this thats fine but anytime libdrm is updated the patch to
kernel
headers will need to be updated and the reason we enforced it so hard
to pin linux-headers to one kernel version looses significance if we
start patching it based upon userspace packages. Unless libdrm starts
honoring backward compatibility with kernel APIs etc. its going to
remain

>
>  - That combination where libdrm isn't being built .. do we want
> incomplete headers ?
>    We could get the kernel to depend on libdrm I suppose :)
>
>  - What about SDK builds ? We need to make sure that libdrm is in the depenency
>    chain and pulled in before they are packaged up, or the headers
> will be missing.


thats only needed for packages depending upon libdrm should not be a problem.

>
> Bruce
>
>>
>> Paul.
>> --
>>
>>>
>>> I agree we likely don't want two sets of those.
>>>
>>> Cheers,
>>>
>>> Richard
>>>
>>>
>> --
>> _______________________________________________
>> Openembedded-core mailing list
>> Openembedded-core@lists.openembedded.org
>> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>
>
>
> --
> "Thou shalt not follow the NULL pointer, for chaos and madness await
> thee at its end"
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [PATCH] linux-libc-headers: exclude drm headers from sysroot

[Richard Purdie]

On Tue, 2015-08-04 at 09:58 -0400, Paul Gortmaker wrote:
> While diagnosing a problem with xf86-video-intel I noticed we had two
> copies of drm headers in the sysroot; one from here and one from
> the libdrm package.   The xf86-video-intel turned out to be another
> thing, but that doesn't mean we want two copies in the sysroot with
> different content and luck of include path indicating which one we
> get.
> 
> This one landed in usr/include/drm and the libdrm one put its files
> at usr/include/libdrm, so there was no obvious over-write conflict.
> 
> The obvious risk here would be unearthing implicit dependencies on
> the libdrm; things trying to build before it has populated the sysroot
> but two full highly parallel builds containing a full desktop graphics
> suite did not show any issues.
> 
> Cc: Bruce Ashfield <bruce.ashfield@windriver.com>
> Cc: Richard Purdie <richard.purdie@linuxfoundation.org>
> Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>

Is this something which should get addressed in the upstream kernel?

I agree we likely don't want two sets of those.

Cheers,

Richard

Re: [PATCH] linux-libc-headers: exclude drm headers from sysroot

[Khem Raj]

On Tue, Aug 4, 2015 at 6:58 AM, Paul Gortmaker
<paul.gortmaker@windriver.com> wrote:
> While diagnosing a problem with xf86-video-intel I noticed we had two
> copies of drm headers in the sysroot; one from here and one from
> the libdrm package.   The xf86-video-intel turned out to be another
> thing, but that doesn't mean we want two copies in the sysroot with
> different content and luck of include path indicating which one we
> get.
>
> This one landed in usr/include/drm and the libdrm one put its files
> at usr/include/libdrm, so there was no obvious over-write conflict.
>
> The obvious risk here would be unearthing implicit dependencies on
> the libdrm; things trying to build before it has populated the sysroot
> but two full highly parallel builds containing a full desktop graphics
> suite did not show any issues.

I dont think this is needed. Both have different namespace. Packages
needing libdrm one will either use
#include <libdrm/drm.h> or if they use #include <drm.h> will add
appropriate -I=/usr/include/drm or -I=/usr/include/libdrm
to demand for which devel packages they require. if they mix it us
then the package should be fixed. by removing this
from kernel headers we are changing the fundamental API that we have
supported like many other distros.
right now there might be packages which are using kernel-headers
instead of libdrm headers their behavior will silently change or
break.

>
> Cc: Bruce Ashfield <bruce.ashfield@windriver.com>
> Cc: Richard Purdie <richard.purdie@linuxfoundation.org>
> Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
> ---
>  meta/recipes-kernel/linux-libc-headers/linux-libc-headers.inc | 4 ++++
>  1 file changed, 4 insertions(+)
>
> diff --git a/meta/recipes-kernel/linux-libc-headers/linux-libc-headers.inc b/meta/recipes-kernel/linux-libc-headers/linux-libc-headers.inc
> index 566bc1e6154b..bda9d3f765ed 100644
> --- a/meta/recipes-kernel/linux-libc-headers/linux-libc-headers.inc
> +++ b/meta/recipes-kernel/linux-libc-headers/linux-libc-headers.inc
> @@ -61,6 +61,10 @@ do_install() {
>         # Kernel should not be exporting this header
>         rm -f ${D}${exec_prefix}/include/scsi/scsi.h
>
> +       # if we take drm from the kernel, and a different set from libdrm
> +       # at different paths in the sysroot, then mayhem can result.
> +       rm -rf ${D}${exec_prefix}/include/drm
> +
>         # The ..install.cmd conflicts between various configure runs
>         find ${D}${includedir} -name ..install.cmd | xargs rm -f
>  }
> --
> 1.8.2.3
>
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core