Kernel Header UAPI Issue

Kernel Header UAPI Issue

[Karthik Poduval]

I have a 5.4 kernel where I applied a patch (using a .scc file) from
kernel 5.6 which brings in dmabuf heaps.

The patch exports a uapi header
include/uapi/linux/dma-heap.h.
I am trying to write an application that uses dmabuf with a recipe (no
special DEPENDS).

It is able to find
#include<linux/dma-buf.h>
but not
#include<linux/dma-heap.h> (this file is introduced in the patch)

I looked at the recipe's sysroot
recipe-sysroot/usr/linux/include
and  dma-buf.h is there but not  dma-heap.h

Why is the header not being imported ?

-- 
Regards,
Karthik Poduval

Re: [yocto] Kernel Header UAPI Issue

[Khem Raj]

On Mon, Feb 22, 2021 at 4:52 PM Karthik Poduval
<karthik.poduval@gmail.com> wrote:
>
> I have a 5.4 kernel where I applied a patch (using a .scc file) from
> kernel 5.6 which brings in dmabuf heaps.
>
> The patch exports a uapi header
> include/uapi/linux/dma-heap.h.
> I am trying to write an application that uses dmabuf with a recipe (no
> special DEPENDS).
>
> It is able to find
> #include<linux/dma-buf.h>
> but not
> #include<linux/dma-heap.h> (this file is introduced in the patch)
>
> I looked at the recipe's sysroot
> recipe-sysroot/usr/linux/include
> and  dma-buf.h is there but not  dma-heap.h
>
> Why is the header not being imported ?
>

What does patch look like? just adding the file is not enough, it
should be added to makefiles to be exported as UAPI header

> --
> Regards,
> Karthik Poduval
>
> 
>

Re: [yocto] Kernel Header UAPI Issue

[Bruce Ashfield]

On Mon, Feb 22, 2021 at 7:52 PM Karthik Poduval
<karthik.poduval@gmail.com> wrote:
>
> I have a 5.4 kernel where I applied a patch (using a .scc file) from
> kernel 5.6 which brings in dmabuf heaps.
>
> The patch exports a uapi header
> include/uapi/linux/dma-heap.h.
> I am trying to write an application that uses dmabuf with a recipe (no
> special DEPENDS).
>
> It is able to find
> #include<linux/dma-buf.h>
> but not
> #include<linux/dma-heap.h> (this file is introduced in the patch)
>
> I looked at the recipe's sysroot
> recipe-sysroot/usr/linux/include


That's coming from the linux-libc-headers, your new upi header is not
part of the libc-headers package, so it won't show up there. And no,
it shouldn't be part of that recipe and the libc-headers should not be
generated from the current state of whatever kernel is being built.

You need to arrange for it to be exported and staged separately (look
for examples of 'sysroot_stage' variants in the meta-data, you can
bbappend the behaviour to your kernel recipe), or your code can look
for the header in the kernel shared workdir (STAGING_KERNEL_BUILDDIR)
by adding that to your include paths in the recipe.

As khem also mentioned, you'll need to make sure that the uapi header
is not just patched in, but properly exported.

Bruce

>
> and  dma-buf.h is there but not  dma-heap.h
>
> Why is the header not being imported ?
>
> --
> Regards,
> Karthik Poduval
>
> 
>


-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Re: [yocto] Kernel Header UAPI Issue

[Karthik Poduval]

[-- Attachment #1: Type: text/plain, Size: 2317 bytes --]

Hi Khem and Bruce,

I got the patch from raspberry pi github and it looked like it put the
header into include/uapi/linux/dma-heap.h, for kernel 5.4 it doesn't look
like there is a need for a Kbuild file. Are any special steps needed for
exporting the header (I thought kernel 5.4 exports everything from
include/uapi directory) ? The  linux-libc-headers sound like the issue.
Thanks for the pointers, I will dig up further based on the suggestions and
revert.

On Mon, Feb 22, 2021 at 5:14 PM Bruce Ashfield <bruce.ashfield@gmail.com>
wrote:

> On Mon, Feb 22, 2021 at 7:52 PM Karthik Poduval
> <karthik.poduval@gmail.com> wrote:
> >
> > I have a 5.4 kernel where I applied a patch (using a .scc file) from
> > kernel 5.6 which brings in dmabuf heaps.
> >
> > The patch exports a uapi header
> > include/uapi/linux/dma-heap.h.
> > I am trying to write an application that uses dmabuf with a recipe (no
> > special DEPENDS).
> >
> > It is able to find
> > #include<linux/dma-buf.h>
> > but not
> > #include<linux/dma-heap.h> (this file is introduced in the patch)
> >
> > I looked at the recipe's sysroot
> > recipe-sysroot/usr/linux/include
>
>
> That's coming from the linux-libc-headers, your new upi header is not
> part of the libc-headers package, so it won't show up there. And no,
> it shouldn't be part of that recipe and the libc-headers should not be
> generated from the current state of whatever kernel is being built.
>
> You need to arrange for it to be exported and staged separately (look
> for examples of 'sysroot_stage' variants in the meta-data, you can
> bbappend the behaviour to your kernel recipe), or your code can look
> for the header in the kernel shared workdir (STAGING_KERNEL_BUILDDIR)
> by adding that to your include paths in the recipe.
>
> As khem also mentioned, you'll need to make sure that the uapi header
> is not just patched in, but properly exported.
>
> Bruce
>
> >
> > and  dma-buf.h is there but not  dma-heap.h
> >
> > Why is the header not being imported ?
> >
> > --
> > Regards,
> > Karthik Poduval
> >
> > 
> >
>
>
> --
> - Thou shalt not follow the NULL pointer, for chaos and madness await
> thee at its end
> - "Use the force Harry" - Gandalf, Star Trek II
>


-- 
Regards,
Karthik Poduval

[-- Attachment #2: Type: text/html, Size: 3010 bytes --]

Re: [yocto] Kernel Header UAPI Issue

[Mikko Rapeli]

Hi,

I know it's not the best or recommended approach, but I find it
hard to avoid merging linux-libc-headers recipe with the actual
kernel recipe that a distro is using. At least a static copy
of some version of uapi headers from that kernel can be used
instead of the poky side linux-libc-headers. This helps to get
the actual BSP SW delivery headers into userspace, SDK etc.

Cheers,

-Mikko

Re: [yocto] Kernel Header UAPI Issue

[Karthik Poduval]

Hi Mikko,

Do you have an example on how you do that ? Do you bbapend the
linux-libc-headers recipe file ?
I have an application that uses dmabuf heap that potentially extends
across multiple BSP's as its BSP agnostic. I don't want to be patching
individual BSP recipes and generating headers. The issue I am facing
is due to backporting the patch from 5.6 to 5.4 so the required header
isn't a part of the linux-libc-headers.bb recipe. Best would have been
a virtual/kernel-keaders target that applications that require BSP
headers would add to their recipe DEPENDS. Why is this not a solved
issue by yocto project ? Why do individual BSP's need to deal with
this differently  when the header install mechanism (make
headers_install) is the same irrespective of the type of BSP ?

On Tue, Feb 23, 2021 at 5:18 AM <Mikko.Rapeli@bmw.de> wrote:
>
> Hi,
>
> I know it's not the best or recommended approach, but I find it
> hard to avoid merging linux-libc-headers recipe with the actual
> kernel recipe that a distro is using. At least a static copy
> of some version of uapi headers from that kernel can be used
> instead of the poky side linux-libc-headers. This helps to get
> the actual BSP SW delivery headers into userspace, SDK etc.
>
> Cheers,
>
> -Mikko



-- 
Regards,
Karthik Poduval

Re: [yocto] Kernel Header UAPI Issue

[Mikko Rapeli]

Hi,

On Tue, Feb 23, 2021 at 06:56:02AM -0800, Karthik Poduval wrote:
> Hi Mikko,
> 
> Do you have an example on how you do that ? Do you bbapend the
> linux-libc-headers recipe file ?

A kernel recipe will provide linux-libc-headers after a
"make headers_install" call...

So the SRC_URI of linux-libc-headers can be the same as from the
linux kernel recipe, or linux kernel recipe can provide (with some
tricks, possibly) the linux-libc-headers binary packages.

> I have an application that uses dmabuf heap that potentially extends
> across multiple BSP's as its BSP agnostic. I don't want to be patching
> individual BSP recipes and generating headers. The issue I am facing
> is due to backporting the patch from 5.6 to 5.4 so the required header
> isn't a part of the linux-libc-headers.bb recipe. Best would have been
> a virtual/kernel-keaders target that applications that require BSP
> headers would add to their recipe DEPENDS. Why is this not a solved
> issue by yocto project ? Why do individual BSP's need to deal with
> this differently  when the header install mechanism (make
> headers_install) is the same irrespective of the type of BSP ?

I guess the reason is to split userspace to BSP/board specific and common
parts. It's not good if whole userspace bootstrap depends on kernel
recipe and any kernel changes requires full bootstrap and recompilation
of all userspace.

But there are BSP/board specific recipes which do need the real effective
kernel headers to interface with kernel drivers, and there are SoC
architecture specific ones which just need enough to build gcc and glibc.

Both have their pross and cons, and to me there is no clear winner.

Cheers,

-Mikko

Re: [yocto] Kernel Header UAPI Issue

[Bruce Ashfield]

On Tue, Feb 23, 2021 at 9:56 AM Karthik Poduval
<karthik.poduval@gmail.com> wrote:
>
> Hi Mikko,
>
> Do you have an example on how you do that ? Do you bbapend the
> linux-libc-headers recipe file ?
> I have an application that uses dmabuf heap that potentially extends
> across multiple BSP's as its BSP agnostic. I don't want to be patching
> individual BSP recipes and generating headers. The issue I am facing
> is due to backporting the patch from 5.6 to 5.4 so the required header
> isn't a part of the linux-libc-headers.bb recipe. Best would have been
> a virtual/kernel-keaders target that applications that require BSP
> headers would add to their recipe DEPENDS. Why is this not a solved
> issue by yocto project ? Why do individual BSP's need to deal with
> this differently  when the header install mechanism (make
> headers_install) is the same irrespective of the type of BSP ?

Because it's not a simple thing to solve (and there's a bugzilla around for it
already, where the thoughts and issues are captured, but that one seems to
have been closed and I can't find it at the moment).  I do have a WIP
class that provides some different modes
(https://bugzilla.yoctoproject.org/show_bug.cgi?id=5305),
but with corner cases and concerns, it keeps slipping. Feel free to try it
out and comment in the bug (I'll try myself to be sure it still applied, it has
been a few months).

What works for your case, doesn't mean it is a general/supporable
solution.

But generally speaking, It's an incredibly bad idea to have your libc-headers
tied to the kernel you are building. Every time that kernel changes, you
basically need to rebuild the entire stack .. hence the bad idea.  It is such
a common question, that we actually put a warning in the libc-headers
recipe itself.

We do not really want a parallel set of headers in the shared workdir, that are
from the currently built kernel. You'd end up with all sorts of mismatches
and cross includes and potential different behaviour per-application.

We already have the kernel source installed into the shared workdir,
which is what the tips in the libc-headers recipe suggest using. And it
honestly isn't so common the need for sanitized headers that the need for
something like I did in that bug has made it necessary.

Bruce

>
> On Tue, Feb 23, 2021 at 5:18 AM <Mikko.Rapeli@bmw.de> wrote:
> >
> > Hi,
> >
> > I know it's not the best or recommended approach, but I find it
> > hard to avoid merging linux-libc-headers recipe with the actual
> > kernel recipe that a distro is using. At least a static copy
> > of some version of uapi headers from that kernel can be used
> > instead of the poky side linux-libc-headers. This helps to get
> > the actual BSP SW delivery headers into userspace, SDK etc.
> >
> > Cheers,
> >
> > -Mikko
>
>
>
> --
> Regards,
> Karthik Poduval


--
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Re: [yocto] Kernel Header UAPI Issue

[Karthik Poduval]

Hi Bruce,

Thanks a lot for your patch (attached in bugzilla
https://bugzilla.yoctoproject.org/show_bug.cgi?id=5305). I am
summarizing all steps once again for the benefit of anyone else who
faces the same issue (including myself in the future) and may use this
email list as a reference. The basic need was to export kernel headers
from kernel source for an application recipe (those headers are not a
part of linux-libc-headers recipe).

I applied the patch pointed by Bruce
https://bugzilla.yoctoproject.org/attachment.cgi?id=4647 to my local
BSP layer. The patch has 2 modes of operation (described in the patch
documentation).

mode1: To make this work I added the following lines to my application recipe.

inherit cmake kernel-alt-headers
DEPENDS = "gtest rsync-native"
EXTRA_OECMAKE =
'-DKERNEL_HEADER_DIR:STRING=${WORKDIR}/${KERNEL_ALT_HEADER_DIR}/include'
Then in my CMakeLists.txt.
include_directories(${KERNEL_HEADER_DIR})

The application compiled fine but there was one side effect that
kernel menuconfig (bitbake virtual/kernel -c menuconfig) wasn't able
to run as it complained that the source tree wasn't clean and make
mrproper was needed. This was resolved by a simple fix to the
do_compile_prepend in the patch (added the mrproper command).
do_compile_prepend() {
        if [ "${KERNEL_SOURCE_IS_LOCAL}" = "False" ]; then
            # install from the staging kernel directory
            oe_runmake -C ${STAGING_KERNEL_DIR} headers_install
INSTALL_HDR_PATH=${WORKDIR}/${KERNEL_ALT_HEADER_DIR}
            oe_runmake -C ${STAGING_KERNEL_DIR} mrproper
        fi
}

mode2: To make this work I added the following line to my kernel recipe.
inherit kernel-alt-headers

to my application recipe following lines were added.
DEPENDS += "virtual/kernel"
EXTRA_OECMAKE =
'-DKERNEL_HEADER_DIR:STRING=${STAGING_DIR_TARGET}/usr/alt-headers/include'
Then in my CMakeLists.txt.
include_directories(${KERNEL_HEADER_DIR})

-- 
Regards,
Karthik Poduval

On Tue, Feb 23, 2021 at 10:50 AM Bruce Ashfield
<bruce.ashfield@gmail.com> wrote:
>
> On Tue, Feb 23, 2021 at 9:56 AM Karthik Poduval
> <karthik.poduval@gmail.com> wrote:
> >
> > Hi Mikko,
> >
> > Do you have an example on how you do that ? Do you bbapend the
> > linux-libc-headers recipe file ?
> > I have an application that uses dmabuf heap that potentially extends
> > across multiple BSP's as its BSP agnostic. I don't want to be patching
> > individual BSP recipes and generating headers. The issue I am facing
> > is due to backporting the patch from 5.6 to 5.4 so the required header
> > isn't a part of the linux-libc-headers.bb recipe. Best would have been
> > a virtual/kernel-keaders target that applications that require BSP
> > headers would add to their recipe DEPENDS. Why is this not a solved
> > issue by yocto project ? Why do individual BSP's need to deal with
> > this differently  when the header install mechanism (make
> > headers_install) is the same irrespective of the type of BSP ?
>
> Because it's not a simple thing to solve (and there's a bugzilla around for it
> already, where the thoughts and issues are captured, but that one seems to
> have been closed and I can't find it at the moment).  I do have a WIP
> class that provides some different modes
> (https://bugzilla.yoctoproject.org/show_bug.cgi?id=5305),
> but with corner cases and concerns, it keeps slipping. Feel free to try it
> out and comment in the bug (I'll try myself to be sure it still applied, it has
> been a few months).
>
> What works for your case, doesn't mean it is a general/supporable
> solution.
>
> But generally speaking, It's an incredibly bad idea to have your libc-headers
> tied to the kernel you are building. Every time that kernel changes, you
> basically need to rebuild the entire stack .. hence the bad idea.  It is such
> a common question, that we actually put a warning in the libc-headers
> recipe itself.
>
> We do not really want a parallel set of headers in the shared workdir, that are
> from the currently built kernel. You'd end up with all sorts of mismatches
> and cross includes and potential different behaviour per-application.
>
> We already have the kernel source installed into the shared workdir,
> which is what the tips in the libc-headers recipe suggest using. And it
> honestly isn't so common the need for sanitized headers that the need for
> something like I did in that bug has made it necessary.
>
> Bruce
>
> >
> > On Tue, Feb 23, 2021 at 5:18 AM <Mikko.Rapeli@bmw.de> wrote:
> > >
> > > Hi,
> > >
> > > I know it's not the best or recommended approach, but I find it
> > > hard to avoid merging linux-libc-headers recipe with the actual
> > > kernel recipe that a distro is using. At least a static copy
> > > of some version of uapi headers from that kernel can be used
> > > instead of the poky side linux-libc-headers. This helps to get
> > > the actual BSP SW delivery headers into userspace, SDK etc.
> > >
> > > Cheers,
> > >
> > > -Mikko
> >
> >
> >
> > --
> > Regards,
> > Karthik Poduval
>
>
> --
> - Thou shalt not follow the NULL pointer, for chaos and madness await
> thee at its end
> - "Use the force Harry" - Gandalf, Star Trek II