Re: [PATCH] kernel.bbclass: allow exporting files from kernel recipes to sysroot

[<Mikko.Rapeli@bmw.de>]

On Mon, Sep 24, 2018 at 08:12:53AM -0400, Bruce Ashfield wrote:
> On Mon, Sep 24, 2018 at 3:25 AM,  <Mikko.Rapeli@bmw.de> wrote:
> >> On Fri, Sep 21, 2018 at 7:49 AM, Mikko Rapeli <mikko.rapeli at bmw.de> wrote:
> >> > This change enables kernel recipes to share files with other
> >> > recipes. Firmware, modules and kernel-depmod are still not shared
> >> > since according to git history they cause problems with multiarch,
> >> > but all others are allowed. Examples of shared files are
> >> > kernel version and recipe specific headers and scripts which
> >> > are not needed by common linux-libc-headers to bootstrap glibc.
> >> >
> >> > For example, if a kernel recipe wants to share headers, it can do:
> >> >
> >> > PACKAGES =+ "${PN}-headers"
> >> >
> >> > do_install_append() {
> >> >     install -d "${D}${includedir}/${PN}"
> >> >     oe_runmake INSTALL_HDR_PATH="${D}${includedir}/${PN}" headers_install
> >> }
> >>
> >> This is what I've always done in the past (and in fact, there's an
> >> open Yocto bug
> >> to track this), but I haven't actually needed to do what you are
> >> modifying in the
> >> bbclass itself.
> >
> > Without this patch, users of kernel.bbclass can provide the headers etc packages,
> > but other recipes can never see the files in their sysroot even if they depend
> > on the kernel recipe because kernel.bbclass has disabled sysroot_stage_all()
> > completely.
> 
> It doesn't need to be in a class, any kernel recipe can create a task to
> do this. The class implementation isn't special in this manner.
> 
> >
> > The recipes could define their own sysroot_stage_all() but I don't see
> > why kernel would be that special and why all of their installed files should
> > be excluded from build sysroots by default.
> 
> Exactly. It is because of the ability to clobber the libc-headers that is is
> special.
> 
> >
> > The old way to do this was to fork linux-libc-headers for the custom kernel
> > but now it has a big fat warning to not do this, but exposing header etc
> > files to other recipes to build against was still not resolved.
> 
> That was one old way, but not the only. And not for exposing non uapi
> headers.

What other ways exist?

I don't care how, but I must export custom kernel specific headers and
other files to other recipes in a build in ways which are compatible with
yocto upstream.

I have not seen any documented ways for this.

> >
> > With this patch the kernel recipe can just install the files needed and
> > users can see them unless they are filtered.
> 
> And as the warning points out, risk the libc-interface, which has caused
> many problems in the past .. and problems that are hard to detect and
> dig out at runtime.
> 
> >
> >> If you call the sysroot stage routines directly in that
> >> install_append, are you really
> >> not seeing the files appear in the recipe's sysroot ?
> >
> > Of course this can be done but why on earth is kernel so special that
> > it's installed files are not visible in sysroots by default?
> 
> because they clobber the uapi libc-headers :D
> 
> >
> > If certain files are problematic, they can be filtered out from the
> > sysroot like my patch tries to do.
> 
> Or we could catch that the clobbering is happening, and not let it happen.
> 
> >
> >> Have you confirmed that we get a warning/error from bitbake about
> >> conflicting files
> >> from multiple recipes if someone doesn't know to use a custom path for their
> >> headers ? That has always been the main concern about allowing something like
> >> this.
> >
> > No, but I do rely on this with other recipes as well. At least I know that
> > conflicting files warning will fail image and SDK generation.
> 
> For something that is being proposed as a general purpose addition to
> the core, it would probably be a good idea to test these extra cases.
> 
> >
> > I would expect that recipe specific sysroots don't allow conflicting files.
> 
> Agreed, but definitely worth testing and logging. That way the change
> won't be blamed for causing issues later.

If files from a recipe can replace files from another recipe when preparing
recipe specific sysroot we have bigger problems than just the kernel.

I can try test this out with the kernel recipe though.

-Mikko