[rocko][PATCH 1/2] make-mod-scripts/kernel-devsrc: Fix objtool issues

[rocko][PATCH 1/2] make-mod-scripts/kernel-devsrc: Fix objtool issues

[Anuj Mittal]

From: Richard Purdie <richard.purdie@linuxfoundation.org>

Kernels which use tools/objtool can now fail when building external modules
due to objtool being missing, the generated files can also cause problems
for kernel-devsrc.

Ensure objtool is generated in make-mod-scripts by also calling
"make prepare".

For devsrc, delete the generated binaries since they'd be native
binaries and unsuitable for the target.

The oeqa kernel module tests also need to have the additional "make prepare"
step added.

(From OE-Core rev: 52fd2993784b4218f5df4f343e7da45d964df305)

Fixes [YOCTO #12860]

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
[Tweaked to add "make prepare" to module-base.bbclass instead of
make-mod-scripts.bb]
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
---
 meta/classes/module-base.bbclass            | 2 +-
 meta/lib/oeqa/runtime/cases/kernelmodule.py | 2 +-
 meta/recipes-kernel/linux/kernel-devsrc.bb  | 7 +++++++
 3 files changed, 9 insertions(+), 2 deletions(-)

diff --git a/meta/classes/module-base.bbclass b/meta/classes/module-base.bbclass
index 6fe77c01b7..64729770c7 100644
--- a/meta/classes/module-base.bbclass
+++ b/meta/classes/module-base.bbclass
@@ -23,5 +23,5 @@ PACKAGE_ARCH = "${MACHINE_ARCH}"
 do_make_scripts() {
 	unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS 
 	make CC="${KERNEL_CC}" LD="${KERNEL_LD}" AR="${KERNEL_AR}" \
-	           -C ${STAGING_KERNEL_DIR} O=${STAGING_KERNEL_BUILDDIR} scripts
+	           -C ${STAGING_KERNEL_DIR} O=${STAGING_KERNEL_BUILDDIR} scripts prepare
 }
diff --git a/meta/lib/oeqa/runtime/cases/kernelmodule.py b/meta/lib/oeqa/runtime/cases/kernelmodule.py
index 11ad7b7f01..de1a5aa445 100644
--- a/meta/lib/oeqa/runtime/cases/kernelmodule.py
+++ b/meta/lib/oeqa/runtime/cases/kernelmodule.py
@@ -28,7 +28,7 @@ class KernelModuleTest(OERuntimeTestCase):
     @OETestDepends(['gcc.GccCompileTest.test_gcc_compile'])
     def test_kernel_module(self):
         cmds = [
-            'cd /usr/src/kernel && make scripts',
+            'cd /usr/src/kernel && make scripts prepare',
             'cd /tmp && make',
             'cd /tmp && insmod hellomod.ko',
             'lsmod | grep hellomod',
diff --git a/meta/recipes-kernel/linux/kernel-devsrc.bb b/meta/recipes-kernel/linux/kernel-devsrc.bb
index c1b5b7786d..8bbfa23e4b 100644
--- a/meta/recipes-kernel/linux/kernel-devsrc.bb
+++ b/meta/recipes-kernel/linux/kernel-devsrc.bb
@@ -69,6 +69,13 @@ do_install() {
                 cp ${B}/arch/powerpc/lib/crtsavres.o $kerneldir/arch/powerpc/lib/crtsavres.o
         fi
 
+        # Remove fixdep/objtool as they won't be target binaries
+        for i in fixdep objtool; do
+                if [ -e $kerneldir/tools/objtool/$i ]; then
+                        rm -rf $kerneldir/tools/objtool/$i
+                fi
+        done
+
         chown -R root:root ${D}
 }
 # Ensure we don't race against "make scripts" during cpio
-- 
2.17.1

[rocko][PATCH 2/2] make-mod-scripts: add build requirements for external modules

[Anuj Mittal]

From: Bruce Ashfield <bruce.ashfield@windriver.com>

Newer kernels (4.14/v4.15+) have dependencies for the build of
modules (and hence external modules). Without these dependencies
explicitly in the build chain, you can end up with build failures like:

  work-shared/qemux86/kernel-source/scripts/extract-cert.c:21:25: fatal
  error: openssl/bio.h: No such file or directory
  |  #include <openssl/bio.h>
  |                          ^
  | compilation terminated.
  | make[2]: *** [scripts/extract-cert] Error 1
  | make[1]: *** [scripts] Error 2

To ensure that these headers are in place, and that the scripts use
our build environment flags, we add a dependency on openssl-native
and use oe_make to invoke the build.

Older kernels have no issues with the extra dependency, so there's no
need to make this conditional.

(From OE-Core rev: 916cb2029d3c97bf12ebf03832b9ba980451dbcf)

Fixes [YOCTO #12860]

Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
[Tweaked to have changes in module*.bbclass instead from where
make-mod-scripts was split in sumo]
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
---
 meta/classes/module-base.bbclass | 5 ++++-
 meta/classes/module.bbclass      | 2 +-
 2 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/meta/classes/module-base.bbclass b/meta/classes/module-base.bbclass
index 64729770c7..9f3465eb9d 100644
--- a/meta/classes/module-base.bbclass
+++ b/meta/classes/module-base.bbclass
@@ -12,6 +12,9 @@ export CROSS_COMPILE = "${TARGET_PREFIX}"
 # we didn't pick the name.
 export KBUILD_OUTPUT = "${STAGING_KERNEL_BUILDDIR}"
 
+DEPENDS += "bc-native"
+EXTRA_OEMAKE += " HOSTCC="${BUILD_CC} ${BUILD_CFLAGS} ${BUILD_LDFLAGS}" HOSTCPP="${BUILD_CPP}""
+
 export KERNEL_VERSION = "${@base_read_file('${STAGING_KERNEL_BUILDDIR}/kernel-abiversion')}"
 KERNEL_OBJECT_SUFFIX = ".ko"
 
@@ -22,6 +25,6 @@ PACKAGE_ARCH = "${MACHINE_ARCH}"
 # be called before do_compile. See module.bbclass for an example.
 do_make_scripts() {
 	unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS 
-	make CC="${KERNEL_CC}" LD="${KERNEL_LD}" AR="${KERNEL_AR}" \
+	oe_runmake CC="${KERNEL_CC}" LD="${KERNEL_LD}" AR="${KERNEL_AR}" \
 	           -C ${STAGING_KERNEL_DIR} O=${STAGING_KERNEL_BUILDDIR} scripts prepare
 }
diff --git a/meta/classes/module.bbclass b/meta/classes/module.bbclass
index 78d1b21dbd..282900dc6d 100644
--- a/meta/classes/module.bbclass
+++ b/meta/classes/module.bbclass
@@ -2,7 +2,7 @@ inherit module-base kernel-module-split pkgconfig
 
 addtask make_scripts after do_prepare_recipe_sysroot before do_configure
 do_make_scripts[lockfiles] = "${TMPDIR}/kernel-scripts.lock"
-do_make_scripts[depends] += "virtual/kernel:do_shared_workdir"
+do_make_scripts[depends] += "virtual/kernel:do_shared_workdir openssl-native:do_populate_sysroot"
 
 EXTRA_OEMAKE += "KERNEL_SRC=${STAGING_KERNEL_DIR}"
 
-- 
2.17.1

Re: [rocko][PATCH 2/2] make-mod-scripts: add build requirements for external modules

[Denys Dmytriyenko]

On Sat, Jul 28, 2018 at 04:49:50PM +0800, Anuj Mittal wrote:
> From: Bruce Ashfield <bruce.ashfield@windriver.com>
> 
> Newer kernels (4.14/v4.15+) have dependencies for the build of
> modules (and hence external modules). Without these dependencies
> explicitly in the build chain, you can end up with build failures like:
> 
>   work-shared/qemux86/kernel-source/scripts/extract-cert.c:21:25: fatal
>   error: openssl/bio.h: No such file or directory
>   |  #include <openssl/bio.h>
>   |                          ^
>   | compilation terminated.
>   | make[2]: *** [scripts/extract-cert] Error 1
>   | make[1]: *** [scripts] Error 2
> 
> To ensure that these headers are in place, and that the scripts use
> our build environment flags, we add a dependency on openssl-native
> and use oe_make to invoke the build.
> 
> Older kernels have no issues with the extra dependency, so there's no
> need to make this conditional.
> 
> (From OE-Core rev: 916cb2029d3c97bf12ebf03832b9ba980451dbcf)
> 
> Fixes [YOCTO #12860]
> 
> Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> [Tweaked to have changes in module*.bbclass instead from where
> make-mod-scripts was split in sumo]
> Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
> ---
>  meta/classes/module-base.bbclass | 5 ++++-
>  meta/classes/module.bbclass      | 2 +-
>  2 files changed, 5 insertions(+), 2 deletions(-)
> 
> diff --git a/meta/classes/module-base.bbclass b/meta/classes/module-base.bbclass
> index 64729770c7..9f3465eb9d 100644
> --- a/meta/classes/module-base.bbclass
> +++ b/meta/classes/module-base.bbclass
> @@ -12,6 +12,9 @@ export CROSS_COMPILE = "${TARGET_PREFIX}"
>  # we didn't pick the name.
>  export KBUILD_OUTPUT = "${STAGING_KERNEL_BUILDDIR}"
>  
> +DEPENDS += "bc-native"
> +EXTRA_OEMAKE += " HOSTCC="${BUILD_CC} ${BUILD_CFLAGS} ${BUILD_LDFLAGS}" HOSTCPP="${BUILD_CPP}""
> +
>  export KERNEL_VERSION = "${@base_read_file('${STAGING_KERNEL_BUILDDIR}/kernel-abiversion')}"
>  KERNEL_OBJECT_SUFFIX = ".ko"
>  
> @@ -22,6 +25,6 @@ PACKAGE_ARCH = "${MACHINE_ARCH}"
>  # be called before do_compile. See module.bbclass for an example.
>  do_make_scripts() {
>  	unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS 
> -	make CC="${KERNEL_CC}" LD="${KERNEL_LD}" AR="${KERNEL_AR}" \
> +	oe_runmake CC="${KERNEL_CC}" LD="${KERNEL_LD}" AR="${KERNEL_AR}" \
>  	           -C ${STAGING_KERNEL_DIR} O=${STAGING_KERNEL_BUILDDIR} scripts prepare

This now breaks out-of-tree kernel modules that provide own non-standard 
Makefiles with something like "-f build.mak" added to EXTRA_OEMAKE.

That is because do_make_scripts() gets executed inside kernel-source directory 
(-C ${STAGING_KERNEL_DIR} above) and supposed to use kernel's Makefile, while 
do_compile()/do_install() are executed within module's sources and use 
whatever Makefile it provides. That is why it historically was using "make" 
here and not "oe_runmake".

When Bruce separated this code into make-mod-scripts, it was possible to also 
use oe_runmake there, as it now had own context. But the backport to rocko 
moved it back into module-base.bbclass and it gets executed within module's 
environment, including custom EXTRA_OEMAKE, which often breaks (i.e. few of my 
modules are now broken).

Should this backport only touch DEPENDS and do_make_scripts[depends], as the 
changelog claims, and leave make/oe_runmake and EXTRA_OEMAKE alone?


>  }
> diff --git a/meta/classes/module.bbclass b/meta/classes/module.bbclass
> index 78d1b21dbd..282900dc6d 100644
> --- a/meta/classes/module.bbclass
> +++ b/meta/classes/module.bbclass
> @@ -2,7 +2,7 @@ inherit module-base kernel-module-split pkgconfig
>  
>  addtask make_scripts after do_prepare_recipe_sysroot before do_configure
>  do_make_scripts[lockfiles] = "${TMPDIR}/kernel-scripts.lock"
> -do_make_scripts[depends] += "virtual/kernel:do_shared_workdir"
> +do_make_scripts[depends] += "virtual/kernel:do_shared_workdir openssl-native:do_populate_sysroot"
>  
>  EXTRA_OEMAKE += "KERNEL_SRC=${STAGING_KERNEL_DIR}"
>  
> -- 
> 2.17.1
> 
> -- 
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [rocko][PATCH 2/2] make-mod-scripts: add build requirements for external modules

[Denys Dmytriyenko]

On Wed, Aug 15, 2018 at 08:26:36PM -0400, Denys Dmytriyenko wrote:
> On Sat, Jul 28, 2018 at 04:49:50PM +0800, Anuj Mittal wrote:
> > From: Bruce Ashfield <bruce.ashfield@windriver.com>
> > 
> > Newer kernels (4.14/v4.15+) have dependencies for the build of
> > modules (and hence external modules). Without these dependencies
> > explicitly in the build chain, you can end up with build failures like:
> > 
> >   work-shared/qemux86/kernel-source/scripts/extract-cert.c:21:25: fatal
> >   error: openssl/bio.h: No such file or directory
> >   |  #include <openssl/bio.h>
> >   |                          ^
> >   | compilation terminated.
> >   | make[2]: *** [scripts/extract-cert] Error 1
> >   | make[1]: *** [scripts] Error 2
> > 
> > To ensure that these headers are in place, and that the scripts use
> > our build environment flags, we add a dependency on openssl-native
> > and use oe_make to invoke the build.
> > 
> > Older kernels have no issues with the extra dependency, so there's no
> > need to make this conditional.
> > 
> > (From OE-Core rev: 916cb2029d3c97bf12ebf03832b9ba980451dbcf)
> > 
> > Fixes [YOCTO #12860]
> > 
> > Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
> > Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> > [Tweaked to have changes in module*.bbclass instead from where
> > make-mod-scripts was split in sumo]
> > Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
> > ---
> >  meta/classes/module-base.bbclass | 5 ++++-
> >  meta/classes/module.bbclass      | 2 +-
> >  2 files changed, 5 insertions(+), 2 deletions(-)
> > 
> > diff --git a/meta/classes/module-base.bbclass b/meta/classes/module-base.bbclass
> > index 64729770c7..9f3465eb9d 100644
> > --- a/meta/classes/module-base.bbclass
> > +++ b/meta/classes/module-base.bbclass
> > @@ -12,6 +12,9 @@ export CROSS_COMPILE = "${TARGET_PREFIX}"
> >  # we didn't pick the name.
> >  export KBUILD_OUTPUT = "${STAGING_KERNEL_BUILDDIR}"
> >  
> > +DEPENDS += "bc-native"
> > +EXTRA_OEMAKE += " HOSTCC="${BUILD_CC} ${BUILD_CFLAGS} ${BUILD_LDFLAGS}" HOSTCPP="${BUILD_CPP}""
> > +
> >  export KERNEL_VERSION = "${@base_read_file('${STAGING_KERNEL_BUILDDIR}/kernel-abiversion')}"
> >  KERNEL_OBJECT_SUFFIX = ".ko"
> >  
> > @@ -22,6 +25,6 @@ PACKAGE_ARCH = "${MACHINE_ARCH}"
> >  # be called before do_compile. See module.bbclass for an example.
> >  do_make_scripts() {
> >  	unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS 
> > -	make CC="${KERNEL_CC}" LD="${KERNEL_LD}" AR="${KERNEL_AR}" \
> > +	oe_runmake CC="${KERNEL_CC}" LD="${KERNEL_LD}" AR="${KERNEL_AR}" \
> >  	           -C ${STAGING_KERNEL_DIR} O=${STAGING_KERNEL_BUILDDIR} scripts prepare
> 
> This now breaks out-of-tree kernel modules that provide own non-standard 
> Makefiles with something like "-f build.mak" added to EXTRA_OEMAKE.
> 
> That is because do_make_scripts() gets executed inside kernel-source directory 
> (-C ${STAGING_KERNEL_DIR} above) and supposed to use kernel's Makefile, while 
> do_compile()/do_install() are executed within module's sources and use 
> whatever Makefile it provides. That is why it historically was using "make" 
> here and not "oe_runmake".
> 
> When Bruce separated this code into make-mod-scripts, it was possible to also 
> use oe_runmake there, as it now had own context. But the backport to rocko 
> moved it back into module-base.bbclass and it gets executed within module's 
> environment, including custom EXTRA_OEMAKE, which often breaks (i.e. few of my 
> modules are now broken).
> 
> Should this backport only touch DEPENDS and do_make_scripts[depends], as the 
> changelog claims, and leave make/oe_runmake and EXTRA_OEMAKE alone?

Patch submitted:
http://lists.openembedded.org/pipermail/openembedded-core/2018-August/154191.html


> >  }
> > diff --git a/meta/classes/module.bbclass b/meta/classes/module.bbclass
> > index 78d1b21dbd..282900dc6d 100644
> > --- a/meta/classes/module.bbclass
> > +++ b/meta/classes/module.bbclass
> > @@ -2,7 +2,7 @@ inherit module-base kernel-module-split pkgconfig
> >  
> >  addtask make_scripts after do_prepare_recipe_sysroot before do_configure
> >  do_make_scripts[lockfiles] = "${TMPDIR}/kernel-scripts.lock"
> > -do_make_scripts[depends] += "virtual/kernel:do_shared_workdir"
> > +do_make_scripts[depends] += "virtual/kernel:do_shared_workdir openssl-native:do_populate_sysroot"
> >  
> >  EXTRA_OEMAKE += "KERNEL_SRC=${STAGING_KERNEL_DIR}"
> >  
> > -- 
> > 2.17.1
> > 
> > -- 
> > _______________________________________________
> > Openembedded-core mailing list
> > Openembedded-core@lists.openembedded.org
> > http://lists.openembedded.org/mailman/listinfo/openembedded-core
> -- 
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core