[PATCH] External modules fail to load due to unknown symbols

[PATCH] External modules fail to load due to unknown symbols

[Amy Fong]

From da12fa5e6daca0c0b7bb15df030b1c79fbefa089 Mon Sep 17 00:00:00 2001
From: Amy Fong <amy.fong@windriver.com>
Date: Mon, 1 Jun 2015 13:16:20 -0400
Subject: [PATCH] External modules fail to load due to unknown symbols

When building external kernel modules, a failure can happen
when the module has dependencies on other kernel modules.
In such cases, the build looks at Module.symvers to do
symbol lookups of the other kernel modules that it depends on.

Module.symvers gets copied by do_shared_workdir
after do_compile and Module.symvers gets updated during
do_compile_kernelmodules.

do_compile_kernelmodules is modified to explicitly copy Module.symvers
to the shared workdir.

Signed-off-by: Amy Fong <amy.fong@windriver.com>
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
---
 meta/classes/kernel.bbclass | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
index 4f22cde..4a177b1 100644
--- a/meta/classes/kernel.bbclass
+++ b/meta/classes/kernel.bbclass
@@ -215,6 +215,14 @@ do_compile_kernelmodules() {
 	unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS MACHINE
 	if (grep -q -i -e '^CONFIG_MODULES=y$' .config); then
 		oe_runmake ${PARALLEL_MAKE} modules CC="${KERNEL_CC}" LD="${KERNEL_LD}" ${KERNEL_EXTRA_ARGS}
+
+		# Module.symvers gets updated during the 
+		# building of the kernel modules. We need to
+		# update this in the shared workdir since some
+		# external kernel modules has a dependency on
+		# other kernel modules and will look at this
+		# file to do symbol lookups
+		cp Module.symvers ${STAGING_KERNEL_BUILDDIR}
 	else
 		bbnote "no modules to compile"
 	fi
-- 
2.0.1

Re: [PATCH] External modules fail to load due to unknown symbols

[Richard Purdie]

On Fri, 2015-06-05 at 13:18 -0400, Amy Fong wrote:
> From da12fa5e6daca0c0b7bb15df030b1c79fbefa089 Mon Sep 17 00:00:00 2001
> From: Amy Fong <amy.fong@windriver.com>
> Date: Mon, 1 Jun 2015 13:16:20 -0400
> Subject: [PATCH] External modules fail to load due to unknown symbols
> 
> When building external kernel modules, a failure can happen
> when the module has dependencies on other kernel modules.
> In such cases, the build looks at Module.symvers to do
> symbol lookups of the other kernel modules that it depends on.
> 
> Module.symvers gets copied by do_shared_workdir
> after do_compile and Module.symvers gets updated during
> do_compile_kernelmodules.
> 
> do_compile_kernelmodules is modified to explicitly copy Module.symvers
> to the shared workdir.
> 
> Signed-off-by: Amy Fong <amy.fong@windriver.com>
> Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
> ---
>  meta/classes/kernel.bbclass | 8 ++++++++
>  1 file changed, 8 insertions(+)
> 
> diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
> index 4f22cde..4a177b1 100644
> --- a/meta/classes/kernel.bbclass
> +++ b/meta/classes/kernel.bbclass
> @@ -215,6 +215,14 @@ do_compile_kernelmodules() {
>  	unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS MACHINE
>  	if (grep -q -i -e '^CONFIG_MODULES=y$' .config); then
>  		oe_runmake ${PARALLEL_MAKE} modules CC="${KERNEL_CC}" LD="${KERNEL_LD}" ${KERNEL_EXTRA_ARGS}
> +
> +		# Module.symvers gets updated during the 
> +		# building of the kernel modules. We need to
> +		# update this in the shared workdir since some
> +		# external kernel modules has a dependency on
> +		# other kernel modules and will look at this
> +		# file to do symbol lookups
> +		cp Module.symvers ${STAGING_KERNEL_BUILDDIR}
>  	else
>  		bbnote "no modules to compile"
>  	fi

Sadly this will give rise to a race. shared_workdir is defined as:

addtask shared_workdir after do_compile before do_compile_kernelmodule

and other modules depend on the shared_workdir task. Your other modules
could therefore compile before the code in compile_kernelmodule has run.

At a guess you will next suggest we change shared_workdir to be after
compile_kernelmodules. The problem there is one of performance. We had
bug reports from the community and Wind River people about the fact that
shared_workdir executes when building kernel modules and doesn't come
from sstate. Forcing shared_workdir to run more tasks every time isn't
going to make people happy either :(

So I'm not sure how we solve this but it is going to need more thought
and the above patch isn't right.

Cheers,

Richard

Re: [PATCH] External modules fail to load due to unknown symbols

[Bruce Ashfield]

On Fri, Jun 5, 2015 at 6:53 PM, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> On Fri, 2015-06-05 at 13:18 -0400, Amy Fong wrote:
>> From da12fa5e6daca0c0b7bb15df030b1c79fbefa089 Mon Sep 17 00:00:00 2001
>> From: Amy Fong <amy.fong@windriver.com>
>> Date: Mon, 1 Jun 2015 13:16:20 -0400
>> Subject: [PATCH] External modules fail to load due to unknown symbols
>>
>> When building external kernel modules, a failure can happen
>> when the module has dependencies on other kernel modules.
>> In such cases, the build looks at Module.symvers to do
>> symbol lookups of the other kernel modules that it depends on.
>>
>> Module.symvers gets copied by do_shared_workdir
>> after do_compile and Module.symvers gets updated during
>> do_compile_kernelmodules.
>>
>> do_compile_kernelmodules is modified to explicitly copy Module.symvers
>> to the shared workdir.
>>
>> Signed-off-by: Amy Fong <amy.fong@windriver.com>
>> Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
>> ---
>>  meta/classes/kernel.bbclass | 8 ++++++++
>>  1 file changed, 8 insertions(+)
>>
>> diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
>> index 4f22cde..4a177b1 100644
>> --- a/meta/classes/kernel.bbclass
>> +++ b/meta/classes/kernel.bbclass
>> @@ -215,6 +215,14 @@ do_compile_kernelmodules() {
>>       unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS MACHINE
>>       if (grep -q -i -e '^CONFIG_MODULES=y$' .config); then
>>               oe_runmake ${PARALLEL_MAKE} modules CC="${KERNEL_CC}" LD="${KERNEL_LD}" ${KERNEL_EXTRA_ARGS}
>> +
>> +             # Module.symvers gets updated during the
>> +             # building of the kernel modules. We need to
>> +             # update this in the shared workdir since some
>> +             # external kernel modules has a dependency on
>> +             # other kernel modules and will look at this
>> +             # file to do symbol lookups
>> +             cp Module.symvers ${STAGING_KERNEL_BUILDDIR}
>>       else
>>               bbnote "no modules to compile"
>>       fi
>
> Sadly this will give rise to a race. shared_workdir is defined as:
>
> addtask shared_workdir after do_compile before do_compile_kernelmodule
>
> and other modules depend on the shared_workdir task. Your other modules
> could therefore compile before the code in compile_kernelmodule has run.
>
> At a guess you will next suggest we change shared_workdir to be after
> compile_kernelmodules. The problem there is one of performance. We had
> bug reports from the community and Wind River people about the fact that
> shared_workdir executes when building kernel modules and doesn't come
> from sstate. Forcing shared_workdir to run more tasks every time isn't
> going to make people happy either :(

:) Actually no, we wouldn't suggest the above, since I already talked Amy out
of a similar patch.

In this case, I was shooting for it to at least be correct, in such that the
symbol file was copied out after the modules built .. and then the plan
was to simply put the rest of the responsibility onto the recipe maintainer.
If they know they depend on symbols from other modules, depend on
do_compile_kernelmodules .. otherwise, the default dependency kicks in
and keeps things sane.

But if we still have a race between the modules building and the copy out,
then we aren't any further ahead.

Bruce

>
> So I'm not sure how we solve this but it is going to need more thought
> and the above patch isn't right.
>
> Cheers,
>
> Richard
>
>
>
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core



-- 
"Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end"

[PATCH] External modules fail to load due to unknown symbols

[Amy Fong]

From da12fa5e6daca0c0b7bb15df030b1c79fbefa089 Mon Sep 17 00:00:00 2001
From: Amy Fong <amy.fong@windriver.com>
Date: Mon, 1 Jun 2015 13:16:20 -0400
Subject: [PATCH] External modules fail to load due to unknown symbols

When building external kernel modules, a failure can happen
when the module has dependencies on other kernel modules.
In such cases, the build looks at Module.symvers to do
symbol lookups of the other kernel modules that it depends on.

Module.symvers gets copied by do_shared_workdir
after do_compile and Module.symvers gets updated during
do_compile_kernelmodules.

do_compile_kernelmodules is modified to explicitly copy Module.symvers
to the shared workdir.

Signed-off-by: Amy Fong <amy.fong@windriver.com>
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
---
 meta/classes/kernel.bbclass | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
index 4f22cde..4a177b1 100644
--- a/meta/classes/kernel.bbclass
+++ b/meta/classes/kernel.bbclass
@@ -215,6 +215,14 @@ do_compile_kernelmodules() {
 	unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS MACHINE
 	if (grep -q -i -e '^CONFIG_MODULES=y$' .config); then
 		oe_runmake ${PARALLEL_MAKE} modules CC="${KERNEL_CC}" LD="${KERNEL_LD}" ${KERNEL_EXTRA_ARGS}
+
+		# Module.symvers gets updated during the 
+		# building of the kernel modules. We need to
+		# update this in the shared workdir since some
+		# external kernel modules has a dependency on
+		# other kernel modules and will look at this
+		# file to do symbol lookups
+		cp Module.symvers ${STAGING_KERNEL_BUILDDIR}
 	else
 		bbnote "no modules to compile"
 	fi
-- 
2.0.1