Changing external kernel module results in rebuild of whole kernel

Changing external kernel module results in rebuild of whole kernel

[Mike Looijmans]

Something in recent OE-core triggered a weird dependency "backfire".

If I change a recipe for a kernel module (a bb recipe that does "inherit 
module") this will trigger a rebuild of the whole kernel.

This turns the 5-second job of just updating a single module into a several 
minute workout for the build machine, and then causes boards to re-write the 
kernel into flash needlessly when upgrading.

I now see this on all projects using OE-core master. I can't really pin what 
caused it though. Anyone else seen this?


Kind regards,

Mike Looijmans
System Expert

TOPIC Embedded Products
Eindhovenseweg 32-C, NL-5683 KH Best
Postbus 440, NL-5680 AK Best
Telefoon: +31 (0) 499 33 69 79
Telefax: +31 (0) 499 33 69 70
E-mail: mike.looijmans@topicproducts.com
Website: www.topicproducts.com

Please consider the environment before printing this e-mail

Re: Changing external kernel module results in rebuild of whole kernel

[Burton, Ross]

[-- Attachment #1: Type: text/plain, Size: 433 bytes --]

On 6 May 2015 at 07:35, Mike Looijmans <mike.looijmans@topic.nl> wrote:

> If I change a recipe for a kernel module (a bb recipe that does "inherit
> module") this will trigger a rebuild of the whole kernel.
>

You can start to debug this by using bitbake-whatchanged. Build the kernel
and modules, make a minor change to the module recipe, and then run
"bitbake-whatchanged virtual/kernel" or some relevant target.

Ross

[-- Attachment #2: Type: text/html, Size: 858 bytes --]

Re: Changing external kernel module results in rebuild of whole kernel

[Mike Looijmans]

Just tried that. Change the SRCREV of the module, and ran
bitbake-whatchanged kernel-module-dyplo

Then it claims nothing changed:
$ bitbake-whatchanged kernel-module-dyplo
Figuring out the STAMPS_DIR ...
Generating the new stamps ... (need several minutes)

=== Summary: (0 changed, 0 unchanged)
Newly added: 0
PV changed: 0
PR changed: 0
Dependencies changed: 0


But if I run bitbake -n kernel-module-dyplo, it really wants to rebuild the 
kernel and module.

I'm merged oe-core master last monday, so it's pretty recent.


On 06-05-15 12:35, Burton, Ross wrote:
>
> On 6 May 2015 at 07:35, Mike Looijmans <mike.looijmans@topic.nl
> <mailto:mike.looijmans@topic.nl>> wrote:
>
>     If I change a recipe for a kernel module (a bb recipe that does "inherit
>     module") this will trigger a rebuild of the whole kernel.
>
>
> You can start to debug this by using bitbake-whatchanged. Build the kernel and
> modules, make a minor change to the module recipe, and then run
> "bitbake-whatchanged virtual/kernel" or some relevant target.
>
> Ross



Kind regards,

Mike Looijmans
System Expert

TOPIC Embedded Products
Eindhovenseweg 32-C, NL-5683 KH Best
Postbus 440, NL-5680 AK Best
Telefoon: +31 (0) 499 33 69 79
Telefax: +31 (0) 499 33 69 70
E-mail: mike.looijmans@topicproducts.com
Website: www.topicproducts.com

Please consider the environment before printing this e-mail

Re: Changing external kernel module results in rebuild of whole kernel

[Mike Looijmans]

On 06-05-15 12:35, Burton, Ross wrote:
>
> On 6 May 2015 at 07:35, Mike Looijmans <mike.looijmans@topic.nl
> <mailto:mike.looijmans@topic.nl>> wrote:
>
>     If I change a recipe for a kernel module (a bb recipe that does "inherit
>     module") this will trigger a rebuild of the whole kernel.
>
>
> You can start to debug this by using bitbake-whatchanged. Build the kernel and
> modules, make a minor change to the module recipe, and then run
> "bitbake-whatchanged virtual/kernel" or some relevant target.

I tried the command "bitbake -S printdiff virtual/kernel" instead, which gave 
the following output:


The differences between the current build and any cached tasks start at the 
following tasks:
/home/mike/projects/zynq-platform/meta-zynq/recipes-kernel/linux-zynq/linux-topic.bb, 
do_rm_work_all
NOTE: Reparsing files to collect dependency data
Writing locked sigs to /home/mike/projects/zynq-platform/build/locked-sigs.inc

Task linux-topic:do_rm_work_all couldn't be used from the cache because:
   We need hash ab23ed87f933003f95ca7bab4d70c1dc, closest matching task was 
99f675e6d7f7eed1bb5c3f2391e8a581
   Hash for dependent task readlinereadline_6.3.bb.do_rm_work:virtual:native 
changed from cc2e3667e52d154d8c9db7cb06ba2515 to de357460cfaed520424e5d7762713df0
   Unable to find matching sigdata for 
virtual:native:/home/mike/projects/zynq-platform/oe-core/meta/recipes-core/readline/readline_6.3.bb.do_rm_work 
with hashes cc2e3667e52d154d8c9db7cb06ba2515 or de357460cfaed520424e5d7762713df0



Kind regards,

Mike Looijmans
System Expert

TOPIC Embedded Products
Eindhovenseweg 32-C, NL-5683 KH Best
Postbus 440, NL-5680 AK Best
Telefoon: +31 (0) 499 33 69 79
Telefax: +31 (0) 499 33 69 70
E-mail: mike.looijmans@topicproducts.com
Website: www.topicproducts.com

Please consider the environment before printing this e-mail

Re: Changing external kernel module results in rebuild of whole kernel

[Richard Purdie]

On Wed, 2015-05-06 at 08:35 +0200, Mike Looijmans wrote:
> Something in recent OE-core triggered a weird dependency "backfire".
> 
> If I change a recipe for a kernel module (a bb recipe that does "inherit 
> module") this will trigger a rebuild of the whole kernel.
> 
> This turns the 5-second job of just updating a single module into a several 
> minute workout for the build machine, and then causes boards to re-write the 
> kernel into flash needlessly when upgrading.
> 
> I now see this on all projects using OE-core master. I can't really pin what 
> caused it though. Anyone else seen this?

I have a suspicion this may be as a result of the changed kernel build
process in 1.8. 

The idea there is that the modules depend on the kernel source and
rather than taring up and then extracting a large (GB) sized sstate
object, we just extract the original kernel source.

So is the kernel really rebuilding, or, is it just extracting source for
the kernel to build against? I noticed rm_work in your other post and
this may also be some bad interaction between rm_work and the kernel
build process changes.

Cheers,

Richard

Re: Changing external kernel module results in rebuild of whole kernel

[Mike Looijmans]

On 06-05-15 14:35, Richard Purdie wrote:
> On Wed, 2015-05-06 at 08:35 +0200, Mike Looijmans wrote:
>> Something in recent OE-core triggered a weird dependency "backfire".
>>
>> If I change a recipe for a kernel module (a bb recipe that does "inherit
>> module") this will trigger a rebuild of the whole kernel.
>>
>> This turns the 5-second job of just updating a single module into a several
>> minute workout for the build machine, and then causes boards to re-write the
>> kernel into flash needlessly when upgrading.
>>
>> I now see this on all projects using OE-core master. I can't really pin what
>> caused it though. Anyone else seen this?
>
> I have a suspicion this may be as a result of the changed kernel build
> process in 1.8.
>
> The idea there is that the modules depend on the kernel source and
> rather than taring up and then extracting a large (GB) sized sstate
> object, we just extract the original kernel source.
>
> So is the kernel really rebuilding, or, is it just extracting source for
> the kernel to build against? I noticed rm_work in your other post and
> this may also be some bad interaction between rm_work and the kernel
> build process changes.

It is really completely rebuilding the kernel (patch, configure, compile, 
install, etc). And as a result, it also recompiles each and every other kernel 
module, because these depend on the kernel as well. It will rebuild the kernel 
only once per run though, even if you change more than one module.

M.


Kind regards,

Mike Looijmans
System Expert

TOPIC Embedded Products
Eindhovenseweg 32-C, NL-5683 KH Best
Postbus 440, NL-5680 AK Best
Telefoon: +31 (0) 499 33 69 79
Telefax: +31 (0) 499 33 69 70
E-mail: mike.looijmans@topicproducts.com
Website: www.topicproducts.com

Please consider the environment before printing this e-mail

Re: Changing external kernel module results in rebuild of whole kernel

[Mike Looijmans]

On 06-05-15 14:35, Richard Purdie wrote:
> On Wed, 2015-05-06 at 08:35 +0200, Mike Looijmans wrote:
>> Something in recent OE-core triggered a weird dependency "backfire".
>>
>> If I change a recipe for a kernel module (a bb recipe that does "inherit
>> module") this will trigger a rebuild of the whole kernel.
>>
>> This turns the 5-second job of just updating a single module into a several
>> minute workout for the build machine, and then causes boards to re-write the
>> kernel into flash needlessly when upgrading.
>>
>> I now see this on all projects using OE-core master. I can't really pin what
>> caused it though. Anyone else seen this?
>
> I have a suspicion this may be as a result of the changed kernel build
> process in 1.8.
>
> The idea there is that the modules depend on the kernel source and
> rather than taring up and then extracting a large (GB) sized sstate
> object, we just extract the original kernel source.
>
> So is the kernel really rebuilding, or, is it just extracting source for
> the kernel to build against? I noticed rm_work in your other post and
> this may also be some bad interaction between rm_work and the kernel
> build process changes.

Any things I can try or provide on this?

As things are now, I'd much prefer the "taring up and then extracting a large 
(GB) sized sstate object" option, since that at least worked correctly.

Mike.


Kind regards,

Mike Looijmans
System Expert

TOPIC Embedded Products
Eindhovenseweg 32-C, NL-5683 KH Best
Postbus 440, NL-5680 AK Best
Telefoon: +31 (0) 499 33 69 79
Telefax: +31 (0) 499 33 69 70
E-mail: mike.looijmans@topicproducts.com
Website: www.topicproducts.com

Please consider the environment before printing this e-mail

Re: Changing external kernel module results in rebuild of whole kernel

[Richard Purdie]

On Tue, 2015-05-12 at 08:01 +0200, Mike Looijmans wrote:
> On 06-05-15 14:35, Richard Purdie wrote:
> > On Wed, 2015-05-06 at 08:35 +0200, Mike Looijmans wrote:
> >> Something in recent OE-core triggered a weird dependency "backfire".
> >>
> >> If I change a recipe for a kernel module (a bb recipe that does "inherit
> >> module") this will trigger a rebuild of the whole kernel.
> >>
> >> This turns the 5-second job of just updating a single module into a several
> >> minute workout for the build machine, and then causes boards to re-write the
> >> kernel into flash needlessly when upgrading.
> >>
> >> I now see this on all projects using OE-core master. I can't really pin what
> >> caused it though. Anyone else seen this?
> >
> > I have a suspicion this may be as a result of the changed kernel build
> > process in 1.8.
> >
> > The idea there is that the modules depend on the kernel source and
> > rather than taring up and then extracting a large (GB) sized sstate
> > object, we just extract the original kernel source.
> >
> > So is the kernel really rebuilding, or, is it just extracting source for
> > the kernel to build against? I noticed rm_work in your other post and
> > this may also be some bad interaction between rm_work and the kernel
> > build process changes.
> 
> Any things I can try or provide on this?
> 
> As things are now, I'd much prefer the "taring up and then extracting a large 
> (GB) sized sstate object" option, since that at least worked correctly.

Sorry for the delayed reply, I didn't understand exactly what was
happening without looking at a build. By far the easiest "fix" right now
is:

RM_WORK_EXCLUDE += "<my kernel recipe PN>"

This won't cost that much diskspace and should avoid the problem you're
seeing.

The problem is module.bbclass has a DEPENDS on virtual/kernel which
means it depends on <kernel>:do_populate_sysroot. As well as triggering
the unpack to repopulate the shared work area for the kernel, this means
it will cause the thing to effectively repackage.

There are various ideas coming to mind:

* We could drop virtual/kernel from DEPENDS which would shorten the
kernel dependency chain by removing populate_sysroot from the equation.

* We could change the task dependencies of populate_sysroot in
kernel.bbclass so that it doesn't depend on do_install (it doesn't
handle files any more now anyway).

* We could figure out why populate_sysroot from sstate isn't good enough
for bitbake and change the sstate code to use sstate more heavily. If I
remember rightly, we currently rerun all a task's tasks rather than pull
it half from sstate and half not.

None of these jumps out at me as an easy or necessarily desirable fix.

Cheers,

Richard

Re: Changing external kernel module results in rebuild of whole kernel

[Richard Purdie]

On Wed, 2015-05-13 at 11:33 +0100, Richard Purdie wrote:
> On Tue, 2015-05-12 at 08:01 +0200, Mike Looijmans wrote:
> > As things are now, I'd much prefer the "taring up and then extracting a large 
> > (GB) sized sstate object" option, since that at least worked correctly.
> 
> Sorry for the delayed reply, I didn't understand exactly what was
> happening without looking at a build. By far the easiest "fix" right now
> is:
> 
> RM_WORK_EXCLUDE += "<my kernel recipe PN>"
> 
> This won't cost that much diskspace and should avoid the problem you're
> seeing.
> 
> The problem is module.bbclass has a DEPENDS on virtual/kernel which
> means it depends on <kernel>:do_populate_sysroot. As well as triggering
> the unpack to repopulate the shared work area for the kernel, this means
> it will cause the thing to effectively repackage.
> 
> There are various ideas coming to mind:
> 
> * We could drop virtual/kernel from DEPENDS which would shorten the
> kernel dependency chain by removing populate_sysroot from the equation.
> 
> * We could change the task dependencies of populate_sysroot in
> kernel.bbclass so that it doesn't depend on do_install (it doesn't
> handle files any more now anyway).
> 
> * We could figure out why populate_sysroot from sstate isn't good enough
> for bitbake and change the sstate code to use sstate more heavily. If I
> remember rightly, we currently rerun all a task's tasks rather than pull
> it half from sstate and half not.
> 
> None of these jumps out at me as an easy or necessarily desirable fix.

I do have some patches:

https://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commit/?h=rpurdie/t222&id=c4aed2ba7ae74e54a4ae8ac7aeda291704bde82a
https://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commit/?h=rpurdie/t222&id=7a20f175c0a1cdec208471e5f7ff5f560f0b609e
https://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commit/?h=rpurdie/t222&id=c5f3cdca3e16d90c4e5bbb3abe5e136b8025f1a4

What I've noticed is that rm_work does not remove the data from
shared-work. What it does do is remove the stamp file which causes all
the tasks to rerun even if the data was there. The above patches make
"do_shared_work" a semi sstate task, one which is never generated into
the sstate feed but does have the right stamp functionality. This
basically stops kernel modules from rerunning kernel tasks even with
rm_work.

Whether these patches are a good idea, I'm less sure.

Bruce: You might want to look at the module.bbclass changes in the first
patch. I think there is some duplication between module and module-base
I need to remove.

Cheers,

Richard

Re: Changing external kernel module results in rebuild of whole kernel

[Bruce Ashfield]

On 2015-05-13 11:44 AM, Richard Purdie wrote:
> On Wed, 2015-05-13 at 11:33 +0100, Richard Purdie wrote:
>> On Tue, 2015-05-12 at 08:01 +0200, Mike Looijmans wrote:
>>> As things are now, I'd much prefer the "taring up and then extracting a large
>>> (GB) sized sstate object" option, since that at least worked correctly.
>>
>> Sorry for the delayed reply, I didn't understand exactly what was
>> happening without looking at a build. By far the easiest "fix" right now
>> is:
>>
>> RM_WORK_EXCLUDE += "<my kernel recipe PN>"
>>
>> This won't cost that much diskspace and should avoid the problem you're
>> seeing.
>>
>> The problem is module.bbclass has a DEPENDS on virtual/kernel which
>> means it depends on <kernel>:do_populate_sysroot. As well as triggering
>> the unpack to repopulate the shared work area for the kernel, this means
>> it will cause the thing to effectively repackage.
>>
>> There are various ideas coming to mind:
>>
>> * We could drop virtual/kernel from DEPENDS which would shorten the
>> kernel dependency chain by removing populate_sysroot from the equation.
>>
>> * We could change the task dependencies of populate_sysroot in
>> kernel.bbclass so that it doesn't depend on do_install (it doesn't
>> handle files any more now anyway).
>>
>> * We could figure out why populate_sysroot from sstate isn't good enough
>> for bitbake and change the sstate code to use sstate more heavily. If I
>> remember rightly, we currently rerun all a task's tasks rather than pull
>> it half from sstate and half not.
>>
>> None of these jumps out at me as an easy or necessarily desirable fix.
>
> I do have some patches:
>
> https://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commit/?h=rpurdie/t222&id=c4aed2ba7ae74e54a4ae8ac7aeda291704bde82a
> https://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commit/?h=rpurdie/t222&id=7a20f175c0a1cdec208471e5f7ff5f560f0b609e
> https://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commit/?h=rpurdie/t222&id=c5f3cdca3e16d90c4e5bbb3abe5e136b8025f1a4
>
> What I've noticed is that rm_work does not remove the data from
> shared-work. What it does do is remove the stamp file which causes all
> the tasks to rerun even if the data was there. The above patches make
> "do_shared_work" a semi sstate task, one which is never generated into
> the sstate feed but does have the right stamp functionality. This
> basically stops kernel modules from rerunning kernel tasks even with
> rm_work.
>
> Whether these patches are a good idea, I'm less sure.
>
> Bruce: You might want to look at the module.bbclass changes in the first
> patch. I think there is some duplication between module and module-base
> I need to remove.


I looked at the series, and it looks sane to me. Nice to move that
dependency to a common location, rather than requiring and end module
recipe (like lttng to get it correct).

For the overlap, I assume you were talking about them both now
having the do_configure[depends] += "virtual/kernel:do_shared_workdir" ?

Since module inherits module base, looks like that can be shuffled down
to the lowest class. It's also a bit odd (to my eyes) that module.bbclass
has the do_make_scripts dependency, while module-base.bbclass has the
actual routines ..

Definitely from sorting out that looks to be in order.

Bruce


>
> Cheers,
>
> Richard
>
>
>