[PATCH] kernel: improve initramfs bundle processing time

[PATCH] kernel: improve initramfs bundle processing time

[bruce.ashfield]

From: Bruce Ashfield <bruce.ashfield@gmail.com>

This is a partial fix for bugzilla 15059 [https://bugzilla.yoctoproject.org/show_bug.cgi?id=15059]

It has been noted by several people that when an initramfs is bundled:

  - a lot of the kernel is rebuilt
  - it takes a really long time

When looking at the logs, the second kernel compilation (that performs
the bundle) is not using the parallel make settings, and builds with
-j1.

We are already explicitly passing PARALLEL_MAKE when building kernel
modules, and by extending that explicit use to the main kernel
compilation, we ensure that we always get a parallel build.

Build times chnaged from more than 30 minutes for the bundle, to
3 minutes in local testing.

The question of whether or not too much is rebuilding during the
bundle step is still an open question, but with this tweak, at least
the build time is back in the realm of acceptable.

Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
---
 meta/classes-recipe/kernel.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes-recipe/kernel.bbclass b/meta/classes-recipe/kernel.bbclass
index aefa0d21bc..e634eabd49 100644
--- a/meta/classes-recipe/kernel.bbclass
+++ b/meta/classes-recipe/kernel.bbclass
@@ -382,7 +382,7 @@ kernel_do_compile() {
 		use_alternate_initrd=CONFIG_INITRAMFS_SOURCE=${B}/usr/${INITRAMFS_IMAGE_NAME}.cpio
 	fi
 	for typeformake in ${KERNEL_IMAGETYPE_FOR_MAKE} ; do
-		oe_runmake ${typeformake} ${KERNEL_EXTRA_ARGS} $use_alternate_initrd
+		oe_runmake ${PARALLEL_MAKE} ${typeformake} ${KERNEL_EXTRA_ARGS} $use_alternate_initrd
 	done
 }
 
-- 
2.25.1

Re: [OE-core] [PATCH] kernel: improve initramfs bundle processing time

[Paul Gortmaker]

[[OE-core] [PATCH] kernel: improve initramfs bundle processing time] On 14/04/2023 (Fri 15:29) Bruce Ashfield via lists.openembedded.org wrote:

> From: Bruce Ashfield <bruce.ashfield@gmail.com>
> 
> This is a partial fix for bugzilla 15059 [https://bugzilla.yoctoproject.org/show_bug.cgi?id=15059]
> 
> It has been noted by several people that when an initramfs is bundled:
> 
>   - a lot of the kernel is rebuilt
>   - it takes a really long time
> 
> When looking at the logs, the second kernel compilation (that performs
> the bundle) is not using the parallel make settings, and builds with
> -j1.
> 
> We are already explicitly passing PARALLEL_MAKE when building kernel
> modules, and by extending that explicit use to the main kernel
> compilation, we ensure that we always get a parallel build.
> 
> Build times chnaged from more than 30 minutes for the bundle, to
> 3 minutes in local testing.

I've seen this a bunch of times in the stuff I'm working on, and the
incremental fix to go from 30m to 3m is super welcome.  Thanks.

> The question of whether or not too much is rebuilding during the
> bundle step is still an open question, but with this tweak, at least
> the build time is back in the realm of acceptable.

Thank you for the incremental fix.  If I can help with the
over-rebuilding aspect, I will.  Let me know.

Paul.
--

> 
> Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
> ---
>  meta/classes-recipe/kernel.bbclass | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/meta/classes-recipe/kernel.bbclass b/meta/classes-recipe/kernel.bbclass
> index aefa0d21bc..e634eabd49 100644
> --- a/meta/classes-recipe/kernel.bbclass
> +++ b/meta/classes-recipe/kernel.bbclass
> @@ -382,7 +382,7 @@ kernel_do_compile() {
>  		use_alternate_initrd=CONFIG_INITRAMFS_SOURCE=${B}/usr/${INITRAMFS_IMAGE_NAME}.cpio
>  	fi
>  	for typeformake in ${KERNEL_IMAGETYPE_FOR_MAKE} ; do
> -		oe_runmake ${typeformake} ${KERNEL_EXTRA_ARGS} $use_alternate_initrd
> +		oe_runmake ${PARALLEL_MAKE} ${typeformake} ${KERNEL_EXTRA_ARGS} $use_alternate_initrd
>  	done
>  }
>  
> -- 
> 2.25.1
> 

> 
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#180005): https://lists.openembedded.org/g/openembedded-core/message/180005
> Mute This Topic: https://lists.openembedded.org/mt/98269754/6329300
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [paul.gortmaker@windriver.com]
> -=-=-=-=-=-=-=-=-=-=-=-
> 

Re: [OE-core] [PATCH] kernel: improve initramfs bundle processing time

[Nick Owens]

[-- Attachment #1: Type: text/plain, Size: 2482 bytes --]

hi bruce,

thanks for your work. looks good to me as a stopgap for now.

could this be backported to kirkstone?

nick

On Fri, Apr 14, 2023 at 12:29 PM Bruce Ashfield <bruce.ashfield@gmail.com>
wrote:

> From: Bruce Ashfield <bruce.ashfield@gmail.com>
>
> This is a partial fix for bugzilla 15059 [
> https://bugzilla.yoctoproject.org/show_bug.cgi?id=15059]
>
> It has been noted by several people that when an initramfs is bundled:
>
>   - a lot of the kernel is rebuilt
>   - it takes a really long time
>
> When looking at the logs, the second kernel compilation (that performs
> the bundle) is not using the parallel make settings, and builds with
> -j1.
>
> We are already explicitly passing PARALLEL_MAKE when building kernel
> modules, and by extending that explicit use to the main kernel
> compilation, we ensure that we always get a parallel build.
>
> Build times chnaged from more than 30 minutes for the bundle, to
> 3 minutes in local testing.
>
> The question of whether or not too much is rebuilding during the
> bundle step is still an open question, but with this tweak, at least
> the build time is back in the realm of acceptable.
>
> Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
> ---
>  meta/classes-recipe/kernel.bbclass | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/meta/classes-recipe/kernel.bbclass
> b/meta/classes-recipe/kernel.bbclass
> index aefa0d21bc..e634eabd49 100644
> --- a/meta/classes-recipe/kernel.bbclass
> +++ b/meta/classes-recipe/kernel.bbclass
> @@ -382,7 +382,7 @@ kernel_do_compile() {
>
> use_alternate_initrd=CONFIG_INITRAMFS_SOURCE=${B}/usr/${INITRAMFS_IMAGE_NAME}.cpio
>         fi
>         for typeformake in ${KERNEL_IMAGETYPE_FOR_MAKE} ; do
> -               oe_runmake ${typeformake} ${KERNEL_EXTRA_ARGS}
> $use_alternate_initrd
> +               oe_runmake ${PARALLEL_MAKE} ${typeformake}
> ${KERNEL_EXTRA_ARGS} $use_alternate_initrd
>         done
>  }
>
> --
> 2.25.1
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#180005):
> https://lists.openembedded.org/g/openembedded-core/message/180005
> Mute This Topic: https://lists.openembedded.org/mt/98269754/4454754
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [
> nick.owens@eero.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>

[-- Attachment #2: Type: text/html, Size: 3690 bytes --]