[PATCH v2 0/6] Use the kernel from sstate when building fitImages

[PATCH v2 0/6] Use the kernel from sstate when building fitImages

[Adrian Freihofer]

Changes in comparison to v1:
- Add the missing dependency from do_image_wic to
  initramfs.do_image_complete which caused the build failure on the AB.
- Support symbolic links for $dtb_path again, as suggested by Mark.
- Simplify the refactoring of the uboot_prep_kimage function.
- Evaluate staging via sysroot instead of taking the kernel artifacts
  from the deploy folder. But there is an issue with this approach: It
  does not allow to remove the dependency from do_deploy on do_install.
- Rebasing to latest master

If the KERNEL_IMAGETYPES(S) contains fitImage, the kernel is always
rebuilt when something changes in the initramfs frequently.
This is even worse if the build runs from an empty TMPDIR. The kernel
re-build starts by fetching the large kernel git repository and
recompiling it from scratch.

This cannot be improved if INITRAMFS_IMAGE_BUNDLE = "1". If the kernel
Makefile is needed to generate the initramfs bundle the kernel build
folder is required.
But for a build configuration with INITRAMFS_IMAGE_BUNDLE = "" the
build folder is not needed. Creating the initramfs bundle requires:
linux.bin, DTBs and the initramfs which are available in the deploy
directory. That means creating the fitImage can be done with artifacts
which are already cached by the sstate.

There is an extra commit providing a html file. This html file provides
some graphics which show the changes in the kernel's task dependencies.
I hope this simplifies the re-view of this patch series.

Testing:
- oe-selftest -a --skip-tests distrodata.Distrodata.test_checkpkg \
     reproducible -T yocto-mirrors -T machine -T toolchain-user \
     -T toolchain-system
- Build for core-image-minimal for MACHINE = "genericarm64"
  (failure from AB was reproducible and is fixed now)
- Build clean followed by build from sstate
  Note: Adding this test to oe-selftest failed because sstate is read only
  - Clean build config:
    KERNEL_IMAGETYPE = "Image"
    KERNEL_IMAGETYPES += " fitImage "
    KERNEL_CLASSES = " kernel-fitimage "
    IMAGE_FSTYPES += "cpio.gz"
    INITRAMFS_IMAGE = "core-image-minimal"
    IMAGE_NAME_SUFFIX:pn-core-image-minimal = ""
    UBOOT_RD_LOADADDRESS = "0x88000000"
    UBOOT_RD_ENTRYPOINT = "0x88000000"
    UBOOT_LOADADDRESS = "0x80080000"
    UBOOT_ENTRYPOINT = "0x80080000"
    FIT_DESC = "A model description"
    FOO_VAR = "1"
    INHERIT += "image-buildinfo"
    IMAGE_BUILDINFO_VARS:append = " FOO_VAR"
  - Append the following and rebuild with sstate:
    FOO_VAR = "2"
    TMPDIR = "${TOPDIR}/tmp-2"
  - Check the log files:
    - tmp/log/cooker/qemux86-64 contains:
      linux-yocto-6.6.35+git-r0: task do_deploy: Succeeded
      linux-yocto-6.6.35+git-r0: task do_deploy_fitimage_unbundled: Succeeded
    - tmp-2/log/cooker/qemux86-64 contains:
      linux-yocto-6.6.35+git-r0: task do_deploy_setscene: Succeeded
      linux-yocto-6.6.35+git-r0: task do_deploy_fitimage_unbundled: Succeeded
- To re-view the chagnes in the task dependencies, the script bellow
  has been used. It confirms:
  - For builds with fitImage and unbundled initramfs:
    - the do_deploy_fitimage_unbundled task runs after do_deploy
    - do_assemble_fitimage_initramfs is not executed
    - do_bundle_initramfs is not executed
    - It works for fitImage in KERNEL_IMAGETYPE as well as for fitImage
      in KERNEL_IMAGETYPES
  - For builds with fitImage and bundled initramfs: No changes
  - For builds with fitImage, without initramfs:
    - do_assemble_fitimage_initramfs is not executed
    - do_bundle_initramfs is not executed

OUTPUT_FILE=task-depends.md
GIT_BRANCH=master
GIT_BRANCH_NEXT=adrianf/kernel-fitimage-sstate

echo "# Task dependeny changes" > "$OUTPUT_FILE"
echo "" >> "$OUTPUT_FILE"

run_bitbake(){
    echo "$1" >> "$OUTPUT_FILE"
    echo "" >> "$OUTPUT_FILE"

    bitbake virtual/kernel -g
    grep -E '(digraph depends|linux-yocto|\})' task-depends.dot \
    | grep -v -E '(spdx|do_kernel_configcheck|do_prepare_recipe_sysroot|do_populate_lic|native.do_populate_sysroot)' \
    | grep -v -E '(depmodwrapper-cross.do_populate_sysroot|binutils-cross-x86_64.do_populate_sysroot|do_package)' \
    | grep -v -E '(do_validate_branches|do_unpack|do_recipe_qa|do_patch|do_kernel_checkout|do_kernel_configme|do_kernel_metadata)' \
    | grep -v -E '(do_sizecheck|do_strip|do_compile_kernelmodules|do_shared_workdir|do_transform_kernel|do_kernel_link_images)' \
    | grep -v -E '(do_kernel_version_sanity_check|do_symlink_kernsrc|do_deploy_source_date_epoch|do_configure|do_fetch)' \
    | sed -e 's;\\n\:.*.bb;;g' -e 's;linux-yocto[. ];;g' > task-depends-filtered.dot

    echo '```plantuml' >> "$OUTPUT_FILE"
    cat task-depends-filtered.dot >> "$OUTPUT_FILE"
    echo '```' >> "$OUTPUT_FILE"
    echo "" >> "$OUTPUT_FILE"
}

run_bitbake_per_branch(){
    echo "## Configuration: $1" >> "$OUTPUT_FILE"
    echo "" >> "$OUTPUT_FILE"
    echo '```raw' >> "$OUTPUT_FILE"
    cat build/conf/auto.conf >> "$OUTPUT_FILE"
    echo '```' >> "$OUTPUT_FILE"
    echo "" >> "$OUTPUT_FILE"

    git checkout $GIT_BRANCH
    run_bitbake "### branch: $GIT_BRANCH"
    mv -f task-depends-filtered.dot task-depends-filtered-old.dot

    git checkout $GIT_BRANCH_NEXT
    run_bitbake "### branch: $GIT_BRANCH_NEXT"

    echo "## Diff" >> "$OUTPUT_FILE"
    echo "" >> "$OUTPUT_FILE"
    echo '```patch' >> "$OUTPUT_FILE"
    diff task-depends-filtered-old.dot task-depends-filtered.dot >> "$OUTPUT_FILE"
    echo '```' >> "$OUTPUT_FILE"
    echo "" >> "$OUTPUT_FILE"
}

cat << EOF > build/conf/auto.conf
KERNEL_IMAGETYPE = "Image"
KERNEL_IMAGETYPES += " fitImage "
KERNEL_CLASSES = " kernel-fitimage "
IMAGE_FSTYPES += "cpio.gz"
INITRAMFS_IMAGE = "core-image-minimal-initramfs"
EOF
run_bitbake_per_branch "image, fitimage, initramfs, unbundled"

cat << EOF > build/conf/auto.conf
KERNEL_IMAGETYPE:forcevariable = "fitImage"
KERNEL_CLASSES = " kernel-fitimage "
IMAGE_FSTYPES += "cpio.gz"
INITRAMFS_IMAGE = "core-image-minimal-initramfs"
EOF
run_bitbake_per_branch "fitimage, initramfs, unbundled"

cat << EOF > build/conf/auto.conf
KERNEL_IMAGETYPE = "Image"
KERNEL_IMAGETYPES += " fitImage "
KERNEL_CLASSES = " kernel-fitimage "
IMAGE_FSTYPES += "cpio.gz"
INITRAMFS_IMAGE = "core-image-minimal-initramfs"
INITRAMFS_IMAGE_BUNDLE = "1"
EOF
run_bitbake_per_branch "image, fitimage, initramfs, bundled"

cat << EOF > build/conf/auto.conf
KERNEL_IMAGETYPE = "Image"
KERNEL_IMAGETYPES += " fitImage "
KERNEL_CLASSES = " kernel-fitimage "
EOF
run_bitbake_per_branch "image, fitimage"

rm -f build/conf/auto.conf task-depends-filtered-old.dot task-depends-filtered.dot task-depends.dot

Adrian Freihofer (6):
  kernel-fitimage: fix intentation
  kernel-fitimage: fix external dtb check
  kernel: refactor linux compression
  kernel-fitimage: refactor fitimage_assemble
  kernel: refactor fitimage
  kernel-fitimage: run unbundled fitimage after deploy

 meta/classes-recipe/image.bbclass           |  12 +-
 meta/classes-recipe/kernel-fitimage.bbclass | 188 ++++++++++++--------
 meta/classes-recipe/kernel-uboot.bbclass    |   1 +
 meta/classes-recipe/kernel.bbclass          |  31 ++--
 4 files changed, 148 insertions(+), 84 deletions(-)

-- 
2.45.2

[PATCH 1/6] kernel-fitimage: fix intentation

[Adrian Freihofer]

From: Adrian Freihofer <adrian.freihofer@siemens.com>

white space changes only.
- python part should be 4 spaces, not 8.
- use tabs for shell

Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
---
 meta/classes-recipe/kernel-fitimage.bbclass | 64 ++++++++++-----------
 1 file changed, 32 insertions(+), 32 deletions(-)

diff --git a/meta/classes-recipe/kernel-fitimage.bbclass b/meta/classes-recipe/kernel-fitimage.bbclass
index 6e6898e5f6f..ce52297aec3 100644
--- a/meta/classes-recipe/kernel-fitimage.bbclass
+++ b/meta/classes-recipe/kernel-fitimage.bbclass
@@ -29,27 +29,27 @@ KERNEL_IMAGETYPE_REPLACEMENT ?= "${@get_fit_replacement_type(d)}"
 DEPENDS:append = " ${@'u-boot-tools-native dtc-native' if 'fitImage' in (d.getVar('KERNEL_IMAGETYPES') or '').split() else ''}"
 
 python __anonymous () {
-        # Override KERNEL_IMAGETYPE_FOR_MAKE variable, which is internal
-        # to kernel.bbclass . We have to override it, since we pack zImage
-        # (at least for now) into the fitImage .
-        typeformake = d.getVar("KERNEL_IMAGETYPE_FOR_MAKE") or ""
-        if 'fitImage' in typeformake.split():
-            d.setVar('KERNEL_IMAGETYPE_FOR_MAKE', typeformake.replace('fitImage', d.getVar('KERNEL_IMAGETYPE_REPLACEMENT')))
-
-        image = d.getVar('INITRAMFS_IMAGE')
-        if image:
-            d.appendVarFlag('do_assemble_fitimage_initramfs', 'depends', ' ${INITRAMFS_IMAGE}:do_image_complete')
-
-        ubootenv = d.getVar('UBOOT_ENV')
-        if ubootenv:
-            d.appendVarFlag('do_assemble_fitimage', 'depends', ' virtual/bootloader:do_populate_sysroot')
-
-        #check if there are any dtb providers
-        providerdtb = d.getVar("PREFERRED_PROVIDER_virtual/dtb")
-        if providerdtb:
-            d.appendVarFlag('do_assemble_fitimage', 'depends', ' virtual/dtb:do_populate_sysroot')
-            d.appendVarFlag('do_assemble_fitimage_initramfs', 'depends', ' virtual/dtb:do_populate_sysroot')
-            d.setVar('EXTERNAL_KERNEL_DEVICETREE', "${RECIPE_SYSROOT}/boot/devicetree")
+    # Override KERNEL_IMAGETYPE_FOR_MAKE variable, which is internal
+    # to kernel.bbclass . We have to override it, since we pack zImage
+    # (at least for now) into the fitImage .
+    typeformake = d.getVar("KERNEL_IMAGETYPE_FOR_MAKE") or ""
+    if 'fitImage' in typeformake.split():
+        d.setVar('KERNEL_IMAGETYPE_FOR_MAKE', typeformake.replace('fitImage', d.getVar('KERNEL_IMAGETYPE_REPLACEMENT')))
+
+    image = d.getVar('INITRAMFS_IMAGE')
+    if image:
+        d.appendVarFlag('do_assemble_fitimage_initramfs', 'depends', ' ${INITRAMFS_IMAGE}:do_image_complete')
+
+    ubootenv = d.getVar('UBOOT_ENV')
+    if ubootenv:
+        d.appendVarFlag('do_assemble_fitimage', 'depends', ' virtual/bootloader:do_populate_sysroot')
+
+    #check if there are any dtb providers
+    providerdtb = d.getVar("PREFERRED_PROVIDER_virtual/dtb")
+    if providerdtb:
+        d.appendVarFlag('do_assemble_fitimage', 'depends', ' virtual/dtb:do_populate_sysroot')
+        d.appendVarFlag('do_assemble_fitimage_initramfs', 'depends', ' virtual/dtb:do_populate_sysroot')
+        d.setVar('EXTERNAL_KERNEL_DEVICETREE', "${RECIPE_SYSROOT}/boot/devicetree")
 }
 
 
@@ -480,13 +480,13 @@ fitimage_emit_section_config() {
 		# default node is selected based on dtb ID if it is present,
 		# otherwise its selected based on kernel ID
 		if [ -n "$dtb_image" ]; then
-		        # Select default node as user specified dtb when
-		        # multiple dtb exists.
-		        if [ -n "$default_dtb_image" ]; then
-			        default_line="default = \"${FIT_CONF_PREFIX}$default_dtb_image\";"
-		        else
-			        default_line="default = \"${FIT_CONF_PREFIX}$dtb_image\";"
-		        fi
+			# Select default node as user specified dtb when
+			# multiple dtb exists.
+			if [ -n "$default_dtb_image" ]; then
+				default_line="default = \"${FIT_CONF_PREFIX}$default_dtb_image\";"
+			else
+				default_line="default = \"${FIT_CONF_PREFIX}$dtb_image\";"
+			fi
 		else
 			default_line="default = \"${FIT_CONF_PREFIX}$kernel_id\";"
 		fi
@@ -605,9 +605,9 @@ fitimage_assemble() {
 				DTB_PATH="${KERNEL_OUTPUT_DIR}/$DTB"
 			fi
 
-		        # Strip off the path component from the filename
+			# Strip off the path component from the filename
 			if "${@'false' if oe.types.boolean(d.getVar('KERNEL_DTBVENDORED')) else 'true'}"; then
-			    DTB=`basename $DTB`
+				DTB=`basename $DTB`
 			fi
 
 			# Set the default dtb image if it exists in the devicetree.
@@ -715,8 +715,8 @@ fitimage_assemble() {
 	# kernel-fitimage.bbclass currently only supports a single kernel (no less or
 	# more) to be added to the FIT image along with 0 or more device trees and
 	# 0 or 1 ramdisk.
-        # It is also possible to include an initramfs bundle (kernel and rootfs in one binary)
-        # When the initramfs bundle is used ramdisk is disabled.
+	# It is also possible to include an initramfs bundle (kernel and rootfs in one binary)
+	# When the initramfs bundle is used ramdisk is disabled.
 	# If a device tree is to be part of the FIT image, then select
 	# the default configuration to be used is based on the dtbcount. If there is
 	# no dtb present than select the default configuation to be based on
-- 
2.45.2

[PATCH 2/6] kernel-fitimage: fix external dtb check

[Adrian Freihofer]

From: Adrian Freihofer <adrian.freihofer@siemens.com>

If EXTERNAL_KERNEL_DEVICETREE and dtb_image_sect are empty variables
dtb_path ends up as "/" which is available on most Unix systems but
probably not the dtb_path which is needed here. Checking for a file
makes more sense and also solves the issue with the "/".

Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
---
 meta/classes-recipe/kernel-fitimage.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes-recipe/kernel-fitimage.bbclass b/meta/classes-recipe/kernel-fitimage.bbclass
index ce52297aec3..67c98adb232 100644
--- a/meta/classes-recipe/kernel-fitimage.bbclass
+++ b/meta/classes-recipe/kernel-fitimage.bbclass
@@ -429,7 +429,7 @@ fitimage_emit_section_config() {
 	fi
 
 	dtb_path="${EXTERNAL_KERNEL_DEVICETREE}/${dtb_image_sect}"
-	if [ -e "$dtb_path" ]; then
+	if [ -f "$dtb_path" ] || [ -L "$dtb_path" ]; then
 		compat=$(fdtget -t s "$dtb_path" / compatible | sed 's/ /", "/g')
 		if [ -n "$compat" ]; then
 			compatible_line="compatible = \"$compat\";"
-- 
2.45.2

[PATCH 3/6] kernel: refactor linux compression

[Adrian Freihofer]

From: Adrian Freihofer <adrian.freihofer@siemens.com>

This is a step to avoid unecessary kernel rebuilds when the initramfs
changes but not the kernel itself. In case of an unbundled fitImage the
kernel should be staged in the sstate instead of re-compiled.

The uboot_prep_kimage function generates the kernel binary which gets
included into the fitImage. It also defines the variable linux_comp
which is used by the fitimage_assemble function later on.
To make the fitimage_assemble function working with kernel binaries from
sstate it must be decoupled from calling uboot_prep_kimage in ${B} by:
- refactoring uboot_prep_kimage to write the result into files
  (linux.bin, linux.comp).
- calling the uboot_prep_kimage before the artifacts are used or cached.
- refactoring the fitimage_assemble function to use files rather than
  calling the uboot_prep_kimage function which depends on ${B}.

Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
---
 meta/classes-recipe/kernel-fitimage.bbclass | 5 ++---
 meta/classes-recipe/kernel-uboot.bbclass    | 1 +
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/meta/classes-recipe/kernel-fitimage.bbclass b/meta/classes-recipe/kernel-fitimage.bbclass
index 67c98adb232..02b5fd8e3f5 100644
--- a/meta/classes-recipe/kernel-fitimage.bbclass
+++ b/meta/classes-recipe/kernel-fitimage.bbclass
@@ -579,9 +579,7 @@ fitimage_assemble() {
 	# Step 1: Prepare a kernel image section.
 	#
 	fitimage_emit_section_maint $1 imagestart
-
-	uboot_prep_kimage
-	fitimage_emit_section_kernel $1 $kernelcount linux.bin "$linux_comp"
+	fitimage_emit_section_kernel $1 $kernelcount linux.bin "$(cat linux.comp)"
 
 	#
 	# Step 2: Prepare a DTB image section
@@ -764,6 +762,7 @@ fitimage_assemble() {
 do_assemble_fitimage() {
 	if echo ${KERNEL_IMAGETYPES} | grep -wq "fitImage"; then
 		cd ${B}
+		uboot_prep_kimage
 		fitimage_assemble fit-image.its fitImage-none ""
 		if [ "${INITRAMFS_IMAGE_BUNDLE}" != "1" ]; then
 			ln -sf fitImage-none ${B}/${KERNEL_OUTPUT_DIR}/fitImage
diff --git a/meta/classes-recipe/kernel-uboot.bbclass b/meta/classes-recipe/kernel-uboot.bbclass
index 30a85ccc287..50be7962d46 100644
--- a/meta/classes-recipe/kernel-uboot.bbclass
+++ b/meta/classes-recipe/kernel-uboot.bbclass
@@ -45,5 +45,6 @@ uboot_prep_kimage() {
 		mv -f "linux.bin${linux_suffix}" linux.bin
 	fi
 
+	[ -n "${vmlinux_path}" ] && echo "${linux_comp}" > linux.comp
 	echo "${linux_comp}"
 }
-- 
2.45.2

[PATCH 4/6] kernel-fitimage: refactor fitimage_assemble

[Adrian Freihofer]

From: Adrian Freihofer <adrian.freihofer@siemens.com>

Make the fitimage_assemble function usable with absolute paths for the
generated its and the fitImage file.

Later on this will allow to take the linux.bin and the DTB files from
the sstated deploy derectory and write the generated files to a separate
folder with independent sstate-cache handling.

Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
---
 meta/classes-recipe/kernel-fitimage.bbclass | 32 +++++++++++++++------
 1 file changed, 24 insertions(+), 8 deletions(-)

diff --git a/meta/classes-recipe/kernel-fitimage.bbclass b/meta/classes-recipe/kernel-fitimage.bbclass
index 02b5fd8e3f5..705b3ab657f 100644
--- a/meta/classes-recipe/kernel-fitimage.bbclass
+++ b/meta/classes-recipe/kernel-fitimage.bbclass
@@ -169,8 +169,9 @@ fitimage_emit_section_kernel() {
 
 	ENTRYPOINT="${UBOOT_ENTRYPOINT}"
 	if [ -n "${UBOOT_ENTRYSYMBOL}" ]; then
+		kernel_base=$(basename $3)
 		ENTRYPOINT=`${HOST_PREFIX}nm vmlinux | \
-			awk '$3=="${UBOOT_ENTRYSYMBOL}" {print "0x"$1;exit}'`
+			awk '$kernel_base=="${UBOOT_ENTRYSYMBOL}" {print "0x"$1;exit}'`
 	fi
 
 	cat << EOF >> $1
@@ -567,7 +568,7 @@ fitimage_assemble() {
 	setupcount=""
 	bootscr_id=""
 	default_dtb_image=""
-	rm -f $1 arch/${ARCH}/boot/$2
+	rm -f "$1" "arch/${ARCH}/boot/$(basename $2)"
 
 	if [ -n "${UBOOT_SIGN_IMG_KEYNAME}" -a "${UBOOT_SIGN_KEYNAME}" = "${UBOOT_SIGN_IMG_KEYNAME}" ]; then
 		bbfatal "Keys used to sign images and configuration nodes must be different."
@@ -608,6 +609,17 @@ fitimage_assemble() {
 				DTB=`basename $DTB`
 			fi
 
+			# Find DTBs without sub-folders when running in deploy folder
+			if [ ! -e "$DTB_PATH" ]; then
+				DTB=$(basename $DTB)
+				DTB_PATH=$(readlink -f $DTB)
+			fi
+
+			# Fail as early as possible if there is still no DTB file found
+			if [ ! -e "$DTB_PATH" ]; then
+				bberror "Cannot find the DTB file at $DTB_PATH"
+			fi
+
 			# Set the default dtb image if it exists in the devicetree.
 			if [ "${FIT_CONF_DEFAULT_DTB}" = "$DTB" ];then
 				default_dtb_image=$(echo "$DTB" | tr '/' '_')
@@ -658,9 +670,8 @@ fitimage_assemble() {
 
 	if [ -n "${UBOOT_ENV}" ] && [ -d "${STAGING_DIR_HOST}/boot" ]; then
 		if [ -e "${STAGING_DIR_HOST}/boot/${UBOOT_ENV_BINARY}" ]; then
-			cp ${STAGING_DIR_HOST}/boot/${UBOOT_ENV_BINARY} ${B}
 			bootscr_id="${UBOOT_ENV_BINARY}"
-			fitimage_emit_section_boot_script $1 "$bootscr_id" ${UBOOT_ENV_BINARY}
+			fitimage_emit_section_boot_script $1 "$bootscr_id" "${STAGING_DIR_HOST}/boot/${UBOOT_ENV_BINARY}"
 		else
 			bbwarn "${STAGING_DIR_HOST}/boot/${UBOOT_ENV_BINARY} not found."
 		fi
@@ -669,9 +680,14 @@ fitimage_assemble() {
 	#
 	# Step 4: Prepare a setup section. (For x86)
 	#
+	# Run from kernel build folder (bundled mode)
 	if [ -e ${KERNEL_OUTPUT_DIR}/setup.bin ]; then
 		setupcount=1
 		fitimage_emit_section_setup $1 $setupcount ${KERNEL_OUTPUT_DIR}/setup.bin
+	# Run from deploy folder (unbundled mode)
+	elif [ -e setup.bin ]; then
+		setupcount=1
+		fitimage_emit_section_setup $1 $setupcount "$(readlink -f setup.bin)"
 	fi
 
 	#
@@ -744,8 +760,8 @@ fitimage_assemble() {
 	#
 	${UBOOT_MKIMAGE} \
 		${@'-D "${UBOOT_MKIMAGE_DTCOPTS}"' if len('${UBOOT_MKIMAGE_DTCOPTS}') else ''} \
-		-f $1 \
-		${KERNEL_OUTPUT_DIR}/$2
+		-f "$1" \
+		"$2"
 
 	#
 	# Step 8: Sign the image
@@ -754,7 +770,7 @@ fitimage_assemble() {
 		${UBOOT_MKIMAGE_SIGN} \
 			${@'-D "${UBOOT_MKIMAGE_DTCOPTS}"' if len('${UBOOT_MKIMAGE_DTCOPTS}') else ''} \
 			-F -k "${UBOOT_SIGN_KEYDIR}" \
-			-r ${KERNEL_OUTPUT_DIR}/$2 \
+			-r "$2" \
 			${UBOOT_MKIMAGE_SIGN_ARGS}
 	fi
 }
@@ -763,7 +779,7 @@ do_assemble_fitimage() {
 	if echo ${KERNEL_IMAGETYPES} | grep -wq "fitImage"; then
 		cd ${B}
 		uboot_prep_kimage
-		fitimage_assemble fit-image.its fitImage-none ""
+		fitimage_assemble fit-image.its ${KERNEL_OUTPUT_DIR}/fitImage-none ""
 		if [ "${INITRAMFS_IMAGE_BUNDLE}" != "1" ]; then
 			ln -sf fitImage-none ${B}/${KERNEL_OUTPUT_DIR}/fitImage
 		fi
-- 
2.45.2

[PATCH 5/6] kernel: refactor fitimage

[Adrian Freihofer]

From: Adrian Freihofer <adrian.freihofer@siemens.com>

* do_bundle_initramfs only when needed
  With INITRAMFS_IMAGE_BUNDLE = "1" the kernel Makefile is used to
  bundle the kernel binary and the initramfs. This works only when the
  build folder of the kernel is available. Building with sstate but with
  an empty TMPDIR requires to rebuild the kernel from scratch whenever
  something in the initramfs changes.

  With INITRAMFS_IMAGE_BUNDLE = "" the fitImage generation is basically
  independent from the kernel Makefile and could therefore work with
  kernel binaries and tools provided by the sstate. But the dependency
  on the do_bundle_initramfs tasks does not allow this. However, if the
  INTIRAMFS_IMAGE is set but the INITRAMFS_IMAGE_BUNDLE is set to 0 the
  do_bundle_initramfs does nothing. There is no real argument for
  running this tasks.

  As a first step towards getting the kernel from sstate when building
  an unbundled fitImage, the task dependencies need to be simplified.
  The do_bundle_initramfs task is now scheduled only when needed.

* If KERNEL_IMAGETYPES does not contain fitImage and INITRAMFS_IMAGE is
  not set, the do_assemble_fitimage_initramfs ends up as an empty task.
  Add the do_assemble_fitimage_initramfs only if a fitImage with
  initramfs needs to be built.

* cd in bitbake task is easy to break e.g. by bbappend. Replace the cd
  by defining the task's dirs variable flag.

* Add 3 comments which are helpful for the next commits.

* Since the tasks and task dependencies are anyway created by an
  anonymous function, align and make the DEPENDS code more specific too.

This refactoring is not very valuable on its own. But it simplifies the
task dependencies which is helpful for the next commit.

Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
---
 meta/classes-recipe/kernel-fitimage.bbclass | 57 ++++++++++++---------
 meta/classes-recipe/kernel.bbclass          | 28 ++++++----
 2 files changed, 50 insertions(+), 35 deletions(-)

diff --git a/meta/classes-recipe/kernel-fitimage.bbclass b/meta/classes-recipe/kernel-fitimage.bbclass
index 705b3ab657f..2dda5aec056 100644
--- a/meta/classes-recipe/kernel-fitimage.bbclass
+++ b/meta/classes-recipe/kernel-fitimage.bbclass
@@ -26,20 +26,8 @@ def get_fit_replacement_type(d):
     return replacementtype
 
 KERNEL_IMAGETYPE_REPLACEMENT ?= "${@get_fit_replacement_type(d)}"
-DEPENDS:append = " ${@'u-boot-tools-native dtc-native' if 'fitImage' in (d.getVar('KERNEL_IMAGETYPES') or '').split() else ''}"
 
 python __anonymous () {
-    # Override KERNEL_IMAGETYPE_FOR_MAKE variable, which is internal
-    # to kernel.bbclass . We have to override it, since we pack zImage
-    # (at least for now) into the fitImage .
-    typeformake = d.getVar("KERNEL_IMAGETYPE_FOR_MAKE") or ""
-    if 'fitImage' in typeformake.split():
-        d.setVar('KERNEL_IMAGETYPE_FOR_MAKE', typeformake.replace('fitImage', d.getVar('KERNEL_IMAGETYPE_REPLACEMENT')))
-
-    image = d.getVar('INITRAMFS_IMAGE')
-    if image:
-        d.appendVarFlag('do_assemble_fitimage_initramfs', 'depends', ' ${INITRAMFS_IMAGE}:do_image_complete')
-
     ubootenv = d.getVar('UBOOT_ENV')
     if ubootenv:
         d.appendVarFlag('do_assemble_fitimage', 'depends', ' virtual/bootloader:do_populate_sysroot')
@@ -48,8 +36,31 @@ python __anonymous () {
     providerdtb = d.getVar("PREFERRED_PROVIDER_virtual/dtb")
     if providerdtb:
         d.appendVarFlag('do_assemble_fitimage', 'depends', ' virtual/dtb:do_populate_sysroot')
-        d.appendVarFlag('do_assemble_fitimage_initramfs', 'depends', ' virtual/dtb:do_populate_sysroot')
         d.setVar('EXTERNAL_KERNEL_DEVICETREE', "${RECIPE_SYSROOT}/boot/devicetree")
+
+    typeformake = d.getVar("KERNEL_IMAGETYPE_FOR_MAKE") or ""
+    if 'fitImage' in typeformake.split():
+        # Override KERNEL_IMAGETYPE_FOR_MAKE variable, which is internal
+        # to kernel.bbclass . We have to override it, since we pack zImage
+        # (at least for now) into the fitImage .
+        d.setVar('KERNEL_IMAGETYPE_FOR_MAKE', typeformake.replace('fitImage', d.getVar('KERNEL_IMAGETYPE_REPLACEMENT')))
+
+        # mkimage and dtc are required by the fitimage_assemble function
+        d.appendVarFlag('do_assemble_fitimage', 'depends',
+                        ' u-boot-tools-native:do_populate_sysroot dtc-native:do_populate_sysroot')
+
+        initramfs_image = d.getVar('INITRAMFS_IMAGE')
+        bundled = bb.utils.to_boolean(d.getVar('INITRAMFS_IMAGE_BUNDLE'))
+        if initramfs_image:
+            if bundled:
+                bb.build.addtask('do_assemble_fitimage_initramfs', 'do_deploy', 'do_bundle_initramfs', d)
+            else:
+                bb.build.addtask('do_assemble_fitimage_initramfs', 'do_deploy', 'do_install', d)
+
+            d.appendVarFlag('do_assemble_fitimage_initramfs', 'depends',
+                            ' u-boot-tools-native:do_populate_sysroot dtc-native:do_populate_sysroot ${INITRAMFS_IMAGE}:do_image_complete')
+            if providerdtb:
+                d.appendVarFlag('do_assemble_fitimage_initramfs', 'depends', ' virtual/dtb:do_populate_sysroot')
 }
 
 
@@ -797,19 +808,14 @@ do_install:append() {
 }
 
 do_assemble_fitimage_initramfs() {
-	if echo ${KERNEL_IMAGETYPES} | grep -wq "fitImage" && \
-		test -n "${INITRAMFS_IMAGE}" ; then
-		cd ${B}
-		if [ "${INITRAMFS_IMAGE_BUNDLE}" = "1" ]; then
-			fitimage_assemble fit-image-${INITRAMFS_IMAGE}.its fitImage-bundle ""
-			ln -sf fitImage-bundle ${B}/${KERNEL_OUTPUT_DIR}/fitImage
-		else
-			fitimage_assemble fit-image-${INITRAMFS_IMAGE}.its fitImage-${INITRAMFS_IMAGE} 1
-		fi
+	if [ "${INITRAMFS_IMAGE_BUNDLE}" = "1" ]; then
+		fitimage_assemble "fit-image-${INITRAMFS_IMAGE}.its" "${KERNEL_OUTPUT_DIR}/fitImage-bundle" ""
+		ln -sf fitImage-bundle "${B}/${KERNEL_OUTPUT_DIR}/fitImage"
+	else
+		fitimage_assemble "fit-image-${INITRAMFS_IMAGE}.its" "${KERNEL_OUTPUT_DIR}/fitImage-${INITRAMFS_IMAGE}" 1
 	fi
 }
-
-addtask assemble_fitimage_initramfs before do_deploy after do_bundle_initramfs
+do_assemble_fitimage_initramfs[dirs] = "${B}"
 
 do_kernel_generate_rsa_keys() {
 	if [ "${UBOOT_SIGN_ENABLE}" = "0" ] && [ "${FIT_GENERATE_KEYS}" = "1" ]; then
@@ -864,6 +870,7 @@ kernel_do_deploy:append() {
 	if echo ${KERNEL_IMAGETYPES} | grep -wq "fitImage"; then
 
 		if [ "${INITRAMFS_IMAGE_BUNDLE}" != "1" ]; then
+			# deploy the artifacts of do_assemble_fitimage
 			bbnote "Copying fit-image.its source file..."
 			install -m 0644 ${B}/fit-image.its "$deployDir/fitImage-its-${KERNEL_FIT_NAME}.its"
 			if [ -n "${KERNEL_FIT_LINK_NAME}" ] ; then
@@ -878,12 +885,14 @@ kernel_do_deploy:append() {
 		fi
 
 		if [ -n "${INITRAMFS_IMAGE}" ]; then
+			# deploy the artifacts of do_assemble_fitimage_initramfs for bundled as well as un-bundled mode
 			bbnote "Copying fit-image-${INITRAMFS_IMAGE}.its source file..."
 			install -m 0644 ${B}/fit-image-${INITRAMFS_IMAGE}.its "$deployDir/fitImage-its-${INITRAMFS_IMAGE_NAME}-${KERNEL_FIT_NAME}.its"
 			if [ -n "${KERNEL_FIT_LINK_NAME}" ] ; then
 				ln -snf fitImage-its-${INITRAMFS_IMAGE_NAME}-${KERNEL_FIT_NAME}.its "$deployDir/fitImage-its-${INITRAMFS_IMAGE_NAME}-${KERNEL_FIT_LINK_NAME}"
 			fi
 
+			# deploy the artifacts of do_assemble_fitimage_initramfs for bundled mode only
 			if [ "${INITRAMFS_IMAGE_BUNDLE}" != "1" ]; then
 				bbnote "Copying fitImage-${INITRAMFS_IMAGE} file..."
 				install -m 0644 ${B}/${KERNEL_OUTPUT_DIR}/fitImage-${INITRAMFS_IMAGE} "$deployDir/fitImage-${INITRAMFS_IMAGE_NAME}-${KERNEL_FIT_NAME}${KERNEL_FIT_BIN_EXT}"
diff --git a/meta/classes-recipe/kernel.bbclass b/meta/classes-recipe/kernel.bbclass
index 89badd90f18..6a8c3c25c07 100644
--- a/meta/classes-recipe/kernel.bbclass
+++ b/meta/classes-recipe/kernel.bbclass
@@ -136,18 +136,24 @@ fi
 set -e
 """ % (type, type, type))
 
-
     image = d.getVar('INITRAMFS_IMAGE')
-    # If the INTIRAMFS_IMAGE is set but the INITRAMFS_IMAGE_BUNDLE is set to 0,
-    # the do_bundle_initramfs does nothing, but the INITRAMFS_IMAGE is built
-    # standalone for use by wic and other tools.
     if image:
-        if d.getVar('INITRAMFS_MULTICONFIG'):
-            d.appendVarFlag('do_bundle_initramfs', 'mcdepends', ' mc::${INITRAMFS_MULTICONFIG}:${INITRAMFS_IMAGE}:do_image_complete')
+        # If the INTIRAMFS_IMAGE is set but the INITRAMFS_IMAGE_BUNDLE is set to 0,
+        # the do_bundle_initramfs is not needed, but the INITRAMFS_IMAGE is built
+        # standalone for use by wic and other tools.
+        def add_initramfs_dep_task(initramfs_dep_task, d):
+            if d.getVar('INITRAMFS_MULTICONFIG'):
+                d.appendVarFlag(initramfs_dep_task, 'mcdepends', ' mc::${INITRAMFS_MULTICONFIG}:${INITRAMFS_IMAGE}:do_image_complete')
+            else:
+                d.appendVarFlag(initramfs_dep_task, 'depends', ' ${INITRAMFS_IMAGE}:do_image_complete')
+
+        if bb.utils.to_boolean(d.getVar('INITRAMFS_IMAGE_BUNDLE')):
+            bb.build.addtask('do_bundle_initramfs', 'do_deploy', 'do_install', d)
+            add_initramfs_dep_task('do_bundle_initramfs', d)
+            bb.build.addtask('do_transform_bundled_initramfs', 'do_deploy', 'do_bundle_initramfs', d)
         else:
-            d.appendVarFlag('do_bundle_initramfs', 'depends', ' ${INITRAMFS_IMAGE}:do_image_complete')
-    if image and bb.utils.to_boolean(d.getVar('INITRAMFS_IMAGE_BUNDLE')):
-        bb.build.addtask('do_transform_bundled_initramfs', 'do_deploy', 'do_bundle_initramfs', d)
+            add_initramfs_dep_task('do_deploy', d)
+            d.appendVarFlag('do_deploy', 'depends', ' ${PN}:do_install')
 
     # NOTE: setting INITRAMFS_TASK is for backward compatibility
     #       The preferred method is to set INITRAMFS_IMAGE, because
@@ -327,6 +333,8 @@ do_bundle_initramfs () {
 				mv -f ${KERNEL_OUTPUT_DIR}/$imageType.bak ${KERNEL_OUTPUT_DIR}/$imageType
 			fi
 		done
+	else
+		bbwarn "Calling do_bundle_initramfs with INITRAMFS_IMAGE_BUNDLE=1 is deprecated"
 	fi
 }
 do_bundle_initramfs[dirs] = "${B}"
@@ -347,8 +355,6 @@ python do_devshell:prepend () {
     os.environ["LDFLAGS"] = ''
 }
 
-addtask bundle_initramfs after do_install before do_deploy
-
 KERNEL_DEBUG_TIMESTAMPS ??= "0"
 
 kernel_do_compile() {
-- 
2.45.2

[PATCH 6/6] kernel-fitimage: run unbundled fitimage after deploy

[Adrian Freihofer]

From: Adrian Freihofer <adrian.freihofer@siemens.com>

Without this change, the kernel is always rebuilt if something changes
in the initramfs. If the kernel is rebuilt with an empty TMPDIR, the
process starts by fetching the large kernel git repository and
recompiling it from scratch. This is very inefficient.

This cannot be improved if INITRAMFS_IMAGE_BUNDLE = "1". If the kernel
Makefile is needed to generate the initramfs bundle the kernel build
folder is required. For this cases nothing changes.

But for a build configuration with INITRAMFS_IMAGE_BUNDLE = "" the
build folder is not needed. Creating the initramfs bundle requires:
linux.bin, DTBs and the initramfs. These artifacts are all available in
the deploy folder which is also sstate cached.

Therefore do_assemble_fitimage_initramfs gets split into two tasks:
- do_assemble_fitimage_initramfs creates the bundled fitImage which runs
  before do_deploy_image. This is without sstate and therefore the
  do_deploy_image task still deploys the artifacts generated by this
  task.
- do_deploy_fitimage_unbundled creates the fitImage in unbundled mode.
  This task runs after do_deploy_image. It uses the linux.bin artifact
  from do_deploy_image to create the fitImage with initramfs.
  This task is done with SSTATE_SKIP_CREATION, because this also works
  with the eSDK installer (oe-selftest -r esdk is successful). It is
  also more effective to quickly create the fit image instead of
  handling it via sstate.

Unfortunately, this change is not 100% backward compatible. Previously,
a dependency on do_deploy deployed all types of fit images. With this
change, deploying the unbundled fit image requires a dependency on
do_deploy_fitimage_unbundled. The assignment of the KERNEL_DEPLOY_DEPEND
is improved to handle this appropriately.

Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
---
 meta/classes-recipe/image.bbclass           | 12 ++-
 meta/classes-recipe/kernel-fitimage.bbclass | 84 +++++++++++++--------
 meta/classes-recipe/kernel.bbclass          |  3 +-
 3 files changed, 66 insertions(+), 33 deletions(-)

diff --git a/meta/classes-recipe/image.bbclass b/meta/classes-recipe/image.bbclass
index 32bafcdbbc3..f5754fb6fd9 100644
--- a/meta/classes-recipe/image.bbclass
+++ b/meta/classes-recipe/image.bbclass
@@ -147,7 +147,17 @@ do_rootfs[vardeps] += "${@rootfs_variables(d)}"
 # This follows many common usecases and user expectations.
 # But if you are building an image which doesn't need the kernel image at all,
 # you can unset this variable manually.
-KERNEL_DEPLOY_DEPEND ?= "virtual/kernel:do_deploy"
+def get_kernel_do_deploy(d):
+    kerneltypes = d.getVar('KERNEL_IMAGETYPES') or ""
+    kerneltype = d.getVar('KERNEL_IMAGETYPE') or ""
+    fitimage = kerneltype == 'fitImage' or 'fitImage' in kerneltypes.split()
+    initramfs_image = d.getVar('INITRAMFS_IMAGE') or ""
+    bundled = d.getVar('INITRAMFS_IMAGE_BUNDLE') == '1'
+    if fitimage and initramfs_image and not bundled:
+        return "virtual/kernel:do_deploy_fitimage_unbundled"
+    return "virtual/kernel:do_deploy"
+
+KERNEL_DEPLOY_DEPEND ?= "${@get_kernel_do_deploy(d)}"
 do_build[depends] += "${KERNEL_DEPLOY_DEPEND}"
 
 
diff --git a/meta/classes-recipe/kernel-fitimage.bbclass b/meta/classes-recipe/kernel-fitimage.bbclass
index 2dda5aec056..f42a0cabeb3 100644
--- a/meta/classes-recipe/kernel-fitimage.bbclass
+++ b/meta/classes-recipe/kernel-fitimage.bbclass
@@ -49,18 +49,28 @@ python __anonymous () {
         d.appendVarFlag('do_assemble_fitimage', 'depends',
                         ' u-boot-tools-native:do_populate_sysroot dtc-native:do_populate_sysroot')
 
+        # A fitImage with the initramfs bundled in the kernel Image is deployed by do_deploy.
+        # In case of a build from an empty TMPDIR a clean kernel re-build is required just because creating the bundled
+        # kernel image requires the kernel Makefile and therefore the kernel's populated build directory.
+        # In case of an unbundled fitImage assembling the fitImage works independenly from the kernel build framework.
+        # This allow to take the kernel binary from the sstated deploy directory. But it requires to assemble the
+        # fitImage in a separate task (do_deploy_fitimage_unbundled) running after do_deploy.
         initramfs_image = d.getVar('INITRAMFS_IMAGE')
         bundled = bb.utils.to_boolean(d.getVar('INITRAMFS_IMAGE_BUNDLE'))
         if initramfs_image:
             if bundled:
-                bb.build.addtask('do_assemble_fitimage_initramfs', 'do_deploy', 'do_bundle_initramfs', d)
+                fit_assemble_task = 'do_assemble_fitimage_initramfs'
+                bb.build.addtask(fit_assemble_task, 'do_deploy', 'do_bundle_initramfs', d)
             else:
-                bb.build.addtask('do_assemble_fitimage_initramfs', 'do_deploy', 'do_install', d)
+                fit_assemble_task = 'do_deploy_fitimage_unbundled'
+                bb.build.addtask(fit_assemble_task, 'do_build', 'do_deploy', d)
+                bb.build.addtask(fit_assemble_task + '_setscene', '', '', d)
+                d.appendVar('SSTATETASKS', fit_assemble_task)
 
-            d.appendVarFlag('do_assemble_fitimage_initramfs', 'depends',
+            d.appendVarFlag(fit_assemble_task, 'depends',
                             ' u-boot-tools-native:do_populate_sysroot dtc-native:do_populate_sysroot ${INITRAMFS_IMAGE}:do_image_complete')
             if providerdtb:
-                d.appendVarFlag('do_assemble_fitimage_initramfs', 'depends', ' virtual/dtb:do_populate_sysroot')
+                d.appendVarFlag(fit_assemble_task, 'depends', ' virtual/dtb:do_populate_sysroot')
 }
 
 
@@ -808,15 +818,30 @@ do_install:append() {
 }
 
 do_assemble_fitimage_initramfs() {
-	if [ "${INITRAMFS_IMAGE_BUNDLE}" = "1" ]; then
-		fitimage_assemble "fit-image-${INITRAMFS_IMAGE}.its" "${KERNEL_OUTPUT_DIR}/fitImage-bundle" ""
-		ln -sf fitImage-bundle "${B}/${KERNEL_OUTPUT_DIR}/fitImage"
-	else
-		fitimage_assemble "fit-image-${INITRAMFS_IMAGE}.its" "${KERNEL_OUTPUT_DIR}/fitImage-${INITRAMFS_IMAGE}" 1
-	fi
+	fitimage_assemble "fit-image-${INITRAMFS_IMAGE}.its" "${KERNEL_OUTPUT_DIR}/fitImage-bundle" ""
+	ln -sf fitImage-bundle "${B}/${KERNEL_OUTPUT_DIR}/fitImage"
 }
 do_assemble_fitimage_initramfs[dirs] = "${B}"
 
+do_deploy_fitimage_unbundled() {
+	fitimage_assemble "${DEPLOY_DIR_IMAGE}/fitImage-its-${INITRAMFS_IMAGE_NAME}-${KERNEL_FIT_NAME}.its"\
+	  "${DEPLOY_DIR_IMAGE}/fitImage-${INITRAMFS_IMAGE_NAME}-${KERNEL_FIT_NAME}${KERNEL_FIT_BIN_EXT}" 1
+	if [ -n "${KERNEL_FIT_LINK_NAME}" ] ; then
+		ln -snf "fitImage-${INITRAMFS_IMAGE_NAME}-${KERNEL_FIT_NAME}${KERNEL_FIT_BIN_EXT}" "${DEPLOY_DIR_IMAGE}/fitImage-${INITRAMFS_IMAGE_NAME}-${KERNEL_FIT_LINK_NAME}"
+		ln -snf fitImage-its-${INITRAMFS_IMAGE_NAME}-${KERNEL_FIT_NAME}.its "${DEPLOY_DIR_IMAGE}/fitImage-its-${INITRAMFS_IMAGE_NAME}-${KERNEL_FIT_LINK_NAME}"
+	fi
+}
+DEPLOYDIR_FITIMAGE_UNBUNDLED = "${WORKDIR}/deploy-fitimage-unbundled-${PN}"
+SSTATE_SKIP_CREATION:task-deploy-fitimage-unbundled = '1'
+do_deploy_fitimage_unbundled[sstate-inputdirs] = "${DEPLOYDIR_FITIMAGE_UNBUNDLED}"
+do_deploy_fitimage_unbundled[sstate-outputdirs] = "${DEPLOY_DIR_IMAGE}"
+python do_deploy_fitimage_unbundled_setscene () {
+    sstate_setscene(d)
+}
+do_deploy_fitimage_unbundled[dirs] = "${DEPLOY_DIR_IMAGE}"
+do_deploy_fitimage_unbundled[cleandirs] = "${DEPLOYDIR_FITIMAGE_UNBUNDLED}"
+do_deploy_fitimage_unbundled[stamp-extra-info] = "${MACHINE_ARCH}"
+
 do_kernel_generate_rsa_keys() {
 	if [ "${UBOOT_SIGN_ENABLE}" = "0" ] && [ "${FIT_GENERATE_KEYS}" = "1" ]; then
 		bbwarn "FIT_GENERATE_KEYS is set to 1 even though UBOOT_SIGN_ENABLE is set to 0. The keys will not be generated as they won't be used."
@@ -868,9 +893,24 @@ kernel_do_deploy[vardepsexclude] = "DATETIME"
 kernel_do_deploy:append() {
 	# Update deploy directory
 	if echo ${KERNEL_IMAGETYPES} | grep -wq "fitImage"; then
-
-		if [ "${INITRAMFS_IMAGE_BUNDLE}" != "1" ]; then
-			# deploy the artifacts of do_assemble_fitimage
+		if  [ -n "${INITRAMFS_IMAGE}" ] && [ "${INITRAMFS_IMAGE_BUNDLE}" != "1" ]; then
+			# do_deploy_fitimage_unbundled needs the linux.bin file for the unbundled fitImage deployment
+			bbnote "Deploying linux.bin and linux.comp file for do_deploy_fitimage_unbundled..."
+			uboot_prep_kimage
+			install -m 0644 ${B}/linux.bin $deployDir/linux.bin
+			install -m 0644 ${B}/linux.comp $deployDir/linux.comp
+			if [ -e "${B}/${KERNEL_OUTPUT_DIR}/setup.bin" ]; then
+				install -m 0644 "${B}/${KERNEL_OUTPUT_DIR}/setup.bin" "$deployDir/setup.bin"
+			fi
+		elif [ -n "${INITRAMFS_IMAGE}" ]; then
+			# deploy the artifacts created by do_assemble_fitimage_initramfs for bundled mode
+			bbnote "Copying fit-image-${INITRAMFS_IMAGE}.its source file..."
+			install -m 0644 ${B}/fit-image-${INITRAMFS_IMAGE}.its "$deployDir/fitImage-its-${INITRAMFS_IMAGE_NAME}-${KERNEL_FIT_NAME}.its"
+			if [ -n "${KERNEL_FIT_LINK_NAME}" ] ; then
+				ln -snf fitImage-its-${INITRAMFS_IMAGE_NAME}-${KERNEL_FIT_NAME}.its "$deployDir/fitImage-its-${INITRAMFS_IMAGE_NAME}-${KERNEL_FIT_LINK_NAME}"
+			fi
+		else
+			# deploy the artifacts of do_assemble_fitimage (fitImage without initramfs)
 			bbnote "Copying fit-image.its source file..."
 			install -m 0644 ${B}/fit-image.its "$deployDir/fitImage-its-${KERNEL_FIT_NAME}.its"
 			if [ -n "${KERNEL_FIT_LINK_NAME}" ] ; then
@@ -883,23 +923,5 @@ kernel_do_deploy:append() {
 				ln -snf fitImage-linux.bin-${KERNEL_FIT_NAME}${KERNEL_FIT_BIN_EXT} "$deployDir/fitImage-linux.bin-${KERNEL_FIT_LINK_NAME}"
 			fi
 		fi
-
-		if [ -n "${INITRAMFS_IMAGE}" ]; then
-			# deploy the artifacts of do_assemble_fitimage_initramfs for bundled as well as un-bundled mode
-			bbnote "Copying fit-image-${INITRAMFS_IMAGE}.its source file..."
-			install -m 0644 ${B}/fit-image-${INITRAMFS_IMAGE}.its "$deployDir/fitImage-its-${INITRAMFS_IMAGE_NAME}-${KERNEL_FIT_NAME}.its"
-			if [ -n "${KERNEL_FIT_LINK_NAME}" ] ; then
-				ln -snf fitImage-its-${INITRAMFS_IMAGE_NAME}-${KERNEL_FIT_NAME}.its "$deployDir/fitImage-its-${INITRAMFS_IMAGE_NAME}-${KERNEL_FIT_LINK_NAME}"
-			fi
-
-			# deploy the artifacts of do_assemble_fitimage_initramfs for bundled mode only
-			if [ "${INITRAMFS_IMAGE_BUNDLE}" != "1" ]; then
-				bbnote "Copying fitImage-${INITRAMFS_IMAGE} file..."
-				install -m 0644 ${B}/${KERNEL_OUTPUT_DIR}/fitImage-${INITRAMFS_IMAGE} "$deployDir/fitImage-${INITRAMFS_IMAGE_NAME}-${KERNEL_FIT_NAME}${KERNEL_FIT_BIN_EXT}"
-				if [ -n "${KERNEL_FIT_LINK_NAME}" ] ; then
-					ln -snf fitImage-${INITRAMFS_IMAGE_NAME}-${KERNEL_FIT_NAME}${KERNEL_FIT_BIN_EXT} "$deployDir/fitImage-${INITRAMFS_IMAGE_NAME}-${KERNEL_FIT_LINK_NAME}"
-				fi
-			fi
-		fi
 	fi
 }
diff --git a/meta/classes-recipe/kernel.bbclass b/meta/classes-recipe/kernel.bbclass
index 6a8c3c25c07..848f4e5e431 100644
--- a/meta/classes-recipe/kernel.bbclass
+++ b/meta/classes-recipe/kernel.bbclass
@@ -488,7 +488,8 @@ kernel_do_install() {
 	# So, at the level of the install task we should not try to install the fitImage. fitImage is still not
 	# generated yet.
 	# After the generation of the fitImage, the deploy task copies the fitImage from the build directory to
-	# the deploy folder.
+	# the deploy folder. If INITRAMFS_IMAGE_BUNDLE != 1 the fitImage with initramfs is deployed after the
+	# deploy task.
 	#
 
 	for imageType in ${KERNEL_IMAGETYPES} ; do
-- 
2.45.2

Re: [PATCH v2 0/6] Use the kernel from sstate when building fitImages

[Adrian Freihofer]

[-- Attachment #1: Type: text/plain, Size: 17003 bytes --]


 task-depends      Task dependeny changes 

Configuration: image, fitimage, initramfs, unbundled 

KERNEL_IMAGETYPE = "Image"
KERNEL_IMAGETYPES += " fitImage "
KERNEL_CLASSES = " kernel-fitimage "
IMAGE_FSTYPES += "cpio.gz"
INITRAMFS_IMAGE = "core-image-minimal-initramfs"
branch: master 





depends
undefined


core-image-minimal.do_build
undefined
core-image-minimal.do_build




do_deploy
undefined
do_deploy




core-image-minimal.do_build->do_deploy
undefined
undefined



do_assemble_fitimage_initramfs
undefined
do_assemble_fitimage_initramfs




do_deploy->do_assemble_fitimage_initramfs
undefined
undefined



do_bundle_initramfs
undefined
do_bundle_initramfs




do_deploy->do_bundle_initramfs
undefined
undefined



do_populate_sysroot
undefined
do_populate_sysroot




do_deploy->do_populate_sysroot
undefined
undefined



do_assemble_fitimage
undefined
do_assemble_fitimage




do_compile
undefined
do_compile




do_assemble_fitimage->do_compile
undefined
undefined



do_kernel_generate_rsa_keys
undefined
do_kernel_generate_rsa_keys




do_assemble_fitimage->do_kernel_generate_rsa_keys
undefined
undefined



do_kernel_generate_rsa_keys->do_compile
undefined
undefined



core-image-minimal.do_image_complete
undefined
core-image-minimal.do_image_complete




do_assemble_fitimage_initramfs->core-image-minimal.do_image_complete
undefined
undefined



do_assemble_fitimage_initramfs->do_bundle_initramfs
undefined
undefined



do_bundle_initramfs->core-image-minimal.do_image_complete
undefined
undefined



do_install
undefined
do_install




do_bundle_initramfs->do_install
undefined
undefined



do_install->do_assemble_fitimage
undefined
undefined



do_install->do_compile
undefined
undefined



do_populate_sysroot->do_install
undefined
undefined



branch: adrianf/kernel-fitimage-sstate 





depends
undefined


core-image-minimal.do_build
undefined
core-image-minimal.do_build




do_deploy
undefined
do_deploy




core-image-minimal.do_build->do_deploy
undefined
undefined



do_deploy_fitimage_unbundled
undefined
do_deploy_fitimage_unbundled




core-image-minimal.do_build->do_deploy_fitimage_unbundled
undefined
undefined



core-image-minimal.do_image_complete
undefined
core-image-minimal.do_image_complete




do_deploy->core-image-minimal.do_image_complete
undefined
undefined



do_install
undefined
do_install




do_deploy->do_install
undefined
undefined



do_populate_sysroot
undefined
do_populate_sysroot




do_deploy->do_populate_sysroot
undefined
undefined



do_deploy_fitimage_unbundled->do_deploy
undefined
undefined



do_deploy_fitimage_unbundled->core-image-minimal.do_image_complete
undefined
undefined



do_assemble_fitimage
undefined
do_assemble_fitimage




do_compile
undefined
do_compile




do_assemble_fitimage->do_compile
undefined
undefined



do_kernel_generate_rsa_keys
undefined
do_kernel_generate_rsa_keys




do_assemble_fitimage->do_kernel_generate_rsa_keys
undefined
undefined



do_kernel_generate_rsa_keys->do_compile
undefined
undefined



do_install->do_assemble_fitimage
undefined
undefined



do_install->do_compile
undefined
undefined



do_populate_sysroot->do_install
undefined
undefined



Diff 

2a3
> "core-image-minimal.do_build" -> "do_deploy_fitimage_unbundled"
6,11d6
< "do_assemble_fitimage_initramfs"
[label="do_assemble_fitimage_initramfs"]
< "do_assemble_fitimage_initramfs" -> "core-image-
minimal.do_image_complete"
< "do_assemble_fitimage_initramfs" -> "do_bundle_initramfs"
< "do_bundle_initramfs" [label="do_bundle_initramfs"]
< "do_bundle_initramfs" -> "core-image-minimal.do_image_complete"
< "do_bundle_initramfs" -> "do_install"
14,15c9,10
< "do_deploy" -> "do_assemble_fitimage_initramfs"
< "do_deploy" -> "do_bundle_initramfs"
---
> "do_deploy" -> "core-image-minimal.do_image_complete"
> "do_deploy" -> "do_install"
16a12,14
> "do_deploy_fitimage_unbundled" [label="do_deploy_fitimage_unbundled"]
> "do_deploy_fitimage_unbundled" -> "core-image-
minimal.do_image_complete"
> "do_deploy_fitimage_unbundled" -> "do_deploy"
Configuration: fitimage, initramfs, unbundled 

KERNEL_IMAGETYPE:forcevariable = "fitImage"
KERNEL_CLASSES = " kernel-fitimage "
IMAGE_FSTYPES += "cpio.gz"
INITRAMFS_IMAGE = "core-image-minimal-initramfs"
branch: master 





depends
undefined


core-image-minimal.do_build
undefined
core-image-minimal.do_build




do_deploy
undefined
do_deploy




core-image-minimal.do_build->do_deploy
undefined
undefined



do_assemble_fitimage_initramfs
undefined
do_assemble_fitimage_initramfs




do_deploy->do_assemble_fitimage_initramfs
undefined
undefined



do_bundle_initramfs
undefined
do_bundle_initramfs




do_deploy->do_bundle_initramfs
undefined
undefined



do_populate_sysroot
undefined
do_populate_sysroot




do_deploy->do_populate_sysroot
undefined
undefined



do_assemble_fitimage
undefined
do_assemble_fitimage




do_compile
undefined
do_compile




do_assemble_fitimage->do_compile
undefined
undefined



do_kernel_generate_rsa_keys
undefined
do_kernel_generate_rsa_keys




do_assemble_fitimage->do_kernel_generate_rsa_keys
undefined
undefined



do_kernel_generate_rsa_keys->do_compile
undefined
undefined



core-image-minimal.do_image_complete
undefined
core-image-minimal.do_image_complete




do_assemble_fitimage_initramfs->core-image-minimal.do_image_complete
undefined
undefined



do_assemble_fitimage_initramfs->do_bundle_initramfs
undefined
undefined



do_bundle_initramfs->core-image-minimal.do_image_complete
undefined
undefined



do_install
undefined
do_install




do_bundle_initramfs->do_install
undefined
undefined



do_install->do_assemble_fitimage
undefined
undefined



do_install->do_compile
undefined
undefined



do_populate_sysroot->do_install
undefined
undefined



branch: adrianf/kernel-fitimage-sstate 





depends
undefined


core-image-minimal.do_build
undefined
core-image-minimal.do_build




do_deploy
undefined
do_deploy




core-image-minimal.do_build->do_deploy
undefined
undefined



do_deploy_fitimage_unbundled
undefined
do_deploy_fitimage_unbundled




core-image-minimal.do_build->do_deploy_fitimage_unbundled
undefined
undefined



core-image-minimal.do_image_complete
undefined
core-image-minimal.do_image_complete




do_deploy->core-image-minimal.do_image_complete
undefined
undefined



do_install
undefined
do_install




do_deploy->do_install
undefined
undefined



do_populate_sysroot
undefined
do_populate_sysroot




do_deploy->do_populate_sysroot
undefined
undefined



do_deploy_fitimage_unbundled->do_deploy
undefined
undefined



do_deploy_fitimage_unbundled->core-image-minimal.do_image_complete
undefined
undefined



do_assemble_fitimage
undefined
do_assemble_fitimage




do_compile
undefined
do_compile




do_assemble_fitimage->do_compile
undefined
undefined



do_kernel_generate_rsa_keys
undefined
do_kernel_generate_rsa_keys




do_assemble_fitimage->do_kernel_generate_rsa_keys
undefined
undefined



do_kernel_generate_rsa_keys->do_compile
undefined
undefined



do_install->do_assemble_fitimage
undefined
undefined



do_install->do_compile
undefined
undefined



do_populate_sysroot->do_install
undefined
undefined



Diff 

2a3
> "core-image-minimal.do_build" -> "do_deploy_fitimage_unbundled"
6,11d6
< "do_assemble_fitimage_initramfs"
[label="do_assemble_fitimage_initramfs"]
< "do_assemble_fitimage_initramfs" -> "core-image-
minimal.do_image_complete"
< "do_assemble_fitimage_initramfs" -> "do_bundle_initramfs"
< "do_bundle_initramfs" [label="do_bundle_initramfs"]
< "do_bundle_initramfs" -> "core-image-minimal.do_image_complete"
< "do_bundle_initramfs" -> "do_install"
14,15c9,10
< "do_deploy" -> "do_assemble_fitimage_initramfs"
< "do_deploy" -> "do_bundle_initramfs"
---
> "do_deploy" -> "core-image-minimal.do_image_complete"
> "do_deploy" -> "do_install"
16a12,14
> "do_deploy_fitimage_unbundled" [label="do_deploy_fitimage_unbundled"]
> "do_deploy_fitimage_unbundled" -> "core-image-
minimal.do_image_complete"
> "do_deploy_fitimage_unbundled" -> "do_deploy"
Configuration: image, fitimage, initramfs, bundled 

KERNEL_IMAGETYPE = "Image"
KERNEL_IMAGETYPES += " fitImage "
KERNEL_CLASSES = " kernel-fitimage "
IMAGE_FSTYPES += "cpio.gz"
INITRAMFS_IMAGE = "core-image-minimal-initramfs"
INITRAMFS_IMAGE_BUNDLE = "1"
branch: master 





depends
undefined


core-image-minimal.do_build
undefined
core-image-minimal.do_build




do_deploy
undefined
do_deploy




core-image-minimal.do_build->do_deploy
undefined
undefined



do_assemble_fitimage_initramfs
undefined
do_assemble_fitimage_initramfs




do_deploy->do_assemble_fitimage_initramfs
undefined
undefined



do_bundle_initramfs
undefined
do_bundle_initramfs




do_deploy->do_bundle_initramfs
undefined
undefined



do_populate_sysroot
undefined
do_populate_sysroot




do_deploy->do_populate_sysroot
undefined
undefined



do_transform_bundled_initramfs
undefined
do_transform_bundled_initramfs




do_deploy->do_transform_bundled_initramfs
undefined
undefined



do_assemble_fitimage
undefined
do_assemble_fitimage




do_compile
undefined
do_compile




do_assemble_fitimage->do_compile
undefined
undefined



do_kernel_generate_rsa_keys
undefined
do_kernel_generate_rsa_keys




do_assemble_fitimage->do_kernel_generate_rsa_keys
undefined
undefined



do_kernel_generate_rsa_keys->do_compile
undefined
undefined



core-image-minimal.do_image_complete
undefined
core-image-minimal.do_image_complete




do_assemble_fitimage_initramfs->core-image-minimal.do_image_complete
undefined
undefined



do_assemble_fitimage_initramfs->do_bundle_initramfs
undefined
undefined



do_bundle_initramfs->core-image-minimal.do_image_complete
undefined
undefined



do_install
undefined
do_install




do_bundle_initramfs->do_install
undefined
undefined



do_install->do_assemble_fitimage
undefined
undefined



do_install->do_compile
undefined
undefined



do_populate_sysroot->do_install
undefined
undefined



do_transform_bundled_initramfs->do_bundle_initramfs
undefined
undefined



branch: adrianf/kernel-fitimage-sstate 





depends
undefined


core-image-minimal.do_build
undefined
core-image-minimal.do_build




do_deploy
undefined
do_deploy




core-image-minimal.do_build->do_deploy
undefined
undefined



do_assemble_fitimage_initramfs
undefined
do_assemble_fitimage_initramfs




do_deploy->do_assemble_fitimage_initramfs
undefined
undefined



do_bundle_initramfs
undefined
do_bundle_initramfs




do_deploy->do_bundle_initramfs
undefined
undefined



do_populate_sysroot
undefined
do_populate_sysroot




do_deploy->do_populate_sysroot
undefined
undefined



do_transform_bundled_initramfs
undefined
do_transform_bundled_initramfs




do_deploy->do_transform_bundled_initramfs
undefined
undefined



do_assemble_fitimage
undefined
do_assemble_fitimage




do_compile
undefined
do_compile




do_assemble_fitimage->do_compile
undefined
undefined



do_kernel_generate_rsa_keys
undefined
do_kernel_generate_rsa_keys




do_assemble_fitimage->do_kernel_generate_rsa_keys
undefined
undefined



do_kernel_generate_rsa_keys->do_compile
undefined
undefined



core-image-minimal.do_image_complete
undefined
core-image-minimal.do_image_complete




do_assemble_fitimage_initramfs->core-image-minimal.do_image_complete
undefined
undefined



do_assemble_fitimage_initramfs->do_bundle_initramfs
undefined
undefined



do_bundle_initramfs->core-image-minimal.do_image_complete
undefined
undefined



do_install
undefined
do_install




do_bundle_initramfs->do_install
undefined
undefined



do_install->do_assemble_fitimage
undefined
undefined



do_install->do_compile
undefined
undefined



do_populate_sysroot->do_install
undefined
undefined



do_transform_bundled_initramfs->do_bundle_initramfs
undefined
undefined



Diff 

Configuration: image, fitimage 

KERNEL_IMAGETYPE = "Image"
KERNEL_IMAGETYPES += " fitImage "
KERNEL_CLASSES = " kernel-fitimage "
branch: master 





depends
undefined


core-image-minimal.do_build
undefined
core-image-minimal.do_build




do_deploy
undefined
do_deploy




core-image-minimal.do_build->do_deploy
undefined
undefined



do_assemble_fitimage_initramfs
undefined
do_assemble_fitimage_initramfs




do_deploy->do_assemble_fitimage_initramfs
undefined
undefined



do_bundle_initramfs
undefined
do_bundle_initramfs




do_deploy->do_bundle_initramfs
undefined
undefined



do_populate_sysroot
undefined
do_populate_sysroot




do_deploy->do_populate_sysroot
undefined
undefined



do_assemble_fitimage
undefined
do_assemble_fitimage




do_compile
undefined
do_compile




do_assemble_fitimage->do_compile
undefined
undefined



do_kernel_generate_rsa_keys
undefined
do_kernel_generate_rsa_keys




do_assemble_fitimage->do_kernel_generate_rsa_keys
undefined
undefined



do_kernel_generate_rsa_keys->do_compile
undefined
undefined



core-image-minimal.do_image_complete
undefined
core-image-minimal.do_image_complete




do_assemble_fitimage_initramfs->core-image-minimal.do_image_complete
undefined
undefined



do_assemble_fitimage_initramfs->do_bundle_initramfs
undefined
undefined



do_bundle_initramfs->core-image-minimal.do_image_complete
undefined
undefined



do_install
undefined
do_install




do_bundle_initramfs->do_install
undefined
undefined



do_install->do_assemble_fitimage
undefined
undefined



do_install->do_compile
undefined
undefined



do_populate_sysroot->do_install
undefined
undefined



branch: adrianf/kernel-fitimage-sstate 





depends
undefined


core-image-minimal.do_build
undefined
core-image-minimal.do_build




do_deploy
undefined
do_deploy




core-image-minimal.do_build->do_deploy
undefined
undefined



do_deploy_fitimage_unbundled
undefined
do_deploy_fitimage_unbundled




core-image-minimal.do_build->do_deploy_fitimage_unbundled
undefined
undefined



core-image-minimal.do_image_complete
undefined
core-image-minimal.do_image_complete




do_deploy->core-image-minimal.do_image_complete
undefined
undefined



do_install
undefined
do_install




do_deploy->do_install
undefined
undefined



do_populate_sysroot
undefined
do_populate_sysroot




do_deploy->do_populate_sysroot
undefined
undefined



do_deploy_fitimage_unbundled->do_deploy
undefined
undefined



do_deploy_fitimage_unbundled->core-image-minimal.do_image_complete
undefined
undefined



do_assemble_fitimage
undefined
do_assemble_fitimage




do_compile
undefined
do_compile




do_assemble_fitimage->do_compile
undefined
undefined



do_kernel_generate_rsa_keys
undefined
do_kernel_generate_rsa_keys




do_assemble_fitimage->do_kernel_generate_rsa_keys
undefined
undefined



do_kernel_generate_rsa_keys->do_compile
undefined
undefined



do_install->do_assemble_fitimage
undefined
undefined



do_install->do_compile
undefined
undefined



do_populate_sysroot->do_install
undefined
undefined



Diff 

2a3
> "core-image-minimal.do_build" -> "do_deploy_fitimage_unbundled"
6,11d6
< "do_assemble_fitimage_initramfs"
[label="do_assemble_fitimage_initramfs"]
< "do_assemble_fitimage_initramfs" -> "core-image-
minimal.do_image_complete"
< "do_assemble_fitimage_initramfs" -> "do_bundle_initramfs"
< "do_bundle_initramfs" [label="do_bundle_initramfs"]
< "do_bundle_initramfs" -> "core-image-minimal.do_image_complete"
< "do_bundle_initramfs" -> "do_install"
14,15c9,10
< "do_deploy" -> "do_assemble_fitimage_initramfs"
< "do_deploy" -> "do_bundle_initramfs"
---
> "do_deploy" -> "core-image-minimal.do_image_complete"
> "do_deploy" -> "do_install"
16a12,14
> "do_deploy_fitimage_unbundled" [label="do_deploy_fitimage_unbundled"]
> "do_deploy_fitimage_unbundled" -> "core-image-
minimal.do_image_complete"
> "do_deploy_fitimage_unbundled" -> "do_deploy"
 

[-- Attachment #2: Type: text/html, Size: 92034 bytes --]

Re: [OE-core] [PATCH v2 0/6] Use the kernel from sstate when building fitImages

[Martin Jansa]

Doesn't it still rebuild from scratch when IMAGE_VERSION_SUFFIX changes?

To be able to re-use the kernel from sstate even when
IMAGE_VERSION_SUFFIX changes I've implemented:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=12937
which was unfortunately rejected due to lack of interest from other
people outside LGE, but maybe you could take advantage of that as
well.

On Mon, Jul 15, 2024 at 4:22 PM Adrian Freihofer via
lists.openembedded.org
<adrian.freihofer=gmail.com@lists.openembedded.org> wrote:
>
> Task dependeny changes
>
> Configuration: image, fitimage, initramfs, unbundled
>
> KERNEL_IMAGETYPE = "Image"
> KERNEL_IMAGETYPES += " fitImage "
> KERNEL_CLASSES = " kernel-fitimage "
> IMAGE_FSTYPES += "cpio.gz"
> INITRAMFS_IMAGE = "core-image-minimal-initramfs"
>
> branch: master
>
> core-image-minimal.do_build do_deploy do_assemble_fitimage_initramfs do_bundle_initramfs do_populate_sysroot do_assemble_fitimage do_compile do_kernel_generate_rsa_keys core-image-minimal.do_image_complete do_install
>
> branch: adrianf/kernel-fitimage-sstate
>
> core-image-minimal.do_build do_deploy do_deploy_fitimage_unbundled core-image-minimal.do_image_complete do_install do_populate_sysroot do_assemble_fitimage do_compile do_kernel_generate_rsa_keys
>
> Diff
>
> 2a3
> > "core-image-minimal.do_build" -> "do_deploy_fitimage_unbundled"
> 6,11d6
> < "do_assemble_fitimage_initramfs" [label="do_assemble_fitimage_initramfs"]
> < "do_assemble_fitimage_initramfs" -> "core-image-minimal.do_image_complete"
> < "do_assemble_fitimage_initramfs" -> "do_bundle_initramfs"
> < "do_bundle_initramfs" [label="do_bundle_initramfs"]
> < "do_bundle_initramfs" -> "core-image-minimal.do_image_complete"
> < "do_bundle_initramfs" -> "do_install"
> 14,15c9,10
> < "do_deploy" -> "do_assemble_fitimage_initramfs"
> < "do_deploy" -> "do_bundle_initramfs"
> ---
> > "do_deploy" -> "core-image-minimal.do_image_complete"
> > "do_deploy" -> "do_install"
> 16a12,14
> > "do_deploy_fitimage_unbundled" [label="do_deploy_fitimage_unbundled"]
> > "do_deploy_fitimage_unbundled" -> "core-image-minimal.do_image_complete"
> > "do_deploy_fitimage_unbundled" -> "do_deploy"
>
> Configuration: fitimage, initramfs, unbundled
>
> KERNEL_IMAGETYPE:forcevariable = "fitImage"
> KERNEL_CLASSES = " kernel-fitimage "
> IMAGE_FSTYPES += "cpio.gz"
> INITRAMFS_IMAGE = "core-image-minimal-initramfs"
>
> branch: master
>
> core-image-minimal.do_build do_deploy do_assemble_fitimage_initramfs do_bundle_initramfs do_populate_sysroot do_assemble_fitimage do_compile do_kernel_generate_rsa_keys core-image-minimal.do_image_complete do_install
>
> branch: adrianf/kernel-fitimage-sstate
>
> core-image-minimal.do_build do_deploy do_deploy_fitimage_unbundled core-image-minimal.do_image_complete do_install do_populate_sysroot do_assemble_fitimage do_compile do_kernel_generate_rsa_keys
>
> Diff
>
> 2a3
> > "core-image-minimal.do_build" -> "do_deploy_fitimage_unbundled"
> 6,11d6
> < "do_assemble_fitimage_initramfs" [label="do_assemble_fitimage_initramfs"]
> < "do_assemble_fitimage_initramfs" -> "core-image-minimal.do_image_complete"
> < "do_assemble_fitimage_initramfs" -> "do_bundle_initramfs"
> < "do_bundle_initramfs" [label="do_bundle_initramfs"]
> < "do_bundle_initramfs" -> "core-image-minimal.do_image_complete"
> < "do_bundle_initramfs" -> "do_install"
> 14,15c9,10
> < "do_deploy" -> "do_assemble_fitimage_initramfs"
> < "do_deploy" -> "do_bundle_initramfs"
> ---
> > "do_deploy" -> "core-image-minimal.do_image_complete"
> > "do_deploy" -> "do_install"
> 16a12,14
> > "do_deploy_fitimage_unbundled" [label="do_deploy_fitimage_unbundled"]
> > "do_deploy_fitimage_unbundled" -> "core-image-minimal.do_image_complete"
> > "do_deploy_fitimage_unbundled" -> "do_deploy"
>
> Configuration: image, fitimage, initramfs, bundled
>
> KERNEL_IMAGETYPE = "Image"
> KERNEL_IMAGETYPES += " fitImage "
> KERNEL_CLASSES = " kernel-fitimage "
> IMAGE_FSTYPES += "cpio.gz"
> INITRAMFS_IMAGE = "core-image-minimal-initramfs"
> INITRAMFS_IMAGE_BUNDLE = "1"
>
> branch: master
>
> core-image-minimal.do_build do_deploy do_assemble_fitimage_initramfs do_bundle_initramfs do_populate_sysroot do_transform_bundled_initramfs do_assemble_fitimage do_compile do_kernel_generate_rsa_keys core-image-minimal.do_image_complete do_install
>
> branch: adrianf/kernel-fitimage-sstate
>
> core-image-minimal.do_build do_deploy do_assemble_fitimage_initramfs do_bundle_initramfs do_populate_sysroot do_transform_bundled_initramfs do_assemble_fitimage do_compile do_kernel_generate_rsa_keys core-image-minimal.do_image_complete do_install
>
> Diff
>
> Configuration: image, fitimage
>
> KERNEL_IMAGETYPE = "Image"
> KERNEL_IMAGETYPES += " fitImage "
> KERNEL_CLASSES = " kernel-fitimage "
>
> branch: master
>
> core-image-minimal.do_build do_deploy do_assemble_fitimage_initramfs do_bundle_initramfs do_populate_sysroot do_assemble_fitimage do_compile do_kernel_generate_rsa_keys core-image-minimal.do_image_complete do_install
>
> branch: adrianf/kernel-fitimage-sstate
>
> core-image-minimal.do_build do_deploy do_deploy_fitimage_unbundled core-image-minimal.do_image_complete do_install do_populate_sysroot do_assemble_fitimage do_compile do_kernel_generate_rsa_keys
>
> Diff
>
> 2a3
> > "core-image-minimal.do_build" -> "do_deploy_fitimage_unbundled"
> 6,11d6
> < "do_assemble_fitimage_initramfs" [label="do_assemble_fitimage_initramfs"]
> < "do_assemble_fitimage_initramfs" -> "core-image-minimal.do_image_complete"
> < "do_assemble_fitimage_initramfs" -> "do_bundle_initramfs"
> < "do_bundle_initramfs" [label="do_bundle_initramfs"]
> < "do_bundle_initramfs" -> "core-image-minimal.do_image_complete"
> < "do_bundle_initramfs" -> "do_install"
> 14,15c9,10
> < "do_deploy" -> "do_assemble_fitimage_initramfs"
> < "do_deploy" -> "do_bundle_initramfs"
> ---
> > "do_deploy" -> "core-image-minimal.do_image_complete"
> > "do_deploy" -> "do_install"
> 16a12,14
> > "do_deploy_fitimage_unbundled" [label="do_deploy_fitimage_unbundled"]
> > "do_deploy_fitimage_unbundled" -> "core-image-minimal.do_image_complete"
> > "do_deploy_fitimage_unbundled" -> "do_deploy"
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#201933): https://lists.openembedded.org/g/openembedded-core/message/201933
> Mute This Topic: https://lists.openembedded.org/mt/107231736/3617156
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [martin.jansa@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>

Re: [OE-core] [PATCH v2 0/6] Use the kernel from sstate when building fitImages

[Adrian Freihofer]

Hi Martin

Thank you for looking at my patches.

On Mon, 2024-07-15 at 16:32 +0200, Martin Jansa wrote:
> Doesn't it still rebuild from scratch when IMAGE_VERSION_SUFFIX
> changes?

I wonder why this happens in WebOS. I think that this line
kernel_do_deploy[vardepsexclude] = "DATETIME"
prevents such unnecessary rebuilds.

So far we haven't had any problems with rebuilding kernels. The trouble
started when we started using fitImages with initramfs for some
devices. It is also worth noting that we build from sstate, but with an
empty TMPDIR.

In such a scenario bitbake does:

 * The initramfs gets assembled. This is expected because images are
   not sstate cached. This runs quickly because all the packages which
   are required to build the initramfs come from sstate-cache.
 * Since the initramfs changes, the fitImage which includes it must be
   re-assembled as well.
 * Now the hassle starts: The do_assemble_fitimage_initramfs needs the
   kernel binaries (linux.bin, DTBs...) which are not available from
   sttate. Bitbake has to start with kernel do_fetch in case of an
   empty TMPDIR just to get the fitImage assembled.

With my patches the kernel binaries used for the fitImage come from
sstate (if the initramfs is not bundled).

I tested this like that:

cat build/conf/local.conf

KERNEL_IMAGETYPE = "Image"
KERNEL_IMAGETYPES += " fitImage "
KERNEL_CLASSES = " kernel-fitimage "and RAM disk
IMAGE_FSTYPES += "cpio.gz"
INITRAMFS_IMAGE = "core-image-minimal"
IMAGE_NAME_SUFFIX:pn-core-image-minimal = ""
UBOOT_RD_LOADADDRESS = "0x88000000"
UBOOT_RD_ENTRYPOINT = "0x88000000"
UBOOT_LOADADDRESS = "0x80080000"
UBOOT_ENTRYPOINT = "0x80080000"
FIT_DESC = "A model description"

# Allow to change the initramfs
FOO_VAR = "1"
INHERIT += "image-buildinfo"
IMAGE_BUILDINFO_VARS:append = " FOO_VAR"


bitbake virtual/kernel

mv build/tmp build/tmp-old-1

bitbake virtual/kernel
-> bitbake does:
- setscene tasks
- core-image-minimal do_rootfs ...
- kernel do_deploy (from sstate)
- kernel do_deploy_fitimage_unbundled


bitbake virtual/kernel
-> bitbake does: nothing


# Enforce a re-build of the initramfs
mv build/tmp build/tmp-old-2
echo 'FOO_VAR = "1"' >> build/conf/local.conf

bitbake virtual/kernel
-> bitbake does:
- setscene tasks
- core-image-minimal do_rootfs ...
- kernel do_deploy (from sstate)
- kernel do_deploy_fitimage_unbundled


bitbake virtual/kernel
-> bitbake does: nothing


diff \
<(ls build/tmp-old-2/deploy/images/qemux86-64 -1) \
<(ls build/tmp/deploy/images/qemux86-64/ -1)
4,10c4,10
< core-image-minimal-qemux86-64-20240718065105.cpio.gz
< core-image-minimal-qemux86-64-20240718065105.ext4
< core-image-minimal-qemux86-64-20240718065105.manifest
< core-image-minimal-qemux86-64-20240718065105.qemuboot.conf
< core-image-minimal-qemux86-64-20240718065105.spdx.tar.zst
< core-image-minimal-qemux86-64-20240718065105.tar.bz2
< core-image-minimal-qemux86-64-20240718065105.testdata.json
---
> core-image-minimal-qemux86-64-20240718073341.cpio.gz
> core-image-minimal-qemux86-64-20240718073341.ext4
> core-image-minimal-qemux86-64-20240718073341.manifest
> core-image-minimal-qemux86-64-20240718073341.qemuboot.conf
> core-image-minimal-qemux86-64-20240718073341.spdx.tar.zst
> core-image-minimal-qemux86-64-20240718073341.tar.bz2
> core-image-minimal-qemux86-64-20240718073341.testdata.json
20c20
< fitImage-core-image-minimal-qemux86-64--6.6.35+git-r0-qemux86-64-
20240718065105.bin
---
> fitImage-core-image-minimal-qemux86-64--6.6.35+git-r0-qemux86-64-
20240718073341.bin
22c22
< fitImage-its-core-image-minimal-qemux86-64--6.6.35+git-r0-qemux86-64-
20240718065105.its
---
> fitImage-its-core-image-minimal-qemux86-64--6.6.35+git-r0-qemux86-64-
20240718073341.its



The answer is then: No it does not re-build. At least not in the
scenario which I would like to fix it, it does not rebuild from
scratch.

Does that make sense?

Regards,
Adrian

> > 
> > -=-=-=-=-=-=-=-=-=-=-=-
> > Links: You receive all messages sent to this group.
> > View/Reply Online (#201933):
> > https://lists.openembedded.org/g/openembedded-core/message/201933
> > Mute This Topic:
> > https://lists.openembedded.org/mt/107231736/3617156
> > Group Owner: openembedded-core+owner@lists.openembedded.org
> > Unsubscribe:
> > https://lists.openembedded.org/g/openembedded-core/unsub [
> > martin.jansa@gmail.com]
> > -=-=-=-=-=-=-=-=-=-=-=-
> > 

Re: [OE-core] [PATCH v2 0/6] Use the kernel from sstate when building fitImages

[Martin Jansa]

On Thu, Jul 18, 2024 at 10:00 AM Adrian Freihofer
<adrian.freihofer@gmail.com> wrote:
>
> Hi Martin
>
> Thank you for looking at my patches.
>
> On Mon, 2024-07-15 at 16:32 +0200, Martin Jansa wrote:
> > Doesn't it still rebuild from scratch when IMAGE_VERSION_SUFFIX
> > changes?
>
> I wonder why this happens in WebOS. I think that this line
> kernel_do_deploy[vardepsexclude] = "DATETIME"
> prevents such unnecessary rebuilds.

Hi Adrian,

yes DATETIME is excluded by default, but it's useful to use different
suffix in IMAGE_VERSION_SUFFIX (e.g. BUILD_NUMBER from CI on jenkins
or release version when building release) and in such cases you don't
want to vardepexclude it, because it's useful to produce matching
version in images as well as kernel (and other artifacts you might
have), so your 1.2.1 release images doesn't end with 1.2.0 kernel
artifacts just because kernel sstate signature didn't change in 1.2.1
bugfix release.

Does this make sense? I bet it would happen in your builds as well.

Whole point of:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=12937
is to allow "renaming" (adding more hardlinks) artifacts with matching
IMAGE_VERSION_SUFFIX without the need of rebuilding anything (e.g.
version-less kernel artifacts are re-used from sstate and then only
quick deploy-links task is executed to add right versioned hardlinks
in deploy directory.)

Cheers,

> So far we haven't had any problems with rebuilding kernels. The trouble
> started when we started using fitImages with initramfs for some
> devices. It is also worth noting that we build from sstate, but with an
> empty TMPDIR.
>
> In such a scenario bitbake does:
>
>  * The initramfs gets assembled. This is expected because images are
>    not sstate cached. This runs quickly because all the packages which
>    are required to build the initramfs come from sstate-cache.
>  * Since the initramfs changes, the fitImage which includes it must be
>    re-assembled as well.
>  * Now the hassle starts: The do_assemble_fitimage_initramfs needs the
>    kernel binaries (linux.bin, DTBs...) which are not available from
>    sttate. Bitbake has to start with kernel do_fetch in case of an
>    empty TMPDIR just to get the fitImage assembled.
>
> With my patches the kernel binaries used for the fitImage come from
> sstate (if the initramfs is not bundled).
>
> I tested this like that:
>
> cat build/conf/local.conf
>
> KERNEL_IMAGETYPE = "Image"
> KERNEL_IMAGETYPES += " fitImage "
> KERNEL_CLASSES = " kernel-fitimage "and RAM disk
> IMAGE_FSTYPES += "cpio.gz"
> INITRAMFS_IMAGE = "core-image-minimal"
> IMAGE_NAME_SUFFIX:pn-core-image-minimal = ""
> UBOOT_RD_LOADADDRESS = "0x88000000"
> UBOOT_RD_ENTRYPOINT = "0x88000000"
> UBOOT_LOADADDRESS = "0x80080000"
> UBOOT_ENTRYPOINT = "0x80080000"
> FIT_DESC = "A model description"
>
> # Allow to change the initramfs
> FOO_VAR = "1"
> INHERIT += "image-buildinfo"
> IMAGE_BUILDINFO_VARS:append = " FOO_VAR"
>
>
> bitbake virtual/kernel
>
> mv build/tmp build/tmp-old-1
>
> bitbake virtual/kernel
> -> bitbake does:
> - setscene tasks
> - core-image-minimal do_rootfs ...
> - kernel do_deploy (from sstate)
> - kernel do_deploy_fitimage_unbundled
>
>
> bitbake virtual/kernel
> -> bitbake does: nothing
>
>
> # Enforce a re-build of the initramfs
> mv build/tmp build/tmp-old-2
> echo 'FOO_VAR = "1"' >> build/conf/local.conf
>
> bitbake virtual/kernel
> -> bitbake does:
> - setscene tasks
> - core-image-minimal do_rootfs ...
> - kernel do_deploy (from sstate)
> - kernel do_deploy_fitimage_unbundled
>
>
> bitbake virtual/kernel
> -> bitbake does: nothing
>
>
> diff \
> <(ls build/tmp-old-2/deploy/images/qemux86-64 -1) \
> <(ls build/tmp/deploy/images/qemux86-64/ -1)
> 4,10c4,10
> < core-image-minimal-qemux86-64-20240718065105.cpio.gz
> < core-image-minimal-qemux86-64-20240718065105.ext4
> < core-image-minimal-qemux86-64-20240718065105.manifest
> < core-image-minimal-qemux86-64-20240718065105.qemuboot.conf
> < core-image-minimal-qemux86-64-20240718065105.spdx.tar.zst
> < core-image-minimal-qemux86-64-20240718065105.tar.bz2
> < core-image-minimal-qemux86-64-20240718065105.testdata.json
> ---
> > core-image-minimal-qemux86-64-20240718073341.cpio.gz
> > core-image-minimal-qemux86-64-20240718073341.ext4
> > core-image-minimal-qemux86-64-20240718073341.manifest
> > core-image-minimal-qemux86-64-20240718073341.qemuboot.conf
> > core-image-minimal-qemux86-64-20240718073341.spdx.tar.zst
> > core-image-minimal-qemux86-64-20240718073341.tar.bz2
> > core-image-minimal-qemux86-64-20240718073341.testdata.json
> 20c20
> < fitImage-core-image-minimal-qemux86-64--6.6.35+git-r0-qemux86-64-
> 20240718065105.bin
> ---
> > fitImage-core-image-minimal-qemux86-64--6.6.35+git-r0-qemux86-64-
> 20240718073341.bin
> 22c22
> < fitImage-its-core-image-minimal-qemux86-64--6.6.35+git-r0-qemux86-64-
> 20240718065105.its
> ---
> > fitImage-its-core-image-minimal-qemux86-64--6.6.35+git-r0-qemux86-64-
> 20240718073341.its
>
>
>
> The answer is then: No it does not re-build. At least not in the
> scenario which I would like to fix it, it does not rebuild from
> scratch.
>
> Does that make sense?
>
> Regards,
> Adrian
>
> > >
> > > -=-=-=-=-=-=-=-=-=-=-=-
> > > Links: You receive all messages sent to this group.
> > > View/Reply Online (#201933):
> > > https://lists.openembedded.org/g/openembedded-core/message/201933
> > > Mute This Topic:
> > > https://lists.openembedded.org/mt/107231736/3617156
> > > Group Owner: openembedded-core+owner@lists.openembedded.org
> > > Unsubscribe:
> > > https://lists.openembedded.org/g/openembedded-core/unsub [
> > > martin.jansa@gmail.com]
> > > -=-=-=-=-=-=-=-=-=-=-=-
> > >
>

Re: [OE-core] [PATCH v2 0/6] Use the kernel from sstate when building fitImages

[Jose Quaresma]

[-- Attachment #1: Type: text/plain, Size: 5424 bytes --]

Adrian Freihofer via lists.openembedded.org <adrian.freihofer=
gmail.com@lists.openembedded.org> escreveu (quinta, 18/07/2024 à(s) 09:00):

> Hi Martin
>
> Thank you for looking at my patches.
>
> On Mon, 2024-07-15 at 16:32 +0200, Martin Jansa wrote:
> > Doesn't it still rebuild from scratch when IMAGE_VERSION_SUFFIX
> > changes?
>
> I wonder why this happens in WebOS. I think that this line
> kernel_do_deploy[vardepsexclude] = "DATETIME"
> prevents such unnecessary rebuilds.
>
> So far we haven't had any problems with rebuilding kernels. The trouble
> started when we started using fitImages with initramfs for some
> devices. It is also worth noting that we build from sstate, but with an
> empty TMPDIR.
>
> In such a scenario bitbake does:
>
>  * The initramfs gets assembled. This is expected because images are
>    not sstate cached. This runs quickly because all the packages which
>    are required to build the initramfs come from sstate-cache.
>  * Since the initramfs changes, the fitImage which includes it must be
>    re-assembled as well.
>

I have faced this problem and sent a patch [1] to support storing the full
images on the sstate.
As it was not a tested and widely used case, it was rejected.
We continue using this in our distro but without the patch we need to
change more parts on our side.

[1] https://lists.openembedded.org/g/openembedded-core/message/184243


>  * Now the hassle starts: The do_assemble_fitimage_initramfs needs the
>    kernel binaries (linux.bin, DTBs...) which are not available from
>    sttate. Bitbake has to start with kernel do_fetch in case of an
>    empty TMPDIR just to get the fitImage assembled.
>
> With my patches the kernel binaries used for the fitImage come from
> sstate (if the initramfs is not bundled).
>

Anyway your overwall patch set is still a good improvement to not
rebuilding the kernel.
That was my concern also and the main reason for my patch.

Jose


>
> I tested this like that:
>
> cat build/conf/local.conf
>
> KERNEL_IMAGETYPE = "Image"
> KERNEL_IMAGETYPES += " fitImage "
> KERNEL_CLASSES = " kernel-fitimage "and RAM disk
> IMAGE_FSTYPES += "cpio.gz"
> INITRAMFS_IMAGE = "core-image-minimal"
> IMAGE_NAME_SUFFIX:pn-core-image-minimal = ""
> UBOOT_RD_LOADADDRESS = "0x88000000"
> UBOOT_RD_ENTRYPOINT = "0x88000000"
> UBOOT_LOADADDRESS = "0x80080000"
> UBOOT_ENTRYPOINT = "0x80080000"
> FIT_DESC = "A model description"
>
> # Allow to change the initramfs
> FOO_VAR = "1"
> INHERIT += "image-buildinfo"
> IMAGE_BUILDINFO_VARS:append = " FOO_VAR"
>
>
> bitbake virtual/kernel
>
> mv build/tmp build/tmp-old-1
>
> bitbake virtual/kernel
> -> bitbake does:
> - setscene tasks
> - core-image-minimal do_rootfs ...
> - kernel do_deploy (from sstate)
> - kernel do_deploy_fitimage_unbundled
>
>
> bitbake virtual/kernel
> -> bitbake does: nothing
>
>
> # Enforce a re-build of the initramfs
> mv build/tmp build/tmp-old-2
> echo 'FOO_VAR = "1"' >> build/conf/local.conf
>
> bitbake virtual/kernel
> -> bitbake does:
> - setscene tasks
> - core-image-minimal do_rootfs ...
> - kernel do_deploy (from sstate)
> - kernel do_deploy_fitimage_unbundled
>
>
> bitbake virtual/kernel
> -> bitbake does: nothing
>
>
> diff \
> <(ls build/tmp-old-2/deploy/images/qemux86-64 -1) \
> <(ls build/tmp/deploy/images/qemux86-64/ -1)
> 4,10c4,10
> < core-image-minimal-qemux86-64-20240718065105.cpio.gz
> < core-image-minimal-qemux86-64-20240718065105.ext4
> < core-image-minimal-qemux86-64-20240718065105.manifest
> < core-image-minimal-qemux86-64-20240718065105.qemuboot.conf
> < core-image-minimal-qemux86-64-20240718065105.spdx.tar.zst
> < core-image-minimal-qemux86-64-20240718065105.tar.bz2
> < core-image-minimal-qemux86-64-20240718065105.testdata.json
> ---
> > core-image-minimal-qemux86-64-20240718073341.cpio.gz
> > core-image-minimal-qemux86-64-20240718073341.ext4
> > core-image-minimal-qemux86-64-20240718073341.manifest
> > core-image-minimal-qemux86-64-20240718073341.qemuboot.conf
> > core-image-minimal-qemux86-64-20240718073341.spdx.tar.zst
> > core-image-minimal-qemux86-64-20240718073341.tar.bz2
> > core-image-minimal-qemux86-64-20240718073341.testdata.json
> 20c20
> < fitImage-core-image-minimal-qemux86-64--6.6.35+git-r0-qemux86-64-
> 20240718065105.bin
> ---
> > fitImage-core-image-minimal-qemux86-64--6.6.35+git-r0-qemux86-64-
> 20240718073341.bin
> 22c22
> < fitImage-its-core-image-minimal-qemux86-64--6.6.35+git-r0-qemux86-64-
> 20240718065105.its
> ---
> > fitImage-its-core-image-minimal-qemux86-64--6.6.35+git-r0-qemux86-64-
> 20240718073341.its
>
>
>
> The answer is then: No it does not re-build. At least not in the
> scenario which I would like to fix it, it does not rebuild from
> scratch.
>
> Does that make sense?
>
> Regards,
> Adrian
>
> > >
> > >
> > >
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#202191):
> https://lists.openembedded.org/g/openembedded-core/message/202191
> Mute This Topic: https://lists.openembedded.org/mt/107231736/5052612
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [
> quaresma.jose@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>

-- 
Best regards,

José Quaresma

[-- Attachment #2: Type: text/html, Size: 7479 bytes --]

Re: [OE-core] [PATCH v2 0/6] Use the kernel from sstate when building fitImages

[Adrian Freihofer]

On Thu, 2024-07-18 at 10:39 +0200, Martin Jansa wrote:
> On Thu, Jul 18, 2024 at 10:00 AM Adrian Freihofer
> <adrian.freihofer@gmail.com> wrote:
> > 
> > Hi Martin
> > 
> > Thank you for looking at my patches.
> > 
> > On Mon, 2024-07-15 at 16:32 +0200, Martin Jansa wrote:
> > > Doesn't it still rebuild from scratch when IMAGE_VERSION_SUFFIX
> > > changes?
> > 
> > I wonder why this happens in WebOS. I think that this line
> > kernel_do_deploy[vardepsexclude] = "DATETIME"
> > prevents such unnecessary rebuilds.
> 
> Hi Adrian,
> 
> yes DATETIME is excluded by default, but it's useful to use different
> suffix in IMAGE_VERSION_SUFFIX (e.g. BUILD_NUMBER from CI on jenkins
> or release version when building release) and in such cases you don't
> want to vardepexclude it, because it's useful to produce matching
> version in images as well as kernel (and other artifacts you might
> have), so your 1.2.1 release images doesn't end with 1.2.0 kernel
> artifacts just because kernel sstate signature didn't change in 1.2.1
> bugfix release.
> 
> Does this make sense? I bet it would happen in your builds as well.

I try to optimize everything towards binary-reproducible builds. This
means that we remove all kinds of variables that change because the
time changes or because the CI build ID changes or something like that.
I see no benefit in compiling such information into the firmware.
But the problem I see is that such ideas clash with efficient and
binary-reproducible builds.

That means that I don't see a reason for changing the
IMAGE_VERSION_SUFFIX variable in such a way.

> 
> Whole point of:
> https://bugzilla.yoctoproject.org/show_bug.cgi?id=12937
> is to allow "renaming" (adding more hardlinks) artifacts with
> matching
> IMAGE_VERSION_SUFFIX without the need of rebuilding anything (e.g.
> version-less kernel artifacts are re-used from sstate and then only
> quick deploy-links task is executed to add right versioned hardlinks
> in deploy directory.)
> 

Well, yes, it would. But, I am confident that it is a better strategy
to simply not do this than to support it.

If such non-reproducible information is needed anyway, it can be
archived independently of Bitbake.

Adrian

> Cheers,
> 
> > So far we haven't had any problems with rebuilding kernels. The
> > trouble
> > started when we started using fitImages with initramfs for some
> > devices. It is also worth noting that we build from sstate, but
> > with an
> > empty TMPDIR.
> > 
> > In such a scenario bitbake does:
> > 
> >  * The initramfs gets assembled. This is expected because images
> > are
> >    not sstate cached. This runs quickly because all the packages
> > which
> >    are required to build the initramfs come from sstate-cache.
> >  * Since the initramfs changes, the fitImage which includes it must
> > be
> >    re-assembled as well.
> >  * Now the hassle starts: The do_assemble_fitimage_initramfs needs
> > the
> >    kernel binaries (linux.bin, DTBs...) which are not available
> > from
> >    sttate. Bitbake has to start with kernel do_fetch in case of an
> >    empty TMPDIR just to get the fitImage assembled.
> > 
> > With my patches the kernel binaries used for the fitImage come from
> > sstate (if the initramfs is not bundled).
> > 
> > I tested this like that:
> > 
> > cat build/conf/local.conf
> > 
> > KERNEL_IMAGETYPE = "Image"
> > KERNEL_IMAGETYPES += " fitImage "
> > KERNEL_CLASSES = " kernel-fitimage "and RAM disk
> > IMAGE_FSTYPES += "cpio.gz"
> > INITRAMFS_IMAGE = "core-image-minimal"
> > IMAGE_NAME_SUFFIX:pn-core-image-minimal = ""
> > UBOOT_RD_LOADADDRESS = "0x88000000"
> > UBOOT_RD_ENTRYPOINT = "0x88000000"
> > UBOOT_LOADADDRESS = "0x80080000"
> > UBOOT_ENTRYPOINT = "0x80080000"
> > FIT_DESC = "A model description"
> > 
> > # Allow to change the initramfs
> > FOO_VAR = "1"
> > INHERIT += "image-buildinfo"
> > IMAGE_BUILDINFO_VARS:append = " FOO_VAR"
> > 
> > 
> > bitbake virtual/kernel
> > 
> > mv build/tmp build/tmp-old-1
> > 
> > bitbake virtual/kernel
> > -> bitbake does:
> > - setscene tasks
> > - core-image-minimal do_rootfs ...
> > - kernel do_deploy (from sstate)
> > - kernel do_deploy_fitimage_unbundled
> > 
> > 
> > bitbake virtual/kernel
> > -> bitbake does: nothing
> > 
> > 
> > # Enforce a re-build of the initramfs
> > mv build/tmp build/tmp-old-2
> > echo 'FOO_VAR = "1"' >> build/conf/local.conf
> > 
> > bitbake virtual/kernel
> > -> bitbake does:
> > - setscene tasks
> > - core-image-minimal do_rootfs ...
> > - kernel do_deploy (from sstate)
> > - kernel do_deploy_fitimage_unbundled
> > 
> > 
> > bitbake virtual/kernel
> > -> bitbake does: nothing
> > 
> > 
> > diff \
> > <(ls build/tmp-old-2/deploy/images/qemux86-64 -1) \
> > <(ls build/tmp/deploy/images/qemux86-64/ -1)
> > 4,10c4,10
> > < core-image-minimal-qemux86-64-20240718065105.cpio.gz
> > < core-image-minimal-qemux86-64-20240718065105.ext4
> > < core-image-minimal-qemux86-64-20240718065105.manifest
> > < core-image-minimal-qemux86-64-20240718065105.qemuboot.conf
> > < core-image-minimal-qemux86-64-20240718065105.spdx.tar.zst
> > < core-image-minimal-qemux86-64-20240718065105.tar.bz2
> > < core-image-minimal-qemux86-64-20240718065105.testdata.json
> > ---
> > > core-image-minimal-qemux86-64-20240718073341.cpio.gz
> > > core-image-minimal-qemux86-64-20240718073341.ext4
> > > core-image-minimal-qemux86-64-20240718073341.manifest
> > > core-image-minimal-qemux86-64-20240718073341.qemuboot.conf
> > > core-image-minimal-qemux86-64-20240718073341.spdx.tar.zst
> > > core-image-minimal-qemux86-64-20240718073341.tar.bz2
> > > core-image-minimal-qemux86-64-20240718073341.testdata.json
> > 20c20
> > < fitImage-core-image-minimal-qemux86-64--6.6.35+git-r0-qemux86-64-
> > 20240718065105.bin
> > ---
> > > fitImage-core-image-minimal-qemux86-64--6.6.35+git-r0-qemux86-64-
> > 20240718073341.bin
> > 22c22
> > < fitImage-its-core-image-minimal-qemux86-64--6.6.35+git-r0-
> > qemux86-64-
> > 20240718065105.its
> > ---
> > > fitImage-its-core-image-minimal-qemux86-64--6.6.35+git-r0-
> > > qemux86-64-
> > 20240718073341.its
> > 
> > 
> > 
> > The answer is then: No it does not re-build. At least not in the
> > scenario which I would like to fix it, it does not rebuild from
> > scratch.
> > 
> > Does that make sense?
> > 
> > Regards,
> > Adrian
> > 
> > > > 
> > > > -=-=-=-=-=-=-=-=-=-=-=-
> > > > Links: You receive all messages sent to this group.
> > > > View/Reply Online (#201933):
> > > > https://lists.openembedded.org/g/openembedded-core/message/201933
> > > > Mute This Topic:
> > > > https://lists.openembedded.org/mt/107231736/3617156
> > > > Group Owner: openembedded-core+owner@lists.openembedded.org
> > > > Unsubscribe:
> > > > https://lists.openembedded.org/g/openembedded-core/unsub [
> > > > martin.jansa@gmail.com]
> > > > -=-=-=-=-=-=-=-=-=-=-=-
> > > > 
> > 

Re: [OE-core] [PATCH v2 0/6] Use the kernel from sstate when building fitImages

[Adrian Freihofer]

On Thu, 2024-07-18 at 11:28 +0100, Jose Quaresma wrote:
> 
> 
> Adrian Freihofer via lists.openembedded.org
> <adrian.freihofer=gmail.com@lists.openembedded.org> escreveu (quinta,
> 18/07/2024 à(s) 09:00):
> > Hi Martin
> > 
> > Thank you for looking at my patches.
> > 
> > On Mon, 2024-07-15 at 16:32 +0200, Martin Jansa wrote:
> > > Doesn't it still rebuild from scratch when IMAGE_VERSION_SUFFIX
> > > changes?
> > 
> > I wonder why this happens in WebOS. I think that this line
> > kernel_do_deploy[vardepsexclude] = "DATETIME"
> > prevents such unnecessary rebuilds.
> > 
> > So far we haven't had any problems with rebuilding kernels. The
> > trouble
> > started when we started using fitImages with initramfs for some
> > devices. It is also worth noting that we build from sstate, but
> > with an
> > empty TMPDIR.
> > 
> > In such a scenario bitbake does:
> > 
> >  * The initramfs gets assembled. This is expected because images
> > are
> >    not sstate cached. This runs quickly because all the packages
> > which
> >    are required to build the initramfs come from sstate-cache.
> >  * Since the initramfs changes, the fitImage which includes it must
> > be
> >    re-assembled as well.
> > 
> 
> 
> I have faced this problem and sent a patch [1] to support storing the
> full images on the sstate.
> As it was not a tested and widely used case, it was rejected.
> We continue using this in our distro but without the patch we need to
> change more parts on our side.

I don't think adding images to the sstate cache can help much. But it
comes at a high cost. It definitely wouldn't help in the case that
should be optimized with my patches here:
 * If the initramfs changes it needs to be re-built and cannot be taken
   from the sstate-cache anyway.
 * The issue is: If the initramfs changes, the kernel gets re-built.
   What we need is an sstate for the kernel artifacts but not for the
   initramfs artifacts.

Adrian

> 
> [1] https://lists.openembedded.org/g/openembedded-core/message/184243
>  
> >  * Now the hassle starts: The do_assemble_fitimage_initramfs needs
> > the
> >    kernel binaries (linux.bin, DTBs...) which are not available
> > from
> >    sttate. Bitbake has to start with kernel do_fetch in case of an
> >    empty TMPDIR just to get the fitImage assembled.
> > 
> > With my patches the kernel binaries used for the fitImage come from
> > sstate (if the initramfs is not bundled).
> > 
> 
> 
> Anyway your overwall patch set is still a good improvement to not
> rebuilding the kernel.
> That was my concern also and the main reason for my patch.
>  
> Jose
>  
> > 
> > I tested this like that:
> > 
> > cat build/conf/local.conf
> > 
> > KERNEL_IMAGETYPE = "Image"
> > KERNEL_IMAGETYPES += " fitImage "
> > KERNEL_CLASSES = " kernel-fitimage "and RAM disk
> > IMAGE_FSTYPES += "cpio.gz"
> > INITRAMFS_IMAGE = "core-image-minimal"
> > IMAGE_NAME_SUFFIX:pn-core-image-minimal = ""
> > UBOOT_RD_LOADADDRESS = "0x88000000"
> > UBOOT_RD_ENTRYPOINT = "0x88000000"
> > UBOOT_LOADADDRESS = "0x80080000"
> > UBOOT_ENTRYPOINT = "0x80080000"
> > FIT_DESC = "A model description"
> > 
> > # Allow to change the initramfs
> > FOO_VAR = "1"
> > INHERIT += "image-buildinfo"
> > IMAGE_BUILDINFO_VARS:append = " FOO_VAR"
> > 
> > 
> > bitbake virtual/kernel
> > 
> > mv build/tmp build/tmp-old-1
> > 
> > bitbake virtual/kernel
> > -> bitbake does:
> > - setscene tasks
> > - core-image-minimal do_rootfs ...
> > - kernel do_deploy (from sstate)
> > - kernel do_deploy_fitimage_unbundled
> > 
> > 
> > bitbake virtual/kernel
> > -> bitbake does: nothing
> > 
> > 
> > # Enforce a re-build of the initramfs
> > mv build/tmp build/tmp-old-2
> > echo 'FOO_VAR = "1"' >> build/conf/local.conf
> > 
> > bitbake virtual/kernel
> > -> bitbake does:
> > - setscene tasks
> > - core-image-minimal do_rootfs ...
> > - kernel do_deploy (from sstate)
> > - kernel do_deploy_fitimage_unbundled
> > 
> > 
> > bitbake virtual/kernel
> > -> bitbake does: nothing
> > 
> > 
> > diff \
> > <(ls build/tmp-old-2/deploy/images/qemux86-64 -1) \
> > <(ls build/tmp/deploy/images/qemux86-64/ -1)
> > 4,10c4,10
> > < core-image-minimal-qemux86-64-20240718065105.cpio.gz
> > < core-image-minimal-qemux86-64-20240718065105.ext4
> > < core-image-minimal-qemux86-64-20240718065105.manifest
> > < core-image-minimal-qemux86-64-20240718065105.qemuboot.conf
> > < core-image-minimal-qemux86-64-20240718065105.spdx.tar.zst
> > < core-image-minimal-qemux86-64-20240718065105.tar.bz2
> > < core-image-minimal-qemux86-64-20240718065105.testdata.json
> > ---
> > > core-image-minimal-qemux86-64-20240718073341.cpio.gz
> > > core-image-minimal-qemux86-64-20240718073341.ext4
> > > core-image-minimal-qemux86-64-20240718073341.manifest
> > > core-image-minimal-qemux86-64-20240718073341.qemuboot.conf
> > > core-image-minimal-qemux86-64-20240718073341.spdx.tar.zst
> > > core-image-minimal-qemux86-64-20240718073341.tar.bz2
> > > core-image-minimal-qemux86-64-20240718073341.testdata.json
> > 20c20
> > < fitImage-core-image-minimal-qemux86-64--6.6.35+git-r0-qemux86-64-
> > 20240718065105.bin
> > ---
> > > fitImage-core-image-minimal-qemux86-64--6.6.35+git-r0-qemux86-64-
> > 20240718073341.bin
> > 22c22
> > < fitImage-its-core-image-minimal-qemux86-64--6.6.35+git-r0-
> > qemux86-64-
> > 20240718065105.its
> > ---
> > > fitImage-its-core-image-minimal-qemux86-64--6.6.35+git-r0-
> > > qemux86-64-
> > 20240718073341.its
> > 
> > 
> > 
> > The answer is then: No it does not re-build. At least not in the
> > scenario which I would like to fix it, it does not rebuild from
> > scratch.
> > 
> > Does that make sense?
> > 
> > Regards,
> > Adrian
> > 
> > > > 
> > > > 
> > > > 
> > 
> > 
> > -=-=-=-=-=-=-=-=-=-=-=-
> > Links: You receive all messages sent to this group.
> > View/Reply Online (#202191):
> > https://lists.openembedded.org/g/openembedded-core/message/202191
> > Mute This Topic:
> > https://lists.openembedded.org/mt/107231736/5052612
> > Group Owner: openembedded-core+owner@lists.openembedded.org
> > Unsubscribe: https://lists.openembedded.org/g/openembedded-
> > core/unsub [quaresma.jose@gmail.com]
> > -=-=-=-=-=-=-=-=-=-=-=-
> > 
> 
> 
> -- 
> Best regards,
> 
> José Quaresma

Re: [OE-core] [PATCH v2 0/6] Use the kernel from sstate when building fitImages

[Martin Jansa]

On Thu, Jul 18, 2024 at 2:37 PM Adrian Freihofer
<adrian.freihofer@gmail.com> wrote:
>
> On Thu, 2024-07-18 at 10:39 +0200, Martin Jansa wrote:
> > On Thu, Jul 18, 2024 at 10:00 AM Adrian Freihofer
> > <adrian.freihofer@gmail.com> wrote:
> > >
> > > Hi Martin
> > >
> > > Thank you for looking at my patches.
> > >
> > > On Mon, 2024-07-15 at 16:32 +0200, Martin Jansa wrote:
> > > > Doesn't it still rebuild from scratch when IMAGE_VERSION_SUFFIX
> > > > changes?
> > >
> > > I wonder why this happens in WebOS. I think that this line
> > > kernel_do_deploy[vardepsexclude] = "DATETIME"
> > > prevents such unnecessary rebuilds.
> >
> > Hi Adrian,
> >
> > yes DATETIME is excluded by default, but it's useful to use different
> > suffix in IMAGE_VERSION_SUFFIX (e.g. BUILD_NUMBER from CI on jenkins
> > or release version when building release) and in such cases you don't
> > want to vardepexclude it, because it's useful to produce matching
> > version in images as well as kernel (and other artifacts you might
> > have), so your 1.2.1 release images doesn't end with 1.2.0 kernel
> > artifacts just because kernel sstate signature didn't change in 1.2.1
> > bugfix release.
> >
> > Does this make sense? I bet it would happen in your builds as well.
>
> I try to optimize everything towards binary-reproducible builds. This
> means that we remove all kinds of variables that change because the
> time changes or because the CI build ID changes or something like that.
> I see no benefit in compiling such information into the firmware.
> But the problem I see is that such ideas clash with efficient and
> binary-reproducible builds.
>
> That means that I don't see a reason for changing the
> IMAGE_VERSION_SUFFIX variable in such a way.
>
> >
> > Whole point of:
> > https://bugzilla.yoctoproject.org/show_bug.cgi?id=12937
> > is to allow "renaming" (adding more hardlinks) artifacts with
> > matching
> > IMAGE_VERSION_SUFFIX without the need of rebuilding anything (e.g.
> > version-less kernel artifacts are re-used from sstate and then only
> > quick deploy-links task is executed to add right versioned hardlinks
> > in deploy directory.)
> >
>
> Well, yes, it would. But, I am confident that it is a better strategy
> to simply not do this than to support it.
>
> If such non-reproducible information is needed anyway, it can be
> archived independently of Bitbake.

Well the changes from:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=12937
make the builds _more_ reproducible, because they move the DATETIME or
whatever IMAGE_VERSION_SUFFIX out of the artifacts in sstate and move
creating versioned names to separate task which can be disabled (by
setting IMAGE_VERSION_SUFFIX to empty if you prefer not to create
them).

Re: [OE-core] [PATCH v2 0/6] Use the kernel from sstate when building fitImages

[Alexandre Belloni]

Hello,

I'm pretty sure this causes this failure:

https://autobuilder.yoctoproject.org/typhoon/#/builders/151/builds/2005/steps/12/logs/stdio

And also those oe-selftest failures:

https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/6945/steps/14/logs/stdio
https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/6987/steps/15/logs/stdio
https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/7000/steps/14/logs/stdio
https://autobuilder.yoctoproject.org/typhoon/#/builders/86/builds/7010/steps/14/logs/stdio


On 15/07/2024 16:10:37+0200, Adrian Freihofer wrote:
> Changes in comparison to v1:
> - Add the missing dependency from do_image_wic to
>   initramfs.do_image_complete which caused the build failure on the AB.
> - Support symbolic links for $dtb_path again, as suggested by Mark.
> - Simplify the refactoring of the uboot_prep_kimage function.
> - Evaluate staging via sysroot instead of taking the kernel artifacts
>   from the deploy folder. But there is an issue with this approach: It
>   does not allow to remove the dependency from do_deploy on do_install.
> - Rebasing to latest master
> 
> If the KERNEL_IMAGETYPES(S) contains fitImage, the kernel is always
> rebuilt when something changes in the initramfs frequently.
> This is even worse if the build runs from an empty TMPDIR. The kernel
> re-build starts by fetching the large kernel git repository and
> recompiling it from scratch.
> 
> This cannot be improved if INITRAMFS_IMAGE_BUNDLE = "1". If the kernel
> Makefile is needed to generate the initramfs bundle the kernel build
> folder is required.
> But for a build configuration with INITRAMFS_IMAGE_BUNDLE = "" the
> build folder is not needed. Creating the initramfs bundle requires:
> linux.bin, DTBs and the initramfs which are available in the deploy
> directory. That means creating the fitImage can be done with artifacts
> which are already cached by the sstate.
> 
> There is an extra commit providing a html file. This html file provides
> some graphics which show the changes in the kernel's task dependencies.
> I hope this simplifies the re-view of this patch series.
> 
> Testing:
> - oe-selftest -a --skip-tests distrodata.Distrodata.test_checkpkg \
>      reproducible -T yocto-mirrors -T machine -T toolchain-user \
>      -T toolchain-system
> - Build for core-image-minimal for MACHINE = "genericarm64"
>   (failure from AB was reproducible and is fixed now)
> - Build clean followed by build from sstate
>   Note: Adding this test to oe-selftest failed because sstate is read only
>   - Clean build config:
>     KERNEL_IMAGETYPE = "Image"
>     KERNEL_IMAGETYPES += " fitImage "
>     KERNEL_CLASSES = " kernel-fitimage "
>     IMAGE_FSTYPES += "cpio.gz"
>     INITRAMFS_IMAGE = "core-image-minimal"
>     IMAGE_NAME_SUFFIX:pn-core-image-minimal = ""
>     UBOOT_RD_LOADADDRESS = "0x88000000"
>     UBOOT_RD_ENTRYPOINT = "0x88000000"
>     UBOOT_LOADADDRESS = "0x80080000"
>     UBOOT_ENTRYPOINT = "0x80080000"
>     FIT_DESC = "A model description"
>     FOO_VAR = "1"
>     INHERIT += "image-buildinfo"
>     IMAGE_BUILDINFO_VARS:append = " FOO_VAR"
>   - Append the following and rebuild with sstate:
>     FOO_VAR = "2"
>     TMPDIR = "${TOPDIR}/tmp-2"
>   - Check the log files:
>     - tmp/log/cooker/qemux86-64 contains:
>       linux-yocto-6.6.35+git-r0: task do_deploy: Succeeded
>       linux-yocto-6.6.35+git-r0: task do_deploy_fitimage_unbundled: Succeeded
>     - tmp-2/log/cooker/qemux86-64 contains:
>       linux-yocto-6.6.35+git-r0: task do_deploy_setscene: Succeeded
>       linux-yocto-6.6.35+git-r0: task do_deploy_fitimage_unbundled: Succeeded
> - To re-view the chagnes in the task dependencies, the script bellow
>   has been used. It confirms:
>   - For builds with fitImage and unbundled initramfs:
>     - the do_deploy_fitimage_unbundled task runs after do_deploy
>     - do_assemble_fitimage_initramfs is not executed
>     - do_bundle_initramfs is not executed
>     - It works for fitImage in KERNEL_IMAGETYPE as well as for fitImage
>       in KERNEL_IMAGETYPES
>   - For builds with fitImage and bundled initramfs: No changes
>   - For builds with fitImage, without initramfs:
>     - do_assemble_fitimage_initramfs is not executed
>     - do_bundle_initramfs is not executed
> 
> OUTPUT_FILE=task-depends.md
> GIT_BRANCH=master
> GIT_BRANCH_NEXT=adrianf/kernel-fitimage-sstate
> 
> echo "# Task dependeny changes" > "$OUTPUT_FILE"
> echo "" >> "$OUTPUT_FILE"
> 
> run_bitbake(){
>     echo "$1" >> "$OUTPUT_FILE"
>     echo "" >> "$OUTPUT_FILE"
> 
>     bitbake virtual/kernel -g
>     grep -E '(digraph depends|linux-yocto|\})' task-depends.dot \
>     | grep -v -E '(spdx|do_kernel_configcheck|do_prepare_recipe_sysroot|do_populate_lic|native.do_populate_sysroot)' \
>     | grep -v -E '(depmodwrapper-cross.do_populate_sysroot|binutils-cross-x86_64.do_populate_sysroot|do_package)' \
>     | grep -v -E '(do_validate_branches|do_unpack|do_recipe_qa|do_patch|do_kernel_checkout|do_kernel_configme|do_kernel_metadata)' \
>     | grep -v -E '(do_sizecheck|do_strip|do_compile_kernelmodules|do_shared_workdir|do_transform_kernel|do_kernel_link_images)' \
>     | grep -v -E '(do_kernel_version_sanity_check|do_symlink_kernsrc|do_deploy_source_date_epoch|do_configure|do_fetch)' \
>     | sed -e 's;\\n\:.*.bb;;g' -e 's;linux-yocto[. ];;g' > task-depends-filtered.dot
> 
>     echo '```plantuml' >> "$OUTPUT_FILE"
>     cat task-depends-filtered.dot >> "$OUTPUT_FILE"
>     echo '```' >> "$OUTPUT_FILE"
>     echo "" >> "$OUTPUT_FILE"
> }
> 
> run_bitbake_per_branch(){
>     echo "## Configuration: $1" >> "$OUTPUT_FILE"
>     echo "" >> "$OUTPUT_FILE"
>     echo '```raw' >> "$OUTPUT_FILE"
>     cat build/conf/auto.conf >> "$OUTPUT_FILE"
>     echo '```' >> "$OUTPUT_FILE"
>     echo "" >> "$OUTPUT_FILE"
> 
>     git checkout $GIT_BRANCH
>     run_bitbake "### branch: $GIT_BRANCH"
>     mv -f task-depends-filtered.dot task-depends-filtered-old.dot
> 
>     git checkout $GIT_BRANCH_NEXT
>     run_bitbake "### branch: $GIT_BRANCH_NEXT"
> 
>     echo "## Diff" >> "$OUTPUT_FILE"
>     echo "" >> "$OUTPUT_FILE"
>     echo '```patch' >> "$OUTPUT_FILE"
>     diff task-depends-filtered-old.dot task-depends-filtered.dot >> "$OUTPUT_FILE"
>     echo '```' >> "$OUTPUT_FILE"
>     echo "" >> "$OUTPUT_FILE"
> }
> 
> cat << EOF > build/conf/auto.conf
> KERNEL_IMAGETYPE = "Image"
> KERNEL_IMAGETYPES += " fitImage "
> KERNEL_CLASSES = " kernel-fitimage "
> IMAGE_FSTYPES += "cpio.gz"
> INITRAMFS_IMAGE = "core-image-minimal-initramfs"
> EOF
> run_bitbake_per_branch "image, fitimage, initramfs, unbundled"
> 
> cat << EOF > build/conf/auto.conf
> KERNEL_IMAGETYPE:forcevariable = "fitImage"
> KERNEL_CLASSES = " kernel-fitimage "
> IMAGE_FSTYPES += "cpio.gz"
> INITRAMFS_IMAGE = "core-image-minimal-initramfs"
> EOF
> run_bitbake_per_branch "fitimage, initramfs, unbundled"
> 
> cat << EOF > build/conf/auto.conf
> KERNEL_IMAGETYPE = "Image"
> KERNEL_IMAGETYPES += " fitImage "
> KERNEL_CLASSES = " kernel-fitimage "
> IMAGE_FSTYPES += "cpio.gz"
> INITRAMFS_IMAGE = "core-image-minimal-initramfs"
> INITRAMFS_IMAGE_BUNDLE = "1"
> EOF
> run_bitbake_per_branch "image, fitimage, initramfs, bundled"
> 
> cat << EOF > build/conf/auto.conf
> KERNEL_IMAGETYPE = "Image"
> KERNEL_IMAGETYPES += " fitImage "
> KERNEL_CLASSES = " kernel-fitimage "
> EOF
> run_bitbake_per_branch "image, fitimage"
> 
> rm -f build/conf/auto.conf task-depends-filtered-old.dot task-depends-filtered.dot task-depends.dot
> 
> Adrian Freihofer (6):
>   kernel-fitimage: fix intentation
>   kernel-fitimage: fix external dtb check
>   kernel: refactor linux compression
>   kernel-fitimage: refactor fitimage_assemble
>   kernel: refactor fitimage
>   kernel-fitimage: run unbundled fitimage after deploy
> 
>  meta/classes-recipe/image.bbclass           |  12 +-
>  meta/classes-recipe/kernel-fitimage.bbclass | 188 ++++++++++++--------
>  meta/classes-recipe/kernel-uboot.bbclass    |   1 +
>  meta/classes-recipe/kernel.bbclass          |  31 ++--
>  4 files changed, 148 insertions(+), 84 deletions(-)
> 
> -- 
> 2.45.2
> 

> 
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#201926): https://lists.openembedded.org/g/openembedded-core/message/201926
> Mute This Topic: https://lists.openembedded.org/mt/107231736/3617179
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alexandre.belloni@bootlin.com]
> -=-=-=-=-=-=-=-=-=-=-=-
> 


-- 
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com