Re: [OE-core] [PATCH v2 0/6] Use the kernel from sstate when building fitImages

Martin Jansa
2024-07-18T08:40:19Z

On Thu, Jul 18, 2024 at 10:00 AM Adrian Freihofer
<adrian.freihofer@gmail.com> wrote:
>
> Hi Martin
>
> Thank you for looking at my patches.
>
> On Mon, 2024-07-15 at 16:32 +0200, Martin Jansa wrote:
> > Doesn't it still rebuild from scratch when IMAGE_VERSION_SUFFIX
> > changes?
>
> I wonder why this happens in WebOS. I think that this line
> kernel_do_deploy[vardepsexclude] = "DATETIME"
> prevents such unnecessary rebuilds.

Hi Adrian,

yes DATETIME is excluded by default, but it's useful to use different
suffix in IMAGE_VERSION_SUFFIX (e.g. BUILD_NUMBER from CI on jenkins
or release version when building release) and in such cases you don't
want to vardepexclude it, because it's useful to produce matching
version in images as well as kernel (and other artifacts you might
have), so your 1.2.1 release images doesn't end with 1.2.0 kernel
artifacts just because kernel sstate signature didn't change in 1.2.1
bugfix release.

Does this make sense? I bet it would happen in your builds as well.

Whole point of:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=12937
is to allow "renaming" (adding more hardlinks) artifacts with matching
IMAGE_VERSION_SUFFIX without the need of rebuilding anything (e.g.
version-less kernel artifacts are re-used from sstate and then only
quick deploy-links task is executed to add right versioned hardlinks
in deploy directory.)

Cheers,

> So far we haven't had any problems with rebuilding kernels. The trouble
> started when we started using fitImages with initramfs for some
> devices. It is also worth noting that we build from sstate, but with an
> empty TMPDIR.
>
> In such a scenario bitbake does:
>
> * The initramfs gets assembled. This is expected because images are
> not sstate cached. This runs quickly because all the packages which
> are required to build the initramfs come from sstate-cache.
> * Since the initramfs changes, the fitImage which includes it must be
> re-assembled as well.
> * Now the hassle starts: The do_assemble_fitimage_initramfs needs the
> kernel binaries (linux.bin, DTBs...) which are not available from
> sttate. Bitbake has to start with kernel do_fetch in case of an
> empty TMPDIR just to get the fitImage assembled.
>
> With my patches the kernel binaries used for the fitImage come from
> sstate (if the initramfs is not bundled).
>
> I tested this like that:
>
> cat build/conf/local.conf
>
> KERNEL_IMAGETYPE = "Image"
> KERNEL_IMAGETYPES += " fitImage "
> KERNEL_CLASSES = " kernel-fitimage "and RAM disk
> IMAGE_FSTYPES += "cpio.gz"
> INITRAMFS_IMAGE = "core-image-minimal"
> IMAGE_NAME_SUFFIX:pn-core-image-minimal = ""
> UBOOT_RD_LOADADDRESS = "0x88000000"
> UBOOT_RD_ENTRYPOINT = "0x88000000"
> UBOOT_LOADADDRESS = "0x80080000"
> UBOOT_ENTRYPOINT = "0x80080000"
> FIT_DESC = "A model description"
>
> # Allow to change the initramfs
> FOO_VAR = "1"
> INHERIT += "image-buildinfo"
> IMAGE_BUILDINFO_VARS:append = " FOO_VAR"
>
>
> bitbake virtual/kernel
>
> mv build/tmp build/tmp-old-1
>
> bitbake virtual/kernel
> -> bitbake does:
> - setscene tasks
> - core-image-minimal do_rootfs ...
> - kernel do_deploy (from sstate)
> - kernel do_deploy_fitimage_unbundled
>
>
> bitbake virtual/kernel
> -> bitbake does: nothing
>
>
> # Enforce a re-build of the initramfs
> mv build/tmp build/tmp-old-2
> echo 'FOO_VAR = "1"' >> build/conf/local.conf
>
> bitbake virtual/kernel
> -> bitbake does:
> - setscene tasks
> - core-image-minimal do_rootfs ...
> - kernel do_deploy (from sstate)
> - kernel do_deploy_fitimage_unbundled
>
>
> bitbake virtual/kernel
> -> bitbake does: nothing
>
>
> diff \
> <(ls build/tmp-old-2/deploy/images/qemux86-64 -1) \
> <(ls build/tmp/deploy/images/qemux86-64/ -1)
> 4,10c4,10
> < core-image-minimal-qemux86-64-20240718065105.cpio.gz
> < core-image-minimal-qemux86-64-20240718065105.ext4
> < core-image-minimal-qemux86-64-20240718065105.manifest
> < core-image-minimal-qemux86-64-20240718065105.qemuboot.conf
> < core-image-minimal-qemux86-64-20240718065105.spdx.tar.zst
> < core-image-minimal-qemux86-64-20240718065105.tar.bz2
> < core-image-minimal-qemux86-64-20240718065105.testdata.json
> ---
> > core-image-minimal-qemux86-64-20240718073341.cpio.gz
> > core-image-minimal-qemux86-64-20240718073341.ext4
> > core-image-minimal-qemux86-64-20240718073341.manifest
> > core-image-minimal-qemux86-64-20240718073341.qemuboot.conf
> > core-image-minimal-qemux86-64-20240718073341.spdx.tar.zst
> > core-image-minimal-qemux86-64-20240718073341.tar.bz2
> > core-image-minimal-qemux86-64-20240718073341.testdata.json
> 20c20
> < fitImage-core-image-minimal-qemux86-64--6.6.35+git-r0-qemux86-64-
> 20240718065105.bin
> ---
> > fitImage-core-image-minimal-qemux86-64--6.6.35+git-r0-qemux86-64-
> 20240718073341.bin
> 22c22
> < fitImage-its-core-image-minimal-qemux86-64--6.6.35+git-r0-qemux86-64-
> 20240718065105.its
> ---
> > fitImage-its-core-image-minimal-qemux86-64--6.6.35+git-r0-qemux86-64-
> 20240718073341.its
>
>
>
> The answer is then: No it does not re-build. At least not in the
> scenario which I would like to fix it, it does not rebuild from
> scratch.
>
> Does that make sense?
>
> Regards,
> Adrian
>
> > >
> > > -=-=-=-=-=-=-=-=-=-=-=-
> > > Links: You receive all messages sent to this group.
> > > View/Reply Online (#201933):
> > > https://lists.openembedded.org/g/openembedded-core/message/201933
> > > Mute This Topic:
> > > https://lists.openembedded.org/mt/107231736/3617156
> > > Group Owner: openembedded-core+owner@lists.openembedded.org
> > > Unsubscribe:
> > > https://lists.openembedded.org/g/openembedded-core/unsub [
> > > martin.jansa@gmail.com]
> > > -=-=-=-=-=-=-=-=-=-=-=-
> > >
>