XZ_THREADS changing do_image_cpio task hash

XZ_THREADS changing do_image_cpio task hash

[Mike Crowe]

It looks like this problem was raised previously[1] but there was some
confusion that made it look like it had been resolved.

Using current master or scarthgap, if I add:

 XZ_THREADS = "32"
 INITRAMFS_FSTYPES = "cpio.xz"

to local.conf, build core-image-minimal-initramfs (for qemuarm64) and then
tweak this to:

 XZ_THREADS = "8"
 INITRAMFS_FSTYPES = "cpio.xz"

and build core-image-minimal-initramfs again then the task hash for
do_image_cpio changes:

 tmp-glibc/stamps/qemuarm64-oe-linux/core-image-minimal-initramfs/1.0.do_image_cpio.sigdata.b1d5686160dfefe3c2aeddc5caa6a7d3e14e6478392abf81ddaf04d5cbaeaa1e
 tmp-glibc/stamps/qemuarm64-oe-linux/core-image-minimal-initramfs/1.0.do_image_cpio.sigdata.28a20d80e712a003b71146fd37ed1158dc99f4ef68f714e19d25fe6f5c83091b

basehash changed from ea836e918b7b1d732979d58e1e1d249355c59dc3e5a7d15e4be1ae66ff5dfda0 to 0029ad7eb64839cb1d6ac8de22d94676ddf9503b685517225546e651fb49935b
Variable do_image_cpio value changed:
@@ -17,5 +17,5 @@
 	fi

 	cd ${TMPDIR}/work/qemuarm64-oe-linux/core-image-minimal-initramfs/1.0/deploy-core-image-minimal-initramfs-image-complete
-	xz -f -k -c -6 --memlimit=50% --threads=32 --check=crc32 core-image-minimal-initramfs-qemuarm64${IMAGE_VERSION_SUFFIX}.cpio > core-image-minimal-initramfs-qemuarm64${IMAGE_VERSION_SUFFIX}.cpio.xz
+	xz -f -k -c -6 --memlimit=50% --threads=8 --check=crc32 core-image-minimal-initramfs-qemuarm64${IMAGE_VERSION_SUFFIX}.cpio > core-image-minimal-initramfs-qemuarm64${IMAGE_VERSION_SUFFIX}.cpio.xz
 	rm core-image-minimal-initramfs-qemuarm64${IMAGE_VERSION_SUFFIX}.cpio

My understanding of vardepsexclude and vardepvalue means that bitbake.conf
containing:

 XZ_THREADS ?= "${@oe.utils.cpu_count(at_least=2)}"
 XZ_THREADS[vardepvalue] = "1"
 XZ_DEFAULTS ?= "--memlimit=${XZ_MEMLIMIT} --threads=${XZ_THREADS}"
 XZ_DEFAULTS[vardepsexclude] += "XZ_MEMLIMIT XZ_THREADS"

ought to avoid the task hash changing. But it doesn't.

Am I missing an obvious reason why the task hash should change? Our build
hosts and developer machines have varying numbers of CPUs and this is
causing a lack of sstate sharing. :(

Thanks.

Mike.

[1] openembedded-core@lists.openembedded.org

Re: [OE-core] XZ_THREADS changing do_image_cpio task hash

[Alexander Kanavin]

On Tue, 10 Feb 2026 at 17:28, Mike Crowe via lists.openembedded.org
<mac=mcrowe.com@lists.openembedded.org> wrote:

> Am I missing an obvious reason why the task hash should change? Our build
> hosts and developer machines have varying numbers of CPUs and this is
> causing a lack of sstate sharing. :(

There might be a real issue in there somewhere but I need to point out
that none of the image tasks contribute anything to sstate, so if you
want to improve sstate reuse, images are the wrong place to start
looking.

Alex

Re: [OE-core] XZ_THREADS changing do_image_cpio task hash

[Mike Crowe]

On Wednesday 11 February 2026 at 12:41:30 +0100, Alexander Kanavin wrote:
> On Tue, 10 Feb 2026 at 17:28, Mike Crowe via lists.openembedded.org
> <mac=mcrowe.com@lists.openembedded.org> wrote:
> 
> > Am I missing an obvious reason why the task hash should change? Our build
> > hosts and developer machines have varying numbers of CPUs and this is
> > causing a lack of sstate sharing. :(
> 
> There might be a real issue in there somewhere but I need to point out
> that none of the image tasks contribute anything to sstate, so if you
> want to improve sstate reuse, images are the wrong place to start
> looking.

It's actually the kernel tasks that include the initramfs image that I'm
interested in getting from sstate.

In my testing, although the image is always rebuilt, the kernel does
successfully come from sstate when XZ_THREADS matches.

(There is also an ugly proliferation of separate siginfo files for the
various hashes in sstate, even if there are no actual artifacts, but these
are small.)

Thanks.

Mike.