Hashserve OEEquivHash unihash issue

["Harro Verton" <wanwizard@openpli.org>]

[-- Attachment #1: Type: text/plain, Size: 4002 bytes --]

Hello Community,

First post, and a relative OE newbie, so please bear with me.

I've inherited a build environment of an open source project ( https://github.com/OpenPLi/openpli-oe-core ). The project builds images for some 120 different devices, with MIPSEL or ARM SoCs, covering 5 different ARCHs.

All are build from a single repository, containing the submodules for Bitbake, Openembedded-Core and meta-openembedded, and all required BSP's for these devices. They all share sstate, a PR server, and a Hashserve server.

In order to avoid packages build for the ARCH feed rebuilding for all subsequent MACHINEs using that same ARCH, my predecessor has introduced HashServe, using OEEquivHash.

This works fine in our previous release (which is based on Hardknott), and worked fine in our current release (which is based on Scarthgap), until a few months ago.
We want to start with the next release, based on Wrynose, but we can't until we can make a final Scarthgap release, which we can't because of this issue.

I've run a  test, building two different MACHINEs with the same ARCH (identical hardware, different brands, different drivers, so separate BSPs), with an empty build directory (so no sstate cache, new PR database, no hash data). For the second machine, 230 tasks are recorded as having a changed unihash, which triggers a recipe rebuild for the second MACHINE.

I've enabled hashserve debugging, and these recipes log something like:
log.do_package_write_ipk.212153:DEBUG: Reported task efe6373f8d36e03f92f0198b5c2891fba8b459bd1e16ee9043f3e3d85a47c65e as unihash efe6373f8d36e03f92f0198b5c2891fba8b459bd1e16ee9043f3e3d85a47c65e to unix:///data/openpli/oe/scarthgap/build/hashserve.sock log.do_package_write_ipk.2223688:DEBUG: Task 8da90b016d0a41cac6ce46c20d8ee64d5e7af35687de43c306544508050227e8 unihash changed 8da90b016d0a41cac6ce46c20d8ee64d5e7af35687de43c306544508050227e8 -> efe6373f8d36e03f92f0198b5c2891fba8b459bd1e16ee9043f3e3d85a47c65e by server unix:///data/openpli/oe/scarthgap/build/hashserve.sock
so the second build records a changed unihash, which triggers a rebuild of the package, and leads to an r0.1 version of the ipk.

All of them but 8 report a unihash change for do_package_qa , those 8 also for do_package_write_ipk.

The downside of this is that after device A is flashed with the result of the first build, and you do an "opkg list-upgradable", you're presented with 182 packages from the ARCH feed shared between the two devices that are supposedly changed (but the only change is r0.0 to r0.1 due to the rebuild). Apart from the fact that most of our users are not tech savvy, several devices are limited on flash memory space, so having to update 180+ packages every time we have a build run is quite a bit of an issue.

I can't find a reason why this suddenly happens, some of these packages are quite basic and don't contain binary code, packages like python3-requests for example. And I have no idea how to debug this further.

For this test build, Bitbake is at yocto-5.0.19, openembedded-core and meta-openembedded are on the scarthgap branch, pulled yesterday.

I've looked at the commit history in our repo, the only time this trio was updated was on July 30th. If I revert that, bitbake goes back to yocto-5.0.15 and both OE submodules to a commit in december 2025, which is when we started with our Scarthgap branch.
When I repeat the test build after the revert, only two recipes for the ARCH feed are rebuild for the second machine, ffmpeg and gstreamer, both are recipes that we override in our own meta layer ( version upgrades to those in openembedded ).

So somewhere along the line, something got broken (or how it worked changed, and we missed that).

re there fixes in later versions for this issue, that are not backported to Scarthgap? How can I debug what exactly causes the unihash to change? Or better, how do I fix this?

TIA,

Harro Verton, member of the PLi Association
https://openpli.org

[-- Attachment #2: Type: text/html, Size: 6250 bytes --]