Progress on AB-INT issues // Help needed!

Progress on AB-INT issues // Help needed!

[Paul Barker]

[-- Attachment #1: Type: text/plain, Size: 3257 bytes --]

Hi all,

Firstly I want to thank everyone who has helped out with the
intermittent autobuilder issues and our current high priority issues so
far. There have been contributions from several different folks and they
are really appreciated.

This help means that we've made good progress!

We've been able to resolve several AB-INT issues so far (#16118, #16121,
#15912, #15188, #15495) and have patches available for others which need
reviewing and merging (#15696, #15885). Some issues are resolved on
master and just waiting on backporting of patches to the relevant stable
branches (#16000, maybe others).

We need to keep up the momentum as we still have a couple of open issues
blocking the Yocto 6.0 M1 build and we still have too many intermittent
failures occurring on the autobuilder.

There are a few issues I can flag where we need extra help:

- segfault in matchbox-windowmanager on qemux86-64 intermittently
  (https://bugzilla.yoctoproject.org/show_bug.cgi?id=13908):
  It's unclear why the simple-looking code here is segfaulting. Ross has
  tried running with ASAN but that didn't give us any new information.
  Can someone add further debugging to the code to catch what's going
  on?

- AB-INT: fetch/sstate/CDN - Missing objects in the cache: Bad file
  descriptor (https://bugzilla.yoctoproject.org/show_bug.cgi?id=15945):
  The error here suggests we may have exceeded limits on the number of
  open file descriptors. Can someone add diagnostics to the relevant
  code path to print how many open file descriptors we have when the
  error occurs?

- AB-INT: runqemu.QemuTest.test_qemu_can_shutdown hangs with "Data
  received serial thread"
  (https://bugzilla.yoctoproject.org/show_bug.cgi?id=15698):
  This needs further investigation.

- AB-INT PTEST: openssl ['Dubious,_test_returned_1', 'openssl'] test
  failure (https://bugzilla.yoctoproject.org/show_bug.cgi?id=15357):
  We need someone to check the logs and confirm that it is always the
  same test case that is failing. If that's true, we can take a patch to
  skip the flaky test.

- Various gstreamer PTEST failures (#16127, #16108, #15408, #15973,
  #15910, #15718):
  Each of these needs to be reviewed to see if upstream have replied to
  our bug reports. If any haven't been filed upstream then they need to
  be. If there isn't an upstream fix available, we can take patches to
  skip the flaky tests.

- Various PTEST failures on RISC-V (#15895, #15884, #15942, #15929,
  #15896):
  We don't use KVM for running qemuriscv64 so tests can be much slower,
  this may be causing various timeouts. Some of the failing cases create
  a large amount of data so there may be issues with buffering. Further
  investigation is needed, we could take patches to skip these tests on
  RISC-V.

We will have another call tomorrow at 09:00 PST / 17:00 UTC to discuss
the issues further and make sure we're focusing effort in the right
places. Please attend via zoom if you're able to help out:
https://lists.yoctoproject.org/g/yocto/topic/yocto_project_ab_int_bug/117364044.

If you're unable to attend the call then you can still help by picking
up one of the issues above.

Best regards,

-- 
Paul Barker


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 252 bytes --]

Re: [yocto] Progress on AB-INT issues // Help needed!

[Yoann Congal]

[-- Attachment #1: Type: text/plain, Size: 5021 bytes --]

Le mar. 20 janv. 2026 à 18:58, Paul Barker via lists.yoctoproject.org <paul=
pbarker.dev@lists.yoctoproject.org> a écrit :

> Hi all,
>
> Firstly I want to thank everyone who has helped out with the
> intermittent autobuilder issues and our current high priority issues so
> far. There have been contributions from several different folks and they
> are really appreciated.
>
> This help means that we've made good progress!
>
> We've been able to resolve several AB-INT issues so far (#16118, #16121,
> #15912, #15188, #15495) and have patches available for others which need
> reviewing and merging (#15696, #15885). Some issues are resolved on
> master and just waiting on backporting of patches to the relevant stable
> branches (#16000, maybe others).
>

As for stable branches, some AB-INT bugs have fixes in review:
* 16000 – AB-INT: ffmpeg build failing => fix currently in review for
integration in whinlatter
* 16099 – devtool: oe-selftest failing on Ubuntu 25.10: couldn't allocate
absolute path for 'null'. => patch in review for integration in whinlatter
* 16137 – AB-INT: core-image-sato.bb:do_testsdk fails on ftpmirror.gnu.org
returning 502 Bad Gateway => patch in review for integration in kirkstone

Some are "in progress":
* 15696 – AB_INT: resulttool: File exists => I've tried a fix but it was
apparently incomplete for stable, I will wait for the recent patches to
land and get tested on master before trying again.
* 16122 – [scarthgap] AB-INT: qemuarm64-tc/qemux86-tc
rust.RustSelfTestSystemEmulated.test_rust fails with "panicked at
src/tools/compiletest/src/lib.rs:1054:31" => I keep to reproduce this one
under an instrumented branch but no luck. On the other hand, I also did not
trigger this one during my tests on the recent scarthgap patch series.

We need to keep up the momentum as we still have a couple of open issues
> blocking the Yocto 6.0 M1 build and we still have too many intermittent
> failures occurring on the autobuilder.
>
> There are a few issues I can flag where we need extra help:
>
> - segfault in matchbox-windowmanager on qemux86-64 intermittently
>   (https://bugzilla.yoctoproject.org/show_bug.cgi?id=13908):
>   It's unclear why the simple-looking code here is segfaulting. Ross has
>   tried running with ASAN but that didn't give us any new information.
>   Can someone add further debugging to the code to catch what's going
>   on?
>
> - AB-INT: fetch/sstate/CDN - Missing objects in the cache: Bad file
>   descriptor (https://bugzilla.yoctoproject.org/show_bug.cgi?id=15945):
>   The error here suggests we may have exceeded limits on the number of
>   open file descriptors. Can someone add diagnostics to the relevant
>   code path to print how many open file descriptors we have when the
>   error occurs?
>
> - AB-INT: runqemu.QemuTest.test_qemu_can_shutdown hangs with "Data
>   received serial thread"
>   (https://bugzilla.yoctoproject.org/show_bug.cgi?id=15698):
>   This needs further investigation.
>
> - AB-INT PTEST: openssl ['Dubious,_test_returned_1', 'openssl'] test
>   failure (https://bugzilla.yoctoproject.org/show_bug.cgi?id=15357):
>   We need someone to check the logs and confirm that it is always the
>   same test case that is failing. If that's true, we can take a patch to
>   skip the flaky test.
>
> - Various gstreamer PTEST failures (#16127, #16108, #15408, #15973,
>   #15910, #15718):
>   Each of these needs to be reviewed to see if upstream have replied to
>   our bug reports. If any haven't been filed upstream then they need to
>   be. If there isn't an upstream fix available, we can take patches to
>   skip the flaky tests.
>
> - Various PTEST failures on RISC-V (#15895, #15884, #15942, #15929,
>   #15896):
>   We don't use KVM for running qemuriscv64 so tests can be much slower,
>   this may be causing various timeouts. Some of the failing cases create
>   a large amount of data so there may be issues with buffering. Further
>   investigation is needed, we could take patches to skip these tests on
>   RISC-V.
>
> We will have another call tomorrow at 09:00 PST / 17:00 UTC to discuss
> the issues further and make sure we're focusing effort in the right
> places. Please attend via zoom if you're able to help out:
>
> https://lists.yoctoproject.org/g/yocto/topic/yocto_project_ab_int_bug/117364044
> .
>
> If you're unable to attend the call then you can still help by picking
> up one of the issues above.
>
> Best regards,
>
> --
> Paul Barker
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#66163):
> https://lists.yoctoproject.org/g/yocto/message/66163
> Mute This Topic: https://lists.yoctoproject.org/mt/117367430/4316185
> Group Owner: yocto+owner@lists.yoctoproject.org
> Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [
> yoann.congal@smile.fr]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>

-- 
Yoann Congal
Smile ECS

[-- Attachment #2: Type: text/html, Size: 7164 bytes --]