Re: [OE-core] [PATCH] oeqa/reproducibility: Work around subunit protocol limitations/bugs

Re: [OE-core] [PATCH] oeqa/reproducibility: Work around subunit protocol limitations/bugs

[Richard Purdie]

On Thu, 2026-04-09 at 21:35 +0100, Richard Purdie via lists.openembedded.org wrote:
> We've seen reproducibility failures with:
> 
>   File "/usr/lib/python3/dist-packages/subunit/chunked.py", line 108, in _read_length
>     raise ValueError("chunk header invalid: %r" % count_str)
> ValueError: chunk header invalid: b'\n'
> 
> which leads to:
> 
> 2026-04-09 01:18:20,886 - oe-selftest - INFO - ERROR: broken-runner
> 
> and UNKNOWN status for builds queued in that runner.
> 
> This appears to be when reproducibility builds fail with:
> 
> Traceback (most recent call last):
>   File "/srv/pokybuild/yocto-worker/bringup-fast/build/layers/openembedded-core/meta/lib/oeqa/selftest/cases/reproducible.py", line 407, in test_reproducible_builds
>     self.fail('\n'.join(fails))
>     ~~~~~~~~~^^^^^^^^^^^^^^^^^^
>   File "/usr/lib/python3.14/unittest/case.py", line 750, in fail
>     raise self.failureException(msg)
> 
> which is possibly from bad placement of '\n' in the byte stream. It doesn't
> 100% reproduce since the placement is critical but from the subunit code, it
> does use <int>\n as a header in the protocol.
> 
> I've wondered if the presence of "\n" in the traceback code is part of the issue
> so remove "\n" from the traceback by writing the python code differently just in
> case it helps.
> 
> Ideally we'd report the issue and fix subunit but I was unable to work out the
> exact sequence to reproduce and any fix would take time to propogate anyway.

There are other better ideas about what is going on here to cause this
and this patch won't help. The issue is thankfully rare so we can drop
this for now until we come up with something better. We can open a bug
to track the issue.

Cheers,

Richard

Re: [OE-core] [PATCH] oeqa/reproducibility: Work around subunit protocol limitations/bugs

[Paul Barker]

[-- Attachment #1: Type: text/plain, Size: 2085 bytes --]

On Thu, 2026-04-09 at 22:54 +0100, Richard Purdie via
lists.openembedded.org wrote:
> On Thu, 2026-04-09 at 21:35 +0100, Richard Purdie via lists.openembedded.org wrote:
> > We've seen reproducibility failures with:
> > 
> >   File "/usr/lib/python3/dist-packages/subunit/chunked.py", line 108, in _read_length
> >     raise ValueError("chunk header invalid: %r" % count_str)
> > ValueError: chunk header invalid: b'\n'
> > 
> > which leads to:
> > 
> > 2026-04-09 01:18:20,886 - oe-selftest - INFO - ERROR: broken-runner
> > 
> > and UNKNOWN status for builds queued in that runner.
> > 
> > This appears to be when reproducibility builds fail with:
> > 
> > Traceback (most recent call last):
> >   File "/srv/pokybuild/yocto-worker/bringup-fast/build/layers/openembedded-core/meta/lib/oeqa/selftest/cases/reproducible.py", line 407, in test_reproducible_builds
> >     self.fail('\n'.join(fails))
> >     ~~~~~~~~~^^^^^^^^^^^^^^^^^^
> >   File "/usr/lib/python3.14/unittest/case.py", line 750, in fail
> >     raise self.failureException(msg)
> > 
> > which is possibly from bad placement of '\n' in the byte stream. It doesn't
> > 100% reproduce since the placement is critical but from the subunit code, it
> > does use <int>\n as a header in the protocol.
> > 
> > I've wondered if the presence of "\n" in the traceback code is part of the issue
> > so remove "\n" from the traceback by writing the python code differently just in
> > case it helps.
> > 
> > Ideally we'd report the issue and fix subunit but I was unable to work out the
> > exact sequence to reproduce and any fix would take time to propogate anyway.
> 
> There are other better ideas about what is going on here to cause this
> and this patch won't help. The issue is thankfully rare so we can drop
> this for now until we come up with something better. We can open a bug
> to track the issue.

For anyone following along, I've filed an issue in bugzilla:
  https://bugzilla.yoctoproject.org/show_bug.cgi?id=16242

Best regards,

-- 
Paul Barker


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 252 bytes --]