snek testsuite breaks in 10.1 for qemu-system-arm on armhf

snek testsuite breaks in 10.1 for qemu-system-arm on armhf

[Christian Ehrhardt]

Hi,
Debian and Ubuntu run the self tests of snek as integration tests like:

`make SNEK_NATIVE=/usr/bin/snek SNEK_ARM=/usr/share/snek/snek-arm
SNEK_RISCV=/usr/share/snek/snek-riscv -C test check`

Those tests recently generated signal by failing in the arm emulation
tests with `qemu-system-arm` - full log [1].

```
224s Running test pass-precedence.py.
224s     pass python3
224s     pass snek
224s pass-precedence.py:72 Syntax error at "".
224s     ***************** snek-arm fail *********************
224s     pass snek-riscv
```

14 tests failed, but all with quite similar signatures.

But that only happens when executed on armhf, the other host
architectures are all happy [2].

I've separated the test and ran a git bisect on qemu 10.0 -> 10.1 as
somewhere here is the trigger.
That worked fine and identified this change [3].

I must admit, I was able to debug it until here, but I can't see how
these snek test failures could be caused by that change.
And yes - arm emulation on an armhf platform isn't the most common scenario.

I can't predict if there is anything wrong in snek which now is
treated differently by qemu to trigger this or if snek is all fine and
qemu broken something - Therefore I've also reported it to snek [4].

The tracking of the initial finding in Ubuntu is here if you want to
see more about how this commit was identified [5].

[1]: https://autopkgtest.ubuntu.com/results/autopkgtest-questing/questing/armhf/s/snek/20250818_045020_246a4@/log.gz
[2]: https://autopkgtest.ubuntu.com/packages/s/snek
[3]: https://salsa.debian.org/qemu-team/qemu/-/commit/cf4905c03135f1181e86c618426f8d6c703b38c0
[4]: https://github.com/keith-packard/snek/issues/103
[5]: https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/2121124

-- 
Christian Ehrhardt
Director of Engineering, Ubuntu Server
Canonical Ltd

Re: snek testsuite breaks in 10.1 for qemu-system-arm on armhf

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 2647 bytes --]

On Thu, Aug 21, 2025 at 02:52:46PM +0200, Christian Ehrhardt wrote:
> Hi,
> Debian and Ubuntu run the self tests of snek as integration tests like:
> 
> `make SNEK_NATIVE=/usr/bin/snek SNEK_ARM=/usr/share/snek/snek-arm
> SNEK_RISCV=/usr/share/snek/snek-riscv -C test check`
> 
> Those tests recently generated signal by failing in the arm emulation
> tests with `qemu-system-arm` - full log [1].
> 
> ```
> 224s Running test pass-precedence.py.
> 224s     pass python3
> 224s     pass snek
> 224s pass-precedence.py:72 Syntax error at "".
> 224s     ***************** snek-arm fail *********************
> 224s     pass snek-riscv
> ```
> 
> 14 tests failed, but all with quite similar signatures.
> 
> But that only happens when executed on armhf, the other host
> architectures are all happy [2].
> 
> I've separated the test and ran a git bisect on qemu 10.0 -> 10.1 as
> somewhere here is the trigger.
> That worked fine and identified this change [3].

CCing Richard Henderson since this was bisected to the following commit:

commit cf4905c03135f1181e86c618426f8d6c703b38c0
Author: Richard Henderson <richard.henderson@linaro.org>
Date:   Sun Jan 12 20:29:41 2025 -0800

    tcg: Convert deposit to TCGOutOpDeposit

> 
> I must admit, I was able to debug it until here, but I can't see how
> these snek test failures could be caused by that change.
> And yes - arm emulation on an armhf platform isn't the most common scenario.

QEMU v10.1.0-rc4 has been tagged and the final QEMU v10.1.0 release is
scheduled for Tuesday, August 26th. I think a fix for this issue can go
through the -stable tree and be released as part of QEMU v10.1.1 since
the number of users affected appears to be small. Please let me know if
you disagree.

Thanks,
Stefan

> 
> I can't predict if there is anything wrong in snek which now is
> treated differently by qemu to trigger this or if snek is all fine and
> qemu broken something - Therefore I've also reported it to snek [4].
> 
> The tracking of the initial finding in Ubuntu is here if you want to
> see more about how this commit was identified [5].
> 
> [1]: https://autopkgtest.ubuntu.com/results/autopkgtest-questing/questing/armhf/s/snek/20250818_045020_246a4@/log.gz
> [2]: https://autopkgtest.ubuntu.com/packages/s/snek
> [3]: https://salsa.debian.org/qemu-team/qemu/-/commit/cf4905c03135f1181e86c618426f8d6c703b38c0
> [4]: https://github.com/keith-packard/snek/issues/103
> [5]: https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/2121124
> 
> -- 
> Christian Ehrhardt
> Director of Engineering, Ubuntu Server
> Canonical Ltd
> 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: snek testsuite breaks in 10.1 for qemu-system-arm on armhf

[Alex Bennée]

Christian Ehrhardt <christian.ehrhardt@canonical.com> writes:

(adding qemu-arm and Richard to the CC)

> Hi,
> Debian and Ubuntu run the self tests of snek as integration tests like:
>
> `make SNEK_NATIVE=/usr/bin/snek SNEK_ARM=/usr/share/snek/snek-arm
> SNEK_RISCV=/usr/share/snek/snek-riscv -C test check`
>
> Those tests recently generated signal by failing in the arm emulation
> tests with `qemu-system-arm` - full log [1].
>
> ```
> 224s Running test pass-precedence.py.
> 224s     pass python3
> 224s     pass snek
> 224s pass-precedence.py:72 Syntax error at "".
> 224s     ***************** snek-arm fail *********************
> 224s     pass snek-riscv
> ```
>
> 14 tests failed, but all with quite similar signatures.
>
> But that only happens when executed on armhf, the other host
> architectures are all happy [2].
>
> I've separated the test and ran a git bisect on qemu 10.0 -> 10.1 as
> somewhere here is the trigger.
> That worked fine and identified this change [3].

This is:

  tcg: Convert deposit to TCGOutOpDeposit

  Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
  Signed-off-by: Richard Henderson <richard.henderson@linaro.org>

> I must admit, I was able to debug it until here, but I can't see how
> these snek test failures could be caused by that change.
> And yes - arm emulation on an armhf platform isn't the most common
> scenario.

Are any of your other hosts 32 bit?

>
> I can't predict if there is anything wrong in snek which now is
> treated differently by qemu to trigger this or if snek is all fine and
> qemu broken something - Therefore I've also reported it to snek [4].
>
> The tracking of the initial finding in Ubuntu is here if you want to
> see more about how this commit was identified [5].
>
> [1]: https://autopkgtest.ubuntu.com/results/autopkgtest-questing/questing/armhf/s/snek/20250818_045020_246a4@/log.gz
> [2]: https://autopkgtest.ubuntu.com/packages/s/snek
> [3]: https://salsa.debian.org/qemu-team/qemu/-/commit/cf4905c03135f1181e86c618426f8d6c703b38c0
> [4]: https://github.com/keith-packard/snek/issues/103
> [5]: https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/2121124

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro

Re: snek testsuite breaks in 10.1 for qemu-system-arm on armhf

[Christian Ehrhardt]

On Thu, Aug 21, 2025 at 4:16 PM Alex Bennée <alex.bennee@linaro.org> wrote:
>
> Christian Ehrhardt <christian.ehrhardt@canonical.com> writes:
>
> (adding qemu-arm and Richard to the CC)
>
> > Hi,
> > Debian and Ubuntu run the self tests of snek as integration tests like:
> >
> > `make SNEK_NATIVE=/usr/bin/snek SNEK_ARM=/usr/share/snek/snek-arm
> > SNEK_RISCV=/usr/share/snek/snek-riscv -C test check`
> >
> > Those tests recently generated signal by failing in the arm emulation
> > tests with `qemu-system-arm` - full log [1].
> >
> > ```
> > 224s Running test pass-precedence.py.
> > 224s     pass python3
> > 224s     pass snek
> > 224s pass-precedence.py:72 Syntax error at "".
> > 224s     ***************** snek-arm fail *********************
> > 224s     pass snek-riscv
> > ```
> >
> > 14 tests failed, but all with quite similar signatures.
> >
> > But that only happens when executed on armhf, the other host
> > architectures are all happy [2].
> >
> > I've separated the test and ran a git bisect on qemu 10.0 -> 10.1 as
> > somewhere here is the trigger.
> > That worked fine and identified this change [3].
>
> This is:
>
>   tcg: Convert deposit to TCGOutOpDeposit
>
>   Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
>   Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
>
> > I must admit, I was able to debug it until here, but I can't see how
> > these snek test failures could be caused by that change.
> > And yes - arm emulation on an armhf platform isn't the most common
> > scenario.
>
> Are any of your other hosts 32 bit?

No, only i386 but that does not have all dependencies and packages to
execute it.





> >
> > I can't predict if there is anything wrong in snek which now is
> > treated differently by qemu to trigger this or if snek is all fine and
> > qemu broken something - Therefore I've also reported it to snek [4].
> >
> > The tracking of the initial finding in Ubuntu is here if you want to
> > see more about how this commit was identified [5].
> >
> > [1]: https://autopkgtest.ubuntu.com/results/autopkgtest-questing/questing/armhf/s/snek/20250818_045020_246a4@/log.gz
> > [2]: https://autopkgtest.ubuntu.com/packages/s/snek
> > [3]: https://salsa.debian.org/qemu-team/qemu/-/commit/cf4905c03135f1181e86c618426f8d6c703b38c0
> > [4]: https://github.com/keith-packard/snek/issues/103
> > [5]: https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/2121124
>
> --
> Alex Bennée
> Virtualisation Tech Lead @ Linaro



-- 
Christian Ehrhardt
Director of Engineering, Ubuntu Server
Canonical Ltd