Re: [OE-core] [PATCH] qemuarm64: maximise the emulated CPU capabilities

[Mikko Rapeli <mikko.rapeli@linaro.org>]

Hi,

On Wed, Feb 14, 2024 at 11:27:52AM +0000, Richard Purdie wrote:
> On Wed, 2024-02-14 at 11:06 +0000, Ross Burton wrote:
> > On 14 Feb 2024, at 11:04, Mikko Rapeli <mikko.rapeli@linaro.org> wrote:
> > > FWIW, this can also be risky and cause instabilities. We've seen boot hangs
> > > at least with old qemu 7.2 and kernel 4.xy where serial console detection
> > > did not correctly work on every boot.
> > > 
> > > If things work now, then great!
> > 
> > It worked on the few tests I did locally. I didn’t run it through the AB 
> > yet to stress it in any meaningful way yet.
> 
> I did put it through last night:
> 
> https://autobuilder.yoctoproject.org/typhoon/#/builders/83/builds/6566
> 
> and things are still running. The ptest issues so far look unrelated.

optee xtests are triggering a reproducible RCU stall in our testing with
--cpu max,pauth-impdef=on and qemu-system 8.2.1 from Debian unstable.
This does not happen with --cpu cortex-a57 on this qemu-machine version.

Example test run:

https://ledge.validation.linaro.org/scheduler/job/80815

+ xtest --clear-storage
[  285.552965] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
[  285.557211] rcu: 	2-...0: (2 GPs behind) idle=a1c4/1/0x4000000000000000 softirq=32180/32180 fqs=2004
[  285.559304] rcu: 	(detected by 0, t=5252 jiffies, g=75961, q=5749 ncpus=4)
[  348.573538] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
[  348.574771] rcu: 	2-...0: (2 GPs behind) idle=a1c4/1/0x4000000000000000 softirq=32180/32180 fqs=8797
[  348.576032] rcu: 	(detected by 0, t=21007 jiffies, g=75961, q=22677 ncpus=4)
[  411.593692] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
[  411.594425] rcu: 	2-...0: (2 GPs behind) idle=a1c4/1/0x4000000000000000 softirq=32180/32180 fqs=15573
[  411.595536] rcu: 	(detected by 0, t=36762 jiffies, g=75961, q=40851 ncpus=4)
[  474.614478] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
[  474.615567] rcu: 	2-...0: (2 GPs behind) idle=a1c4/1/0x4000000000000000 softirq=32180/32180 fqs=22375
[  474.616871] rcu: 	(detected by 0, t=52517 jiffies, g=75961, q=57789 ncpus=4)
[  537.635868] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
[  537.637129] rcu: 	2-...0: (2 GPs behind) idle=a1c4/1/0x4000000000000000 softirq=32180/32180 fqs=29112
[  537.638621] rcu: 	(detected by 0, t=68272 jiffies, g=75961, q=74773 ncpus=4)

The qemu system runs UEFI compatible firmware with u-boot, tf-a, optee etc configured for
secureboot and Linux kernel 6.5.10, initramfs and rootfs with systemd is slightly
behind poky master branch.

This was the kind of issues that we saw also one year ago with max CPU settings on qemu
so the root cause is still not identified or fixed.

Of course the problems could be in our specific configuration with optee and u-boot
based UEFI firmware also on qemu which doesn't need them for basic usage.

I have not yet tried to reproduce this with a local build using native qemu from yocto
build via runqemu/testimage.bbclass.

Cheers,

-Mikko