SVSM: Updated Linux and QEMU branches available for testing

SVSM: Updated Linux and QEMU branches available for testing

[Jörg Rödel]

Hi,

I just pushed out new branches for COCONUT-SVSMs downstream Linux and QEMU
changes. The new branches need to be used together, as there have been rebases
which changed parts of the user-space interface. For Linux the branch is:

	https://github.com/coconut-svsm/linux/tree/svsm-v7.2

For QEMU:

	https://github.com/coconut-svsm/qemu/tree/svsm-v7.2

These new branches bring a set of changes and new features:

	- KVM planes patches rebased to v7.2

	- QEMU patches rewritten on v11.1.0

	- Support for KVM planes with in-kernel IOAPIC/PIC/PIT/RTC

	- Support for PCI device assignment with KVM planes

	- Use of version 2 of the direct-VMSA patch-set

The last point deserves closer attention, as it will lead to a breaking change
going forward. The new version of the direct-VMSA patch-set requires a valid
VMSA GPA to be passed to KVM. The currently used GPA (0xFFFF_FFFF_F000) is NOT
a valid GPA.

For now QEMU will fall back to legacy behavior when it finds an invalid GPA in
the IGVM file, so everything keeps booting as ususal, except that the launch
measurement will not match anymore.

This can be easily fixed in the SVSM and I have a draft-PR open for that:

	https://github.com/coconut-svsm/svsm/pull/1209

This PR moves the VMSA page into guest memory for QEMU, just like it is on
Hyper-V. The problem with that PR is that is breaks all existing Linux/QEMU
combinations, as they all have strict checks for the VMSA GPA to be a
0xFFFF_FFFF_F000.

My plan forward here is to switch to the new 7.2-based branches after everyone
had enough time to test them. After switching there will be even more time for
everyone else to switch over to the new downstream code. And only after that we
will merge PR #1209.

This is not an ideal situation and I'd like to have avoided more breaking
changes. But in this case I believe it is justified as avoiding this needs ugly
changes in the KVM direct-vmsa patch-set.

With that in mind I wish everyone happy testing. Please report any issues you
might find so that they can be fixed.

Regards,

	Joerg

Re: SVSM: Updated Linux and QEMU branches available for testing

[Liam Merwick]

On 27/08/2026 17:17, Jörg Rödel wrote:
> Hi, I just pushed out new branches for COCONUT-SVSMs downstream Linux 
> and QEMU changes. The new branches need to be used together, as there 
> have been rebases which changed parts of the user-space interface. For 
> Linux the branch is: https: //github. com/coconut-svsm/linux/tree/svsm-v7. 2
> 


On a system that was running the 7.1 branches, I've upgraded the host
kernel and qemu to the 7.2 branches and am now getting CPU stalls in
the guest (and eventual boot hang).

I'm using ./scripts/launch_guest.sh (and 2026.08-devel)

One thing I've noticed with kvm_stat is that the top host functions are

  Event                                         Total %Total CurAvg/s
  kvm_set_irq                                 2304824   22.2     2095
  kvm_ioapic_set_irq                          2304824   22.2     2095
  kvm_pic_set_irq                             2304768   22.2     2095
  kvm_exit                                     626814    6.0      159
  kvm_entry                                    626814    6.0      159
  kvm_vmgexit_exit                             299778    2.9      105
  kvm_vmgexit_enter                            299777    2.9      105
  (...)

Are there any new kernel boot args needed for host or guest kernels?
I've confirmed that regular SEV-SNP guest (VMPL0) still launches OK
with the same components. I've also tried dropping the last

Regards,
Liam

FWIW, sample guest stall stack.

[ 1230.794758] INFO: task kworker/u16:0:12 blocked for more than 122 
seconds.
[ 1230.796595]       Not tainted 7.2.0-0.svsm.6.el9uek.rc.x86_64 #1
[ 1230.798135] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" 
disables this message.
[ 1230.800083] task:kworker/u16:0   state:D stack:0     pid:12 
tgid:12    ppid:2      task_flags:0x4208060 flags:0x00080000
[ 1230.802818] Workqueue: async async_run_entry_fn
[ 1230.803993] Call Trace:
[ 1230.804646]  <TASK>
[ 1230.805250]  __schedule+0x2b1/0x6d0
[ 1230.806178]  schedule+0x27/0xa0
[ 1230.807014]  async_synchronize_cookie_domain+0x125/0x16c
[ 1230.808354]  ? __pfx_autoremove_wake_function+0x10/0x10
[ 1230.809708]  async_port_probe+0x3d/0x60 [libata]
[ 1230.810923]  async_run_entry_fn+0x34/0x140
[ 1230.812009]  process_one_work+0x196/0x35e
[ 1230.813054]  worker_thread+0x1af/0x330
[ 1230.814039]  ? __pfx_worker_thread+0x10/0x10
[ 1230.815144]  kthread+0xe3/0x120
[ 1230.815982]  ? __pfx_kthread+0x10/0x10
[ 1230.816956]  ret_from_fork+0x1ab/0x270
[ 1230.817961]  ? __pfx_kthread+0x10/0x10
[ 1230.818949]  ret_from_fork_asm+0x1a/0x30
[ 1230.819973]  </TASK>



> Hi,
> 
> I just pushed out new branches for COCONUT-SVSMs downstream Linux and QEMU
> changes. The new branches need to be used together, as there have been rebases
> which changed parts of the user-space interface. For Linux the branch is:
> 
> 	https://github.com/coconut-svsm/linux/tree/svsm-v7.2
> 
> For QEMU:
> 
> 	https://github.com/coconut-svsm/qemu/tree/svsm-v7.2
> 
> These new branches bring a set of changes and new features:
> 
> 	- KVM planes patches rebased to v7.2
> 
> 	- QEMU patches rewritten on v11.1.0
> 
> 	- Support for KVM planes with in-kernel IOAPIC/PIC/PIT/RTC
> 
> 	- Support for PCI device assignment with KVM planes
> 
> 	- Use of version 2 of the direct-VMSA patch-set
> 
> The last point deserves closer attention, as it will lead to a breaking change
> going forward. The new version of the direct-VMSA patch-set requires a valid
> VMSA GPA to be passed to KVM. The currently used GPA (0xFFFF_FFFF_F000) is NOT
> a valid GPA.
> 
> For now QEMU will fall back to legacy behavior when it finds an invalid GPA in
> the IGVM file, so everything keeps booting as ususal, except that the launch
> measurement will not match anymore.
> 
> This can be easily fixed in the SVSM and I have a draft-PR open for that:
> 
> 	https://github.com/coconut-svsm/svsm/pull/1209
> 
> This PR moves the VMSA page into guest memory for QEMU, just like it is on
> Hyper-V. The problem with that PR is that is breaks all existing Linux/QEMU
> combinations, as they all have strict checks for the VMSA GPA to be a
> 0xFFFF_FFFF_F000.
> 
> My plan forward here is to switch to the new 7.2-based branches after everyone
> had enough time to test them. After switching there will be even more time for
> everyone else to switch over to the new downstream code. And only after that we
> will merge PR #1209.
> 
> This is not an ideal situation and I'd like to have avoided more breaking
> changes. But in this case I believe it is justified as avoiding this needs ugly
> changes in the KVM direct-vmsa patch-set.
> 
> With that in mind I wish everyone happy testing. Please report any issues you
> might find so that they can be fixed.
> 
> Regards,
> 
> 	Joerg
> 

Re: SVSM: Updated Linux and QEMU branches available for testing

[Jörg Rödel]

Hi Liam,

On Mon, Aug 31, 2026 at 09:41:13AM +0100, Liam Merwick wrote:
> 
> On 27/08/2026 17:17, Jörg Rödel wrote:
> > Hi, I just pushed out new branches for COCONUT-SVSMs downstream Linux
> > and QEMU changes. The new branches need to be used together, as there
> > have been rebases which changed parts of the user-space interface. For
> > Linux the branch is:
> > https: //github. com/coconut-svsm/linux/tree/svsm-v7. 2
> > 
> 
> 
> On a system that was running the 7.1 branches, I've upgraded the host
> kernel and qemu to the 7.2 branches and am now getting CPU stalls in
> the guest (and eventual boot hang).
> 
> I'm using ./scripts/launch_guest.sh (and 2026.08-devel)
> 
> One thing I've noticed with kvm_stat is that the top host functions are
> 
>  Event                                         Total %Total CurAvg/s
>  kvm_set_irq                                 2304824   22.2     2095
>  kvm_ioapic_set_irq                          2304824   22.2     2095
>  kvm_pic_set_irq                             2304768   22.2     2095
>  kvm_exit                                     626814    6.0      159
>  kvm_entry                                    626814    6.0      159
>  kvm_vmgexit_exit                             299778    2.9      105
>  kvm_vmgexit_enter                            299777    2.9      105
>  (...)

Thanks for testing the new branches. Can you please share your exact
launch_guest.sh command line? Also, what type of guest image are you running,
something I can download as well?

-Joerg

Re: SVSM: Updated Linux and QEMU branches available for testing

[Liam Merwick]

Hi Jörg,

On 31/08/2026 16:24, Jörg Rödel wrote:
> Hi Liam, On Mon, Aug 31, 2026 at 09: 41: 13AM +0100, Liam Merwick wrote: 
>  > > On 27/08/2026 17: 17, Jörg Rödel wrote: > > Hi, I just pushed out 
> new branches for COCONUT-SVSMs downstream Linux > > and QEMU changes. 
> The new branches
> 
> Hi Liam,
> 
> On Mon, Aug 31, 2026 at 09:41:13AM +0100, Liam Merwick wrote:
>> 
>> On 27/08/2026 17:17, Jörg Rödel wrote:
>> > Hi, I just pushed out new branches for COCONUT-SVSMs downstream Linux
>> > and QEMU changes. The new branches need to be used together, as there
>> > have been rebases which changed parts of the user-space interface. For
>> > Linux the branch is:
>> > https: //github. com/coconut-svsm/linux/tree/svsm-v7. 2
>> > 
>> 
>> 
>> On a system that was running the 7.1 branches, I've upgraded the host
>> kernel and qemu to the 7.2 branches and am now getting CPU stalls in
>> the guest (and eventual boot hang).
>> 
>> I'm using ./scripts/launch_guest.sh (and 2026.08-devel)
>> 
>> One thing I've noticed with kvm_stat is that the top host functions are
>> 
>>  Event                                         Total %Total CurAvg/s
>>  kvm_set_irq                                 2304824   22.2     2095
>>  kvm_ioapic_set_irq                          2304824   22.2     2095
>>  kvm_pic_set_irq                             2304768   22.2     2095
>>  kvm_exit                                     626814    6.0      159
>>  kvm_entry                                    626814    6.0      159
>>  kvm_vmgexit_exit                             299778    2.9      105
>>  kvm_vmgexit_enter                            299777    2.9      105
>>  (...)
> 
> Thanks for testing the new branches. Can you please share your exact
> launch_guest.sh command line? Also, what type of guest image are you running,
> something I can download as well?
> 
In a svsm repo, at tag v2026.08-devel where I've built svsm with
FW_FILE pointing to the svsm edk2 repo at tag
db97fa0a4b29 ("OvmfPkg/PlatformPei: Initialize 
PcdConfidentialComputingGuestAttr early")
[ I tried both DEBUG and RELEASE builds ] and qemu is built from
the svsm-v7.2 branch.


./scripts/launch_guest.sh --qemu 
/root/src/coconut/qemu/build/qemu-system-x86_64 -i bin/coconut-qemu.igvm 
--image ./resolute-server-cloudimg-amd64.img

Initially I was using OL9 in the guest but I reproduced it with an 
unmodified Ubuntu 26.04 [1]

Regards,
Liam

[1] 
https://cloud-images.ubuntu.com/resolute/current/resolute-server-cloudimg-amd64.img

Re: SVSM: Updated Linux and QEMU branches available for testing

[Jörg Rödel]

Hi Liam,

On Tue, Sep 01, 2026 at 01:58:34AM +0100, Liam Merwick wrote:
> ./scripts/launch_guest.sh --qemu
> /root/src/coconut/qemu/build/qemu-system-x86_64 -i bin/coconut-qemu.igvm
> --image ./resolute-server-cloudimg-amd64.img

I found the issue, it is in the interaction between QEMU and the
launch_guest.sh script. The script does no longer detect that the device-plane=
property is available and does not add it to the command line. This explains
why IOAPIC IRQs (and also other IRQs) do not get through to Linux anymore.

I will prepare and push a fix.

-Joerg

Re: SVSM: Updated Linux and QEMU branches available for testing

[Jörg Rödel]

On Tue, Sep 01, 2026 at 01:58:34AM +0100, Liam Merwick wrote:
> ./scripts/launch_guest.sh --qemu
> /root/src/coconut/qemu/build/qemu-system-x86_64 -i bin/coconut-qemu.igvm
> --image ./resolute-server-cloudimg-amd64.img

I updated the QEMU branch with a fix. The launch_guest.sh script can now detect
that the device-plane property is available and adds it to the command line.
Can you please test the fix? It fixed the issue on my side.

-Joerg

Re: SVSM: Updated Linux and QEMU branches available for testing

[Liam Merwick]

On 01/09/2026 01:58, Liam Merwick wrote:
> Hi Jörg, On 31/08/2026 16: 24, Jörg Rödel wrote: > Hi Liam, On Mon, Aug 
> 31, 2026 at 09: 41: 13AM +0100, Liam Merwick wrote: > > > On 27/08/2026 
> 17: 17, Jörg Rödel wrote: > > Hi, I just pushed out > new branches for 
> COCONUT-SVSMs
> 
> Hi Jörg,
> 
> On 31/08/2026 16:24, Jörg Rödel wrote:
>> Hi Liam, On Mon, Aug 31, 2026 at 09: 41: 13AM +0100, Liam Merwick wrote: 
>>  > > On 27/08/2026 17: 17, Jörg Rödel wrote: > > Hi, I just pushed out 
>> new branches for COCONUT-SVSMs downstream Linux > > and QEMU changes. 
>> The new branches
>> 
>> Hi Liam,
>> 
>> On Mon, Aug 31, 2026 at 09:41:13AM +0100, Liam Merwick wrote:
>>> 
>>> On 27/08/2026 17:17, Jörg Rödel wrote:
>>> > Hi, I just pushed out new branches for COCONUT-SVSMs downstream Linux
>>> > and QEMU changes. The new branches need to be used together, as there
>>> > have been rebases which changed parts of the user-space interface. For
>>> > Linux the branch is:
>>> > https: //github. com/coconut-svsm/linux/tree/svsm-v7. 2
>>> > 
>>> 
>>> 
>>> On a system that was running the 7.1 branches, I've upgraded the host
>>> kernel and qemu to the 7.2 branches and am now getting CPU stalls in
>>> the guest (and eventual boot hang).
>>> 
>>> I'm using ./scripts/launch_guest.sh (and 2026.08-devel)
>>> 
>>> One thing I've noticed with kvm_stat is that the top host functions are
>>> 
>>>  Event                                         Total %Total CurAvg/s
>>>  kvm_set_irq                                 2304824   22.2     2095
>>>  kvm_ioapic_set_irq                          2304824   22.2     2095
>>>  kvm_pic_set_irq                             2304768   22.2     2095
>>>  kvm_exit                                     626814    6.0      159
>>>  kvm_entry                                    626814    6.0      159
>>>  kvm_vmgexit_exit                             299778    2.9      105
>>>  kvm_vmgexit_enter                            299777    2.9      105
>>>  (...)
>> 
>> Thanks for testing the new branches. Can you please share your exact
>> launch_guest.sh command line? Also, what type of guest image are you running,
>> something I can download as well?

Another data point. The machine I encountered this issue is a Genoa
machine. I did a fresh install on a Turin and another Genoa and those
do not reproduce the issue - the latest SVSM branches work on those.

On the problematic Genoa, switching back to the 7.1 based kernel/qemu
combo works. This machine has been running SNP/SVSM for a while so not
a fully clean config - it could have accumulated some setting that is
incompatible but I don't see anything yet in /proc/cmdline, etc.

Regards,
Liam


>> 
> In a svsm repo, at tag v2026.08-devel where I've built svsm with
> FW_FILE pointing to the svsm edk2 repo at tag
> db97fa0a4b29 ("OvmfPkg/PlatformPei: Initialize
> PcdConfidentialComputingGuestAttr early")
> [ I tried both DEBUG and RELEASE builds ] and qemu is built from
> the svsm-v7.2 branch.
> 
> 
> ./scripts/launch_guest.sh --qemu
> /root/src/coconut/qemu/build/qemu-system-x86_64 -i bin/coconut-qemu.igvm
> --image ./resolute-server-cloudimg-amd64.img
> 
> Initially I was using OL9 in the guest but I reproduced it with an
> unmodified Ubuntu 26.04 [1]
> 
> Regards,
> Liam
> 
> [1]
> https://cloud-images.ubuntu.com/resolute/current/resolute-server-cloudimg-amd64.img
> 
> 

Re: SVSM: Updated Linux and QEMU branches available for testing

[Jörg Rödel]

Hi Liam,

On Wed, Sep 02, 2026 at 12:00:54AM +0100, Liam Merwick wrote:
> Another data point. The machine I encountered this issue is a Genoa
> machine. I did a fresh install on a Turin and another Genoa and those
> do not reproduce the issue - the latest SVSM branches work on those.

I pushed a fix to the svsm-v7.2 branch of

	https://github.com/coconut-svsm/qemu

Did you test with this latest version of QEMU on the failing Genoa machine? For
reference, the HEAD points at commit 5c1c05b2c3aa0dd1248e0cf58e401626e30887f8.

-Joerg

Re: SVSM: Updated Linux and QEMU branches available for testing

[Liam Merwick]

On 02/09/2026 10:18, Jörg Rödel wrote:
> Hi Liam, On Wed, Sep 02, 2026 at 12: 00: 54AM +0100, Liam Merwick wrote: 
>  > Another data point. The machine I encountered this issue is a Genoa > 
> machine. I did a fresh install on a Turin and another Genoa and those > 
> do not reproduce
> 
> Hi Liam,
> 
> On Wed, Sep 02, 2026 at 12:00:54AM +0100, Liam Merwick wrote:
>> Another data point. The machine I encountered this issue is a Genoa
>> machine. I did a fresh install on a Turin and another Genoa and those
>> do not reproduce the issue - the latest SVSM branches work on those.
> 
> I pushed a fix to the svsm-v7.2 branch of
> 
> 	https://github.com/coconut-svsm/qemu
> 
> Did you test with this latest version of QEMU on the failing Genoa machine? For
> reference, the HEAD points at commit 5c1c05b2c3aa0dd1248e0cf58e401626e30887f8.
>

Yes, thanks. With this QEMU fix everything works correctly.

Regards,
Liam