Re: KVM: SVM: 32-bit PAE Windows 2000 guest crashes with AVIC on Zen 2 (EPYC 7402), reproducible in seconds

[Sean Christopherson <seanjc@google.com>]

On Tue, Oct 27, 2026, Sam H wrote:
> Hey KVM Team,
> 
> I have a 32-bit Windows 2000 guest running the PAE kernel

On purpose?  :-D

> that bugchecks within 15 to 134 seconds under a process churn load on an AMD
> Zen 2 host with AVIC enabled. The same guest survives the same load with AVIC
> off, or with PAE off. I have a disk clone that reproduces it on demand and
> can test patches against it.
> 
> Host
> - 2 x AMD EPYC 7402 (family 17h), microcode 0x830107d

Do you have access to Zen4+ hardware?  If this only reproduces on Rome (Zen2),
then unfortunately the answer is going to be "don't run AVIC on Rome".

> - Proxmox VE 9.2.21, kernel 7.0.14-25-pve, QEMU 11.0.3 (pve-qemu-kvm 11.0.3-4)
> - kvm_amd: avic=Y, enable_ipiv=N, npt=Y
> - Also crashes on 6.14.8-2-pve (no enable_ipiv parameter on that kernel)
> 
> Guest
> - Windows 2000 Datacenter Server SP4, boot.ini /PAE /3GB (ntkrpamp)
> - Crashes on both the SP4 kernel (5.00.2195.6717) and the Update
> Rollup 1 kernel (5.00.2195.7045)
> - -cpu kvm64, machine pc-i440fx-5.1, 2 sockets x 4 cores (8 vCPU),
> 8192 MiB, numa on
> - Extra args: -machine hpet=off -rtc driftfix=slew -global
> kvm-pit.lost_tick_policy=discard (the PIT policy is what keeps AVIC
> active for this guest)
> 
> Crash
> - STOP 0x0000000A IRQL_NOT_LESS_OR_EQUAL, IRQL 2, always in ntkrnlpa.exe.
> - SP4 kernel: 0x0A (0x00000154, 2, 0, 0xDD537EAF), ntkrnlpa+0x137EAF,
> mov ecx,[ebx+0x154] with ebx = 0 or 0xF000FF53 (the real mode IVT
> pattern, i.e. the contents of guest physical page 0).
> - UR1 kernel: 0x0A (0x00000098, 2, 0, 0xDD53C9E6), ntkrnlpa+0x13C9E6,
> read of [esi+0x98] with esi = 0.
> - One write variant was also seen: 0x0A (0x00000098, 2, 1, 0xDD53C84C).
> - The faulting code is the PAE memory manager's hyperspace page table
> walk: it maps a page table page into a per-CPU hyperspace slot by
> writing a PTE, then reads through the slot, and the read comes back
> zero.
> - When it crashes, 7 of the 8 vCPUs are stopped at the same EIP
> (0xDD4335C3) and the crashing vCPU is at 0xDD06A1A6.
> 
> Reproducer
> 6 concurrent sessions, each looping 200 short cmd.exe processes plus
> one debug.exe, so heavy process create and exit.
> 
> Results on the clone (stress started after boot, same disk each time;
> 8 vCPU 2x4, kvm64, numa on, 8192 MiB unless stated)
> - AVIC on, /PAE /3GB (baseline): crashed at 134 s
> - AVIC on, /3GB only, no /PAE: survived 300 s
> - AVIC on, /PAE /3GB, 3584 MiB (no RAM above 4 GB): crashed at 26 s
> - AVIC on, /PAE without /3GB: crashed at 30 s
> - AVIC on, /PAE /3GB, QEMU vapic=off: crashed at 26 s
> - AVIC off (PIT reinject), /PAE /3GB: survived 600 s, twice
> - AVIC on, /PAE /3GB, repeat with tracing: crashed at 15 s
> - AVIC on, /PAE /3GB, 1 socket x 8 cores: crashed at 50 s
> - AVIC on, no /PAE, 30 minute soak: survived 1800 s
> 
> The production copy of the guest also crashed 3 times in one day on a
> single vCPU with AVIC on (7.0.14-25, enable_ipiv=N), so one vCPU
> lowers the rate but does not remove it. Setting the CPU vendor to
> GenuineIntel made no difference. During a traced run the AVIC IRQ
> window toggled about 1 to 2 times a second, so I do not think it is
> the IRQ window inhibit issue.
> 
> The same guest image ran on an Intel Xeon E5-2620 v4 host (Proxmox VE
> 5.4, kernel 4.15) for years and has no crash dumps on its disk from
> that time.
> 
> What I have not tested: npt=0, because AVIC needs NPT, so npt=0 would
> also turn AVIC off and could not tell the two apart from the AVIC off
> run above. It would also be a host wide change on a host running ~70
> other production workloads/guests, and we can't risk performance
> impacts. Also kernels other than 6.14 and 7.0.
> 
> A possibly relevant difference: AMD APM Vol 2 section 15.25.10 says
> that under nested paging the guest's PAE PDPTEs are not loaded at MOV
> CR3 the way they are natively or under Intel EPT. I have not proven
> that is the mechanism, and the AVIC off result says interrupt delivery
> or timing plays a part as well.
> 
> I can provide the two minidumps, register captures of all vCPUs at the
> moment of a crash, and KVM traces, and I can run any patch against the
> clone.
> 
> Thanks,
> Sam Howell
>