[Bug 216045] New: KVM x2APIC virtualization causes EOI to be ignored

[Bug 216045] New: KVM x2APIC virtualization causes EOI to be ignored

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=216045

            Bug ID: 216045
           Summary: KVM x2APIC virtualization causes EOI to be ignored
           Product: Virtualization
           Version: unspecified
    Kernel Version: 5.17.9
          Hardware: Intel
                OS: Linux
              Tree: Mainline
            Status: NEW
          Severity: normal
          Priority: P1
         Component: kvm
          Assignee: virtualization_kvm@kernel-bugs.osdl.org
          Reporter: ercli@ucdavis.edu
        Regression: No

Created attachment 301071
  --> https://bugzilla.kernel.org/attachment.cgi?id=301071&action=edit
Guest hypervisor to reproduce this bug (xz compressed)

CPU model: 11th Gen Intel(R) Core(TM) i7-1185G7 @ 3.00GHz
Host kernel version: 5.17.9
Host kernel arch: x86_64
Guest: a micro-hypervisor (called XMHF), which runs Linux (64-bit Debian,
5.10.0-9-amd64)
QEMU command line: qemu-system-x86_64 -m 512M -smp 2 -cpu Haswell,vmx=yes
-enable-kvm -serial stdio -drive media=disk,file=c.img,index=1 -drive
media=disk,file=debian11x64.qcow2,index=2
The problem goes away if using -machine kernel_irqchip=off
This problem cannot be tested with -accel tcg , because the guest requires
nested virtualization

Files used to reproduce this bug:

debian11x64.qcow2 (3.3G) is uploaded at
https://drive.google.com/uc?id=1LtUwnzH8pDvjoBhJhxk5wIz9_UTlGaBA . It should be
equivalent to a freshly installed Debian 5.10.0-9-amd64 on QEMU. Its kaslr is
disabled for debugging convenience.

c.img (8M) is uploaded at
https://drive.google.com/uc?id=1g3c9KMAoh_Yvb9bzSuOBMG5L-2VX6twU . It is also
compressed as c.img.xz and uploaded with this bug. It is built from
https://github.com/lxylxy123456/uberxmhf/tree/ab7968ed8017f43978081862526636f75c80a3b7
.

Actual behavior:

After running the QEMU command above, the hypervisor (XMHF) will boot and then
chain load Debian as a guest OS. Debian starts to boot and stuck. In QEMU's
monitor, if I enter "info lapic", I see

...
ISR      253
IRR      48 236 253

Expected behavior:

When running the QEMU command above, the hypervisor should boot Debian, and
Debian should boot successfully. The VGA screen should show "debian login:".

The expected behavior can be achieved in a few ways:
1. Remove "-drive media=disk,file=c.img,index=1" from QEMU command line. This
will boot Debian directly (instead of booting XMHF and then boot Debian)
2. Add "-machine kernel_irqchip=off" to the QEMU command line
3. Apply the following patch to KVM and re-compile. This patch will let KVM
think that x2APIC virtualization is not available.

diff --git a/arch/x86/kvm/vmx/capabilities.h b/arch/x86/kvm/vmx/capabilities.h
index 3f430e218..29b412ae4 100644
--- a/arch/x86/kvm/vmx/capabilities.h
+++ b/arch/x86/kvm/vmx/capabilities.h
@@ -157,8 +157,7 @@ static inline bool cpu_has_vmx_rdtscp(void)

 static inline bool cpu_has_vmx_virtualize_x2apic_mode(void)
 {
-       return vmcs_config.cpu_based_2nd_exec_ctrl &
-               SECONDARY_EXEC_VIRTUALIZE_X2APIC_MODE;
+       return 0;
 }

 static inline bool cpu_has_vmx_vpid(void)

-- 
You may reply to this email to add a comment.

You are receiving this mail because:
You are watching the assignee of the bug.

[Bug 216045] KVM x2APIC virtualization causes EOI to be ignored

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=216045

--- Comment #1 from Eric Li (ercli@ucdavis.edu) ---
Explanation of guest behavior

1. BIOS boots XMHF
2. XMHF starts GRUB as a nested guest
3. GRUB by default boots Debian (5 seconds timeout)
4. Debian executes 0xffffffff81062179 (can set a hardware breakpoint using
GDB). At this instruction is a WRMSR to 0x80b (IA32_X2APIC_EOI)
5. The WRMSR instruction above will VMEXIT to hypervisor mode (XMHF). The
VMEXIT entry point is 0x10206411 (xmhf_parteventhub_arch_x86vmx_entry)
6. At 0x10206434, the hypervisor calls 0x10207eb9
xmhf_parteventhub_arch_x86vmx_intercept_handler()
7. At 0x10208390, the hypervisor calls 0x10206d9f _vmx_handle_intercept_wrmsr()
8. At 0x102070c6, the hypervisor performs WRMSR to 0x80b. The intention is to
forward EOI from the guest (Debian) to KVM. Before WRMSR, "info lapic" in QEMU
gives

...
ISR      48
IRR      48

9. After WRMSR, RIP lands at 0x102070c8. "info lapic" in QEMU gives

...
ISR      48
IRR      48

However, I am expecting it to give

...
ISR      (none)
IRR      48

10. After a lot of instructions, the nested Debian guest halts. "info lapic" is

...
ISR      236
IRR      48 236 253

-- 
You may reply to this email to add a comment.

You are receiving this mail because:
You are watching the assignee of the bug.