Re: Nested virtualization not working with hyperv guest/windows 11

[Sean Christopherson <seanjc@google.com>]

On Fri, Mar 24, 2023, Michał Zegan wrote:
> Hi,
> 
> I've sent this some time ago, but was not subscribed here, so unsure if I
> didn't get a reply or maybe missed it, so repeating:
> 
> I have a linux host with cpu intel core i7 12700h, kernel currently 6.2,
> fedora37.
> 
> I have a kvm/qemu/libvirt virtual machine, cpu model set to host, machine
> type q35, uefi with secureboot enabled, smm on.
> 
> The kvm_intel module has nested=y set in parameters so nested virtualization
> is enabled on host.
> 
> The virtual machine has windows11 pro guest installed.
> 
> When I install hyperv/virtualization platform/other similar functions, after
> reboot, the windows does not boot. Namely it reboots three times and then
> goes to recovery.

This is going to be nearly impossible to debug without more information.  Assuming
you can't extract more information from the guest, can you try enabling KVM
tracepoints?  E.g. to see if KVM is injecting an exception or a nested VM-Entry
failure that leads to the reboot.

I.e. enable tracing

    echo 1 > /sys/kernel/debug/tracing/tracing_on

and then to get the full blast from the trace firehose:

    echo 1 > /sys/kernel/debug/tracing/events/kvm/enable

or to get slightly less noisy log:

    echo 1 > /sys/kernel/debug/tracing/events/kvm/kvm_entry/enable
    echo 1 > /sys/kernel/debug/tracing/events/kvm/kvm_exit/enable
    echo 1 > /sys/kernel/debug/tracing/events/kvm/kvm_inj_exception/enable
    echo 1 > /sys/kernel/debug/tracing/events/kvm/kvm_nested_intercepts/enable
    echo 1 > /sys/kernel/debug/tracing/events/kvm/kvm_nested_intr_vmexit/enable
    echo 1 > /sys/kernel/debug/tracing/events/kvm/kvm_nested_vmenter_failed/enable
    echo 1 > /sys/kernel/debug/tracing/events/kvm/kvm_nested_vmexit/enable
    echo 1 > /sys/kernel/debug/tracing/events/kvm/kvm_nested_vmexit_inject/enable
    echo 1 > /sys/kernel/debug/tracing/events/kvm/kvm_nested_vmenter/enable

To capture something useful, you may need to (significantly) increase the size of
the buffer, 

    echo 131072 > /sys/kernel/debug/tracing/buffer_size_kb

The log itself can be found at

    /sys/kernel/debug/tracing/trace