Re: [PATCH] target/i386: KVM: add hack for Windows vCPU hotplug with SGX

[Sean Christopherson <seanjc@google.com>]

On Mon, Jun 09, 2025, Denis V. Lunev wrote:
> On 6/9/25 18:12, Paolo Bonzini wrote:
> > On 6/9/25 15:23, Andrey Zhadchenko wrote:
> > > When hotplugging vCPUs to the Windows vms, we observed strange instance
> > > crash on Intel(R) Xeon(R) CPU E3-1230 v6:
> > > panic hyper-v: arg1='0x3e', arg2='0x46d359bbdff',
> > > arg3='0x56d359bbdff', arg4='0x0', arg5='0x0'
> > > 
> > > Presumably, Windows thinks that hotplugged CPU is not "equivalent
> > > enough"
> > > to the previous ones. The problem lies within msr 3a. During the
> > > startup,
> > > Windows assigns some value to this register. During the hotplug it
> > > expects similar value on the new vCPU in msr 3a. But by default it
> > > is zero.
> > 
> > If I understand correctly, you checked that it's Windows that writes
> > 0x40005 to the MSR on non-hotplugged CPUs.

...

> > > Bit #18 probably means that Intel SGX is supported, because disabling
> > > it via CPU arguments results is successfull hotplug (and msr value 0x5).
> > 
> > What is the trace like in this case?  Does Windows "accept" 0x0 and
> > write 0x5?
> > 
> > Does anything in edk2 run during the hotplug process (on real hardware
> > it does, because the whole hotplug is managed via SMM)? If so maybe that
> > could be a better place to write the value.

Yeah, I would expect firmware to write and lock IA32_FEATURE_CONTROL.

> > So many questions, but I'd really prefer to avoid this hack if the only
> > reason for it is SGX...

Does your setup actually support SGX?  I.e. expose EPC sections to the guest?
If not, can't you simply disable SGX in CPUID?

> Linux by itself handles this well and assigns MSRs properly (we observe
> corresponding set_msr on the hotplugged CPU).

Linux is much more tolerant of oddities, and quite a bit of effort went into
making sure that IA32_FEATURE_CONTROL was initialized if firmware left it unlocked.