[PATCH] KVM: x86: expose MSR_TSC_AUX to userspace

[PATCH] KVM: x86: expose MSR_TSC_AUX to userspace

[Paolo Bonzini]

If we do not do this, it is not properly saved and restored across
migration.  Windows notices due to its self-protection mechanisms,
and is very upset about it (blue screen of death).

Cc: Radim Krcmar <rkrcmar@redhat.com>
Cc: stable@vger.kernel.org
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 arch/x86/kvm/x86.c | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index aba7f95..377160d 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -950,7 +950,7 @@ static u32 msrs_to_save[] = {
 	MSR_CSTAR, MSR_KERNEL_GS_BASE, MSR_SYSCALL_MASK, MSR_LSTAR,
 #endif
 	MSR_IA32_TSC, MSR_IA32_CR_PAT, MSR_VM_HSAVE_PA,
-	MSR_IA32_FEATURE_CONTROL, MSR_IA32_BNDCFGS
+	MSR_IA32_FEATURE_CONTROL, MSR_IA32_BNDCFGS, MSR_TSC_AUX,
 };
 
 static unsigned num_msrs_to_save;
@@ -3979,16 +3979,17 @@ static void kvm_init_msr_list(void)
 
 		/*
 		 * Even MSRs that are valid in the host may not be exposed
-		 * to the guests in some cases.  We could work around this
-		 * in VMX with the generic MSR save/load machinery, but it
-		 * is not really worthwhile since it will really only
-		 * happen with nested virtualization.
+		 * to the guests in some cases.
 		 */
 		switch (msrs_to_save[i]) {
 		case MSR_IA32_BNDCFGS:
 			if (!kvm_x86_ops->mpx_supported())
 				continue;
 			break;
+		case MSR_TSC_AUX:
+			if (!kvm_x86_ops->rdtscp_supported())
+				continue;
+			break;
 		default:
 			break;
 		}
-- 
2.4.3

Re: [PATCH] KVM: x86: expose MSR_TSC_AUX to userspace

[Paolo Bonzini]

> Paolo, under what circumstances (which versions of Windows? Anything
> special running in the guest?) has this failure happened? I'd like to repro
> this, I'm not sure if we've observed it before.

We saw it with migration under Windows 10, nothing special running in the
guest.  It's very hard to reproduce, we've only seen it once but the BSOD
parameters provided surprisingly good evidence:

----------------------------------
CRITICAL_STRUCTURE_CORRUPTION (109)
This bugcheck is generated when the kernel detects that critical kernel code or
data have been corrupted. There are generally three causes for a corruption:
1) A driver has inadvertently or deliberately modified critical kernel code
 or data. See http://www.microsoft.com/whdc/driver/kernel/64bitPatching.mspx
2) A developer attempted to set a normal kernel breakpoint using a kernel
 debugger that was not attached when the system was booted. Normal breakpoints,
 "bp", can only be set if the debugger is attached at boot time. Hardware
 breakpoints, "ba", can be set at any time.
3) A hardware corruption occurred, e.g. failing RAM holding kernel code or data.
Arguments:
Arg1: a3a01f58a88e3638, Reserved
Arg2: b3b72bdefb0f076f, Reserved
Arg3: 00000001c0000103, Failure type dependent information
Arg4: 0000000000000007, Type of corrupted region, can be
	...
	7   : Critical MSR modification
----------------------------------

Argument 1 and 2 might be related to the old and new value (perhaps some
kind of hash).

Argument 3 is not documented either, but the low 32 bits look a lot like
the MSR_TSC_AUX index. :)

Paolo

Re: [PATCH] KVM: x86: expose MSR_TSC_AUX to userspace

[Peter Hornyack]

On Fri, Nov 13, 2015 at 1:42 AM, Paolo Bonzini <pbonzini@redhat.com> wrote:
>
>> Paolo, under what circumstances (which versions of Windows? Anything
>> special running in the guest?) has this failure happened? I'd like to repro
>> this, I'm not sure if we've observed it before.
>
> We saw it with migration under Windows 10, nothing special running in the
> guest.  It's very hard to reproduce, we've only seen it once but the BSOD
> parameters provided surprisingly good evidence:

Great, thanks for that information and for the patch. I'll let you
know if I successfully reproduce the issue here.

> ----------------------------------
> CRITICAL_STRUCTURE_CORRUPTION (109)
> This bugcheck is generated when the kernel detects that critical kernel code or
> data have been corrupted. There are generally three causes for a corruption:
> 1) A driver has inadvertently or deliberately modified critical kernel code
>  or data. See http://www.microsoft.com/whdc/driver/kernel/64bitPatching.mspx
> 2) A developer attempted to set a normal kernel breakpoint using a kernel
>  debugger that was not attached when the system was booted. Normal breakpoints,
>  "bp", can only be set if the debugger is attached at boot time. Hardware
>  breakpoints, "ba", can be set at any time.
> 3) A hardware corruption occurred, e.g. failing RAM holding kernel code or data.
> Arguments:
> Arg1: a3a01f58a88e3638, Reserved
> Arg2: b3b72bdefb0f076f, Reserved
> Arg3: 00000001c0000103, Failure type dependent information
> Arg4: 0000000000000007, Type of corrupted region, can be
>         ...
>         7   : Critical MSR modification
> ----------------------------------
>
> Argument 1 and 2 might be related to the old and new value (perhaps some
> kind of hash).
>
> Argument 3 is not documented either, but the low 32 bits look a lot like
> the MSR_TSC_AUX index. :)
>
> Paolo

Peter