Re: [PATCH v2] KVM: x86: Allow guests to see MSR_IA32_TSX_CTRL even if tsx=off

Re: [PATCH v2] KVM: x86: Allow guests to see MSR_IA32_TSX_CTRL even if tsx=off

[Sean Christopherson]

On Fri, Jan 29, 2021, Paolo Bonzini wrote:
> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
> index 76bce832cade..15733013b266 100644
> --- a/arch/x86/kvm/x86.c
> +++ b/arch/x86/kvm/x86.c
> @@ -1401,7 +1401,7 @@ static u64 kvm_get_arch_capabilities(void)
>  	 *	  This lets the guest use VERW to clear CPU buffers.


This comment be updated to call out the new TSX_CTRL behavior.

	/*
	 * On TAA affected systems:
	 *      - nothing to do if TSX is disabled on the host.
	 *      - we emulate TSX_CTRL if present on the host.
	 *	  This lets the guest use VERW to clear CPU buffers.
	 */

>  	 */
>  	if (!boot_cpu_has(X86_FEATURE_RTM))
> -		data &= ~(ARCH_CAP_TAA_NO | ARCH_CAP_TSX_CTRL_MSR);
> +		data &= ~ARCH_CAP_TAA_NO;

Hmm, simply clearing TSX_CTRL will only preserve the host value.  Since
ARCH_CAPABILITIES is unconditionally emulated by KVM, wouldn't it make sense to
unconditionally expose TSX_CTRL as well, as opposed to exposing it only if it's
supported in the host?  I.e. allow migrating a TSX-disabled guest to a host
without TSX.  Or am I misunderstanding how TSX_CTRL is checked/used?

>  	else if (!boot_cpu_has_bug(X86_BUG_TAA))
>  		data |= ARCH_CAP_TAA_NO;
>  
> -- 
> 2.26.2
> 

Re: [PATCH v2] KVM: x86: Allow guests to see MSR_IA32_TSX_CTRL even if tsx=off

[Paolo Bonzini]

On 29/01/21 17:58, Sean Christopherson wrote:
> On Fri, Jan 29, 2021, Paolo Bonzini wrote:
>> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
>> index 76bce832cade..15733013b266 100644
>> --- a/arch/x86/kvm/x86.c
>> +++ b/arch/x86/kvm/x86.c
>> @@ -1401,7 +1401,7 @@ static u64 kvm_get_arch_capabilities(void)
>>   	 *	  This lets the guest use VERW to clear CPU buffers.
> 
> 
> This comment be updated to call out the new TSX_CTRL behavior.
> 
> 	/*
> 	 * On TAA affected systems:
> 	 *      - nothing to do if TSX is disabled on the host.
> 	 *      - we emulate TSX_CTRL if present on the host.
> 	 *	  This lets the guest use VERW to clear CPU buffers.
> 	 */

Ok.

>>   	 */
>>   	if (!boot_cpu_has(X86_FEATURE_RTM))
>> -		data &= ~(ARCH_CAP_TAA_NO | ARCH_CAP_TSX_CTRL_MSR);
>> +		data &= ~ARCH_CAP_TAA_NO;
> 
> Hmm, simply clearing TSX_CTRL will only preserve the host value.  Since
> ARCH_CAPABILITIES is unconditionally emulated by KVM, wouldn't it make sense to
> unconditionally expose TSX_CTRL as well, as opposed to exposing it only if it's
> supported in the host?  I.e. allow migrating a TSX-disabled guest to a host
> without TSX.  Or am I misunderstanding how TSX_CTRL is checked/used?

I'm a bit wary of having a combination (MDS_NO=0, TSX_CTRL=1) that does 
not exist on bare metal.  There are other cases where such combinations 
can happen, especially with the Spectre and SSBD mitigations (for 
example due to AMD CPUID bits for Intel processors), but at least those 
are just redundancies in the CPUID bits and it's more likely that the 
guest does something sensible with them.

Paolo

>>   	else if (!boot_cpu_has_bug(X86_BUG_TAA))
>>   		data |= ARCH_CAP_TAA_NO;
>>   
>> -- 
>> 2.26.2
>>
> 

Re: [PATCH v2] KVM: x86: Allow guests to see MSR_IA32_TSX_CTRL even if tsx=off

[Paolo Bonzini]

On 01/02/21 09:46, Paolo Bonzini wrote:
>>
>> This comment be updated to call out the new TSX_CTRL behavior.
>>
>>     /*
>>      * On TAA affected systems:
>>      *      - nothing to do if TSX is disabled on the host.
>>      *      - we emulate TSX_CTRL if present on the host.
>>      *      This lets the guest use VERW to clear CPU buffers.
>>      */
> 
> Ok.

Hmm, but the comment is even more accurate now than before, isn't it? 
It said nothing about hiding TSX_CTRL, so now it matches the code below.

Paolo

Re: [PATCH v2] KVM: x86: Allow guests to see MSR_IA32_TSX_CTRL even if tsx=off

[Sean Christopherson]

On Mon, Feb 01, 2021, Paolo Bonzini wrote:
> On 01/02/21 09:46, Paolo Bonzini wrote:
> > > 
> > > This comment be updated to call out the new TSX_CTRL behavior.
> > > 
> > >     /*
> > >      * On TAA affected systems:
> > >      *      - nothing to do if TSX is disabled on the host.
> > >      *      - we emulate TSX_CTRL if present on the host.
> > >      *      This lets the guest use VERW to clear CPU buffers.
> > >      */
> > 
> > Ok.
> 
> Hmm, but the comment is even more accurate now than before, isn't it? It
> said nothing about hiding TSX_CTRL, so now it matches the code below.

Ha, that is technically true.  But it says "nothing to do..." and then clears a
flag.  The other interpretation of "nothing to do... at runtime" is also wrong
as KVM emulates the MSR as a nop.

I guess I just find the whole comment more confusing than the code itself.

Re: [PATCH v2] KVM: x86: Allow guests to see MSR_IA32_TSX_CTRL even if tsx=off

[Paolo Bonzini]

On 01/02/21 17:38, Sean Christopherson wrote:
>>>>      /*
>>>>       * On TAA affected systems:
>>>>       *      - nothing to do if TSX is disabled on the host.
>>>>       *      - we emulate TSX_CTRL if present on the host.
>>>>       *      This lets the guest use VERW to clear CPU buffers.
>>>>       */
> 
> it says "nothing to do..." and then clears a
> flag.  The other interpretation of "nothing to do... at runtime" is also wrong
> as KVM emulates the MSR as a nop.
> 
> I guess I just find the whole comment more confusing than the code itself.

What about:


         if (!boot_cpu_has(X86_FEATURE_RTM)) {
                 /*
                  * If RTM=0 because the kernel has disabled TSX, the 
host might
                  * have TAA_NO or TSX_CTRL.  Clear TAA_NO (the guest 
sees RTM=0
                  * and therefore knows that there cannot be TAA) but keep
                  * TSX_CTRL: some buggy userspaces leave it set on 
tsx=on hosts,
                  * and we want to allow migrating those guests to 
tsx=off hosts.
                  */
                 data &= ~ARCH_CAP_TAA_NO;
         } else if (!boot_cpu_has_bug(X86_BUG_TAA)) {
                 data |= ARCH_CAP_TAA_NO;
         } else {
                 /*
                  * Nothing to do here; we emulate TSX_CTRL if present 
on the
                  * host so the guest can choose between disabling TSX or
                  * using VERW to clear CPU buffers.
                  */
         }

Paolo

Re: [PATCH v2] KVM: x86: Allow guests to see MSR_IA32_TSX_CTRL even if tsx=off

[Sean Christopherson]

On Mon, Feb 01, 2021, Paolo Bonzini wrote:
> On 01/02/21 17:38, Sean Christopherson wrote:
> > > > >      /*
> > > > >       * On TAA affected systems:
> > > > >       *      - nothing to do if TSX is disabled on the host.
> > > > >       *      - we emulate TSX_CTRL if present on the host.
> > > > >       *      This lets the guest use VERW to clear CPU buffers.
> > > > >       */
> > 
> > it says "nothing to do..." and then clears a
> > flag.  The other interpretation of "nothing to do... at runtime" is also wrong
> > as KVM emulates the MSR as a nop.
> > 
> > I guess I just find the whole comment more confusing than the code itself.
> 
> What about:
> 
> 
>         if (!boot_cpu_has(X86_FEATURE_RTM)) {
>                 /*
>                  * If RTM=0 because the kernel has disabled TSX, the host might
>                  * have TAA_NO or TSX_CTRL.  Clear TAA_NO (the guest sees RTM=0
>                  * and therefore knows that there cannot be TAA) but keep
>                  * TSX_CTRL: some buggy userspaces leave it set on tsx=on hosts,
>                  * and we want to allow migrating those guests to tsx=off hosts.
>                  */
>                 data &= ~ARCH_CAP_TAA_NO;
>         } else if (!boot_cpu_has_bug(X86_BUG_TAA)) {
>                 data |= ARCH_CAP_TAA_NO;
>         } else {
>                 /*
>                  * Nothing to do here; we emulate TSX_CTRL if present on the
>                  * host so the guest can choose between disabling TSX or
>                  * using VERW to clear CPU buffers.
>                  */
>         }

Awesome!  Thanks much!

Re: [PATCH v2] KVM: x86: Allow guests to see MSR_IA32_TSX_CTRL even if tsx=off

[Sean Christopherson]

On Mon, Feb 01, 2021, Paolo Bonzini wrote:
> On 29/01/21 17:58, Sean Christopherson wrote:
> > On Fri, Jan 29, 2021, Paolo Bonzini wrote:
> > >   	 */
> > >   	if (!boot_cpu_has(X86_FEATURE_RTM))
> > > -		data &= ~(ARCH_CAP_TAA_NO | ARCH_CAP_TSX_CTRL_MSR);
> > > +		data &= ~ARCH_CAP_TAA_NO;
> > 
> > Hmm, simply clearing TSX_CTRL will only preserve the host value.  Since
> > ARCH_CAPABILITIES is unconditionally emulated by KVM, wouldn't it make sense to
> > unconditionally expose TSX_CTRL as well, as opposed to exposing it only if it's
> > supported in the host?  I.e. allow migrating a TSX-disabled guest to a host
> > without TSX.  Or am I misunderstanding how TSX_CTRL is checked/used?
> 
> I'm a bit wary of having a combination (MDS_NO=0, TSX_CTRL=1) that does not
> exist on bare metal.  There are other cases where such combinations can
> happen, especially with the Spectre and SSBD mitigations (for example due to
> AMD CPUID bits for Intel processors), but at least those are just
> redundancies in the CPUID bits and it's more likely that the guest does
> something sensible with them.

Gotcha.  The vulnerability combos and all the double and triple negatives make
my head spin.

Thanks!