[MODERATED] Re: [PATCH] L1TF KVM ARCH_CAPABILITIES #4

[Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>]

On Sun, Jul 15, 2018 at 03:57:16PM +0200, speck for Paolo Bonzini wrote:
> From: Paolo Bonzini <pbonzini@redhat.com>
> Subject: [PATCH 4/4] KVM: VMX: tell nested hypervisor to skip L1D flush on
>  vmentry
> 
> When nested virtualization is in use, VMENTER operations from the
> nested hypervisor into the nested guest will always be processed by
> the bare metal hypervisor, and KVM's "conditional cache flushes"
> mode in particular does a flush on nested vmentry.  Therefore,
> include the "skip L1D flush on vmentry" bit in KVM's suggested
> ARCH_CAPABILITIES setting.
> 
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> ---
>  Documentation/admin-guide/l1tf.rst | 24 ++++++++++++++++++++++++
>  arch/x86/include/asm/kvm_host.h    |  1 +
>  arch/x86/kvm/vmx.c                 |  2 +-
>  arch/x86/kvm/x86.c                 | 15 ++++++++++++++-
>  4 files changed, 40 insertions(+), 2 deletions(-)
> 
> diff --git a/Documentation/admin-guide/l1tf.rst b/Documentation/admin-guide/l1tf.rst
> index 5adf7d7c2b4e..3b22561af512 100644
> --- a/Documentation/admin-guide/l1tf.rst
> +++ b/Documentation/admin-guide/l1tf.rst
> @@ -528,6 +528,30 @@ available:
>      EPT can be disabled in the hypervisor via the 'kvm-intel.ept'
>      parameter.
>  
> +3.4. Nested virtual machines
> +""""""""""""""""""""""""""""
> +
> +When nested virtualization is in use, three operating systems are involved:
> +the bare metal hypervisor, the nested hypervisor, and the nested virtual
> +machine.  VMENTER operations from the nested hypervisor into the nested
> +guest will always be processed by the bare metal hypervisor.  Therefore,
> +when running as a bare metal hypervisor, instead, KVM will:
> +
> + - flush the L1D cache on every switch from nested hypervisor to
> +   nested virtual machine, so that the nested hypervisor's secrets
> +   are not exposed to the nested virtual machine;
> +
> + - flush the L1D cache on every switch from nested virtual machine to
> +   nested hypervisor; this is a complex operation, and flushing the L1D
> +   cache avoids that the bare metal hypervisor's secrets be exposed
> +   to the nested virtual machine;
> +
> + - instruct the nested hypervisor to not perform any L1D cache flush
> +   on VMENTER.
> +
> +This behavior is disabled when KVM is loaded with the module option
> +"kvm-intel.vmentry_l1d_flush=never".
> +
>  
>  .. _default_mitigations:
>  
> diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
> index 57d418061c55..e375011108d1 100644
> --- a/arch/x86/include/asm/kvm_host.h
> +++ b/arch/x86/include/asm/kvm_host.h
> @@ -1417,6 +1417,7 @@ enum {
>  void kvm_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event);
>  void kvm_vcpu_reload_apic_access_page(struct kvm_vcpu *vcpu);
>  
> +u64 kvm_get_arch_capabilities(void);
>  void kvm_define_shared_msr(unsigned index, u32 msr);
>  int kvm_set_shared_msr(unsigned index, u64 val, u64 mask);
>  
> diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
> index bf0801399cb4..e2692920053d 100644
> --- a/arch/x86/kvm/vmx.c
> +++ b/arch/x86/kvm/vmx.c
> @@ -6423,7 +6423,7 @@ static void vmx_vcpu_setup(struct vcpu_vmx *vmx)
>  	}
>  
>  	if (boot_cpu_has(X86_FEATURE_ARCH_CAPABILITIES))
> -		rdmsrl(MSR_IA32_ARCH_CAPABILITIES, vmx->arch_capabilities);
> +		vmx->arch_capabilities = kvm_get_arch_capabilities();
>  
>  	vm_exit_controls_init(vmx, vmcs_config.vmexit_ctrl);
>  
> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
> index 79c8ca2c2ad9..442d16228e29 100644
> --- a/arch/x86/kvm/x86.c
> +++ b/arch/x86/kvm/x86.c
> @@ -1103,11 +1103,24 @@ bool kvm_rdpmc(struct kvm_vcpu *vcpu)
>  
>  static unsigned int num_msr_based_features;
>  
> +u64 kvm_get_arch_capabilities(void)
> +{
> +	u64 data;
> +
> +	rdmsrl_safe(MSR_IA32_ARCH_CAPABILITIES, &data);
> +	if (l1tf_vmx_mitigation != VMENTER_L1D_FLUSH_NEVER)
> +		data |= ARCH_CAP_SKIP_L1DFL_VMENTRY;
> +
> +	return data;
> +}


ODPOST 262 modules
ERROR: "kvm_get_arch_capabilities" [arch/x86/kvm/kvm-intel.ko] undefined!
make[2]: *** [/home/konrad/linux/scripts/Makefile.modpost:92: __modpost] Error 1
make[1]: *** [/home/konrad/linux/Makefile:1213: modules] Error 2
make[1]: *** Waiting for unfinished jobs....