[PATCH 0/2] KVM: x86: VMX: cleanup VMXON region allocation

[PATCH 0/2] KVM: x86: VMX: cleanup VMXON region allocation

[Vitaly Kuznetsov]

Minor cleanup with no functional change (intended):
- Rename 'kvm_area' to 'vmxon_region'
- Simplify setting revision_id for VMXON region when eVMCS is in use

Vitaly Kuznetsov (2):
  KVM: x86: VMX: rename 'kvm_area' to 'vmxon_region'
  KVM: x86: VMX: untangle VMXON revision_id setting when using eVMCS

 arch/x86/kvm/vmx/vmx.c | 41 ++++++++++++++++++-----------------------
 arch/x86/kvm/vmx/vmx.h | 12 +++++++++---
 2 files changed, 27 insertions(+), 26 deletions(-)

-- 
2.24.1

[PATCH 1/2] KVM: x86: VMX: rename 'kvm_area' to 'vmxon_region'

[Vitaly Kuznetsov]

The name 'kvm_area' is misleading (as we have way too many areas which are
KVM related), what alloc_kvm_area()/free_kvm_area() functions really do is
allocate/free VMXON region for all CPUs. Rename accordingly.

No functional change intended.

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
---
 arch/x86/kvm/vmx/vmx.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
index 40b1e6138cd5..dab19e4e5f2b 100644
--- a/arch/x86/kvm/vmx/vmx.c
+++ b/arch/x86/kvm/vmx/vmx.c
@@ -2635,7 +2635,7 @@ int alloc_loaded_vmcs(struct loaded_vmcs *loaded_vmcs)
 	return -ENOMEM;
 }
 
-static void free_kvm_area(void)
+static void free_vmxon_regions(void)
 {
 	int cpu;
 
@@ -2645,7 +2645,7 @@ static void free_kvm_area(void)
 	}
 }
 
-static __init int alloc_kvm_area(void)
+static __init int alloc_vmxon_regions(void)
 {
 	int cpu;
 
@@ -2654,7 +2654,7 @@ static __init int alloc_kvm_area(void)
 
 		vmcs = alloc_vmcs_cpu(false, cpu, GFP_KERNEL);
 		if (!vmcs) {
-			free_kvm_area();
+			free_vmxon_regions();
 			return -ENOMEM;
 		}
 
@@ -7815,7 +7815,7 @@ static __init int hardware_setup(void)
 			return r;
 	}
 
-	r = alloc_kvm_area();
+	r = alloc_vmxon_regions();
 	if (r)
 		nested_vmx_hardware_unsetup();
 	return r;
@@ -7826,7 +7826,7 @@ static __exit void hardware_unsetup(void)
 	if (nested)
 		nested_vmx_hardware_unsetup();
 
-	free_kvm_area();
+	free_vmxon_regions();
 }
 
 static bool vmx_check_apicv_inhibit_reasons(ulong bit)
-- 
2.24.1

[PATCH 2/2] KVM: x86: VMX: untangle VMXON revision_id setting when using eVMCS

[Vitaly Kuznetsov]

As stated in alloc_vmxon_regions(), VMXON region needs to be tagged with
revision id from MSR_IA32_VMX_BASIC even in case of eVMCS. The logic to
do so is not very straightforward: first, we set
hdr.revision_id = KVM_EVMCS_VERSION in alloc_vmcs_cpu() just to reset it
back to vmcs_config.revision_id in alloc_vmxon_regions(). Simplify this by
introducing 'enum vmx_area_type' parameter to what is now known as
alloc_vmx_area_cpu().

No functional change intended.

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
---
 arch/x86/kvm/vmx/vmx.c | 31 +++++++++++++------------------
 arch/x86/kvm/vmx/vmx.h | 12 +++++++++---
 2 files changed, 22 insertions(+), 21 deletions(-)

diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
index dab19e4e5f2b..4ee19fb35cde 100644
--- a/arch/x86/kvm/vmx/vmx.c
+++ b/arch/x86/kvm/vmx/vmx.c
@@ -2554,7 +2554,7 @@ static __init int setup_vmcs_config(struct vmcs_config *vmcs_conf,
 	return 0;
 }
 
-struct vmcs *alloc_vmcs_cpu(bool shadow, int cpu, gfp_t flags)
+struct vmcs *alloc_vmx_area_cpu(enum vmx_area_type type, int cpu, gfp_t flags)
 {
 	int node = cpu_to_node(cpu);
 	struct page *pages;
@@ -2566,13 +2566,21 @@ struct vmcs *alloc_vmcs_cpu(bool shadow, int cpu, gfp_t flags)
 	vmcs = page_address(pages);
 	memset(vmcs, 0, vmcs_config.size);
 
-	/* KVM supports Enlightened VMCS v1 only */
-	if (static_branch_unlikely(&enable_evmcs))
+	/*
+	 * When eVMCS is enabled, vmcs->revision_id needs to be set to the
+	 * supported eVMCS version (KVM_EVMCS_VERSION) instead of revision_id
+	 * reported by MSR_IA32_VMX_BASIC.
+	 *
+	 * However, even though not explicitly documented by TLFS, VMXArea
+	 * passed as VMXON argument should still be marked with revision_id
+	 * reported by physical CPU.
+	 */
+	if (type != VMXON_REGION && static_branch_unlikely(&enable_evmcs))
 		vmcs->hdr.revision_id = KVM_EVMCS_VERSION;
 	else
 		vmcs->hdr.revision_id = vmcs_config.revision_id;
 
-	if (shadow)
+	if (type == SHADOW_VMCS_REGION)
 		vmcs->hdr.shadow_vmcs = 1;
 	return vmcs;
 }
@@ -2652,25 +2660,12 @@ static __init int alloc_vmxon_regions(void)
 	for_each_possible_cpu(cpu) {
 		struct vmcs *vmcs;
 
-		vmcs = alloc_vmcs_cpu(false, cpu, GFP_KERNEL);
+		vmcs = alloc_vmx_area_cpu(VMXON_REGION, cpu, GFP_KERNEL);
 		if (!vmcs) {
 			free_vmxon_regions();
 			return -ENOMEM;
 		}
 
-		/*
-		 * When eVMCS is enabled, alloc_vmcs_cpu() sets
-		 * vmcs->revision_id to KVM_EVMCS_VERSION instead of
-		 * revision_id reported by MSR_IA32_VMX_BASIC.
-		 *
-		 * However, even though not explicitly documented by
-		 * TLFS, VMXArea passed as VMXON argument should
-		 * still be marked with revision_id reported by
-		 * physical CPU.
-		 */
-		if (static_branch_unlikely(&enable_evmcs))
-			vmcs->hdr.revision_id = vmcs_config.revision_id;
-
 		per_cpu(vmxarea, cpu) = vmcs;
 	}
 	return 0;
diff --git a/arch/x86/kvm/vmx/vmx.h b/arch/x86/kvm/vmx/vmx.h
index e64da06c7009..7bdac5a50432 100644
--- a/arch/x86/kvm/vmx/vmx.h
+++ b/arch/x86/kvm/vmx/vmx.h
@@ -489,7 +489,13 @@ static inline struct pi_desc *vcpu_to_pi_desc(struct kvm_vcpu *vcpu)
 	return &(to_vmx(vcpu)->pi_desc);
 }
 
-struct vmcs *alloc_vmcs_cpu(bool shadow, int cpu, gfp_t flags);
+enum vmx_area_type {
+	VMXON_REGION,
+	VMCS_REGION,
+	SHADOW_VMCS_REGION,
+};
+
+struct vmcs *alloc_vmx_area_cpu(enum vmx_area_type type, int cpu, gfp_t flags);
 void free_vmcs(struct vmcs *vmcs);
 int alloc_loaded_vmcs(struct loaded_vmcs *loaded_vmcs);
 void free_loaded_vmcs(struct loaded_vmcs *loaded_vmcs);
@@ -498,8 +504,8 @@ void loaded_vmcs_clear(struct loaded_vmcs *loaded_vmcs);
 
 static inline struct vmcs *alloc_vmcs(bool shadow)
 {
-	return alloc_vmcs_cpu(shadow, raw_smp_processor_id(),
-			      GFP_KERNEL_ACCOUNT);
+	return alloc_vmx_area_cpu(shadow ? SHADOW_VMCS_REGION : VMCS_REGION,
+				  raw_smp_processor_id(), GFP_KERNEL_ACCOUNT);
 }
 
 u64 construct_eptp(struct kvm_vcpu *vcpu, unsigned long root_hpa);
-- 
2.24.1

Re: [PATCH 0/2] KVM: x86: VMX: cleanup VMXON region allocation

[Paolo Bonzini]

On 05/03/20 11:01, Vitaly Kuznetsov wrote:
> Minor cleanup with no functional change (intended):
> - Rename 'kvm_area' to 'vmxon_region'
> - Simplify setting revision_id for VMXON region when eVMCS is in use
> 
> Vitaly Kuznetsov (2):
>   KVM: x86: VMX: rename 'kvm_area' to 'vmxon_region'
>   KVM: x86: VMX: untangle VMXON revision_id setting when using eVMCS
> 
>  arch/x86/kvm/vmx/vmx.c | 41 ++++++++++++++++++-----------------------
>  arch/x86/kvm/vmx/vmx.h | 12 +++++++++---
>  2 files changed, 27 insertions(+), 26 deletions(-)
> 

Queued, thanks.

Paolo

Re: [PATCH 1/2] KVM: x86: VMX: rename 'kvm_area' to 'vmxon_region'

[Sean Christopherson]

Super nit: can I convince you to use "KVM: VMX:" instead of "KVM: x86: VMX:"?

  $ glo | grep -e "KVM: x86: nVMX" -e "KVM: x86: VMX:" | wc -l
  8
  $ glo | grep -e "KVM: nVMX" -e "KVM: VMX:" | wc -l
  1032

I'm very conditioned to scan for "KVM: *VMX:", e.g. I was about to complain
that this used the wrong scope :-)   And in the event that Intel adds a new
technology I'd like to be able to use "KVM: Intel:" and "KVM: ***X:"
instead of "KVM: x86: Intel:" and "KVM: x86: Intel: ***X:" for code that is
common to Intel versus specific to a technology.

On Thu, Mar 05, 2020 at 11:01:22AM +0100, Vitaly Kuznetsov wrote:
> The name 'kvm_area' is misleading (as we have way too many areas which are
> KVM related), what alloc_kvm_area()/free_kvm_area() functions really do is
> allocate/free VMXON region for all CPUs. Rename accordingly.
> 
> No functional change intended.
> 
> Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
> ---

+1000

Reviewed-by: Sean Christopherson <sean.j.christopherson@intel.com>

Re: [PATCH 2/2] KVM: x86: VMX: untangle VMXON revision_id setting when using eVMCS

[Sean Christopherson]

On Thu, Mar 05, 2020 at 11:01:23AM +0100, Vitaly Kuznetsov wrote:
> As stated in alloc_vmxon_regions(), VMXON region needs to be tagged with
> revision id from MSR_IA32_VMX_BASIC even in case of eVMCS. The logic to
> do so is not very straightforward: first, we set
> hdr.revision_id = KVM_EVMCS_VERSION in alloc_vmcs_cpu() just to reset it
> back to vmcs_config.revision_id in alloc_vmxon_regions(). Simplify this by
> introducing 'enum vmx_area_type' parameter to what is now known as
> alloc_vmx_area_cpu().

I'd strongly prefer to keep the alloc_vmcs_cpu() name and call the new enum
"vmcs_type".  The discrepancy could be resolved by a comment above the
VMXON_REGION usage, e.g.

		/* The VMXON region is really just a special type of VMCS. */
		vmcs = alloc_vmcs_cpu(VMXON_REGION, cpu, GFP_KERNEL);

> No functional change intended.
> 
> Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
> ---
>  arch/x86/kvm/vmx/vmx.c | 31 +++++++++++++------------------
>  arch/x86/kvm/vmx/vmx.h | 12 +++++++++---
>  2 files changed, 22 insertions(+), 21 deletions(-)
> 
> diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
> index dab19e4e5f2b..4ee19fb35cde 100644
> --- a/arch/x86/kvm/vmx/vmx.c
> +++ b/arch/x86/kvm/vmx/vmx.c
> @@ -2554,7 +2554,7 @@ static __init int setup_vmcs_config(struct vmcs_config *vmcs_conf,
>  	return 0;
>  }
>  
> -struct vmcs *alloc_vmcs_cpu(bool shadow, int cpu, gfp_t flags)
> +struct vmcs *alloc_vmx_area_cpu(enum vmx_area_type type, int cpu, gfp_t flags)
>  {
>  	int node = cpu_to_node(cpu);
>  	struct page *pages;
> @@ -2566,13 +2566,21 @@ struct vmcs *alloc_vmcs_cpu(bool shadow, int cpu, gfp_t flags)
>  	vmcs = page_address(pages);
>  	memset(vmcs, 0, vmcs_config.size);
>  
> -	/* KVM supports Enlightened VMCS v1 only */
> -	if (static_branch_unlikely(&enable_evmcs))
> +	/*
> +	 * When eVMCS is enabled, vmcs->revision_id needs to be set to the
> +	 * supported eVMCS version (KVM_EVMCS_VERSION) instead of revision_id
> +	 * reported by MSR_IA32_VMX_BASIC.
> +	 *
> +	 * However, even though not explicitly documented by TLFS, VMXArea
> +	 * passed as VMXON argument should still be marked with revision_id
> +	 * reported by physical CPU.
> +	 */
> +	if (type != VMXON_REGION && static_branch_unlikely(&enable_evmcs))
>  		vmcs->hdr.revision_id = KVM_EVMCS_VERSION;
>  	else
>  		vmcs->hdr.revision_id = vmcs_config.revision_id;
>  
> -	if (shadow)
> +	if (type == SHADOW_VMCS_REGION)
>  		vmcs->hdr.shadow_vmcs = 1;
>  	return vmcs;
>  }
> @@ -2652,25 +2660,12 @@ static __init int alloc_vmxon_regions(void)
>  	for_each_possible_cpu(cpu) {
>  		struct vmcs *vmcs;
>  
> -		vmcs = alloc_vmcs_cpu(false, cpu, GFP_KERNEL);
> +		vmcs = alloc_vmx_area_cpu(VMXON_REGION, cpu, GFP_KERNEL);
>  		if (!vmcs) {
>  			free_vmxon_regions();
>  			return -ENOMEM;
>  		}
>  
> -		/*
> -		 * When eVMCS is enabled, alloc_vmcs_cpu() sets
> -		 * vmcs->revision_id to KVM_EVMCS_VERSION instead of
> -		 * revision_id reported by MSR_IA32_VMX_BASIC.
> -		 *
> -		 * However, even though not explicitly documented by
> -		 * TLFS, VMXArea passed as VMXON argument should
> -		 * still be marked with revision_id reported by
> -		 * physical CPU.
> -		 */
> -		if (static_branch_unlikely(&enable_evmcs))
> -			vmcs->hdr.revision_id = vmcs_config.revision_id;
> -
>  		per_cpu(vmxarea, cpu) = vmcs;
>  	}
>  	return 0;
> diff --git a/arch/x86/kvm/vmx/vmx.h b/arch/x86/kvm/vmx/vmx.h
> index e64da06c7009..7bdac5a50432 100644
> --- a/arch/x86/kvm/vmx/vmx.h
> +++ b/arch/x86/kvm/vmx/vmx.h
> @@ -489,7 +489,13 @@ static inline struct pi_desc *vcpu_to_pi_desc(struct kvm_vcpu *vcpu)
>  	return &(to_vmx(vcpu)->pi_desc);
>  }
>  
> -struct vmcs *alloc_vmcs_cpu(bool shadow, int cpu, gfp_t flags);
> +enum vmx_area_type {
> +	VMXON_REGION,
> +	VMCS_REGION,
> +	SHADOW_VMCS_REGION,
> +};
> +
> +struct vmcs *alloc_vmx_area_cpu(enum vmx_area_type type, int cpu, gfp_t flags);
>  void free_vmcs(struct vmcs *vmcs);
>  int alloc_loaded_vmcs(struct loaded_vmcs *loaded_vmcs);
>  void free_loaded_vmcs(struct loaded_vmcs *loaded_vmcs);
> @@ -498,8 +504,8 @@ void loaded_vmcs_clear(struct loaded_vmcs *loaded_vmcs);
>  
>  static inline struct vmcs *alloc_vmcs(bool shadow)
>  {
> -	return alloc_vmcs_cpu(shadow, raw_smp_processor_id(),
> -			      GFP_KERNEL_ACCOUNT);
> +	return alloc_vmx_area_cpu(shadow ? SHADOW_VMCS_REGION : VMCS_REGION,
> +				  raw_smp_processor_id(), GFP_KERNEL_ACCOUNT);
>  }
>  
>  u64 construct_eptp(struct kvm_vcpu *vcpu, unsigned long root_hpa);
> -- 
> 2.24.1
> 

Re: [PATCH 1/2] KVM: x86: VMX: rename 'kvm_area' to 'vmxon_region'

[Vitaly Kuznetsov]

Sean Christopherson <sean.j.christopherson@intel.com> writes:

> Super nit: can I convince you to use "KVM: VMX:" instead of "KVM: x86: VMX:"?
>
>   $ glo | grep -e "KVM: x86: nVMX" -e "KVM: x86: VMX:" | wc -l
>   8
>   $ glo | grep -e "KVM: nVMX" -e "KVM: VMX:" | wc -l
>   1032
>
> I'm very conditioned to scan for "KVM: *VMX:", e.g. I was about to complain
> that this used the wrong scope :-)   And in the event that Intel adds a new
> technology I'd like to be able to use "KVM: Intel:" and "KVM: ***X:"
> instead of "KVM: x86: Intel:" and "KVM: x86: Intel: ***X:" for code that is
> common to Intel versus specific to a technology.

What if someone else adds VMX instead? :-)

Point taken, will use 'KVM: VMX:' in the future (and I'm in no way
object to changing this in the queue if it's not too late).

-- 
Vitaly

Re: [PATCH 1/2] KVM: x86: VMX: rename 'kvm_area' to 'vmxon_region'

[Sean Christopherson]

On Thu, Mar 05, 2020 at 04:58:35PM +0100, Vitaly Kuznetsov wrote:
> Sean Christopherson <sean.j.christopherson@intel.com> writes:
> 
> > Super nit: can I convince you to use "KVM: VMX:" instead of "KVM: x86: VMX:"?
> >
> >   $ glo | grep -e "KVM: x86: nVMX" -e "KVM: x86: VMX:" | wc -l
> >   8
> >   $ glo | grep -e "KVM: nVMX" -e "KVM: VMX:" | wc -l
> >   1032
> >
> > I'm very conditioned to scan for "KVM: *VMX:", e.g. I was about to complain
> > that this used the wrong scope :-)   And in the event that Intel adds a new
> > technology I'd like to be able to use "KVM: Intel:" and "KVM: ***X:"
> > instead of "KVM: x86: Intel:" and "KVM: x86: Intel: ***X:" for code that is
> > common to Intel versus specific to a technology.
> 
> What if someone else adds VMX instead? :-)

I never said it was a _good_ plan :-D

Re: [PATCH 2/2] KVM: x86: VMX: untangle VMXON revision_id setting when using eVMCS

[Vitaly Kuznetsov]

Sean Christopherson <sean.j.christopherson@intel.com> writes:

> On Thu, Mar 05, 2020 at 11:01:23AM +0100, Vitaly Kuznetsov wrote:
>> As stated in alloc_vmxon_regions(), VMXON region needs to be tagged with
>> revision id from MSR_IA32_VMX_BASIC even in case of eVMCS. The logic to
>> do so is not very straightforward: first, we set
>> hdr.revision_id = KVM_EVMCS_VERSION in alloc_vmcs_cpu() just to reset it
>> back to vmcs_config.revision_id in alloc_vmxon_regions(). Simplify this by
>> introducing 'enum vmx_area_type' parameter to what is now known as
>> alloc_vmx_area_cpu().
>
> I'd strongly prefer to keep the alloc_vmcs_cpu() name and call the new enum
> "vmcs_type".  The discrepancy could be resolved by a comment above the
> VMXON_REGION usage, e.g.
>
> 		/* The VMXON region is really just a special type of VMCS. */
> 		vmcs = alloc_vmcs_cpu(VMXON_REGION, cpu, GFP_KERNEL);

I have no strong opinion (but honestly I don't really know what VMXON
region is being used for), Paolo already said 'queued' but I think I'll
send v2 with the suggested changes (including patch prefixes :-)

-- 
Vitaly

Re: [PATCH 2/2] KVM: x86: VMX: untangle VMXON revision_id setting when using eVMCS

[Paolo Bonzini]

On 05/03/20 17:34, Vitaly Kuznetsov wrote:
>> I'd strongly prefer to keep the alloc_vmcs_cpu() name and call the new enum
>> "vmcs_type".  The discrepancy could be resolved by a comment above the
>> VMXON_REGION usage, e.g.
>>
>> 		/* The VMXON region is really just a special type of VMCS. */
>> 		vmcs = alloc_vmcs_cpu(VMXON_REGION, cpu, GFP_KERNEL);
> I have no strong opinion (but honestly I don't really know what VMXON
> region is being used for), Paolo already said 'queued' but I think I'll
> send v2 with the suggested changes (including patch prefixes :-)

Yes, please do.

Paolo