* [PATCH 0/3] KVM/Hyper-V: Add Hyper-V direct tlb flush support @ 2019-08-09 9:49 lantianyu1986 2019-08-09 9:49 ` [PATCH 1/3] x86/Hyper-V: Fix definition of struct hv_vp_assist_page lantianyu1986 2019-08-09 9:49 ` [PATCH 2/3] KVM/Hyper-V: Add new KVM cap KVM_CAP_HYPERV_DIRECT_TLBFLUSH lantianyu1986 0 siblings, 2 replies; 5+ messages in thread From: lantianyu1986 @ 2019-08-09 9:49 UTC (permalink / raw) To: pbonzini, rkrcmar, corbet, kys, haiyangz, sthemmin, sashal, tglx, mingo, bp, hpa, x86, michael.h.kelley Cc: Tianyu Lan, kvm, linux-doc, linux-hyperv, linux-kernel, vkuznets From: Tianyu Lan <Tianyu.Lan@microsoft.com> This patchset is to add Hyper-V direct tlb support in KVM. Hyper-V in L0 can delegate L1 hypervisor to handle tlb flush request from L2 guest when direct tlb flush is enabled in L1. Patch 2 introduces new cap KVM_CAP_HYPERV_DIRECT_TLBFLUSH to enable feature from user space. User space should enable this feature only when Hyper-V hypervisor capability is exposed to guest and KVM profile is hided. There is a parameter conflict between KVM and Hyper-V hypercall. We hope L2 guest doesn't use KVM hypercall when the feature is enabled. Detail please see comment of new API "KVM_CAP_HYPERV_DIRECT_TLBFLUSH" Tianyu Lan (2): x86/Hyper-V: Fix definition of struct hv_vp_assist_page KVM/Hyper-V: Add new KVM cap KVM_CAP_HYPERV_DIRECT_TLBFLUSH Vitaly Kuznetsov (1): KVM/Hyper-V/VMX: Add direct tlb flush support Documentation/virtual/kvm/api.txt | 10 ++++++++++ arch/x86/include/asm/hyperv-tlfs.h | 24 +++++++++++++++++++----- arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/vmx/evmcs.h | 2 ++ arch/x86/kvm/vmx/vmx.c | 38 ++++++++++++++++++++++++++++++++++++++ arch/x86/kvm/x86.c | 8 ++++++++ include/linux/kvm_host.h | 1 + include/uapi/linux/kvm.h | 1 + 8 files changed, 81 insertions(+), 5 deletions(-) -- 2.14.2 ^ permalink raw reply [flat|nested] 5+ messages in thread
* [PATCH 1/3] x86/Hyper-V: Fix definition of struct hv_vp_assist_page 2019-08-09 9:49 [PATCH 0/3] KVM/Hyper-V: Add Hyper-V direct tlb flush support lantianyu1986 @ 2019-08-09 9:49 ` lantianyu1986 2019-08-09 10:25 ` Vitaly Kuznetsov 2019-08-09 9:49 ` [PATCH 2/3] KVM/Hyper-V: Add new KVM cap KVM_CAP_HYPERV_DIRECT_TLBFLUSH lantianyu1986 1 sibling, 1 reply; 5+ messages in thread From: lantianyu1986 @ 2019-08-09 9:49 UTC (permalink / raw) To: pbonzini, rkrcmar, corbet, kys, haiyangz, sthemmin, sashal, tglx, mingo, bp, hpa, x86, michael.h.kelley Cc: Tianyu Lan, kvm, linux-doc, linux-kernel, linux-hyperv, vkuznets From: Tianyu Lan <Tianyu.Lan@microsoft.com> The struct hv_vp_assist_page was defined incorrectly. The "vtl_control" should be u64[3], "nested_enlightenments_control" should be a u64 and there is 7 reserved bytes following "enlighten_vmentry". This patch is to fix it. Signed-off-by: Tianyu Lan <Tianyu.Lan@microsoft.com> --- arch/x86/include/asm/hyperv-tlfs.h | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/arch/x86/include/asm/hyperv-tlfs.h b/arch/x86/include/asm/hyperv-tlfs.h index af78cd72b8f3..a79703c56ebe 100644 --- a/arch/x86/include/asm/hyperv-tlfs.h +++ b/arch/x86/include/asm/hyperv-tlfs.h @@ -517,11 +517,11 @@ struct hv_timer_message_payload { /* Define virtual processor assist page structure. */ struct hv_vp_assist_page { __u32 apic_assist; - __u32 reserved; - __u64 vtl_control[2]; + __u32 reserved1; + __u64 vtl_control[3]; __u64 nested_enlightenments_control[2]; - __u32 enlighten_vmentry; - __u32 padding; + __u8 enlighten_vmentry; + __u8 reserved2[7]; __u64 current_nested_vmcs; } __packed; -- 2.14.2 ^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [PATCH 1/3] x86/Hyper-V: Fix definition of struct hv_vp_assist_page 2019-08-09 9:49 ` [PATCH 1/3] x86/Hyper-V: Fix definition of struct hv_vp_assist_page lantianyu1986 @ 2019-08-09 10:25 ` Vitaly Kuznetsov 0 siblings, 0 replies; 5+ messages in thread From: Vitaly Kuznetsov @ 2019-08-09 10:25 UTC (permalink / raw) To: lantianyu1986 Cc: Tianyu Lan, kvm, linux-doc, linux-kernel, linux-hyperv, pbonzini, rkrcmar, corbet, kys, haiyangz, sthemmin, sashal, tglx, mingo, bp, hpa, x86, michael.h.kelley lantianyu1986@gmail.com writes: > From: Tianyu Lan <Tianyu.Lan@microsoft.com> > > The struct hv_vp_assist_page was defined incorrectly. > The "vtl_control" should be u64[3], "nested_enlightenments_control" > should be a u64 and there is 7 reserved bytes following "enlighten_vmentry". > This patch is to fix it. > > Signed-off-by: Tianyu Lan <Tianyu.Lan@microsoft.com> > --- > arch/x86/include/asm/hyperv-tlfs.h | 8 ++++---- > 1 file changed, 4 insertions(+), 4 deletions(-) > > diff --git a/arch/x86/include/asm/hyperv-tlfs.h b/arch/x86/include/asm/hyperv-tlfs.h > index af78cd72b8f3..a79703c56ebe 100644 > --- a/arch/x86/include/asm/hyperv-tlfs.h > +++ b/arch/x86/include/asm/hyperv-tlfs.h > @@ -517,11 +517,11 @@ struct hv_timer_message_payload { > /* Define virtual processor assist page structure. */ > struct hv_vp_assist_page { > __u32 apic_assist; > - __u32 reserved; > - __u64 vtl_control[2]; > + __u32 reserved1; > + __u64 vtl_control[3]; > __u64 nested_enlightenments_control[2]; In PATCH3 you define 'struct hv_nested_enlightenments_control' and it is 64bit long, not 128. We should change it here too as ... > - __u32 enlighten_vmentry; enlighten_vmentry filed will get a very different offset breaking Enlightened VMCS. > - __u32 padding; > + __u8 enlighten_vmentry; > + __u8 reserved2[7]; > __u64 current_nested_vmcs; > } __packed; -- Vitaly ^ permalink raw reply [flat|nested] 5+ messages in thread
* [PATCH 2/3] KVM/Hyper-V: Add new KVM cap KVM_CAP_HYPERV_DIRECT_TLBFLUSH 2019-08-09 9:49 [PATCH 0/3] KVM/Hyper-V: Add Hyper-V direct tlb flush support lantianyu1986 2019-08-09 9:49 ` [PATCH 1/3] x86/Hyper-V: Fix definition of struct hv_vp_assist_page lantianyu1986 @ 2019-08-09 9:49 ` lantianyu1986 2019-08-09 10:44 ` Vitaly Kuznetsov 1 sibling, 1 reply; 5+ messages in thread From: lantianyu1986 @ 2019-08-09 9:49 UTC (permalink / raw) To: pbonzini, rkrcmar, corbet, kys, haiyangz, sthemmin, sashal, tglx, mingo, bp, hpa, x86, michael.h.kelley Cc: Tianyu Lan, kvm, linux-doc, linux-kernel, linux-hyperv, vkuznets From: Tianyu Lan <Tianyu.Lan@microsoft.com> This patch adds new KVM cap KVM_CAP_HYPERV_DIRECT_TLBFLUSH and let user space to enable direct tlb flush function when only Hyper-V hypervsior capability is exposed to VM. This patch also adds enable_direct_tlbflush callback in the struct kvm_x86_ops and platforms may use it to implement direct tlb flush support. Signed-off-by: Tianyu Lan <Tianyu.Lan@microsoft.com> --- Documentation/virtual/kvm/api.txt | 10 ++++++++++ arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/x86.c | 8 ++++++++ include/uapi/linux/kvm.h | 1 + 4 files changed, 21 insertions(+) diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt index 2cd6250b2896..45308ed6dd75 100644 --- a/Documentation/virtual/kvm/api.txt +++ b/Documentation/virtual/kvm/api.txt @@ -5289,3 +5289,13 @@ Architectures: x86 This capability indicates that KVM supports paravirtualized Hyper-V IPI send hypercalls: HvCallSendSyntheticClusterIpi, HvCallSendSyntheticClusterIpiEx. +8.21 KVM_CAP_HYPERV_DIRECT_TLBFLUSH + +Architecture: x86 + +This capability indicates that KVM supports Hyper-V direct tlb flush function. +User space should enable this feature only when Hyper-V hypervisor capability +is exposed to guest and KVM profile is hided. Both Hyper-V and KVM hypercalls +use RAX and RCX registers to pass parameters. If KVM hypercall is exposed +to L2 guest with direct tlbflush enabled, Hyper-V may mistake KVM hypercall +for Hyper-V tlb flush Hypercall due to paremeter register overlap. diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 0cc5b611a113..667d154e89d4 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1205,6 +1205,8 @@ struct kvm_x86_ops { uint16_t (*nested_get_evmcs_version)(struct kvm_vcpu *vcpu); bool (*need_emulation_on_page_fault)(struct kvm_vcpu *vcpu); + + int (*enable_direct_tlbflush)(struct kvm_vcpu *vcpu); }; struct kvm_arch_async_pf { diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 9d7b9e6a0939..a9d8ee7f7bf0 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -3183,6 +3183,9 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) r = kvm_x86_ops->get_nested_state ? kvm_x86_ops->get_nested_state(NULL, NULL, 0) : 0; break; + case KVM_CAP_HYPERV_DIRECT_TLBFLUSH: + r = kvm_x86_ops->enable_direct_tlbflush ? 1 : 0; + break; default: break; } @@ -3953,6 +3956,11 @@ static int kvm_vcpu_ioctl_enable_cap(struct kvm_vcpu *vcpu, r = -EFAULT; } return r; + case KVM_CAP_HYPERV_DIRECT_TLBFLUSH: + if (!kvm_x86_ops->enable_direct_tlbflush) + return -ENOTTY; + + return kvm_x86_ops->enable_direct_tlbflush(vcpu); default: return -EINVAL; diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index a7c19540ce21..cb959bc925b1 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -996,6 +996,7 @@ struct kvm_ppc_resize_hpt { #define KVM_CAP_ARM_PTRAUTH_ADDRESS 171 #define KVM_CAP_ARM_PTRAUTH_GENERIC 172 #define KVM_CAP_PMU_EVENT_FILTER 173 +#define KVM_CAP_HYPERV_DIRECT_TLBFLUSH 174 #ifdef KVM_CAP_IRQ_ROUTING -- 2.14.2 ^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [PATCH 2/3] KVM/Hyper-V: Add new KVM cap KVM_CAP_HYPERV_DIRECT_TLBFLUSH 2019-08-09 9:49 ` [PATCH 2/3] KVM/Hyper-V: Add new KVM cap KVM_CAP_HYPERV_DIRECT_TLBFLUSH lantianyu1986 @ 2019-08-09 10:44 ` Vitaly Kuznetsov 0 siblings, 0 replies; 5+ messages in thread From: Vitaly Kuznetsov @ 2019-08-09 10:44 UTC (permalink / raw) To: lantianyu1986 Cc: Tianyu Lan, kvm, linux-doc, linux-kernel, linux-hyperv, pbonzini, rkrcmar, corbet, kys, haiyangz, sthemmin, sashal, tglx, mingo, bp, hpa, x86, michael.h.kelley lantianyu1986@gmail.com writes: > From: Tianyu Lan <Tianyu.Lan@microsoft.com> > > This patch adds new KVM cap KVM_CAP_HYPERV_DIRECT_TLBFLUSH and let > user space to enable direct tlb flush function when only Hyper-V > hypervsior capability is exposed to VM. This patch also adds > enable_direct_tlbflush callback in the struct kvm_x86_ops and > platforms may use it to implement direct tlb flush support. > > Signed-off-by: Tianyu Lan <Tianyu.Lan@microsoft.com> > --- > Documentation/virtual/kvm/api.txt | 10 ++++++++++ > arch/x86/include/asm/kvm_host.h | 2 ++ > arch/x86/kvm/x86.c | 8 ++++++++ > include/uapi/linux/kvm.h | 1 + > 4 files changed, 21 insertions(+) > > diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt > index 2cd6250b2896..45308ed6dd75 100644 > --- a/Documentation/virtual/kvm/api.txt > +++ b/Documentation/virtual/kvm/api.txt > @@ -5289,3 +5289,13 @@ Architectures: x86 > This capability indicates that KVM supports paravirtualized Hyper-V IPI send > hypercalls: > HvCallSendSyntheticClusterIpi, HvCallSendSyntheticClusterIpiEx. > +8.21 KVM_CAP_HYPERV_DIRECT_TLBFLUSH > + > +Architecture: x86 > + > +This capability indicates that KVM supports Hyper-V direct tlb flush function. > +User space should enable this feature only when Hyper-V hypervisor capability > +is exposed to guest and KVM profile is hided. Both Hyper-V and KVM hypercalls > +use RAX and RCX registers to pass parameters. If KVM hypercall is exposed > +to L2 guest with direct tlbflush enabled, Hyper-V may mistake KVM hypercall > +for Hyper-V tlb flush Hypercall due to paremeter register overlap. First, we need to explicitly state that this is for KVM on Hyper-V and second, that this disables normal hypercall handling by KVM. My take: This capability indicates that KVM running on top of Hyper-V hypervisor enables Direct TLB flush for its guests meaning that TLB flush hypercalls are handled by Level 1 hypervisor (Hyper-V) bypassing KVM. Due to the different ABI for hypercall parameters between Hyper-V and KVM, enabling this capability effectively disables all hypercall handling by KVM (as some KVM hypercall may be mistakenly treated as TLB flush hypercalls by Hyper-C) so userspace should disable KVM identification in CPUID. I think we should also enforce this somehow leaving only Hyper-V style hypercalls handling (for Windows guests) in place. > diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h > index 0cc5b611a113..667d154e89d4 100644 > --- a/arch/x86/include/asm/kvm_host.h > +++ b/arch/x86/include/asm/kvm_host.h > @@ -1205,6 +1205,8 @@ struct kvm_x86_ops { > uint16_t (*nested_get_evmcs_version)(struct kvm_vcpu *vcpu); > > bool (*need_emulation_on_page_fault)(struct kvm_vcpu *vcpu); > + > + int (*enable_direct_tlbflush)(struct kvm_vcpu *vcpu); > }; > > struct kvm_arch_async_pf { > diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c > index 9d7b9e6a0939..a9d8ee7f7bf0 100644 > --- a/arch/x86/kvm/x86.c > +++ b/arch/x86/kvm/x86.c > @@ -3183,6 +3183,9 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) > r = kvm_x86_ops->get_nested_state ? > kvm_x86_ops->get_nested_state(NULL, NULL, 0) : 0; > break; > + case KVM_CAP_HYPERV_DIRECT_TLBFLUSH: > + r = kvm_x86_ops->enable_direct_tlbflush ? 1 : 0; > + break; > default: > break; > } > @@ -3953,6 +3956,11 @@ static int kvm_vcpu_ioctl_enable_cap(struct kvm_vcpu *vcpu, > r = -EFAULT; > } > return r; > + case KVM_CAP_HYPERV_DIRECT_TLBFLUSH: > + if (!kvm_x86_ops->enable_direct_tlbflush) > + return -ENOTTY; > + > + return kvm_x86_ops->enable_direct_tlbflush(vcpu); > > default: > return -EINVAL; > diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h > index a7c19540ce21..cb959bc925b1 100644 > --- a/include/uapi/linux/kvm.h > +++ b/include/uapi/linux/kvm.h > @@ -996,6 +996,7 @@ struct kvm_ppc_resize_hpt { > #define KVM_CAP_ARM_PTRAUTH_ADDRESS 171 > #define KVM_CAP_ARM_PTRAUTH_GENERIC 172 > #define KVM_CAP_PMU_EVENT_FILTER 173 > +#define KVM_CAP_HYPERV_DIRECT_TLBFLUSH 174 > > #ifdef KVM_CAP_IRQ_ROUTING -- Vitaly ^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2019-08-09 10:44 UTC | newest] Thread overview: 5+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2019-08-09 9:49 [PATCH 0/3] KVM/Hyper-V: Add Hyper-V direct tlb flush support lantianyu1986 2019-08-09 9:49 ` [PATCH 1/3] x86/Hyper-V: Fix definition of struct hv_vp_assist_page lantianyu1986 2019-08-09 10:25 ` Vitaly Kuznetsov 2019-08-09 9:49 ` [PATCH 2/3] KVM/Hyper-V: Add new KVM cap KVM_CAP_HYPERV_DIRECT_TLBFLUSH lantianyu1986 2019-08-09 10:44 ` Vitaly Kuznetsov
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).