From mboxrd@z Thu Jan 1 00:00:00 1970 From: Wei Liu Subject: Re: [PATCH v5] x86/hvm/viridian: flush remote tlbs by hypercall Date: Thu, 26 Nov 2015 12:03:19 +0000 Message-ID: <20151126120318.GM15747@citrix.com> References: <1448013312-930-1-git-send-email-paul.durrant@citrix.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: Received: from mail6.bemta3.messagelabs.com ([195.245.230.39]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1a1vGr-0002Ip-QK for xen-devel@lists.xenproject.org; Thu, 26 Nov 2015 12:03:30 +0000 Content-Disposition: inline In-Reply-To: <1448013312-930-1-git-send-email-paul.durrant@citrix.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Paul Durrant Cc: Wei Liu , Ian Campbell , Stefano Stabellini , Andrew Cooper , Ian Jackson , xen-devel@lists.xenproject.org, Keir Fraser List-Id: xen-devel@lists.xenproject.org On Fri, Nov 20, 2015 at 09:55:12AM +0000, Paul Durrant wrote: > The Microsoft Hypervisor Top Level Functional Spec. (section 3.4) defines > two bits in CPUID leaf 0x40000004:EAX for the hypervisor to recommend > whether or not to issue a hypercall for local or remote TLB flush. > > Whilst it's doubtful whether using a hypercall for local TLB flush would > be any more efficient than a specific INVLPG VMEXIT, a remote TLB flush > may well be more efficiently done. This is because the alternative > mechanism is to IPI all the vCPUs in question which (in the absence of > APIC virtualisation) will require emulation and scheduling of the vCPUs > only to have them immediately VMEXIT for local TLB flush. > > This patch therefore adds a viridian option which, if selected, enables > the hypercall for remote TLB flush and implements it using ASID > invalidation for targetted vCPUs followed by an IPI only to the set of > CPUs that happened to be running a targetted vCPU (which may be the empty > set). The flush may be more severe than requested since the hypercall can > request flush only for a specific address space (CR3) but Xen neither > keeps a mapping of ASID to guest CR3 nor allows invalidation of a specific > ASID, but on a host with contended CPUs performance is still likely to > be better than a more specific flush using IPIs. > > The implementation of the patch introduces per-vCPU viridian_init() and > viridian_deinit() functions to allow a scratch cpumask to be allocated. > This avoids needing to put this potentially large data structure on stack > during hypercall processing. It also modifies the hypercall input and > output bit-fields to allow a check for the 'fast' calling convention, > and a white-space fix in the definition of HVMPV_feature_mask (to remove > hard tabs). > > Signed-off-by: Paul Durrant > Cc: Ian Jackson > Cc: Stefano Stabellini > Cc: Ian Campbell > Cc: Wei Liu > Cc: Keir Fraser > Reviewed-by: Jan Beulich > Cc: Andrew Cooper > --- > > v5: > - Make sure vcpu_mask is only compared against vcpu_id < 64 > > v4: > - Remove extraneous blank line in params.h > - Use __cpumask_set_cpu() rather than cpumask_set_cpu() > > v3: > - Correct use of cpumask_var_t > - Extend comment to explain pcpu_mask flush > - Other cosmetic changes > > v2: > - Re-name viridian_init/deinit() to viridian_vcpu_init/deinit() > - Use alloc/free_cpumask_var() > - Use hvm_copy_from_guest_phys() to get hypercall arguments > --- > docs/man/xl.cfg.pod.5 | 6 ++ > tools/libxl/libxl_dom.c | 3 + > tools/libxl/libxl_types.idl | 1 + > xen/arch/x86/hvm/hvm.c | 12 ++++ > xen/arch/x86/hvm/viridian.c | 123 +++++++++++++++++++++++++++++++++---- > xen/include/asm-x86/hvm/viridian.h | 4 ++ > xen/include/asm-x86/perfc_defn.h | 1 + > xen/include/public/hvm/params.h | 13 ++-- > 8 files changed, 146 insertions(+), 17 deletions(-) > > diff --git a/docs/man/xl.cfg.pod.5 b/docs/man/xl.cfg.pod.5 > index b63846a..1a88e36 100644 > --- a/docs/man/xl.cfg.pod.5 > +++ b/docs/man/xl.cfg.pod.5 > @@ -1466,6 +1466,12 @@ This set incorporates the Partition Reference TSC MSR. This > enlightenment can improve performance of Windows 7 and Windows > Server 2008 R2 onwards. > > +=item B > + > +This set incorporates use of hypercalls for remote TLB flushing. > +This enlightenment may improve performance of Windows guests running > +on hosts with higher levels of (physical) CPU contention. > + > =item B > > This is a special value that enables the default set of groups, which > diff --git a/tools/libxl/libxl_dom.c b/tools/libxl/libxl_dom.c > index 44d481b..009ca9c 100644 > --- a/tools/libxl/libxl_dom.c > +++ b/tools/libxl/libxl_dom.c > @@ -251,6 +251,9 @@ static int hvm_set_viridian_features(libxl__gc *gc, uint32_t domid, > if (libxl_bitmap_test(&enlightenments, LIBXL_VIRIDIAN_ENLIGHTENMENT_REFERENCE_TSC)) > mask |= HVMPV_reference_tsc; > > + if (libxl_bitmap_test(&enlightenments, LIBXL_VIRIDIAN_ENLIGHTENMENT_HCALL_REMOTE_TLB_FLUSH)) > + mask |= HVMPV_hcall_remote_tlb_flush; > + > if (mask != 0 && > xc_hvm_param_set(CTX->xch, > domid, > diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl > index 4d78f86..0aa5b9d 100644 > --- a/tools/libxl/libxl_types.idl > +++ b/tools/libxl/libxl_types.idl > @@ -219,6 +219,7 @@ libxl_viridian_enlightenment = Enumeration("viridian_enlightenment", [ > (1, "freq"), > (2, "time_ref_count"), > (3, "reference_tsc"), > + (4, "hcall_remote_tlb_flush"), > ]) > Acked-by: Wei Liu