From: christoffer.dall@linaro.org (Christoffer Dall)
To: linux-arm-kernel@lists.infradead.org
Subject: [PATCH v2 06/36] KVM: arm64: Defer restoring host VFP state to vcpu_put
Date: Mon, 11 Dec 2017 10:31:22 +0100 [thread overview]
Message-ID: <20171211093122.GG910@cbox> (raw)
In-Reply-To: <864lozssy6.wl-marc.zyngier@arm.com>
On Sat, Dec 09, 2017 at 05:37:53PM +0000, Marc Zyngier wrote:
> On Thu, 07 Dec 2017 17:06:00 +0000,
> Christoffer Dall wrote:
> >
> > Avoid saving the guest VFP registers and restoring the host VFP
> > registers on every exit from the VM. Only when we're about to run
> > userspace or other threads in the kernel do we really have to switch the
> > state back to the host state.
> >
> > We still initially configure the VFP registers to trap when entering the
> > VM, but the difference is that we now leave the guest state in the
> > hardware registers as long as we're running this VCPU, even if we
> > occasionally trap to the host, and we only restore the host state when
> > we return to user space or when scheduling another thread.
> >
> > Reviewed-by: Andrew Jones <drjones@redhat.com>
> > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> > ---
> >
> > Notes:
> > Changes since v1:
> > - Cosmetic changes
> > - Change the flags variable to a u8
> > - Expanded the commit message
> >
> > arch/arm64/include/asm/kvm_emulate.h | 5 ++++
> > arch/arm64/include/asm/kvm_host.h | 3 +++
> > arch/arm64/kernel/asm-offsets.c | 1 +
> > arch/arm64/kvm/hyp/entry.S | 3 +++
> > arch/arm64/kvm/hyp/switch.c | 48 +++++++++++-------------------------
> > arch/arm64/kvm/hyp/sysreg-sr.c | 22 ++++++++++++++---
> > 6 files changed, 46 insertions(+), 36 deletions(-)
> >
> > diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
> > index b36aaa1fe332..635137e6ed1c 100644
> > --- a/arch/arm64/include/asm/kvm_emulate.h
> > +++ b/arch/arm64/include/asm/kvm_emulate.h
> > @@ -67,6 +67,11 @@ static inline unsigned long *vcpu_hcr(struct kvm_vcpu *vcpu)
> > return (unsigned long *)&vcpu->arch.hcr_el2;
> > }
> >
> > +static inline bool vcpu_el1_is_32bit(struct kvm_vcpu *vcpu)
> > +{
> > + return !(vcpu->arch.hcr_el2 & HCR_RW);
> > +}
>
> Since you now introduce this helper, could you use it to repaint
> inject_fault.c which could make use of it too? This could actually be
> a separate patch.
>
Yes, I'll do that first, and the have this patch follow.
> > +
> > static inline unsigned long *vcpu_pc(const struct kvm_vcpu *vcpu)
> > {
> > return (unsigned long *)&vcpu_gp_regs(vcpu)->regs.pc;
> > diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> > index 20fab9194794..c841eeeeb5c5 100644
> > --- a/arch/arm64/include/asm/kvm_host.h
> > +++ b/arch/arm64/include/asm/kvm_host.h
> > @@ -211,6 +211,9 @@ struct kvm_vcpu_arch {
> > /* Guest debug state */
> > u64 debug_flags;
> >
> > + /* 1 if the guest VFP state is loaded into the hardware */
> > + u8 guest_vfp_loaded;
> > +
> > /*
> > * We maintain more than a single set of debug registers to support
> > * debugging the guest from the host and to maintain separate host and
> > diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c
> > index 612021dce84f..99467327c043 100644
> > --- a/arch/arm64/kernel/asm-offsets.c
> > +++ b/arch/arm64/kernel/asm-offsets.c
> > @@ -133,6 +133,7 @@ int main(void)
> > DEFINE(CPU_GP_REGS, offsetof(struct kvm_cpu_context, gp_regs));
> > DEFINE(CPU_USER_PT_REGS, offsetof(struct kvm_regs, regs));
> > DEFINE(CPU_FP_REGS, offsetof(struct kvm_regs, fp_regs));
> > + DEFINE(VCPU_GUEST_VFP_LOADED, offsetof(struct kvm_vcpu, arch.guest_vfp_loaded));
> > DEFINE(VCPU_FPEXC32_EL2, offsetof(struct kvm_vcpu, arch.ctxt.sys_regs[FPEXC32_EL2]));
> > DEFINE(VCPU_HOST_CONTEXT, offsetof(struct kvm_vcpu, arch.host_cpu_context));
> > DEFINE(HOST_CONTEXT_VCPU, offsetof(struct kvm_cpu_context, __hyp_running_vcpu));
> > diff --git a/arch/arm64/kvm/hyp/entry.S b/arch/arm64/kvm/hyp/entry.S
> > index a360ac6e89e9..53652287a236 100644
> > --- a/arch/arm64/kvm/hyp/entry.S
> > +++ b/arch/arm64/kvm/hyp/entry.S
> > @@ -184,6 +184,9 @@ alternative_endif
> > add x0, x2, #CPU_GP_REG_OFFSET(CPU_FP_REGS)
> > bl __fpsimd_restore_state
> >
> > + mov x0, #1
> > + strb w0, [x3, #VCPU_GUEST_VFP_LOADED]
> > +
> > // Skip restoring fpexc32 for AArch64 guests
> > mrs x1, hcr_el2
> > tbnz x1, #HCR_RW_SHIFT, 1f
> > diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> > index 11ec1c6f3b84..f5d53ef9ca79 100644
> > --- a/arch/arm64/kvm/hyp/switch.c
> > +++ b/arch/arm64/kvm/hyp/switch.c
> > @@ -24,43 +24,32 @@
> > #include <asm/fpsimd.h>
> > #include <asm/debug-monitors.h>
> >
> > -static bool __hyp_text __fpsimd_enabled_nvhe(void)
> > -{
> > - return !(read_sysreg(cptr_el2) & CPTR_EL2_TFP);
> > -}
> > -
> > -static bool __hyp_text __fpsimd_enabled_vhe(void)
> > -{
> > - return !!(read_sysreg(cpacr_el1) & CPACR_EL1_FPEN);
> > -}
> > -
> > -static hyp_alternate_select(__fpsimd_is_enabled,
> > - __fpsimd_enabled_nvhe, __fpsimd_enabled_vhe,
> > - ARM64_HAS_VIRT_HOST_EXTN);
> > -
> > -bool __hyp_text __fpsimd_enabled(void)
> > -{
> > - return __fpsimd_is_enabled()();
> > -}
> > -
> > -static void __hyp_text __activate_traps_vhe(void)
> > +static void __hyp_text __activate_traps_vhe(struct kvm_vcpu *vcpu)
> > {
> > u64 val;
> >
> > val = read_sysreg(cpacr_el1);
> > val |= CPACR_EL1_TTA;
> > - val &= ~(CPACR_EL1_FPEN | CPACR_EL1_ZEN);
> > + val &= ~CPACR_EL1_ZEN;
> > + if (vcpu->arch.guest_vfp_loaded)
> > + val |= CPACR_EL1_FPEN;
> > + else
> > + val &= ~CPACR_EL1_FPEN;
> > write_sysreg(val, cpacr_el1);
> >
> > write_sysreg(__kvm_hyp_vector, vbar_el1);
> > }
> >
> > -static void __hyp_text __activate_traps_nvhe(void)
> > +static void __hyp_text __activate_traps_nvhe(struct kvm_vcpu *vcpu)
> > {
> > u64 val;
> >
> > val = CPTR_EL2_DEFAULT;
> > - val |= CPTR_EL2_TTA | CPTR_EL2_TFP | CPTR_EL2_TZ;
> > + val |= CPTR_EL2_TTA | CPTR_EL2_TZ;
> > + if (vcpu->arch.guest_vfp_loaded)
> > + val &= ~CPTR_EL2_TFP;
> > + else
> > + val |= CPTR_EL2_TFP;
> > write_sysreg(val, cptr_el2);
> > }
> >
> > @@ -83,7 +72,8 @@ static void __hyp_text __activate_traps(struct kvm_vcpu *vcpu)
> > */
> > val = vcpu->arch.hcr_el2;
> >
> > - if (!(val & HCR_RW) && system_supports_fpsimd()) {
> > + if (vcpu_el1_is_32bit(vcpu) && system_supports_fpsimd() &&
> > + !vcpu->arch.guest_vfp_loaded) {
> > write_sysreg(1 << 30, fpexc32_el2);
> > isb();
> > }
> > @@ -100,7 +90,7 @@ static void __hyp_text __activate_traps(struct kvm_vcpu *vcpu)
> > write_sysreg(0, pmselr_el0);
> > write_sysreg(ARMV8_PMU_USERENR_MASK, pmuserenr_el0);
> > write_sysreg(vcpu->arch.mdcr_el2, mdcr_el2);
> > - __activate_traps_arch()();
> > + __activate_traps_arch()(vcpu);
> > }
> >
> > static void __hyp_text __deactivate_traps_vhe(void)
> > @@ -288,7 +278,6 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> > {
> > struct kvm_cpu_context *host_ctxt;
> > struct kvm_cpu_context *guest_ctxt;
> > - bool fp_enabled;
> > u64 exit_code;
> >
> > vcpu = kern_hyp_va(vcpu);
> > @@ -380,8 +369,6 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> > /* 0 falls through to be handled out of EL2 */
> > }
> >
> > - fp_enabled = __fpsimd_enabled();
> > -
> > __sysreg_save_guest_state(guest_ctxt);
> > __sysreg32_save_state(vcpu);
> > __timer_disable_traps(vcpu);
> > @@ -392,11 +379,6 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> >
> > __sysreg_restore_host_state(host_ctxt);
> >
> > - if (fp_enabled) {
> > - __fpsimd_save_state(&guest_ctxt->gp_regs.fp_regs);
> > - __fpsimd_restore_state(&host_ctxt->gp_regs.fp_regs);
> > - }
> > -
> > __debug_save_state(vcpu, kern_hyp_va(vcpu->arch.debug_ptr), guest_ctxt);
> > /*
> > * This must come after restoring the host sysregs, since a non-VHE
> > diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c
> > index cbbcd6f410a8..68a7d164e5e1 100644
> > --- a/arch/arm64/kvm/hyp/sysreg-sr.c
> > +++ b/arch/arm64/kvm/hyp/sysreg-sr.c
> > @@ -19,6 +19,7 @@
> > #include <linux/kvm_host.h>
> >
> > #include <asm/kvm_asm.h>
> > +#include <asm/kvm_emulate.h>
> > #include <asm/kvm_hyp.h>
> >
> > /* Yes, this does nothing, on purpose */
> > @@ -137,6 +138,11 @@ void __hyp_text __sysreg_restore_guest_state(struct kvm_cpu_context *ctxt)
> > __sysreg_restore_common_state(ctxt);
> > }
> >
> > +static void __hyp_text __fpsimd32_save_state(struct kvm_cpu_context *ctxt)
> > +{
> > + ctxt->sys_regs[FPEXC32_EL2] = read_sysreg(fpexc32_el2);
> > +}
> > +
> > void __hyp_text __sysreg32_save_state(struct kvm_vcpu *vcpu)
> > {
> > u64 *spsr, *sysreg;
> > @@ -155,9 +161,6 @@ void __hyp_text __sysreg32_save_state(struct kvm_vcpu *vcpu)
> > sysreg[DACR32_EL2] = read_sysreg(dacr32_el2);
> > sysreg[IFSR32_EL2] = read_sysreg(ifsr32_el2);
> >
> > - if (__fpsimd_enabled())
> > - sysreg[FPEXC32_EL2] = read_sysreg(fpexc32_el2);
> > -
> > if (vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY)
> > sysreg[DBGVCR32_EL2] = read_sysreg(dbgvcr32_el2);
> > }
> > @@ -212,6 +215,19 @@ void kvm_vcpu_load_sysregs(struct kvm_vcpu *vcpu)
> > */
> > void kvm_vcpu_put_sysregs(struct kvm_vcpu *vcpu)
> > {
> > + struct kvm_cpu_context *host_ctxt = vcpu->arch.host_cpu_context;
> > + struct kvm_cpu_context *guest_ctxt = &vcpu->arch.ctxt;
> > +
> > + /* Restore host FP/SIMD state */
> > + if (vcpu->arch.guest_vfp_loaded) {
> > + if (vcpu_el1_is_32bit(vcpu)) {
> > + kvm_call_hyp(__fpsimd32_save_state,
> > + kern_hyp_va(guest_ctxt));
> > + }
> > + __fpsimd_save_state(&guest_ctxt->gp_regs.fp_regs);
> > + __fpsimd_restore_state(&host_ctxt->gp_regs.fp_regs);
> > + vcpu->arch.guest_vfp_loaded = 0;
> > + }
> > }
> >
> > void __hyp_text __kvm_set_tpidr_el2(u64 tpidr_el2)
> > --
> > 2.14.2
> >
>
> Otherwise,
>
> Reviewed-by: Marc Zyngier <marc.zyngier@arm.com>
>
Thanks!
-Christoffer
next prev parent reply other threads:[~2017-12-11 9:31 UTC|newest]
Thread overview: 79+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-12-07 17:05 [PATCH v2 00/36] Optimize KVM/ARM for VHE systems Christoffer Dall
2017-12-07 17:05 ` [PATCH v2 01/36] KVM: arm64: Avoid storing the vcpu pointer on the stack Christoffer Dall
2017-12-09 17:19 ` Marc Zyngier
2017-12-11 9:30 ` Christoffer Dall
2017-12-11 9:35 ` Marc Zyngier
2017-12-07 17:05 ` [PATCH v2 02/36] KVM: arm64: Rework hyp_panic for VHE and non-VHE Christoffer Dall
2017-12-09 17:24 ` Marc Zyngier
2017-12-07 17:05 ` [PATCH v2 03/36] KVM: arm64: Move HCR_INT_OVERRIDE to default HCR_EL2 guest flag Christoffer Dall
2017-12-07 17:05 ` [PATCH v2 04/36] KVM: arm/arm64: Get rid of vcpu->arch.irq_lines Christoffer Dall
2017-12-07 17:05 ` [PATCH v2 05/36] KVM: arm/arm64: Add kvm_vcpu_load_sysregs and kvm_vcpu_put_sysregs Christoffer Dall
2017-12-09 17:30 ` Marc Zyngier
2017-12-07 17:06 ` [PATCH v2 06/36] KVM: arm64: Defer restoring host VFP state to vcpu_put Christoffer Dall
2017-12-09 17:37 ` Marc Zyngier
2017-12-11 9:31 ` Christoffer Dall [this message]
2017-12-07 17:06 ` [PATCH v2 07/36] KVM: arm64: Move debug dirty flag calculation out of world switch Christoffer Dall
2017-12-09 19:20 ` Marc Zyngier
2017-12-07 17:06 ` [PATCH v2 08/36] KVM: arm64: Slightly improve debug save/restore functions Christoffer Dall
2017-12-09 19:37 ` Marc Zyngier
2017-12-07 17:06 ` [PATCH v2 09/36] KVM: arm64: Improve debug register save/restore flow Christoffer Dall
2017-12-11 9:40 ` Marc Zyngier
2017-12-07 17:06 ` [PATCH v2 10/36] KVM: arm64: Factor out fault info population and gic workarounds Christoffer Dall
2017-12-11 9:45 ` Marc Zyngier
2017-12-07 17:06 ` [PATCH v2 11/36] KVM: arm64: Introduce VHE-specific kvm_vcpu_run Christoffer Dall
2017-12-11 9:53 ` Marc Zyngier
2017-12-14 12:12 ` Christoffer Dall
2017-12-14 12:17 ` Marc Zyngier
2017-12-07 17:06 ` [PATCH v2 12/36] KVM: arm64: Remove kern_hyp_va() use in VHE switch function Christoffer Dall
2017-12-11 9:54 ` Marc Zyngier
2017-12-07 17:06 ` [PATCH v2 13/36] KVM: arm64: Don't deactivate VM on VHE systems Christoffer Dall
2017-12-11 9:58 ` Marc Zyngier
2017-12-07 17:06 ` [PATCH v2 14/36] KVM: arm64: Remove noop calls to timer save/restore from VHE switch Christoffer Dall
2017-12-11 10:02 ` Marc Zyngier
2017-12-14 12:30 ` Christoffer Dall
2017-12-07 17:06 ` [PATCH v2 15/36] KVM: arm64: Move userspace system registers into separate function Christoffer Dall
2017-12-11 10:14 ` Marc Zyngier
2017-12-14 12:53 ` Christoffer Dall
2017-12-07 17:06 ` [PATCH v2 16/36] KVM: arm64: Rewrite sysreg alternatives to static keys Christoffer Dall
2017-12-11 10:15 ` Marc Zyngier
2017-12-07 17:06 ` [PATCH v2 17/36] KVM: arm64: Introduce separate VHE/non-VHE sysreg save/restore functions Christoffer Dall
2017-12-11 10:22 ` Marc Zyngier
2017-12-07 17:06 ` [PATCH v2 18/36] KVM: arm/arm64: Remove leftover comment from kvm_vcpu_run_vhe Christoffer Dall
2017-12-11 10:30 ` Marc Zyngier
2017-12-07 17:06 ` [PATCH v2 19/36] KVM: arm64: Unify non-VHE host/guest sysreg save and restore functions Christoffer Dall
2017-12-11 10:40 ` Marc Zyngier
2017-12-07 17:06 ` [PATCH v2 20/36] KVM: arm64: Don't save the host ELR_EL2 and SPSR_EL2 on VHE systems Christoffer Dall
2017-12-11 10:44 ` Marc Zyngier
2017-12-14 13:46 ` Christoffer Dall
2017-12-07 17:06 ` [PATCH v2 21/36] KVM: arm64: Change 32-bit handling of VM system registers Christoffer Dall
2017-12-11 10:57 ` Marc Zyngier
2017-12-07 17:06 ` [PATCH v2 22/36] KVM: arm64: Prepare to handle traps on deferred VM sysregs Christoffer Dall
2017-12-11 11:10 ` Marc Zyngier
2017-12-11 11:24 ` Christoffer Dall
2017-12-11 11:46 ` Marc Zyngier
2017-12-12 13:08 ` Marc Zyngier
2017-12-12 15:46 ` Christoffer Dall
2017-12-07 17:06 ` [PATCH v2 23/36] KVM: arm64: Prepare to handle traps on deferred EL0 sysregs Christoffer Dall
2017-12-07 17:06 ` [PATCH v2 24/36] KVM: arm64: Prepare to handle traps on remaining deferred EL1 sysregs Christoffer Dall
2017-12-07 17:06 ` [PATCH v2 25/36] KVM: arm64: Prepare to handle traps on deferred AArch32 sysregs Christoffer Dall
2017-12-07 17:06 ` [PATCH v2 26/36] KVM: arm64: Defer saving/restoring system registers to vcpu load/put on VHE Christoffer Dall
2017-12-11 13:20 ` Marc Zyngier
2017-12-15 16:29 ` Christoffer Dall
2017-12-29 16:39 ` Christoffer Dall
2017-12-07 17:06 ` [PATCH v2 27/36] KVM: arm64: Move common VHE/non-VHE trap config in separate functions Christoffer Dall
2017-12-11 13:53 ` Marc Zyngier
2017-12-07 17:06 ` [PATCH v2 28/36] KVM: arm64: Configure FPSIMD traps on vcpu load/put for VHE Christoffer Dall
2017-12-11 14:18 ` Marc Zyngier
2017-12-07 17:06 ` [PATCH v2 29/36] KVM: arm64: Configure c15, PMU, and debug register traps on cpu " Christoffer Dall
2017-12-07 17:06 ` [PATCH v2 30/36] KVM: arm64: Separate activate_traps and deactive_traps for VHE and non-VHE Christoffer Dall
2017-12-07 17:06 ` [PATCH v2 31/36] KVM: arm/arm64: Get rid of vgic_elrsr Christoffer Dall
2017-12-07 17:06 ` [PATCH v2 32/36] KVM: arm/arm64: Handle VGICv2 save/restore from the main VGIC code Christoffer Dall
2017-12-07 17:06 ` [PATCH v2 33/36] KVM: arm/arm64: Move arm64-only vgic-v2-sr.c file to arm64 Christoffer Dall
2017-12-07 17:06 ` [PATCH v2 34/36] KVM: arm/arm64: Handle VGICv3 save/restore from the main VGIC code on VHE Christoffer Dall
2017-12-07 17:06 ` [PATCH v2 35/36] KVM: arm/arm64: Move VGIC APR save/restore to vgic put/load Christoffer Dall
2017-12-07 17:06 ` [PATCH v2 36/36] KVM: arm/arm64: Avoid VGICv3 save/restore on VHE with no IRQs Christoffer Dall
2017-12-11 14:43 ` [PATCH v2 00/36] Optimize KVM/ARM for VHE systems Yury Norov
2017-12-11 14:56 ` Marc Zyngier
2017-12-11 15:14 ` Yury Norov
2017-12-11 15:34 ` Christoffer Dall
2017-12-11 16:30 ` Yury Norov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20171211093122.GG910@cbox \
--to=christoffer.dall@linaro.org \
--cc=linux-arm-kernel@lists.infradead.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).