From: Nicholas Piggin <npiggin@gmail.com>
To: kvm-ppc@vger.kernel.org, linuxppc-dev@lists.ozlabs.org
Cc: Nicholas Piggin <npiggin@gmail.com>
Subject: [PATCH v3 35/52] KVM: PPC: Book3S HV P9: More SPR speed improvements
Date: Tue, 5 Oct 2021 02:00:32 +1000 [thread overview]
Message-ID: <20211004160049.1338837-36-npiggin@gmail.com> (raw)
In-Reply-To: <20211004160049.1338837-1-npiggin@gmail.com>
This avoids more scoreboard stalls and reduces mtSPRs.
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
---
arch/powerpc/kvm/book3s_hv_p9_entry.c | 73 ++++++++++++++++-----------
1 file changed, 43 insertions(+), 30 deletions(-)
diff --git a/arch/powerpc/kvm/book3s_hv_p9_entry.c b/arch/powerpc/kvm/book3s_hv_p9_entry.c
index 67f57b03a896..a23f09fa7d2d 100644
--- a/arch/powerpc/kvm/book3s_hv_p9_entry.c
+++ b/arch/powerpc/kvm/book3s_hv_p9_entry.c
@@ -645,24 +645,29 @@ int kvmhv_vcpu_entry_p9(struct kvm_vcpu *vcpu, u64 time_limit, unsigned long lpc
vc->tb_offset_applied = vc->tb_offset;
}
- if (vc->pcr)
- mtspr(SPRN_PCR, vc->pcr | PCR_MASK);
- mtspr(SPRN_DPDES, vc->dpdes);
mtspr(SPRN_VTB, vc->vtb);
-
mtspr(SPRN_PURR, vcpu->arch.purr);
mtspr(SPRN_SPURR, vcpu->arch.spurr);
+ if (vc->pcr)
+ mtspr(SPRN_PCR, vc->pcr | PCR_MASK);
+ if (vc->dpdes)
+ mtspr(SPRN_DPDES, vc->dpdes);
+
if (dawr_enabled()) {
- mtspr(SPRN_DAWR0, vcpu->arch.dawr0);
- mtspr(SPRN_DAWRX0, vcpu->arch.dawrx0);
+ if (vcpu->arch.dawr0 != host_dawr0)
+ mtspr(SPRN_DAWR0, vcpu->arch.dawr0);
+ if (vcpu->arch.dawrx0 != host_dawrx0)
+ mtspr(SPRN_DAWRX0, vcpu->arch.dawrx0);
if (cpu_has_feature(CPU_FTR_DAWR1)) {
- mtspr(SPRN_DAWR1, vcpu->arch.dawr1);
- mtspr(SPRN_DAWRX1, vcpu->arch.dawrx1);
+ if (vcpu->arch.dawr1 != host_dawr1)
+ mtspr(SPRN_DAWR1, vcpu->arch.dawr1);
+ if (vcpu->arch.dawrx1 != host_dawrx1)
+ mtspr(SPRN_DAWRX1, vcpu->arch.dawrx1);
}
}
- mtspr(SPRN_CIABR, vcpu->arch.ciabr);
- mtspr(SPRN_IC, vcpu->arch.ic);
+ if (vcpu->arch.ciabr != host_ciabr)
+ mtspr(SPRN_CIABR, vcpu->arch.ciabr);
mtspr(SPRN_PSSCR, vcpu->arch.psscr | PSSCR_EC |
(local_paca->kvm_hstate.fake_suspend << PSSCR_FAKE_SUSPEND_LG));
@@ -881,20 +886,6 @@ int kvmhv_vcpu_entry_p9(struct kvm_vcpu *vcpu, u64 time_limit, unsigned long lpc
vc->dpdes = mfspr(SPRN_DPDES);
vc->vtb = mfspr(SPRN_VTB);
- save_clear_guest_mmu(kvm, vcpu);
- switch_mmu_to_host(kvm, host_pidr);
-
- /*
- * If we are in real mode, only switch MMU on after the MMU is
- * switched to host, to avoid the P9_RADIX_PREFETCH_BUG.
- */
- if (IS_ENABLED(CONFIG_PPC_TRANSACTIONAL_MEM) &&
- vcpu->arch.shregs.msr & MSR_TS_MASK)
- msr |= MSR_TS_S;
- __mtmsrd(msr, 0);
-
- store_vcpu_state(vcpu);
-
dec = mfspr(SPRN_DEC);
if (!(lpcr & LPCR_LD)) /* Sign extend if not using large decrementer */
dec = (s32) dec;
@@ -912,6 +903,22 @@ int kvmhv_vcpu_entry_p9(struct kvm_vcpu *vcpu, u64 time_limit, unsigned long lpc
vc->tb_offset_applied = 0;
}
+ save_clear_guest_mmu(kvm, vcpu);
+ switch_mmu_to_host(kvm, host_pidr);
+
+ /*
+ * Enable MSR here in order to have facilities enabled to save
+ * guest registers. This enables MMU (if we were in realmode), so
+ * only switch MMU on after the MMU is switched to host, to avoid
+ * the P9_RADIX_PREFETCH_BUG or hash guest context.
+ */
+ if (IS_ENABLED(CONFIG_PPC_TRANSACTIONAL_MEM) &&
+ vcpu->arch.shregs.msr & MSR_TS_MASK)
+ msr |= MSR_TS_S;
+ __mtmsrd(msr, 0);
+
+ store_vcpu_state(vcpu);
+
mtspr(SPRN_PURR, local_paca->kvm_hstate.host_purr);
mtspr(SPRN_SPURR, local_paca->kvm_hstate.host_spurr);
@@ -919,15 +926,21 @@ int kvmhv_vcpu_entry_p9(struct kvm_vcpu *vcpu, u64 time_limit, unsigned long lpc
mtspr(SPRN_PSSCR, host_psscr |
(local_paca->kvm_hstate.fake_suspend << PSSCR_FAKE_SUSPEND_LG));
mtspr(SPRN_HFSCR, host_hfscr);
- mtspr(SPRN_CIABR, host_ciabr);
- mtspr(SPRN_DAWR0, host_dawr0);
- mtspr(SPRN_DAWRX0, host_dawrx0);
+ if (vcpu->arch.ciabr != host_ciabr)
+ mtspr(SPRN_CIABR, host_ciabr);
+ if (vcpu->arch.dawr0 != host_dawr0)
+ mtspr(SPRN_DAWR0, host_dawr0);
+ if (vcpu->arch.dawrx0 != host_dawrx0)
+ mtspr(SPRN_DAWRX0, host_dawrx0);
if (cpu_has_feature(CPU_FTR_DAWR1)) {
- mtspr(SPRN_DAWR1, host_dawr1);
- mtspr(SPRN_DAWRX1, host_dawrx1);
+ if (vcpu->arch.dawr1 != host_dawr1)
+ mtspr(SPRN_DAWR1, host_dawr1);
+ if (vcpu->arch.dawrx1 != host_dawrx1)
+ mtspr(SPRN_DAWRX1, host_dawrx1);
}
- mtspr(SPRN_DPDES, 0);
+ if (vc->dpdes)
+ mtspr(SPRN_DPDES, 0);
if (vc->pcr)
mtspr(SPRN_PCR, PCR_MASK);
--
2.23.0
next prev parent reply other threads:[~2021-10-04 16:24 UTC|newest]
Thread overview: 62+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-10-04 15:59 [PATCH v3 00/52] KVM: PPC: Book3S HV P9: entry/exit optimisations Nicholas Piggin
2021-10-04 15:59 ` [PATCH v3 01/52] powerpc/64s: Remove WORT SPR from POWER9/10 (take 2) Nicholas Piggin
2021-10-04 15:59 ` [PATCH v3 02/52] powerpc/64s: guard optional TIDR SPR with CPU ftr test Nicholas Piggin
2021-10-11 18:44 ` Fabiano Rosas
2021-10-12 2:08 ` Michael Ellerman
2021-10-13 16:51 ` Fabiano Rosas
2021-10-04 16:00 ` [PATCH v3 03/52] KMV: PPC: Book3S HV P9: Use set_dec to set decrementer to host Nicholas Piggin
2021-10-04 16:00 ` [PATCH v3 04/52] KVM: PPC: Book3S HV P9: Use host timer accounting to avoid decrementer read Nicholas Piggin
2021-10-04 16:00 ` [PATCH v3 05/52] KVM: PPC: Book3S HV P9: Use large decrementer for HDEC Nicholas Piggin
2021-10-04 16:00 ` [PATCH v3 06/52] KVM: PPC: Book3S HV P9: Reduce mftb per guest entry/exit Nicholas Piggin
2021-10-04 16:00 ` [PATCH v3 07/52] powerpc/time: add API for KVM to re-arm the host timer/decrementer Nicholas Piggin
2021-10-04 16:00 ` [PATCH v3 08/52] KVM: PPC: Book3S HV: POWER10 enable HAIL when running radix guests Nicholas Piggin
2021-10-04 16:00 ` [PATCH v3 09/52] powerpc/64s: Keep AMOR SPR a constant ~0 at runtime Nicholas Piggin
2021-10-04 16:00 ` [PATCH v3 10/52] KVM: PPC: Book3S HV: Don't always save PMU for guest capable of nesting Nicholas Piggin
2021-10-16 12:38 ` Fabiano Rosas
2021-10-20 5:26 ` Nicholas Piggin
2021-10-04 16:00 ` [PATCH v3 11/52] powerpc/64s: Always set PMU control registers to frozen/disabled when not in use Nicholas Piggin
2021-10-04 16:00 ` [PATCH v3 12/52] powerpc/64s: Implement PMU override command line option Nicholas Piggin
2021-10-04 16:00 ` [PATCH v3 13/52] KVM: PPC: Book3S HV P9: Implement PMU save/restore in C Nicholas Piggin
2021-10-04 16:00 ` [PATCH v3 14/52] KVM: PPC: Book3S HV P9: Factor PMU save/load into context switch functions Nicholas Piggin
2021-10-04 16:00 ` [PATCH v3 15/52] KVM: PPC: Book3S HV P9: Demand fault PMU SPRs when marked not inuse Nicholas Piggin
2021-10-04 16:00 ` [PATCH v3 16/52] KVM: PPC: Book3S HV P9: Factor out yield_count increment Nicholas Piggin
2021-10-04 16:00 ` [PATCH v3 17/52] KVM: PPC: Book3S HV: CTRL SPR does not require read-modify-write Nicholas Piggin
2021-10-16 12:54 ` Fabiano Rosas
2021-10-04 16:00 ` [PATCH v3 18/52] KVM: PPC: Book3S HV P9: Move SPRG restore to restore_p9_host_os_sprs Nicholas Piggin
2021-10-16 12:59 ` Fabiano Rosas
2021-10-04 16:00 ` [PATCH v3 19/52] KVM: PPC: Book3S HV P9: Reduce mtmsrd instructions required to save host SPRs Nicholas Piggin
2021-10-16 13:45 ` Fabiano Rosas
2021-10-20 5:35 ` Nicholas Piggin
2021-10-04 16:00 ` [PATCH v3 20/52] KVM: PPC: Book3S HV P9: Improve mtmsrd scheduling by delaying MSR[EE] disable Nicholas Piggin
2021-10-04 16:00 ` [PATCH v3 21/52] KVM: PPC: Book3S HV P9: Add kvmppc_stop_thread to match kvmppc_start_thread Nicholas Piggin
2021-10-04 16:00 ` [PATCH v3 22/52] KVM: PPC: Book3S HV: Change dec_expires to be relative to guest timebase Nicholas Piggin
2021-10-04 16:00 ` [PATCH v3 23/52] KVM: PPC: Book3S HV P9: Move TB updates Nicholas Piggin
2021-10-04 16:00 ` [PATCH v3 24/52] KVM: PPC: Book3S HV P9: Optimise timebase reads Nicholas Piggin
2021-10-04 16:00 ` [PATCH v3 25/52] KVM: PPC: Book3S HV P9: Avoid SPR scoreboard stalls Nicholas Piggin
2021-10-04 16:00 ` [PATCH v3 26/52] KVM: PPC: Book3S HV P9: Only execute mtSPR if the value changed Nicholas Piggin
2021-10-04 16:00 ` [PATCH v3 27/52] KVM: PPC: Book3S HV P9: Juggle SPR switching around Nicholas Piggin
2021-10-04 16:00 ` [PATCH v3 28/52] KVM: PPC: Book3S HV P9: Move vcpu register save/restore into functions Nicholas Piggin
2021-10-04 16:00 ` [PATCH v3 29/52] KVM: PPC: Book3S HV P9: Move host OS save/restore functions to built-in Nicholas Piggin
2021-10-04 16:00 ` [PATCH v3 30/52] KVM: PPC: Book3S HV P9: Move nested guest entry into its own function Nicholas Piggin
2021-10-04 16:00 ` [PATCH v3 31/52] KVM: PPC: Book3S HV P9: Move remaining SPR and MSR access into low level entry Nicholas Piggin
2021-10-04 16:00 ` [PATCH v3 32/52] KVM: PPC: Book3S HV P9: Implement TM fastpath for guest entry/exit Nicholas Piggin
2021-10-04 16:00 ` [PATCH v3 33/52] KVM: PPC: Book3S HV P9: Switch PMU to guest as late as possible Nicholas Piggin
2021-10-04 16:00 ` [PATCH v3 34/52] KVM: PPC: Book3S HV P9: Restrict DSISR canary workaround to processors that require it Nicholas Piggin
2021-10-04 16:00 ` Nicholas Piggin [this message]
2021-10-04 16:00 ` [PATCH v3 36/52] KVM: PPC: Book3S HV P9: Demand fault EBB facility registers Nicholas Piggin
2021-10-04 16:00 ` [PATCH v3 37/52] KVM: PPC: Book3S HV P9: Demand fault TM " Nicholas Piggin
2021-10-04 16:00 ` [PATCH v3 38/52] KVM: PPC: Book3S HV P9: Use Linux SPR save/restore to manage some host SPRs Nicholas Piggin
2021-10-04 16:00 ` [PATCH v3 39/52] KVM: PPC: Book3S HV P9: Comment and fix MMU context switching code Nicholas Piggin
2021-10-04 16:00 ` [PATCH v3 40/52] KVM: PPC: Book3S HV P9: Test dawr_enabled() before saving host DAWR SPRs Nicholas Piggin
2021-10-04 16:00 ` [PATCH v3 41/52] KVM: PPC: Book3S HV P9: Don't restore PSSCR if not needed Nicholas Piggin
2021-10-04 16:00 ` [PATCH v3 42/52] KVM: PPC: Book3S HV P9: Avoid tlbsync sequence on radix guest exit Nicholas Piggin
2021-10-04 16:00 ` [PATCH v3 43/52] KVM: PPC: Book3S HV Nested: Avoid extra mftb() in nested entry Nicholas Piggin
2021-10-04 16:00 ` [PATCH v3 44/52] KVM: PPC: Book3S HV P9: Improve mfmsr performance on entry Nicholas Piggin
2021-10-04 16:00 ` [PATCH v3 45/52] KVM: PPC: Book3S HV P9: Optimise hash guest SLB saving Nicholas Piggin
2021-10-04 16:00 ` [PATCH v3 46/52] KVM: PPC: Book3S HV P9: Avoid changing MSR[RI] in entry and exit Nicholas Piggin
2021-10-04 16:00 ` [PATCH v3 47/52] KVM: PPC: Book3S HV P9: Add unlikely annotation for !mmu_ready Nicholas Piggin
2021-10-04 16:00 ` [PATCH v3 48/52] KVM: PPC: Book3S HV P9: Avoid cpu_in_guest atomics on entry and exit Nicholas Piggin
2021-10-04 16:00 ` [PATCH v3 49/52] KVM: PPC: Book3S HV P9: Remove most of the vcore logic Nicholas Piggin
2021-10-04 16:00 ` [PATCH v3 50/52] KVM: PPC: Book3S HV P9: Tidy kvmppc_create_dtl_entry Nicholas Piggin
2021-10-04 16:00 ` [PATCH v3 51/52] KVM: PPC: Book3S HV P9: Stop using vc->dpdes Nicholas Piggin
2021-10-04 16:00 ` [PATCH v3 52/52] KVM: PPC: Book3S HV P9: Remove subcore HMI handling Nicholas Piggin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20211004160049.1338837-36-npiggin@gmail.com \
--to=npiggin@gmail.com \
--cc=kvm-ppc@vger.kernel.org \
--cc=linuxppc-dev@lists.ozlabs.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).