From: Nicholas Piggin <npiggin@gmail.com>
To: linuxppc-dev@lists.ozlabs.org
Cc: Nicholas Piggin <npiggin@gmail.com>,
Fabiano Rosas <farosas@linux.ibm.com>
Subject: [PATCH v4 37/53] KVM: PPC: Book3S HV P9: Demand fault TM facility registers
Date: Tue, 23 Nov 2021 19:52:15 +1000 [thread overview]
Message-ID: <20211123095231.1036501-38-npiggin@gmail.com> (raw)
In-Reply-To: <20211123095231.1036501-1-npiggin@gmail.com>
Use HFSCR facility disabling to implement demand faulting for TM, with
a hysteresis counter similar to the load_fp etc counters in context
switching that implement the equivalent demand faulting for userspace
facilities.
This speeds up guest entry/exit by avoiding the register save/restore
when a guest is not frequently using them. When a guest does use them
often, there will be some additional demand fault overhead, but these
are not commonly used facilities.
Reviewed-by: Fabiano Rosas <farosas@linux.ibm.com>
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
---
arch/powerpc/include/asm/kvm_host.h | 3 +++
arch/powerpc/kvm/book3s_hv.c | 26 ++++++++++++++++++++------
arch/powerpc/kvm/book3s_hv_p9_entry.c | 15 +++++++++++----
3 files changed, 34 insertions(+), 10 deletions(-)
diff --git a/arch/powerpc/include/asm/kvm_host.h b/arch/powerpc/include/asm/kvm_host.h
index 7a55b19eb6c0..d7004412b859 100644
--- a/arch/powerpc/include/asm/kvm_host.h
+++ b/arch/powerpc/include/asm/kvm_host.h
@@ -580,6 +580,9 @@ struct kvm_vcpu_arch {
ulong ppr;
u32 pspb;
u8 load_ebb;
+#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
+ u8 load_tm;
+#endif
ulong fscr;
ulong shadow_fscr;
ulong ebbhr;
diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c
index da29cf9236c8..198f6d997330 100644
--- a/arch/powerpc/kvm/book3s_hv.c
+++ b/arch/powerpc/kvm/book3s_hv.c
@@ -1446,6 +1446,16 @@ static int kvmppc_ebb_unavailable(struct kvm_vcpu *vcpu)
return RESUME_GUEST;
}
+static int kvmppc_tm_unavailable(struct kvm_vcpu *vcpu)
+{
+ if (!(vcpu->arch.hfscr_permitted & HFSCR_TM))
+ return EMULATE_FAIL;
+
+ vcpu->arch.hfscr |= HFSCR_TM;
+
+ return RESUME_GUEST;
+}
+
static int kvmppc_handle_exit_hv(struct kvm_vcpu *vcpu,
struct task_struct *tsk)
{
@@ -1739,6 +1749,8 @@ static int kvmppc_handle_exit_hv(struct kvm_vcpu *vcpu,
r = kvmppc_pmu_unavailable(vcpu);
if (cause == FSCR_EBB_LG)
r = kvmppc_ebb_unavailable(vcpu);
+ if (cause == FSCR_TM_LG)
+ r = kvmppc_tm_unavailable(vcpu);
}
if (r == EMULATE_FAIL) {
kvmppc_core_queue_program(vcpu, SRR1_PROGILL);
@@ -2783,9 +2795,9 @@ static int kvmppc_core_vcpu_create_hv(struct kvm_vcpu *vcpu)
vcpu->arch.hfscr_permitted = vcpu->arch.hfscr;
/*
- * PM, EBB is demand-faulted so start with it clear.
+ * PM, EBB, TM are demand-faulted so start with it clear.
*/
- vcpu->arch.hfscr &= ~(HFSCR_PM | HFSCR_EBB);
+ vcpu->arch.hfscr &= ~(HFSCR_PM | HFSCR_EBB | HFSCR_TM);
kvmppc_mmu_book3s_hv_init(vcpu);
@@ -3868,8 +3880,9 @@ static int kvmhv_vcpu_entry_p9_nested(struct kvm_vcpu *vcpu, u64 time_limit, uns
msr |= MSR_VEC;
if (cpu_has_feature(CPU_FTR_VSX))
msr |= MSR_VSX;
- if (cpu_has_feature(CPU_FTR_TM) ||
- cpu_has_feature(CPU_FTR_P9_TM_HV_ASSIST))
+ if ((cpu_has_feature(CPU_FTR_TM) ||
+ cpu_has_feature(CPU_FTR_P9_TM_HV_ASSIST)) &&
+ (vcpu->arch.hfscr & HFSCR_TM))
msr |= MSR_TM;
msr = msr_check_and_set(msr);
@@ -4608,8 +4621,9 @@ static int kvmppc_vcpu_run_hv(struct kvm_vcpu *vcpu)
msr |= MSR_VEC;
if (cpu_has_feature(CPU_FTR_VSX))
msr |= MSR_VSX;
- if (cpu_has_feature(CPU_FTR_TM) ||
- cpu_has_feature(CPU_FTR_P9_TM_HV_ASSIST))
+ if ((cpu_has_feature(CPU_FTR_TM) ||
+ cpu_has_feature(CPU_FTR_P9_TM_HV_ASSIST)) &&
+ (vcpu->arch.hfscr & HFSCR_TM))
msr |= MSR_TM;
msr = msr_check_and_set(msr);
diff --git a/arch/powerpc/kvm/book3s_hv_p9_entry.c b/arch/powerpc/kvm/book3s_hv_p9_entry.c
index 929a7c336b09..8499e8a9ca8f 100644
--- a/arch/powerpc/kvm/book3s_hv_p9_entry.c
+++ b/arch/powerpc/kvm/book3s_hv_p9_entry.c
@@ -310,7 +310,7 @@ bool load_vcpu_state(struct kvm_vcpu *vcpu,
if (MSR_TM_ACTIVE(guest_msr)) {
kvmppc_restore_tm_hv(vcpu, guest_msr, true);
ret = true;
- } else {
+ } else if (vcpu->arch.hfscr & HFSCR_TM) {
mtspr(SPRN_TEXASR, vcpu->arch.texasr);
mtspr(SPRN_TFHAR, vcpu->arch.tfhar);
mtspr(SPRN_TFIAR, vcpu->arch.tfiar);
@@ -346,10 +346,16 @@ void store_vcpu_state(struct kvm_vcpu *vcpu)
unsigned long guest_msr = vcpu->arch.shregs.msr;
if (MSR_TM_ACTIVE(guest_msr)) {
kvmppc_save_tm_hv(vcpu, guest_msr, true);
- } else {
+ } else if (vcpu->arch.hfscr & HFSCR_TM) {
vcpu->arch.texasr = mfspr(SPRN_TEXASR);
vcpu->arch.tfhar = mfspr(SPRN_TFHAR);
vcpu->arch.tfiar = mfspr(SPRN_TFIAR);
+
+ if (!vcpu->arch.nested) {
+ vcpu->arch.load_tm++; /* see load_ebb comment */
+ if (!vcpu->arch.load_tm)
+ vcpu->arch.hfscr &= ~HFSCR_TM;
+ }
}
}
#endif
@@ -641,8 +647,9 @@ int kvmhv_vcpu_entry_p9(struct kvm_vcpu *vcpu, u64 time_limit, unsigned long lpc
msr |= MSR_VEC;
if (cpu_has_feature(CPU_FTR_VSX))
msr |= MSR_VSX;
- if (cpu_has_feature(CPU_FTR_TM) ||
- cpu_has_feature(CPU_FTR_P9_TM_HV_ASSIST))
+ if ((cpu_has_feature(CPU_FTR_TM) ||
+ cpu_has_feature(CPU_FTR_P9_TM_HV_ASSIST)) &&
+ (vcpu->arch.hfscr & HFSCR_TM))
msr |= MSR_TM;
msr = msr_check_and_set(msr);
/* Save MSR for restore. This is after hard disable, so EE is clear. */
--
2.23.0
next prev parent reply other threads:[~2021-11-23 10:17 UTC|newest]
Thread overview: 55+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-11-23 9:51 [PATCH v4 00/53] KVM: PPC: Book3S HV P9: entry/exit optimisations Nicholas Piggin
2021-11-23 9:51 ` [PATCH v4 01/53] powerpc/64s: Remove WORT SPR from POWER9/10 (take 2) Nicholas Piggin
2021-11-23 9:51 ` [PATCH v4 02/53] powerpc/64s: guard optional TIDR SPR with CPU ftr test Nicholas Piggin
2021-11-23 9:51 ` [PATCH v4 03/53] KMV: PPC: Book3S HV P9: Use set_dec to set decrementer to host Nicholas Piggin
2021-11-23 9:51 ` [PATCH v4 04/53] KVM: PPC: Book3S HV P9: Use host timer accounting to avoid decrementer read Nicholas Piggin
2021-11-23 9:51 ` [PATCH v4 05/53] KVM: PPC: Book3S HV P9: Use large decrementer for HDEC Nicholas Piggin
2021-11-23 9:51 ` [PATCH v4 06/53] KVM: PPC: Book3S HV P9: Reduce mftb per guest entry/exit Nicholas Piggin
2021-11-23 9:51 ` [PATCH v4 07/53] powerpc/time: add API for KVM to re-arm the host timer/decrementer Nicholas Piggin
2021-11-23 9:51 ` [PATCH v4 08/53] KVM: PPC: Book3S HV: POWER10 enable HAIL when running radix guests Nicholas Piggin
2021-11-23 9:51 ` [PATCH v4 09/53] powerpc/64s: Keep AMOR SPR a constant ~0 at runtime Nicholas Piggin
2021-11-23 9:51 ` [PATCH v4 10/53] KVM: PPC: Book3S HV: Don't always save PMU for guest capable of nesting Nicholas Piggin
2021-11-23 9:51 ` [PATCH v4 11/53] powerpc/64s: Always set PMU control registers to frozen/disabled when not in use Nicholas Piggin
2021-11-23 9:51 ` [PATCH v4 12/53] powerpc/64s: Implement PMU override command line option Nicholas Piggin
2021-11-23 9:51 ` [PATCH v4 13/53] KVM: PPC: Book3S HV P9: Implement PMU save/restore in C Nicholas Piggin
2021-11-23 9:51 ` [PATCH v4 14/53] KVM: PPC: Book3S HV P9: Factor PMU save/load into context switch functions Nicholas Piggin
2021-11-23 9:51 ` [PATCH v4 15/53] KVM: PPC: Book3S HV P9: Demand fault PMU SPRs when marked not inuse Nicholas Piggin
2021-11-23 9:51 ` [PATCH v4 16/53] KVM: PPC: Book3S HV P9: Factor out yield_count increment Nicholas Piggin
2021-11-23 9:51 ` [PATCH v4 17/53] KVM: PPC: Book3S HV: CTRL SPR does not require read-modify-write Nicholas Piggin
2021-11-23 9:51 ` [PATCH v4 18/53] KVM: PPC: Book3S HV P9: Move SPRG restore to restore_p9_host_os_sprs Nicholas Piggin
2021-11-23 9:51 ` [PATCH v4 19/53] KVM: PPC: Book3S HV P9: Reduce mtmsrd instructions required to save host SPRs Nicholas Piggin
2021-11-23 9:51 ` [PATCH v4 20/53] KVM: PPC: Book3S HV P9: Improve mtmsrd scheduling by delaying MSR[EE] disable Nicholas Piggin
2021-11-23 9:51 ` [PATCH v4 21/53] KVM: PPC: Book3S HV P9: Add kvmppc_stop_thread to match kvmppc_start_thread Nicholas Piggin
2021-11-23 9:52 ` [PATCH v4 22/53] KVM: PPC: Book3S HV: Change dec_expires to be relative to guest timebase Nicholas Piggin
2021-11-23 9:52 ` [PATCH v4 23/53] KVM: PPC: Book3S HV P9: Move TB updates Nicholas Piggin
2021-11-23 9:52 ` [PATCH v4 24/53] KVM: PPC: Book3S HV P9: Optimise timebase reads Nicholas Piggin
2021-11-23 9:52 ` [PATCH v4 25/53] KVM: PPC: Book3S HV P9: Avoid SPR scoreboard stalls Nicholas Piggin
2021-11-23 9:52 ` [PATCH v4 26/53] KVM: PPC: Book3S HV P9: Only execute mtSPR if the value changed Nicholas Piggin
2021-11-23 9:52 ` [PATCH v4 27/53] KVM: PPC: Book3S HV P9: Juggle SPR switching around Nicholas Piggin
2021-11-23 9:52 ` [PATCH v4 28/53] KVM: PPC: Book3S HV P9: Move vcpu register save/restore into functions Nicholas Piggin
2021-11-23 9:52 ` [PATCH v4 29/53] KVM: PPC: Book3S HV P9: Move host OS save/restore functions to built-in Nicholas Piggin
2021-11-23 9:52 ` [PATCH v4 30/53] KVM: PPC: Book3S HV P9: Move nested guest entry into its own function Nicholas Piggin
2021-11-23 9:52 ` [PATCH v4 31/53] KVM: PPC: Book3S HV P9: Move remaining SPR and MSR access into low level entry Nicholas Piggin
2021-11-23 9:52 ` [PATCH v4 32/53] KVM: PPC: Book3S HV P9: Implement TM fastpath for guest entry/exit Nicholas Piggin
2021-11-23 9:52 ` [PATCH v4 33/53] KVM: PPC: Book3S HV P9: Switch PMU to guest as late as possible Nicholas Piggin
2021-11-23 9:52 ` [PATCH v4 34/53] KVM: PPC: Book3S HV P9: Restrict DSISR canary workaround to processors that require it Nicholas Piggin
2021-11-23 9:52 ` [PATCH v4 35/53] KVM: PPC: Book3S HV P9: More SPR speed improvements Nicholas Piggin
2021-11-23 9:52 ` [PATCH v4 36/53] KVM: PPC: Book3S HV P9: Demand fault EBB facility registers Nicholas Piggin
2021-11-23 9:52 ` Nicholas Piggin [this message]
2021-11-23 9:52 ` [PATCH v4 38/53] KVM: PPC: Book3S HV P9: Use Linux SPR save/restore to manage some host SPRs Nicholas Piggin
2021-11-23 9:52 ` [PATCH v4 39/53] KVM: PPC: Book3S HV P9: Comment and fix MMU context switching code Nicholas Piggin
2021-11-23 9:52 ` [PATCH v4 40/53] KVM: PPC: Book3S HV P9: Test dawr_enabled() before saving host DAWR SPRs Nicholas Piggin
2021-11-23 9:52 ` [PATCH v4 41/53] KVM: PPC: Book3S HV P9: Don't restore PSSCR if not needed Nicholas Piggin
2021-11-23 9:52 ` [PATCH v4 42/53] KVM: PPC: Book3S HV: Split P8 from P9 path guest vCPU TLB flushing Nicholas Piggin
2021-11-23 9:52 ` [PATCH v4 43/53] KVM: PPC: Book3S HV P9: Avoid tlbsync sequence on radix guest exit Nicholas Piggin
2021-11-23 9:52 ` [PATCH v4 44/53] KVM: PPC: Book3S HV Nested: Avoid extra mftb() in nested entry Nicholas Piggin
2021-11-23 9:52 ` [PATCH v4 45/53] KVM: PPC: Book3S HV P9: Improve mfmsr performance on entry Nicholas Piggin
2021-11-23 9:52 ` [PATCH v4 46/53] KVM: PPC: Book3S HV P9: Optimise hash guest SLB saving Nicholas Piggin
2021-11-23 9:52 ` [PATCH v4 47/53] KVM: PPC: Book3S HV P9: Avoid changing MSR[RI] in entry and exit Nicholas Piggin
2021-11-23 9:52 ` [PATCH v4 48/53] KVM: PPC: Book3S HV P9: Add unlikely annotation for !mmu_ready Nicholas Piggin
2021-11-23 9:52 ` [PATCH v4 49/53] KVM: PPC: Book3S HV P9: Avoid cpu_in_guest atomics on entry and exit Nicholas Piggin
2021-11-23 9:52 ` [PATCH v4 50/53] KVM: PPC: Book3S HV P9: Remove most of the vcore logic Nicholas Piggin
2021-11-23 9:52 ` [PATCH v4 51/53] KVM: PPC: Book3S HV P9: Tidy kvmppc_create_dtl_entry Nicholas Piggin
2021-11-23 9:52 ` [PATCH v4 52/53] KVM: PPC: Book3S HV P9: Stop using vc->dpdes Nicholas Piggin
2021-11-23 9:52 ` [PATCH v4 53/53] KVM: PPC: Book3S HV P9: Remove subcore HMI handling Nicholas Piggin
2021-11-25 9:38 ` [PATCH v4 00/53] KVM: PPC: Book3S HV P9: entry/exit optimisations Michael Ellerman
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20211123095231.1036501-38-npiggin@gmail.com \
--to=npiggin@gmail.com \
--cc=farosas@linux.ibm.com \
--cc=linuxppc-dev@lists.ozlabs.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).