* [PATCH V3 0/2] KVM: PPC: Book3S HV: Support POWER9's large decrementer mode @ 2017-05-29 10:11 Paul Mackerras 2017-05-29 10:11 ` [PATCH V3 1/2] KVM: PPC: Book3S HV: Cope with host using " Paul Mackerras 2017-05-29 10:12 ` [PATCH V3 2/2] KVM: PPC: Book3S HV: Enable guests to use large decrementer mode on POWER9 Paul Mackerras 0 siblings, 2 replies; 4+ messages in thread From: Paul Mackerras @ 2017-05-29 10:11 UTC (permalink / raw) To: kvm; +Cc: kvm-ppc, linuxppc-dev One of the new features of POWER9 is that the decrementer (the facility that provides an interrupt after a programmable length of time) has been increased in size from 32 bits to 56 bits, allowing time intervals of up to about 814 days, compared to 4 seconds previously. This patch series adds support for the large decrementer mode to HV KVM. There is already code in the host kernel to enable large decrementer mode for the host, which means that some of the KVM entry/exit code is currently incorrect; the first patch fixes that. The second patch allows userspace to enable large decrementer mode for the guest, by setting the appropriate bit in the guest LPCR value. Changes in v3: Don't use up a CPU feature bit, since there is actually only one place that needs a feature section. For that, we can test the value of the LPCR_LD bit in the image of the host LPCR that is kept in the kvm struct. This means that this patch set now only touches KVM code. Paul. --- arch/powerpc/include/asm/kvm_host.h | 2 +- arch/powerpc/kvm/book3s_hv.c | 6 ++++ arch/powerpc/kvm/book3s_hv_interrupts.S | 12 +++++++- arch/powerpc/kvm/book3s_hv_rmhandlers.S | 50 ++++++++++++++++++++++++++------- arch/powerpc/kvm/emulate.c | 4 +-- 5 files changed, 60 insertions(+), 14 deletions(-) ^ permalink raw reply [flat|nested] 4+ messages in thread
* [PATCH V3 1/2] KVM: PPC: Book3S HV: Cope with host using large decrementer mode 2017-05-29 10:11 [PATCH V3 0/2] KVM: PPC: Book3S HV: Support POWER9's large decrementer mode Paul Mackerras @ 2017-05-29 10:11 ` Paul Mackerras 2017-05-29 10:12 ` [PATCH V3 2/2] KVM: PPC: Book3S HV: Enable guests to use large decrementer mode on POWER9 Paul Mackerras 1 sibling, 0 replies; 4+ messages in thread From: Paul Mackerras @ 2017-05-29 10:11 UTC (permalink / raw) To: kvm; +Cc: kvm-ppc, linuxppc-dev POWER9 introduces a new mode for the decrementer register, called large decrementer mode, in which the decrementer counter is 56 bits wide rather than 32, and reads are sign-extended rather than zero-extended. For the decrementer, this new mode is optional and controlled by a bit in the LPCR. The hypervisor decrementer (HDEC) is 56 bits wide on POWER9 and has no mode control. Since KVM code reads and writes the decrementer and hypervisor decrementer registers in a few places, it needs to be aware of the need to treat the decrementer value as a 64-bit quantity, and only do a 32-bit sign extension when large decrementer mode is not in effect. Similarly, the HDEC should always be treated as a 64-bit quantity on POWER9. We define a new EXTEND_HDEC macro to encapsulate the feature test for POWER9 and the sign extension. To enable the sign extension to be removed in large decrementer mode, we test the LPCR_LD bit in the host LPCR image stored in the struct kvm for the guest. If is set then large decrementer mode is enabled and the sign extension should be skipped. This is partly based on an earlier patch by Oliver O'Halloran. Cc: stable@vger.kernel.org # v4.10+ Signed-off-by: Paul Mackerras <paulus@ozlabs.org> --- arch/powerpc/kvm/book3s_hv_interrupts.S | 12 +++++++++++- arch/powerpc/kvm/book3s_hv_rmhandlers.S | 23 +++++++++++++++++------ 2 files changed, 28 insertions(+), 7 deletions(-) diff --git a/arch/powerpc/kvm/book3s_hv_interrupts.S b/arch/powerpc/kvm/book3s_hv_interrupts.S index 0fdc4a2..404deb5 100644 --- a/arch/powerpc/kvm/book3s_hv_interrupts.S +++ b/arch/powerpc/kvm/book3s_hv_interrupts.S @@ -121,10 +121,20 @@ END_FTR_SECTION_IFSET(CPU_FTR_ARCH_207S) * Put whatever is in the decrementer into the * hypervisor decrementer. */ +BEGIN_FTR_SECTION + ld r5, HSTATE_KVM_VCORE(r13) + ld r6, VCORE_KVM(r5) + ld r9, KVM_HOST_LPCR(r6) + andis. r9, r9, LPCR_LD@h +END_FTR_SECTION_IFSET(CPU_FTR_ARCH_300) mfspr r8,SPRN_DEC mftb r7 - mtspr SPRN_HDEC,r8 +BEGIN_FTR_SECTION + /* On POWER9, don't sign-extend if host LPCR[LD] bit is set */ + bne 32f +END_FTR_SECTION_IFSET(CPU_FTR_ARCH_300) extsw r8,r8 +32: mtspr SPRN_HDEC,r8 add r8,r8,r7 std r8,HSTATE_DECEXP(r13) diff --git a/arch/powerpc/kvm/book3s_hv_rmhandlers.S b/arch/powerpc/kvm/book3s_hv_rmhandlers.S index bdb3f76..e390b38 100644 --- a/arch/powerpc/kvm/book3s_hv_rmhandlers.S +++ b/arch/powerpc/kvm/book3s_hv_rmhandlers.S @@ -32,6 +32,12 @@ #include <asm/opal.h> #include <asm/xive-regs.h> +/* Sign-extend HDEC if not on POWER9 */ +#define EXTEND_HDEC(reg) \ +BEGIN_FTR_SECTION; \ + extsw reg, reg; \ +END_FTR_SECTION_IFCLR(CPU_FTR_ARCH_300) + #define VCPU_GPRS_TM(reg) (((reg) * ULONG_SIZE) + VCPU_GPR_TM) /* Values in HSTATE_NAPPING(r13) */ @@ -214,6 +220,8 @@ END_FTR_SECTION_IFSET(CPU_FTR_ARCH_207S) kvmppc_primary_no_guest: /* We handle this much like a ceded vcpu */ /* put the HDEC into the DEC, since HDEC interrupts don't wake us */ + /* HDEC may be larger than DEC for arch >= v3.00, but since the */ + /* HDEC value came from DEC in the first place, it will fit */ mfspr r3, SPRN_HDEC mtspr SPRN_DEC, r3 /* @@ -295,8 +303,9 @@ kvm_novcpu_wakeup: /* See if our timeslice has expired (HDEC is negative) */ mfspr r0, SPRN_HDEC + EXTEND_HDEC(r0) li r12, BOOK3S_INTERRUPT_HV_DECREMENTER - cmpwi r0, 0 + cmpdi r0, 0 blt kvm_novcpu_exit /* Got an IPI but other vcpus aren't yet exiting, must be a latecomer */ @@ -390,8 +399,8 @@ kvm_secondary_got_guest: lbz r4, HSTATE_PTID(r13) cmpwi r4, 0 bne 63f - lis r6, 0x7fff - ori r6, r6, 0xffff + LOAD_REG_ADDR(r6, decrementer_max) + ld r6, 0(r6) mtspr SPRN_HDEC, r6 /* and set per-LPAR registers, if doing dynamic micro-threading */ ld r6, HSTATE_SPLIT_MODE(r13) @@ -968,7 +977,8 @@ ALT_FTR_SECTION_END_IFCLR(CPU_FTR_ARCH_300) /* Check if HDEC expires soon */ mfspr r3, SPRN_HDEC - cmpwi r3, 512 /* 1 microsecond */ + EXTEND_HDEC(r3) + cmpdi r3, 512 /* 1 microsecond */ blt hdec_soon #ifdef CONFIG_KVM_XICS @@ -2366,12 +2376,13 @@ END_FTR_SECTION_IFSET(CPU_FTR_TM) mfspr r3, SPRN_DEC mfspr r4, SPRN_HDEC mftb r5 - cmpw r3, r4 + extsw r3, r3 + EXTEND_HDEC(r4) + cmpd r3, r4 ble 67f mtspr SPRN_DEC, r4 67: /* save expiry time of guest decrementer */ - extsw r3, r3 add r3, r3, r5 ld r4, HSTATE_KVM_VCPU(r13) ld r5, HSTATE_KVM_VCORE(r13) -- 2.7.4 ^ permalink raw reply related [flat|nested] 4+ messages in thread
* [PATCH V3 2/2] KVM: PPC: Book3S HV: Enable guests to use large decrementer mode on POWER9 2017-05-29 10:11 [PATCH V3 0/2] KVM: PPC: Book3S HV: Support POWER9's large decrementer mode Paul Mackerras 2017-05-29 10:11 ` [PATCH V3 1/2] KVM: PPC: Book3S HV: Cope with host using " Paul Mackerras @ 2017-05-29 10:12 ` Paul Mackerras 2017-06-01 6:26 ` Suraj Jitindar Singh 1 sibling, 1 reply; 4+ messages in thread From: Paul Mackerras @ 2017-05-29 10:12 UTC (permalink / raw) To: kvm; +Cc: kvm-ppc, linuxppc-dev This allows userspace (e.g. QEMU) to enable large decrementer mode for the guest when running on a POWER9 host, by setting the LPCR_LD bit in the guest LPCR value. With this, the guest exit code saves 64 bits of the guest DEC value on exit. Other places that use the guest DEC value check the LPCR_LD bit in the guest LPCR value, and if it is set, omit the 32-bit sign extension that would otherwise be done. This doesn't change the DEC emulation used by PR KVM because PR KVM is not supported on POWER9 yet. This is partly based on an earlier patch by Oliver O'Halloran. Signed-off-by: Paul Mackerras <paulus@ozlabs.org> --- arch/powerpc/include/asm/kvm_host.h | 2 +- arch/powerpc/kvm/book3s_hv.c | 6 ++++++ arch/powerpc/kvm/book3s_hv_rmhandlers.S | 29 ++++++++++++++++++++++++----- arch/powerpc/kvm/emulate.c | 4 ++-- 4 files changed, 33 insertions(+), 8 deletions(-) diff --git a/arch/powerpc/include/asm/kvm_host.h b/arch/powerpc/include/asm/kvm_host.h index 9c51ac4..3f879c8 100644 --- a/arch/powerpc/include/asm/kvm_host.h +++ b/arch/powerpc/include/asm/kvm_host.h @@ -579,7 +579,7 @@ struct kvm_vcpu_arch { ulong mcsrr0; ulong mcsrr1; ulong mcsr; - u32 dec; + ulong dec; #ifdef CONFIG_BOOKE u32 decar; #endif diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c index 42b7a4f..9b2eb66 100644 --- a/arch/powerpc/kvm/book3s_hv.c +++ b/arch/powerpc/kvm/book3s_hv.c @@ -1143,6 +1143,12 @@ static void kvmppc_set_lpcr(struct kvm_vcpu *vcpu, u64 new_lpcr, mask = LPCR_DPFD | LPCR_ILE | LPCR_TC; if (cpu_has_feature(CPU_FTR_ARCH_207S)) mask |= LPCR_AIL; + /* + * On POWER9, allow userspace to enable large decrementer for the + * guest, whether or not the host has it enabled. + */ + if (cpu_has_feature(CPU_FTR_ARCH_300)) + mask |= LPCR_LD; /* Broken 32-bit version of LPCR must not clear top bits */ if (preserve_top32) diff --git a/arch/powerpc/kvm/book3s_hv_rmhandlers.S b/arch/powerpc/kvm/book3s_hv_rmhandlers.S index e390b38..3c901b5 100644 --- a/arch/powerpc/kvm/book3s_hv_rmhandlers.S +++ b/arch/powerpc/kvm/book3s_hv_rmhandlers.S @@ -920,7 +920,7 @@ ALT_FTR_SECTION_END_IFCLR(CPU_FTR_ARCH_300) mftb r7 subf r3,r7,r8 mtspr SPRN_DEC,r3 - stw r3,VCPU_DEC(r4) + std r3,VCPU_DEC(r4) ld r5, VCPU_SPRG0(r4) ld r6, VCPU_SPRG1(r4) @@ -1032,7 +1032,13 @@ kvmppc_cede_reentry: /* r4 = vcpu, r13 = paca */ li r0, BOOK3S_INTERRUPT_EXTERNAL bne cr1, 12f mfspr r0, SPRN_DEC - cmpwi r0, 0 +BEGIN_FTR_SECTION + /* On POWER9 check whether the guest has large decrementer enabled */ + andis. r8, r8, LPCR_LD@h + bne 15f +END_FTR_SECTION_IFSET(CPU_FTR_ARCH_300) + extsw r0, r0 +15: cmpdi r0, 0 li r0, BOOK3S_INTERRUPT_DECREMENTER bge 5f @@ -1459,12 +1465,18 @@ mc_cont: mtspr SPRN_SPURR,r4 /* Save DEC */ + ld r3, HSTATE_KVM_VCORE(r13) mfspr r5,SPRN_DEC mftb r6 + /* On P9, if the guest has large decr enabled, don't sign extend */ +BEGIN_FTR_SECTION + ld r4, VCORE_LPCR(r3) + andis. r4, r4, LPCR_LD@h + bne 16f +END_FTR_SECTION_IFSET(CPU_FTR_ARCH_300) extsw r5,r5 - add r5,r5,r6 +16: add r5,r5,r6 /* r5 is a guest timebase value here, convert to host TB */ - ld r3,HSTATE_KVM_VCORE(r13) ld r4,VCORE_TB_OFFSET(r3) subf r5,r4,r5 std r5,VCPU_DEC_EXPIRES(r9) @@ -2376,8 +2388,15 @@ END_FTR_SECTION_IFSET(CPU_FTR_TM) mfspr r3, SPRN_DEC mfspr r4, SPRN_HDEC mftb r5 +BEGIN_FTR_SECTION + /* On P9 check whether the guest has large decrementer mode enabled */ + ld r6, HSTATE_KVM_VCORE(r13) + ld r6, VCORE_LPCR(r6) + andis. r6, r6, LPCR_LD@h + bne 68f +END_FTR_SECTION_IFSET(CPU_FTR_ARCH_300) extsw r3, r3 - EXTEND_HDEC(r4) +68: EXTEND_HDEC(r4) cmpd r3, r4 ble 67f mtspr SPRN_DEC, r4 diff --git a/arch/powerpc/kvm/emulate.c b/arch/powerpc/kvm/emulate.c index c873ffe..4d8b4d6 100644 --- a/arch/powerpc/kvm/emulate.c +++ b/arch/powerpc/kvm/emulate.c @@ -39,7 +39,7 @@ void kvmppc_emulate_dec(struct kvm_vcpu *vcpu) unsigned long dec_nsec; unsigned long long dec_time; - pr_debug("mtDEC: %x\n", vcpu->arch.dec); + pr_debug("mtDEC: %lx\n", vcpu->arch.dec); hrtimer_try_to_cancel(&vcpu->arch.dec_timer); #ifdef CONFIG_PPC_BOOK3S @@ -109,7 +109,7 @@ static int kvmppc_emulate_mtspr(struct kvm_vcpu *vcpu, int sprn, int rs) case SPRN_TBWU: break; case SPRN_DEC: - vcpu->arch.dec = spr_val; + vcpu->arch.dec = (u32) spr_val; kvmppc_emulate_dec(vcpu); break; -- 2.7.4 ^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: [PATCH V3 2/2] KVM: PPC: Book3S HV: Enable guests to use large decrementer mode on POWER9 2017-05-29 10:12 ` [PATCH V3 2/2] KVM: PPC: Book3S HV: Enable guests to use large decrementer mode on POWER9 Paul Mackerras @ 2017-06-01 6:26 ` Suraj Jitindar Singh 0 siblings, 0 replies; 4+ messages in thread From: Suraj Jitindar Singh @ 2017-06-01 6:26 UTC (permalink / raw) To: Paul Mackerras, kvm; +Cc: kvm-ppc, linuxppc-dev On Mon, 2017-05-29 at 20:12 +1000, Paul Mackerras wrote: > This allows userspace (e.g. QEMU) to enable large decrementer mode > for > the guest when running on a POWER9 host, by setting the LPCR_LD bit > in > the guest LPCR value. With this, the guest exit code saves 64 bits > of > the guest DEC value on exit. Other places that use the guest DEC > value check the LPCR_LD bit in the guest LPCR value, and if it is > set, > omit the 32-bit sign extension that would otherwise be done. > > This doesn't change the DEC emulation used by PR KVM because PR KVM > is not supported on POWER9 yet. > > This is partly based on an earlier patch by Oliver O'Halloran. > > Signed-off-by: Paul Mackerras <paulus@ozlabs.org> Tested with a hacked up qemu and upstream guest/host (with these patches). Tested-by: Suraj Jitindar Singh <sjitindarsingh@gmail.com> > --- > arch/powerpc/include/asm/kvm_host.h | 2 +- > arch/powerpc/kvm/book3s_hv.c | 6 ++++++ > arch/powerpc/kvm/book3s_hv_rmhandlers.S | 29 > ++++++++++++++++++++++++----- > arch/powerpc/kvm/emulate.c | 4 ++-- > 4 files changed, 33 insertions(+), 8 deletions(-) > > diff --git a/arch/powerpc/include/asm/kvm_host.h > b/arch/powerpc/include/asm/kvm_host.h > index 9c51ac4..3f879c8 100644 > --- a/arch/powerpc/include/asm/kvm_host.h > +++ b/arch/powerpc/include/asm/kvm_host.h > @@ -579,7 +579,7 @@ struct kvm_vcpu_arch { > ulong mcsrr0; > ulong mcsrr1; > ulong mcsr; > - u32 dec; > + ulong dec; > #ifdef CONFIG_BOOKE > u32 decar; > #endif > diff --git a/arch/powerpc/kvm/book3s_hv.c > b/arch/powerpc/kvm/book3s_hv.c > index 42b7a4f..9b2eb66 100644 > --- a/arch/powerpc/kvm/book3s_hv.c > +++ b/arch/powerpc/kvm/book3s_hv.c > @@ -1143,6 +1143,12 @@ static void kvmppc_set_lpcr(struct kvm_vcpu > *vcpu, u64 new_lpcr, > mask = LPCR_DPFD | LPCR_ILE | LPCR_TC; > if (cpu_has_feature(CPU_FTR_ARCH_207S)) > mask |= LPCR_AIL; > + /* > + * On POWER9, allow userspace to enable large decrementer > for the > + * guest, whether or not the host has it enabled. > + */ > + if (cpu_has_feature(CPU_FTR_ARCH_300)) > + mask |= LPCR_LD; > > /* Broken 32-bit version of LPCR must not clear top bits */ > if (preserve_top32) > diff --git a/arch/powerpc/kvm/book3s_hv_rmhandlers.S > b/arch/powerpc/kvm/book3s_hv_rmhandlers.S > index e390b38..3c901b5 100644 > --- a/arch/powerpc/kvm/book3s_hv_rmhandlers.S > +++ b/arch/powerpc/kvm/book3s_hv_rmhandlers.S > @@ -920,7 +920,7 @@ ALT_FTR_SECTION_END_IFCLR(CPU_FTR_ARCH_300) > mftb r7 > subf r3,r7,r8 > mtspr SPRN_DEC,r3 > - stw r3,VCPU_DEC(r4) > + std r3,VCPU_DEC(r4) > > ld r5, VCPU_SPRG0(r4) > ld r6, VCPU_SPRG1(r4) > @@ -1032,7 +1032,13 @@ kvmppc_cede_reentry: /* r4 = > vcpu, r13 = paca */ > li r0, BOOK3S_INTERRUPT_EXTERNAL > bne cr1, 12f > mfspr r0, SPRN_DEC > - cmpwi r0, 0 > +BEGIN_FTR_SECTION > + /* On POWER9 check whether the guest has large decrementer > enabled */ > + andis. r8, r8, LPCR_LD@h > + bne 15f > +END_FTR_SECTION_IFSET(CPU_FTR_ARCH_300) > + extsw r0, r0 > +15: cmpdi r0, 0 > li r0, BOOK3S_INTERRUPT_DECREMENTER > bge 5f > > @@ -1459,12 +1465,18 @@ mc_cont: > mtspr SPRN_SPURR,r4 > > /* Save DEC */ > + ld r3, HSTATE_KVM_VCORE(r13) > mfspr r5,SPRN_DEC > mftb r6 > + /* On P9, if the guest has large decr enabled, don't sign > extend */ > +BEGIN_FTR_SECTION > + ld r4, VCORE_LPCR(r3) > + andis. r4, r4, LPCR_LD@h > + bne 16f > +END_FTR_SECTION_IFSET(CPU_FTR_ARCH_300) > extsw r5,r5 > - add r5,r5,r6 > +16: add r5,r5,r6 > /* r5 is a guest timebase value here, convert to host TB */ > - ld r3,HSTATE_KVM_VCORE(r13) > ld r4,VCORE_TB_OFFSET(r3) > subf r5,r4,r5 > std r5,VCPU_DEC_EXPIRES(r9) > @@ -2376,8 +2388,15 @@ END_FTR_SECTION_IFSET(CPU_FTR_TM) > mfspr r3, SPRN_DEC > mfspr r4, SPRN_HDEC > mftb r5 > +BEGIN_FTR_SECTION > + /* On P9 check whether the guest has large decrementer mode > enabled */ > + ld r6, HSTATE_KVM_VCORE(r13) > + ld r6, VCORE_LPCR(r6) > + andis. r6, r6, LPCR_LD@h > + bne 68f > +END_FTR_SECTION_IFSET(CPU_FTR_ARCH_300) > extsw r3, r3 > - EXTEND_HDEC(r4) > +68: EXTEND_HDEC(r4) > cmpd r3, r4 > ble 67f > mtspr SPRN_DEC, r4 > diff --git a/arch/powerpc/kvm/emulate.c b/arch/powerpc/kvm/emulate.c > index c873ffe..4d8b4d6 100644 > --- a/arch/powerpc/kvm/emulate.c > +++ b/arch/powerpc/kvm/emulate.c > @@ -39,7 +39,7 @@ void kvmppc_emulate_dec(struct kvm_vcpu *vcpu) > unsigned long dec_nsec; > unsigned long long dec_time; > > - pr_debug("mtDEC: %x\n", vcpu->arch.dec); > + pr_debug("mtDEC: %lx\n", vcpu->arch.dec); > hrtimer_try_to_cancel(&vcpu->arch.dec_timer); > > #ifdef CONFIG_PPC_BOOK3S > @@ -109,7 +109,7 @@ static int kvmppc_emulate_mtspr(struct kvm_vcpu > *vcpu, int sprn, int rs) > case SPRN_TBWU: break; > > case SPRN_DEC: > - vcpu->arch.dec = spr_val; > + vcpu->arch.dec = (u32) spr_val; > kvmppc_emulate_dec(vcpu); > break; > ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2017-06-01 6:26 UTC | newest] Thread overview: 4+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2017-05-29 10:11 [PATCH V3 0/2] KVM: PPC: Book3S HV: Support POWER9's large decrementer mode Paul Mackerras 2017-05-29 10:11 ` [PATCH V3 1/2] KVM: PPC: Book3S HV: Cope with host using " Paul Mackerras 2017-05-29 10:12 ` [PATCH V3 2/2] KVM: PPC: Book3S HV: Enable guests to use large decrementer mode on POWER9 Paul Mackerras 2017-06-01 6:26 ` Suraj Jitindar Singh
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).