From: Alexey Kardashevskiy <aik@ozlabs.ru>
To: Nicholas Piggin <npiggin@gmail.com>, kvm-ppc@vger.kernel.org
Cc: linuxppc-dev@lists.ozlabs.org
Subject: Re: [PATCH v6 30/48] KVM: PPC: Book3S HV P9: Implement the rest of the P9 path in C
Date: Fri, 09 Apr 2021 03:57:22 +0000 [thread overview]
Message-ID: <e3be3cb3-23a1-2e28-1bda-a7437f2fd2ca@ozlabs.ru> (raw)
In-Reply-To: <20210405011948.675354-31-npiggin@gmail.com>
On 05/04/2021 11:19, Nicholas Piggin wrote:
> Almost all logic is moved to C, by introducing a new in_guest mode for
> the P9 path that branches very early in the KVM interrupt handler to
> P9 exit code.
>
> The main P9 entry and exit assembly is now only about 160 lines of low
> level stack setup and register save/restore, plus a bad-interrupt
> handler.
>
> There are two motivations for this, the first is just make the code more
> maintainable being in C. The second is to reduce the amount of code
> running in a special KVM mode, "realmode". In quotes because with radix
> it is no longer necessarily real-mode in the MMU, but it still has to be
> treated specially because it may be in real-mode, and has various
> important registers like PID, DEC, TB, etc set to guest. This is hostile
> to the rest of Linux and can't use arbitrary kernel functionality or be
> instrumented well.
>
> This initial patch is a reasonably faithful conversion of the asm code,
> but it does lack any loop to return quickly back into the guest without
> switching out of realmode in the case of unimportant or easily handled
> interrupts. As explained in previous changes, handling HV interrupts
> in real mode is not so important for P9.
>
> Use of Linux 64s interrupt entry code register conventions including
> paca EX_ save areas are brought into the KVM code. There is no point
> shuffling things into different paca save areas and making up a
> different calling convention for KVM.
>
> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
> ---
> arch/powerpc/include/asm/asm-prototypes.h | 3 +-
> arch/powerpc/include/asm/kvm_asm.h | 3 +-
> arch/powerpc/include/asm/kvm_book3s_64.h | 8 +
> arch/powerpc/include/asm/kvm_host.h | 7 +-
> arch/powerpc/kernel/security.c | 5 +-
> arch/powerpc/kvm/Makefile | 1 +
> arch/powerpc/kvm/book3s_64_entry.S | 247 ++++++++++++++++++++++
> arch/powerpc/kvm/book3s_hv.c | 9 +-
> arch/powerpc/kvm/book3s_hv_interrupt.c | 218 +++++++++++++++++++
> arch/powerpc/kvm/book3s_hv_rmhandlers.S | 125 +----------
> 10 files changed, 501 insertions(+), 125 deletions(-)
> create mode 100644 arch/powerpc/kvm/book3s_hv_interrupt.c
>
> diff --git a/arch/powerpc/include/asm/asm-prototypes.h b/arch/powerpc/include/asm/asm-prototypes.h
> index 939f3c94c8f3..7c74c80ed994 100644
> --- a/arch/powerpc/include/asm/asm-prototypes.h
> +++ b/arch/powerpc/include/asm/asm-prototypes.h
> @@ -122,6 +122,7 @@ extern s32 patch__call_flush_branch_caches3;
> extern s32 patch__flush_count_cache_return;
> extern s32 patch__flush_link_stack_return;
> extern s32 patch__call_kvm_flush_link_stack;
> +extern s32 patch__call_kvm_flush_link_stack_p9;
> extern s32 patch__memset_nocache, patch__memcpy_nocache;
>
> extern long flush_branch_caches;
> @@ -142,7 +143,7 @@ void kvmhv_load_host_pmu(void);
> void kvmhv_save_guest_pmu(struct kvm_vcpu *vcpu, bool pmu_in_use);
> void kvmhv_load_guest_pmu(struct kvm_vcpu *vcpu);
>
> -int __kvmhv_vcpu_entry_p9(struct kvm_vcpu *vcpu);
> +void kvmppc_p9_enter_guest(struct kvm_vcpu *vcpu);
>
> long kvmppc_h_set_dabr(struct kvm_vcpu *vcpu, unsigned long dabr);
> long kvmppc_h_set_xdabr(struct kvm_vcpu *vcpu, unsigned long dabr,
> diff --git a/arch/powerpc/include/asm/kvm_asm.h b/arch/powerpc/include/asm/kvm_asm.h
> index a3633560493b..b4f9996bd331 100644
> --- a/arch/powerpc/include/asm/kvm_asm.h
> +++ b/arch/powerpc/include/asm/kvm_asm.h
> @@ -146,7 +146,8 @@
> #define KVM_GUEST_MODE_GUEST 1
> #define KVM_GUEST_MODE_SKIP 2
> #define KVM_GUEST_MODE_GUEST_HV 3
> -#define KVM_GUEST_MODE_HOST_HV 4
> +#define KVM_GUEST_MODE_GUEST_HV_FAST 4 /* ISA v3.0 with host radix mode */
> +#define KVM_GUEST_MODE_HOST_HV 5
>
> #define KVM_INST_FETCH_FAILED -1
>
> diff --git a/arch/powerpc/include/asm/kvm_book3s_64.h b/arch/powerpc/include/asm/kvm_book3s_64.h
> index 9bb9bb370b53..c214bcffb441 100644
> --- a/arch/powerpc/include/asm/kvm_book3s_64.h
> +++ b/arch/powerpc/include/asm/kvm_book3s_64.h
> @@ -153,9 +153,17 @@ static inline bool kvmhv_vcpu_is_radix(struct kvm_vcpu *vcpu)
> return radix;
> }
>
> +int __kvmhv_vcpu_entry_p9(struct kvm_vcpu *vcpu);
> +
> #define KVM_DEFAULT_HPT_ORDER 24 /* 16MB HPT by default */
> #endif
>
> +/*
> + * Invalid HDSISR value which is used to indicate when HW has not set the reg.
> + * Used to work around an errata.
> + */
> +#define HDSISR_CANARY 0x7fff
> +
> /*
> * We use a lock bit in HPTE dword 0 to synchronize updates and
> * accesses to each HPTE, and another bit to indicate non-present
> diff --git a/arch/powerpc/include/asm/kvm_host.h b/arch/powerpc/include/asm/kvm_host.h
> index 05fb00d37609..fa0083345b11 100644
> --- a/arch/powerpc/include/asm/kvm_host.h
> +++ b/arch/powerpc/include/asm/kvm_host.h
> @@ -690,7 +690,12 @@ struct kvm_vcpu_arch {
> ulong fault_dar;
> u32 fault_dsisr;
> unsigned long intr_msr;
> - ulong fault_gpa; /* guest real address of page fault (POWER9) */
> + /*
> + * POWER9 and later, fault_gpa contains the guest real address of page
> + * fault for a radix guest, or segment descriptor (equivalent to result
> + * from slbmfev of SLB entry that translated the EA) for hash guests.
> + */
> + ulong fault_gpa;
> #endif
>
> #ifdef CONFIG_BOOKE
> diff --git a/arch/powerpc/kernel/security.c b/arch/powerpc/kernel/security.c
> index e4e1a94ccf6a..3a607c11f20f 100644
> --- a/arch/powerpc/kernel/security.c
> +++ b/arch/powerpc/kernel/security.c
> @@ -430,16 +430,19 @@ device_initcall(stf_barrier_debugfs_init);
>
> static void update_branch_cache_flush(void)
> {
> - u32 *site;
> + u32 *site, __maybe_unused *site2;
>
> #ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
> site = &patch__call_kvm_flush_link_stack;
> + site2 = &patch__call_kvm_flush_link_stack_p9;
> // This controls the branch from guest_exit_cont to kvm_flush_link_stack
> if (link_stack_flush_type = BRANCH_CACHE_FLUSH_NONE) {
> patch_instruction_site(site, ppc_inst(PPC_INST_NOP));
> + patch_instruction_site(site2, ppc_inst(PPC_INST_NOP));
> } else {
> // Could use HW flush, but that could also flush count cache
> patch_branch_site(site, (u64)&kvm_flush_link_stack, BRANCH_SET_LINK);
> + patch_branch_site(site2, (u64)&kvm_flush_link_stack, BRANCH_SET_LINK);
> }
> #endif
>
> diff --git a/arch/powerpc/kvm/Makefile b/arch/powerpc/kvm/Makefile
> index cdd119028f64..ca7c86aa9360 100644
> --- a/arch/powerpc/kvm/Makefile
> +++ b/arch/powerpc/kvm/Makefile
> @@ -88,6 +88,7 @@ kvm-book3s_64-builtin-tm-objs-$(CONFIG_PPC_TRANSACTIONAL_MEM) += \
>
> ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
> kvm-book3s_64-builtin-objs-$(CONFIG_KVM_BOOK3S_64_HANDLER) += \
> + book3s_hv_interrupt.o \
> book3s_hv_hmi.o \
> book3s_hv_rmhandlers.o \
> book3s_hv_rm_mmu.o \
> diff --git a/arch/powerpc/kvm/book3s_64_entry.S b/arch/powerpc/kvm/book3s_64_entry.S
> index 0c79c89c6a4b..d98ad580fd98 100644
> --- a/arch/powerpc/kvm/book3s_64_entry.S
> +++ b/arch/powerpc/kvm/book3s_64_entry.S
> @@ -1,11 +1,16 @@
> /* SPDX-License-Identifier: GPL-2.0-only */
> #include <asm/asm-offsets.h>
> #include <asm/cache.h>
> +#include <asm/code-patching-asm.h>
> #include <asm/exception-64s.h>
> +#include <asm/export.h>
> #include <asm/kvm_asm.h>
> #include <asm/kvm_book3s_asm.h>
> +#include <asm/mmu.h>
> #include <asm/ppc_asm.h>
> +#include <asm/ptrace.h>
> #include <asm/reg.h>
> +#include <asm/ultravisor-api.h>
>
> /*
> * These are branched to from interrupt handlers in exception-64s.S which set
> @@ -29,10 +34,15 @@
> .global kvmppc_hcall
> .balign IFETCH_ALIGN_BYTES
> kvmppc_hcall:
> + lbz r10,HSTATE_IN_GUEST(r13)
> + cmpwi r10,KVM_GUEST_MODE_GUEST_HV_FAST
> + beq kvmppc_p9_exit_hcall
> ld r10,PACA_EXGEN+EX_R13(r13)
> SET_SCRATCH0(r10)
> li r10,0xc00
> /* Now we look like kvmppc_interrupt */
> + li r11,PACA_EXGEN
> + b 1f
>
> /*
> * KVM interrupt entry occurs after GEN_INT_ENTRY runs, and follows that
> @@ -53,6 +63,12 @@ kvmppc_hcall:
> .global kvmppc_interrupt
> .balign IFETCH_ALIGN_BYTES
> kvmppc_interrupt:
> + std r10,HSTATE_SCRATCH0(r13)
> + lbz r10,HSTATE_IN_GUEST(r13)
> + cmpwi r10,KVM_GUEST_MODE_GUEST_HV_FAST
> + beq kvmppc_p9_exit_interrupt
> + ld r10,HSTATE_SCRATCH0(r13)
> + lbz r11,HSTATE_IN_GUEST(r13)
> li r11,PACA_EXGEN
> cmpdi r10,0x200
> bgt+ 1f
> @@ -154,3 +170,234 @@ END_FTR_SECTION_IFSET(CPU_FTR_HAS_PPR)
> GET_SCRATCH0(r13)
> HRFI_TO_KERNEL
> #endif
> +
> +/* Stack frame offsets for kvmppc_hv_entry */
> +#define SFS (144 + STACK_FRAME_MIN_SIZE)
> +#define STACK_SLOT_NVGPRS (SFS - 144) /* 18 gprs */
> +
> +/*
> + * void kvmppc_p9_enter_guest(struct vcpu *vcpu);
> + *
> + * Enter the guest on a ISAv3.0 or later system where we have exactly
> + * one vcpu per vcore, and both the host and guest are radix, and threads
> + * are set to "indepdent mode".
> + */
> +.balign IFETCH_ALIGN_BYTES
> +_GLOBAL(kvmppc_p9_enter_guest)
> +EXPORT_SYMBOL_GPL(kvmppc_p9_enter_guest)
> + mflr r0
> + std r0,PPC_LR_STKOFF(r1)
> + stdu r1,-SFS(r1)
> +
> + std r1,HSTATE_HOST_R1(r13)
> +
> + mfcr r4
> + stw r4,SFS+8(r1)
> +
> + reg = 14
> + .rept 18
> + std reg,STACK_SLOT_NVGPRS + ((reg - 14) * 8)(r1)
> + reg = reg + 1
> + .endr
> +
> + ld r4,VCPU_LR(r3)
> + mtlr r4
> + ld r4,VCPU_CTR(r3)
> + mtctr r4
> + ld r4,VCPU_XER(r3)
> + mtspr SPRN_XER,r4
> +
> + ld r1,VCPU_CR(r3)
> +
> +BEGIN_FTR_SECTION
> + ld r4,VCPU_CFAR(r3)
> + mtspr SPRN_CFAR,r4
> +END_FTR_SECTION_IFSET(CPU_FTR_CFAR)
> +BEGIN_FTR_SECTION
> + ld r4,VCPU_PPR(r3)
> + mtspr SPRN_PPR,r4
> +END_FTR_SECTION_IFSET(CPU_FTR_HAS_PPR)
> +
> + reg = 4
> + .rept 28
> + ld reg,__VCPU_GPR(reg)(r3)
> + reg = reg + 1
> + .endr
> +
> + ld r4,VCPU_KVM(r3)
> + lbz r4,KVM_SECURE_GUEST(r4)
This does not compile when CONFIG_KVM_BOOK3S_HV_POSSIBLE is not defined.
--
Alexey
next prev parent reply other threads:[~2021-04-09 3:57 UTC|newest]
Thread overview: 65+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-04-05 1:19 [PATCH v6 00/48] KVM: PPC: Book3S: C-ify the P9 entry/exit code Nicholas Piggin
2021-04-05 1:19 ` [PATCH v6 01/48] KVM: PPC: Book3S HV: Nested move LPCR sanitising to sanitise_hv_regs Nicholas Piggin
2021-04-05 1:19 ` [PATCH v6 02/48] KVM: PPC: Book3S HV: Add a function to filter guest LPCR bits Nicholas Piggin
2021-04-05 1:19 ` [PATCH v6 03/48] KVM: PPC: Book3S HV: Disallow LPCR[AIL] to be set to 1 or 2 Nicholas Piggin
2021-04-05 1:19 ` [PATCH v6 04/48] KVM: PPC: Book3S HV: Prevent radix guests setting LPCR[TC] Nicholas Piggin
2021-04-05 1:19 ` [PATCH v6 05/48] KVM: PPC: Book3S HV: Remove redundant mtspr PSPB Nicholas Piggin
2021-04-05 1:19 ` [PATCH v6 06/48] KVM: PPC: Book3S HV: remove unused kvmppc_h_protect argument Nicholas Piggin
2021-04-05 1:19 ` [PATCH v6 07/48] KVM: PPC: Book3S HV: Fix CONFIG_SPAPR_TCE_IOMMU=n default hcalls Nicholas Piggin
2021-04-05 1:19 ` [PATCH v6 08/48] powerpc/64s: Remove KVM handler support from CBE_RAS interrupts Nicholas Piggin
2021-04-05 1:19 ` [PATCH v6 09/48] powerpc/64s: remove KVM SKIP test from instruction breakpoint handler Nicholas Piggin
2021-04-06 2:44 ` Nicholas Piggin
2021-04-06 4:17 ` Paul Mackerras
2021-04-05 1:19 ` [PATCH v6 10/48] KVM: PPC: Book3S HV: Ensure MSR[ME] is always set in guest MSR Nicholas Piggin
2021-04-05 1:19 ` [PATCH v6 11/48] KVM: PPC: Book3S HV: Ensure MSR[HV] is always clear " Nicholas Piggin
2021-04-05 1:19 ` [PATCH v6 12/48] KVM: PPC: Book3S 64: move KVM interrupt entry to a common entry point Nicholas Piggin
2021-04-05 1:19 ` [PATCH v6 13/48] KVM: PPC: Book3S 64: Move GUEST_MODE_SKIP test into KVM Nicholas Piggin
2021-04-05 1:19 ` [PATCH v6 14/48] KVM: PPC: Book3S 64: add hcall interrupt handler Nicholas Piggin
2021-04-05 1:19 ` [PATCH v6 15/48] KVM: PPC: Book3S 64: Move hcall early register setup to KVM Nicholas Piggin
2021-04-05 1:19 ` [PATCH v6 16/48] KVM: PPC: Book3S 64: Move interrupt " Nicholas Piggin
2021-04-06 4:37 ` Paul Mackerras
2021-04-06 7:04 ` Nicholas Piggin
2021-04-05 1:19 ` [PATCH v6 17/48] KVM: PPC: Book3S 64: move bad_host_intr check to HV handler Nicholas Piggin
2021-04-05 1:19 ` [PATCH v6 18/48] KVM: PPC: Book3S 64: Minimise hcall handler calling convention differences Nicholas Piggin
2021-04-05 1:19 ` [PATCH v6 19/48] KVM: PPC: Book3S HV P9: Move radix MMU switching instructions together Nicholas Piggin
2021-04-05 1:19 ` [PATCH v6 20/48] KVM: PPC: Book3S HV P9: implement kvmppc_xive_pull_vcpu in C Nicholas Piggin
2021-04-05 1:19 ` [PATCH v6 21/48] KVM: PPC: Book3S HV P9: Move xive vcpu context management into kvmhv_p9_guest_entry Nicholas Piggin
2021-04-05 1:19 ` [PATCH v6 22/48] KVM: PPC: Book3S HV P9: Stop handling hcalls in real-mode in the P9 path Nicholas Piggin
2021-04-05 1:19 ` [PATCH v6 23/48] KVM: PPC: Book3S HV P9: Move setting HDEC after switching to guest LPCR Nicholas Piggin
2021-04-05 1:19 ` [PATCH v6 24/48] KVM: PPC: Book3S HV P9: Use large decrementer for HDEC Nicholas Piggin
2021-04-05 1:19 ` [PATCH v6 25/48] KVM: PPC: Book3S HV P9: Use host timer accounting to avoid decrementer read Nicholas Piggin
2021-04-05 1:19 ` [PATCH v6 26/48] KVM: PPC: Book3S HV P9: Reduce mftb per guest entry/exit Nicholas Piggin
2021-04-05 1:19 ` [PATCH v6 27/48] KVM: PPC: Book3S HV P9: Reduce irq_work vs guest decrementer races Nicholas Piggin
2021-04-05 1:19 ` [PATCH v6 28/48] KMV: PPC: Book3S HV: Use set_dec to set decrementer to host Nicholas Piggin
2021-04-05 1:19 ` [PATCH v6 29/48] powerpc/time: add API for KVM to re-arm the host timer/decrementer Nicholas Piggin
2021-04-05 1:19 ` [PATCH v6 30/48] KVM: PPC: Book3S HV P9: Implement the rest of the P9 path in C Nicholas Piggin
2021-04-06 7:27 ` Paul Mackerras
2021-04-06 9:12 ` Nicholas Piggin
2021-04-06 13:02 ` Nicholas Piggin
2021-04-09 3:57 ` Alexey Kardashevskiy [this message]
2021-04-10 0:37 ` Nicholas Piggin
2021-04-05 1:19 ` [PATCH v6 31/48] KVM: PPC: Book3S HV P9: inline kvmhv_load_hv_regs_and_go into __kvmhv_vcpu_entry_p9 Nicholas Piggin
2021-04-05 1:19 ` [PATCH v6 32/48] KVM: PPC: Book3S HV P9: Read machine check registers while MSR[RI] is 0 Nicholas Piggin
2021-04-09 8:55 ` Alexey Kardashevskiy
2021-04-10 0:39 ` Nicholas Piggin
2021-04-05 1:19 ` [PATCH v6 33/48] KVM: PPC: Book3S HV P9: Improve exit timing accounting coverage Nicholas Piggin
2021-04-09 9:40 ` Alexey Kardashevskiy
2021-04-05 1:19 ` [PATCH v6 34/48] KVM: PPC: Book3S HV P9: Move SPR loading after expiry time check Nicholas Piggin
2021-04-05 1:19 ` [PATCH v6 35/48] KVM: PPC: Book3S HV P9: Add helpers for OS SPR handling Nicholas Piggin
2021-04-05 1:19 ` [PATCH v6 36/48] KVM: PPC: Book3S HV P9: Switch to guest MMU context as late as possible Nicholas Piggin
2021-04-05 1:19 ` [PATCH v6 37/48] KVM: PPC: Book3S HV: Implement radix prefetch workaround by disabling MMU Nicholas Piggin
2021-04-05 1:19 ` [PATCH v6 38/48] KVM: PPC: Book3S HV: Remove support for dependent threads mode on P9 Nicholas Piggin
2021-04-07 6:51 ` Paul Mackerras
2021-04-07 7:44 ` Nicholas Piggin
2021-04-07 9:35 ` Nicholas Piggin
2021-04-05 1:19 ` [PATCH v6 39/48] KVM: PPC: Book3S HV: Remove radix guest support from P7/8 path Nicholas Piggin
2021-04-05 1:19 ` [PATCH v6 40/48] KVM: PPC: Book3S HV: Remove virt mode checks from real mode handlers Nicholas Piggin
2021-04-05 1:19 ` [PATCH v6 41/48] KVM: PPC: Book3S HV: Remove unused nested HV tests in XICS emulation Nicholas Piggin
2021-04-05 1:19 ` [PATCH v6 42/48] KVM: PPC: Book3S HV P9: Allow all P9 processors to enable nested HV Nicholas Piggin
2021-04-05 1:19 ` [PATCH v6 43/48] KVM: PPC: Book3S HV: small pseries_do_hcall cleanup Nicholas Piggin
2021-04-05 1:19 ` [PATCH v6 44/48] KVM: PPC: Book3S HV: add virtual mode handlers for HPT hcalls and page faults Nicholas Piggin
2021-04-05 1:19 ` [PATCH v6 45/48] KVM: PPC: Book3S HV P9: Reflect userspace hcalls to hash guests to support PR KVM Nicholas Piggin
2021-04-05 1:19 ` [PATCH v6 46/48] KVM: PPC: Book3S HV P9: implement hash guest support Nicholas Piggin
2021-04-05 1:19 ` [PATCH v6 47/48] KVM: PPC: Book3S HV P9: implement hash host / " Nicholas Piggin
2021-04-05 1:19 ` [PATCH v6 48/48] KVM: PPC: Book3S HV: remove ISA v3.0 and v3.1 support from P7/8 path Nicholas Piggin
2021-04-08 7:33 ` [PATCH v6 00/48] KVM: PPC: Book3S: C-ify the P9 entry/exit code Nicholas Piggin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=e3be3cb3-23a1-2e28-1bda-a7437f2fd2ca@ozlabs.ru \
--to=aik@ozlabs.ru \
--cc=kvm-ppc@vger.kernel.org \
--cc=linuxppc-dev@lists.ozlabs.org \
--cc=npiggin@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox