* [PATCH 00/14] MIPS: KVM: Dynamically generate exception code
@ 2016-06-23 16:34 ` James Hogan
0 siblings, 0 replies; 36+ messages in thread
From: James Hogan @ 2016-06-23 16:34 UTC (permalink / raw)
To: Paolo Bonzini, Radim Krčmář, Ralf Baechle
Cc: James Hogan, linux-mips, kvm
These patches change the MIPS KVM exception entry code to be dynamically
assembled by the MIPS "uasm" in-kernel assembler, directly into unmapped
memory at run time by a new entry.c. Previously this code was statically
assembled from locore.S at build time and later copied into unmapped
memory at run time.
Patches 1-5 add support for the necessary instructions to uasm.
Patches 6-8 do the minimal-change conversion of locore.S to entry.c
using uasm (I've used -M10% so the diff is shown as a file move).
Patches 9-14 make some related improvements that are possible now that
it is dynamically generated, such as avoiding messy runtime conditionals
in assembly code, making use of KScratch registers when available, and
simplifying the initial GP register save sequence & jump to common code.
Ralf: Since the uasm patches (1-5) are needed for the later patches, I
suggest these all go together via the KVM tree (on which the whole
patchset is based), so Acks are welcome if they're okay with you.
James Hogan (14):
MIPS: uasm: Add CFC1/CTC1 instructions
MIPS: uasm: Add CFCMSA/CTCMSA instructions
MIPS: uasm: Add DI instruction
MIPS: uasm: Add MTHI/MTLO instructions
MIPS: uasm: Add r6 MUL encoding
MIPS; KVM: Convert exception entry to uasm
MIPS: KVM: Add dumping of generated entry code
MIPS: KVM: Drop now unused asm offsets
MIPS: KVM: Omit FPU handling entry code if possible
MIPS: KVM: Check MSA presence at uasm time
MIPS: KVM: Drop redundant restore of DDATA_LO
MIPS: KVM: Dynamically choose scratch registers
MIPS: KVM: Relative branch to common exit handler
MIPS: KVM: Save k0 straight into VCPU structure
arch/mips/include/asm/kvm_host.h | 9 +-
arch/mips/include/asm/uasm.h | 7 +
arch/mips/include/uapi/asm/inst.h | 71 ++-
arch/mips/kernel/asm-offsets.c | 66 ---
arch/mips/kvm/Kconfig | 1 +
arch/mips/kvm/Makefile | 2 +-
arch/mips/kvm/{locore.S => entry.c} | 976 +++++++++++++++++++-----------------
arch/mips/kvm/interrupt.h | 4 -
arch/mips/kvm/mips.c | 66 ++-
arch/mips/mm/uasm-micromips.c | 13 +-
arch/mips/mm/uasm-mips.c | 11 +
arch/mips/mm/uasm.c | 24 +-
12 files changed, 695 insertions(+), 555 deletions(-)
rename arch/mips/kvm/{locore.S => entry.c} (15%)
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Radim Krčmář <rkrcmar@redhat.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: linux-mips@linux-mips.org
Cc: kvm@vger.kernel.org
--
2.4.10
^ permalink raw reply [flat|nested] 36+ messages in thread* [PATCH 01/14] MIPS: uasm: Add CFC1/CTC1 instructions
@ 2016-06-23 16:34 ` James Hogan
0 siblings, 0 replies; 36+ messages in thread
From: James Hogan @ 2016-06-23 16:34 UTC (permalink / raw)
To: Paolo Bonzini, Radim Krčmář, Ralf Baechle
Cc: James Hogan, linux-mips
Add CFC1/CTC1 instructions for accessing FP control registers to uasm so
that KVM can use uasm for generating its entry point code at runtime.
Signed-off-by: James Hogan <james.hogan@imgtec.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: linux-mips@linux-mips.org
---
arch/mips/include/asm/uasm.h | 2 ++
arch/mips/mm/uasm-micromips.c | 8 ++++++--
arch/mips/mm/uasm-mips.c | 2 ++
arch/mips/mm/uasm.c | 26 ++++++++++++++------------
4 files changed, 24 insertions(+), 14 deletions(-)
diff --git a/arch/mips/include/asm/uasm.h b/arch/mips/include/asm/uasm.h
index b6ecfeee4dbe..3153ada46e9a 100644
--- a/arch/mips/include/asm/uasm.h
+++ b/arch/mips/include/asm/uasm.h
@@ -104,6 +104,8 @@ Ip_u1s2(_bltz);
Ip_u1s2(_bltzl);
Ip_u1u2s3(_bne);
Ip_u2s3u1(_cache);
+Ip_u1u2(_cfc1);
+Ip_u1u2(_ctc1);
Ip_u2u1s3(_daddiu);
Ip_u3u1u2(_daddu);
Ip_u2u1msbu3(_dins);
diff --git a/arch/mips/mm/uasm-micromips.c b/arch/mips/mm/uasm-micromips.c
index d78178daea4b..8b1acb2f6b8b 100644
--- a/arch/mips/mm/uasm-micromips.c
+++ b/arch/mips/mm/uasm-micromips.c
@@ -53,6 +53,8 @@ static struct insn insn_table_MM[] = {
{ insn_bltzl, 0, 0 },
{ insn_bne, M(mm_bne32_op, 0, 0, 0, 0, 0), RT | RS | BIMM },
{ insn_cache, M(mm_pool32b_op, 0, 0, mm_cache_func, 0, 0), RT | RS | SIMM },
+ { insn_cfc1, M(mm_pool32f_op, 0, 0, 0, mm_cfc1_op, mm_32f_73_op), RT | RS },
+ { insn_ctc1, M(mm_pool32f_op, 0, 0, 0, mm_ctc1_op, mm_32f_73_op), RT | RS },
{ insn_daddu, 0, 0 },
{ insn_daddiu, 0, 0 },
{ insn_divu, M(mm_pool32a_op, 0, 0, 0, mm_divu_op, mm_pool32axf_op), RT | RS },
@@ -166,13 +168,15 @@ static void build_insn(u32 **buf, enum opcode opc, ...)
op = ip->match;
va_start(ap, opc);
if (ip->fields & RS) {
- if (opc == insn_mfc0 || opc == insn_mtc0)
+ if (opc == insn_mfc0 || opc == insn_mtc0 ||
+ opc == insn_cfc1 || opc == insn_ctc1)
op |= build_rt(va_arg(ap, u32));
else
op |= build_rs(va_arg(ap, u32));
}
if (ip->fields & RT) {
- if (opc == insn_mfc0 || opc == insn_mtc0)
+ if (opc == insn_mfc0 || opc == insn_mtc0 ||
+ opc == insn_cfc1 || opc == insn_ctc1)
op |= build_rs(va_arg(ap, u32));
else
op |= build_rt(va_arg(ap, u32));
diff --git a/arch/mips/mm/uasm-mips.c b/arch/mips/mm/uasm-mips.c
index 9c2220a45189..5152544962c3 100644
--- a/arch/mips/mm/uasm-mips.c
+++ b/arch/mips/mm/uasm-mips.c
@@ -67,6 +67,8 @@ static struct insn insn_table[] = {
#else
{ insn_cache, M6(cache_op, 0, 0, 0, cache6_op), RS | RT | SIMM9 },
#endif
+ { insn_cfc1, M(cop1_op, cfc_op, 0, 0, 0, 0), RT | RD },
+ { insn_ctc1, M(cop1_op, ctc_op, 0, 0, 0, 0), RT | RD },
{ insn_daddiu, M(daddiu_op, 0, 0, 0, 0, 0), RS | RT | SIMM },
{ insn_daddu, M(spec_op, 0, 0, 0, 0, daddu_op), RS | RT | RD },
{ insn_dinsm, M(spec3_op, 0, 0, 0, 0, dinsm_op), RS | RT | RD | RE },
diff --git a/arch/mips/mm/uasm.c b/arch/mips/mm/uasm.c
index ad718debc35a..4731893db3f7 100644
--- a/arch/mips/mm/uasm.c
+++ b/arch/mips/mm/uasm.c
@@ -49,18 +49,18 @@ enum opcode {
insn_invalid,
insn_addiu, insn_addu, insn_and, insn_andi, insn_bbit0, insn_bbit1,
insn_beq, insn_beql, insn_bgez, insn_bgezl, insn_bltz, insn_bltzl,
- insn_bne, insn_cache, insn_daddiu, insn_daddu, insn_dins, insn_dinsm,
- insn_divu, insn_dmfc0, insn_dmtc0, insn_drotr, insn_drotr32, insn_dsll,
- insn_dsll32, insn_dsra, insn_dsrl, insn_dsrl32, insn_dsubu, insn_eret,
- insn_ext, insn_ins, insn_j, insn_jal, insn_jalr, insn_jr, insn_lb,
- insn_ld, insn_ldx, insn_lh, insn_ll, insn_lld, insn_lui, insn_lw,
- insn_lwx, insn_mfc0, insn_mfhc0, insn_mfhi, insn_mflo, insn_mtc0,
- insn_mthc0, insn_mul, insn_or, insn_ori, insn_pref, insn_rfe,
- insn_rotr, insn_sc, insn_scd, insn_sd, insn_sll, insn_sllv, insn_slt,
- insn_sltiu, insn_sltu, insn_sra, insn_srl, insn_srlv, insn_subu,
- insn_sw, insn_sync, insn_syscall, insn_tlbp, insn_tlbr, insn_tlbwi,
- insn_tlbwr, insn_wait, insn_wsbh, insn_xor, insn_xori, insn_yield,
- insn_lddir, insn_ldpte,
+ insn_bne, insn_cache, insn_cfc1, insn_ctc1, insn_daddiu, insn_daddu,
+ insn_dins, insn_dinsm, insn_divu, insn_dmfc0, insn_dmtc0, insn_drotr,
+ insn_drotr32, insn_dsll, insn_dsll32, insn_dsra, insn_dsrl, insn_dsrl32,
+ insn_dsubu, insn_eret, insn_ext, insn_ins, insn_j, insn_jal, insn_jalr,
+ insn_jr, insn_lb, insn_ld, insn_ldx, insn_lh, insn_ll, insn_lld,
+ insn_lui, insn_lw, insn_lwx, insn_mfc0, insn_mfhc0, insn_mfhi,
+ insn_mflo, insn_mtc0, insn_mthc0, insn_mul, insn_or, insn_ori,
+ insn_pref, insn_rfe, insn_rotr, insn_sc, insn_scd, insn_sd, insn_sll,
+ insn_sllv, insn_slt, insn_sltiu, insn_sltu, insn_sra, insn_srl,
+ insn_srlv, insn_subu, insn_sw, insn_sync, insn_syscall, insn_tlbp,
+ insn_tlbr, insn_tlbwi, insn_tlbwr, insn_wait, insn_wsbh, insn_xor,
+ insn_xori, insn_yield, insn_lddir, insn_ldpte,
};
struct insn {
@@ -268,6 +268,8 @@ I_u1s2(_bltz)
I_u1s2(_bltzl)
I_u1u2s3(_bne)
I_u2s3u1(_cache)
+I_u1u2(_cfc1)
+I_u1u2(_ctc1)
I_u1u2u3(_dmfc0)
I_u1u2u3(_dmtc0)
I_u2u1s3(_daddiu)
--
2.4.10
^ permalink raw reply related [flat|nested] 36+ messages in thread* [PATCH 01/14] MIPS: uasm: Add CFC1/CTC1 instructions
@ 2016-06-23 16:34 ` James Hogan
0 siblings, 0 replies; 36+ messages in thread
From: James Hogan @ 2016-06-23 16:34 UTC (permalink / raw)
To: Paolo Bonzini, Radim Krčmář, Ralf Baechle
Cc: James Hogan, linux-mips
Add CFC1/CTC1 instructions for accessing FP control registers to uasm so
that KVM can use uasm for generating its entry point code at runtime.
Signed-off-by: James Hogan <james.hogan@imgtec.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: linux-mips@linux-mips.org
---
arch/mips/include/asm/uasm.h | 2 ++
arch/mips/mm/uasm-micromips.c | 8 ++++++--
arch/mips/mm/uasm-mips.c | 2 ++
arch/mips/mm/uasm.c | 26 ++++++++++++++------------
4 files changed, 24 insertions(+), 14 deletions(-)
diff --git a/arch/mips/include/asm/uasm.h b/arch/mips/include/asm/uasm.h
index b6ecfeee4dbe..3153ada46e9a 100644
--- a/arch/mips/include/asm/uasm.h
+++ b/arch/mips/include/asm/uasm.h
@@ -104,6 +104,8 @@ Ip_u1s2(_bltz);
Ip_u1s2(_bltzl);
Ip_u1u2s3(_bne);
Ip_u2s3u1(_cache);
+Ip_u1u2(_cfc1);
+Ip_u1u2(_ctc1);
Ip_u2u1s3(_daddiu);
Ip_u3u1u2(_daddu);
Ip_u2u1msbu3(_dins);
diff --git a/arch/mips/mm/uasm-micromips.c b/arch/mips/mm/uasm-micromips.c
index d78178daea4b..8b1acb2f6b8b 100644
--- a/arch/mips/mm/uasm-micromips.c
+++ b/arch/mips/mm/uasm-micromips.c
@@ -53,6 +53,8 @@ static struct insn insn_table_MM[] = {
{ insn_bltzl, 0, 0 },
{ insn_bne, M(mm_bne32_op, 0, 0, 0, 0, 0), RT | RS | BIMM },
{ insn_cache, M(mm_pool32b_op, 0, 0, mm_cache_func, 0, 0), RT | RS | SIMM },
+ { insn_cfc1, M(mm_pool32f_op, 0, 0, 0, mm_cfc1_op, mm_32f_73_op), RT | RS },
+ { insn_ctc1, M(mm_pool32f_op, 0, 0, 0, mm_ctc1_op, mm_32f_73_op), RT | RS },
{ insn_daddu, 0, 0 },
{ insn_daddiu, 0, 0 },
{ insn_divu, M(mm_pool32a_op, 0, 0, 0, mm_divu_op, mm_pool32axf_op), RT | RS },
@@ -166,13 +168,15 @@ static void build_insn(u32 **buf, enum opcode opc, ...)
op = ip->match;
va_start(ap, opc);
if (ip->fields & RS) {
- if (opc == insn_mfc0 || opc == insn_mtc0)
+ if (opc == insn_mfc0 || opc == insn_mtc0 ||
+ opc == insn_cfc1 || opc == insn_ctc1)
op |= build_rt(va_arg(ap, u32));
else
op |= build_rs(va_arg(ap, u32));
}
if (ip->fields & RT) {
- if (opc == insn_mfc0 || opc == insn_mtc0)
+ if (opc == insn_mfc0 || opc == insn_mtc0 ||
+ opc == insn_cfc1 || opc == insn_ctc1)
op |= build_rs(va_arg(ap, u32));
else
op |= build_rt(va_arg(ap, u32));
diff --git a/arch/mips/mm/uasm-mips.c b/arch/mips/mm/uasm-mips.c
index 9c2220a45189..5152544962c3 100644
--- a/arch/mips/mm/uasm-mips.c
+++ b/arch/mips/mm/uasm-mips.c
@@ -67,6 +67,8 @@ static struct insn insn_table[] = {
#else
{ insn_cache, M6(cache_op, 0, 0, 0, cache6_op), RS | RT | SIMM9 },
#endif
+ { insn_cfc1, M(cop1_op, cfc_op, 0, 0, 0, 0), RT | RD },
+ { insn_ctc1, M(cop1_op, ctc_op, 0, 0, 0, 0), RT | RD },
{ insn_daddiu, M(daddiu_op, 0, 0, 0, 0, 0), RS | RT | SIMM },
{ insn_daddu, M(spec_op, 0, 0, 0, 0, daddu_op), RS | RT | RD },
{ insn_dinsm, M(spec3_op, 0, 0, 0, 0, dinsm_op), RS | RT | RD | RE },
diff --git a/arch/mips/mm/uasm.c b/arch/mips/mm/uasm.c
index ad718debc35a..4731893db3f7 100644
--- a/arch/mips/mm/uasm.c
+++ b/arch/mips/mm/uasm.c
@@ -49,18 +49,18 @@ enum opcode {
insn_invalid,
insn_addiu, insn_addu, insn_and, insn_andi, insn_bbit0, insn_bbit1,
insn_beq, insn_beql, insn_bgez, insn_bgezl, insn_bltz, insn_bltzl,
- insn_bne, insn_cache, insn_daddiu, insn_daddu, insn_dins, insn_dinsm,
- insn_divu, insn_dmfc0, insn_dmtc0, insn_drotr, insn_drotr32, insn_dsll,
- insn_dsll32, insn_dsra, insn_dsrl, insn_dsrl32, insn_dsubu, insn_eret,
- insn_ext, insn_ins, insn_j, insn_jal, insn_jalr, insn_jr, insn_lb,
- insn_ld, insn_ldx, insn_lh, insn_ll, insn_lld, insn_lui, insn_lw,
- insn_lwx, insn_mfc0, insn_mfhc0, insn_mfhi, insn_mflo, insn_mtc0,
- insn_mthc0, insn_mul, insn_or, insn_ori, insn_pref, insn_rfe,
- insn_rotr, insn_sc, insn_scd, insn_sd, insn_sll, insn_sllv, insn_slt,
- insn_sltiu, insn_sltu, insn_sra, insn_srl, insn_srlv, insn_subu,
- insn_sw, insn_sync, insn_syscall, insn_tlbp, insn_tlbr, insn_tlbwi,
- insn_tlbwr, insn_wait, insn_wsbh, insn_xor, insn_xori, insn_yield,
- insn_lddir, insn_ldpte,
+ insn_bne, insn_cache, insn_cfc1, insn_ctc1, insn_daddiu, insn_daddu,
+ insn_dins, insn_dinsm, insn_divu, insn_dmfc0, insn_dmtc0, insn_drotr,
+ insn_drotr32, insn_dsll, insn_dsll32, insn_dsra, insn_dsrl, insn_dsrl32,
+ insn_dsubu, insn_eret, insn_ext, insn_ins, insn_j, insn_jal, insn_jalr,
+ insn_jr, insn_lb, insn_ld, insn_ldx, insn_lh, insn_ll, insn_lld,
+ insn_lui, insn_lw, insn_lwx, insn_mfc0, insn_mfhc0, insn_mfhi,
+ insn_mflo, insn_mtc0, insn_mthc0, insn_mul, insn_or, insn_ori,
+ insn_pref, insn_rfe, insn_rotr, insn_sc, insn_scd, insn_sd, insn_sll,
+ insn_sllv, insn_slt, insn_sltiu, insn_sltu, insn_sra, insn_srl,
+ insn_srlv, insn_subu, insn_sw, insn_sync, insn_syscall, insn_tlbp,
+ insn_tlbr, insn_tlbwi, insn_tlbwr, insn_wait, insn_wsbh, insn_xor,
+ insn_xori, insn_yield, insn_lddir, insn_ldpte,
};
struct insn {
@@ -268,6 +268,8 @@ I_u1s2(_bltz)
I_u1s2(_bltzl)
I_u1u2s3(_bne)
I_u2s3u1(_cache)
+I_u1u2(_cfc1)
+I_u1u2(_ctc1)
I_u1u2u3(_dmfc0)
I_u1u2u3(_dmtc0)
I_u2u1s3(_daddiu)
--
2.4.10
^ permalink raw reply related [flat|nested] 36+ messages in thread* Re: [PATCH 01/14] MIPS: uasm: Add CFC1/CTC1 instructions
2016-06-23 16:34 ` James Hogan
(?)
@ 2016-07-05 13:56 ` Paolo Bonzini
2016-07-05 14:04 ` Ralf Baechle
-1 siblings, 1 reply; 36+ messages in thread
From: Paolo Bonzini @ 2016-07-05 13:56 UTC (permalink / raw)
To: James Hogan, Radim Krčmář, Ralf Baechle; +Cc: linux-mips
On 23/06/2016 18:34, James Hogan wrote:
> diff --git a/arch/mips/mm/uasm.c b/arch/mips/mm/uasm.c
> index ad718debc35a..4731893db3f7 100644
> --- a/arch/mips/mm/uasm.c
> +++ b/arch/mips/mm/uasm.c
> @@ -49,18 +49,18 @@ enum opcode {
This "enum opcode" looks like a pretty bad conflict magnet. Ralf, can
you check if you have any patches affecting it too?
Paolo
^ permalink raw reply [flat|nested] 36+ messages in thread* Re: [PATCH 01/14] MIPS: uasm: Add CFC1/CTC1 instructions
2016-07-05 13:56 ` Paolo Bonzini
@ 2016-07-05 14:04 ` Ralf Baechle
0 siblings, 0 replies; 36+ messages in thread
From: Ralf Baechle @ 2016-07-05 14:04 UTC (permalink / raw)
To: Paolo Bonzini; +Cc: James Hogan, Radim Krčmář, linux-mips
On Tue, Jul 05, 2016 at 03:56:31PM +0200, Paolo Bonzini wrote:
> On 23/06/2016 18:34, James Hogan wrote:
> > diff --git a/arch/mips/mm/uasm.c b/arch/mips/mm/uasm.c
> > index ad718debc35a..4731893db3f7 100644
> > --- a/arch/mips/mm/uasm.c
> > +++ b/arch/mips/mm/uasm.c
> > @@ -49,18 +49,18 @@ enum opcode {
>
> This "enum opcode" looks like a pretty bad conflict magnet. Ralf, can
> you check if you have any patches affecting it too?
Nothing at this point.
Ralf
^ permalink raw reply [flat|nested] 36+ messages in thread
* [PATCH 02/14] MIPS: uasm: Add CFCMSA/CTCMSA instructions
@ 2016-06-23 16:34 ` James Hogan
0 siblings, 0 replies; 36+ messages in thread
From: James Hogan @ 2016-06-23 16:34 UTC (permalink / raw)
To: Paolo Bonzini, Radim Krčmář, Ralf Baechle
Cc: James Hogan, linux-mips
Add CFCMSA/CTCMSA instructions for accessing MSA control registers to
uasm so that KVM can use uasm for generating its entry point code at
runtime.
Signed-off-by: James Hogan <james.hogan@imgtec.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: linux-mips@linux-mips.org
---
arch/mips/include/asm/uasm.h | 2 ++
arch/mips/include/uapi/asm/inst.h | 24 +++++++++++++++++++++++-
arch/mips/mm/uasm-micromips.c | 2 ++
arch/mips/mm/uasm-mips.c | 2 ++
arch/mips/mm/uasm.c | 26 ++++++++++++++------------
5 files changed, 43 insertions(+), 13 deletions(-)
diff --git a/arch/mips/include/asm/uasm.h b/arch/mips/include/asm/uasm.h
index 3153ada46e9a..edc02687016e 100644
--- a/arch/mips/include/asm/uasm.h
+++ b/arch/mips/include/asm/uasm.h
@@ -105,7 +105,9 @@ Ip_u1s2(_bltzl);
Ip_u1u2s3(_bne);
Ip_u2s3u1(_cache);
Ip_u1u2(_cfc1);
+Ip_u2u1(_cfcmsa);
Ip_u1u2(_ctc1);
+Ip_u2u1(_ctcmsa);
Ip_u2u1s3(_daddiu);
Ip_u3u1u2(_daddu);
Ip_u2u1msbu3(_dins);
diff --git a/arch/mips/include/uapi/asm/inst.h b/arch/mips/include/uapi/asm/inst.h
index a1ebf973725c..2e624dd058ef 100644
--- a/arch/mips/include/uapi/asm/inst.h
+++ b/arch/mips/include/uapi/asm/inst.h
@@ -238,6 +238,21 @@ enum bshfl_func {
};
/*
+ * MSA minor opcodes.
+ */
+enum msa_func {
+ msa_elm_op = 0x19,
+};
+
+/*
+ * MSA ELM opcodes.
+ */
+enum msa_elm {
+ msa_ctc_op = 0x3e,
+ msa_cfc_op = 0x7e,
+};
+
+/*
* func field for MSA MI10 format.
*/
enum msa_mi10_func {
@@ -264,7 +279,7 @@ enum mm_major_op {
mm_pool32b_op, mm_pool16b_op, mm_lhu16_op, mm_andi16_op,
mm_addiu32_op, mm_lhu32_op, mm_sh32_op, mm_lh32_op,
mm_pool32i_op, mm_pool16c_op, mm_lwsp16_op, mm_pool16d_op,
- mm_ori32_op, mm_pool32f_op, mm_reserved1_op, mm_reserved2_op,
+ mm_ori32_op, mm_pool32f_op, mm_pool32s_op, mm_reserved2_op,
mm_pool32c_op, mm_lwgp16_op, mm_lw16_op, mm_pool16e_op,
mm_xori32_op, mm_jals32_op, mm_addiupc_op, mm_reserved3_op,
mm_reserved4_op, mm_pool16f_op, mm_sb16_op, mm_beqz16_op,
@@ -479,6 +494,13 @@ enum mm_32f_73_minor_op {
};
/*
+ * (microMIPS) POOL32S minor opcodes.
+ */
+enum mm_32s_minor_op {
+ mm_32s_elm_op = 0x16,
+};
+
+/*
* (microMIPS) POOL16C minor opcodes.
*/
enum mm_16c_minor_op {
diff --git a/arch/mips/mm/uasm-micromips.c b/arch/mips/mm/uasm-micromips.c
index 8b1acb2f6b8b..eba5018961eb 100644
--- a/arch/mips/mm/uasm-micromips.c
+++ b/arch/mips/mm/uasm-micromips.c
@@ -54,7 +54,9 @@ static struct insn insn_table_MM[] = {
{ insn_bne, M(mm_bne32_op, 0, 0, 0, 0, 0), RT | RS | BIMM },
{ insn_cache, M(mm_pool32b_op, 0, 0, mm_cache_func, 0, 0), RT | RS | SIMM },
{ insn_cfc1, M(mm_pool32f_op, 0, 0, 0, mm_cfc1_op, mm_32f_73_op), RT | RS },
+ { insn_cfcmsa, M(mm_pool32s_op, 0, msa_cfc_op, 0, 0, mm_32s_elm_op), RD | RE },
{ insn_ctc1, M(mm_pool32f_op, 0, 0, 0, mm_ctc1_op, mm_32f_73_op), RT | RS },
+ { insn_ctcmsa, M(mm_pool32s_op, 0, msa_ctc_op, 0, 0, mm_32s_elm_op), RD | RE },
{ insn_daddu, 0, 0 },
{ insn_daddiu, 0, 0 },
{ insn_divu, M(mm_pool32a_op, 0, 0, 0, mm_divu_op, mm_pool32axf_op), RT | RS },
diff --git a/arch/mips/mm/uasm-mips.c b/arch/mips/mm/uasm-mips.c
index 5152544962c3..9f77783f23a6 100644
--- a/arch/mips/mm/uasm-mips.c
+++ b/arch/mips/mm/uasm-mips.c
@@ -68,7 +68,9 @@ static struct insn insn_table[] = {
{ insn_cache, M6(cache_op, 0, 0, 0, cache6_op), RS | RT | SIMM9 },
#endif
{ insn_cfc1, M(cop1_op, cfc_op, 0, 0, 0, 0), RT | RD },
+ { insn_cfcmsa, M(msa_op, 0, msa_cfc_op, 0, 0, msa_elm_op), RD | RE },
{ insn_ctc1, M(cop1_op, ctc_op, 0, 0, 0, 0), RT | RD },
+ { insn_ctcmsa, M(msa_op, 0, msa_ctc_op, 0, 0, msa_elm_op), RD | RE },
{ insn_daddiu, M(daddiu_op, 0, 0, 0, 0, 0), RS | RT | SIMM },
{ insn_daddu, M(spec_op, 0, 0, 0, 0, daddu_op), RS | RT | RD },
{ insn_dinsm, M(spec3_op, 0, 0, 0, 0, dinsm_op), RS | RT | RD | RE },
diff --git a/arch/mips/mm/uasm.c b/arch/mips/mm/uasm.c
index 4731893db3f7..3affd08a262b 100644
--- a/arch/mips/mm/uasm.c
+++ b/arch/mips/mm/uasm.c
@@ -49,18 +49,18 @@ enum opcode {
insn_invalid,
insn_addiu, insn_addu, insn_and, insn_andi, insn_bbit0, insn_bbit1,
insn_beq, insn_beql, insn_bgez, insn_bgezl, insn_bltz, insn_bltzl,
- insn_bne, insn_cache, insn_cfc1, insn_ctc1, insn_daddiu, insn_daddu,
- insn_dins, insn_dinsm, insn_divu, insn_dmfc0, insn_dmtc0, insn_drotr,
- insn_drotr32, insn_dsll, insn_dsll32, insn_dsra, insn_dsrl, insn_dsrl32,
- insn_dsubu, insn_eret, insn_ext, insn_ins, insn_j, insn_jal, insn_jalr,
- insn_jr, insn_lb, insn_ld, insn_ldx, insn_lh, insn_ll, insn_lld,
- insn_lui, insn_lw, insn_lwx, insn_mfc0, insn_mfhc0, insn_mfhi,
- insn_mflo, insn_mtc0, insn_mthc0, insn_mul, insn_or, insn_ori,
- insn_pref, insn_rfe, insn_rotr, insn_sc, insn_scd, insn_sd, insn_sll,
- insn_sllv, insn_slt, insn_sltiu, insn_sltu, insn_sra, insn_srl,
- insn_srlv, insn_subu, insn_sw, insn_sync, insn_syscall, insn_tlbp,
- insn_tlbr, insn_tlbwi, insn_tlbwr, insn_wait, insn_wsbh, insn_xor,
- insn_xori, insn_yield, insn_lddir, insn_ldpte,
+ insn_bne, insn_cache, insn_cfc1, insn_cfcmsa, insn_ctc1, insn_ctcmsa,
+ insn_daddiu, insn_daddu, insn_dins, insn_dinsm, insn_divu, insn_dmfc0,
+ insn_dmtc0, insn_drotr, insn_drotr32, insn_dsll, insn_dsll32, insn_dsra,
+ insn_dsrl, insn_dsrl32, insn_dsubu, insn_eret, insn_ext, insn_ins,
+ insn_j, insn_jal, insn_jalr, insn_jr, insn_lb, insn_ld, insn_ldx,
+ insn_lh, insn_ll, insn_lld, insn_lui, insn_lw, insn_lwx, insn_mfc0,
+ insn_mfhc0, insn_mfhi, insn_mflo, insn_mtc0, insn_mthc0, insn_mul,
+ insn_or, insn_ori, insn_pref, insn_rfe, insn_rotr, insn_sc, insn_scd,
+ insn_sd, insn_sll, insn_sllv, insn_slt, insn_sltiu, insn_sltu, insn_sra,
+ insn_srl, insn_srlv, insn_subu, insn_sw, insn_sync, insn_syscall,
+ insn_tlbp, insn_tlbr, insn_tlbwi, insn_tlbwr, insn_wait, insn_wsbh,
+ insn_xor, insn_xori, insn_yield, insn_lddir, insn_ldpte,
};
struct insn {
@@ -269,7 +269,9 @@ I_u1s2(_bltzl)
I_u1u2s3(_bne)
I_u2s3u1(_cache)
I_u1u2(_cfc1)
+I_u2u1(_cfcmsa)
I_u1u2(_ctc1)
+I_u2u1(_ctcmsa)
I_u1u2u3(_dmfc0)
I_u1u2u3(_dmtc0)
I_u2u1s3(_daddiu)
--
2.4.10
^ permalink raw reply related [flat|nested] 36+ messages in thread* [PATCH 02/14] MIPS: uasm: Add CFCMSA/CTCMSA instructions
@ 2016-06-23 16:34 ` James Hogan
0 siblings, 0 replies; 36+ messages in thread
From: James Hogan @ 2016-06-23 16:34 UTC (permalink / raw)
To: Paolo Bonzini, Radim Krčmář, Ralf Baechle
Cc: James Hogan, linux-mips
Add CFCMSA/CTCMSA instructions for accessing MSA control registers to
uasm so that KVM can use uasm for generating its entry point code at
runtime.
Signed-off-by: James Hogan <james.hogan@imgtec.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: linux-mips@linux-mips.org
---
arch/mips/include/asm/uasm.h | 2 ++
arch/mips/include/uapi/asm/inst.h | 24 +++++++++++++++++++++++-
arch/mips/mm/uasm-micromips.c | 2 ++
arch/mips/mm/uasm-mips.c | 2 ++
arch/mips/mm/uasm.c | 26 ++++++++++++++------------
5 files changed, 43 insertions(+), 13 deletions(-)
diff --git a/arch/mips/include/asm/uasm.h b/arch/mips/include/asm/uasm.h
index 3153ada46e9a..edc02687016e 100644
--- a/arch/mips/include/asm/uasm.h
+++ b/arch/mips/include/asm/uasm.h
@@ -105,7 +105,9 @@ Ip_u1s2(_bltzl);
Ip_u1u2s3(_bne);
Ip_u2s3u1(_cache);
Ip_u1u2(_cfc1);
+Ip_u2u1(_cfcmsa);
Ip_u1u2(_ctc1);
+Ip_u2u1(_ctcmsa);
Ip_u2u1s3(_daddiu);
Ip_u3u1u2(_daddu);
Ip_u2u1msbu3(_dins);
diff --git a/arch/mips/include/uapi/asm/inst.h b/arch/mips/include/uapi/asm/inst.h
index a1ebf973725c..2e624dd058ef 100644
--- a/arch/mips/include/uapi/asm/inst.h
+++ b/arch/mips/include/uapi/asm/inst.h
@@ -238,6 +238,21 @@ enum bshfl_func {
};
/*
+ * MSA minor opcodes.
+ */
+enum msa_func {
+ msa_elm_op = 0x19,
+};
+
+/*
+ * MSA ELM opcodes.
+ */
+enum msa_elm {
+ msa_ctc_op = 0x3e,
+ msa_cfc_op = 0x7e,
+};
+
+/*
* func field for MSA MI10 format.
*/
enum msa_mi10_func {
@@ -264,7 +279,7 @@ enum mm_major_op {
mm_pool32b_op, mm_pool16b_op, mm_lhu16_op, mm_andi16_op,
mm_addiu32_op, mm_lhu32_op, mm_sh32_op, mm_lh32_op,
mm_pool32i_op, mm_pool16c_op, mm_lwsp16_op, mm_pool16d_op,
- mm_ori32_op, mm_pool32f_op, mm_reserved1_op, mm_reserved2_op,
+ mm_ori32_op, mm_pool32f_op, mm_pool32s_op, mm_reserved2_op,
mm_pool32c_op, mm_lwgp16_op, mm_lw16_op, mm_pool16e_op,
mm_xori32_op, mm_jals32_op, mm_addiupc_op, mm_reserved3_op,
mm_reserved4_op, mm_pool16f_op, mm_sb16_op, mm_beqz16_op,
@@ -479,6 +494,13 @@ enum mm_32f_73_minor_op {
};
/*
+ * (microMIPS) POOL32S minor opcodes.
+ */
+enum mm_32s_minor_op {
+ mm_32s_elm_op = 0x16,
+};
+
+/*
* (microMIPS) POOL16C minor opcodes.
*/
enum mm_16c_minor_op {
diff --git a/arch/mips/mm/uasm-micromips.c b/arch/mips/mm/uasm-micromips.c
index 8b1acb2f6b8b..eba5018961eb 100644
--- a/arch/mips/mm/uasm-micromips.c
+++ b/arch/mips/mm/uasm-micromips.c
@@ -54,7 +54,9 @@ static struct insn insn_table_MM[] = {
{ insn_bne, M(mm_bne32_op, 0, 0, 0, 0, 0), RT | RS | BIMM },
{ insn_cache, M(mm_pool32b_op, 0, 0, mm_cache_func, 0, 0), RT | RS | SIMM },
{ insn_cfc1, M(mm_pool32f_op, 0, 0, 0, mm_cfc1_op, mm_32f_73_op), RT | RS },
+ { insn_cfcmsa, M(mm_pool32s_op, 0, msa_cfc_op, 0, 0, mm_32s_elm_op), RD | RE },
{ insn_ctc1, M(mm_pool32f_op, 0, 0, 0, mm_ctc1_op, mm_32f_73_op), RT | RS },
+ { insn_ctcmsa, M(mm_pool32s_op, 0, msa_ctc_op, 0, 0, mm_32s_elm_op), RD | RE },
{ insn_daddu, 0, 0 },
{ insn_daddiu, 0, 0 },
{ insn_divu, M(mm_pool32a_op, 0, 0, 0, mm_divu_op, mm_pool32axf_op), RT | RS },
diff --git a/arch/mips/mm/uasm-mips.c b/arch/mips/mm/uasm-mips.c
index 5152544962c3..9f77783f23a6 100644
--- a/arch/mips/mm/uasm-mips.c
+++ b/arch/mips/mm/uasm-mips.c
@@ -68,7 +68,9 @@ static struct insn insn_table[] = {
{ insn_cache, M6(cache_op, 0, 0, 0, cache6_op), RS | RT | SIMM9 },
#endif
{ insn_cfc1, M(cop1_op, cfc_op, 0, 0, 0, 0), RT | RD },
+ { insn_cfcmsa, M(msa_op, 0, msa_cfc_op, 0, 0, msa_elm_op), RD | RE },
{ insn_ctc1, M(cop1_op, ctc_op, 0, 0, 0, 0), RT | RD },
+ { insn_ctcmsa, M(msa_op, 0, msa_ctc_op, 0, 0, msa_elm_op), RD | RE },
{ insn_daddiu, M(daddiu_op, 0, 0, 0, 0, 0), RS | RT | SIMM },
{ insn_daddu, M(spec_op, 0, 0, 0, 0, daddu_op), RS | RT | RD },
{ insn_dinsm, M(spec3_op, 0, 0, 0, 0, dinsm_op), RS | RT | RD | RE },
diff --git a/arch/mips/mm/uasm.c b/arch/mips/mm/uasm.c
index 4731893db3f7..3affd08a262b 100644
--- a/arch/mips/mm/uasm.c
+++ b/arch/mips/mm/uasm.c
@@ -49,18 +49,18 @@ enum opcode {
insn_invalid,
insn_addiu, insn_addu, insn_and, insn_andi, insn_bbit0, insn_bbit1,
insn_beq, insn_beql, insn_bgez, insn_bgezl, insn_bltz, insn_bltzl,
- insn_bne, insn_cache, insn_cfc1, insn_ctc1, insn_daddiu, insn_daddu,
- insn_dins, insn_dinsm, insn_divu, insn_dmfc0, insn_dmtc0, insn_drotr,
- insn_drotr32, insn_dsll, insn_dsll32, insn_dsra, insn_dsrl, insn_dsrl32,
- insn_dsubu, insn_eret, insn_ext, insn_ins, insn_j, insn_jal, insn_jalr,
- insn_jr, insn_lb, insn_ld, insn_ldx, insn_lh, insn_ll, insn_lld,
- insn_lui, insn_lw, insn_lwx, insn_mfc0, insn_mfhc0, insn_mfhi,
- insn_mflo, insn_mtc0, insn_mthc0, insn_mul, insn_or, insn_ori,
- insn_pref, insn_rfe, insn_rotr, insn_sc, insn_scd, insn_sd, insn_sll,
- insn_sllv, insn_slt, insn_sltiu, insn_sltu, insn_sra, insn_srl,
- insn_srlv, insn_subu, insn_sw, insn_sync, insn_syscall, insn_tlbp,
- insn_tlbr, insn_tlbwi, insn_tlbwr, insn_wait, insn_wsbh, insn_xor,
- insn_xori, insn_yield, insn_lddir, insn_ldpte,
+ insn_bne, insn_cache, insn_cfc1, insn_cfcmsa, insn_ctc1, insn_ctcmsa,
+ insn_daddiu, insn_daddu, insn_dins, insn_dinsm, insn_divu, insn_dmfc0,
+ insn_dmtc0, insn_drotr, insn_drotr32, insn_dsll, insn_dsll32, insn_dsra,
+ insn_dsrl, insn_dsrl32, insn_dsubu, insn_eret, insn_ext, insn_ins,
+ insn_j, insn_jal, insn_jalr, insn_jr, insn_lb, insn_ld, insn_ldx,
+ insn_lh, insn_ll, insn_lld, insn_lui, insn_lw, insn_lwx, insn_mfc0,
+ insn_mfhc0, insn_mfhi, insn_mflo, insn_mtc0, insn_mthc0, insn_mul,
+ insn_or, insn_ori, insn_pref, insn_rfe, insn_rotr, insn_sc, insn_scd,
+ insn_sd, insn_sll, insn_sllv, insn_slt, insn_sltiu, insn_sltu, insn_sra,
+ insn_srl, insn_srlv, insn_subu, insn_sw, insn_sync, insn_syscall,
+ insn_tlbp, insn_tlbr, insn_tlbwi, insn_tlbwr, insn_wait, insn_wsbh,
+ insn_xor, insn_xori, insn_yield, insn_lddir, insn_ldpte,
};
struct insn {
@@ -269,7 +269,9 @@ I_u1s2(_bltzl)
I_u1u2s3(_bne)
I_u2s3u1(_cache)
I_u1u2(_cfc1)
+I_u2u1(_cfcmsa)
I_u1u2(_ctc1)
+I_u2u1(_ctcmsa)
I_u1u2u3(_dmfc0)
I_u1u2u3(_dmtc0)
I_u2u1s3(_daddiu)
--
2.4.10
^ permalink raw reply related [flat|nested] 36+ messages in thread
* [PATCH 03/14] MIPS: uasm: Add DI instruction
@ 2016-06-23 16:34 ` James Hogan
0 siblings, 0 replies; 36+ messages in thread
From: James Hogan @ 2016-06-23 16:34 UTC (permalink / raw)
To: Paolo Bonzini, Radim Krčmář, Ralf Baechle
Cc: James Hogan, linux-mips
Add DI instruction for disabling interrupts to uasm so that KVM can use
uasm for generating its entry point code at runtime.
Signed-off-by: James Hogan <james.hogan@imgtec.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: linux-mips@linux-mips.org
---
arch/mips/include/asm/uasm.h | 1 +
arch/mips/include/uapi/asm/inst.h | 1 +
arch/mips/mm/uasm-micromips.c | 1 +
arch/mips/mm/uasm-mips.c | 1 +
arch/mips/mm/uasm.c | 23 ++++++++++++-----------
5 files changed, 16 insertions(+), 11 deletions(-)
diff --git a/arch/mips/include/asm/uasm.h b/arch/mips/include/asm/uasm.h
index edc02687016e..4af8a5becbbb 100644
--- a/arch/mips/include/asm/uasm.h
+++ b/arch/mips/include/asm/uasm.h
@@ -110,6 +110,7 @@ Ip_u1u2(_ctc1);
Ip_u2u1(_ctcmsa);
Ip_u2u1s3(_daddiu);
Ip_u3u1u2(_daddu);
+Ip_u1(_di);
Ip_u2u1msbu3(_dins);
Ip_u2u1msbu3(_dinsm);
Ip_u1u2(_divu);
diff --git a/arch/mips/include/uapi/asm/inst.h b/arch/mips/include/uapi/asm/inst.h
index 2e624dd058ef..7010d0b7b752 100644
--- a/arch/mips/include/uapi/asm/inst.h
+++ b/arch/mips/include/uapi/asm/inst.h
@@ -376,6 +376,7 @@ enum mm_32axf_minor_op {
mm_jalrhb_op = 0x07c,
mm_tlbwi_op = 0x08d,
mm_tlbwr_op = 0x0cd,
+ mm_di_op = 0x11d,
mm_jalrs_op = 0x13c,
mm_jalrshb_op = 0x17c,
mm_sync_op = 0x1ad,
diff --git a/arch/mips/mm/uasm-micromips.c b/arch/mips/mm/uasm-micromips.c
index eba5018961eb..40bef28f192c 100644
--- a/arch/mips/mm/uasm-micromips.c
+++ b/arch/mips/mm/uasm-micromips.c
@@ -59,6 +59,7 @@ static struct insn insn_table_MM[] = {
{ insn_ctcmsa, M(mm_pool32s_op, 0, msa_ctc_op, 0, 0, mm_32s_elm_op), RD | RE },
{ insn_daddu, 0, 0 },
{ insn_daddiu, 0, 0 },
+ { insn_di, M(mm_pool32a_op, 0, 0, 0, mm_di_op, mm_pool32axf_op), RS },
{ insn_divu, M(mm_pool32a_op, 0, 0, 0, mm_divu_op, mm_pool32axf_op), RT | RS },
{ insn_dmfc0, 0, 0 },
{ insn_dmtc0, 0, 0 },
diff --git a/arch/mips/mm/uasm-mips.c b/arch/mips/mm/uasm-mips.c
index 9f77783f23a6..2b7d85b8241f 100644
--- a/arch/mips/mm/uasm-mips.c
+++ b/arch/mips/mm/uasm-mips.c
@@ -74,6 +74,7 @@ static struct insn insn_table[] = {
{ insn_daddiu, M(daddiu_op, 0, 0, 0, 0, 0), RS | RT | SIMM },
{ insn_daddu, M(spec_op, 0, 0, 0, 0, daddu_op), RS | RT | RD },
{ insn_dinsm, M(spec3_op, 0, 0, 0, 0, dinsm_op), RS | RT | RD | RE },
+ { insn_di, M(cop0_op, mfmc0_op, 0, 12, 0, 0), RT },
{ insn_dins, M(spec3_op, 0, 0, 0, 0, dins_op), RS | RT | RD | RE },
{ insn_divu, M(spec_op, 0, 0, 0, 0, divu_op), RS | RT },
{ insn_dmfc0, M(cop0_op, dmfc_op, 0, 0, 0, 0), RT | RD | SET},
diff --git a/arch/mips/mm/uasm.c b/arch/mips/mm/uasm.c
index 3affd08a262b..006fb05b74a7 100644
--- a/arch/mips/mm/uasm.c
+++ b/arch/mips/mm/uasm.c
@@ -50,17 +50,17 @@ enum opcode {
insn_addiu, insn_addu, insn_and, insn_andi, insn_bbit0, insn_bbit1,
insn_beq, insn_beql, insn_bgez, insn_bgezl, insn_bltz, insn_bltzl,
insn_bne, insn_cache, insn_cfc1, insn_cfcmsa, insn_ctc1, insn_ctcmsa,
- insn_daddiu, insn_daddu, insn_dins, insn_dinsm, insn_divu, insn_dmfc0,
- insn_dmtc0, insn_drotr, insn_drotr32, insn_dsll, insn_dsll32, insn_dsra,
- insn_dsrl, insn_dsrl32, insn_dsubu, insn_eret, insn_ext, insn_ins,
- insn_j, insn_jal, insn_jalr, insn_jr, insn_lb, insn_ld, insn_ldx,
- insn_lh, insn_ll, insn_lld, insn_lui, insn_lw, insn_lwx, insn_mfc0,
- insn_mfhc0, insn_mfhi, insn_mflo, insn_mtc0, insn_mthc0, insn_mul,
- insn_or, insn_ori, insn_pref, insn_rfe, insn_rotr, insn_sc, insn_scd,
- insn_sd, insn_sll, insn_sllv, insn_slt, insn_sltiu, insn_sltu, insn_sra,
- insn_srl, insn_srlv, insn_subu, insn_sw, insn_sync, insn_syscall,
- insn_tlbp, insn_tlbr, insn_tlbwi, insn_tlbwr, insn_wait, insn_wsbh,
- insn_xor, insn_xori, insn_yield, insn_lddir, insn_ldpte,
+ insn_daddiu, insn_daddu, insn_di, insn_dins, insn_dinsm, insn_divu,
+ insn_dmfc0, insn_dmtc0, insn_drotr, insn_drotr32, insn_dsll,
+ insn_dsll32, insn_dsra, insn_dsrl, insn_dsrl32, insn_dsubu, insn_eret,
+ insn_ext, insn_ins, insn_j, insn_jal, insn_jalr, insn_jr, insn_lb,
+ insn_ld, insn_ldx, insn_lh, insn_ll, insn_lld, insn_lui, insn_lw,
+ insn_lwx, insn_mfc0, insn_mfhc0, insn_mfhi, insn_mflo, insn_mtc0,
+ insn_mthc0, insn_mul, insn_or, insn_ori, insn_pref, insn_rfe, insn_rotr,
+ insn_sc, insn_scd, insn_sd, insn_sll, insn_sllv, insn_slt, insn_sltiu,
+ insn_sltu, insn_sra, insn_srl, insn_srlv, insn_subu, insn_sw, insn_sync,
+ insn_syscall, insn_tlbp, insn_tlbr, insn_tlbwi, insn_tlbwr, insn_wait,
+ insn_wsbh, insn_xor, insn_xori, insn_yield, insn_lddir, insn_ldpte,
};
struct insn {
@@ -276,6 +276,7 @@ I_u1u2u3(_dmfc0)
I_u1u2u3(_dmtc0)
I_u2u1s3(_daddiu)
I_u3u1u2(_daddu)
+I_u1(_di);
I_u1u2(_divu)
I_u2u1u3(_dsll)
I_u2u1u3(_dsll32)
--
2.4.10
^ permalink raw reply related [flat|nested] 36+ messages in thread* [PATCH 03/14] MIPS: uasm: Add DI instruction
@ 2016-06-23 16:34 ` James Hogan
0 siblings, 0 replies; 36+ messages in thread
From: James Hogan @ 2016-06-23 16:34 UTC (permalink / raw)
To: Paolo Bonzini, Radim Krčmář, Ralf Baechle
Cc: James Hogan, linux-mips
Add DI instruction for disabling interrupts to uasm so that KVM can use
uasm for generating its entry point code at runtime.
Signed-off-by: James Hogan <james.hogan@imgtec.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: linux-mips@linux-mips.org
---
arch/mips/include/asm/uasm.h | 1 +
arch/mips/include/uapi/asm/inst.h | 1 +
arch/mips/mm/uasm-micromips.c | 1 +
arch/mips/mm/uasm-mips.c | 1 +
arch/mips/mm/uasm.c | 23 ++++++++++++-----------
5 files changed, 16 insertions(+), 11 deletions(-)
diff --git a/arch/mips/include/asm/uasm.h b/arch/mips/include/asm/uasm.h
index edc02687016e..4af8a5becbbb 100644
--- a/arch/mips/include/asm/uasm.h
+++ b/arch/mips/include/asm/uasm.h
@@ -110,6 +110,7 @@ Ip_u1u2(_ctc1);
Ip_u2u1(_ctcmsa);
Ip_u2u1s3(_daddiu);
Ip_u3u1u2(_daddu);
+Ip_u1(_di);
Ip_u2u1msbu3(_dins);
Ip_u2u1msbu3(_dinsm);
Ip_u1u2(_divu);
diff --git a/arch/mips/include/uapi/asm/inst.h b/arch/mips/include/uapi/asm/inst.h
index 2e624dd058ef..7010d0b7b752 100644
--- a/arch/mips/include/uapi/asm/inst.h
+++ b/arch/mips/include/uapi/asm/inst.h
@@ -376,6 +376,7 @@ enum mm_32axf_minor_op {
mm_jalrhb_op = 0x07c,
mm_tlbwi_op = 0x08d,
mm_tlbwr_op = 0x0cd,
+ mm_di_op = 0x11d,
mm_jalrs_op = 0x13c,
mm_jalrshb_op = 0x17c,
mm_sync_op = 0x1ad,
diff --git a/arch/mips/mm/uasm-micromips.c b/arch/mips/mm/uasm-micromips.c
index eba5018961eb..40bef28f192c 100644
--- a/arch/mips/mm/uasm-micromips.c
+++ b/arch/mips/mm/uasm-micromips.c
@@ -59,6 +59,7 @@ static struct insn insn_table_MM[] = {
{ insn_ctcmsa, M(mm_pool32s_op, 0, msa_ctc_op, 0, 0, mm_32s_elm_op), RD | RE },
{ insn_daddu, 0, 0 },
{ insn_daddiu, 0, 0 },
+ { insn_di, M(mm_pool32a_op, 0, 0, 0, mm_di_op, mm_pool32axf_op), RS },
{ insn_divu, M(mm_pool32a_op, 0, 0, 0, mm_divu_op, mm_pool32axf_op), RT | RS },
{ insn_dmfc0, 0, 0 },
{ insn_dmtc0, 0, 0 },
diff --git a/arch/mips/mm/uasm-mips.c b/arch/mips/mm/uasm-mips.c
index 9f77783f23a6..2b7d85b8241f 100644
--- a/arch/mips/mm/uasm-mips.c
+++ b/arch/mips/mm/uasm-mips.c
@@ -74,6 +74,7 @@ static struct insn insn_table[] = {
{ insn_daddiu, M(daddiu_op, 0, 0, 0, 0, 0), RS | RT | SIMM },
{ insn_daddu, M(spec_op, 0, 0, 0, 0, daddu_op), RS | RT | RD },
{ insn_dinsm, M(spec3_op, 0, 0, 0, 0, dinsm_op), RS | RT | RD | RE },
+ { insn_di, M(cop0_op, mfmc0_op, 0, 12, 0, 0), RT },
{ insn_dins, M(spec3_op, 0, 0, 0, 0, dins_op), RS | RT | RD | RE },
{ insn_divu, M(spec_op, 0, 0, 0, 0, divu_op), RS | RT },
{ insn_dmfc0, M(cop0_op, dmfc_op, 0, 0, 0, 0), RT | RD | SET},
diff --git a/arch/mips/mm/uasm.c b/arch/mips/mm/uasm.c
index 3affd08a262b..006fb05b74a7 100644
--- a/arch/mips/mm/uasm.c
+++ b/arch/mips/mm/uasm.c
@@ -50,17 +50,17 @@ enum opcode {
insn_addiu, insn_addu, insn_and, insn_andi, insn_bbit0, insn_bbit1,
insn_beq, insn_beql, insn_bgez, insn_bgezl, insn_bltz, insn_bltzl,
insn_bne, insn_cache, insn_cfc1, insn_cfcmsa, insn_ctc1, insn_ctcmsa,
- insn_daddiu, insn_daddu, insn_dins, insn_dinsm, insn_divu, insn_dmfc0,
- insn_dmtc0, insn_drotr, insn_drotr32, insn_dsll, insn_dsll32, insn_dsra,
- insn_dsrl, insn_dsrl32, insn_dsubu, insn_eret, insn_ext, insn_ins,
- insn_j, insn_jal, insn_jalr, insn_jr, insn_lb, insn_ld, insn_ldx,
- insn_lh, insn_ll, insn_lld, insn_lui, insn_lw, insn_lwx, insn_mfc0,
- insn_mfhc0, insn_mfhi, insn_mflo, insn_mtc0, insn_mthc0, insn_mul,
- insn_or, insn_ori, insn_pref, insn_rfe, insn_rotr, insn_sc, insn_scd,
- insn_sd, insn_sll, insn_sllv, insn_slt, insn_sltiu, insn_sltu, insn_sra,
- insn_srl, insn_srlv, insn_subu, insn_sw, insn_sync, insn_syscall,
- insn_tlbp, insn_tlbr, insn_tlbwi, insn_tlbwr, insn_wait, insn_wsbh,
- insn_xor, insn_xori, insn_yield, insn_lddir, insn_ldpte,
+ insn_daddiu, insn_daddu, insn_di, insn_dins, insn_dinsm, insn_divu,
+ insn_dmfc0, insn_dmtc0, insn_drotr, insn_drotr32, insn_dsll,
+ insn_dsll32, insn_dsra, insn_dsrl, insn_dsrl32, insn_dsubu, insn_eret,
+ insn_ext, insn_ins, insn_j, insn_jal, insn_jalr, insn_jr, insn_lb,
+ insn_ld, insn_ldx, insn_lh, insn_ll, insn_lld, insn_lui, insn_lw,
+ insn_lwx, insn_mfc0, insn_mfhc0, insn_mfhi, insn_mflo, insn_mtc0,
+ insn_mthc0, insn_mul, insn_or, insn_ori, insn_pref, insn_rfe, insn_rotr,
+ insn_sc, insn_scd, insn_sd, insn_sll, insn_sllv, insn_slt, insn_sltiu,
+ insn_sltu, insn_sra, insn_srl, insn_srlv, insn_subu, insn_sw, insn_sync,
+ insn_syscall, insn_tlbp, insn_tlbr, insn_tlbwi, insn_tlbwr, insn_wait,
+ insn_wsbh, insn_xor, insn_xori, insn_yield, insn_lddir, insn_ldpte,
};
struct insn {
@@ -276,6 +276,7 @@ I_u1u2u3(_dmfc0)
I_u1u2u3(_dmtc0)
I_u2u1s3(_daddiu)
I_u3u1u2(_daddu)
+I_u1(_di);
I_u1u2(_divu)
I_u2u1u3(_dsll)
I_u2u1u3(_dsll32)
--
2.4.10
^ permalink raw reply related [flat|nested] 36+ messages in thread
* [PATCH 04/14] MIPS: uasm: Add MTHI/MTLO instructions
@ 2016-06-23 16:34 ` James Hogan
0 siblings, 0 replies; 36+ messages in thread
From: James Hogan @ 2016-06-23 16:34 UTC (permalink / raw)
To: Paolo Bonzini, Radim Krčmář, Ralf Baechle
Cc: James Hogan, linux-mips
Add MTHI/MTLO instructions for writing to the hi & lo registers to uasm
so that KVM can use uasm for generating its entry point code at runtime.
Signed-off-by: James Hogan <james.hogan@imgtec.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: linux-mips@linux-mips.org
---
arch/mips/include/asm/uasm.h | 2 ++
arch/mips/include/uapi/asm/inst.h | 2 ++
arch/mips/mm/uasm-micromips.c | 2 ++
arch/mips/mm/uasm-mips.c | 2 ++
arch/mips/mm/uasm.c | 13 ++++++++-----
5 files changed, 16 insertions(+), 5 deletions(-)
diff --git a/arch/mips/include/asm/uasm.h b/arch/mips/include/asm/uasm.h
index 4af8a5becbbb..f7929f65f7ca 100644
--- a/arch/mips/include/asm/uasm.h
+++ b/arch/mips/include/asm/uasm.h
@@ -146,6 +146,8 @@ Ip_u1(_mfhi);
Ip_u1(_mflo);
Ip_u1u2u3(_mtc0);
Ip_u1u2u3(_mthc0);
+Ip_u1(_mthi);
+Ip_u1(_mtlo);
Ip_u3u1u2(_mul);
Ip_u3u1u2(_or);
Ip_u2u1u3(_ori);
diff --git a/arch/mips/include/uapi/asm/inst.h b/arch/mips/include/uapi/asm/inst.h
index 7010d0b7b752..6319c5037e66 100644
--- a/arch/mips/include/uapi/asm/inst.h
+++ b/arch/mips/include/uapi/asm/inst.h
@@ -375,7 +375,9 @@ enum mm_32axf_minor_op {
mm_mflo32_op = 0x075,
mm_jalrhb_op = 0x07c,
mm_tlbwi_op = 0x08d,
+ mm_mthi32_op = 0x0b5,
mm_tlbwr_op = 0x0cd,
+ mm_mtlo32_op = 0x0f5,
mm_di_op = 0x11d,
mm_jalrs_op = 0x13c,
mm_jalrshb_op = 0x17c,
diff --git a/arch/mips/mm/uasm-micromips.c b/arch/mips/mm/uasm-micromips.c
index 40bef28f192c..277cf52d80e1 100644
--- a/arch/mips/mm/uasm-micromips.c
+++ b/arch/mips/mm/uasm-micromips.c
@@ -89,6 +89,8 @@ static struct insn insn_table_MM[] = {
{ insn_mfhi, M(mm_pool32a_op, 0, 0, 0, mm_mfhi32_op, mm_pool32axf_op), RS },
{ insn_mflo, M(mm_pool32a_op, 0, 0, 0, mm_mflo32_op, mm_pool32axf_op), RS },
{ insn_mtc0, M(mm_pool32a_op, 0, 0, 0, mm_mtc0_op, mm_pool32axf_op), RT | RS | RD },
+ { insn_mthi, M(mm_pool32a_op, 0, 0, 0, mm_mthi32_op, mm_pool32axf_op), RS },
+ { insn_mtlo, M(mm_pool32a_op, 0, 0, 0, mm_mtlo32_op, mm_pool32axf_op), RS },
{ insn_mul, M(mm_pool32a_op, 0, 0, 0, 0, mm_mul_op), RT | RS | RD },
{ insn_or, M(mm_pool32a_op, 0, 0, 0, 0, mm_or32_op), RT | RS | RD },
{ insn_ori, M(mm_ori32_op, 0, 0, 0, 0, 0), RT | RS | UIMM },
diff --git a/arch/mips/mm/uasm-mips.c b/arch/mips/mm/uasm-mips.c
index 2b7d85b8241f..86a3c76a1ad8 100644
--- a/arch/mips/mm/uasm-mips.c
+++ b/arch/mips/mm/uasm-mips.c
@@ -119,6 +119,8 @@ static struct insn insn_table[] = {
{ insn_mflo, M(spec_op, 0, 0, 0, 0, mflo_op), RD },
{ insn_mtc0, M(cop0_op, mtc_op, 0, 0, 0, 0), RT | RD | SET},
{ insn_mthc0, M(cop0_op, mthc0_op, 0, 0, 0, 0), RT | RD | SET},
+ { insn_mthi, M(spec_op, 0, 0, 0, 0, mthi_op), RS },
+ { insn_mtlo, M(spec_op, 0, 0, 0, 0, mtlo_op), RS },
{ insn_mul, M(spec2_op, 0, 0, 0, 0, mul_op), RS | RT | RD},
{ insn_ori, M(ori_op, 0, 0, 0, 0, 0), RS | RT | UIMM },
{ insn_or, M(spec_op, 0, 0, 0, 0, or_op), RS | RT | RD },
diff --git a/arch/mips/mm/uasm.c b/arch/mips/mm/uasm.c
index 006fb05b74a7..3e0282d301d6 100644
--- a/arch/mips/mm/uasm.c
+++ b/arch/mips/mm/uasm.c
@@ -56,11 +56,12 @@ enum opcode {
insn_ext, insn_ins, insn_j, insn_jal, insn_jalr, insn_jr, insn_lb,
insn_ld, insn_ldx, insn_lh, insn_ll, insn_lld, insn_lui, insn_lw,
insn_lwx, insn_mfc0, insn_mfhc0, insn_mfhi, insn_mflo, insn_mtc0,
- insn_mthc0, insn_mul, insn_or, insn_ori, insn_pref, insn_rfe, insn_rotr,
- insn_sc, insn_scd, insn_sd, insn_sll, insn_sllv, insn_slt, insn_sltiu,
- insn_sltu, insn_sra, insn_srl, insn_srlv, insn_subu, insn_sw, insn_sync,
- insn_syscall, insn_tlbp, insn_tlbr, insn_tlbwi, insn_tlbwr, insn_wait,
- insn_wsbh, insn_xor, insn_xori, insn_yield, insn_lddir, insn_ldpte,
+ insn_mthc0, insn_mthi, insn_mtlo, insn_mul, insn_or, insn_ori,
+ insn_pref, insn_rfe, insn_rotr, insn_sc, insn_scd, insn_sd, insn_sll,
+ insn_sllv, insn_slt, insn_sltiu, insn_sltu, insn_sra, insn_srl,
+ insn_srlv, insn_subu, insn_sw, insn_sync, insn_syscall, insn_tlbp,
+ insn_tlbr, insn_tlbwi, insn_tlbwr, insn_wait, insn_wsbh, insn_xor,
+ insn_xori, insn_yield, insn_lddir, insn_ldpte,
};
struct insn {
@@ -306,6 +307,8 @@ I_u1(_mfhi)
I_u1(_mflo)
I_u1u2u3(_mtc0)
I_u1u2u3(_mthc0)
+I_u1(_mthi)
+I_u1(_mtlo)
I_u3u1u2(_mul)
I_u2u1u3(_ori)
I_u3u1u2(_or)
--
2.4.10
^ permalink raw reply related [flat|nested] 36+ messages in thread* [PATCH 04/14] MIPS: uasm: Add MTHI/MTLO instructions
@ 2016-06-23 16:34 ` James Hogan
0 siblings, 0 replies; 36+ messages in thread
From: James Hogan @ 2016-06-23 16:34 UTC (permalink / raw)
To: Paolo Bonzini, Radim Krčmář, Ralf Baechle
Cc: James Hogan, linux-mips
Add MTHI/MTLO instructions for writing to the hi & lo registers to uasm
so that KVM can use uasm for generating its entry point code at runtime.
Signed-off-by: James Hogan <james.hogan@imgtec.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: linux-mips@linux-mips.org
---
arch/mips/include/asm/uasm.h | 2 ++
arch/mips/include/uapi/asm/inst.h | 2 ++
arch/mips/mm/uasm-micromips.c | 2 ++
arch/mips/mm/uasm-mips.c | 2 ++
arch/mips/mm/uasm.c | 13 ++++++++-----
5 files changed, 16 insertions(+), 5 deletions(-)
diff --git a/arch/mips/include/asm/uasm.h b/arch/mips/include/asm/uasm.h
index 4af8a5becbbb..f7929f65f7ca 100644
--- a/arch/mips/include/asm/uasm.h
+++ b/arch/mips/include/asm/uasm.h
@@ -146,6 +146,8 @@ Ip_u1(_mfhi);
Ip_u1(_mflo);
Ip_u1u2u3(_mtc0);
Ip_u1u2u3(_mthc0);
+Ip_u1(_mthi);
+Ip_u1(_mtlo);
Ip_u3u1u2(_mul);
Ip_u3u1u2(_or);
Ip_u2u1u3(_ori);
diff --git a/arch/mips/include/uapi/asm/inst.h b/arch/mips/include/uapi/asm/inst.h
index 7010d0b7b752..6319c5037e66 100644
--- a/arch/mips/include/uapi/asm/inst.h
+++ b/arch/mips/include/uapi/asm/inst.h
@@ -375,7 +375,9 @@ enum mm_32axf_minor_op {
mm_mflo32_op = 0x075,
mm_jalrhb_op = 0x07c,
mm_tlbwi_op = 0x08d,
+ mm_mthi32_op = 0x0b5,
mm_tlbwr_op = 0x0cd,
+ mm_mtlo32_op = 0x0f5,
mm_di_op = 0x11d,
mm_jalrs_op = 0x13c,
mm_jalrshb_op = 0x17c,
diff --git a/arch/mips/mm/uasm-micromips.c b/arch/mips/mm/uasm-micromips.c
index 40bef28f192c..277cf52d80e1 100644
--- a/arch/mips/mm/uasm-micromips.c
+++ b/arch/mips/mm/uasm-micromips.c
@@ -89,6 +89,8 @@ static struct insn insn_table_MM[] = {
{ insn_mfhi, M(mm_pool32a_op, 0, 0, 0, mm_mfhi32_op, mm_pool32axf_op), RS },
{ insn_mflo, M(mm_pool32a_op, 0, 0, 0, mm_mflo32_op, mm_pool32axf_op), RS },
{ insn_mtc0, M(mm_pool32a_op, 0, 0, 0, mm_mtc0_op, mm_pool32axf_op), RT | RS | RD },
+ { insn_mthi, M(mm_pool32a_op, 0, 0, 0, mm_mthi32_op, mm_pool32axf_op), RS },
+ { insn_mtlo, M(mm_pool32a_op, 0, 0, 0, mm_mtlo32_op, mm_pool32axf_op), RS },
{ insn_mul, M(mm_pool32a_op, 0, 0, 0, 0, mm_mul_op), RT | RS | RD },
{ insn_or, M(mm_pool32a_op, 0, 0, 0, 0, mm_or32_op), RT | RS | RD },
{ insn_ori, M(mm_ori32_op, 0, 0, 0, 0, 0), RT | RS | UIMM },
diff --git a/arch/mips/mm/uasm-mips.c b/arch/mips/mm/uasm-mips.c
index 2b7d85b8241f..86a3c76a1ad8 100644
--- a/arch/mips/mm/uasm-mips.c
+++ b/arch/mips/mm/uasm-mips.c
@@ -119,6 +119,8 @@ static struct insn insn_table[] = {
{ insn_mflo, M(spec_op, 0, 0, 0, 0, mflo_op), RD },
{ insn_mtc0, M(cop0_op, mtc_op, 0, 0, 0, 0), RT | RD | SET},
{ insn_mthc0, M(cop0_op, mthc0_op, 0, 0, 0, 0), RT | RD | SET},
+ { insn_mthi, M(spec_op, 0, 0, 0, 0, mthi_op), RS },
+ { insn_mtlo, M(spec_op, 0, 0, 0, 0, mtlo_op), RS },
{ insn_mul, M(spec2_op, 0, 0, 0, 0, mul_op), RS | RT | RD},
{ insn_ori, M(ori_op, 0, 0, 0, 0, 0), RS | RT | UIMM },
{ insn_or, M(spec_op, 0, 0, 0, 0, or_op), RS | RT | RD },
diff --git a/arch/mips/mm/uasm.c b/arch/mips/mm/uasm.c
index 006fb05b74a7..3e0282d301d6 100644
--- a/arch/mips/mm/uasm.c
+++ b/arch/mips/mm/uasm.c
@@ -56,11 +56,12 @@ enum opcode {
insn_ext, insn_ins, insn_j, insn_jal, insn_jalr, insn_jr, insn_lb,
insn_ld, insn_ldx, insn_lh, insn_ll, insn_lld, insn_lui, insn_lw,
insn_lwx, insn_mfc0, insn_mfhc0, insn_mfhi, insn_mflo, insn_mtc0,
- insn_mthc0, insn_mul, insn_or, insn_ori, insn_pref, insn_rfe, insn_rotr,
- insn_sc, insn_scd, insn_sd, insn_sll, insn_sllv, insn_slt, insn_sltiu,
- insn_sltu, insn_sra, insn_srl, insn_srlv, insn_subu, insn_sw, insn_sync,
- insn_syscall, insn_tlbp, insn_tlbr, insn_tlbwi, insn_tlbwr, insn_wait,
- insn_wsbh, insn_xor, insn_xori, insn_yield, insn_lddir, insn_ldpte,
+ insn_mthc0, insn_mthi, insn_mtlo, insn_mul, insn_or, insn_ori,
+ insn_pref, insn_rfe, insn_rotr, insn_sc, insn_scd, insn_sd, insn_sll,
+ insn_sllv, insn_slt, insn_sltiu, insn_sltu, insn_sra, insn_srl,
+ insn_srlv, insn_subu, insn_sw, insn_sync, insn_syscall, insn_tlbp,
+ insn_tlbr, insn_tlbwi, insn_tlbwr, insn_wait, insn_wsbh, insn_xor,
+ insn_xori, insn_yield, insn_lddir, insn_ldpte,
};
struct insn {
@@ -306,6 +307,8 @@ I_u1(_mfhi)
I_u1(_mflo)
I_u1u2u3(_mtc0)
I_u1u2u3(_mthc0)
+I_u1(_mthi)
+I_u1(_mtlo)
I_u3u1u2(_mul)
I_u2u1u3(_ori)
I_u3u1u2(_or)
--
2.4.10
^ permalink raw reply related [flat|nested] 36+ messages in thread
* [PATCH 05/14] MIPS: uasm: Add r6 MUL encoding
@ 2016-06-23 16:34 ` James Hogan
0 siblings, 0 replies; 36+ messages in thread
From: James Hogan @ 2016-06-23 16:34 UTC (permalink / raw)
To: Paolo Bonzini, Radim Krčmář, Ralf Baechle
Cc: James Hogan, linux-mips
Add the R6 MUL instruction encoding for 3 operand signed multiply to
uasm so that KVM can use uasm for generating its entry point code at
runtime on R6.
Signed-off-by: James Hogan <james.hogan@imgtec.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: linux-mips@linux-mips.org
---
arch/mips/include/uapi/asm/inst.h | 44 +++++++++++++++++++++++++++++++++++++++
arch/mips/mm/uasm-mips.c | 4 ++++
2 files changed, 48 insertions(+)
diff --git a/arch/mips/include/uapi/asm/inst.h b/arch/mips/include/uapi/asm/inst.h
index 6319c5037e66..fc96012c75d1 100644
--- a/arch/mips/include/uapi/asm/inst.h
+++ b/arch/mips/include/uapi/asm/inst.h
@@ -93,6 +93,50 @@ enum spec3_op {
};
/*
+ * Bits 10-6 minor opcode for r6 spec mult/div encodings
+ */
+enum mult_op {
+ mult_mult_op = 0x0,
+ mult_mul_op = 0x2,
+ mult_muh_op = 0x3,
+};
+enum multu_op {
+ multu_multu_op = 0x0,
+ multu_mulu_op = 0x2,
+ multu_muhu_op = 0x3,
+};
+enum div_op {
+ div_div_op = 0x0,
+ div_div6_op = 0x2,
+ div_mod_op = 0x3,
+};
+enum divu_op {
+ divu_divu_op = 0x0,
+ divu_divu6_op = 0x2,
+ divu_modu_op = 0x3,
+};
+enum dmult_op {
+ dmult_dmult_op = 0x0,
+ dmult_dmul_op = 0x2,
+ dmult_dmuh_op = 0x3,
+};
+enum dmultu_op {
+ dmultu_dmultu_op = 0x0,
+ dmultu_dmulu_op = 0x2,
+ dmultu_dmuhu_op = 0x3,
+};
+enum ddiv_op {
+ ddiv_ddiv_op = 0x0,
+ ddiv_ddiv6_op = 0x2,
+ ddiv_dmod_op = 0x3,
+};
+enum ddivu_op {
+ ddivu_ddivu_op = 0x0,
+ ddivu_ddivu6_op = 0x2,
+ ddivu_dmodu_op = 0x3,
+};
+
+/*
* rt field of bcond opcodes.
*/
enum rt_op {
diff --git a/arch/mips/mm/uasm-mips.c b/arch/mips/mm/uasm-mips.c
index 86a3c76a1ad8..cec524167822 100644
--- a/arch/mips/mm/uasm-mips.c
+++ b/arch/mips/mm/uasm-mips.c
@@ -121,7 +121,11 @@ static struct insn insn_table[] = {
{ insn_mthc0, M(cop0_op, mthc0_op, 0, 0, 0, 0), RT | RD | SET},
{ insn_mthi, M(spec_op, 0, 0, 0, 0, mthi_op), RS },
{ insn_mtlo, M(spec_op, 0, 0, 0, 0, mtlo_op), RS },
+#ifndef CONFIG_CPU_MIPSR6
{ insn_mul, M(spec2_op, 0, 0, 0, 0, mul_op), RS | RT | RD},
+#else
+ { insn_mul, M(spec_op, 0, 0, 0, mult_mul_op, mult_op), RS | RT | RD},
+#endif
{ insn_ori, M(ori_op, 0, 0, 0, 0, 0), RS | RT | UIMM },
{ insn_or, M(spec_op, 0, 0, 0, 0, or_op), RS | RT | RD },
#ifndef CONFIG_CPU_MIPSR6
--
2.4.10
^ permalink raw reply related [flat|nested] 36+ messages in thread* [PATCH 05/14] MIPS: uasm: Add r6 MUL encoding
@ 2016-06-23 16:34 ` James Hogan
0 siblings, 0 replies; 36+ messages in thread
From: James Hogan @ 2016-06-23 16:34 UTC (permalink / raw)
To: Paolo Bonzini, Radim Krčmář, Ralf Baechle
Cc: James Hogan, linux-mips
Add the R6 MUL instruction encoding for 3 operand signed multiply to
uasm so that KVM can use uasm for generating its entry point code at
runtime on R6.
Signed-off-by: James Hogan <james.hogan@imgtec.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: linux-mips@linux-mips.org
---
arch/mips/include/uapi/asm/inst.h | 44 +++++++++++++++++++++++++++++++++++++++
arch/mips/mm/uasm-mips.c | 4 ++++
2 files changed, 48 insertions(+)
diff --git a/arch/mips/include/uapi/asm/inst.h b/arch/mips/include/uapi/asm/inst.h
index 6319c5037e66..fc96012c75d1 100644
--- a/arch/mips/include/uapi/asm/inst.h
+++ b/arch/mips/include/uapi/asm/inst.h
@@ -93,6 +93,50 @@ enum spec3_op {
};
/*
+ * Bits 10-6 minor opcode for r6 spec mult/div encodings
+ */
+enum mult_op {
+ mult_mult_op = 0x0,
+ mult_mul_op = 0x2,
+ mult_muh_op = 0x3,
+};
+enum multu_op {
+ multu_multu_op = 0x0,
+ multu_mulu_op = 0x2,
+ multu_muhu_op = 0x3,
+};
+enum div_op {
+ div_div_op = 0x0,
+ div_div6_op = 0x2,
+ div_mod_op = 0x3,
+};
+enum divu_op {
+ divu_divu_op = 0x0,
+ divu_divu6_op = 0x2,
+ divu_modu_op = 0x3,
+};
+enum dmult_op {
+ dmult_dmult_op = 0x0,
+ dmult_dmul_op = 0x2,
+ dmult_dmuh_op = 0x3,
+};
+enum dmultu_op {
+ dmultu_dmultu_op = 0x0,
+ dmultu_dmulu_op = 0x2,
+ dmultu_dmuhu_op = 0x3,
+};
+enum ddiv_op {
+ ddiv_ddiv_op = 0x0,
+ ddiv_ddiv6_op = 0x2,
+ ddiv_dmod_op = 0x3,
+};
+enum ddivu_op {
+ ddivu_ddivu_op = 0x0,
+ ddivu_ddivu6_op = 0x2,
+ ddivu_dmodu_op = 0x3,
+};
+
+/*
* rt field of bcond opcodes.
*/
enum rt_op {
diff --git a/arch/mips/mm/uasm-mips.c b/arch/mips/mm/uasm-mips.c
index 86a3c76a1ad8..cec524167822 100644
--- a/arch/mips/mm/uasm-mips.c
+++ b/arch/mips/mm/uasm-mips.c
@@ -121,7 +121,11 @@ static struct insn insn_table[] = {
{ insn_mthc0, M(cop0_op, mthc0_op, 0, 0, 0, 0), RT | RD | SET},
{ insn_mthi, M(spec_op, 0, 0, 0, 0, mthi_op), RS },
{ insn_mtlo, M(spec_op, 0, 0, 0, 0, mtlo_op), RS },
+#ifndef CONFIG_CPU_MIPSR6
{ insn_mul, M(spec2_op, 0, 0, 0, 0, mul_op), RS | RT | RD},
+#else
+ { insn_mul, M(spec_op, 0, 0, 0, mult_mul_op, mult_op), RS | RT | RD},
+#endif
{ insn_ori, M(ori_op, 0, 0, 0, 0, 0), RS | RT | UIMM },
{ insn_or, M(spec_op, 0, 0, 0, 0, or_op), RS | RT | RD },
#ifndef CONFIG_CPU_MIPSR6
--
2.4.10
^ permalink raw reply related [flat|nested] 36+ messages in thread
* [PATCH 06/14] MIPS; KVM: Convert exception entry to uasm
@ 2016-06-23 16:34 ` James Hogan
0 siblings, 0 replies; 36+ messages in thread
From: James Hogan @ 2016-06-23 16:34 UTC (permalink / raw)
To: Paolo Bonzini, Radim Krčmář, Ralf Baechle
Cc: James Hogan, linux-mips, kvm
Convert the whole of locore.S (assembly to enter guest and handle
exception entry) to be generated dynamically with uasm. This is done
with minimal changes to the resulting code.
The main changes are:
- Some constants are generated by uasm using LUI+ADDIU instead of
LUI+ORI.
- Loading of lo and hi are swapped around in vcpu_run but not when
resuming the guest after an exit. Both bits of logic are now generated
by the same code.
- Register MOVEs in uasm use different ADDU operand ordering to GNU as,
putting zero register into rs instead of rt.
- The JALR.HB to call the C exit handler is switched to JALR, since the
hazard barrier would appear to be unnecessary.
This will allow further optimisation in the future to dynamically handle
the capabilities of the CPU.
Signed-off-by: James Hogan <james.hogan@imgtec.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Radim Krčmář <rkrcmar@redhat.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: linux-mips@linux-mips.org
Cc: kvm@vger.kernel.org
---
arch/mips/include/asm/kvm_host.h | 8 +-
arch/mips/kvm/Kconfig | 1 +
arch/mips/kvm/Makefile | 2 +-
arch/mips/kvm/{locore.S => entry.c} | 884 ++++++++++++++++++------------------
arch/mips/kvm/interrupt.h | 4 -
arch/mips/kvm/mips.c | 37 +-
6 files changed, 472 insertions(+), 464 deletions(-)
rename arch/mips/kvm/{locore.S => entry.c} (20%)
diff --git a/arch/mips/include/asm/kvm_host.h b/arch/mips/include/asm/kvm_host.h
index b0773c6d622f..2e76e899079c 100644
--- a/arch/mips/include/asm/kvm_host.h
+++ b/arch/mips/include/asm/kvm_host.h
@@ -533,8 +533,12 @@ int kvm_mips_emulation_init(struct kvm_mips_callbacks **install_callbacks);
/* Debug: dump vcpu state */
int kvm_arch_vcpu_dump_regs(struct kvm_vcpu *vcpu);
-/* Trampoline ASM routine to start running in "Guest" context */
-extern int __kvm_mips_vcpu_run(struct kvm_run *run, struct kvm_vcpu *vcpu);
+extern int kvm_mips_handle_exit(struct kvm_run *run, struct kvm_vcpu *vcpu);
+
+/* Building of entry/exception code */
+void *kvm_mips_build_vcpu_run(void *addr);
+void *kvm_mips_build_exception(void *addr);
+void *kvm_mips_build_exit(void *addr);
/* FPU/MSA context management */
void __kvm_save_fpu(struct kvm_vcpu_arch *vcpu);
diff --git a/arch/mips/kvm/Kconfig b/arch/mips/kvm/Kconfig
index 2ae12825529f..7c56d6b124d1 100644
--- a/arch/mips/kvm/Kconfig
+++ b/arch/mips/kvm/Kconfig
@@ -17,6 +17,7 @@ if VIRTUALIZATION
config KVM
tristate "Kernel-based Virtual Machine (KVM) support"
depends on HAVE_KVM
+ select EXPORT_UASM
select PREEMPT_NOTIFIERS
select ANON_INODES
select KVM_MMIO
diff --git a/arch/mips/kvm/Makefile b/arch/mips/kvm/Makefile
index 0aabe40fcac9..847429de780d 100644
--- a/arch/mips/kvm/Makefile
+++ b/arch/mips/kvm/Makefile
@@ -7,7 +7,7 @@ EXTRA_CFLAGS += -Ivirt/kvm -Iarch/mips/kvm
common-objs-$(CONFIG_CPU_HAS_MSA) += msa.o
-kvm-objs := $(common-objs-y) mips.o emulate.o locore.o \
+kvm-objs := $(common-objs-y) mips.o emulate.o entry.o \
interrupt.o stats.o commpage.o \
dyntrans.o trap_emul.o fpu.o
kvm-objs += mmu.o
diff --git a/arch/mips/kvm/locore.S b/arch/mips/kvm/entry.c
similarity index 20%
rename from arch/mips/kvm/locore.S
rename to arch/mips/kvm/entry.c
index 698286c0f732..9a18b4939b35 100644
--- a/arch/mips/kvm/locore.S
+++ b/arch/mips/kvm/entry.c
@@ -3,373 +3,445 @@
* License. See the file "COPYING" in the main directory of this archive
* for more details.
*
- * Main entry point for the guest, exception handling.
+ * Generation of main entry point for the guest, exception handling.
*
- * Copyright (C) 2012 MIPS Technologies, Inc. All rights reserved.
+ * Copyright (C) 2012 MIPS Technologies, Inc.
* Authors: Sanjay Lal <sanjayl@kymasys.com>
+ *
+ * Copyright (C) 2016 Imagination Technologies Ltd.
*/
-#include <asm/asm.h>
-#include <asm/asmmacro.h>
-#include <asm/regdef.h>
-#include <asm/mipsregs.h>
-#include <asm/stackframe.h>
-#include <asm/asm-offsets.h>
-
-#define _C_LABEL(x) x
-#define MIPSX(name) mips32_ ## name
+#include <linux/kvm_host.h>
+#include <asm/msa.h>
+#include <asm/setup.h>
+#include <asm/uasm.h>
+
+/* Register names */
+#define ZERO 0
+#define AT 1
+#define V0 2
+#define V1 3
+#define A0 4
+#define A1 5
+
+#if _MIPS_SIM == _MIPS_SIM_ABI32
+#define T0 8
+#define T1 9
+#define T2 10
+#define T3 11
+#endif /* _MIPS_SIM == _MIPS_SIM_ABI32 */
+
+#if _MIPS_SIM == _MIPS_SIM_ABI64 || _MIPS_SIM == _MIPS_SIM_NABI32
+#define T0 12
+#define T1 13
+#define T2 14
+#define T3 15
+#endif /* _MIPS_SIM == _MIPS_SIM_ABI64 || _MIPS_SIM == _MIPS_SIM_NABI32 */
+
+#define S0 16
+#define S1 17
+#define T9 25
+#define K0 26
+#define K1 27
+#define GP 28
+#define SP 29
+#define RA 31
+
+/* Some CP0 registers */
+#define C0_HWRENA 7, 0
+#define C0_BADVADDR 8, 0
+#define C0_ENTRYHI 10, 0
+#define C0_STATUS 12, 0
+#define C0_CAUSE 13, 0
+#define C0_EPC 14, 0
+#define C0_EBASE 15, 1
+#define C0_CONFIG3 16, 3
+#define C0_CONFIG5 16, 5
+#define C0_DDATA_LO 28, 3
+#define C0_ERROREPC 30, 0
+
#define CALLFRAME_SIZ 32
-/*
- * VECTOR
- * exception vector entrypoint
- */
-#define VECTOR(x, regmask) \
- .ent _C_LABEL(x),0; \
- EXPORT(x);
-
-#define VECTOR_END(x) \
- EXPORT(x);
-
-/* Overload, Danger Will Robinson!! */
-#define PT_HOST_USERLOCAL PT_EPC
+enum label_id {
+ label_fpu_1 = 1,
+ label_msa_1,
+ label_return_to_host,
+ label_kernel_asid,
+};
-#define CP0_DDATA_LO $28,3
+UASM_L_LA(_fpu_1)
+UASM_L_LA(_msa_1)
+UASM_L_LA(_return_to_host)
+UASM_L_LA(_kernel_asid)
-/* Resume Flags */
-#define RESUME_FLAG_HOST (1<<1) /* Resume host? */
+static void *kvm_mips_build_enter_guest(void *addr);
+static void *kvm_mips_build_ret_from_exit(void *addr);
+static void *kvm_mips_build_ret_to_guest(void *addr);
+static void *kvm_mips_build_ret_to_host(void *addr);
-#define RESUME_GUEST 0
-#define RESUME_HOST RESUME_FLAG_HOST
-
-/*
- * __kvm_mips_vcpu_run: entry point to the guest
- * a0: run
- * a1: vcpu
+/**
+ * kvm_mips_build_vcpu_run() - Assemble function to start running a guest VCPU.
+ * @addr: Address to start writing code.
+ *
+ * Assemble the start of the vcpu_run function to run a guest VCPU. The function
+ * conforms to the following prototype:
+ *
+ * int vcpu_run(struct kvm_run *run, struct kvm_vcpu *vcpu);
+ *
+ * The exit from the guest and return to the caller is handled by the code
+ * generated by kvm_mips_build_ret_to_host().
+ *
+ * Returns: Next address after end of written function.
*/
- .set noreorder
+void *kvm_mips_build_vcpu_run(void *addr)
+{
+ u32 *p = addr;
+ unsigned int i;
+
+ /*
+ * A0: run
+ * A1: vcpu
+ */
-FEXPORT(__kvm_mips_vcpu_run)
/* k0/k1 not being used in host kernel context */
- INT_ADDIU k1, sp, -PT_SIZE
- LONG_S $16, PT_R16(k1)
- LONG_S $17, PT_R17(k1)
- LONG_S $18, PT_R18(k1)
- LONG_S $19, PT_R19(k1)
- LONG_S $20, PT_R20(k1)
- LONG_S $21, PT_R21(k1)
- LONG_S $22, PT_R22(k1)
- LONG_S $23, PT_R23(k1)
-
- LONG_S $28, PT_R28(k1)
- LONG_S $29, PT_R29(k1)
- LONG_S $30, PT_R30(k1)
- LONG_S $31, PT_R31(k1)
+ uasm_i_addiu(&p, K1, SP, -(int)sizeof(struct pt_regs));
+ for (i = 16; i < 32; ++i) {
+ if (i == 24)
+ i = 28;
+ UASM_i_SW(&p, i, offsetof(struct pt_regs, regs[i]), K1);
+ }
/* Save hi/lo */
- mflo v0
- LONG_S v0, PT_LO(k1)
- mfhi v1
- LONG_S v1, PT_HI(k1)
+ uasm_i_mflo(&p, V0);
+ UASM_i_SW(&p, V0, offsetof(struct pt_regs, lo), K1);
+ uasm_i_mfhi(&p, V1);
+ UASM_i_SW(&p, V1, offsetof(struct pt_regs, hi), K1);
/* Save host status */
- mfc0 v0, CP0_STATUS
- LONG_S v0, PT_STATUS(k1)
+ uasm_i_mfc0(&p, V0, C0_STATUS);
+ UASM_i_SW(&p, V0, offsetof(struct pt_regs, cp0_status), K1);
/* Save DDATA_LO, will be used to store pointer to vcpu */
- mfc0 v1, CP0_DDATA_LO
- LONG_S v1, PT_HOST_USERLOCAL(k1)
+ uasm_i_mfc0(&p, V1, C0_DDATA_LO);
+ UASM_i_SW(&p, V1, offsetof(struct pt_regs, cp0_epc), K1);
/* DDATA_LO has pointer to vcpu */
- mtc0 a1, CP0_DDATA_LO
+ uasm_i_mtc0(&p, A1, C0_DDATA_LO);
/* Offset into vcpu->arch */
- INT_ADDIU k1, a1, VCPU_HOST_ARCH
+ uasm_i_addiu(&p, K1, A1, offsetof(struct kvm_vcpu, arch));
/*
* Save the host stack to VCPU, used for exception processing
* when we exit from the Guest
*/
- LONG_S sp, VCPU_HOST_STACK(k1)
+ UASM_i_SW(&p, SP, offsetof(struct kvm_vcpu_arch, host_stack), K1);
/* Save the kernel gp as well */
- LONG_S gp, VCPU_HOST_GP(k1)
+ UASM_i_SW(&p, GP, offsetof(struct kvm_vcpu_arch, host_gp), K1);
/*
* Setup status register for running the guest in UM, interrupts
* are disabled
*/
- li k0, (ST0_EXL | KSU_USER | ST0_BEV)
- mtc0 k0, CP0_STATUS
- ehb
+ UASM_i_LA(&p, K0, ST0_EXL | KSU_USER | ST0_BEV);
+ uasm_i_mtc0(&p, K0, C0_STATUS);
+ uasm_i_ehb(&p);
/* load up the new EBASE */
- LONG_L k0, VCPU_GUEST_EBASE(k1)
- mtc0 k0, CP0_EBASE
+ UASM_i_LW(&p, K0, offsetof(struct kvm_vcpu_arch, guest_ebase), K1);
+ uasm_i_mtc0(&p, K0, C0_EBASE);
/*
* Now that the new EBASE has been loaded, unset BEV, set
* interrupt mask as it was but make sure that timer interrupts
* are enabled
*/
- li k0, (ST0_EXL | KSU_USER | ST0_IE)
- andi v0, v0, ST0_IM
- or k0, k0, v0
- mtc0 k0, CP0_STATUS
- ehb
+ uasm_i_addiu(&p, K0, ZERO, ST0_EXL | KSU_USER | ST0_IE);
+ uasm_i_andi(&p, V0, V0, ST0_IM);
+ uasm_i_or(&p, K0, K0, V0);
+ uasm_i_mtc0(&p, K0, C0_STATUS);
+ uasm_i_ehb(&p);
+
+ p = kvm_mips_build_enter_guest(p);
+
+ return p;
+}
+
+/**
+ * kvm_mips_build_enter_guest() - Assemble code to resume guest execution.
+ * @addr: Address to start writing code.
+ *
+ * Assemble the code to resume guest execution. This code is common between the
+ * initial entry into the guest from the host, and returning from the exit
+ * handler back to the guest.
+ *
+ * Returns: Next address after end of written function.
+ */
+static void *kvm_mips_build_enter_guest(void *addr)
+{
+ u32 *p = addr;
+ unsigned int i;
+ struct uasm_label labels[2];
+ struct uasm_reloc relocs[2];
+ struct uasm_label *l = labels;
+ struct uasm_reloc *r = relocs;
+
+ memset(labels, 0, sizeof(labels));
+ memset(relocs, 0, sizeof(relocs));
/* Set Guest EPC */
- LONG_L t0, VCPU_PC(k1)
- mtc0 t0, CP0_EPC
+ UASM_i_LW(&p, T0, offsetof(struct kvm_vcpu_arch, pc), K1);
+ uasm_i_mtc0(&p, T0, C0_EPC);
-FEXPORT(__kvm_mips_load_asid)
/* Set the ASID for the Guest Kernel */
- PTR_L t0, VCPU_COP0(k1)
- LONG_L t0, COP0_STATUS(t0)
- andi t0, KSU_USER | ST0_ERL | ST0_EXL
- xori t0, KSU_USER
- bnez t0, 1f /* If kernel */
- INT_ADDIU t1, k1, VCPU_GUEST_KERNEL_ASID /* (BD) */
- INT_ADDIU t1, k1, VCPU_GUEST_USER_ASID /* else user */
-1:
- /* t1: contains the base of the ASID array, need to get the cpu id */
- LONG_L t2, TI_CPU($28) /* smp_processor_id */
- INT_SLL t2, t2, 2 /* x4 */
- REG_ADDU t3, t1, t2
- LONG_L k0, (t3)
+ UASM_i_LW(&p, T0, offsetof(struct kvm_vcpu_arch, cop0), K1);
+ UASM_i_LW(&p, T0, offsetof(struct mips_coproc, reg[MIPS_CP0_STATUS][0]),
+ T0);
+ uasm_i_andi(&p, T0, T0, KSU_USER | ST0_ERL | ST0_EXL);
+ uasm_i_xori(&p, T0, T0, KSU_USER);
+ uasm_il_bnez(&p, &r, T0, label_kernel_asid);
+ uasm_i_addiu(&p, T1, K1,
+ offsetof(struct kvm_vcpu_arch, guest_kernel_asid));
+ /* else user */
+ uasm_i_addiu(&p, T1, K1,
+ offsetof(struct kvm_vcpu_arch, guest_user_asid));
+ uasm_l_kernel_asid(&l, p);
+
+ /* t1: contains the base of the ASID array, need to get the cpu id */
+ /* smp_processor_id */
+ UASM_i_LW(&p, T2, offsetof(struct thread_info, cpu), GP);
+ /* x4 */
+ uasm_i_sll(&p, T2, T2, 2);
+ UASM_i_ADDU(&p, T3, T1, T2);
+ UASM_i_LW(&p, K0, 0, T3);
#ifdef CONFIG_MIPS_ASID_BITS_VARIABLE
- li t3, CPUINFO_SIZE/4
- mul t2, t2, t3 /* x sizeof(struct cpuinfo_mips)/4 */
- LONG_L t2, (cpu_data + CPUINFO_ASID_MASK)(t2)
- and k0, k0, t2
+ /* x sizeof(struct cpuinfo_mips)/4 */
+ uasm_i_addiu(&p, T3, ZERO, sizeof(struct cpuinfo_mips)/4);
+ uasm_i_mul(&p, T2, T2, T3);
+
+ UASM_i_LA_mostly(&p, AT, (long)&cpu_data[0].asid_mask);
+ UASM_i_ADDU(&p, AT, AT, T2);
+ UASM_i_LW(&p, T2, uasm_rel_lo((long)&cpu_data[0].asid_mask), AT);
+ uasm_i_and(&p, K0, K0, T2);
#else
- andi k0, k0, MIPS_ENTRYHI_ASID
+ uasm_i_andi(&p, K0, K0, MIPS_ENTRYHI_ASID);
#endif
- mtc0 k0, CP0_ENTRYHI
- ehb
+ uasm_i_mtc0(&p, K0, C0_ENTRYHI);
+ uasm_i_ehb(&p);
/* Disable RDHWR access */
- mtc0 zero, CP0_HWRENA
-
- .set noat
- /* Now load up the Guest Context from VCPU */
- LONG_L $1, VCPU_R1(k1)
- LONG_L $2, VCPU_R2(k1)
- LONG_L $3, VCPU_R3(k1)
-
- LONG_L $4, VCPU_R4(k1)
- LONG_L $5, VCPU_R5(k1)
- LONG_L $6, VCPU_R6(k1)
- LONG_L $7, VCPU_R7(k1)
-
- LONG_L $8, VCPU_R8(k1)
- LONG_L $9, VCPU_R9(k1)
- LONG_L $10, VCPU_R10(k1)
- LONG_L $11, VCPU_R11(k1)
- LONG_L $12, VCPU_R12(k1)
- LONG_L $13, VCPU_R13(k1)
- LONG_L $14, VCPU_R14(k1)
- LONG_L $15, VCPU_R15(k1)
- LONG_L $16, VCPU_R16(k1)
- LONG_L $17, VCPU_R17(k1)
- LONG_L $18, VCPU_R18(k1)
- LONG_L $19, VCPU_R19(k1)
- LONG_L $20, VCPU_R20(k1)
- LONG_L $21, VCPU_R21(k1)
- LONG_L $22, VCPU_R22(k1)
- LONG_L $23, VCPU_R23(k1)
- LONG_L $24, VCPU_R24(k1)
- LONG_L $25, VCPU_R25(k1)
-
- /* k0/k1 loaded up later */
-
- LONG_L $28, VCPU_R28(k1)
- LONG_L $29, VCPU_R29(k1)
- LONG_L $30, VCPU_R30(k1)
- LONG_L $31, VCPU_R31(k1)
+ uasm_i_mtc0(&p, ZERO, C0_HWRENA);
+
+ /* load the guest context from VCPU and return */
+ for (i = 1; i < 32; ++i) {
+ /* Guest k0/k1 loaded later */
+ if (i == K0 || i == K1)
+ continue;
+ UASM_i_LW(&p, i, offsetof(struct kvm_vcpu_arch, gprs[i]), K1);
+ }
/* Restore hi/lo */
- LONG_L k0, VCPU_LO(k1)
- mtlo k0
+ UASM_i_LW(&p, K0, offsetof(struct kvm_vcpu_arch, hi), K1);
+ uasm_i_mthi(&p, K0);
- LONG_L k0, VCPU_HI(k1)
- mthi k0
+ UASM_i_LW(&p, K0, offsetof(struct kvm_vcpu_arch, lo), K1);
+ uasm_i_mtlo(&p, K0);
-FEXPORT(__kvm_mips_load_k0k1)
/* Restore the guest's k0/k1 registers */
- LONG_L k0, VCPU_R26(k1)
- LONG_L k1, VCPU_R27(k1)
+ UASM_i_LW(&p, K0, offsetof(struct kvm_vcpu_arch, gprs[K0]), K1);
+ UASM_i_LW(&p, K1, offsetof(struct kvm_vcpu_arch, gprs[K1]), K1);
/* Jump to guest */
- eret
-EXPORT(__kvm_mips_vcpu_run_end)
-
-VECTOR(MIPSX(exception), unknown)
-/* Find out what mode we came from and jump to the proper handler. */
- mtc0 k0, CP0_ERROREPC #01: Save guest k0
- ehb #02:
-
- mfc0 k0, CP0_EBASE #02: Get EBASE
- INT_SRL k0, k0, 10 #03: Get rid of CPUNum
- INT_SLL k0, k0, 10 #04
- LONG_S k1, 0x3000(k0) #05: Save k1 @ offset 0x3000
- INT_ADDIU k0, k0, 0x2000 #06: Exception handler is
- # installed @ offset 0x2000
- j k0 #07: jump to the function
- nop #08: branch delay slot
-VECTOR_END(MIPSX(exceptionEnd))
-.end MIPSX(exception)
-
-/*
- * Generic Guest exception handler. We end up here when the guest
- * does something that causes a trap to kernel mode.
+ uasm_i_eret(&p);
+
+ uasm_resolve_relocs(relocs, labels);
+
+ return p;
+}
+
+/**
+ * kvm_mips_build_exception() - Assemble first level guest exception handler.
+ * @addr: Address to start writing code.
+ *
+ * Assemble exception vector code for guest execution. The generated vector will
+ * jump to the common exception handler generated by kvm_mips_build_exit().
+ *
+ * Returns: Next address after end of written function.
+ */
+void *kvm_mips_build_exception(void *addr)
+{
+ u32 *p = addr;
+
+ /* Save guest k0 */
+ uasm_i_mtc0(&p, K0, C0_ERROREPC);
+ uasm_i_ehb(&p);
+
+ /* Get EBASE */
+ uasm_i_mfc0(&p, K0, C0_EBASE);
+ /* Get rid of CPUNum */
+ uasm_i_srl(&p, K0, K0, 10);
+ uasm_i_sll(&p, K0, K0, 10);
+ /* Save k1 @ offset 0x3000 */
+ UASM_i_SW(&p, K1, 0x3000, K0);
+
+ /* Exception handler is installed @ offset 0x2000 */
+ uasm_i_addiu(&p, K0, K0, 0x2000);
+ /* Jump to the function */
+ uasm_i_jr(&p, K0);
+ uasm_i_nop(&p);
+
+ return p;
+}
+
+/**
+ * kvm_mips_build_exit() - Assemble common guest exit handler.
+ * @addr: Address to start writing code.
+ *
+ * Assemble the generic guest exit handling code. This is called by the
+ * exception vectors (generated by kvm_mips_build_exception()), and calls
+ * kvm_mips_handle_exit(), then either resumes the guest or returns to the host
+ * depending on the return value.
+ *
+ * Returns: Next address after end of written function.
*/
-NESTED (MIPSX(GuestException), CALLFRAME_SIZ, ra)
- /* Get the VCPU pointer from DDTATA_LO */
- mfc0 k1, CP0_DDATA_LO
- INT_ADDIU k1, k1, VCPU_HOST_ARCH
+void *kvm_mips_build_exit(void *addr)
+{
+ u32 *p = addr;
+ unsigned int i;
+ struct uasm_label labels[3];
+ struct uasm_reloc relocs[3];
+ struct uasm_label *l = labels;
+ struct uasm_reloc *r = relocs;
+
+ memset(labels, 0, sizeof(labels));
+ memset(relocs, 0, sizeof(relocs));
+
+ /*
+ * Generic Guest exception handler. We end up here when the guest
+ * does something that causes a trap to kernel mode.
+ */
+
+ /* Get the VCPU pointer from DDATA_LO */
+ uasm_i_mfc0(&p, K1, C0_DDATA_LO);
+ uasm_i_addiu(&p, K1, K1, offsetof(struct kvm_vcpu, arch));
/* Start saving Guest context to VCPU */
- LONG_S $0, VCPU_R0(k1)
- LONG_S $1, VCPU_R1(k1)
- LONG_S $2, VCPU_R2(k1)
- LONG_S $3, VCPU_R3(k1)
- LONG_S $4, VCPU_R4(k1)
- LONG_S $5, VCPU_R5(k1)
- LONG_S $6, VCPU_R6(k1)
- LONG_S $7, VCPU_R7(k1)
- LONG_S $8, VCPU_R8(k1)
- LONG_S $9, VCPU_R9(k1)
- LONG_S $10, VCPU_R10(k1)
- LONG_S $11, VCPU_R11(k1)
- LONG_S $12, VCPU_R12(k1)
- LONG_S $13, VCPU_R13(k1)
- LONG_S $14, VCPU_R14(k1)
- LONG_S $15, VCPU_R15(k1)
- LONG_S $16, VCPU_R16(k1)
- LONG_S $17, VCPU_R17(k1)
- LONG_S $18, VCPU_R18(k1)
- LONG_S $19, VCPU_R19(k1)
- LONG_S $20, VCPU_R20(k1)
- LONG_S $21, VCPU_R21(k1)
- LONG_S $22, VCPU_R22(k1)
- LONG_S $23, VCPU_R23(k1)
- LONG_S $24, VCPU_R24(k1)
- LONG_S $25, VCPU_R25(k1)
-
- /* Guest k0/k1 saved later */
-
- LONG_S $28, VCPU_R28(k1)
- LONG_S $29, VCPU_R29(k1)
- LONG_S $30, VCPU_R30(k1)
- LONG_S $31, VCPU_R31(k1)
-
- .set at
+ for (i = 0; i < 32; ++i) {
+ /* Guest k0/k1 saved later */
+ if (i == K0 || i == K1)
+ continue;
+ UASM_i_SW(&p, i, offsetof(struct kvm_vcpu_arch, gprs[i]), K1);
+ }
/* We need to save hi/lo and restore them on the way out */
- mfhi t0
- LONG_S t0, VCPU_HI(k1)
+ uasm_i_mfhi(&p, T0);
+ UASM_i_SW(&p, T0, offsetof(struct kvm_vcpu_arch, hi), K1);
- mflo t0
- LONG_S t0, VCPU_LO(k1)
+ uasm_i_mflo(&p, T0);
+ UASM_i_SW(&p, T0, offsetof(struct kvm_vcpu_arch, lo), K1);
/* Finally save guest k0/k1 to VCPU */
- mfc0 t0, CP0_ERROREPC
- LONG_S t0, VCPU_R26(k1)
+ uasm_i_mfc0(&p, T0, C0_ERROREPC);
+ UASM_i_SW(&p, T0, offsetof(struct kvm_vcpu_arch, gprs[K0]), K1);
/* Get GUEST k1 and save it in VCPU */
- PTR_LI t1, ~0x2ff
- mfc0 t0, CP0_EBASE
- and t0, t0, t1
- LONG_L t0, 0x3000(t0)
- LONG_S t0, VCPU_R27(k1)
+ uasm_i_addiu(&p, T1, ZERO, ~0x2ff);
+ uasm_i_mfc0(&p, T0, C0_EBASE);
+ uasm_i_and(&p, T0, T0, T1);
+ UASM_i_LW(&p, T0, 0x3000, T0);
+ UASM_i_SW(&p, T0, offsetof(struct kvm_vcpu_arch, gprs[K1]), K1);
/* Now that context has been saved, we can use other registers */
/* Restore vcpu */
- mfc0 a1, CP0_DDATA_LO
- move s1, a1
+ uasm_i_mfc0(&p, A1, C0_DDATA_LO);
+ uasm_i_move(&p, S1, A1);
/* Restore run (vcpu->run) */
- LONG_L a0, VCPU_RUN(a1)
+ UASM_i_LW(&p, A0, offsetof(struct kvm_vcpu, run), A1);
/* Save pointer to run in s0, will be saved by the compiler */
- move s0, a0
+ uasm_i_move(&p, S0, A0);
/*
- * Save Host level EPC, BadVaddr and Cause to VCPU, useful to
- * process the exception
+ * Save Host level EPC, BadVaddr and Cause to VCPU, useful to process
+ * the exception
*/
- mfc0 k0,CP0_EPC
- LONG_S k0, VCPU_PC(k1)
+ uasm_i_mfc0(&p, K0, C0_EPC);
+ UASM_i_SW(&p, K0, offsetof(struct kvm_vcpu_arch, pc), K1);
- mfc0 k0, CP0_BADVADDR
- LONG_S k0, VCPU_HOST_CP0_BADVADDR(k1)
+ uasm_i_mfc0(&p, K0, C0_BADVADDR);
+ UASM_i_SW(&p, K0, offsetof(struct kvm_vcpu_arch, host_cp0_badvaddr),
+ K1);
- mfc0 k0, CP0_CAUSE
- sw k0, VCPU_HOST_CP0_CAUSE(k1)
+ uasm_i_mfc0(&p, K0, C0_CAUSE);
+ uasm_i_sw(&p, K0, offsetof(struct kvm_vcpu_arch, host_cp0_cause), K1);
/* Now restore the host state just enough to run the handlers */
/* Switch EBASE to the one used by Linux */
/* load up the host EBASE */
- mfc0 v0, CP0_STATUS
+ uasm_i_mfc0(&p, V0, C0_STATUS);
- or k0, v0, ST0_BEV
+ uasm_i_lui(&p, AT, ST0_BEV >> 16);
+ uasm_i_or(&p, K0, V0, AT);
- mtc0 k0, CP0_STATUS
- ehb
+ uasm_i_mtc0(&p, K0, C0_STATUS);
+ uasm_i_ehb(&p);
- LONG_L k0, ebase
- mtc0 k0,CP0_EBASE
+ UASM_i_LA_mostly(&p, K0, (long)&ebase);
+ UASM_i_LW(&p, K0, uasm_rel_lo((long)&ebase), K0);
+ uasm_i_mtc0(&p, K0, C0_EBASE);
/*
* If FPU is enabled, save FCR31 and clear it so that later ctc1's don't
* trigger FPE for pending exceptions.
*/
- and v1, v0, ST0_CU1
- beqz v1, 1f
- nop
- .set push
- SET_HARDFLOAT
- cfc1 t0, fcr31
- sw t0, VCPU_FCR31(k1)
- ctc1 zero,fcr31
- .set pop
-1:
+ uasm_i_lui(&p, AT, ST0_CU1 >> 16);
+ uasm_i_and(&p, V1, V0, AT);
+ uasm_il_beqz(&p, &r, V1, label_fpu_1);
+ uasm_i_nop(&p);
+ uasm_i_cfc1(&p, T0, 31);
+ uasm_i_sw(&p, T0, offsetof(struct kvm_vcpu_arch, fpu.fcr31), K1);
+ uasm_i_ctc1(&p, ZERO, 31);
+ uasm_l_fpu_1(&l, p);
#ifdef CONFIG_CPU_HAS_MSA
/*
* If MSA is enabled, save MSACSR and clear it so that later
* instructions don't trigger MSAFPE for pending exceptions.
*/
- mfc0 t0, CP0_CONFIG3
- ext t0, t0, 28, 1 /* MIPS_CONF3_MSAP */
- beqz t0, 1f
- nop
- mfc0 t0, CP0_CONFIG5
- ext t0, t0, 27, 1 /* MIPS_CONF5_MSAEN */
- beqz t0, 1f
- nop
- _cfcmsa t0, MSA_CSR
- sw t0, VCPU_MSA_CSR(k1)
- _ctcmsa MSA_CSR, zero
-1:
+ uasm_i_mfc0(&p, T0, C0_CONFIG3);
+ uasm_i_ext(&p, T0, T0, 28, 1); /* MIPS_CONF3_MSAP */
+ uasm_il_beqz(&p, &r, T0, label_msa_1);
+ uasm_i_nop(&p);
+ uasm_i_mfc0(&p, T0, C0_CONFIG5);
+ uasm_i_ext(&p, T0, T0, 27, 1); /* MIPS_CONF5_MSAEN */
+ uasm_il_beqz(&p, &r, T0, label_msa_1);
+ uasm_i_nop(&p);
+ uasm_i_cfcmsa(&p, T0, MSA_CSR);
+ uasm_i_sw(&p, T0, offsetof(struct kvm_vcpu_arch, fpu.msacsr),
+ K1);
+ uasm_i_ctcmsa(&p, MSA_CSR, ZERO);
+ uasm_l_msa_1(&l, p);
#endif
/* Now that the new EBASE has been loaded, unset BEV and KSU_USER */
- and v0, v0, ~(ST0_EXL | KSU_USER | ST0_IE)
- or v0, v0, ST0_CU0
- mtc0 v0, CP0_STATUS
- ehb
+ uasm_i_addiu(&p, AT, ZERO, ~(ST0_EXL | KSU_USER | ST0_IE));
+ uasm_i_and(&p, V0, V0, AT);
+ uasm_i_lui(&p, AT, ST0_CU0 >> 16);
+ uasm_i_or(&p, V0, V0, AT);
+ uasm_i_mtc0(&p, V0, C0_STATUS);
+ uasm_i_ehb(&p);
/* Load up host GP */
- LONG_L gp, VCPU_HOST_GP(k1)
+ UASM_i_LW(&p, GP, offsetof(struct kvm_vcpu_arch, host_gp), K1);
/* Need a stack before we can jump to "C" */
- LONG_L sp, VCPU_HOST_STACK(k1)
+ UASM_i_LW(&p, SP, offsetof(struct kvm_vcpu_arch, host_stack), K1);
/* Saved host state */
- INT_ADDIU sp, sp, -PT_SIZE
+ uasm_i_addiu(&p, SP, SP, -(int)sizeof(struct pt_regs));
/*
* XXXKYMA do we need to load the host ASID, maybe not because the
@@ -377,27 +449,54 @@ NESTED (MIPSX(GuestException), CALLFRAME_SIZ, ra)
*/
/* Restore host DDATA_LO */
- LONG_L k0, PT_HOST_USERLOCAL(sp)
- mtc0 k0, CP0_DDATA_LO
+ UASM_i_LW(&p, K0, offsetof(struct pt_regs, cp0_epc), SP);
+ uasm_i_mtc0(&p, K0, C0_DDATA_LO);
/* Restore RDHWR access */
- INT_L k0, hwrena
- mtc0 k0, CP0_HWRENA
+ UASM_i_LA_mostly(&p, K0, (long)&hwrena);
+ uasm_i_lw(&p, K0, uasm_rel_lo((long)&hwrena), K0);
+ uasm_i_mtc0(&p, K0, C0_HWRENA);
/* Jump to handler */
-FEXPORT(__kvm_mips_jump_to_handler)
/*
* XXXKYMA: not sure if this is safe, how large is the stack??
* Now jump to the kvm_mips_handle_exit() to see if we can deal
* with this in the kernel
*/
- PTR_LA t9, kvm_mips_handle_exit
- jalr.hb t9
- INT_ADDIU sp, sp, -CALLFRAME_SIZ /* BD Slot */
+ UASM_i_LA(&p, T9, (unsigned long)kvm_mips_handle_exit);
+ uasm_i_jalr(&p, RA, T9);
+ uasm_i_addiu(&p, SP, SP, -CALLFRAME_SIZ);
+
+ uasm_resolve_relocs(relocs, labels);
+
+ p = kvm_mips_build_ret_from_exit(p);
+
+ return p;
+}
+
+/**
+ * kvm_mips_build_ret_from_exit() - Assemble guest exit return handler.
+ * @addr: Address to start writing code.
+ *
+ * Assemble the code to handle the return from kvm_mips_handle_exit(), either
+ * resuming the guest or returning to the host depending on the return value.
+ *
+ * Returns: Next address after end of written function.
+ */
+static void *kvm_mips_build_ret_from_exit(void *addr)
+{
+ u32 *p = addr;
+ struct uasm_label labels[2];
+ struct uasm_reloc relocs[2];
+ struct uasm_label *l = labels;
+ struct uasm_reloc *r = relocs;
+
+ memset(labels, 0, sizeof(labels));
+ memset(relocs, 0, sizeof(relocs));
/* Return from handler Make sure interrupts are disabled */
- di
- ehb
+ uasm_i_di(&p, ZERO);
+ uasm_i_ehb(&p);
/*
* XXXKYMA: k0/k1 could have been blown away if we processed
@@ -405,198 +504,119 @@ FEXPORT(__kvm_mips_jump_to_handler)
* guest, reload k1
*/
- move k1, s1
- INT_ADDIU k1, k1, VCPU_HOST_ARCH
+ uasm_i_move(&p, K1, S1);
+ uasm_i_addiu(&p, K1, K1, offsetof(struct kvm_vcpu, arch));
/*
* Check return value, should tell us if we are returning to the
* host (handle I/O etc)or resuming the guest
*/
- andi t0, v0, RESUME_HOST
- bnez t0, __kvm_mips_return_to_host
- nop
+ uasm_i_andi(&p, T0, V0, RESUME_HOST);
+ uasm_il_bnez(&p, &r, T0, label_return_to_host);
+ uasm_i_nop(&p);
+
+ p = kvm_mips_build_ret_to_guest(p);
+
+ uasm_l_return_to_host(&l, p);
+ p = kvm_mips_build_ret_to_host(p);
+
+ uasm_resolve_relocs(relocs, labels);
+
+ return p;
+}
+
+/**
+ * kvm_mips_build_ret_to_guest() - Assemble code to return to the guest.
+ * @addr: Address to start writing code.
+ *
+ * Assemble the code to handle return from the guest exit handler
+ * (kvm_mips_handle_exit()) back to the guest.
+ *
+ * Returns: Next address after end of written function.
+ */
+static void *kvm_mips_build_ret_to_guest(void *addr)
+{
+ u32 *p = addr;
-__kvm_mips_return_to_guest:
/* Put the saved pointer to vcpu (s1) back into the DDATA_LO Register */
- mtc0 s1, CP0_DDATA_LO
+ uasm_i_mtc0(&p, S1, C0_DDATA_LO);
/* Load up the Guest EBASE to minimize the window where BEV is set */
- LONG_L t0, VCPU_GUEST_EBASE(k1)
+ UASM_i_LW(&p, T0, offsetof(struct kvm_vcpu_arch, guest_ebase), K1);
/* Switch EBASE back to the one used by KVM */
- mfc0 v1, CP0_STATUS
- or k0, v1, ST0_BEV
- mtc0 k0, CP0_STATUS
- ehb
- mtc0 t0, CP0_EBASE
+ uasm_i_mfc0(&p, V1, C0_STATUS);
+ uasm_i_lui(&p, AT, ST0_BEV >> 16);
+ uasm_i_or(&p, K0, V1, AT);
+ uasm_i_mtc0(&p, K0, C0_STATUS);
+ uasm_i_ehb(&p);
+ uasm_i_mtc0(&p, T0, C0_EBASE);
/* Setup status register for running guest in UM */
- or v1, v1, (ST0_EXL | KSU_USER | ST0_IE)
- and v1, v1, ~(ST0_CU0 | ST0_MX)
- mtc0 v1, CP0_STATUS
- ehb
+ uasm_i_ori(&p, V1, V1, ST0_EXL | KSU_USER | ST0_IE);
+ UASM_i_LA(&p, AT, ~(ST0_CU0 | ST0_MX));
+ uasm_i_and(&p, V1, V1, AT);
+ uasm_i_mtc0(&p, V1, C0_STATUS);
+ uasm_i_ehb(&p);
+
+ p = kvm_mips_build_enter_guest(p);
+
+ return p;
+}
+
+/**
+ * kvm_mips_build_ret_to_host() - Assemble code to return to the host.
+ * @addr: Address to start writing code.
+ *
+ * Assemble the code to handle return from the guest exit handler
+ * (kvm_mips_handle_exit()) back to the host, i.e. to the caller of the vcpu_run
+ * function generated by kvm_mips_build_vcpu_run().
+ *
+ * Returns: Next address after end of written function.
+ */
+static void *kvm_mips_build_ret_to_host(void *addr)
+{
+ u32 *p = addr;
+ unsigned int i;
- /* Set Guest EPC */
- LONG_L t0, VCPU_PC(k1)
- mtc0 t0, CP0_EPC
-
- /* Set the ASID for the Guest Kernel */
- PTR_L t0, VCPU_COP0(k1)
- LONG_L t0, COP0_STATUS(t0)
- andi t0, KSU_USER | ST0_ERL | ST0_EXL
- xori t0, KSU_USER
- bnez t0, 1f /* If kernel */
- INT_ADDIU t1, k1, VCPU_GUEST_KERNEL_ASID /* (BD) */
- INT_ADDIU t1, k1, VCPU_GUEST_USER_ASID /* else user */
-1:
- /* t1: contains the base of the ASID array, need to get the cpu id */
- LONG_L t2, TI_CPU($28) /* smp_processor_id */
- INT_SLL t2, t2, 2 /* x4 */
- REG_ADDU t3, t1, t2
- LONG_L k0, (t3)
-#ifdef CONFIG_MIPS_ASID_BITS_VARIABLE
- li t3, CPUINFO_SIZE/4
- mul t2, t2, t3 /* x sizeof(struct cpuinfo_mips)/4 */
- LONG_L t2, (cpu_data + CPUINFO_ASID_MASK)(t2)
- and k0, k0, t2
-#else
- andi k0, k0, MIPS_ENTRYHI_ASID
-#endif
- mtc0 k0, CP0_ENTRYHI
- ehb
-
- /* Disable RDHWR access */
- mtc0 zero, CP0_HWRENA
-
- .set noat
- /* load the guest context from VCPU and return */
- LONG_L $0, VCPU_R0(k1)
- LONG_L $1, VCPU_R1(k1)
- LONG_L $2, VCPU_R2(k1)
- LONG_L $3, VCPU_R3(k1)
- LONG_L $4, VCPU_R4(k1)
- LONG_L $5, VCPU_R5(k1)
- LONG_L $6, VCPU_R6(k1)
- LONG_L $7, VCPU_R7(k1)
- LONG_L $8, VCPU_R8(k1)
- LONG_L $9, VCPU_R9(k1)
- LONG_L $10, VCPU_R10(k1)
- LONG_L $11, VCPU_R11(k1)
- LONG_L $12, VCPU_R12(k1)
- LONG_L $13, VCPU_R13(k1)
- LONG_L $14, VCPU_R14(k1)
- LONG_L $15, VCPU_R15(k1)
- LONG_L $16, VCPU_R16(k1)
- LONG_L $17, VCPU_R17(k1)
- LONG_L $18, VCPU_R18(k1)
- LONG_L $19, VCPU_R19(k1)
- LONG_L $20, VCPU_R20(k1)
- LONG_L $21, VCPU_R21(k1)
- LONG_L $22, VCPU_R22(k1)
- LONG_L $23, VCPU_R23(k1)
- LONG_L $24, VCPU_R24(k1)
- LONG_L $25, VCPU_R25(k1)
-
- /* $/k1 loaded later */
- LONG_L $28, VCPU_R28(k1)
- LONG_L $29, VCPU_R29(k1)
- LONG_L $30, VCPU_R30(k1)
- LONG_L $31, VCPU_R31(k1)
-
-FEXPORT(__kvm_mips_skip_guest_restore)
- LONG_L k0, VCPU_HI(k1)
- mthi k0
-
- LONG_L k0, VCPU_LO(k1)
- mtlo k0
-
- LONG_L k0, VCPU_R26(k1)
- LONG_L k1, VCPU_R27(k1)
-
- eret
- .set at
-
-__kvm_mips_return_to_host:
/* EBASE is already pointing to Linux */
- LONG_L k1, VCPU_HOST_STACK(k1)
- INT_ADDIU k1,k1, -PT_SIZE
+ UASM_i_LW(&p, K1, offsetof(struct kvm_vcpu_arch, host_stack), K1);
+ uasm_i_addiu(&p, K1, K1, -(int)sizeof(struct pt_regs));
/* Restore host DDATA_LO */
- LONG_L k0, PT_HOST_USERLOCAL(k1)
- mtc0 k0, CP0_DDATA_LO
+ UASM_i_LW(&p, K0, offsetof(struct pt_regs, cp0_epc), K1);
+ uasm_i_mtc0(&p, K0, C0_DDATA_LO);
/*
* r2/v0 is the return code, shift it down by 2 (arithmetic)
* to recover the err code
*/
- INT_SRA k0, v0, 2
- move $2, k0
+ uasm_i_sra(&p, K0, V0, 2);
+ uasm_i_move(&p, V0, K0);
/* Load context saved on the host stack */
- LONG_L $16, PT_R16(k1)
- LONG_L $17, PT_R17(k1)
- LONG_L $18, PT_R18(k1)
- LONG_L $19, PT_R19(k1)
- LONG_L $20, PT_R20(k1)
- LONG_L $21, PT_R21(k1)
- LONG_L $22, PT_R22(k1)
- LONG_L $23, PT_R23(k1)
+ for (i = 16; i < 31; ++i) {
+ if (i == 24)
+ i = 28;
+ UASM_i_LW(&p, i, offsetof(struct pt_regs, regs[i]), K1);
+ }
- LONG_L $28, PT_R28(k1)
- LONG_L $29, PT_R29(k1)
- LONG_L $30, PT_R30(k1)
+ UASM_i_LW(&p, K0, offsetof(struct pt_regs, hi), K1);
+ uasm_i_mthi(&p, K0);
- LONG_L k0, PT_HI(k1)
- mthi k0
-
- LONG_L k0, PT_LO(k1)
- mtlo k0
+ UASM_i_LW(&p, K0, offsetof(struct pt_regs, lo), K1);
+ uasm_i_mtlo(&p, K0);
/* Restore RDHWR access */
- INT_L k0, hwrena
- mtc0 k0, CP0_HWRENA
+ UASM_i_LA_mostly(&p, K0, (long)&hwrena);
+ uasm_i_lw(&p, K0, uasm_rel_lo((long)&hwrena), K0);
+ uasm_i_mtc0(&p, K0, C0_HWRENA);
/* Restore RA, which is the address we will return to */
- LONG_L ra, PT_R31(k1)
- j ra
- nop
+ UASM_i_LW(&p, RA, offsetof(struct pt_regs, regs[RA]), K1);
+ uasm_i_jr(&p, RA);
+ uasm_i_nop(&p);
-VECTOR_END(MIPSX(GuestExceptionEnd))
-.end MIPSX(GuestException)
+ return p;
+}
-MIPSX(exceptions):
- ####
- ##### The exception handlers.
- #####
- .word _C_LABEL(MIPSX(GuestException)) # 0
- .word _C_LABEL(MIPSX(GuestException)) # 1
- .word _C_LABEL(MIPSX(GuestException)) # 2
- .word _C_LABEL(MIPSX(GuestException)) # 3
- .word _C_LABEL(MIPSX(GuestException)) # 4
- .word _C_LABEL(MIPSX(GuestException)) # 5
- .word _C_LABEL(MIPSX(GuestException)) # 6
- .word _C_LABEL(MIPSX(GuestException)) # 7
- .word _C_LABEL(MIPSX(GuestException)) # 8
- .word _C_LABEL(MIPSX(GuestException)) # 9
- .word _C_LABEL(MIPSX(GuestException)) # 10
- .word _C_LABEL(MIPSX(GuestException)) # 11
- .word _C_LABEL(MIPSX(GuestException)) # 12
- .word _C_LABEL(MIPSX(GuestException)) # 13
- .word _C_LABEL(MIPSX(GuestException)) # 14
- .word _C_LABEL(MIPSX(GuestException)) # 15
- .word _C_LABEL(MIPSX(GuestException)) # 16
- .word _C_LABEL(MIPSX(GuestException)) # 17
- .word _C_LABEL(MIPSX(GuestException)) # 18
- .word _C_LABEL(MIPSX(GuestException)) # 19
- .word _C_LABEL(MIPSX(GuestException)) # 20
- .word _C_LABEL(MIPSX(GuestException)) # 21
- .word _C_LABEL(MIPSX(GuestException)) # 22
- .word _C_LABEL(MIPSX(GuestException)) # 23
- .word _C_LABEL(MIPSX(GuestException)) # 24
- .word _C_LABEL(MIPSX(GuestException)) # 25
- .word _C_LABEL(MIPSX(GuestException)) # 26
- .word _C_LABEL(MIPSX(GuestException)) # 27
- .word _C_LABEL(MIPSX(GuestException)) # 28
- .word _C_LABEL(MIPSX(GuestException)) # 29
- .word _C_LABEL(MIPSX(GuestException)) # 30
- .word _C_LABEL(MIPSX(GuestException)) # 31
diff --git a/arch/mips/kvm/interrupt.h b/arch/mips/kvm/interrupt.h
index d661c100b219..fb118a2c8379 100644
--- a/arch/mips/kvm/interrupt.h
+++ b/arch/mips/kvm/interrupt.h
@@ -28,10 +28,6 @@
#define MIPS_EXC_MAX 12
/* XXXSL More to follow */
-extern char __kvm_mips_vcpu_run_end[];
-extern char mips32_exception[], mips32_exceptionEnd[];
-extern char mips32_GuestException[], mips32_GuestExceptionEnd[];
-
#define C_TI (_ULCAST_(1) << 30)
#define KVM_MIPS_IRQ_DELIVER_ALL_AT_ONCE (0)
diff --git a/arch/mips/kvm/mips.c b/arch/mips/kvm/mips.c
index 5a2b9034a05c..f00cf5f9ac52 100644
--- a/arch/mips/kvm/mips.c
+++ b/arch/mips/kvm/mips.c
@@ -247,8 +247,8 @@ void kvm_arch_commit_memory_region(struct kvm *kvm,
struct kvm_vcpu *kvm_arch_vcpu_create(struct kvm *kvm, unsigned int id)
{
- int err, size, offset;
- void *gebase;
+ int err, size;
+ void *gebase, *p;
int i;
struct kvm_vcpu *vcpu = kzalloc(sizeof(struct kvm_vcpu), GFP_KERNEL);
@@ -286,41 +286,28 @@ struct kvm_vcpu *kvm_arch_vcpu_create(struct kvm *kvm, unsigned int id)
/* Save new ebase */
vcpu->arch.guest_ebase = gebase;
- /* Copy L1 Guest Exception handler to correct offset */
+ /* Build guest exception vectors dynamically in unmapped memory */
/* TLB Refill, EXL = 0 */
- memcpy(gebase, mips32_exception,
- mips32_exceptionEnd - mips32_exception);
+ kvm_mips_build_exception(gebase);
/* General Exception Entry point */
- memcpy(gebase + 0x180, mips32_exception,
- mips32_exceptionEnd - mips32_exception);
+ kvm_mips_build_exception(gebase + 0x180);
/* For vectored interrupts poke the exception code @ all offsets 0-7 */
for (i = 0; i < 8; i++) {
kvm_debug("L1 Vectored handler @ %p\n",
gebase + 0x200 + (i * VECTORSPACING));
- memcpy(gebase + 0x200 + (i * VECTORSPACING), mips32_exception,
- mips32_exceptionEnd - mips32_exception);
+ kvm_mips_build_exception(gebase + 0x200 + i * VECTORSPACING);
}
- /* General handler, relocate to unmapped space for sanity's sake */
- offset = 0x2000;
- kvm_debug("Installing KVM Exception handlers @ %p, %#x bytes\n",
- gebase + offset,
- mips32_GuestExceptionEnd - mips32_GuestException);
+ /* General exit handler */
+ p = gebase + 0x2000;
+ p = kvm_mips_build_exit(p);
- memcpy(gebase + offset, mips32_GuestException,
- mips32_GuestExceptionEnd - mips32_GuestException);
-
-#ifdef MODULE
- offset += mips32_GuestExceptionEnd - mips32_GuestException;
- memcpy(gebase + offset, (char *)__kvm_mips_vcpu_run,
- __kvm_mips_vcpu_run_end - (char *)__kvm_mips_vcpu_run);
- vcpu->arch.vcpu_run = gebase + offset;
-#else
- vcpu->arch.vcpu_run = __kvm_mips_vcpu_run;
-#endif
+ /* Guest entry routine */
+ vcpu->arch.vcpu_run = p;
+ p = kvm_mips_build_vcpu_run(p);
/* Invalidate the icache for these ranges */
local_flush_icache_range((unsigned long)gebase,
--
2.4.10
^ permalink raw reply related [flat|nested] 36+ messages in thread* [PATCH 06/14] MIPS; KVM: Convert exception entry to uasm
@ 2016-06-23 16:34 ` James Hogan
0 siblings, 0 replies; 36+ messages in thread
From: James Hogan @ 2016-06-23 16:34 UTC (permalink / raw)
To: Paolo Bonzini, Radim Krčmář, Ralf Baechle
Cc: James Hogan, linux-mips, kvm
Convert the whole of locore.S (assembly to enter guest and handle
exception entry) to be generated dynamically with uasm. This is done
with minimal changes to the resulting code.
The main changes are:
- Some constants are generated by uasm using LUI+ADDIU instead of
LUI+ORI.
- Loading of lo and hi are swapped around in vcpu_run but not when
resuming the guest after an exit. Both bits of logic are now generated
by the same code.
- Register MOVEs in uasm use different ADDU operand ordering to GNU as,
putting zero register into rs instead of rt.
- The JALR.HB to call the C exit handler is switched to JALR, since the
hazard barrier would appear to be unnecessary.
This will allow further optimisation in the future to dynamically handle
the capabilities of the CPU.
Signed-off-by: James Hogan <james.hogan@imgtec.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Radim Krčmář <rkrcmar@redhat.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: linux-mips@linux-mips.org
Cc: kvm@vger.kernel.org
---
arch/mips/include/asm/kvm_host.h | 8 +-
arch/mips/kvm/Kconfig | 1 +
arch/mips/kvm/Makefile | 2 +-
arch/mips/kvm/{locore.S => entry.c} | 884 ++++++++++++++++++------------------
arch/mips/kvm/interrupt.h | 4 -
arch/mips/kvm/mips.c | 37 +-
6 files changed, 472 insertions(+), 464 deletions(-)
rename arch/mips/kvm/{locore.S => entry.c} (20%)
diff --git a/arch/mips/include/asm/kvm_host.h b/arch/mips/include/asm/kvm_host.h
index b0773c6d622f..2e76e899079c 100644
--- a/arch/mips/include/asm/kvm_host.h
+++ b/arch/mips/include/asm/kvm_host.h
@@ -533,8 +533,12 @@ int kvm_mips_emulation_init(struct kvm_mips_callbacks **install_callbacks);
/* Debug: dump vcpu state */
int kvm_arch_vcpu_dump_regs(struct kvm_vcpu *vcpu);
-/* Trampoline ASM routine to start running in "Guest" context */
-extern int __kvm_mips_vcpu_run(struct kvm_run *run, struct kvm_vcpu *vcpu);
+extern int kvm_mips_handle_exit(struct kvm_run *run, struct kvm_vcpu *vcpu);
+
+/* Building of entry/exception code */
+void *kvm_mips_build_vcpu_run(void *addr);
+void *kvm_mips_build_exception(void *addr);
+void *kvm_mips_build_exit(void *addr);
/* FPU/MSA context management */
void __kvm_save_fpu(struct kvm_vcpu_arch *vcpu);
diff --git a/arch/mips/kvm/Kconfig b/arch/mips/kvm/Kconfig
index 2ae12825529f..7c56d6b124d1 100644
--- a/arch/mips/kvm/Kconfig
+++ b/arch/mips/kvm/Kconfig
@@ -17,6 +17,7 @@ if VIRTUALIZATION
config KVM
tristate "Kernel-based Virtual Machine (KVM) support"
depends on HAVE_KVM
+ select EXPORT_UASM
select PREEMPT_NOTIFIERS
select ANON_INODES
select KVM_MMIO
diff --git a/arch/mips/kvm/Makefile b/arch/mips/kvm/Makefile
index 0aabe40fcac9..847429de780d 100644
--- a/arch/mips/kvm/Makefile
+++ b/arch/mips/kvm/Makefile
@@ -7,7 +7,7 @@ EXTRA_CFLAGS += -Ivirt/kvm -Iarch/mips/kvm
common-objs-$(CONFIG_CPU_HAS_MSA) += msa.o
-kvm-objs := $(common-objs-y) mips.o emulate.o locore.o \
+kvm-objs := $(common-objs-y) mips.o emulate.o entry.o \
interrupt.o stats.o commpage.o \
dyntrans.o trap_emul.o fpu.o
kvm-objs += mmu.o
diff --git a/arch/mips/kvm/locore.S b/arch/mips/kvm/entry.c
similarity index 20%
rename from arch/mips/kvm/locore.S
rename to arch/mips/kvm/entry.c
index 698286c0f732..9a18b4939b35 100644
--- a/arch/mips/kvm/locore.S
+++ b/arch/mips/kvm/entry.c
@@ -3,373 +3,445 @@
* License. See the file "COPYING" in the main directory of this archive
* for more details.
*
- * Main entry point for the guest, exception handling.
+ * Generation of main entry point for the guest, exception handling.
*
- * Copyright (C) 2012 MIPS Technologies, Inc. All rights reserved.
+ * Copyright (C) 2012 MIPS Technologies, Inc.
* Authors: Sanjay Lal <sanjayl@kymasys.com>
+ *
+ * Copyright (C) 2016 Imagination Technologies Ltd.
*/
-#include <asm/asm.h>
-#include <asm/asmmacro.h>
-#include <asm/regdef.h>
-#include <asm/mipsregs.h>
-#include <asm/stackframe.h>
-#include <asm/asm-offsets.h>
-
-#define _C_LABEL(x) x
-#define MIPSX(name) mips32_ ## name
+#include <linux/kvm_host.h>
+#include <asm/msa.h>
+#include <asm/setup.h>
+#include <asm/uasm.h>
+
+/* Register names */
+#define ZERO 0
+#define AT 1
+#define V0 2
+#define V1 3
+#define A0 4
+#define A1 5
+
+#if _MIPS_SIM == _MIPS_SIM_ABI32
+#define T0 8
+#define T1 9
+#define T2 10
+#define T3 11
+#endif /* _MIPS_SIM == _MIPS_SIM_ABI32 */
+
+#if _MIPS_SIM == _MIPS_SIM_ABI64 || _MIPS_SIM == _MIPS_SIM_NABI32
+#define T0 12
+#define T1 13
+#define T2 14
+#define T3 15
+#endif /* _MIPS_SIM == _MIPS_SIM_ABI64 || _MIPS_SIM == _MIPS_SIM_NABI32 */
+
+#define S0 16
+#define S1 17
+#define T9 25
+#define K0 26
+#define K1 27
+#define GP 28
+#define SP 29
+#define RA 31
+
+/* Some CP0 registers */
+#define C0_HWRENA 7, 0
+#define C0_BADVADDR 8, 0
+#define C0_ENTRYHI 10, 0
+#define C0_STATUS 12, 0
+#define C0_CAUSE 13, 0
+#define C0_EPC 14, 0
+#define C0_EBASE 15, 1
+#define C0_CONFIG3 16, 3
+#define C0_CONFIG5 16, 5
+#define C0_DDATA_LO 28, 3
+#define C0_ERROREPC 30, 0
+
#define CALLFRAME_SIZ 32
-/*
- * VECTOR
- * exception vector entrypoint
- */
-#define VECTOR(x, regmask) \
- .ent _C_LABEL(x),0; \
- EXPORT(x);
-
-#define VECTOR_END(x) \
- EXPORT(x);
-
-/* Overload, Danger Will Robinson!! */
-#define PT_HOST_USERLOCAL PT_EPC
+enum label_id {
+ label_fpu_1 = 1,
+ label_msa_1,
+ label_return_to_host,
+ label_kernel_asid,
+};
-#define CP0_DDATA_LO $28,3
+UASM_L_LA(_fpu_1)
+UASM_L_LA(_msa_1)
+UASM_L_LA(_return_to_host)
+UASM_L_LA(_kernel_asid)
-/* Resume Flags */
-#define RESUME_FLAG_HOST (1<<1) /* Resume host? */
+static void *kvm_mips_build_enter_guest(void *addr);
+static void *kvm_mips_build_ret_from_exit(void *addr);
+static void *kvm_mips_build_ret_to_guest(void *addr);
+static void *kvm_mips_build_ret_to_host(void *addr);
-#define RESUME_GUEST 0
-#define RESUME_HOST RESUME_FLAG_HOST
-
-/*
- * __kvm_mips_vcpu_run: entry point to the guest
- * a0: run
- * a1: vcpu
+/**
+ * kvm_mips_build_vcpu_run() - Assemble function to start running a guest VCPU.
+ * @addr: Address to start writing code.
+ *
+ * Assemble the start of the vcpu_run function to run a guest VCPU. The function
+ * conforms to the following prototype:
+ *
+ * int vcpu_run(struct kvm_run *run, struct kvm_vcpu *vcpu);
+ *
+ * The exit from the guest and return to the caller is handled by the code
+ * generated by kvm_mips_build_ret_to_host().
+ *
+ * Returns: Next address after end of written function.
*/
- .set noreorder
+void *kvm_mips_build_vcpu_run(void *addr)
+{
+ u32 *p = addr;
+ unsigned int i;
+
+ /*
+ * A0: run
+ * A1: vcpu
+ */
-FEXPORT(__kvm_mips_vcpu_run)
/* k0/k1 not being used in host kernel context */
- INT_ADDIU k1, sp, -PT_SIZE
- LONG_S $16, PT_R16(k1)
- LONG_S $17, PT_R17(k1)
- LONG_S $18, PT_R18(k1)
- LONG_S $19, PT_R19(k1)
- LONG_S $20, PT_R20(k1)
- LONG_S $21, PT_R21(k1)
- LONG_S $22, PT_R22(k1)
- LONG_S $23, PT_R23(k1)
-
- LONG_S $28, PT_R28(k1)
- LONG_S $29, PT_R29(k1)
- LONG_S $30, PT_R30(k1)
- LONG_S $31, PT_R31(k1)
+ uasm_i_addiu(&p, K1, SP, -(int)sizeof(struct pt_regs));
+ for (i = 16; i < 32; ++i) {
+ if (i == 24)
+ i = 28;
+ UASM_i_SW(&p, i, offsetof(struct pt_regs, regs[i]), K1);
+ }
/* Save hi/lo */
- mflo v0
- LONG_S v0, PT_LO(k1)
- mfhi v1
- LONG_S v1, PT_HI(k1)
+ uasm_i_mflo(&p, V0);
+ UASM_i_SW(&p, V0, offsetof(struct pt_regs, lo), K1);
+ uasm_i_mfhi(&p, V1);
+ UASM_i_SW(&p, V1, offsetof(struct pt_regs, hi), K1);
/* Save host status */
- mfc0 v0, CP0_STATUS
- LONG_S v0, PT_STATUS(k1)
+ uasm_i_mfc0(&p, V0, C0_STATUS);
+ UASM_i_SW(&p, V0, offsetof(struct pt_regs, cp0_status), K1);
/* Save DDATA_LO, will be used to store pointer to vcpu */
- mfc0 v1, CP0_DDATA_LO
- LONG_S v1, PT_HOST_USERLOCAL(k1)
+ uasm_i_mfc0(&p, V1, C0_DDATA_LO);
+ UASM_i_SW(&p, V1, offsetof(struct pt_regs, cp0_epc), K1);
/* DDATA_LO has pointer to vcpu */
- mtc0 a1, CP0_DDATA_LO
+ uasm_i_mtc0(&p, A1, C0_DDATA_LO);
/* Offset into vcpu->arch */
- INT_ADDIU k1, a1, VCPU_HOST_ARCH
+ uasm_i_addiu(&p, K1, A1, offsetof(struct kvm_vcpu, arch));
/*
* Save the host stack to VCPU, used for exception processing
* when we exit from the Guest
*/
- LONG_S sp, VCPU_HOST_STACK(k1)
+ UASM_i_SW(&p, SP, offsetof(struct kvm_vcpu_arch, host_stack), K1);
/* Save the kernel gp as well */
- LONG_S gp, VCPU_HOST_GP(k1)
+ UASM_i_SW(&p, GP, offsetof(struct kvm_vcpu_arch, host_gp), K1);
/*
* Setup status register for running the guest in UM, interrupts
* are disabled
*/
- li k0, (ST0_EXL | KSU_USER | ST0_BEV)
- mtc0 k0, CP0_STATUS
- ehb
+ UASM_i_LA(&p, K0, ST0_EXL | KSU_USER | ST0_BEV);
+ uasm_i_mtc0(&p, K0, C0_STATUS);
+ uasm_i_ehb(&p);
/* load up the new EBASE */
- LONG_L k0, VCPU_GUEST_EBASE(k1)
- mtc0 k0, CP0_EBASE
+ UASM_i_LW(&p, K0, offsetof(struct kvm_vcpu_arch, guest_ebase), K1);
+ uasm_i_mtc0(&p, K0, C0_EBASE);
/*
* Now that the new EBASE has been loaded, unset BEV, set
* interrupt mask as it was but make sure that timer interrupts
* are enabled
*/
- li k0, (ST0_EXL | KSU_USER | ST0_IE)
- andi v0, v0, ST0_IM
- or k0, k0, v0
- mtc0 k0, CP0_STATUS
- ehb
+ uasm_i_addiu(&p, K0, ZERO, ST0_EXL | KSU_USER | ST0_IE);
+ uasm_i_andi(&p, V0, V0, ST0_IM);
+ uasm_i_or(&p, K0, K0, V0);
+ uasm_i_mtc0(&p, K0, C0_STATUS);
+ uasm_i_ehb(&p);
+
+ p = kvm_mips_build_enter_guest(p);
+
+ return p;
+}
+
+/**
+ * kvm_mips_build_enter_guest() - Assemble code to resume guest execution.
+ * @addr: Address to start writing code.
+ *
+ * Assemble the code to resume guest execution. This code is common between the
+ * initial entry into the guest from the host, and returning from the exit
+ * handler back to the guest.
+ *
+ * Returns: Next address after end of written function.
+ */
+static void *kvm_mips_build_enter_guest(void *addr)
+{
+ u32 *p = addr;
+ unsigned int i;
+ struct uasm_label labels[2];
+ struct uasm_reloc relocs[2];
+ struct uasm_label *l = labels;
+ struct uasm_reloc *r = relocs;
+
+ memset(labels, 0, sizeof(labels));
+ memset(relocs, 0, sizeof(relocs));
/* Set Guest EPC */
- LONG_L t0, VCPU_PC(k1)
- mtc0 t0, CP0_EPC
+ UASM_i_LW(&p, T0, offsetof(struct kvm_vcpu_arch, pc), K1);
+ uasm_i_mtc0(&p, T0, C0_EPC);
-FEXPORT(__kvm_mips_load_asid)
/* Set the ASID for the Guest Kernel */
- PTR_L t0, VCPU_COP0(k1)
- LONG_L t0, COP0_STATUS(t0)
- andi t0, KSU_USER | ST0_ERL | ST0_EXL
- xori t0, KSU_USER
- bnez t0, 1f /* If kernel */
- INT_ADDIU t1, k1, VCPU_GUEST_KERNEL_ASID /* (BD) */
- INT_ADDIU t1, k1, VCPU_GUEST_USER_ASID /* else user */
-1:
- /* t1: contains the base of the ASID array, need to get the cpu id */
- LONG_L t2, TI_CPU($28) /* smp_processor_id */
- INT_SLL t2, t2, 2 /* x4 */
- REG_ADDU t3, t1, t2
- LONG_L k0, (t3)
+ UASM_i_LW(&p, T0, offsetof(struct kvm_vcpu_arch, cop0), K1);
+ UASM_i_LW(&p, T0, offsetof(struct mips_coproc, reg[MIPS_CP0_STATUS][0]),
+ T0);
+ uasm_i_andi(&p, T0, T0, KSU_USER | ST0_ERL | ST0_EXL);
+ uasm_i_xori(&p, T0, T0, KSU_USER);
+ uasm_il_bnez(&p, &r, T0, label_kernel_asid);
+ uasm_i_addiu(&p, T1, K1,
+ offsetof(struct kvm_vcpu_arch, guest_kernel_asid));
+ /* else user */
+ uasm_i_addiu(&p, T1, K1,
+ offsetof(struct kvm_vcpu_arch, guest_user_asid));
+ uasm_l_kernel_asid(&l, p);
+
+ /* t1: contains the base of the ASID array, need to get the cpu id */
+ /* smp_processor_id */
+ UASM_i_LW(&p, T2, offsetof(struct thread_info, cpu), GP);
+ /* x4 */
+ uasm_i_sll(&p, T2, T2, 2);
+ UASM_i_ADDU(&p, T3, T1, T2);
+ UASM_i_LW(&p, K0, 0, T3);
#ifdef CONFIG_MIPS_ASID_BITS_VARIABLE
- li t3, CPUINFO_SIZE/4
- mul t2, t2, t3 /* x sizeof(struct cpuinfo_mips)/4 */
- LONG_L t2, (cpu_data + CPUINFO_ASID_MASK)(t2)
- and k0, k0, t2
+ /* x sizeof(struct cpuinfo_mips)/4 */
+ uasm_i_addiu(&p, T3, ZERO, sizeof(struct cpuinfo_mips)/4);
+ uasm_i_mul(&p, T2, T2, T3);
+
+ UASM_i_LA_mostly(&p, AT, (long)&cpu_data[0].asid_mask);
+ UASM_i_ADDU(&p, AT, AT, T2);
+ UASM_i_LW(&p, T2, uasm_rel_lo((long)&cpu_data[0].asid_mask), AT);
+ uasm_i_and(&p, K0, K0, T2);
#else
- andi k0, k0, MIPS_ENTRYHI_ASID
+ uasm_i_andi(&p, K0, K0, MIPS_ENTRYHI_ASID);
#endif
- mtc0 k0, CP0_ENTRYHI
- ehb
+ uasm_i_mtc0(&p, K0, C0_ENTRYHI);
+ uasm_i_ehb(&p);
/* Disable RDHWR access */
- mtc0 zero, CP0_HWRENA
-
- .set noat
- /* Now load up the Guest Context from VCPU */
- LONG_L $1, VCPU_R1(k1)
- LONG_L $2, VCPU_R2(k1)
- LONG_L $3, VCPU_R3(k1)
-
- LONG_L $4, VCPU_R4(k1)
- LONG_L $5, VCPU_R5(k1)
- LONG_L $6, VCPU_R6(k1)
- LONG_L $7, VCPU_R7(k1)
-
- LONG_L $8, VCPU_R8(k1)
- LONG_L $9, VCPU_R9(k1)
- LONG_L $10, VCPU_R10(k1)
- LONG_L $11, VCPU_R11(k1)
- LONG_L $12, VCPU_R12(k1)
- LONG_L $13, VCPU_R13(k1)
- LONG_L $14, VCPU_R14(k1)
- LONG_L $15, VCPU_R15(k1)
- LONG_L $16, VCPU_R16(k1)
- LONG_L $17, VCPU_R17(k1)
- LONG_L $18, VCPU_R18(k1)
- LONG_L $19, VCPU_R19(k1)
- LONG_L $20, VCPU_R20(k1)
- LONG_L $21, VCPU_R21(k1)
- LONG_L $22, VCPU_R22(k1)
- LONG_L $23, VCPU_R23(k1)
- LONG_L $24, VCPU_R24(k1)
- LONG_L $25, VCPU_R25(k1)
-
- /* k0/k1 loaded up later */
-
- LONG_L $28, VCPU_R28(k1)
- LONG_L $29, VCPU_R29(k1)
- LONG_L $30, VCPU_R30(k1)
- LONG_L $31, VCPU_R31(k1)
+ uasm_i_mtc0(&p, ZERO, C0_HWRENA);
+
+ /* load the guest context from VCPU and return */
+ for (i = 1; i < 32; ++i) {
+ /* Guest k0/k1 loaded later */
+ if (i == K0 || i == K1)
+ continue;
+ UASM_i_LW(&p, i, offsetof(struct kvm_vcpu_arch, gprs[i]), K1);
+ }
/* Restore hi/lo */
- LONG_L k0, VCPU_LO(k1)
- mtlo k0
+ UASM_i_LW(&p, K0, offsetof(struct kvm_vcpu_arch, hi), K1);
+ uasm_i_mthi(&p, K0);
- LONG_L k0, VCPU_HI(k1)
- mthi k0
+ UASM_i_LW(&p, K0, offsetof(struct kvm_vcpu_arch, lo), K1);
+ uasm_i_mtlo(&p, K0);
-FEXPORT(__kvm_mips_load_k0k1)
/* Restore the guest's k0/k1 registers */
- LONG_L k0, VCPU_R26(k1)
- LONG_L k1, VCPU_R27(k1)
+ UASM_i_LW(&p, K0, offsetof(struct kvm_vcpu_arch, gprs[K0]), K1);
+ UASM_i_LW(&p, K1, offsetof(struct kvm_vcpu_arch, gprs[K1]), K1);
/* Jump to guest */
- eret
-EXPORT(__kvm_mips_vcpu_run_end)
-
-VECTOR(MIPSX(exception), unknown)
-/* Find out what mode we came from and jump to the proper handler. */
- mtc0 k0, CP0_ERROREPC #01: Save guest k0
- ehb #02:
-
- mfc0 k0, CP0_EBASE #02: Get EBASE
- INT_SRL k0, k0, 10 #03: Get rid of CPUNum
- INT_SLL k0, k0, 10 #04
- LONG_S k1, 0x3000(k0) #05: Save k1 @ offset 0x3000
- INT_ADDIU k0, k0, 0x2000 #06: Exception handler is
- # installed @ offset 0x2000
- j k0 #07: jump to the function
- nop #08: branch delay slot
-VECTOR_END(MIPSX(exceptionEnd))
-.end MIPSX(exception)
-
-/*
- * Generic Guest exception handler. We end up here when the guest
- * does something that causes a trap to kernel mode.
+ uasm_i_eret(&p);
+
+ uasm_resolve_relocs(relocs, labels);
+
+ return p;
+}
+
+/**
+ * kvm_mips_build_exception() - Assemble first level guest exception handler.
+ * @addr: Address to start writing code.
+ *
+ * Assemble exception vector code for guest execution. The generated vector will
+ * jump to the common exception handler generated by kvm_mips_build_exit().
+ *
+ * Returns: Next address after end of written function.
+ */
+void *kvm_mips_build_exception(void *addr)
+{
+ u32 *p = addr;
+
+ /* Save guest k0 */
+ uasm_i_mtc0(&p, K0, C0_ERROREPC);
+ uasm_i_ehb(&p);
+
+ /* Get EBASE */
+ uasm_i_mfc0(&p, K0, C0_EBASE);
+ /* Get rid of CPUNum */
+ uasm_i_srl(&p, K0, K0, 10);
+ uasm_i_sll(&p, K0, K0, 10);
+ /* Save k1 @ offset 0x3000 */
+ UASM_i_SW(&p, K1, 0x3000, K0);
+
+ /* Exception handler is installed @ offset 0x2000 */
+ uasm_i_addiu(&p, K0, K0, 0x2000);
+ /* Jump to the function */
+ uasm_i_jr(&p, K0);
+ uasm_i_nop(&p);
+
+ return p;
+}
+
+/**
+ * kvm_mips_build_exit() - Assemble common guest exit handler.
+ * @addr: Address to start writing code.
+ *
+ * Assemble the generic guest exit handling code. This is called by the
+ * exception vectors (generated by kvm_mips_build_exception()), and calls
+ * kvm_mips_handle_exit(), then either resumes the guest or returns to the host
+ * depending on the return value.
+ *
+ * Returns: Next address after end of written function.
*/
-NESTED (MIPSX(GuestException), CALLFRAME_SIZ, ra)
- /* Get the VCPU pointer from DDTATA_LO */
- mfc0 k1, CP0_DDATA_LO
- INT_ADDIU k1, k1, VCPU_HOST_ARCH
+void *kvm_mips_build_exit(void *addr)
+{
+ u32 *p = addr;
+ unsigned int i;
+ struct uasm_label labels[3];
+ struct uasm_reloc relocs[3];
+ struct uasm_label *l = labels;
+ struct uasm_reloc *r = relocs;
+
+ memset(labels, 0, sizeof(labels));
+ memset(relocs, 0, sizeof(relocs));
+
+ /*
+ * Generic Guest exception handler. We end up here when the guest
+ * does something that causes a trap to kernel mode.
+ */
+
+ /* Get the VCPU pointer from DDATA_LO */
+ uasm_i_mfc0(&p, K1, C0_DDATA_LO);
+ uasm_i_addiu(&p, K1, K1, offsetof(struct kvm_vcpu, arch));
/* Start saving Guest context to VCPU */
- LONG_S $0, VCPU_R0(k1)
- LONG_S $1, VCPU_R1(k1)
- LONG_S $2, VCPU_R2(k1)
- LONG_S $3, VCPU_R3(k1)
- LONG_S $4, VCPU_R4(k1)
- LONG_S $5, VCPU_R5(k1)
- LONG_S $6, VCPU_R6(k1)
- LONG_S $7, VCPU_R7(k1)
- LONG_S $8, VCPU_R8(k1)
- LONG_S $9, VCPU_R9(k1)
- LONG_S $10, VCPU_R10(k1)
- LONG_S $11, VCPU_R11(k1)
- LONG_S $12, VCPU_R12(k1)
- LONG_S $13, VCPU_R13(k1)
- LONG_S $14, VCPU_R14(k1)
- LONG_S $15, VCPU_R15(k1)
- LONG_S $16, VCPU_R16(k1)
- LONG_S $17, VCPU_R17(k1)
- LONG_S $18, VCPU_R18(k1)
- LONG_S $19, VCPU_R19(k1)
- LONG_S $20, VCPU_R20(k1)
- LONG_S $21, VCPU_R21(k1)
- LONG_S $22, VCPU_R22(k1)
- LONG_S $23, VCPU_R23(k1)
- LONG_S $24, VCPU_R24(k1)
- LONG_S $25, VCPU_R25(k1)
-
- /* Guest k0/k1 saved later */
-
- LONG_S $28, VCPU_R28(k1)
- LONG_S $29, VCPU_R29(k1)
- LONG_S $30, VCPU_R30(k1)
- LONG_S $31, VCPU_R31(k1)
-
- .set at
+ for (i = 0; i < 32; ++i) {
+ /* Guest k0/k1 saved later */
+ if (i == K0 || i == K1)
+ continue;
+ UASM_i_SW(&p, i, offsetof(struct kvm_vcpu_arch, gprs[i]), K1);
+ }
/* We need to save hi/lo and restore them on the way out */
- mfhi t0
- LONG_S t0, VCPU_HI(k1)
+ uasm_i_mfhi(&p, T0);
+ UASM_i_SW(&p, T0, offsetof(struct kvm_vcpu_arch, hi), K1);
- mflo t0
- LONG_S t0, VCPU_LO(k1)
+ uasm_i_mflo(&p, T0);
+ UASM_i_SW(&p, T0, offsetof(struct kvm_vcpu_arch, lo), K1);
/* Finally save guest k0/k1 to VCPU */
- mfc0 t0, CP0_ERROREPC
- LONG_S t0, VCPU_R26(k1)
+ uasm_i_mfc0(&p, T0, C0_ERROREPC);
+ UASM_i_SW(&p, T0, offsetof(struct kvm_vcpu_arch, gprs[K0]), K1);
/* Get GUEST k1 and save it in VCPU */
- PTR_LI t1, ~0x2ff
- mfc0 t0, CP0_EBASE
- and t0, t0, t1
- LONG_L t0, 0x3000(t0)
- LONG_S t0, VCPU_R27(k1)
+ uasm_i_addiu(&p, T1, ZERO, ~0x2ff);
+ uasm_i_mfc0(&p, T0, C0_EBASE);
+ uasm_i_and(&p, T0, T0, T1);
+ UASM_i_LW(&p, T0, 0x3000, T0);
+ UASM_i_SW(&p, T0, offsetof(struct kvm_vcpu_arch, gprs[K1]), K1);
/* Now that context has been saved, we can use other registers */
/* Restore vcpu */
- mfc0 a1, CP0_DDATA_LO
- move s1, a1
+ uasm_i_mfc0(&p, A1, C0_DDATA_LO);
+ uasm_i_move(&p, S1, A1);
/* Restore run (vcpu->run) */
- LONG_L a0, VCPU_RUN(a1)
+ UASM_i_LW(&p, A0, offsetof(struct kvm_vcpu, run), A1);
/* Save pointer to run in s0, will be saved by the compiler */
- move s0, a0
+ uasm_i_move(&p, S0, A0);
/*
- * Save Host level EPC, BadVaddr and Cause to VCPU, useful to
- * process the exception
+ * Save Host level EPC, BadVaddr and Cause to VCPU, useful to process
+ * the exception
*/
- mfc0 k0,CP0_EPC
- LONG_S k0, VCPU_PC(k1)
+ uasm_i_mfc0(&p, K0, C0_EPC);
+ UASM_i_SW(&p, K0, offsetof(struct kvm_vcpu_arch, pc), K1);
- mfc0 k0, CP0_BADVADDR
- LONG_S k0, VCPU_HOST_CP0_BADVADDR(k1)
+ uasm_i_mfc0(&p, K0, C0_BADVADDR);
+ UASM_i_SW(&p, K0, offsetof(struct kvm_vcpu_arch, host_cp0_badvaddr),
+ K1);
- mfc0 k0, CP0_CAUSE
- sw k0, VCPU_HOST_CP0_CAUSE(k1)
+ uasm_i_mfc0(&p, K0, C0_CAUSE);
+ uasm_i_sw(&p, K0, offsetof(struct kvm_vcpu_arch, host_cp0_cause), K1);
/* Now restore the host state just enough to run the handlers */
/* Switch EBASE to the one used by Linux */
/* load up the host EBASE */
- mfc0 v0, CP0_STATUS
+ uasm_i_mfc0(&p, V0, C0_STATUS);
- or k0, v0, ST0_BEV
+ uasm_i_lui(&p, AT, ST0_BEV >> 16);
+ uasm_i_or(&p, K0, V0, AT);
- mtc0 k0, CP0_STATUS
- ehb
+ uasm_i_mtc0(&p, K0, C0_STATUS);
+ uasm_i_ehb(&p);
- LONG_L k0, ebase
- mtc0 k0,CP0_EBASE
+ UASM_i_LA_mostly(&p, K0, (long)&ebase);
+ UASM_i_LW(&p, K0, uasm_rel_lo((long)&ebase), K0);
+ uasm_i_mtc0(&p, K0, C0_EBASE);
/*
* If FPU is enabled, save FCR31 and clear it so that later ctc1's don't
* trigger FPE for pending exceptions.
*/
- and v1, v0, ST0_CU1
- beqz v1, 1f
- nop
- .set push
- SET_HARDFLOAT
- cfc1 t0, fcr31
- sw t0, VCPU_FCR31(k1)
- ctc1 zero,fcr31
- .set pop
-1:
+ uasm_i_lui(&p, AT, ST0_CU1 >> 16);
+ uasm_i_and(&p, V1, V0, AT);
+ uasm_il_beqz(&p, &r, V1, label_fpu_1);
+ uasm_i_nop(&p);
+ uasm_i_cfc1(&p, T0, 31);
+ uasm_i_sw(&p, T0, offsetof(struct kvm_vcpu_arch, fpu.fcr31), K1);
+ uasm_i_ctc1(&p, ZERO, 31);
+ uasm_l_fpu_1(&l, p);
#ifdef CONFIG_CPU_HAS_MSA
/*
* If MSA is enabled, save MSACSR and clear it so that later
* instructions don't trigger MSAFPE for pending exceptions.
*/
- mfc0 t0, CP0_CONFIG3
- ext t0, t0, 28, 1 /* MIPS_CONF3_MSAP */
- beqz t0, 1f
- nop
- mfc0 t0, CP0_CONFIG5
- ext t0, t0, 27, 1 /* MIPS_CONF5_MSAEN */
- beqz t0, 1f
- nop
- _cfcmsa t0, MSA_CSR
- sw t0, VCPU_MSA_CSR(k1)
- _ctcmsa MSA_CSR, zero
-1:
+ uasm_i_mfc0(&p, T0, C0_CONFIG3);
+ uasm_i_ext(&p, T0, T0, 28, 1); /* MIPS_CONF3_MSAP */
+ uasm_il_beqz(&p, &r, T0, label_msa_1);
+ uasm_i_nop(&p);
+ uasm_i_mfc0(&p, T0, C0_CONFIG5);
+ uasm_i_ext(&p, T0, T0, 27, 1); /* MIPS_CONF5_MSAEN */
+ uasm_il_beqz(&p, &r, T0, label_msa_1);
+ uasm_i_nop(&p);
+ uasm_i_cfcmsa(&p, T0, MSA_CSR);
+ uasm_i_sw(&p, T0, offsetof(struct kvm_vcpu_arch, fpu.msacsr),
+ K1);
+ uasm_i_ctcmsa(&p, MSA_CSR, ZERO);
+ uasm_l_msa_1(&l, p);
#endif
/* Now that the new EBASE has been loaded, unset BEV and KSU_USER */
- and v0, v0, ~(ST0_EXL | KSU_USER | ST0_IE)
- or v0, v0, ST0_CU0
- mtc0 v0, CP0_STATUS
- ehb
+ uasm_i_addiu(&p, AT, ZERO, ~(ST0_EXL | KSU_USER | ST0_IE));
+ uasm_i_and(&p, V0, V0, AT);
+ uasm_i_lui(&p, AT, ST0_CU0 >> 16);
+ uasm_i_or(&p, V0, V0, AT);
+ uasm_i_mtc0(&p, V0, C0_STATUS);
+ uasm_i_ehb(&p);
/* Load up host GP */
- LONG_L gp, VCPU_HOST_GP(k1)
+ UASM_i_LW(&p, GP, offsetof(struct kvm_vcpu_arch, host_gp), K1);
/* Need a stack before we can jump to "C" */
- LONG_L sp, VCPU_HOST_STACK(k1)
+ UASM_i_LW(&p, SP, offsetof(struct kvm_vcpu_arch, host_stack), K1);
/* Saved host state */
- INT_ADDIU sp, sp, -PT_SIZE
+ uasm_i_addiu(&p, SP, SP, -(int)sizeof(struct pt_regs));
/*
* XXXKYMA do we need to load the host ASID, maybe not because the
@@ -377,27 +449,54 @@ NESTED (MIPSX(GuestException), CALLFRAME_SIZ, ra)
*/
/* Restore host DDATA_LO */
- LONG_L k0, PT_HOST_USERLOCAL(sp)
- mtc0 k0, CP0_DDATA_LO
+ UASM_i_LW(&p, K0, offsetof(struct pt_regs, cp0_epc), SP);
+ uasm_i_mtc0(&p, K0, C0_DDATA_LO);
/* Restore RDHWR access */
- INT_L k0, hwrena
- mtc0 k0, CP0_HWRENA
+ UASM_i_LA_mostly(&p, K0, (long)&hwrena);
+ uasm_i_lw(&p, K0, uasm_rel_lo((long)&hwrena), K0);
+ uasm_i_mtc0(&p, K0, C0_HWRENA);
/* Jump to handler */
-FEXPORT(__kvm_mips_jump_to_handler)
/*
* XXXKYMA: not sure if this is safe, how large is the stack??
* Now jump to the kvm_mips_handle_exit() to see if we can deal
* with this in the kernel
*/
- PTR_LA t9, kvm_mips_handle_exit
- jalr.hb t9
- INT_ADDIU sp, sp, -CALLFRAME_SIZ /* BD Slot */
+ UASM_i_LA(&p, T9, (unsigned long)kvm_mips_handle_exit);
+ uasm_i_jalr(&p, RA, T9);
+ uasm_i_addiu(&p, SP, SP, -CALLFRAME_SIZ);
+
+ uasm_resolve_relocs(relocs, labels);
+
+ p = kvm_mips_build_ret_from_exit(p);
+
+ return p;
+}
+
+/**
+ * kvm_mips_build_ret_from_exit() - Assemble guest exit return handler.
+ * @addr: Address to start writing code.
+ *
+ * Assemble the code to handle the return from kvm_mips_handle_exit(), either
+ * resuming the guest or returning to the host depending on the return value.
+ *
+ * Returns: Next address after end of written function.
+ */
+static void *kvm_mips_build_ret_from_exit(void *addr)
+{
+ u32 *p = addr;
+ struct uasm_label labels[2];
+ struct uasm_reloc relocs[2];
+ struct uasm_label *l = labels;
+ struct uasm_reloc *r = relocs;
+
+ memset(labels, 0, sizeof(labels));
+ memset(relocs, 0, sizeof(relocs));
/* Return from handler Make sure interrupts are disabled */
- di
- ehb
+ uasm_i_di(&p, ZERO);
+ uasm_i_ehb(&p);
/*
* XXXKYMA: k0/k1 could have been blown away if we processed
@@ -405,198 +504,119 @@ FEXPORT(__kvm_mips_jump_to_handler)
* guest, reload k1
*/
- move k1, s1
- INT_ADDIU k1, k1, VCPU_HOST_ARCH
+ uasm_i_move(&p, K1, S1);
+ uasm_i_addiu(&p, K1, K1, offsetof(struct kvm_vcpu, arch));
/*
* Check return value, should tell us if we are returning to the
* host (handle I/O etc)or resuming the guest
*/
- andi t0, v0, RESUME_HOST
- bnez t0, __kvm_mips_return_to_host
- nop
+ uasm_i_andi(&p, T0, V0, RESUME_HOST);
+ uasm_il_bnez(&p, &r, T0, label_return_to_host);
+ uasm_i_nop(&p);
+
+ p = kvm_mips_build_ret_to_guest(p);
+
+ uasm_l_return_to_host(&l, p);
+ p = kvm_mips_build_ret_to_host(p);
+
+ uasm_resolve_relocs(relocs, labels);
+
+ return p;
+}
+
+/**
+ * kvm_mips_build_ret_to_guest() - Assemble code to return to the guest.
+ * @addr: Address to start writing code.
+ *
+ * Assemble the code to handle return from the guest exit handler
+ * (kvm_mips_handle_exit()) back to the guest.
+ *
+ * Returns: Next address after end of written function.
+ */
+static void *kvm_mips_build_ret_to_guest(void *addr)
+{
+ u32 *p = addr;
-__kvm_mips_return_to_guest:
/* Put the saved pointer to vcpu (s1) back into the DDATA_LO Register */
- mtc0 s1, CP0_DDATA_LO
+ uasm_i_mtc0(&p, S1, C0_DDATA_LO);
/* Load up the Guest EBASE to minimize the window where BEV is set */
- LONG_L t0, VCPU_GUEST_EBASE(k1)
+ UASM_i_LW(&p, T0, offsetof(struct kvm_vcpu_arch, guest_ebase), K1);
/* Switch EBASE back to the one used by KVM */
- mfc0 v1, CP0_STATUS
- or k0, v1, ST0_BEV
- mtc0 k0, CP0_STATUS
- ehb
- mtc0 t0, CP0_EBASE
+ uasm_i_mfc0(&p, V1, C0_STATUS);
+ uasm_i_lui(&p, AT, ST0_BEV >> 16);
+ uasm_i_or(&p, K0, V1, AT);
+ uasm_i_mtc0(&p, K0, C0_STATUS);
+ uasm_i_ehb(&p);
+ uasm_i_mtc0(&p, T0, C0_EBASE);
/* Setup status register for running guest in UM */
- or v1, v1, (ST0_EXL | KSU_USER | ST0_IE)
- and v1, v1, ~(ST0_CU0 | ST0_MX)
- mtc0 v1, CP0_STATUS
- ehb
+ uasm_i_ori(&p, V1, V1, ST0_EXL | KSU_USER | ST0_IE);
+ UASM_i_LA(&p, AT, ~(ST0_CU0 | ST0_MX));
+ uasm_i_and(&p, V1, V1, AT);
+ uasm_i_mtc0(&p, V1, C0_STATUS);
+ uasm_i_ehb(&p);
+
+ p = kvm_mips_build_enter_guest(p);
+
+ return p;
+}
+
+/**
+ * kvm_mips_build_ret_to_host() - Assemble code to return to the host.
+ * @addr: Address to start writing code.
+ *
+ * Assemble the code to handle return from the guest exit handler
+ * (kvm_mips_handle_exit()) back to the host, i.e. to the caller of the vcpu_run
+ * function generated by kvm_mips_build_vcpu_run().
+ *
+ * Returns: Next address after end of written function.
+ */
+static void *kvm_mips_build_ret_to_host(void *addr)
+{
+ u32 *p = addr;
+ unsigned int i;
- /* Set Guest EPC */
- LONG_L t0, VCPU_PC(k1)
- mtc0 t0, CP0_EPC
-
- /* Set the ASID for the Guest Kernel */
- PTR_L t0, VCPU_COP0(k1)
- LONG_L t0, COP0_STATUS(t0)
- andi t0, KSU_USER | ST0_ERL | ST0_EXL
- xori t0, KSU_USER
- bnez t0, 1f /* If kernel */
- INT_ADDIU t1, k1, VCPU_GUEST_KERNEL_ASID /* (BD) */
- INT_ADDIU t1, k1, VCPU_GUEST_USER_ASID /* else user */
-1:
- /* t1: contains the base of the ASID array, need to get the cpu id */
- LONG_L t2, TI_CPU($28) /* smp_processor_id */
- INT_SLL t2, t2, 2 /* x4 */
- REG_ADDU t3, t1, t2
- LONG_L k0, (t3)
-#ifdef CONFIG_MIPS_ASID_BITS_VARIABLE
- li t3, CPUINFO_SIZE/4
- mul t2, t2, t3 /* x sizeof(struct cpuinfo_mips)/4 */
- LONG_L t2, (cpu_data + CPUINFO_ASID_MASK)(t2)
- and k0, k0, t2
-#else
- andi k0, k0, MIPS_ENTRYHI_ASID
-#endif
- mtc0 k0, CP0_ENTRYHI
- ehb
-
- /* Disable RDHWR access */
- mtc0 zero, CP0_HWRENA
-
- .set noat
- /* load the guest context from VCPU and return */
- LONG_L $0, VCPU_R0(k1)
- LONG_L $1, VCPU_R1(k1)
- LONG_L $2, VCPU_R2(k1)
- LONG_L $3, VCPU_R3(k1)
- LONG_L $4, VCPU_R4(k1)
- LONG_L $5, VCPU_R5(k1)
- LONG_L $6, VCPU_R6(k1)
- LONG_L $7, VCPU_R7(k1)
- LONG_L $8, VCPU_R8(k1)
- LONG_L $9, VCPU_R9(k1)
- LONG_L $10, VCPU_R10(k1)
- LONG_L $11, VCPU_R11(k1)
- LONG_L $12, VCPU_R12(k1)
- LONG_L $13, VCPU_R13(k1)
- LONG_L $14, VCPU_R14(k1)
- LONG_L $15, VCPU_R15(k1)
- LONG_L $16, VCPU_R16(k1)
- LONG_L $17, VCPU_R17(k1)
- LONG_L $18, VCPU_R18(k1)
- LONG_L $19, VCPU_R19(k1)
- LONG_L $20, VCPU_R20(k1)
- LONG_L $21, VCPU_R21(k1)
- LONG_L $22, VCPU_R22(k1)
- LONG_L $23, VCPU_R23(k1)
- LONG_L $24, VCPU_R24(k1)
- LONG_L $25, VCPU_R25(k1)
-
- /* $/k1 loaded later */
- LONG_L $28, VCPU_R28(k1)
- LONG_L $29, VCPU_R29(k1)
- LONG_L $30, VCPU_R30(k1)
- LONG_L $31, VCPU_R31(k1)
-
-FEXPORT(__kvm_mips_skip_guest_restore)
- LONG_L k0, VCPU_HI(k1)
- mthi k0
-
- LONG_L k0, VCPU_LO(k1)
- mtlo k0
-
- LONG_L k0, VCPU_R26(k1)
- LONG_L k1, VCPU_R27(k1)
-
- eret
- .set at
-
-__kvm_mips_return_to_host:
/* EBASE is already pointing to Linux */
- LONG_L k1, VCPU_HOST_STACK(k1)
- INT_ADDIU k1,k1, -PT_SIZE
+ UASM_i_LW(&p, K1, offsetof(struct kvm_vcpu_arch, host_stack), K1);
+ uasm_i_addiu(&p, K1, K1, -(int)sizeof(struct pt_regs));
/* Restore host DDATA_LO */
- LONG_L k0, PT_HOST_USERLOCAL(k1)
- mtc0 k0, CP0_DDATA_LO
+ UASM_i_LW(&p, K0, offsetof(struct pt_regs, cp0_epc), K1);
+ uasm_i_mtc0(&p, K0, C0_DDATA_LO);
/*
* r2/v0 is the return code, shift it down by 2 (arithmetic)
* to recover the err code
*/
- INT_SRA k0, v0, 2
- move $2, k0
+ uasm_i_sra(&p, K0, V0, 2);
+ uasm_i_move(&p, V0, K0);
/* Load context saved on the host stack */
- LONG_L $16, PT_R16(k1)
- LONG_L $17, PT_R17(k1)
- LONG_L $18, PT_R18(k1)
- LONG_L $19, PT_R19(k1)
- LONG_L $20, PT_R20(k1)
- LONG_L $21, PT_R21(k1)
- LONG_L $22, PT_R22(k1)
- LONG_L $23, PT_R23(k1)
+ for (i = 16; i < 31; ++i) {
+ if (i == 24)
+ i = 28;
+ UASM_i_LW(&p, i, offsetof(struct pt_regs, regs[i]), K1);
+ }
- LONG_L $28, PT_R28(k1)
- LONG_L $29, PT_R29(k1)
- LONG_L $30, PT_R30(k1)
+ UASM_i_LW(&p, K0, offsetof(struct pt_regs, hi), K1);
+ uasm_i_mthi(&p, K0);
- LONG_L k0, PT_HI(k1)
- mthi k0
-
- LONG_L k0, PT_LO(k1)
- mtlo k0
+ UASM_i_LW(&p, K0, offsetof(struct pt_regs, lo), K1);
+ uasm_i_mtlo(&p, K0);
/* Restore RDHWR access */
- INT_L k0, hwrena
- mtc0 k0, CP0_HWRENA
+ UASM_i_LA_mostly(&p, K0, (long)&hwrena);
+ uasm_i_lw(&p, K0, uasm_rel_lo((long)&hwrena), K0);
+ uasm_i_mtc0(&p, K0, C0_HWRENA);
/* Restore RA, which is the address we will return to */
- LONG_L ra, PT_R31(k1)
- j ra
- nop
+ UASM_i_LW(&p, RA, offsetof(struct pt_regs, regs[RA]), K1);
+ uasm_i_jr(&p, RA);
+ uasm_i_nop(&p);
-VECTOR_END(MIPSX(GuestExceptionEnd))
-.end MIPSX(GuestException)
+ return p;
+}
-MIPSX(exceptions):
- ####
- ##### The exception handlers.
- #####
- .word _C_LABEL(MIPSX(GuestException)) # 0
- .word _C_LABEL(MIPSX(GuestException)) # 1
- .word _C_LABEL(MIPSX(GuestException)) # 2
- .word _C_LABEL(MIPSX(GuestException)) # 3
- .word _C_LABEL(MIPSX(GuestException)) # 4
- .word _C_LABEL(MIPSX(GuestException)) # 5
- .word _C_LABEL(MIPSX(GuestException)) # 6
- .word _C_LABEL(MIPSX(GuestException)) # 7
- .word _C_LABEL(MIPSX(GuestException)) # 8
- .word _C_LABEL(MIPSX(GuestException)) # 9
- .word _C_LABEL(MIPSX(GuestException)) # 10
- .word _C_LABEL(MIPSX(GuestException)) # 11
- .word _C_LABEL(MIPSX(GuestException)) # 12
- .word _C_LABEL(MIPSX(GuestException)) # 13
- .word _C_LABEL(MIPSX(GuestException)) # 14
- .word _C_LABEL(MIPSX(GuestException)) # 15
- .word _C_LABEL(MIPSX(GuestException)) # 16
- .word _C_LABEL(MIPSX(GuestException)) # 17
- .word _C_LABEL(MIPSX(GuestException)) # 18
- .word _C_LABEL(MIPSX(GuestException)) # 19
- .word _C_LABEL(MIPSX(GuestException)) # 20
- .word _C_LABEL(MIPSX(GuestException)) # 21
- .word _C_LABEL(MIPSX(GuestException)) # 22
- .word _C_LABEL(MIPSX(GuestException)) # 23
- .word _C_LABEL(MIPSX(GuestException)) # 24
- .word _C_LABEL(MIPSX(GuestException)) # 25
- .word _C_LABEL(MIPSX(GuestException)) # 26
- .word _C_LABEL(MIPSX(GuestException)) # 27
- .word _C_LABEL(MIPSX(GuestException)) # 28
- .word _C_LABEL(MIPSX(GuestException)) # 29
- .word _C_LABEL(MIPSX(GuestException)) # 30
- .word _C_LABEL(MIPSX(GuestException)) # 31
diff --git a/arch/mips/kvm/interrupt.h b/arch/mips/kvm/interrupt.h
index d661c100b219..fb118a2c8379 100644
--- a/arch/mips/kvm/interrupt.h
+++ b/arch/mips/kvm/interrupt.h
@@ -28,10 +28,6 @@
#define MIPS_EXC_MAX 12
/* XXXSL More to follow */
-extern char __kvm_mips_vcpu_run_end[];
-extern char mips32_exception[], mips32_exceptionEnd[];
-extern char mips32_GuestException[], mips32_GuestExceptionEnd[];
-
#define C_TI (_ULCAST_(1) << 30)
#define KVM_MIPS_IRQ_DELIVER_ALL_AT_ONCE (0)
diff --git a/arch/mips/kvm/mips.c b/arch/mips/kvm/mips.c
index 5a2b9034a05c..f00cf5f9ac52 100644
--- a/arch/mips/kvm/mips.c
+++ b/arch/mips/kvm/mips.c
@@ -247,8 +247,8 @@ void kvm_arch_commit_memory_region(struct kvm *kvm,
struct kvm_vcpu *kvm_arch_vcpu_create(struct kvm *kvm, unsigned int id)
{
- int err, size, offset;
- void *gebase;
+ int err, size;
+ void *gebase, *p;
int i;
struct kvm_vcpu *vcpu = kzalloc(sizeof(struct kvm_vcpu), GFP_KERNEL);
@@ -286,41 +286,28 @@ struct kvm_vcpu *kvm_arch_vcpu_create(struct kvm *kvm, unsigned int id)
/* Save new ebase */
vcpu->arch.guest_ebase = gebase;
- /* Copy L1 Guest Exception handler to correct offset */
+ /* Build guest exception vectors dynamically in unmapped memory */
/* TLB Refill, EXL = 0 */
- memcpy(gebase, mips32_exception,
- mips32_exceptionEnd - mips32_exception);
+ kvm_mips_build_exception(gebase);
/* General Exception Entry point */
- memcpy(gebase + 0x180, mips32_exception,
- mips32_exceptionEnd - mips32_exception);
+ kvm_mips_build_exception(gebase + 0x180);
/* For vectored interrupts poke the exception code @ all offsets 0-7 */
for (i = 0; i < 8; i++) {
kvm_debug("L1 Vectored handler @ %p\n",
gebase + 0x200 + (i * VECTORSPACING));
- memcpy(gebase + 0x200 + (i * VECTORSPACING), mips32_exception,
- mips32_exceptionEnd - mips32_exception);
+ kvm_mips_build_exception(gebase + 0x200 + i * VECTORSPACING);
}
- /* General handler, relocate to unmapped space for sanity's sake */
- offset = 0x2000;
- kvm_debug("Installing KVM Exception handlers @ %p, %#x bytes\n",
- gebase + offset,
- mips32_GuestExceptionEnd - mips32_GuestException);
+ /* General exit handler */
+ p = gebase + 0x2000;
+ p = kvm_mips_build_exit(p);
- memcpy(gebase + offset, mips32_GuestException,
- mips32_GuestExceptionEnd - mips32_GuestException);
-
-#ifdef MODULE
- offset += mips32_GuestExceptionEnd - mips32_GuestException;
- memcpy(gebase + offset, (char *)__kvm_mips_vcpu_run,
- __kvm_mips_vcpu_run_end - (char *)__kvm_mips_vcpu_run);
- vcpu->arch.vcpu_run = gebase + offset;
-#else
- vcpu->arch.vcpu_run = __kvm_mips_vcpu_run;
-#endif
+ /* Guest entry routine */
+ vcpu->arch.vcpu_run = p;
+ p = kvm_mips_build_vcpu_run(p);
/* Invalidate the icache for these ranges */
local_flush_icache_range((unsigned long)gebase,
--
2.4.10
^ permalink raw reply related [flat|nested] 36+ messages in thread* Re: [PATCH 06/14] MIPS; KVM: Convert exception entry to uasm
@ 2016-07-04 18:53 ` James Hogan
0 siblings, 0 replies; 36+ messages in thread
From: James Hogan @ 2016-07-04 18:53 UTC (permalink / raw)
To: Paolo Bonzini, Radim Krčmář, Ralf Baechle; +Cc: linux-mips, kvm
[-- Attachment #1: Type: text/plain, Size: 39246 bytes --]
Hi Paolo,
On Thu, Jun 23, 2016 at 05:34:39PM +0100, James Hogan wrote:
> Convert the whole of locore.S (assembly to enter guest and handle
> exception entry) to be generated dynamically with uasm. This is done
> with minimal changes to the resulting code.
Before I forget, please feel free to s/;/:/ in the subject line if/when
you apply (but I know I need some acks yet).
Thanks
James
>
> The main changes are:
> - Some constants are generated by uasm using LUI+ADDIU instead of
> LUI+ORI.
> - Loading of lo and hi are swapped around in vcpu_run but not when
> resuming the guest after an exit. Both bits of logic are now generated
> by the same code.
> - Register MOVEs in uasm use different ADDU operand ordering to GNU as,
> putting zero register into rs instead of rt.
> - The JALR.HB to call the C exit handler is switched to JALR, since the
> hazard barrier would appear to be unnecessary.
>
> This will allow further optimisation in the future to dynamically handle
> the capabilities of the CPU.
>
> Signed-off-by: James Hogan <james.hogan@imgtec.com>
> Cc: Paolo Bonzini <pbonzini@redhat.com>
> Cc: Radim Krčmář <rkrcmar@redhat.com>
> Cc: Ralf Baechle <ralf@linux-mips.org>
> Cc: linux-mips@linux-mips.org
> Cc: kvm@vger.kernel.org
> ---
> arch/mips/include/asm/kvm_host.h | 8 +-
> arch/mips/kvm/Kconfig | 1 +
> arch/mips/kvm/Makefile | 2 +-
> arch/mips/kvm/{locore.S => entry.c} | 884 ++++++++++++++++++------------------
> arch/mips/kvm/interrupt.h | 4 -
> arch/mips/kvm/mips.c | 37 +-
> 6 files changed, 472 insertions(+), 464 deletions(-)
> rename arch/mips/kvm/{locore.S => entry.c} (20%)
>
> diff --git a/arch/mips/include/asm/kvm_host.h b/arch/mips/include/asm/kvm_host.h
> index b0773c6d622f..2e76e899079c 100644
> --- a/arch/mips/include/asm/kvm_host.h
> +++ b/arch/mips/include/asm/kvm_host.h
> @@ -533,8 +533,12 @@ int kvm_mips_emulation_init(struct kvm_mips_callbacks **install_callbacks);
> /* Debug: dump vcpu state */
> int kvm_arch_vcpu_dump_regs(struct kvm_vcpu *vcpu);
>
> -/* Trampoline ASM routine to start running in "Guest" context */
> -extern int __kvm_mips_vcpu_run(struct kvm_run *run, struct kvm_vcpu *vcpu);
> +extern int kvm_mips_handle_exit(struct kvm_run *run, struct kvm_vcpu *vcpu);
> +
> +/* Building of entry/exception code */
> +void *kvm_mips_build_vcpu_run(void *addr);
> +void *kvm_mips_build_exception(void *addr);
> +void *kvm_mips_build_exit(void *addr);
>
> /* FPU/MSA context management */
> void __kvm_save_fpu(struct kvm_vcpu_arch *vcpu);
> diff --git a/arch/mips/kvm/Kconfig b/arch/mips/kvm/Kconfig
> index 2ae12825529f..7c56d6b124d1 100644
> --- a/arch/mips/kvm/Kconfig
> +++ b/arch/mips/kvm/Kconfig
> @@ -17,6 +17,7 @@ if VIRTUALIZATION
> config KVM
> tristate "Kernel-based Virtual Machine (KVM) support"
> depends on HAVE_KVM
> + select EXPORT_UASM
> select PREEMPT_NOTIFIERS
> select ANON_INODES
> select KVM_MMIO
> diff --git a/arch/mips/kvm/Makefile b/arch/mips/kvm/Makefile
> index 0aabe40fcac9..847429de780d 100644
> --- a/arch/mips/kvm/Makefile
> +++ b/arch/mips/kvm/Makefile
> @@ -7,7 +7,7 @@ EXTRA_CFLAGS += -Ivirt/kvm -Iarch/mips/kvm
>
> common-objs-$(CONFIG_CPU_HAS_MSA) += msa.o
>
> -kvm-objs := $(common-objs-y) mips.o emulate.o locore.o \
> +kvm-objs := $(common-objs-y) mips.o emulate.o entry.o \
> interrupt.o stats.o commpage.o \
> dyntrans.o trap_emul.o fpu.o
> kvm-objs += mmu.o
> diff --git a/arch/mips/kvm/locore.S b/arch/mips/kvm/entry.c
> similarity index 20%
> rename from arch/mips/kvm/locore.S
> rename to arch/mips/kvm/entry.c
> index 698286c0f732..9a18b4939b35 100644
> --- a/arch/mips/kvm/locore.S
> +++ b/arch/mips/kvm/entry.c
> @@ -3,373 +3,445 @@
> * License. See the file "COPYING" in the main directory of this archive
> * for more details.
> *
> - * Main entry point for the guest, exception handling.
> + * Generation of main entry point for the guest, exception handling.
> *
> - * Copyright (C) 2012 MIPS Technologies, Inc. All rights reserved.
> + * Copyright (C) 2012 MIPS Technologies, Inc.
> * Authors: Sanjay Lal <sanjayl@kymasys.com>
> + *
> + * Copyright (C) 2016 Imagination Technologies Ltd.
> */
>
> -#include <asm/asm.h>
> -#include <asm/asmmacro.h>
> -#include <asm/regdef.h>
> -#include <asm/mipsregs.h>
> -#include <asm/stackframe.h>
> -#include <asm/asm-offsets.h>
> -
> -#define _C_LABEL(x) x
> -#define MIPSX(name) mips32_ ## name
> +#include <linux/kvm_host.h>
> +#include <asm/msa.h>
> +#include <asm/setup.h>
> +#include <asm/uasm.h>
> +
> +/* Register names */
> +#define ZERO 0
> +#define AT 1
> +#define V0 2
> +#define V1 3
> +#define A0 4
> +#define A1 5
> +
> +#if _MIPS_SIM == _MIPS_SIM_ABI32
> +#define T0 8
> +#define T1 9
> +#define T2 10
> +#define T3 11
> +#endif /* _MIPS_SIM == _MIPS_SIM_ABI32 */
> +
> +#if _MIPS_SIM == _MIPS_SIM_ABI64 || _MIPS_SIM == _MIPS_SIM_NABI32
> +#define T0 12
> +#define T1 13
> +#define T2 14
> +#define T3 15
> +#endif /* _MIPS_SIM == _MIPS_SIM_ABI64 || _MIPS_SIM == _MIPS_SIM_NABI32 */
> +
> +#define S0 16
> +#define S1 17
> +#define T9 25
> +#define K0 26
> +#define K1 27
> +#define GP 28
> +#define SP 29
> +#define RA 31
> +
> +/* Some CP0 registers */
> +#define C0_HWRENA 7, 0
> +#define C0_BADVADDR 8, 0
> +#define C0_ENTRYHI 10, 0
> +#define C0_STATUS 12, 0
> +#define C0_CAUSE 13, 0
> +#define C0_EPC 14, 0
> +#define C0_EBASE 15, 1
> +#define C0_CONFIG3 16, 3
> +#define C0_CONFIG5 16, 5
> +#define C0_DDATA_LO 28, 3
> +#define C0_ERROREPC 30, 0
> +
> #define CALLFRAME_SIZ 32
>
> -/*
> - * VECTOR
> - * exception vector entrypoint
> - */
> -#define VECTOR(x, regmask) \
> - .ent _C_LABEL(x),0; \
> - EXPORT(x);
> -
> -#define VECTOR_END(x) \
> - EXPORT(x);
> -
> -/* Overload, Danger Will Robinson!! */
> -#define PT_HOST_USERLOCAL PT_EPC
> +enum label_id {
> + label_fpu_1 = 1,
> + label_msa_1,
> + label_return_to_host,
> + label_kernel_asid,
> +};
>
> -#define CP0_DDATA_LO $28,3
> +UASM_L_LA(_fpu_1)
> +UASM_L_LA(_msa_1)
> +UASM_L_LA(_return_to_host)
> +UASM_L_LA(_kernel_asid)
>
> -/* Resume Flags */
> -#define RESUME_FLAG_HOST (1<<1) /* Resume host? */
> +static void *kvm_mips_build_enter_guest(void *addr);
> +static void *kvm_mips_build_ret_from_exit(void *addr);
> +static void *kvm_mips_build_ret_to_guest(void *addr);
> +static void *kvm_mips_build_ret_to_host(void *addr);
>
> -#define RESUME_GUEST 0
> -#define RESUME_HOST RESUME_FLAG_HOST
> -
> -/*
> - * __kvm_mips_vcpu_run: entry point to the guest
> - * a0: run
> - * a1: vcpu
> +/**
> + * kvm_mips_build_vcpu_run() - Assemble function to start running a guest VCPU.
> + * @addr: Address to start writing code.
> + *
> + * Assemble the start of the vcpu_run function to run a guest VCPU. The function
> + * conforms to the following prototype:
> + *
> + * int vcpu_run(struct kvm_run *run, struct kvm_vcpu *vcpu);
> + *
> + * The exit from the guest and return to the caller is handled by the code
> + * generated by kvm_mips_build_ret_to_host().
> + *
> + * Returns: Next address after end of written function.
> */
> - .set noreorder
> +void *kvm_mips_build_vcpu_run(void *addr)
> +{
> + u32 *p = addr;
> + unsigned int i;
> +
> + /*
> + * A0: run
> + * A1: vcpu
> + */
>
> -FEXPORT(__kvm_mips_vcpu_run)
> /* k0/k1 not being used in host kernel context */
> - INT_ADDIU k1, sp, -PT_SIZE
> - LONG_S $16, PT_R16(k1)
> - LONG_S $17, PT_R17(k1)
> - LONG_S $18, PT_R18(k1)
> - LONG_S $19, PT_R19(k1)
> - LONG_S $20, PT_R20(k1)
> - LONG_S $21, PT_R21(k1)
> - LONG_S $22, PT_R22(k1)
> - LONG_S $23, PT_R23(k1)
> -
> - LONG_S $28, PT_R28(k1)
> - LONG_S $29, PT_R29(k1)
> - LONG_S $30, PT_R30(k1)
> - LONG_S $31, PT_R31(k1)
> + uasm_i_addiu(&p, K1, SP, -(int)sizeof(struct pt_regs));
> + for (i = 16; i < 32; ++i) {
> + if (i == 24)
> + i = 28;
> + UASM_i_SW(&p, i, offsetof(struct pt_regs, regs[i]), K1);
> + }
>
> /* Save hi/lo */
> - mflo v0
> - LONG_S v0, PT_LO(k1)
> - mfhi v1
> - LONG_S v1, PT_HI(k1)
> + uasm_i_mflo(&p, V0);
> + UASM_i_SW(&p, V0, offsetof(struct pt_regs, lo), K1);
> + uasm_i_mfhi(&p, V1);
> + UASM_i_SW(&p, V1, offsetof(struct pt_regs, hi), K1);
>
> /* Save host status */
> - mfc0 v0, CP0_STATUS
> - LONG_S v0, PT_STATUS(k1)
> + uasm_i_mfc0(&p, V0, C0_STATUS);
> + UASM_i_SW(&p, V0, offsetof(struct pt_regs, cp0_status), K1);
>
> /* Save DDATA_LO, will be used to store pointer to vcpu */
> - mfc0 v1, CP0_DDATA_LO
> - LONG_S v1, PT_HOST_USERLOCAL(k1)
> + uasm_i_mfc0(&p, V1, C0_DDATA_LO);
> + UASM_i_SW(&p, V1, offsetof(struct pt_regs, cp0_epc), K1);
>
> /* DDATA_LO has pointer to vcpu */
> - mtc0 a1, CP0_DDATA_LO
> + uasm_i_mtc0(&p, A1, C0_DDATA_LO);
>
> /* Offset into vcpu->arch */
> - INT_ADDIU k1, a1, VCPU_HOST_ARCH
> + uasm_i_addiu(&p, K1, A1, offsetof(struct kvm_vcpu, arch));
>
> /*
> * Save the host stack to VCPU, used for exception processing
> * when we exit from the Guest
> */
> - LONG_S sp, VCPU_HOST_STACK(k1)
> + UASM_i_SW(&p, SP, offsetof(struct kvm_vcpu_arch, host_stack), K1);
>
> /* Save the kernel gp as well */
> - LONG_S gp, VCPU_HOST_GP(k1)
> + UASM_i_SW(&p, GP, offsetof(struct kvm_vcpu_arch, host_gp), K1);
>
> /*
> * Setup status register for running the guest in UM, interrupts
> * are disabled
> */
> - li k0, (ST0_EXL | KSU_USER | ST0_BEV)
> - mtc0 k0, CP0_STATUS
> - ehb
> + UASM_i_LA(&p, K0, ST0_EXL | KSU_USER | ST0_BEV);
> + uasm_i_mtc0(&p, K0, C0_STATUS);
> + uasm_i_ehb(&p);
>
> /* load up the new EBASE */
> - LONG_L k0, VCPU_GUEST_EBASE(k1)
> - mtc0 k0, CP0_EBASE
> + UASM_i_LW(&p, K0, offsetof(struct kvm_vcpu_arch, guest_ebase), K1);
> + uasm_i_mtc0(&p, K0, C0_EBASE);
>
> /*
> * Now that the new EBASE has been loaded, unset BEV, set
> * interrupt mask as it was but make sure that timer interrupts
> * are enabled
> */
> - li k0, (ST0_EXL | KSU_USER | ST0_IE)
> - andi v0, v0, ST0_IM
> - or k0, k0, v0
> - mtc0 k0, CP0_STATUS
> - ehb
> + uasm_i_addiu(&p, K0, ZERO, ST0_EXL | KSU_USER | ST0_IE);
> + uasm_i_andi(&p, V0, V0, ST0_IM);
> + uasm_i_or(&p, K0, K0, V0);
> + uasm_i_mtc0(&p, K0, C0_STATUS);
> + uasm_i_ehb(&p);
> +
> + p = kvm_mips_build_enter_guest(p);
> +
> + return p;
> +}
> +
> +/**
> + * kvm_mips_build_enter_guest() - Assemble code to resume guest execution.
> + * @addr: Address to start writing code.
> + *
> + * Assemble the code to resume guest execution. This code is common between the
> + * initial entry into the guest from the host, and returning from the exit
> + * handler back to the guest.
> + *
> + * Returns: Next address after end of written function.
> + */
> +static void *kvm_mips_build_enter_guest(void *addr)
> +{
> + u32 *p = addr;
> + unsigned int i;
> + struct uasm_label labels[2];
> + struct uasm_reloc relocs[2];
> + struct uasm_label *l = labels;
> + struct uasm_reloc *r = relocs;
> +
> + memset(labels, 0, sizeof(labels));
> + memset(relocs, 0, sizeof(relocs));
>
> /* Set Guest EPC */
> - LONG_L t0, VCPU_PC(k1)
> - mtc0 t0, CP0_EPC
> + UASM_i_LW(&p, T0, offsetof(struct kvm_vcpu_arch, pc), K1);
> + uasm_i_mtc0(&p, T0, C0_EPC);
>
> -FEXPORT(__kvm_mips_load_asid)
> /* Set the ASID for the Guest Kernel */
> - PTR_L t0, VCPU_COP0(k1)
> - LONG_L t0, COP0_STATUS(t0)
> - andi t0, KSU_USER | ST0_ERL | ST0_EXL
> - xori t0, KSU_USER
> - bnez t0, 1f /* If kernel */
> - INT_ADDIU t1, k1, VCPU_GUEST_KERNEL_ASID /* (BD) */
> - INT_ADDIU t1, k1, VCPU_GUEST_USER_ASID /* else user */
> -1:
> - /* t1: contains the base of the ASID array, need to get the cpu id */
> - LONG_L t2, TI_CPU($28) /* smp_processor_id */
> - INT_SLL t2, t2, 2 /* x4 */
> - REG_ADDU t3, t1, t2
> - LONG_L k0, (t3)
> + UASM_i_LW(&p, T0, offsetof(struct kvm_vcpu_arch, cop0), K1);
> + UASM_i_LW(&p, T0, offsetof(struct mips_coproc, reg[MIPS_CP0_STATUS][0]),
> + T0);
> + uasm_i_andi(&p, T0, T0, KSU_USER | ST0_ERL | ST0_EXL);
> + uasm_i_xori(&p, T0, T0, KSU_USER);
> + uasm_il_bnez(&p, &r, T0, label_kernel_asid);
> + uasm_i_addiu(&p, T1, K1,
> + offsetof(struct kvm_vcpu_arch, guest_kernel_asid));
> + /* else user */
> + uasm_i_addiu(&p, T1, K1,
> + offsetof(struct kvm_vcpu_arch, guest_user_asid));
> + uasm_l_kernel_asid(&l, p);
> +
> + /* t1: contains the base of the ASID array, need to get the cpu id */
> + /* smp_processor_id */
> + UASM_i_LW(&p, T2, offsetof(struct thread_info, cpu), GP);
> + /* x4 */
> + uasm_i_sll(&p, T2, T2, 2);
> + UASM_i_ADDU(&p, T3, T1, T2);
> + UASM_i_LW(&p, K0, 0, T3);
> #ifdef CONFIG_MIPS_ASID_BITS_VARIABLE
> - li t3, CPUINFO_SIZE/4
> - mul t2, t2, t3 /* x sizeof(struct cpuinfo_mips)/4 */
> - LONG_L t2, (cpu_data + CPUINFO_ASID_MASK)(t2)
> - and k0, k0, t2
> + /* x sizeof(struct cpuinfo_mips)/4 */
> + uasm_i_addiu(&p, T3, ZERO, sizeof(struct cpuinfo_mips)/4);
> + uasm_i_mul(&p, T2, T2, T3);
> +
> + UASM_i_LA_mostly(&p, AT, (long)&cpu_data[0].asid_mask);
> + UASM_i_ADDU(&p, AT, AT, T2);
> + UASM_i_LW(&p, T2, uasm_rel_lo((long)&cpu_data[0].asid_mask), AT);
> + uasm_i_and(&p, K0, K0, T2);
> #else
> - andi k0, k0, MIPS_ENTRYHI_ASID
> + uasm_i_andi(&p, K0, K0, MIPS_ENTRYHI_ASID);
> #endif
> - mtc0 k0, CP0_ENTRYHI
> - ehb
> + uasm_i_mtc0(&p, K0, C0_ENTRYHI);
> + uasm_i_ehb(&p);
>
> /* Disable RDHWR access */
> - mtc0 zero, CP0_HWRENA
> -
> - .set noat
> - /* Now load up the Guest Context from VCPU */
> - LONG_L $1, VCPU_R1(k1)
> - LONG_L $2, VCPU_R2(k1)
> - LONG_L $3, VCPU_R3(k1)
> -
> - LONG_L $4, VCPU_R4(k1)
> - LONG_L $5, VCPU_R5(k1)
> - LONG_L $6, VCPU_R6(k1)
> - LONG_L $7, VCPU_R7(k1)
> -
> - LONG_L $8, VCPU_R8(k1)
> - LONG_L $9, VCPU_R9(k1)
> - LONG_L $10, VCPU_R10(k1)
> - LONG_L $11, VCPU_R11(k1)
> - LONG_L $12, VCPU_R12(k1)
> - LONG_L $13, VCPU_R13(k1)
> - LONG_L $14, VCPU_R14(k1)
> - LONG_L $15, VCPU_R15(k1)
> - LONG_L $16, VCPU_R16(k1)
> - LONG_L $17, VCPU_R17(k1)
> - LONG_L $18, VCPU_R18(k1)
> - LONG_L $19, VCPU_R19(k1)
> - LONG_L $20, VCPU_R20(k1)
> - LONG_L $21, VCPU_R21(k1)
> - LONG_L $22, VCPU_R22(k1)
> - LONG_L $23, VCPU_R23(k1)
> - LONG_L $24, VCPU_R24(k1)
> - LONG_L $25, VCPU_R25(k1)
> -
> - /* k0/k1 loaded up later */
> -
> - LONG_L $28, VCPU_R28(k1)
> - LONG_L $29, VCPU_R29(k1)
> - LONG_L $30, VCPU_R30(k1)
> - LONG_L $31, VCPU_R31(k1)
> + uasm_i_mtc0(&p, ZERO, C0_HWRENA);
> +
> + /* load the guest context from VCPU and return */
> + for (i = 1; i < 32; ++i) {
> + /* Guest k0/k1 loaded later */
> + if (i == K0 || i == K1)
> + continue;
> + UASM_i_LW(&p, i, offsetof(struct kvm_vcpu_arch, gprs[i]), K1);
> + }
>
> /* Restore hi/lo */
> - LONG_L k0, VCPU_LO(k1)
> - mtlo k0
> + UASM_i_LW(&p, K0, offsetof(struct kvm_vcpu_arch, hi), K1);
> + uasm_i_mthi(&p, K0);
>
> - LONG_L k0, VCPU_HI(k1)
> - mthi k0
> + UASM_i_LW(&p, K0, offsetof(struct kvm_vcpu_arch, lo), K1);
> + uasm_i_mtlo(&p, K0);
>
> -FEXPORT(__kvm_mips_load_k0k1)
> /* Restore the guest's k0/k1 registers */
> - LONG_L k0, VCPU_R26(k1)
> - LONG_L k1, VCPU_R27(k1)
> + UASM_i_LW(&p, K0, offsetof(struct kvm_vcpu_arch, gprs[K0]), K1);
> + UASM_i_LW(&p, K1, offsetof(struct kvm_vcpu_arch, gprs[K1]), K1);
>
> /* Jump to guest */
> - eret
> -EXPORT(__kvm_mips_vcpu_run_end)
> -
> -VECTOR(MIPSX(exception), unknown)
> -/* Find out what mode we came from and jump to the proper handler. */
> - mtc0 k0, CP0_ERROREPC #01: Save guest k0
> - ehb #02:
> -
> - mfc0 k0, CP0_EBASE #02: Get EBASE
> - INT_SRL k0, k0, 10 #03: Get rid of CPUNum
> - INT_SLL k0, k0, 10 #04
> - LONG_S k1, 0x3000(k0) #05: Save k1 @ offset 0x3000
> - INT_ADDIU k0, k0, 0x2000 #06: Exception handler is
> - # installed @ offset 0x2000
> - j k0 #07: jump to the function
> - nop #08: branch delay slot
> -VECTOR_END(MIPSX(exceptionEnd))
> -.end MIPSX(exception)
> -
> -/*
> - * Generic Guest exception handler. We end up here when the guest
> - * does something that causes a trap to kernel mode.
> + uasm_i_eret(&p);
> +
> + uasm_resolve_relocs(relocs, labels);
> +
> + return p;
> +}
> +
> +/**
> + * kvm_mips_build_exception() - Assemble first level guest exception handler.
> + * @addr: Address to start writing code.
> + *
> + * Assemble exception vector code for guest execution. The generated vector will
> + * jump to the common exception handler generated by kvm_mips_build_exit().
> + *
> + * Returns: Next address after end of written function.
> + */
> +void *kvm_mips_build_exception(void *addr)
> +{
> + u32 *p = addr;
> +
> + /* Save guest k0 */
> + uasm_i_mtc0(&p, K0, C0_ERROREPC);
> + uasm_i_ehb(&p);
> +
> + /* Get EBASE */
> + uasm_i_mfc0(&p, K0, C0_EBASE);
> + /* Get rid of CPUNum */
> + uasm_i_srl(&p, K0, K0, 10);
> + uasm_i_sll(&p, K0, K0, 10);
> + /* Save k1 @ offset 0x3000 */
> + UASM_i_SW(&p, K1, 0x3000, K0);
> +
> + /* Exception handler is installed @ offset 0x2000 */
> + uasm_i_addiu(&p, K0, K0, 0x2000);
> + /* Jump to the function */
> + uasm_i_jr(&p, K0);
> + uasm_i_nop(&p);
> +
> + return p;
> +}
> +
> +/**
> + * kvm_mips_build_exit() - Assemble common guest exit handler.
> + * @addr: Address to start writing code.
> + *
> + * Assemble the generic guest exit handling code. This is called by the
> + * exception vectors (generated by kvm_mips_build_exception()), and calls
> + * kvm_mips_handle_exit(), then either resumes the guest or returns to the host
> + * depending on the return value.
> + *
> + * Returns: Next address after end of written function.
> */
> -NESTED (MIPSX(GuestException), CALLFRAME_SIZ, ra)
> - /* Get the VCPU pointer from DDTATA_LO */
> - mfc0 k1, CP0_DDATA_LO
> - INT_ADDIU k1, k1, VCPU_HOST_ARCH
> +void *kvm_mips_build_exit(void *addr)
> +{
> + u32 *p = addr;
> + unsigned int i;
> + struct uasm_label labels[3];
> + struct uasm_reloc relocs[3];
> + struct uasm_label *l = labels;
> + struct uasm_reloc *r = relocs;
> +
> + memset(labels, 0, sizeof(labels));
> + memset(relocs, 0, sizeof(relocs));
> +
> + /*
> + * Generic Guest exception handler. We end up here when the guest
> + * does something that causes a trap to kernel mode.
> + */
> +
> + /* Get the VCPU pointer from DDATA_LO */
> + uasm_i_mfc0(&p, K1, C0_DDATA_LO);
> + uasm_i_addiu(&p, K1, K1, offsetof(struct kvm_vcpu, arch));
>
> /* Start saving Guest context to VCPU */
> - LONG_S $0, VCPU_R0(k1)
> - LONG_S $1, VCPU_R1(k1)
> - LONG_S $2, VCPU_R2(k1)
> - LONG_S $3, VCPU_R3(k1)
> - LONG_S $4, VCPU_R4(k1)
> - LONG_S $5, VCPU_R5(k1)
> - LONG_S $6, VCPU_R6(k1)
> - LONG_S $7, VCPU_R7(k1)
> - LONG_S $8, VCPU_R8(k1)
> - LONG_S $9, VCPU_R9(k1)
> - LONG_S $10, VCPU_R10(k1)
> - LONG_S $11, VCPU_R11(k1)
> - LONG_S $12, VCPU_R12(k1)
> - LONG_S $13, VCPU_R13(k1)
> - LONG_S $14, VCPU_R14(k1)
> - LONG_S $15, VCPU_R15(k1)
> - LONG_S $16, VCPU_R16(k1)
> - LONG_S $17, VCPU_R17(k1)
> - LONG_S $18, VCPU_R18(k1)
> - LONG_S $19, VCPU_R19(k1)
> - LONG_S $20, VCPU_R20(k1)
> - LONG_S $21, VCPU_R21(k1)
> - LONG_S $22, VCPU_R22(k1)
> - LONG_S $23, VCPU_R23(k1)
> - LONG_S $24, VCPU_R24(k1)
> - LONG_S $25, VCPU_R25(k1)
> -
> - /* Guest k0/k1 saved later */
> -
> - LONG_S $28, VCPU_R28(k1)
> - LONG_S $29, VCPU_R29(k1)
> - LONG_S $30, VCPU_R30(k1)
> - LONG_S $31, VCPU_R31(k1)
> -
> - .set at
> + for (i = 0; i < 32; ++i) {
> + /* Guest k0/k1 saved later */
> + if (i == K0 || i == K1)
> + continue;
> + UASM_i_SW(&p, i, offsetof(struct kvm_vcpu_arch, gprs[i]), K1);
> + }
>
> /* We need to save hi/lo and restore them on the way out */
> - mfhi t0
> - LONG_S t0, VCPU_HI(k1)
> + uasm_i_mfhi(&p, T0);
> + UASM_i_SW(&p, T0, offsetof(struct kvm_vcpu_arch, hi), K1);
>
> - mflo t0
> - LONG_S t0, VCPU_LO(k1)
> + uasm_i_mflo(&p, T0);
> + UASM_i_SW(&p, T0, offsetof(struct kvm_vcpu_arch, lo), K1);
>
> /* Finally save guest k0/k1 to VCPU */
> - mfc0 t0, CP0_ERROREPC
> - LONG_S t0, VCPU_R26(k1)
> + uasm_i_mfc0(&p, T0, C0_ERROREPC);
> + UASM_i_SW(&p, T0, offsetof(struct kvm_vcpu_arch, gprs[K0]), K1);
>
> /* Get GUEST k1 and save it in VCPU */
> - PTR_LI t1, ~0x2ff
> - mfc0 t0, CP0_EBASE
> - and t0, t0, t1
> - LONG_L t0, 0x3000(t0)
> - LONG_S t0, VCPU_R27(k1)
> + uasm_i_addiu(&p, T1, ZERO, ~0x2ff);
> + uasm_i_mfc0(&p, T0, C0_EBASE);
> + uasm_i_and(&p, T0, T0, T1);
> + UASM_i_LW(&p, T0, 0x3000, T0);
> + UASM_i_SW(&p, T0, offsetof(struct kvm_vcpu_arch, gprs[K1]), K1);
>
> /* Now that context has been saved, we can use other registers */
>
> /* Restore vcpu */
> - mfc0 a1, CP0_DDATA_LO
> - move s1, a1
> + uasm_i_mfc0(&p, A1, C0_DDATA_LO);
> + uasm_i_move(&p, S1, A1);
>
> /* Restore run (vcpu->run) */
> - LONG_L a0, VCPU_RUN(a1)
> + UASM_i_LW(&p, A0, offsetof(struct kvm_vcpu, run), A1);
> /* Save pointer to run in s0, will be saved by the compiler */
> - move s0, a0
> + uasm_i_move(&p, S0, A0);
>
> /*
> - * Save Host level EPC, BadVaddr and Cause to VCPU, useful to
> - * process the exception
> + * Save Host level EPC, BadVaddr and Cause to VCPU, useful to process
> + * the exception
> */
> - mfc0 k0,CP0_EPC
> - LONG_S k0, VCPU_PC(k1)
> + uasm_i_mfc0(&p, K0, C0_EPC);
> + UASM_i_SW(&p, K0, offsetof(struct kvm_vcpu_arch, pc), K1);
>
> - mfc0 k0, CP0_BADVADDR
> - LONG_S k0, VCPU_HOST_CP0_BADVADDR(k1)
> + uasm_i_mfc0(&p, K0, C0_BADVADDR);
> + UASM_i_SW(&p, K0, offsetof(struct kvm_vcpu_arch, host_cp0_badvaddr),
> + K1);
>
> - mfc0 k0, CP0_CAUSE
> - sw k0, VCPU_HOST_CP0_CAUSE(k1)
> + uasm_i_mfc0(&p, K0, C0_CAUSE);
> + uasm_i_sw(&p, K0, offsetof(struct kvm_vcpu_arch, host_cp0_cause), K1);
>
> /* Now restore the host state just enough to run the handlers */
>
> /* Switch EBASE to the one used by Linux */
> /* load up the host EBASE */
> - mfc0 v0, CP0_STATUS
> + uasm_i_mfc0(&p, V0, C0_STATUS);
>
> - or k0, v0, ST0_BEV
> + uasm_i_lui(&p, AT, ST0_BEV >> 16);
> + uasm_i_or(&p, K0, V0, AT);
>
> - mtc0 k0, CP0_STATUS
> - ehb
> + uasm_i_mtc0(&p, K0, C0_STATUS);
> + uasm_i_ehb(&p);
>
> - LONG_L k0, ebase
> - mtc0 k0,CP0_EBASE
> + UASM_i_LA_mostly(&p, K0, (long)&ebase);
> + UASM_i_LW(&p, K0, uasm_rel_lo((long)&ebase), K0);
> + uasm_i_mtc0(&p, K0, C0_EBASE);
>
> /*
> * If FPU is enabled, save FCR31 and clear it so that later ctc1's don't
> * trigger FPE for pending exceptions.
> */
> - and v1, v0, ST0_CU1
> - beqz v1, 1f
> - nop
> - .set push
> - SET_HARDFLOAT
> - cfc1 t0, fcr31
> - sw t0, VCPU_FCR31(k1)
> - ctc1 zero,fcr31
> - .set pop
> -1:
> + uasm_i_lui(&p, AT, ST0_CU1 >> 16);
> + uasm_i_and(&p, V1, V0, AT);
> + uasm_il_beqz(&p, &r, V1, label_fpu_1);
> + uasm_i_nop(&p);
> + uasm_i_cfc1(&p, T0, 31);
> + uasm_i_sw(&p, T0, offsetof(struct kvm_vcpu_arch, fpu.fcr31), K1);
> + uasm_i_ctc1(&p, ZERO, 31);
> + uasm_l_fpu_1(&l, p);
>
> #ifdef CONFIG_CPU_HAS_MSA
> /*
> * If MSA is enabled, save MSACSR and clear it so that later
> * instructions don't trigger MSAFPE for pending exceptions.
> */
> - mfc0 t0, CP0_CONFIG3
> - ext t0, t0, 28, 1 /* MIPS_CONF3_MSAP */
> - beqz t0, 1f
> - nop
> - mfc0 t0, CP0_CONFIG5
> - ext t0, t0, 27, 1 /* MIPS_CONF5_MSAEN */
> - beqz t0, 1f
> - nop
> - _cfcmsa t0, MSA_CSR
> - sw t0, VCPU_MSA_CSR(k1)
> - _ctcmsa MSA_CSR, zero
> -1:
> + uasm_i_mfc0(&p, T0, C0_CONFIG3);
> + uasm_i_ext(&p, T0, T0, 28, 1); /* MIPS_CONF3_MSAP */
> + uasm_il_beqz(&p, &r, T0, label_msa_1);
> + uasm_i_nop(&p);
> + uasm_i_mfc0(&p, T0, C0_CONFIG5);
> + uasm_i_ext(&p, T0, T0, 27, 1); /* MIPS_CONF5_MSAEN */
> + uasm_il_beqz(&p, &r, T0, label_msa_1);
> + uasm_i_nop(&p);
> + uasm_i_cfcmsa(&p, T0, MSA_CSR);
> + uasm_i_sw(&p, T0, offsetof(struct kvm_vcpu_arch, fpu.msacsr),
> + K1);
> + uasm_i_ctcmsa(&p, MSA_CSR, ZERO);
> + uasm_l_msa_1(&l, p);
> #endif
>
> /* Now that the new EBASE has been loaded, unset BEV and KSU_USER */
> - and v0, v0, ~(ST0_EXL | KSU_USER | ST0_IE)
> - or v0, v0, ST0_CU0
> - mtc0 v0, CP0_STATUS
> - ehb
> + uasm_i_addiu(&p, AT, ZERO, ~(ST0_EXL | KSU_USER | ST0_IE));
> + uasm_i_and(&p, V0, V0, AT);
> + uasm_i_lui(&p, AT, ST0_CU0 >> 16);
> + uasm_i_or(&p, V0, V0, AT);
> + uasm_i_mtc0(&p, V0, C0_STATUS);
> + uasm_i_ehb(&p);
>
> /* Load up host GP */
> - LONG_L gp, VCPU_HOST_GP(k1)
> + UASM_i_LW(&p, GP, offsetof(struct kvm_vcpu_arch, host_gp), K1);
>
> /* Need a stack before we can jump to "C" */
> - LONG_L sp, VCPU_HOST_STACK(k1)
> + UASM_i_LW(&p, SP, offsetof(struct kvm_vcpu_arch, host_stack), K1);
>
> /* Saved host state */
> - INT_ADDIU sp, sp, -PT_SIZE
> + uasm_i_addiu(&p, SP, SP, -(int)sizeof(struct pt_regs));
>
> /*
> * XXXKYMA do we need to load the host ASID, maybe not because the
> @@ -377,27 +449,54 @@ NESTED (MIPSX(GuestException), CALLFRAME_SIZ, ra)
> */
>
> /* Restore host DDATA_LO */
> - LONG_L k0, PT_HOST_USERLOCAL(sp)
> - mtc0 k0, CP0_DDATA_LO
> + UASM_i_LW(&p, K0, offsetof(struct pt_regs, cp0_epc), SP);
> + uasm_i_mtc0(&p, K0, C0_DDATA_LO);
>
> /* Restore RDHWR access */
> - INT_L k0, hwrena
> - mtc0 k0, CP0_HWRENA
> + UASM_i_LA_mostly(&p, K0, (long)&hwrena);
> + uasm_i_lw(&p, K0, uasm_rel_lo((long)&hwrena), K0);
> + uasm_i_mtc0(&p, K0, C0_HWRENA);
>
> /* Jump to handler */
> -FEXPORT(__kvm_mips_jump_to_handler)
> /*
> * XXXKYMA: not sure if this is safe, how large is the stack??
> * Now jump to the kvm_mips_handle_exit() to see if we can deal
> * with this in the kernel
> */
> - PTR_LA t9, kvm_mips_handle_exit
> - jalr.hb t9
> - INT_ADDIU sp, sp, -CALLFRAME_SIZ /* BD Slot */
> + UASM_i_LA(&p, T9, (unsigned long)kvm_mips_handle_exit);
> + uasm_i_jalr(&p, RA, T9);
> + uasm_i_addiu(&p, SP, SP, -CALLFRAME_SIZ);
> +
> + uasm_resolve_relocs(relocs, labels);
> +
> + p = kvm_mips_build_ret_from_exit(p);
> +
> + return p;
> +}
> +
> +/**
> + * kvm_mips_build_ret_from_exit() - Assemble guest exit return handler.
> + * @addr: Address to start writing code.
> + *
> + * Assemble the code to handle the return from kvm_mips_handle_exit(), either
> + * resuming the guest or returning to the host depending on the return value.
> + *
> + * Returns: Next address after end of written function.
> + */
> +static void *kvm_mips_build_ret_from_exit(void *addr)
> +{
> + u32 *p = addr;
> + struct uasm_label labels[2];
> + struct uasm_reloc relocs[2];
> + struct uasm_label *l = labels;
> + struct uasm_reloc *r = relocs;
> +
> + memset(labels, 0, sizeof(labels));
> + memset(relocs, 0, sizeof(relocs));
>
> /* Return from handler Make sure interrupts are disabled */
> - di
> - ehb
> + uasm_i_di(&p, ZERO);
> + uasm_i_ehb(&p);
>
> /*
> * XXXKYMA: k0/k1 could have been blown away if we processed
> @@ -405,198 +504,119 @@ FEXPORT(__kvm_mips_jump_to_handler)
> * guest, reload k1
> */
>
> - move k1, s1
> - INT_ADDIU k1, k1, VCPU_HOST_ARCH
> + uasm_i_move(&p, K1, S1);
> + uasm_i_addiu(&p, K1, K1, offsetof(struct kvm_vcpu, arch));
>
> /*
> * Check return value, should tell us if we are returning to the
> * host (handle I/O etc)or resuming the guest
> */
> - andi t0, v0, RESUME_HOST
> - bnez t0, __kvm_mips_return_to_host
> - nop
> + uasm_i_andi(&p, T0, V0, RESUME_HOST);
> + uasm_il_bnez(&p, &r, T0, label_return_to_host);
> + uasm_i_nop(&p);
> +
> + p = kvm_mips_build_ret_to_guest(p);
> +
> + uasm_l_return_to_host(&l, p);
> + p = kvm_mips_build_ret_to_host(p);
> +
> + uasm_resolve_relocs(relocs, labels);
> +
> + return p;
> +}
> +
> +/**
> + * kvm_mips_build_ret_to_guest() - Assemble code to return to the guest.
> + * @addr: Address to start writing code.
> + *
> + * Assemble the code to handle return from the guest exit handler
> + * (kvm_mips_handle_exit()) back to the guest.
> + *
> + * Returns: Next address after end of written function.
> + */
> +static void *kvm_mips_build_ret_to_guest(void *addr)
> +{
> + u32 *p = addr;
>
> -__kvm_mips_return_to_guest:
> /* Put the saved pointer to vcpu (s1) back into the DDATA_LO Register */
> - mtc0 s1, CP0_DDATA_LO
> + uasm_i_mtc0(&p, S1, C0_DDATA_LO);
>
> /* Load up the Guest EBASE to minimize the window where BEV is set */
> - LONG_L t0, VCPU_GUEST_EBASE(k1)
> + UASM_i_LW(&p, T0, offsetof(struct kvm_vcpu_arch, guest_ebase), K1);
>
> /* Switch EBASE back to the one used by KVM */
> - mfc0 v1, CP0_STATUS
> - or k0, v1, ST0_BEV
> - mtc0 k0, CP0_STATUS
> - ehb
> - mtc0 t0, CP0_EBASE
> + uasm_i_mfc0(&p, V1, C0_STATUS);
> + uasm_i_lui(&p, AT, ST0_BEV >> 16);
> + uasm_i_or(&p, K0, V1, AT);
> + uasm_i_mtc0(&p, K0, C0_STATUS);
> + uasm_i_ehb(&p);
> + uasm_i_mtc0(&p, T0, C0_EBASE);
>
> /* Setup status register for running guest in UM */
> - or v1, v1, (ST0_EXL | KSU_USER | ST0_IE)
> - and v1, v1, ~(ST0_CU0 | ST0_MX)
> - mtc0 v1, CP0_STATUS
> - ehb
> + uasm_i_ori(&p, V1, V1, ST0_EXL | KSU_USER | ST0_IE);
> + UASM_i_LA(&p, AT, ~(ST0_CU0 | ST0_MX));
> + uasm_i_and(&p, V1, V1, AT);
> + uasm_i_mtc0(&p, V1, C0_STATUS);
> + uasm_i_ehb(&p);
> +
> + p = kvm_mips_build_enter_guest(p);
> +
> + return p;
> +}
> +
> +/**
> + * kvm_mips_build_ret_to_host() - Assemble code to return to the host.
> + * @addr: Address to start writing code.
> + *
> + * Assemble the code to handle return from the guest exit handler
> + * (kvm_mips_handle_exit()) back to the host, i.e. to the caller of the vcpu_run
> + * function generated by kvm_mips_build_vcpu_run().
> + *
> + * Returns: Next address after end of written function.
> + */
> +static void *kvm_mips_build_ret_to_host(void *addr)
> +{
> + u32 *p = addr;
> + unsigned int i;
>
> - /* Set Guest EPC */
> - LONG_L t0, VCPU_PC(k1)
> - mtc0 t0, CP0_EPC
> -
> - /* Set the ASID for the Guest Kernel */
> - PTR_L t0, VCPU_COP0(k1)
> - LONG_L t0, COP0_STATUS(t0)
> - andi t0, KSU_USER | ST0_ERL | ST0_EXL
> - xori t0, KSU_USER
> - bnez t0, 1f /* If kernel */
> - INT_ADDIU t1, k1, VCPU_GUEST_KERNEL_ASID /* (BD) */
> - INT_ADDIU t1, k1, VCPU_GUEST_USER_ASID /* else user */
> -1:
> - /* t1: contains the base of the ASID array, need to get the cpu id */
> - LONG_L t2, TI_CPU($28) /* smp_processor_id */
> - INT_SLL t2, t2, 2 /* x4 */
> - REG_ADDU t3, t1, t2
> - LONG_L k0, (t3)
> -#ifdef CONFIG_MIPS_ASID_BITS_VARIABLE
> - li t3, CPUINFO_SIZE/4
> - mul t2, t2, t3 /* x sizeof(struct cpuinfo_mips)/4 */
> - LONG_L t2, (cpu_data + CPUINFO_ASID_MASK)(t2)
> - and k0, k0, t2
> -#else
> - andi k0, k0, MIPS_ENTRYHI_ASID
> -#endif
> - mtc0 k0, CP0_ENTRYHI
> - ehb
> -
> - /* Disable RDHWR access */
> - mtc0 zero, CP0_HWRENA
> -
> - .set noat
> - /* load the guest context from VCPU and return */
> - LONG_L $0, VCPU_R0(k1)
> - LONG_L $1, VCPU_R1(k1)
> - LONG_L $2, VCPU_R2(k1)
> - LONG_L $3, VCPU_R3(k1)
> - LONG_L $4, VCPU_R4(k1)
> - LONG_L $5, VCPU_R5(k1)
> - LONG_L $6, VCPU_R6(k1)
> - LONG_L $7, VCPU_R7(k1)
> - LONG_L $8, VCPU_R8(k1)
> - LONG_L $9, VCPU_R9(k1)
> - LONG_L $10, VCPU_R10(k1)
> - LONG_L $11, VCPU_R11(k1)
> - LONG_L $12, VCPU_R12(k1)
> - LONG_L $13, VCPU_R13(k1)
> - LONG_L $14, VCPU_R14(k1)
> - LONG_L $15, VCPU_R15(k1)
> - LONG_L $16, VCPU_R16(k1)
> - LONG_L $17, VCPU_R17(k1)
> - LONG_L $18, VCPU_R18(k1)
> - LONG_L $19, VCPU_R19(k1)
> - LONG_L $20, VCPU_R20(k1)
> - LONG_L $21, VCPU_R21(k1)
> - LONG_L $22, VCPU_R22(k1)
> - LONG_L $23, VCPU_R23(k1)
> - LONG_L $24, VCPU_R24(k1)
> - LONG_L $25, VCPU_R25(k1)
> -
> - /* $/k1 loaded later */
> - LONG_L $28, VCPU_R28(k1)
> - LONG_L $29, VCPU_R29(k1)
> - LONG_L $30, VCPU_R30(k1)
> - LONG_L $31, VCPU_R31(k1)
> -
> -FEXPORT(__kvm_mips_skip_guest_restore)
> - LONG_L k0, VCPU_HI(k1)
> - mthi k0
> -
> - LONG_L k0, VCPU_LO(k1)
> - mtlo k0
> -
> - LONG_L k0, VCPU_R26(k1)
> - LONG_L k1, VCPU_R27(k1)
> -
> - eret
> - .set at
> -
> -__kvm_mips_return_to_host:
> /* EBASE is already pointing to Linux */
> - LONG_L k1, VCPU_HOST_STACK(k1)
> - INT_ADDIU k1,k1, -PT_SIZE
> + UASM_i_LW(&p, K1, offsetof(struct kvm_vcpu_arch, host_stack), K1);
> + uasm_i_addiu(&p, K1, K1, -(int)sizeof(struct pt_regs));
>
> /* Restore host DDATA_LO */
> - LONG_L k0, PT_HOST_USERLOCAL(k1)
> - mtc0 k0, CP0_DDATA_LO
> + UASM_i_LW(&p, K0, offsetof(struct pt_regs, cp0_epc), K1);
> + uasm_i_mtc0(&p, K0, C0_DDATA_LO);
>
> /*
> * r2/v0 is the return code, shift it down by 2 (arithmetic)
> * to recover the err code
> */
> - INT_SRA k0, v0, 2
> - move $2, k0
> + uasm_i_sra(&p, K0, V0, 2);
> + uasm_i_move(&p, V0, K0);
>
> /* Load context saved on the host stack */
> - LONG_L $16, PT_R16(k1)
> - LONG_L $17, PT_R17(k1)
> - LONG_L $18, PT_R18(k1)
> - LONG_L $19, PT_R19(k1)
> - LONG_L $20, PT_R20(k1)
> - LONG_L $21, PT_R21(k1)
> - LONG_L $22, PT_R22(k1)
> - LONG_L $23, PT_R23(k1)
> + for (i = 16; i < 31; ++i) {
> + if (i == 24)
> + i = 28;
> + UASM_i_LW(&p, i, offsetof(struct pt_regs, regs[i]), K1);
> + }
>
> - LONG_L $28, PT_R28(k1)
> - LONG_L $29, PT_R29(k1)
> - LONG_L $30, PT_R30(k1)
> + UASM_i_LW(&p, K0, offsetof(struct pt_regs, hi), K1);
> + uasm_i_mthi(&p, K0);
>
> - LONG_L k0, PT_HI(k1)
> - mthi k0
> -
> - LONG_L k0, PT_LO(k1)
> - mtlo k0
> + UASM_i_LW(&p, K0, offsetof(struct pt_regs, lo), K1);
> + uasm_i_mtlo(&p, K0);
>
> /* Restore RDHWR access */
> - INT_L k0, hwrena
> - mtc0 k0, CP0_HWRENA
> + UASM_i_LA_mostly(&p, K0, (long)&hwrena);
> + uasm_i_lw(&p, K0, uasm_rel_lo((long)&hwrena), K0);
> + uasm_i_mtc0(&p, K0, C0_HWRENA);
>
> /* Restore RA, which is the address we will return to */
> - LONG_L ra, PT_R31(k1)
> - j ra
> - nop
> + UASM_i_LW(&p, RA, offsetof(struct pt_regs, regs[RA]), K1);
> + uasm_i_jr(&p, RA);
> + uasm_i_nop(&p);
>
> -VECTOR_END(MIPSX(GuestExceptionEnd))
> -.end MIPSX(GuestException)
> + return p;
> +}
>
> -MIPSX(exceptions):
> - ####
> - ##### The exception handlers.
> - #####
> - .word _C_LABEL(MIPSX(GuestException)) # 0
> - .word _C_LABEL(MIPSX(GuestException)) # 1
> - .word _C_LABEL(MIPSX(GuestException)) # 2
> - .word _C_LABEL(MIPSX(GuestException)) # 3
> - .word _C_LABEL(MIPSX(GuestException)) # 4
> - .word _C_LABEL(MIPSX(GuestException)) # 5
> - .word _C_LABEL(MIPSX(GuestException)) # 6
> - .word _C_LABEL(MIPSX(GuestException)) # 7
> - .word _C_LABEL(MIPSX(GuestException)) # 8
> - .word _C_LABEL(MIPSX(GuestException)) # 9
> - .word _C_LABEL(MIPSX(GuestException)) # 10
> - .word _C_LABEL(MIPSX(GuestException)) # 11
> - .word _C_LABEL(MIPSX(GuestException)) # 12
> - .word _C_LABEL(MIPSX(GuestException)) # 13
> - .word _C_LABEL(MIPSX(GuestException)) # 14
> - .word _C_LABEL(MIPSX(GuestException)) # 15
> - .word _C_LABEL(MIPSX(GuestException)) # 16
> - .word _C_LABEL(MIPSX(GuestException)) # 17
> - .word _C_LABEL(MIPSX(GuestException)) # 18
> - .word _C_LABEL(MIPSX(GuestException)) # 19
> - .word _C_LABEL(MIPSX(GuestException)) # 20
> - .word _C_LABEL(MIPSX(GuestException)) # 21
> - .word _C_LABEL(MIPSX(GuestException)) # 22
> - .word _C_LABEL(MIPSX(GuestException)) # 23
> - .word _C_LABEL(MIPSX(GuestException)) # 24
> - .word _C_LABEL(MIPSX(GuestException)) # 25
> - .word _C_LABEL(MIPSX(GuestException)) # 26
> - .word _C_LABEL(MIPSX(GuestException)) # 27
> - .word _C_LABEL(MIPSX(GuestException)) # 28
> - .word _C_LABEL(MIPSX(GuestException)) # 29
> - .word _C_LABEL(MIPSX(GuestException)) # 30
> - .word _C_LABEL(MIPSX(GuestException)) # 31
> diff --git a/arch/mips/kvm/interrupt.h b/arch/mips/kvm/interrupt.h
> index d661c100b219..fb118a2c8379 100644
> --- a/arch/mips/kvm/interrupt.h
> +++ b/arch/mips/kvm/interrupt.h
> @@ -28,10 +28,6 @@
> #define MIPS_EXC_MAX 12
> /* XXXSL More to follow */
>
> -extern char __kvm_mips_vcpu_run_end[];
> -extern char mips32_exception[], mips32_exceptionEnd[];
> -extern char mips32_GuestException[], mips32_GuestExceptionEnd[];
> -
> #define C_TI (_ULCAST_(1) << 30)
>
> #define KVM_MIPS_IRQ_DELIVER_ALL_AT_ONCE (0)
> diff --git a/arch/mips/kvm/mips.c b/arch/mips/kvm/mips.c
> index 5a2b9034a05c..f00cf5f9ac52 100644
> --- a/arch/mips/kvm/mips.c
> +++ b/arch/mips/kvm/mips.c
> @@ -247,8 +247,8 @@ void kvm_arch_commit_memory_region(struct kvm *kvm,
>
> struct kvm_vcpu *kvm_arch_vcpu_create(struct kvm *kvm, unsigned int id)
> {
> - int err, size, offset;
> - void *gebase;
> + int err, size;
> + void *gebase, *p;
> int i;
>
> struct kvm_vcpu *vcpu = kzalloc(sizeof(struct kvm_vcpu), GFP_KERNEL);
> @@ -286,41 +286,28 @@ struct kvm_vcpu *kvm_arch_vcpu_create(struct kvm *kvm, unsigned int id)
> /* Save new ebase */
> vcpu->arch.guest_ebase = gebase;
>
> - /* Copy L1 Guest Exception handler to correct offset */
> + /* Build guest exception vectors dynamically in unmapped memory */
>
> /* TLB Refill, EXL = 0 */
> - memcpy(gebase, mips32_exception,
> - mips32_exceptionEnd - mips32_exception);
> + kvm_mips_build_exception(gebase);
>
> /* General Exception Entry point */
> - memcpy(gebase + 0x180, mips32_exception,
> - mips32_exceptionEnd - mips32_exception);
> + kvm_mips_build_exception(gebase + 0x180);
>
> /* For vectored interrupts poke the exception code @ all offsets 0-7 */
> for (i = 0; i < 8; i++) {
> kvm_debug("L1 Vectored handler @ %p\n",
> gebase + 0x200 + (i * VECTORSPACING));
> - memcpy(gebase + 0x200 + (i * VECTORSPACING), mips32_exception,
> - mips32_exceptionEnd - mips32_exception);
> + kvm_mips_build_exception(gebase + 0x200 + i * VECTORSPACING);
> }
>
> - /* General handler, relocate to unmapped space for sanity's sake */
> - offset = 0x2000;
> - kvm_debug("Installing KVM Exception handlers @ %p, %#x bytes\n",
> - gebase + offset,
> - mips32_GuestExceptionEnd - mips32_GuestException);
> + /* General exit handler */
> + p = gebase + 0x2000;
> + p = kvm_mips_build_exit(p);
>
> - memcpy(gebase + offset, mips32_GuestException,
> - mips32_GuestExceptionEnd - mips32_GuestException);
> -
> -#ifdef MODULE
> - offset += mips32_GuestExceptionEnd - mips32_GuestException;
> - memcpy(gebase + offset, (char *)__kvm_mips_vcpu_run,
> - __kvm_mips_vcpu_run_end - (char *)__kvm_mips_vcpu_run);
> - vcpu->arch.vcpu_run = gebase + offset;
> -#else
> - vcpu->arch.vcpu_run = __kvm_mips_vcpu_run;
> -#endif
> + /* Guest entry routine */
> + vcpu->arch.vcpu_run = p;
> + p = kvm_mips_build_vcpu_run(p);
>
> /* Invalidate the icache for these ranges */
> local_flush_icache_range((unsigned long)gebase,
> --
> 2.4.10
>
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply [flat|nested] 36+ messages in thread* Re: [PATCH 06/14] MIPS; KVM: Convert exception entry to uasm
@ 2016-07-04 18:53 ` James Hogan
0 siblings, 0 replies; 36+ messages in thread
From: James Hogan @ 2016-07-04 18:53 UTC (permalink / raw)
To: Paolo Bonzini, Radim Krčmář, Ralf Baechle; +Cc: linux-mips, kvm
[-- Attachment #1: Type: text/plain, Size: 39246 bytes --]
Hi Paolo,
On Thu, Jun 23, 2016 at 05:34:39PM +0100, James Hogan wrote:
> Convert the whole of locore.S (assembly to enter guest and handle
> exception entry) to be generated dynamically with uasm. This is done
> with minimal changes to the resulting code.
Before I forget, please feel free to s/;/:/ in the subject line if/when
you apply (but I know I need some acks yet).
Thanks
James
>
> The main changes are:
> - Some constants are generated by uasm using LUI+ADDIU instead of
> LUI+ORI.
> - Loading of lo and hi are swapped around in vcpu_run but not when
> resuming the guest after an exit. Both bits of logic are now generated
> by the same code.
> - Register MOVEs in uasm use different ADDU operand ordering to GNU as,
> putting zero register into rs instead of rt.
> - The JALR.HB to call the C exit handler is switched to JALR, since the
> hazard barrier would appear to be unnecessary.
>
> This will allow further optimisation in the future to dynamically handle
> the capabilities of the CPU.
>
> Signed-off-by: James Hogan <james.hogan@imgtec.com>
> Cc: Paolo Bonzini <pbonzini@redhat.com>
> Cc: Radim Krčmář <rkrcmar@redhat.com>
> Cc: Ralf Baechle <ralf@linux-mips.org>
> Cc: linux-mips@linux-mips.org
> Cc: kvm@vger.kernel.org
> ---
> arch/mips/include/asm/kvm_host.h | 8 +-
> arch/mips/kvm/Kconfig | 1 +
> arch/mips/kvm/Makefile | 2 +-
> arch/mips/kvm/{locore.S => entry.c} | 884 ++++++++++++++++++------------------
> arch/mips/kvm/interrupt.h | 4 -
> arch/mips/kvm/mips.c | 37 +-
> 6 files changed, 472 insertions(+), 464 deletions(-)
> rename arch/mips/kvm/{locore.S => entry.c} (20%)
>
> diff --git a/arch/mips/include/asm/kvm_host.h b/arch/mips/include/asm/kvm_host.h
> index b0773c6d622f..2e76e899079c 100644
> --- a/arch/mips/include/asm/kvm_host.h
> +++ b/arch/mips/include/asm/kvm_host.h
> @@ -533,8 +533,12 @@ int kvm_mips_emulation_init(struct kvm_mips_callbacks **install_callbacks);
> /* Debug: dump vcpu state */
> int kvm_arch_vcpu_dump_regs(struct kvm_vcpu *vcpu);
>
> -/* Trampoline ASM routine to start running in "Guest" context */
> -extern int __kvm_mips_vcpu_run(struct kvm_run *run, struct kvm_vcpu *vcpu);
> +extern int kvm_mips_handle_exit(struct kvm_run *run, struct kvm_vcpu *vcpu);
> +
> +/* Building of entry/exception code */
> +void *kvm_mips_build_vcpu_run(void *addr);
> +void *kvm_mips_build_exception(void *addr);
> +void *kvm_mips_build_exit(void *addr);
>
> /* FPU/MSA context management */
> void __kvm_save_fpu(struct kvm_vcpu_arch *vcpu);
> diff --git a/arch/mips/kvm/Kconfig b/arch/mips/kvm/Kconfig
> index 2ae12825529f..7c56d6b124d1 100644
> --- a/arch/mips/kvm/Kconfig
> +++ b/arch/mips/kvm/Kconfig
> @@ -17,6 +17,7 @@ if VIRTUALIZATION
> config KVM
> tristate "Kernel-based Virtual Machine (KVM) support"
> depends on HAVE_KVM
> + select EXPORT_UASM
> select PREEMPT_NOTIFIERS
> select ANON_INODES
> select KVM_MMIO
> diff --git a/arch/mips/kvm/Makefile b/arch/mips/kvm/Makefile
> index 0aabe40fcac9..847429de780d 100644
> --- a/arch/mips/kvm/Makefile
> +++ b/arch/mips/kvm/Makefile
> @@ -7,7 +7,7 @@ EXTRA_CFLAGS += -Ivirt/kvm -Iarch/mips/kvm
>
> common-objs-$(CONFIG_CPU_HAS_MSA) += msa.o
>
> -kvm-objs := $(common-objs-y) mips.o emulate.o locore.o \
> +kvm-objs := $(common-objs-y) mips.o emulate.o entry.o \
> interrupt.o stats.o commpage.o \
> dyntrans.o trap_emul.o fpu.o
> kvm-objs += mmu.o
> diff --git a/arch/mips/kvm/locore.S b/arch/mips/kvm/entry.c
> similarity index 20%
> rename from arch/mips/kvm/locore.S
> rename to arch/mips/kvm/entry.c
> index 698286c0f732..9a18b4939b35 100644
> --- a/arch/mips/kvm/locore.S
> +++ b/arch/mips/kvm/entry.c
> @@ -3,373 +3,445 @@
> * License. See the file "COPYING" in the main directory of this archive
> * for more details.
> *
> - * Main entry point for the guest, exception handling.
> + * Generation of main entry point for the guest, exception handling.
> *
> - * Copyright (C) 2012 MIPS Technologies, Inc. All rights reserved.
> + * Copyright (C) 2012 MIPS Technologies, Inc.
> * Authors: Sanjay Lal <sanjayl@kymasys.com>
> + *
> + * Copyright (C) 2016 Imagination Technologies Ltd.
> */
>
> -#include <asm/asm.h>
> -#include <asm/asmmacro.h>
> -#include <asm/regdef.h>
> -#include <asm/mipsregs.h>
> -#include <asm/stackframe.h>
> -#include <asm/asm-offsets.h>
> -
> -#define _C_LABEL(x) x
> -#define MIPSX(name) mips32_ ## name
> +#include <linux/kvm_host.h>
> +#include <asm/msa.h>
> +#include <asm/setup.h>
> +#include <asm/uasm.h>
> +
> +/* Register names */
> +#define ZERO 0
> +#define AT 1
> +#define V0 2
> +#define V1 3
> +#define A0 4
> +#define A1 5
> +
> +#if _MIPS_SIM == _MIPS_SIM_ABI32
> +#define T0 8
> +#define T1 9
> +#define T2 10
> +#define T3 11
> +#endif /* _MIPS_SIM == _MIPS_SIM_ABI32 */
> +
> +#if _MIPS_SIM == _MIPS_SIM_ABI64 || _MIPS_SIM == _MIPS_SIM_NABI32
> +#define T0 12
> +#define T1 13
> +#define T2 14
> +#define T3 15
> +#endif /* _MIPS_SIM == _MIPS_SIM_ABI64 || _MIPS_SIM == _MIPS_SIM_NABI32 */
> +
> +#define S0 16
> +#define S1 17
> +#define T9 25
> +#define K0 26
> +#define K1 27
> +#define GP 28
> +#define SP 29
> +#define RA 31
> +
> +/* Some CP0 registers */
> +#define C0_HWRENA 7, 0
> +#define C0_BADVADDR 8, 0
> +#define C0_ENTRYHI 10, 0
> +#define C0_STATUS 12, 0
> +#define C0_CAUSE 13, 0
> +#define C0_EPC 14, 0
> +#define C0_EBASE 15, 1
> +#define C0_CONFIG3 16, 3
> +#define C0_CONFIG5 16, 5
> +#define C0_DDATA_LO 28, 3
> +#define C0_ERROREPC 30, 0
> +
> #define CALLFRAME_SIZ 32
>
> -/*
> - * VECTOR
> - * exception vector entrypoint
> - */
> -#define VECTOR(x, regmask) \
> - .ent _C_LABEL(x),0; \
> - EXPORT(x);
> -
> -#define VECTOR_END(x) \
> - EXPORT(x);
> -
> -/* Overload, Danger Will Robinson!! */
> -#define PT_HOST_USERLOCAL PT_EPC
> +enum label_id {
> + label_fpu_1 = 1,
> + label_msa_1,
> + label_return_to_host,
> + label_kernel_asid,
> +};
>
> -#define CP0_DDATA_LO $28,3
> +UASM_L_LA(_fpu_1)
> +UASM_L_LA(_msa_1)
> +UASM_L_LA(_return_to_host)
> +UASM_L_LA(_kernel_asid)
>
> -/* Resume Flags */
> -#define RESUME_FLAG_HOST (1<<1) /* Resume host? */
> +static void *kvm_mips_build_enter_guest(void *addr);
> +static void *kvm_mips_build_ret_from_exit(void *addr);
> +static void *kvm_mips_build_ret_to_guest(void *addr);
> +static void *kvm_mips_build_ret_to_host(void *addr);
>
> -#define RESUME_GUEST 0
> -#define RESUME_HOST RESUME_FLAG_HOST
> -
> -/*
> - * __kvm_mips_vcpu_run: entry point to the guest
> - * a0: run
> - * a1: vcpu
> +/**
> + * kvm_mips_build_vcpu_run() - Assemble function to start running a guest VCPU.
> + * @addr: Address to start writing code.
> + *
> + * Assemble the start of the vcpu_run function to run a guest VCPU. The function
> + * conforms to the following prototype:
> + *
> + * int vcpu_run(struct kvm_run *run, struct kvm_vcpu *vcpu);
> + *
> + * The exit from the guest and return to the caller is handled by the code
> + * generated by kvm_mips_build_ret_to_host().
> + *
> + * Returns: Next address after end of written function.
> */
> - .set noreorder
> +void *kvm_mips_build_vcpu_run(void *addr)
> +{
> + u32 *p = addr;
> + unsigned int i;
> +
> + /*
> + * A0: run
> + * A1: vcpu
> + */
>
> -FEXPORT(__kvm_mips_vcpu_run)
> /* k0/k1 not being used in host kernel context */
> - INT_ADDIU k1, sp, -PT_SIZE
> - LONG_S $16, PT_R16(k1)
> - LONG_S $17, PT_R17(k1)
> - LONG_S $18, PT_R18(k1)
> - LONG_S $19, PT_R19(k1)
> - LONG_S $20, PT_R20(k1)
> - LONG_S $21, PT_R21(k1)
> - LONG_S $22, PT_R22(k1)
> - LONG_S $23, PT_R23(k1)
> -
> - LONG_S $28, PT_R28(k1)
> - LONG_S $29, PT_R29(k1)
> - LONG_S $30, PT_R30(k1)
> - LONG_S $31, PT_R31(k1)
> + uasm_i_addiu(&p, K1, SP, -(int)sizeof(struct pt_regs));
> + for (i = 16; i < 32; ++i) {
> + if (i == 24)
> + i = 28;
> + UASM_i_SW(&p, i, offsetof(struct pt_regs, regs[i]), K1);
> + }
>
> /* Save hi/lo */
> - mflo v0
> - LONG_S v0, PT_LO(k1)
> - mfhi v1
> - LONG_S v1, PT_HI(k1)
> + uasm_i_mflo(&p, V0);
> + UASM_i_SW(&p, V0, offsetof(struct pt_regs, lo), K1);
> + uasm_i_mfhi(&p, V1);
> + UASM_i_SW(&p, V1, offsetof(struct pt_regs, hi), K1);
>
> /* Save host status */
> - mfc0 v0, CP0_STATUS
> - LONG_S v0, PT_STATUS(k1)
> + uasm_i_mfc0(&p, V0, C0_STATUS);
> + UASM_i_SW(&p, V0, offsetof(struct pt_regs, cp0_status), K1);
>
> /* Save DDATA_LO, will be used to store pointer to vcpu */
> - mfc0 v1, CP0_DDATA_LO
> - LONG_S v1, PT_HOST_USERLOCAL(k1)
> + uasm_i_mfc0(&p, V1, C0_DDATA_LO);
> + UASM_i_SW(&p, V1, offsetof(struct pt_regs, cp0_epc), K1);
>
> /* DDATA_LO has pointer to vcpu */
> - mtc0 a1, CP0_DDATA_LO
> + uasm_i_mtc0(&p, A1, C0_DDATA_LO);
>
> /* Offset into vcpu->arch */
> - INT_ADDIU k1, a1, VCPU_HOST_ARCH
> + uasm_i_addiu(&p, K1, A1, offsetof(struct kvm_vcpu, arch));
>
> /*
> * Save the host stack to VCPU, used for exception processing
> * when we exit from the Guest
> */
> - LONG_S sp, VCPU_HOST_STACK(k1)
> + UASM_i_SW(&p, SP, offsetof(struct kvm_vcpu_arch, host_stack), K1);
>
> /* Save the kernel gp as well */
> - LONG_S gp, VCPU_HOST_GP(k1)
> + UASM_i_SW(&p, GP, offsetof(struct kvm_vcpu_arch, host_gp), K1);
>
> /*
> * Setup status register for running the guest in UM, interrupts
> * are disabled
> */
> - li k0, (ST0_EXL | KSU_USER | ST0_BEV)
> - mtc0 k0, CP0_STATUS
> - ehb
> + UASM_i_LA(&p, K0, ST0_EXL | KSU_USER | ST0_BEV);
> + uasm_i_mtc0(&p, K0, C0_STATUS);
> + uasm_i_ehb(&p);
>
> /* load up the new EBASE */
> - LONG_L k0, VCPU_GUEST_EBASE(k1)
> - mtc0 k0, CP0_EBASE
> + UASM_i_LW(&p, K0, offsetof(struct kvm_vcpu_arch, guest_ebase), K1);
> + uasm_i_mtc0(&p, K0, C0_EBASE);
>
> /*
> * Now that the new EBASE has been loaded, unset BEV, set
> * interrupt mask as it was but make sure that timer interrupts
> * are enabled
> */
> - li k0, (ST0_EXL | KSU_USER | ST0_IE)
> - andi v0, v0, ST0_IM
> - or k0, k0, v0
> - mtc0 k0, CP0_STATUS
> - ehb
> + uasm_i_addiu(&p, K0, ZERO, ST0_EXL | KSU_USER | ST0_IE);
> + uasm_i_andi(&p, V0, V0, ST0_IM);
> + uasm_i_or(&p, K0, K0, V0);
> + uasm_i_mtc0(&p, K0, C0_STATUS);
> + uasm_i_ehb(&p);
> +
> + p = kvm_mips_build_enter_guest(p);
> +
> + return p;
> +}
> +
> +/**
> + * kvm_mips_build_enter_guest() - Assemble code to resume guest execution.
> + * @addr: Address to start writing code.
> + *
> + * Assemble the code to resume guest execution. This code is common between the
> + * initial entry into the guest from the host, and returning from the exit
> + * handler back to the guest.
> + *
> + * Returns: Next address after end of written function.
> + */
> +static void *kvm_mips_build_enter_guest(void *addr)
> +{
> + u32 *p = addr;
> + unsigned int i;
> + struct uasm_label labels[2];
> + struct uasm_reloc relocs[2];
> + struct uasm_label *l = labels;
> + struct uasm_reloc *r = relocs;
> +
> + memset(labels, 0, sizeof(labels));
> + memset(relocs, 0, sizeof(relocs));
>
> /* Set Guest EPC */
> - LONG_L t0, VCPU_PC(k1)
> - mtc0 t0, CP0_EPC
> + UASM_i_LW(&p, T0, offsetof(struct kvm_vcpu_arch, pc), K1);
> + uasm_i_mtc0(&p, T0, C0_EPC);
>
> -FEXPORT(__kvm_mips_load_asid)
> /* Set the ASID for the Guest Kernel */
> - PTR_L t0, VCPU_COP0(k1)
> - LONG_L t0, COP0_STATUS(t0)
> - andi t0, KSU_USER | ST0_ERL | ST0_EXL
> - xori t0, KSU_USER
> - bnez t0, 1f /* If kernel */
> - INT_ADDIU t1, k1, VCPU_GUEST_KERNEL_ASID /* (BD) */
> - INT_ADDIU t1, k1, VCPU_GUEST_USER_ASID /* else user */
> -1:
> - /* t1: contains the base of the ASID array, need to get the cpu id */
> - LONG_L t2, TI_CPU($28) /* smp_processor_id */
> - INT_SLL t2, t2, 2 /* x4 */
> - REG_ADDU t3, t1, t2
> - LONG_L k0, (t3)
> + UASM_i_LW(&p, T0, offsetof(struct kvm_vcpu_arch, cop0), K1);
> + UASM_i_LW(&p, T0, offsetof(struct mips_coproc, reg[MIPS_CP0_STATUS][0]),
> + T0);
> + uasm_i_andi(&p, T0, T0, KSU_USER | ST0_ERL | ST0_EXL);
> + uasm_i_xori(&p, T0, T0, KSU_USER);
> + uasm_il_bnez(&p, &r, T0, label_kernel_asid);
> + uasm_i_addiu(&p, T1, K1,
> + offsetof(struct kvm_vcpu_arch, guest_kernel_asid));
> + /* else user */
> + uasm_i_addiu(&p, T1, K1,
> + offsetof(struct kvm_vcpu_arch, guest_user_asid));
> + uasm_l_kernel_asid(&l, p);
> +
> + /* t1: contains the base of the ASID array, need to get the cpu id */
> + /* smp_processor_id */
> + UASM_i_LW(&p, T2, offsetof(struct thread_info, cpu), GP);
> + /* x4 */
> + uasm_i_sll(&p, T2, T2, 2);
> + UASM_i_ADDU(&p, T3, T1, T2);
> + UASM_i_LW(&p, K0, 0, T3);
> #ifdef CONFIG_MIPS_ASID_BITS_VARIABLE
> - li t3, CPUINFO_SIZE/4
> - mul t2, t2, t3 /* x sizeof(struct cpuinfo_mips)/4 */
> - LONG_L t2, (cpu_data + CPUINFO_ASID_MASK)(t2)
> - and k0, k0, t2
> + /* x sizeof(struct cpuinfo_mips)/4 */
> + uasm_i_addiu(&p, T3, ZERO, sizeof(struct cpuinfo_mips)/4);
> + uasm_i_mul(&p, T2, T2, T3);
> +
> + UASM_i_LA_mostly(&p, AT, (long)&cpu_data[0].asid_mask);
> + UASM_i_ADDU(&p, AT, AT, T2);
> + UASM_i_LW(&p, T2, uasm_rel_lo((long)&cpu_data[0].asid_mask), AT);
> + uasm_i_and(&p, K0, K0, T2);
> #else
> - andi k0, k0, MIPS_ENTRYHI_ASID
> + uasm_i_andi(&p, K0, K0, MIPS_ENTRYHI_ASID);
> #endif
> - mtc0 k0, CP0_ENTRYHI
> - ehb
> + uasm_i_mtc0(&p, K0, C0_ENTRYHI);
> + uasm_i_ehb(&p);
>
> /* Disable RDHWR access */
> - mtc0 zero, CP0_HWRENA
> -
> - .set noat
> - /* Now load up the Guest Context from VCPU */
> - LONG_L $1, VCPU_R1(k1)
> - LONG_L $2, VCPU_R2(k1)
> - LONG_L $3, VCPU_R3(k1)
> -
> - LONG_L $4, VCPU_R4(k1)
> - LONG_L $5, VCPU_R5(k1)
> - LONG_L $6, VCPU_R6(k1)
> - LONG_L $7, VCPU_R7(k1)
> -
> - LONG_L $8, VCPU_R8(k1)
> - LONG_L $9, VCPU_R9(k1)
> - LONG_L $10, VCPU_R10(k1)
> - LONG_L $11, VCPU_R11(k1)
> - LONG_L $12, VCPU_R12(k1)
> - LONG_L $13, VCPU_R13(k1)
> - LONG_L $14, VCPU_R14(k1)
> - LONG_L $15, VCPU_R15(k1)
> - LONG_L $16, VCPU_R16(k1)
> - LONG_L $17, VCPU_R17(k1)
> - LONG_L $18, VCPU_R18(k1)
> - LONG_L $19, VCPU_R19(k1)
> - LONG_L $20, VCPU_R20(k1)
> - LONG_L $21, VCPU_R21(k1)
> - LONG_L $22, VCPU_R22(k1)
> - LONG_L $23, VCPU_R23(k1)
> - LONG_L $24, VCPU_R24(k1)
> - LONG_L $25, VCPU_R25(k1)
> -
> - /* k0/k1 loaded up later */
> -
> - LONG_L $28, VCPU_R28(k1)
> - LONG_L $29, VCPU_R29(k1)
> - LONG_L $30, VCPU_R30(k1)
> - LONG_L $31, VCPU_R31(k1)
> + uasm_i_mtc0(&p, ZERO, C0_HWRENA);
> +
> + /* load the guest context from VCPU and return */
> + for (i = 1; i < 32; ++i) {
> + /* Guest k0/k1 loaded later */
> + if (i == K0 || i == K1)
> + continue;
> + UASM_i_LW(&p, i, offsetof(struct kvm_vcpu_arch, gprs[i]), K1);
> + }
>
> /* Restore hi/lo */
> - LONG_L k0, VCPU_LO(k1)
> - mtlo k0
> + UASM_i_LW(&p, K0, offsetof(struct kvm_vcpu_arch, hi), K1);
> + uasm_i_mthi(&p, K0);
>
> - LONG_L k0, VCPU_HI(k1)
> - mthi k0
> + UASM_i_LW(&p, K0, offsetof(struct kvm_vcpu_arch, lo), K1);
> + uasm_i_mtlo(&p, K0);
>
> -FEXPORT(__kvm_mips_load_k0k1)
> /* Restore the guest's k0/k1 registers */
> - LONG_L k0, VCPU_R26(k1)
> - LONG_L k1, VCPU_R27(k1)
> + UASM_i_LW(&p, K0, offsetof(struct kvm_vcpu_arch, gprs[K0]), K1);
> + UASM_i_LW(&p, K1, offsetof(struct kvm_vcpu_arch, gprs[K1]), K1);
>
> /* Jump to guest */
> - eret
> -EXPORT(__kvm_mips_vcpu_run_end)
> -
> -VECTOR(MIPSX(exception), unknown)
> -/* Find out what mode we came from and jump to the proper handler. */
> - mtc0 k0, CP0_ERROREPC #01: Save guest k0
> - ehb #02:
> -
> - mfc0 k0, CP0_EBASE #02: Get EBASE
> - INT_SRL k0, k0, 10 #03: Get rid of CPUNum
> - INT_SLL k0, k0, 10 #04
> - LONG_S k1, 0x3000(k0) #05: Save k1 @ offset 0x3000
> - INT_ADDIU k0, k0, 0x2000 #06: Exception handler is
> - # installed @ offset 0x2000
> - j k0 #07: jump to the function
> - nop #08: branch delay slot
> -VECTOR_END(MIPSX(exceptionEnd))
> -.end MIPSX(exception)
> -
> -/*
> - * Generic Guest exception handler. We end up here when the guest
> - * does something that causes a trap to kernel mode.
> + uasm_i_eret(&p);
> +
> + uasm_resolve_relocs(relocs, labels);
> +
> + return p;
> +}
> +
> +/**
> + * kvm_mips_build_exception() - Assemble first level guest exception handler.
> + * @addr: Address to start writing code.
> + *
> + * Assemble exception vector code for guest execution. The generated vector will
> + * jump to the common exception handler generated by kvm_mips_build_exit().
> + *
> + * Returns: Next address after end of written function.
> + */
> +void *kvm_mips_build_exception(void *addr)
> +{
> + u32 *p = addr;
> +
> + /* Save guest k0 */
> + uasm_i_mtc0(&p, K0, C0_ERROREPC);
> + uasm_i_ehb(&p);
> +
> + /* Get EBASE */
> + uasm_i_mfc0(&p, K0, C0_EBASE);
> + /* Get rid of CPUNum */
> + uasm_i_srl(&p, K0, K0, 10);
> + uasm_i_sll(&p, K0, K0, 10);
> + /* Save k1 @ offset 0x3000 */
> + UASM_i_SW(&p, K1, 0x3000, K0);
> +
> + /* Exception handler is installed @ offset 0x2000 */
> + uasm_i_addiu(&p, K0, K0, 0x2000);
> + /* Jump to the function */
> + uasm_i_jr(&p, K0);
> + uasm_i_nop(&p);
> +
> + return p;
> +}
> +
> +/**
> + * kvm_mips_build_exit() - Assemble common guest exit handler.
> + * @addr: Address to start writing code.
> + *
> + * Assemble the generic guest exit handling code. This is called by the
> + * exception vectors (generated by kvm_mips_build_exception()), and calls
> + * kvm_mips_handle_exit(), then either resumes the guest or returns to the host
> + * depending on the return value.
> + *
> + * Returns: Next address after end of written function.
> */
> -NESTED (MIPSX(GuestException), CALLFRAME_SIZ, ra)
> - /* Get the VCPU pointer from DDTATA_LO */
> - mfc0 k1, CP0_DDATA_LO
> - INT_ADDIU k1, k1, VCPU_HOST_ARCH
> +void *kvm_mips_build_exit(void *addr)
> +{
> + u32 *p = addr;
> + unsigned int i;
> + struct uasm_label labels[3];
> + struct uasm_reloc relocs[3];
> + struct uasm_label *l = labels;
> + struct uasm_reloc *r = relocs;
> +
> + memset(labels, 0, sizeof(labels));
> + memset(relocs, 0, sizeof(relocs));
> +
> + /*
> + * Generic Guest exception handler. We end up here when the guest
> + * does something that causes a trap to kernel mode.
> + */
> +
> + /* Get the VCPU pointer from DDATA_LO */
> + uasm_i_mfc0(&p, K1, C0_DDATA_LO);
> + uasm_i_addiu(&p, K1, K1, offsetof(struct kvm_vcpu, arch));
>
> /* Start saving Guest context to VCPU */
> - LONG_S $0, VCPU_R0(k1)
> - LONG_S $1, VCPU_R1(k1)
> - LONG_S $2, VCPU_R2(k1)
> - LONG_S $3, VCPU_R3(k1)
> - LONG_S $4, VCPU_R4(k1)
> - LONG_S $5, VCPU_R5(k1)
> - LONG_S $6, VCPU_R6(k1)
> - LONG_S $7, VCPU_R7(k1)
> - LONG_S $8, VCPU_R8(k1)
> - LONG_S $9, VCPU_R9(k1)
> - LONG_S $10, VCPU_R10(k1)
> - LONG_S $11, VCPU_R11(k1)
> - LONG_S $12, VCPU_R12(k1)
> - LONG_S $13, VCPU_R13(k1)
> - LONG_S $14, VCPU_R14(k1)
> - LONG_S $15, VCPU_R15(k1)
> - LONG_S $16, VCPU_R16(k1)
> - LONG_S $17, VCPU_R17(k1)
> - LONG_S $18, VCPU_R18(k1)
> - LONG_S $19, VCPU_R19(k1)
> - LONG_S $20, VCPU_R20(k1)
> - LONG_S $21, VCPU_R21(k1)
> - LONG_S $22, VCPU_R22(k1)
> - LONG_S $23, VCPU_R23(k1)
> - LONG_S $24, VCPU_R24(k1)
> - LONG_S $25, VCPU_R25(k1)
> -
> - /* Guest k0/k1 saved later */
> -
> - LONG_S $28, VCPU_R28(k1)
> - LONG_S $29, VCPU_R29(k1)
> - LONG_S $30, VCPU_R30(k1)
> - LONG_S $31, VCPU_R31(k1)
> -
> - .set at
> + for (i = 0; i < 32; ++i) {
> + /* Guest k0/k1 saved later */
> + if (i == K0 || i == K1)
> + continue;
> + UASM_i_SW(&p, i, offsetof(struct kvm_vcpu_arch, gprs[i]), K1);
> + }
>
> /* We need to save hi/lo and restore them on the way out */
> - mfhi t0
> - LONG_S t0, VCPU_HI(k1)
> + uasm_i_mfhi(&p, T0);
> + UASM_i_SW(&p, T0, offsetof(struct kvm_vcpu_arch, hi), K1);
>
> - mflo t0
> - LONG_S t0, VCPU_LO(k1)
> + uasm_i_mflo(&p, T0);
> + UASM_i_SW(&p, T0, offsetof(struct kvm_vcpu_arch, lo), K1);
>
> /* Finally save guest k0/k1 to VCPU */
> - mfc0 t0, CP0_ERROREPC
> - LONG_S t0, VCPU_R26(k1)
> + uasm_i_mfc0(&p, T0, C0_ERROREPC);
> + UASM_i_SW(&p, T0, offsetof(struct kvm_vcpu_arch, gprs[K0]), K1);
>
> /* Get GUEST k1 and save it in VCPU */
> - PTR_LI t1, ~0x2ff
> - mfc0 t0, CP0_EBASE
> - and t0, t0, t1
> - LONG_L t0, 0x3000(t0)
> - LONG_S t0, VCPU_R27(k1)
> + uasm_i_addiu(&p, T1, ZERO, ~0x2ff);
> + uasm_i_mfc0(&p, T0, C0_EBASE);
> + uasm_i_and(&p, T0, T0, T1);
> + UASM_i_LW(&p, T0, 0x3000, T0);
> + UASM_i_SW(&p, T0, offsetof(struct kvm_vcpu_arch, gprs[K1]), K1);
>
> /* Now that context has been saved, we can use other registers */
>
> /* Restore vcpu */
> - mfc0 a1, CP0_DDATA_LO
> - move s1, a1
> + uasm_i_mfc0(&p, A1, C0_DDATA_LO);
> + uasm_i_move(&p, S1, A1);
>
> /* Restore run (vcpu->run) */
> - LONG_L a0, VCPU_RUN(a1)
> + UASM_i_LW(&p, A0, offsetof(struct kvm_vcpu, run), A1);
> /* Save pointer to run in s0, will be saved by the compiler */
> - move s0, a0
> + uasm_i_move(&p, S0, A0);
>
> /*
> - * Save Host level EPC, BadVaddr and Cause to VCPU, useful to
> - * process the exception
> + * Save Host level EPC, BadVaddr and Cause to VCPU, useful to process
> + * the exception
> */
> - mfc0 k0,CP0_EPC
> - LONG_S k0, VCPU_PC(k1)
> + uasm_i_mfc0(&p, K0, C0_EPC);
> + UASM_i_SW(&p, K0, offsetof(struct kvm_vcpu_arch, pc), K1);
>
> - mfc0 k0, CP0_BADVADDR
> - LONG_S k0, VCPU_HOST_CP0_BADVADDR(k1)
> + uasm_i_mfc0(&p, K0, C0_BADVADDR);
> + UASM_i_SW(&p, K0, offsetof(struct kvm_vcpu_arch, host_cp0_badvaddr),
> + K1);
>
> - mfc0 k0, CP0_CAUSE
> - sw k0, VCPU_HOST_CP0_CAUSE(k1)
> + uasm_i_mfc0(&p, K0, C0_CAUSE);
> + uasm_i_sw(&p, K0, offsetof(struct kvm_vcpu_arch, host_cp0_cause), K1);
>
> /* Now restore the host state just enough to run the handlers */
>
> /* Switch EBASE to the one used by Linux */
> /* load up the host EBASE */
> - mfc0 v0, CP0_STATUS
> + uasm_i_mfc0(&p, V0, C0_STATUS);
>
> - or k0, v0, ST0_BEV
> + uasm_i_lui(&p, AT, ST0_BEV >> 16);
> + uasm_i_or(&p, K0, V0, AT);
>
> - mtc0 k0, CP0_STATUS
> - ehb
> + uasm_i_mtc0(&p, K0, C0_STATUS);
> + uasm_i_ehb(&p);
>
> - LONG_L k0, ebase
> - mtc0 k0,CP0_EBASE
> + UASM_i_LA_mostly(&p, K0, (long)&ebase);
> + UASM_i_LW(&p, K0, uasm_rel_lo((long)&ebase), K0);
> + uasm_i_mtc0(&p, K0, C0_EBASE);
>
> /*
> * If FPU is enabled, save FCR31 and clear it so that later ctc1's don't
> * trigger FPE for pending exceptions.
> */
> - and v1, v0, ST0_CU1
> - beqz v1, 1f
> - nop
> - .set push
> - SET_HARDFLOAT
> - cfc1 t0, fcr31
> - sw t0, VCPU_FCR31(k1)
> - ctc1 zero,fcr31
> - .set pop
> -1:
> + uasm_i_lui(&p, AT, ST0_CU1 >> 16);
> + uasm_i_and(&p, V1, V0, AT);
> + uasm_il_beqz(&p, &r, V1, label_fpu_1);
> + uasm_i_nop(&p);
> + uasm_i_cfc1(&p, T0, 31);
> + uasm_i_sw(&p, T0, offsetof(struct kvm_vcpu_arch, fpu.fcr31), K1);
> + uasm_i_ctc1(&p, ZERO, 31);
> + uasm_l_fpu_1(&l, p);
>
> #ifdef CONFIG_CPU_HAS_MSA
> /*
> * If MSA is enabled, save MSACSR and clear it so that later
> * instructions don't trigger MSAFPE for pending exceptions.
> */
> - mfc0 t0, CP0_CONFIG3
> - ext t0, t0, 28, 1 /* MIPS_CONF3_MSAP */
> - beqz t0, 1f
> - nop
> - mfc0 t0, CP0_CONFIG5
> - ext t0, t0, 27, 1 /* MIPS_CONF5_MSAEN */
> - beqz t0, 1f
> - nop
> - _cfcmsa t0, MSA_CSR
> - sw t0, VCPU_MSA_CSR(k1)
> - _ctcmsa MSA_CSR, zero
> -1:
> + uasm_i_mfc0(&p, T0, C0_CONFIG3);
> + uasm_i_ext(&p, T0, T0, 28, 1); /* MIPS_CONF3_MSAP */
> + uasm_il_beqz(&p, &r, T0, label_msa_1);
> + uasm_i_nop(&p);
> + uasm_i_mfc0(&p, T0, C0_CONFIG5);
> + uasm_i_ext(&p, T0, T0, 27, 1); /* MIPS_CONF5_MSAEN */
> + uasm_il_beqz(&p, &r, T0, label_msa_1);
> + uasm_i_nop(&p);
> + uasm_i_cfcmsa(&p, T0, MSA_CSR);
> + uasm_i_sw(&p, T0, offsetof(struct kvm_vcpu_arch, fpu.msacsr),
> + K1);
> + uasm_i_ctcmsa(&p, MSA_CSR, ZERO);
> + uasm_l_msa_1(&l, p);
> #endif
>
> /* Now that the new EBASE has been loaded, unset BEV and KSU_USER */
> - and v0, v0, ~(ST0_EXL | KSU_USER | ST0_IE)
> - or v0, v0, ST0_CU0
> - mtc0 v0, CP0_STATUS
> - ehb
> + uasm_i_addiu(&p, AT, ZERO, ~(ST0_EXL | KSU_USER | ST0_IE));
> + uasm_i_and(&p, V0, V0, AT);
> + uasm_i_lui(&p, AT, ST0_CU0 >> 16);
> + uasm_i_or(&p, V0, V0, AT);
> + uasm_i_mtc0(&p, V0, C0_STATUS);
> + uasm_i_ehb(&p);
>
> /* Load up host GP */
> - LONG_L gp, VCPU_HOST_GP(k1)
> + UASM_i_LW(&p, GP, offsetof(struct kvm_vcpu_arch, host_gp), K1);
>
> /* Need a stack before we can jump to "C" */
> - LONG_L sp, VCPU_HOST_STACK(k1)
> + UASM_i_LW(&p, SP, offsetof(struct kvm_vcpu_arch, host_stack), K1);
>
> /* Saved host state */
> - INT_ADDIU sp, sp, -PT_SIZE
> + uasm_i_addiu(&p, SP, SP, -(int)sizeof(struct pt_regs));
>
> /*
> * XXXKYMA do we need to load the host ASID, maybe not because the
> @@ -377,27 +449,54 @@ NESTED (MIPSX(GuestException), CALLFRAME_SIZ, ra)
> */
>
> /* Restore host DDATA_LO */
> - LONG_L k0, PT_HOST_USERLOCAL(sp)
> - mtc0 k0, CP0_DDATA_LO
> + UASM_i_LW(&p, K0, offsetof(struct pt_regs, cp0_epc), SP);
> + uasm_i_mtc0(&p, K0, C0_DDATA_LO);
>
> /* Restore RDHWR access */
> - INT_L k0, hwrena
> - mtc0 k0, CP0_HWRENA
> + UASM_i_LA_mostly(&p, K0, (long)&hwrena);
> + uasm_i_lw(&p, K0, uasm_rel_lo((long)&hwrena), K0);
> + uasm_i_mtc0(&p, K0, C0_HWRENA);
>
> /* Jump to handler */
> -FEXPORT(__kvm_mips_jump_to_handler)
> /*
> * XXXKYMA: not sure if this is safe, how large is the stack??
> * Now jump to the kvm_mips_handle_exit() to see if we can deal
> * with this in the kernel
> */
> - PTR_LA t9, kvm_mips_handle_exit
> - jalr.hb t9
> - INT_ADDIU sp, sp, -CALLFRAME_SIZ /* BD Slot */
> + UASM_i_LA(&p, T9, (unsigned long)kvm_mips_handle_exit);
> + uasm_i_jalr(&p, RA, T9);
> + uasm_i_addiu(&p, SP, SP, -CALLFRAME_SIZ);
> +
> + uasm_resolve_relocs(relocs, labels);
> +
> + p = kvm_mips_build_ret_from_exit(p);
> +
> + return p;
> +}
> +
> +/**
> + * kvm_mips_build_ret_from_exit() - Assemble guest exit return handler.
> + * @addr: Address to start writing code.
> + *
> + * Assemble the code to handle the return from kvm_mips_handle_exit(), either
> + * resuming the guest or returning to the host depending on the return value.
> + *
> + * Returns: Next address after end of written function.
> + */
> +static void *kvm_mips_build_ret_from_exit(void *addr)
> +{
> + u32 *p = addr;
> + struct uasm_label labels[2];
> + struct uasm_reloc relocs[2];
> + struct uasm_label *l = labels;
> + struct uasm_reloc *r = relocs;
> +
> + memset(labels, 0, sizeof(labels));
> + memset(relocs, 0, sizeof(relocs));
>
> /* Return from handler Make sure interrupts are disabled */
> - di
> - ehb
> + uasm_i_di(&p, ZERO);
> + uasm_i_ehb(&p);
>
> /*
> * XXXKYMA: k0/k1 could have been blown away if we processed
> @@ -405,198 +504,119 @@ FEXPORT(__kvm_mips_jump_to_handler)
> * guest, reload k1
> */
>
> - move k1, s1
> - INT_ADDIU k1, k1, VCPU_HOST_ARCH
> + uasm_i_move(&p, K1, S1);
> + uasm_i_addiu(&p, K1, K1, offsetof(struct kvm_vcpu, arch));
>
> /*
> * Check return value, should tell us if we are returning to the
> * host (handle I/O etc)or resuming the guest
> */
> - andi t0, v0, RESUME_HOST
> - bnez t0, __kvm_mips_return_to_host
> - nop
> + uasm_i_andi(&p, T0, V0, RESUME_HOST);
> + uasm_il_bnez(&p, &r, T0, label_return_to_host);
> + uasm_i_nop(&p);
> +
> + p = kvm_mips_build_ret_to_guest(p);
> +
> + uasm_l_return_to_host(&l, p);
> + p = kvm_mips_build_ret_to_host(p);
> +
> + uasm_resolve_relocs(relocs, labels);
> +
> + return p;
> +}
> +
> +/**
> + * kvm_mips_build_ret_to_guest() - Assemble code to return to the guest.
> + * @addr: Address to start writing code.
> + *
> + * Assemble the code to handle return from the guest exit handler
> + * (kvm_mips_handle_exit()) back to the guest.
> + *
> + * Returns: Next address after end of written function.
> + */
> +static void *kvm_mips_build_ret_to_guest(void *addr)
> +{
> + u32 *p = addr;
>
> -__kvm_mips_return_to_guest:
> /* Put the saved pointer to vcpu (s1) back into the DDATA_LO Register */
> - mtc0 s1, CP0_DDATA_LO
> + uasm_i_mtc0(&p, S1, C0_DDATA_LO);
>
> /* Load up the Guest EBASE to minimize the window where BEV is set */
> - LONG_L t0, VCPU_GUEST_EBASE(k1)
> + UASM_i_LW(&p, T0, offsetof(struct kvm_vcpu_arch, guest_ebase), K1);
>
> /* Switch EBASE back to the one used by KVM */
> - mfc0 v1, CP0_STATUS
> - or k0, v1, ST0_BEV
> - mtc0 k0, CP0_STATUS
> - ehb
> - mtc0 t0, CP0_EBASE
> + uasm_i_mfc0(&p, V1, C0_STATUS);
> + uasm_i_lui(&p, AT, ST0_BEV >> 16);
> + uasm_i_or(&p, K0, V1, AT);
> + uasm_i_mtc0(&p, K0, C0_STATUS);
> + uasm_i_ehb(&p);
> + uasm_i_mtc0(&p, T0, C0_EBASE);
>
> /* Setup status register for running guest in UM */
> - or v1, v1, (ST0_EXL | KSU_USER | ST0_IE)
> - and v1, v1, ~(ST0_CU0 | ST0_MX)
> - mtc0 v1, CP0_STATUS
> - ehb
> + uasm_i_ori(&p, V1, V1, ST0_EXL | KSU_USER | ST0_IE);
> + UASM_i_LA(&p, AT, ~(ST0_CU0 | ST0_MX));
> + uasm_i_and(&p, V1, V1, AT);
> + uasm_i_mtc0(&p, V1, C0_STATUS);
> + uasm_i_ehb(&p);
> +
> + p = kvm_mips_build_enter_guest(p);
> +
> + return p;
> +}
> +
> +/**
> + * kvm_mips_build_ret_to_host() - Assemble code to return to the host.
> + * @addr: Address to start writing code.
> + *
> + * Assemble the code to handle return from the guest exit handler
> + * (kvm_mips_handle_exit()) back to the host, i.e. to the caller of the vcpu_run
> + * function generated by kvm_mips_build_vcpu_run().
> + *
> + * Returns: Next address after end of written function.
> + */
> +static void *kvm_mips_build_ret_to_host(void *addr)
> +{
> + u32 *p = addr;
> + unsigned int i;
>
> - /* Set Guest EPC */
> - LONG_L t0, VCPU_PC(k1)
> - mtc0 t0, CP0_EPC
> -
> - /* Set the ASID for the Guest Kernel */
> - PTR_L t0, VCPU_COP0(k1)
> - LONG_L t0, COP0_STATUS(t0)
> - andi t0, KSU_USER | ST0_ERL | ST0_EXL
> - xori t0, KSU_USER
> - bnez t0, 1f /* If kernel */
> - INT_ADDIU t1, k1, VCPU_GUEST_KERNEL_ASID /* (BD) */
> - INT_ADDIU t1, k1, VCPU_GUEST_USER_ASID /* else user */
> -1:
> - /* t1: contains the base of the ASID array, need to get the cpu id */
> - LONG_L t2, TI_CPU($28) /* smp_processor_id */
> - INT_SLL t2, t2, 2 /* x4 */
> - REG_ADDU t3, t1, t2
> - LONG_L k0, (t3)
> -#ifdef CONFIG_MIPS_ASID_BITS_VARIABLE
> - li t3, CPUINFO_SIZE/4
> - mul t2, t2, t3 /* x sizeof(struct cpuinfo_mips)/4 */
> - LONG_L t2, (cpu_data + CPUINFO_ASID_MASK)(t2)
> - and k0, k0, t2
> -#else
> - andi k0, k0, MIPS_ENTRYHI_ASID
> -#endif
> - mtc0 k0, CP0_ENTRYHI
> - ehb
> -
> - /* Disable RDHWR access */
> - mtc0 zero, CP0_HWRENA
> -
> - .set noat
> - /* load the guest context from VCPU and return */
> - LONG_L $0, VCPU_R0(k1)
> - LONG_L $1, VCPU_R1(k1)
> - LONG_L $2, VCPU_R2(k1)
> - LONG_L $3, VCPU_R3(k1)
> - LONG_L $4, VCPU_R4(k1)
> - LONG_L $5, VCPU_R5(k1)
> - LONG_L $6, VCPU_R6(k1)
> - LONG_L $7, VCPU_R7(k1)
> - LONG_L $8, VCPU_R8(k1)
> - LONG_L $9, VCPU_R9(k1)
> - LONG_L $10, VCPU_R10(k1)
> - LONG_L $11, VCPU_R11(k1)
> - LONG_L $12, VCPU_R12(k1)
> - LONG_L $13, VCPU_R13(k1)
> - LONG_L $14, VCPU_R14(k1)
> - LONG_L $15, VCPU_R15(k1)
> - LONG_L $16, VCPU_R16(k1)
> - LONG_L $17, VCPU_R17(k1)
> - LONG_L $18, VCPU_R18(k1)
> - LONG_L $19, VCPU_R19(k1)
> - LONG_L $20, VCPU_R20(k1)
> - LONG_L $21, VCPU_R21(k1)
> - LONG_L $22, VCPU_R22(k1)
> - LONG_L $23, VCPU_R23(k1)
> - LONG_L $24, VCPU_R24(k1)
> - LONG_L $25, VCPU_R25(k1)
> -
> - /* $/k1 loaded later */
> - LONG_L $28, VCPU_R28(k1)
> - LONG_L $29, VCPU_R29(k1)
> - LONG_L $30, VCPU_R30(k1)
> - LONG_L $31, VCPU_R31(k1)
> -
> -FEXPORT(__kvm_mips_skip_guest_restore)
> - LONG_L k0, VCPU_HI(k1)
> - mthi k0
> -
> - LONG_L k0, VCPU_LO(k1)
> - mtlo k0
> -
> - LONG_L k0, VCPU_R26(k1)
> - LONG_L k1, VCPU_R27(k1)
> -
> - eret
> - .set at
> -
> -__kvm_mips_return_to_host:
> /* EBASE is already pointing to Linux */
> - LONG_L k1, VCPU_HOST_STACK(k1)
> - INT_ADDIU k1,k1, -PT_SIZE
> + UASM_i_LW(&p, K1, offsetof(struct kvm_vcpu_arch, host_stack), K1);
> + uasm_i_addiu(&p, K1, K1, -(int)sizeof(struct pt_regs));
>
> /* Restore host DDATA_LO */
> - LONG_L k0, PT_HOST_USERLOCAL(k1)
> - mtc0 k0, CP0_DDATA_LO
> + UASM_i_LW(&p, K0, offsetof(struct pt_regs, cp0_epc), K1);
> + uasm_i_mtc0(&p, K0, C0_DDATA_LO);
>
> /*
> * r2/v0 is the return code, shift it down by 2 (arithmetic)
> * to recover the err code
> */
> - INT_SRA k0, v0, 2
> - move $2, k0
> + uasm_i_sra(&p, K0, V0, 2);
> + uasm_i_move(&p, V0, K0);
>
> /* Load context saved on the host stack */
> - LONG_L $16, PT_R16(k1)
> - LONG_L $17, PT_R17(k1)
> - LONG_L $18, PT_R18(k1)
> - LONG_L $19, PT_R19(k1)
> - LONG_L $20, PT_R20(k1)
> - LONG_L $21, PT_R21(k1)
> - LONG_L $22, PT_R22(k1)
> - LONG_L $23, PT_R23(k1)
> + for (i = 16; i < 31; ++i) {
> + if (i == 24)
> + i = 28;
> + UASM_i_LW(&p, i, offsetof(struct pt_regs, regs[i]), K1);
> + }
>
> - LONG_L $28, PT_R28(k1)
> - LONG_L $29, PT_R29(k1)
> - LONG_L $30, PT_R30(k1)
> + UASM_i_LW(&p, K0, offsetof(struct pt_regs, hi), K1);
> + uasm_i_mthi(&p, K0);
>
> - LONG_L k0, PT_HI(k1)
> - mthi k0
> -
> - LONG_L k0, PT_LO(k1)
> - mtlo k0
> + UASM_i_LW(&p, K0, offsetof(struct pt_regs, lo), K1);
> + uasm_i_mtlo(&p, K0);
>
> /* Restore RDHWR access */
> - INT_L k0, hwrena
> - mtc0 k0, CP0_HWRENA
> + UASM_i_LA_mostly(&p, K0, (long)&hwrena);
> + uasm_i_lw(&p, K0, uasm_rel_lo((long)&hwrena), K0);
> + uasm_i_mtc0(&p, K0, C0_HWRENA);
>
> /* Restore RA, which is the address we will return to */
> - LONG_L ra, PT_R31(k1)
> - j ra
> - nop
> + UASM_i_LW(&p, RA, offsetof(struct pt_regs, regs[RA]), K1);
> + uasm_i_jr(&p, RA);
> + uasm_i_nop(&p);
>
> -VECTOR_END(MIPSX(GuestExceptionEnd))
> -.end MIPSX(GuestException)
> + return p;
> +}
>
> -MIPSX(exceptions):
> - ####
> - ##### The exception handlers.
> - #####
> - .word _C_LABEL(MIPSX(GuestException)) # 0
> - .word _C_LABEL(MIPSX(GuestException)) # 1
> - .word _C_LABEL(MIPSX(GuestException)) # 2
> - .word _C_LABEL(MIPSX(GuestException)) # 3
> - .word _C_LABEL(MIPSX(GuestException)) # 4
> - .word _C_LABEL(MIPSX(GuestException)) # 5
> - .word _C_LABEL(MIPSX(GuestException)) # 6
> - .word _C_LABEL(MIPSX(GuestException)) # 7
> - .word _C_LABEL(MIPSX(GuestException)) # 8
> - .word _C_LABEL(MIPSX(GuestException)) # 9
> - .word _C_LABEL(MIPSX(GuestException)) # 10
> - .word _C_LABEL(MIPSX(GuestException)) # 11
> - .word _C_LABEL(MIPSX(GuestException)) # 12
> - .word _C_LABEL(MIPSX(GuestException)) # 13
> - .word _C_LABEL(MIPSX(GuestException)) # 14
> - .word _C_LABEL(MIPSX(GuestException)) # 15
> - .word _C_LABEL(MIPSX(GuestException)) # 16
> - .word _C_LABEL(MIPSX(GuestException)) # 17
> - .word _C_LABEL(MIPSX(GuestException)) # 18
> - .word _C_LABEL(MIPSX(GuestException)) # 19
> - .word _C_LABEL(MIPSX(GuestException)) # 20
> - .word _C_LABEL(MIPSX(GuestException)) # 21
> - .word _C_LABEL(MIPSX(GuestException)) # 22
> - .word _C_LABEL(MIPSX(GuestException)) # 23
> - .word _C_LABEL(MIPSX(GuestException)) # 24
> - .word _C_LABEL(MIPSX(GuestException)) # 25
> - .word _C_LABEL(MIPSX(GuestException)) # 26
> - .word _C_LABEL(MIPSX(GuestException)) # 27
> - .word _C_LABEL(MIPSX(GuestException)) # 28
> - .word _C_LABEL(MIPSX(GuestException)) # 29
> - .word _C_LABEL(MIPSX(GuestException)) # 30
> - .word _C_LABEL(MIPSX(GuestException)) # 31
> diff --git a/arch/mips/kvm/interrupt.h b/arch/mips/kvm/interrupt.h
> index d661c100b219..fb118a2c8379 100644
> --- a/arch/mips/kvm/interrupt.h
> +++ b/arch/mips/kvm/interrupt.h
> @@ -28,10 +28,6 @@
> #define MIPS_EXC_MAX 12
> /* XXXSL More to follow */
>
> -extern char __kvm_mips_vcpu_run_end[];
> -extern char mips32_exception[], mips32_exceptionEnd[];
> -extern char mips32_GuestException[], mips32_GuestExceptionEnd[];
> -
> #define C_TI (_ULCAST_(1) << 30)
>
> #define KVM_MIPS_IRQ_DELIVER_ALL_AT_ONCE (0)
> diff --git a/arch/mips/kvm/mips.c b/arch/mips/kvm/mips.c
> index 5a2b9034a05c..f00cf5f9ac52 100644
> --- a/arch/mips/kvm/mips.c
> +++ b/arch/mips/kvm/mips.c
> @@ -247,8 +247,8 @@ void kvm_arch_commit_memory_region(struct kvm *kvm,
>
> struct kvm_vcpu *kvm_arch_vcpu_create(struct kvm *kvm, unsigned int id)
> {
> - int err, size, offset;
> - void *gebase;
> + int err, size;
> + void *gebase, *p;
> int i;
>
> struct kvm_vcpu *vcpu = kzalloc(sizeof(struct kvm_vcpu), GFP_KERNEL);
> @@ -286,41 +286,28 @@ struct kvm_vcpu *kvm_arch_vcpu_create(struct kvm *kvm, unsigned int id)
> /* Save new ebase */
> vcpu->arch.guest_ebase = gebase;
>
> - /* Copy L1 Guest Exception handler to correct offset */
> + /* Build guest exception vectors dynamically in unmapped memory */
>
> /* TLB Refill, EXL = 0 */
> - memcpy(gebase, mips32_exception,
> - mips32_exceptionEnd - mips32_exception);
> + kvm_mips_build_exception(gebase);
>
> /* General Exception Entry point */
> - memcpy(gebase + 0x180, mips32_exception,
> - mips32_exceptionEnd - mips32_exception);
> + kvm_mips_build_exception(gebase + 0x180);
>
> /* For vectored interrupts poke the exception code @ all offsets 0-7 */
> for (i = 0; i < 8; i++) {
> kvm_debug("L1 Vectored handler @ %p\n",
> gebase + 0x200 + (i * VECTORSPACING));
> - memcpy(gebase + 0x200 + (i * VECTORSPACING), mips32_exception,
> - mips32_exceptionEnd - mips32_exception);
> + kvm_mips_build_exception(gebase + 0x200 + i * VECTORSPACING);
> }
>
> - /* General handler, relocate to unmapped space for sanity's sake */
> - offset = 0x2000;
> - kvm_debug("Installing KVM Exception handlers @ %p, %#x bytes\n",
> - gebase + offset,
> - mips32_GuestExceptionEnd - mips32_GuestException);
> + /* General exit handler */
> + p = gebase + 0x2000;
> + p = kvm_mips_build_exit(p);
>
> - memcpy(gebase + offset, mips32_GuestException,
> - mips32_GuestExceptionEnd - mips32_GuestException);
> -
> -#ifdef MODULE
> - offset += mips32_GuestExceptionEnd - mips32_GuestException;
> - memcpy(gebase + offset, (char *)__kvm_mips_vcpu_run,
> - __kvm_mips_vcpu_run_end - (char *)__kvm_mips_vcpu_run);
> - vcpu->arch.vcpu_run = gebase + offset;
> -#else
> - vcpu->arch.vcpu_run = __kvm_mips_vcpu_run;
> -#endif
> + /* Guest entry routine */
> + vcpu->arch.vcpu_run = p;
> + p = kvm_mips_build_vcpu_run(p);
>
> /* Invalidate the icache for these ranges */
> local_flush_icache_range((unsigned long)gebase,
> --
> 2.4.10
>
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply [flat|nested] 36+ messages in thread
* [PATCH 07/14] MIPS: KVM: Add dumping of generated entry code
@ 2016-06-23 16:34 ` James Hogan
0 siblings, 0 replies; 36+ messages in thread
From: James Hogan @ 2016-06-23 16:34 UTC (permalink / raw)
To: Paolo Bonzini, Radim Krčmář, Ralf Baechle
Cc: James Hogan, linux-mips, kvm
Dump the generated entry code with pr_debug(), similar to how it is done
in tlbex.c, so it can be more easily debugged.
Signed-off-by: James Hogan <james.hogan@imgtec.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Radim Krčmář <rkrcmar@redhat.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: linux-mips@linux-mips.org
Cc: kvm@vger.kernel.org
---
arch/mips/kvm/mips.c | 25 +++++++++++++++++++++++++
1 file changed, 25 insertions(+)
diff --git a/arch/mips/kvm/mips.c b/arch/mips/kvm/mips.c
index f00cf5f9ac52..f809ad84196d 100644
--- a/arch/mips/kvm/mips.c
+++ b/arch/mips/kvm/mips.c
@@ -245,6 +245,23 @@ void kvm_arch_commit_memory_region(struct kvm *kvm,
}
}
+static inline void dump_handler(const char *symbol, void *start, void *end)
+{
+ u32 *p;
+
+ pr_debug("LEAF(%s)\n", symbol);
+
+ pr_debug("\t.set push\n");
+ pr_debug("\t.set noreorder\n");
+
+ for (p = start; p < (u32 *)end; ++p)
+ pr_debug("\t.word\t0x%08x\t\t# %p\n", *p, p);
+
+ pr_debug("\t.set\tpop\n");
+
+ pr_debug("\tEND(%s)\n", symbol);
+}
+
struct kvm_vcpu *kvm_arch_vcpu_create(struct kvm *kvm, unsigned int id)
{
int err, size;
@@ -309,6 +326,14 @@ struct kvm_vcpu *kvm_arch_vcpu_create(struct kvm *kvm, unsigned int id)
vcpu->arch.vcpu_run = p;
p = kvm_mips_build_vcpu_run(p);
+ /* Dump the generated code */
+ pr_debug("#include <asm/asm.h>\n");
+ pr_debug("#include <asm/regdef.h>\n");
+ pr_debug("\n");
+ dump_handler("kvm_vcpu_run", vcpu->arch.vcpu_run, p);
+ dump_handler("kvm_gen_exc", gebase + 0x180, gebase + 0x200);
+ dump_handler("kvm_exit", gebase + 0x2000, vcpu->arch.vcpu_run);
+
/* Invalidate the icache for these ranges */
local_flush_icache_range((unsigned long)gebase,
(unsigned long)gebase + ALIGN(size, PAGE_SIZE));
--
2.4.10
^ permalink raw reply related [flat|nested] 36+ messages in thread* [PATCH 07/14] MIPS: KVM: Add dumping of generated entry code
@ 2016-06-23 16:34 ` James Hogan
0 siblings, 0 replies; 36+ messages in thread
From: James Hogan @ 2016-06-23 16:34 UTC (permalink / raw)
To: Paolo Bonzini, Radim Krčmář, Ralf Baechle
Cc: James Hogan, linux-mips, kvm
Dump the generated entry code with pr_debug(), similar to how it is done
in tlbex.c, so it can be more easily debugged.
Signed-off-by: James Hogan <james.hogan@imgtec.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Radim Krčmář <rkrcmar@redhat.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: linux-mips@linux-mips.org
Cc: kvm@vger.kernel.org
---
arch/mips/kvm/mips.c | 25 +++++++++++++++++++++++++
1 file changed, 25 insertions(+)
diff --git a/arch/mips/kvm/mips.c b/arch/mips/kvm/mips.c
index f00cf5f9ac52..f809ad84196d 100644
--- a/arch/mips/kvm/mips.c
+++ b/arch/mips/kvm/mips.c
@@ -245,6 +245,23 @@ void kvm_arch_commit_memory_region(struct kvm *kvm,
}
}
+static inline void dump_handler(const char *symbol, void *start, void *end)
+{
+ u32 *p;
+
+ pr_debug("LEAF(%s)\n", symbol);
+
+ pr_debug("\t.set push\n");
+ pr_debug("\t.set noreorder\n");
+
+ for (p = start; p < (u32 *)end; ++p)
+ pr_debug("\t.word\t0x%08x\t\t# %p\n", *p, p);
+
+ pr_debug("\t.set\tpop\n");
+
+ pr_debug("\tEND(%s)\n", symbol);
+}
+
struct kvm_vcpu *kvm_arch_vcpu_create(struct kvm *kvm, unsigned int id)
{
int err, size;
@@ -309,6 +326,14 @@ struct kvm_vcpu *kvm_arch_vcpu_create(struct kvm *kvm, unsigned int id)
vcpu->arch.vcpu_run = p;
p = kvm_mips_build_vcpu_run(p);
+ /* Dump the generated code */
+ pr_debug("#include <asm/asm.h>\n");
+ pr_debug("#include <asm/regdef.h>\n");
+ pr_debug("\n");
+ dump_handler("kvm_vcpu_run", vcpu->arch.vcpu_run, p);
+ dump_handler("kvm_gen_exc", gebase + 0x180, gebase + 0x200);
+ dump_handler("kvm_exit", gebase + 0x2000, vcpu->arch.vcpu_run);
+
/* Invalidate the icache for these ranges */
local_flush_icache_range((unsigned long)gebase,
(unsigned long)gebase + ALIGN(size, PAGE_SIZE));
--
2.4.10
^ permalink raw reply related [flat|nested] 36+ messages in thread
* [PATCH 08/14] MIPS: KVM: Drop now unused asm offsets
@ 2016-06-23 16:34 ` James Hogan
0 siblings, 0 replies; 36+ messages in thread
From: James Hogan @ 2016-06-23 16:34 UTC (permalink / raw)
To: Paolo Bonzini, Radim Krčmář, Ralf Baechle
Cc: James Hogan, linux-mips, kvm
Now that locore.S is converted to uasm, remove a bunch of the assembly
offset definitions created by asm-offsets.c, including the CPUINFO_ ones
for reading the variable asid mask, and the non FPU/MSA related VCPU_
definitions. KVM's fpu.S and msa.S still use the remaining definitions.
Signed-off-by: James Hogan <james.hogan@imgtec.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Radim Krčmář <rkrcmar@redhat.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: linux-mips@linux-mips.org
Cc: kvm@vger.kernel.org
---
arch/mips/kernel/asm-offsets.c | 66 ------------------------------------------
1 file changed, 66 deletions(-)
diff --git a/arch/mips/kernel/asm-offsets.c b/arch/mips/kernel/asm-offsets.c
index a1263d188a5a..fae2f9447792 100644
--- a/arch/mips/kernel/asm-offsets.c
+++ b/arch/mips/kernel/asm-offsets.c
@@ -339,67 +339,9 @@ void output_pm_defines(void)
}
#endif
-void output_cpuinfo_defines(void)
-{
- COMMENT(" MIPS cpuinfo offsets. ");
- DEFINE(CPUINFO_SIZE, sizeof(struct cpuinfo_mips));
-#ifdef CONFIG_MIPS_ASID_BITS_VARIABLE
- OFFSET(CPUINFO_ASID_MASK, cpuinfo_mips, asid_mask);
-#endif
-}
-
void output_kvm_defines(void)
{
COMMENT(" KVM/MIPS Specfic offsets. ");
- DEFINE(VCPU_ARCH_SIZE, sizeof(struct kvm_vcpu_arch));
- OFFSET(VCPU_RUN, kvm_vcpu, run);
- OFFSET(VCPU_HOST_ARCH, kvm_vcpu, arch);
-
- OFFSET(VCPU_GUEST_EBASE, kvm_vcpu_arch, guest_ebase);
-
- OFFSET(VCPU_HOST_STACK, kvm_vcpu_arch, host_stack);
- OFFSET(VCPU_HOST_GP, kvm_vcpu_arch, host_gp);
-
- OFFSET(VCPU_HOST_CP0_BADVADDR, kvm_vcpu_arch, host_cp0_badvaddr);
- OFFSET(VCPU_HOST_CP0_CAUSE, kvm_vcpu_arch, host_cp0_cause);
- OFFSET(VCPU_HOST_EPC, kvm_vcpu_arch, host_cp0_epc);
-
- OFFSET(VCPU_R0, kvm_vcpu_arch, gprs[0]);
- OFFSET(VCPU_R1, kvm_vcpu_arch, gprs[1]);
- OFFSET(VCPU_R2, kvm_vcpu_arch, gprs[2]);
- OFFSET(VCPU_R3, kvm_vcpu_arch, gprs[3]);
- OFFSET(VCPU_R4, kvm_vcpu_arch, gprs[4]);
- OFFSET(VCPU_R5, kvm_vcpu_arch, gprs[5]);
- OFFSET(VCPU_R6, kvm_vcpu_arch, gprs[6]);
- OFFSET(VCPU_R7, kvm_vcpu_arch, gprs[7]);
- OFFSET(VCPU_R8, kvm_vcpu_arch, gprs[8]);
- OFFSET(VCPU_R9, kvm_vcpu_arch, gprs[9]);
- OFFSET(VCPU_R10, kvm_vcpu_arch, gprs[10]);
- OFFSET(VCPU_R11, kvm_vcpu_arch, gprs[11]);
- OFFSET(VCPU_R12, kvm_vcpu_arch, gprs[12]);
- OFFSET(VCPU_R13, kvm_vcpu_arch, gprs[13]);
- OFFSET(VCPU_R14, kvm_vcpu_arch, gprs[14]);
- OFFSET(VCPU_R15, kvm_vcpu_arch, gprs[15]);
- OFFSET(VCPU_R16, kvm_vcpu_arch, gprs[16]);
- OFFSET(VCPU_R17, kvm_vcpu_arch, gprs[17]);
- OFFSET(VCPU_R18, kvm_vcpu_arch, gprs[18]);
- OFFSET(VCPU_R19, kvm_vcpu_arch, gprs[19]);
- OFFSET(VCPU_R20, kvm_vcpu_arch, gprs[20]);
- OFFSET(VCPU_R21, kvm_vcpu_arch, gprs[21]);
- OFFSET(VCPU_R22, kvm_vcpu_arch, gprs[22]);
- OFFSET(VCPU_R23, kvm_vcpu_arch, gprs[23]);
- OFFSET(VCPU_R24, kvm_vcpu_arch, gprs[24]);
- OFFSET(VCPU_R25, kvm_vcpu_arch, gprs[25]);
- OFFSET(VCPU_R26, kvm_vcpu_arch, gprs[26]);
- OFFSET(VCPU_R27, kvm_vcpu_arch, gprs[27]);
- OFFSET(VCPU_R28, kvm_vcpu_arch, gprs[28]);
- OFFSET(VCPU_R29, kvm_vcpu_arch, gprs[29]);
- OFFSET(VCPU_R30, kvm_vcpu_arch, gprs[30]);
- OFFSET(VCPU_R31, kvm_vcpu_arch, gprs[31]);
- OFFSET(VCPU_LO, kvm_vcpu_arch, lo);
- OFFSET(VCPU_HI, kvm_vcpu_arch, hi);
- OFFSET(VCPU_PC, kvm_vcpu_arch, pc);
- BLANK();
OFFSET(VCPU_FPR0, kvm_vcpu_arch, fpu.fpr[0]);
OFFSET(VCPU_FPR1, kvm_vcpu_arch, fpu.fpr[1]);
@@ -437,14 +379,6 @@ void output_kvm_defines(void)
OFFSET(VCPU_FCR31, kvm_vcpu_arch, fpu.fcr31);
OFFSET(VCPU_MSA_CSR, kvm_vcpu_arch, fpu.msacsr);
BLANK();
-
- OFFSET(VCPU_COP0, kvm_vcpu_arch, cop0);
- OFFSET(VCPU_GUEST_KERNEL_ASID, kvm_vcpu_arch, guest_kernel_asid);
- OFFSET(VCPU_GUEST_USER_ASID, kvm_vcpu_arch, guest_user_asid);
-
- OFFSET(COP0_TLB_HI, mips_coproc, reg[MIPS_CP0_TLB_HI][0]);
- OFFSET(COP0_STATUS, mips_coproc, reg[MIPS_CP0_STATUS][0]);
- BLANK();
}
#ifdef CONFIG_MIPS_CPS
--
2.4.10
^ permalink raw reply related [flat|nested] 36+ messages in thread* [PATCH 08/14] MIPS: KVM: Drop now unused asm offsets
@ 2016-06-23 16:34 ` James Hogan
0 siblings, 0 replies; 36+ messages in thread
From: James Hogan @ 2016-06-23 16:34 UTC (permalink / raw)
To: Paolo Bonzini, Radim Krčmář, Ralf Baechle
Cc: James Hogan, linux-mips, kvm
Now that locore.S is converted to uasm, remove a bunch of the assembly
offset definitions created by asm-offsets.c, including the CPUINFO_ ones
for reading the variable asid mask, and the non FPU/MSA related VCPU_
definitions. KVM's fpu.S and msa.S still use the remaining definitions.
Signed-off-by: James Hogan <james.hogan@imgtec.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Radim Krčmář <rkrcmar@redhat.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: linux-mips@linux-mips.org
Cc: kvm@vger.kernel.org
---
arch/mips/kernel/asm-offsets.c | 66 ------------------------------------------
1 file changed, 66 deletions(-)
diff --git a/arch/mips/kernel/asm-offsets.c b/arch/mips/kernel/asm-offsets.c
index a1263d188a5a..fae2f9447792 100644
--- a/arch/mips/kernel/asm-offsets.c
+++ b/arch/mips/kernel/asm-offsets.c
@@ -339,67 +339,9 @@ void output_pm_defines(void)
}
#endif
-void output_cpuinfo_defines(void)
-{
- COMMENT(" MIPS cpuinfo offsets. ");
- DEFINE(CPUINFO_SIZE, sizeof(struct cpuinfo_mips));
-#ifdef CONFIG_MIPS_ASID_BITS_VARIABLE
- OFFSET(CPUINFO_ASID_MASK, cpuinfo_mips, asid_mask);
-#endif
-}
-
void output_kvm_defines(void)
{
COMMENT(" KVM/MIPS Specfic offsets. ");
- DEFINE(VCPU_ARCH_SIZE, sizeof(struct kvm_vcpu_arch));
- OFFSET(VCPU_RUN, kvm_vcpu, run);
- OFFSET(VCPU_HOST_ARCH, kvm_vcpu, arch);
-
- OFFSET(VCPU_GUEST_EBASE, kvm_vcpu_arch, guest_ebase);
-
- OFFSET(VCPU_HOST_STACK, kvm_vcpu_arch, host_stack);
- OFFSET(VCPU_HOST_GP, kvm_vcpu_arch, host_gp);
-
- OFFSET(VCPU_HOST_CP0_BADVADDR, kvm_vcpu_arch, host_cp0_badvaddr);
- OFFSET(VCPU_HOST_CP0_CAUSE, kvm_vcpu_arch, host_cp0_cause);
- OFFSET(VCPU_HOST_EPC, kvm_vcpu_arch, host_cp0_epc);
-
- OFFSET(VCPU_R0, kvm_vcpu_arch, gprs[0]);
- OFFSET(VCPU_R1, kvm_vcpu_arch, gprs[1]);
- OFFSET(VCPU_R2, kvm_vcpu_arch, gprs[2]);
- OFFSET(VCPU_R3, kvm_vcpu_arch, gprs[3]);
- OFFSET(VCPU_R4, kvm_vcpu_arch, gprs[4]);
- OFFSET(VCPU_R5, kvm_vcpu_arch, gprs[5]);
- OFFSET(VCPU_R6, kvm_vcpu_arch, gprs[6]);
- OFFSET(VCPU_R7, kvm_vcpu_arch, gprs[7]);
- OFFSET(VCPU_R8, kvm_vcpu_arch, gprs[8]);
- OFFSET(VCPU_R9, kvm_vcpu_arch, gprs[9]);
- OFFSET(VCPU_R10, kvm_vcpu_arch, gprs[10]);
- OFFSET(VCPU_R11, kvm_vcpu_arch, gprs[11]);
- OFFSET(VCPU_R12, kvm_vcpu_arch, gprs[12]);
- OFFSET(VCPU_R13, kvm_vcpu_arch, gprs[13]);
- OFFSET(VCPU_R14, kvm_vcpu_arch, gprs[14]);
- OFFSET(VCPU_R15, kvm_vcpu_arch, gprs[15]);
- OFFSET(VCPU_R16, kvm_vcpu_arch, gprs[16]);
- OFFSET(VCPU_R17, kvm_vcpu_arch, gprs[17]);
- OFFSET(VCPU_R18, kvm_vcpu_arch, gprs[18]);
- OFFSET(VCPU_R19, kvm_vcpu_arch, gprs[19]);
- OFFSET(VCPU_R20, kvm_vcpu_arch, gprs[20]);
- OFFSET(VCPU_R21, kvm_vcpu_arch, gprs[21]);
- OFFSET(VCPU_R22, kvm_vcpu_arch, gprs[22]);
- OFFSET(VCPU_R23, kvm_vcpu_arch, gprs[23]);
- OFFSET(VCPU_R24, kvm_vcpu_arch, gprs[24]);
- OFFSET(VCPU_R25, kvm_vcpu_arch, gprs[25]);
- OFFSET(VCPU_R26, kvm_vcpu_arch, gprs[26]);
- OFFSET(VCPU_R27, kvm_vcpu_arch, gprs[27]);
- OFFSET(VCPU_R28, kvm_vcpu_arch, gprs[28]);
- OFFSET(VCPU_R29, kvm_vcpu_arch, gprs[29]);
- OFFSET(VCPU_R30, kvm_vcpu_arch, gprs[30]);
- OFFSET(VCPU_R31, kvm_vcpu_arch, gprs[31]);
- OFFSET(VCPU_LO, kvm_vcpu_arch, lo);
- OFFSET(VCPU_HI, kvm_vcpu_arch, hi);
- OFFSET(VCPU_PC, kvm_vcpu_arch, pc);
- BLANK();
OFFSET(VCPU_FPR0, kvm_vcpu_arch, fpu.fpr[0]);
OFFSET(VCPU_FPR1, kvm_vcpu_arch, fpu.fpr[1]);
@@ -437,14 +379,6 @@ void output_kvm_defines(void)
OFFSET(VCPU_FCR31, kvm_vcpu_arch, fpu.fcr31);
OFFSET(VCPU_MSA_CSR, kvm_vcpu_arch, fpu.msacsr);
BLANK();
-
- OFFSET(VCPU_COP0, kvm_vcpu_arch, cop0);
- OFFSET(VCPU_GUEST_KERNEL_ASID, kvm_vcpu_arch, guest_kernel_asid);
- OFFSET(VCPU_GUEST_USER_ASID, kvm_vcpu_arch, guest_user_asid);
-
- OFFSET(COP0_TLB_HI, mips_coproc, reg[MIPS_CP0_TLB_HI][0]);
- OFFSET(COP0_STATUS, mips_coproc, reg[MIPS_CP0_STATUS][0]);
- BLANK();
}
#ifdef CONFIG_MIPS_CPS
--
2.4.10
^ permalink raw reply related [flat|nested] 36+ messages in thread
* [PATCH 09/14] MIPS: KVM: Omit FPU handling entry code if possible
@ 2016-06-23 16:34 ` James Hogan
0 siblings, 0 replies; 36+ messages in thread
From: James Hogan @ 2016-06-23 16:34 UTC (permalink / raw)
To: Paolo Bonzini, Radim Krčmář, Ralf Baechle
Cc: James Hogan, linux-mips, kvm
The FPU handling code on entry from guest is unnecessary if no FPU is
present, so allow it to be dropped at uasm assembly time.
Signed-off-by: James Hogan <james.hogan@imgtec.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Radim Krčmář <rkrcmar@redhat.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: linux-mips@linux-mips.org
Cc: kvm@vger.kernel.org
---
arch/mips/kvm/entry.c | 27 +++++++++++++++------------
1 file changed, 15 insertions(+), 12 deletions(-)
diff --git a/arch/mips/kvm/entry.c b/arch/mips/kvm/entry.c
index 9a18b4939b35..c0d9f551c1c1 100644
--- a/arch/mips/kvm/entry.c
+++ b/arch/mips/kvm/entry.c
@@ -393,18 +393,21 @@ void *kvm_mips_build_exit(void *addr)
UASM_i_LW(&p, K0, uasm_rel_lo((long)&ebase), K0);
uasm_i_mtc0(&p, K0, C0_EBASE);
- /*
- * If FPU is enabled, save FCR31 and clear it so that later ctc1's don't
- * trigger FPE for pending exceptions.
- */
- uasm_i_lui(&p, AT, ST0_CU1 >> 16);
- uasm_i_and(&p, V1, V0, AT);
- uasm_il_beqz(&p, &r, V1, label_fpu_1);
- uasm_i_nop(&p);
- uasm_i_cfc1(&p, T0, 31);
- uasm_i_sw(&p, T0, offsetof(struct kvm_vcpu_arch, fpu.fcr31), K1);
- uasm_i_ctc1(&p, ZERO, 31);
- uasm_l_fpu_1(&l, p);
+ if (raw_cpu_has_fpu) {
+ /*
+ * If FPU is enabled, save FCR31 and clear it so that later
+ * ctc1's don't trigger FPE for pending exceptions.
+ */
+ uasm_i_lui(&p, AT, ST0_CU1 >> 16);
+ uasm_i_and(&p, V1, V0, AT);
+ uasm_il_beqz(&p, &r, V1, label_fpu_1);
+ uasm_i_nop(&p);
+ uasm_i_cfc1(&p, T0, 31);
+ uasm_i_sw(&p, T0, offsetof(struct kvm_vcpu_arch, fpu.fcr31),
+ K1);
+ uasm_i_ctc1(&p, ZERO, 31);
+ uasm_l_fpu_1(&l, p);
+ }
#ifdef CONFIG_CPU_HAS_MSA
/*
--
2.4.10
^ permalink raw reply related [flat|nested] 36+ messages in thread* [PATCH 09/14] MIPS: KVM: Omit FPU handling entry code if possible
@ 2016-06-23 16:34 ` James Hogan
0 siblings, 0 replies; 36+ messages in thread
From: James Hogan @ 2016-06-23 16:34 UTC (permalink / raw)
To: Paolo Bonzini, Radim Krčmář, Ralf Baechle
Cc: James Hogan, linux-mips, kvm
The FPU handling code on entry from guest is unnecessary if no FPU is
present, so allow it to be dropped at uasm assembly time.
Signed-off-by: James Hogan <james.hogan@imgtec.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Radim Krčmář <rkrcmar@redhat.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: linux-mips@linux-mips.org
Cc: kvm@vger.kernel.org
---
arch/mips/kvm/entry.c | 27 +++++++++++++++------------
1 file changed, 15 insertions(+), 12 deletions(-)
diff --git a/arch/mips/kvm/entry.c b/arch/mips/kvm/entry.c
index 9a18b4939b35..c0d9f551c1c1 100644
--- a/arch/mips/kvm/entry.c
+++ b/arch/mips/kvm/entry.c
@@ -393,18 +393,21 @@ void *kvm_mips_build_exit(void *addr)
UASM_i_LW(&p, K0, uasm_rel_lo((long)&ebase), K0);
uasm_i_mtc0(&p, K0, C0_EBASE);
- /*
- * If FPU is enabled, save FCR31 and clear it so that later ctc1's don't
- * trigger FPE for pending exceptions.
- */
- uasm_i_lui(&p, AT, ST0_CU1 >> 16);
- uasm_i_and(&p, V1, V0, AT);
- uasm_il_beqz(&p, &r, V1, label_fpu_1);
- uasm_i_nop(&p);
- uasm_i_cfc1(&p, T0, 31);
- uasm_i_sw(&p, T0, offsetof(struct kvm_vcpu_arch, fpu.fcr31), K1);
- uasm_i_ctc1(&p, ZERO, 31);
- uasm_l_fpu_1(&l, p);
+ if (raw_cpu_has_fpu) {
+ /*
+ * If FPU is enabled, save FCR31 and clear it so that later
+ * ctc1's don't trigger FPE for pending exceptions.
+ */
+ uasm_i_lui(&p, AT, ST0_CU1 >> 16);
+ uasm_i_and(&p, V1, V0, AT);
+ uasm_il_beqz(&p, &r, V1, label_fpu_1);
+ uasm_i_nop(&p);
+ uasm_i_cfc1(&p, T0, 31);
+ uasm_i_sw(&p, T0, offsetof(struct kvm_vcpu_arch, fpu.fcr31),
+ K1);
+ uasm_i_ctc1(&p, ZERO, 31);
+ uasm_l_fpu_1(&l, p);
+ }
#ifdef CONFIG_CPU_HAS_MSA
/*
--
2.4.10
^ permalink raw reply related [flat|nested] 36+ messages in thread
* [PATCH 10/14] MIPS: KVM: Check MSA presence at uasm time
@ 2016-06-23 16:34 ` James Hogan
0 siblings, 0 replies; 36+ messages in thread
From: James Hogan @ 2016-06-23 16:34 UTC (permalink / raw)
To: Paolo Bonzini, Radim Krčmář, Ralf Baechle
Cc: James Hogan, linux-mips, kvm
Check for presence of MSA at uasm assembly time rather than at runtime
in the generated KVM host entry code. This optimises the guest exit path
by eliminating the MSA code entirely if not present, and eliminating the
read of Config3.MSAP and conditional branch if MSA is present.
Signed-off-by: James Hogan <james.hogan@imgtec.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Radim Krčmář <rkrcmar@redhat.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: linux-mips@linux-mips.org
Cc: kvm@vger.kernel.org
---
arch/mips/kvm/entry.c | 35 +++++++++++++++--------------------
1 file changed, 15 insertions(+), 20 deletions(-)
diff --git a/arch/mips/kvm/entry.c b/arch/mips/kvm/entry.c
index c0d9f551c1c1..53e1e576d18a 100644
--- a/arch/mips/kvm/entry.c
+++ b/arch/mips/kvm/entry.c
@@ -55,7 +55,6 @@
#define C0_CAUSE 13, 0
#define C0_EPC 14, 0
#define C0_EBASE 15, 1
-#define C0_CONFIG3 16, 3
#define C0_CONFIG5 16, 5
#define C0_DDATA_LO 28, 3
#define C0_ERROREPC 30, 0
@@ -409,25 +408,21 @@ void *kvm_mips_build_exit(void *addr)
uasm_l_fpu_1(&l, p);
}
-#ifdef CONFIG_CPU_HAS_MSA
- /*
- * If MSA is enabled, save MSACSR and clear it so that later
- * instructions don't trigger MSAFPE for pending exceptions.
- */
- uasm_i_mfc0(&p, T0, C0_CONFIG3);
- uasm_i_ext(&p, T0, T0, 28, 1); /* MIPS_CONF3_MSAP */
- uasm_il_beqz(&p, &r, T0, label_msa_1);
- uasm_i_nop(&p);
- uasm_i_mfc0(&p, T0, C0_CONFIG5);
- uasm_i_ext(&p, T0, T0, 27, 1); /* MIPS_CONF5_MSAEN */
- uasm_il_beqz(&p, &r, T0, label_msa_1);
- uasm_i_nop(&p);
- uasm_i_cfcmsa(&p, T0, MSA_CSR);
- uasm_i_sw(&p, T0, offsetof(struct kvm_vcpu_arch, fpu.msacsr),
- K1);
- uasm_i_ctcmsa(&p, MSA_CSR, ZERO);
- uasm_l_msa_1(&l, p);
-#endif
+ if (cpu_has_msa) {
+ /*
+ * If MSA is enabled, save MSACSR and clear it so that later
+ * instructions don't trigger MSAFPE for pending exceptions.
+ */
+ uasm_i_mfc0(&p, T0, C0_CONFIG5);
+ uasm_i_ext(&p, T0, T0, 27, 1); /* MIPS_CONF5_MSAEN */
+ uasm_il_beqz(&p, &r, T0, label_msa_1);
+ uasm_i_nop(&p);
+ uasm_i_cfcmsa(&p, T0, MSA_CSR);
+ uasm_i_sw(&p, T0, offsetof(struct kvm_vcpu_arch, fpu.msacsr),
+ K1);
+ uasm_i_ctcmsa(&p, MSA_CSR, ZERO);
+ uasm_l_msa_1(&l, p);
+ }
/* Now that the new EBASE has been loaded, unset BEV and KSU_USER */
uasm_i_addiu(&p, AT, ZERO, ~(ST0_EXL | KSU_USER | ST0_IE));
--
2.4.10
^ permalink raw reply related [flat|nested] 36+ messages in thread* [PATCH 10/14] MIPS: KVM: Check MSA presence at uasm time
@ 2016-06-23 16:34 ` James Hogan
0 siblings, 0 replies; 36+ messages in thread
From: James Hogan @ 2016-06-23 16:34 UTC (permalink / raw)
To: Paolo Bonzini, Radim Krčmář, Ralf Baechle
Cc: James Hogan, linux-mips, kvm
Check for presence of MSA at uasm assembly time rather than at runtime
in the generated KVM host entry code. This optimises the guest exit path
by eliminating the MSA code entirely if not present, and eliminating the
read of Config3.MSAP and conditional branch if MSA is present.
Signed-off-by: James Hogan <james.hogan@imgtec.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Radim Krčmář <rkrcmar@redhat.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: linux-mips@linux-mips.org
Cc: kvm@vger.kernel.org
---
arch/mips/kvm/entry.c | 35 +++++++++++++++--------------------
1 file changed, 15 insertions(+), 20 deletions(-)
diff --git a/arch/mips/kvm/entry.c b/arch/mips/kvm/entry.c
index c0d9f551c1c1..53e1e576d18a 100644
--- a/arch/mips/kvm/entry.c
+++ b/arch/mips/kvm/entry.c
@@ -55,7 +55,6 @@
#define C0_CAUSE 13, 0
#define C0_EPC 14, 0
#define C0_EBASE 15, 1
-#define C0_CONFIG3 16, 3
#define C0_CONFIG5 16, 5
#define C0_DDATA_LO 28, 3
#define C0_ERROREPC 30, 0
@@ -409,25 +408,21 @@ void *kvm_mips_build_exit(void *addr)
uasm_l_fpu_1(&l, p);
}
-#ifdef CONFIG_CPU_HAS_MSA
- /*
- * If MSA is enabled, save MSACSR and clear it so that later
- * instructions don't trigger MSAFPE for pending exceptions.
- */
- uasm_i_mfc0(&p, T0, C0_CONFIG3);
- uasm_i_ext(&p, T0, T0, 28, 1); /* MIPS_CONF3_MSAP */
- uasm_il_beqz(&p, &r, T0, label_msa_1);
- uasm_i_nop(&p);
- uasm_i_mfc0(&p, T0, C0_CONFIG5);
- uasm_i_ext(&p, T0, T0, 27, 1); /* MIPS_CONF5_MSAEN */
- uasm_il_beqz(&p, &r, T0, label_msa_1);
- uasm_i_nop(&p);
- uasm_i_cfcmsa(&p, T0, MSA_CSR);
- uasm_i_sw(&p, T0, offsetof(struct kvm_vcpu_arch, fpu.msacsr),
- K1);
- uasm_i_ctcmsa(&p, MSA_CSR, ZERO);
- uasm_l_msa_1(&l, p);
-#endif
+ if (cpu_has_msa) {
+ /*
+ * If MSA is enabled, save MSACSR and clear it so that later
+ * instructions don't trigger MSAFPE for pending exceptions.
+ */
+ uasm_i_mfc0(&p, T0, C0_CONFIG5);
+ uasm_i_ext(&p, T0, T0, 27, 1); /* MIPS_CONF5_MSAEN */
+ uasm_il_beqz(&p, &r, T0, label_msa_1);
+ uasm_i_nop(&p);
+ uasm_i_cfcmsa(&p, T0, MSA_CSR);
+ uasm_i_sw(&p, T0, offsetof(struct kvm_vcpu_arch, fpu.msacsr),
+ K1);
+ uasm_i_ctcmsa(&p, MSA_CSR, ZERO);
+ uasm_l_msa_1(&l, p);
+ }
/* Now that the new EBASE has been loaded, unset BEV and KSU_USER */
uasm_i_addiu(&p, AT, ZERO, ~(ST0_EXL | KSU_USER | ST0_IE));
--
2.4.10
^ permalink raw reply related [flat|nested] 36+ messages in thread
* [PATCH 11/14] MIPS: KVM: Drop redundant restore of DDATA_LO
@ 2016-06-23 16:34 ` James Hogan
0 siblings, 0 replies; 36+ messages in thread
From: James Hogan @ 2016-06-23 16:34 UTC (permalink / raw)
To: Paolo Bonzini, Radim Krčmář, Ralf Baechle
Cc: James Hogan, linux-mips, kvm
On return from the exit handler to the host (without re-entering the
guest) we restore the saved value of the DDATA_LO register which we use
as a scratch register. However we've already restored it ready for
calling the exit handler so there is no need to do it again, so drop
that code.
Signed-off-by: James Hogan <james.hogan@imgtec.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Radim Krčmář <rkrcmar@redhat.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: linux-mips@linux-mips.org
Cc: kvm@vger.kernel.org
---
arch/mips/kvm/entry.c | 4 ----
1 file changed, 4 deletions(-)
diff --git a/arch/mips/kvm/entry.c b/arch/mips/kvm/entry.c
index 53e1e576d18a..6395bfa7e542 100644
--- a/arch/mips/kvm/entry.c
+++ b/arch/mips/kvm/entry.c
@@ -581,10 +581,6 @@ static void *kvm_mips_build_ret_to_host(void *addr)
UASM_i_LW(&p, K1, offsetof(struct kvm_vcpu_arch, host_stack), K1);
uasm_i_addiu(&p, K1, K1, -(int)sizeof(struct pt_regs));
- /* Restore host DDATA_LO */
- UASM_i_LW(&p, K0, offsetof(struct pt_regs, cp0_epc), K1);
- uasm_i_mtc0(&p, K0, C0_DDATA_LO);
-
/*
* r2/v0 is the return code, shift it down by 2 (arithmetic)
* to recover the err code
--
2.4.10
^ permalink raw reply related [flat|nested] 36+ messages in thread
* [PATCH 11/14] MIPS: KVM: Drop redundant restore of DDATA_LO
@ 2016-06-23 16:34 ` James Hogan
0 siblings, 0 replies; 36+ messages in thread
From: James Hogan @ 2016-06-23 16:34 UTC (permalink / raw)
To: Paolo Bonzini, Radim Krčmář, Ralf Baechle
Cc: James Hogan, linux-mips, kvm
On return from the exit handler to the host (without re-entering the
guest) we restore the saved value of the DDATA_LO register which we use
as a scratch register. However we've already restored it ready for
calling the exit handler so there is no need to do it again, so drop
that code.
Signed-off-by: James Hogan <james.hogan@imgtec.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Radim Krčmář <rkrcmar@redhat.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: linux-mips@linux-mips.org
Cc: kvm@vger.kernel.org
---
arch/mips/kvm/entry.c | 4 ----
1 file changed, 4 deletions(-)
diff --git a/arch/mips/kvm/entry.c b/arch/mips/kvm/entry.c
index 53e1e576d18a..6395bfa7e542 100644
--- a/arch/mips/kvm/entry.c
+++ b/arch/mips/kvm/entry.c
@@ -581,10 +581,6 @@ static void *kvm_mips_build_ret_to_host(void *addr)
UASM_i_LW(&p, K1, offsetof(struct kvm_vcpu_arch, host_stack), K1);
uasm_i_addiu(&p, K1, K1, -(int)sizeof(struct pt_regs));
- /* Restore host DDATA_LO */
- UASM_i_LW(&p, K0, offsetof(struct pt_regs, cp0_epc), K1);
- uasm_i_mtc0(&p, K0, C0_DDATA_LO);
-
/*
* r2/v0 is the return code, shift it down by 2 (arithmetic)
* to recover the err code
--
2.4.10
^ permalink raw reply related [flat|nested] 36+ messages in thread
* [PATCH 12/14] MIPS: KVM: Dynamically choose scratch registers
@ 2016-06-23 16:34 ` James Hogan
0 siblings, 0 replies; 36+ messages in thread
From: James Hogan @ 2016-06-23 16:34 UTC (permalink / raw)
To: Paolo Bonzini, Radim Krčmář, Ralf Baechle
Cc: James Hogan, linux-mips, kvm
Scratch cop0 registers are needed by KVM to be able to save/restore all
the GPRs, including k0/k1, and for storing the VCPU pointer. However no
registers are universally suitable for these purposes, so the decision
should be made at runtime.
Until now, we've used DDATA_LO to store the VCPU pointer, and ErrorEPC
as a temporary. It could be argued that this is abuse of those
registers, and DDATA_LO is known not to be usable on certain
implementations (Cavium Octeon). If KScratch registers are present, use
them instead.
We save & restore the temporary register in addition to the VCPU pointer
register when using a KScratch register for it, as it may be used for
normal host TLB handling too.
Signed-off-by: James Hogan <james.hogan@imgtec.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Radim Krčmář <rkrcmar@redhat.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: linux-mips@linux-mips.org
Cc: kvm@vger.kernel.org
---
arch/mips/include/asm/kvm_host.h | 1 +
arch/mips/kvm/entry.c | 94 +++++++++++++++++++++++++++++++++-------
arch/mips/kvm/mips.c | 4 ++
3 files changed, 84 insertions(+), 15 deletions(-)
diff --git a/arch/mips/include/asm/kvm_host.h b/arch/mips/include/asm/kvm_host.h
index 2e76e899079c..a80c3208b234 100644
--- a/arch/mips/include/asm/kvm_host.h
+++ b/arch/mips/include/asm/kvm_host.h
@@ -536,6 +536,7 @@ int kvm_arch_vcpu_dump_regs(struct kvm_vcpu *vcpu);
extern int kvm_mips_handle_exit(struct kvm_run *run, struct kvm_vcpu *vcpu);
/* Building of entry/exception code */
+int kvm_mips_entry_setup(void);
void *kvm_mips_build_vcpu_run(void *addr);
void *kvm_mips_build_exception(void *addr);
void *kvm_mips_build_exit(void *addr);
diff --git a/arch/mips/kvm/entry.c b/arch/mips/kvm/entry.c
index 6395bfa7e542..b6e7fd9f12f0 100644
--- a/arch/mips/kvm/entry.c
+++ b/arch/mips/kvm/entry.c
@@ -61,6 +61,9 @@
#define CALLFRAME_SIZ 32
+static unsigned int scratch_vcpu[2] = { C0_DDATA_LO };
+static unsigned int scratch_tmp[2] = { C0_ERROREPC };
+
enum label_id {
label_fpu_1 = 1,
label_msa_1,
@@ -79,6 +82,69 @@ static void *kvm_mips_build_ret_to_guest(void *addr);
static void *kvm_mips_build_ret_to_host(void *addr);
/**
+ * kvm_mips_entry_setup() - Perform global setup for entry code.
+ *
+ * Perform global setup for entry code, such as choosing a scratch register.
+ *
+ * Returns: 0 on success.
+ * -errno on failure.
+ */
+int kvm_mips_entry_setup(void)
+{
+ /*
+ * We prefer to use KScratchN registers if they are available over the
+ * defaults above, which may not work on all cores.
+ */
+ unsigned int kscratch_mask = cpu_data[0].kscratch_mask & 0xfc;
+
+ /* Pick a scratch register for storing VCPU */
+ if (kscratch_mask) {
+ scratch_vcpu[0] = 31;
+ scratch_vcpu[1] = ffs(kscratch_mask) - 1;
+ kscratch_mask &= ~BIT(scratch_vcpu[1]);
+ }
+
+ /* Pick a scratch register to use as a temp for saving state */
+ if (kscratch_mask) {
+ scratch_tmp[0] = 31;
+ scratch_tmp[1] = ffs(kscratch_mask) - 1;
+ kscratch_mask &= ~BIT(scratch_tmp[1]);
+ }
+
+ return 0;
+}
+
+static void kvm_mips_build_save_scratch(u32 **p, unsigned int tmp,
+ unsigned int frame)
+{
+ /* Save the VCPU scratch register value in cp0_epc of the stack frame */
+ uasm_i_mfc0(p, tmp, scratch_vcpu[0], scratch_vcpu[1]);
+ UASM_i_SW(p, tmp, offsetof(struct pt_regs, cp0_epc), frame);
+
+ /* Save the temp scratch register value in cp0_cause of stack frame */
+ if (scratch_tmp[0] == 31) {
+ uasm_i_mfc0(p, tmp, scratch_tmp[0], scratch_tmp[1]);
+ UASM_i_SW(p, tmp, offsetof(struct pt_regs, cp0_cause), frame);
+ }
+}
+
+static void kvm_mips_build_restore_scratch(u32 **p, unsigned int tmp,
+ unsigned int frame)
+{
+ /*
+ * Restore host scratch register values saved by
+ * kvm_mips_build_save_scratch().
+ */
+ UASM_i_LW(p, tmp, offsetof(struct pt_regs, cp0_epc), frame);
+ uasm_i_mtc0(p, tmp, scratch_vcpu[0], scratch_vcpu[1]);
+
+ if (scratch_tmp[0] == 31) {
+ UASM_i_LW(p, tmp, offsetof(struct pt_regs, cp0_cause), frame);
+ uasm_i_mtc0(p, tmp, scratch_tmp[0], scratch_tmp[1]);
+ }
+}
+
+/**
* kvm_mips_build_vcpu_run() - Assemble function to start running a guest VCPU.
* @addr: Address to start writing code.
*
@@ -120,12 +186,11 @@ void *kvm_mips_build_vcpu_run(void *addr)
uasm_i_mfc0(&p, V0, C0_STATUS);
UASM_i_SW(&p, V0, offsetof(struct pt_regs, cp0_status), K1);
- /* Save DDATA_LO, will be used to store pointer to vcpu */
- uasm_i_mfc0(&p, V1, C0_DDATA_LO);
- UASM_i_SW(&p, V1, offsetof(struct pt_regs, cp0_epc), K1);
+ /* Save scratch registers, will be used to store pointer to vcpu etc */
+ kvm_mips_build_save_scratch(&p, V1, K1);
- /* DDATA_LO has pointer to vcpu */
- uasm_i_mtc0(&p, A1, C0_DDATA_LO);
+ /* VCPU scratch register has pointer to vcpu */
+ uasm_i_mtc0(&p, A1, scratch_vcpu[0], scratch_vcpu[1]);
/* Offset into vcpu->arch */
uasm_i_addiu(&p, K1, A1, offsetof(struct kvm_vcpu, arch));
@@ -273,7 +338,7 @@ void *kvm_mips_build_exception(void *addr)
u32 *p = addr;
/* Save guest k0 */
- uasm_i_mtc0(&p, K0, C0_ERROREPC);
+ uasm_i_mtc0(&p, K0, scratch_tmp[0], scratch_tmp[1]);
uasm_i_ehb(&p);
/* Get EBASE */
@@ -321,8 +386,8 @@ void *kvm_mips_build_exit(void *addr)
* does something that causes a trap to kernel mode.
*/
- /* Get the VCPU pointer from DDATA_LO */
- uasm_i_mfc0(&p, K1, C0_DDATA_LO);
+ /* Get the VCPU pointer from the scratch register */
+ uasm_i_mfc0(&p, K1, scratch_vcpu[0], scratch_vcpu[1]);
uasm_i_addiu(&p, K1, K1, offsetof(struct kvm_vcpu, arch));
/* Start saving Guest context to VCPU */
@@ -341,7 +406,7 @@ void *kvm_mips_build_exit(void *addr)
UASM_i_SW(&p, T0, offsetof(struct kvm_vcpu_arch, lo), K1);
/* Finally save guest k0/k1 to VCPU */
- uasm_i_mfc0(&p, T0, C0_ERROREPC);
+ uasm_i_mfc0(&p, T0, scratch_tmp[0], scratch_tmp[1]);
UASM_i_SW(&p, T0, offsetof(struct kvm_vcpu_arch, gprs[K0]), K1);
/* Get GUEST k1 and save it in VCPU */
@@ -354,7 +419,7 @@ void *kvm_mips_build_exit(void *addr)
/* Now that context has been saved, we can use other registers */
/* Restore vcpu */
- uasm_i_mfc0(&p, A1, C0_DDATA_LO);
+ uasm_i_mfc0(&p, A1, scratch_vcpu[0], scratch_vcpu[1]);
uasm_i_move(&p, S1, A1);
/* Restore run (vcpu->run) */
@@ -446,9 +511,8 @@ void *kvm_mips_build_exit(void *addr)
* kernel entries are marked GLOBAL, need to verify
*/
- /* Restore host DDATA_LO */
- UASM_i_LW(&p, K0, offsetof(struct pt_regs, cp0_epc), SP);
- uasm_i_mtc0(&p, K0, C0_DDATA_LO);
+ /* Restore host scratch registers, as we'll have clobbered them */
+ kvm_mips_build_restore_scratch(&p, K0, SP);
/* Restore RDHWR access */
UASM_i_LA_mostly(&p, K0, (long)&hwrena);
@@ -536,8 +600,8 @@ static void *kvm_mips_build_ret_to_guest(void *addr)
{
u32 *p = addr;
- /* Put the saved pointer to vcpu (s1) back into the DDATA_LO Register */
- uasm_i_mtc0(&p, S1, C0_DDATA_LO);
+ /* Put the saved pointer to vcpu (s1) back into the scratch register */
+ uasm_i_mtc0(&p, S1, scratch_vcpu[0], scratch_vcpu[1]);
/* Load up the Guest EBASE to minimize the window where BEV is set */
UASM_i_LW(&p, T0, offsetof(struct kvm_vcpu_arch, guest_ebase), K1);
diff --git a/arch/mips/kvm/mips.c b/arch/mips/kvm/mips.c
index f809ad84196d..e45b505e2ad7 100644
--- a/arch/mips/kvm/mips.c
+++ b/arch/mips/kvm/mips.c
@@ -1775,6 +1775,10 @@ static int __init kvm_mips_init(void)
{
int ret;
+ ret = kvm_mips_entry_setup();
+ if (ret)
+ return ret;
+
ret = kvm_init(NULL, sizeof(struct kvm_vcpu), 0, THIS_MODULE);
if (ret)
--
2.4.10
^ permalink raw reply related [flat|nested] 36+ messages in thread* [PATCH 12/14] MIPS: KVM: Dynamically choose scratch registers
@ 2016-06-23 16:34 ` James Hogan
0 siblings, 0 replies; 36+ messages in thread
From: James Hogan @ 2016-06-23 16:34 UTC (permalink / raw)
To: Paolo Bonzini, Radim Krčmář, Ralf Baechle
Cc: James Hogan, linux-mips, kvm
Scratch cop0 registers are needed by KVM to be able to save/restore all
the GPRs, including k0/k1, and for storing the VCPU pointer. However no
registers are universally suitable for these purposes, so the decision
should be made at runtime.
Until now, we've used DDATA_LO to store the VCPU pointer, and ErrorEPC
as a temporary. It could be argued that this is abuse of those
registers, and DDATA_LO is known not to be usable on certain
implementations (Cavium Octeon). If KScratch registers are present, use
them instead.
We save & restore the temporary register in addition to the VCPU pointer
register when using a KScratch register for it, as it may be used for
normal host TLB handling too.
Signed-off-by: James Hogan <james.hogan@imgtec.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Radim Krčmář <rkrcmar@redhat.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: linux-mips@linux-mips.org
Cc: kvm@vger.kernel.org
---
arch/mips/include/asm/kvm_host.h | 1 +
arch/mips/kvm/entry.c | 94 +++++++++++++++++++++++++++++++++-------
arch/mips/kvm/mips.c | 4 ++
3 files changed, 84 insertions(+), 15 deletions(-)
diff --git a/arch/mips/include/asm/kvm_host.h b/arch/mips/include/asm/kvm_host.h
index 2e76e899079c..a80c3208b234 100644
--- a/arch/mips/include/asm/kvm_host.h
+++ b/arch/mips/include/asm/kvm_host.h
@@ -536,6 +536,7 @@ int kvm_arch_vcpu_dump_regs(struct kvm_vcpu *vcpu);
extern int kvm_mips_handle_exit(struct kvm_run *run, struct kvm_vcpu *vcpu);
/* Building of entry/exception code */
+int kvm_mips_entry_setup(void);
void *kvm_mips_build_vcpu_run(void *addr);
void *kvm_mips_build_exception(void *addr);
void *kvm_mips_build_exit(void *addr);
diff --git a/arch/mips/kvm/entry.c b/arch/mips/kvm/entry.c
index 6395bfa7e542..b6e7fd9f12f0 100644
--- a/arch/mips/kvm/entry.c
+++ b/arch/mips/kvm/entry.c
@@ -61,6 +61,9 @@
#define CALLFRAME_SIZ 32
+static unsigned int scratch_vcpu[2] = { C0_DDATA_LO };
+static unsigned int scratch_tmp[2] = { C0_ERROREPC };
+
enum label_id {
label_fpu_1 = 1,
label_msa_1,
@@ -79,6 +82,69 @@ static void *kvm_mips_build_ret_to_guest(void *addr);
static void *kvm_mips_build_ret_to_host(void *addr);
/**
+ * kvm_mips_entry_setup() - Perform global setup for entry code.
+ *
+ * Perform global setup for entry code, such as choosing a scratch register.
+ *
+ * Returns: 0 on success.
+ * -errno on failure.
+ */
+int kvm_mips_entry_setup(void)
+{
+ /*
+ * We prefer to use KScratchN registers if they are available over the
+ * defaults above, which may not work on all cores.
+ */
+ unsigned int kscratch_mask = cpu_data[0].kscratch_mask & 0xfc;
+
+ /* Pick a scratch register for storing VCPU */
+ if (kscratch_mask) {
+ scratch_vcpu[0] = 31;
+ scratch_vcpu[1] = ffs(kscratch_mask) - 1;
+ kscratch_mask &= ~BIT(scratch_vcpu[1]);
+ }
+
+ /* Pick a scratch register to use as a temp for saving state */
+ if (kscratch_mask) {
+ scratch_tmp[0] = 31;
+ scratch_tmp[1] = ffs(kscratch_mask) - 1;
+ kscratch_mask &= ~BIT(scratch_tmp[1]);
+ }
+
+ return 0;
+}
+
+static void kvm_mips_build_save_scratch(u32 **p, unsigned int tmp,
+ unsigned int frame)
+{
+ /* Save the VCPU scratch register value in cp0_epc of the stack frame */
+ uasm_i_mfc0(p, tmp, scratch_vcpu[0], scratch_vcpu[1]);
+ UASM_i_SW(p, tmp, offsetof(struct pt_regs, cp0_epc), frame);
+
+ /* Save the temp scratch register value in cp0_cause of stack frame */
+ if (scratch_tmp[0] == 31) {
+ uasm_i_mfc0(p, tmp, scratch_tmp[0], scratch_tmp[1]);
+ UASM_i_SW(p, tmp, offsetof(struct pt_regs, cp0_cause), frame);
+ }
+}
+
+static void kvm_mips_build_restore_scratch(u32 **p, unsigned int tmp,
+ unsigned int frame)
+{
+ /*
+ * Restore host scratch register values saved by
+ * kvm_mips_build_save_scratch().
+ */
+ UASM_i_LW(p, tmp, offsetof(struct pt_regs, cp0_epc), frame);
+ uasm_i_mtc0(p, tmp, scratch_vcpu[0], scratch_vcpu[1]);
+
+ if (scratch_tmp[0] == 31) {
+ UASM_i_LW(p, tmp, offsetof(struct pt_regs, cp0_cause), frame);
+ uasm_i_mtc0(p, tmp, scratch_tmp[0], scratch_tmp[1]);
+ }
+}
+
+/**
* kvm_mips_build_vcpu_run() - Assemble function to start running a guest VCPU.
* @addr: Address to start writing code.
*
@@ -120,12 +186,11 @@ void *kvm_mips_build_vcpu_run(void *addr)
uasm_i_mfc0(&p, V0, C0_STATUS);
UASM_i_SW(&p, V0, offsetof(struct pt_regs, cp0_status), K1);
- /* Save DDATA_LO, will be used to store pointer to vcpu */
- uasm_i_mfc0(&p, V1, C0_DDATA_LO);
- UASM_i_SW(&p, V1, offsetof(struct pt_regs, cp0_epc), K1);
+ /* Save scratch registers, will be used to store pointer to vcpu etc */
+ kvm_mips_build_save_scratch(&p, V1, K1);
- /* DDATA_LO has pointer to vcpu */
- uasm_i_mtc0(&p, A1, C0_DDATA_LO);
+ /* VCPU scratch register has pointer to vcpu */
+ uasm_i_mtc0(&p, A1, scratch_vcpu[0], scratch_vcpu[1]);
/* Offset into vcpu->arch */
uasm_i_addiu(&p, K1, A1, offsetof(struct kvm_vcpu, arch));
@@ -273,7 +338,7 @@ void *kvm_mips_build_exception(void *addr)
u32 *p = addr;
/* Save guest k0 */
- uasm_i_mtc0(&p, K0, C0_ERROREPC);
+ uasm_i_mtc0(&p, K0, scratch_tmp[0], scratch_tmp[1]);
uasm_i_ehb(&p);
/* Get EBASE */
@@ -321,8 +386,8 @@ void *kvm_mips_build_exit(void *addr)
* does something that causes a trap to kernel mode.
*/
- /* Get the VCPU pointer from DDATA_LO */
- uasm_i_mfc0(&p, K1, C0_DDATA_LO);
+ /* Get the VCPU pointer from the scratch register */
+ uasm_i_mfc0(&p, K1, scratch_vcpu[0], scratch_vcpu[1]);
uasm_i_addiu(&p, K1, K1, offsetof(struct kvm_vcpu, arch));
/* Start saving Guest context to VCPU */
@@ -341,7 +406,7 @@ void *kvm_mips_build_exit(void *addr)
UASM_i_SW(&p, T0, offsetof(struct kvm_vcpu_arch, lo), K1);
/* Finally save guest k0/k1 to VCPU */
- uasm_i_mfc0(&p, T0, C0_ERROREPC);
+ uasm_i_mfc0(&p, T0, scratch_tmp[0], scratch_tmp[1]);
UASM_i_SW(&p, T0, offsetof(struct kvm_vcpu_arch, gprs[K0]), K1);
/* Get GUEST k1 and save it in VCPU */
@@ -354,7 +419,7 @@ void *kvm_mips_build_exit(void *addr)
/* Now that context has been saved, we can use other registers */
/* Restore vcpu */
- uasm_i_mfc0(&p, A1, C0_DDATA_LO);
+ uasm_i_mfc0(&p, A1, scratch_vcpu[0], scratch_vcpu[1]);
uasm_i_move(&p, S1, A1);
/* Restore run (vcpu->run) */
@@ -446,9 +511,8 @@ void *kvm_mips_build_exit(void *addr)
* kernel entries are marked GLOBAL, need to verify
*/
- /* Restore host DDATA_LO */
- UASM_i_LW(&p, K0, offsetof(struct pt_regs, cp0_epc), SP);
- uasm_i_mtc0(&p, K0, C0_DDATA_LO);
+ /* Restore host scratch registers, as we'll have clobbered them */
+ kvm_mips_build_restore_scratch(&p, K0, SP);
/* Restore RDHWR access */
UASM_i_LA_mostly(&p, K0, (long)&hwrena);
@@ -536,8 +600,8 @@ static void *kvm_mips_build_ret_to_guest(void *addr)
{
u32 *p = addr;
- /* Put the saved pointer to vcpu (s1) back into the DDATA_LO Register */
- uasm_i_mtc0(&p, S1, C0_DDATA_LO);
+ /* Put the saved pointer to vcpu (s1) back into the scratch register */
+ uasm_i_mtc0(&p, S1, scratch_vcpu[0], scratch_vcpu[1]);
/* Load up the Guest EBASE to minimize the window where BEV is set */
UASM_i_LW(&p, T0, offsetof(struct kvm_vcpu_arch, guest_ebase), K1);
diff --git a/arch/mips/kvm/mips.c b/arch/mips/kvm/mips.c
index f809ad84196d..e45b505e2ad7 100644
--- a/arch/mips/kvm/mips.c
+++ b/arch/mips/kvm/mips.c
@@ -1775,6 +1775,10 @@ static int __init kvm_mips_init(void)
{
int ret;
+ ret = kvm_mips_entry_setup();
+ if (ret)
+ return ret;
+
ret = kvm_init(NULL, sizeof(struct kvm_vcpu), 0, THIS_MODULE);
if (ret)
--
2.4.10
^ permalink raw reply related [flat|nested] 36+ messages in thread
* [PATCH 13/14] MIPS: KVM: Relative branch to common exit handler
@ 2016-06-23 16:34 ` James Hogan
0 siblings, 0 replies; 36+ messages in thread
From: James Hogan @ 2016-06-23 16:34 UTC (permalink / raw)
To: Paolo Bonzini, Radim Krčmář, Ralf Baechle
Cc: James Hogan, linux-mips, kvm
Use a relative branch to get from the individual exception vectors to
the common guest exit handler, rather than loading the address of the
exit handler and jumping to it.
This is made easier due to the fact we are now generating the entry code
dynamically. This will also allow the exception code to be further
reduced in future patches.
Signed-off-by: James Hogan <james.hogan@imgtec.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Radim Krčmář <rkrcmar@redhat.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: linux-mips@linux-mips.org
Cc: kvm@vger.kernel.org
---
arch/mips/include/asm/kvm_host.h | 2 +-
arch/mips/kvm/entry.c | 23 +++++++++++++++++------
arch/mips/kvm/mips.c | 12 +++++++-----
3 files changed, 25 insertions(+), 12 deletions(-)
diff --git a/arch/mips/include/asm/kvm_host.h b/arch/mips/include/asm/kvm_host.h
index a80c3208b234..b32785543787 100644
--- a/arch/mips/include/asm/kvm_host.h
+++ b/arch/mips/include/asm/kvm_host.h
@@ -538,7 +538,7 @@ extern int kvm_mips_handle_exit(struct kvm_run *run, struct kvm_vcpu *vcpu);
/* Building of entry/exception code */
int kvm_mips_entry_setup(void);
void *kvm_mips_build_vcpu_run(void *addr);
-void *kvm_mips_build_exception(void *addr);
+void *kvm_mips_build_exception(void *addr, void *handler);
void *kvm_mips_build_exit(void *addr);
/* FPU/MSA context management */
diff --git a/arch/mips/kvm/entry.c b/arch/mips/kvm/entry.c
index b6e7fd9f12f0..fb2cbf653474 100644
--- a/arch/mips/kvm/entry.c
+++ b/arch/mips/kvm/entry.c
@@ -69,12 +69,14 @@ enum label_id {
label_msa_1,
label_return_to_host,
label_kernel_asid,
+ label_exit_common,
};
UASM_L_LA(_fpu_1)
UASM_L_LA(_msa_1)
UASM_L_LA(_return_to_host)
UASM_L_LA(_kernel_asid)
+UASM_L_LA(_exit_common)
static void *kvm_mips_build_enter_guest(void *addr);
static void *kvm_mips_build_ret_from_exit(void *addr);
@@ -327,15 +329,23 @@ static void *kvm_mips_build_enter_guest(void *addr)
/**
* kvm_mips_build_exception() - Assemble first level guest exception handler.
* @addr: Address to start writing code.
+ * @handler: Address of common handler (within range of @addr).
*
* Assemble exception vector code for guest execution. The generated vector will
- * jump to the common exception handler generated by kvm_mips_build_exit().
+ * branch to the common exception handler generated by kvm_mips_build_exit().
*
* Returns: Next address after end of written function.
*/
-void *kvm_mips_build_exception(void *addr)
+void *kvm_mips_build_exception(void *addr, void *handler)
{
u32 *p = addr;
+ struct uasm_label labels[2];
+ struct uasm_reloc relocs[2];
+ struct uasm_label *l = labels;
+ struct uasm_reloc *r = relocs;
+
+ memset(labels, 0, sizeof(labels));
+ memset(relocs, 0, sizeof(relocs));
/* Save guest k0 */
uasm_i_mtc0(&p, K0, scratch_tmp[0], scratch_tmp[1]);
@@ -349,12 +359,13 @@ void *kvm_mips_build_exception(void *addr)
/* Save k1 @ offset 0x3000 */
UASM_i_SW(&p, K1, 0x3000, K0);
- /* Exception handler is installed @ offset 0x2000 */
- uasm_i_addiu(&p, K0, K0, 0x2000);
- /* Jump to the function */
- uasm_i_jr(&p, K0);
+ /* Branch to the common handler */
+ uasm_il_b(&p, &r, label_exit_common);
uasm_i_nop(&p);
+ uasm_l_exit_common(&l, handler);
+ uasm_resolve_relocs(relocs, labels);
+
return p;
}
diff --git a/arch/mips/kvm/mips.c b/arch/mips/kvm/mips.c
index e45b505e2ad7..a62267f6fb07 100644
--- a/arch/mips/kvm/mips.c
+++ b/arch/mips/kvm/mips.c
@@ -265,7 +265,7 @@ static inline void dump_handler(const char *symbol, void *start, void *end)
struct kvm_vcpu *kvm_arch_vcpu_create(struct kvm *kvm, unsigned int id)
{
int err, size;
- void *gebase, *p;
+ void *gebase, *p, *handler;
int i;
struct kvm_vcpu *vcpu = kzalloc(sizeof(struct kvm_vcpu), GFP_KERNEL);
@@ -304,22 +304,24 @@ struct kvm_vcpu *kvm_arch_vcpu_create(struct kvm *kvm, unsigned int id)
vcpu->arch.guest_ebase = gebase;
/* Build guest exception vectors dynamically in unmapped memory */
+ handler = gebase + 0x2000;
/* TLB Refill, EXL = 0 */
- kvm_mips_build_exception(gebase);
+ kvm_mips_build_exception(gebase, handler);
/* General Exception Entry point */
- kvm_mips_build_exception(gebase + 0x180);
+ kvm_mips_build_exception(gebase + 0x180, handler);
/* For vectored interrupts poke the exception code @ all offsets 0-7 */
for (i = 0; i < 8; i++) {
kvm_debug("L1 Vectored handler @ %p\n",
gebase + 0x200 + (i * VECTORSPACING));
- kvm_mips_build_exception(gebase + 0x200 + i * VECTORSPACING);
+ kvm_mips_build_exception(gebase + 0x200 + i * VECTORSPACING,
+ handler);
}
/* General exit handler */
- p = gebase + 0x2000;
+ p = handler;
p = kvm_mips_build_exit(p);
/* Guest entry routine */
--
2.4.10
^ permalink raw reply related [flat|nested] 36+ messages in thread* [PATCH 13/14] MIPS: KVM: Relative branch to common exit handler
@ 2016-06-23 16:34 ` James Hogan
0 siblings, 0 replies; 36+ messages in thread
From: James Hogan @ 2016-06-23 16:34 UTC (permalink / raw)
To: Paolo Bonzini, Radim Krčmář, Ralf Baechle
Cc: James Hogan, linux-mips, kvm
Use a relative branch to get from the individual exception vectors to
the common guest exit handler, rather than loading the address of the
exit handler and jumping to it.
This is made easier due to the fact we are now generating the entry code
dynamically. This will also allow the exception code to be further
reduced in future patches.
Signed-off-by: James Hogan <james.hogan@imgtec.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Radim Krčmář <rkrcmar@redhat.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: linux-mips@linux-mips.org
Cc: kvm@vger.kernel.org
---
arch/mips/include/asm/kvm_host.h | 2 +-
arch/mips/kvm/entry.c | 23 +++++++++++++++++------
arch/mips/kvm/mips.c | 12 +++++++-----
3 files changed, 25 insertions(+), 12 deletions(-)
diff --git a/arch/mips/include/asm/kvm_host.h b/arch/mips/include/asm/kvm_host.h
index a80c3208b234..b32785543787 100644
--- a/arch/mips/include/asm/kvm_host.h
+++ b/arch/mips/include/asm/kvm_host.h
@@ -538,7 +538,7 @@ extern int kvm_mips_handle_exit(struct kvm_run *run, struct kvm_vcpu *vcpu);
/* Building of entry/exception code */
int kvm_mips_entry_setup(void);
void *kvm_mips_build_vcpu_run(void *addr);
-void *kvm_mips_build_exception(void *addr);
+void *kvm_mips_build_exception(void *addr, void *handler);
void *kvm_mips_build_exit(void *addr);
/* FPU/MSA context management */
diff --git a/arch/mips/kvm/entry.c b/arch/mips/kvm/entry.c
index b6e7fd9f12f0..fb2cbf653474 100644
--- a/arch/mips/kvm/entry.c
+++ b/arch/mips/kvm/entry.c
@@ -69,12 +69,14 @@ enum label_id {
label_msa_1,
label_return_to_host,
label_kernel_asid,
+ label_exit_common,
};
UASM_L_LA(_fpu_1)
UASM_L_LA(_msa_1)
UASM_L_LA(_return_to_host)
UASM_L_LA(_kernel_asid)
+UASM_L_LA(_exit_common)
static void *kvm_mips_build_enter_guest(void *addr);
static void *kvm_mips_build_ret_from_exit(void *addr);
@@ -327,15 +329,23 @@ static void *kvm_mips_build_enter_guest(void *addr)
/**
* kvm_mips_build_exception() - Assemble first level guest exception handler.
* @addr: Address to start writing code.
+ * @handler: Address of common handler (within range of @addr).
*
* Assemble exception vector code for guest execution. The generated vector will
- * jump to the common exception handler generated by kvm_mips_build_exit().
+ * branch to the common exception handler generated by kvm_mips_build_exit().
*
* Returns: Next address after end of written function.
*/
-void *kvm_mips_build_exception(void *addr)
+void *kvm_mips_build_exception(void *addr, void *handler)
{
u32 *p = addr;
+ struct uasm_label labels[2];
+ struct uasm_reloc relocs[2];
+ struct uasm_label *l = labels;
+ struct uasm_reloc *r = relocs;
+
+ memset(labels, 0, sizeof(labels));
+ memset(relocs, 0, sizeof(relocs));
/* Save guest k0 */
uasm_i_mtc0(&p, K0, scratch_tmp[0], scratch_tmp[1]);
@@ -349,12 +359,13 @@ void *kvm_mips_build_exception(void *addr)
/* Save k1 @ offset 0x3000 */
UASM_i_SW(&p, K1, 0x3000, K0);
- /* Exception handler is installed @ offset 0x2000 */
- uasm_i_addiu(&p, K0, K0, 0x2000);
- /* Jump to the function */
- uasm_i_jr(&p, K0);
+ /* Branch to the common handler */
+ uasm_il_b(&p, &r, label_exit_common);
uasm_i_nop(&p);
+ uasm_l_exit_common(&l, handler);
+ uasm_resolve_relocs(relocs, labels);
+
return p;
}
diff --git a/arch/mips/kvm/mips.c b/arch/mips/kvm/mips.c
index e45b505e2ad7..a62267f6fb07 100644
--- a/arch/mips/kvm/mips.c
+++ b/arch/mips/kvm/mips.c
@@ -265,7 +265,7 @@ static inline void dump_handler(const char *symbol, void *start, void *end)
struct kvm_vcpu *kvm_arch_vcpu_create(struct kvm *kvm, unsigned int id)
{
int err, size;
- void *gebase, *p;
+ void *gebase, *p, *handler;
int i;
struct kvm_vcpu *vcpu = kzalloc(sizeof(struct kvm_vcpu), GFP_KERNEL);
@@ -304,22 +304,24 @@ struct kvm_vcpu *kvm_arch_vcpu_create(struct kvm *kvm, unsigned int id)
vcpu->arch.guest_ebase = gebase;
/* Build guest exception vectors dynamically in unmapped memory */
+ handler = gebase + 0x2000;
/* TLB Refill, EXL = 0 */
- kvm_mips_build_exception(gebase);
+ kvm_mips_build_exception(gebase, handler);
/* General Exception Entry point */
- kvm_mips_build_exception(gebase + 0x180);
+ kvm_mips_build_exception(gebase + 0x180, handler);
/* For vectored interrupts poke the exception code @ all offsets 0-7 */
for (i = 0; i < 8; i++) {
kvm_debug("L1 Vectored handler @ %p\n",
gebase + 0x200 + (i * VECTORSPACING));
- kvm_mips_build_exception(gebase + 0x200 + i * VECTORSPACING);
+ kvm_mips_build_exception(gebase + 0x200 + i * VECTORSPACING,
+ handler);
}
/* General exit handler */
- p = gebase + 0x2000;
+ p = handler;
p = kvm_mips_build_exit(p);
/* Guest entry routine */
--
2.4.10
^ permalink raw reply related [flat|nested] 36+ messages in thread
* [PATCH 14/14] MIPS: KVM: Save k0 straight into VCPU structure
@ 2016-06-23 16:34 ` James Hogan
0 siblings, 0 replies; 36+ messages in thread
From: James Hogan @ 2016-06-23 16:34 UTC (permalink / raw)
To: Paolo Bonzini, Radim Krčmář, Ralf Baechle
Cc: James Hogan, linux-mips, kvm
Currently on a guest exception the guest's k0 register is saved to the
scratch temp register and the guest k1 saved to the exception base
address + 0x3000 using k0 to extract the Exception Base field of the
EBase register and as the base operand to the store. Both are then
copied into the VCPU structure after the other general purpose registers
have been saved there.
This bouncing to exception base + 0x3000 is not actually necessary as
the VCPU pointer can be determined and written through just as easily
with only a single spare register. The VCPU pointer is already needed in
k1 for saving the other GP registers, so lets save the guest k0 register
straight into the VCPU structure through k1, first saving k1 into the
scratch temp register instead of k0.
This could potentially pave the way for having a single exception base
area for use by all guests.
The ehb after saving the k register to the scratch temp register is also
delayed until just before it needs to be read back.
Signed-off-by: James Hogan <james.hogan@imgtec.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Radim Krčmář <rkrcmar@redhat.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: linux-mips@linux-mips.org
Cc: kvm@vger.kernel.org
---
arch/mips/kvm/entry.c | 37 +++++++++++++++----------------------
1 file changed, 15 insertions(+), 22 deletions(-)
diff --git a/arch/mips/kvm/entry.c b/arch/mips/kvm/entry.c
index fb2cbf653474..de8b6ec5573f 100644
--- a/arch/mips/kvm/entry.c
+++ b/arch/mips/kvm/entry.c
@@ -347,17 +347,15 @@ void *kvm_mips_build_exception(void *addr, void *handler)
memset(labels, 0, sizeof(labels));
memset(relocs, 0, sizeof(relocs));
- /* Save guest k0 */
- uasm_i_mtc0(&p, K0, scratch_tmp[0], scratch_tmp[1]);
- uasm_i_ehb(&p);
+ /* Save guest k1 into scratch register */
+ uasm_i_mtc0(&p, K1, scratch_tmp[0], scratch_tmp[1]);
- /* Get EBASE */
- uasm_i_mfc0(&p, K0, C0_EBASE);
- /* Get rid of CPUNum */
- uasm_i_srl(&p, K0, K0, 10);
- uasm_i_sll(&p, K0, K0, 10);
- /* Save k1 @ offset 0x3000 */
- UASM_i_SW(&p, K1, 0x3000, K0);
+ /* Get the VCPU pointer from the VCPU scratch register */
+ uasm_i_mfc0(&p, K1, scratch_vcpu[0], scratch_vcpu[1]);
+ uasm_i_addiu(&p, K1, K1, offsetof(struct kvm_vcpu, arch));
+
+ /* Save guest k0 into VCPU structure */
+ UASM_i_SW(&p, K0, offsetof(struct kvm_vcpu_arch, gprs[K0]), K1);
/* Branch to the common handler */
uasm_il_b(&p, &r, label_exit_common);
@@ -395,12 +393,13 @@ void *kvm_mips_build_exit(void *addr)
/*
* Generic Guest exception handler. We end up here when the guest
* does something that causes a trap to kernel mode.
+ *
+ * Both k0/k1 registers will have already been saved (k0 into the vcpu
+ * structure, and k1 into the scratch_tmp register).
+ *
+ * The k1 register will already contain the kvm_vcpu_arch pointer.
*/
- /* Get the VCPU pointer from the scratch register */
- uasm_i_mfc0(&p, K1, scratch_vcpu[0], scratch_vcpu[1]);
- uasm_i_addiu(&p, K1, K1, offsetof(struct kvm_vcpu, arch));
-
/* Start saving Guest context to VCPU */
for (i = 0; i < 32; ++i) {
/* Guest k0/k1 saved later */
@@ -416,15 +415,9 @@ void *kvm_mips_build_exit(void *addr)
uasm_i_mflo(&p, T0);
UASM_i_SW(&p, T0, offsetof(struct kvm_vcpu_arch, lo), K1);
- /* Finally save guest k0/k1 to VCPU */
+ /* Finally save guest k1 to VCPU */
+ uasm_i_ehb(&p);
uasm_i_mfc0(&p, T0, scratch_tmp[0], scratch_tmp[1]);
- UASM_i_SW(&p, T0, offsetof(struct kvm_vcpu_arch, gprs[K0]), K1);
-
- /* Get GUEST k1 and save it in VCPU */
- uasm_i_addiu(&p, T1, ZERO, ~0x2ff);
- uasm_i_mfc0(&p, T0, C0_EBASE);
- uasm_i_and(&p, T0, T0, T1);
- UASM_i_LW(&p, T0, 0x3000, T0);
UASM_i_SW(&p, T0, offsetof(struct kvm_vcpu_arch, gprs[K1]), K1);
/* Now that context has been saved, we can use other registers */
--
2.4.10
^ permalink raw reply related [flat|nested] 36+ messages in thread* [PATCH 14/14] MIPS: KVM: Save k0 straight into VCPU structure
@ 2016-06-23 16:34 ` James Hogan
0 siblings, 0 replies; 36+ messages in thread
From: James Hogan @ 2016-06-23 16:34 UTC (permalink / raw)
To: Paolo Bonzini, Radim Krčmář, Ralf Baechle
Cc: James Hogan, linux-mips, kvm
Currently on a guest exception the guest's k0 register is saved to the
scratch temp register and the guest k1 saved to the exception base
address + 0x3000 using k0 to extract the Exception Base field of the
EBase register and as the base operand to the store. Both are then
copied into the VCPU structure after the other general purpose registers
have been saved there.
This bouncing to exception base + 0x3000 is not actually necessary as
the VCPU pointer can be determined and written through just as easily
with only a single spare register. The VCPU pointer is already needed in
k1 for saving the other GP registers, so lets save the guest k0 register
straight into the VCPU structure through k1, first saving k1 into the
scratch temp register instead of k0.
This could potentially pave the way for having a single exception base
area for use by all guests.
The ehb after saving the k register to the scratch temp register is also
delayed until just before it needs to be read back.
Signed-off-by: James Hogan <james.hogan@imgtec.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Radim Krčmář <rkrcmar@redhat.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: linux-mips@linux-mips.org
Cc: kvm@vger.kernel.org
---
arch/mips/kvm/entry.c | 37 +++++++++++++++----------------------
1 file changed, 15 insertions(+), 22 deletions(-)
diff --git a/arch/mips/kvm/entry.c b/arch/mips/kvm/entry.c
index fb2cbf653474..de8b6ec5573f 100644
--- a/arch/mips/kvm/entry.c
+++ b/arch/mips/kvm/entry.c
@@ -347,17 +347,15 @@ void *kvm_mips_build_exception(void *addr, void *handler)
memset(labels, 0, sizeof(labels));
memset(relocs, 0, sizeof(relocs));
- /* Save guest k0 */
- uasm_i_mtc0(&p, K0, scratch_tmp[0], scratch_tmp[1]);
- uasm_i_ehb(&p);
+ /* Save guest k1 into scratch register */
+ uasm_i_mtc0(&p, K1, scratch_tmp[0], scratch_tmp[1]);
- /* Get EBASE */
- uasm_i_mfc0(&p, K0, C0_EBASE);
- /* Get rid of CPUNum */
- uasm_i_srl(&p, K0, K0, 10);
- uasm_i_sll(&p, K0, K0, 10);
- /* Save k1 @ offset 0x3000 */
- UASM_i_SW(&p, K1, 0x3000, K0);
+ /* Get the VCPU pointer from the VCPU scratch register */
+ uasm_i_mfc0(&p, K1, scratch_vcpu[0], scratch_vcpu[1]);
+ uasm_i_addiu(&p, K1, K1, offsetof(struct kvm_vcpu, arch));
+
+ /* Save guest k0 into VCPU structure */
+ UASM_i_SW(&p, K0, offsetof(struct kvm_vcpu_arch, gprs[K0]), K1);
/* Branch to the common handler */
uasm_il_b(&p, &r, label_exit_common);
@@ -395,12 +393,13 @@ void *kvm_mips_build_exit(void *addr)
/*
* Generic Guest exception handler. We end up here when the guest
* does something that causes a trap to kernel mode.
+ *
+ * Both k0/k1 registers will have already been saved (k0 into the vcpu
+ * structure, and k1 into the scratch_tmp register).
+ *
+ * The k1 register will already contain the kvm_vcpu_arch pointer.
*/
- /* Get the VCPU pointer from the scratch register */
- uasm_i_mfc0(&p, K1, scratch_vcpu[0], scratch_vcpu[1]);
- uasm_i_addiu(&p, K1, K1, offsetof(struct kvm_vcpu, arch));
-
/* Start saving Guest context to VCPU */
for (i = 0; i < 32; ++i) {
/* Guest k0/k1 saved later */
@@ -416,15 +415,9 @@ void *kvm_mips_build_exit(void *addr)
uasm_i_mflo(&p, T0);
UASM_i_SW(&p, T0, offsetof(struct kvm_vcpu_arch, lo), K1);
- /* Finally save guest k0/k1 to VCPU */
+ /* Finally save guest k1 to VCPU */
+ uasm_i_ehb(&p);
uasm_i_mfc0(&p, T0, scratch_tmp[0], scratch_tmp[1]);
- UASM_i_SW(&p, T0, offsetof(struct kvm_vcpu_arch, gprs[K0]), K1);
-
- /* Get GUEST k1 and save it in VCPU */
- uasm_i_addiu(&p, T1, ZERO, ~0x2ff);
- uasm_i_mfc0(&p, T0, C0_EBASE);
- uasm_i_and(&p, T0, T0, T1);
- UASM_i_LW(&p, T0, 0x3000, T0);
UASM_i_SW(&p, T0, offsetof(struct kvm_vcpu_arch, gprs[K1]), K1);
/* Now that context has been saved, we can use other registers */
--
2.4.10
^ permalink raw reply related [flat|nested] 36+ messages in thread
* Re: [PATCH 00/14] MIPS: KVM: Dynamically generate exception code
2016-06-23 16:34 ` James Hogan
` (14 preceding siblings ...)
(?)
@ 2016-07-05 13:48 ` Ralf Baechle
2016-07-05 13:53 ` Paolo Bonzini
-1 siblings, 1 reply; 36+ messages in thread
From: Ralf Baechle @ 2016-07-05 13:48 UTC (permalink / raw)
To: James Hogan; +Cc: Paolo Bonzini, Radim Krčmář, linux-mips, kvm
On Thu, Jun 23, 2016 at 05:34:33PM +0100, James Hogan wrote:
> These patches change the MIPS KVM exception entry code to be dynamically
> assembled by the MIPS "uasm" in-kernel assembler, directly into unmapped
> memory at run time by a new entry.c. Previously this code was statically
> assembled from locore.S at build time and later copied into unmapped
> memory at run time.
>
> Patches 1-5 add support for the necessary instructions to uasm.
>
> Patches 6-8 do the minimal-change conversion of locore.S to entry.c
> using uasm (I've used -M10% so the diff is shown as a file move).
>
> Patches 9-14 make some related improvements that are possible now that
> it is dynamically generated, such as avoiding messy runtime conditionals
> in assembly code, making use of KScratch registers when available, and
> simplifying the initial GP register save sequence & jump to common code.
>
> Ralf: Since the uasm patches (1-5) are needed for the later patches, I
> suggest these all go together via the KVM tree (on which the whole
> patchset is based), so Acks are welcome if they're okay with you.
Yes, please, so for the MIPS bits, that is patche 01..05:
Acked-by: Ralf Baechle <ralf@linux-mips.org>
Ralf
^ permalink raw reply [flat|nested] 36+ messages in thread* Re: [PATCH 00/14] MIPS: KVM: Dynamically generate exception code
2016-07-05 13:48 ` [PATCH 00/14] MIPS: KVM: Dynamically generate exception code Ralf Baechle
@ 2016-07-05 13:53 ` Paolo Bonzini
0 siblings, 0 replies; 36+ messages in thread
From: Paolo Bonzini @ 2016-07-05 13:53 UTC (permalink / raw)
To: Ralf Baechle, James Hogan; +Cc: Radim Krčmář, linux-mips, kvm
On 05/07/2016 15:48, Ralf Baechle wrote:
> On Thu, Jun 23, 2016 at 05:34:33PM +0100, James Hogan wrote:
>
>> These patches change the MIPS KVM exception entry code to be dynamically
>> assembled by the MIPS "uasm" in-kernel assembler, directly into unmapped
>> memory at run time by a new entry.c. Previously this code was statically
>> assembled from locore.S at build time and later copied into unmapped
>> memory at run time.
>>
>> Patches 1-5 add support for the necessary instructions to uasm.
>>
>> Patches 6-8 do the minimal-change conversion of locore.S to entry.c
>> using uasm (I've used -M10% so the diff is shown as a file move).
>>
>> Patches 9-14 make some related improvements that are possible now that
>> it is dynamically generated, such as avoiding messy runtime conditionals
>> in assembly code, making use of KScratch registers when available, and
>> simplifying the initial GP register save sequence & jump to common code.
>>
>> Ralf: Since the uasm patches (1-5) are needed for the later patches, I
>> suggest these all go together via the KVM tree (on which the whole
>> patchset is based), so Acks are welcome if they're okay with you.
>
> Yes, please, so for the MIPS bits, that is patche 01..05:
>
> Acked-by: Ralf Baechle <ralf@linux-mips.org>
Good, I'll apply the whole bunch.
Paolo
^ permalink raw reply [flat|nested] 36+ messages in thread