* [PATCH 0/5] armv7 BE kvm support
@ 2013-12-20 7:26 Victor Kamensky
2013-12-20 7:26 ` [PATCH 1/5] ARM: kvm: replace push and pop with stdmb and ldmia instrs to enable assembler.h inclusion Victor Kamensky
` (5 more replies)
0 siblings, 6 replies; 8+ messages in thread
From: Victor Kamensky @ 2013-12-20 7:26 UTC (permalink / raw)
To: linux-arm-kernel
Hi Guys,
Here is series that enables KVM support for V7 big endian kernels. Mostly
it deals with BE KVM host support. Marc Zyngier showed before with his patches
how BE guest could run on top LE host. With these patches BE guest runs on
top of BE host. If Marc's kvmtool is used with few additional changes I tested
that BE host could run LE guest. Also I verified that there were no
regressions in BE guest on top of LE host case.
Note that posted series covers only kernel side changes. The changes were
tested inside of bigger setup with additional changes in qemu and kvmtool.
I will post those changes separately in proper aliases but for completeness
sake Appendix A gives pointers to git repositories and branches with all
needed changes.
Please note first patch is not related to BE KVM per se. I've run
into an issue of conflicting 'push' identifier use while trying to include
assembler.h into KVM .S files. Details of an issue I observed covered in
Appendix B. The first patch is my take on solving it.
Victor Kamensky (5):
ARM: kvm: replace push and pop with stdmb and ldmia instrs to enable
assembler.h inclusion
ARM: fix KVM assembler files to work in BE case
ARM: kvm one_reg coproc set and get BE fixes
ARM: kvm vgic mmio should return data in BE format in BE case
ARM: kvm MMIO support BE host running LE code
arch/arm/include/asm/assembler.h | 7 +++
arch/arm/include/asm/kvm_asm.h | 4 +-
arch/arm/include/asm/kvm_emulate.h | 22 +++++++--
arch/arm/kvm/coproc.c | 94 ++++++++++++++++++++++++++++----------
arch/arm/kvm/init.S | 7 ++-
arch/arm/kvm/interrupts.S | 50 +++++++++++---------
arch/arm/kvm/interrupts_head.S | 61 +++++++++++++++----------
virt/kvm/arm/vgic.c | 4 +-
8 files changed, 168 insertions(+), 81 deletions(-)
--
1.8.1.4
Thanks,
Victor
Appendix A: Testing and Full Setup Description
----------------------------------------------
I) No mixed mode setup - i.e BE guest on BE host; and LE guest
on LE host tested to make sure no regressions.
KVM host and guest kernels:
TC2 on top of Linus 3.13-rc4 (this patch series):
git: git://git.linaro.org/people/victor.kamensky/linux-linaro-tracking-be.git
branch: armv7be-kvm-3.13-rc4
TC2 and Arndale on top of Linaro BE tree:
git: git://git.linaro.org/people/victor.kamensky/linux-linaro-tracking-be.git
branch: llct-be-20131216-kvm
- TC1 kernels used as guests
qemu:
git: git://git.linaro.org/people/victor.kamensky/qemu-be.git
branch: armv7be-v1
description: changes to run qemu on armeb target; and other
changes to work with be image on top of be host
kvmtool:
git: git://git.linaro.org/people/victor.kamensky/linux-linaro-tracking-be.git
branch: kvmtool-armv7be-v1
desciption: minimal changes to build kvmtool for armeb target; and
tiny change with virtio magic
II) Mixed mode setup all possible combinations within V7 (LE guest on BE host;
BE guest on LE host as Marc's setup tested to make sure no regressions) only
with kvmtool.
This work is based on Marc Zyngier's work that made BE guest to run on top
of LE host. For this setup special version of kvmtool should be used and
in addition I had to apply patch to guest kernel that would switch reading
virtio configs reads to be LE only, that is made on top of previous Rusty
Russell's changes. Effectively I just had to do very minor addition to make
LE guest to work on BE host, most of heavy lifting was done before by Marc.
KVM host kernels: as in previous setup
Guest TC1 kernels with LE virtio config patch:
git: git://git.linaro.org/people/victor.kamensky/linux-linaro-tracking-be.git
branch: virtio-leconfig-3.13-rc4
kvmtool:
git: git://git.linaro.org/people/victor.kamensky/linux-linaro-tracking-be.git
branch: kvmtool-mixed-v1
description: based on git://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms.git
branch kvm-arm64/kvmtool-be-on-le; adds missing include fix; above armeb target
build patches; and one fix related to BE mode
qemu:
git: git://git.linaro.org/people/victor.kamensky/qemu-be.git
branch: armv7be-leconfig-v1
description: change virtio-blk that so qemu could work with guest image
where virtio leconfig is made; note it does not work in mixed mode; to do
so qemu would need bunch of similar changes that Marc did in kvmtool
Appendix B: kvm asm file and asm/assembler.h file issue
-------------------------------------------------------
diff --git a/arch/arm/kvm/interrupts.S b/arch/arm/kvm/interrupts.S
index ddc1553..5d3b511 100644
--- a/arch/arm/kvm/interrupts.S
+++ b/arch/arm/kvm/interrupts.S
@@ -25,6 +25,7 @@
#include <asm/kvm_asm.h>
#include <asm/kvm_arm.h>
#include <asm/vfpmacros.h>
+#include <asm/assembler.h>
#include "interrupts_head.S"
.text
produce the following compilation errors:
/run/media/kamensky/wd/linaro/linux-linaro-core-tracking/092913/linux-linaro-tracking-be/arch/arm/kvm/interrupts.S: Assembler messages:
/run/media/kamensky/wd/linaro/linux-linaro-core-tracking/092913/linux-linaro-tracking-be/arch/arm/kvm/interrupts.S:51: Error: ARM register expected -- `lsr {r2,r3}'
/run/media/kamensky/wd/linaro/linux-linaro-core-tracking/092913/linux-linaro-tracking-be/arch/arm/kvm/interrupts.S:100: Error: ARM register expected -- `lsr {r2}'
/run/media/kamensky/wd/linaro/linux-linaro-core-tracking/092913/linux-linaro-tracking-be/arch/arm/kvm/interrupts.S:100: Error: ARM register expected -- `lsr {r4-r12}'
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH 1/5] ARM: kvm: replace push and pop with stdmb and ldmia instrs to enable assembler.h inclusion
2013-12-20 7:26 [PATCH 0/5] armv7 BE kvm support Victor Kamensky
@ 2013-12-20 7:26 ` Victor Kamensky
2014-01-07 16:01 ` Dave Martin
2013-12-20 7:26 ` [PATCH 2/5] ARM: fix KVM assembler files to work in BE case Victor Kamensky
` (4 subsequent siblings)
5 siblings, 1 reply; 8+ messages in thread
From: Victor Kamensky @ 2013-12-20 7:26 UTC (permalink / raw)
To: linux-arm-kernel
Before fix kvm interrupt.S and interrupt_head.S used push and pop assembler
instruction. It causes problem if <asm/assembler.h> file should be include. In
assembler.h "push" is defined as macro so it causes compilation errors like
this:
arch/arm/kvm/interrupts.S: Assembler messages:
arch/arm/kvm/interrupts.S:51: Error: ARM register expected -- `lsr {r2,r3}'
Solution implemented by this patch replaces all 'push {...}' with
'stdmb sp!, {...}' instruction; and all 'pop {...}' with 'ldmia sp!, {...}'.
Signed-off-by: Victor Kamensky <victor.kamensky@linaro.org>
---
arch/arm/kvm/interrupts.S | 38 +++++++++++++++++++-------------------
arch/arm/kvm/interrupts_head.S | 34 +++++++++++++++++-----------------
2 files changed, 36 insertions(+), 36 deletions(-)
diff --git a/arch/arm/kvm/interrupts.S b/arch/arm/kvm/interrupts.S
index ddc1553..df19133 100644
--- a/arch/arm/kvm/interrupts.S
+++ b/arch/arm/kvm/interrupts.S
@@ -47,7 +47,7 @@ __kvm_hyp_code_start:
* instead, ignoring the ipa value.
*/
ENTRY(__kvm_tlb_flush_vmid_ipa)
- push {r2, r3}
+ stmdb sp!, {r2, r3}
dsb ishst
add r0, r0, #KVM_VTTBR
@@ -62,7 +62,7 @@ ENTRY(__kvm_tlb_flush_vmid_ipa)
mcrr p15, 6, r2, r3, c2 @ Back to VMID #0
isb @ Not necessary if followed by eret
- pop {r2, r3}
+ ldmia sp!, {r2, r3}
bx lr
ENDPROC(__kvm_tlb_flush_vmid_ipa)
@@ -110,7 +110,7 @@ ENTRY(__kvm_vcpu_run)
#ifdef CONFIG_VFPv3
@ Set FPEXC_EN so the guest doesn't trap floating point instructions
VFPFMRX r2, FPEXC @ VMRS
- push {r2}
+ stmdb sp!, {r2}
orr r2, r2, #FPEXC_EN
VFPFMXR FPEXC, r2 @ VMSR
#endif
@@ -175,7 +175,7 @@ __kvm_vcpu_return:
after_vfp_restore:
@ Restore FPEXC_EN which we clobbered on entry
- pop {r2}
+ ldmia sp!, {r2}
VFPFMXR FPEXC, r2
#endif
@@ -260,7 +260,7 @@ ENTRY(kvm_call_hyp)
/* Handle undef, svc, pabt, or dabt by crashing with a user notice */
.macro bad_exception exception_code, panic_str
- push {r0-r2}
+ stmdb sp!, {r0-r2}
mrrc p15, 6, r0, r1, c2 @ Read VTTBR
lsr r1, r1, #16
ands r1, r1, #0xff
@@ -338,7 +338,7 @@ hyp_hvc:
* Getting here is either becuase of a trap from a guest or from calling
* HVC from the host kernel, which means "switch to Hyp mode".
*/
- push {r0, r1, r2}
+ stmdb sp!, {r0, r1, r2}
@ Check syndrome register
mrc p15, 4, r1, c5, c2, 0 @ HSR
@@ -361,11 +361,11 @@ hyp_hvc:
bne guest_trap @ Guest called HVC
host_switch_to_hyp:
- pop {r0, r1, r2}
+ ldmia sp!, {r0, r1, r2}
- push {lr}
+ stmdb sp!, {lr}
mrs lr, SPSR
- push {lr}
+ stmdb sp!, {lr}
mov lr, r0
mov r0, r1
@@ -375,9 +375,9 @@ host_switch_to_hyp:
THUMB( orr lr, #1)
blx lr @ Call the HYP function
- pop {lr}
+ ldmia sp!, {lr}
msr SPSR_csxf, lr
- pop {lr}
+ ldmia sp!, {lr}
eret
guest_trap:
@@ -418,7 +418,7 @@ guest_trap:
/* Preserve PAR */
mrrc p15, 0, r0, r1, c7 @ PAR
- push {r0, r1}
+ stmdb sp!, {r0, r1}
/* Resolve IPA using the xFAR */
mcr p15, 0, r2, c7, c8, 0 @ ATS1CPR
@@ -431,7 +431,7 @@ guest_trap:
orr r2, r2, r1, lsl #24
/* Restore PAR */
- pop {r0, r1}
+ ldmia sp!, {r0, r1}
mcrr p15, 0, r0, r1, c7 @ PAR
3: load_vcpu @ Load VCPU pointer to r0
@@ -440,10 +440,10 @@ guest_trap:
1: mov r1, #ARM_EXCEPTION_HVC
b __kvm_vcpu_return
-4: pop {r0, r1} @ Failed translation, return to guest
+4: ldmia sp!, {r0, r1} @ Failed translation, return to guest
mcrr p15, 0, r0, r1, c7 @ PAR
clrex
- pop {r0, r1, r2}
+ ldmia sp!, {r0, r1, r2}
eret
/*
@@ -455,7 +455,7 @@ guest_trap:
#ifdef CONFIG_VFPv3
switch_to_guest_vfp:
load_vcpu @ Load VCPU pointer to r0
- push {r3-r7}
+ stmdb sp!, {r3-r7}
@ NEON/VFP used. Turn on VFP access.
set_hcptr vmexit, (HCPTR_TCP(10) | HCPTR_TCP(11))
@@ -467,15 +467,15 @@ switch_to_guest_vfp:
add r7, r0, #VCPU_VFP_GUEST
restore_vfp_state r7
- pop {r3-r7}
- pop {r0-r2}
+ ldmia sp!, {r3-r7}
+ ldmia sp!, {r0-r2}
clrex
eret
#endif
.align
hyp_irq:
- push {r0, r1, r2}
+ stmdb sp!, {r0, r1, r2}
mov r1, #ARM_EXCEPTION_IRQ
load_vcpu @ Load VCPU pointer to r0
b __kvm_vcpu_return
diff --git a/arch/arm/kvm/interrupts_head.S b/arch/arm/kvm/interrupts_head.S
index 6f18695..c371db7 100644
--- a/arch/arm/kvm/interrupts_head.S
+++ b/arch/arm/kvm/interrupts_head.S
@@ -63,7 +63,7 @@ vcpu .req r0 @ vcpu pointer always in r0
mrs r2, SP_\mode
mrs r3, LR_\mode
mrs r4, SPSR_\mode
- push {r2, r3, r4}
+ stmdb sp!, {r2, r3, r4}
.endm
/*
@@ -73,13 +73,13 @@ vcpu .req r0 @ vcpu pointer always in r0
.macro save_host_regs
/* Hyp regs. Only ELR_hyp (SPSR_hyp already saved) */
mrs r2, ELR_hyp
- push {r2}
+ stmdb sp!, {r2}
/* usr regs */
- push {r4-r12} @ r0-r3 are always clobbered
+ stmdb sp!, {r4-r12} @ r0-r3 are always clobbered
mrs r2, SP_usr
mov r3, lr
- push {r2, r3}
+ stmdb sp!, {r2, r3}
push_host_regs_mode svc
push_host_regs_mode abt
@@ -95,11 +95,11 @@ vcpu .req r0 @ vcpu pointer always in r0
mrs r7, SP_fiq
mrs r8, LR_fiq
mrs r9, SPSR_fiq
- push {r2-r9}
+ stmdb sp!, {r2-r9}
.endm
.macro pop_host_regs_mode mode
- pop {r2, r3, r4}
+ ldmia sp!, {r2, r3, r4}
msr SP_\mode, r2
msr LR_\mode, r3
msr SPSR_\mode, r4
@@ -110,7 +110,7 @@ vcpu .req r0 @ vcpu pointer always in r0
* Clobbers all registers, in all modes, except r0 and r1.
*/
.macro restore_host_regs
- pop {r2-r9}
+ ldmia sp!, {r2-r9}
msr r8_fiq, r2
msr r9_fiq, r3
msr r10_fiq, r4
@@ -125,12 +125,12 @@ vcpu .req r0 @ vcpu pointer always in r0
pop_host_regs_mode abt
pop_host_regs_mode svc
- pop {r2, r3}
+ ldmia sp!, {r2, r3}
msr SP_usr, r2
mov lr, r3
- pop {r4-r12}
+ ldmia sp!, {r4-r12}
- pop {r2}
+ ldmia sp!, {r2}
msr ELR_hyp, r2
.endm
@@ -218,7 +218,7 @@ vcpu .req r0 @ vcpu pointer always in r0
add r2, vcpu, #VCPU_USR_REG(3)
stm r2, {r3-r12}
add r2, vcpu, #VCPU_USR_REG(0)
- pop {r3, r4, r5} @ r0, r1, r2
+ ldmia sp!, {r3, r4, r5} @ r0, r1, r2
stm r2, {r3, r4, r5}
mrs r2, SP_usr
mov r3, lr
@@ -258,7 +258,7 @@ vcpu .req r0 @ vcpu pointer always in r0
mrc p15, 2, r12, c0, c0, 0 @ CSSELR
.if \store_to_vcpu == 0
- push {r2-r12} @ Push CP15 registers
+ stmdb sp!, {r2-r12} @ Push CP15 registers
.else
str r2, [vcpu, #CP15_OFFSET(c1_SCTLR)]
str r3, [vcpu, #CP15_OFFSET(c1_CPACR)]
@@ -286,7 +286,7 @@ vcpu .req r0 @ vcpu pointer always in r0
mrc p15, 0, r12, c12, c0, 0 @ VBAR
.if \store_to_vcpu == 0
- push {r2-r12} @ Push CP15 registers
+ stmdb sp!, {r2-r12} @ Push CP15 registers
.else
str r2, [vcpu, #CP15_OFFSET(c13_CID)]
str r3, [vcpu, #CP15_OFFSET(c13_TID_URW)]
@@ -305,7 +305,7 @@ vcpu .req r0 @ vcpu pointer always in r0
mrrc p15, 0, r4, r5, c7 @ PAR
.if \store_to_vcpu == 0
- push {r2,r4-r5}
+ stmdb sp!, {r2,r4-r5}
.else
str r2, [vcpu, #CP15_OFFSET(c14_CNTKCTL)]
add r12, vcpu, #CP15_OFFSET(c7_PAR)
@@ -322,7 +322,7 @@ vcpu .req r0 @ vcpu pointer always in r0
*/
.macro write_cp15_state read_from_vcpu
.if \read_from_vcpu == 0
- pop {r2,r4-r5}
+ ldmia sp!, {r2,r4-r5}
.else
ldr r2, [vcpu, #CP15_OFFSET(c14_CNTKCTL)]
add r12, vcpu, #CP15_OFFSET(c7_PAR)
@@ -333,7 +333,7 @@ vcpu .req r0 @ vcpu pointer always in r0
mcrr p15, 0, r4, r5, c7 @ PAR
.if \read_from_vcpu == 0
- pop {r2-r12}
+ ldmia sp!, {r2-r12}
.else
ldr r2, [vcpu, #CP15_OFFSET(c13_CID)]
ldr r3, [vcpu, #CP15_OFFSET(c13_TID_URW)]
@@ -361,7 +361,7 @@ vcpu .req r0 @ vcpu pointer always in r0
mcr p15, 0, r12, c12, c0, 0 @ VBAR
.if \read_from_vcpu == 0
- pop {r2-r12}
+ ldmia sp!, {r2-r12}
.else
ldr r2, [vcpu, #CP15_OFFSET(c1_SCTLR)]
ldr r3, [vcpu, #CP15_OFFSET(c1_CPACR)]
--
1.8.1.4
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH 2/5] ARM: fix KVM assembler files to work in BE case
2013-12-20 7:26 [PATCH 0/5] armv7 BE kvm support Victor Kamensky
2013-12-20 7:26 ` [PATCH 1/5] ARM: kvm: replace push and pop with stdmb and ldmia instrs to enable assembler.h inclusion Victor Kamensky
@ 2013-12-20 7:26 ` Victor Kamensky
2013-12-20 7:26 ` [PATCH 3/5] ARM: kvm one_reg coproc set and get BE fixes Victor Kamensky
` (3 subsequent siblings)
5 siblings, 0 replies; 8+ messages in thread
From: Victor Kamensky @ 2013-12-20 7:26 UTC (permalink / raw)
To: linux-arm-kernel
ARM v7 KVM assembler files fixes to work in big endian mode:
vgic h/w registers are little endian; when asm code reads/writes from/to
them, it needs to do byteswap after/before. Byte swap code uses ARM_BE8
wrapper to add swap only if BIG_ENDIAN kernel is configured
mcrr and mrrc instructions take couple 32 bit registers as argument, one
is supposed to be high part of 64 bit value and another is low part of
64 bit. Typically those values are loaded/stored with ldrd and strd
instructions and those will load high and low parts in opposite register
depending on endianity. Introduce and use rr_lo_hi macro that swap
registers in BE mode when they are passed to mcrr and mrrc instructions.
function that returns 64 bit result __kvm_vcpu_run in couple registers
has to be adjusted for BE case.
Signed-off-by: Victor Kamensky <victor.kamensky@linaro.org>
---
arch/arm/include/asm/assembler.h | 7 +++++++
arch/arm/include/asm/kvm_asm.h | 4 ++--
arch/arm/kvm/init.S | 7 +++++--
arch/arm/kvm/interrupts.S | 12 +++++++++---
arch/arm/kvm/interrupts_head.S | 27 ++++++++++++++++++++-------
5 files changed, 43 insertions(+), 14 deletions(-)
diff --git a/arch/arm/include/asm/assembler.h b/arch/arm/include/asm/assembler.h
index 5c22851..ad1ad31 100644
--- a/arch/arm/include/asm/assembler.h
+++ b/arch/arm/include/asm/assembler.h
@@ -60,6 +60,13 @@
#define ARM_BE8(code...)
#endif
+/* swap pair of registers position depending on current endianity */
+#ifdef CONFIG_CPU_ENDIAN_BE8
+#define rr_lo_hi(a1, a2) a2, a1
+#else
+#define rr_lo_hi(a1, a2) a1, a2
+#endif
+
/*
* Data preload for architectures that support it
*/
diff --git a/arch/arm/include/asm/kvm_asm.h b/arch/arm/include/asm/kvm_asm.h
index 661da11..12981d6 100644
--- a/arch/arm/include/asm/kvm_asm.h
+++ b/arch/arm/include/asm/kvm_asm.h
@@ -26,9 +26,9 @@
#define c1_ACTLR 4 /* Auxilliary Control Register */
#define c1_CPACR 5 /* Coprocessor Access Control */
#define c2_TTBR0 6 /* Translation Table Base Register 0 */
-#define c2_TTBR0_high 7 /* TTBR0 top 32 bits */
+#define c2_TTBR0_hilo 7 /* TTBR0 top 32 bits in LE case, low 32 bits in BE case */
#define c2_TTBR1 8 /* Translation Table Base Register 1 */
-#define c2_TTBR1_high 9 /* TTBR1 top 32 bits */
+#define c2_TTBR1_hilo 9 /* TTBR1 top 32 bits in LE case, low 32 bits in BE case */
#define c2_TTBCR 10 /* Translation Table Base Control R. */
#define c3_DACR 11 /* Domain Access Control Register */
#define c5_DFSR 12 /* Data Fault Status Register */
diff --git a/arch/arm/kvm/init.S b/arch/arm/kvm/init.S
index 1b9844d..2d10b2d 100644
--- a/arch/arm/kvm/init.S
+++ b/arch/arm/kvm/init.S
@@ -22,6 +22,7 @@
#include <asm/kvm_asm.h>
#include <asm/kvm_arm.h>
#include <asm/kvm_mmu.h>
+#include <asm/assembler.h>
/********************************************************************
* Hypervisor initialization
@@ -70,8 +71,10 @@ __do_hyp_init:
cmp r0, #0 @ We have a SP?
bne phase2 @ Yes, second stage init
+ARM_BE8(setend be) @ Switch to Big Endian mode if needed
+
@ Set the HTTBR to point to the hypervisor PGD pointer passed
- mcrr p15, 4, r2, r3, c2
+ mcrr p15, 4, rr_lo_hi(r2, r3), c2
@ Set the HTCR and VTCR to the same shareability and cacheability
@ settings as the non-secure TTBCR and with T0SZ == 0.
@@ -137,7 +140,7 @@ phase2:
mov pc, r0
target: @ We're now in the trampoline code, switch page tables
- mcrr p15, 4, r2, r3, c2
+ mcrr p15, 4, rr_lo_hi(r2, r3), c2
isb
@ Invalidate the old TLBs
diff --git a/arch/arm/kvm/interrupts.S b/arch/arm/kvm/interrupts.S
index df19133..0784ec3 100644
--- a/arch/arm/kvm/interrupts.S
+++ b/arch/arm/kvm/interrupts.S
@@ -25,6 +25,7 @@
#include <asm/kvm_asm.h>
#include <asm/kvm_arm.h>
#include <asm/vfpmacros.h>
+#include <asm/assembler.h>
#include "interrupts_head.S"
.text
@@ -52,14 +53,14 @@ ENTRY(__kvm_tlb_flush_vmid_ipa)
dsb ishst
add r0, r0, #KVM_VTTBR
ldrd r2, r3, [r0]
- mcrr p15, 6, r2, r3, c2 @ Write VTTBR
+ mcrr p15, 6, rr_lo_hi(r2, r3), c2 @ Write VTTBR
isb
mcr p15, 0, r0, c8, c3, 0 @ TLBIALLIS (rt ignored)
dsb ish
isb
mov r2, #0
mov r3, #0
- mcrr p15, 6, r2, r3, c2 @ Back to VMID #0
+ mcrr p15, 6, rr_lo_hi(r2, r3), c2 @ Back to VMID #0
isb @ Not necessary if followed by eret
ldmia sp!, {r2, r3}
@@ -135,7 +136,7 @@ ENTRY(__kvm_vcpu_run)
ldr r1, [vcpu, #VCPU_KVM]
add r1, r1, #KVM_VTTBR
ldrd r2, r3, [r1]
- mcrr p15, 6, r2, r3, c2 @ Write VTTBR
+ mcrr p15, 6, rr_lo_hi(r2, r3), c2 @ Write VTTBR
@ We're all done, just restore the GPRs and go to the guest
restore_guest_regs
@@ -199,8 +200,13 @@ after_vfp_restore:
restore_host_regs
clrex @ Clear exclusive monitor
+#ifndef __ARMEB__
mov r0, r1 @ Return the return code
mov r1, #0 @ Clear upper bits in return value
+#else
+ @ r1 already has return code
+ mov r0, #0 @ Clear upper bits in return value
+#endif /* __ARMEB__ */
bx lr @ return to IOCTL
/********************************************************************
diff --git a/arch/arm/kvm/interrupts_head.S b/arch/arm/kvm/interrupts_head.S
index c371db7..67b4002 100644
--- a/arch/arm/kvm/interrupts_head.S
+++ b/arch/arm/kvm/interrupts_head.S
@@ -251,8 +251,8 @@ vcpu .req r0 @ vcpu pointer always in r0
mrc p15, 0, r3, c1, c0, 2 @ CPACR
mrc p15, 0, r4, c2, c0, 2 @ TTBCR
mrc p15, 0, r5, c3, c0, 0 @ DACR
- mrrc p15, 0, r6, r7, c2 @ TTBR 0
- mrrc p15, 1, r8, r9, c2 @ TTBR 1
+ mrrc p15, 0, rr_lo_hi(r6, r7), c2 @ TTBR 0
+ mrrc p15, 1, rr_lo_hi(r8, r9), c2 @ TTBR 1
mrc p15, 0, r10, c10, c2, 0 @ PRRR
mrc p15, 0, r11, c10, c2, 1 @ NMRR
mrc p15, 2, r12, c0, c0, 0 @ CSSELR
@@ -380,8 +380,8 @@ vcpu .req r0 @ vcpu pointer always in r0
mcr p15, 0, r3, c1, c0, 2 @ CPACR
mcr p15, 0, r4, c2, c0, 2 @ TTBCR
mcr p15, 0, r5, c3, c0, 0 @ DACR
- mcrr p15, 0, r6, r7, c2 @ TTBR 0
- mcrr p15, 1, r8, r9, c2 @ TTBR 1
+ mcrr p15, 0, rr_lo_hi(r6, r7), c2 @ TTBR 0
+ mcrr p15, 1, rr_lo_hi(r8, r9), c2 @ TTBR 1
mcr p15, 0, r10, c10, c2, 0 @ PRRR
mcr p15, 0, r11, c10, c2, 1 @ NMRR
mcr p15, 2, r12, c0, c0, 0 @ CSSELR
@@ -413,13 +413,21 @@ vcpu .req r0 @ vcpu pointer always in r0
ldr r9, [r2, #GICH_ELRSR1]
ldr r10, [r2, #GICH_APR]
+ARM_BE8(rev r3, r3 )
str r3, [r11, #VGIC_CPU_HCR]
+ARM_BE8(rev r4, r4 )
str r4, [r11, #VGIC_CPU_VMCR]
+ARM_BE8(rev r5, r5 )
str r5, [r11, #VGIC_CPU_MISR]
+ARM_BE8(rev r6, r6 )
str r6, [r11, #VGIC_CPU_EISR]
+ARM_BE8(rev r7, r7 )
str r7, [r11, #(VGIC_CPU_EISR + 4)]
+ARM_BE8(rev r8, r8 )
str r8, [r11, #VGIC_CPU_ELRSR]
+ARM_BE8(rev r9, r9 )
str r9, [r11, #(VGIC_CPU_ELRSR + 4)]
+ARM_BE8(rev r10, r10 )
str r10, [r11, #VGIC_CPU_APR]
/* Clear GICH_HCR */
@@ -431,6 +439,7 @@ vcpu .req r0 @ vcpu pointer always in r0
add r3, r11, #VGIC_CPU_LR
ldr r4, [r11, #VGIC_CPU_NR_LR]
1: ldr r6, [r2], #4
+ARM_BE8(rev r6, r6 )
str r6, [r3], #4
subs r4, r4, #1
bne 1b
@@ -459,8 +468,11 @@ vcpu .req r0 @ vcpu pointer always in r0
ldr r4, [r11, #VGIC_CPU_VMCR]
ldr r8, [r11, #VGIC_CPU_APR]
+ARM_BE8(rev r3, r3 )
str r3, [r2, #GICH_HCR]
+ARM_BE8(rev r4, r4 )
str r4, [r2, #GICH_VMCR]
+ARM_BE8(rev r8, r8 )
str r8, [r2, #GICH_APR]
/* Restore list registers */
@@ -468,6 +480,7 @@ vcpu .req r0 @ vcpu pointer always in r0
add r3, r11, #VGIC_CPU_LR
ldr r4, [r11, #VGIC_CPU_NR_LR]
1: ldr r6, [r3], #4
+ARM_BE8(rev r6, r6 )
str r6, [r2], #4
subs r4, r4, #1
bne 1b
@@ -498,7 +511,7 @@ vcpu .req r0 @ vcpu pointer always in r0
mcr p15, 0, r2, c14, c3, 1 @ CNTV_CTL
isb
- mrrc p15, 3, r2, r3, c14 @ CNTV_CVAL
+ mrrc p15, 3, rr_lo_hi(r2, r3), c14 @ CNTV_CVAL
ldr r4, =VCPU_TIMER_CNTV_CVAL
add r5, vcpu, r4
strd r2, r3, [r5]
@@ -538,12 +551,12 @@ vcpu .req r0 @ vcpu pointer always in r0
ldr r2, [r4, #KVM_TIMER_CNTVOFF]
ldr r3, [r4, #(KVM_TIMER_CNTVOFF + 4)]
- mcrr p15, 4, r2, r3, c14 @ CNTVOFF
+ mcrr p15, 4, rr_lo_hi(r2, r3), c14 @ CNTVOFF
ldr r4, =VCPU_TIMER_CNTV_CVAL
add r5, vcpu, r4
ldrd r2, r3, [r5]
- mcrr p15, 3, r2, r3, c14 @ CNTV_CVAL
+ mcrr p15, 3, rr_lo_hi(r2, r3), c14 @ CNTV_CVAL
isb
ldr r2, [vcpu, #VCPU_TIMER_CNTV_CTL]
--
1.8.1.4
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH 3/5] ARM: kvm one_reg coproc set and get BE fixes
2013-12-20 7:26 [PATCH 0/5] armv7 BE kvm support Victor Kamensky
2013-12-20 7:26 ` [PATCH 1/5] ARM: kvm: replace push and pop with stdmb and ldmia instrs to enable assembler.h inclusion Victor Kamensky
2013-12-20 7:26 ` [PATCH 2/5] ARM: fix KVM assembler files to work in BE case Victor Kamensky
@ 2013-12-20 7:26 ` Victor Kamensky
2013-12-20 7:26 ` [PATCH 4/5] ARM: kvm vgic mmio should return data in BE format in BE case Victor Kamensky
` (2 subsequent siblings)
5 siblings, 0 replies; 8+ messages in thread
From: Victor Kamensky @ 2013-12-20 7:26 UTC (permalink / raw)
To: linux-arm-kernel
This patch fixes issue of reading and writing
ARM V7 registers values from/to user land. Existing code was designed to
work only in LE case.
struct kvm_one_reg
------------------
registers value passed through kvm_one_reg structure. It is used by
KVM_SET_ONE_REG, KVM_GET_ONE_REG ioctls. Note by looking at structure
itself we cannot tell what is size of register. Note that structure carries
address of user memory, 'addr' where register should be read or written
Setting register (from user-land to kvm)
----------------------------------------
kvm_arm_set_reg takes vcpu and pointer to struct kvm_one_reg which already
read from user space
kvm_arm_set_reg calls set_core_reg or kvm_arm_coproc_set_reg
set_core_reg deals only with 4 bytes registers, it just reads 4 bytes from
user space and store it properly into vcpu->arch.regs
kvm_arm_coproc_set_reg deals with registers of different size. At certain
point code reaches phase where it retrieves description of register by id
and it knows register size, which could be either 4 or 8 bytes. Kernel code
is ready to read values from user space, but destination type may vary. It
could be pointer to 32 bit integer or it could be pointer to 64 bit
integer. And all possible permutation of size and destination pointer are
possible. Depending on destination pointer type, 4 bytes or 8 bytes, two
new helper functions are introduced - reg_from_user32 and reg_from_user64.
They are used instead of reg_from_user function which could work only in
LE case.
Size sizeof(*DstInt) Function used to read from user
4 4 reg_from_user32
8 4 reg_from_user32 - read two registers
4 8 reg_from_user64 - need special handling for BE
8 8 reg_from_user64
Getting register (to user-land from kvm)
----------------------------------------
Situation with reading registers is similar to writing. Integer pointer
type of register to be copied could be 4 or 8 bytes. And size passed in
struct kvm_one_reg could be 4 or 8. And any permutation is possible.
Depending on src pointer type, 4 bytes or 8 bytes, two new helper functions
are introduced - reg_from_user32 and reg_to_user64. They are used instead
of reg_to_user function, which could work only in LE case.
Size sizeof(*SrcInt) Function used to write to user
4 4 reg_to_user32
8 4 reg_to_user32 - writes two registers
4 8 reg_to_user64 - need special handleing for BE
8 8 reg_to_user64
Note code does assume that it can only deals with 4 or 8 byte registers.
Signed-off-by: Victor Kamensky <victor.kamensky@linaro.org>
---
arch/arm/kvm/coproc.c | 94 +++++++++++++++++++++++++++++++++++++--------------
1 file changed, 69 insertions(+), 25 deletions(-)
diff --git a/arch/arm/kvm/coproc.c b/arch/arm/kvm/coproc.c
index 78c0885..64b2b94 100644
--- a/arch/arm/kvm/coproc.c
+++ b/arch/arm/kvm/coproc.c
@@ -634,17 +634,61 @@ static struct coproc_reg invariant_cp15[] = {
{ CRn( 0), CRm( 0), Op1( 1), Op2( 7), is32, NULL, get_AIDR },
};
-static int reg_from_user(void *val, const void __user *uaddr, u64 id)
+static int reg_from_user64(u64 *val, const void __user *uaddr, u64 id)
+{
+ unsigned long regsize = KVM_REG_SIZE(id);
+ union {
+ u32 word;
+ u64 dword;
+ } tmp = {0};
+
+ if (copy_from_user(&tmp, uaddr, regsize) != 0)
+ return -EFAULT;
+
+ switch (regsize) {
+ case 4:
+ *val = tmp.word;
+ break;
+ case 8:
+ *val = tmp.dword;
+ break;
+ }
+ return 0;
+}
+
+/* Note it may really copy two u32 registers */
+static int reg_from_user32(u32 *val, const void __user *uaddr, u64 id)
{
- /* This Just Works because we are little endian. */
if (copy_from_user(val, uaddr, KVM_REG_SIZE(id)) != 0)
return -EFAULT;
return 0;
}
-static int reg_to_user(void __user *uaddr, const void *val, u64 id)
+static int reg_to_user64(void __user *uaddr, const u64 *val, u64 id)
+{
+ unsigned long regsize = KVM_REG_SIZE(id);
+ union {
+ u32 word;
+ u64 dword;
+ } tmp;
+
+ switch (regsize) {
+ case 4:
+ tmp.word = *val;
+ break;
+ case 8:
+ tmp.dword = *val;
+ break;
+ }
+
+ if (copy_to_user(uaddr, &tmp, regsize) != 0)
+ return -EFAULT;
+ return 0;
+}
+
+/* Note it may really copy two u32 registers */
+static int reg_to_user32(void __user *uaddr, const u32 *val, u64 id)
{
- /* This Just Works because we are little endian. */
if (copy_to_user(uaddr, val, KVM_REG_SIZE(id)) != 0)
return -EFAULT;
return 0;
@@ -662,7 +706,7 @@ static int get_invariant_cp15(u64 id, void __user *uaddr)
if (!r)
return -ENOENT;
- return reg_to_user(uaddr, &r->val, id);
+ return reg_to_user64(uaddr, &r->val, id);
}
static int set_invariant_cp15(u64 id, void __user *uaddr)
@@ -678,7 +722,7 @@ static int set_invariant_cp15(u64 id, void __user *uaddr)
if (!r)
return -ENOENT;
- err = reg_from_user(&val, uaddr, id);
+ err = reg_from_user64(&val, uaddr, id);
if (err)
return err;
@@ -846,7 +890,7 @@ static int vfp_get_reg(const struct kvm_vcpu *vcpu, u64 id, void __user *uaddr)
if (vfpid < num_fp_regs()) {
if (KVM_REG_SIZE(id) != 8)
return -ENOENT;
- return reg_to_user(uaddr, &vcpu->arch.vfp_guest.fpregs[vfpid],
+ return reg_to_user64(uaddr, &vcpu->arch.vfp_guest.fpregs[vfpid],
id);
}
@@ -856,22 +900,22 @@ static int vfp_get_reg(const struct kvm_vcpu *vcpu, u64 id, void __user *uaddr)
switch (vfpid) {
case KVM_REG_ARM_VFP_FPEXC:
- return reg_to_user(uaddr, &vcpu->arch.vfp_guest.fpexc, id);
+ return reg_to_user32(uaddr, &vcpu->arch.vfp_guest.fpexc, id);
case KVM_REG_ARM_VFP_FPSCR:
- return reg_to_user(uaddr, &vcpu->arch.vfp_guest.fpscr, id);
+ return reg_to_user32(uaddr, &vcpu->arch.vfp_guest.fpscr, id);
case KVM_REG_ARM_VFP_FPINST:
- return reg_to_user(uaddr, &vcpu->arch.vfp_guest.fpinst, id);
+ return reg_to_user32(uaddr, &vcpu->arch.vfp_guest.fpinst, id);
case KVM_REG_ARM_VFP_FPINST2:
- return reg_to_user(uaddr, &vcpu->arch.vfp_guest.fpinst2, id);
+ return reg_to_user32(uaddr, &vcpu->arch.vfp_guest.fpinst2, id);
case KVM_REG_ARM_VFP_MVFR0:
val = fmrx(MVFR0);
- return reg_to_user(uaddr, &val, id);
+ return reg_to_user32(uaddr, &val, id);
case KVM_REG_ARM_VFP_MVFR1:
val = fmrx(MVFR1);
- return reg_to_user(uaddr, &val, id);
+ return reg_to_user32(uaddr, &val, id);
case KVM_REG_ARM_VFP_FPSID:
val = fmrx(FPSID);
- return reg_to_user(uaddr, &val, id);
+ return reg_to_user32(uaddr, &val, id);
default:
return -ENOENT;
}
@@ -890,8 +934,8 @@ static int vfp_set_reg(struct kvm_vcpu *vcpu, u64 id, const void __user *uaddr)
if (vfpid < num_fp_regs()) {
if (KVM_REG_SIZE(id) != 8)
return -ENOENT;
- return reg_from_user(&vcpu->arch.vfp_guest.fpregs[vfpid],
- uaddr, id);
+ return reg_from_user64(&vcpu->arch.vfp_guest.fpregs[vfpid],
+ uaddr, id);
}
/* FP control registers are all 32 bit. */
@@ -900,28 +944,28 @@ static int vfp_set_reg(struct kvm_vcpu *vcpu, u64 id, const void __user *uaddr)
switch (vfpid) {
case KVM_REG_ARM_VFP_FPEXC:
- return reg_from_user(&vcpu->arch.vfp_guest.fpexc, uaddr, id);
+ return reg_from_user32(&vcpu->arch.vfp_guest.fpexc, uaddr, id);
case KVM_REG_ARM_VFP_FPSCR:
- return reg_from_user(&vcpu->arch.vfp_guest.fpscr, uaddr, id);
+ return reg_from_user32(&vcpu->arch.vfp_guest.fpscr, uaddr, id);
case KVM_REG_ARM_VFP_FPINST:
- return reg_from_user(&vcpu->arch.vfp_guest.fpinst, uaddr, id);
+ return reg_from_user32(&vcpu->arch.vfp_guest.fpinst, uaddr, id);
case KVM_REG_ARM_VFP_FPINST2:
- return reg_from_user(&vcpu->arch.vfp_guest.fpinst2, uaddr, id);
+ return reg_from_user32(&vcpu->arch.vfp_guest.fpinst2, uaddr, id);
/* These are invariant. */
case KVM_REG_ARM_VFP_MVFR0:
- if (reg_from_user(&val, uaddr, id))
+ if (reg_from_user32(&val, uaddr, id))
return -EFAULT;
if (val != fmrx(MVFR0))
return -EINVAL;
return 0;
case KVM_REG_ARM_VFP_MVFR1:
- if (reg_from_user(&val, uaddr, id))
+ if (reg_from_user32(&val, uaddr, id))
return -EFAULT;
if (val != fmrx(MVFR1))
return -EINVAL;
return 0;
case KVM_REG_ARM_VFP_FPSID:
- if (reg_from_user(&val, uaddr, id))
+ if (reg_from_user32(&val, uaddr, id))
return -EFAULT;
if (val != fmrx(FPSID))
return -EINVAL;
@@ -968,7 +1012,7 @@ int kvm_arm_coproc_get_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg)
return get_invariant_cp15(reg->id, uaddr);
/* Note: copies two regs if size is 64 bit. */
- return reg_to_user(uaddr, &vcpu->arch.cp15[r->reg], reg->id);
+ return reg_to_user32(uaddr, &vcpu->arch.cp15[r->reg], reg->id);
}
int kvm_arm_coproc_set_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg)
@@ -987,7 +1031,7 @@ int kvm_arm_coproc_set_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg)
return set_invariant_cp15(reg->id, uaddr);
/* Note: copies two regs if size is 64 bit */
- return reg_from_user(&vcpu->arch.cp15[r->reg], uaddr, reg->id);
+ return reg_from_user32(&vcpu->arch.cp15[r->reg], uaddr, reg->id);
}
static unsigned int num_demux_regs(void)
--
1.8.1.4
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH 4/5] ARM: kvm vgic mmio should return data in BE format in BE case
2013-12-20 7:26 [PATCH 0/5] armv7 BE kvm support Victor Kamensky
` (2 preceding siblings ...)
2013-12-20 7:26 ` [PATCH 3/5] ARM: kvm one_reg coproc set and get BE fixes Victor Kamensky
@ 2013-12-20 7:26 ` Victor Kamensky
2013-12-20 7:26 ` [PATCH 5/5] ARM: kvm MMIO support BE host running LE code Victor Kamensky
2013-12-20 8:12 ` [PATCH 0/5] armv7 BE kvm support Christoffer Dall
5 siblings, 0 replies; 8+ messages in thread
From: Victor Kamensky @ 2013-12-20 7:26 UTC (permalink / raw)
To: linux-arm-kernel
KVM mmio in BE case assumes that data it recieves is in BE format. Vgic
operates in LE, so need byteswap data in BE case.
Signed-off-by: Victor Kamensky <victor.kamensky@linaro.org>
---
virt/kvm/arm/vgic.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
index 685fc72..7e11458 100644
--- a/virt/kvm/arm/vgic.c
+++ b/virt/kvm/arm/vgic.c
@@ -236,12 +236,12 @@ static void vgic_cpu_irq_clear(struct kvm_vcpu *vcpu, int irq)
static u32 mmio_data_read(struct kvm_exit_mmio *mmio, u32 mask)
{
- return *((u32 *)mmio->data) & mask;
+ return le32_to_cpu(*((u32 *)mmio->data)) & mask;
}
static void mmio_data_write(struct kvm_exit_mmio *mmio, u32 mask, u32 value)
{
- *((u32 *)mmio->data) = value & mask;
+ *((u32 *)mmio->data) = cpu_to_le32(value) & mask;
}
/**
--
1.8.1.4
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH 5/5] ARM: kvm MMIO support BE host running LE code
2013-12-20 7:26 [PATCH 0/5] armv7 BE kvm support Victor Kamensky
` (3 preceding siblings ...)
2013-12-20 7:26 ` [PATCH 4/5] ARM: kvm vgic mmio should return data in BE format in BE case Victor Kamensky
@ 2013-12-20 7:26 ` Victor Kamensky
2013-12-20 8:12 ` [PATCH 0/5] armv7 BE kvm support Christoffer Dall
5 siblings, 0 replies; 8+ messages in thread
From: Victor Kamensky @ 2013-12-20 7:26 UTC (permalink / raw)
To: linux-arm-kernel
In case of status register E bit is not set (LE mode) and host runs in
BE mode we need byteswap data, so read/write is emulated correctly.
Signed-off-by: Victor Kamensky <victor.kamensky@linaro.org>
---
arch/arm/include/asm/kvm_emulate.h | 22 ++++++++++++++++++----
1 file changed, 18 insertions(+), 4 deletions(-)
diff --git a/arch/arm/include/asm/kvm_emulate.h b/arch/arm/include/asm/kvm_emulate.h
index 0fa90c9..69b7469 100644
--- a/arch/arm/include/asm/kvm_emulate.h
+++ b/arch/arm/include/asm/kvm_emulate.h
@@ -185,9 +185,16 @@ static inline unsigned long vcpu_data_guest_to_host(struct kvm_vcpu *vcpu,
default:
return be32_to_cpu(data);
}
+ } else {
+ switch (len) {
+ case 1:
+ return data & 0xff;
+ case 2:
+ return le16_to_cpu(data & 0xffff);
+ default:
+ return le32_to_cpu(data);
+ }
}
-
- return data; /* Leave LE untouched */
}
static inline unsigned long vcpu_data_host_to_guest(struct kvm_vcpu *vcpu,
@@ -203,9 +210,16 @@ static inline unsigned long vcpu_data_host_to_guest(struct kvm_vcpu *vcpu,
default:
return cpu_to_be32(data);
}
+ } else {
+ switch (len) {
+ case 1:
+ return data & 0xff;
+ case 2:
+ return cpu_to_le16(data & 0xffff);
+ default:
+ return cpu_to_le32(data);
+ }
}
-
- return data; /* Leave LE untouched */
}
#endif /* __ARM_KVM_EMULATE_H__ */
--
1.8.1.4
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH 0/5] armv7 BE kvm support
2013-12-20 7:26 [PATCH 0/5] armv7 BE kvm support Victor Kamensky
` (4 preceding siblings ...)
2013-12-20 7:26 ` [PATCH 5/5] ARM: kvm MMIO support BE host running LE code Victor Kamensky
@ 2013-12-20 8:12 ` Christoffer Dall
5 siblings, 0 replies; 8+ messages in thread
From: Christoffer Dall @ 2013-12-20 8:12 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Dec 19, 2013 at 11:26:54PM -0800, Victor Kamensky wrote:
> Hi Guys,
Hi Victor,
>
> Here is series that enables KVM support for V7 big endian kernels. Mostly
> it deals with BE KVM host support. Marc Zyngier showed before with his patches
> how BE guest could run on top LE host. With these patches BE guest runs on
> top of BE host. If Marc's kvmtool is used with few additional changes I tested
> that BE host could run LE guest. Also I verified that there were no
> regressions in BE guest on top of LE host case.
Can you please cc the kvmarm list as per the MAINTAINERS files for
patches to kvm/arm in the future?
Looking forward to reading your code!
-Christoffer
>
> Note that posted series covers only kernel side changes. The changes were
> tested inside of bigger setup with additional changes in qemu and kvmtool.
> I will post those changes separately in proper aliases but for completeness
> sake Appendix A gives pointers to git repositories and branches with all
> needed changes.
>
> Please note first patch is not related to BE KVM per se. I've run
> into an issue of conflicting 'push' identifier use while trying to include
> assembler.h into KVM .S files. Details of an issue I observed covered in
> Appendix B. The first patch is my take on solving it.
>
> Victor Kamensky (5):
> ARM: kvm: replace push and pop with stdmb and ldmia instrs to enable
> assembler.h inclusion
> ARM: fix KVM assembler files to work in BE case
> ARM: kvm one_reg coproc set and get BE fixes
> ARM: kvm vgic mmio should return data in BE format in BE case
> ARM: kvm MMIO support BE host running LE code
>
> arch/arm/include/asm/assembler.h | 7 +++
> arch/arm/include/asm/kvm_asm.h | 4 +-
> arch/arm/include/asm/kvm_emulate.h | 22 +++++++--
> arch/arm/kvm/coproc.c | 94 ++++++++++++++++++++++++++++----------
> arch/arm/kvm/init.S | 7 ++-
> arch/arm/kvm/interrupts.S | 50 +++++++++++---------
> arch/arm/kvm/interrupts_head.S | 61 +++++++++++++++----------
> virt/kvm/arm/vgic.c | 4 +-
> 8 files changed, 168 insertions(+), 81 deletions(-)
>
> --
> 1.8.1.4
>
> Thanks,
> Victor
>
> Appendix A: Testing and Full Setup Description
> ----------------------------------------------
>
> I) No mixed mode setup - i.e BE guest on BE host; and LE guest
> on LE host tested to make sure no regressions.
>
> KVM host and guest kernels:
> TC2 on top of Linus 3.13-rc4 (this patch series):
> git: git://git.linaro.org/people/victor.kamensky/linux-linaro-tracking-be.git
> branch: armv7be-kvm-3.13-rc4
>
> TC2 and Arndale on top of Linaro BE tree:
> git: git://git.linaro.org/people/victor.kamensky/linux-linaro-tracking-be.git
> branch: llct-be-20131216-kvm
>
> - TC1 kernels used as guests
>
> qemu:
> git: git://git.linaro.org/people/victor.kamensky/qemu-be.git
> branch: armv7be-v1
> description: changes to run qemu on armeb target; and other
> changes to work with be image on top of be host
>
> kvmtool:
> git: git://git.linaro.org/people/victor.kamensky/linux-linaro-tracking-be.git
> branch: kvmtool-armv7be-v1
> desciption: minimal changes to build kvmtool for armeb target; and
> tiny change with virtio magic
>
> II) Mixed mode setup all possible combinations within V7 (LE guest on BE host;
> BE guest on LE host as Marc's setup tested to make sure no regressions) only
> with kvmtool.
>
> This work is based on Marc Zyngier's work that made BE guest to run on top
> of LE host. For this setup special version of kvmtool should be used and
> in addition I had to apply patch to guest kernel that would switch reading
> virtio configs reads to be LE only, that is made on top of previous Rusty
> Russell's changes. Effectively I just had to do very minor addition to make
> LE guest to work on BE host, most of heavy lifting was done before by Marc.
>
> KVM host kernels: as in previous setup
>
> Guest TC1 kernels with LE virtio config patch:
> git: git://git.linaro.org/people/victor.kamensky/linux-linaro-tracking-be.git
> branch: virtio-leconfig-3.13-rc4
>
> kvmtool:
> git: git://git.linaro.org/people/victor.kamensky/linux-linaro-tracking-be.git
> branch: kvmtool-mixed-v1
> description: based on git://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms.git
> branch kvm-arm64/kvmtool-be-on-le; adds missing include fix; above armeb target
> build patches; and one fix related to BE mode
>
> qemu:
> git: git://git.linaro.org/people/victor.kamensky/qemu-be.git
> branch: armv7be-leconfig-v1
> description: change virtio-blk that so qemu could work with guest image
> where virtio leconfig is made; note it does not work in mixed mode; to do
> so qemu would need bunch of similar changes that Marc did in kvmtool
>
>
> Appendix B: kvm asm file and asm/assembler.h file issue
> -------------------------------------------------------
>
> diff --git a/arch/arm/kvm/interrupts.S b/arch/arm/kvm/interrupts.S
> index ddc1553..5d3b511 100644
> --- a/arch/arm/kvm/interrupts.S
> +++ b/arch/arm/kvm/interrupts.S
> @@ -25,6 +25,7 @@
> #include <asm/kvm_asm.h>
> #include <asm/kvm_arm.h>
> #include <asm/vfpmacros.h>
> +#include <asm/assembler.h>
> #include "interrupts_head.S"
>
> .text
>
> produce the following compilation errors:
>
> /run/media/kamensky/wd/linaro/linux-linaro-core-tracking/092913/linux-linaro-tracking-be/arch/arm/kvm/interrupts.S: Assembler messages:
> /run/media/kamensky/wd/linaro/linux-linaro-core-tracking/092913/linux-linaro-tracking-be/arch/arm/kvm/interrupts.S:51: Error: ARM register expected -- `lsr {r2,r3}'
> /run/media/kamensky/wd/linaro/linux-linaro-core-tracking/092913/linux-linaro-tracking-be/arch/arm/kvm/interrupts.S:100: Error: ARM register expected -- `lsr {r2}'
> /run/media/kamensky/wd/linaro/linux-linaro-core-tracking/092913/linux-linaro-tracking-be/arch/arm/kvm/interrupts.S:100: Error: ARM register expected -- `lsr {r4-r12}'
>
--
Christoffer
^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH 1/5] ARM: kvm: replace push and pop with stdmb and ldmia instrs to enable assembler.h inclusion
2013-12-20 7:26 ` [PATCH 1/5] ARM: kvm: replace push and pop with stdmb and ldmia instrs to enable assembler.h inclusion Victor Kamensky
@ 2014-01-07 16:01 ` Dave Martin
0 siblings, 0 replies; 8+ messages in thread
From: Dave Martin @ 2014-01-07 16:01 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Dec 19, 2013 at 11:26:55PM -0800, Victor Kamensky wrote:
> Before fix kvm interrupt.S and interrupt_head.S used push and pop assembler
> instruction. It causes problem if <asm/assembler.h> file should be include. In
> assembler.h "push" is defined as macro so it causes compilation errors like
> this:
>
> arch/arm/kvm/interrupts.S: Assembler messages:
> arch/arm/kvm/interrupts.S:51: Error: ARM register expected -- `lsr {r2,r3}'
>
> Solution implemented by this patch replaces all 'push {...}' with
> 'stdmb sp!, {...}' instruction; and all 'pop {...}' with 'ldmia sp!, {...}'.
>
> Signed-off-by: Victor Kamensky <victor.kamensky@linaro.org>
> ---
> arch/arm/kvm/interrupts.S | 38 +++++++++++++++++++-------------------
> arch/arm/kvm/interrupts_head.S | 34 +++++++++++++++++-----------------
> 2 files changed, 36 insertions(+), 36 deletions(-)
>
> diff --git a/arch/arm/kvm/interrupts.S b/arch/arm/kvm/interrupts.S
> index ddc1553..df19133 100644
> --- a/arch/arm/kvm/interrupts.S
> +++ b/arch/arm/kvm/interrupts.S
> @@ -47,7 +47,7 @@ __kvm_hyp_code_start:
> * instead, ignoring the ipa value.
> */
> ENTRY(__kvm_tlb_flush_vmid_ipa)
> - push {r2, r3}
> + stmdb sp!, {r2, r3}
Minor nit: when pushing and popping the stack, the convention for kernel
code is to write
stmfd sp!,
ldmfd sp!,
instead of
stmdb sp!,
ldmia sp!,
This will have no effect on the assembly, but may be slightly easier to
read.
>
> dsb ishst
> add r0, r0, #KVM_VTTBR
> @@ -62,7 +62,7 @@ ENTRY(__kvm_tlb_flush_vmid_ipa)
> mcrr p15, 6, r2, r3, c2 @ Back to VMID #0
> isb @ Not necessary if followed by eret
>
> - pop {r2, r3}
> + ldmia sp!, {r2, r3}
> bx lr
> ENDPROC(__kvm_tlb_flush_vmid_ipa)
>
> @@ -110,7 +110,7 @@ ENTRY(__kvm_vcpu_run)
> #ifdef CONFIG_VFPv3
> @ Set FPEXC_EN so the guest doesn't trap floating point instructions
> VFPFMRX r2, FPEXC @ VMRS
> - push {r2}
> + stmdb sp!, {r2}
For saving or restoring a single register, you may want to do:
str r2, [sp, #-4]!
...
ldr r2, [sp], #4
This is the commonly used convention, and may be marginally more
efficient. The "push" and "pop" assembler mnemonics generate
these instead of stm/ldm if the list only contains a single register,
whereas ldm/stm always generate a real load or store multiple even
if there is only one register in the list.
This is a trivial issue, so only worth fixing if you are going to repost
the patch anyway.
> orr r2, r2, #FPEXC_EN
> VFPFMXR FPEXC, r2 @ VMSR
> #endif
> @@ -175,7 +175,7 @@ __kvm_vcpu_return:
>
> after_vfp_restore:
> @ Restore FPEXC_EN which we clobbered on entry
> - pop {r2}
> + ldmia sp!, {r2}
> VFPFMXR FPEXC, r2
> #endif
>
> @@ -260,7 +260,7 @@ ENTRY(kvm_call_hyp)
>
> /* Handle undef, svc, pabt, or dabt by crashing with a user notice */
> .macro bad_exception exception_code, panic_str
> - push {r0-r2}
> + stmdb sp!, {r0-r2}
> mrrc p15, 6, r0, r1, c2 @ Read VTTBR
> lsr r1, r1, #16
> ands r1, r1, #0xff
> @@ -338,7 +338,7 @@ hyp_hvc:
> * Getting here is either becuase of a trap from a guest or from calling
> * HVC from the host kernel, which means "switch to Hyp mode".
> */
> - push {r0, r1, r2}
> + stmdb sp!, {r0, r1, r2}
>
> @ Check syndrome register
> mrc p15, 4, r1, c5, c2, 0 @ HSR
> @@ -361,11 +361,11 @@ hyp_hvc:
> bne guest_trap @ Guest called HVC
>
> host_switch_to_hyp:
> - pop {r0, r1, r2}
> + ldmia sp!, {r0, r1, r2}
>
> - push {lr}
> + stmdb sp!, {lr}
> mrs lr, SPSR
> - push {lr}
> + stmdb sp!, {lr}
>
> mov lr, r0
> mov r0, r1
> @@ -375,9 +375,9 @@ host_switch_to_hyp:
> THUMB( orr lr, #1)
> blx lr @ Call the HYP function
>
> - pop {lr}
> + ldmia sp!, {lr}
> msr SPSR_csxf, lr
> - pop {lr}
> + ldmia sp!, {lr}
> eret
>
> guest_trap:
> @@ -418,7 +418,7 @@ guest_trap:
>
> /* Preserve PAR */
> mrrc p15, 0, r0, r1, c7 @ PAR
> - push {r0, r1}
> + stmdb sp!, {r0, r1}
>
> /* Resolve IPA using the xFAR */
> mcr p15, 0, r2, c7, c8, 0 @ ATS1CPR
> @@ -431,7 +431,7 @@ guest_trap:
> orr r2, r2, r1, lsl #24
>
> /* Restore PAR */
> - pop {r0, r1}
> + ldmia sp!, {r0, r1}
> mcrr p15, 0, r0, r1, c7 @ PAR
>
> 3: load_vcpu @ Load VCPU pointer to r0
> @@ -440,10 +440,10 @@ guest_trap:
> 1: mov r1, #ARM_EXCEPTION_HVC
> b __kvm_vcpu_return
>
> -4: pop {r0, r1} @ Failed translation, return to guest
> +4: ldmia sp!, {r0, r1} @ Failed translation, return to guest
> mcrr p15, 0, r0, r1, c7 @ PAR
> clrex
> - pop {r0, r1, r2}
> + ldmia sp!, {r0, r1, r2}
> eret
>
> /*
> @@ -455,7 +455,7 @@ guest_trap:
> #ifdef CONFIG_VFPv3
> switch_to_guest_vfp:
> load_vcpu @ Load VCPU pointer to r0
> - push {r3-r7}
> + stmdb sp!, {r3-r7}
>
> @ NEON/VFP used. Turn on VFP access.
> set_hcptr vmexit, (HCPTR_TCP(10) | HCPTR_TCP(11))
> @@ -467,15 +467,15 @@ switch_to_guest_vfp:
> add r7, r0, #VCPU_VFP_GUEST
> restore_vfp_state r7
>
> - pop {r3-r7}
> - pop {r0-r2}
> + ldmia sp!, {r3-r7}
> + ldmia sp!, {r0-r2}
> clrex
> eret
> #endif
>
> .align
> hyp_irq:
> - push {r0, r1, r2}
> + stmdb sp!, {r0, r1, r2}
> mov r1, #ARM_EXCEPTION_IRQ
> load_vcpu @ Load VCPU pointer to r0
> b __kvm_vcpu_return
> diff --git a/arch/arm/kvm/interrupts_head.S b/arch/arm/kvm/interrupts_head.S
> index 6f18695..c371db7 100644
> --- a/arch/arm/kvm/interrupts_head.S
> +++ b/arch/arm/kvm/interrupts_head.S
> @@ -63,7 +63,7 @@ vcpu .req r0 @ vcpu pointer always in r0
> mrs r2, SP_\mode
> mrs r3, LR_\mode
> mrs r4, SPSR_\mode
> - push {r2, r3, r4}
> + stmdb sp!, {r2, r3, r4}
> .endm
>
> /*
> @@ -73,13 +73,13 @@ vcpu .req r0 @ vcpu pointer always in r0
> .macro save_host_regs
> /* Hyp regs. Only ELR_hyp (SPSR_hyp already saved) */
> mrs r2, ELR_hyp
> - push {r2}
> + stmdb sp!, {r2}
>
> /* usr regs */
> - push {r4-r12} @ r0-r3 are always clobbered
> + stmdb sp!, {r4-r12} @ r0-r3 are always clobbered
> mrs r2, SP_usr
> mov r3, lr
> - push {r2, r3}
> + stmdb sp!, {r2, r3}
>
> push_host_regs_mode svc
> push_host_regs_mode abt
> @@ -95,11 +95,11 @@ vcpu .req r0 @ vcpu pointer always in r0
> mrs r7, SP_fiq
> mrs r8, LR_fiq
> mrs r9, SPSR_fiq
> - push {r2-r9}
> + stmdb sp!, {r2-r9}
> .endm
>
> .macro pop_host_regs_mode mode
> - pop {r2, r3, r4}
> + ldmia sp!, {r2, r3, r4}
> msr SP_\mode, r2
> msr LR_\mode, r3
> msr SPSR_\mode, r4
> @@ -110,7 +110,7 @@ vcpu .req r0 @ vcpu pointer always in r0
> * Clobbers all registers, in all modes, except r0 and r1.
> */
> .macro restore_host_regs
> - pop {r2-r9}
> + ldmia sp!, {r2-r9}
> msr r8_fiq, r2
> msr r9_fiq, r3
> msr r10_fiq, r4
> @@ -125,12 +125,12 @@ vcpu .req r0 @ vcpu pointer always in r0
> pop_host_regs_mode abt
> pop_host_regs_mode svc
>
> - pop {r2, r3}
> + ldmia sp!, {r2, r3}
> msr SP_usr, r2
> mov lr, r3
> - pop {r4-r12}
> + ldmia sp!, {r4-r12}
>
> - pop {r2}
> + ldmia sp!, {r2}
> msr ELR_hyp, r2
> .endm
>
> @@ -218,7 +218,7 @@ vcpu .req r0 @ vcpu pointer always in r0
> add r2, vcpu, #VCPU_USR_REG(3)
> stm r2, {r3-r12}
> add r2, vcpu, #VCPU_USR_REG(0)
> - pop {r3, r4, r5} @ r0, r1, r2
> + ldmia sp!, {r3, r4, r5} @ r0, r1, r2
> stm r2, {r3, r4, r5}
> mrs r2, SP_usr
> mov r3, lr
> @@ -258,7 +258,7 @@ vcpu .req r0 @ vcpu pointer always in r0
> mrc p15, 2, r12, c0, c0, 0 @ CSSELR
>
> .if \store_to_vcpu == 0
> - push {r2-r12} @ Push CP15 registers
> + stmdb sp!, {r2-r12} @ Push CP15 registers
> .else
> str r2, [vcpu, #CP15_OFFSET(c1_SCTLR)]
> str r3, [vcpu, #CP15_OFFSET(c1_CPACR)]
> @@ -286,7 +286,7 @@ vcpu .req r0 @ vcpu pointer always in r0
> mrc p15, 0, r12, c12, c0, 0 @ VBAR
>
> .if \store_to_vcpu == 0
> - push {r2-r12} @ Push CP15 registers
> + stmdb sp!, {r2-r12} @ Push CP15 registers
> .else
> str r2, [vcpu, #CP15_OFFSET(c13_CID)]
> str r3, [vcpu, #CP15_OFFSET(c13_TID_URW)]
> @@ -305,7 +305,7 @@ vcpu .req r0 @ vcpu pointer always in r0
> mrrc p15, 0, r4, r5, c7 @ PAR
>
> .if \store_to_vcpu == 0
> - push {r2,r4-r5}
> + stmdb sp!, {r2,r4-r5}
> .else
> str r2, [vcpu, #CP15_OFFSET(c14_CNTKCTL)]
> add r12, vcpu, #CP15_OFFSET(c7_PAR)
> @@ -322,7 +322,7 @@ vcpu .req r0 @ vcpu pointer always in r0
> */
> .macro write_cp15_state read_from_vcpu
> .if \read_from_vcpu == 0
> - pop {r2,r4-r5}
> + ldmia sp!, {r2,r4-r5}
> .else
> ldr r2, [vcpu, #CP15_OFFSET(c14_CNTKCTL)]
> add r12, vcpu, #CP15_OFFSET(c7_PAR)
> @@ -333,7 +333,7 @@ vcpu .req r0 @ vcpu pointer always in r0
> mcrr p15, 0, r4, r5, c7 @ PAR
>
> .if \read_from_vcpu == 0
> - pop {r2-r12}
> + ldmia sp!, {r2-r12}
> .else
> ldr r2, [vcpu, #CP15_OFFSET(c13_CID)]
> ldr r3, [vcpu, #CP15_OFFSET(c13_TID_URW)]
> @@ -361,7 +361,7 @@ vcpu .req r0 @ vcpu pointer always in r0
> mcr p15, 0, r12, c12, c0, 0 @ VBAR
>
> .if \read_from_vcpu == 0
> - pop {r2-r12}
> + ldmia sp!, {r2-r12}
> .else
> ldr r2, [vcpu, #CP15_OFFSET(c1_SCTLR)]
> ldr r3, [vcpu, #CP15_OFFSET(c1_CPACR)]
> --
> 1.8.1.4
>
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2014-01-07 16:01 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2013-12-20 7:26 [PATCH 0/5] armv7 BE kvm support Victor Kamensky
2013-12-20 7:26 ` [PATCH 1/5] ARM: kvm: replace push and pop with stdmb and ldmia instrs to enable assembler.h inclusion Victor Kamensky
2014-01-07 16:01 ` Dave Martin
2013-12-20 7:26 ` [PATCH 2/5] ARM: fix KVM assembler files to work in BE case Victor Kamensky
2013-12-20 7:26 ` [PATCH 3/5] ARM: kvm one_reg coproc set and get BE fixes Victor Kamensky
2013-12-20 7:26 ` [PATCH 4/5] ARM: kvm vgic mmio should return data in BE format in BE case Victor Kamensky
2013-12-20 7:26 ` [PATCH 5/5] ARM: kvm MMIO support BE host running LE code Victor Kamensky
2013-12-20 8:12 ` [PATCH 0/5] armv7 BE kvm support Christoffer Dall
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).