linux-arm-kernel.lists.infradead.org archive mirror
 help / color / mirror / Atom feed
* [PATCH 0/5] armv7 BE kvm support
@ 2013-12-20  7:26 Victor Kamensky
  2013-12-20  7:26 ` [PATCH 1/5] ARM: kvm: replace push and pop with stdmb and ldmia instrs to enable assembler.h inclusion Victor Kamensky
                   ` (5 more replies)
  0 siblings, 6 replies; 8+ messages in thread
From: Victor Kamensky @ 2013-12-20  7:26 UTC (permalink / raw)
  To: linux-arm-kernel

Hi Guys,

Here is series that enables KVM support for V7 big endian kernels. Mostly
it deals with BE KVM host support. Marc Zyngier showed before with his patches
how BE guest could run on top LE host. With these patches BE guest runs on 
top of BE host. If Marc's kvmtool is used with few additional changes I tested
that BE host could run LE guest. Also I verified that there were no 
regressions in BE guest on top of LE host case.

Note that posted series covers only kernel side changes. The changes were
tested inside of bigger setup with additional changes in qemu and kvmtool.
I will post those changes separately in proper aliases but for completeness 
sake Appendix A gives pointers to git repositories and branches with all
needed changes.

Please note first patch is not related to BE KVM per se. I've run
into an issue of conflicting 'push' identifier use while trying to include 
assembler.h into KVM .S files. Details of an issue I observed covered in
Appendix B. The first patch is my take on solving it.

Victor Kamensky (5):
  ARM: kvm: replace push and pop with stdmb and ldmia instrs to enable
    assembler.h inclusion
  ARM: fix KVM assembler files to work in BE case
  ARM: kvm one_reg coproc set and get BE fixes
  ARM: kvm vgic mmio should return data in BE format in BE case
  ARM: kvm MMIO support BE host running LE code

 arch/arm/include/asm/assembler.h   |  7 +++
 arch/arm/include/asm/kvm_asm.h     |  4 +-
 arch/arm/include/asm/kvm_emulate.h | 22 +++++++--
 arch/arm/kvm/coproc.c              | 94 ++++++++++++++++++++++++++++----------
 arch/arm/kvm/init.S                |  7 ++-
 arch/arm/kvm/interrupts.S          | 50 +++++++++++---------
 arch/arm/kvm/interrupts_head.S     | 61 +++++++++++++++----------
 virt/kvm/arm/vgic.c                |  4 +-
 8 files changed, 168 insertions(+), 81 deletions(-)

-- 
1.8.1.4

Thanks,
Victor

      Appendix A: Testing and Full Setup Description
      ----------------------------------------------

I) No mixed mode setup - i.e BE guest on BE host; and LE guest
on LE host tested to make sure no regressions.

KVM host and guest kernels:
  TC2 on top of Linus 3.13-rc4 (this patch series):
      git: git://git.linaro.org/people/victor.kamensky/linux-linaro-tracking-be.git
      branch: armv7be-kvm-3.13-rc4
      
  TC2 and Arndale on top of Linaro BE tree:
      git: git://git.linaro.org/people/victor.kamensky/linux-linaro-tracking-be.git
      branch: llct-be-20131216-kvm

- TC1 kernels used as guests

qemu:
      git: git://git.linaro.org/people/victor.kamensky/qemu-be.git 
      branch: armv7be-v1
      description: changes to run qemu on armeb target; and other
      changes to work with be image on top of be host

kvmtool:
      git: git://git.linaro.org/people/victor.kamensky/linux-linaro-tracking-be.git
      branch: kvmtool-armv7be-v1
      desciption: minimal changes to build kvmtool for armeb target; and
      tiny change with virtio magic

II) Mixed mode setup all possible combinations within V7 (LE guest on BE host;
BE guest on LE host as Marc's setup tested to make sure no regressions) only
with kvmtool.

This work is based on Marc Zyngier's work that made BE guest to run on top
of LE host. For this setup special version of kvmtool should be used and
in addition I had to apply patch to guest kernel that would switch reading
virtio configs reads to be LE only, that is made on top of previous Rusty
Russell's changes. Effectively I just had to do very minor addition to make
LE guest to work on BE host, most of heavy lifting was done before by Marc.

KVM host kernels: as in previous setup

Guest TC1 kernels with LE virtio config patch:
      git: git://git.linaro.org/people/victor.kamensky/linux-linaro-tracking-be.git
      branch: virtio-leconfig-3.13-rc4

kvmtool:
      git: git://git.linaro.org/people/victor.kamensky/linux-linaro-tracking-be.git
      branch: kvmtool-mixed-v1
      description: based on git://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms.git
      branch kvm-arm64/kvmtool-be-on-le; adds missing include fix; above armeb target
      build patches; and one fix related to BE mode

qemu:
      git: git://git.linaro.org/people/victor.kamensky/qemu-be.git
      branch: armv7be-leconfig-v1
      description: change virtio-blk that so qemu could work with guest image
      where virtio leconfig is made; note it does not work in mixed mode; to do
      so qemu would need bunch of similar changes that Marc did in kvmtool


      Appendix B: kvm asm file and asm/assembler.h file issue
      -------------------------------------------------------

diff --git a/arch/arm/kvm/interrupts.S b/arch/arm/kvm/interrupts.S
index ddc1553..5d3b511 100644
--- a/arch/arm/kvm/interrupts.S
+++ b/arch/arm/kvm/interrupts.S
@@ -25,6 +25,7 @@
 #include <asm/kvm_asm.h>
 #include <asm/kvm_arm.h>
 #include <asm/vfpmacros.h>
+#include <asm/assembler.h>
 #include "interrupts_head.S"
 
        .text

produce the following compilation errors:

/run/media/kamensky/wd/linaro/linux-linaro-core-tracking/092913/linux-linaro-tracking-be/arch/arm/kvm/interrupts.S: Assembler messages:
/run/media/kamensky/wd/linaro/linux-linaro-core-tracking/092913/linux-linaro-tracking-be/arch/arm/kvm/interrupts.S:51: Error: ARM register expected -- `lsr {r2,r3}'
/run/media/kamensky/wd/linaro/linux-linaro-core-tracking/092913/linux-linaro-tracking-be/arch/arm/kvm/interrupts.S:100: Error: ARM register expected -- `lsr {r2}'
/run/media/kamensky/wd/linaro/linux-linaro-core-tracking/092913/linux-linaro-tracking-be/arch/arm/kvm/interrupts.S:100: Error: ARM register expected -- `lsr {r4-r12}'

^ permalink raw reply related	[flat|nested] 8+ messages in thread

* [PATCH 1/5] ARM: kvm: replace push and pop with stdmb and ldmia instrs to enable assembler.h inclusion
  2013-12-20  7:26 [PATCH 0/5] armv7 BE kvm support Victor Kamensky
@ 2013-12-20  7:26 ` Victor Kamensky
  2014-01-07 16:01   ` Dave Martin
  2013-12-20  7:26 ` [PATCH 2/5] ARM: fix KVM assembler files to work in BE case Victor Kamensky
                   ` (4 subsequent siblings)
  5 siblings, 1 reply; 8+ messages in thread
From: Victor Kamensky @ 2013-12-20  7:26 UTC (permalink / raw)
  To: linux-arm-kernel

Before fix kvm interrupt.S and interrupt_head.S used push and pop assembler
instruction. It causes problem if <asm/assembler.h> file should be include. In
assembler.h "push" is defined as macro so it causes compilation errors like
this:

arch/arm/kvm/interrupts.S: Assembler messages:
arch/arm/kvm/interrupts.S:51: Error: ARM register expected -- `lsr {r2,r3}'

Solution implemented by this patch replaces all 'push {...}' with
'stdmb sp!, {...}' instruction; and all 'pop {...}' with 'ldmia sp!, {...}'.

Signed-off-by: Victor Kamensky <victor.kamensky@linaro.org>
---
 arch/arm/kvm/interrupts.S      | 38 +++++++++++++++++++-------------------
 arch/arm/kvm/interrupts_head.S | 34 +++++++++++++++++-----------------
 2 files changed, 36 insertions(+), 36 deletions(-)

diff --git a/arch/arm/kvm/interrupts.S b/arch/arm/kvm/interrupts.S
index ddc1553..df19133 100644
--- a/arch/arm/kvm/interrupts.S
+++ b/arch/arm/kvm/interrupts.S
@@ -47,7 +47,7 @@ __kvm_hyp_code_start:
  * instead, ignoring the ipa value.
  */
 ENTRY(__kvm_tlb_flush_vmid_ipa)
-	push	{r2, r3}
+	stmdb	sp!, {r2, r3}
 
 	dsb	ishst
 	add	r0, r0, #KVM_VTTBR
@@ -62,7 +62,7 @@ ENTRY(__kvm_tlb_flush_vmid_ipa)
 	mcrr	p15, 6, r2, r3, c2	@ Back to VMID #0
 	isb				@ Not necessary if followed by eret
 
-	pop	{r2, r3}
+	ldmia	sp!, {r2, r3}
 	bx	lr
 ENDPROC(__kvm_tlb_flush_vmid_ipa)
 
@@ -110,7 +110,7 @@ ENTRY(__kvm_vcpu_run)
 #ifdef CONFIG_VFPv3
 	@ Set FPEXC_EN so the guest doesn't trap floating point instructions
 	VFPFMRX r2, FPEXC		@ VMRS
-	push	{r2}
+	stmdb	sp!, {r2}
 	orr	r2, r2, #FPEXC_EN
 	VFPFMXR FPEXC, r2		@ VMSR
 #endif
@@ -175,7 +175,7 @@ __kvm_vcpu_return:
 
 after_vfp_restore:
 	@ Restore FPEXC_EN which we clobbered on entry
-	pop	{r2}
+	ldmia	sp!, {r2}
 	VFPFMXR FPEXC, r2
 #endif
 
@@ -260,7 +260,7 @@ ENTRY(kvm_call_hyp)
 
 /* Handle undef, svc, pabt, or dabt by crashing with a user notice */
 .macro bad_exception exception_code, panic_str
-	push	{r0-r2}
+	stmdb   sp!, {r0-r2}
 	mrrc	p15, 6, r0, r1, c2	@ Read VTTBR
 	lsr	r1, r1, #16
 	ands	r1, r1, #0xff
@@ -338,7 +338,7 @@ hyp_hvc:
 	 * Getting here is either becuase of a trap from a guest or from calling
 	 * HVC from the host kernel, which means "switch to Hyp mode".
 	 */
-	push	{r0, r1, r2}
+	stmdb	sp!, {r0, r1, r2}
 
 	@ Check syndrome register
 	mrc	p15, 4, r1, c5, c2, 0	@ HSR
@@ -361,11 +361,11 @@ hyp_hvc:
 	bne	guest_trap		@ Guest called HVC
 
 host_switch_to_hyp:
-	pop	{r0, r1, r2}
+	ldmia	sp!, {r0, r1, r2}
 
-	push	{lr}
+	stmdb	sp!, {lr}
 	mrs	lr, SPSR
-	push	{lr}
+	stmdb	sp!, {lr}
 
 	mov	lr, r0
 	mov	r0, r1
@@ -375,9 +375,9 @@ host_switch_to_hyp:
 THUMB(	orr	lr, #1)
 	blx	lr			@ Call the HYP function
 
-	pop	{lr}
+	ldmia	sp!, {lr}
 	msr	SPSR_csxf, lr
-	pop	{lr}
+	ldmia	sp!, {lr}
 	eret
 
 guest_trap:
@@ -418,7 +418,7 @@ guest_trap:
 
 	/* Preserve PAR */
 	mrrc	p15, 0, r0, r1, c7	@ PAR
-	push	{r0, r1}
+	stmdb	sp!, {r0, r1}
 
 	/* Resolve IPA using the xFAR */
 	mcr	p15, 0, r2, c7, c8, 0	@ ATS1CPR
@@ -431,7 +431,7 @@ guest_trap:
 	orr	r2, r2, r1, lsl #24
 
 	/* Restore PAR */
-	pop	{r0, r1}
+	ldmia	sp!, {r0, r1}
 	mcrr	p15, 0, r0, r1, c7	@ PAR
 
 3:	load_vcpu			@ Load VCPU pointer to r0
@@ -440,10 +440,10 @@ guest_trap:
 1:	mov	r1, #ARM_EXCEPTION_HVC
 	b	__kvm_vcpu_return
 
-4:	pop	{r0, r1}		@ Failed translation, return to guest
+4:	ldmia	sp!, {r0, r1}		@ Failed translation, return to guest
 	mcrr	p15, 0, r0, r1, c7	@ PAR
 	clrex
-	pop	{r0, r1, r2}
+	ldmia	sp!, {r0, r1, r2}
 	eret
 
 /*
@@ -455,7 +455,7 @@ guest_trap:
 #ifdef CONFIG_VFPv3
 switch_to_guest_vfp:
 	load_vcpu			@ Load VCPU pointer to r0
-	push	{r3-r7}
+	stmdb	sp!, {r3-r7}
 
 	@ NEON/VFP used.  Turn on VFP access.
 	set_hcptr vmexit, (HCPTR_TCP(10) | HCPTR_TCP(11))
@@ -467,15 +467,15 @@ switch_to_guest_vfp:
 	add	r7, r0, #VCPU_VFP_GUEST
 	restore_vfp_state r7
 
-	pop	{r3-r7}
-	pop	{r0-r2}
+	ldmia	sp!, {r3-r7}
+	ldmia	sp!, {r0-r2}
 	clrex
 	eret
 #endif
 
 	.align
 hyp_irq:
-	push	{r0, r1, r2}
+	stmdb	sp!, {r0, r1, r2}
 	mov	r1, #ARM_EXCEPTION_IRQ
 	load_vcpu			@ Load VCPU pointer to r0
 	b	__kvm_vcpu_return
diff --git a/arch/arm/kvm/interrupts_head.S b/arch/arm/kvm/interrupts_head.S
index 6f18695..c371db7 100644
--- a/arch/arm/kvm/interrupts_head.S
+++ b/arch/arm/kvm/interrupts_head.S
@@ -63,7 +63,7 @@ vcpu	.req	r0		@ vcpu pointer always in r0
 	mrs	r2, SP_\mode
 	mrs	r3, LR_\mode
 	mrs	r4, SPSR_\mode
-	push	{r2, r3, r4}
+	stmdb	sp!, {r2, r3, r4}
 .endm
 
 /*
@@ -73,13 +73,13 @@ vcpu	.req	r0		@ vcpu pointer always in r0
 .macro save_host_regs
 	/* Hyp regs. Only ELR_hyp (SPSR_hyp already saved) */
 	mrs	r2, ELR_hyp
-	push	{r2}
+	stmdb	sp!, {r2}
 
 	/* usr regs */
-	push	{r4-r12}	@ r0-r3 are always clobbered
+	stmdb	sp!, {r4-r12}	@ r0-r3 are always clobbered
 	mrs	r2, SP_usr
 	mov	r3, lr
-	push	{r2, r3}
+	stmdb	sp!, {r2, r3}
 
 	push_host_regs_mode svc
 	push_host_regs_mode abt
@@ -95,11 +95,11 @@ vcpu	.req	r0		@ vcpu pointer always in r0
 	mrs	r7, SP_fiq
 	mrs	r8, LR_fiq
 	mrs	r9, SPSR_fiq
-	push	{r2-r9}
+	stmdb	sp!, {r2-r9}
 .endm
 
 .macro pop_host_regs_mode mode
-	pop	{r2, r3, r4}
+	ldmia	sp!, {r2, r3, r4}
 	msr	SP_\mode, r2
 	msr	LR_\mode, r3
 	msr	SPSR_\mode, r4
@@ -110,7 +110,7 @@ vcpu	.req	r0		@ vcpu pointer always in r0
  * Clobbers all registers, in all modes, except r0 and r1.
  */
 .macro restore_host_regs
-	pop	{r2-r9}
+	ldmia	sp!, {r2-r9}
 	msr	r8_fiq, r2
 	msr	r9_fiq, r3
 	msr	r10_fiq, r4
@@ -125,12 +125,12 @@ vcpu	.req	r0		@ vcpu pointer always in r0
 	pop_host_regs_mode abt
 	pop_host_regs_mode svc
 
-	pop	{r2, r3}
+	ldmia	sp!, {r2, r3}
 	msr	SP_usr, r2
 	mov	lr, r3
-	pop	{r4-r12}
+	ldmia	sp!, {r4-r12}
 
-	pop	{r2}
+	ldmia	sp!, {r2}
 	msr	ELR_hyp, r2
 .endm
 
@@ -218,7 +218,7 @@ vcpu	.req	r0		@ vcpu pointer always in r0
 	add	r2, vcpu, #VCPU_USR_REG(3)
 	stm	r2, {r3-r12}
 	add	r2, vcpu, #VCPU_USR_REG(0)
-	pop	{r3, r4, r5}		@ r0, r1, r2
+	ldmia	sp!, {r3, r4, r5}		@ r0, r1, r2
 	stm	r2, {r3, r4, r5}
 	mrs	r2, SP_usr
 	mov	r3, lr
@@ -258,7 +258,7 @@ vcpu	.req	r0		@ vcpu pointer always in r0
 	mrc	p15, 2, r12, c0, c0, 0	@ CSSELR
 
 	.if \store_to_vcpu == 0
-	push	{r2-r12}		@ Push CP15 registers
+	stmdb	sp!, {r2-r12}		@ Push CP15 registers
 	.else
 	str	r2, [vcpu, #CP15_OFFSET(c1_SCTLR)]
 	str	r3, [vcpu, #CP15_OFFSET(c1_CPACR)]
@@ -286,7 +286,7 @@ vcpu	.req	r0		@ vcpu pointer always in r0
 	mrc	p15, 0, r12, c12, c0, 0	@ VBAR
 
 	.if \store_to_vcpu == 0
-	push	{r2-r12}		@ Push CP15 registers
+	stmdb	sp!, {r2-r12}		@ Push CP15 registers
 	.else
 	str	r2, [vcpu, #CP15_OFFSET(c13_CID)]
 	str	r3, [vcpu, #CP15_OFFSET(c13_TID_URW)]
@@ -305,7 +305,7 @@ vcpu	.req	r0		@ vcpu pointer always in r0
 	mrrc	p15, 0, r4, r5, c7	@ PAR
 
 	.if \store_to_vcpu == 0
-	push	{r2,r4-r5}
+	stmdb	sp!, {r2,r4-r5}
 	.else
 	str	r2, [vcpu, #CP15_OFFSET(c14_CNTKCTL)]
 	add	r12, vcpu, #CP15_OFFSET(c7_PAR)
@@ -322,7 +322,7 @@ vcpu	.req	r0		@ vcpu pointer always in r0
  */
 .macro write_cp15_state read_from_vcpu
 	.if \read_from_vcpu == 0
-	pop	{r2,r4-r5}
+	ldmia	sp!, {r2,r4-r5}
 	.else
 	ldr	r2, [vcpu, #CP15_OFFSET(c14_CNTKCTL)]
 	add	r12, vcpu, #CP15_OFFSET(c7_PAR)
@@ -333,7 +333,7 @@ vcpu	.req	r0		@ vcpu pointer always in r0
 	mcrr	p15, 0, r4, r5, c7	@ PAR
 
 	.if \read_from_vcpu == 0
-	pop	{r2-r12}
+	ldmia	sp!, {r2-r12}
 	.else
 	ldr	r2, [vcpu, #CP15_OFFSET(c13_CID)]
 	ldr	r3, [vcpu, #CP15_OFFSET(c13_TID_URW)]
@@ -361,7 +361,7 @@ vcpu	.req	r0		@ vcpu pointer always in r0
 	mcr	p15, 0, r12, c12, c0, 0	@ VBAR
 
 	.if \read_from_vcpu == 0
-	pop	{r2-r12}
+	ldmia	sp!, {r2-r12}
 	.else
 	ldr	r2, [vcpu, #CP15_OFFSET(c1_SCTLR)]
 	ldr	r3, [vcpu, #CP15_OFFSET(c1_CPACR)]
-- 
1.8.1.4

^ permalink raw reply related	[flat|nested] 8+ messages in thread

* [PATCH 2/5] ARM: fix KVM assembler files to work in BE case
  2013-12-20  7:26 [PATCH 0/5] armv7 BE kvm support Victor Kamensky
  2013-12-20  7:26 ` [PATCH 1/5] ARM: kvm: replace push and pop with stdmb and ldmia instrs to enable assembler.h inclusion Victor Kamensky
@ 2013-12-20  7:26 ` Victor Kamensky
  2013-12-20  7:26 ` [PATCH 3/5] ARM: kvm one_reg coproc set and get BE fixes Victor Kamensky
                   ` (3 subsequent siblings)
  5 siblings, 0 replies; 8+ messages in thread
From: Victor Kamensky @ 2013-12-20  7:26 UTC (permalink / raw)
  To: linux-arm-kernel

ARM v7 KVM assembler files fixes to work in big endian mode:

vgic h/w registers are little endian; when asm code reads/writes from/to
them, it needs to do byteswap after/before. Byte swap code uses ARM_BE8
wrapper to add swap only if BIG_ENDIAN kernel is configured

mcrr and mrrc instructions take couple 32 bit registers as argument, one
is supposed to be high part of 64 bit value and another is low part of
64 bit. Typically those values are loaded/stored with ldrd and strd
instructions and those will load high and low parts in opposite register
depending on endianity. Introduce and use rr_lo_hi macro that swap
registers in BE mode when they are passed to mcrr and mrrc instructions.

function that returns 64 bit result __kvm_vcpu_run in couple registers
has to be adjusted for BE case.

Signed-off-by: Victor Kamensky <victor.kamensky@linaro.org>
---
 arch/arm/include/asm/assembler.h |  7 +++++++
 arch/arm/include/asm/kvm_asm.h   |  4 ++--
 arch/arm/kvm/init.S              |  7 +++++--
 arch/arm/kvm/interrupts.S        | 12 +++++++++---
 arch/arm/kvm/interrupts_head.S   | 27 ++++++++++++++++++++-------
 5 files changed, 43 insertions(+), 14 deletions(-)

diff --git a/arch/arm/include/asm/assembler.h b/arch/arm/include/asm/assembler.h
index 5c22851..ad1ad31 100644
--- a/arch/arm/include/asm/assembler.h
+++ b/arch/arm/include/asm/assembler.h
@@ -60,6 +60,13 @@
 #define ARM_BE8(code...)
 #endif
 
+/* swap pair of registers position depending on current endianity */
+#ifdef CONFIG_CPU_ENDIAN_BE8
+#define rr_lo_hi(a1, a2) a2, a1
+#else
+#define rr_lo_hi(a1, a2) a1, a2
+#endif
+
 /*
  * Data preload for architectures that support it
  */
diff --git a/arch/arm/include/asm/kvm_asm.h b/arch/arm/include/asm/kvm_asm.h
index 661da11..12981d6 100644
--- a/arch/arm/include/asm/kvm_asm.h
+++ b/arch/arm/include/asm/kvm_asm.h
@@ -26,9 +26,9 @@
 #define c1_ACTLR	4	/* Auxilliary Control Register */
 #define c1_CPACR	5	/* Coprocessor Access Control */
 #define c2_TTBR0	6	/* Translation Table Base Register 0 */
-#define c2_TTBR0_high	7	/* TTBR0 top 32 bits */
+#define c2_TTBR0_hilo	7	/* TTBR0 top 32 bits in LE case, low 32 bits in BE case */
 #define c2_TTBR1	8	/* Translation Table Base Register 1 */
-#define c2_TTBR1_high	9	/* TTBR1 top 32 bits */
+#define c2_TTBR1_hilo	9	/* TTBR1 top 32 bits in LE case, low 32 bits in BE case */
 #define c2_TTBCR	10	/* Translation Table Base Control R. */
 #define c3_DACR		11	/* Domain Access Control Register */
 #define c5_DFSR		12	/* Data Fault Status Register */
diff --git a/arch/arm/kvm/init.S b/arch/arm/kvm/init.S
index 1b9844d..2d10b2d 100644
--- a/arch/arm/kvm/init.S
+++ b/arch/arm/kvm/init.S
@@ -22,6 +22,7 @@
 #include <asm/kvm_asm.h>
 #include <asm/kvm_arm.h>
 #include <asm/kvm_mmu.h>
+#include <asm/assembler.h>
 
 /********************************************************************
  * Hypervisor initialization
@@ -70,8 +71,10 @@ __do_hyp_init:
 	cmp	r0, #0			@ We have a SP?
 	bne	phase2			@ Yes, second stage init
 
+ARM_BE8(setend	be) @ Switch to Big Endian mode if needed
+
 	@ Set the HTTBR to point to the hypervisor PGD pointer passed
-	mcrr	p15, 4, r2, r3, c2
+	mcrr	p15, 4, rr_lo_hi(r2, r3), c2
 
 	@ Set the HTCR and VTCR to the same shareability and cacheability
 	@ settings as the non-secure TTBCR and with T0SZ == 0.
@@ -137,7 +140,7 @@ phase2:
 	mov	pc, r0
 
 target:	@ We're now in the trampoline code, switch page tables
-	mcrr	p15, 4, r2, r3, c2
+	mcrr	p15, 4, rr_lo_hi(r2, r3), c2
 	isb
 
 	@ Invalidate the old TLBs
diff --git a/arch/arm/kvm/interrupts.S b/arch/arm/kvm/interrupts.S
index df19133..0784ec3 100644
--- a/arch/arm/kvm/interrupts.S
+++ b/arch/arm/kvm/interrupts.S
@@ -25,6 +25,7 @@
 #include <asm/kvm_asm.h>
 #include <asm/kvm_arm.h>
 #include <asm/vfpmacros.h>
+#include <asm/assembler.h>
 #include "interrupts_head.S"
 
 	.text
@@ -52,14 +53,14 @@ ENTRY(__kvm_tlb_flush_vmid_ipa)
 	dsb	ishst
 	add	r0, r0, #KVM_VTTBR
 	ldrd	r2, r3, [r0]
-	mcrr	p15, 6, r2, r3, c2	@ Write VTTBR
+	mcrr	p15, 6, rr_lo_hi(r2, r3), c2	@ Write VTTBR
 	isb
 	mcr     p15, 0, r0, c8, c3, 0	@ TLBIALLIS (rt ignored)
 	dsb	ish
 	isb
 	mov	r2, #0
 	mov	r3, #0
-	mcrr	p15, 6, r2, r3, c2	@ Back to VMID #0
+	mcrr	p15, 6, rr_lo_hi(r2, r3), c2	@ Back to VMID #0
 	isb				@ Not necessary if followed by eret
 
 	ldmia	sp!, {r2, r3}
@@ -135,7 +136,7 @@ ENTRY(__kvm_vcpu_run)
 	ldr	r1, [vcpu, #VCPU_KVM]
 	add	r1, r1, #KVM_VTTBR
 	ldrd	r2, r3, [r1]
-	mcrr	p15, 6, r2, r3, c2	@ Write VTTBR
+	mcrr	p15, 6, rr_lo_hi(r2, r3), c2	@ Write VTTBR
 
 	@ We're all done, just restore the GPRs and go to the guest
 	restore_guest_regs
@@ -199,8 +200,13 @@ after_vfp_restore:
 
 	restore_host_regs
 	clrex				@ Clear exclusive monitor
+#ifndef __ARMEB__
 	mov	r0, r1			@ Return the return code
 	mov	r1, #0			@ Clear upper bits in return value
+#else
+	@ r1 already has return code
+	mov	r0, #0			@ Clear upper bits in return value
+#endif /* __ARMEB__ */
 	bx	lr			@ return to IOCTL
 
 /********************************************************************
diff --git a/arch/arm/kvm/interrupts_head.S b/arch/arm/kvm/interrupts_head.S
index c371db7..67b4002 100644
--- a/arch/arm/kvm/interrupts_head.S
+++ b/arch/arm/kvm/interrupts_head.S
@@ -251,8 +251,8 @@ vcpu	.req	r0		@ vcpu pointer always in r0
 	mrc	p15, 0, r3, c1, c0, 2	@ CPACR
 	mrc	p15, 0, r4, c2, c0, 2	@ TTBCR
 	mrc	p15, 0, r5, c3, c0, 0	@ DACR
-	mrrc	p15, 0, r6, r7, c2	@ TTBR 0
-	mrrc	p15, 1, r8, r9, c2	@ TTBR 1
+	mrrc	p15, 0, rr_lo_hi(r6, r7), c2	@ TTBR 0
+	mrrc	p15, 1, rr_lo_hi(r8, r9), c2	@ TTBR 1
 	mrc	p15, 0, r10, c10, c2, 0	@ PRRR
 	mrc	p15, 0, r11, c10, c2, 1	@ NMRR
 	mrc	p15, 2, r12, c0, c0, 0	@ CSSELR
@@ -380,8 +380,8 @@ vcpu	.req	r0		@ vcpu pointer always in r0
 	mcr	p15, 0, r3, c1, c0, 2	@ CPACR
 	mcr	p15, 0, r4, c2, c0, 2	@ TTBCR
 	mcr	p15, 0, r5, c3, c0, 0	@ DACR
-	mcrr	p15, 0, r6, r7, c2	@ TTBR 0
-	mcrr	p15, 1, r8, r9, c2	@ TTBR 1
+	mcrr	p15, 0, rr_lo_hi(r6, r7), c2	@ TTBR 0
+	mcrr	p15, 1, rr_lo_hi(r8, r9), c2	@ TTBR 1
 	mcr	p15, 0, r10, c10, c2, 0	@ PRRR
 	mcr	p15, 0, r11, c10, c2, 1	@ NMRR
 	mcr	p15, 2, r12, c0, c0, 0	@ CSSELR
@@ -413,13 +413,21 @@ vcpu	.req	r0		@ vcpu pointer always in r0
 	ldr	r9, [r2, #GICH_ELRSR1]
 	ldr	r10, [r2, #GICH_APR]
 
+ARM_BE8(rev	r3, r3	)
 	str	r3, [r11, #VGIC_CPU_HCR]
+ARM_BE8(rev	r4, r4	)
 	str	r4, [r11, #VGIC_CPU_VMCR]
+ARM_BE8(rev	r5, r5	)
 	str	r5, [r11, #VGIC_CPU_MISR]
+ARM_BE8(rev	r6, r6	)
 	str	r6, [r11, #VGIC_CPU_EISR]
+ARM_BE8(rev	r7, r7	)
 	str	r7, [r11, #(VGIC_CPU_EISR + 4)]
+ARM_BE8(rev	r8, r8	)
 	str	r8, [r11, #VGIC_CPU_ELRSR]
+ARM_BE8(rev	r9, r9	)
 	str	r9, [r11, #(VGIC_CPU_ELRSR + 4)]
+ARM_BE8(rev	r10, r10	)
 	str	r10, [r11, #VGIC_CPU_APR]
 
 	/* Clear GICH_HCR */
@@ -431,6 +439,7 @@ vcpu	.req	r0		@ vcpu pointer always in r0
 	add	r3, r11, #VGIC_CPU_LR
 	ldr	r4, [r11, #VGIC_CPU_NR_LR]
 1:	ldr	r6, [r2], #4
+ARM_BE8(rev	r6, r6	)
 	str	r6, [r3], #4
 	subs	r4, r4, #1
 	bne	1b
@@ -459,8 +468,11 @@ vcpu	.req	r0		@ vcpu pointer always in r0
 	ldr	r4, [r11, #VGIC_CPU_VMCR]
 	ldr	r8, [r11, #VGIC_CPU_APR]
 
+ARM_BE8(rev	r3, r3	)
 	str	r3, [r2, #GICH_HCR]
+ARM_BE8(rev	r4, r4	)
 	str	r4, [r2, #GICH_VMCR]
+ARM_BE8(rev	r8, r8	)
 	str	r8, [r2, #GICH_APR]
 
 	/* Restore list registers */
@@ -468,6 +480,7 @@ vcpu	.req	r0		@ vcpu pointer always in r0
 	add	r3, r11, #VGIC_CPU_LR
 	ldr	r4, [r11, #VGIC_CPU_NR_LR]
 1:	ldr	r6, [r3], #4
+ARM_BE8(rev	r6, r6  )
 	str	r6, [r2], #4
 	subs	r4, r4, #1
 	bne	1b
@@ -498,7 +511,7 @@ vcpu	.req	r0		@ vcpu pointer always in r0
 	mcr	p15, 0, r2, c14, c3, 1	@ CNTV_CTL
 	isb
 
-	mrrc	p15, 3, r2, r3, c14	@ CNTV_CVAL
+	mrrc	p15, 3, rr_lo_hi(r2, r3), c14	@ CNTV_CVAL
 	ldr	r4, =VCPU_TIMER_CNTV_CVAL
 	add	r5, vcpu, r4
 	strd	r2, r3, [r5]
@@ -538,12 +551,12 @@ vcpu	.req	r0		@ vcpu pointer always in r0
 
 	ldr	r2, [r4, #KVM_TIMER_CNTVOFF]
 	ldr	r3, [r4, #(KVM_TIMER_CNTVOFF + 4)]
-	mcrr	p15, 4, r2, r3, c14	@ CNTVOFF
+	mcrr	p15, 4, rr_lo_hi(r2, r3), c14	@ CNTVOFF
 
 	ldr	r4, =VCPU_TIMER_CNTV_CVAL
 	add	r5, vcpu, r4
 	ldrd	r2, r3, [r5]
-	mcrr	p15, 3, r2, r3, c14	@ CNTV_CVAL
+	mcrr	p15, 3, rr_lo_hi(r2, r3), c14	@ CNTV_CVAL
 	isb
 
 	ldr	r2, [vcpu, #VCPU_TIMER_CNTV_CTL]
-- 
1.8.1.4

^ permalink raw reply related	[flat|nested] 8+ messages in thread

* [PATCH 3/5] ARM: kvm one_reg coproc set and get BE fixes
  2013-12-20  7:26 [PATCH 0/5] armv7 BE kvm support Victor Kamensky
  2013-12-20  7:26 ` [PATCH 1/5] ARM: kvm: replace push and pop with stdmb and ldmia instrs to enable assembler.h inclusion Victor Kamensky
  2013-12-20  7:26 ` [PATCH 2/5] ARM: fix KVM assembler files to work in BE case Victor Kamensky
@ 2013-12-20  7:26 ` Victor Kamensky
  2013-12-20  7:26 ` [PATCH 4/5] ARM: kvm vgic mmio should return data in BE format in BE case Victor Kamensky
                   ` (2 subsequent siblings)
  5 siblings, 0 replies; 8+ messages in thread
From: Victor Kamensky @ 2013-12-20  7:26 UTC (permalink / raw)
  To: linux-arm-kernel

This patch fixes issue of reading and writing
ARM V7 registers values from/to user land. Existing code was designed to
work only in LE case.

struct kvm_one_reg
------------------

registers value passed through kvm_one_reg structure. It is used by
KVM_SET_ONE_REG, KVM_GET_ONE_REG ioctls. Note by looking at structure
itself we cannot tell what is size of register. Note that structure carries
address of user memory, 'addr' where register should be read or written

Setting register (from user-land to kvm)
----------------------------------------

kvm_arm_set_reg takes vcpu and pointer to struct kvm_one_reg which already
read from user space

kvm_arm_set_reg calls set_core_reg or kvm_arm_coproc_set_reg

set_core_reg deals only with 4 bytes registers, it just reads 4 bytes from
user space and store it properly into vcpu->arch.regs

kvm_arm_coproc_set_reg deals with registers of different size. At certain
point code reaches phase where it retrieves description of register by id
and it knows register size, which could be either 4 or 8 bytes. Kernel code
is ready to read values from user space, but destination type may vary. It
could be pointer to 32 bit integer or it could be pointer to 64 bit
integer. And all possible permutation of size and destination pointer are
possible. Depending on destination pointer type, 4 bytes or 8 bytes, two
new helper functions are introduced  - reg_from_user32 and reg_from_user64.
They are used instead of reg_from_user function which could work only in
LE case.

Size  sizeof(*DstInt)   Function used to read from user
   4                4   reg_from_user32
   8                4   reg_from_user32 - read two registers
   4                8   reg_from_user64 - need special handling for BE
   8                8   reg_from_user64

Getting register (to user-land from kvm)
----------------------------------------

Situation with reading registers is similar to writing. Integer pointer
type of register to be copied could be 4 or 8 bytes. And size passed in
struct kvm_one_reg could be 4 or 8. And any permutation is possible.
Depending on src pointer type, 4 bytes or 8 bytes, two new helper functions
are introduced - reg_from_user32 and reg_to_user64. They are used instead
of reg_to_user function, which could work only in LE case.

Size  sizeof(*SrcInt)   Function used to write to user
   4                4   reg_to_user32
   8                4   reg_to_user32 - writes two registers
   4                8   reg_to_user64 - need special handleing for BE
   8                8   reg_to_user64

Note code does assume that it can only deals with 4 or 8 byte registers.

Signed-off-by: Victor Kamensky <victor.kamensky@linaro.org>
---
 arch/arm/kvm/coproc.c | 94 +++++++++++++++++++++++++++++++++++++--------------
 1 file changed, 69 insertions(+), 25 deletions(-)

diff --git a/arch/arm/kvm/coproc.c b/arch/arm/kvm/coproc.c
index 78c0885..64b2b94 100644
--- a/arch/arm/kvm/coproc.c
+++ b/arch/arm/kvm/coproc.c
@@ -634,17 +634,61 @@ static struct coproc_reg invariant_cp15[] = {
 	{ CRn( 0), CRm( 0), Op1( 1), Op2( 7), is32, NULL, get_AIDR },
 };
 
-static int reg_from_user(void *val, const void __user *uaddr, u64 id)
+static int reg_from_user64(u64 *val, const void __user *uaddr, u64 id)
+{
+	unsigned long regsize = KVM_REG_SIZE(id);
+	union {
+		u32	word;
+		u64	dword;
+	} tmp = {0};
+
+	if (copy_from_user(&tmp, uaddr, regsize) != 0)
+		return -EFAULT;
+
+	switch (regsize) {
+	case 4:
+		*val = tmp.word;
+		break;
+	case 8:
+		*val = tmp.dword;
+		break;
+	}
+	return 0;
+}
+
+/* Note it may really copy two u32 registers */
+static int reg_from_user32(u32 *val, const void __user *uaddr, u64 id)
 {
-	/* This Just Works because we are little endian. */
 	if (copy_from_user(val, uaddr, KVM_REG_SIZE(id)) != 0)
 		return -EFAULT;
 	return 0;
 }
 
-static int reg_to_user(void __user *uaddr, const void *val, u64 id)
+static int reg_to_user64(void __user *uaddr, const u64 *val, u64 id)
+{
+	unsigned long regsize = KVM_REG_SIZE(id);
+	union {
+		u32	word;
+		u64	dword;
+	} tmp;
+
+	switch (regsize) {
+	case 4:
+		tmp.word = *val;
+		break;
+	case 8:
+		tmp.dword = *val;
+		break;
+	}
+
+	if (copy_to_user(uaddr, &tmp, regsize) != 0)
+		return -EFAULT;
+	return 0;
+}
+
+/* Note it may really copy two u32 registers */
+static int reg_to_user32(void __user *uaddr, const u32 *val, u64 id)
 {
-	/* This Just Works because we are little endian. */
 	if (copy_to_user(uaddr, val, KVM_REG_SIZE(id)) != 0)
 		return -EFAULT;
 	return 0;
@@ -662,7 +706,7 @@ static int get_invariant_cp15(u64 id, void __user *uaddr)
 	if (!r)
 		return -ENOENT;
 
-	return reg_to_user(uaddr, &r->val, id);
+	return reg_to_user64(uaddr, &r->val, id);
 }
 
 static int set_invariant_cp15(u64 id, void __user *uaddr)
@@ -678,7 +722,7 @@ static int set_invariant_cp15(u64 id, void __user *uaddr)
 	if (!r)
 		return -ENOENT;
 
-	err = reg_from_user(&val, uaddr, id);
+	err = reg_from_user64(&val, uaddr, id);
 	if (err)
 		return err;
 
@@ -846,7 +890,7 @@ static int vfp_get_reg(const struct kvm_vcpu *vcpu, u64 id, void __user *uaddr)
 	if (vfpid < num_fp_regs()) {
 		if (KVM_REG_SIZE(id) != 8)
 			return -ENOENT;
-		return reg_to_user(uaddr, &vcpu->arch.vfp_guest.fpregs[vfpid],
+		return reg_to_user64(uaddr, &vcpu->arch.vfp_guest.fpregs[vfpid],
 				   id);
 	}
 
@@ -856,22 +900,22 @@ static int vfp_get_reg(const struct kvm_vcpu *vcpu, u64 id, void __user *uaddr)
 
 	switch (vfpid) {
 	case KVM_REG_ARM_VFP_FPEXC:
-		return reg_to_user(uaddr, &vcpu->arch.vfp_guest.fpexc, id);
+		return reg_to_user32(uaddr, &vcpu->arch.vfp_guest.fpexc, id);
 	case KVM_REG_ARM_VFP_FPSCR:
-		return reg_to_user(uaddr, &vcpu->arch.vfp_guest.fpscr, id);
+		return reg_to_user32(uaddr, &vcpu->arch.vfp_guest.fpscr, id);
 	case KVM_REG_ARM_VFP_FPINST:
-		return reg_to_user(uaddr, &vcpu->arch.vfp_guest.fpinst, id);
+		return reg_to_user32(uaddr, &vcpu->arch.vfp_guest.fpinst, id);
 	case KVM_REG_ARM_VFP_FPINST2:
-		return reg_to_user(uaddr, &vcpu->arch.vfp_guest.fpinst2, id);
+		return reg_to_user32(uaddr, &vcpu->arch.vfp_guest.fpinst2, id);
 	case KVM_REG_ARM_VFP_MVFR0:
 		val = fmrx(MVFR0);
-		return reg_to_user(uaddr, &val, id);
+		return reg_to_user32(uaddr, &val, id);
 	case KVM_REG_ARM_VFP_MVFR1:
 		val = fmrx(MVFR1);
-		return reg_to_user(uaddr, &val, id);
+		return reg_to_user32(uaddr, &val, id);
 	case KVM_REG_ARM_VFP_FPSID:
 		val = fmrx(FPSID);
-		return reg_to_user(uaddr, &val, id);
+		return reg_to_user32(uaddr, &val, id);
 	default:
 		return -ENOENT;
 	}
@@ -890,8 +934,8 @@ static int vfp_set_reg(struct kvm_vcpu *vcpu, u64 id, const void __user *uaddr)
 	if (vfpid < num_fp_regs()) {
 		if (KVM_REG_SIZE(id) != 8)
 			return -ENOENT;
-		return reg_from_user(&vcpu->arch.vfp_guest.fpregs[vfpid],
-				     uaddr, id);
+		return reg_from_user64(&vcpu->arch.vfp_guest.fpregs[vfpid],
+				       uaddr, id);
 	}
 
 	/* FP control registers are all 32 bit. */
@@ -900,28 +944,28 @@ static int vfp_set_reg(struct kvm_vcpu *vcpu, u64 id, const void __user *uaddr)
 
 	switch (vfpid) {
 	case KVM_REG_ARM_VFP_FPEXC:
-		return reg_from_user(&vcpu->arch.vfp_guest.fpexc, uaddr, id);
+		return reg_from_user32(&vcpu->arch.vfp_guest.fpexc, uaddr, id);
 	case KVM_REG_ARM_VFP_FPSCR:
-		return reg_from_user(&vcpu->arch.vfp_guest.fpscr, uaddr, id);
+		return reg_from_user32(&vcpu->arch.vfp_guest.fpscr, uaddr, id);
 	case KVM_REG_ARM_VFP_FPINST:
-		return reg_from_user(&vcpu->arch.vfp_guest.fpinst, uaddr, id);
+		return reg_from_user32(&vcpu->arch.vfp_guest.fpinst, uaddr, id);
 	case KVM_REG_ARM_VFP_FPINST2:
-		return reg_from_user(&vcpu->arch.vfp_guest.fpinst2, uaddr, id);
+		return reg_from_user32(&vcpu->arch.vfp_guest.fpinst2, uaddr, id);
 	/* These are invariant. */
 	case KVM_REG_ARM_VFP_MVFR0:
-		if (reg_from_user(&val, uaddr, id))
+		if (reg_from_user32(&val, uaddr, id))
 			return -EFAULT;
 		if (val != fmrx(MVFR0))
 			return -EINVAL;
 		return 0;
 	case KVM_REG_ARM_VFP_MVFR1:
-		if (reg_from_user(&val, uaddr, id))
+		if (reg_from_user32(&val, uaddr, id))
 			return -EFAULT;
 		if (val != fmrx(MVFR1))
 			return -EINVAL;
 		return 0;
 	case KVM_REG_ARM_VFP_FPSID:
-		if (reg_from_user(&val, uaddr, id))
+		if (reg_from_user32(&val, uaddr, id))
 			return -EFAULT;
 		if (val != fmrx(FPSID))
 			return -EINVAL;
@@ -968,7 +1012,7 @@ int kvm_arm_coproc_get_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg)
 		return get_invariant_cp15(reg->id, uaddr);
 
 	/* Note: copies two regs if size is 64 bit. */
-	return reg_to_user(uaddr, &vcpu->arch.cp15[r->reg], reg->id);
+	return reg_to_user32(uaddr, &vcpu->arch.cp15[r->reg], reg->id);
 }
 
 int kvm_arm_coproc_set_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg)
@@ -987,7 +1031,7 @@ int kvm_arm_coproc_set_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg)
 		return set_invariant_cp15(reg->id, uaddr);
 
 	/* Note: copies two regs if size is 64 bit */
-	return reg_from_user(&vcpu->arch.cp15[r->reg], uaddr, reg->id);
+	return reg_from_user32(&vcpu->arch.cp15[r->reg], uaddr, reg->id);
 }
 
 static unsigned int num_demux_regs(void)
-- 
1.8.1.4

^ permalink raw reply related	[flat|nested] 8+ messages in thread

* [PATCH 4/5] ARM: kvm vgic mmio should return data in BE format in BE case
  2013-12-20  7:26 [PATCH 0/5] armv7 BE kvm support Victor Kamensky
                   ` (2 preceding siblings ...)
  2013-12-20  7:26 ` [PATCH 3/5] ARM: kvm one_reg coproc set and get BE fixes Victor Kamensky
@ 2013-12-20  7:26 ` Victor Kamensky
  2013-12-20  7:26 ` [PATCH 5/5] ARM: kvm MMIO support BE host running LE code Victor Kamensky
  2013-12-20  8:12 ` [PATCH 0/5] armv7 BE kvm support Christoffer Dall
  5 siblings, 0 replies; 8+ messages in thread
From: Victor Kamensky @ 2013-12-20  7:26 UTC (permalink / raw)
  To: linux-arm-kernel

KVM mmio in BE case assumes that data it recieves is in BE format. Vgic
operates in LE, so need byteswap data in BE case.

Signed-off-by: Victor Kamensky <victor.kamensky@linaro.org>
---
 virt/kvm/arm/vgic.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
index 685fc72..7e11458 100644
--- a/virt/kvm/arm/vgic.c
+++ b/virt/kvm/arm/vgic.c
@@ -236,12 +236,12 @@ static void vgic_cpu_irq_clear(struct kvm_vcpu *vcpu, int irq)
 
 static u32 mmio_data_read(struct kvm_exit_mmio *mmio, u32 mask)
 {
-	return *((u32 *)mmio->data) & mask;
+	return le32_to_cpu(*((u32 *)mmio->data)) & mask;
 }
 
 static void mmio_data_write(struct kvm_exit_mmio *mmio, u32 mask, u32 value)
 {
-	*((u32 *)mmio->data) = value & mask;
+	*((u32 *)mmio->data) = cpu_to_le32(value) & mask;
 }
 
 /**
-- 
1.8.1.4

^ permalink raw reply related	[flat|nested] 8+ messages in thread

* [PATCH 5/5] ARM: kvm MMIO support BE host running LE code
  2013-12-20  7:26 [PATCH 0/5] armv7 BE kvm support Victor Kamensky
                   ` (3 preceding siblings ...)
  2013-12-20  7:26 ` [PATCH 4/5] ARM: kvm vgic mmio should return data in BE format in BE case Victor Kamensky
@ 2013-12-20  7:26 ` Victor Kamensky
  2013-12-20  8:12 ` [PATCH 0/5] armv7 BE kvm support Christoffer Dall
  5 siblings, 0 replies; 8+ messages in thread
From: Victor Kamensky @ 2013-12-20  7:26 UTC (permalink / raw)
  To: linux-arm-kernel

In case of status register E bit is not set (LE mode) and host runs in
BE mode we need byteswap data, so read/write is emulated correctly.

Signed-off-by: Victor Kamensky <victor.kamensky@linaro.org>
---
 arch/arm/include/asm/kvm_emulate.h | 22 ++++++++++++++++++----
 1 file changed, 18 insertions(+), 4 deletions(-)

diff --git a/arch/arm/include/asm/kvm_emulate.h b/arch/arm/include/asm/kvm_emulate.h
index 0fa90c9..69b7469 100644
--- a/arch/arm/include/asm/kvm_emulate.h
+++ b/arch/arm/include/asm/kvm_emulate.h
@@ -185,9 +185,16 @@ static inline unsigned long vcpu_data_guest_to_host(struct kvm_vcpu *vcpu,
 		default:
 			return be32_to_cpu(data);
 		}
+	} else {
+		switch (len) {
+		case 1:
+			return data & 0xff;
+		case 2:
+			return le16_to_cpu(data & 0xffff);
+		default:
+			return le32_to_cpu(data);
+		}
 	}
-
-	return data;		/* Leave LE untouched */
 }
 
 static inline unsigned long vcpu_data_host_to_guest(struct kvm_vcpu *vcpu,
@@ -203,9 +210,16 @@ static inline unsigned long vcpu_data_host_to_guest(struct kvm_vcpu *vcpu,
 		default:
 			return cpu_to_be32(data);
 		}
+	} else {
+		switch (len) {
+		case 1:
+			return data & 0xff;
+		case 2:
+			return cpu_to_le16(data & 0xffff);
+		default:
+			return cpu_to_le32(data);
+		}
 	}
-
-	return data;		/* Leave LE untouched */
 }
 
 #endif /* __ARM_KVM_EMULATE_H__ */
-- 
1.8.1.4

^ permalink raw reply related	[flat|nested] 8+ messages in thread

* [PATCH 0/5] armv7 BE kvm support
  2013-12-20  7:26 [PATCH 0/5] armv7 BE kvm support Victor Kamensky
                   ` (4 preceding siblings ...)
  2013-12-20  7:26 ` [PATCH 5/5] ARM: kvm MMIO support BE host running LE code Victor Kamensky
@ 2013-12-20  8:12 ` Christoffer Dall
  5 siblings, 0 replies; 8+ messages in thread
From: Christoffer Dall @ 2013-12-20  8:12 UTC (permalink / raw)
  To: linux-arm-kernel

On Thu, Dec 19, 2013 at 11:26:54PM -0800, Victor Kamensky wrote:
> Hi Guys,

Hi Victor,

> 
> Here is series that enables KVM support for V7 big endian kernels. Mostly
> it deals with BE KVM host support. Marc Zyngier showed before with his patches
> how BE guest could run on top LE host. With these patches BE guest runs on 
> top of BE host. If Marc's kvmtool is used with few additional changes I tested
> that BE host could run LE guest. Also I verified that there were no 
> regressions in BE guest on top of LE host case.

Can you please cc the kvmarm list as per the MAINTAINERS files for
patches to kvm/arm in the future?

Looking forward to reading your code!

-Christoffer

> 
> Note that posted series covers only kernel side changes. The changes were
> tested inside of bigger setup with additional changes in qemu and kvmtool.
> I will post those changes separately in proper aliases but for completeness 
> sake Appendix A gives pointers to git repositories and branches with all
> needed changes.
> 
> Please note first patch is not related to BE KVM per se. I've run
> into an issue of conflicting 'push' identifier use while trying to include 
> assembler.h into KVM .S files. Details of an issue I observed covered in
> Appendix B. The first patch is my take on solving it.
> 
> Victor Kamensky (5):
>   ARM: kvm: replace push and pop with stdmb and ldmia instrs to enable
>     assembler.h inclusion
>   ARM: fix KVM assembler files to work in BE case
>   ARM: kvm one_reg coproc set and get BE fixes
>   ARM: kvm vgic mmio should return data in BE format in BE case
>   ARM: kvm MMIO support BE host running LE code
> 
>  arch/arm/include/asm/assembler.h   |  7 +++
>  arch/arm/include/asm/kvm_asm.h     |  4 +-
>  arch/arm/include/asm/kvm_emulate.h | 22 +++++++--
>  arch/arm/kvm/coproc.c              | 94 ++++++++++++++++++++++++++++----------
>  arch/arm/kvm/init.S                |  7 ++-
>  arch/arm/kvm/interrupts.S          | 50 +++++++++++---------
>  arch/arm/kvm/interrupts_head.S     | 61 +++++++++++++++----------
>  virt/kvm/arm/vgic.c                |  4 +-
>  8 files changed, 168 insertions(+), 81 deletions(-)
> 
> -- 
> 1.8.1.4
> 
> Thanks,
> Victor
> 
>       Appendix A: Testing and Full Setup Description
>       ----------------------------------------------
> 
> I) No mixed mode setup - i.e BE guest on BE host; and LE guest
> on LE host tested to make sure no regressions.
> 
> KVM host and guest kernels:
>   TC2 on top of Linus 3.13-rc4 (this patch series):
>       git: git://git.linaro.org/people/victor.kamensky/linux-linaro-tracking-be.git
>       branch: armv7be-kvm-3.13-rc4
>       
>   TC2 and Arndale on top of Linaro BE tree:
>       git: git://git.linaro.org/people/victor.kamensky/linux-linaro-tracking-be.git
>       branch: llct-be-20131216-kvm
> 
> - TC1 kernels used as guests
> 
> qemu:
>       git: git://git.linaro.org/people/victor.kamensky/qemu-be.git 
>       branch: armv7be-v1
>       description: changes to run qemu on armeb target; and other
>       changes to work with be image on top of be host
> 
> kvmtool:
>       git: git://git.linaro.org/people/victor.kamensky/linux-linaro-tracking-be.git
>       branch: kvmtool-armv7be-v1
>       desciption: minimal changes to build kvmtool for armeb target; and
>       tiny change with virtio magic
> 
> II) Mixed mode setup all possible combinations within V7 (LE guest on BE host;
> BE guest on LE host as Marc's setup tested to make sure no regressions) only
> with kvmtool.
> 
> This work is based on Marc Zyngier's work that made BE guest to run on top
> of LE host. For this setup special version of kvmtool should be used and
> in addition I had to apply patch to guest kernel that would switch reading
> virtio configs reads to be LE only, that is made on top of previous Rusty
> Russell's changes. Effectively I just had to do very minor addition to make
> LE guest to work on BE host, most of heavy lifting was done before by Marc.
> 
> KVM host kernels: as in previous setup
> 
> Guest TC1 kernels with LE virtio config patch:
>       git: git://git.linaro.org/people/victor.kamensky/linux-linaro-tracking-be.git
>       branch: virtio-leconfig-3.13-rc4
> 
> kvmtool:
>       git: git://git.linaro.org/people/victor.kamensky/linux-linaro-tracking-be.git
>       branch: kvmtool-mixed-v1
>       description: based on git://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms.git
>       branch kvm-arm64/kvmtool-be-on-le; adds missing include fix; above armeb target
>       build patches; and one fix related to BE mode
> 
> qemu:
>       git: git://git.linaro.org/people/victor.kamensky/qemu-be.git
>       branch: armv7be-leconfig-v1
>       description: change virtio-blk that so qemu could work with guest image
>       where virtio leconfig is made; note it does not work in mixed mode; to do
>       so qemu would need bunch of similar changes that Marc did in kvmtool
> 
> 
>       Appendix B: kvm asm file and asm/assembler.h file issue
>       -------------------------------------------------------
> 
> diff --git a/arch/arm/kvm/interrupts.S b/arch/arm/kvm/interrupts.S
> index ddc1553..5d3b511 100644
> --- a/arch/arm/kvm/interrupts.S
> +++ b/arch/arm/kvm/interrupts.S
> @@ -25,6 +25,7 @@
>  #include <asm/kvm_asm.h>
>  #include <asm/kvm_arm.h>
>  #include <asm/vfpmacros.h>
> +#include <asm/assembler.h>
>  #include "interrupts_head.S"
>  
>         .text
> 
> produce the following compilation errors:
> 
> /run/media/kamensky/wd/linaro/linux-linaro-core-tracking/092913/linux-linaro-tracking-be/arch/arm/kvm/interrupts.S: Assembler messages:
> /run/media/kamensky/wd/linaro/linux-linaro-core-tracking/092913/linux-linaro-tracking-be/arch/arm/kvm/interrupts.S:51: Error: ARM register expected -- `lsr {r2,r3}'
> /run/media/kamensky/wd/linaro/linux-linaro-core-tracking/092913/linux-linaro-tracking-be/arch/arm/kvm/interrupts.S:100: Error: ARM register expected -- `lsr {r2}'
> /run/media/kamensky/wd/linaro/linux-linaro-core-tracking/092913/linux-linaro-tracking-be/arch/arm/kvm/interrupts.S:100: Error: ARM register expected -- `lsr {r4-r12}'
> 

-- 
Christoffer

^ permalink raw reply	[flat|nested] 8+ messages in thread

* [PATCH 1/5] ARM: kvm: replace push and pop with stdmb and ldmia instrs to enable assembler.h inclusion
  2013-12-20  7:26 ` [PATCH 1/5] ARM: kvm: replace push and pop with stdmb and ldmia instrs to enable assembler.h inclusion Victor Kamensky
@ 2014-01-07 16:01   ` Dave Martin
  0 siblings, 0 replies; 8+ messages in thread
From: Dave Martin @ 2014-01-07 16:01 UTC (permalink / raw)
  To: linux-arm-kernel

On Thu, Dec 19, 2013 at 11:26:55PM -0800, Victor Kamensky wrote:
> Before fix kvm interrupt.S and interrupt_head.S used push and pop assembler
> instruction. It causes problem if <asm/assembler.h> file should be include. In
> assembler.h "push" is defined as macro so it causes compilation errors like
> this:
> 
> arch/arm/kvm/interrupts.S: Assembler messages:
> arch/arm/kvm/interrupts.S:51: Error: ARM register expected -- `lsr {r2,r3}'
> 
> Solution implemented by this patch replaces all 'push {...}' with
> 'stdmb sp!, {...}' instruction; and all 'pop {...}' with 'ldmia sp!, {...}'.
> 
> Signed-off-by: Victor Kamensky <victor.kamensky@linaro.org>
> ---
>  arch/arm/kvm/interrupts.S      | 38 +++++++++++++++++++-------------------
>  arch/arm/kvm/interrupts_head.S | 34 +++++++++++++++++-----------------
>  2 files changed, 36 insertions(+), 36 deletions(-)
> 
> diff --git a/arch/arm/kvm/interrupts.S b/arch/arm/kvm/interrupts.S
> index ddc1553..df19133 100644
> --- a/arch/arm/kvm/interrupts.S
> +++ b/arch/arm/kvm/interrupts.S
> @@ -47,7 +47,7 @@ __kvm_hyp_code_start:
>   * instead, ignoring the ipa value.
>   */
>  ENTRY(__kvm_tlb_flush_vmid_ipa)
> -	push	{r2, r3}
> +	stmdb	sp!, {r2, r3}

Minor nit: when pushing and popping the stack, the convention for kernel
code is to write

	stmfd	sp!,
	ldmfd	sp!,

instead of

	stmdb	sp!,
	ldmia	sp!,

This will have no effect on the assembly, but may be slightly easier to
read.

>  
>  	dsb	ishst
>  	add	r0, r0, #KVM_VTTBR
> @@ -62,7 +62,7 @@ ENTRY(__kvm_tlb_flush_vmid_ipa)
>  	mcrr	p15, 6, r2, r3, c2	@ Back to VMID #0
>  	isb				@ Not necessary if followed by eret
>  
> -	pop	{r2, r3}
> +	ldmia	sp!, {r2, r3}
>  	bx	lr
>  ENDPROC(__kvm_tlb_flush_vmid_ipa)
>  
> @@ -110,7 +110,7 @@ ENTRY(__kvm_vcpu_run)
>  #ifdef CONFIG_VFPv3
>  	@ Set FPEXC_EN so the guest doesn't trap floating point instructions
>  	VFPFMRX r2, FPEXC		@ VMRS
> -	push	{r2}
> +	stmdb	sp!, {r2}

For saving or restoring a single register, you may want to do:

	str	r2, [sp, #-4]!

	...

	ldr	r2, [sp], #4

This is the commonly used convention, and may be marginally more
efficient.  The "push" and "pop" assembler mnemonics generate
these instead of stm/ldm if the list only contains a single register,
whereas ldm/stm always generate a real load or store multiple even
if there is only one register in the list.

This is a trivial issue, so only worth fixing if you are going to repost
the patch anyway.

>  	orr	r2, r2, #FPEXC_EN
>  	VFPFMXR FPEXC, r2		@ VMSR
>  #endif
> @@ -175,7 +175,7 @@ __kvm_vcpu_return:
>  
>  after_vfp_restore:
>  	@ Restore FPEXC_EN which we clobbered on entry
> -	pop	{r2}
> +	ldmia	sp!, {r2}
>  	VFPFMXR FPEXC, r2
>  #endif
>  
> @@ -260,7 +260,7 @@ ENTRY(kvm_call_hyp)
>  
>  /* Handle undef, svc, pabt, or dabt by crashing with a user notice */
>  .macro bad_exception exception_code, panic_str
> -	push	{r0-r2}
> +	stmdb   sp!, {r0-r2}
>  	mrrc	p15, 6, r0, r1, c2	@ Read VTTBR
>  	lsr	r1, r1, #16
>  	ands	r1, r1, #0xff
> @@ -338,7 +338,7 @@ hyp_hvc:
>  	 * Getting here is either becuase of a trap from a guest or from calling
>  	 * HVC from the host kernel, which means "switch to Hyp mode".
>  	 */
> -	push	{r0, r1, r2}
> +	stmdb	sp!, {r0, r1, r2}
>  
>  	@ Check syndrome register
>  	mrc	p15, 4, r1, c5, c2, 0	@ HSR
> @@ -361,11 +361,11 @@ hyp_hvc:
>  	bne	guest_trap		@ Guest called HVC
>  
>  host_switch_to_hyp:
> -	pop	{r0, r1, r2}
> +	ldmia	sp!, {r0, r1, r2}
>  
> -	push	{lr}
> +	stmdb	sp!, {lr}
>  	mrs	lr, SPSR
> -	push	{lr}
> +	stmdb	sp!, {lr}
>  
>  	mov	lr, r0
>  	mov	r0, r1
> @@ -375,9 +375,9 @@ host_switch_to_hyp:
>  THUMB(	orr	lr, #1)
>  	blx	lr			@ Call the HYP function
>  
> -	pop	{lr}
> +	ldmia	sp!, {lr}
>  	msr	SPSR_csxf, lr
> -	pop	{lr}
> +	ldmia	sp!, {lr}
>  	eret
>  
>  guest_trap:
> @@ -418,7 +418,7 @@ guest_trap:
>  
>  	/* Preserve PAR */
>  	mrrc	p15, 0, r0, r1, c7	@ PAR
> -	push	{r0, r1}
> +	stmdb	sp!, {r0, r1}
>  
>  	/* Resolve IPA using the xFAR */
>  	mcr	p15, 0, r2, c7, c8, 0	@ ATS1CPR
> @@ -431,7 +431,7 @@ guest_trap:
>  	orr	r2, r2, r1, lsl #24
>  
>  	/* Restore PAR */
> -	pop	{r0, r1}
> +	ldmia	sp!, {r0, r1}
>  	mcrr	p15, 0, r0, r1, c7	@ PAR
>  
>  3:	load_vcpu			@ Load VCPU pointer to r0
> @@ -440,10 +440,10 @@ guest_trap:
>  1:	mov	r1, #ARM_EXCEPTION_HVC
>  	b	__kvm_vcpu_return
>  
> -4:	pop	{r0, r1}		@ Failed translation, return to guest
> +4:	ldmia	sp!, {r0, r1}		@ Failed translation, return to guest
>  	mcrr	p15, 0, r0, r1, c7	@ PAR
>  	clrex
> -	pop	{r0, r1, r2}
> +	ldmia	sp!, {r0, r1, r2}
>  	eret
>  
>  /*
> @@ -455,7 +455,7 @@ guest_trap:
>  #ifdef CONFIG_VFPv3
>  switch_to_guest_vfp:
>  	load_vcpu			@ Load VCPU pointer to r0
> -	push	{r3-r7}
> +	stmdb	sp!, {r3-r7}
>  
>  	@ NEON/VFP used.  Turn on VFP access.
>  	set_hcptr vmexit, (HCPTR_TCP(10) | HCPTR_TCP(11))
> @@ -467,15 +467,15 @@ switch_to_guest_vfp:
>  	add	r7, r0, #VCPU_VFP_GUEST
>  	restore_vfp_state r7
>  
> -	pop	{r3-r7}
> -	pop	{r0-r2}
> +	ldmia	sp!, {r3-r7}
> +	ldmia	sp!, {r0-r2}
>  	clrex
>  	eret
>  #endif
>  
>  	.align
>  hyp_irq:
> -	push	{r0, r1, r2}
> +	stmdb	sp!, {r0, r1, r2}
>  	mov	r1, #ARM_EXCEPTION_IRQ
>  	load_vcpu			@ Load VCPU pointer to r0
>  	b	__kvm_vcpu_return
> diff --git a/arch/arm/kvm/interrupts_head.S b/arch/arm/kvm/interrupts_head.S
> index 6f18695..c371db7 100644
> --- a/arch/arm/kvm/interrupts_head.S
> +++ b/arch/arm/kvm/interrupts_head.S
> @@ -63,7 +63,7 @@ vcpu	.req	r0		@ vcpu pointer always in r0
>  	mrs	r2, SP_\mode
>  	mrs	r3, LR_\mode
>  	mrs	r4, SPSR_\mode
> -	push	{r2, r3, r4}
> +	stmdb	sp!, {r2, r3, r4}
>  .endm
>  
>  /*
> @@ -73,13 +73,13 @@ vcpu	.req	r0		@ vcpu pointer always in r0
>  .macro save_host_regs
>  	/* Hyp regs. Only ELR_hyp (SPSR_hyp already saved) */
>  	mrs	r2, ELR_hyp
> -	push	{r2}
> +	stmdb	sp!, {r2}
>  
>  	/* usr regs */
> -	push	{r4-r12}	@ r0-r3 are always clobbered
> +	stmdb	sp!, {r4-r12}	@ r0-r3 are always clobbered
>  	mrs	r2, SP_usr
>  	mov	r3, lr
> -	push	{r2, r3}
> +	stmdb	sp!, {r2, r3}
>  
>  	push_host_regs_mode svc
>  	push_host_regs_mode abt
> @@ -95,11 +95,11 @@ vcpu	.req	r0		@ vcpu pointer always in r0
>  	mrs	r7, SP_fiq
>  	mrs	r8, LR_fiq
>  	mrs	r9, SPSR_fiq
> -	push	{r2-r9}
> +	stmdb	sp!, {r2-r9}
>  .endm
>  
>  .macro pop_host_regs_mode mode
> -	pop	{r2, r3, r4}
> +	ldmia	sp!, {r2, r3, r4}
>  	msr	SP_\mode, r2
>  	msr	LR_\mode, r3
>  	msr	SPSR_\mode, r4
> @@ -110,7 +110,7 @@ vcpu	.req	r0		@ vcpu pointer always in r0
>   * Clobbers all registers, in all modes, except r0 and r1.
>   */
>  .macro restore_host_regs
> -	pop	{r2-r9}
> +	ldmia	sp!, {r2-r9}
>  	msr	r8_fiq, r2
>  	msr	r9_fiq, r3
>  	msr	r10_fiq, r4
> @@ -125,12 +125,12 @@ vcpu	.req	r0		@ vcpu pointer always in r0
>  	pop_host_regs_mode abt
>  	pop_host_regs_mode svc
>  
> -	pop	{r2, r3}
> +	ldmia	sp!, {r2, r3}
>  	msr	SP_usr, r2
>  	mov	lr, r3
> -	pop	{r4-r12}
> +	ldmia	sp!, {r4-r12}
>  
> -	pop	{r2}
> +	ldmia	sp!, {r2}
>  	msr	ELR_hyp, r2
>  .endm
>  
> @@ -218,7 +218,7 @@ vcpu	.req	r0		@ vcpu pointer always in r0
>  	add	r2, vcpu, #VCPU_USR_REG(3)
>  	stm	r2, {r3-r12}
>  	add	r2, vcpu, #VCPU_USR_REG(0)
> -	pop	{r3, r4, r5}		@ r0, r1, r2
> +	ldmia	sp!, {r3, r4, r5}		@ r0, r1, r2
>  	stm	r2, {r3, r4, r5}
>  	mrs	r2, SP_usr
>  	mov	r3, lr
> @@ -258,7 +258,7 @@ vcpu	.req	r0		@ vcpu pointer always in r0
>  	mrc	p15, 2, r12, c0, c0, 0	@ CSSELR
>  
>  	.if \store_to_vcpu == 0
> -	push	{r2-r12}		@ Push CP15 registers
> +	stmdb	sp!, {r2-r12}		@ Push CP15 registers
>  	.else
>  	str	r2, [vcpu, #CP15_OFFSET(c1_SCTLR)]
>  	str	r3, [vcpu, #CP15_OFFSET(c1_CPACR)]
> @@ -286,7 +286,7 @@ vcpu	.req	r0		@ vcpu pointer always in r0
>  	mrc	p15, 0, r12, c12, c0, 0	@ VBAR
>  
>  	.if \store_to_vcpu == 0
> -	push	{r2-r12}		@ Push CP15 registers
> +	stmdb	sp!, {r2-r12}		@ Push CP15 registers
>  	.else
>  	str	r2, [vcpu, #CP15_OFFSET(c13_CID)]
>  	str	r3, [vcpu, #CP15_OFFSET(c13_TID_URW)]
> @@ -305,7 +305,7 @@ vcpu	.req	r0		@ vcpu pointer always in r0
>  	mrrc	p15, 0, r4, r5, c7	@ PAR
>  
>  	.if \store_to_vcpu == 0
> -	push	{r2,r4-r5}
> +	stmdb	sp!, {r2,r4-r5}
>  	.else
>  	str	r2, [vcpu, #CP15_OFFSET(c14_CNTKCTL)]
>  	add	r12, vcpu, #CP15_OFFSET(c7_PAR)
> @@ -322,7 +322,7 @@ vcpu	.req	r0		@ vcpu pointer always in r0
>   */
>  .macro write_cp15_state read_from_vcpu
>  	.if \read_from_vcpu == 0
> -	pop	{r2,r4-r5}
> +	ldmia	sp!, {r2,r4-r5}
>  	.else
>  	ldr	r2, [vcpu, #CP15_OFFSET(c14_CNTKCTL)]
>  	add	r12, vcpu, #CP15_OFFSET(c7_PAR)
> @@ -333,7 +333,7 @@ vcpu	.req	r0		@ vcpu pointer always in r0
>  	mcrr	p15, 0, r4, r5, c7	@ PAR
>  
>  	.if \read_from_vcpu == 0
> -	pop	{r2-r12}
> +	ldmia	sp!, {r2-r12}
>  	.else
>  	ldr	r2, [vcpu, #CP15_OFFSET(c13_CID)]
>  	ldr	r3, [vcpu, #CP15_OFFSET(c13_TID_URW)]
> @@ -361,7 +361,7 @@ vcpu	.req	r0		@ vcpu pointer always in r0
>  	mcr	p15, 0, r12, c12, c0, 0	@ VBAR
>  
>  	.if \read_from_vcpu == 0
> -	pop	{r2-r12}
> +	ldmia	sp!, {r2-r12}
>  	.else
>  	ldr	r2, [vcpu, #CP15_OFFSET(c1_SCTLR)]
>  	ldr	r3, [vcpu, #CP15_OFFSET(c1_CPACR)]
> -- 
> 1.8.1.4
> 
> 
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2014-01-07 16:01 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2013-12-20  7:26 [PATCH 0/5] armv7 BE kvm support Victor Kamensky
2013-12-20  7:26 ` [PATCH 1/5] ARM: kvm: replace push and pop with stdmb and ldmia instrs to enable assembler.h inclusion Victor Kamensky
2014-01-07 16:01   ` Dave Martin
2013-12-20  7:26 ` [PATCH 2/5] ARM: fix KVM assembler files to work in BE case Victor Kamensky
2013-12-20  7:26 ` [PATCH 3/5] ARM: kvm one_reg coproc set and get BE fixes Victor Kamensky
2013-12-20  7:26 ` [PATCH 4/5] ARM: kvm vgic mmio should return data in BE format in BE case Victor Kamensky
2013-12-20  7:26 ` [PATCH 5/5] ARM: kvm MMIO support BE host running LE code Victor Kamensky
2013-12-20  8:12 ` [PATCH 0/5] armv7 BE kvm support Christoffer Dall

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).