[PATCH 0/4] powerpc: Enable ILE on pSeries without H_MODE_SET

[PATCH 0/4] powerpc: Enable ILE on pSeries without H_MODE_SET

[Alexander Graf]

Howdy,

There are a few machines out there that would be pretty ppc64le capable
if only it wasn't for the hypervisor that's running on them.

The problem is that we need to run in ILE (interrupts delivered in little
endian) mode to run our normal interrupt vectors. The hypercall to enable
this mode is not available on PowerVM.

However, we can try to be smart here. Instead of configuring the hypervisor
to run interrupts in little endian mode, we can just bring ourselves back
to little endian on every interrupt.

We do this by patching every interrupt handler with a "branch" instruction
at the beginning which jumps into a tiny helper that turns on MSR.LE and
resumes the interrupt handler.

This is not the most smart thing to do performance wise, but it does have
a really nice side effect: We do not impact hypervisors that do support
H_SET_MODE. This is important, as we want to be as fast as possible on machines
that are supposed to run Little Endian guests.

As a tiny side effect we also clobber CFAR in every interrupt, rendering
the whole thing pretty useless. So we don't get nice and shiny perf and
debugging helps. Oh well.

Given that the alternatives to all of this would either be

  a) Not run at all or
  b) Incur performance penalties for "good" systems or
  c) Add significant maintenance burden on us

I'm quite convinced this is the way we want to go.


Enjoy,

Alex

Alexander Graf (4):
  powerpc: Add global exports for all interrupt vectors
  powerpc: Add relocation code for fixups
  powerpc: Add hack to make ppc64le work on hosts without ILE
  powerpc: Don't return to BE mode when we are already there

 arch/powerpc/include/asm/cputable.h    |   2 +
 arch/powerpc/kernel/Makefile           |   3 +
 arch/powerpc/kernel/exceptions-64s.S   |   5 ++
 arch/powerpc/kernel/fake_ile.S         | 101 +++++++++++++++++++++++++++++++++
 arch/powerpc/kernel/vmlinux.lds.S      |  14 +++++
 arch/powerpc/lib/feature-fixups.c      |  22 +++++++
 arch/powerpc/platforms/pseries/setup.c |  61 +++++++++++++++++++-
 7 files changed, 207 insertions(+), 1 deletion(-)
 create mode 100644 arch/powerpc/kernel/fake_ile.S

-- 
1.8.1.4

[PATCH 1/4] powerpc: Add global exports for all interrupt vectors

[Alexander Graf]

We need to access every interrupt vector we can find soon, so let's
make them all visible through names to outside code.

Signed-off-by: Alexander Graf <agraf@suse.de>
---
 arch/powerpc/kernel/exceptions-64s.S | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/arch/powerpc/kernel/exceptions-64s.S b/arch/powerpc/kernel/exceptions-64s.S
index 9f905e4..fcd039f 100644
--- a/arch/powerpc/kernel/exceptions-64s.S
+++ b/arch/powerpc/kernel/exceptions-64s.S
@@ -148,6 +148,7 @@ END_FTR_SECTION_IFSET(CPU_FTR_HVMODE | CPU_FTR_ARCH_206)
 				 NOTEST, 0x100)
 
 	. = 0x200
+	.globl machine_check_pSeries_1
 machine_check_pSeries_1:
 	/* This is moved out of line as it can be patched by FW, but
 	 * some code path might still want to branch into the original
@@ -328,24 +329,28 @@ hv_doorbell_trampoline:
 	 * trickery is thus necessary
 	 */
 	. = 0xf00
+	.global performance_monitor_pseries_trampoline
 performance_monitor_pseries_trampoline:
 	SET_SCRATCH0(r13)
 	EXCEPTION_PROLOG_0(PACA_EXGEN)
 	b	performance_monitor_pSeries
 
 	. = 0xf20
+	.global altivec_unavailable_pseries_trampoline
 altivec_unavailable_pseries_trampoline:
 	SET_SCRATCH0(r13)
 	EXCEPTION_PROLOG_0(PACA_EXGEN)
 	b	altivec_unavailable_pSeries
 
 	. = 0xf40
+	.global vsx_unavailable_pseries_trampoline
 vsx_unavailable_pseries_trampoline:
 	SET_SCRATCH0(r13)
 	EXCEPTION_PROLOG_0(PACA_EXGEN)
 	b	vsx_unavailable_pSeries
 
 	. = 0xf60
+	.global facility_unavailable_trampoline
 facility_unavailable_trampoline:
 	SET_SCRATCH0(r13)
 	EXCEPTION_PROLOG_0(PACA_EXGEN)
-- 
1.8.1.4

[PATCH 2/4] powerpc: Add relocation code for fixups

[Alexander Graf]

We need to patch an instruction that is covered by the fixup
framework. If we don't do anything about it we end up getting
our own patched instruction unpatched by nops by the fixups.

So add an export to the fixup code that allows us to tell it
that an instruction moved location in memory. This works because
we move the instruction into a different location, but still
execute it.

Signed-off-by: Alexander Graf <agraf@suse.de>
---
 arch/powerpc/include/asm/cputable.h |  2 ++
 arch/powerpc/lib/feature-fixups.c   | 22 ++++++++++++++++++++++
 2 files changed, 24 insertions(+)

diff --git a/arch/powerpc/include/asm/cputable.h b/arch/powerpc/include/asm/cputable.h
index 0d4939b..c981f99 100644
--- a/arch/powerpc/include/asm/cputable.h
+++ b/arch/powerpc/include/asm/cputable.h
@@ -99,6 +99,8 @@ extern unsigned int __start___ftr_fixup, __stop___ftr_fixup;
 extern struct cpu_spec *identify_cpu(unsigned long offset, unsigned int pvr);
 extern void do_feature_fixups(unsigned long value, void *fixup_start,
 			      void *fixup_end);
+extern void relocate_fixup_entry(void *fixup_start, void *fixup_end,
+				 void *old_addr, void *new_addr);
 
 extern const char *powerpc_base_platform;
 
diff --git a/arch/powerpc/lib/feature-fixups.c b/arch/powerpc/lib/feature-fixups.c
index 7a8a748..33996a4 100644
--- a/arch/powerpc/lib/feature-fixups.c
+++ b/arch/powerpc/lib/feature-fixups.c
@@ -151,6 +151,28 @@ void do_final_fixups(void)
 #endif
 }
 
+/*
+ * This changes the internal fixup location of a code block from
+ * old_addr to new_addr.
+ */
+void relocate_fixup_entry(void *fixup_start, void *fixup_end,
+			  void *old_addr, void *new_addr)
+{
+	struct fixup_entry *fcur, *fend;
+
+	fcur = fixup_start;
+	fend = fixup_end;
+
+	for (; fcur < fend; fcur++) {
+		long diff = (long)new_addr -
+			    (long)calc_addr(fcur, fcur->start_off);
+		if (calc_addr(fcur, fcur->start_off) == old_addr) {
+			fcur->start_off += diff;
+			fcur->end_off += diff;
+		}
+	}
+}
+
 #ifdef CONFIG_FTR_FIXUP_SELFTEST
 
 #define check(x)	\
-- 
1.8.1.4

[PATCH 3/4] powerpc: Add hack to make ppc64le work on hosts without ILE

[Alexander Graf]

Some hypervisors don't implement the H_SET_MODE hypercall that we
need to set the ILE bit in LPCR which allows us to execute interrupts
in little endian mode.

However otherwise we would be able to run on those hypervisors just
fine.

So let's be creative. This patch creates a few small helpers that
work without register clobbering that execute in big endian mode.

Every interrupt gets patched with a branch instruction as the first
instruction which jumps into their respective helper. That helper
enables MSR.LE and returns back to the original interrupt.

That way we can keep our interrupt handlers in little endian code
while the hypervisor is unaware that we need them to be in little
endian.

Signed-off-by: Alexander Graf <agraf@suse.de>
---
 arch/powerpc/kernel/Makefile           |   3 +
 arch/powerpc/kernel/fake_ile.S         | 101 +++++++++++++++++++++++++++++++++
 arch/powerpc/kernel/vmlinux.lds.S      |  14 +++++
 arch/powerpc/platforms/pseries/setup.c |  50 ++++++++++++++++
 4 files changed, 168 insertions(+)
 create mode 100644 arch/powerpc/kernel/fake_ile.S

diff --git a/arch/powerpc/kernel/Makefile b/arch/powerpc/kernel/Makefile
index 445cb6e..31842e6 100644
--- a/arch/powerpc/kernel/Makefile
+++ b/arch/powerpc/kernel/Makefile
@@ -131,6 +131,9 @@ endif
 
 obj-$(CONFIG_EPAPR_PARAVIRT)	+= epapr_paravirt.o epapr_hcalls.o
 obj-$(CONFIG_KVM_GUEST)		+= kvm.o kvm_emul.o
+ifeq ($(CONFIG_CPU_LITTLE_ENDIAN),y)
+obj-$(CONFIG_PPC_BOOK3S_64)	+= fake_ile.o
+endif
 
 # Disable GCOV in odd or sensitive code
 GCOV_PROFILE_prom_init.o := n
diff --git a/arch/powerpc/kernel/fake_ile.S b/arch/powerpc/kernel/fake_ile.S
new file mode 100644
index 0000000..21e9bd7
--- /dev/null
+++ b/arch/powerpc/kernel/fake_ile.S
@@ -0,0 +1,101 @@
+/*
+ * PowerPC helpers for hypervisors without ILE implementation
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License, version 2, as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.
+ *
+ * Copyright SUSE Linux Products GmbH 2013
+ *
+ * Authors: Alexander Graf <agraf@suse.de>
+ */
+
+#include <asm/reg.h>
+#include <asm/ppc_asm.h>
+#include <asm/asm-offsets.h>
+#include <asm/exception-64s.h>
+
+/* Little Endian fixups for hosts that don't support Little Endian */
+
+#define FAKE_ILE_HANDLER(handler, area) 			 \
+								 \
+/* This runs in BE mode */					 \
+fake_ile_##handler:						 \
+	.section __be_patch,"a"					;\
+	.llong fake_ile_##handler				;\
+	.previous						;\
+	SET_SCRATCH0(r13)					;\
+	GET_PACA(r13)						;\
+	std     r9, area + EX_R9(r13)				;\
+	std     r10, area + EX_R10(r13)				;\
+	mfsrr0	r9						;\
+	mfsrr1	r10						;\
+	std	r9, area + EX_SRR0(r13)				;\
+	std	r10, area + EX_R11(r13)				;\
+	mflr	r9						;\
+	bl	1f						;\
+	1:							;\
+	mflr	r10						;\
+	mtlr	r9						;\
+	addi	r9, r10, back_to_interrupt_##handler - 1b	;\
+	mfmsr	r10						;\
+	ori	r10, r10, MSR_LE				;\
+	mtsrr0	r9						;\
+	mtsrr1	r10						;\
+	ld	r9, area + EX_SRR0(r13)				;\
+	ld	r10, area + EX_R11(r13)				;\
+	RFI							;\
+	end_fake_ile_##handler:					;\
+	.section __be_patch,"a"					;\
+	.llong end_fake_ile_##handler				;\
+	.previous						;\
+								;\
+/* This runs in LE mode */					 \
+back_to_interrupt_##handler:					;\
+	mtsrr0	r9						;\
+	mtsrr1	r10						;\
+	li	r9, area + EX_R9				;\
+	li	r10, area + EX_R10				;\
+	ldbrx	r9, r13, r9					;\
+	ldbrx	r10, r13, r10					;\
+	GET_SCRATCH0(r13)					;\
+	/* This becomes the instruction we patched away */	 \
+	patched_insn_##handler:					;\
+	.long 0							;\
+	b 	handler + 4					;\
+								 \
+	.section __fake_ile,"a"					;\
+	.llong handler						;\
+	.llong patched_insn_##handler				;\
+	.llong fake_ile_##handler				;\
+	.previous						;\
+
+FAKE_ILE_HANDLER(system_reset_pSeries, PACA_EXMC)
+FAKE_ILE_HANDLER(machine_check_pSeries_1, PACA_EXMC)
+FAKE_ILE_HANDLER(data_access_pSeries, PACA_EXGEN)
+FAKE_ILE_HANDLER(data_access_slb_pSeries, PACA_EXSLB)
+FAKE_ILE_HANDLER(instruction_access_pSeries, PACA_EXGEN)
+FAKE_ILE_HANDLER(instruction_access_slb_pSeries, PACA_EXSLB)
+FAKE_ILE_HANDLER(hardware_interrupt_pSeries, PACA_EXGEN)
+FAKE_ILE_HANDLER(alignment_pSeries, PACA_EXGEN)
+FAKE_ILE_HANDLER(program_check_pSeries, PACA_EXGEN)
+FAKE_ILE_HANDLER(fp_unavailable_pSeries, PACA_EXGEN)
+FAKE_ILE_HANDLER(decrementer_pSeries, PACA_EXGEN)
+FAKE_ILE_HANDLER(doorbell_super_pSeries, PACA_EXGEN)
+FAKE_ILE_HANDLER(trap_0b_pSeries, PACA_EXGEN)
+FAKE_ILE_HANDLER(system_call_pSeries, PACA_EXGEN)
+FAKE_ILE_HANDLER(performance_monitor_pseries_trampoline, PACA_EXGEN)
+FAKE_ILE_HANDLER(altivec_unavailable_pseries_trampoline, PACA_EXGEN)
+FAKE_ILE_HANDLER(vsx_unavailable_pseries_trampoline, PACA_EXGEN)
+FAKE_ILE_HANDLER(facility_unavailable_trampoline, PACA_EXGEN)
+FAKE_ILE_HANDLER(instruction_breakpoint_pSeries, PACA_EXGEN)
+FAKE_ILE_HANDLER(altivec_assist_pSeries, PACA_EXGEN)
diff --git a/arch/powerpc/kernel/vmlinux.lds.S b/arch/powerpc/kernel/vmlinux.lds.S
index f096e72..7fdb7c9 100644
--- a/arch/powerpc/kernel/vmlinux.lds.S
+++ b/arch/powerpc/kernel/vmlinux.lds.S
@@ -147,6 +147,20 @@ SECTIONS
 		*(__fw_ftr_fixup)
 		__stop___fw_ftr_fixup = .;
 	}
+
+	. = ALIGN(8);
+	__fake_ile : AT(ADDR(__fake_ile) - LOAD_OFFSET) {
+		__start___fake_ile = .;
+		*(__fake_ile)
+		__stop___fake_ile = .;
+	}
+
+	. = ALIGN(8);
+	__be_patch : AT(ADDR(__be_patch) - LOAD_OFFSET) {
+		__start___be_patch = .;
+		*(__be_patch)
+		__stop___be_patch = .;
+	}
 #endif
 	.init.ramfs : AT(ADDR(.init.ramfs) - LOAD_OFFSET) {
 		INIT_RAM_FS
diff --git a/arch/powerpc/platforms/pseries/setup.c b/arch/powerpc/platforms/pseries/setup.c
index c1f1908..fb5d98c 100644
--- a/arch/powerpc/platforms/pseries/setup.c
+++ b/arch/powerpc/platforms/pseries/setup.c
@@ -67,6 +67,9 @@
 #include <asm/eeh.h>
 #include <asm/reg.h>
 #include <asm/plpar_wrappers.h>
+#include <asm/cacheflush.h>
+#include <asm/cputable.h>
+#include <asm/code-patching.h>
 
 #include "pseries.h"
 
@@ -455,6 +458,40 @@ long pseries_big_endian_exceptions(void)
 	}
 }
 
+static void swizzle_endian(u32 *start, u32 *end)
+{
+	for (; (long)start < (long)end; start++)
+		patch_instruction(start, swab32(*start));
+}
+
+static void fixup_missing_little_endian_exceptions(void)
+{
+	extern u32 *__start___fake_ile, *__stop___fake_ile;
+	extern u32 *__start___be_patch, *__stop___be_patch;
+	u32 **be_table = &__start___be_patch;
+	u32 **fake_table = &__start___fake_ile;
+
+	/* Make our big endian code look like big endian code */
+	for (; (long)be_table < (long)&__stop___be_patch; be_table += 2)
+		swizzle_endian(be_table[0], be_table[1]);
+
+	/* Now patch the interrupt handlers to branch to our BE code */
+	for (; (long)fake_table < (long)&__stop___fake_ile; fake_table += 3) {
+		u32 *le_handler = fake_table[0];
+		u32 *patched_insn = fake_table[1];
+		u32 *be_handler = fake_table[2];
+		u32 le_be_diff = (long)be_handler - (long)le_handler;
+		patch_instruction(patched_insn, *le_handler);
+		/* This patches the interrupt handler's first instruction into
+		   a branch that jumps to our BE handler that enables MSR_LE */
+		patch_instruction(le_handler, swab32(0x48000000 | le_be_diff));
+		/* Make sure that feature fixups use the new address for its
+		   code patching */
+		relocate_fixup_entry(&__start___ftr_fixup, &__stop___ftr_fixup,
+				     le_handler, patched_insn);
+	}
+}
+
 static long pseries_little_endian_exceptions(void)
 {
 	long rc;
@@ -737,6 +774,19 @@ static int __init pSeries_probe(void)
 			ppc_md.progress("H_SET_MODE LE exception fail", 0);
 			panic("Could not enable little endian exceptions");
 		}
+	} else {
+		/*
+		 * The hypervisor we're running on does not know how to
+		 * configure us to run interrupts in little endian mode,
+		 * so we have to cheat a bit.
+		 *
+		 * This call reprograms all interrupt handlers' first
+		 * instruction into a branch to a big endian fixup section
+		 * which only transitions us into little endian mode, then
+		 * returns back to the normal little endian interrupt
+		 * handler.
+		 */
+		fixup_missing_little_endian_exceptions();
 	}
 #endif
 
-- 
1.8.1.4

[PATCH 4/4] powerpc: Don't return to BE mode when we are already there

[Alexander Graf]

Our Little Endian kernels can now live in a world where they are
running with Big Endian interrupts enabled. That is great for kexec,
because now we don't have to switch back to Big Endian mode.

Indicate this in the code. Only try to go into Big Endian mode when
we're not already there yet.

Signed-off-by: Alexander Graf <agraf@suse.de>
---
 arch/powerpc/platforms/pseries/setup.c | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/arch/powerpc/platforms/pseries/setup.c b/arch/powerpc/platforms/pseries/setup.c
index fb5d98c..7db1cf1 100644
--- a/arch/powerpc/platforms/pseries/setup.c
+++ b/arch/powerpc/platforms/pseries/setup.c
@@ -446,10 +446,15 @@ static void pSeries_machine_kexec(struct kimage *image)
 #endif
 
 #ifdef __LITTLE_ENDIAN__
+static bool ile_enabled;
+
 long pseries_big_endian_exceptions(void)
 {
 	long rc;
 
+	if (!ile_enabled)
+		return H_SUCCESS;
+
 	while (1) {
 		rc = enable_big_endian_exceptions();
 		if (!H_IS_LONG_BUSY(rc))
@@ -498,8 +503,12 @@ static long pseries_little_endian_exceptions(void)
 
 	while (1) {
 		rc = enable_little_endian_exceptions();
-		if (!H_IS_LONG_BUSY(rc))
+
+		if (!H_IS_LONG_BUSY(rc)) {
+			ile_enabled = true;
 			return rc;
+		}
+
 		mdelay(get_longbusy_msecs(rc));
 	}
 }
-- 
1.8.1.4