[PATCH v5 0/6] irqchip, gicv3: Updates and Cavium ThunderX errata workarounds

[PATCH v5 0/6] irqchip, gicv3: Updates and Cavium ThunderX errata workarounds

[Robert Richter]

From: Robert Richter <rrichter@cavium.com>

This patch series adds gicv3 updates and workarounds for HW errata in
Cavium's ThunderX GICV3.

The patches has been rebased onto 4.3-rc1. Note that there are two
important fixes. See below for all changes.

The first one is an unchanged resubmission of a patch from a gicv3
series I sent a while ago.

The next patches implement the workarounds for ThunderX's gicv3. Patch
#2 implements the cpu workaround for gicv3 on ThunderX. Patch #3 is a
prerequisit for patch #5. Patch #4 adds generic code to parse the hw
revision provided by an IIDR. This patch is used for the implementa-
tion of the actual gicv3-its workaround in #5. Patch #6 updates to the
new jump label API.

All current review comments addressed so far with v5

v5:
 * fixed calling gic_read_iar_cavium_thunderx() if jump label is
   enabled
 * fix table size wrongly allocating only 4MB
 * made is_cavium_thunderx static
 * removed ARCH_THUNDER dependency for Cavium errata options to
   make it available for generic kernels
 * renamed caps names to quirk
 * introduced CAVIUM_ERRATUM_22375 config option
 * introduced ITS_FLAGS_WORKAROUND_CAVIUM_22375
 * added config descriptions
 * update to new jump label API

v4:
 * simplify code to only use cpus_have_cap() in gicv3_enable_quirks()
 * only enable hw detection for its in its_enable_quirks()
 * removed gicv3_check_capabilities()
 * drop special cpu capability for zero

v3:
 * use arm64 errata framework for midr check
 * fix mixup of errata to be dependend from midr/iidr

v2:
 * Workaround for 23154:
   * implement code in a single asm() to keep instruction sequence
   * added comment to the code that explains the erratum
   * apply workaround also if running as guest, thus check MIDR
 * adding MIDR check

Robert Richter (6):
  irqchip, gicv3-its: Add range check for number of allocated pages
  irqchip, gicv3: Workaround for Cavium ThunderX erratum 23154
  irqchip, gicv3-its: Read typer register outside the loop
  irqchip, gicv3-its: Add HW revision detection and configuration
  irqchip, gicv3-its: Workaround for Cavium ThunderX errata 22375, 24313
  irqchip, gicv3-its: Use new jump label API

 arch/arm64/Kconfig                  | 27 +++++++++++++++
 arch/arm64/include/asm/cpufeature.h |  3 +-
 arch/arm64/include/asm/cputype.h    | 17 ++++++----
 arch/arm64/kernel/cpu_errata.c      |  9 +++++
 drivers/irqchip/irq-gic-common.c    | 11 +++++++
 drivers/irqchip/irq-gic-common.h    |  9 +++++
 drivers/irqchip/irq-gic-v3-its.c    | 65 +++++++++++++++++++++++++++++++++----
 drivers/irqchip/irq-gic-v3.c        | 42 +++++++++++++++++++++++-
 include/linux/irqchip/arm-gic-v3.h  |  1 +
 9 files changed, 169 insertions(+), 15 deletions(-)

-- 
2.1.1

[PATCH v5 1/6] irqchip, gicv3-its: Add range check for number of allocated pages

[Robert Richter]

From: Robert Richter <rrichter@cavium.com>

The number of pages for the its table may exceed the maximum of 256.
Adding a range check and limitting the number to its maximum.

Based on a patch from Tirumalesh Chalamarla <tchalamarla@cavium.com>.

Signed-off-by: Tirumalesh Chalamarla <tchalamarla@cavium.com>
Reviewed-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Robert Richter <rrichter@cavium.com>
---
 drivers/irqchip/irq-gic-v3-its.c   | 11 ++++++++++-
 include/linux/irqchip/arm-gic-v3.h |  1 +
 2 files changed, 11 insertions(+), 1 deletion(-)

diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c
index 26b55c53755f..0636cb33aeea 100644
--- a/drivers/irqchip/irq-gic-v3-its.c
+++ b/drivers/irqchip/irq-gic-v3-its.c
@@ -822,6 +822,7 @@ static int its_alloc_tables(const char *node_name, struct its_node *its)
 		u64 entry_size = GITS_BASER_ENTRY_SIZE(val);
 		int order = get_order(psz);
 		int alloc_size;
+		int alloc_pages;
 		u64 tmp;
 		void *base;
 
@@ -856,6 +857,14 @@ static int its_alloc_tables(const char *node_name, struct its_node *its)
 		}
 
 		alloc_size = (1 << order) * PAGE_SIZE;
+		alloc_pages = (alloc_size / psz);
+		if (alloc_pages > GITS_BASER_PAGES_MAX) {
+			alloc_pages = GITS_BASER_PAGES_MAX;
+			order = get_order(GITS_BASER_PAGES_MAX * psz);
+			pr_warn("%s: Device Table too large, reduce its page order to %u (%u pages)\n",
+				node_name, order, alloc_pages);
+		}
+
 		base = (void *)__get_free_pages(GFP_KERNEL | __GFP_ZERO, order);
 		if (!base) {
 			err = -ENOMEM;
@@ -884,7 +893,7 @@ static int its_alloc_tables(const char *node_name, struct its_node *its)
 			break;
 		}
 
-		val |= (alloc_size / psz) - 1;
+		val |= alloc_pages - 1;
 
 		writeq_relaxed(val, its->base + GITS_BASER + i * 8);
 		tmp = readq_relaxed(its->base + GITS_BASER + i * 8);
diff --git a/include/linux/irqchip/arm-gic-v3.h b/include/linux/irqchip/arm-gic-v3.h
index 9eeeb9589acf..c0c8a2ef9d90 100644
--- a/include/linux/irqchip/arm-gic-v3.h
+++ b/include/linux/irqchip/arm-gic-v3.h
@@ -231,6 +231,7 @@
 #define GITS_BASER_PAGE_SIZE_16K	(1UL << GITS_BASER_PAGE_SIZE_SHIFT)
 #define GITS_BASER_PAGE_SIZE_64K	(2UL << GITS_BASER_PAGE_SIZE_SHIFT)
 #define GITS_BASER_PAGE_SIZE_MASK	(3UL << GITS_BASER_PAGE_SIZE_SHIFT)
+#define GITS_BASER_PAGES_MAX		256
 
 #define GITS_BASER_TYPE_NONE		0
 #define GITS_BASER_TYPE_DEVICE		1
-- 
2.1.1

[PATCH v5 2/6] irqchip, gicv3: Workaround for Cavium ThunderX erratum 23154

[Robert Richter]

From: Robert Richter <rrichter@cavium.com>

This patch implements Cavium ThunderX erratum 23154.

The gicv3 of ThunderX requires a modified version for reading the IAR
status to ensure data synchronization. Since this is in the fast-path
and called with each interrupt, runtime patching is used using jump
label patching for smallest overhead (no-op). This is the same
technique as used for tracepoints.

v5:
 * fixed calling gic_read_iar_cavium_thunderx() if jump label is
   enabled
 * made is_cavium_thunderx static
 * removed ARCH_THUNDER dependency for Cavium errata options to
   make it available for generic kernels

v4:
 * simplify code to only use cpus_have_cap() in gicv3_enable_quirks()

v3:
 * fix erratum to be dependend from midr
 * use arm64 errata framework

v2:
 * implement code in a single asm() to keep instruction sequence
 * added comment to the code that explains the erratum
 * apply workaround also if running as guest, thus check MIDR

Signed-off-by: Robert Richter <rrichter@cavium.com>
---
 arch/arm64/Kconfig                  | 10 +++++++++
 arch/arm64/include/asm/cpufeature.h |  3 ++-
 arch/arm64/include/asm/cputype.h    | 17 ++++++++-------
 arch/arm64/kernel/cpu_errata.c      |  9 ++++++++
 drivers/irqchip/irq-gic-v3.c        | 42 ++++++++++++++++++++++++++++++++++++-
 5 files changed, 72 insertions(+), 9 deletions(-)

diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 7d95663c0160..2e65757ae7be 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -331,6 +331,16 @@ config ARM64_ERRATUM_845719
 
 	  If unsure, say Y.
 
+config CAVIUM_ERRATUM_23154
+	bool "Cavium erratum 23154: Access to ICC_IAR1_EL1 is not sync'ed"
+	default y
+	help
+	  The gicv3 of ThunderX requires a modified version for
+	  reading the IAR status to ensure data synchronization
+	  (access to icc_iar1_el1 is not sync'ed before and after).
+
+	  If unsure, say Y.
+
 endmenu
 
 
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index 171570702bb8..dbc78d2b8cc6 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -27,8 +27,9 @@
 #define ARM64_HAS_SYSREG_GIC_CPUIF		3
 #define ARM64_HAS_PAN				4
 #define ARM64_HAS_LSE_ATOMICS			5
+#define ARM64_WORKAROUND_CAVIUM_23154		6
 
-#define ARM64_NCAPS				6
+#define ARM64_NCAPS				7
 
 #ifndef __ASSEMBLY__
 
diff --git a/arch/arm64/include/asm/cputype.h b/arch/arm64/include/asm/cputype.h
index ee6403df9fe4..100a3d1b17c8 100644
--- a/arch/arm64/include/asm/cputype.h
+++ b/arch/arm64/include/asm/cputype.h
@@ -62,15 +62,18 @@
 	(0xf			<< MIDR_ARCHITECTURE_SHIFT) | \
 	((partnum)		<< MIDR_PARTNUM_SHIFT))
 
-#define ARM_CPU_IMP_ARM		0x41
-#define ARM_CPU_IMP_APM		0x50
+#define ARM_CPU_IMP_ARM			0x41
+#define ARM_CPU_IMP_APM			0x50
+#define ARM_CPU_IMP_CAVIUM		0x43
 
-#define ARM_CPU_PART_AEM_V8	0xD0F
-#define ARM_CPU_PART_FOUNDATION	0xD00
-#define ARM_CPU_PART_CORTEX_A57	0xD07
-#define ARM_CPU_PART_CORTEX_A53	0xD03
+#define ARM_CPU_PART_AEM_V8		0xD0F
+#define ARM_CPU_PART_FOUNDATION		0xD00
+#define ARM_CPU_PART_CORTEX_A57		0xD07
+#define ARM_CPU_PART_CORTEX_A53		0xD03
 
-#define APM_CPU_PART_POTENZA	0x000
+#define APM_CPU_PART_POTENZA		0x000
+
+#define CAVIUM_CPU_PART_THUNDERX	0x0A1
 
 #define ID_AA64MMFR0_BIGENDEL0_SHIFT	16
 #define ID_AA64MMFR0_BIGENDEL0_MASK	(0xf << ID_AA64MMFR0_BIGENDEL0_SHIFT)
diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
index 6ffd91438560..574450c257a4 100644
--- a/arch/arm64/kernel/cpu_errata.c
+++ b/arch/arm64/kernel/cpu_errata.c
@@ -23,6 +23,7 @@
 
 #define MIDR_CORTEX_A53 MIDR_CPU_PART(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A53)
 #define MIDR_CORTEX_A57 MIDR_CPU_PART(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A57)
+#define MIDR_THUNDERX	MIDR_CPU_PART(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX)
 
 #define CPU_MODEL_MASK (MIDR_IMPLEMENTOR_MASK | MIDR_PARTNUM_MASK | \
 			MIDR_ARCHITECTURE_MASK)
@@ -82,6 +83,14 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
 		MIDR_RANGE(MIDR_CORTEX_A53, 0x00, 0x04),
 	},
 #endif
+#ifdef CONFIG_CAVIUM_ERRATUM_23154
+	{
+	/* Cavium ThunderX, pass 1.x */
+		.desc = "Cavium erratum 23154",
+		.capability = ARM64_WORKAROUND_CAVIUM_23154,
+		MIDR_RANGE(MIDR_THUNDERX, 0x00, 0x01),
+	},
+#endif
 	{
 	}
 };
diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c
index 7deed6ef54c2..5899e471df4c 100644
--- a/drivers/irqchip/irq-gic-v3.c
+++ b/drivers/irqchip/irq-gic-v3.c
@@ -114,7 +114,7 @@ static void gic_redist_wait_for_rwp(void)
 }
 
 /* Low level accessors */
-static u64 __maybe_unused gic_read_iar(void)
+static u64 gic_read_iar_common(void)
 {
 	u64 irqstat;
 
@@ -122,6 +122,38 @@ static u64 __maybe_unused gic_read_iar(void)
 	return irqstat;
 }
 
+/*
+ * Cavium ThunderX erratum 23154
+ *
+ * The gicv3 of ThunderX requires a modified version for reading the
+ * IAR status to ensure data synchronization (access to icc_iar1_el1
+ * is not sync'ed before and after).
+ */
+static u64 gic_read_iar_cavium_thunderx(void)
+{
+	u64 irqstat;
+
+	asm volatile(
+		"nop;nop;nop;nop\n\t"
+		"nop;nop;nop;nop\n\t"
+		"mrs_s %0, " __stringify(ICC_IAR1_EL1) "\n\t"
+		"nop;nop;nop;nop"
+		: "=r" (irqstat));
+	mb();
+
+	return irqstat;
+}
+
+static struct static_key is_cavium_thunderx = STATIC_KEY_INIT_FALSE;
+
+static u64 __maybe_unused gic_read_iar(void)
+{
+	if (static_key_false(&is_cavium_thunderx))
+		return gic_read_iar_cavium_thunderx();
+	else
+		return gic_read_iar_common();
+}
+
 static void __maybe_unused gic_write_pmr(u64 val)
 {
 	asm volatile("msr_s " __stringify(ICC_PMR_EL1) ", %0" : : "r" (val));
@@ -839,6 +871,12 @@ static const struct irq_domain_ops gic_irq_domain_ops = {
 	.free = gic_irq_domain_free,
 };
 
+static void gicv3_enable_quirks(void)
+{
+	if (cpus_have_cap(ARM64_WORKAROUND_CAVIUM_23154))
+		static_key_slow_inc(&is_cavium_thunderx);
+}
+
 static int __init gic_of_init(struct device_node *node, struct device_node *parent)
 {
 	void __iomem *dist_base;
@@ -904,6 +942,8 @@ static int __init gic_of_init(struct device_node *node, struct device_node *pare
 	gic_data.nr_redist_regions = nr_redist_regions;
 	gic_data.redist_stride = redist_stride;
 
+	gicv3_enable_quirks();
+
 	/*
 	 * Find out how many interrupts are supported.
 	 * The GIC only supports up to 1020 interrupt sources (SGI+PPI+SPI)
-- 
2.1.1

[PATCH v5 3/6] irqchip, gicv3-its: Read typer register outside the loop

[Robert Richter]

From: Robert Richter <rrichter@cavium.com>

No need to read the typer register in the loop. Values do not change.

This patch is basically a prerequisite for a follow-on patch that adds
errata code for Cavium ThunderX. It moves the calculation of the
number of id entries to the beginning of the function close to other
setup values that are needed to allocate the its table. Now we have a
central location to modify the setup parameters and the errata code
can be implemented in a single block.

Acked-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Robert Richter <rrichter@cavium.com>
---
 drivers/irqchip/irq-gic-v3-its.c | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c
index 0636cb33aeea..b073f28ea00d 100644
--- a/drivers/irqchip/irq-gic-v3-its.c
+++ b/drivers/irqchip/irq-gic-v3-its.c
@@ -815,6 +815,8 @@ static int its_alloc_tables(const char *node_name, struct its_node *its)
 	int psz = SZ_64K;
 	u64 shr = GITS_BASER_InnerShareable;
 	u64 cache = GITS_BASER_WaWb;
+	u64 typer = readq_relaxed(its->base + GITS_TYPER);
+	u32 ids = GITS_TYPER_DEVBITS(typer);
 
 	for (i = 0; i < GITS_BASER_NR_REGS; i++) {
 		u64 val = readq_relaxed(its->base + GITS_BASER + i * 8);
@@ -838,9 +840,6 @@ static int its_alloc_tables(const char *node_name, struct its_node *its)
 		 * For other tables, only allocate a single page.
 		 */
 		if (type == GITS_BASER_TYPE_DEVICE) {
-			u64 typer = readq_relaxed(its->base + GITS_TYPER);
-			u32 ids = GITS_TYPER_DEVBITS(typer);
-
 			/*
 			 * 'order' was initialized earlier to the default page
 			 * granule of the the ITS.  We can't have an allocation
-- 
2.1.1

[PATCH v5 4/6] irqchip, gicv3-its: Add HW revision detection and configuration

[Robert Richter]

From: Robert Richter <rrichter@cavium.com>

Some GIC revisions require an individual configuration to esp. add
workarounds for HW bugs. This patch implements generic code to parse
the hw revision provided by an IIDR register value and runs specific
code if hw matches. A function is added that reads the IIDR registers
for ITS (GITS_IIDR) and then goes through a list of init functions to
be called for specific versions. Same could be done for GICV3
(GICD_IIDR), but there are no users yet for it.

The patch is needed to implement workarounds for HW errata in Cavium's
ThunderX GICV3 ITS.

v5:
 * renamed caps names to quirk

v4:
 * only enable hw detection for its in its_enable_quirks()
 * removed gicv3_check_capabilities()

v3:
 * use arm64 errata framework for midr check

v2:
 * adding MIDR check

Signed-off-by: Robert Richter <rrichter@cavium.com>
---
 drivers/irqchip/irq-gic-common.c | 11 +++++++++++
 drivers/irqchip/irq-gic-common.h |  9 +++++++++
 drivers/irqchip/irq-gic-v3-its.c | 16 ++++++++++++++++
 3 files changed, 36 insertions(+)

diff --git a/drivers/irqchip/irq-gic-common.c b/drivers/irqchip/irq-gic-common.c
index 9448e391cb71..44a077f3a4a2 100644
--- a/drivers/irqchip/irq-gic-common.c
+++ b/drivers/irqchip/irq-gic-common.c
@@ -21,6 +21,17 @@
 
 #include "irq-gic-common.h"
 
+void gic_enable_quirks(u32 iidr, const struct gic_quirk *quirks,
+		void *data)
+{
+	for (; quirks->desc; quirks++) {
+		if (quirks->iidr != (quirks->mask & iidr))
+			continue;
+		quirks->init(data);
+		pr_info("GIC: enabling workaround for %s\n", quirks->desc);
+	}
+}
+
 int gic_configure_irq(unsigned int irq, unsigned int type,
 		       void __iomem *base, void (*sync_access)(void))
 {
diff --git a/drivers/irqchip/irq-gic-common.h b/drivers/irqchip/irq-gic-common.h
index 35a9884778bd..fff697db8e22 100644
--- a/drivers/irqchip/irq-gic-common.h
+++ b/drivers/irqchip/irq-gic-common.h
@@ -20,10 +20,19 @@
 #include <linux/of.h>
 #include <linux/irqdomain.h>
 
+struct gic_quirk {
+	const char *desc;
+	void (*init)(void *data);
+	u32 iidr;
+	u32 mask;
+};
+
 int gic_configure_irq(unsigned int irq, unsigned int type,
                        void __iomem *base, void (*sync_access)(void));
 void gic_dist_config(void __iomem *base, int gic_irqs,
 		     void (*sync_access)(void));
 void gic_cpu_config(void __iomem *base, void (*sync_access)(void));
+void gic_enable_quirks(u32 iidr, const struct gic_quirk *quirks,
+		void *data);
 
 #endif /* _IRQ_GIC_COMMON_H */
diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c
index b073f28ea00d..5c6023e80f6d 100644
--- a/drivers/irqchip/irq-gic-v3-its.c
+++ b/drivers/irqchip/irq-gic-v3-its.c
@@ -37,6 +37,8 @@
 #include <asm/cputype.h>
 #include <asm/exception.h>
 
+#include "irq-gic-common.h"
+
 #define ITS_FLAGS_CMDQ_NEEDS_FLUSHING		(1 << 0)
 
 #define RDIST_FLAGS_PROPBASE_NEEDS_FLUSHING	(1 << 0)
@@ -1371,6 +1373,18 @@ static int its_force_quiescent(void __iomem *base)
 	}
 }
 
+static const struct gic_quirk its_quirks[] = {
+	{
+	}
+};
+
+static void its_enable_quirks(struct its_node *its)
+{
+	u32 iidr = readl_relaxed(its->base + GITS_IIDR);
+
+	gic_enable_quirks(iidr, its_quirks, its);
+}
+
 static int its_probe(struct device_node *node, struct irq_domain *parent)
 {
 	struct resource res;
@@ -1429,6 +1443,8 @@ static int its_probe(struct device_node *node, struct irq_domain *parent)
 	}
 	its->cmd_write = its->cmd_base;
 
+	its_enable_quirks(its);
+
 	err = its_alloc_tables(node->full_name, its);
 	if (err)
 		goto out_free_cmd;
-- 
2.1.1

[PATCH v5 5/6] irqchip, gicv3-its: Workaround for Cavium ThunderX errata 22375, 24313

[Robert Richter]

From: Robert Richter <rrichter@cavium.com>

This implements two gicv3-its errata workarounds for ThunderX. Both
with small impact affecting only ITS table allocation.

 erratum 22375: only alloc 8MB table size
 erratum 24313: ignore memory access type

The fixes are in ITS initialization and basically ignore memory access
type and table size provided by the TYPER and BASER registers.

v5:
 * fix table size wrongly allocating only 4MB
 * introduced CAVIUM_ERRATUM_22375 config option
 * added config description
 * introduced ITS_FLAGS_WORKAROUND_CAVIUM_22375

v3:
 * fix erratum to be dependend from iidr

Signed-off-by: Ganapatrao Kulkarni <gkulkarni@caviumnetworks.com>
Signed-off-by: Robert Richter <rrichter@cavium.com>
---
 arch/arm64/Kconfig               | 17 +++++++++++++++++
 drivers/irqchip/irq-gic-v3-its.c | 37 +++++++++++++++++++++++++++++++++----
 2 files changed, 50 insertions(+), 4 deletions(-)

diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 2e65757ae7be..08a948f15650 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -331,6 +331,23 @@ config ARM64_ERRATUM_845719
 
 	  If unsure, say Y.
 
+config CAVIUM_ERRATUM_22375
+	bool "Cavium erratum 22375, 24313"
+	default y
+	help
+	  Enable workaround for erratum 22375, 24313.
+
+	  This implements two gicv3-its errata workarounds for ThunderX. Both
+	  with small impact affecting only ITS table allocation.
+
+	    erratum 22375: only alloc 8MB table size
+	    erratum 24313: ignore memory access type
+
+	  The fixes are in ITS initialization and basically ignore memory access
+	  type and table size provided by the TYPER and BASER registers.
+
+	  If unsure, say Y.
+
 config CAVIUM_ERRATUM_23154
 	bool "Cavium erratum 23154: Access to ICC_IAR1_EL1 is not sync'ed"
 	default y
diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c
index 5c6023e80f6d..4e9ef1f80c63 100644
--- a/drivers/irqchip/irq-gic-v3-its.c
+++ b/drivers/irqchip/irq-gic-v3-its.c
@@ -39,7 +39,8 @@
 
 #include "irq-gic-common.h"
 
-#define ITS_FLAGS_CMDQ_NEEDS_FLUSHING		(1 << 0)
+#define ITS_FLAGS_CMDQ_NEEDS_FLUSHING		(1ULL << 0)
+#define ITS_FLAGS_WORKAROUND_CAVIUM_22375	(1ULL << 1)
 
 #define RDIST_FLAGS_PROPBASE_NEEDS_FLUSHING	(1 << 0)
 
@@ -816,9 +817,22 @@ static int its_alloc_tables(const char *node_name, struct its_node *its)
 	int i;
 	int psz = SZ_64K;
 	u64 shr = GITS_BASER_InnerShareable;
-	u64 cache = GITS_BASER_WaWb;
-	u64 typer = readq_relaxed(its->base + GITS_TYPER);
-	u32 ids = GITS_TYPER_DEVBITS(typer);
+	u64 cache;
+	u64 typer;
+	u32 ids;
+
+	if (its->flags & ITS_FLAGS_WORKAROUND_CAVIUM_22375) {
+		/*
+		 * erratum 22375: only alloc 8MB table size
+		 * erratum 24313: ignore memory access type
+		 */
+		cache	= 0;
+		ids	= 0x14;			/* 20 bits, 8MB */
+	} else {
+		cache	= GITS_BASER_WaWb;
+		typer	= readq_relaxed(its->base + GITS_TYPER);
+		ids	= GITS_TYPER_DEVBITS(typer);
+	}
 
 	for (i = 0; i < GITS_BASER_NR_REGS; i++) {
 		u64 val = readq_relaxed(its->base + GITS_BASER + i * 8);
@@ -1373,7 +1387,22 @@ static int its_force_quiescent(void __iomem *base)
 	}
 }
 
+static void __maybe_unused its_enable_quirk_cavium_22375(void *data)
+{
+	struct its_node *its = data;
+
+	its->flags |= ITS_FLAGS_WORKAROUND_CAVIUM_22375;
+}
+
 static const struct gic_quirk its_quirks[] = {
+#ifdef CONFIG_CAVIUM_ERRATUM_22375
+	{
+		.desc	= "ITS: Cavium errata 22375, 24313",
+		.iidr	= 0xa100034c,	/* ThunderX pass 1.x */
+		.mask	= 0xffff0fff,
+		.init	= its_enable_quirk_cavium_22375,
+	},
+#endif
 	{
 	}
 };
-- 
2.1.1

[PATCH v5 6/6] irqchip, gicv3-its: Use new jump label API

[Robert Richter]

From: Robert Richter <rrichter@cavium.com>

Use newly introduced jump label API.

Make this a separate patch for easier backporting to older kernels of
the errata patch set.

Signed-off-by: Robert Richter <rrichter@cavium.com>
---
 drivers/irqchip/irq-gic-v3.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c
index 5899e471df4c..8954763ef08a 100644
--- a/drivers/irqchip/irq-gic-v3.c
+++ b/drivers/irqchip/irq-gic-v3.c
@@ -144,11 +144,11 @@ static u64 gic_read_iar_cavium_thunderx(void)
 	return irqstat;
 }
 
-static struct static_key is_cavium_thunderx = STATIC_KEY_INIT_FALSE;
+static DEFINE_STATIC_KEY_FALSE(is_cavium_thunderx);
 
 static u64 __maybe_unused gic_read_iar(void)
 {
-	if (static_key_false(&is_cavium_thunderx))
+	if (static_branch_unlikely(&is_cavium_thunderx))
 		return gic_read_iar_cavium_thunderx();
 	else
 		return gic_read_iar_common();
@@ -874,7 +874,7 @@ static const struct irq_domain_ops gic_irq_domain_ops = {
 static void gicv3_enable_quirks(void)
 {
 	if (cpus_have_cap(ARM64_WORKAROUND_CAVIUM_23154))
-		static_key_slow_inc(&is_cavium_thunderx);
+		static_branch_enable(&is_cavium_thunderx);
 }
 
 static int __init gic_of_init(struct device_node *node, struct device_node *parent)
-- 
2.1.1

[PATCH v5 2/6] irqchip, gicv3: Workaround for Cavium ThunderX erratum 23154

[Marc Zyngier]

On Mon, 21 Sep 2015 22:58:35 +0200
Robert Richter <rric@kernel.org> wrote:

> From: Robert Richter <rrichter@cavium.com>
> 
> This patch implements Cavium ThunderX erratum 23154.
> 
> The gicv3 of ThunderX requires a modified version for reading the IAR
> status to ensure data synchronization. Since this is in the fast-path
> and called with each interrupt, runtime patching is used using jump
> label patching for smallest overhead (no-op). This is the same
> technique as used for tracepoints.
> 
> v5:
>  * fixed calling gic_read_iar_cavium_thunderx() if jump label is
>    enabled
>  * made is_cavium_thunderx static
>  * removed ARCH_THUNDER dependency for Cavium errata options to
>    make it available for generic kernels
> 
> v4:
>  * simplify code to only use cpus_have_cap() in gicv3_enable_quirks()
> 
> v3:
>  * fix erratum to be dependend from midr
>  * use arm64 errata framework
> 
> v2:
>  * implement code in a single asm() to keep instruction sequence
>  * added comment to the code that explains the erratum
>  * apply workaround also if running as guest, thus check MIDR
> 
> Signed-off-by: Robert Richter <rrichter@cavium.com>

Reviewed-by: Marc Zyngier <marc.zyngier@arm.com>

	M.
-- 
Jazz is not dead. It just smells funny.

[PATCH v5 4/6] irqchip, gicv3-its: Add HW revision detection and configuration

[Marc Zyngier]

On Mon, 21 Sep 2015 22:58:37 +0200
Robert Richter <rric@kernel.org> wrote:

> From: Robert Richter <rrichter@cavium.com>
> 
> Some GIC revisions require an individual configuration to esp. add
> workarounds for HW bugs. This patch implements generic code to parse
> the hw revision provided by an IIDR register value and runs specific
> code if hw matches. A function is added that reads the IIDR registers
> for ITS (GITS_IIDR) and then goes through a list of init functions to
> be called for specific versions. Same could be done for GICV3
> (GICD_IIDR), but there are no users yet for it.
> 
> The patch is needed to implement workarounds for HW errata in Cavium's
> ThunderX GICV3 ITS.
> 
> v5:
>  * renamed caps names to quirk
> 
> v4:
>  * only enable hw detection for its in its_enable_quirks()
>  * removed gicv3_check_capabilities()
> 
> v3:
>  * use arm64 errata framework for midr check
> 
> v2:
>  * adding MIDR check
> 
> Signed-off-by: Robert Richter <rrichter@cavium.com>

Reviewed-by: Marc Zyngier <marc.zyngier@arm.com>

	M.
-- 
Jazz is not dead. It just smells funny.

[PATCH v5 5/6] irqchip, gicv3-its: Workaround for Cavium ThunderX errata 22375, 24313

[Marc Zyngier]

On Mon, 21 Sep 2015 22:58:38 +0200
Robert Richter <rric@kernel.org> wrote:

> From: Robert Richter <rrichter@cavium.com>
> 
> This implements two gicv3-its errata workarounds for ThunderX. Both
> with small impact affecting only ITS table allocation.
> 
>  erratum 22375: only alloc 8MB table size
>  erratum 24313: ignore memory access type
> 
> The fixes are in ITS initialization and basically ignore memory access
> type and table size provided by the TYPER and BASER registers.
> 
> v5:
>  * fix table size wrongly allocating only 4MB
>  * introduced CAVIUM_ERRATUM_22375 config option
>  * added config description
>  * introduced ITS_FLAGS_WORKAROUND_CAVIUM_22375
> 
> v3:
>  * fix erratum to be dependend from iidr
> 
> Signed-off-by: Ganapatrao Kulkarni <gkulkarni@caviumnetworks.com>
> Signed-off-by: Robert Richter <rrichter@cavium.com>

Reviewed-by: Marc Zyngier <marc.zyngier@arm.com>

	M.
-- 
Jazz is not dead. It just smells funny.

[PATCH v5 6/6] irqchip, gicv3-its: Use new jump label API

[Marc Zyngier]

On Mon, 21 Sep 2015 22:58:39 +0200
Robert Richter <rric@kernel.org> wrote:

> From: Robert Richter <rrichter@cavium.com>
> 
> Use newly introduced jump label API.
> 
> Make this a separate patch for easier backporting to older kernels of
> the errata patch set.
> 
> Signed-off-by: Robert Richter <rrichter@cavium.com>

Acked-by: Marc Zyngier <marc.zyngier@arm.com>

	M.
-- 
Jazz is not dead. It just smells funny.

[PATCH v5 0/6] irqchip, gicv3: Updates and Cavium ThunderX errata workarounds

[Marc Zyngier]

On Mon, 21 Sep 2015 22:58:33 +0200
Robert Richter <rric@kernel.org> wrote:

> From: Robert Richter <rrichter@cavium.com>
> 
> This patch series adds gicv3 updates and workarounds for HW errata in
> Cavium's ThunderX GICV3.
> 
> The patches has been rebased onto 4.3-rc1. Note that there are two
> important fixes. See below for all changes.
> 
> The first one is an unchanged resubmission of a patch from a gicv3
> series I sent a while ago.
> 
> The next patches implement the workarounds for ThunderX's gicv3. Patch
> #2 implements the cpu workaround for gicv3 on ThunderX. Patch #3 is a
> prerequisit for patch #5. Patch #4 adds generic code to parse the hw
> revision provided by an IIDR. This patch is used for the implementa-
> tion of the actual gicv3-its workaround in #5. Patch #6 updates to the
> new jump label API.
> 
> All current review comments addressed so far with v5

Catalin, Will: assuming you don't have any objection to this series,
how do you want to deal with patch 2?

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny.

[PATCH v5 0/6] irqchip, gicv3: Updates and Cavium ThunderX errata workarounds

[Marc Zyngier]

On Tue, 22 Sep 2015 17:57:01 +0100
Marc Zyngier <marc.zyngier@arm.com> wrote:

[Duh. Now with Will and Catalin on CC]

> On Mon, 21 Sep 2015 22:58:33 +0200
> Robert Richter <rric@kernel.org> wrote:
> 
> > From: Robert Richter <rrichter@cavium.com>
> > 
> > This patch series adds gicv3 updates and workarounds for HW errata in
> > Cavium's ThunderX GICV3.
> > 
> > The patches has been rebased onto 4.3-rc1. Note that there are two
> > important fixes. See below for all changes.
> > 
> > The first one is an unchanged resubmission of a patch from a gicv3
> > series I sent a while ago.
> > 
> > The next patches implement the workarounds for ThunderX's gicv3. Patch
> > #2 implements the cpu workaround for gicv3 on ThunderX. Patch #3 is a
> > prerequisit for patch #5. Patch #4 adds generic code to parse the hw
> > revision provided by an IIDR. This patch is used for the implementa-
> > tion of the actual gicv3-its workaround in #5. Patch #6 updates to the
> > new jump label API.
> > 
> > All current review comments addressed so far with v5
> 
> Catalin, Will: assuming you don't have any objection to this series,
> how do you want to deal with patch 2?
> 
> Thanks,
> 
> 	M.



-- 
Jazz is not dead. It just smells funny.

[PATCH v5 0/6] irqchip, gicv3: Updates and Cavium ThunderX errata workarounds

[Will Deacon]

On Tue, Sep 22, 2015 at 07:09:32PM +0100, Marc Zyngier wrote:
> On Tue, 22 Sep 2015 17:57:01 +0100
> Marc Zyngier <marc.zyngier@arm.com> wrote:
> 
> [Duh. Now with Will and Catalin on CC]
> 
> > On Mon, 21 Sep 2015 22:58:33 +0200
> > Robert Richter <rric@kernel.org> wrote:
> > 
> > > From: Robert Richter <rrichter@cavium.com>
> > > 
> > > This patch series adds gicv3 updates and workarounds for HW errata in
> > > Cavium's ThunderX GICV3.
> > > 
> > > The patches has been rebased onto 4.3-rc1. Note that there are two
> > > important fixes. See below for all changes.
> > > 
> > > The first one is an unchanged resubmission of a patch from a gicv3
> > > series I sent a while ago.
> > > 
> > > The next patches implement the workarounds for ThunderX's gicv3. Patch
> > > #2 implements the cpu workaround for gicv3 on ThunderX. Patch #3 is a
> > > prerequisit for patch #5. Patch #4 adds generic code to parse the hw
> > > revision provided by an IIDR. This patch is used for the implementa-
> > > tion of the actual gicv3-its workaround in #5. Patch #6 updates to the
> > > new jump label API.
> > > 
> > > All current review comments addressed so far with v5
> > 
> > Catalin, Will: assuming you don't have any objection to this series,
> > how do you want to deal with patch 2?

What are the actual dependencies here? AFAICT, the series is addressing
multiple errata, so would it be possible to make the arm64 bits somewhat
independent from the gic parts?

Also, I assume this is targetting 4.4?

Will

[PATCH v5 0/6] irqchip, gicv3: Updates and Cavium ThunderX errata workarounds

[Marc Zyngier]

On Tue, 22 Sep 2015 19:27:26 +0100
Will Deacon <will.deacon@arm.com> wrote:

> On Tue, Sep 22, 2015 at 07:09:32PM +0100, Marc Zyngier wrote:
> > On Tue, 22 Sep 2015 17:57:01 +0100
> > Marc Zyngier <marc.zyngier@arm.com> wrote:
> > 
> > [Duh. Now with Will and Catalin on CC]
> > 
> > > On Mon, 21 Sep 2015 22:58:33 +0200
> > > Robert Richter <rric@kernel.org> wrote:
> > > 
> > > > From: Robert Richter <rrichter@cavium.com>
> > > > 
> > > > This patch series adds gicv3 updates and workarounds for HW errata in
> > > > Cavium's ThunderX GICV3.
> > > > 
> > > > The patches has been rebased onto 4.3-rc1. Note that there are two
> > > > important fixes. See below for all changes.
> > > > 
> > > > The first one is an unchanged resubmission of a patch from a gicv3
> > > > series I sent a while ago.
> > > > 
> > > > The next patches implement the workarounds for ThunderX's gicv3. Patch
> > > > #2 implements the cpu workaround for gicv3 on ThunderX. Patch #3 is a
> > > > prerequisit for patch #5. Patch #4 adds generic code to parse the hw
> > > > revision provided by an IIDR. This patch is used for the implementa-
> > > > tion of the actual gicv3-its workaround in #5. Patch #6 updates to the
> > > > new jump label API.
> > > > 
> > > > All current review comments addressed so far with v5
> > > 
> > > Catalin, Will: assuming you don't have any objection to this series,
> > > how do you want to deal with patch 2?
> 
> What are the actual dependencies here? AFAICT, the series is addressing
> multiple errata, so would it be possible to make the arm64 bits somewhat
> independent from the gic parts?

Patch 2 could be split into an arm64-specific part and a gic part, with
a bit of #ifdef-ery in gicv3_enable_quirks().

> Also, I assume this is targetting 4.4?

Up to you, really. It is not a regression, but it would still be nice
to have 4.3 working reliably on this HW.

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny.

[PATCH v5 0/6] irqchip, gicv3: Updates and Cavium ThunderX errata workarounds

[Catalin Marinas]

On Tue, Sep 22, 2015 at 08:41:34PM +0100, Marc Zyngier wrote:
> On Tue, 22 Sep 2015 19:27:26 +0100
> Will Deacon <will.deacon@arm.com> wrote:
> > On Tue, Sep 22, 2015 at 07:09:32PM +0100, Marc Zyngier wrote:
> > > > Catalin, Will: assuming you don't have any objection to this series,
> > > > how do you want to deal with patch 2?
> > 
> > What are the actual dependencies here? AFAICT, the series is addressing
> > multiple errata, so would it be possible to make the arm64 bits somewhat
> > independent from the gic parts?
> 
> Patch 2 could be split into an arm64-specific part and a gic part, with
> a bit of #ifdef-ery in gicv3_enable_quirks().

The arm64 part without CONFIG_CAVIUM_ERRATUM_* wouldn't cause any
problem. Anyway, I'm not too bothered about separate patches, I think
the whole series could go in via a single tree (irqchip).

> > Also, I assume this is targetting 4.4?
> 
> Up to you, really. It is not a regression, but it would still be nice
> to have 4.3 working reliably on this HW.

I don't have any objection to this patchset but it looks like quite a
lot of code for 4.3 and it is not a regression. Anyway, for the
arch/arm64 bits:

Acked-by: Catalin Marinas <catalin.marinas@arm.com>

-- 
Catalin

[PATCH v5 0/6] irqchip, gicv3: Updates and Cavium ThunderX errata workarounds

[Robert Richter]

Marc, Will,

On 22.09.15 20:41:34, Marc Zyngier wrote:
> On Tue, 22 Sep 2015 19:27:26 +0100
> Will Deacon <will.deacon@arm.com> wrote:
> 
> > On Tue, Sep 22, 2015 at 07:09:32PM +0100, Marc Zyngier wrote:
> > > On Tue, 22 Sep 2015 17:57:01 +0100
> > > Marc Zyngier <marc.zyngier@arm.com> wrote:
> > > 
> > > [Duh. Now with Will and Catalin on CC]
> > > 
> > > > On Mon, 21 Sep 2015 22:58:33 +0200
> > > > Robert Richter <rric@kernel.org> wrote:
> > > > 
> > > > > From: Robert Richter <rrichter@cavium.com>
> > > > > 
> > > > > This patch series adds gicv3 updates and workarounds for HW errata in
> > > > > Cavium's ThunderX GICV3.
> > > > > 
> > > > > The patches has been rebased onto 4.3-rc1. Note that there are two
> > > > > important fixes. See below for all changes.
> > > > > 
> > > > > The first one is an unchanged resubmission of a patch from a gicv3
> > > > > series I sent a while ago.
> > > > > 
> > > > > The next patches implement the workarounds for ThunderX's gicv3. Patch
> > > > > #2 implements the cpu workaround for gicv3 on ThunderX. Patch #3 is a
> > > > > prerequisit for patch #5. Patch #4 adds generic code to parse the hw
> > > > > revision provided by an IIDR. This patch is used for the implementa-
> > > > > tion of the actual gicv3-its workaround in #5. Patch #6 updates to the
> > > > > new jump label API.
> > > > > 
> > > > > All current review comments addressed so far with v5
> > > > 
> > > > Catalin, Will: assuming you don't have any objection to this series,
> > > > how do you want to deal with patch 2?

thanks for your review.

> > What are the actual dependencies here? AFAICT, the series is addressing
> > multiple errata, so would it be possible to make the arm64 bits somewhat
> > independent from the gic parts?
> 
> Patch 2 could be split into an arm64-specific part and a gic part, with
> a bit of #ifdef-ery in gicv3_enable_quirks().

Patch 2 addresses a gicv3 erratum (#23154). Thus needs to be enable in
host and guest, thus MIDR is used together with the arm64 errata
framework. There are no other arm64 errata than this one, so it is
closely related to gicv3.

Or, do you mean separating arm64 code by putting the relevant parts
into #ifdefs for CONFIG_ARM64? Isn't ARM_GIC_V3 only enabled for ARM64?

Do you have any changes of this patch in mind?

Also note that patch #6 is for #2. So if you pass the patches through
different trees, put those both patches together.

> > Also, I assume this is targetting 4.4?
> 
> Up to you, really. It is not a regression, but it would still be nice
> to have 4.3 working reliably on this HW.

Thanks,

-Robert