[RFC PATCH 0/6] riscv: support EIC770X/JH7110 noncoherent devices with XPbmtUC

[RFC PATCH 0/6] riscv: support EIC770X/JH7110 noncoherent devices with XPbmtUC

[Bo Gan]

Starfive JH7110 and ESWIN EIC770X both have non cache-coherent
peripherals. On JH7110[1], GPU/VOUT/VPU/ISP are routed to the sys port,
making them not cache-coherent. On EIC770X, all peripherals are routed
to the sys port, and none is cache-coherent. To make drivers work on
such platforms, the standard solution is to use Svpbmt and map the DMA
buffer as uncacheable. However, neither SoC supports Svpbmt. Instead,
they map the system memory twice, as cached and uncached. The uncached
alias implicitly applies the uncacheable PMA. To support such platform,
a special form of Svpbmt, namely "XPbmtUC" is introduced in this patch.
It's a synthetical PTE format where a single bit (UC) is controlling
the cacheability and the bit position can be configured at runtime. It
is intended to model the physical memory aliasing with minimal effort.

On JH7110, it aligns perfectly with the HW, as the aliased UC region
happens to be offsetted by 2^34. Thus, configuring the XPbmtUC with
bit=32 (PPN is shifted by 2) is all that needs to be done.

On EIC770X, the aliased UC region is put to a awkward offset, and given
there can be 2 NUMA node (dual-die) with 2 separate memory regions and
their UC alias counterpart, we instead ask the firmware to provide a
thin-layer hypervisor to re-arrange the memory map. The XPbmtUC will be
enabled with bit=38, thus map all UC pages to 2^40 (the upper-half of
2^41), and the underlaying hypervisor will re-map the 2^40+ addresses
to the appropriate UC alias regions. (See description in PATCH 1/6)

We chose bit 38 (PPN bit 40) to make the 2-stage translation efficient.
Hypervisor can utilize Sv39x4 G-stage scheme, and map all pages as 1GB
huge page, consuming only the first-level page table (16KB total), and
several TLB entries. In practice, it's the firmware/bootloader that
configures XPbmtUC through device-tree, based on firmware capabilities,
and skip the enablement on stock firmware. This is tested on Hifive
Premier P550 with the modified OpenSBI[2]. It runs the host Linux in VS
mode, and provide the aforementioned remapping. The performance penalty
(if not running KVM in Linux) is minimal, as the CPU is never switched
to HS mode. A very slight, unavoidable, slow down is with the external
interrupt delivery. Due to the lack of AIA in EIC770X, all device irq
now needs to trap to M mode first, before forwarding to VS mode. The
overhead of running KVM in such setup is yet unknown, and may well be
noticeable, as all HS-qualified instructions will trap to M mode, and
there's also the extra cost of flushing G/VS-stage TLBs. I'm analyzing
it in parallel.

I'm aware there's an ongoing series that Samuel sent for physical
memory aliases. I haven't been following too closely, but if you're
worried about it touching to many areas, I hope my series can shed some
light on the problem. My change is very minimal and local, also fairly
easy to remove if we later decide deprecating it down the road.

[1] https://github.com/starfive-tech/JH7100_Docs/blob/main/JH7100%20Cache%20Coherence%20V1.0.pdf
[2] https://github.com/ganboing/opensbi/tree/eic77x-vspt-physalias-wip

Bo Gan (6):
  riscv: Add a custom, simplified version of Svpbmt "XPbmtUC"
  riscv: alternatives: support auipc+load pair
  riscv: apply page table attribute bits for XPbmtUC
  riscv: select RISCV_ISA_XPBMTUC in STARFIVE and ESWIN SoC
  riscv: dts: starfive: jh7110: activate XPbmtUC
  [TESTING-ONLY] riscv: dts: eswin: eic7700: activate XPbmtUC

 arch/riscv/Kconfig                       | 12 ++++++++++++
 arch/riscv/Kconfig.socs                  |  2 ++
 arch/riscv/boot/dts/eswin/eic7700.dtsi   |  1 +
 arch/riscv/boot/dts/starfive/jh7110.dtsi |  1 +
 arch/riscv/include/asm/errata_list.h     | 17 +++++++++++++++--
 arch/riscv/include/asm/hwcap.h           |  1 +
 arch/riscv/include/asm/insn.h            |  8 ++++++++
 arch/riscv/include/asm/pgtable-64.h      | 17 ++++++++++++++++-
 arch/riscv/kernel/alternative.c          | 11 ++++++-----
 arch/riscv/kernel/cpufeature.c           |  8 ++++++++
 arch/riscv/mm/pgtable.c                  |  7 +++++++
 11 files changed, 77 insertions(+), 8 deletions(-)

-- 
2.34.1

[RFC PATCH 1/6] riscv: Add a custom, simplified version of Svpbmt "XPbmtUC"

[Bo Gan]

On platforms that doesn't support Svpbmt or XTheadMae, SoC vendors
sometimes map the system memory twice in physical address space, one
as cached, and the other as uncached. Through the uncached window,
device drivers will be able to map DMA buffer for noncoherent devices.
Such setup is usually found in SoC with pre-Svpbmt Sifive cores.
Make use of such feature by modeling it as "XPbmtUC", a customized
version of Svpbmt, where a single bit in PTE is used for UC control.
There's no IO bit with such scheme, as it's assumed that the PMA
(usually hard-wired on these SoCs) will properly convey the strongly-
ordered, non-idempotent attribute of the MMIO region.

The enablement of such position of "XPbmtUC" is controlled by the
device-tree property "riscv,xpbmt-uncache-bit".

Example:

Starfive JH7110 (Sifive U74):
           [0x0,   0x40000000) Low MMIO
    [0x40000000, 0x2_40000000) Cached Mem
  [0x4_40000000, 0x6_40000000) Uncached Mem UC+
  [0x9_00000000, 0x9_d0000000) High MMIO

Device-tree:
  riscv,xpbmt-uncache-bit = <32>;

Use PTE bit 32 (PPN bit 34) as UC (uncache) control to perfectly
match the memory map of the SoC.

ESWIN EIC770X (Sifive U84/P550):
           [0x0,    0x20000000) Core Internal
    [0x20000000,    0x40000000) Core Internal (Die 1)
    [0x40000000,    0x60000000) Low MMIO
    [0x60000000,    0x80000000) Low MMIO (Die 1)
    [0x80000000, 0x10_80000000) Cached Mem
 [0x20_00000000, 0x30_00000000) Cached Mem (Die 1)
 [0x80_00000000, 0xa0_00000000) High MMIO
 [0xa0_00000000, 0xc0_00000000) High MMIO (Die 1)
 [0xc0_00000000, 0xd0_00000000) Uncached Mem
 [0xe0_00000000, 0xf0_00000000) Uncached Mem (Die 1)

EIC770X is not directly compatible to this model, as the uncached
regions are offsetted, and the offset is different among the Dies
in the dual-die version (EIC7702). so we expect the firmware to
provide a thin layer of hypervisor to transparently re-map:

    [0x80000000,  0x10_80000000) Cached Mem
 [0x20_00000000,  0x30_00000000) Cached Mem (Die 1)
 [0xc0_00000000,  0xd0_00000000) Uncached Mem <----------.
 [0xe0_00000000,  0xf0_00000000) Uncached Mem (Die 1) <--+--.
[0x100_80000000, 0x110_80000000) Mem UC+ ----------------'  |
[0x120_00000000, 0x130_00000000) Mem UC+ (Die 1) -----------'

With that, the firmware/bootloader can set the following at boot:
  riscv,xpbmt-uncache-bit = <38>;

Signed-off-by: Bo Gan <ganboing@gmail.com>
---
 arch/riscv/Kconfig                  | 12 ++++++++++++
 arch/riscv/include/asm/hwcap.h      |  1 +
 arch/riscv/include/asm/pgtable-64.h |  8 ++++++++
 arch/riscv/kernel/cpufeature.c      |  8 ++++++++
 arch/riscv/mm/pgtable.c             |  7 +++++++
 5 files changed, 36 insertions(+)

diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index 6b39f37f769a2..f2b4da6a3deb1 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -893,6 +893,18 @@ config TOOLCHAIN_NEEDS_OLD_ISA_SPEC
 	  versions of clang and GCC to be passed to GAS, which has the same result
 	  as passing zicsr and zifencei to -march.
 
+config RISCV_ISA_XPBMTUC
+	bool "Support XPbmtUC (customized pbmt uncache bit)"
+	depends on 64BIT && MMU
+	depends on RISCV_ALTERNATIVE
+	default n
+	select DMA_DIRECT_REMAP
+	help
+	  Add support for "riscv,xpbmt-uncache-bit" device-tree property.
+	  The bit denotes the bit in PTE that marks the page as uncached.
+
+	  If you don't know what to do here, say N.
+
 config FPU
 	bool "FPU support"
 	default y
diff --git a/arch/riscv/include/asm/hwcap.h b/arch/riscv/include/asm/hwcap.h
index 4369a23385413..6baa6566cf4cc 100644
--- a/arch/riscv/include/asm/hwcap.h
+++ b/arch/riscv/include/asm/hwcap.h
@@ -111,6 +111,7 @@
 #define RISCV_ISA_EXT_ZILSD		102
 #define RISCV_ISA_EXT_ZCLSD		103
 
+#define RISCV_ISA_EXT_XPBMTUC		126
 #define RISCV_ISA_EXT_XLINUXENVCFG	127
 
 #define RISCV_ISA_EXT_MAX		128
diff --git a/arch/riscv/include/asm/pgtable-64.h b/arch/riscv/include/asm/pgtable-64.h
index 6e789fa58514c..1a6d04884111d 100644
--- a/arch/riscv/include/asm/pgtable-64.h
+++ b/arch/riscv/include/asm/pgtable-64.h
@@ -140,6 +140,14 @@ enum napot_cont_order {
 #define _PAGE_IO_THEAD		((1UL << 63) | (1UL << 60))
 #define _PAGE_MTMASK_THEAD	(_PAGE_PMA_THEAD | _PAGE_IO_THEAD | (1UL << 59))
 
+#ifdef CONFIG_RISCV_ISA_XPBMTUC
+extern int riscv_xpbmtuc_bit;
+extern u64 riscv_xpbmtuc_mask;
+#endif
+
+#define XPBMTUC_HAS_PAGE_NOCACHE CONFIG_RISCV_ISA_XPBMTUC
+#define XPBMTUC_HAS_PAGE_MTMASK  CONFIG_RISCV_ISA_XPBMTUC
+
 static inline u64 riscv_page_mtmask(void)
 {
 	u64 val;
diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c
index fa591aff9d335..faec169004b4a 100644
--- a/arch/riscv/kernel/cpufeature.c
+++ b/arch/riscv/kernel/cpufeature.c
@@ -1118,6 +1118,14 @@ void __init riscv_fill_hwcap(void)
 		riscv_v_setup_vsize();
 	}
 
+#ifdef CONFIG_RISCV_ISA_XPBMTUC
+	if (!of_property_read_u32(of_root, "riscv,xpbmt-uncache-bit",
+				  &riscv_xpbmtuc_bit)) {
+		riscv_xpbmtuc_mask = 1UL << riscv_xpbmtuc_bit;
+		set_bit(RISCV_ISA_EXT_XPBMTUC, riscv_isa);
+		pr_info("Using XPbmtUC bit=%d\n", riscv_xpbmtuc_bit);
+	}
+#endif
 	memset(print_str, 0, sizeof(print_str));
 	for (i = 0, j = 0; i < NUM_ALPHA_EXTS; i++)
 		if (riscv_isa[0] & BIT_MASK(i))
diff --git a/arch/riscv/mm/pgtable.c b/arch/riscv/mm/pgtable.c
index 807c0a0de1827..4ca442bc8595d 100644
--- a/arch/riscv/mm/pgtable.c
+++ b/arch/riscv/mm/pgtable.c
@@ -5,6 +5,13 @@
 #include <linux/kernel.h>
 #include <linux/pgtable.h>
 
+#ifdef CONFIG_RISCV_ISA_XPBMTUC
+int riscv_xpbmtuc_bit;
+
+u64 riscv_xpbmtuc_mask;
+EXPORT_SYMBOL(riscv_xpbmtuc_mask);
+#endif
+
 int ptep_set_access_flags(struct vm_area_struct *vma,
 			  unsigned long address, pte_t *ptep,
 			  pte_t entry, int dirty)
-- 
2.34.1

[RFC PATCH 2/6] riscv: alternatives: support auipc+load pair

[Bo Gan]

Previously only auipc+jalr pair is supported. Add auipc+load pair
to support PC-relative memory load instruction as well.

Signed-off-by: Bo Gan <ganboing@gmail.com>
---
 arch/riscv/include/asm/insn.h   |  8 ++++++++
 arch/riscv/kernel/alternative.c | 11 ++++++-----
 2 files changed, 14 insertions(+), 5 deletions(-)

diff --git a/arch/riscv/include/asm/insn.h b/arch/riscv/include/asm/insn.h
index c3005573e8c99..1c791a8732efc 100644
--- a/arch/riscv/include/asm/insn.h
+++ b/arch/riscv/include/asm/insn.h
@@ -135,6 +135,8 @@
 #define RVC_C2_RS1_MASK		GENMASK(4, 0)
 
 /* parts of opcode for RVG*/
+#define RVG_OPCODE_LOAD		0x03
+#define RVG_OPCODE_STORE	0x23
 #define RVG_OPCODE_FENCE	0x0f
 #define RVG_OPCODE_AUIPC	0x17
 #define RVG_OPCODE_BRANCH	0x63
@@ -198,6 +200,8 @@
 #define RVG_MATCH_BGE		(RV_ENCODE_FUNCT3(BGE) | RVG_OPCODE_BRANCH)
 #define RVG_MATCH_BLTU		(RV_ENCODE_FUNCT3(BLTU) | RVG_OPCODE_BRANCH)
 #define RVG_MATCH_BGEU		(RV_ENCODE_FUNCT3(BGEU) | RVG_OPCODE_BRANCH)
+#define RVG_MATCH_LOAD		(RVG_OPCODE_LOAD)
+#define RVG_MATCH_STORE		(RVG_OPCODE_STORE)
 #define RVG_MATCH_EBREAK	(RV_ENCODE_FUNCT12(EBREAK) | RVG_OPCODE_SYSTEM)
 #define RVG_MATCH_SRET		(RV_ENCODE_FUNCT12(SRET) | RVG_OPCODE_SYSTEM)
 #define RVC_MATCH_C_BEQZ	(RVC_ENCODE_FUNCT3(C_BEQZ) | RVC_OPCODE_C1)
@@ -222,6 +226,8 @@
 #define RVG_MASK_BGE		(RV_INSN_FUNCT3_MASK | RV_INSN_OPCODE_MASK)
 #define RVG_MASK_BLTU		(RV_INSN_FUNCT3_MASK | RV_INSN_OPCODE_MASK)
 #define RVG_MASK_BGEU		(RV_INSN_FUNCT3_MASK | RV_INSN_OPCODE_MASK)
+#define RVG_MASK_LOAD		(RV_INSN_OPCODE_MASK)
+#define RVG_MASK_STORE		(RV_INSN_OPCODE_MASK)
 #define RVC_MASK_C_BEQZ		(RVC_INSN_FUNCT3_MASK | RVC_INSN_OPCODE_MASK)
 #define RVC_MASK_C_BNEZ		(RVC_INSN_FUNCT3_MASK | RVC_INSN_OPCODE_MASK)
 #define RVC_MASK_C_EBREAK	0xffff
@@ -262,6 +268,8 @@ __RISCV_INSN_FUNCS(c_ebreak, RVC_MASK_C_EBREAK, RVC_MATCH_C_EBREAK)
 __RISCV_INSN_FUNCS(ebreak, RVG_MASK_EBREAK, RVG_MATCH_EBREAK)
 __RISCV_INSN_FUNCS(sret, RVG_MASK_SRET, RVG_MATCH_SRET)
 __RISCV_INSN_FUNCS(fence, RVG_MASK_FENCE, RVG_MATCH_FENCE);
+__RISCV_INSN_FUNCS(load, RVG_MASK_LOAD, RVG_MATCH_LOAD);
+__RISCV_INSN_FUNCS(store, RVG_MASK_STORE, RVG_MATCH_STORE);
 
 /* special case to catch _any_ system instruction */
 static __always_inline bool riscv_insn_is_system(u32 code)
diff --git a/arch/riscv/kernel/alternative.c b/arch/riscv/kernel/alternative.c
index 7642704c7f184..04a9d3aed4647 100644
--- a/arch/riscv/kernel/alternative.c
+++ b/arch/riscv/kernel/alternative.c
@@ -74,7 +74,7 @@ static u32 riscv_instruction_at(void *p)
 	return (u32)parcel[0] | (u32)parcel[1] << 16;
 }
 
-static void riscv_alternative_fix_auipc_jalr(void *ptr, u32 auipc_insn,
+static void riscv_alternative_fix_auipc_pair(void *ptr, u32 auipc_insn,
 					     u32 jalr_insn, int patch_offset)
 {
 	u32 call[2] = { auipc_insn, jalr_insn };
@@ -123,14 +123,15 @@ void riscv_alternative_fix_offsets(void *alt_ptr, unsigned int len,
 		if (riscv_insn_is_auipc(insn) && i < num_insn - 1) {
 			u32 insn2 = riscv_instruction_at(alt_ptr + (i + 1) * sizeof(u32));
 
-			if (!riscv_insn_is_jalr(insn2))
+			if (!riscv_insn_is_jalr(insn2) &&
+			    !riscv_insn_is_load(insn2))
 				continue;
 
-			/* if instruction pair is a call, it will use the ra register */
-			if (RV_EXTRACT_RD_REG(insn) != 1)
+			if (RV_EXTRACT_RD_REG(insn) != RV_EXTRACT_RS1_REG(insn2))
 				continue;
 
-			riscv_alternative_fix_auipc_jalr(alt_ptr + i * sizeof(u32),
+			/* insn2 use rd of insn as rs1, patch it */
+			riscv_alternative_fix_auipc_pair(alt_ptr + i * sizeof(u32),
 							 insn, insn2, patch_offset);
 			i++;
 		}
-- 
2.34.1

[RFC PATCH 3/6] riscv: apply page table attribute bits for XPbmtUC

[Bo Gan]

Apply the UC bit like Svpbmt and THEAD_MAE does. Also changed the
_PAGE_PFN_MASK definition to exclude the UC bit, as it's position
is now determined at runtime, and can be part of PPN.

Signed-off-by: Bo Gan <ganboing@gmail.com>
---
 arch/riscv/include/asm/errata_list.h | 17 +++++++++++++++--
 arch/riscv/include/asm/pgtable-64.h  |  9 ++++++++-
 2 files changed, 23 insertions(+), 3 deletions(-)

diff --git a/arch/riscv/include/asm/errata_list.h b/arch/riscv/include/asm/errata_list.h
index 6694b5ccdcf85..ba0f3d4dd0cbb 100644
--- a/arch/riscv/include/asm/errata_list.h
+++ b/arch/riscv/include/asm/errata_list.h
@@ -53,6 +53,16 @@ asm(ALTERNATIVE(	\
 	: /* no inputs */	\
 	: "memory")
 
+#ifdef CONFIG_64BIT
+#define ALT_PAGE_CUST_BIT(_bit)						\
+asm(ALTERNATIVE("li %0, 0\t\nnop",					\
+		"1: auipc %0, %%pcrel_hi(riscv_xpbmtuc_mask)\t\n"	\
+		      "ld %0, %%pcrel_lo(1b)(%0)", 0,			\
+			RISCV_ISA_EXT_XPBMTUC,				\
+			CONFIG_RISCV_ISA_XPBMTUC)			\
+		: "=r"(_bit))
+#endif
+
 /*
  * _val is marked as "will be overwritten", so need to set it to 0
  * in the default case.
@@ -60,11 +70,14 @@ asm(ALTERNATIVE(	\
 #define ALT_SVPBMT_SHIFT 61
 #define ALT_THEAD_MAE_SHIFT 59
 #define ALT_SVPBMT(_val, prot)						\
-asm(ALTERNATIVE_2("li %0, 0\t\nnop",					\
+asm(ALTERNATIVE_3("li %0, 0\t\nnop",					\
 		  "li %0, %1\t\nslli %0,%0,%3", 0,			\
 			RISCV_ISA_EXT_SVPBMT, CONFIG_RISCV_ISA_SVPBMT,	\
 		  "li %0, %2\t\nslli %0,%0,%4", THEAD_VENDOR_ID,	\
-			ERRATA_THEAD_MAE, CONFIG_ERRATA_THEAD_MAE)	\
+			ERRATA_THEAD_MAE, CONFIG_ERRATA_THEAD_MAE,	\
+		  "1: auipc %0, %%pcrel_hi(riscv_xpbmtuc_mask)\t\n"	\
+			"ld %0, %%pcrel_lo(1b)(%0)", 0,			\
+			RISCV_ISA_EXT_XPBMTUC, XPBMTUC_HAS##prot)	\
 		: "=r"(_val)						\
 		: "I"(prot##_SVPBMT >> ALT_SVPBMT_SHIFT),		\
 		  "I"(prot##_THEAD >> ALT_THEAD_MAE_SHIFT),		\
diff --git a/arch/riscv/include/asm/pgtable-64.h b/arch/riscv/include/asm/pgtable-64.h
index 1a6d04884111d..aab6990d92238 100644
--- a/arch/riscv/include/asm/pgtable-64.h
+++ b/arch/riscv/include/asm/pgtable-64.h
@@ -76,7 +76,14 @@ typedef struct {
  * | 63 | 62 61 | 60 54 | 53  10 | 9             8 | 7 | 6 | 5 | 4 | 3 | 2 | 1 | 0
  *   N      MT     RSV    PFN      reserved for SW   D   A   G   U   X   W   R   V
  */
-#define _PAGE_PFN_MASK  GENMASK(53, 10)
+static inline u64 riscv_pfn_mask(void)
+{
+	u64 cust_bit;
+
+	ALT_PAGE_CUST_BIT(cust_bit);
+	return GENMASK(53, 10) ^ cust_bit;
+}
+#define _PAGE_PFN_MASK  riscv_pfn_mask()
 
 /*
  * [63] Svnapot definitions:
-- 
2.34.1

[RFC PATCH 4/6] riscv: select RISCV_ISA_XPBMTUC in STARFIVE and ESWIN SoC

[Bo Gan]

Enable the XPbmtUC feature for Starfive and ESWIN SoC

Signed-off-by: Bo Gan <ganboing@gmail.com>
---
 arch/riscv/Kconfig.socs | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/arch/riscv/Kconfig.socs b/arch/riscv/Kconfig.socs
index d621b85dd63bd..0584511707c7c 100644
--- a/arch/riscv/Kconfig.socs
+++ b/arch/riscv/Kconfig.socs
@@ -14,6 +14,7 @@ config ARCH_ANLOGIC
 
 config ARCH_ESWIN
 	bool "ESWIN SoCs"
+	select RISCV_ISA_XPBMTUC
 	help
 	  This enables support for ESWIN SoC platform hardware,
 	  including the ESWIN EIC7700 SoC.
@@ -56,6 +57,7 @@ config SOC_STARFIVE
 	select PINCTRL
 	select RESET_CONTROLLER
 	select ARM_AMBA
+	select RISCV_ISA_XPBMTUC
 	help
 	  This enables support for StarFive SoC platform hardware.
 
-- 
2.34.1

[RFC PATCH 5/6] riscv: dts: starfive: jh7110: activate XPbmtUC

[Bo Gan]

Set riscv,xpbmt-uncache-bit to 32 to match SoC memory map:

           [0x0,   0x40000000) Low MMIO
    [0x40000000, 0x2_40000000) Cached Mem
  [0x4_40000000, 0x6_40000000) Uncached Mem UC+
  [0x9_00000000, 0x9_d0000000) High MMIO

Signed-off-by: Bo Gan <ganboing@gmail.com>
---
 arch/riscv/boot/dts/starfive/jh7110.dtsi | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/riscv/boot/dts/starfive/jh7110.dtsi b/arch/riscv/boot/dts/starfive/jh7110.dtsi
index 6e56e9d20bb06..6dfeb31538fba 100644
--- a/arch/riscv/boot/dts/starfive/jh7110.dtsi
+++ b/arch/riscv/boot/dts/starfive/jh7110.dtsi
@@ -14,6 +14,7 @@ / {
 	compatible = "starfive,jh7110";
 	#address-cells = <2>;
 	#size-cells = <2>;
+	riscv,xpbmt-uncache-bit = <32>;
 
 	cpus: cpus {
 		#address-cells = <1>;
-- 
2.34.1

[RFC PATCH 6/6] [TESTING-ONLY] riscv: dts: eswin: eic7700: activate XPbmtUC

[Bo Gan]

Set riscv,xpbmt-uncache-bit to 38 for testing only.
(Make sure your firmware remaps it as the following)

            [0x0,    0x20000000) Core Internal
     [0x20000000,    0x40000000) Core Internal (Die 1)
     [0x40000000,    0x60000000) Low MMIO
     [0x60000000,    0x80000000) Low MMIO (Die 1)
     [0x80000000, 0x10_80000000) Cached Mem
  [0x20_00000000, 0x30_00000000) Cached Mem (Die 1)
  [0x80_00000000, 0xa0_00000000) High MMIO
  [0xa0_00000000, 0xc0_00000000) High MMIO (Die 1)
  [0xc0_00000000, 0xd0_00000000) Uncached Mem  <----------.
  [0xe0_00000000, 0xf0_00000000) Uncached Mem (Die 1)  <--+--.
with firmware/hypervisor re-mapping:                      |  |
------------------------------------                      |  |
 [0x100_80000000, 0x110_80000000) Mem UC+ ----------------'  |
 [0x120_00000000, 0x130_00000000) Mem UC+ (Die 1) -----------'

Signed-off-by: Bo Gan <ganboing@gmail.com>
---
 arch/riscv/boot/dts/eswin/eic7700.dtsi | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/riscv/boot/dts/eswin/eic7700.dtsi b/arch/riscv/boot/dts/eswin/eic7700.dtsi
index f16ec76fb130c..5c413439daf0a 100644
--- a/arch/riscv/boot/dts/eswin/eic7700.dtsi
+++ b/arch/riscv/boot/dts/eswin/eic7700.dtsi
@@ -10,6 +10,7 @@
 / {
 	#address-cells = <2>;
 	#size-cells = <2>;
+	riscv,xpbmt-uncache-bit = <38>;
 
 	cpus {
 		#address-cells = <1>;
-- 
2.34.1

Re: [RFC PATCH 0/6] riscv: support EIC770X/JH7110 noncoherent devices with XPbmtUC

[Conor Dooley]

[-- Attachment #1: Type: text/plain, Size: 3547 bytes --]

On Fri, Mar 13, 2026 at 01:44:01AM -0700, Bo Gan wrote:
> Starfive JH7110 and ESWIN EIC770X both have non cache-coherent
> peripherals. On JH7110[1], GPU/VOUT/VPU/ISP are routed to the sys port,
> making them not cache-coherent. On EIC770X, all peripherals are routed
> to the sys port, and none is cache-coherent. To make drivers work on
> such platforms, the standard solution is to use Svpbmt and map the DMA
> buffer as uncacheable. However, neither SoC supports Svpbmt. Instead,
> they map the system memory twice, as cached and uncached. The uncached
> alias implicitly applies the uncacheable PMA. To support such platform,
> a special form of Svpbmt, namely "XPbmtUC" is introduced in this patch.
> It's a synthetical PTE format where a single bit (UC) is controlling
> the cacheability and the bit position can be configured at runtime. It
> is intended to model the physical memory aliasing with minimal effort.
> 
> On JH7110, it aligns perfectly with the HW, as the aliased UC region
> happens to be offsetted by 2^34. Thus, configuring the XPbmtUC with
> bit=32 (PPN is shifted by 2) is all that needs to be done.
> 
> On EIC770X, the aliased UC region is put to a awkward offset, and given
> there can be 2 NUMA node (dual-die) with 2 separate memory regions and
> their UC alias counterpart, we instead ask the firmware to provide a
> thin-layer hypervisor to re-arrange the memory map. The XPbmtUC will be
> enabled with bit=38, thus map all UC pages to 2^40 (the upper-half of
> 2^41), and the underlaying hypervisor will re-map the 2^40+ addresses
> to the appropriate UC alias regions. (See description in PATCH 1/6)
> 
> We chose bit 38 (PPN bit 40) to make the 2-stage translation efficient.
> Hypervisor can utilize Sv39x4 G-stage scheme, and map all pages as 1GB
> huge page, consuming only the first-level page table (16KB total), and
> several TLB entries. In practice, it's the firmware/bootloader that
> configures XPbmtUC through device-tree, based on firmware capabilities,
> and skip the enablement on stock firmware. This is tested on Hifive
> Premier P550 with the modified OpenSBI[2]. It runs the host Linux in VS
> mode, and provide the aforementioned remapping. The performance penalty
> (if not running KVM in Linux) is minimal, as the CPU is never switched
> to HS mode. A very slight, unavoidable, slow down is with the external
> interrupt delivery. Due to the lack of AIA in EIC770X, all device irq
> now needs to trap to M mode first, before forwarding to VS mode. The
> overhead of running KVM in such setup is yet unknown, and may well be
> noticeable, as all HS-qualified instructions will trap to M mode, and
> there's also the extra cost of flushing G/VS-stage TLBs. I'm analyzing
> it in parallel.
> 
> I'm aware there's an ongoing series that Samuel sent for physical
> memory aliases. I haven't been following too closely, but if you're
> worried about it touching to many areas, I hope my series can shed some
> light on the problem. My change is very minimal and local, also fairly
> easy to remove if we later decide deprecating it down the road.
> 
> [1] https://github.com/starfive-tech/JH7100_Docs/blob/main/JH7100%20Cache%20Coherence%20V1.0.pdf
> [2] https://github.com/ganboing/opensbi/tree/eic77x-vspt-physalias-wip

For those following along at home, Samuel's series is:
https://lore.kernel.org/all/20251113014656.2605447-20-samuel.holland@sifive.com/

I've been meaning to try it, but never conjured up the time to dig into
it...

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: [RFC PATCH 1/6] riscv: Add a custom, simplified version of Svpbmt "XPbmtUC"

[Conor Dooley]

[-- Attachment #1: Type: text/plain, Size: 8120 bytes --]

Hey,

Gonna offer some feedback on the detail of what's been done in this
series, without providing any commentary on whether this is the correct
approach to take.

On Fri, Mar 13, 2026 at 01:44:02AM -0700, Bo Gan wrote:
> On platforms that doesn't support Svpbmt or XTheadMae, SoC vendors
> sometimes map the system memory twice in physical address space, one
> as cached, and the other as uncached. Through the uncached window,
> device drivers will be able to map DMA buffer for noncoherent devices.
> Such setup is usually found in SoC with pre-Svpbmt Sifive cores.
> Make use of such feature by modeling it as "XPbmtUC", a customized
> version of Svpbmt, where a single bit in PTE is used for UC control.
> There's no IO bit with such scheme, as it's assumed that the PMA
> (usually hard-wired on these SoCs) will properly convey the strongly-
> ordered, non-idempotent attribute of the MMIO region.
> 
> The enablement of such position of "XPbmtUC" is controlled by the
> device-tree property "riscv,xpbmt-uncache-bit".

Firstly, the naming generally I take some exception to. If this is some
fake vendor extension for linux purposes, it needs to have "xlinux" in
it, like our xlinuxenvcfg does. It should also be consistent, don't use
"xpmbtuc" and "xpbmt-uncache-bit", pick one and stick to it.

Athough, I think I disagree fundamentally with this property, as it seems
to me like "software configuration" that shouldn't be permitted in
devicetree. Maybe I am misunderstanding, but the numbers you chose are
convenient, not set in stone by the specific hardware, right?

I'd be much more comfortable with adding xlinuxwhatever to
riscv,isa-extensions, to signal that a soc supports this stuff than with
a property for the bit itself. I suppose that bit information could then
come from a LUT in the vendor extensions, that a validate callback could
check (via root compatible) before enabling. There's not a super neat
way to do that at the moment though I don't think, code currently
expects that vendor extensions are in a different "namespace" to
standard ones, and this would blur the lines because it's not from a
specific vendor, nor is it a standard extension.
I guess, it could be done by keeping it as a standard number, but then
it's a bit trickier to neatly access the LUT while keeping it split
apart.
I know this means having to modify the kernel if there's a new device,
but I'm inclined to say "deal with it" because they could've done
something standard and opted not to.

Could also argue that this should be shoved into a sifive specific
thing, but I don't expect that they're the only ones with devices like
this that could benefit.

> 
> Example:
> 
> Starfive JH7110 (Sifive U74):
>            [0x0,   0x40000000) Low MMIO
>     [0x40000000, 0x2_40000000) Cached Mem
>   [0x4_40000000, 0x6_40000000) Uncached Mem UC+
>   [0x9_00000000, 0x9_d0000000) High MMIO
> 
> Device-tree:
>   riscv,xpbmt-uncache-bit = <32>;
> 
> Use PTE bit 32 (PPN bit 34) as UC (uncache) control to perfectly
> match the memory map of the SoC.
> 
> ESWIN EIC770X (Sifive U84/P550):
>            [0x0,    0x20000000) Core Internal
>     [0x20000000,    0x40000000) Core Internal (Die 1)
>     [0x40000000,    0x60000000) Low MMIO
>     [0x60000000,    0x80000000) Low MMIO (Die 1)
>     [0x80000000, 0x10_80000000) Cached Mem
>  [0x20_00000000, 0x30_00000000) Cached Mem (Die 1)
>  [0x80_00000000, 0xa0_00000000) High MMIO
>  [0xa0_00000000, 0xc0_00000000) High MMIO (Die 1)
>  [0xc0_00000000, 0xd0_00000000) Uncached Mem
>  [0xe0_00000000, 0xf0_00000000) Uncached Mem (Die 1)
> 
> EIC770X is not directly compatible to this model, as the uncached
> regions are offsetted, and the offset is different among the Dies
> in the dual-die version (EIC7702). so we expect the firmware to
> provide a thin layer of hypervisor to transparently re-map:
> 
>     [0x80000000,  0x10_80000000) Cached Mem
>  [0x20_00000000,  0x30_00000000) Cached Mem (Die 1)
>  [0xc0_00000000,  0xd0_00000000) Uncached Mem <----------.
>  [0xe0_00000000,  0xf0_00000000) Uncached Mem (Die 1) <--+--.
> [0x100_80000000, 0x110_80000000) Mem UC+ ----------------'  |
> [0x120_00000000, 0x130_00000000) Mem UC+ (Die 1) -----------'
> 
> With that, the firmware/bootloader can set the following at boot:
>   riscv,xpbmt-uncache-bit = <38>;
> 
> Signed-off-by: Bo Gan <ganboing@gmail.com>
> ---
>  arch/riscv/Kconfig                  | 12 ++++++++++++
>  arch/riscv/include/asm/hwcap.h      |  1 +
>  arch/riscv/include/asm/pgtable-64.h |  8 ++++++++
>  arch/riscv/kernel/cpufeature.c      |  8 ++++++++
>  arch/riscv/mm/pgtable.c             |  7 +++++++
>  5 files changed, 36 insertions(+)
> 
> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> index 6b39f37f769a2..f2b4da6a3deb1 100644
> --- a/arch/riscv/Kconfig
> +++ b/arch/riscv/Kconfig
> @@ -893,6 +893,18 @@ config TOOLCHAIN_NEEDS_OLD_ISA_SPEC
>  	  versions of clang and GCC to be passed to GAS, which has the same result
>  	  as passing zicsr and zifencei to -march.
>  
> +config RISCV_ISA_XPBMTUC
> +	bool "Support XPbmtUC (customized pbmt uncache bit)"
> +	depends on 64BIT && MMU
> +	depends on RISCV_ALTERNATIVE
> +	default n
> +	select DMA_DIRECT_REMAP
> +	help
> +	  Add support for "riscv,xpbmt-uncache-bit" device-tree property.
> +	  The bit denotes the bit in PTE that marks the page as uncached.
> +
> +	  If you don't know what to do here, say N.
> +
>  config FPU
>  	bool "FPU support"
>  	default y
> diff --git a/arch/riscv/include/asm/hwcap.h b/arch/riscv/include/asm/hwcap.h
> index 4369a23385413..6baa6566cf4cc 100644
> --- a/arch/riscv/include/asm/hwcap.h
> +++ b/arch/riscv/include/asm/hwcap.h
> @@ -111,6 +111,7 @@
>  #define RISCV_ISA_EXT_ZILSD		102
>  #define RISCV_ISA_EXT_ZCLSD		103
>  
> +#define RISCV_ISA_EXT_XPBMTUC		126
>  #define RISCV_ISA_EXT_XLINUXENVCFG	127
>  
>  #define RISCV_ISA_EXT_MAX		128
> diff --git a/arch/riscv/include/asm/pgtable-64.h b/arch/riscv/include/asm/pgtable-64.h
> index 6e789fa58514c..1a6d04884111d 100644
> --- a/arch/riscv/include/asm/pgtable-64.h
> +++ b/arch/riscv/include/asm/pgtable-64.h
> @@ -140,6 +140,14 @@ enum napot_cont_order {
>  #define _PAGE_IO_THEAD		((1UL << 63) | (1UL << 60))
>  #define _PAGE_MTMASK_THEAD	(_PAGE_PMA_THEAD | _PAGE_IO_THEAD | (1UL << 59))
>  
> +#ifdef CONFIG_RISCV_ISA_XPBMTUC
> +extern int riscv_xpbmtuc_bit;
> +extern u64 riscv_xpbmtuc_mask;
> +#endif
> +
> +#define XPBMTUC_HAS_PAGE_NOCACHE CONFIG_RISCV_ISA_XPBMTUC
> +#define XPBMTUC_HAS_PAGE_MTMASK  CONFIG_RISCV_ISA_XPBMTUC
> +
>  static inline u64 riscv_page_mtmask(void)
>  {
>  	u64 val;
> diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c
> index fa591aff9d335..faec169004b4a 100644
> --- a/arch/riscv/kernel/cpufeature.c
> +++ b/arch/riscv/kernel/cpufeature.c
> @@ -1118,6 +1118,14 @@ void __init riscv_fill_hwcap(void)
>  		riscv_v_setup_vsize();
>  	}
>  
> +#ifdef CONFIG_RISCV_ISA_XPBMTUC

Code like this needs to be unconditionally compiled.

> +	if (!of_property_read_u32(of_root, "riscv,xpbmt-uncache-bit",
> +				  &riscv_xpbmtuc_bit)) {
> +		riscv_xpbmtuc_mask = 1UL << riscv_xpbmtuc_bit;
> +		set_bit(RISCV_ISA_EXT_XPBMTUC, riscv_isa);
> +		pr_info("Using XPbmtUC bit=%d\n", riscv_xpbmtuc_bit);
> +	}
> +#endif
>  	memset(print_str, 0, sizeof(print_str));
>  	for (i = 0, j = 0; i < NUM_ALPHA_EXTS; i++)
>  		if (riscv_isa[0] & BIT_MASK(i))
> diff --git a/arch/riscv/mm/pgtable.c b/arch/riscv/mm/pgtable.c
> index 807c0a0de1827..4ca442bc8595d 100644
> --- a/arch/riscv/mm/pgtable.c
> +++ b/arch/riscv/mm/pgtable.c
> @@ -5,6 +5,13 @@
>  #include <linux/kernel.h>
>  #include <linux/pgtable.h>
>  
> +#ifdef CONFIG_RISCV_ISA_XPBMTUC
> +int riscv_xpbmtuc_bit;
> +
> +u64 riscv_xpbmtuc_mask;
> +EXPORT_SYMBOL(riscv_xpbmtuc_mask);
> +#endif
> +
>  int ptep_set_access_flags(struct vm_area_struct *vma,
>  			  unsigned long address, pte_t *ptep,
>  			  pte_t entry, int dirty)
> -- 
> 2.34.1
> 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: [RFC PATCH 3/6] riscv: apply page table attribute bits for XPbmtUC

[Conor Dooley]

[-- Attachment #1: Type: text/plain, Size: 2867 bytes --]

On Fri, Mar 13, 2026 at 01:44:04AM -0700, Bo Gan wrote:
> Apply the UC bit like Svpbmt and THEAD_MAE does. Also changed the
> _PAGE_PFN_MASK definition to exclude the UC bit, as it's position
> is now determined at runtime, and can be part of PPN.
> 
> Signed-off-by: Bo Gan <ganboing@gmail.com>

This should be squashed with the patch adding detection and the Kconfig
option.

> ---
>  arch/riscv/include/asm/errata_list.h | 17 +++++++++++++++--
>  arch/riscv/include/asm/pgtable-64.h  |  9 ++++++++-
>  2 files changed, 23 insertions(+), 3 deletions(-)
> 
> diff --git a/arch/riscv/include/asm/errata_list.h b/arch/riscv/include/asm/errata_list.h
> index 6694b5ccdcf85..ba0f3d4dd0cbb 100644
> --- a/arch/riscv/include/asm/errata_list.h
> +++ b/arch/riscv/include/asm/errata_list.h
> @@ -53,6 +53,16 @@ asm(ALTERNATIVE(	\
>  	: /* no inputs */	\
>  	: "memory")
>  
> +#ifdef CONFIG_64BIT
> +#define ALT_PAGE_CUST_BIT(_bit)						\
> +asm(ALTERNATIVE("li %0, 0\t\nnop",					\
> +		"1: auipc %0, %%pcrel_hi(riscv_xpbmtuc_mask)\t\n"	\
> +		      "ld %0, %%pcrel_lo(1b)(%0)", 0,			\
> +			RISCV_ISA_EXT_XPBMTUC,				\
> +			CONFIG_RISCV_ISA_XPBMTUC)			\
> +		: "=r"(_bit))
> +#endif
> +
>  /*
>   * _val is marked as "will be overwritten", so need to set it to 0
>   * in the default case.
> @@ -60,11 +70,14 @@ asm(ALTERNATIVE(	\
>  #define ALT_SVPBMT_SHIFT 61
>  #define ALT_THEAD_MAE_SHIFT 59
>  #define ALT_SVPBMT(_val, prot)						\
> -asm(ALTERNATIVE_2("li %0, 0\t\nnop",					\
> +asm(ALTERNATIVE_3("li %0, 0\t\nnop",					\
>  		  "li %0, %1\t\nslli %0,%0,%3", 0,			\
>  			RISCV_ISA_EXT_SVPBMT, CONFIG_RISCV_ISA_SVPBMT,	\
>  		  "li %0, %2\t\nslli %0,%0,%4", THEAD_VENDOR_ID,	\
> -			ERRATA_THEAD_MAE, CONFIG_ERRATA_THEAD_MAE)	\
> +			ERRATA_THEAD_MAE, CONFIG_ERRATA_THEAD_MAE,	\
> +		  "1: auipc %0, %%pcrel_hi(riscv_xpbmtuc_mask)\t\n"	\
> +			"ld %0, %%pcrel_lo(1b)(%0)", 0,			\
> +			RISCV_ISA_EXT_XPBMTUC, XPBMTUC_HAS##prot)	\
>  		: "=r"(_val)						\
>  		: "I"(prot##_SVPBMT >> ALT_SVPBMT_SHIFT),		\
>  		  "I"(prot##_THEAD >> ALT_THEAD_MAE_SHIFT),		\
> diff --git a/arch/riscv/include/asm/pgtable-64.h b/arch/riscv/include/asm/pgtable-64.h
> index 1a6d04884111d..aab6990d92238 100644
> --- a/arch/riscv/include/asm/pgtable-64.h
> +++ b/arch/riscv/include/asm/pgtable-64.h
> @@ -76,7 +76,14 @@ typedef struct {
>   * | 63 | 62 61 | 60 54 | 53  10 | 9             8 | 7 | 6 | 5 | 4 | 3 | 2 | 1 | 0
>   *   N      MT     RSV    PFN      reserved for SW   D   A   G   U   X   W   R   V
>   */
> -#define _PAGE_PFN_MASK  GENMASK(53, 10)
> +static inline u64 riscv_pfn_mask(void)
> +{
> +	u64 cust_bit;
> +
> +	ALT_PAGE_CUST_BIT(cust_bit);
> +	return GENMASK(53, 10) ^ cust_bit;
> +}
> +#define _PAGE_PFN_MASK  riscv_pfn_mask()
>  
>  /*
>   * [63] Svnapot definitions:
> -- 
> 2.34.1
> 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: [RFC PATCH 4/6] riscv: select RISCV_ISA_XPBMTUC in STARFIVE and ESWIN SoC

[Conor Dooley]

[-- Attachment #1: Type: text/plain, Size: 1052 bytes --]

On Fri, Mar 13, 2026 at 01:44:05AM -0700, Bo Gan wrote:
> Enable the XPbmtUC feature for Starfive and ESWIN SoC
> 
> Signed-off-by: Bo Gan <ganboing@gmail.com>
> ---
>  arch/riscv/Kconfig.socs | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/arch/riscv/Kconfig.socs b/arch/riscv/Kconfig.socs
> index d621b85dd63bd..0584511707c7c 100644
> --- a/arch/riscv/Kconfig.socs
> +++ b/arch/riscv/Kconfig.socs
> @@ -14,6 +14,7 @@ config ARCH_ANLOGIC
>  
>  config ARCH_ESWIN
>  	bool "ESWIN SoCs"
> +	select RISCV_ISA_XPBMTUC

Nah, you can't do this. Either RISCV_ISA_XPBMTUC is user selectable or
it is mandatory for these platforms and selected. Don't mix and match
please.

>  	help
>  	  This enables support for ESWIN SoC platform hardware,
>  	  including the ESWIN EIC7700 SoC.
> @@ -56,6 +57,7 @@ config SOC_STARFIVE
>  	select PINCTRL
>  	select RESET_CONTROLLER
>  	select ARM_AMBA
> +	select RISCV_ISA_XPBMTUC
>  	help
>  	  This enables support for StarFive SoC platform hardware.
>  
> -- 
> 2.34.1
> 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: [RFC PATCH 5/6] riscv: dts: starfive: jh7110: activate XPbmtUC

[Conor Dooley]

[-- Attachment #1: Type: text/plain, Size: 1455 bytes --]

On Fri, Mar 13, 2026 at 01:44:06AM -0700, Bo Gan wrote:
> Set riscv,xpbmt-uncache-bit to 32 to match SoC memory map:
> 
>            [0x0,   0x40000000) Low MMIO
>     [0x40000000, 0x2_40000000) Cached Mem
>   [0x4_40000000, 0x6_40000000) Uncached Mem UC+
>   [0x9_00000000, 0x9_d0000000) High MMIO
> 
> Signed-off-by: Bo Gan <ganboing@gmail.com>


What I want know is how this whole setup interacts with the existing
support that we have for these devices?
Samuel's patchetset removed from the devicetree all of the nodes related
to having two mappings of the same memory, and modified the existing
erratum to only be required for older devicetrees.
You've not removed them, only added a new property. The non-coherent
peripherals on jh7110 already work prior to this patchset, is there not
going to be funky behaviour with both of these things operating in
parallel?

> ---
>  arch/riscv/boot/dts/starfive/jh7110.dtsi | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/arch/riscv/boot/dts/starfive/jh7110.dtsi b/arch/riscv/boot/dts/starfive/jh7110.dtsi
> index 6e56e9d20bb06..6dfeb31538fba 100644
> --- a/arch/riscv/boot/dts/starfive/jh7110.dtsi
> +++ b/arch/riscv/boot/dts/starfive/jh7110.dtsi
> @@ -14,6 +14,7 @@ / {
>  	compatible = "starfive,jh7110";
>  	#address-cells = <2>;
>  	#size-cells = <2>;
> +	riscv,xpbmt-uncache-bit = <32>;
>  
>  	cpus: cpus {
>  		#address-cells = <1>;
> -- 
> 2.34.1
> 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: [RFC PATCH 1/6] riscv: Add a custom, simplified version of Svpbmt "XPbmtUC"

[Bo Gan]

Hi Conor,

Thanks so much for the prompt review. See inline.

On 3/13/26 06:24, Conor Dooley wrote:
> Hey,
> 
> Gonna offer some feedback on the detail of what's been done in this
> series, without providing any commentary on whether this is the correct
> approach to take.
> 
> On Fri, Mar 13, 2026 at 01:44:02AM -0700, Bo Gan wrote:
>> On platforms that doesn't support Svpbmt or XTheadMae, SoC vendors
>> sometimes map the system memory twice in physical address space, one
>> as cached, and the other as uncached. Through the uncached window,
>> device drivers will be able to map DMA buffer for noncoherent devices.
>> Such setup is usually found in SoC with pre-Svpbmt Sifive cores.
>> Make use of such feature by modeling it as "XPbmtUC", a customized
>> version of Svpbmt, where a single bit in PTE is used for UC control.
>> There's no IO bit with such scheme, as it's assumed that the PMA
>> (usually hard-wired on these SoCs) will properly convey the strongly-
>> ordered, non-idempotent attribute of the MMIO region.
>>
>> The enablement of such position of "XPbmtUC" is controlled by the
>> device-tree property "riscv,xpbmt-uncache-bit".
> 
> Firstly, the naming generally I take some exception to. If this is some
> fake vendor extension for linux purposes, it needs to have "xlinux" in
> it, like our xlinuxenvcfg does. It should also be consistent, don't use
> "xpmbtuc" and "xpbmt-uncache-bit", pick one and stick to it.
> 
Makes sense. I can certainly change that to be conformant.

> Athough, I think I disagree fundamentally with this property, as it seems
> to me like "software configuration" that shouldn't be permitted in
> devicetree. Maybe I am misunderstanding, but the numbers you chose are
> convenient, not set in stone by the specific hardware, right?

For JH7110, the bit 32 (PPN bit 34) matches exactly with the HW. Meaning
toggling this bit would re-map the page to the uncached window, which
matches perfectly with the synthetic UC bit in the scheme.

For EIC770X, the bit 38 (PPN bit 40) is hand picked to be able to map all
physical memory space (40 bit), while making it very easy for the thin-
hypervisor, which can utilize Sv39x4 (41 bit) page scheme in G-stage.

I also considered the sbi call approach, where the kernel can query for
the support and position of the uncache bit. The thing is that JH7110
can just hard-code the bit without any changes to firmware, and I want
to have a consistent way for both SoC, thus the device-tree approach, to
let the EIC770X firmware/bootloader adding the property to dt at runtime.
Any better ideas?

> 
> I'd be much more comfortable with adding xlinuxwhatever to
> riscv,isa-extensions, to signal that a soc supports this stuff than with
> a property for the bit itself. I suppose that bit information could then
> come from a LUT in the vendor extensions, that a validate callback could
> check (via root compatible) before enabling. There's not a super neat
> way to do that at the moment though I don't think, code currently
> expects that vendor extensions are in a different "namespace" to
> standard ones, and this would blur the lines because it's not from a
> specific vendor, nor is it a standard extension.
> I guess, it could be done by keeping it as a standard number, but then
> it's a bit trickier to neatly access the LUT while keeping it split
> apart.
> I know this means having to modify the kernel if there's a new device,
> but I'm inclined to say "deal with it" because they could've done
> something standard and opted not to.
> 
> Could also argue that this should be shoved into a sifive specific
> thing, but I don't expect that they're the only ones with devices like
> this that could benefit.
> 

I've thought about riscv,isa-extensions. The issue with that is that it's
a per-CPU thing, but I'm adding a global extension, and I don't want to
pollute the isa-extension string. Thus, I followed Samuel's approach --
He uses "riscv,physical-memory-regions" in the root node.

>>
>> Example:
>>
>> Starfive JH7110 (Sifive U74):
>>             [0x0,   0x40000000) Low MMIO
>>      [0x40000000, 0x2_40000000) Cached Mem
>>    [0x4_40000000, 0x6_40000000) Uncached Mem UC+
>>    [0x9_00000000, 0x9_d0000000) High MMIO
>>
>> Device-tree:
>>    riscv,xpbmt-uncache-bit = <32>;
>>
>> Use PTE bit 32 (PPN bit 34) as UC (uncache) control to perfectly
>> match the memory map of the SoC.
>>
>> ESWIN EIC770X (Sifive U84/P550):
>>             [0x0,    0x20000000) Core Internal
>>      [0x20000000,    0x40000000) Core Internal (Die 1)
>>      [0x40000000,    0x60000000) Low MMIO
>>      [0x60000000,    0x80000000) Low MMIO (Die 1)
>>      [0x80000000, 0x10_80000000) Cached Mem
>>   [0x20_00000000, 0x30_00000000) Cached Mem (Die 1)
>>   [0x80_00000000, 0xa0_00000000) High MMIO
>>   [0xa0_00000000, 0xc0_00000000) High MMIO (Die 1)
>>   [0xc0_00000000, 0xd0_00000000) Uncached Mem
>>   [0xe0_00000000, 0xf0_00000000) Uncached Mem (Die 1)
>>
>> EIC770X is not directly compatible to this model, as the uncached
>> regions are offsetted, and the offset is different among the Dies
>> in the dual-die version (EIC7702). so we expect the firmware to
>> provide a thin layer of hypervisor to transparently re-map:
>>
>>      [0x80000000,  0x10_80000000) Cached Mem
>>   [0x20_00000000,  0x30_00000000) Cached Mem (Die 1)
>>   [0xc0_00000000,  0xd0_00000000) Uncached Mem <----------.
>>   [0xe0_00000000,  0xf0_00000000) Uncached Mem (Die 1) <--+--.
>> [0x100_80000000, 0x110_80000000) Mem UC+ ----------------'  |
>> [0x120_00000000, 0x130_00000000) Mem UC+ (Die 1) -----------'
>>
>> With that, the firmware/bootloader can set the following at boot:
>>    riscv,xpbmt-uncache-bit = <38>;
>>
>> Signed-off-by: Bo Gan <ganboing@gmail.com>
>> ---
>>   arch/riscv/Kconfig                  | 12 ++++++++++++
>>   arch/riscv/include/asm/hwcap.h      |  1 +
>>   arch/riscv/include/asm/pgtable-64.h |  8 ++++++++
>>   arch/riscv/kernel/cpufeature.c      |  8 ++++++++
>>   arch/riscv/mm/pgtable.c             |  7 +++++++
>>   5 files changed, 36 insertions(+)
>>
>> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
>> index 6b39f37f769a2..f2b4da6a3deb1 100644
>> --- a/arch/riscv/Kconfig
>> +++ b/arch/riscv/Kconfig
>> @@ -893,6 +893,18 @@ config TOOLCHAIN_NEEDS_OLD_ISA_SPEC
>>   	  versions of clang and GCC to be passed to GAS, which has the same result
>>   	  as passing zicsr and zifencei to -march.
>>   
>> +config RISCV_ISA_XPBMTUC
>> +	bool "Support XPbmtUC (customized pbmt uncache bit)"
>> +	depends on 64BIT && MMU
>> +	depends on RISCV_ALTERNATIVE
>> +	default n
>> +	select DMA_DIRECT_REMAP
>> +	help
>> +	  Add support for "riscv,xpbmt-uncache-bit" device-tree property.
>> +	  The bit denotes the bit in PTE that marks the page as uncached.
>> +
>> +	  If you don't know what to do here, say N.
>> +
>>   config FPU
>>   	bool "FPU support"
>>   	default y
>> diff --git a/arch/riscv/include/asm/hwcap.h b/arch/riscv/include/asm/hwcap.h
>> index 4369a23385413..6baa6566cf4cc 100644
>> --- a/arch/riscv/include/asm/hwcap.h
>> +++ b/arch/riscv/include/asm/hwcap.h
>> @@ -111,6 +111,7 @@
>>   #define RISCV_ISA_EXT_ZILSD		102
>>   #define RISCV_ISA_EXT_ZCLSD		103
>>   
>> +#define RISCV_ISA_EXT_XPBMTUC		126
>>   #define RISCV_ISA_EXT_XLINUXENVCFG	127
>>   
>>   #define RISCV_ISA_EXT_MAX		128
>> diff --git a/arch/riscv/include/asm/pgtable-64.h b/arch/riscv/include/asm/pgtable-64.h
>> index 6e789fa58514c..1a6d04884111d 100644
>> --- a/arch/riscv/include/asm/pgtable-64.h
>> +++ b/arch/riscv/include/asm/pgtable-64.h
>> @@ -140,6 +140,14 @@ enum napot_cont_order {
>>   #define _PAGE_IO_THEAD		((1UL << 63) | (1UL << 60))
>>   #define _PAGE_MTMASK_THEAD	(_PAGE_PMA_THEAD | _PAGE_IO_THEAD | (1UL << 59))
>>   
>> +#ifdef CONFIG_RISCV_ISA_XPBMTUC
>> +extern int riscv_xpbmtuc_bit;
>> +extern u64 riscv_xpbmtuc_mask;
>> +#endif
>> +
>> +#define XPBMTUC_HAS_PAGE_NOCACHE CONFIG_RISCV_ISA_XPBMTUC
>> +#define XPBMTUC_HAS_PAGE_MTMASK  CONFIG_RISCV_ISA_XPBMTUC
>> +
>>   static inline u64 riscv_page_mtmask(void)
>>   {
>>   	u64 val;
>> diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c
>> index fa591aff9d335..faec169004b4a 100644
>> --- a/arch/riscv/kernel/cpufeature.c
>> +++ b/arch/riscv/kernel/cpufeature.c
>> @@ -1118,6 +1118,14 @@ void __init riscv_fill_hwcap(void)
>>   		riscv_v_setup_vsize();
>>   	}
>>   
>> +#ifdef CONFIG_RISCV_ISA_XPBMTUC
> 
> Code like this needs to be unconditionally compiled.
> 
>> +	if (!of_property_read_u32(of_root, "riscv,xpbmt-uncache-bit",
>> +				  &riscv_xpbmtuc_bit)) {
>> +		riscv_xpbmtuc_mask = 1UL << riscv_xpbmtuc_bit;
>> +		set_bit(RISCV_ISA_EXT_XPBMTUC, riscv_isa);
>> +		pr_info("Using XPbmtUC bit=%d\n", riscv_xpbmtuc_bit);
>> +	}
>> +#endif
>>   	memset(print_str, 0, sizeof(print_str));
>>   	for (i = 0, j = 0; i < NUM_ALPHA_EXTS; i++)
>>   		if (riscv_isa[0] & BIT_MASK(i))
>> diff --git a/arch/riscv/mm/pgtable.c b/arch/riscv/mm/pgtable.c
>> index 807c0a0de1827..4ca442bc8595d 100644
>> --- a/arch/riscv/mm/pgtable.c
>> +++ b/arch/riscv/mm/pgtable.c
>> @@ -5,6 +5,13 @@
>>   #include <linux/kernel.h>
>>   #include <linux/pgtable.h>
>>   
>> +#ifdef CONFIG_RISCV_ISA_XPBMTUC
>> +int riscv_xpbmtuc_bit;
>> +
>> +u64 riscv_xpbmtuc_mask;
>> +EXPORT_SYMBOL(riscv_xpbmtuc_mask);
>> +#endif
>> +
>>   int ptep_set_access_flags(struct vm_area_struct *vma,
>>   			  unsigned long address, pte_t *ptep,
>>   			  pte_t entry, int dirty)
>> -- 
>> 2.34.1
>>

Bo

Re: [RFC PATCH 3/6] riscv: apply page table attribute bits for XPbmtUC

[Bo Gan]

Hi Conor,

On 3/13/26 06:24, Conor Dooley wrote:
> On Fri, Mar 13, 2026 at 01:44:04AM -0700, Bo Gan wrote:
>> Apply the UC bit like Svpbmt and THEAD_MAE does. Also changed the
>> _PAGE_PFN_MASK definition to exclude the UC bit, as it's position
>> is now determined at runtime, and can be part of PPN.
>>
>> Signed-off-by: Bo Gan <ganboing@gmail.com>
> 
> This should be squashed with the patch adding detection and the Kconfig
> option.
> 

Sure.

>> ---
>>   arch/riscv/include/asm/errata_list.h | 17 +++++++++++++++--
>>   arch/riscv/include/asm/pgtable-64.h  |  9 ++++++++-
>>   2 files changed, 23 insertions(+), 3 deletions(-)
>>
>> diff --git a/arch/riscv/include/asm/errata_list.h b/arch/riscv/include/asm/errata_list.h
>> index 6694b5ccdcf85..ba0f3d4dd0cbb 100644
>> --- a/arch/riscv/include/asm/errata_list.h
>> +++ b/arch/riscv/include/asm/errata_list.h
>> @@ -53,6 +53,16 @@ asm(ALTERNATIVE(	\
>>   	: /* no inputs */	\
>>   	: "memory")
>>   
>> +#ifdef CONFIG_64BIT
>> +#define ALT_PAGE_CUST_BIT(_bit)						\
>> +asm(ALTERNATIVE("li %0, 0\t\nnop",					\
>> +		"1: auipc %0, %%pcrel_hi(riscv_xpbmtuc_mask)\t\n"	\
>> +		      "ld %0, %%pcrel_lo(1b)(%0)", 0,			\
>> +			RISCV_ISA_EXT_XPBMTUC,				\
>> +			CONFIG_RISCV_ISA_XPBMTUC)			\
>> +		: "=r"(_bit))
>> +#endif
>> +
>>   /*
>>    * _val is marked as "will be overwritten", so need to set it to 0
>>    * in the default case.
>> @@ -60,11 +70,14 @@ asm(ALTERNATIVE(	\
>>   #define ALT_SVPBMT_SHIFT 61
>>   #define ALT_THEAD_MAE_SHIFT 59
>>   #define ALT_SVPBMT(_val, prot)						\
>> -asm(ALTERNATIVE_2("li %0, 0\t\nnop",					\
>> +asm(ALTERNATIVE_3("li %0, 0\t\nnop",					\
>>   		  "li %0, %1\t\nslli %0,%0,%3", 0,			\
>>   			RISCV_ISA_EXT_SVPBMT, CONFIG_RISCV_ISA_SVPBMT,	\
>>   		  "li %0, %2\t\nslli %0,%0,%4", THEAD_VENDOR_ID,	\
>> -			ERRATA_THEAD_MAE, CONFIG_ERRATA_THEAD_MAE)	\
>> +			ERRATA_THEAD_MAE, CONFIG_ERRATA_THEAD_MAE,	\
>> +		  "1: auipc %0, %%pcrel_hi(riscv_xpbmtuc_mask)\t\n"	\
>> +			"ld %0, %%pcrel_lo(1b)(%0)", 0,			\
>> +			RISCV_ISA_EXT_XPBMTUC, XPBMTUC_HAS##prot)	\
>>   		: "=r"(_val)						\
>>   		: "I"(prot##_SVPBMT >> ALT_SVPBMT_SHIFT),		\
>>   		  "I"(prot##_THEAD >> ALT_THEAD_MAE_SHIFT),		\
>> diff --git a/arch/riscv/include/asm/pgtable-64.h b/arch/riscv/include/asm/pgtable-64.h
>> index 1a6d04884111d..aab6990d92238 100644
>> --- a/arch/riscv/include/asm/pgtable-64.h
>> +++ b/arch/riscv/include/asm/pgtable-64.h
>> @@ -76,7 +76,14 @@ typedef struct {
>>    * | 63 | 62 61 | 60 54 | 53  10 | 9             8 | 7 | 6 | 5 | 4 | 3 | 2 | 1 | 0
>>    *   N      MT     RSV    PFN      reserved for SW   D   A   G   U   X   W   R   V
>>    */
>> -#define _PAGE_PFN_MASK  GENMASK(53, 10)
>> +static inline u64 riscv_pfn_mask(void)
>> +{
>> +	u64 cust_bit;
>> +
>> +	ALT_PAGE_CUST_BIT(cust_bit);
>> +	return GENMASK(53, 10) ^ cust_bit;
>> +}
>> +#define _PAGE_PFN_MASK  riscv_pfn_mask()
>>   
>>   /*
>>    * [63] Svnapot definitions:
>> -- 
>> 2.34.1
>>

Bo

Re: [RFC PATCH 4/6] riscv: select RISCV_ISA_XPBMTUC in STARFIVE and ESWIN SoC

[Bo Gan]

Hi Conor,

On 3/13/26 06:28, Conor Dooley wrote:
> On Fri, Mar 13, 2026 at 01:44:05AM -0700, Bo Gan wrote:
>> Enable the XPbmtUC feature for Starfive and ESWIN SoC
>>
>> Signed-off-by: Bo Gan <ganboing@gmail.com>
>> ---
>>   arch/riscv/Kconfig.socs | 2 ++
>>   1 file changed, 2 insertions(+)
>>
>> diff --git a/arch/riscv/Kconfig.socs b/arch/riscv/Kconfig.socs
>> index d621b85dd63bd..0584511707c7c 100644
>> --- a/arch/riscv/Kconfig.socs
>> +++ b/arch/riscv/Kconfig.socs
>> @@ -14,6 +14,7 @@ config ARCH_ANLOGIC
>>   
>>   config ARCH_ESWIN
>>   	bool "ESWIN SoCs"
>> +	select RISCV_ISA_XPBMTUC
> 
> Nah, you can't do this. Either RISCV_ISA_XPBMTUC is user selectable or
> it is mandatory for these platforms and selected. Don't mix and match
> please.
> 

Sure.

>>   	help
>>   	  This enables support for ESWIN SoC platform hardware,
>>   	  including the ESWIN EIC7700 SoC.
>> @@ -56,6 +57,7 @@ config SOC_STARFIVE
>>   	select PINCTRL
>>   	select RESET_CONTROLLER
>>   	select ARM_AMBA
>> +	select RISCV_ISA_XPBMTUC
>>   	help
>>   	  This enables support for StarFive SoC platform hardware.
>>   
>> -- 
>> 2.34.1
>>

Bo

Re: [RFC PATCH 5/6] riscv: dts: starfive: jh7110: activate XPbmtUC

[Bo Gan]

Hi Conor,

On 3/13/26 06:48, Conor Dooley wrote:
> On Fri, Mar 13, 2026 at 01:44:06AM -0700, Bo Gan wrote:
>> Set riscv,xpbmt-uncache-bit to 32 to match SoC memory map:
>>
>>             [0x0,   0x40000000) Low MMIO
>>      [0x40000000, 0x2_40000000) Cached Mem
>>    [0x4_40000000, 0x6_40000000) Uncached Mem UC+
>>    [0x9_00000000, 0x9_d0000000) High MMIO
>>
>> Signed-off-by: Bo Gan <ganboing@gmail.com>
> 
> 
> What I want know is how this whole setup interacts with the existing
> support that we have for these devices?
> Samuel's patchetset removed from the devicetree all of the nodes related
> to having two mappings of the same memory, and modified the existing
> erratum to only be required for older devicetrees.
> You've not removed them, only added a new property. The non-coherent
> peripherals on jh7110 already work prior to this patchset, is there not
> going to be funky behaviour with both of these things operating in
> parallel?
> 

I just want to clarify that Samuel's change is not touching JH7110, but
*JH7100*. They are very similar chips, can can confuse people sometimes,
but JH7110 evolved to put more devices such as gmac/sdio/usb/pcie through
the front port to make them cache coherent. The left over noncoherent
device are dealt by the driver through cache flushes, I believe. That's
why we have out-of-box support for JH7110 even without Samuel's patch. On
the older JH7100, none of the peripheral is DMA coherent, so we either
use ERRATA_STARFIVE_JH7100, making kernel NONPORTABLE or Samuel's patch.
Refer to this pdf for a detailed comparison:
https://github.com/starfive-tech/JH7100_Docs/blob/main/JH7100%20Cache%20Coherence%20V1.0.pdf

My patch doesn't touch anything JH7100. It's an enhancement to JH7110 (the
newer chip). The issue is that even though basic peripherals are coherent,
the video subsystem, GPU/VPU/ISP... is still not. (See page 6 in that pdf)
Hence technically, we still need to deal with the same issue on JH7110.
E.g., part of the reason we can't have GPU driver working is because we
can't map a DMA page as uncached in userspace. There's no cache flush insn
in userspace, either. This change tries to solve the issue in a simple yet
elegant way.

There's no other changes to the device-tree required, just adding the prop
for JH7110, and things should just work. As for JH7100 (the older chip), I
don't plan to support it, as the cached/uncached region are offsetted by
0xf80000000, so it can't be modeled by a single UC bit (w/o hypervisor
re-mapping it underneath, and there's no H for JH71x0) The chip was also
superseded very quickly after the release of JH7110. I don't imagine too
many users demanding support.

>> ---
>>   arch/riscv/boot/dts/starfive/jh7110.dtsi | 1 +
>>   1 file changed, 1 insertion(+)
>>
>> diff --git a/arch/riscv/boot/dts/starfive/jh7110.dtsi b/arch/riscv/boot/dts/starfive/jh7110.dtsi
>> index 6e56e9d20bb06..6dfeb31538fba 100644
>> --- a/arch/riscv/boot/dts/starfive/jh7110.dtsi
>> +++ b/arch/riscv/boot/dts/starfive/jh7110.dtsi
>> @@ -14,6 +14,7 @@ / {
>>   	compatible = "starfive,jh7110";
>>   	#address-cells = <2>;
>>   	#size-cells = <2>;
>> +	riscv,xpbmt-uncache-bit = <32>;
>>   
>>   	cpus: cpus {
>>   		#address-cells = <1>;
>> -- 
>> 2.34.1
>>

Bo

Re: [RFC PATCH 0/6] riscv: support EIC770X/JH7110 noncoherent devices with XPbmtUC

[Bo Gan]

Hi

On 3/13/26 05:30, Conor Dooley wrote:
> On Fri, Mar 13, 2026 at 01:44:01AM -0700, Bo Gan wrote:
>> Starfive JH7110 and ESWIN EIC770X both have non cache-coherent
>> peripherals. On JH7110[1], GPU/VOUT/VPU/ISP are routed to the sys port,
>> making them not cache-coherent. On EIC770X, all peripherals are routed
>> to the sys port, and none is cache-coherent. To make drivers work on
>> such platforms, the standard solution is to use Svpbmt and map the DMA
>> buffer as uncacheable. However, neither SoC supports Svpbmt. Instead,
>> they map the system memory twice, as cached and uncached. The uncached
>> alias implicitly applies the uncacheable PMA. To support such platform,
>> a special form of Svpbmt, namely "XPbmtUC" is introduced in this patch.
>> It's a synthetical PTE format where a single bit (UC) is controlling
>> the cacheability and the bit position can be configured at runtime. It
>> is intended to model the physical memory aliasing with minimal effort.
>>
>> On JH7110, it aligns perfectly with the HW, as the aliased UC region
>> happens to be offsetted by 2^34. Thus, configuring the XPbmtUC with
>> bit=32 (PPN is shifted by 2) is all that needs to be done.
>>
>> On EIC770X, the aliased UC region is put to a awkward offset, and given
>> there can be 2 NUMA node (dual-die) with 2 separate memory regions and
>> their UC alias counterpart, we instead ask the firmware to provide a
>> thin-layer hypervisor to re-arrange the memory map. The XPbmtUC will be
>> enabled with bit=38, thus map all UC pages to 2^40 (the upper-half of
>> 2^41), and the underlaying hypervisor will re-map the 2^40+ addresses
>> to the appropriate UC alias regions. (See description in PATCH 1/6)
>>
>> We chose bit 38 (PPN bit 40) to make the 2-stage translation efficient.
>> Hypervisor can utilize Sv39x4 G-stage scheme, and map all pages as 1GB
>> huge page, consuming only the first-level page table (16KB total), and
>> several TLB entries. In practice, it's the firmware/bootloader that
>> configures XPbmtUC through device-tree, based on firmware capabilities,
>> and skip the enablement on stock firmware. This is tested on Hifive
>> Premier P550 with the modified OpenSBI[2]. It runs the host Linux in VS
>> mode, and provide the aforementioned remapping. The performance penalty
>> (if not running KVM in Linux) is minimal, as the CPU is never switched
>> to HS mode. A very slight, unavoidable, slow down is with the external
>> interrupt delivery. Due to the lack of AIA in EIC770X, all device irq
>> now needs to trap to M mode first, before forwarding to VS mode. The
>> overhead of running KVM in such setup is yet unknown, and may well be
>> noticeable, as all HS-qualified instructions will trap to M mode, and
>> there's also the extra cost of flushing G/VS-stage TLBs. I'm analyzing
>> it in parallel.
>>
>> I'm aware there's an ongoing series that Samuel sent for physical
>> memory aliases. I haven't been following too closely, but if you're
>> worried about it touching to many areas, I hope my series can shed some
>> light on the problem. My change is very minimal and local, also fairly
>> easy to remove if we later decide deprecating it down the road.
>>
>> [1] https://github.com/starfive-tech/JH7100_Docs/blob/main/JH7100%20Cache%20Coherence%20V1.0.pdf
>> [2] https://github.com/ganboing/opensbi/tree/eic77x-vspt-physalias-wip
> 
> For those following along at home, Samuel's series is:
> https://lore.kernel.org/all/20251113014656.2605447-20-samuel.holland@sifive.com/
> 
> I've been meaning to try it, but never conjured up the time to dig into
> it...

Thanks, Conor. I don't mean to step ahead of Samuel, just want to find a
middle ground that's easier to maintain from kernel perspective. I know
riscv HW is evolving rapidly, and we just don't want to add way too many
workarounds for each SoC. Hence I decided to move the re-mapping logic to
firmware/hypervisor to keep the kernel clean. If you'd like, use my v6.19
tree for testing on Hifive P550 if you have one:
https://github.com/ganboing/linux-eic77/tree/ganboing-xpbmt-uc-v1-eic77-clk-v15
I'm also sanity checking on my JH7110.

Bo

Re: [RFC PATCH 5/6] riscv: dts: starfive: jh7110: activate XPbmtUC

[Conor Dooley]

[-- Attachment #1: Type: text/plain, Size: 1465 bytes --]

On Fri, Mar 13, 2026 at 02:59:44PM -0700, Bo Gan wrote:
> Hi Conor,
> 
> On 3/13/26 06:48, Conor Dooley wrote:
> > On Fri, Mar 13, 2026 at 01:44:06AM -0700, Bo Gan wrote:
> > > Set riscv,xpbmt-uncache-bit to 32 to match SoC memory map:
> > > 
> > >             [0x0,   0x40000000) Low MMIO
> > >      [0x40000000, 0x2_40000000) Cached Mem
> > >    [0x4_40000000, 0x6_40000000) Uncached Mem UC+
> > >    [0x9_00000000, 0x9_d0000000) High MMIO
> > > 
> > > Signed-off-by: Bo Gan <ganboing@gmail.com>
> > 
> > 
> > What I want know is how this whole setup interacts with the existing
> > support that we have for these devices?
> > Samuel's patchetset removed from the devicetree all of the nodes related
> > to having two mappings of the same memory, and modified the existing
> > erratum to only be required for older devicetrees.
> > You've not removed them, only added a new property. The non-coherent
> > peripherals on jh7110 already work prior to this patchset, is there not
> > going to be funky behaviour with both of these things operating in
> > parallel?
> > 
> 
> I just want to clarify that Samuel's change is not touching JH7110, but
> *JH7100*. They are very similar chips, can can confuse people sometimes,
> but JH7110 evolved to put more devices such as gmac/sdio/usb/pcie through
> the front port to make them cache coherent. The left over noncoherent

Believe it or not, I know that! I just misread the filename ;)

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: [RFC PATCH 1/6] riscv: Add a custom, simplified version of Svpbmt "XPbmtUC"

[Conor Dooley]

[-- Attachment #1: Type: text/plain, Size: 4742 bytes --]

On Fri, Mar 13, 2026 at 02:33:16PM -0700, Bo Gan wrote:
> Hi Conor,
> 
> Thanks so much for the prompt review. See inline.
> 
> On 3/13/26 06:24, Conor Dooley wrote:
> > Hey,
> > 
> > Gonna offer some feedback on the detail of what's been done in this
> > series, without providing any commentary on whether this is the correct
> > approach to take.
> > 
> > On Fri, Mar 13, 2026 at 01:44:02AM -0700, Bo Gan wrote:
> > > On platforms that doesn't support Svpbmt or XTheadMae, SoC vendors
> > > sometimes map the system memory twice in physical address space, one
> > > as cached, and the other as uncached. Through the uncached window,
> > > device drivers will be able to map DMA buffer for noncoherent devices.
> > > Such setup is usually found in SoC with pre-Svpbmt Sifive cores.
> > > Make use of such feature by modeling it as "XPbmtUC", a customized
> > > version of Svpbmt, where a single bit in PTE is used for UC control.
> > > There's no IO bit with such scheme, as it's assumed that the PMA
> > > (usually hard-wired on these SoCs) will properly convey the strongly-
> > > ordered, non-idempotent attribute of the MMIO region.
> > > 
> > > The enablement of such position of "XPbmtUC" is controlled by the
> > > device-tree property "riscv,xpbmt-uncache-bit".
> > 
> > Firstly, the naming generally I take some exception to. If this is some
> > fake vendor extension for linux purposes, it needs to have "xlinux" in
> > it, like our xlinuxenvcfg does. It should also be consistent, don't use
> > "xpmbtuc" and "xpbmt-uncache-bit", pick one and stick to it.
> > 
> Makes sense. I can certainly change that to be conformant.
> 
> > Athough, I think I disagree fundamentally with this property, as it seems
> > to me like "software configuration" that shouldn't be permitted in
> > devicetree. Maybe I am misunderstanding, but the numbers you chose are
> > convenient, not set in stone by the specific hardware, right?
> 
> For JH7110, the bit 32 (PPN bit 34) matches exactly with the HW. Meaning
> toggling this bit would re-map the page to the uncached window, which
> matches perfectly with the synthetic UC bit in the scheme.

What does "matches exactly with the hardware" mean? AFAICT, you picked
it because it was the best value, but you could also have picked another
less optimal value?

> 
> For EIC770X, the bit 38 (PPN bit 40) is hand picked to be able to map all
> physical memory space (40 bit), while making it very easy for the thin-
> hypervisor, which can utilize Sv39x4 (41 bit) page scheme in G-stage.
> 
> I also considered the sbi call approach, where the kernel can query for
> the support and position of the uncache bit. The thing is that JH7110
> can just hard-code the bit without any changes to firmware, and I want
> to have a consistent way for both SoC, thus the device-tree approach, to
> let the EIC770X firmware/bootloader adding the property to dt at runtime.
> Any better ideas?

Is the only thing that's variable on your eic770x platform whether or
not the bit is enabled? Or are you looking to vary the bit depending on
the specific platform?

> > I'd be much more comfortable with adding xlinuxwhatever to
> > riscv,isa-extensions, to signal that a soc supports this stuff than with
> > a property for the bit itself. I suppose that bit information could then
> > come from a LUT in the vendor extensions, that a validate callback could
> > check (via root compatible) before enabling. There's not a super neat
> > way to do that at the moment though I don't think, code currently
> > expects that vendor extensions are in a different "namespace" to
> > standard ones, and this would blur the lines because it's not from a
> > specific vendor, nor is it a standard extension.
> > I guess, it could be done by keeping it as a standard number, but then
> > it's a bit trickier to neatly access the LUT while keeping it split
> > apart.
> > I know this means having to modify the kernel if there's a new device,
> > but I'm inclined to say "deal with it" because they could've done
> > something standard and opted not to.
> > 
> > Could also argue that this should be shoved into a sifive specific
> > thing, but I don't expect that they're the only ones with devices like
> > this that could benefit.
> > 
> 
> I've thought about riscv,isa-extensions. The issue with that is that it's
> a per-CPU thing, but I'm adding a global extension, and I don't want to

Most of the extensions in that string are effectively global. There's no
need to worry about "polluting" it.

> pollute the isa-extension string. Thus, I followed Samuel's approach --
> He uses "riscv,physical-memory-regions" in the root node.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: [RFC PATCH 1/6] riscv: Add a custom, simplified version of Svpbmt "XPbmtUC"

[Bo Gan]

Hi Conor,

On 3/13/26 16:55, Conor Dooley wrote:
> On Fri, Mar 13, 2026 at 02:33:16PM -0700, Bo Gan wrote:
>> Hi Conor,
>>
>> Thanks so much for the prompt review. See inline.
>>
>> On 3/13/26 06:24, Conor Dooley wrote:
>>> Hey,
>>>
>>> Gonna offer some feedback on the detail of what's been done in this
>>> series, without providing any commentary on whether this is the correct
>>> approach to take.
>>>
>>> On Fri, Mar 13, 2026 at 01:44:02AM -0700, Bo Gan wrote:
>>>> On platforms that doesn't support Svpbmt or XTheadMae, SoC vendors
>>>> sometimes map the system memory twice in physical address space, one
>>>> as cached, and the other as uncached. Through the uncached window,
>>>> device drivers will be able to map DMA buffer for noncoherent devices.
>>>> Such setup is usually found in SoC with pre-Svpbmt Sifive cores.
>>>> Make use of such feature by modeling it as "XPbmtUC", a customized
>>>> version of Svpbmt, where a single bit in PTE is used for UC control.
>>>> There's no IO bit with such scheme, as it's assumed that the PMA
>>>> (usually hard-wired on these SoCs) will properly convey the strongly-
>>>> ordered, non-idempotent attribute of the MMIO region.
>>>>
>>>> The enablement of such position of "XPbmtUC" is controlled by the
>>>> device-tree property "riscv,xpbmt-uncache-bit".
>>>
>>> Firstly, the naming generally I take some exception to. If this is some
>>> fake vendor extension for linux purposes, it needs to have "xlinux" in
>>> it, like our xlinuxenvcfg does. It should also be consistent, don't use
>>> "xpmbtuc" and "xpbmt-uncache-bit", pick one and stick to it.
>>>
>> Makes sense. I can certainly change that to be conformant.
>>
>>> Athough, I think I disagree fundamentally with this property, as it seems
>>> to me like "software configuration" that shouldn't be permitted in
>>> devicetree. Maybe I am misunderstanding, but the numbers you chose are
>>> convenient, not set in stone by the specific hardware, right?
>>
>> For JH7110, the bit 32 (PPN bit 34) matches exactly with the HW. Meaning
>> toggling this bit would re-map the page to the uncached window, which
>> matches perfectly with the synthetic UC bit in the scheme.
> 
> What does "matches exactly with the hardware" mean? AFAICT, you picked
> it because it was the best value, but you could also have picked another
> less optimal value?
> 
>>
>> For EIC770X, the bit 38 (PPN bit 40) is hand picked to be able to map all
>> physical memory space (40 bit), while making it very easy for the thin-
>> hypervisor, which can utilize Sv39x4 (41 bit) page scheme in G-stage.
>>
>> I also considered the sbi call approach, where the kernel can query for
>> the support and position of the uncache bit. The thing is that JH7110
>> can just hard-code the bit without any changes to firmware, and I want
>> to have a consistent way for both SoC, thus the device-tree approach, to
>> let the EIC770X firmware/bootloader adding the property to dt at runtime.
>> Any better ideas?
> 
> Is the only thing that's variable on your eic770x platform whether or
> not the bit is enabled? Or are you looking to vary the bit depending on
> the specific platform?
> 

It'll be "fixed" for eic770x if a thin-hypervisor re-mapping is enabled
underneath. It just so happens that the physical address space is 40 bits
(ignoring the 40bit+ upper uncached region for interleaved memory, which
we don't need when the "xpbmt-uc" is enabled anyway), and the hypervisor
can use Sv39x4 (also 41bit) to re-map everything.

The variation comes with different SoCs, JH7110 vs. EIC770X. I'd like to
make it a variable, to make a unified kernel binary boot on all SoCs, so
I need to fix the alternative logic for PC-relative instructions to read
from a global variable "xpbmtuc_bit/mask". Also I want to avoid adding
too many branches to the alternative macro.

>>> I'd be much more comfortable with adding xlinuxwhatever to
>>> riscv,isa-extensions, to signal that a soc supports this stuff than with
>>> a property for the bit itself. I suppose that bit information could then
>>> come from a LUT in the vendor extensions, that a validate callback could
>>> check (via root compatible) before enabling. There's not a super neat
>>> way to do that at the moment though I don't think, code currently
>>> expects that vendor extensions are in a different "namespace" to
>>> standard ones, and this would blur the lines because it's not from a
>>> specific vendor, nor is it a standard extension.
>>> I guess, it could be done by keeping it as a standard number, but then
>>> it's a bit trickier to neatly access the LUT while keeping it split
>>> apart.
>>> I know this means having to modify the kernel if there's a new device,
>>> but I'm inclined to say "deal with it" because they could've done
>>> something standard and opted not to.
>>>
>>> Could also argue that this should be shoved into a sifive specific
>>> thing, but I don't expect that they're the only ones with devices like
>>> this that could benefit.
>>>
>>
>> I've thought about riscv,isa-extensions. The issue with that is that it's
>> a per-CPU thing, but I'm adding a global extension, and I don't want to
> 
> Most of the extensions in that string are effectively global. There's no
> need to worry about "polluting" it.
> 

Got it. So I can use something like "xlinuxpbmtuc38" in isa-string? (until
someone comes up with a better naming. Naming things is hard...)

>> pollute the isa-extension string. Thus, I followed Samuel's approach --
>> He uses "riscv,physical-memory-regions" in the root node.

Bo

Re: [RFC PATCH 1/6] riscv: Add a custom, simplified version of Svpbmt "XPbmtUC"

[Conor Dooley]

[-- Attachment #1: Type: text/plain, Size: 7089 bytes --]

On Fri, Mar 13, 2026 at 05:29:53PM -0700, Bo Gan wrote:
> Hi Conor,
> 
> On 3/13/26 16:55, Conor Dooley wrote:
> > On Fri, Mar 13, 2026 at 02:33:16PM -0700, Bo Gan wrote:
> > > Hi Conor,
> > > 
> > > Thanks so much for the prompt review. See inline.
> > > 
> > > On 3/13/26 06:24, Conor Dooley wrote:
> > > > Hey,
> > > > 
> > > > Gonna offer some feedback on the detail of what's been done in this
> > > > series, without providing any commentary on whether this is the correct
> > > > approach to take.
> > > > 
> > > > On Fri, Mar 13, 2026 at 01:44:02AM -0700, Bo Gan wrote:
> > > > > On platforms that doesn't support Svpbmt or XTheadMae, SoC vendors
> > > > > sometimes map the system memory twice in physical address space, one
> > > > > as cached, and the other as uncached. Through the uncached window,
> > > > > device drivers will be able to map DMA buffer for noncoherent devices.
> > > > > Such setup is usually found in SoC with pre-Svpbmt Sifive cores.
> > > > > Make use of such feature by modeling it as "XPbmtUC", a customized
> > > > > version of Svpbmt, where a single bit in PTE is used for UC control.
> > > > > There's no IO bit with such scheme, as it's assumed that the PMA
> > > > > (usually hard-wired on these SoCs) will properly convey the strongly-
> > > > > ordered, non-idempotent attribute of the MMIO region.
> > > > > 
> > > > > The enablement of such position of "XPbmtUC" is controlled by the
> > > > > device-tree property "riscv,xpbmt-uncache-bit".
> > > > 
> > > > Firstly, the naming generally I take some exception to. If this is some
> > > > fake vendor extension for linux purposes, it needs to have "xlinux" in
> > > > it, like our xlinuxenvcfg does. It should also be consistent, don't use
> > > > "xpmbtuc" and "xpbmt-uncache-bit", pick one and stick to it.
> > > > 
> > > Makes sense. I can certainly change that to be conformant.
> > > 
> > > > Athough, I think I disagree fundamentally with this property, as it seems
> > > > to me like "software configuration" that shouldn't be permitted in
> > > > devicetree. Maybe I am misunderstanding, but the numbers you chose are
> > > > convenient, not set in stone by the specific hardware, right?
> > > 
> > > For JH7110, the bit 32 (PPN bit 34) matches exactly with the HW. Meaning
> > > toggling this bit would re-map the page to the uncached window, which
> > > matches perfectly with the synthetic UC bit in the scheme.
> > 
> > What does "matches exactly with the hardware" mean? AFAICT, you picked
> > it because it was the best value, but you could also have picked another
> > less optimal value?
> > 
> > > 
> > > For EIC770X, the bit 38 (PPN bit 40) is hand picked to be able to map all
> > > physical memory space (40 bit), while making it very easy for the thin-
> > > hypervisor, which can utilize Sv39x4 (41 bit) page scheme in G-stage.
> > > 
> > > I also considered the sbi call approach, where the kernel can query for
> > > the support and position of the uncache bit. The thing is that JH7110
> > > can just hard-code the bit without any changes to firmware, and I want
> > > to have a consistent way for both SoC, thus the device-tree approach, to
> > > let the EIC770X firmware/bootloader adding the property to dt at runtime.
> > > Any better ideas?
> > 
> > Is the only thing that's variable on your eic770x platform whether or
> > not the bit is enabled? Or are you looking to vary the bit depending on
> > the specific platform?
> > 
> 
> It'll be "fixed" for eic770x if a thin-hypervisor re-mapping is enabled
> underneath. It just so happens that the physical address space is 40 bits
> (ignoring the 40bit+ upper uncached region for interleaved memory, which
> we don't need when the "xpbmt-uc" is enabled anyway), and the hypervisor
> can use Sv39x4 (also 41bit) to re-map everything.
> 
> The variation comes with different SoCs, JH7110 vs. EIC770X. I'd like to
> make it a variable, to make a unified kernel binary boot on all SoCs, so

FWIW, I have no interest in things that are not multiplatform-safe, so
anything I've been suggesting has been with that in mind. When I was
talking about not conveying the bit via DT, but storing the value in the
kernel, I was still considering that the values would be stored for
specific soc compatibles.

To be honest, I'm not completely dead-set opposed to a property that has
the bit positioning, but any property being added for what is
effectively an erratum needs to pass a high bar when the info could be
gathered in another way. That the eic7700 one depends on firmware for
what the bit may be is points in your favour, since firmware variability
is part of what dt is there to do. The jh7110 is points against, since
it could be fished out of the errata handling code.

> I need to fix the alternative logic for PC-relative instructions to read
> from a global variable "xpbmtuc_bit/mask". Also I want to avoid adding
> too many branches to the alternative macro.
> 
> > > > I'd be much more comfortable with adding xlinuxwhatever to
> > > > riscv,isa-extensions, to signal that a soc supports this stuff than with
> > > > a property for the bit itself. I suppose that bit information could then
> > > > come from a LUT in the vendor extensions, that a validate callback could
> > > > check (via root compatible) before enabling. There's not a super neat
> > > > way to do that at the moment though I don't think, code currently
> > > > expects that vendor extensions are in a different "namespace" to
> > > > standard ones, and this would blur the lines because it's not from a
> > > > specific vendor, nor is it a standard extension.
> > > > I guess, it could be done by keeping it as a standard number, but then
> > > > it's a bit trickier to neatly access the LUT while keeping it split
> > > > apart.
> > > > I know this means having to modify the kernel if there's a new device,
> > > > but I'm inclined to say "deal with it" because they could've done
> > > > something standard and opted not to.
> > > > 
> > > > Could also argue that this should be shoved into a sifive specific
> > > > thing, but I don't expect that they're the only ones with devices like
> > > > this that could benefit.
> > > > 
> > > 
> > > I've thought about riscv,isa-extensions. The issue with that is that it's
> > > a per-CPU thing, but I'm adding a global extension, and I don't want to
> > 
> > Most of the extensions in that string are effectively global. There's no
> > need to worry about "polluting" it.
> > 
> 
> Got it. So I can use something like "xlinuxpbmtuc38" in isa-string? (until
> someone comes up with a better naming. Naming things is hard...)

I don't think the encoding of the bit should be in the name, otherwise
we'd need to many different variations, if using riscv,isa-extensions is
the approach that ends up being used.

> > > pollute the isa-extension string. Thus, I followed Samuel's approach --
> > > He uses "riscv,physical-memory-regions" in the root node.
> 
> Bo

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: [RFC PATCH 1/6] riscv: Add a custom, simplified version of Svpbmt "XPbmtUC"

[Bo Gan]

Hi Conor,

On 3/13/26 18:18, Conor Dooley wrote:
> On Fri, Mar 13, 2026 at 05:29:53PM -0700, Bo Gan wrote:
>> Hi Conor,
>>
>> On 3/13/26 16:55, Conor Dooley wrote:
>>> On Fri, Mar 13, 2026 at 02:33:16PM -0700, Bo Gan wrote:
>>>> Hi Conor,
>>>>
>>>> Thanks so much for the prompt review. See inline.
>>>>
>>>> On 3/13/26 06:24, Conor Dooley wrote:
>>>>> Hey,
>>>>>
>>>>> Gonna offer some feedback on the detail of what's been done in this
>>>>> series, without providing any commentary on whether this is the correct
>>>>> approach to take.
>>>>>
>>>>> On Fri, Mar 13, 2026 at 01:44:02AM -0700, Bo Gan wrote:
>>>>>> On platforms that doesn't support Svpbmt or XTheadMae, SoC vendors
>>>>>> sometimes map the system memory twice in physical address space, one
>>>>>> as cached, and the other as uncached. Through the uncached window,
>>>>>> device drivers will be able to map DMA buffer for noncoherent devices.
>>>>>> Such setup is usually found in SoC with pre-Svpbmt Sifive cores.
>>>>>> Make use of such feature by modeling it as "XPbmtUC", a customized
>>>>>> version of Svpbmt, where a single bit in PTE is used for UC control.
>>>>>> There's no IO bit with such scheme, as it's assumed that the PMA
>>>>>> (usually hard-wired on these SoCs) will properly convey the strongly-
>>>>>> ordered, non-idempotent attribute of the MMIO region.
>>>>>>
>>>>>> The enablement of such position of "XPbmtUC" is controlled by the
>>>>>> device-tree property "riscv,xpbmt-uncache-bit".
>>>>>
>>>>> Firstly, the naming generally I take some exception to. If this is some
>>>>> fake vendor extension for linux purposes, it needs to have "xlinux" in
>>>>> it, like our xlinuxenvcfg does. It should also be consistent, don't use
>>>>> "xpmbtuc" and "xpbmt-uncache-bit", pick one and stick to it.
>>>>>
>>>> Makes sense. I can certainly change that to be conformant.
>>>>
>>>>> Athough, I think I disagree fundamentally with this property, as it seems
>>>>> to me like "software configuration" that shouldn't be permitted in
>>>>> devicetree. Maybe I am misunderstanding, but the numbers you chose are
>>>>> convenient, not set in stone by the specific hardware, right?
>>>>
>>>> For JH7110, the bit 32 (PPN bit 34) matches exactly with the HW. Meaning
>>>> toggling this bit would re-map the page to the uncached window, which
>>>> matches perfectly with the synthetic UC bit in the scheme.
>>>
>>> What does "matches exactly with the hardware" mean? AFAICT, you picked
>>> it because it was the best value, but you could also have picked another
>>> less optimal value?
>>>
>>>>
>>>> For EIC770X, the bit 38 (PPN bit 40) is hand picked to be able to map all
>>>> physical memory space (40 bit), while making it very easy for the thin-
>>>> hypervisor, which can utilize Sv39x4 (41 bit) page scheme in G-stage.
>>>>
>>>> I also considered the sbi call approach, where the kernel can query for
>>>> the support and position of the uncache bit. The thing is that JH7110
>>>> can just hard-code the bit without any changes to firmware, and I want
>>>> to have a consistent way for both SoC, thus the device-tree approach, to
>>>> let the EIC770X firmware/bootloader adding the property to dt at runtime.
>>>> Any better ideas?
>>>
>>> Is the only thing that's variable on your eic770x platform whether or
>>> not the bit is enabled? Or are you looking to vary the bit depending on
>>> the specific platform?
>>>
>>
>> It'll be "fixed" for eic770x if a thin-hypervisor re-mapping is enabled
>> underneath. It just so happens that the physical address space is 40 bits
>> (ignoring the 40bit+ upper uncached region for interleaved memory, which
>> we don't need when the "xpbmt-uc" is enabled anyway), and the hypervisor
>> can use Sv39x4 (also 41bit) to re-map everything.
>>
>> The variation comes with different SoCs, JH7110 vs. EIC770X. I'd like to
>> make it a variable, to make a unified kernel binary boot on all SoCs, so
> 
> FWIW, I have no interest in things that are not multiplatform-safe, so
> anything I've been suggesting has been with that in mind. When I was
> talking about not conveying the bit via DT, but storing the value in the
> kernel, I was still considering that the values would be stored for
> specific soc compatibles.
> 
> To be honest, I'm not completely dead-set opposed to a property that has
> the bit positioning, but any property being added for what is
> effectively an erratum needs to pass a high bar when the info could be
> gathered in another way. That the eic7700 one depends on firmware for
> what the bit may be is points in your favour, since firmware variability
> is part of what dt is there to do. The jh7110 is points against, since
> it could be fished out of the errata handling code.
> 
Even for JH7110, I don't think it can be handled through the errata. It
describes the errata of the core (if I'm not mistaken), and there can be
other SoCs using the same core with the same archid/impid, but maps the
peripherals differently, and the UC bit position doesn't apply there. I
think you are probably looking for "SoC level errata" handling. It's not
there AFAIK. Hence I guess both SoC cases point in favor of the dt prop?

>> I need to fix the alternative logic for PC-relative instructions to read
>> from a global variable "xpbmtuc_bit/mask". Also I want to avoid adding
>> too many branches to the alternative macro.
>>
>>>>> I'd be much more comfortable with adding xlinuxwhatever to
>>>>> riscv,isa-extensions, to signal that a soc supports this stuff than with
>>>>> a property for the bit itself. I suppose that bit information could then
>>>>> come from a LUT in the vendor extensions, that a validate callback could
>>>>> check (via root compatible) before enabling. There's not a super neat
>>>>> way to do that at the moment though I don't think, code currently
>>>>> expects that vendor extensions are in a different "namespace" to
>>>>> standard ones, and this would blur the lines because it's not from a
>>>>> specific vendor, nor is it a standard extension.
>>>>> I guess, it could be done by keeping it as a standard number, but then
>>>>> it's a bit trickier to neatly access the LUT while keeping it split
>>>>> apart.
>>>>> I know this means having to modify the kernel if there's a new device,
>>>>> but I'm inclined to say "deal with it" because they could've done
>>>>> something standard and opted not to.
>>>>>
>>>>> Could also argue that this should be shoved into a sifive specific
>>>>> thing, but I don't expect that they're the only ones with devices like
>>>>> this that could benefit.
>>>>>
>>>>
>>>> I've thought about riscv,isa-extensions. The issue with that is that it's
>>>> a per-CPU thing, but I'm adding a global extension, and I don't want to
>>>
>>> Most of the extensions in that string are effectively global. There's no
>>> need to worry about "polluting" it.
>>>
>>
>> Got it. So I can use something like "xlinuxpbmtuc38" in isa-string? (until
>> someone comes up with a better naming. Naming things is hard...)
> 
> I don't think the encoding of the bit should be in the name, otherwise
> we'd need to many different variations, if using riscv,isa-extensions is
> the approach that ends up being used.
> 
>>>> pollute the isa-extension string. Thus, I followed Samuel's approach --
>>>> He uses "riscv,physical-memory-regions" in the root node.
>>
>> Bo

Bo

Re: [RFC PATCH 1/6] riscv: Add a custom, simplified version of Svpbmt "XPbmtUC"

[Conor Dooley]

[-- Attachment #1: Type: text/plain, Size: 1984 bytes --]

On Fri, Mar 13, 2026 at 10:06:42PM -0700, Bo Gan wrote:
> > To be honest, I'm not completely dead-set opposed to a property that has
> > the bit positioning, but any property being added for what is
> > effectively an erratum needs to pass a high bar when the info could be
> > gathered in another way. That the eic7700 one depends on firmware for
> > what the bit may be is points in your favour, since firmware variability
> > is part of what dt is there to do. The jh7110 is points against, since
> > it could be fished out of the errata handling code.
> > 
> Even for JH7110, I don't think it can be handled through the errata. It
> describes the errata of the core (if I'm not mistaken), and there can be
> other SoCs using the same core with the same archid/impid, but maps the
> peripherals differently, and the UC bit position doesn't apply there. I
> think you are probably looking for "SoC level errata" handling. It's not
> there AFAIK. Hence I guess both SoC cases point in favor of the dt prop?

I dunno, nothing wrong with checking the devicetree during the errata
"probe" code. Checks are not limited to imp/arch ids, can do ecalls etc
etc in there too, so looking at the root compatible would be possible.

Either way, if people like what you've done here generally (because
coming up with our own use of PTE bits could be controversial), and a
custom property of some sort is to be used, you need to provide a good
justification of why it is needed in the commit messages because you're
setting a precedent of being the first "extension" conjured up to suit
linux that would need that kind of functionality.
Need to demonstrate that it describes an aspect of the hardware, and
isn't being conjured up to configure software to use one out of several
possible values, that it may even be able to determine heuristically
from information already provided in the devicetree (like the root
compatible or a completely described memory node).

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: [RFC PATCH 1/6] riscv: Add a custom, simplified version of Svpbmt "XPbmtUC"

[Conor Dooley]

[-- Attachment #1: Type: text/plain, Size: 1048 bytes --]

On Fri, Mar 13, 2026 at 02:33:16PM -0700, Bo Gan wrote:
> > > diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c
> > > index fa591aff9d335..faec169004b4a 100644
> > > --- a/arch/riscv/kernel/cpufeature.c
> > > +++ b/arch/riscv/kernel/cpufeature.c
> > > @@ -1118,6 +1118,14 @@ void __init riscv_fill_hwcap(void)
> > >   		riscv_v_setup_vsize();
> > >   	}
> > > +#ifdef CONFIG_RISCV_ISA_XPBMTUC
> > 
> > Code like this needs to be unconditionally compiled.
> > 
> > > +	if (!of_property_read_u32(of_root, "riscv,xpbmt-uncache-bit",
> > > +				  &riscv_xpbmtuc_bit)) {
> > > +		riscv_xpbmtuc_mask = 1UL << riscv_xpbmtuc_bit;
> > > +		set_bit(RISCV_ISA_EXT_XPBMTUC, riscv_isa);
> > > +		pr_info("Using XPbmtUC bit=%d\n", riscv_xpbmtuc_bit);
> > > +	}
> > > +#endif
> > >   	memset(print_str, 0, sizeof(print_str));
> > >   	for (i = 0, j = 0; i < NUM_ALPHA_EXTS; i++)
> > >   		if (riscv_isa[0] & BIT_MASK(i))


btw, not sure if you saw this comment, about using IS_ENABLED() rather
than ifdeffery.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: [RFC PATCH 1/6] riscv: Add a custom, simplified version of Svpbmt "XPbmtUC"

[Bo Gan]

On 3/14/26 05:17, Conor Dooley wrote:
> On Fri, Mar 13, 2026 at 10:06:42PM -0700, Bo Gan wrote:
>>> To be honest, I'm not completely dead-set opposed to a property that has
>>> the bit positioning, but any property being added for what is
>>> effectively an erratum needs to pass a high bar when the info could be
>>> gathered in another way. That the eic7700 one depends on firmware for
>>> what the bit may be is points in your favour, since firmware variability
>>> is part of what dt is there to do. The jh7110 is points against, since
>>> it could be fished out of the errata handling code.
>>>
>> Even for JH7110, I don't think it can be handled through the errata. It
>> describes the errata of the core (if I'm not mistaken), and there can be
>> other SoCs using the same core with the same archid/impid, but maps the
>> peripherals differently, and the UC bit position doesn't apply there. I
>> think you are probably looking for "SoC level errata" handling. It's not
>> there AFAIK. Hence I guess both SoC cases point in favor of the dt prop?
> 
> I dunno, nothing wrong with checking the devicetree during the errata
> "probe" code. Checks are not limited to imp/arch ids, can do ecalls etc
> etc in there too, so looking at the root compatible would be possible.
> 
> Either way, if people like what you've done here generally (because
> coming up with our own use of PTE bits could be controversial), and a
> custom property of some sort is to be used, you need to provide a good
> justification of why it is needed in the commit messages because you're
> setting a precedent of being the first "extension" conjured up to suit
> linux that would need that kind of functionality.
> Need to demonstrate that it describes an aspect of the hardware, and
> isn't being conjured up to configure software to use one out of several
> possible values, that it may even be able to determine heuristically
> from information already provided in the devicetree (like the root
> compatible or a completely described memory node).

Got your point. I've moved away from DT and "extension" in v2 and made
it a sifive "errata", given that mapping memory twice through front/sys
port is more of a sifive concept, not something generic to other core
vendors. The DT is kept untouched, and the detection logic is done by
a LUT and sbi ecalls if not predefined. Link:

https://lore.kernel.org/linux-riscv/20260316060328.1173634-1-ganboing@gmail.com

Bo