Re: [RFC PATCH 0/6] riscv: support EIC770X/JH7110 noncoherent devices with XPbmtUC

[Conor Dooley <conor@kernel.org>]

[-- Attachment #1: Type: text/plain, Size: 3547 bytes --]

On Fri, Mar 13, 2026 at 01:44:01AM -0700, Bo Gan wrote:
> Starfive JH7110 and ESWIN EIC770X both have non cache-coherent
> peripherals. On JH7110[1], GPU/VOUT/VPU/ISP are routed to the sys port,
> making them not cache-coherent. On EIC770X, all peripherals are routed
> to the sys port, and none is cache-coherent. To make drivers work on
> such platforms, the standard solution is to use Svpbmt and map the DMA
> buffer as uncacheable. However, neither SoC supports Svpbmt. Instead,
> they map the system memory twice, as cached and uncached. The uncached
> alias implicitly applies the uncacheable PMA. To support such platform,
> a special form of Svpbmt, namely "XPbmtUC" is introduced in this patch.
> It's a synthetical PTE format where a single bit (UC) is controlling
> the cacheability and the bit position can be configured at runtime. It
> is intended to model the physical memory aliasing with minimal effort.
> 
> On JH7110, it aligns perfectly with the HW, as the aliased UC region
> happens to be offsetted by 2^34. Thus, configuring the XPbmtUC with
> bit=32 (PPN is shifted by 2) is all that needs to be done.
> 
> On EIC770X, the aliased UC region is put to a awkward offset, and given
> there can be 2 NUMA node (dual-die) with 2 separate memory regions and
> their UC alias counterpart, we instead ask the firmware to provide a
> thin-layer hypervisor to re-arrange the memory map. The XPbmtUC will be
> enabled with bit=38, thus map all UC pages to 2^40 (the upper-half of
> 2^41), and the underlaying hypervisor will re-map the 2^40+ addresses
> to the appropriate UC alias regions. (See description in PATCH 1/6)
> 
> We chose bit 38 (PPN bit 40) to make the 2-stage translation efficient.
> Hypervisor can utilize Sv39x4 G-stage scheme, and map all pages as 1GB
> huge page, consuming only the first-level page table (16KB total), and
> several TLB entries. In practice, it's the firmware/bootloader that
> configures XPbmtUC through device-tree, based on firmware capabilities,
> and skip the enablement on stock firmware. This is tested on Hifive
> Premier P550 with the modified OpenSBI[2]. It runs the host Linux in VS
> mode, and provide the aforementioned remapping. The performance penalty
> (if not running KVM in Linux) is minimal, as the CPU is never switched
> to HS mode. A very slight, unavoidable, slow down is with the external
> interrupt delivery. Due to the lack of AIA in EIC770X, all device irq
> now needs to trap to M mode first, before forwarding to VS mode. The
> overhead of running KVM in such setup is yet unknown, and may well be
> noticeable, as all HS-qualified instructions will trap to M mode, and
> there's also the extra cost of flushing G/VS-stage TLBs. I'm analyzing
> it in parallel.
> 
> I'm aware there's an ongoing series that Samuel sent for physical
> memory aliases. I haven't been following too closely, but if you're
> worried about it touching to many areas, I hope my series can shed some
> light on the problem. My change is very minimal and local, also fairly
> easy to remove if we later decide deprecating it down the road.
> 
> [1] https://github.com/starfive-tech/JH7100_Docs/blob/main/JH7100%20Cache%20Coherence%20V1.0.pdf
> [2] https://github.com/ganboing/opensbi/tree/eic77x-vspt-physalias-wip

For those following along at home, Samuel's series is:
https://lore.kernel.org/all/20251113014656.2605447-20-samuel.holland@sifive.com/

I've been meaning to try it, but never conjured up the time to dig into
it...

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]