Re: [PATCH v2 0/5] Initial ESWIN/EIC7700 support

Bo Gan
2025-11-18T07:05:31Z

Hi Anup,

On 11/17/25 07:09, Anup Patel wrote:
> Hi Bo Gan,
>
> On Mon, Nov 17, 2025 at 3:00 PM Bo Gan <ganboing@gmail.com> wrote:
>>
>> Hi Anup,
>>
>> Thanks for the prompt reply. See inline.
>>
>> On 11/17/25 00:04, Anup Patel wrote:
>>> On Mon, Nov 17, 2025 at 11:20 AM Bo Gan <ganboing@gmail.com> wrote:
>>>>
>>>> EIC7700 is the SoC used in HiFive P550 and Milk-V Megrez. This SoC is
>>>> currently one of the only off-the-shelf board/chips that support H
>>>> extension, although it's v0.6.1. It also supports pre-ratified N-trace.
>>>> Add support for it so people can benefit from latest OpenSBI features.
>>>>
>>>> The device-tree of HiFive P550 has been upstreamed to Linux:
>>>> https://lore.kernel.org/all/20250825132427.1618089-1-pinkesh.vaghela@einfochips.com/
>>>> However U-boot is not, and there are bugs in vendor U-boot device-tree,
>>>> and also inconsistencies between the two. Thus, this patch is coded with
>>>> the upstreamed device-tree as the reference, but tested with the patched
>>>> vendor U-boot device tree as `FW_FDT_PATH`. The patched vendor U-boot is
>>>> hosted here: https://github.com/ganboing/u-boot-eic7x/tree/eic7x-dt-fix
>>>> Refer to PATCH 5/5 for the instructions on building the firmware blob
>>>> and launch it through UART boot.
>>>>
>>>> The major complication of this chip is that it requires certain memory
>>>> regions to be blocked with PMP entries to prevent speculative execution
>>>> or HW prefetcher from touching them to avoid bus errors. Also due to the
>>>> fact that this SoC handles cache incoherent DMA by mapping memory twice,
>>>> one as cached, and the other as uncached, we also need an extra PMP to
>>>> protect the OpenSBI in the uncached portion in address space. Following
>>>> changes are made to lib/ and firmware/ to make it possible:
>>>>
>>>> - Allow platform to override pmp_(un)configure
>>>> - Add helper function for settings PMP TOR
>>>> - Add helper function to check if memregions are disjoint
>>>> - Introduce FIRMWARE_PACKED_RXRW for disabling power-of-2 RW split
>>>
>>> PMP TOR entries are not going to fly. Also, adding separate config
>>> option FIRMWARE_PACKED_RXRW and separate defconfig for
>>> EIC770X means distros can't use the same fw_dyanmic.bin on
>>> EIC770X and other platforms.
>>>
>>> If PMP TOR is being used only to save one PMP entry then there
>>> are better ways to achieve this.
>>>
>>
>> Okay. I think I've found a way to use NAPOT to achieve the same effect.
>>
>> Memory Port (die 0)
>> ├─ 0x00_8000_0000 - 0x10_8000_0000 die 0 memory (cached)
>> └─ 0x10_8000_0000 - 0x80_0000_0000 (hole)
>>
>> Memory Port (die 1)
>> ├─ 0x00_8000_0000 - 0x20_0000_0000 (hole)
>> ├─ 0x20_0000_0000 - 0x30_0000_0000 die 1 memory (cached)
>> └─ 0x30_0000_0000 - 0x80_0000_0000 (hole)
>>
>> By default, for both die 0/1:
>>
>> NAPOT[0] ...... MRWX,SU() ____ OpenSBI in cached mem
>> NAPOT[1] ...... MRWX,SU() ____ OpenSBI in uncached mem
>> NAPOT[2-5] <free>
>> NAPOT[6] [0x0, 0x80_0000_0000) L___
>> NAPOT[7] ...... <reserved>
>>
>> For die 0 in root domain:
>>
>> NAPOT[0] ...... M(RWX)SU() ____ OpenSBI in cached mem
>> NAPOT[1] ...... M(RWX)SU() ____ OpenSBI in uncached mem
>> NAPOT[2] [0x200_0000, 0x201_0000) ____ M(RW)SU() CLINT
>> NAPOT[3] [0x2000_0000, 0x4000_0000) L___ for die 1 p550
>> NAPOT[4] [0x0, 0x10_0000_0000) _RWX M() SU(RWX)
>> NAPOT[5] [0x10_0000_0000, 0x10_8000_0000) _RWX M() SU(RWX)
>> NAPOT[6] [0x0, 0x80_0000_0000) L___
>> NAPOT[7] [0x0, 0xffffffffffffffff) _RWX M() SU(RWX)
>>
>> Similarly, for die 1, use:
>>
>>
>> NAPOT[0] ...... M(RWX)SU() ____ OpenSBI in cached mem
>> NAPOT[1] ...... M(RWX)SU() ____ OpenSBI in uncached mem
>> NAPOT[2] [0x400_0000, 0x401_0000) ____ M(RW)SU() CLINT
>> NAPOT[3] [0x0, 0x2000_0000) L___ for die 0 p550
>> NAPOT[4] [0x0, 0x8000_0000) _RWX M() SU(RWX)
>> NAPOT[5] [0x20_0000_0000, 0x30_8000_0000) _RWX M() SU(RWX)
>> NAPOT[6] [0x0, 0x80_0000_0000) L___
>> NAPOT[7] [0x0, 0xffffffffffffffff) _RWX M() SU(RWX)
>>
>> It'll work also for non-root domain harts, and actually better if it
>> doesn't contain a "catch all" region (order == __riscv_xlen). However,
>> compared to the TOR approach, it consumes 1 more PMP on die 0 for root
>> domain or any domain with a "catch all" region, and we've used up all
>> entries. There's no room for more, so we can't do something like a
>> trusted/untrusted-domain scenario as documented in domain_support.md,
>> because we can't block the access of tdomain memory region in udomain.
>> I'm not sure if this is an issue at this point. Perhaps it could be
>> when someone tries out TEE? I don't have the answer today.
>
> There is one more approach which can save PMP entries for
> you where you can create a large region in platform early init
> covering multiple regions added by OpenSBI drivers.
>
> For example, you can create a single region for all M-mode
> only MMIO devices if platform address space allows you.
>

Unfortunately, the address space layout makes it impossible. we need to
block CLINT from S/U, but allow the Trace components, PLIC and L3 Cache
Controller from S/U.

Die 0 P550 internal region (quoting from datasheet):
R:Read, W:Write, X:Exec, I:Inst Cacheable, D:Data Cacheable, A:Atomics

0x00000000 - 0x00000FFF Debug
0x00001000 - 0x00003FFF RWX A Error Device
0x00004000 - 0x00004FFF RW A Test Indicator
0x00018000 - 0x00018FFF RWX A Trace Funnel
0x00100000 - 0x00100FFF RWX A Trace Encoder 0
0x00101000 - 0x00101FFF RWX A Trace Encoder 1
0x00102000 - 0x00102FFF RWX A Trace Encoder 2
0x00103000 - 0x00103FFF RWX A Trace Encoder 3
0x00104000 - 0x00107FFF RWX A Hart 0 Private L2 Cache
0x00108000 - 0x0010BFFF RWX A Hart 1 Private L2 Cache
0x0010C000 - 0x0010FFFF RWX A Hart 2 Private L2 Cache
0x00110000 - 0x00113FFF RWX A Hart 3 Private L2 Cache
0x00200000 - 0x00200FFF RW A Bus blocker TL64D2D_Out
0x00202000 - 0x00202FFF RW A Bus blocker TL256D2D_Out
0x00204000 - 0x00204FFF RW A Bus blocker TL256D2D_In
0x01700000 - 0x01700FFF RWX A Hart 0 Bus-Error Unit
0x01701000 - 0x01701FFF RWX A Hart 1 Bus-Error Unit
0x01702000 - 0x01702FFF RWX A Hart 2 Bus-Error Unit
0x01703000 - 0x01703FFF RWX A Hart 3 Bus-Error Unit
0x02000000 - 0x0200FFFF RWX A CLINT
0x02010000 - 0x02013FFF RWX A L3 Cache Controller
0x08000000 - 0x083FFFFF RWX A L3 LIM
0x0C000000 - 0x0FFFFFFF RWX A PLIC
0x10010000 - 0x10010FFF RWX A Burst Bundler on Front Port
0x10030000 - 0x10033FFF RWX A TL-UH to TL-C Adapter
0x1A000000 - 0x1A3FFFFF RWXIDA L3 Zero Device

I re-read the EIC7700 datasheet, and I finally got a theory on the reason
behind bus errors. There are 3 regions having data cacheable attribute in
the chip hard-coded PMA:

- [0x00_1a00_0000, 0x00_1a40_0000) -- L3 zero device for die 0
- [0x00_3a00_0000, 0x00_3a40_0000) -- L3 zero device for die 1
- [0x00_8000_0000, 0x80_0000_0000) -- entire memory port

They can be speculatively accessed or HW prefetched, so if there's only
1 die and there're holes in these regions, you'll get bus error from
time to time if the core happens to touch it. Thus, whatever belongs to
the other (remote) die needs to be blocked. Hopefully this only applies
for data cacheable regions, otherwise, I'd say it's horribly broken. I
already dislike the fact that we have to use PMP to do it, but it's what
it is. In the PMP configuration above, I chose to block die 1 L3 zero
device from die 0 by disallowing the entire internal port for die 1. Or
I can do it with finer granularity. Either way, it can't be omitted.

What I can do is configure the PMPs with NAPOTs as you suggested. It'll
be like the detailed list above. This should cover 99% of use cases. For
users with a udomain-tdomain like setup, they can still try it, but put
a note in the comment and say due to HW limitation, it's experimental
only and you can't properly protect tdomain from udomain.

Does it sound good?

Bo

> Regards,
> Anup