Re: [PATCH 0/4 v3] cxl/core: Enable Region creation on x86 with Low Mem Hole

Robert Richter
2025-03-21T10:34:43Z

Fabio,

On 14.03.25 12:36:29, Fabio M. De Francesco wrote:
> The CXL Fixed Memory Window Structure (CFMWS) describes zero or more Host
> Physical Address (HPA) windows that are associated with each CXL Host
> Bridge. Each window represents a contiguous HPA that may be interleaved
> with one or more targets (CXL v3.1 - 9.18.1.3).
>
> The Low Memory Hole (LMH) of x86 is a range of addresses of physical low
> memory to which systems cannot send transactions. On those systems, BIOS
> publishes CFMWS which communicate the active System Physical Address (SPA)
> ranges that map to a subset of the Host Physical Address (HPA) ranges. The
> SPA range trims out the hole, and capacity in the endpoint is lost with no
> SPA to map to CXL HPA in that hole.
>
> In the early stages of CXL Regions construction and attach on platforms
> with Low Memory Holes, the driver fails and returns an error because it
> expects that the CXL Endpoint Decoder range is a subset of the Root
> Decoder's (SPA >= HPA). On x86 with LMH's, it happens that SPA < HPA.
>
> Therefore, detect x86 Low Memory Holes, match CXL Root and Endpoint
> Decoders or already made CXL Regions and Decoders to allow the
> construction of new CXL Regions and the attachment of Endpoint Decoders,
> even if SPA < HPA. If needed because of LMH's, adjust the Endpoint Decoder
> range end to match Root Decoder's.
>
> - Patch 1/4 changes the calling conventions of three match_*_by_range()
> helpers in preparation of 3/4.
> - Patch 2/4 Introduces helpers to detect LMH's and also one to adjust
> the HPA range end for CXL Regions construction.
> - Patch 3/4 enables CXL Regions construction and Endpoint Decoders
> attachment by matching Root Decoders or Regions with Endpoint
> Decoders, adjusting Endpoint Decoders HPA range end, and relaxing
> constraints while Endpoints decoders' attachment.
> - Patch 4/4 simulates a LMH for the CXL tests on patched CXL driver.
>
> Many thanks to Alison, Dan, and Ira for their help and for their reviews
> of my RFC on Intel's internal ML.
>
> Commenting on v1, Alison wrote a couple of observations on what users
> will see. I suggest anyone interested to see how this series affect
> users to take a look at her observations.[0] Thank you!
>
> Changes for v3:
>
> Re-base the series on cxl/next.
>
> 1/4 - 2/4:
> Constify local variables.
> 3/4:
> Call arch_match_region() from region_res_match_cxl_range().
> 4/4:
> arch_match_region() - Check that region end is under start + 4G;
> arch_match_spa() - Check that SPA range start is cfmws_range_start.

I have sent comments for version 1 and suggested a simpler approach
for this to implement. My comments haven't been addressed yet, but we
need better isolation to reduce interference with other platforms and
archs. Please take a look again.

Many thanks,

-Robert