Re: [PATCH v3] cxl: docs/driver-api/conventions resolve conflicts btw CFMWS, LMH, ED

[Gregory Price <gourry@gourry.net>]

On Fri, Jul 04, 2025 at 12:05:33PM +0200, Fabio M. De Francesco wrote:
> On Thursday, July 3, 2025 9:40:21 PM Central European Summer Time Gregory Price wrote:
> > On Tue, Jul 01, 2025 at 08:23:57AM -0700, Dave Jiang wrote:
> > > 
> > > 
> > > On 6/23/25 12:19 PM, Gregory Price wrote:
> > > > On Mon, Jun 23, 2025 at 05:29:02PM +0200, Fabio M. De Francesco wrote:
> > > >> Add documentation on how to resolve conflicts between CXL Fixed Memory
> > > >> Windows, Platform Memory Holes, and Endpoint Decoders.
> > > >>
> > > >> Signed-off-by: Fabio M. De Francesco <fabio.m.de.francesco@linux.intel.com>
> > > > 
> > > > I won't block a doc update on a suggestion so
> > > > 
> > > > Reviewed-by: Gregory Price <gourry@gourry.net>
> > > > 
> > > >> +Platform Firmware (BIOS) might reserve part of physical addresses below
> > > >> +4 GB (e.g., the Low Memory Hole that describes PCIe memory space for MMIO
> > > >> +or a requirement for the greater than 8 way interleave CXL regions starting
> > > >> +at address 0). In that case the Window Size value cannot be anymore
> > > >> +constrained to the NIW * 256 MB above-mentioned rule.
> > > > 
> > > > It might be nice to have a diagram that explains this visually, as it's
> > > > difficult for me to understand the implications through words alone...
> > > 
> > > +1 on request for diagram to explain. We should try to document this issue as clearly as possible. Thank you.
> > >
> > 
> > At the very least, it would be nice to have an explicitly example that
> > explains the expected cfmws/decoder configurations that are valid but
> > "technically" violate the spec
> > 
> > I *think* this basically boils down to "CFMWS size is not aligned, but
> > all the decoders it targets are"?  If I understand this correctly?
> > 
> Yes, this is the intended meaning. 
> 
> E.g, two windows, 384 GB total memory, LMH at 2 GB:
> 
> Window | CFMWS Base | CFMWS Size | HDM Decoder Base | HDM Decoder Size | Ways | Granularity
>   0    |   0 GB     |     2 GB   |      0 GB        |       3 GB       |  12  |    256
>   1    |   4 GB     |   380 GB   |      0 GB        |     380 GB       |  12  |    256


In this example, does the root decoder have 0/2 and the host bridge
decoder have 0/3?  Because root decoders objects are created by reading
the CFMWS Base/Size - while host bridge decoder objects are created by
reading the host bridge DVSECS.

Or does the linux code read CFMWS Base/Size and create a root decoder
with 0/3 because "that's what it should have"?  etc

I think you need to describe what the expected behavior is for what linux
will produce in terms of the decoder objects given the above.

> 
> 12 *256MB = 3GB, the Top of Low memory below 4GB is set as 2GB in the platform for MMIO Low Mem Hole (2-4GB). Only 2GB will be potentially available for user, but currently the CXL driver expects a CFMWS range's size to be greater or equal to the HDM Decoder's, so it  doesn't match them and we lose those 2GB described by CFMWS0. If we allow the matching, we make those 2GB available to users.
> 
> Is a table like the one above good enough to make this subject clearer?
> 
> Thanks,
> 
> Fabio
> >
> > > > 
> > > > which is likely why the conflict exists in the first place :]
> > > > 
> > > > ~Gregory
> > > 
> > 
> 
> 
> 
>