Re: [PATCH RFC 0/3] cxl: Initial support for Back-Invalidate

[Jonathan Cameron <Jonathan.Cameron@huawei.com>]

On Mon, 11 Aug 2025 18:02:25 -0700
Davidlohr Bueso <dave@stgolabs.net> wrote:

> Hello,
> 
> The following is some initial plumbing to enabling HDM-DB in Linux. This model allows devices,
> specifically Type 2 and Type 3 devices, to expose their local memory to the host CPU in a
> coherent manner, In alignment with what was discussed at last year's LPC type2 support session,
> this series takes the type3 memory expander approach, which is more direct.
> 
> While this is an early RFC and I'm sure many thoughts, the next phase of this would be to
> integrate Bi with Alejandro's Type2 work as well as with Jonathan's Cache Coherency subsystem
> series (aka memregion inv)... this might be a good topic for the upcoming LPC's devmem session:
> 	https://lore.kernel.org/linux-cxl/20250624141355.269056-1-alejandro.lucero-palau@amd.com
> 	https://lore.kernel.org/linux-cxl/20250624154805.66985-1-Jonathan.Cameron@huawei.com

Hi Davidlohr,

For type 3 devices, when are HDM-DB / BI flows this useful?
I think it's worth calling those cases out.

IIRC 
- Shared coherent memory
- Media operations (8.2.10.9.5.3 in 3.2 spec)
   So the range based sanitize and zero stuff.

But not I think on DCD extent add or remove.  Which is kind of fair enough as
the device would have to track prefetches that occurred to addresses it wasn't
backing at the time. 

So I'm not immediately sure how it will combine with explicit cache coherency management.
So far we haven't hooked anything up for non BI enabled shared memory.

For type 2, do we have a BI capable example?

> 
> o Patch 1 adds the BI cachemem register discovery along with two interfaces around cxlds to allow
> the setup and deallocations of BI-IDs. The idea is for type3 memdevs and future type2 devices
> to make use of cxlds->bi when committing HDM decoders, such that different device coherence models
> can be differentiated as:
> 
> 	type2 hdm-db: cxlds->type == CXL_DEVTYPE_DEVMEM && cxlds->bi == true
> 	type2 hdm-d:  cxlds->type == CXL_DEVTYPE_DEVMEM && cxlds->bi == false
> 	type3 hdm-h:  cxlds->type == CXL_DEVTYPE_CLASSMEM && cxlds->bi == false
> 	type3 hdm-db: cxlds->type == CXL_DEVTYPE_CLASSMEM && cxlds->bi == true
> 	
> Because ->bi becoming true does not depend on auto-committing upon HDM decoder port/enumeration
> (port driver), for now this is set as unsupported and will error out when initializing the HDM
> decoder that has its BI bit set.
> 
> o Patch 2 renames/updates some of the CXL Window coherency restrictions. This should be picked
> up regardless as the spec has been updated already.
> 
> o Patch 3 deals with the HDM decoder programming changes around whether or not to set the
> BI bit. Based on the model above, decoder target types are straightforward: DEVMEM or HOSTONLY
> for type2 and regular type3, but for type3 HDM-DB, this is not as clear, for which this patch
> will 1) rely on the HDM capability for supporting coherence models, and 2) allow, when possible,
> to change it by the user when configuring the BI-capable HDM decoder. It gives the user sysfs
> tools to create BI-enabled memory regions (see testing below).