Re: Deep flush support for CXL pmem?

[Jonathan Cameron <jonathan.cameron@huawei.com>]

On Fri, 17 Oct 2025 10:25:26 -0400
Gregory Price <gourry@gourry.net> wrote:

> On Fri, Oct 17, 2025 at 02:11:16PM +0800, Huang, Ying wrote:
> > Hi, All,
> > 
> > When reading the documentation of PMDK (https://github.com/pmem/pmdk), I
> > found that deep flush is used for data loss recovery in addition to GPF.
> > With that, we can identify that a file isn't affected by the GPF failure
> > because it wasn't opened during the GPF failure.
> > 
> > However, IIUC, the Linux kernel doesn't support deep flush (via
> > nvdimm_flush()) for CXL pmem at least for now.  For NVDIMM, we have WPQ,
> > but it appears that we don't have that for CXL.  Is deep flush for CXL
> > defined in some spec?
> >  
> 
> Should note that this is relevant not just for pmem, but also for shared
> memory use cases.  Last I looked there's nothing defined / implemented.
> 
> When I reviewed the cxl.mem protocol about a year or so ago, I came away
> thinking a deep flush was going to be problematic for shared devices
> (multiple separate coherency domains / non-coherent memory).

Two things to deal with.  Getting it to the device.  For that what happens
is CPU architecture / host system architecture dependent. Not all (possibly
none) of the architectures have this well defined today - on some it
just works (cross your fingers that this is always the case). In some systems
using the persistent flushes gets it out on the bus whether or not the
CXL device is persistent.

There are ways around that, such as 'device' like entities on particular
systems that do this.  We have one in our Hydra Home Agent that is handled
by the cache maintenance 'subsystem' series (I'll send v4 of that early
next week). You can use that to guarantee caches are written back to
the device. We don't currently use it outside of capacity additions / removal
(DCD or address decoder programming routes).

The other thing is getting the device to actually write it to persistent
memory rather than relying on local or remote power to keep things up long
enough for writes to complete.

So I understand that there are gaps in what is guaranteed to do the job for both
deep flushes to persist memory and for shared memory use cases.

Jonathan

> 
> ~Gregory