Re: [PATCH v4 07/10] cxl: add host cache flush and multi-function reset

[Jonathan Cameron <jonathan.cameron@huawei.com>]

On Thu, 22 Jan 2026 19:24:49 +0000
Vikram Sethi <vsethi@nvidia.com> wrote:

> Hi Jonathan,
> Happy new year!
> 
> > From: Jonathan Cameron <jonathan.cameron@huawei.com>
> > Date: Thursday, January 22, 2026 4:31 AM
> > To: Srirangan Madhavan <smadhavan@nvidia.com>
> > Cc: dave@stgolabs.net <dave@stgolabs.net>, dave.jiang@intel.com <dave.jiang@intel.com>, alison.schofield@intel.com <alison.schofield@intel.com>, vishal.l.verma@intel.com <vishal.l.verma@intel.com>, ira.weiny@intel.com <ira.weiny@intel.com>, dan.j.williams@intel.com <dan.j.williams@intel.com>, bhelgaas@google.com <bhelgaas@google.com>, ming.li@zohomail.com <ming.li@zohomail.com>, rrichter@amd.com <rrichter@amd.com>, Smita.KoralahalliChannabasappa@amd.com <Smita.KoralahalliChannabasappa@amd.com>, huaisheng.ye@intel.com <huaisheng.ye@intel.com>, linux-cxl@vger.kernel.org <linux-cxl@vger.kernel.org>, linux-pci@vger.kernel.org <linux-pci@vger.kernel.org>, Vishal Aslot <vaslot@nvidia.com>, Vikram Sethi <vsethi@nvidia.com>, Shanker Donthineni <sdonthineni@nvidia.com>, Vidya Sagar <vidyas@nvidia.com>, Matt Ochs <mochs@nvidia.com>, Jason Sequeira <jsequeira@nvidia.com>
> > Subject: Re: [PATCH v4 07/10] cxl: add host cache flush and multi-function reset  
> 
> 
> >Prefetching should (hopefully) not make any dirty lines. Hopefully
> >no one does clean writebacks (and there is a way to check if they
> >do in the ACPI tables).  You need to flush again after to force out
> >that stale stuff though before any demand fetches occur.  
> 
> I am aware of some systems which do clean writebacks to device memory. 
> What exact ACPI table has this discovery?

CEDT (so in the CXL spec, it's there in 3.2 and 4, I'm too lazy to open
earlier specs) Specifically the CXL System Description Structure (CSDS)
System Capabilities bit[1]

No Clean Writeback: Specifies the clean writeback beahvior of hte host
- 0 = The host may or may not generate clean writebacks.
- 1 = The host guarantees to never generate clean writeback at the host's
      cacheline granularity.


> 
> >I need to think a bit more on this as there are some scary
> >comments in the spec on CXL.reset such as all CXL.mem reads
> >are dropped (timeout fun). Mind you a device is permitted
> >to do that anyway before cxl.mem is enabled, so hopefully n
> >one times out if a prefetcher hits the device before that's on..  
> 
> Yes, any sensible coherent memory device reset will have to include first offlining the memory such that there is no demand or speculative fetch possible, else you get to deal with CXL error isolation fun.
> 
> >From a kernel point of view SMCCC needs to be just one option as a
> >bunch of hardware (I'll only point at ours as not sure what else is
> >public) provide MMIO accessible agents to do this stuff and going via
> >EL3 to talk to an engine the kernel can poke directly is silly.  
> 
> We have a similar custom engine for efficient flushing, but the interface is not available to the kernel, so the SMCCC is preferred for our implementation. Like you say, the AML wrapper for SMCCC is another option, although we also have some device tree based systems where the efficient custom flushing is desirable. 

Great.  So all that work they did before dropping the PSCI call
was worth doing.  I'll keep an eye open for the new spec.

> 
> >I'm not against the PSCI thing coming back though if someone needs it.
> >Preferably without the CPU rendezvous stuff though -> or Linux can
> >just reject anyone who does that.  
> 
> Agreed. I think Souvik also agreed offline that rendezvous is not needed for the SMC based cache flush.

Great.

Thanks,

Jonathan

> 
> Vikram
>