Re: [PATCH v4 07/10] cxl: add host cache flush and multi-function reset

Vikram Sethi
2026-01-21T22:19:37Z

Hi Jonathan,

________________________________________
From: Jonathan Cameron <jonathan.cameron@huawei.com>
Sent: Wednesday, January 21, 2026 5:20 AM
To: Srirangan Madhavan
Cc: dave@stgolabs.net; dave.jiang@intel.com; alison.schofield@intel.com; vishal.l.verma@intel.com; ira.weiny@intel.com; dan.j.williams@intel.com; bhelgaas@google.com; ming.li@zohomail.com; rrichter@amd.com; Smita.KoralahalliChannabasappa@amd.com; huaisheng.ye@intel.com; linux-cxl@vger.kernel.org; linux-pci@vger.kernel.org; Vishal Aslot; Vikram Sethi; Shanker Donthineni; Vidya Sagar; Matt Ochs; Jason Sequeira
Subject: Re: [PATCH v4 07/10] cxl: add host cache flush and multi-function reset

On Tue, 20 Jan 2026 22:26:07 +0000
smadhavan@nvidia.com wrote:

>> From: Srirangan Madhavan <smadhavan@nvidia.com>
>>
>> Flush host CPU caches for mapped HDM ranges after teardown and prepare
>> sibling Type 2 functions on multi-function devices. The host cache
>> maintenance uses wbinvd_on_all_cpus() on x86 and VA-based PoC clean+

>That's not sufficient in general on arm64. It might flush far enough
>or there might be buffers beyond the point of coherence that are not
>flushed.

>Until there are clarifications from Arm we need something like the
>agents in drivers/cache to do equivalent of wbinvd_on_all_cpus()
>(be it on a PA range to making a tiny bit less horrible)

>Or we need an opt in list for platforms where a flush to PoC is enough.

>Needs to some sort of arch_ call as well, not hidden in the cxl driver.

Yes, and in fact it cannot be a cache flush by VA while PTE mappings to the memory are valid, as the core can prefetch the lines back right away, and later evictions post device reset can cause device errors as the device snoop filter isn’t aware of host having any lines.

The only way to do this correctly is via a SMC call to Arm Trusted Firmware to handle in a SOC specific way (after the memory has been offlined and removed from the PTEs), since there is no architectural way to flush by PA (set/way isn’t usable in the kernel). There was a SMC call defined for this in the PSCI spec 2 years ago, but was removed for some reason. I had a discussion with some ARM folks maintaining the PSCI and SMCCC specifications and the direction was to bring the SMC call back in SMCCC specification. Like you say, that can be a separate patch series. Adding Souvik for SMCCC specification details once it is available.

Thanks