Re: [PATCH v2 0/8] Cache coherency management subsystem

["H. Peter Anvin" <hpa@zytor.com>]

On July 10, 2025 12:11:13 PM PDT, dan.j.williams@intel.com wrote:
>H. Peter Anvin wrote:
>[..]
>> >> > In the near term though, current CXL platforms that do not support
>> >> > device-initiated-invalidate still need coarse cache management for that
>> >> > original infrequent provisioning events. Folks that want to go further
>> >> > and attempt frequent DCD events with WBINVD get to keep all the pieces.
>> >> 
>> >> I would strongly prefer those pieces to include WARNs and or worse.
>> >
>> >That is fair. It is not productive for the CXL subsystem to sit back and
>> >hope that people notice the destructive side-effects of wbinvd and hope
>> >that leads to device changes.
>> >
>> >This discussion has me reconsidering that yes, it would indeed be better
>> >to clflushopt loop over potentially terabytes on all CPUs. That should
>> >only be suffered rarely for the provisioning case, and for the DCD case
>> >the potential add/remove events should be more manageable.
>> >
>> >drm already has drm_clflush_pages() for bulk cache management, CXL
>> >should just align on that approach.
>> 
>> Let's not be flippant; looping over terabytes could take *hours*. But those are hours during which the system is alive, and only one CPU needs to be looping.
>
>Do not all CPUs need to perform the invalidation for L1 copies of the
>line?
>
>Not trying to be flippant, but if wbinvd is only a one-shot per Peter's
>proposed policy and the system experiences another CXL reconfiguration
>event, then looping is the only option or fail the memory plug event.
>
>> The other question is: what happens if memory is unplugged and then a
>> cache line evicted? I'm guessing that existing memory hotplug
>> solutions simply drop the writeback, since the OS knows there is no
>> valid memory there, and so any cached data is inherently worthless.
>
>Right, the expectation is that unplug is always coordinated and that
>surprise unplug is unsupported / might lead to system instability.
>
>

CLFLUSH goes through the cache coherency protocol and is therefore system wide, which WBINVD is not.