Re: [PATCH] x86, memregion: Avoid big hammer from cpu_cache_invalidate_memregion()

[<dan.j.williams@intel.com>]

Hi Davidlohr,

Davidlohr Bueso wrote:
> The reason for getting away with wbinvd_on_all_cpus() was originally
> that the users at the time were a one time at boot occurrence, so it
> mitigated a lot of the effects of the system-wide disruptiveness and
> cache destruction. This has now changed with users such as provisioning
> memory through CXL Dynamic Capacity Devices.

Except the kernel does not support CXL Dynamic Capacity yet.

> Lets instead use clflushopt and only invalidate the range in question.
> The performance of course scales poorly with the region size but is
> ultimately less invasive.
> 
> Signed-off-by: Davidlohr Bueso <dave@stgolabs.net>
> ---
>  arch/x86/mm/pat/set_memory.c | 13 +++++++++++++
>  1 file changed, 13 insertions(+)
> 
> diff --git a/arch/x86/mm/pat/set_memory.c b/arch/x86/mm/pat/set_memory.c
> index 6c6eb486f7a6..4a1c4f6bec17 100644
> --- a/arch/x86/mm/pat/set_memory.c
> +++ b/arch/x86/mm/pat/set_memory.c
> @@ -372,6 +372,19 @@ int cpu_cache_invalidate_memregion(phys_addr_t start, size_t len)
>  {
>  	if (WARN_ON_ONCE(!cpu_cache_has_invalidate_memregion()))
>  		return -ENXIO;
> +
> +	if (static_cpu_has(X86_FEATURE_CLFLUSHOPT)) {
> +		void *vaddr = memremap(start, len, MEMREMAP_WB);

How much of the cost is in the mapping management?

I was not expecting that virtual address based flushing would be
reasonable to call from all the places where
cpu_cache_invalidate_memregion() is called to do physical flushing. If
the concern is increased frequency of flushing due to dynamic capacity,
and that dynamic capacity updates have a chance to be finer grained,
then I would then expect some kind of tie into memory hotplug that can
invalidate cache using the direct map.