[PATCH] x86, memregion: Avoid big hammer from cpu_cache_invalidate_memregion()

[PATCH] x86, memregion: Avoid big hammer from cpu_cache_invalidate_memregion()

[Davidlohr Bueso]

The reason for getting away with wbinvd_on_all_cpus() was originally
that the users at the time were a one time at boot occurrence, so it
mitigated a lot of the effects of the system-wide disruptiveness and
cache destruction. This has now changed with users such as provisioning
memory through CXL Dynamic Capacity Devices.

Lets instead use clflushopt and only invalidate the range in question.
The performance of course scales poorly with the region size but is
ultimately less invasive.

Signed-off-by: Davidlohr Bueso <dave@stgolabs.net>
---
 arch/x86/mm/pat/set_memory.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/arch/x86/mm/pat/set_memory.c b/arch/x86/mm/pat/set_memory.c
index 6c6eb486f7a6..4a1c4f6bec17 100644
--- a/arch/x86/mm/pat/set_memory.c
+++ b/arch/x86/mm/pat/set_memory.c
@@ -372,6 +372,19 @@ int cpu_cache_invalidate_memregion(phys_addr_t start, size_t len)
 {
 	if (WARN_ON_ONCE(!cpu_cache_has_invalidate_memregion()))
 		return -ENXIO;
+
+	if (static_cpu_has(X86_FEATURE_CLFLUSHOPT)) {
+		void *vaddr = memremap(start, len, MEMREMAP_WB);
+
+		if (!vaddr)
+			goto fallback;
+
+		clflush_cache_range(vaddr, len);
+		memunmap(vaddr);
+
+		return 0;
+	}
+fallback:
 	wbinvd_on_all_cpus();
 	return 0;
 }
-- 
2.39.5

Re: [PATCH] x86, memregion: Avoid big hammer from cpu_cache_invalidate_memregion()

[dan.j.williams]

Hi Davidlohr,

Davidlohr Bueso wrote:
> The reason for getting away with wbinvd_on_all_cpus() was originally
> that the users at the time were a one time at boot occurrence, so it
> mitigated a lot of the effects of the system-wide disruptiveness and
> cache destruction. This has now changed with users such as provisioning
> memory through CXL Dynamic Capacity Devices.

Except the kernel does not support CXL Dynamic Capacity yet.

> Lets instead use clflushopt and only invalidate the range in question.
> The performance of course scales poorly with the region size but is
> ultimately less invasive.
> 
> Signed-off-by: Davidlohr Bueso <dave@stgolabs.net>
> ---
>  arch/x86/mm/pat/set_memory.c | 13 +++++++++++++
>  1 file changed, 13 insertions(+)
> 
> diff --git a/arch/x86/mm/pat/set_memory.c b/arch/x86/mm/pat/set_memory.c
> index 6c6eb486f7a6..4a1c4f6bec17 100644
> --- a/arch/x86/mm/pat/set_memory.c
> +++ b/arch/x86/mm/pat/set_memory.c
> @@ -372,6 +372,19 @@ int cpu_cache_invalidate_memregion(phys_addr_t start, size_t len)
>  {
>  	if (WARN_ON_ONCE(!cpu_cache_has_invalidate_memregion()))
>  		return -ENXIO;
> +
> +	if (static_cpu_has(X86_FEATURE_CLFLUSHOPT)) {
> +		void *vaddr = memremap(start, len, MEMREMAP_WB);

How much of the cost is in the mapping management?

I was not expecting that virtual address based flushing would be
reasonable to call from all the places where
cpu_cache_invalidate_memregion() is called to do physical flushing. If
the concern is increased frequency of flushing due to dynamic capacity,
and that dynamic capacity updates have a chance to be finer grained,
then I would then expect some kind of tie into memory hotplug that can
invalidate cache using the direct map.