[PATCH] x86: improve algorithm in clflush_cache_range

Ross Zwisler
2015-04-28T22:13:30Z

The current algorithm used in clflush_cache_range() can cause the last
cache line of the buffer to be flushed twice. Fix that algorithm so
that each cache line will only be flushed once.

Signed-off-by: Ross Zwisler <ross.zwisler@linux.intel.com>
Reported-by: H. Peter Anvin <hpa@zytor.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: x86@kernel.org
Cc: Borislav Petkov <bp@suse.de>
---
arch/x86/mm/pageattr.c | 13 ++++++-------
1 file changed, 6 insertions(+), 7 deletions(-)

diff --git a/arch/x86/mm/pageattr.c b/arch/x86/mm/pageattr.c
index 89af288ec674..338e507f95b8 100644
--- a/arch/x86/mm/pageattr.c
+++ b/arch/x86/mm/pageattr.c
@@ -129,16 +129,15 @@ within(unsigned long addr, unsigned long start, unsigned long end)
*/
void clflush_cache_range(void *vaddr, unsigned int size)
{
- void *vend = vaddr + size - 1;
+ unsigned long clflush_mask = boot_cpu_data.x86_clflush_size - 1;
+ char *vend = (char *)vaddr + size;
+ char *p;

mb();

- for (; vaddr < vend; vaddr += boot_cpu_data.x86_clflush_size)
- clflushopt(vaddr);
- /*
- * Flush any possible final partial cacheline:
- */
- clflushopt(vend);
+ for (p = (char *)((unsigned long)vaddr & ~clflush_mask);
+ p < vend; p += boot_cpu_data.x86_clflush_size)
+ clflushopt(p);

mb();
}
--
1.9.3