Re: PROBLEM: Probabilistic segfault on AMD hardware with INVLPGB

[Rik van Riel <riel@surriel.com>]

On Tue, 7 Jul 2026 13:18:30 +0100
Matt Fleming <matt@readmodwrite.com> wrote:

> Hi Rik and Henrik,
> 
> I've got a small C reproducer that triggers this within the first round
> on Cloudflare's AMD Turin machines:
> 
>   https://gist.github.com/mfleming/ca26ad3f8d65a12d23d62fb176480fc1
> 
> Build:
> 
>   gcc -O2 -Wall -Wextra -pthread -static -o repro-invlpgb repro-invlpgb.c
> 
> Run:
> 
>   ./repro-invlpgb --batch --rounds 20 --jobs 32 -d 5 -w 8 -m 2 -s 512 -q
> 
> The mutator side holds a pthread rwlock write lock while doing
> munmap() + mmap(MAP_FIXED) on the same VA and filling the range with
> {cookie, slot, generation, offset} markers. Reader threads hold the read
> lock while checking those markers.
> 
> The failure mode is usually a CORRUPTION line rather than a direct
> SIGSEGV: a read from one virtual offset returns a valid marker for a
> different offset. That looks consistent with a stale or wrong translation
> after VA reuse.

I poked at the reproducer yesterday, and it seems like the pages
in the buffer get reversed between uses, so in one pass they will
be ABCDEFGH while the next run they'll be HGFEDCBA.

This showed up in the observed corruption pattern,
where the second slot in run N+1 would see the data
from the second to last slot in run N, etc.

All the observed corruptions end up seeing data from
the last run, from the flipped location (Nth slot,
where the page used to be in the Nth-from-last slot).

The change below seems to close off the race in
practice with this reproducer, but since I do not
know what really goes on inside the CPU, there are
likely better ways to fix it.

AMD friends, do we need to gather any additional
data for the hardware and/or firmware people to
take a look at this, or is there anything else
needed?

diff --git a/arch/x86/include/asm/tlb.h b/arch/x86/include/asm/tlb.h
index 866ea78ba156..2d30426f9183 100644
--- a/arch/x86/include/asm/tlb.h
+++ b/arch/x86/include/asm/tlb.h
@@ -110,6 +110,13 @@ static inline void __tlbsync(void)
 
 	/* TLBSYNC: supported in binutils >= 0.36. */
 	asm volatile(".byte 0x0f, 0x01, 0xff" ::: "memory");
+
+	/*
+	 * TLBSYNC: we should not have to do this, but it seems to
+	 * mitigate the stale TLB visibility issue reported by
+	 * https://lore.kernel.org/lkml/CAAuFnRTyva1_3tsF3vrMBL+TLS1YL4EgUPT2c3O9k7A9hWUMnA@mail.gmail.com/
+	 */
+	asm volatile(".byte 0x0f, 0x01, 0xff" ::: "memory");
 }
 #else
 /* Some compilers (I'm looking at you clang!) simply can't do DCE */