Re: Excessive TLB flush ranges

Uladzislau Rezki
2023-05-15T18:17:52Z

On Mon, May 15, 2023 at 06:43:40PM +0200, Thomas Gleixner wrote:
> Folks!
>
> We're observing massive latencies and slowdowns on ARM32 machines due to
> excessive TLB flush ranges.
>
> Those can be observed when tearing down a process, which has a seccomp
> BPF filter installed. ARM32 uses the vmalloc area for module space.
>
> bpf_prog_free_deferred()
> vfree()
> _vm_unmap_aliases()
> collect_per_cpu_vmap_blocks: start:0x95c8d000 end:0x95c8e000 size:0x1000
> __purge_vmap_area_lazy(start:0x95c8d000, end:0x95c8e000)
>
> va_start:0xf08a1000 va_end:0xf08a5000 size:0x00004000 gap:0x5ac13000 (371731 pages)
> va_start:0xf08a5000 va_end:0xf08a9000 size:0x00004000 gap:0x00000000 ( 0 pages)
> va_start:0xf08a9000 va_end:0xf08ad000 size:0x00004000 gap:0x00000000 ( 0 pages)
> va_start:0xf08ad000 va_end:0xf08b1000 size:0x00004000 gap:0x00000000 ( 0 pages)
> va_start:0xf08b3000 va_end:0xf08b7000 size:0x00004000 gap:0x00002000 ( 2 pages)
> va_start:0xf08b7000 va_end:0xf08bb000 size:0x00004000 gap:0x00000000 ( 0 pages)
> va_start:0xf08bb000 va_end:0xf08bf000 size:0x00004000 gap:0x00000000 ( 0 pages)
> va_start:0xf0a15000 va_end:0xf0a17000 size:0x00002000 gap:0x00156000 ( 342 pages)
>
> flush_tlb_kernel_range(start:0x95c8d000, end:0xf0a17000)
>
> Does 372106 flush operations where only 31 are useful
>
> So for all architectures which lack a mechanism to do a full TLB flush
> in flush_tlb_kernel_range() this takes ages (4-8ms) and slows down
> realtime processes on the other CPUs by a factor of two and larger.
>
> So while ARM32, CSKY, NIOS, PPC (some variants), _should_ arguably have
> a fallback to tlb_flush_all() when the range is too large, there is
> another issue. I've seen a couple of instances where _vm_unmap_aliases()
> collects one page and the actual va list has only 2 pages, which might
> be eventually worth to flush one by one.
>
> I'm not sure whether that's worth it as checking for those gaps might be
> too expensive for the case where a large number of va entries needs to
> be flushed.
>
> We'll experiment with a tlb_flush_all() fallback on that ARM32 system in
> the next days and see how that works out.
>
For systems which lack a full TLB flush and to flush a long range is
a problem(it takes time), probably we can flush VA one by one. Because
currently we calculate a flush range [min:max] and that range includes
the space that might not be mapped at all. Like below:

VA_1 VA_2
|....|-------------------------|............|
10 12 60 68

. mapped;
- not mapped.

so we flush from 10 until 68. Instead, probably we can do a flush of VA_1
range and VA_2 range. On modern systems with many CPUs, it could be a big
slow down.

Just some thoughts.

--
Uladzislau Rezki