Re: [PATCH v5 00/25] context_tracking,x86: Defer some IPIs until a user->kernel transition

[Dave Hansen <dave.hansen@intel.com>]

On 5/2/25 04:22, Peter Zijlstra wrote:
> On Wed, Apr 30, 2025 at 11:07:35AM -0700, Dave Hansen wrote:
> 
>> Both AMD and Intel have hardware to do it. ARM CPUs do it too, I think.
>> You can go buy the Intel hardware off the shelf today.
> To be fair, the Intel RAR thing is pretty horrific 🙁 Definitely
> sub-par compared to the AMD and ARM things.
> 
> Furthermore, the paper states it is a uarch feature for SPR with no
> guarantee future uarchs will get it (and to be fair, I'd prefer it if
> they didn't).

I don't think any of that is set in stone, fwiw. It should be entirely
possible to obtain a longer promise about its availability.

Or ask that AMD and Intel put their heads together in their fancy new
x86 advisory group and figure out a single way forward. If you're right
that RAR stinks and INVLPGB rocks, then it'll be an easy thing to advise.

> Furthermore, I suspect it will actually be slower than IPIs for anything
> with more than 64 logical CPUs due to reduced parallelism.

Maybe my brain is crusty and I need to go back and read the spec, but I
remember RAR using the normal old APIC programming that normal old TLB
flush IPIs use. So they have similar restrictions. If it's inefficient
to program a wide IPI, it's also inefficient to program a RAR operation.
So the (theoretical) pro is that you program it like an IPI and it slots
into the IPI code fairly easily. But the con is that it has the same
limitations as IPIs.

I was actually concerned that INVLPGB won't be scalable. Since it
doesn't have the ability to target specific CPUs in the ISA, it
fundamentally need to either have a mechanism to reach all CPUs, or some
way to know which TLB entries each CPU might have.

Maybe AMD has something super duper clever to limit the broadcast scope.
But if they don't, then a small range flush on a small number of CPUs
might end up being pretty expensive, relatively.

I don't think this is a big problem in Rik's series because he had a
floor on the size of processes that get INVLPGB applied. Also, if it
turns out to be a problem, it's dirt simple to revert back to IPIs for
problematic TLB flushes.

But I am deeply curious how the system will behave if there are a
boatload of processes doing modestly-sized INVLPGBs that only apply to a
handful of CPUs on a very large system.

AMD and Intel came at this from very different angles (go figure). The
designs are prioritizing different things for sure. I can't wait to see
both of them fighting it out under real workloads.