Re: [x86/mm/tlb] 6035152d8e: will-it-scale.per_thread_ops -13.2% regression

[Dave Hansen <dave.hansen@intel.com>]

On 3/17/22 12:02, Nadav Amit wrote:
>> This new "early lazy check" behavior could theoretically work both ways.
>> If threads tended to be waking up from idle when TLB flushes were being
>> sent, this would tend to reduce the number of IPIs.  But, since they
>> tend to be going to sleep it increases the number of IPIs.
>>
>> Anybody have a better theory?  I think we should probably revert the commit.
> 
> Let’s get back to the motivation behind this patch.
> 
> Originally we had an indirect branch that on system which are
> vulnerable to Spectre v2 translates into a retpoline.
> 
> So I would not paraphrase this patch purpose as “early lazy check”
> but instead “more efficient lazy check”. There is very little code
> that was executed between the call to on_each_cpu_cond_mask() and
> the actual check of tlb_is_not_lazy(). So what it seems to happen
> in this test-case - according to what you say - is that *slower*
> checks of is-lazy allows to send fewer IPIs since some cores go
> into idle-state.
> 
> Was this test run with retpolines? If there is a difference in
> performance without retpoline - I am probably wrong.

Nope, no retpolines:

> /sys/devices/system/cpu/vulnerabilities/spectre_v2:Mitigation: Enhanced IBRS, IBPB: conditional, RSB filling

which is the same situation as the "Xeon Platinum 8358" which found this
in 0day.

Maybe the increased IPIs with this approach end up being a wash with the
reduced retpoline overhead.

Did you have any specific performance numbers that show the benefit on
retpoline systems?