Re: [PATCH v4 1/5] x86/bugs: Only harden syscalls when needed

[Yujie Liu <yujie.liu@intel.com>]

On Mon, May 06, 2024 at 10:17:41PM -0700, Josh Poimboeuf wrote:
> On Mon, Apr 22, 2024 at 04:09:33PM +0800, Yujie Liu wrote:
> > On Fri, Apr 19, 2024 at 02:09:47PM -0700, Josh Poimboeuf wrote:
> > > Syscall hardening (converting the syscall indirect branch to a series of
> > > direct branches) has shown some performance regressions:
> > >
> > > - Red Hat internal testing showed up to 12% slowdowns in database
> > >   benchmark testing on Sapphire Rapids when the DB was stressed with 80+
> > >   users to cause contention.
> > >
> > > - The kernel test robot's will-it-scale benchmarks showed significant
> > >   regressions on Skylake with IBRS:
> > >   https://lkml.kernel.org/lkml/202404191333.178a0eed-yujie.liu@intel.com
> > 
> > To clarify, we reported a +1.4% improvement (not regression) of
> > will-it-scale futex4 benchmark on Skylake. Meanwhile we did observe some
> > regressions by running other benchmarks on Ice Lake, such as:
> > 
> >     stress-ng.null.ops_per_sec -4.0% regression on Intel Xeon Gold 6346 (Ice Lake)
> >     unixbench.fsbuffer.throughput -1.4% regression on Intel Xeon Gold 6346 (Ice Lake)
> 
> Thanks for clarifying that.  I'm not sure what I was looking at.
> 
> I also saw your email where Ice Lake showed a ~10% regression for
> 1e3ad78334a6.  Unfortunately my patch wouldn't help with that, as it's
> designed to help with older systems (e.g., Skylake) and newer (e.g.,
> Sapphire Rapids) but not Ice/Cascade Lake.
> 
> Whether 1e3ad78334a6 helps or hurts seems very workload-dependent.
> 
> It would be especially interesting to see if my patch helps on the newer
> systems which have the HW mitigation: Raptor Lake, Meteor Lake, Sapphire
> Rapids, Emerald Rapids.
> 
> For now, maybe I'll just table this patch until we have more data.

FYI, we tested this patch on a Sapphire Rapids machine in our test
infrastructure. There are both improvements and regressions across
different benchmarks, which does show that the impact may be very
workload-dependent.

Intel Xeon Platinum 8480CTDX (Sapphire Rapids) with 512G Memory:

+-----------------------+----------+---------------+------------------+
|                       | mainline |  + this patch | (percent change) |
+-----------------------+----------+---------------+------------------+
| will-it-scale.futex4  | 5015892  | 5118744       | +2.05%           |
+-----------------------+----------+---------------+------------------+
| will-it-scale.pread1  | 2720498  | 2721206       | +0.03%           |
+-----------------------+----------+---------------+------------------+
| will-it-scale.pwrite1 | 2143302  | 2164511       | +0.99%           |
+-----------------------+----------+---------------+------------------+
| will-it-scale.poll1   | 4046943  | 4095046       | +1.19%           |
+-----------------------+----------+---------------+------------------+
| stress-ng.null        | 19400    | 18689         | -3.66%           |
+-----------------------+----------+---------------+------------------+
| unixbench.fsbuffer    | 369494   | 364181        | -1.44%           |
+-----------------------+----------+---------------+------------------+

Thanks,
Yujie