Re: Mainline kernel OLTP performance update

[Andrew Morton <akpm@linux-foundation.org>]

On Wed, 14 Jan 2009 18:21:47 -0700 Matthew Wilcox <matthew@wil.cx> wrote:

> On Wed, Jan 14, 2009 at 04:35:57PM -0800, Andrew Morton wrote:
> > On Tue, 13 Jan 2009 15:44:17 -0700
> > "Wilcox, Matthew R" <matthew.r.wilcox@intel.com> wrote:
> > >
> > 
> > (top-posting repaired.  That @intel.com address is a bad influence ;))
> 
> Alas, that email address goes to an Outlook client.  Not much to be done
> about that.

aspirin?

> > (cc linux-scsi)
> > 
> > > > This is latest 2.6.29-rc1 kernel OLTP performance result. Compare to
> > > > 2.6.24.2 the regression is around 3.5%.
> > > > 
> > > > Linux OLTP Performance summary
> > > > Kernel#            Speedup(x)   Intr/s  CtxSw/s us%  sys%   idle%  iowait%
> > > > 2.6.24.2                1.000   21969   43425   76   24     0      0
> > > > 2.6.27.2                0.973   30402   43523   74   25     0      1
> > > > 2.6.29-rc1              0.965   30331   41970   74   26     0      0
> 
> > But the interrupt rate went through the roof.
> 
> Yes.  I forget why that was; I'll have to dig through my archives for
> that.

Oh.  I'd have thought that this alone could account for 3.5%.

> > A 3.5% slowdown in this workload is considered pretty serious, isn't it?
> 
> Yes.  Anything above 0.3% is statistically significant.  1% is a big
> deal.  The fact that we've lost 3.5% in the last year doesn't make
> people happy.  There's a few things we've identified that have a big
> effect:
> 
>  - Per-partition statistics.  Putting in a sysctl to stop doing them gets
>    some of that back, but not as much as taking them out (even when
>    the sysctl'd variable is in a __read_mostly section).  We tried a
>    patch from Jens to speed up the search for a new partition, but it
>    had no effect.

I find this surprising.

>  - The RT scheduler changes.  They're better for some RT tasks, but not
>    the database benchmark workload.  Chinang has posted about
>    this before, but the thread didn't really go anywhere.
>    http://marc.info/?t=122903815000001&r=1&w=2

Well.  It's more a case that it wasn't taken anywhere.  I appear to
have recently been informed that there have never been any
CPU-scheduler-caused regressions.  Please persist!

> SLUB would have had a huge negative effect if we were using it -- on the
> order of 7% iirc.  SLQB is at least performance-neutral with SLAB.

We really need to unblock that problem somehow.  I assume that
enterprise distros are shipping slab?