Re: lmbench ctxsw regression with CFS

Nick Piggin
2007-08-03T00:14:58Z

On Thu, Aug 02, 2007 at 05:44:47PM +0200, Ingo Molnar wrote:
>
> * Nick Piggin <npiggin@suse.de> wrote:
>
> > > > > One thing to check out is whether the lmbench numbers are
> > > > > "correct". Especially on SMP systems, the lmbench numbers are
> > > > > actually *best* when the two processes run on the same CPU, even
> > > > > though that's not really at all the best scheduling - it's just
> > > > > that it artificially improves lmbench numbers because of the
> > > > > close cache affinity for the pipe data structures.
> > > >
> > > > Yes, I bound them to a single core.
> > >
> > > could you send me the .config you used?
> >
> > Sure, attached...
> >
> > You don't see a regression? If not, then can you send me the .config
> > you used? [...]
>
> i used your config to get a few numbers and to see what happens. Here's
> the numbers of 10 consecutive "lat_ctx -s 0 2" runs:
>
> [ time in micro-seconds, smaller is better ]
>
> v2.6.22 v2.6.23-git v2.6.23-git+const-param
> ------- ----------- -----------------------
> 1.30 1.60 1.19
> 1.30 1.36 1.18
> 1.14 1.50 1.01
> 1.26 1.27 1.23
> 1.22 1.40 1.04
> 1.13 1.34 1.09
> 1.27 1.39 1.05
> 1.20 1.30 1.16
> 1.20 1.17 1.16
> 1.25 1.33 1.01
> -------------------------------------------------------------
> avg: 1.22 1.36 (+11.3%) 1.11 (-10.3%)
> min: 1.13 1.17 ( +3.5%) 1.01 (-11.8%)
> max: 1.27 1.60 (+26.0%) 1.23 ( -3.2%)
>
> one reason for the extra overhead is the current tunability of CFS, but
> that is not fundamental, it's caused by the many knobs that CFS has at
> the moment. The const-tuning patch (attached below, results in the
> rightmost column) changes those knobs to constants, allowing the
> compiler to optimize the math better and reduce code size. (the code
> movement in the patch makes up for most of its size, the change that it
> does is simple otherwise.)

[...]

Oh good. Thanks for getting to the bottom of it. We have normally
disliked too much runtime tunables in the scheduler, so I assume
these are mostly going away or under a CONFIG option for 2.6.23?
Or...?

What CPU did you get these numbers on? Do the indirect calls hurt
much on those without an indirect predictor? (I'll try running some
tests).

I must say that I don't really like the indirect calls a great deal,
and they could be eliminated just with a couple of branches and
direct calls.