Re: Phoronix CFS vs BFS bencharks

[Mike Galbraith <efault@gmx.de>]

On Mon, 2009-09-14 at 22:49 +0200, Willy Tarreau wrote:
> On Mon, Sep 14, 2009 at 09:14:35PM +0200, Marcin Letyns wrote:
> > 2009/9/14 Mike Galbraith <efault@gmx.de>
> > >
> > > On Mon, 2009-09-14 at 16:27 +0200, Marcin Letyns wrote:
> > > > Hello,
> > > >
> > > > Disabling NEW_FAIR_SLEEPERS makes a lot of difference here in the
> > > > Apache benchmark:
> > > >
> > > > 2.6.30.6-bfs: 7311.05
> > > >
> > > > 2.6.30.6-cfs-fair_sl_disabled: 8249.17
> > > >
> > > > 2.6.30.6-cfs-fair_sl_enabled: 4894.99
> > >
> > > Wow.
> > >
> > > Some loads like wakeup preemption (mysql+oltp), and some hate it.  This
> > > load appears to REALLY hate it (as does volanomark, but that thing is
> > > extremely overloaded).  How many threads does that benchmark run
> > > concurrently?
> > 
> > >From the benchmark description:
> > 
> > This is a test of ab, which is the Apache Benchmark program. This test
> > profile measures how many requests per second a given system can
> > sustain when carrying out 500,000 requests with 100 requests being
> > carried out concurrently.
> 
> Be careful not to run ab on the same machine as you run apache, otherwise
> the numerous apache processes can limit ab's throughput. This is the same
> reason as why I educate people so that they don't run a single-process
> proxy in front of a multi-process/multi-thread web server. Apparently
> it's not obvious to everyone.

I turned on apache, and played with ab a bit, and yup, ab is a hog, so
any fairness hurts it a badly.  Ergo, running ab on the same box as
apache suffers with CFS when NEW_FAIR_SLEEPERS are turned on.  Issuing
ab bandwidth to match it's 1:N pig nature brings throughput right back.

(In all the comparison testing I've done, BFS favors hogs, and with
NEW_FAIR_SLEEPERS turned off, so does CFS, though not as much.)

Running apache on one core and ab on another (with shared cache tho),
something went south with BFS. I would have expected it to be much
closer (shrug).  

Some likely not very uninteresting numbers below.  I wasted a lot more
of my time generating them than anyone will downloading them :)

ab -n 500000 -c 100 http://localhost/openSUSE.org.html

2.6.31-bfs221-smp
Concurrency Level:      100
Time taken for tests:   43.556 seconds
Complete requests:      500000
Failed requests:        0
Write errors:           0
Total transferred:      7158558404 bytes
HTML transferred:       7027047358 bytes
Requests per second:    11479.50 [#/sec] (mean)
Time per request:       8.711 [ms] (mean)
Time per request:       0.087 [ms] (mean, across all concurrent requests)
Transfer rate:          160501.38 [Kbytes/sec] received

2.6.32-tip-smp NO_NEW_FAIR_SLEEPERS
Concurrency Level:      100
Time taken for tests:   42.834 seconds
Complete requests:      500000
Failed requests:        0
Write errors:           0
Total transferred:      7158429480 bytes
HTML transferred:       7026921590 bytes
Requests per second:    11672.84 [#/sec] (mean)
Time per request:       8.567 [ms] (mean)
Time per request:       0.086 [ms] (mean, across all concurrent requests)
Transfer rate:          163201.63 [Kbytes/sec] received

2.6.32-tip-smp NEW_FAIR_SLEEPERS
Concurrency Level:      100
Time taken for tests:   68.221 seconds
Complete requests:      500000
Failed requests:        0
Write errors:           0
Total transferred:      7158357900 bytes
HTML transferred:       7026851325 bytes
Requests per second:    7329.12 [#/sec] (mean)
Time per request:       13.644 [ms] (mean)
Time per request:       0.136 [ms] (mean, across all concurrent requests)
Transfer rate:          102469.65 [Kbytes/sec] received

2.6.32-tip-smp NEW_FAIR_SLEEPERS + ab at nice -15
Concurrency Level:      100
Time taken for tests:   42.824 seconds
Complete requests:      500000
Failed requests:        0
Write errors:           0
Total transferred:      7158451988 bytes
HTML transferred:       7026943572 bytes
Requests per second:    11675.68 [#/sec] (mean)
Time per request:       8.565 [ms] (mean)
Time per request:       0.086 [ms] (mean, across all concurrent requests)
Transfer rate:          163241.78 [Kbytes/sec] received

taskset -c 2 /etc/init.d/apache2 restart
taskset -c 3 ab -n 500000 -c 100 http://localhost/openSUSE.org.html

2.6.31-bfs221-smp
Concurrency Level:      100
Time taken for tests:   86.590 seconds
Complete requests:      500000
Failed requests:        0
Write errors:           0
Total transferred:      7158000000 bytes
HTML transferred:       7026500000 bytes
Requests per second:    5774.37 [#/sec] (mean)
Time per request:       17.318 [ms] (mean)
Time per request:       0.173 [ms] (mean, across all concurrent requests)
Transfer rate:          80728.41 [Kbytes/sec] received

2.6.32-tip-smp
Concurrency Level:      100
Time taken for tests:   48.640 seconds
Complete requests:      500000
Failed requests:        0
Write errors:           0
Total transferred:      7158000000 bytes
HTML transferred:       7026500000 bytes
Requests per second:    10279.71 [#/sec] (mean)
Time per request:       9.728 [ms] (mean)
Time per request:       0.097 [ms] (mean, across all concurrent requests)
Transfer rate:          143715.15 [Kbytes/sec] received