Re: BFS vs. mainline scheduler benchmarks and measurements

[Nikos Chantziaras <realnc@arcor.de>]

On 09/08/2009 06:23 PM, Peter Zijlstra wrote:
> On Tue, 2009-09-08 at 11:13 +0200, Jens Axboe wrote:
>> And here's a newer version.
>
> I tinkered a bit with your proglet and finally found the problem.
>
> You used a single pipe per child, this means the loop in run_child()
> would consume what it just wrote out until it got force preempted by the
> parent which would also get woken.
>
> This results in the child spinning a while (its full quota) and only
> reporting the last timestamp to the parent.
>
> Since consumer (parent) is a single thread the program basically
> measures the worst delay in a thundering herd wakeup of N children.
>
> The below version yields:
>
> idle
>
> [root@opteron sched]# ./latt -c8 sleep 30
> Entries: 664 (clients=8)
>
> Averages:
> ------------------------------
>          Max           128 usec
>          Avg            26 usec
>          Stdev          16 usec
>
>
> make -j4
>
> [root@opteron sched]# ./latt -c8 sleep 30
> Entries: 648 (clients=8)
>
> Averages:
> ------------------------------
>          Max         20861 usec
>          Avg          3763 usec
>          Stdev        4637 usec
>
>
> Mike's patch, make -j4
>
> [root@opteron sched]# ./latt -c8 sleep 30
> Entries: 648 (clients=8)
>
> Averages:
> ------------------------------
>          Max         17854 usec
>          Avg          6298 usec
>          Stdev        4735 usec

I've run two tests with this tool.  One with mainline (2.6.31-rc9) and 
one patched with 2.6.31-rc9-sched-bfs-210.patch.

Before running this test, I disabled the cron daemon in order not to 
have something pop-up in the background out of a sudden.

The test consisted of starting a "make -j2" in the kernel tree inside a 
3GB tmpfs mountpoint and then running 'latt "mplayer -vo gl2 -framedrop 
videofile.mkv"'  (mplayer in this case is a single-threaded 
application.)  Caches were warmed-up first; the results below are from 
the second run of each test.

The kernel .config file used by the running kernels and also for "make 
-j2" is:

   http://foss.math.aegean.gr/~realnc/kernel/config-2.6.31-rc9-latt-test

The video file used for mplayer is:

   http://foss.math.aegean.gr/~realnc/vids/3DMark2000.mkv (100MB)
   (The reason this was used is that it's a 60FPS video,
   therefore very smooth and makes all skips stand out
   clearly.)


Results for mainline:

   Averages:
   ------------------------------
           Max         29930 usec
           Avg         11043 usec
           Stdev        5752 usec


Results for BFS:

   Averages:
   ------------------------------
           Max         14017 usec
           Avg            49 usec
           Stdev         697 usec


One thing that's worth noting is that with mainline, mplayer would 
occasionally spit this out:

    YOUR SYSTEM IS TOO SLOW TO PLAY THIS

which doesn't happen with BFS.