[test] hackbench.c interactivity results: vanilla versus SD/RSDL

[Ingo Molnar <mingo@elte.hu>]

* Ingo Molnar <mingo@elte.hu> wrote:

> * Con Kolivas <kernel@kolivas.org> wrote:
> 
> > I'm cautiously optimistic that we're at the thin edge of the bugfix 
> > wedge now.
[...]

> and the numbers he posted:
> 
>  http://marc.info/?l=linux-kernel&m=117448900626028&w=2
> 
> his test conclusion was that under CPU load, RSDL (SD) generally does 
> not hold up to mainline's interactivity.

Today i've done some hackbench.c (which is similar to VolanoMark) 
interactivity testing on SD-latest (v0.37), doing "hackbench 10", 
"hackbench 50" and "hackbench 100" runs, comparing it to 
vanilla+Mike's-task-promotion-patch.

Performance
-----------

performance was largely comparable, within noise: the vanilla scheduler 
was slightly (~5%) better at "hackbench 10", SD and vanilla was within 
noise on "hackbench 50" and "hackbench 100" runs. (I think vanilla's 
better results might be due to the longer timeslices vanilla can give 
out - SD has to operate via short timeslices, by design.)

Shell interactivity
-------------------

The vanilla scheduler kept the shell completely usable during all tests: 
'ls' output was instantaneous and characters could be typed without 
delay.

The SD/RSDL scheduler kept the shell barely usable during the hackbench 
10 test, and it was completely unusable during the hackbench 50 and 100 
tests. A simple 'ls' took 2-3 seconds to complete (up to 6 seconds at 
times) and the shell printed characters in a very laggy way. 'vi' was 
totally unusable, etc. etc.

[ i've also re-tested this with RSDL 0.34, and there interactivity got 
  better although it still didnt match that of the vanilla scheduler's
  interactivity. So this is a definitive SD regression. ]

[ A quick guess: could SD's substandard interactivity in this test be 
  due to the SMP migration logic inconsistencies Mike noticed? This is 
  an SMP system and the hackbench workload is very scheduling intense 
  and tasks are frequently queued from one CPU to another. ]

Conclusion
----------

i consider this a must-fix item as SD badly misbehaves under this 
workload.

Test environment details
------------------------

the test system is a 2GHz Athlon64 dual-core box. All tests were running 
on default nice 0 levels. All tests were done 10 times on a totally idle 
test-system. Mike's patch is the one that improves vanilla scheduler's 
anti-starvation code:

   http://marc.info/?l=linux-kernel&m=117507110922009&w=2

( Mike's patch does not make a measurable performance difference in
  hackbench, nor does it make a visible interactivity difference for 
  this workload, but since i think Mike's patch improves the vanilla 
  scheduler i included it in the test for completeness, so that both 
  variants of interactivity code are at the 'bleeding edge'. )

	Ingo