Re: [patch] CFS scheduler, -v19

[Ingo Molnar <mingo@elte.hu>]

* Willy Tarreau <w@1wt.eu> wrote:

> > The biggest user-visible change in -v19 is reworked sleeper 
> > fairness: it's similar in behavior to -v18 but works more 
> > consistently across nice levels. Fork-happy workloads (like kernel 
> > builds) should behave better as well. There are also a handful of 
> > speedups: unsigned math, 32-bit speedups, O(1) task pickup, 
> > debloating and other micro-optimizations.
> 
> Interestingly, I also noticed the possibility of O(1) task pickup when 
> playing with v18, but did not detect any noticeable improvement with 
> it. Of course, it depends on the workload and I probably didn't 
> perform the most relevant tests.

yeah - it's a small tweak. CFS is O(31) in sleep/wakeup so it's now all 
a big O(1) family again :)

> V19 works very well here on 2.6.20.14. I could start 32k busy loops at 
> nice +19 (I exhausted the 32k pids limit), and could still perform 
> normal operations. I noticed that 'vmstat' scans all the pid entries 
> under /proc, which takes ages to collect data before displaying a 
> line. Obviously, the system sometimes shows some short latencies, but 
> not much more than what you get from and SSH through a remote DSL 
> connection.

great! I did not try to push it this far, yet.

> Here's a vmstat 1 output :
> 
>  r  b  w   swpd   free   buff  cache   si   so    bi    bo   in    cs us sy id
> 32437  0  0      0 809724    488   6196    0    0     1     0  135     0 24 72 4
> 32436  0  0      0 811336    488   6196    0    0     0     0  717     0 78 22 0

crazy :-)

> Amusingly, I started mpg123 during this test and it skipped quite a 
> bit. After setting all tasks to SCHED_IDLE, it did not skip anymore. 
> All this seems to behave like one could expect.

yeah. It behaves better than i expected in fact - 32K tasks is pushing 
things quite a bit. (we've got a 32K PID limit for example)

> I also started 30k processes distributed in 130 groups of 234 chained 
> by pipes in which one byte is passed. I get an average of 8000 in the 
> run queue. The context switch rate is very low and sometimes even null 
> in this test, maybe some of them are starving, I really do not know :
> 
>  r  b  w   swpd   free   buff  cache   si   so    bi    bo   in    cs us sy id
> 7752  0  1      0 656892    244   4196    0    0     0     0  725     0 16 84  0

hm, could you profile this? We could have some bottleneck somewhere 
(likely not in the scheduler) with that many tasks being runnable. [ 
With CFS you can actually run a profiler under this workload ;-) ]

> In my tree, I have replaced the rbtree with the ebtree we talked 
> about, but it did not bring any performance boost because, eventhough 
> insert() and delete() are faster, the scheduler is already quite good 
> at avoiding them as much as possible, mostly relying on rb_next() 
> which has the same cost in both trees. All in all, the only variations 
> I noticed were caused by cacheline alignment when I tried to reorder 
> fields in the eb_node. So I will stop my experimentations here since I 
> don't see any more room for improvement.

well, just a little bit of improvement would be nice to have too :)

	Ingo