Re: [peterz-queue:sched/eevdf] [sched/fair] 23669fce72: aim7.jobs-per-min -18.6% regression

[Chen Yu <yu.c.chen@intel.com>]

On 2023-03-26 at 15:38:51 +0200, Peter Zijlstra wrote:
> On Sun, Mar 26, 2023 at 01:00:24PM +0200, Peter Zijlstra wrote:
> > sched/eevdf + place_bonus (based on tip/sched/urgent -- will push out
> > later today)
> > 
> > root@ivb-ep:~# echo NO_PLACE_BONUS > /debug/sched/features ; stress-ng --futex 40 -t 5 --metrics 2>&1 | awk '{ if ($4 == "futex") print $0 }'
> > stress-ng: info:  [2373] futex           6541589      5.00      4.28    194.83   1308211.07       32854.97        99.54          2288
> > root@ivb-ep:~# echo PLACE_BONUS > /debug/sched/features ; stress-ng --futex 40 -t 5 --metrics 2>&1 | awk '{ if ($4 == "futex") print $0 }'
> > stress-ng: info:  [2537] futex           9745715      5.00      5.38    194.55   1948945.01       48745.49        99.96          2288
> > 
> > 
> > The whole sleeper bonus is something that's fairly controversial but it
> > was needed in CFS to make some 'starvation' cases go away -- the lag
> > based placement cures them too. And given the whole (recent) trainwreck with:
> > 
> >   829c1651e9c4 ("sched/fair: sanitize vruntime of entity being placed")
> >   a53ce18cacb4 ("sched/fair: Sanitize vruntime of entity being migrated")
> > 
> > I'm happy to delete all that. Still, let me think a little, perhaps I
> > can come up with something slightly less horrible than all that which we
> > can default-disable for now...
> 
> OK, pushed a new version of sched/eevdf that includes a default
> disabled PLACE_BONUS.
> 
> (and lots of other cleanups, eg. moved to scale_load_down() for
> avg_vruntime and measured the max delta to be ~44 bits for a
> systemd/cgroup using kernel build -- after I fixed a cgroup placement
> issue).
I tested stress-ng 100% futex(instance equals to cpu number)
on top of sched/eevdf:

baseline: Commit a53ce18cacb4 ("sched/fair: Sanitize vruntime of entity being migrated"),
NO_PLACE_BONUS + Commit 757a8ca40738 ("sched/eevdf: Debug / validation crud")
PLACE_BONUS + Commit 757a8ca40738

In summary the PLACE_BONUS has restored the throughput(and even better than baseline):

baseline:
2023-03-27 13:25:36 stress-ng --timeout 60 --times --verify --metrics-brief --futex 128
stress-ng: info:  [5930] setting to a 60 second run per stressor
stress-ng: info:  [5930] dispatching hogs: 128 futex
stress-ng: info:  [5930] stressor       bogo ops real time  usr time  sys time   bogo ops/s     bogo ops/s
stress-ng: info:  [5930]                           (secs)    (secs)    (secs)   (real time) (usr+sys time)
stress-ng: info:  [5930] futex         129599510     60.00    205.28   7296.59   2159865.69       17275.62

757a8ca40738 + NO_PLACE_BONUS:
2023-03-27 12:51:02 stress-ng --timeout 60 --times --verify --metrics-brief --futex 128
stress-ng: info:  [5968] setting to a 60 second run per stressor
stress-ng: info:  [5968] dispatching hogs: 128 futex
stress-ng: info:  [5968] stressor       bogo ops real time  usr time  sys time   bogo ops/s     bogo ops/s
stress-ng: info:  [5968]                           (secs)    (secs)    (secs)   (real time) (usr+sys time)
stress-ng: info:  [5968] futex          63869149     60.00     87.55   7289.74   1064478.29        8657.53

757a8ca40738 + PLACE_BONUS:
2023-03-27 12:46:38 stress-ng --timeout 60 --times --verify --metrics-brief --futex 128
stress-ng: info:  [5902] setting to a 60 second run per stressor
stress-ng: info:  [5902] dispatching hogs: 128 futex
stress-ng: info:  [5902] stressor       bogo ops real time  usr time  sys time   bogo ops/s     bogo ops/s
stress-ng: info:  [5902]                           (secs)    (secs)    (secs)   (real time) (usr+sys time)
stress-ng: info:  [5902] futex         134332423     60.27    152.41   7224.27   2228839.17       18210.41

I'll launch more tests tomorrow with/without PLACE_BONUS.

thanks,
Chenyu