Re: [RFC PATCH v1 00/11] Create fast idle path for short idle periods

["Li, Aubrey" <aubrey.li@linux.intel.com>]

On 2017/7/14 12:05, Paul E. McKenney wrote:
>
> More specifically: rcu_needs_cpu(), rcu_prepare_for_idle(),
> rcu_cleanup_after_idle(), rcu_eqs_enter(), rcu_eqs_enter_common(),
> rcu_dynticks_eqs_enter(), do_nocb_deferred_wakeup(),
> rcu_dynticks_task_enter(), rcu_eqs_exit(), rcu_eqs_exit_common(),
> rcu_dynticks_task_exit(), rcu_dynticks_eqs_exit().
>
> The first three (rcu_needs_cpu(), rcu_prepare_for_idle(), and
> rcu_cleanup_after_idle()) should not be significant unless you have
> CONFIG_RCU_FAST_NO_HZ=y.  If you do, it would be interesting to learn
> how often invoke_rcu_core() is invoked from rcu_prepare_for_idle()
> and rcu_cleanup_after_idle(), as this can raise softirq.  Also
> rcu_accelerate_cbs() and rcu_try_advance_all_cbs().
>
> Knowing which of these is causing the most trouble might help me
> reduce the overhead in the current idle path.

I measured two cases, nothing notable found.

The one is CONFIG_NO_HZ_IDLE=y, so the following function is just empty.

rcu_prepare_for_idle():         NULL
rcu_cleanup_after_idle():       NULL
do_nocb_deferred_wakeup():      NULL
rcu_dynticks_task_enter():      NULL
rcu_dynticks_task_exit():       NULL

And the following functions are traced separately, for each function I
traced 3 times by intel_PT, for each time the sampling period is 1-second.
num means the times the function is invoked in 1-second. (min, max, avg) is
the function duration, the unit is nano-second. 

rcu_needs_cpu():
1) num: 6110    min: 3  max: 564        avg: 17.0
2) num: 16535   min: 3  max: 683        avg: 18.0
3) num: 8815    min: 3  max: 394        avg: 20.0

rcu_eqs_enter():
1) num: 7956    min: 17 max: 656        avg: 32.0
2) num: 9170    min: 17 max: 1075       avg: 35.0
3) num: 8604    min: 17 max: 859        avg: 29.0

rcu_eqs_enter_common():
1) num: 14676   min: 15 max: 620        avg: 28.0
2) num: 11180   min: 15 max: 795        avg: 30.0
3) num: 11484   min: 15 max: 725        avg: 29.0

rcu_dynticks_eqs_enter():
1) num: 11035   min: 10 max: 580        avg: 17.0
2) num: 15518   min: 10 max: 456        avg: 16.0
3) num: 15320   min: 10 max: 454        avg: 19.0

rcu_eqs_exit():
1) num: 11080   min: 14 max: 893        avg: 23.0
2) num: 13526   min: 14 max: 640        avg: 23.0
3) num: 12534   min: 14 max: 630        avg: 22.0

rcu_eqs_exit_common():
1) num: 18002   min: 12 max: 553        avg: 17.0
2) num: 10570   min: 11 max: 485        avg: 17.0
3) num: 13628   min: 11 max: 567        avg: 16.0

rcu_dynticks_eqs_exit():
1) num: 11195   min: 11 max: 436        avg: 16.0
2) num: 11808   min: 10 max: 506        avg: 16.0
3) num: 8132    min: 10 max: 546        avg: 15.0

==============================================================================

The other case is CONFIG_NO_HZ_FULL, I also enabled the required config to make
all the functions not empty.

rcu_needs_cpu():
1) num: 8530    min: 5  max: 770        avg: 13.0
2) num: 9965    min: 5  max: 518        avg: 12.0
3) num: 12503   min: 5  max: 755        avg: 16.0

rcu_prepare_for_idle():
1) num: 11662   min: 5  max: 684        avg: 9.0
2) num: 15294   min: 5  max: 676        avg: 9.0
3) num: 14332   min: 5  max: 524        avg: 9.0

rcu_cleanup_after_idle():
1) num: 13584   min: 4  max: 657        avg: 6.0
2) num: 9102    min: 4  max: 529        avg: 5.0
3) num: 10648   min: 4  max: 471        avg: 6.0

rcu_eqs_enter():
1) num: 14222   min: 26 max: 745        avg: 54.0
2) num: 12502   min: 26 max: 650        avg: 53.0
3) num: 11834   min: 26 max: 863        avg: 52.0

rcu_eqs_enter_common():
1) num: 16792   min: 24 max: 973        avg: 43.0
2) num: 19755   min: 24 max: 898        avg: 45.0
3) num: 8167    min: 24 max: 722        avg: 42.0

rcu_dynticks_eqs_enter():
1) num: 11605   min: 10 max: 532        avg: 14.0
2) num: 10438   min: 9  max: 554        avg: 14.0
3) num: 19816   min: 9  max: 701        avg: 14.0

do_nocb_deferred_wakeup():
1) num: 15348   min: 1  max: 459        avg: 3.0
2) num: 12822   min: 1  max: 564        avg: 4.0
3) num: 8272    min: 0  max: 458        avg: 3.0

rcu_dynticks_task_enter():
1) num: 6358    min: 1  max: 268        avg: 1.0
2) num: 11128   min: 1  max: 360        avg: 1.0
3) num: 20516   min: 1  max: 214        avg: 1.0

rcu_eqs_exit():
1) num: 16117   min: 20 max: 782        avg: 43.0
2) num: 11042   min: 20 max: 775        avg: 47.0
3) num: 16499   min: 20 max: 752        avg: 46.0

rcu_eqs_exit_common():
1) num: 12584   min: 17 max: 703        avg: 28.0
2) num: 17412   min: 17 max: 759        avg: 28.0
3) num: 16733   min: 17 max: 798        avg: 29.0

rcu_dynticks_task_exit():
1) num: 11730   min: 1  max: 528        avg: 4.0
2) num: 18840   min: 1  max: 581        avg: 5.0
3) num: 9815    min: 1  max: 381        avg: 4.0

rcu_dynticks_eqs_exit():
1) num: 10902   min: 9  max: 557        avg: 13.0
2) num: 19474   min: 9  max: 563        avg: 13.0
3) num: 11865   min: 9  max: 672        avg: 12.0

Please let me know if there is some data not reasonable, I can revisit again.

Thanks,
-Aubrey