Re: [PATCH 0/1] cpuidle/menu: Address performance drop from favoring physical over polling cpuidle state

[Christian Loehle <christian.loehle@arm.com>]

Hi Aboorva,

On 10/21/24 06:27, Aboorva Devarajan wrote:
> [...] 
> I ran the benchmark again and collected the idle statistics on a different
> system since I no longer have access to the previous one.
> 
> Here are the stats I gathered, including data for the menu, teo, and ladder governors:
> 
> Metric                        | Ladder        | Teo           | Menu          | Menu Patched
> --------------------------------------------------------------------------------------------
> Transactions Processed        | 42,902,188    | 48,499,709    | 42,653,477    | 48,648,867   
> Latency (ms)                  | 0.313         | 0.277         | 0.315         | 0.276        
> TPS                           | 715,340.96    | 808,672.59    | 711,123.13    | 811,137.40  
> --------------------------------------------------------------------------------------------
> Total Usage Difference        | 46,680,184    | 66,348,992    | 47,892,509    | 62,366,337
> Usage Diff (CEDE)             | 46,680,184    | 7,612,960     | 45,421,436    | 19,237,430
> Usage Diff (Snooze)           | 0             | 58,736,032    | 2,471,073     | 43,128,907
> --------------------------------------------------------------------------------------------
> Total Time Difference (s)     | 5,552.20      | 5,242.75      | 5,534.62      | 5,238.46     
> Time Diff (CEDE)              | 5,552.20      | 2,497.89      | 5,431.91      | 3,324.99
> Time Diff (Snooze)            | 0.00          | 2,744.86      | 102.71        | 1,913.47
> --------------------------------------------------------------------------------------------
> Total Above Diff              | 40,381,398    | 4,520,998     | 38,942,725    | 15,072,345
> Above Diff (CEDE)             | 40,381,398    | 4,520,998     | 38,942,725    | 15,072,345
> Above Diff (Snooze)           | 0             | 0             | 0             | 0
> --------------------------------------------------------------------------------------------
> Total Below Diff              | 0             | 12,362,392    | 405,357       | 8,172,845
> Below Diff (CEDE)             | 0             | 0             | 0             | 0
> Below Diff (Snooze)           | 0             | 12,362,392    | 405,357       | 8,172,845
> --------------------------------------------------------------------------------------------
> % Above w.r.t. Usage          | 86.51%        | 6.82%         | 81.30%        | 24.17%
> --------------------------------------------------------------------------------------------
> % Below w.r.t. Usage          | 0%            | 18.64%        | 0.85%         | 13.10%
> --------------------------------------------------------------------------------------------
> 
> --------------------------------------------------------------------------------------------
> 
> Note:
> 
> % Above w.r.t Usage = (Total Above Difference / Total Usage Difference) * 100
> % Below w.r.t Usage = (Total Below Difference / Total Usage Difference) * 100
> 
> 
> Menu, Teo, Ladder: This is with menu, teo and ladder governor enabled respectively on v6.12-rc1.
> 
> Menu Patched: This is with menu governor enabled on v6.12-rc + 
>               https://lore.kernel.org/all/20240809073120.250974-2-aboorvad@linux.ibm.com/
> --------------------------------------------------------------------------------------------
> 
> --------------------------------------------------------------------------------------------
> Inference:
> --------------------------------------------------------------------------------------------
> 
> Transactions Processed: The number of transactions processed in Menu Patched is
> 14.06% higher compared to Menu.
> 
> Latency: There is a 12.38% reduction in latency in Menu Patched compared to Menu.
> 
> TPS (Transactions Per Second): The TPS in Menu Patched is 14.06% higher than in
> Menu.
> --------------------------------------------------------------------------------------------
> 
> While this patch hasn't completely eliminated the cpuidle miss ratio, but it
> has improved see Above w.r.t Usage, Below w.r.t Usage.
> 
> I'll keep investigating why there's still a 24% miss rate in the "above" and
> 13% in the "below" stats after the patch. This could be a different issue.
> Additionally, I've seen good performance improvements using the teo governor
> with the pgbench benchmark, although it didn't provide the same benefit in the
> original test.

Good to hear and thank you for testing.
Indeed the menu idle misses sound somewhat high, given that teo scores fewer.
I'd be very curious about the workload where menu patched significantly outperforms
teo if you ever come across that again.


> [...] 
> 
> I ran a similar benchmark using teo cpuidle governor, presented the averaged out
> values across 10 runs (has low standard deviation). Below are the results for a
> pipe-based wakeup with an approximate 50 microsecond sleep interval:
> 
> 
> Pipe based wakeup with approx 50 us as sleep interval:
> 
> Metric                    Ladder           Menu            Teo              Menu Patched
> ----------------------------------------------------------------------------------------
> Wakeups                   579,690          579,951         578,328          578,363
> Avg wakeup latency (us)   7.456            7.112           4.46067          4.48733
> ========================================================================================
> Sleep interval (us)       51.7333          51.7097         51.8533          51.8477
> ========================================================================================
> State 0 Usage diff        0                7,324           578,361          578,407
> State 0 Time diff (ns)    0                340,115         2.75953e+07      2.75696e+07
> State 0 Above diff        0                0               0                0
> State 0 Below diff        0                0               0.333333         0.666667
> ========================================================================================
> State 1 Usage diff        580,114          572,644         0.666667         9.33333
> State 1 Time diff (ns)    2.74189e+07      2.73901e+07     20.6667          445.333
> State 1 Above diff        579,993          572,535         0.666667         9.33333
> State 1 Below diff        0                0               0                0
> Total Above               579,993          572,535         0.666667         9.33333
> Total Below               0                0               0.333333         0.666667
> ========================================================================================
> Total Above Percent       99.98%           98.7167%        0%               0%             --> [1]
> Total Below Percent       0%               0%              0%               0%
> ========================================================================================
> 
> Timer based wakeup with approx 50 us as sleep interval:
> 
> Metric                    Ladder           Menu            Teo              Menu Patched
> ----------------------------------------------------------------------------------------
> Sleep interval (us)        53.775           52.3687         52.746           52.327
> ========================================================================================
> State 0 Usage diff         0                572,575         568,419          573,109
> State 0 Time diff (ns)     0                2.84142e+07     2.81956e+07      2.84548e+07
> State 0 Above diff         0                0               0                0
> State 0 Below diff         0                0.333333        1                0.333333
> ========================================================================================
> State 1 Usage diff         558,285          0.333333        0                0
> State 1 Time diff (ns)     2.80965e+07      17              0                0
> State 1 Above diff         558,215          0.333333        0                0
> State 1 Below diff         0                0               0                0
> ========================================================================================
> Total Above                558,215          0.333333        0                0
> Total Below                0                0.333333        1                0.333333
> ========================================================================================
> Total Above Percent        99.99%           0%              0%               0%
> Total Below Percent        0%               0%              0%               0%
> ========================================================================================
> 
> 
> [1] The results does show that the teo governor doesn't have this issue.

Glad to see that confirmed, thank you.

Regards,
Christian