[PATCH] cpuidle: Add 'high' and 'low' idle state metrics

[PATCH] cpuidle: Add 'high' and 'low' idle state metrics

[Rafael J. Wysocki]

From: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

Add two new metrics for CPU idle states, "high" and "low", to count
the number of times the given state had been asked for (or entered
from the kernel's perspective), but the observed idle duration turned
out to be too high or too low for it (respectively).

These mertics help to estimat the quality of the CPU idle governor
in use.

Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
---

On top of https://patchwork.kernel.org/patch/10705317/

---
 Documentation/ABI/testing/sysfs-devices-system-cpu |    7 +++++
 Documentation/admin-guide/pm/cpuidle.rst           |    8 +++++
 drivers/cpuidle/cpuidle.c                          |   29 +++++++++++++++++++++
 drivers/cpuidle/sysfs.c                            |    6 ++++
 include/linux/cpuidle.h                            |    2 +
 5 files changed, 52 insertions(+)

Index: linux-pm/drivers/cpuidle/cpuidle.c
===================================================================
--- linux-pm.orig/drivers/cpuidle/cpuidle.c
+++ linux-pm/drivers/cpuidle/cpuidle.c
@@ -248,6 +248,8 @@ int cpuidle_enter_state(struct cpuidle_d
 		local_irq_enable();
 
 	if (entered_state >= 0) {
+		int i;
+
 		/*
 		 * Update cpuidle counters
 		 * This can be moved to within driver enter routine,
@@ -260,6 +262,33 @@ int cpuidle_enter_state(struct cpuidle_d
 		dev->last_residency = (int)diff;
 		dev->states_usage[entered_state].time += dev->last_residency;
 		dev->states_usage[entered_state].usage++;
+
+		if (diff < drv->states[entered_state].target_residency) {
+			for (i = entered_state - 1; i >= 0; i--) {
+				if (drv->states[i].disabled ||
+				    dev->states_usage[i].disable)
+					continue;
+
+				/* Shallower states are enabled, so update. */
+				dev->states_usage[entered_state].high++;
+				break;
+			}
+		} else {
+			for (i = entered_state + 1; i < drv->state_count; i++) {
+				if (drv->states[i].disabled ||
+				    dev->states_usage[i].disable)
+					continue;
+
+				/*
+				 * Update if a deeper state would have been a
+				 * better match for the observed idle duration.
+				 */
+				if (diff >= drv->states[i].target_residency)
+					dev->states_usage[entered_state].low++;
+
+				break;
+			}
+		}
 	} else {
 		dev->last_residency = 0;
 	}
Index: linux-pm/include/linux/cpuidle.h
===================================================================
--- linux-pm.orig/include/linux/cpuidle.h
+++ linux-pm/include/linux/cpuidle.h
@@ -33,6 +33,8 @@ struct cpuidle_state_usage {
 	unsigned long long	disable;
 	unsigned long long	usage;
 	unsigned long long	time; /* in US */
+	unsigned long long	high; /* Number of times it's been too deep */
+	unsigned long long	low;  /* Number of times it's been too shallow */
 #ifdef CONFIG_SUSPEND
 	unsigned long long	s2idle_usage;
 	unsigned long long	s2idle_time; /* in US */
Index: linux-pm/drivers/cpuidle/sysfs.c
===================================================================
--- linux-pm.orig/drivers/cpuidle/sysfs.c
+++ linux-pm/drivers/cpuidle/sysfs.c
@@ -301,6 +301,8 @@ define_show_state_str_function(name)
 define_show_state_str_function(desc)
 define_show_state_ull_function(disable)
 define_store_state_ull_function(disable)
+define_show_state_ull_function(high)
+define_show_state_ull_function(low)
 
 define_one_state_ro(name, show_state_name);
 define_one_state_ro(desc, show_state_desc);
@@ -310,6 +312,8 @@ define_one_state_ro(power, show_state_po
 define_one_state_ro(usage, show_state_usage);
 define_one_state_ro(time, show_state_time);
 define_one_state_rw(disable, show_state_disable, store_state_disable);
+define_one_state_ro(high, show_state_high);
+define_one_state_ro(low, show_state_low);
 
 static struct attribute *cpuidle_state_default_attrs[] = {
 	&attr_name.attr,
@@ -320,6 +324,8 @@ static struct attribute *cpuidle_state_d
 	&attr_usage.attr,
 	&attr_time.attr,
 	&attr_disable.attr,
+	&attr_high.attr,
+	&attr_low.attr,
 	NULL
 };
 
Index: linux-pm/Documentation/admin-guide/pm/cpuidle.rst
===================================================================
--- linux-pm.orig/Documentation/admin-guide/pm/cpuidle.rst
+++ linux-pm/Documentation/admin-guide/pm/cpuidle.rst
@@ -404,9 +404,17 @@ object corresponding to it, as follows:
 ``disable``
 	Whether or not this idle state is disabled.
 
+``high``
+	Total number of times this idle state had been asked for, but the
+	observed idle duration was too short to match its target residency.
+
 ``latency``
 	Exit latency of the idle state in microseconds.
 
+``low``
+	Total number of times this idle state had been asked for, but a deeper
+	idle state would have been a better match for the observed idle duration.
+
 ``name``
 	Name of the idle state.
 
Index: linux-pm/Documentation/ABI/testing/sysfs-devices-system-cpu
===================================================================
--- linux-pm.orig/Documentation/ABI/testing/sysfs-devices-system-cpu
+++ linux-pm/Documentation/ABI/testing/sysfs-devices-system-cpu
@@ -145,6 +145,8 @@ What:		/sys/devices/system/cpu/cpuX/cpui
 		/sys/devices/system/cpu/cpuX/cpuidle/stateN/power
 		/sys/devices/system/cpu/cpuX/cpuidle/stateN/time
 		/sys/devices/system/cpu/cpuX/cpuidle/stateN/usage
+		/sys/devices/system/cpu/cpuX/cpuidle/stateN/high
+		/sys/devices/system/cpu/cpuX/cpuidle/stateN/low
 Date:		September 2007
 KernelVersion:	v2.6.24
 Contact:	Linux power management list <linux-pm@vger.kernel.org>
@@ -166,6 +168,11 @@ Description:
 
 		usage: (RO) Number of times this state was entered (a count).
 
+		high: (RO) Number of times this state was entered, but the
+		      observed CPU target residency was too high for it (a count).
+
+		low: (RO) Number of times this state was entered, but the
+		     observed CPU target residency was too low for it (a count).
 
 What:		/sys/devices/system/cpu/cpuX/cpuidle/stateN/desc
 Date:		February 2008

RE: [PATCH] cpuidle: Add 'high' and 'low' idle state metrics

[Doug Smythies]

Hi Rafael,

On 2018.12.03 04:32 Rafael J. Wysocki wrote:

> Add two new metrics for CPU idle states, "high" and "low", to count
> the number of times the given state had been asked for (or entered
> from the kernel's perspective), but the observed idle duration turned
> out to be too high or too low for it (respectively).

I wonder about the "high" "low" terminology here.

> These mertics help to estimat the quality of the CPU idle governor
> in use.

Yes, very useful. Thanks.

Here the terms are mixed with "deep" and "shallow"

> +	unsigned long long	high; /* Number of times it's been too deep */
> +	unsigned long long	low;  /* Number of times it's been too shallow */

> +``high``
> +	Total number of times this idle state had been asked for, but the
> +	observed idle duration was too short to match its target residency.
> +

O.K. To avoid ambiguity, how about naming them "too_short" and "too_long"?

> +``low``
> +	Total number of times this idle state had been asked for, but a deeper
> +	idle state would have been a better match for the observed idle duration.

Even though I read the patch, by the time I actually looked at the numbers I had
forgotten the meaning. I had look at idle state 0 and 4 (the deepest for my computer)
to figure it out:

doug@s15:~/c$ cat /sys/devices/system/cpu/cpu0/cpuidle/state0/low
259871
doug@s15:~/c$ cat /sys/devices/system/cpu/cpu0/cpuidle/state0/high
0
doug@s15:~/c$ cat /sys/devices/system/cpu/cpu0/cpuidle/state4/low
0
doug@s15:~/c$ cat /sys/devices/system/cpu/cpu0/cpuidle/state4/high
5584

Because state 0 can not be too short and state 4 can not be too long.

... Doug

Re: [PATCH] cpuidle: Add 'high' and 'low' idle state metrics

[Rafael J. Wysocki]

On Thu, Dec 6, 2018 at 12:08 AM Doug Smythies <dsmythies@telus.net> wrote:
>
> Hi Rafael,
>
> On 2018.12.03 04:32 Rafael J. Wysocki wrote:
>
> > Add two new metrics for CPU idle states, "high" and "low", to count
> > the number of times the given state had been asked for (or entered
> > from the kernel's perspective), but the observed idle duration turned
> > out to be too high or too low for it (respectively).
>
> I wonder about the "high" "low" terminology here.

I took these names, because they are concise and simple.  I could use
"below" and "above" respectively I guess.  What about these?

> > These mertics help to estimat the quality of the CPU idle governor
> > in use.
>
> Yes, very useful. Thanks.
>
> Here the terms are mixed with "deep" and "shallow"
>
> > +     unsigned long long      high; /* Number of times it's been too deep */
> > +     unsigned long long      low;  /* Number of times it's been too shallow */
>
> > +``high``
> > +     Total number of times this idle state had been asked for, but the
> > +     observed idle duration was too short to match its target residency.
> > +
>
> O.K. To avoid ambiguity, how about naming them "too_short" and "too_long"?

Well, I guess the "too_" prefix may be skipped, so "short" and "long"
could be used, but that's about the state, not about the idle
duration.  The state cannot be "too long" or "too short", arguably.
It might be "too deep" or "too shallow".

> > +``low``
> > +     Total number of times this idle state had been asked for, but a deeper
> > +     idle state would have been a better match for the observed idle duration.
>
> Even though I read the patch, by the time I actually looked at the numbers I had
> forgotten the meaning. I had look at idle state 0 and 4 (the deepest for my computer)
> to figure it out:
>
> doug@s15:~/c$ cat /sys/devices/system/cpu/cpu0/cpuidle/state0/low
> 259871
> doug@s15:~/c$ cat /sys/devices/system/cpu/cpu0/cpuidle/state0/high
> 0
> doug@s15:~/c$ cat /sys/devices/system/cpu/cpu0/cpuidle/state4/low
> 0
> doug@s15:~/c$ cat /sys/devices/system/cpu/cpu0/cpuidle/state4/high
> 5584
>
> Because state 0 can not be too short and state 4 can not be too long.

Where "state" actually means "the target residency of state" I suppose? :-)

RE: [PATCH] cpuidle: Add 'high' and 'low' idle state metrics

[Doug Smythies]

On 2018.12.06 01:09 Rafael J. Wysocki wrote:
> On Thu, Dec 6, 2018 at 12:08 AM Doug Smythies <dsmythies@telus.net> wrote:
>> On 2018.12.03 04:32 Rafael J. Wysocki wrote:
>>
>>> Add two new metrics for CPU idle states, "high" and "low", to count
>>> the number of times the given state had been asked for (or entered
>>> from the kernel's perspective), but the observed idle duration turned
>>> out to be too high or too low for it (respectively).
>>
>> I wonder about the "high" "low" terminology here.
>
> I took these names, because they are concise and simple.  I could use
> "below" and "above" respectively I guess.  What about these?

I see you already sent a new patch with these names. Yes, myself I like
them better.

I am going to try to add these counts to my next sets of graphs.

... Doug