From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Doug Smythies" Subject: CPUs do not go idle - excessive energy consumption Date: Thu, 12 Oct 2017 08:28:14 -0700 Message-ID: <000001d3436e$b9b58160$2d208420$@net> Mime-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit Return-path: Received: from cmta20.telus.net ([209.171.16.93]:51683 "EHLO cmta20.telus.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752044AbdJLPgZ (ORCPT ); Thu, 12 Oct 2017 11:36:25 -0400 Content-Language: en-ca Sender: linux-pm-owner@vger.kernel.org List-Id: linux-pm@vger.kernel.org To: linux-pm@vger.kernel.org Cc: Doug Smythies Hi all, I am observing higher than nominal processor package power consumption, under some conditions. The worst case, so far, was an extra 6.3 watts or 25%, however more typically it is between 0 and 4 watts (over 1 minute sampling intervals). The simplest use case found so far is a single threaded 100% load on one CPU. The problem appears to be CPUs that should be idle and in a deep C state, sometimes are not. It is as though they have been forgotten and only when they hit the watchdog timer, or some other event, do they generally get sorted out. I have looked at kernels back as far as 4.5, and the issue is always present. I am looking for assistance investigating towards finding the root cause, because I am unfamiliar with the idle code and have made little (O.K. no) progress on my own. Details (for my test server, with an Intel i7-2600K, currently kernel 4.14-rc4): In an otherwise idle system, with one processor loaded 100%, processor power consumption is expected to be about 24 watts. However, sometimes power consumptions of up to about 28 watts have been observed. Of course, the first thought is that the extra power consumption is just due to some task, because the "otherwise idle" system isn't really completely so. If that were true, we would expect the intel_pstate CPU frequency scaling driver to be called often. It is not. Although it is difficult to correlate higher power samples from turbostat, with intel_pstate trace data, it seems they are due to occurrences of high load and long durations. For a few years now, we would never expect to see high load without high frequency calls to the driver. Also, typically for my test server, with many services turned off, other tasks should take very little run time per time slice. Examples (from trace data acquired and post processed with intel_pstate_tracer.py): CPU 3: Load 87.2%; Duration 4000.091 msec; Comment watchdog. CPU 5: Load 100%; Duration 2536.22 msec; Comment idle. CPU 3: Load 100%; Duration 1184.027 msec; Comment idle. (I have thousands more examples.) Higher power consumption with no user load at all seems to be rather rare, (Between 0 and 40 in one hour, arbitrary thresholds), and hard to detect via turbostat output. However, if the system is booted with intel_idle.max_cstate=1, then higher power consumption with no user load is very common, as are much wilder variations on power consumption (from ~9 watts to ~32 watts). For reference, the average rate is about 600 per hour with one 100% load, but it does vary. Note: While I tend to use taskset so as to know which CPU should be busy, the issue also occurs when taskset is not used. Note: I have arbitrarily set the threshold for this condition at >= 10% load and >= 250 milliseconds duration (the time between calls to the intel_pstate driver), and written a program to parse such samples out of the .csv files generated by intel_pstate_tracer.py. Note: This might be a separate problem, but the issue is made worse by taking CPUs off-line and then bringing them back on-line (I think). ... Doug