No c2-c7 states on core i7

No c2-c7 states on core i7

[Erik Slagter]

Hi there,

I already put on my flame retardant underwear, I know this is a FAQ, but 
I really need a bit of a jumpstart, google won't tell me where to start, 
so please be gentle ;-)

I bought a Gigabyte EX58-UD3R motherboard recently and put a core i7 920 
on it. This seems to be a fairly common combination.

In short: I know the core i7 does implement several C-states, from 
memory c1, c1e, c3, c6 and c7. Linux (vanilla, 2.6.31.6) does not 
recognise any of them.

The kernel configuration is very similar to the one I am running on my 
laptop with a recent mobile core2duo processor, and this one reports 
C1,C2,C3 (although it should also report yet another C state, but I 
guess that one is disabled by the bios or something alike).

Is this normal (WIP?)?

If not, where should I start debugging? I know there is "something" with 
decoding the DSDT table, but what should I look for?

I seem to remember that with an earlier linux version (before 2.6.30.5) 
it actually did work, but I am not completely sure.

Thanks for your help in advance!

Additional info that might be useful:

- all options related to power saving and C-states are ENABLED in the BIOS
- all options related to power management and idling are ENABLED in the 
kernel, which runs in 64 bits mode
- frequency switching runs fine using acpi-cpufreq and ondemand governor

If I compare the dmesg from both computers after booting, the laptop 
says "ACPI: CPU1 (power states: C1[C1] C2[C2] C3[C3])" at some point, 
this message is not output at all by the server.

Output from proc/acpi/processor/CPU0/info
processor id:            0
acpi id:                 0
bus mastering control:   yes
power management:        no
throttling control:      yes
limit interface:         yes

Output from proc/acpi/processor/CPU0/power
active state:            C0
max_cstate:              C8
maximum allowed latency: 2000000000 usec
states:
     C1:                  type[C1] promotion[--] demotion[--] 
latency[000] usage[00000000] duration[00000000000000000000]

Re: No c2-c7 states on core i7

[Erik Slagter]

As always this appears to be a BIOS problem (bug?)

As soon as you change the BASECLOCK setting, all C states are withdrawn. 
I think it's really idiotic, and I am really going to try to enable them 
anyway.

Interesting fact: the system uses almost exactly the same amount of 
power when C states are enabled by the BIOS and when they are disabled. 
Makes you wonder whether they actually do something at all. The linux 
counters really say they're called, anyway.

So if you don't want a power-wasting board, don't buy a Gigabyte EX58-UD3R.

c1/c1e/c3/c6/c7 on linux

[Erik Slagter]

Ik have two machines, a dell laptop with a mobile core2duo T9300 and a 
gigabyte desktop with a core i7 920. Both advertise all of these 
C-states c1/c1e/c3/c6/c7 in the specs (although the core2duo doesn't 
have c7 I believe).

It's not clear at all how linux handles these. The kernel on the T9300 
says it uesde C1/C2/C3 of "type" C1/C2/C3, the core i7 says exactly the 
same.

Is this a WiP? A bios issue? It looks like I am missing some states?

The kernel I am using is 2.6.31.6

Re: No c2-c7 states on core i7

[ykzhao]

On Fri, 2009-11-20 at 22:28 +0800, Erik Slagter wrote:
> Hi there,
> 
> I already put on my flame retardant underwear, I know this is a FAQ, but 
> I really need a bit of a jumpstart, google won't tell me where to start, 
> so please be gentle ;-)
> 
> I bought a Gigabyte EX58-UD3R motherboard recently and put a core i7 920 
> on it. This seems to be a fairly common combination.
> 
> In short: I know the core i7 does implement several C-states, from 
> memory c1, c1e, c3, c6 and c7. Linux (vanilla, 2.6.31.6) does not 
> recognise any of them.
> 
> The kernel configuration is very similar to the one I am running on my 
> laptop with a recent mobile core2duo processor, and this one reports 
> C1,C2,C3 (although it should also report yet another C state, but I 
> guess that one is disabled by the bios or something alike).
> 
> Is this normal (WIP?)?
> 
> If not, where should I start debugging? I know there is "something" with 
> decoding the DSDT table, but what should I look for?
> 
> I seem to remember that with an earlier linux version (before 2.6.30.5) 
> it actually did work, but I am not completely sure.
> 
> Thanks for your help in advance!
> 
> Additional info that might be useful:
> 
> - all options related to power saving and C-states are ENABLED in the BIOS
> - all options related to power management and idling are ENABLED in the 
> kernel, which runs in 64 bits mode
> - frequency switching runs fine using acpi-cpufreq and ondemand governor
> 
> If I compare the dmesg from both computers after booting, the laptop 
> says "ACPI: CPU1 (power states: C1[C1] C2[C2] C3[C3])" at some point, 
> this message is not output at all by the server.

Will you please attach the output of acpidump on the machine with the
core i7 cpu?
Please attach the output of every file
under /sys/firmware/acpi/tables/dynamic/SSDT*? 
   cat /sys/firmware/acpi/tables/dynamic/SSDT1 > ssdt1

Please also attach the output of /proc/cpuinfo.

Thanks.
> 
> Output from proc/acpi/processor/CPU0/info
> processor id:            0
> acpi id:                 0
> bus mastering control:   yes
> power management:        no
> throttling control:      yes
> limit interface:         yes
> 
> Output from proc/acpi/processor/CPU0/power
> active state:            C0
> max_cstate:              C8
> maximum allowed latency: 2000000000 usec
> states:
>      C1:                  type[C1] promotion[--] demotion[--] 
> latency[000] usage[00000000] duration[00000000000000000000]
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: No c2-c7 states on core i7

[Erik Slagter]

On 23-11-09 04:05, ykzhao wrote:
> On Fri, 2009-11-20 at 22:28 +0800, Erik Slagter wrote:
>> Hi there,
>>
>> I already put on my flame retardant underwear, I know this is a FAQ, but
>> I really need a bit of a jumpstart, google won't tell me where to start,
>> so please be gentle ;-)
>>
>> I bought a Gigabyte EX58-UD3R motherboard recently and put a core i7 920
>> on it. This seems to be a fairly common combination.
>>
>> In short: I know the core i7 does implement several C-states, from
>> memory c1, c1e, c3, c6 and c7. Linux (vanilla, 2.6.31.6) does not
>> recognise any of them.
>>
>> The kernel configuration is very similar to the one I am running on my
>> laptop with a recent mobile core2duo processor, and this one reports
>> C1,C2,C3 (although it should also report yet another C state, but I
>> guess that one is disabled by the bios or something alike).
>>
>> Is this normal (WIP?)?
>>
>> If not, where should I start debugging? I know there is "something" with
>> decoding the DSDT table, but what should I look for?
>>
>> I seem to remember that with an earlier linux version (before 2.6.30.5)
>> it actually did work, but I am not completely sure.
>>
>> Thanks for your help in advance!
>>
>> Additional info that might be useful:
>>
>> - all options related to power saving and C-states are ENABLED in the BIOS
>> - all options related to power management and idling are ENABLED in the
>> kernel, which runs in 64 bits mode
>> - frequency switching runs fine using acpi-cpufreq and ondemand governor
>>
>> If I compare the dmesg from both computers after booting, the laptop
>> says "ACPI: CPU1 (power states: C1[C1] C2[C2] C3[C3])" at some point,
>> this message is not output at all by the server.
>>
>> Output from proc/acpi/processor/CPU0/info
>> processor id:            0
>> acpi id:                 0
>> bus mastering control:   yes
>> power management:        no
>> throttling control:      yes
>> limit interface:         yes
>>
>> Output from proc/acpi/processor/CPU0/power
>> active state:            C0
>> max_cstate:              C8
>> maximum allowed latency: 2000000000 usec
>> states:
>>       C1:                  type[C1] promotion[--] demotion[--]
>> latency[000] usage[00000000] duration[00000000000000000000]
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html

 > Will you please attach the output of acpidump on the machine with the
 > core i7 cpu?
 > Please attach the output of every file
 > under /sys/firmware/acpi/tables/dynamic/SSDT*?
 >     cat /sys/firmware/acpi/tables/dynamic/SSDT1>  ssdt1
 >
 > Please also attach the output of /proc/cpuinfo.

Hi,

Thank you for your attention. This specific issue has already been 
solved for me, it turns out be yet another bios issue (as I have 
experienced before). Enable base clock setting and all C states are gone!

But now I have a few other issues that maybe you can help me with:

- How are intel core i7 C states mapped to linux C states? Both my 
laptop (core2duo T9300) and desktop (core i7 920) should have 
c1-c3-c6-c7, but on both machines linux only sees c1-c2-c3.

- On said desktop, I see no difference in power consumption with c 
states enabled or disabled; if they're enabled (and base clock control 
is off...) linux is using c3 almost all of the time, but power 
consumption remains the same (about 110 watts for the complete 
motherboard, is that normal?).

- From a quick inspection of the acpi tables (I am not quite an expert 
on this...) it looks like a number of _CST objects are exported with 
default bios settings, while with base clock control enabled, these are 
gone. I would to try (yes I know, not recommended, etc.) to copy the 
_CST objects from the one boot instance's tables to the other. Now I 
think of it, it would probably even suffice to boot one time with 
baseclock control disabled, record the acpi tables (SSDT?) and then make 
the kernel use that in subsequent boots with the base clock control 
enabled (with all other bios settings the same, off course). Is that 
difficult to achieve? I seem to remember that's possible one way or another.

Thanks for your help!

P.S. I am going to have a look at the /sys/* files you mentioned anyway, 
I didn't even know they were there!

Re: No c2-c7 states on core i7

[Len Brown]

> > Will you please attach the output of acpidump on the machine with the
> > core i7 cpu?
> > Please attach the output of every file
> > under /sys/firmware/acpi/tables/dynamic/SSDT*?
> >     cat /sys/firmware/acpi/tables/dynamic/SSDT1>  ssdt1
> >
> > Please also attach the output of /proc/cpuinfo.

> Thank you for your attention. This specific issue has already been solved for
> me, it turns out be yet another bios issue (as I have experienced before).
> Enable base clock setting and all C states are gone!
> 
> But now I have a few other issues that maybe you can help me with:
> 
> - How are intel core i7 C states mapped to linux C states? Both my laptop
> (core2duo T9300) and desktop (core i7 920) should have c1-c3-c6-c7, but on
> both machines linux only sees c1-c2-c3.

It is up to the BIOS to map hardware C-states to software visible 
C-states.  How the BIOS does that would be answered by the information
that Yakui requested above.

In particular, most BIOS today use the _CST object in the DSDT
or in an SSDT.

> - On said desktop, I see no difference in power consumption with c states
> enabled or disabled; if they're enabled (and base clock control is off...)
> linux is using c3 almost all of the time, but power consumption remains the
> same (about 110 watts for the complete motherboard, is that normal?).

No, that isn't normal. C-states on modern processors generally
save a lot of energy.

If you run powertop and you find that you are over 99% idle
and you save no energy compared to when you are 0% idle
(say a copy of "cat /dev/zero > /dev/null" for each core)
then something is wrong with your system.

> - From a quick inspection of the acpi tables (I am not quite an expert on
> this...) it looks like a number of _CST objects are exported with default bios
> settings, while with base clock control enabled, these are gone. I would to
> try (yes I know, not recommended, etc.) to copy the _CST objects from the one
> boot instance's tables to the other. Now I think of it, it would probably even
> suffice to boot one time with baseclock control disabled, record the acpi
> tables (SSDT?) and then make the kernel use that in subsequent boots with the
> base clock control enabled (with all other bios settings the same, off
> course). Is that difficult to achieve? I seem to remember that's possible one
> way or another.

It is possible, but as soon as you reverse engineer and over-ride
something in the BIOS, you are on very thin ice.  Presumably
the BIOS engineer made a concious decision to disable C-states 
when you over-clock your board and had a reason to do so.

Maybe the more important question is what measurable benefit
you get when you over-clock your board, and if you really need that...

-Len Brown
Intel Open Source Technology Center

Re: No c2-c7 states on core i7

[Erik Slagter]

[-- Attachment #1: Type: text/plain, Size: 1232 bytes --]

I am getting more and puzzled about this system/board/bios with the 
minute. I hope your guys will help me find out what's happening!

>>> Will you please attach the output of acpidump on the machine with the
>>> core i7 cpu?
>>> Please attach the output of every file
>>> under /sys/firmware/acpi/tables/dynamic/SSDT*?
>>>      cat /sys/firmware/acpi/tables/dynamic/SSDT1>   ssdt1
>>>
>>> Please also attach the output of /proc/cpuinfo.

I have attached a tar file with these tables from acpidump and from the 
sys directory, because acpidump yields two tables more than available in 
the directory /sys/firmware/acpi/tables, also the directory 
/sys/firmware/acpi/tables/dynamic is empty.

I have included both versions from "default" bios settings and slightly 
overclocked settings (where the _CST's are gone).

Please also have a look on this little table I created from tests, imho 
that cpufreq does nothing when idle (which I'd expect when c states are 
enabled and the system is using these, but not when no c states are 
availabled...)

Also it suggests that the presence and or usage of c states doesn't 
matter much in power consumption (3 watts in idle...)

More in the next message to keep things a little structured.

[-- Attachment #2: acpi.tgz --]
[-- Type: application/x-compressed-tar, Size: 28204 bytes --]

[-- Attachment #3: acpi --]
[-- Type: text/plain, Size: 457 bytes --]

results on very idle system, booted with init=/bin/sh

frequency							c states					idle cpufreq=lowest freq		idle cpufreq=highest freq		full load cpufreq=ondemand
overclocked 3.4 Ghz					disabled by bios			108								111								190
2.6 Ghz								enabled and active			100								100								170
default bios settings 2.6 Ghz		default disabled in setup	90								92								150
default bios settings 2.6 Ghz		enabled by user in setup	87								87								150

Re: No c2-c7 states on core i7

[Erik Slagter]

 > No, that isn't normal. C-states on modern processors generally
 > save a lot of energy.
 >
 > If you run powertop and you find that you are over 99% idle
 > and you save no energy compared to when you are 0% idle
 > (say a copy of "cat /dev/zero>  /dev/null" for each core)
 > then something is wrong with your system.

I just removed a large story here, I guess the table from my other 
message is a lot more informative.

 > It is possible, but as soon as you reverse engineer and over-ride
 > something in the BIOS, you are on very thin ice.  Presumably
 > the BIOS engineer made a concious decision to disable C-states
 > when you over-clock your board and had a reason to do so.

As long as nothing gets fried that's no problem for me.

 > Maybe the more important question is what measurable benefit
 > you get when you over-clock your board, and if you really need that...

I can understand your doubts on this matter, but I think I do have a 
legitimate reasoning. This is a server that almost all of the time does 
next to nothing. Load 0.02 or similar. It needs to be running 24/24 
though because it receives e-mail and answers the telephone. So that's 
why I want it to be low on power usage. On the other hand I need to 
transcode movie clips to h264 very regularly. I can use every 4*2 core 
for the process using x264 and indeed it works very fast. Also I noticed 
that every mhz higher clock means shorter encoding time, all 
(virtual-)cores get completely loaded. The "normal" speed of the 920 is 
2.8 Ghz (or in fact 2.63 Ghz) and a change to 3.4 Ghz really does make a 
difference in encoding speed, theoretically 30%, in practise even more.

Intel would really make me happy if they would design a processor with 
four or more cores and a chipset that would implement C7 and also could 
scale from 100 Mhz to 3.6 Ghz, in two or three steps, I wouldn't mind, 
and then would use something like 20 watts in idle, like my laptop does.

Re: No c2-c7 states on core i7

[Jindrich Makovicka]

On Mon, 23 Nov 2009 20:26:53 +0100
Erik Slagter <erik@slagter.name> wrote:

>  > No, that isn't normal. C-states on modern processors generally
>  > save a lot of energy.
>  >
>  > If you run powertop and you find that you are over 99% idle
>  > and you save no energy compared to when you are 0% idle
>  > (say a copy of "cat /dev/zero>  /dev/null" for each core)
>  > then something is wrong with your system.
> 
> I just removed a large story here, I guess the table from my other 
> message is a lot more informative.
> 
>  > It is possible, but as soon as you reverse engineer and over-ride
>  > something in the BIOS, you are on very thin ice.  Presumably
>  > the BIOS engineer made a concious decision to disable C-states
>  > when you over-clock your board and had a reason to do so.
> 
> As long as nothing gets fried that's no problem for me.
> 
>  > Maybe the more important question is what measurable benefit
>  > you get when you over-clock your board, and if you really need
>  > that...
> 
> I can understand your doubts on this matter, but I think I do have a 
> legitimate reasoning. This is a server that almost all of the time
> does next to nothing. Load 0.02 or similar. It needs to be running
> 24/24 though because it receives e-mail and answers the telephone. So
> that's why I want it to be low on power usage. On the other hand I
> need to transcode movie clips to h264 very regularly. I can use every
> 4*2 core for the process using x264 and indeed it works very fast.
> Also I noticed that every mhz higher clock means shorter encoding
> time, all (virtual-)cores get completely loaded. The "normal" speed
> of the 920 is 2.8 Ghz (or in fact 2.63 Ghz) and a change to 3.4 Ghz
> really does make a difference in encoding speed, theoretically 30%,
> in practise even more.
> 
> Intel would really make me happy if they would design a processor
> with four or more cores and a chipset that would implement C7 and
> also could scale from 100 Mhz to 3.6 Ghz, in two or three steps, I
> wouldn't mind, and then would use something like 20 watts in idle,
> like my laptop does.

Most of the overclockers disable C3/C6/C7 anyway for stability reasons,
probably because sudden changes in core voltage disturb the CPU
operation. My i5 Lynnfield desktop was actually unusable with C3/C6/C7
even with default settings (fsck reporting random errors etc.).  I
ended up disabling the higher C-states and just raised the Base clock
so I do not need the Turbo mode. As long as you do not disable C1,
the difference in idle power consumption is rather small.

-- 
Jindrich Makovicka

Re: No c2-c7 states on core i7

[Erik Slagter]

> Most of the overclockers disable C3/C6/C7 anyway for stability reasons,
> probably because sudden changes in core voltage disturb the CPU
> operation. My i5 Lynnfield desktop was actually unusable with C3/C6/C7
> even with default settings (fsck reporting random errors etc.).  I
> ended up disabling the higher C-states and just raised the Base clock
> so I do not need the Turbo mode. As long as you do not disable C1,
> the difference in idle power consumption is rather small.

This is very interesting.

I really would not want to disable the higher C states because according 
to intel marketing, but also according to tech sheets these promise to 
save a lot of energy by progressively completely turning off parts of 
the core(s).

What you describe is actually exactly what I see, power usage differs 
greatly from C0 to C1E (between 150 and 100 wats, 50%...), after that, 
in higher C states, the power usage doesn't change that much.

It seems I have a combination of a very stable power supply and a very 
stable motherboard because I have all features enabled that make the 
power supply fluctuate (c3-c7, cpufreq), and even more, until yesterday 
I used to use overclocking to 3.2 Ghz with very slightly raised cpu 
voltage, that combination has been running for months now without any 
flaw or any related kernel message.

Yesterday I raised the clock to 3.4 Ghz and I set all voltages to 
"normal" and it seems to run fine. I will have to check under really 
heavy load yet, I will do that tonight. My motherboard has an option 
"power line load calibration", I guess that has something to do with it :-)

Interesting: I pulled out a "modern" (but cheapo, no fans) pcie vga card 
and replaced it with a very old pci card (I guess from 1996) and that 
saves me 20 watts! That's more power than I problably ever could save 
using C states!

BTW on my ancient athlon-mp board, I could enable c2/c3 (although not 
supported by the bios) and that would save about ~40 watts of power. But 
it looks I won't see such a difference on modern CPU's, maybe because of 
the dynamic frequency scaling and enhanced c1? Is there anyone who can 
give me an indication of what to expect in powersavings by c2-c7 on 
modern cpu's?

Re: No c2-c7 states on core i7

[Len Brown]

> Yesterday I raised the clock to 3.4 Ghz and I set all voltages to "normal" and
> it seems to run fine. I will have to check under really heavy load yet, I will
> do that tonight. My motherboard has an option "power line load calibration", I
> guess that has something to do with it :-)
> 
> Interesting: I pulled out a "modern" (but cheapo, no fans) pcie vga card and
> replaced it with a very old pci card (I guess from 1996) and that saves me 20
> watts! That's more power than I problably ever could save using C states!

Yep, beware graphics card power.

> BTW on my ancient athlon-mp board, I could enable c2/c3 (although not
> supported by the bios) and that would save about ~40 watts of power. But it
> looks I won't see such a difference on modern CPU's, maybe because of the
> dynamic frequency scaling and enhanced c1? Is there anyone who can give me an
> indication of what to expect in powersavings by c2-c7 on modern cpu's?

On the mobile parts they used to publish the power limits for C-states,
but I don't see these numbers in the core i7 datasheet...

However, many have done comparisons of what you might expect for
idle vs loaded power.  eg.
http://www.anandtech.com/cpuchipsets/showdoc.aspx?i=3634&p=17

In general, I don't view it as a Linux bug that the MB's BIOS
disables c-states when it is over-clocked.  Indeed, it is
quite possibly a BIOS feature.

Len Brown, Intel Open Source Technology Center