Bug Report for Linux Kernel 3.x

Bug Report for Linux Kernel 3.x

[Steevven1]

Hello,

I would like to report what myself and several others believe to be a
serious bug affecting all Linux kernels 3.0 and up, and none below 3.0
(2.6.x and lower).

One-sentence description: On certain hardware, the 3.x series kernels
drain a considerable amount more power than the 2.6-series kernels, up
to 40% more, even when idle.

The specific hardware I have personally tested is a Lenovo ThinkPad
X220 with an Intel SandyBridge i7 2620M processor. We think that this
bug is POSSIBLY specific to this processor, or family of processors,
commonly used in notebooks. A complete thread of mine and others'
results, evidence, and troubleshooting process is located at:
http://ubuntuforums.org/showthread.php?t=1822629

We have tried several different 3.0 and 3.1-series kernels, all with
the same problems. One part of the problem was that the following
(revealed by powertop) was causing excessive processor wakeups
compared to the 2.6-series kernels: "[Rescheduling interrupts] <kernel
IPI>"

Someone much more advanced than I figured out how to eliminate this
problem and recompiled the kernel (which he posted in the thread I
linked to previously), but power usage is still improved only
slightly, nowhere near kernel 2.6 levels.

There is a lot more information I could give you here, but all of it
is contained within the thread I linked to. If you have any specific
questions, I will answer as quickly as possible.

Thanks a lot for your assistance. Please confirm that you received this email.

Steven Keys!

Re: Bug Report for Linux Kernel 3.x

[Frederic Weisbecker]

On Tue, Aug 23, 2011 at 07:19:35PM -0400, Steevven1 wrote:
> Hello,
> 
> I would like to report what myself and several others believe to be a
> serious bug affecting all Linux kernels 3.0 and up, and none below 3.0
> (2.6.x and lower).
> 
> One-sentence description: On certain hardware, the 3.x series kernels
> drain a considerable amount more power than the 2.6-series kernels, up
> to 40% more, even when idle.
> 
> The specific hardware I have personally tested is a Lenovo ThinkPad
> X220 with an Intel SandyBridge i7 2620M processor. We think that this
> bug is POSSIBLY specific to this processor, or family of processors,
> commonly used in notebooks. A complete thread of mine and others'
> results, evidence, and troubleshooting process is located at:
> http://ubuntuforums.org/showthread.php?t=1822629
> 
> We have tried several different 3.0 and 3.1-series kernels, all with
> the same problems. One part of the problem was that the following
> (revealed by powertop) was causing excessive processor wakeups
> compared to the 2.6-series kernels: "[Rescheduling interrupts] <kernel
> IPI>"
> 
> Someone much more advanced than I figured out how to eliminate this
> problem and recompiled the kernel (which he posted in the thread I
> linked to previously), but power usage is still improved only
> slightly, nowhere near kernel 2.6 levels.
> 
> There is a lot more information I could give you here, but all of it
> is contained within the thread I linked to. If you have any specific
> questions, I will answer as quickly as possible.
> 
> Thanks a lot for your assistance. Please confirm that you received this email.
> 
> Steven Keys!

Hi Steve,


You said the problem with rescheduling interrupt disappeared but you
haven't said why. After reading the thread it seems it was about
Sparse irq config beeing enabled? And that disappeared after
sparse irq got disabled?

If so that looks like a first bug.

Also now that you don't see that problem anymore but still
a problem with power consumption, what does powertop report to
you?

Thanks.

Re: Bug Report for Linux Kernel 3.x

[Steevven1]

QUICK NOTE - Thanks for replying directly to me as well as the mailing
list. I am NOT on that mailing list. Please continue to do this.

That is correct about sparse irq. That was causing the first problem,
and solving that did in fact result in better power performance, but
it was apparently NOT the bulk of the problem.

We still have no diagnosis at all for the apparently still-present
other (mystery) problem. Powertop now reports to me very low processor
wakeups (basically the same as the 2.6-series kernels), which
indicates to me that the problem is PROBABLY not related to processor
wakeups. I am by no means an expert. The top wakeup-causers with the
modified 3.0 kernel (no sparse irq) and an idle system are "[iwlagn]
<interrupt>" (wifi), "[i915] <interrupt>" (I don't know what this is),
and "[kernel scheduler] Load balancing tick"

P-states/frequencies are handled about the same as with the 2.6
kernels, so I don't see a problem there either.

What else can I check? It seems like powertop is telling me nothing
about this mystery power drain.

Thanks for the reply,
SK!


On Tue, Aug 23, 2011 at 8:11 PM, Frederic Weisbecker <fweisbec@gmail.com> wrote:
> On Tue, Aug 23, 2011 at 07:19:35PM -0400, Steevven1 wrote:
>> Hello,
>>
>> I would like to report what myself and several others believe to be a
>> serious bug affecting all Linux kernels 3.0 and up, and none below 3.0
>> (2.6.x and lower).
>>
>> One-sentence description: On certain hardware, the 3.x series kernels
>> drain a considerable amount more power than the 2.6-series kernels, up
>> to 40% more, even when idle.
>>
>> The specific hardware I have personally tested is a Lenovo ThinkPad
>> X220 with an Intel SandyBridge i7 2620M processor. We think that this
>> bug is POSSIBLY specific to this processor, or family of processors,
>> commonly used in notebooks. A complete thread of mine and others'
>> results, evidence, and troubleshooting process is located at:
>> http://ubuntuforums.org/showthread.php?t=1822629
>>
>> We have tried several different 3.0 and 3.1-series kernels, all with
>> the same problems. One part of the problem was that the following
>> (revealed by powertop) was causing excessive processor wakeups
>> compared to the 2.6-series kernels: "[Rescheduling interrupts] <kernel
>> IPI>"
>>
>> Someone much more advanced than I figured out how to eliminate this
>> problem and recompiled the kernel (which he posted in the thread I
>> linked to previously), but power usage is still improved only
>> slightly, nowhere near kernel 2.6 levels.
>>
>> There is a lot more information I could give you here, but all of it
>> is contained within the thread I linked to. If you have any specific
>> questions, I will answer as quickly as possible.
>>
>> Thanks a lot for your assistance. Please confirm that you received this email.
>>
>> Steven Keys!
>
> Hi Steve,
>
>
> You said the problem with rescheduling interrupt disappeared but you
> haven't said why. After reading the thread it seems it was about
> Sparse irq config beeing enabled? And that disappeared after
> sparse irq got disabled?
>
> If so that looks like a first bug.
>
> Also now that you don't see that problem anymore but still
> a problem with power consumption, what does powertop report to
> you?
>
> Thanks.
>

Re: Bug Report for Linux Kernel 3.x

[Frederic Weisbecker]

(Please don't top-post)

On Tue, Aug 23, 2011 at 08:21:50PM -0400, Steevven1 wrote:
> QUICK NOTE - Thanks for replying directly to me as well as the mailing
> list. I am NOT on that mailing list. Please continue to do this.
> 
> That is correct about sparse irq. That was causing the first problem,
> and solving that did in fact result in better power performance, but
> it was apparently NOT the bulk of the problem.
> 
> We still have no diagnosis at all for the apparently still-present
> other (mystery) problem. Powertop now reports to me very low processor
> wakeups (basically the same as the 2.6-series kernels), which
> indicates to me that the problem is PROBABLY not related to processor
> wakeups. I am by no means an expert. The top wakeup-causers with the
> modified 3.0 kernel (no sparse irq) and an idle system are "[iwlagn]
> <interrupt>" (wifi), "[i915] <interrupt>" (I don't know what this is),
> and "[kernel scheduler] Load balancing tick"
> 
> P-states/frequencies are handled about the same as with the 2.6
> kernels, so I don't see a problem there either.
> 
> What else can I check? It seems like powertop is telling me nothing
> about this mystery power drain.

Well, if powertop has nothing to tell about that, I have no clue where
to go. I'm adding more people in Cc in the hope they can be more
helpful.

> 
> 
> On Tue, Aug 23, 2011 at 8:11 PM, Frederic Weisbecker <fweisbec@gmail.com> wrote:
> > On Tue, Aug 23, 2011 at 07:19:35PM -0400, Steevven1 wrote:
> >> Hello,
> >>
> >> I would like to report what myself and several others believe to be a
> >> serious bug affecting all Linux kernels 3.0 and up, and none below 3.0
> >> (2.6.x and lower).
> >>
> >> One-sentence description: On certain hardware, the 3.x series kernels
> >> drain a considerable amount more power than the 2.6-series kernels, up
> >> to 40% more, even when idle.
> >>
> >> The specific hardware I have personally tested is a Lenovo ThinkPad
> >> X220 with an Intel SandyBridge i7 2620M processor. We think that this
> >> bug is POSSIBLY specific to this processor, or family of processors,
> >> commonly used in notebooks. A complete thread of mine and others'
> >> results, evidence, and troubleshooting process is located at:
> >> http://ubuntuforums.org/showthread.php?t=1822629
> >>
> >> We have tried several different 3.0 and 3.1-series kernels, all with
> >> the same problems. One part of the problem was that the following
> >> (revealed by powertop) was causing excessive processor wakeups
> >> compared to the 2.6-series kernels: "[Rescheduling interrupts] <kernel
> >> IPI>"
> >>
> >> Someone much more advanced than I figured out how to eliminate this
> >> problem and recompiled the kernel (which he posted in the thread I
> >> linked to previously), but power usage is still improved only
> >> slightly, nowhere near kernel 2.6 levels.
> >>
> >> There is a lot more information I could give you here, but all of it
> >> is contained within the thread I linked to. If you have any specific
> >> questions, I will answer as quickly as possible.
> >>
> >> Thanks a lot for your assistance. Please confirm that you received this email.
> >>
> >> Steven Keys!
> >
> > Hi Steve,
> >
> >
> > You said the problem with rescheduling interrupt disappeared but you
> > haven't said why. After reading the thread it seems it was about
> > Sparse irq config beeing enabled? And that disappeared after
> > sparse irq got disabled?
> >
> > If so that looks like a first bug.
> >
> > Also now that you don't see that problem anymore but still
> > a problem with power consumption, what does powertop report to
> > you?
> >
> > Thanks.
> >

Re: Bug Report for Linux Kernel 3.x

[Arjan van de Ven]

On 8/24/2011 6:58 AM, Frederic Weisbecker wrote:
> (Please don't top-post)
>
> On Tue, Aug 23, 2011 at 08:21:50PM -0400, Steevven1 wrote:
>> QUICK NOTE - Thanks for replying directly to me as well as the mailing
>> list. I am NOT on that mailing list. Please continue to do this.
>>
>> That is correct about sparse irq. That was causing the first problem,
>> and solving that did in fact result in better power performance, but
>> it was apparently NOT the bulk of the problem.
>>
>> We still have no diagnosis at all for the apparently still-present
>> other (mystery) problem. Powertop now reports to me very low processor
>> wakeups (basically the same as the 2.6-series kernels), which
>> indicates to me that the problem is PROBABLY not related to processor
>> wakeups. I am by no means an expert. The top wakeup-causers with the
>> modified 3.0 kernel (no sparse irq) and an idle system are "[iwlagn]
>> <interrupt>" (wifi), "[i915]<interrupt>" (I don't know what this is),
>> and "[kernel scheduler] Load balancing tick"
>>
>> P-states/frequencies are handled about the same as with the 2.6
>> kernels, so I don't see a problem there either.
>>
>> What else can I check? It seems like powertop is telling me nothing
>> about this mystery power drain.
> Well, if powertop has nothing to tell about that, I have no clue where
> to go. I'm adding more people in Cc in the hope they can be more
> helpful.
>


make sure to use powertop 1.98

Re: Bug Report for Linux Kernel 3.x

[Anca Emanuel]

Hi all,


Make sure you read this:

Tweaks To Extend The Battery Life Of Intel Linux Notebooks
http://www.phoronix.com/scan.php?page=article&item=intel_i915_power&num=1

Linux 3.1 Kernel Draws More Power With Another Regression
http://www.phoronix.com/scan.php?page=article&item=linux_31_power_regress&num=1

Re: Bug Report for Linux Kernel 3.x

[Steevven1]

On Wed, Aug 24, 2011 at 11:52 AM, Anca Emanuel <anca.emanuel@gmail.com> wrote:
> Hi all,
>
>
> Make sure you read this:
>
> Tweaks To Extend The Battery Life Of Intel Linux Notebooks
> http://www.phoronix.com/scan.php?page=article&item=intel_i915_power&num=1
>
> Linux 3.1 Kernel Draws More Power With Another Regression
> http://www.phoronix.com/scan.php?page=article&item=linux_31_power_regress&num=1
>

VERY interesting! The second link seems to imply that Phoronix has
independently found this same bug that we've been talking about in my
originally-linked ubuntuforums discussion. Specifically, the
interesting quote is:

"Besides the new Linux 3.1 kernel power regression, there's also a
power regression introduced in the Linux 3.0 kernel that has
previously not been talked about on Phoronix. The Linux 3.0 power draw
is up by 24% over the Linux 2.6.39 kernel."

Re: Bug Report for Linux Kernel 3.x

[Arjan van de Ven]

On 8/24/2011 9:44 AM, Steevven1 wrote:

> VERY interesting! The second link seems to imply that Phoronix has
> independently found this same bug that we've been talking about in my
> originally-linked ubuntuforums discussion. Specifically, the
> interesting quote is:
>
> "Besides the new Linux 3.1 kernel power regression, there's also a
> power regression introduced in the Linux 3.0 kernel that has
> previously not been talked about on Phoronix. The Linux 3.0 power draw
> is up by 24% over the Linux 2.6.39 kernel."


sounds like good news; a 24% regression is MUCH easier to bisect than a 
0.05% regression ;-)

Re: Bug Report for Linux Kernel 3.x

[Steevven1]

On Wed, Aug 24, 2011 at 12:46 PM, Arjan van de Ven
<arjan@linux.intel.com> wrote:
> On 8/24/2011 9:44 AM, Steevven1 wrote:
>
>> VERY interesting! The second link seems to imply that Phoronix has
>> independently found this same bug that we've been talking about in my
>> originally-linked ubuntuforums discussion. Specifically, the
>> interesting quote is:
>>
>> "Besides the new Linux 3.1 kernel power regression, there's also a
>> power regression introduced in the Linux 3.0 kernel that has
>> previously not been talked about on Phoronix. The Linux 3.0 power draw
>> is up by 24% over the Linux 2.6.39 kernel."
>
>
> sounds like good news; a 24% regression is MUCH easier to bisect than a
> 0.05% regression ;-)
>
>


Is there any news on this bug, or is there an official kernel.org
place to track it?

Here are the only other two places I know of where this bug is being tracked:
http://ubuntuforums.org/showthread.php?t=1822629 (as linked before)
https://bugs.launchpad.net/ubuntu/+bug/834037 (Ubuntu bug tracking system)

Re: Bug Report for Linux Kernel 3.x

[Dave Jones]

On Tue, Aug 23, 2011 at 07:19:35PM -0400, Steevven1 wrote:
 > Hello,
 > 
 > I would like to report what myself and several others believe to be a
 > serious bug affecting all Linux kernels 3.0 and up, and none below 3.0
 > (2.6.x and lower).
 > 
 > One-sentence description: On certain hardware, the 3.x series kernels
 > drain a considerable amount more power than the 2.6-series kernels, up
 > to 40% more, even when idle.
 >
 > The specific hardware I have personally tested is a Lenovo ThinkPad
 > X220 with an Intel SandyBridge i7 2620M processor.
 
Sounds like https://bugzilla.redhat.com/show_bug.cgi?id=727579
Try booting with i915.i915_enable_rc6=1
 
	Dave

Re: Bug Report for Linux Kernel 3.x

[Steevven1]

Thanks for the comment. I have already tried this with several
different 3-series kernels. There was no effect on power consumption
or output from powertop.

SK!


On Tue, Aug 23, 2011 at 11:58 PM, Dave Jones <davej@redhat.com> wrote:
> On Tue, Aug 23, 2011 at 07:19:35PM -0400, Steevven1 wrote:
>  > Hello,
>  >
>  > I would like to report what myself and several others believe to be a
>  > serious bug affecting all Linux kernels 3.0 and up, and none below 3.0
>  > (2.6.x and lower).
>  >
>  > One-sentence description: On certain hardware, the 3.x series kernels
>  > drain a considerable amount more power than the 2.6-series kernels, up
>  > to 40% more, even when idle.
>  >
>  > The specific hardware I have personally tested is a Lenovo ThinkPad
>  > X220 with an Intel SandyBridge i7 2620M processor.
>
> Sounds like https://bugzilla.redhat.com/show_bug.cgi?id=727579
> Try booting with i915.i915_enable_rc6=1
>
>        Dave
>
>

Re: Bug Report for Linux Kernel 3.x

[Jeff Chua]

Your might have  a different issue or perhaps you should make sure
you're up to date with the latest git pull. I'm on
14c62e78dc1379185515be41903c4a667efc6d54 (3.1.0-rc3)

I'm using Lenovo X201s and X was very very unstable and screen freezes
after a while. Now after adding i915.i915_enable_rc6=1, X is running
happily again!!!

Thanks,
Jeff.


On Wed, Aug 24, 2011 at 12:00 PM, Steevven1 <steevven1@gmail.com> wrote:
>
> Thanks for the comment. I have already tried this with several
> different 3-series kernels. There was no effect on power consumption
> or output from powertop.
>
> SK!
>
>
> On Tue, Aug 23, 2011 at 11:58 PM, Dave Jones <davej@redhat.com> wrote:
> > On Tue, Aug 23, 2011 at 07:19:35PM -0400, Steevven1 wrote:
> >  > Hello,
> >  >
> >  > I would like to report what myself and several others believe to be a
> >  > serious bug affecting all Linux kernels 3.0 and up, and none below 3.0
> >  > (2.6.x and lower).
> >  >
> >  > One-sentence description: On certain hardware, the 3.x series kernels
> >  > drain a considerable amount more power than the 2.6-series kernels, up
> >  > to 40% more, even when idle.
> >  >
> >  > The specific hardware I have personally tested is a Lenovo ThinkPad
> >  > X220 with an Intel SandyBridge i7 2620M processor.
> >
> > Sounds like https://bugzilla.redhat.com/show_bug.cgi?id=727579
> > Try booting with i915.i915_enable_rc6=1
> >
> >        Dave
> >
> >
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/