possible hrtimer related problem on tegra

possible hrtimer related problem on tegra

[Marc Dietrich]

Hi,

I've been plagued with random lockups starting with Linux 3.0.13 (pulled from 
ChromeOS-3.0 tree) on a Tegra board (paz00). First symtoms are slowly losing 
responsiveness followed by a total lockup (nothing in dmesg, no oops). Using 
performance governor instead of ondemand, made it go away.

So I started bisecting 3.0.12..3.0.13 and finially found that reverting 
24ee8bfe (hrtimer: Fix extra wakeups from __remove_hrtimer()) fixes the 
problem.

As Olof mentioned on IRC it is likely that the patch does not directly causes 
lockups but something else which was hidden before. It isn't clear to me if 
other tegra boards (seaboard, harmony) also suffer from this.

Any hints on this issue are appreciated.

Thanks

Marc

Re: possible hrtimer related problem on tegra

[Olof Johansson]

Hi,

On Sun, Jan 22, 2012 at 7:18 AM, Marc Dietrich <marvin24-Mmb7MZpHnFY@public.gmane.org> wrote:
> Hi,
>
> I've been plagued with random lockups starting with Linux 3.0.13 (pulled from
> ChromeOS-3.0 tree) on a Tegra board (paz00). First symtoms are slowly losing
> responsiveness followed by a total lockup (nothing in dmesg, no oops). Using
> performance governor instead of ondemand, made it go away.
>
> So I started bisecting 3.0.12..3.0.13 and finially found that reverting
> 24ee8bfe (hrtimer: Fix extra wakeups from __remove_hrtimer()) fixes the
> problem.
>
> As Olof mentioned on IRC it is likely that the patch does not directly causes
> lockups but something else which was hidden before. It isn't clear to me if
> other tegra boards (seaboard, harmony) also suffer from this.
>
> Any hints on this issue are appreciated.

You're asking on a community list about issues with a distro kernel.
In general that is frowned upon.

Since I'm one of the people looking after the distro kernel in
particular though; The current 3.0 tegra branch has a handful of
stability issues with power management/cpufreq/dvfs enabled, and we
have chosen to (at least for now) disable those features in our
configs. Chances are good that that patch in question is tickling a
bug that is specific to the Chrome OS tree so without further
investigation I don't see a reason to pull in external folks right now
either.

The other tegra maintainers don't particularly know what's in our
distro tree either, and I wouldn't expect them to spend time on
working on out-of-tree code that isn't their own.

Best way forward is probably for you to do a local revert on the
offending patch and to open a new issue at http://crosbug.com/ for it.
Feel free to cc me (olofj-F7+t8E8rja9g9hUCZPvPmw@public.gmane.org) on the bug.


-Olof

Re: Re: possible hrtimer related problem on tegra

[Marc Dietrich]

Hi Olof,

Am Montag, 23. Januar 2012, 00:09:42 schrieb Olof Johansson:
> On Sun, Jan 22, 2012 at 7:18 AM, Marc Dietrich <marvin24-Mmb7MZpHnFY@public.gmane.org> wrote:
> > I've been plagued with random lockups starting with Linux 3.0.13 (pulled from
> > ChromeOS-3.0 tree) on a Tegra board (paz00). First symtoms are slowly losing
> > responsiveness followed by a total lockup (nothing in dmesg, no oops). Using
> > performance governor instead of ondemand, made it go away.
> > 
> > So I started bisecting 3.0.12..3.0.13 and finially found that reverting
> > 24ee8bfe (hrtimer: Fix extra wakeups from __remove_hrtimer()) fixes the
> > problem.
> > 
> > As Olof mentioned on IRC it is likely that the patch does not directly causes
> > lockups but something else which was hidden before. It isn't clear to me if
> > other tegra boards (seaboard, harmony) also suffer from this.
> > 
> > Any hints on this issue are appreciated.
> 
> You're asking on a community list about issues with a distro kernel.
> In general that is frowned upon.

I'm sorry to have this reported on a non appropriate place. I had the impression that 
the ChromeOS kernel, and especially the timer related code, doesn't differ too much 
from the mainline kernel (only suspend code added?).

On the other hand, there is no other place I know of to report problems with Android 
or ChromeOS kernels and which is reaching most of tegra kernel developers/users 
(sorry again if readers feel that I just spammed the list). There are at least some 
chances other users also see this issue, if it is not strictly related to the 
ChromeOS kernel only.

> Since I'm one of the people looking after the distro kernel in
> particular though; The current 3.0 tegra branch has a handful of
> stability issues with power management/cpufreq/dvfs enabled, and we
> have chosen to (at least for now) disable those features in our
> configs. Chances are good that that patch in question is tickling a
> bug that is specific to the Chrome OS tree so without further
> investigation I don't see a reason to pull in external folks right now
> either.
> 
> The other tegra maintainers don't particularly know what's in our
> distro tree either, and I wouldn't expect them to spend time on
> working on out-of-tree code that isn't their own.
> 
> Best way forward is probably for you to do a local revert on the
> offending patch and to open a new issue at http://crosbug.com/ for it.
> Feel free to cc me (olofj-F7+t8E8rja9g9hUCZPvPmw@public.gmane.org) on the bug.

Ok, I filed a bug report: http://code.google.com/p/chromium-os/issues/detail?id=25334 
(no idea how to add a CC).

I'll also try to reproduce it on a mainline kernel.

Marc