Powerbook hard shutdown after boot if it's hot

Powerbook hard shutdown after boot if it's hot

[Michael Buesch]

Hi,

I have a strange problem that my Powerbook shuts down hard
right after boot, if the machine is hot.
For example, if I compile a kernel and then want to reboot
into it, it will shutdown the machine right after userspace
is booted. Cooling down the machine will fix the issue.
I first thought this might be a problem with the adt746x
chip driver (thermostat). But this turns out to not be
the case. I disabled the module (put return 0; early into
the module_init function), but the problem still exists.
I'm not sure how to debug this, as I don't see what's
going on. I tried to boot with init=/bin/bash. It will
boot into the shell but shutdown hard after 1 or 2 seconds.
It seems like there is some timer hitting in right after
userspace is up and running (some workqueue?). Strange is
that it _only_ shuts down the machine when it's hot
and it _only_ does this right after boot. If it survived
the first few seconds after boot, it's rock-stable and
it won't show any problems when getting hot (will drive
the fans correctly, etc etc...)

Any idea how to debug this?

-- 
Greetings Michael.

Re: Powerbook hard shutdown after boot if it's hot

[Gabriel Paubert]

On Thu, Jun 14, 2007 at 05:39:12PM +0200, Michael Buesch wrote:
> Hi,
> 
> I have a strange problem that my Powerbook shuts down hard
> right after boot, if the machine is hot.
> For example, if I compile a kernel and then want to reboot
> into it, it will shutdown the machine right after userspace
> is booted. Cooling down the machine will fix the issue.
> I first thought this might be a problem with the adt746x
> chip driver (thermostat). But this turns out to not be
> the case. I disabled the module (put return 0; early into
> the module_init function), but the problem still exists.
> I'm not sure how to debug this, as I don't see what's
> going on. I tried to boot with init=/bin/bash. It will
> boot into the shell but shutdown hard after 1 or 2 seconds.
> It seems like there is some timer hitting in right after
> userspace is up and running (some workqueue?). Strange is
> that it _only_ shuts down the machine when it's hot
> and it _only_ does this right after boot. If it survived
> the first few seconds after boot, it's rock-stable and
> it won't show any problems when getting hot (will drive
> the fans correctly, etc etc...)
> 
> Any idea how to debug this?

I believe that the hardware (PMU?) forces a power off
when the fan speed control registers are not accessed
for some time. This delay may depend on the temperature. 

When my PB motherboard failed, the fans were driven at full
speed at power-up and then ran for about a minute before
they were stopped and the machine completely shut-down.

As a stab in the dark, you might try to force the fans at 
full speed in an __init function. This will make the machine 
noisy for some time before the loop starts regulating fan 
speed but might buy you some time.

	Regards,
	Gabriel

Re: Powerbook hard shutdown after boot if it's hot

[Segher Boessenkool]

> I believe that the hardware (PMU?) forces a power off
> when the fan speed control registers are not accessed
> for some time. This delay may depend on the temperature.

Yes, to me as well it sounds like the PMU is unhappy
because you didn't talk to it for too long while
things are hot.  Pretty hard to prove this assumption
of course -- but try Gabriel's suggestion, if that
works, it means this diagnosis is at least slightly
correct.  A real fix is something different of course.

> As a stab in the dark, you might try to force the fans at
> full speed in an __init function. This will make the machine
> noisy for some time before the loop starts regulating fan
> speed but might buy you some time.


Segher

Re: Powerbook hard shutdown after boot if it's hot

[Benjamin Herrenschmidt]

On Thu, 2007-06-14 at 17:39 +0200, Michael Buesch wrote:
> Hi,
> 
> I have a strange problem that my Powerbook shuts down hard
> right after boot, if the machine is hot.
> For example, if I compile a kernel and then want to reboot
> into it, it will shutdown the machine right after userspace
> is booted. Cooling down the machine will fix the issue.
> I first thought this might be a problem with the adt746x
> chip driver (thermostat). But this turns out to not be
> the case. I disabled the module (put return 0; early into
> the module_init function), but the problem still exists.
> I'm not sure how to debug this, as I don't see what's
> going on. I tried to boot with init=/bin/bash. It will
> boot into the shell but shutdown hard after 1 or 2 seconds.
> It seems like there is some timer hitting in right after
> userspace is up and running (some workqueue?). Strange is
> that it _only_ shuts down the machine when it's hot
> and it _only_ does this right after boot. If it survived
> the first few seconds after boot, it's rock-stable and
> it won't show any problems when getting hot (will drive
> the fans correctly, etc etc...)
> 
> Any idea how to debug this?

Not really, that's weird.. maybe trying to print the values in the adt
sensor to see how hot it is vs. the various thresholds set by the
firmware ? Maybe the PMU is trying to send us a "high temp" alert via a
PMU message we don't know how to parse and expcts us to reply in a given
amount of time (for example by slowing the CPU down).

Ben.

Re: Powerbook hard shutdown after boot if it's hot

[Michael Buesch]

On Saturday 16 June 2007 02:51:13 Benjamin Herrenschmidt wrote:
> On Thu, 2007-06-14 at 17:39 +0200, Michael Buesch wrote:
> > Hi,
> > 
> > I have a strange problem that my Powerbook shuts down hard
> > right after boot, if the machine is hot.
> > For example, if I compile a kernel and then want to reboot
> > into it, it will shutdown the machine right after userspace
> > is booted. Cooling down the machine will fix the issue.
> > I first thought this might be a problem with the adt746x
> > chip driver (thermostat). But this turns out to not be
> > the case. I disabled the module (put return 0; early into
> > the module_init function), but the problem still exists.
> > I'm not sure how to debug this, as I don't see what's
> > going on. I tried to boot with init=/bin/bash. It will
> > boot into the shell but shutdown hard after 1 or 2 seconds.
> > It seems like there is some timer hitting in right after
> > userspace is up and running (some workqueue?). Strange is
> > that it _only_ shuts down the machine when it's hot
> > and it _only_ does this right after boot. If it survived
> > the first few seconds after boot, it's rock-stable and
> > it won't show any problems when getting hot (will drive
> > the fans correctly, etc etc...)
> > 
> > Any idea how to debug this?
> 
> Not really, that's weird.. maybe trying to print the values in the adt
> sensor to see how hot it is vs. the various thresholds set by the

The thresholds are the same as when the machine is cold:
81 80 87

I also tried to spinup the fans at full speed on boot (by setting
the fan_speed variable in the adt driver to 255), but it kills
the machine before the fans have the ability to spin up.

I also tried to move the 2000ms sleep in the adt monitor task down
to the bottom of the task function, so it immediately runs the task
once on boot. But that didn't help either.

I think this powerdown is not really related to the adt chip/driver.
I think it's more likely some PMU issue and the PMU forces the machine
down. How's the PMU involved into temperatures? Does it get
the temperature values from the adt chip?

> firmware ? Maybe the PMU is trying to send us a "high temp" alert via a
> PMU message we don't know how to parse and expcts us to reply in a given
> amount of time (for example by slowing the CPU down).

Where in the kernel code are the PMU messages handled?

-- 
Greetings Michael.

Re: Powerbook hard shutdown after boot if it's hot

[Michael Buesch]

On Saturday 16 June 2007 15:04:41 Michael Buesch wrote:
> On Saturday 16 June 2007 02:51:13 Benjamin Herrenschmidt wrote:
> > On Thu, 2007-06-14 at 17:39 +0200, Michael Buesch wrote:
> > > Hi,
> > > 
> > > I have a strange problem that my Powerbook shuts down hard
> > > right after boot, if the machine is hot.
> > > For example, if I compile a kernel and then want to reboot
> > > into it, it will shutdown the machine right after userspace
> > > is booted. Cooling down the machine will fix the issue.
> > > I first thought this might be a problem with the adt746x
> > > chip driver (thermostat). But this turns out to not be
> > > the case. I disabled the module (put return 0; early into
> > > the module_init function), but the problem still exists.
> > > I'm not sure how to debug this, as I don't see what's
> > > going on. I tried to boot with init=/bin/bash. It will
> > > boot into the shell but shutdown hard after 1 or 2 seconds.
> > > It seems like there is some timer hitting in right after
> > > userspace is up and running (some workqueue?). Strange is
> > > that it _only_ shuts down the machine when it's hot
> > > and it _only_ does this right after boot. If it survived
> > > the first few seconds after boot, it's rock-stable and
> > > it won't show any problems when getting hot (will drive
> > > the fans correctly, etc etc...)
> > > 
> > > Any idea how to debug this?
> > 
> > Not really, that's weird.. maybe trying to print the values in the adt
> > sensor to see how hot it is vs. the various thresholds set by the
> 
> The thresholds are the same as when the machine is cold:
> 81 80 87
> 
> I also tried to spinup the fans at full speed on boot (by setting
> the fan_speed variable in the adt driver to 255), but it kills
> the machine before the fans have the ability to spin up.
> 
> I also tried to move the 2000ms sleep in the adt monitor task down
> to the bottom of the task function, so it immediately runs the task
> once on boot. But that didn't help either.
> 
> I think this powerdown is not really related to the adt chip/driver.
> I think it's more likely some PMU issue and the PMU forces the machine
> down. How's the PMU involved into temperatures? Does it get
> the temperature values from the adt chip?
> 
> > firmware ? Maybe the PMU is trying to send us a "high temp" alert via a
> > PMU message we don't know how to parse and expcts us to reply in a given
> > amount of time (for example by slowing the CPU down).
> 
> Where in the kernel code are the PMU messages handled?
> 

Oh, and I completely forgot to say that this problem does
not exist when booting OSX. So if I try to boot linux and it
fails because of the issue and I immediately boot OSX it will succeed.

-- 
Greetings Michael.

Re: Powerbook hard shutdown after boot if it's hot

[Segher Boessenkool]

> I think this powerdown is not really related to the adt chip/driver.
> I think it's more likely some PMU issue and the PMU forces the machine
> down. How's the PMU involved into temperatures? Does it get
> the temperature values from the adt chip?

It is connected to the overtemp alarm interrupt pin of
the temperature sensors at least.

Does this shutdown only happen if you boot Linux while
you were running Linux before, too?


Segher

Re: Powerbook hard shutdown after boot if it's hot

[Segher Boessenkool]

> Oh, and I completely forgot to say that this problem does
> not exist when booting OSX. So if I try to boot linux and it
> fails because of the issue and I immediately boot OSX it will succeed.

Right -- and if you're running OSX (and run the box hot),
and then boot Linux, does it happen then?


Segher

Re: Powerbook hard shutdown after boot if it's hot

[Michael Buesch]

On Saturday 16 June 2007 15:40:39 Segher Boessenkool wrote:
> > I think this powerdown is not really related to the adt chip/driver.
> > I think it's more likely some PMU issue and the PMU forces the machine
> > down. How's the PMU involved into temperatures? Does it get
> > the temperature values from the adt chip?
> 
> It is connected to the overtemp alarm interrupt pin of
> the temperature sensors at least.
> 
> Does this shutdown only happen if you boot Linux while
> you were running Linux before, too?

No, it also happens, if I heat up the machine in OSX
and boot linux. Booting OSX does not cause it, then.

-- 
Greetings Michael.

Re: Powerbook hard shutdown after boot if it's hot

[Brad Boyer]

On Sat, Jun 16, 2007 at 03:04:41PM +0200, Michael Buesch wrote:
> Where in the kernel code are the PMU messages handled?

The main place to look would be drivers/macintosh/via-pmu.c, and
it looks like the bulk of the work is being done in the function
pmu_handle_data(). You might want to take a look at other parts
of the driver as well.

	Brad Boyer
	flar@allandria.com