windfarm_fcu_controls: cpu-pump-0 <HW FAULT>

windfarm_fcu_controls: cpu-pump-0 <HW FAULT>

[Andreas Schwab]

wf_fcu_fan_get_rpm is returning EFAULT when reading the values for the
cpu pump controls on the PowerMac7,3.  Also, wf_fcu_get_pump_minmax is
unable to get the real limits and falls back to using the defaults.

Here is the output when DEBUG is defined:

wf_fcu: FCU Initialized, RPM fan shift is 3
Looking up FCU controls in device-tree...
 control: rpm0, type: fan-rpm-control
 matching location: CPU B PUMP, reg: 0x00000010
 location match, name: cpu-pump-1
wf_fcu: pump min/max for cpu-pump-1 set to: [1250..3200] RPM
wf_pm72: Liquid cooling pump(s) detected, using new algorithm !
 control: rpm1, type: fan-rpm-control
 matching location: CPU A PUMP, reg: 0x00000012
 location match, name: cpu-pump-0
wf_fcu: pump min/max for cpu-pump-0 set to: [1250..3200] RPM
 control: rpm2, type: fan-rpm-control
 matching location: DRIVE BAY, reg: 0x00000014
 location match, name: drive-bay-fan
wf_fcu: fan min/max for drive-bay-fan set to: [300..7000] RPM
 control: rpm3, type: fan-rpm-control
 matching location: CPU A INTAKE, reg: 0x00000016
 location match, name: cpu-front-fan-0
wf_fcu: fan min/max for cpu-front-fan-0 set to: [300..3200] RPM
 control: rpm4, type: fan-rpm-control
 matching location: CPU A EXHAUST, reg: 0x00000018
 location match, name: cpu-rear-fan-0
wf_fcu: fan min/max for cpu-rear-fan-0 set to: [300..3200] RPM
 control: rpm5, type: fan-rpm-control
 matching location: CPU B INTAKE, reg: 0x0000001a
 location match, name: cpu-front-fan-1
wf_fcu: fan min/max for cpu-front-fan-1 set to: [300..3200] RPM
 control: rpm6, type: fan-rpm-control
 matching location: CPU B EXHAUST, reg: 0x0000001c
 location match, name: cpu-rear-fan-1
wf_fcu: fan min/max for cpu-rear-fan-1 set to: [300..3200] RPM
 control: pwm1, type: fan-pwm-control
 matching location: BACKSIDE, reg: 0x00000032
 location match, name: backside-fan
 control: pwm2, type: fan-pwm-control
 matching location: SLOT, reg: 0x00000034
 location match, name: slots-fan
 control: adc0, type: power-sensor
 control: adc1, type: power-sensor
 control: adc2, type: power-sensor
 control: adc3, type: power-sensor
 control: gpi0, type: gpi-sensor
 control: gpi1, type: gpi-sensor

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: windfarm_fcu_controls: cpu-pump-0 <HW FAULT>

[Benjamin Herrenschmidt]

On Fri, 2013-08-02 at 21:08 +0200, Andreas Schwab wrote:
> wf_fcu_fan_get_rpm is returning EFAULT when reading the values for the
> cpu pump controls on the PowerMac7,3.  Also, wf_fcu_get_pump_minmax is
> unable to get the real limits and falls back to using the defaults.
> 
> Here is the output when DEBUG is defined:

Can you add some more printk's in there to check what's going on inside
wf_fcu_get_pump_minmax() ?

Also is it getting faults for both pumps ?

Does it work with the older driver ? (both the minmax and the reading of
the pump). What is the "failures" bitmask value ?

Cheers,
Ben.

Re: windfarm_fcu_controls: cpu-pump-0 <HW FAULT>

[Andreas Schwab]

Benjamin Herrenschmidt <benh@kernel.crashing.org> writes:

> Can you add some more printk's in there to check what's going on inside
> wf_fcu_get_pump_minmax() ?

All values from mpu->processor_part_num are 0xffff.

> Also is it getting faults for both pumps ?

Yes.

> Does it work with the older driver ? (both the minmax and the reading of
> the pump).

The minmax situation is the same, but otherwise appears to work (no fan
reading errors logged).  Here is an example debug output:

** CPU 1 RPM: 300 Ex, 300, Pump: 1250, In, overtemp: 0
  cpu 0, exhaust RPM: 300
  cpu 0, temp raw: 023c, m_diode: 9982, b_diode: fffff799
  temp: 52.139
  cpu 0, current: 8.789, voltage: 1.286, power: 11.308 W
  cpu 1, exhaust RPM: 300
  cpu 1, temp raw: 021c, m_diode: a047, b_diode: fffff777
  temp: 50.380
  cpu 1, current: 8.666, voltage: 1.281, power: 11.108 W
  power target: 55.000, error: 43.691
  integral: 00f42f8d
   integ_p: 10
   adj_in_target: 65.011, ttarget: 74
   deriv_p: -15
   prop_p: -103
   sum: -118

> What is the "failures" bitmask value ?

3 for both.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: windfarm_fcu_controls: cpu-pump-0 <HW FAULT>

[Benjamin Herrenschmidt]

On Sat, 2013-08-03 at 10:43 +0200, Andreas Schwab wrote:
> Benjamin Herrenschmidt <benh@kernel.crashing.org> writes:
> 
> > Can you add some more printk's in there to check what's going on inside
> > wf_fcu_get_pump_minmax() ?
> 
> All values from mpu->processor_part_num are 0xffff.
> 
> > Also is it getting faults for both pumps ?
> 
> Yes.
> 
> > Does it work with the older driver ? (both the minmax and the reading of
> > the pump).
> 
> The minmax situation is the same, but otherwise appears to work (no fan
> reading errors logged).  Here is an example debug output:

Odd. Can you try to trace if there is any significant difference in the
i2c messages used ? Some typo I might have done somewhere ?

Something I might do at init time that puts them into a faulty state ?

I don't have one of these anymore (mine died) so I can't really test.

Cheers,
Ben.

> ** CPU 1 RPM: 300 Ex, 300, Pump: 1250, In, overtemp: 0
>   cpu 0, exhaust RPM: 300
>   cpu 0, temp raw: 023c, m_diode: 9982, b_diode: fffff799
>   temp: 52.139
>   cpu 0, current: 8.789, voltage: 1.286, power: 11.308 W
>   cpu 1, exhaust RPM: 300
>   cpu 1, temp raw: 021c, m_diode: a047, b_diode: fffff777
>   temp: 50.380
>   cpu 1, current: 8.666, voltage: 1.281, power: 11.108 W
>   power target: 55.000, error: 43.691
>   integral: 00f42f8d
>    integ_p: 10
>    adj_in_target: 65.011, ttarget: 74
>    deriv_p: -15
>    prop_p: -103
>    sum: -118
> 
> > What is the "failures" bitmask value ?
> 
> 3 for both.
> 
> Andreas.
> 

Re: windfarm_fcu_controls: cpu-pump-0 <HW FAULT>

[Andreas Schwab]

Actually I see the same failure mask with the therm_pm72 driver, I
didn't notice since the driver ignores the return value from set_rpm_fan
and there is no sysfs file to monitor the cpu pumps.  So it looks like
some real hardware fault?

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: windfarm_fcu_controls: cpu-pump-0 <HW FAULT>

[Benjamin Herrenschmidt]

On Sat, 2013-08-03 at 15:49 +0200, Andreas Schwab wrote:
> Actually I see the same failure mask with the therm_pm72 driver, I
> didn't notice since the driver ignores the return value from set_rpm_fan
> and there is no sysfs file to monitor the cpu pumps.  So it looks like
> some real hardware fault?

Either that, or the fault indication might be something to ignore on
those machines, I don't know at this point. It might be worth trying to
find another of these machines to compare.

Maybe the water leaked out ? :-)

Cheers,
Ben.

> 
> Andreas.
> 

Re: windfarm_fcu_controls: cpu-pump-0 <HW FAULT>

[Andreas Schwab]

But then, MacOS is happy and the hardware test didn't find anything
either...

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: windfarm_fcu_controls: cpu-pump-0 <HW FAULT>

[Benjamin Herrenschmidt]

On Sat, 2013-08-03 at 16:33 +0200, Andreas Schwab wrote:
> But then, MacOS is happy and the hardware test didn't find anything
> either...

Ok, I'll take another peek at the MacOS code see if there's anything
meaningful there (though we don't have the complete stuff, I think the
fault mask we decoded from the OFW methods).

Cheers,
Ben.