[Fwd: Re: [lm-sensors] Kernel hangs with i2c-i801 driver?]

[Fwd: Re: [lm-sensors] Kernel hangs with i2c-i801 driver?]

[Daniel Nilsson]

On Fri, Nov 25, 2005 at 08:56:49PM +0100, Rudolf Marek wrote:
> 
> Hello all,
> 
> I have a theory that changing limit values will assert SMI interrupt
> and SMI code will hang the machine. 
> Maybe the interrupt is raised via SMBALERT line which can be
> disabled in the i2c-i801.c driver. 
> 
> Eventualy i think if we switch the interrupt generation to IRQ9 so
> inux should report "IRQ9 nobody cared" instead of crarsh ... if I'm
> correct with the SMI assumption. 
> 
> you can try to:
> setpci  -s 0:1f.3 40.b=1
> 
> This will set the SMB routing interrupt to IRQ9 (or some other but
> not on SMI)
> 
> and then repeat with sensors -s
> This is a bit dangerous so please do it with filesystems mouted RO.
> 
> If you get instead of crash no crash and/or some message in dmesg we
> have won :) 

This sounds like an interesting theory, but I have a small problem
here... I can't reproduct the problem on my machine :-( I know it
happened at least 4 times before, but now it doesn't seem to happen. I
have not changed any software or recompiled the kernel. All that might
have changed are the sensors limits in sensors.conf which I did
restore to default values. There are also a few reboots that have
happened since then.

I obviously need to be able to reproduce fairly easily so that we can
see if a change has any significant effect or not, so any tips or
tricks that might force the issue would be welcome! I'm leaning
towards writing a script that will loop and write all possible values
to the limits files under sysfs. Other then that I'm out of ideas...

/Daniel

[Fwd: Re: [lm-sensors] Kernel hangs with i2c-i801 driver?]

[Henrique de Moraes Holschuh]

On Sat, 26 Nov 2005, Daniel Nilsson wrote:
> tricks that might force the issue would be welcome! I'm leaning

Try setting limits one by one in such a way to cause critical alarms.

-- 
  "One disk to rule them all, One disk to find them. One disk to bring
  them all and in the darkness grind them. In the Land of Redmond
  where the shadows lie." -- The Silicon Valley Tarot
  Henrique Holschuh

[Fwd: Re: [lm-sensors] Kernel hangs with i2c-i801 driver?]

[Daniel Nilsson]

On Fri, Nov 25, 2005 at 08:56:49PM +0100, Rudolf Marek wrote:
> 
> Hello all,
> 
> I have a theory that changing limit values will assert SMI interrupt
> and SMI code will hang the machine. 
> Maybe the interrupt is raised via SMBALERT line which can be
> disabled in the i2c-i801.c driver. 
> 
> Eventualy i think if we switch the interrupt generation to IRQ9 so
> inux should report "IRQ9 nobody cared" instead of crarsh ... if I'm
> correct with the SMI assumption. 
> 
> you can try to:
> setpci  -s 0:1f.3 40.b=1
> 
> This will set the SMB routing interrupt to IRQ9 (or some other but not on SMI)

Rudolf,

I was finally able to get into a situation where I can reproduce the
hang in a repeatable fashion. I'm using this script to set the values:

find /sys/bus/i2c/devices/0-002c/ -name 'in*_min' -exec /bin/sh -c '/bin/echo 0 > {}' \; 
find /sys/bus/i2c/devices/0-002c/ -name 'in*_max' -exec /bin/sh -c '/bin/echo 10000 > {}' \; 
find /sys/bus/i2c/devices/0-002c/ -name 'temp*_min' -exec /bin/sh -c '/bin/echo 0 > {}' \;
find /sys/bus/i2c/devices/0-002c/ -name 'temp*_max' -exec /bin/sh -c '/bin/echo 100000 > {}' \;
find /sys/bus/i2c/devices/0-002c/ -name 'temp*_max_hyst' -exec /bin/sh -c '/bin/echo 0 > {}' \;
#find /sys/bus/i2c/devices/0-002c/ -name 'fan*_min' -exec /bin/sh -c '/bin/echo 0 > {}' \;
#find /sys/bus/i2c/devices/0-002c/ -name 'fan*_div' -exec /bin/sh -c '/bin/echo 2 > {}' \;

If I uncommment the last two lines I can always get the system to
hang, but I have sometimes been able to get it to hang by just setting
the in* and temp* limits as well.

I tried executing 'setpci  -s 0:1f.3 40.b=1', that command completed
without complaining but I didn't see any difference in the
behaviour. But I don't know how to verify that the setpci command
actually did what you were expecting it to do.

I noticed one more thing that might not make a difference here, in my
initial post I probably showed the output from sensors with the fan
sensors that are not used hidden (by setting the ignore option in
sensors.conf). I noticed that Fan7 is giving a negative value and has a
different default divisor and  min value. It is also the only fan
giving an alarm in the default state. I thought this might be
significant since setting the fan limits in the script above seem to
be what always caused the hang.

oden:~# sensors
w83792d-i2c-0-2c
Adapter: SMBus I801 adapter at 3080
VCoreA:    +1.69 V  (min =  +1.40 V, max =  +1.60 V)       ALARM
VCoreB:    +1.70 V  (min =  +1.48 V, max =  +1.60 V)       ALARM
VIN0:      +2.96 V  (min =  +3.20 V, max =  +3.39 V)       ALARM
VIN1:      +2.97 V  (min =  +3.09 V, max =  +3.30 V)       ALARM
VIN2:      +2.96 V  (min =  +1.39 V, max =  +1.49 V)       ALARM
VIN3:      +2.96 V  (min =  +2.59 V, max =  +2.64 V)       ALARM
5VCC:      +4.94 V  (min =  +4.73 V, max =  +5.23 V)
5VSB:      +5.06 V  (min =  +4.73 V, max =  +5.23 V)
VBAT:      +3.23 V  (min =  +2.85 V, max =  +3.14 V)       ALARM
Fan1:     2616 RPM  (min = 1500 RPM, div = 4)
Fan2:        0 RPM  (min =  703 RPM, div = 8)
Fan3:     1480 RPM  (min =  703 RPM, div = 8)
Fan4:        0 RPM  (min =  703 RPM, div = 8)
Fan5:        0 RPM  (min =  703 RPM, div = 8)
Fan6:        0 RPM  (min =  703 RPM, div = 8)
Fan7:       -1 RPM  (min =    0 RPM, div = 2)              ALARM
Temp1:     +22.0?C  (high = +36.0?C, hyst = +31.0?C)   ALARM
Temp2:     +41.0?C  (high = +60.0?C, hyst = +57.0?C)   ALARM
Temp3:     +46.5?C  (high = +96.0?C, hyst = +93.0?C)   ALARM
chassis:  Chassis is normal.

Should I still try the UP kernel and an older kernel version?

-- 
Daniel Nilsson

[Fwd: Re: [lm-sensors] Kernel hangs with i2c-i801 driver?]

[Daniel Nilsson]

On Sun, Nov 27, 2005 at 12:31:22PM +0100, Daniel Nilsson wrote:
> I was finally able to get into a situation where I can reproduce the
> hang in a repeatable fashion. I'm using this script to set the values:
> 
> find /sys/bus/i2c/devices/0-002c/ -name 'in*_min' -exec /bin/sh -c '/bin/echo 0 > {}' \; 
> find /sys/bus/i2c/devices/0-002c/ -name 'in*_max' -exec /bin/sh -c '/bin/echo 10000 > {}' \; 
> find /sys/bus/i2c/devices/0-002c/ -name 'temp*_min' -exec /bin/sh -c '/bin/echo 0 > {}' \;
> find /sys/bus/i2c/devices/0-002c/ -name 'temp*_max' -exec /bin/sh -c '/bin/echo 100000 > {}' \;
> find /sys/bus/i2c/devices/0-002c/ -name 'temp*_max_hyst' -exec /bin/sh -c '/bin/echo 0 > {}' \;
> #find /sys/bus/i2c/devices/0-002c/ -name 'fan*_min' -exec /bin/sh -c '/bin/echo 0 > {}' \;
> #find /sys/bus/i2c/devices/0-002c/ -name 'fan*_div' -exec /bin/sh -c '/bin/echo 2 > {}' \;
> 
> If I uncommment the last two lines I can always get the system to
> hang, but I have sometimes been able to get it to hang by just setting
> the in* and temp* limits as well.

I've done some more testing, and the results are unfortunately not
conclusive to me so I'm posting them here to see if they tell someone
else more then they tell me.

I modified my initial script, so that it now looks like this:

#!/bin/sh

find /sys/bus/i2c/devices/0-002c/ -name 'in*_min' -exec ./echo_n_sleep.sh {} 0 \;
find /sys/bus/i2c/devices/0-002c/ -name 'in*_max' -exec ./echo_n_sleep.sh {} 10000 \;
find /sys/bus/i2c/devices/0-002c/ -name 'temp*_min' -exec ./echo_n_sleep.sh {} 0 \;
find /sys/bus/i2c/devices/0-002c/ -name 'temp*_max' -exec ./echo_n_sleep.sh {} 100000 \;
find /sys/bus/i2c/devices/0-002c/ -name 'temp*_max_hyst' -exec ./echo_n_sleep.sh {} 0  \;
find /sys/bus/i2c/devices/0-002c/ -name 'fan*_min' -exec ./echo_n_sleep.sh {} 0 \;
find /sys/bus/i2c/devices/0-002c/ -name 'fan*_div' -exec ./echo_n_sleep.sh {} 2 \;

The script ./echo_n_sleep.sh looks like this:

#!/bin/sh

WHERE=$1
WHAT=$2

echo -n "Setting $1 to $2 , "
/bin/echo $2 > $1
echo -n "sleeping 1s..."
sleep 1
echo "ok."

Running the above scripts always results in a hang at the exact same
place:

Setting /sys/bus/i2c/devices/0-002c/in8_min to 0 , sleeping 1s...ok.
Setting /sys/bus/i2c/devices/0-002c/in7_min to 0 , sleeping 1s...ok.
Setting /sys/bus/i2c/devices/0-002c/in6_min to 0 , sleeping 1s...ok.
Setting /sys/bus/i2c/devices/0-002c/in5_min to 0 , sleeping 1s...ok.
Setting /sys/bus/i2c/devices/0-002c/in4_min to 0 , sleeping 1s...ok.
Setting /sys/bus/i2c/devices/0-002c/in3_min to 0 , sleeping 1s...ok.
Setting /sys/bus/i2c/devices/0-002c/in2_min to 0 , sleeping 1s...ok.
Setting /sys/bus/i2c/devices/0-002c/in1_min to 0 , sleeping 1s...ok.
Setting /sys/bus/i2c/devices/0-002c/in0_min to 0 , sleeping 1s...ok.
Setting /sys/bus/i2c/devices/0-002c/in8_max to 10000 , sleeping 1s...ok.
Setting /sys/bus/i2c/devices/0-002c/in7_max to 10000 , sleeping 1s...ok.
Setting /sys/bus/i2c/devices/0-002c/in6_max to 10000 , sleeping 1s...ok.
Setting /sys/bus/i2c/devices/0-002c/in5_max to 10000 , sleeping 1s...ok.
Setting /sys/bus/i2c/devices/0-002c/in4_max to 10000 , sleeping 1s...ok.
Setting /sys/bus/i2c/devices/0-002c/in3_max to 10000 , sleeping 1s...ok.
Setting /sys/bus/i2c/devices/0-002c/in2_max to 10000 , sleeping 1s...ok.
Setting /sys/bus/i2c/devices/0-002c/in1_max to 10000 , sleeping 1s...ok.
Setting /sys/bus/i2c/devices/0-002c/in0_max to 10000 , sleeping 1s...ok.
Setting /sys/bus/i2c/devices/0-002c/temp3_max to 100000 , sleeping 1s...ok.
Setting /sys/bus/i2c/devices/0-002c/temp2_max to 100000 , sleeping 1s...ok.
Setting /sys/bus/i2c/devices/0-002c/temp1_max to 100000 , sleeping 1s...ok.
Setting /sys/bus/i2c/devices/0-002c/temp3_max_hyst to 0 , sleeping 1s...ok.
Setting /sys/bus/i2c/devices/0-002c/temp2_max_hyst to 0 , sleeping 1s...ok.
Setting /sys/bus/i2c/devices/0-002c/temp1_max_hyst to 0 , sleeping 1s...ok.
Setting /sys/bus/i2c/devices/0-002c/fan7_min to 0 , sleeping 1s...ok.
Setting /sys/bus/i2c/devices/0-002c/fan6_min to 0 ,

It crashed when fan6_min is set to 0 for some reason. I thought it
might have to do with timing, so I increased the sleep to 5s
between each write but that gives the same result. I also tried this
with Hyperthreading turned off, which also gave the same result (a
kernel hang).

The issue is that if I just set /sys/bus/i2c/devices/0-002c/fan6_min
to 0 directly after a reboot no hang happens. So when I manually set
the various limits in a random order the machine will not hang. I then
modified the script to that the order in which the limits are like this:

#/bin/sh

find /sys/bus/i2c/devices/0-002c/ -name 'fan*_min' -exec ./echo_n_sleep.sh {} 0 \;
find /sys/bus/i2c/devices/0-002c/ -name 'fan*_div' -exec ./echo_n_sleep.sh {} 2 \;
find /sys/bus/i2c/devices/0-002c/ -name 'in*_min' -exec ./echo_n_sleep.sh {} 0 \;
find /sys/bus/i2c/devices/0-002c/ -name 'in*_max' -exec ./echo_n_sleep.sh {} 10000 \;
find /sys/bus/i2c/devices/0-002c/ -name 'temp*_min' -exec ./echo_n_sleep.sh {} 0 \;
find /sys/bus/i2c/devices/0-002c/ -name 'temp*_max' -exec ./echo_n_sleep.sh {} 100000 \;
find /sys/bus/i2c/devices/0-002c/ -name 'temp*_max_hyst' -exec ./echo_n_sleep.sh {} 0  \;

With this new order, I can execute that script as many times as I want
without any hangs. I can after executing this script that works fine
go back and execute the one giving the problem, and no I don't get a
hang at all! So it seems that once the limits are set correctly, I can
rewrite the same values over and over in any order.

I don't know what more to try at this point. One of the suggestions
were to try the 2.6.13.4 kernel which I attempted, but since there is
no w83792d driver in that kernel I can't really test. I tried to build
the w83792d from 2.6.14 under 2.6.13.4 but I didn't succeed. There
seems to be quite a few changes in the header files and there were
lots of warnings and errors. If there is a version of the w83792d
driver that will compile under 2.6.13.4 I should be able to test that
fairly quickly.

Thanks
Daniel

[Fwd: Re: [lm-sensors] Kernel hangs with i2c-i801 driver?]

[Rudolf Marek]

Hello all,

Sorry for a delay.

> I was finally able to get into a situation where I can reproduce the
> hang in a repeatable fashion. I'm using this script to set the values:
> 
> find /sys/bus/i2c/devices/0-002c/ -name 'in*_min' -exec /bin/sh -c '/bin/echo 0 > {}' \; 
> find /sys/bus/i2c/devices/0-002c/ -name 'in*_max' -exec /bin/sh -c '/bin/echo 10000 > {}' \; 
> find /sys/bus/i2c/devices/0-002c/ -name 'temp*_min' -exec /bin/sh -c '/bin/echo 0 > {}' \;
> find /sys/bus/i2c/devices/0-002c/ -name 'temp*_max' -exec /bin/sh -c '/bin/echo 100000 > {}' \;
> find /sys/bus/i2c/devices/0-002c/ -name 'temp*_max_hyst' -exec /bin/sh -c '/bin/echo 0 > {}' \;
> #find /sys/bus/i2c/devices/0-002c/ -name 'fan*_min' -exec /bin/sh -c '/bin/echo 0 > {}' \;
> #find /sys/bus/i2c/devices/0-002c/ -name 'fan*_div' -exec /bin/sh -c '/bin/echo 2 > {}' \;
> 
> If I uncommment the last two lines I can always get the system to
> hang, but I have sometimes been able to get it to hang by just setting
> the in* and temp* limits as well.
> 
> I tried executing 'setpci  -s 0:1f.3 40.b=1', that command completed
> without complaining but I didn't see any difference in the
> behaviour. But I don't know how to verify that the setpci command
> actually did what you were expecting it to do.

This just disables the smbalert signal. Maybe the output from a chip is routed
to other pin. Please can you provide the dump after cold boot and BEFORE loading
the w83792d driver?

i2cdump 0 0x2c

> I noticed one more thing that might not make a difference here, in my
> initial post I probably showed the output from sensors with the fan
> sensors that are not used hidden (by setting the ignore option in
> sensors.conf). I noticed that Fan7 is giving a negative value and has a
> different default divisor and  min value. It is also the only fan
> giving an alarm in the default state. I thought this might be
> significant since setting the fan limits in the script above seem to
> be what always caused the hang.

We have a patch from Yuan, I hope it is not malformed by my stupid mail client.

--- linux-2.6.14-rc5.orig/drivers/hwmon/w83792d.c	2005-11-09 11:27:44.000000000 +0800
+++ linux-2.6.14-rc5/drivers/hwmon/w83792d.c	2005-11-11 17:05:50.000000000 +0800
@@ -257,7 +257,7 @@ DIV_TO_REG(long val)
 {
 	int i;
 	val = SENSORS_LIMIT(val, 1, 128) >> 1;
-	for (i = 0; i < 6; i++) {
+	for (i = 0; i < 7; i++) {
 		if (val = 0)
 			break;
 		val >>= 1;
@@ -1284,8 +1284,8 @@ w83792d_detect(struct i2c_adapter *adapt
 	w83792d_init_client(new_client);

 	/* A few vars need to be filled upon startup */
-	for (i = 1; i <= 7; i++) {
-		data->fan_min[i - 1] = w83792d_read_value(new_client,
+	for (i = 0; i < 7; i++) {
+		data->fan_min[i] = w83792d_read_value(new_client,
 					W83792D_REG_FAN_MIN[i]);
 	}

@@ -1311,7 +1311,9 @@ w83792d_detect(struct i2c_adapter *adapt
 	device_create_file_fan(new_client, 4);
 	device_create_file_fan(new_client, 5);
 	device_create_file_fan(new_client, 6);
-	device_create_file_fan(new_client, 7);
+	val1 = w83792d_read_value(new_client, W83792D_REG_PIN);
+	if ( val1 & 0x04 )
+		device_create_file_fan(new_client, 7);

 	device_create_file_temp1(new_client);		/* Temp1 */
 	device_create_file_temp_add(new_client, 2);	/* Temp2 */

You may extend this from:

+	if ( val1 & 0x04 )
+		device_create_file_fan(new_client, 7);

add:

- 	device_create_file_fan(new_client, 6);
+	if ( val1 & 0x40 )
+		device_create_file_fan(new_client, 6);

Because fan6 and fan7 - mean their pins are programmable so correct would be to create their files when
they actual outputs are really used for fans...


> oden:~# sensors
> w83792d-i2c-0-2c
> Adapter: SMBus I801 adapter at 3080
> VCoreA:    +1.69 V  (min =  +1.40 V, max =  +1.60 V)       ALARM
> VCoreB:    +1.70 V  (min =  +1.48 V, max =  +1.60 V)       ALARM
> VIN0:      +2.96 V  (min =  +3.20 V, max =  +3.39 V)       ALARM
> VIN1:      +2.97 V  (min =  +3.09 V, max =  +3.30 V)       ALARM
> VIN2:      +2.96 V  (min =  +1.39 V, max =  +1.49 V)       ALARM
> VIN3:      +2.96 V  (min =  +2.59 V, max =  +2.64 V)       ALARM
> 5VCC:      +4.94 V  (min =  +4.73 V, max =  +5.23 V)
> 5VSB:      +5.06 V  (min =  +4.73 V, max =  +5.23 V)
> VBAT:      +3.23 V  (min =  +2.85 V, max =  +3.14 V)       ALARM
> Fan1:     2616 RPM  (min = 1500 RPM, div = 4)
> Fan2:        0 RPM  (min =  703 RPM, div = 8)
> Fan3:     1480 RPM  (min =  703 RPM, div = 8)
> Fan4:        0 RPM  (min =  703 RPM, div = 8)
> Fan5:        0 RPM  (min =  703 RPM, div = 8)
> Fan6:        0 RPM  (min =  703 RPM, div = 8)
> Fan7:       -1 RPM  (min =    0 RPM, div = 2)              ALARM
> Temp1:     +22.0?C  (high = +36.0?C, hyst = +31.0?C)   ALARM
> Temp2:     +41.0?C  (high = +60.0?C, hyst = +57.0?C)   ALARM
> Temp3:     +46.5?C  (high = +96.0?C, hyst = +93.0?C)   ALARM
> chassis:  Chassis is normal.
> 
> Should I still try the UP kernel and an older kernel version?

Lets wait if the patch solve this. I think it should solve the fan7 for sure.
Thanks
Regards
Rudolf

[Fwd: Re: [lm-sensors] Kernel hangs with i2c-i801 driver?]

[Keith]

--- Daniel Nilsson <daniel.n.nilsson@home.se> wrote:

> On Sun, Nov 27, 2005 at 12:31:22PM +0100, Daniel
> Nilsson wrote:
> > I was finally able to get into a situation where I
> can reproduce the
> > hang in a repeatable fashion. I'm using this

...

I did some similar testing (writing values manually)
and was not able to make the box lock up until I
mistakenly wrote too small a value to vrm (at least I
think that's what did it); the box chugged along fine
until I started doing some work and it choked.  But
again I'm pretty sure it was my fault; after reboot
the BIOS complained about a previous voltage failure.
:)

I'm now at the point of assuming it was too tight of a
temperature range in the default sensors.conf.  With
temp1_min/max at 10-80 and temp2/temp3 at 10-50 I've
run sensors -s multiple times with no lockup.  I've
not changed anything else.

I still have fan readings of zero -- and it's not a
divisor problem considering there are no fan*_div
files in sysfs -- but this is another problem...

Keith




		
__________________________________________
Yahoo! DSL ? Something to write home about.
Just $16.99/mo. or less.
dsl.yahoo.com