[lm-sensors] lm87 alarm threshold changed mysteriously

[lm-sensors] lm87 alarm threshold changed mysteriously

[David Knierim]

We have a product based on the Intel E7501 chipset which uses two LM87
sensor chips and the i2c-i801 bus driver.  We have fielded several
hundred of these boxes over the past two years.  They are all running
Red Hat 9 with the version of sensors that came with it (the rpm
version is 2.6.5-5).  Yes, I know this an ancient release...

I got a call yesterday that one of the boxes had started getting
voltage alarms.  After some investigation, I determined the problem
was that the low voltage alarm threshold for the +V2.5 power plane had
changed to 3.32V (from the normal value of 2.37V.   The high voltage
threshold had not changed for the normal setting of  2.61V.   The
measured voltage of 2.48V looked fine, too.

After running sensors -s, the problem went away (as I expected it to).

So I have a bunch of questions:
1 - is it likely that this problem would be fixed in a newer release
of lm_sensors?
2 - if it's not fixed, how would one track down the problem so we could fix it?
3 - is it possible this is a hardware problem?
4 - Any thoughs on recovering from problem in a more automated fashion
until a fix is found?

Any thoughts are welcome.

David

[lm-sensors] lm87 alarm threshold changed mysteriously

[Mark Studebaker]

I don't see anything in CHANGES or the CVS log that sounds like it would fix this.
Strange since I assume there are no writes happening to any registers in normal operation, just reads for monitoring.
Could be noise on the I2C bus or even cosmic rays that flipped a bit in the LM87.
The server wasn't in Denver, was it (high altitude)?
Sorry no great ideas on how to answer 2-4.
Hope it doesn't happen again...
mds


David Knierim wrote:
> We have a product based on the Intel E7501 chipset which uses two LM87
> sensor chips and the i2c-i801 bus driver.  We have fielded several
> hundred of these boxes over the past two years.  They are all running
> Red Hat 9 with the version of sensors that came with it (the rpm
> version is 2.6.5-5).  Yes, I know this an ancient release...
> 
> I got a call yesterday that one of the boxes had started getting
> voltage alarms.  After some investigation, I determined the problem
> was that the low voltage alarm threshold for the +V2.5 power plane had
> changed to 3.32V (from the normal value of 2.37V.   The high voltage
> threshold had not changed for the normal setting of  2.61V.   The
> measured voltage of 2.48V looked fine, too.
> 
> After running sensors -s, the problem went away (as I expected it to).
> 
> So I have a bunch of questions:
> 1 - is it likely that this problem would be fixed in a newer release
> of lm_sensors?
> 2 - if it's not fixed, how would one track down the problem so we could fix it?
> 3 - is it possible this is a hardware problem?
> 4 - Any thoughs on recovering from problem in a more automated fashion
> until a fix is found?
> 
> Any thoughts are welcome.
> 
> David
> 
> _______________________________________________
> lm-sensors mailing list
> lm-sensors@lm-sensors.org
> http://lists.lm-sensors.org/mailman/listinfo/lm-sensors

[lm-sensors] lm87 alarm threshold changed mysteriously

[Jean Delvare]

Hi David,

Trying to complement Mark's answer:

> We have a product based on the Intel E7501 chipset which uses two LM87
> sensor chips and the i2c-i801 bus driver.  We have fielded several
> hundred of these boxes over the past two years.  They are all running
> Red Hat 9 with the version of sensors that came with it (the rpm
> version is 2.6.5-5).  Yes, I know this an ancient release...
> 
> I got a call yesterday that one of the boxes had started getting
> voltage alarms.  After some investigation, I determined the problem
> was that the low voltage alarm threshold for the +V2.5 power plane had
> changed to 3.32V (from the normal value of 2.37V.   The high voltage
> threshold had not changed for the normal setting of  2.61V.   The
> measured voltage of 2.48V looked fine, too.
> 
> After running sensors -s, the problem went away (as I expected it to).
> 
> So I have a bunch of questions:
> 1 - is it likely that this problem would be fixed in a newer release
> of lm_sensors?

Probably not, for the simple reason that I don't think this is a
lm_sensors bug in the first place. I'd rather suspect a hardware defect.

> 2 - if it's not fixed, how would one track down the problem so we
> could fix it?

If the problem happens frequently enough, you could enable debugging in
i2c-i801 so that all bus writes are logged. This would tell you whether
the faulty limit was actually set through the linux drivers, or if the
chip changed the value on its own. In the latter case, there's nothing
we can do, obviously.

> 3 - is it possible this is a hardware problem?

Note that 3.32V is no random register value, it's 0xff. So my theory is
that this was a hardware problem in the LM87 itself, which lost a
register value.

> 4 - Any thoughs on recovering from problem in a more automated fashion
> until a fix is found?

Not without voiding the whole point of hardware monitoring.

-- 
Jean Delvare