Kernel Locking Up

Kernel Locking Up

[Travis Shirk]

Hello,

Ever since I upgraded to the 2.4.x (currently running 2.4.8)
kernels, my machine has been locking up every other day
or so.  Does anyone have any hints/tips for figuring out
what is going on.

The symptons are total lock-up of the machine.  No mouse
movement, all GUI monoitors freeze, and I cannot switch to a
virtual console.  I'm not able to ping the locked machine or
ssh/telnet into it either.  So I'm left wondering....how and
the hell to I debug this problem.  It'd be nice to have some
more information to go on or post to the list.

I'm running on a dual PIII 850, and this problem occurs with
2.4.7 and 2.4.8.

Any suggestions?

Travis

-- 
Travis Shirk <travis at pobox dot com>
Mathematics is God and Knuth is our prophet.

Re: Kernel Locking Up

[Tim Waugh]

[-- Attachment #1: Type: text/plain, Size: 312 bytes --]

On Wed, Aug 22, 2001 at 09:46:14AM -0600, Travis Shirk wrote:

> Ever since I upgraded to the 2.4.x (currently running 2.4.8)
> kernels, my machine has been locking up every other day
> or so.  Does anyone have any hints/tips for figuring out
> what is going on.

Stick a serial/parallel console on it.

Tim.
*/

[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]

Re: Kernel Locking Up

[Admin Mailing Lists]

1) re-direct console messages to a serial port
2) don't run a GUI for a while (so you can see console messages)

-Tony
.-._.-._.-._.-._.-._.-._.-._.-._.-._.-._.-._.-._.-._.-._.-._.-._.-._.-.
Anthony J. Biacco                       Network Administrator/Engineer
thelittleprince@asteroid-b612.org       Intergrafix Internet Services

    "Dream as if you'll live forever, live as if you'll die today"
http://www.asteroid-b612.org                http://www.intergrafix.net
.-._.-._.-._.-._.-._.-._.-._.-._.-._.-._.-._.-._.-._.-._.-._.-._.-._.-.

On Wed, 22 Aug 2001, Travis Shirk wrote:

> Hello,
> 
> Ever since I upgraded to the 2.4.x (currently running 2.4.8)
> kernels, my machine has been locking up every other day
> or so.  Does anyone have any hints/tips for figuring out
> what is going on.
> 
> The symptons are total lock-up of the machine.  No mouse
> movement, all GUI monoitors freeze, and I cannot switch to a
> virtual console.  I'm not able to ping the locked machine or
> ssh/telnet into it either.  So I'm left wondering....how and
> the hell to I debug this problem.  It'd be nice to have some
> more information to go on or post to the list.
> 
> I'm running on a dual PIII 850, and this problem occurs with
> 2.4.7 and 2.4.8.
> 
> Any suggestions?
> 
> Travis
> 
> -- 
> Travis Shirk <travis at pobox dot com>
> Mathematics is God and Knuth is our prophet.
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

Re: Kernel Locking Up

[Tim Walberg]

[-- Attachment #1: Type: text/plain, Size: 425 bytes --]

On 08/22/2001 11:52 -0400, Admin Mailing Lists wrote:
>>	
>>	1) re-direct console messages to a serial port

Don't have one available right now, although I could use
my laptop if I can find a couple days to live without it...

>>	2) don't run a GUI for a while (so you can see console messages)
>>	

Tried this, and even with maximum logging levels, there was no
message.

			tw

-- 
twalberg@mindspring.com

[-- Attachment #2: Type: application/pgp-signature, Size: 175 bytes --]

Re: Kernel Locking Up

[Tim Walberg]

[-- Attachment #1: Type: text/plain, Size: 2290 bytes --]

Not much help debugging it, but I can say I've seen the same
thing numerous times on kernels 2.4.[4678] and various -ac
patches (as well as stock kernels). I've tried disabling certain
"newer" features (kernel pppoe for example) and upgrading some
of the other software on the box that was suspect due to it being
relatively active at the time of crash (samba, X, etc.) but have
had no luck yet. I wish I had a serial console to plug in to
see if that turned anything up, because SysRQ doesn't even respond.

On a lighter note, I'm currently running 2.4.8-ac2, and while
the problem isn't gone, it does seem to be less frequent, so I'm
thinking I might try moving to a later 2.4.8-ac or even 2.4.9 if
I get the time this weekend.

I didn't see this at all on 2.4.2, which I ran for a couple
months, so my guess is it's related to something that changed
in 2.4.3 (which I skipped) and 2.4.4, but I haven't had the time
to even consider an exhaustive search through all those possibilities.

Good luck, and let me know if you find anything...

				tw

On 08/22/2001 09:46 -0600, Travis Shirk wrote:
>>	Hello,
>>	
>>	Ever since I upgraded to the 2.4.x (currently running 2.4.8)
>>	kernels, my machine has been locking up every other day
>>	or so.  Does anyone have any hints/tips for figuring out
>>	what is going on.
>>	
>>	The symptons are total lock-up of the machine.  No mouse
>>	movement, all GUI monoitors freeze, and I cannot switch to a
>>	virtual console.  I'm not able to ping the locked machine or
>>	ssh/telnet into it either.  So I'm left wondering....how and
>>	the hell to I debug this problem.  It'd be nice to have some
>>	more information to go on or post to the list.
>>	
>>	I'm running on a dual PIII 850, and this problem occurs with
>>	2.4.7 and 2.4.8.
>>	
>>	Any suggestions?
>>	
>>	Travis
>>	
>>	-- 
>>	Travis Shirk <travis at pobox dot com>
>>	Mathematics is God and Knuth is our prophet.
>>	
>>	-
>>	To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>>	the body of a message to majordomo@vger.kernel.org
>>	More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>	Please read the FAQ at  http://www.tux.org/lkml/
End of included message



-- 
twalberg@mindspring.com

[-- Attachment #2: Type: application/pgp-signature, Size: 175 bytes --]

Re: Kernel Locking Up

[Alan Cox]

> The symptons are total lock-up of the machine.  No mouse
> movement, all GUI monoitors freeze, and I cannot switch to a
> virtual console.  I'm not able to ping the locked machine or
> ssh/telnet into it either.  So I'm left wondering....how and
> the hell to I debug this problem.  It'd be nice to have some
> more information to go on or post to the list.

Can you get it to crash when you are not in X11 at all ?

Re: Kernel Locking Up

[Tim Walberg]

[-- Attachment #1: Type: text/plain, Size: 937 bytes --]

Yes, I have seen it happen a couple times before
I started X, within a couple minutes of boot completing.

		tw

On 08/22/2001 18:23 +0100, Alan Cox wrote:
>>	> The symptons are total lock-up of the machine.  No mouse
>>	> movement, all GUI monoitors freeze, and I cannot switch to a
>>	> virtual console.  I'm not able to ping the locked machine or
>>	> ssh/telnet into it either.  So I'm left wondering....how and
>>	> the hell to I debug this problem.  It'd be nice to have some
>>	> more information to go on or post to the list.
>>	
>>	Can you get it to crash when you are not in X11 at all ?
>>	-
>>	To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>>	the body of a message to majordomo@vger.kernel.org
>>	More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>	Please read the FAQ at  http://www.tux.org/lkml/
End of included message



-- 
twalberg@mindspring.com

[-- Attachment #2: Type: application/pgp-signature, Size: 175 bytes --]

RE: Kernel Locking Up

[Kevin Krieser]

I had some lockups too when I went to the 2.4 kernels.  I ended up removing
a SCSI card, and adding a fan to cool off my computer.  I was up for 12 days
recently before I noticed that my USB printer wasn't working, and I saw I
had compiled the wrong USB controllers into my kernel.  It has now been up
for almost 4 days now with a properly configured stock 2.4.8 kernel.

-----Original Message-----
From: linux-kernel-owner@vger.kernel.org
[mailto:linux-kernel-owner@vger.kernel.org]On Behalf Of Tim Walberg
Sent: Wednesday, August 22, 2001 12:34 PM
To: Alan Cox
Cc: Travis Shirk; Linux Kernel Mailing List
Subject: Re: Kernel Locking Up


Yes, I have seen it happen a couple times before
I started X, within a couple minutes of boot completing.

		tw

Re: Kernel Locking Up

[Travis Shirk]

On Wed, 22 Aug 2001, Alan Cox wrote:

>> The symptons are total lock-up of the machine.  No mouse
>> movement, all GUI monoitors freeze, and I cannot switch to a
>> virtual console.  I'm not able to ping the locked machine or
>> ssh/telnet into it either.  So I'm left wondering....how and
>> the hell to I debug this problem.  It'd be nice to have some
>> more information to go on or post to the list.
>
>Can you get it to crash when you are not in X11 at all ?

I have not, but I do not spend too much time outside of
the GUI.  I don't boot into X, so unless I'm logged out
X is running.

Travis
-- 
Travis Shirk <travis at pobox dot com>
Mathematics is God and Knuth is our prophet.

Re: Kernel Locking Up

[Ted Deppner]

On Wed, Aug 22, 2001 at 09:46:14AM -0600, Travis Shirk wrote:
> Ever since I upgraded to the 2.4.x (currently running 2.4.8)
> kernels, my machine has been locking up every other day
> or so.  Does anyone have any hints/tips for figuring out
> what is going on.

As another data point, I've had similar problems with one machine (the
heaviest utilized), but none others.  I'm running about 20 2.4.x machines,
in various uses (I work for an ISP).

Kernels 2.4.6 through 2.4.7, and even a 2.4.7-ac8 I tried for good
measure.

The one running on a Dell PowerEdge 2450, dual P3-750s, 512mb ram, Mylex
ExcelRaid 2000, Intel EEPRO100, running a qmail setup transiting 20 to 40k
messages per day regularly locks up every 3 to 8 days.  No dmesg, no error
logs, no oops, nothing on the console.  

The death spiral didn't seem triggered by any particular thing, and
logged in ssh terminals were the still slightly usable for 2 to 5 commands
but then they'd stop working.

2.4.8 hasn't had any problems so far in 5 days, but I'm not holding my
breath too tightly.

The ONLY similarity in the death throws has been what looks like physical
cable or network card interrupt problems.  The packets in my ssh session
seem to block until I hit enter 5 to 10 times, then I get a burst of
traffic.  I've been able to do some simple commands in this situation, but
nothing complex.

ifconfig eth0 has shown millions of various errors (carrier, collisions),
and hundreds of thousands of them between typing the command in twice.
The Cisco 6000 series switch on the other side of the cable shows no such
errors.

-- 
Ted Deppner
http://www.psyber.com/~ted/

Re: Kernel Locking Up

[Ben Greear]

Ted Deppner wrote:
> 
> On Wed, Aug 22, 2001 at 09:46:14AM -0600, Travis Shirk wrote:
> > Ever since I upgraded to the 2.4.x (currently running 2.4.8)
> > kernels, my machine has been locking up every other day
> > or so.  Does anyone have any hints/tips for figuring out
> > what is going on.
> 
> As another data point, I've had similar problems with one machine (the
> heaviest utilized), but none others.  I'm running about 20 2.4.x machines,
> in various uses (I work for an ISP).
> 
> Kernels 2.4.6 through 2.4.7, and even a 2.4.7-ac8 I tried for good
> measure.
> 
> The one running on a Dell PowerEdge 2450, dual P3-750s, 512mb ram, Mylex
> ExcelRaid 2000, Intel EEPRO100, running a qmail setup transiting 20 to 40k
> messages per day regularly locks up every 3 to 8 days.  No dmesg, no error
> logs, no oops, nothing on the console.

If the latest kernel still blows up, try using Intel's e100 driver..
The eepro100 driver has been flaky from time to time on certain
chipsets...

Ben

> ifconfig eth0 has shown millions of various errors (carrier, collisions),
> and hundreds of thousands of them between typing the command in twice.
> The Cisco 6000 series switch on the other side of the cable shows no such
> errors.
> 
> --
> Ted Deppner
> http://www.psyber.com/~ted/
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

-- 
Ben Greear <greearb@candelatech.com>          <Ben_Greear@excite.com>
President of Candela Technologies Inc      http://www.candelatech.com
ScryMUD:  http://scry.wanfear.com     http://scry.wanfear.com/~greear

Re: Kernel Locking Up

[Keith Owens]

On Wed, 22 Aug 2001 09:46:14 -0600 (MDT), 
Travis Shirk <travis@pobox.com> wrote:
>Ever since I upgraded to the 2.4.x (currently running 2.4.8)
>kernels, my machine has been locking up every other day
>or so.  Does anyone have any hints/tips for figuring out
>what is going on.

Kernel Debugger + serial console.
ftp://oss.sgi.com/projects/kdb/download/ix86/.  man Documentation/kdb.mm.
Documentation/serial-console.txt.

Make sure that you enable the NMI watchdog, boot 2.4.x kernels with
"nmi_watchdog=1".  Unless the problem is a total hardware lockup, nmi
will trip after 5 seconds and drop into kdb.