2.6.33-rt4 core2duo oopses: what method to catch

2.6.33-rt4 core2duo oopses: what method to catch

[ulmo]

I'm getting kernel faults in 2.6.33-rt4 on a core 2 duo.  It takes a
moment to hard-crash, so there's a chance for the output to be diagnosed. 
A few questions regarding that:

Please point me the way to best get the diagnostic data (kernel trace).

There used to be utilities to use System.map to give names to traces (like
sysklogd).  There also used to be a built-in method for System.map so the
kernel could do that.  What's the option?  What are my options?  I
currently use metalog for the syslog daemon.

What's the easiest way to write the oops to disk while in the initrd?  I'm
getting the oopses while it's doing reiserfsck, and I only have a bit of
room in /boot to stick some output.  Is there a way to log to /boot the
oopses, with some small initrd-fitting binary that will cull kernel
logging for those and write them out?  Just a standard logger?

After that I can try to get some output for us to look at.  Thanks.

Re: 2.6.33-rt4 core2duo oopses: what method to catch

[Uwe Kleine-König]

Hi,

On Sun, Feb 28, 2010 at 03:49:38AM -0800, ulmo@sonic.net wrote:
> I'm getting kernel faults in 2.6.33-rt4 on a core 2 duo.  It takes a
> moment to hard-crash, so there's a chance for the output to be diagnosed. 
> A few questions regarding that:
> 
> Please point me the way to best get the diagnostic data (kernel trace).
> 
> There used to be utilities to use System.map to give names to traces (like
> sysklogd).  There also used to be a built-in method for System.map so the
> kernel could do that.  What's the option?  What are my options?  I
> currently use metalog for the syslog daemon.
I don't know if this is what you mean, but KALLSYMS (General setup ->
Configure standard kernel features -> Load all symbols for
debugging/ksymoops) and its subitems should be helpful.

> What's the easiest way to write the oops to disk while in the initrd?  I'm
> getting the oopses while it's doing reiserfsck, and I only have a bit of
> room in /boot to stick some output.  Is there a way to log to /boot the
> oopses, with some small initrd-fitting binary that will cull kernel
> logging for those and write them out?  Just a standard logger?
If you get an oops, the best way to catch it is via a serial console.
Then you can just copy&paste from another machine.

Best regards,
Uwe

-- 
Pengutronix e.K.                           | Uwe Kleine-König            |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
--
To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: 2.6.33-rt4 core2duo oopses: what method to catch

[Thomas Gleixner]

On Sun, 28 Feb 2010, ulmo@sonic.net wrote:

> I'm getting kernel faults in 2.6.33-rt4 on a core 2 duo.  It takes a
> moment to hard-crash, so there's a chance for the output to be diagnosed. 

Does the same problem happen with vanilla 2.6.33 ?
 
> Please point me the way to best get the diagnostic data (kernel trace).
> 
> There used to be utilities to use System.map to give names to traces (like
> sysklogd).  There also used to be a built-in method for System.map so the
> kernel could do that.  What's the option?  What are my options?  I
> currently use metalog for the syslog daemon.

CONFIG_KALLSYMS=y
CONFIG_KALLSYMS_ALL=y
CONFIG_KALLSYMS_EXTRA_PASS=y

CONFIG_DEBUG_BUGVERBOSE=y
 
> What's the easiest way to write the oops to disk while in the initrd?  I'm
> getting the oopses while it's doing reiserfsck, and I only have a bit of
> room in /boot to stick some output.  Is there a way to log to /boot the
> oopses, with some small initrd-fitting binary that will cull kernel
> logging for those and write them out?  Just a standard logger?
> 
> After that I can try to get some output for us to look at.  Thanks.

Either serial console - requires a serial port - or netconsole. For
both you need a second computer to log the output.

Thanks,

	tglx

Re: 2.6.33-rt4 core2duo oopses: what method to catch

[Mark Knecht]

On Mon, Mar 1, 2010 at 5:11 AM, Thomas Gleixner <tglx@linutronix.de> wrote:
> On Sun, 28 Feb 2010, ulmo@sonic.net wrote:
>
>> I'm getting kernel faults in 2.6.33-rt4 on a core 2 duo.  It takes a
>> moment to hard-crash, so there's a chance for the output to be diagnosed.
>
> Does the same problem happen with vanilla 2.6.33 ?
>
>> Please point me the way to best get the diagnostic data (kernel trace).
>>
>> There used to be utilities to use System.map to give names to traces (like
>> sysklogd).  There also used to be a built-in method for System.map so the
>> kernel could do that.  What's the option?  What are my options?  I
>> currently use metalog for the syslog daemon.
>
> CONFIG_KALLSYMS=y
> CONFIG_KALLSYMS_ALL=y
> CONFIG_KALLSYMS_EXTRA_PASS=y
>
> CONFIG_DEBUG_BUGVERBOSE=y
>
>> What's the easiest way to write the oops to disk while in the initrd?  I'm
>> getting the oopses while it's doing reiserfsck, and I only have a bit of
>> room in /boot to stick some output.  Is there a way to log to /boot the
>> oopses, with some small initrd-fitting binary that will cull kernel
>> logging for those and write them out?  Just a standard logger?
>>
>> After that I can try to get some output for us to look at.  Thanks.
>
> Either serial console - requires a serial port - or netconsole. For
> both you need a second computer to log the output.
>
> Thanks,
>
>        tglx

I haven't heard of netconsole before but it makes sense. I did this
using the serial console a couple of times but none of my machines
have serial ports anymore (and could I find the right cable these days
anyway?)

Is there a good page describing how to set this up and use it?

Thanks,
Mark
--
To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: 2.6.33-rt4 core2duo oopses: what method to catch

[John Kacur]

On Mon, Mar 1, 2010 at 6:17 PM, Mark Knecht <markknecht@gmail.com> wrote:
> On Mon, Mar 1, 2010 at 5:11 AM, Thomas Gleixner <tglx@linutronix.de> wrote:
>> On Sun, 28 Feb 2010, ulmo@sonic.net wrote:
>>
>>> I'm getting kernel faults in 2.6.33-rt4 on a core 2 duo.  It takes a
>>> moment to hard-crash, so there's a chance for the output to be diagnosed.
>>
>> Does the same problem happen with vanilla 2.6.33 ?
>>
>>> Please point me the way to best get the diagnostic data (kernel trace).
>>>
>>> There used to be utilities to use System.map to give names to traces (like
>>> sysklogd).  There also used to be a built-in method for System.map so the
>>> kernel could do that.  What's the option?  What are my options?  I
>>> currently use metalog for the syslog daemon.
>>
>> CONFIG_KALLSYMS=y
>> CONFIG_KALLSYMS_ALL=y
>> CONFIG_KALLSYMS_EXTRA_PASS=y
>>
>> CONFIG_DEBUG_BUGVERBOSE=y
>>
>>> What's the easiest way to write the oops to disk while in the initrd?  I'm
>>> getting the oopses while it's doing reiserfsck, and I only have a bit of
>>> room in /boot to stick some output.  Is there a way to log to /boot the
>>> oopses, with some small initrd-fitting binary that will cull kernel
>>> logging for those and write them out?  Just a standard logger?
>>>
>>> After that I can try to get some output for us to look at.  Thanks.
>>
>> Either serial console - requires a serial port - or netconsole. For
>> both you need a second computer to log the output.
>>
>> Thanks,
>>
>>        tglx
>
> I haven't heard of netconsole before but it makes sense. I did this
> using the serial console a couple of times but none of my machines
> have serial ports anymore (and could I find the right cable these days
> anyway?)
>
> Is there a good page describing how to set this up and use it?
>

In the kernel itself, Documentation/networking/netconsole.txt
--
To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html