how to capture kernel panics

how to capture kernel panics

[shabanip]

is there any way to capture and log kernel panics on disk or ...?

Re: how to capture kernel panics

[James Colannino]

shabanip wrote:

>is there any way to capture and log kernel panics on disk or ...?
>  
>

My guess would be, at the very least, it depends on what part of the 
kernel is causing the panic.  Most likely I would say no, although 
here's another question: if running a second kernel under user-mode 
Linux, can this be done?

James

Re: how to capture kernel panics

[Trond Hasle Amundsen]

James Colannino <lkml@colannino.org> writes:

> shabanip wrote:
>
>>is there any way to capture and log kernel panics on disk or ...?
>
> My guess would be, at the very least, it depends on what part of the
> kernel is causing the panic.

A kernel panic means that the kernel no longer knows what it's doing,
and therefore stops doing anything immediately. Hence it won't use the
filesystems and cannot log the panic to anything but the console. I
would think the best solution to your problem is to set up a serial
console to another machine, and log everything to disk on that
machine. See Documentation/serial-console.txt for how to set up a
serial console.

Regards,

-- 
Trond Hasle Amundsen <t.h.amundsen@usit.uio.no>
Center for Information Technology Services, University of Oslo

Re: how to capture kernel panics

[Christian Borntraeger]

shabanip wrote:
> is there any way to capture and log kernel panics on disk or ...?

In former times, the Linux kernel tried to sync in the panic function. (If 
the panic did not happen in interrupt context) Unfortunately this had 
severe side effects in cases where the panic was triggered by file system 
block device code or any other part which is necessary for syncing. In most 
cases the call trace never made it onto disk anyway. So currently the 
kernel does not support saving a panic.

Apart from using a serial console, you might have a look at several 
kexec/kdump/lkcd tools where people are working on being able to dump the 
memory of a paniced kernel.

cheers

Christian

Re: how to capture kernel panics

[Jeff Moyer]

==> Regarding Re: how to capture kernel panics; Christian Borntraeger <linux-kernel@borntraeger.net> adds:

linux-kernel> shabanip wrote:
>> is there any way to capture and log kernel panics on disk or ...?

linux-kernel> In former times, the Linux kernel tried to sync in the panic
linux-kernel> function. (If the panic did not happen in interrupt context)
linux-kernel> Unfortunately this had severe side effects in cases where the
linux-kernel> panic was triggered by file system block device code or any
linux-kernel> other part which is necessary for syncing. In most cases the
linux-kernel> call trace never made it onto disk anyway. So currently the
linux-kernel> kernel does not support saving a panic.

linux-kernel> Apart from using a serial console, you might have a look at
linux-kernel> several kexec/kdump/lkcd tools where people are working on
linux-kernel> being able to dump the memory of a paniced kernel.

Or netconsole, which will dump printk's do the server:port of your
choosing.

-Jeff

Re: how to capture kernel panics

[shabanip]

as i see netconsole is a kernel module.
so i just need to load netconsole module with server:port parameters.
am i right?

Payam Shabanian
shabanip -at- avapajoohesh.com

> ==> Regarding Re: how to capture kernel panics; Christian Borntraeger
> <linux-kernel@borntraeger.net> adds:
>
> linux-kernel> shabanip wrote:
>>> is there any way to capture and log kernel panics on disk or ...?
>
> linux-kernel> In former times, the Linux kernel tried to sync in the panic
> linux-kernel> function. (If the panic did not happen in interrupt context)
> linux-kernel> Unfortunately this had severe side effects in cases where
> the
> linux-kernel> panic was triggered by file system block device code or any
> linux-kernel> other part which is necessary for syncing. In most cases the
> linux-kernel> call trace never made it onto disk anyway. So currently the
> linux-kernel> kernel does not support saving a panic.
>
> linux-kernel> Apart from using a serial console, you might have a look at
> linux-kernel> several kexec/kdump/lkcd tools where people are working on
> linux-kernel> being able to dump the memory of a paniced kernel.
>
> Or netconsole, which will dump printk's do the server:port of your
> choosing.
>
> -Jeff
>

Re: how to capture kernel panics

[Jeff Moyer]

==> Regarding Re: how to capture kernel panics; "shabanip" <shabanip@avapajoohesh.com> adds:

shabanip> as i see netconsole is a kernel module.  so i just need to load
shabanip> netconsole module with server:port parameters.  am i right?

MODULE_PARM_DESC(netconsole, " netconsole=[src-port]@[src-ip]/[dev],[tgt-port]@<tgt-ip>/[tgt-macaddr]\n");

So, for example:

modprobe netconsole netconsole=6666@192.168.1.1/eth0,6666@192.168.1.100/00:40:95:9A:12:34

-Jeff


shabanip> Payam Shabanian shabanip -at- avapajoohesh.com

>> ==> Regarding Re: how to capture kernel panics; Christian Borntraeger
>> <linux-kernel@borntraeger.net> adds:
>> 
linux-kernel> shabanip wrote:
>>>> is there any way to capture and log kernel panics on disk or ...?
>>
linux-kernel> In former times, the Linux kernel tried to sync in the panic
linux-kernel> function. (If the panic did not happen in interrupt context)
linux-kernel> Unfortunately this had severe side effects in cases where
>> the
linux-kernel> panic was triggered by file system block device code or any
linux-kernel> other part which is necessary for syncing. In most cases the
linux-kernel> call trace never made it onto disk anyway. So currently the
linux-kernel> kernel does not support saving a panic.
>>
linux-kernel> Apart from using a serial console, you might have a look at
linux-kernel> several kexec/kdump/lkcd tools where people are working on
linux-kernel> being able to dump the memory of a paniced kernel.
>> Or netconsole, which will dump printk's do the server:port of your
>> choosing.
>> 
>> -Jeff
>> 

shabanip> - To unsubscribe from this list: send the line "unsubscribe
shabanip> linux-kernel" in the body of a message to
shabanip> majordomo@vger.kernel.org More majordomo info at
shabanip> http://vger.kernel.org/majordomo-info.html Please read the FAQ at
shabanip> http://www.tux.org/lkml/

Re: how to capture kernel panics

[shabanip]

thanks for the help.
does anything else left to configure?

Payam Shabanian
shabanip -at- avapajoohesh.com


> ==> Regarding Re: how to capture kernel panics; "shabanip"
> <shabanip@avapajoohesh.com> adds:
>
> shabanip> as i see netconsole is a kernel module.  so i just need to load
> shabanip> netconsole module with server:port parameters.  am i right?
>
> MODULE_PARM_DESC(netconsole, "
> netconsole=[src-port]@[src-ip]/[dev],[tgt-port]@<tgt-ip>/[tgt-macaddr]\n");
>
> So, for example:
>
> modprobe netconsole
> netconsole=6666@192.168.1.1/eth0,6666@192.168.1.100/00:40:95:9A:12:34
>
> -Jeff
>
>
> shabanip> Payam Shabanian shabanip -at- avapajoohesh.com
>
>>> ==> Regarding Re: how to capture kernel panics; Christian Borntraeger
>>> <linux-kernel@borntraeger.net> adds:
>>>
> linux-kernel> shabanip wrote:
>>>>> is there any way to capture and log kernel panics on disk or ...?
>>>
> linux-kernel> In former times, the Linux kernel tried to sync in the panic
> linux-kernel> function. (If the panic did not happen in interrupt context)
> linux-kernel> Unfortunately this had severe side effects in cases where
>>> the
> linux-kernel> panic was triggered by file system block device code or any
> linux-kernel> other part which is necessary for syncing. In most cases the
> linux-kernel> call trace never made it onto disk anyway. So currently the
> linux-kernel> kernel does not support saving a panic.
>>>
> linux-kernel> Apart from using a serial console, you might have a look at
> linux-kernel> several kexec/kdump/lkcd tools where people are working on
> linux-kernel> being able to dump the memory of a paniced kernel.
>>> Or netconsole, which will dump printk's do the server:port of your
>>> choosing.
>>>
>>> -Jeff
>>>
>
> shabanip> - To unsubscribe from this list: send the line "unsubscribe
> shabanip> linux-kernel" in the body of a message to
> shabanip> majordomo@vger.kernel.org More majordomo info at
> shabanip> http://vger.kernel.org/majordomo-info.html Please read the FAQ
> at
> shabanip> http://www.tux.org/lkml/
>

Re: how to capture kernel panics

[Jeff Moyer]

==> Regarding Re: how to capture kernel panics; "shabanip" <shabanip@avapajoohesh.com> adds:

shabanip> thanks for the help.  does anything else left to configure?

On the log target system, you will probably want to run something similar
to netcat to capture the logs.  To simply output to a terminal, you could
do something like this:

nc -l -p 6666 -u

-Jeff

shabanip> Payam Shabanian shabanip -at- avapajoohesh.com


>> ==> Regarding Re: how to capture kernel panics; "shabanip"
>> <shabanip@avapajoohesh.com> adds:
>> 
shabanip> as i see netconsole is a kernel module.  so i just need to load
shabanip> netconsole module with server:port parameters.  am i right?
>> MODULE_PARM_DESC(netconsole, "
>> netconsole=[src-port]@[src-ip]/[dev],[tgt-port]@<tgt-ip>/[tgt-macaddr]\n");
>> 
>> So, for example:
>> 
>> modprobe netconsole
>> netconsole=6666@192.168.1.1/eth0,6666@192.168.1.100/00:40:95:9A:12:34
>> 
>> -Jeff
>> 
>> 
shabanip> Payam Shabanian shabanip -at- avapajoohesh.com
>>
>>>> ==> Regarding Re: how to capture kernel panics; Christian Borntraeger
>>>> <linux-kernel@borntraeger.net> adds:
>>>> 
linux-kernel> shabanip wrote:
>>>>>> is there any way to capture and log kernel panics on disk or ...?
>>>>
linux-kernel> In former times, the Linux kernel tried to sync in the panic
linux-kernel> function. (If the panic did not happen in interrupt context)
linux-kernel> Unfortunately this had severe side effects in cases where
>>>> the
linux-kernel> panic was triggered by file system block device code or any
linux-kernel> other part which is necessary for syncing. In most cases the
linux-kernel> call trace never made it onto disk anyway. So currently the
linux-kernel> kernel does not support saving a panic.
>>>>
linux-kernel> Apart from using a serial console, you might have a look at
linux-kernel> several kexec/kdump/lkcd tools where people are working on
linux-kernel> being able to dump the memory of a paniced kernel.
>>>> Or netconsole, which will dump printk's do the server:port of your
>>>> choosing.
>>>> 
>>>> -Jeff
>>>> 
>>
shabanip> - To unsubscribe from this list: send the line "unsubscribe
shabanip> linux-kernel" in the body of a message to
shabanip> majordomo@vger.kernel.org More majordomo info at
shabanip> http://vger.kernel.org/majordomo-info.html Please read the FAQ
>> at
shabanip> http://www.tux.org/lkml/
>>