kernel debugging

kernel debugging

[Steve Iribarne (GMail)]

Hello.

This is more a general question to see what others do out here.  I am
begining to get sick of printk debugging.  I work on two different PPC
boards.  An 860 and 8260.

I want to get some feedback on the best kernel debugger to use.  I
have been looking at three.

1.  kgdb
2.  kdb
3.  UML

I am leaning towards kgdb, but before I jump in I thought I'd put this
out to the best group I could think of linuxppc.  Because I am sure
most of you are using something!  :)

Thanks.

-stv

Re: kernel debugging

[David Hawkins]

> This is more a general question to see what others do out here.  I am
> begining to get sick of printk debugging.  I work on two different PPC
> boards.  An 860 and 8260.
> 
> I want to get some feedback on the best kernel debugger to use.  I
> have been looking at three.
> 
> 1.  kgdb
> 2.  kdb
> 3.  UML
> 
> I am leaning towards kgdb, but before I jump in I thought I'd put this
> out to the best group I could think of linuxppc.  Because I am sure
> most of you are using something!  :)

Hey Steve,

You've missed the most important one; a hardware debugger. I believe
most people using PPC use a BDI2000 from Abatron. They're about $3k.
I recently received one, but haven't had time to work on my PPC
stuff, so can't comment further. I'm sure everyone else on the
list will help with comments.

Note that if you use a bunch of different PPC cores, you need to
add $1k per core for the firmware uploads to the BDI2000. So if
you are working with say AMCC 440, Freescale e300/G2, e500, etc
you'll need a firmware image for each due to the debug core
differences (I guess).

The US distributor for the BDI2000 is Ultimate Solutions,
www.ultsol.com.

Cheers
Dave

Re: kernel debugging

[David H. Lynch Jr.]

    Everyone has their own debugging style.

    Engineers seem to like hardware debugging tools. I have used some
very fancy debugging hardware, but except for extremely rare instances
it is more work to get setup
    and figure out what you are trying to do than inserting some
debugging and rebuilding.

    My idea of debugging hardware is a port with an LED on it I can try
to blink.

    I also only rarely use software debuggers.

    Most of the time when things go off the rails the critical question
for me is Where did things go wrong. Once I know that usually the
problem is obvious and I do nto need dumps of variables or memory.

    I also do development across numerous platforms, OS's and languages.
I need debugging tools and techniques that are broadly portable. A
hardware debugging tool might help with board bringup, but it would be
of little use
    for web or perl programming. Investing time and capitol in highly
specialized tools or knowledge requires being narrowly focused to get a
worthwhile payback.
   
    Regardless, I think debugging is a sort of religious preference. You
need to know who you are and what you need. Other peoples experience is
useful but should not be determinative.

Steve Iribarne (GMail) wrote:
> Hello.
>
> This is more a general question to see what others do out here.  I am
> begining to get sick of printk debugging.  I work on two different PPC
> boards.  An 860 and 8260.
>
> I want to get some feedback on the best kernel debugger to use.  I
> have been looking at three.
>
> 1.  kgdb
> 2.  kdb
> 3.  UML
>
> I am leaning towards kgdb, but before I jump in I thought I'd put this
> out to the best group I could think of linuxppc.  Because I am sure
> most of you are using something!  :)
>
> Thanks.
>
> -stv
> _______________________________________________
> Linuxppc-embedded mailing list
> Linuxppc-embedded@ozlabs.org
> https://ozlabs.org/mailman/listinfo/linuxppc-embedded
>   


-- 
Dave Lynch 						DLA Systems
Software Development:  				     Embedded Linux
717.627.3770 	     dhlii@dlasys.net 	      http://www.dlasys.net
fax: 1.253.369.9244 			       Cell: 1.717.587.7774
Over 25 years' experience in platforms, languages, and technologies too numerous to list.

Re: kernel debugging

[Steve Iribarne (GMail)]

On 5/4/06, David H. Lynch Jr. <dhlii@dlasys.net> wrote:
>     Everyone has their own debugging style.
>
>     Engineers seem to like hardware debugging tools. I have used some
> very fancy debugging hardware, but except for extremely rare instances
> it is more work to get setup
>     and figure out what you are trying to do than inserting some
> debugging and rebuilding.
>
>     My idea of debugging hardware is a port with an LED on it I can try
> to blink.
>
>     I also only rarely use software debuggers.
>
>     Most of the time when things go off the rails the critical question
> for me is Where did things go wrong. Once I know that usually the
> problem is obvious and I do nto need dumps of variables or memory.
>
>     I also do development across numerous platforms, OS's and languages.
> I need debugging tools and techniques that are broadly portable. A
> hardware debugging tool might help with board bringup, but it would be
> of little use
>     for web or perl programming. Investing time and capitol in highly
> specialized tools or knowledge requires being narrowly focused to get a
> worthwhile payback.
>
>     Regardless, I think debugging is a sort of religious preference. You
> need to know who you are and what you need. Other peoples experience is
> useful but should not be determinative.
>

Yes I am finding this out.  It's a long journey!  :)

Most of the issues I have to deal with are systems that are already up
and running in the PPC world and I come in afterwords to clean up.  So
for example, right now I think I have a DMA problem that is crashing
my PPC and rebooting the system.

To be sure, I would love to step over a few instructions and slow the
world down a bit.  That's why I was going to setup kgdb or something
like that.

Thanks.

-stv


> Steve Iribarne (GMail) wrote:
> > Hello.
> >
> > This is more a general question to see what others do out here.  I am
> > begining to get sick of printk debugging.  I work on two different PPC
> > boards.  An 860 and 8260.
> >
> > I want to get some feedback on the best kernel debugger to use.  I
> > have been looking at three.
> >
> > 1.  kgdb
> > 2.  kdb
> > 3.  UML
> >
> > I am leaning towards kgdb, but before I jump in I thought I'd put this
> > out to the best group I could think of linuxppc.  Because I am sure
> > most of you are using something!  :)
> >
> > Thanks.
> >
> > -stv
> > _______________________________________________
> > Linuxppc-embedded mailing list
> > Linuxppc-embedded@ozlabs.org
> > https://ozlabs.org/mailman/listinfo/linuxppc-embedded
> >
>
>
> --
> Dave Lynch                                              DLA Systems
> Software Development:                                Embedded Linux
> 717.627.3770         dhlii@dlasys.net         http://www.dlasys.net
> fax: 1.253.369.9244                            Cell: 1.717.587.7774
> Over 25 years' experience in platforms, languages, and technologies too n=
umerous to list.
>
>

Re: kernel debugging

[Mark Chambers]

>> >
>> > This is more a general question to see what others do out here.  I am
>> > begining to get sick of printk debugging.  I work on two different PPC
>> > boards.  An 860 and 8260.
>> >
>> > I want to get some feedback on the best kernel debugger to use.  I
>> > have been looking at three.
>> >
>> > 1.  kgdb
>> > 2.  kdb
>> > 3.  UML
>> >

For the 860 you can purchase a hardware debugger from www.denx.de for
50 Euros.  For the 8260 you must buy the more expensive BDI2000, but
that is Freescale's fault.  But the look and feel of BDI2000 is the same as
BDI4GDB, just faster, so you can decide whether it's worth the money for 
you.
When you say 'kgdb' you imply 'gdb' which is the standard GNU-world
debugger.  kgdb is a means for letting a kernel communicate with a PC based
(or equivalent) gdb and is an alternative to a hardware debugger.  Also,
check out ddd, a front end for gdb.

IMHO, these serial debugging ports on PPC are the greatest thing since 
sliced
bread and it would be foolish not to take advantage of them.

Mark C. 

Kernel debugging

[Pankaj Vinadrao Joshi]

[-- Attachment #1.1: Type: text/plain, Size: 207 bytes --]

Hi,
I am using linux 5.4.3 with our custom Yocto distro on RISC v machine i want to get kernel crash log(hard panic) since RISC v does mot have support for the kxec how i can collect the crash logs?




[-- Attachment #1.2: Type: text/html, Size: 1005 bytes --]

[-- Attachment #2: Type: text/plain, Size: 170 bytes --]

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Kernel debugging

[Valdis Klētnieks]

[-- Attachment #1.1: Type: text/plain, Size: 491 bytes --]

On Wed, 19 Feb 2020 06:19:10 +0000, "Pankaj Vinadrao Joshi" said:

> I am using linux 5.4.3 with our custom Yocto distro on RISC v machine i want
> to get kernel crash log(hard panic) since RISC v does mot have support for the
> kxec how i can collect the crash logs?

Is netconsole an option, if ethernet is available?  If it has a serial port, use
another system as a serial console?  (I admit not knowing how well "console
on a USB serial port" works - if it does, that may be an option)

[-- Attachment #1.2: Type: application/pgp-signature, Size: 832 bytes --]

[-- Attachment #2: Type: text/plain, Size: 170 bytes --]

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Kernel Debugging

[Saket Sinha]

 I am facing an issue where I have been developing a filesystem driver(over
x86) which has become  "unstable"  by which I mean ,  at compile time it is
building but during runtime it fails when I call the corresponding APIs
from the user-space.

In order to find out exactly where it is failing, I need to debug the
driver.
I am very new to kernel development. I have heard about KGDB and KDB but it
takes quite a lot to get them working so waht are the other tools available?

Regards,
Saket Sinha
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20130806/6a3b2911/attachment.html 

Kernel Debugging

[Chaitanya Gotkhindikar]

Hi Saket,

You may configure kdump and use crash utility for debugging offline.

Here are some useful links.
https://sites.google.com/site/syscookbook/rhel/rhel-kdump-rhel6
http://www.novell.com/support/kb/doc.php?id=3374462

Otherwise systemtap is good option too.
http://www.linuxforu.com/2010/09/systemtap-tutorial-part-1/

Thanks,
Chaitanya


On Tue, Aug 6, 2013 at 3:55 PM, Saket Sinha <saket.sinha89@gmail.com> wrote:

>
>
>  I am facing an issue where I have been developing a filesystem
> driver(over x86) which has become  "unstable"  by which I mean ,  at
> compile time it is building but during runtime it fails when I call the
> corresponding APIs from the user-space.
>
> In order to find out exactly where it is failing, I need to debug the
> driver.
> I am very new to kernel development. I have heard about KGDB and KDB but
> it takes quite a lot to get them working so waht are the other tools
> available?
>
> Regards,
> Saket Sinha
>
>
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20130806/4eca3d14/attachment-0001.html 

kernel debugging

[Siju George]

[-- Attachment #1.1: Type: text/plain, Size: 305 bytes --]

Folks,
 
Does Xen support kernel debugging in Dom0 or in DomU ?
make ARCH=xen menuconfig doesn't show the kernel hacking menu. Is this because kernel debugging is not supported by Xen ?
 
Thanks,
Siju



		
---------------------------------
 Yahoo! FareChase - Search multiple travel sites in one click.  

[-- Attachment #1.2: Type: text/html, Size: 579 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: kernel debugging

[Kip Macy]

[-- Attachment #1.1: Type: text/plain, Size: 858 bytes --]

You can debug domUs by attaching with gdbserver-xen from
xen-unstable.hg/tools/debugger/gdb - pdb might work as well although it
probably hasn't tracked recent changes.

On 11/3/05, Siju George <sijugeorge@yahoo.com> wrote:
>
>  Folks,
>  Does Xen support kernel debugging in Dom0 or in DomU ?
> make ARCH=xen menuconfig doesn't show the kernel hacking menu. Is this
> because kernel debugging is not supported by Xen ?
>  Thanks,
> Siju
>
> ------------------------------
> Yahoo! FareChase - Search multiple travel sites in one click.<http://us.lrd.yahoo.com/_ylc=X3oDMTFqODRtdXQ4BF9TAzMyOTc1MDIEX3MDOTY2ODgxNjkEcG9zAzEEc2VjA21haWwtZm9vdGVyBHNsawNmYw--/SIG=110oav78o/**http%3a//farechase.yahoo.com/>
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>
>
>

[-- Attachment #1.2: Type: text/html, Size: 1610 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

kernel debugging

[UmaMaheswari Devi]

I am new to kernel hacking and am facing problems in trying to peek at the
runtime values of some kernel variables using gdb.

I am issuing the gdb command as follows:
     gdb vmlinux /proc/kcore
This displays the message
    /proc/kcore: Operation not permitted
before the (gdb) prompt is displayed.
gdb then prints a value of 0 for any valid variable that is requested.

vmlinux appears to be OK, as gdb correctly identifies undefined variables.
The problem seems to be with /proc/kcore. This file has a permission of 400. I
am using the Red Hat distribution.

Any help is appreciated. 

Thanks,
Uma.

Re: kernel debugging

[Jesper Juhl]

On 7/24/05, UmaMaheswari Devi <uma@cs.unc.edu> wrote:
> I am new to kernel hacking and am facing problems in trying to peek at the
> runtime values of some kernel variables using gdb.
> 
> I am issuing the gdb command as follows:
>      gdb vmlinux /proc/kcore
> This displays the message
>     /proc/kcore: Operation not permitted
> before the (gdb) prompt is displayed.
> gdb then prints a value of 0 for any valid variable that is requested.
> 
> vmlinux appears to be OK, as gdb correctly identifies undefined variables.
> The problem seems to be with /proc/kcore. This file has a permission of 400. I
> am using the Red Hat distribution.
> 
> Any help is appreciated.
> 
If you want to use gdb to debug the kernel you should probably
investigate UML (User Mode Linux). Take a look at this link :
http://user-mode-linux.sourceforge.net/debugging.html

Alternatives include kgdb - http://kgdb.sourceforge.net/ 
and kdb - http://oss.sgi.com/projects/kdb/

You can also find many documents on Linux Kernel debugging aids and
techniques via google.

-- 
Jesper Juhl <jesper.juhl@gmail.com>
Don't top-post  http://www.catb.org/~esr/jargon/html/T/top-post.html
Plain text mails only, please      http://www.expita.com/nomime.html

Re: kernel debugging

[Jesper Juhl]

On 7/24/05, Jesper Juhl <jesper.juhl@gmail.com> wrote:
> On 7/24/05, UmaMaheswari Devi <uma@cs.unc.edu> wrote:
> > I am new to kernel hacking and am facing problems in trying to peek at the
> > runtime values of some kernel variables using gdb.
> >
> > I am issuing the gdb command as follows:
> >      gdb vmlinux /proc/kcore
> > This displays the message
> >     /proc/kcore: Operation not permitted
> > before the (gdb) prompt is displayed.
> > gdb then prints a value of 0 for any valid variable that is requested.
> >
> > vmlinux appears to be OK, as gdb correctly identifies undefined variables.
> > The problem seems to be with /proc/kcore. This file has a permission of 400. I
> > am using the Red Hat distribution.
> >
> > Any help is appreciated.
> >
> If you want to use gdb to debug the kernel you should probably
> investigate UML (User Mode Linux). Take a look at this link :
> http://user-mode-linux.sourceforge.net/debugging.html
> 
> Alternatives include kgdb - http://kgdb.sourceforge.net/
> and kdb - http://oss.sgi.com/projects/kdb/
> 
> You can also find many documents on Linux Kernel debugging aids and
> techniques via google.
> 
Ohh, and do search the LKML archives, kernel debuggers have been
discussed endlessly over the years on the list, so you should be able
to find many threads covering that.

-- 
Jesper Juhl <jesper.juhl@gmail.com>
Don't top-post  http://www.catb.org/~esr/jargon/html/T/top-post.html
Plain text mails only, please      http://www.expita.com/nomime.html

Re: kernel debugging

[uma]

I am using Red Hat sources, which has function open_kcore() hardcoded to
return -EPERM always.

Changing this function to the way it is defined in the public sources (as
shown below) did the trick.

open_kcore(struct inode * inode, struct file * filp)
{
    return capable(CAP_SYS_RAWIO) ? 0 : -EPERM;
}

I am now able to use gdb to examine kernel symbols.

Uma.


Quoting UmaMaheswari Devi <uma@cs.unc.edu>:

> I am new to kernel hacking and am facing problems in tryingstatic int
to peek at the
> runtime values of some kernel variables using gdb.
>
> I am issuing the gdb command as follows:
>     gdb vmlinux /proc/kcore
> This displays the message
>    /proc/kcore: Operation not permitted
> before the (gdb) prompt is displayed.
> gdb then prints a value of 0 for any valid variable that is requested.
>
> vmlinux appears to be OK, as gdb correctly identifies undefined variables.
> The problem seems to be with /proc/kcore. This file has a permission 
> of 400. I
> am using the Red Hat distribution.
>
> Any help is appreciated. Thanks,
> Uma.
>
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

Re: kernel debugging

[Lee Revell]

On Mon, 2005-07-25 at 15:23 -0400, uma@email.unc.edu wrote:
> I am using Red Hat sources, which has function open_kcore() hardcoded to
> return -EPERM always.
> 
> Changing this function to the way it is defined in the public sources (as
> shown below) did the trick.

All these Red Hat / RHEL threads are completely OT.  Please take it to a
Red Hat list.

Lee

Re: Kernel Debugging

[Wolfgang Denk]

Dear Bo Yang,

in message <406B9D084FAFD4118C980008C79FADB901B57FF9@ALBMAIL> you wrote:
>
> Thank you for your prompt reply.

You are welcome.

> Could you please tell me where I can get or how to build a PPC cross version
> of GDB.

You can download the latest  version  of  our  ELDK  (Embedded  Linux
Development  Kit)  from ftp://ftp.leo.org/pub/eldk/2.1.0/, please see
ftp://ftp.leo.org/pub/eldk/2.1.0/eldk-ppc-linux-x86/README.html    or
http:www.denx.de/re/ELDK.html for detailed instructions.

> Also, I have following questions I can't find the answer:
> Is it enough just to include kgdb support when configuring kernel to build
> linux kernel with kgdb support?
> Has DENX's linux-2.4.4 kernel been patched with kgdb already or not?

Actually, we don't use kgdb. We  use  the  Abatron  BDI2000  for  all
kernel and device driver debugging (on PPC, ARM, and MIPS).

> Are the ? marks shown on serial console when kernel built with kgdb support
> starts run supposed to see or some else?

kgdb  will  stop  at  a  built-in  breakpoint  pretty  early  in  the
initialization  sequence  to  allow  you  to set more breakpoints for
example to debug drivers etc. It expects  that  you  connect  with  a
debugger on the serial line.

Best regards,

Wolfgang Denk

--
Software Engineering:  Embedded and Realtime Systems,  Embedded Linux
Phone: (+49)-8142-4596-87  Fax: (+49)-8142-4596-88  Email: wd@denx.de
Systems programmers are the high priests of a low cult.
                                                       -- R.S. Barton

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Kernel Debugging

[Yang, Bo]

Hi Everyone,

I am trying to debug the linux kernel using remote GDB via serial port on
MPC8xxFADS board. I built the kernel (2.4.4 from denx) with "kernel
hacking-->Include kgdb kernel debugger" turned on. (I am not sure that
turning on this flag is enough to support kernel debugging).

I got a bunch of ????? marks on serial console once the kernel gets run from
"ppcboot", and then I start GDB with cmd "target remote /dev/ttyS0" on the
host. The GDB got messages like "Ignoring packet error, continuing..." and
then timeed out to close connection.

Who can tell me what I am missing? Is it possible to debug linux kernel
without any BDM/JTAG debugger?

Any useful feedbacks are appreciated.

Bo Yang

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: Kernel Debugging

[Wolfgang Denk]

In message <406B9D084FAFD4118C980008C79FADB901B57FF7@ALBMAIL> you wrote:
>
> I got a bunch of ????? marks on serial console once the kernel gets run from
> "ppcboot", and then I start GDB with cmd "target remote /dev/ttyS0" on the
> host. The GDB got messages like "Ignoring packet error, continuing..." and
> then timeed out to close connection.

This is usually an indication that you tried to rn the  native  (x86)
GDB  which  will not be anle to communicate with a PPC target. Please
use the PPC cross version of GDB instead (ppc_8xx-gdb).

Best regards,

Wolfgang Denk

--
Software Engineering:  Embedded and Realtime Systems,  Embedded Linux
Phone: (+49)-8142-4596-87  Fax: (+49)-8142-4596-88  Email: wd@denx.de
He had quite a powerful intellect, but it  was  as  powerful  like  a
locomotive,  and  ran on rails and was therefore almost impossible to
steer.                          - Terry Pratchett, _Lords and Ladies_

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/