How to debug a kernel thread?

How to debug a kernel thread?

[Parmenides]

Hi,

  It is said that the kernel can be debugged in qemu and I take a
try. First, I started the qemu with

                 qemu -m 64M -kernel arch/x86/boot/bzImage -initrd
~/image.cpio.gz  -net nic -net tap,ifname=tap0  -s

 In another console

                  gdb vmlinux
                  (gdb) target remote localhost:1234
                  (gdb) continue

A LKM (mymodule.ko) which starts a kernel thread is made with debug
info, and was 'scp' to the guest. In guest, it is inserted by

                  insmod mymodule.ko

Then, back to gdb

                  (gdb) add-symbol-file mymodule.ko 0xc482e000
                  (gdb) break mymodules.c:37
                  (gdb) continue

The 37th line of mymodules.c is in a loop of kernel thread, which
ensures the breakpoint should be triggered every time the loop go
through.  But, the breakpoint doesn't triggered as expected. Instead,
the kernel thread is running over and over indicated by its repeated
output messages. So, I think a kernel thread can not be break by any
breakpoint.

However, I think maybe the gdb want to attach to the kernel thread.
Then, I checked the kernel thread's PID with ps and got 62.

                 (gdb) control+C
                 (gdb) attach 62

The gdb promted me it will kill the program being debugged. I answered
with 'yes', the gdb told me

                 ptrace: No such process.

then the debug session is terminated and the guest is closed.

I started the qemu  with the above command again

                 qemu -m 64M -kernel arch/x86/boot/bzImage -initrd
~/image.cpio.gz  -net nic -net tap,ifname=tap0  -s

And, without quitting the gdb

                  (gdb) target remote localhost:1234
                  (gdb) continue

In the guest, 'mymodule.ko' is inserted again

                  insmod mymodule.ko

I found that the breakpoint set at mymodule.c:37 is triggered this
time surprisingly, and the 'insmod' didn't return immediately until
the gdb is given with another 'continue' command.

                  (gdb) continue

Then, the breakpoint doesn't triggered anymore as usual,


There is two questions:

1. Why the kernel thread can not be break?
2. Why is the breakpoint triggered just when the 'mymodule.ko' is loaded?

Thanks.

How to debug a kernel thread?

[Mulyadi Santosa]

Hi Parmenides..

On Fri, Mar 30, 2012 at 19:00, Parmenides <mobile.parmenides@gmail.com> wrote:
>
> In the guest, 'mymodule.ko' is inserted again
>
> ? ? ? ? ? ? ? ? ?insmod mymodule.ko
>
> I found that the breakpoint set at mymodule.c:37 is triggered this
> time surprisingly, and the 'insmod' didn't return immediately until
> the gdb is given with another 'continue' command.
>
> ? ? ? ? ? ? ? ? ?(gdb) continue
>
> Then, the breakpoint doesn't triggered anymore as usual,

Reading your above explanation, assuming that you inserted the symbol
file into the right offset, I think you might found a bug in Qemu's
gdb stub itself.

I highly suggest you report this problem directly to qemu-devel list
to get quick fix.


-- 
regards,

Mulyadi Santosa
Freelance Linux trainer and consultant

blog: the-hydra.blogspot.com
training: mulyaditraining.blogspot.com

How to debug a kernel thread?

[Mulyadi Santosa]

Hi :)

On Tue, Apr 3, 2012 at 17:37, Parmenides <mobile.parmenides@gmail.com> wrote:
> source tree. I compiled the kernel with kgdb and run it in qemu with
> virtual serial port:
>
> ? ? qemu -kernel bzImage -initrd image.cpio.gz -append
> "kgdboc=ttyS0,115200 kgdbwait" -serial tcp::1234,server
>
> In another console, the gdb was given command:
>
> ? ? (gdb) target remote local:1234
> ? ? (gdb) continue
>
> Then I pressed Ctrl+C to suspend the target, but got no reply. The gdb
> got stuck and lost control to the target. As such, there is no

IIRC, years ago I did the same thing like yours, using kgdb and qemu.
What I forgot is how I hook into the remote kgdb stub.

Here's my suggestion: try other option outside tcp, perhaps UNIX
socket...or anything that's supported by gdb. And see if it makes any
difference.

-- 
regards,

Mulyadi Santosa
Freelance Linux trainer and consultant

blog: the-hydra.blogspot.com
training: mulyaditraining.blogspot.com