module debugging

module debugging

[Salam Farhat]

Hi,

I am not sure if this is the place to ask, if so please let me know where
to do so.

I am using a kernel module that I modified slightly to work on my current
kernel version. It is running on a virtual machine and on a heavy load it
goes into an infinite loop and freezes the system. I can tell that the
virtual machine is using 100% of its assigned cpu. There are print
statements almost everywhere in the module, but after a few seconds none
get printed.



The kernel has the following options already enabled:
CONFIG_DETECT_SOFTLOCKUP  and

CONFIG_DETECT_HUNG_TASK. To my understanding the kernel is supposed to
print the stack trace when a hung task is detected, however, nothing gets
printed in kern.log. I am not sure if this is the place where the kernel
would dump the stack or if the print process is affected by the hang since
the print statements in the module are not being printed as well.



I would like to know how to debug this issue. Any help is appreciated.



Thank you.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20150821/2eaed6b2/attachment.html 

module debugging

[leo kirotawa]

IMO, I believe if you module is using/hanging all processors it's
impossible to kernel use any to print stuffs. What about try Sysrq [1]
?  I used it once in a power system plus xmon, but I'll be honest not
sure how it works on x86.

https://www.kernel.org/doc/Documentation/sysrq.txt

On Fri, Aug 21, 2015 at 8:06 AM, Salam Farhat <salalimo@gmail.com> wrote:
> Hi,
>
> I am not sure if this is the place to ask, if so please let me know where to
> do so.
>
> I am using a kernel module that I modified slightly to work on my current
> kernel version. It is running on a virtual machine and on a heavy load it
> goes into an infinite loop and freezes the system. I can tell that the
> virtual machine is using 100% of its assigned cpu. There are print
> statements almost everywhere in the module, but after a few seconds none get
> printed.
>
>
>
> The kernel has the following options already enabled:
> CONFIG_DETECT_SOFTLOCKUP  and
>
> CONFIG_DETECT_HUNG_TASK. To my understanding the kernel is supposed to print
> the stack trace when a hung task is detected, however, nothing gets printed
> in kern.log. I am not sure if this is the place where the kernel would dump
> the stack or if the print process is affected by the hang since the print
> statements in the module are not being printed as well.
>
>
>
> I would like to know how to debug this issue. Any help is appreciated.
>
>
>
> Thank you.
>
>
>
>
>
>
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>



-- 

----------------------------------------------
Le?nidas S. Barbosa (Kirotawa)
blog: corecode.wordpress.com

module debugging

[Valdis.Kletnieks at vt.edu]

On Fri, 21 Aug 2015 07:06:58 -0400, Salam Farhat said:

> I am using a kernel module that I modified slightly to work on my current
> kernel version. It is running on a virtual machine and on a heavy load it
> goes into an infinite loop and freezes the system. I can tell that the
> virtual machine is using 100% of its assigned cpu.

Try giving the VM a second virtual CPU, if you have only one CPU and it
gets hung sufficiently hard, the debugging stuff won't have a chance to run.

Oh, and make sure you configure your kernel for SMP.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 848 bytes
Desc: not available
Url : http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20150821/5b14ed45/attachment-0001.bin 

module debugging

[Salam Farhat]

On Fri, Aug 21, 2015 at 8:33 AM, <Valdis.Kletnieks@vt.edu> wrote:

> On Fri, 21 Aug 2015 07:06:58 -0400, Salam Farhat said:
>
> > I am using a kernel module that I modified slightly to work on my current
> > kernel version. It is running on a virtual machine and on a heavy load it
> > goes into an infinite loop and freezes the system. I can tell that the
> > virtual machine is using 100% of its assigned cpu.
>
> Try giving the VM a second virtual CPU, if you have only one CPU and it
> gets hung sufficiently hard, the debugging stuff won't have a chance to
> run.
>
> Oh, and make sure you configure your kernel for SMP.
>

It has 4 cores and kernel is already configured for smp.

Sysrq is a pretty cool option, but other than reboot almost no other
commands run including 'e', 'f', ;i;.

It seems I might have to resort to debugging by the process of elimination
(removing code until it doesn't hang and so on.. ).

Any other ideas I would appreciate it.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20150822/c8f2a445/attachment.html 

module debugging

[Andrey Skvortsov]

On 22 Aug, Salam Farhat wrote:
> On Fri, Aug 21, 2015 at 8:33 AM, <Valdis.Kletnieks@vt.edu> wrote:
> 
> > On Fri, 21 Aug 2015 07:06:58 -0400, Salam Farhat said:
> >
> > > I am using a kernel module that I modified slightly to work on my current
> > > kernel version. It is running on a virtual machine and on a heavy load it
> > > goes into an infinite loop and freezes the system. I can tell that the
> > > virtual machine is using 100% of its assigned cpu.
> >
> > Try giving the VM a second virtual CPU, if you have only one CPU and it
> > gets hung sufficiently hard, the debugging stuff won't have a chance to
> > run.
> >
> > Oh, and make sure you configure your kernel for SMP.
> >
> 
> It has 4 cores and kernel is already configured for smp.
> 
> Sysrq is a pretty cool option, but other than reboot almost no other
> commands run including 'e', 'f', ;i;.
> 
> It seems I might have to resort to debugging by the process of elimination
> (removing code until it doesn't hang and so on.. ).
> 
> Any other ideas I would appreciate it.
If you use qemu-kvm to run your VM, you could try to use gdb to debug
your kernel running inside of VM.

man qemu:

 -s  Shorthand for -gdb tcp::1234, i.e. open a gdbserver on TCP port 1234.


-- 
Best regards,
Andrey Skvortsov

Secure eMail with gnupg: See http://www.gnupg.org/
PGP Key ID: 0x57A3AEAD
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
Url : http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20150825/89f7fc7d/attachment.bin 

module debugging

[Fabien R]

I used kgdb over serial port many years ago.
Perhaps there's a version for ethernet.

--
Fabien