GDB, pthreads, and kernel threads

GDB, pthreads, and kernel threads

[John Clark]

Most of my work has been in the kernel and I had not paid attention to
user 'threads'. However, I have at the moment to a need to debug a
user 'pthread' based applicaiton, that I may want to move into the kernel.

However, I can't seem to figure out how to get GDB to debug my user
pthreads app. What is the correct setup to debug pthreads based applications
now that it seems that pthreads implementation generates processes/threads
in the kernel.

Thanks
John Clark

Re: GDB, pthreads, and kernel threads

[Nix]

On 18 May 2005, John Clark announced authoritatively:
> Most of my work has been in the kernel and I had not paid attention to
> user 'threads'. However, I have at the moment to a need to debug a
> user 'pthread' based applicaiton, that I may want to move into the kernel.
> 
> However, I can't seem to figure out how to get GDB to debug my user
> pthreads app. What is the correct setup to debug pthreads based applications
> now that it seems that pthreads implementation generates processes/threads
> in the kernel.

Use a recent GDB (>=6.2) and things should just work. (At least, they do
for me.)

-- 
`End users are just test loads for verifying that the system works, kind of
 like resistors in an electrical circuit.' - Kaz Kylheku in c.o.l.d.s

Re: GDB, pthreads, and kernel threads

[John Clark]

Nix wrote:

>On 18 May 2005, John Clark announced authoritatively:
>  
>
>>Most of my work has been in the kernel and I had not paid attention to
>>user 'threads'. However, I have at the moment to a need to debug a
>>user 'pthread' based applicaiton, that I may want to move into the kernel.
>>
>>However, I can't seem to figure out how to get GDB to debug my user
>>pthreads app. What is the correct setup to debug pthreads based applications
>>now that it seems that pthreads implementation generates processes/threads
>>in the kernel.
>>    
>>
>
>Use a recent GDB (>=6.2) and things should just work. (At least, they do
>for me.)
>  
>

I built the latest GDB-6.3, as well as rebuilt glibc-2.3.5, and now when 
I step through the
main code line, which creates the tasks (I'm using the pthreads.c from 
the GDB testsuite), I do
not getany output from:

info threads

When I set a break point on the entry point of one of the 
soon-to-be-created threads,
I get a diagnostic message:

Program terminated with signal SIGTRAP, Trace/Breakpoint trap.
The program no longer exists.


On the machine being used to debug the kernel is: 2.6.5.

Is there any problems with that kernel, or should I upgrade to a more 
recent vintage
version?

Thanks
John Clark

Re: GDB, pthreads, and kernel threads

[Robert Hancock]

John Clark wrote:
> I built the latest GDB-6.3, as well as rebuilt glibc-2.3.5, and now when 
> I step through the
> main code line, which creates the tasks (I'm using the pthreads.c from 
> the GDB testsuite), I do
> not getany output from:
> 
> info threads
> 
> When I set a break point on the entry point of one of the 
> soon-to-be-created threads,
> I get a diagnostic message:
> 
> Program terminated with signal SIGTRAP, Trace/Breakpoint trap.
> The program no longer exists.

Are you sure your glibc and gdb were both configured to support threads 
when they were compiled?

-- 
Robert Hancock      Saskatoon, SK, Canada
To email, remove "nospam" from hancockr@nospamshaw.ca
Home Page: http://www.roberthancock.com/

Re: GDB, pthreads, and kernel threads

[Ajay Patel]

I have an exactly same problem.
My platform is PPC, Linux 2.6.11.4, glibc 2.3.4 with linux threads.

One thing I observed that the problem only occurs
when break point is set in non-main thread.

Thanks
Ajay


On 5/18/05, Robert Hancock <hancockr@shaw.ca> wrote:
> John Clark wrote:
> > I built the latest GDB-6.3, as well as rebuilt glibc-2.3.5, and now when
> > I step through the
> > main code line, which creates the tasks (I'm using the pthreads.c from
> > the GDB testsuite), I do
> > not getany output from:
> >
> > info threads
> >
> > When I set a break point on the entry point of one of the
> > soon-to-be-created threads,
> > I get a diagnostic message:
> >
> > Program terminated with signal SIGTRAP, Trace/Breakpoint trap.
> > The program no longer exists.
> 
> Are you sure your glibc and gdb were both configured to support threads
> when they were compiled?
> 
> --
> Robert Hancock      Saskatoon, SK, Canada
> To email, remove "nospam" from hancockr@nospamshaw.ca
> Home Page: http://www.roberthancock.com/
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

Re: GDB, pthreads, and kernel threads

[John Clark]

Robert Hancock wrote:

> John Clark wrote:
>
>> I built the latest GDB-6.3, as well as rebuilt glibc-2.3.5, and now 
>> when I step through the
>> main code line, which creates the tasks (I'm using the pthreads.c 
>> from the GDB testsuite), I do
>> not getany output from:
>>
>> info threads
>>
>> When I set a break point on the entry point of one of the 
>> soon-to-be-created threads,
>> I get a diagnostic message:
>>
>> Program terminated with signal SIGTRAP, Trace/Breakpoint trap.
>> The program no longer exists.
>
>
> Are you sure your glibc and gdb were both configured to support 
> threads when they were compiled?


The application that I'm working with 'works', in the sense that when I 
do a 'ps' there are several processes
listed under the app name, corresponding to the created threads.

When I run gdb with the app, it does load /lib/libthread_db.so.1, so my 
presumption here is that gdb has
been thread enabled.

Since the app is pretty portable, and I've been using NetBSD to develop 
co-develop it, I can run gdb
on the NetBSD side, and 'things' seem to work better. There are still 
some wyrd operational issues
on the NetBSD, but at least when I set a break point in a thread, it 
works, and when I do a
'info threads' command to gdb, it gives a reasonable out put of the list 
of threads. Since the NetBSD
implementation of threads is 'new' to the kernel, I do expect some problems.

Now, because the thread implementation is different between NetBSD and 
Linux, and it seems that Linux
creates distinct 'processes' for the threads, I'm wondering if I have 
not correctly configured my kernel,
or if there is something special one has to do to allow gdb to write to 
one of the created thread processes.

Also, I do believe I'm using the NPTL package for threads. Is there a 
way to absolutely tell without
question?

Thanks
John Clark

Re: GDB, pthreads, and kernel threads

[Douglas McNaught]

John Clark <jclark@metricsystems.com> writes:

> Also, I do believe I'm using the NPTL package for threads. Is there a
> way to absolutely tell without
> question?

If you see multiple 'ps' entries for threads (without using any
special flags to ps) you are not using NPTL.  NPTL is in 2.6 and in
some vendor 2.4 kernels, but not in kernel.org 2.4.X.  

Hope this helps!

-Doug

Re: GDB, pthreads, and kernel threads

[John Clark]

Douglas McNaught wrote:

>John Clark <jclark@metricsystems.com> writes:
>
>  
>
>>Also, I do believe I'm using the NPTL package for threads. Is there a
>>way to absolutely tell without
>>question?
>>    
>>
>
>If you see multiple 'ps' entries for threads (without using any
>special flags to ps) you are not using NPTL.  NPTL is in 2.6 and in
>some vendor 2.4 kernels, but not in kernel.org 2.4.X.
>  
>

I used the hint from Ajay Patel and found that in my 'installed' glibc 
on my host 'linuxthreads' was being
used. However, I did compile glibc-2.3.5 for the host and results from 
that indicate that the nptl threads
version is begin used.

I will see if 1) the Gentoo distribution which was installed on the host 
machine I'm using has an updated
version of everything to the nptl threads, or 2) compile gdb against the 
glibc-2.3.5 which has the
ntpl threads.

Also, I'm using a 2.6.5 kernel in my host development environment, and 
using a 2.6.11.6 kernel
in my target. The target is actually using uClibc, and so I will check 
for the type of threads package
that is being supported there as well.

Thanks
John Clark

Re: GDB, pthreads, and kernel threads

[Ajay Patel]

> Also, I do believe I'm using the NPTL package for threads. Is there a
> way to absolutely tell without
> question?

If you run /lib/libc-{version}.so your output will show you what are
you running.

For example:

$/lib/libc-2.3.2.so
GNU C Library stable release version 2.3.2, by Roland McGrath et al.
Copyright (C) 2003 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
Compiled by GNU CC version 3.2.3 20030502 (Red Hat Linux 3.2.3-20).
Compiled on a Linux 2.4.20 system on 2003-10-02.
Available extensions:
        GNU libio by Per Bothner
        crypt add-on version 2.1 by Michael Glad and others
        linuxthreads-0.10 by Xavier Leroy 
-------------------------------->linuxthreads
        The C stubs add-on version 2.1.2.
        BIND-8.2.3-T5B
        NIS(YP)/NIS+ NSS modules 0.19 by Thorsten Kukuk
        Glibc-2.0 compatibility add-on by Cristian Gafton
        libthread_db work sponsored by Alpha Processor Inc


Thanks
Ajay