gdb and multi-threaded applications.

gdb and multi-threaded applications.

[jjs]

I recently installed linuxppc-1999 on my beige G3. The main motivation for
doing this was to debug a multi-threaded application, which I could not do
under the Release 4 that I previously had installed.

The version of gdb that comes with the distribution (which I got from
ftp.linuxppc.org) is 4.17.0.11, which is not multi-thread aware. I therefore
got hold of the sources for 4.17 from ftp.gnu.org and H.J.Lu's patches for
4.17.0.14 and compiled up the resulting sources. According to the release
notes for 4.17.0.14, the change form 4.17.0.11 to 4.17.0.12 adds support for
LinuxThreads on PowerPC, so the version I have should work for
multi-threaded applications. However, typing 'info threads' on the
resultingexecutable, just returns me to the gdb prompt. Typing 'thread 1',
yields the message 'Thread ID 1 not known. Use the "info threads" command to
see the IDs of currently known threads.'. According to the gdb info, this is
typical behaviour of a gdb that does not have multi-threaded support.

Also, running the gdb.threads test in the testsuite that comes with the
distribution, tells me that my gdb does not support pthreads for this
machine.

All the evidence points to my gdb build not supporting multiple threads. Can
anybody tell me what I am doing wrong?

Cheers, Jerry.


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: gdb and multi-threaded applications.

[Martin Costabel]

jjs wrote:
>
> I recently installed linuxppc-1999 on my beige G3. The main motivation for
> doing this was to debug a multi-threaded application, which I could not do
> under the Release 4 that I previously had installed.
>
> The version of gdb that comes with the distribution (which I got from
> ftp.linuxppc.org) is 4.17.0.11, which is not multi-thread aware. I therefore
> got hold of the sources for 4.17 from ftp.gnu.org and H.J.Lu's patches for
> 4.17.0.14 and compiled up the resulting sources. According to the release
> notes for 4.17.0.14, the change form 4.17.0.11 to 4.17.0.12 adds support for
> LinuxThreads on PowerPC, so the version I have should work for
> multi-threaded applications. However, typing 'info threads' on the
> resultingexecutable, just returns me to the gdb prompt. Typing 'thread 1',
> yields the message 'Thread ID 1 not known. Use the "info threads" command to
> see the IDs of currently known threads.'. According to the gdb info, this is
> typical behaviour of a gdb that does not have multi-threaded support.

Franz Sirl has a gdb-4.18-4d RPM in his directory at devel.linuxppc.org.
His version 4.18-4c is also contained in the dev-rel-1.1. You might want
to try these.

--
Martin

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: gdb and multi-threaded applications.

[Kevin Buettner]

On Feb 23, 10:09am, jjs wrote:

> I recently installed linuxppc-1999 on my beige G3. The main motivation for
> doing this was to debug a multi-threaded application, which I could not do
> under the Release 4 that I previously had installed.
>
> The version of gdb that comes with the distribution (which I got from
> ftp.linuxppc.org) is 4.17.0.11, which is not multi-thread aware. I therefore
> got hold of the sources for 4.17 from ftp.gnu.org and H.J.Lu's patches for
> 4.17.0.14 and compiled up the resulting sources. According to the release
> notes for 4.17.0.14, the change form 4.17.0.11 to 4.17.0.12 adds support for
> LinuxThreads on PowerPC, so the version I have should work for
> multi-threaded applications. However, typing 'info threads' on the
> resultingexecutable, just returns me to the gdb prompt. Typing 'thread 1',
> yields the message 'Thread ID 1 not known. Use the "info threads" command to
> see the IDs of currently known threads.'. According to the gdb info, this is
> typical behaviour of a gdb that does not have multi-threaded support.
>
> Also, running the gdb.threads test in the testsuite that comes with the
> distribution, tells me that my gdb does not support pthreads for this
> machine.
>
> All the evidence points to my gdb build not supporting multiple threads. Can
> anybody tell me what I am doing wrong?

I'm not sure that you're doing anything wrong.  Was the program in
question multithreaded?  I see the same behavior on linux/x86 when
debugging a single threaded program.

Regardless, you may want to try the latest gdb sources from the CVS
repository at sourceware.cygnus.com.  I've recently merged the gdb
patches for linux/ppc into the "official" source base and need help
testing this code.  This version is thread aware and passes all of the
thread related tests in the test suite.  In fact overall, I'm seeing
only 31 failures.  (This number rises to 70 if you also build/test
insight (gdbtk)).

See http://sourceware.cygnus.com/gdb/ for information on how to obtain
the sources.

Kevin

--
Kevin Buettner
kev@primenet.com, kevinb@redhat.com

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: gdb and multi-threaded applications.

[jjs]

>On Feb 23, 10:09am, jjs wrote:
>
>> I recently installed linuxppc-1999 on my beige G3. The main motivation
for
>> doing this was to debug a multi-threaded application, which I could not
do
>> under the Release 4 that I previously had installed.
>>
>> The version of gdb that comes with the distribution (which I got from
>> ftp.linuxppc.org) is 4.17.0.11, which is not multi-thread aware. I
therefore
>> got hold of the sources for 4.17 from ftp.gnu.org and H.J.Lu's patches
for
>> 4.17.0.14 and compiled up the resulting sources. According to the release
>> notes for 4.17.0.14, the change form 4.17.0.11 to 4.17.0.12 adds support
for
>> LinuxThreads on PowerPC, so the version I have should work for
>> multi-threaded applications. However, typing 'info threads' on the
>> resultingexecutable, just returns me to the gdb prompt. Typing 'thread
1',
>> yields the message 'Thread ID 1 not known. Use the "info threads" command
to
>> see the IDs of currently known threads.'. According to the gdb info, this
is
>> typical behaviour of a gdb that does not have multi-threaded support.
>>
>> Also, running the gdb.threads test in the testsuite that comes with the
>> distribution, tells me that my gdb does not support pthreads for this
>> machine.
>>
>> All the evidence points to my gdb build not supporting multiple threads.
Can
>> anybody tell me what I am doing wrong?
>
>I'm not sure that you're doing anything wrong.  Was the program in
>question multithreaded?  I see the same behavior on linux/x86 when
>debugging a single threaded program.
>
>Regardless, you may want to try the latest gdb sources from the CVS
>repository at sourceware.cygnus.com.  I've recently merged the gdb
>patches for linux/ppc into the "official" source base and need help
>testing this code.  This version is thread aware and passes all of the
>thread related tests in the test suite.  In fact overall, I'm seeing
>only 31 failures.  (This number rises to 70 if you also build/test
>insight (gdbtk)).
>
>See http://sourceware.cygnus.com/gdb/ for information on how to obtain
>the sources.
>
>Kevin
>
>--
>Kevin Buettner
>kev@primenet.com, kevinb@redhat.com
>

Kevin,

Thanks for your reply the other day, I have tried taking the gdb-000222
snapshot from the ftp site and compiled it, but I am getting the same
behaviour as I was with 4.17.0.14, which unfortunately I did not fully
describe in my original message. I am seeing the described behaviour with
'info threads' but also when I run my program under gdb, it hangs when
trying to create a new thread via a call to pthread_create. It works as
expected outside the debugger, creating the thread and continuing.

I have tried attaching gdb to the gdb process that is hanging and get the
following stack trace:

#0  0x15c64c0 in __syscall_rt_sigsuspend () at soinit.c:59
#1  0x15c51f0 in __sigsuspend (set=0x19da444)
    at ../sysdeps/unix/sysv/linux/sigsuspend.c:48
#2  0x184aa88 in linux_child_wait (pid=-1, rpid=0x7fffe0f0,
status=0x7fffe0f4)
    at linux-thread.c:1338
#3  0x184ad14 in linuxthreads_wait (pid=-1, ourstatus=0x7fffe150)
    at linux-thread.c:1415
#4  0x18316e4 in wait_for_inferior () at infrun.c:1261
#5  0x18314ec in proceed (addr=1, siggnal=TARGET_SIGNAL_DEFAULT, step=1)
    at infrun.c:1058
#6  0x182e5a8 in step_1 (skip_subroutines=0, single_inst=27131904,
    count_string=0x0) at infcmd.c:542
#7  0x182e230 in step_command (count_string=0x1 <Address 0x1 out of bounds>,
    from_tty=8) at infcmd.c:423
#8  0x186e814 in execute_command (p=0x19efbc1 "", from_tty=1) at top.c:1502
#9  0x183a5fc in command_handler (command=0x19efbc0 "s") at event-top.c:508
#10 0x183adb8 in command_line_handler (rl=0x19d9864 "") at event-top.c:810
#11 0x192d094 in rl_callback_read_char () at callback.c:122
#12 0x1839980 in rl_callback_read_char_wrapper (client_data=0xfffffffc)
    at event-top.c:166
#13 0x183a464 in stdin_event_handler (error=-4, client_data=0xfffffffc)
    at event-top.c:415
#14 0x188e2dc in handle_file_event (event_file_desc=-4) at event-loop.c:726
#15 0x188de70 in process_event () at event-loop.c:381
#16 0x188dec8 in gdb_do_one_event (data=0xfffffffc) at event-loop.c:418
#17 0x186d66c in catch_errors (func=0x188de94 <gdb_do_one_event>, args=0x0,
    errstring=0x1960d80 "", mask=3) at top.c:612
#18 0x188df10 in start_event_loop () at event-loop.c:442
#19 0x1839a84 in cli_command_loop () at event-top.c:196
#20 0x1802768 in captured_command_loop (data=0xfffffffc) at main.c:99
#21 0x186d66c in catch_errors (func=0x180273c <captured_command_loop>,
    args=0x0, errstring=0x193bcf4 "", mask=3) at top.c:612
#22 0x18031c8 in captured_main (data=0xfffffffc) at main.c:737
#23 0x186d66c in catch_errors (func=0x18027a8 <captured_main>,
    args=0x7ffff4f8, errstring=0x193bcf4 "", mask=3) at top.c:612
#24 0x18031fc in main (argc=-4, argv=0x8) at main.c:749
#25 0x15bd7d4 in __libc_start_main (argc=2, argv=0x7ffff564,
envp=0x7ffff570,
    auxvec=0x7ffff664, rtld_fini=0xfffffffc, stinfo=0x193bbb0,
    stack_on_entry=0x16bdc90) at ../sysdeps/powerpc/elf/libc-start.c:106

I am running with kernel 2.2.6-15apmac, which came with the linuxppc-1999
distribution.

Any assistance you can give me will be gratefully received.

Cheers, Jerry.


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: gdb and multi-threaded applications.

[Kevin Buettner]

On Feb 26, 12:38pm, jjs wrote:

> Thanks for your reply the other day, I have tried taking the gdb-000222
> snapshot from the ftp site and compiled it, but I am getting the same
> behaviour as I was with 4.17.0.14, which unfortunately I did not fully
> describe in my original message. I am seeing the described behaviour with
> 'info threads' but also when I run my program under gdb, it hangs when
> trying to create a new thread via a call to pthread_create. It works as
> expected outside the debugger, creating the thread and continuing.
>
> I have tried attaching gdb to the gdb process that is hanging and get the
> following stack trace:
>
> #0  0x15c64c0 in __syscall_rt_sigsuspend () at soinit.c:59
> #1  0x15c51f0 in __sigsuspend (set=0x19da444)

Would it be possible for you to print out the contents of set at this
point?  (Go "up" a frame and do "print *set".)

>     at ../sysdeps/unix/sysv/linux/sigsuspend.c:48
> #2  0x184aa88 in linux_child_wait (pid=-1, rpid=0x7fffe0f0,
> status=0x7fffe0f4)
>     at linux-thread.c:1338

And I'd like to see what linux_threads_block_mask is at this point too.

> #3  0x184ad14 in linuxthreads_wait (pid=-1, ourstatus=0x7fffe150)
>     at linux-thread.c:1415
> #4  0x18316e4 in wait_for_inferior () at infrun.c:1261

I have seen this backtrace before.  For a time, I was seeing hangs
under certain circumstances when debugging single threaded
applications too.  The problem was that the SIGCHLD signal was being
blocked.  This is why I want to know the signal mask is which is being
passed to sigsuspend().

> I am running with kernel 2.2.6-15apmac, which came with the linuxppc-1999
> distribution.

We're running the same kernels at least.

> Any assistance you can give me will be gratefully received.

Are you running a proprietary applicaton?  If not, it may be useful at
some point for me to get a copy of it and try to reproduce the problems
you're seeing myself.

In the interim, it may be useful for you to try out the threaded
program testsuite/gdb.threads/linux-dp which is (obviously) in the gdb
testsuite.  Here's part of a sample run when I try it.  (Note:  If you
do "make check" from the gdb directory, all of the programs will be
built and the test suite will be run.  After the testsuite finishes
running, all of the programs are built and you can try running them
yourself.)

    [kev@arroyo testsuite]$ ../gdb gdb.threads/linux-dp
    GNU gdb 20000204
    Copyright 1998 Free Software Foundation, Inc.
    GDB is free software, covered by the GNU General Public License, and you are
    welcome to change it and/or distribute copies of it under certain conditions.
    Type "show copying" to see the conditions.
    There is absolutely no warranty for GDB.  Type "show warranty" for details.
    This GDB was configured as "powerpc-unknown-linux-gnu"...
    (gdb) b 187
    Breakpoint 1 at 0x10000d7c: file /ocotillo1/kev/sourceware/src/gdb/testsuite/gdb.threads/linux-dp.c, line 187.
    (gdb) r
    Starting program: /home/kev/sourceware-gdb-bld/gdb/testsuite/gdb.threads/linux-dp

    Breakpoint 1, main (argc=1, argv=0x7ffffd04)
	at /ocotillo1/kev/sourceware/src/gdb/testsuite/gdb.threads/linux-dp.c:187
    187         for (i = 0; i < num_philosophers; i++)
    (gdb) info thread
    (gdb) n
    189             numbers[i] = i;
    (gdb) n
    191             pthread_create (&philosophers[i], &ta, philosopher, &numbers[i]);
    (gdb) s
    [New Thread 32622 (manager thread)]
    [New Thread 32621 (initial thread)]
    [New Thread 32623]
    187         for (i = 0; i < num_philosophers; i++)
    (gdb) info thread
      3 Thread 32623  0xfe094c0 in __syscall_rt_sigsuspend () at soinit.c:59
    * 2 Thread 32621 (initial thread)  main (argc=1, argv=0x7ffffd04)
	at /ocotillo1/kev/sourceware/src/gdb/testsuite/gdb.threads/linux-dp.c:187
      1 Thread 32622 (manager thread)  0xfe08178 in kill () at soinit.c:59
    (gdb) print num_philosophers
    $1 = 5
    (gdb) n
      _ 0 _
      ! 0 _
    189             numbers[i] = i;
    (gdb)
    191             pthread_create (&philosophers[i], &ta, philosopher, &numbers[i]);
    (gdb)
    [New Thread 32624]
	  _ 1 _
	  ! 1 _
	  ! 1 !
	  _ 1 _
    187         for (i = 0; i < num_philosophers; i++)

The thing to notice in the above run is that we make it through several
calls of pthread_create().  (If I would've sent you the rest of my trace
you'd see that it makes it through all five calls successfully.)

Also... I doubt that it'll matter, but I've fixed a few bugs in the last
several days.  (I'm now seeing only 14 testsuite failures.)  You may wish
to download a newer version.

One last thing... could you do "rpm -q -f /lib/libpthread-0.8.so" and tell
me what the output is?  Here's what happens when I try it:

    [kev@arroyo kev]$ rpm -q -f /lib/libpthread-0.8.so
    glibc-2.1.1-6c

Kevin

--
Kevin Buettner
kev@primenet.com, kevinb@redhat.com

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: gdb and multi-threaded applications.

[jjs]

Kevin,

>On Feb 26, 12:38pm, jjs wrote:
>
>> Thanks for your reply the other day, I have tried taking the gdb-000222
>> snapshot from the ftp site and compiled it, but I am getting the same
>> behaviour as I was with 4.17.0.14, which unfortunately I did not fully
>> describe in my original message. I am seeing the described behaviour with
>> 'info threads' but also when I run my program under gdb, it hangs when
>> trying to create a new thread via a call to pthread_create. It works as
>> expected outside the debugger, creating the thread and continuing.
>>
>> I have tried attaching gdb to the gdb process that is hanging and get the
>> following stack trace:
>>
>> #0  0x15c64c0 in __syscall_rt_sigsuspend () at soinit.c:59
>> #1  0x15c51f0 in __sigsuspend (set=0x19da444)
>
>Would it be possible for you to print out the contents of set at this
>point?  (Go "up" a frame and do "print *set".)
>
>>     at ../sysdeps/unix/sysv/linux/sigsuspend.c:48
>> #2  0x184aa88 in linux_child_wait (pid=-1, rpid=0x7fffe0f0,
>> status=0x7fffe0f4)
>>     at linux-thread.c:1338

(gdb) p *set
$2 = {__val = {0 <repeats 32 times>}}

>
>And I'd like to see what linux_threads_block_mask is at this point too.
>


(gdb) p linuxthreads_block_mask
$3 = {__val = {0 <repeats 32 times>}}

>> #3  0x184ad14 in linuxthreads_wait (pid=-1, ourstatus=0x7fffe150)
>>     at linux-thread.c:1415
>> #4  0x18316e4 in wait_for_inferior () at infrun.c:1261
>
>I have seen this backtrace before.  For a time, I was seeing hangs
>under certain circumstances when debugging single threaded
>applications too.  The problem was that the SIGCHLD signal was being
>blocked.  This is why I want to know the signal mask is which is being
>passed to sigsuspend().
>
>> I am running with kernel 2.2.6-15apmac, which came with the linuxppc-1999
>> distribution.
>
>We're running the same kernels at least.
>
>> Any assistance you can give me will be gratefully received.
>
>Are you running a proprietary applicaton?  If not, it may be useful at
>some point for me to get a copy of it and try to reproduce the problems
>you're seeing myself.

Unfortunately, I am running a proprietory application.

>
>In the interim, it may be useful for you to try out the threaded
>program testsuite/gdb.threads/linux-dp which is (obviously) in the gdb
>testsuite.  Here's part of a sample run when I try it.  (Note:  If you
>do "make check" from the gdb directory, all of the programs will be
>built and the test suite will be run.  After the testsuite finishes
>running, all of the programs are built and you can try running them
>yourself.)
>
>    [kev@arroyo testsuite]$ ../gdb gdb.threads/linux-dp
>    GNU gdb 20000204
>    Copyright 1998 Free Software Foundation, Inc.
>    GDB is free software, covered by the GNU General Public License, and
you are
>    welcome to change it and/or distribute copies of it under certain
conditions.
>    Type "show copying" to see the conditions.
>    There is absolutely no warranty for GDB.  Type "show warranty" for
details.
>    This GDB was configured as "powerpc-unknown-linux-gnu"...
>    (gdb) b 187
>    Breakpoint 1 at 0x10000d7c: file
/ocotillo1/kev/sourceware/src/gdb/testsuite/gdb.threads/linux-dp.c, line
187.
>    (gdb) r
>    Starting program:
/home/kev/sourceware-gdb-bld/gdb/testsuite/gdb.threads/linux-dp
>
>    Breakpoint 1, main (argc=1, argv=0x7ffffd04)
> at /ocotillo1/kev/sourceware/src/gdb/testsuite/gdb.threads/linux-dp.c:187
>    187         for (i = 0; i < num_philosophers; i++)
>    (gdb) info thread
>    (gdb) n
>    189             numbers[i] = i;
>    (gdb) n
>    191             pthread_create (&philosophers[i], &ta, philosopher,
&numbers[i]);
>    (gdb) s
>    [New Thread 32622 (manager thread)]
>    [New Thread 32621 (initial thread)]
>    [New Thread 32623]
>    187         for (i = 0; i < num_philosophers; i++)
>    (gdb) info thread
>      3 Thread 32623  0xfe094c0 in __syscall_rt_sigsuspend () at
soinit.c:59
>    * 2 Thread 32621 (initial thread)  main (argc=1, argv=0x7ffffd04)
> at /ocotillo1/kev/sourceware/src/gdb/testsuite/gdb.threads/linux-dp.c:187
>      1 Thread 32622 (manager thread)  0xfe08178 in kill () at soinit.c:59
>    (gdb) print num_philosophers
>    $1 = 5
>    (gdb) n
>      _ 0 _
>      ! 0 _
>    189             numbers[i] = i;
>    (gdb)
>    191             pthread_create (&philosophers[i], &ta, philosopher,
&numbers[i]);
>    (gdb)
>    [New Thread 32624]
>   _ 1 _
>   ! 1 _
>   ! 1 !
>   _ 1 _
>    187         for (i = 0; i < num_philosophers; i++)
>
>The thing to notice in the above run is that we make it through several
>calls of pthread_create().  (If I would've sent you the rest of my trace
>you'd see that it makes it through all five calls successfully.)
>

I tried running linux-dp under gdb and it works exactly as you  describe, so
there is no fundamental problem with gdb debugging a program that calls
pthread_create.

>Also... I doubt that it'll matter, but I've fixed a few bugs in the last
>several days.  (I'm now seeing only 14 testsuite failures.)  You may wish
>to download a newer version.
>
>One last thing... could you do "rpm -q -f /lib/libpthread-0.8.so" and tell
>me what the output is?  Here's what happens when I try it:
>
>    [kev@arroyo kev]$ rpm -q -f /lib/libpthread-0.8.so
>    glibc-2.1.1-6c
>

This gave the same for me as it did you.

>Kevin
>
>--
>Kevin Buettner
>kev@primenet.com, kevinb@redhat.com
>

I hope this helps. If there is any other information I can give you please
don't hesitate to ask.

Cheers, Jerry.


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: gdb and multi-threaded applications.

[Kevin Buettner]

On Feb 28,  3:31pm, jjs wrote:

> >Would it be possible for you to print out the contents of set at this
> >point?  (Go "up" a frame and do "print *set".)
> >
> >>     at ../sysdeps/unix/sysv/linux/sigsuspend.c:48
> >> #2  0x184aa88 in linux_child_wait (pid=-1, rpid=0x7fffe0f0,
> >> status=0x7fffe0f4)
> >>     at linux-thread.c:1338
>
> (gdb) p *set
> $2 = {__val = {0 <repeats 32 times>}}

This looks okay.

> >And I'd like to see what linux_threads_block_mask is at this point too.
>
> (gdb) p linuxthreads_block_mask
> $3 = {__val = {0 <repeats 32 times>}}

This looks okay too.

> >Are you running a proprietary applicaton?  If not, it may be useful at
> >some point for me to get a copy of it and try to reproduce the problems
> >you're seeing myself.
>
> Unfortunately, I am running a proprietory application.

[...]

> I tried running linux-dp under gdb and it works exactly as you  describe, so
> there is no fundamental problem with gdb debugging a program that calls
> pthread_create.

In order for me to be able to go further on this problem, I'll need to
be able to reproduce it on my machine so that I can debug it.  Could
you look at what your proprietary application is doing (with regard to
thread creation) and attempt to create a small program which illustrates
the problem?

Kevin

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: gdb and multi-threaded applications.

[jjs]

>
>In order for me to be able to go further on this problem, I'll need to
>be able to reproduce it on my machine so that I can debug it.  Could
>you look at what your proprietary application is doing (with regard to
>thread creation) and attempt to create a small program which illustrates
>the problem?
>
>Kevin
>

Kevin,

Sorry I have not been back to you sooner, I have been distracted by other
things. I tried getting a test application to exhibit the same behaviour
with no luck. However, I did notice that before the call to pthread_create
to start up the new thread we have the line:

    pthread_sigmask(SIG_BLOCK, &fullSigMask, &sigMask);

All the bits in fullSigMask are set, so this call blocks all signals to the
calling thread. Commenting this line out makes everything work under gdb,
the new thread gets created and doing 'info threads' after this gives me a
list of all the threads running in the process. This implies that gdb
requires a signal to be unblocked at the point the thread is created. If
this is fact the case, which signal must we not block in order for correct
operation under gdb?

Cheers, Jerry.


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: gdb and multi-threaded applications.

[Kevin Buettner]

On Mar 10,  1:03pm, jjs wrote:

> Sorry I have not been back to you sooner, I have been distracted by other
> things. I tried getting a test application to exhibit the same behaviour
> with no luck. However, I did notice that before the call to pthread_create
> to start up the new thread we have the line:
>
>     pthread_sigmask(SIG_BLOCK, &fullSigMask, &sigMask);
>
> All the bits in fullSigMask are set, so this call blocks all signals to the
> calling thread. Commenting this line out makes everything work under gdb,
> the new thread gets created and doing 'info threads' after this gives me a
> list of all the threads running in the process. This implies that gdb
> requires a signal to be unblocked at the point the thread is created. If
> this is fact the case, which signal must we not block in order for correct
> operation under gdb?

SIGCHLD.

(I think.)

Background for Michael: Jerry reported a problem in which gdb (on
linux/ppc) hangs on a pthread_create().   Jerry has verified that
the thread tests in the test suite work properly on his platform,
so we know that gdb does work (some of the time) when creating
new threads.  Jerry's application is proprietary and I've asked him
to try to distill the problem into a small example that we can test.

Note to Jerry: I'd still like a small example, preferably one that
we could eventually put into the testsuite.

Kevin

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: gdb and multi-threaded applications.

[Michael Snyder]

Kevin Buettner wrote:
>
> On Mar 10,  1:03pm, jjs wrote:
>
> > Sorry I have not been back to you sooner, I have been distracted by other
> > things. I tried getting a test application to exhibit the same behaviour
> > with no luck. However, I did notice that before the call to pthread_create
> > to start up the new thread we have the line:
> >
> >     pthread_sigmask(SIG_BLOCK, &fullSigMask, &sigMask);
> >
> > All the bits in fullSigMask are set, so this call blocks all signals to the
> > calling thread. Commenting this line out makes everything work under gdb,
> > the new thread gets created and doing 'info threads' after this gives me a
> > list of all the threads running in the process. This implies that gdb
> > requires a signal to be unblocked at the point the thread is created.

Your analysis is correct.  The linuxthreads library uses certain
signals to communicate and synchronize between threads.  GDB
listens in on these signals.  Most commonly they are SIGUSR32,
SIGUSR33 and SIGUSR34, but this is not always the case.  I would
think that blocking these signals might affect more than just
GDB debugging (ie. it might affect the program's behavior directly).

> > If
> > this is fact the case, which signal must we not block in order for correct
> > operation under gdb?
>
> SIGCHLD.
>
> (I think.)

SIGCHLD does come into play as well, although I have never
fully understood its role.  Try unblocking SIGCHLD, but also
be prepared to try unblocking the ones I mentioned above.

> Background for Michael: Jerry reported a problem in which gdb (on
> linux/ppc) hangs on a pthread_create().   Jerry has verified that
> the thread tests in the test suite work properly on his platform,
> so we know that gdb does work (some of the time) when creating
> new threads.  Jerry's application is proprietary and I've asked him
> to try to distill the problem into a small example that we can test.
>
> Note to Jerry: I'd still like a small example, preferably one that
> we could eventually put into the testsuite.
>
> Kevin

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: gdb and multi-threaded applications.

[Michael Snyder]

Kevin Buettner wrote:
>
> On Mar 10,  1:03pm, jjs wrote:
>
> > Sorry I have not been back to you sooner, I have been distracted by other
> > things. I tried getting a test application to exhibit the same behaviour
> > with no luck. However, I did notice that before the call to pthread_create
> > to start up the new thread we have the line:
> >
> >     pthread_sigmask(SIG_BLOCK, &fullSigMask, &sigMask);
> >
> > All the bits in fullSigMask are set, so this call blocks all signals to the
> > calling thread. Commenting this line out makes everything work under gdb,
> > the new thread gets created and doing 'info threads' after this gives me a
> > list of all the threads running in the process. This implies that gdb
> > requires a signal to be unblocked at the point the thread is created. If
> > this is fact the case, which signal must we not block in order for correct
> > operation under gdb?
>
> SIGCHLD.
>
> (I think.)
>
> Background for Michael: Jerry reported a problem in which gdb (on
> linux/ppc) hangs on a pthread_create().   Jerry has verified that
> the thread tests in the test suite work properly on his platform,
> so we know that gdb does work (some of the time) when creating
> new threads.  Jerry's application is proprietary and I've asked him
> to try to distill the problem into a small example that we can test.
>
> Note to Jerry: I'd still like a small example, preferably one that
> we could eventually put into the testsuite.
>
> Kevin

Afterthought -- GDB also uses SIGSTOP to stop threads when
one of them hits a breakpoint etc.  You shouldn't block that.

				Michael

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: gdb and multi-threaded applications.

[jjs]

>Note to Jerry: I'd still like a small example, preferably one that
>we could eventually put into the testsuite.
>
>Kevin
>

Kevin,

OK, here is an example.

gdb_thread.c
=================================================================

#include <stdio.h>
#include <pthread.h>
#include <signal.h>
#include <errno.h>

char *signame[] =
{
  "0",
  "SIGHUP",
  "SIGINT",
  "SIGQUIT",
  "SIGILL",
  "SIGTRAP",
  "SIGABRT",
  "SIGBUS",
  "SIGFPE",
  "SIGKILL",
  "SIGUSR1",
  "SIGSEGV",
  "SIGUSR2",
  "SIGPIPE",
  "SIGALRM",
  "SIGTERM",
  "SIGSTKFLT",
  "SIGCHLD",
  "SIGCONT",
  "SIGSTOP",
  "SIGTSTP",
  "SIGTTIN",
  "SIGTTOU",
  "SIGURG",
  "SIGXCPU",
  "SIGXFSZ",
  "SIGVTALRM",
  "SIGPROF",
  "SIGWINCH",
  "SIGIO",
  "SIGPWR",
  "SIGUNUSED",
  "SIG32",
  "SIG33",
  "SIG34",
  "SIG35",
  "SIG36",
  "SIG37",
  "SIG38",
  "SIG39",
  "SIG40",
  "SIG41",
  "SIG42",
  "SIG43",
  "SIG44",
  "SIG45",
  "SIG46",
  "SIG47",
  "SIG48",
  "SIG49",
  "SIG50",
  "SIG51",
  "SIG52",
  "SIG53",
  "SIG54",
  "SIG55",
  "SIG56",
  "SIG57",
  "SIG58",
  "SIG59",
  "SIG60",
  "SIG61",
  "SIG62",
  "SIG63"
};
#define NSIGS (sizeof(signame) / sizeof(signame[0]))

void
attrInit(pthread_attr_t *attrPtr)
{
 struct sched_param schedParam;

 schedParam.sched_priority = 0;
 pthread_attr_init(attrPtr);
 pthread_attr_setdetachstate(attrPtr, PTHREAD_CREATE_DETACHED);
 pthread_attr_setschedpolicy(attrPtr, SCHED_OTHER);
 pthread_attr_setschedparam(attrPtr, &schedParam);
 pthread_attr_setinheritsched(attrPtr, PTHREAD_EXPLICIT_SCHED);
 pthread_attr_setscope(attrPtr, PTHREAD_SCOPE_PROCESS);
 pthread_setschedparam(pthread_self(), SCHED_OTHER, &schedParam);
 pthread_setcanceltype(PTHREAD_CANCEL_ASYNCHRONOUS, NULL);
}

void *
threadStartPoint(void *arg)
{
 int retVal;

 printf("Thread started successfully.\n");
 return NULL;
}

main(int argc, char *argv[])
{
 pthread_t  newThread;
 sigset_t  sigmask;
 sigset_t  fullSigMask;
 pthread_attr_t threadAttr;
 int    err;
 int    signal;

 if (argc == 2)
 {
  signal = atoi(argv[1]);
 }
 else
 {
  fprintf(stderr, "usage: thread signal\n");
  exit(-1);
 }
 attrInit(&threadAttr);
 sigemptyset(&sigmask);
 sigaddset(&sigmask, signal);
 sigfillset(&fullSigMask);
 if (pthread_sigmask(SIG_BLOCK, &sigmask, NULL) == 0)
 {
  printf("Blocked signal %s before creating thread.\n", signame[signal]);
 }
 else
 {
  printf("Failed to block signal %s before creating thread.\n",
      signame[signal]);
  exit(-1);
 }
 err = pthread_create(&newThread, &threadAttr,
       (void *(*)(void *)) threadStartPoint,
       (void *) NULL);

 switch(err)
 {
 case 0:
  printf("pthread_create succeeded\n");
  break;
 case EAGAIN:
  printf("pthread_create failed: System resources not available\n");
  break;
 case EINVAL:
  printf("pthread_create failed: Invalid thread attributes\n");
  break;
 default:
  printf("pthread_create failed: Unknown error - %d\n", err);
  break;
 }
 exit(0);
}

Makefile
====================================================================

CFLAGS=-g
CC=gcc

gdb_thread: gdb_thread.o
   gcc -g -o $@ gdb_thread.o -lpthread

===========================================================================

Running this program under gdb, I get the following output:

(gdb) r 1
Starting program: /tmp_mnt/home/wolf/jjs/src/thread/gdb_thread 1
Blocked signal SIGHUP before creating thread.
[New Thread 12358 (manager thread)]
[New Thread 12352 (initial thread)]
[New Thread 12359]
pthread_create succeeded

Program exited normally.
(gdb) r 2
Starting program: /tmp_mnt/home/wolf/jjs/src/thread/gdb_thread 2
Blocked signal SIGINT before creating thread.
[New Thread 12366 (manager thread)]
[New Thread 12360 (initial thread)]
[New Thread 12367]
pthread_create succeeded

Program exited normally.
(gdb)

Note that the "Thread started successfully." message from threadStartPoint
is not printed out. I think this is just because we fall off the end of the
program before we get a chance to ececute the printf. This pattern is
repeated for all signal numbers except 34, which gives the following output:

(gdb) r 34
Starting program: /tmp_mnt/home/wolf/jjs/src/thread/gdb_thread 34
Blocked signal SIG34 before creating thread.

and then gdb hangs and I have to kill gdb_thread externally to continue.

Note that I tried blocking each signal in turn in a loop first of all and
the program sailed past signal 34 without a problem. Experimenting a little,
it turns out that after pthread_create succeeds once, it will subsequently
succeed under gdb even if signal 34 is blocked.

I have tried running our proprietory application under gdb with all signals
except signal 34 blocked when pthread_create is called and it works fine. I
will make sure that all the signals Michael suggests are unblocked in our
application and hopefully every thing will work OK.

Thanks for your help, guys.

Cheers, Jerry.


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/