ptrace.2: BUGS (missing WIFEXITED notification)

ptrace.2: BUGS (missing WIFEXITED notification)

[Vegard Nossum]

[resend with Cc: linux-man]

Hi again :-)

We hit another edge case in the ptrace() interface and after several
hours of chasing it down, we found that it was already described in the
"BUGS" section:

"If a thread group leader is traced and exits by calling _exit(2), a
PTRACE_EVENT_EXIT stop will happen for it (if requested), but the
subsequent WIFEXITED notification will not be delivered until all other
threads exit. As explained above, if one of other threads calls
execve(2), the death of the thread group leader will never be reported.
If the execed thread is not traced by this tracer, the tracer will never
know that execve(2) happened. One possible workaround is to
PTRACE_DETACH the thread group leader instead of restarting it in this
case. Last confirmed on 2.6.38.6."

I wanted to write that we've also noticed the same thing not only for
_exit() but also for terminating signals, however we also came across
this bit in the manual source:

.\" Note from Denys Vlasenko:
.\" Here "exits" means any kind of death - _exit, exit_group,
.\" signal death. Signal death and exit_group cases are trivial,
.\" though: since signal death and exit_group kill all other threads
.\" too, "until all other threads exit" thing happens rather soon
.\" in these cases. Therefore, only _exit presents observably
.\" puzzling behavior to ptrace users: thread leader _exit's,
.\" but WIFEXITED isn't reported! We are trying to explain here
.\" why it is so.

There is a difference, however -- this behaviour can also be observed
for the other types of death if you are currently tracing the other
threads too!

In other words, when multiple threads are being traced and the group
leader exits, waitpid() on this group leader will hang indefinitely
(because the other threads won't exit until we wait for and CONT/DETACH
them, and we don't receive the exit notification for the group leader
until the other threads have really exited).

To me, this means that not only _exit() but also other types of death
present "observably puzzling behavior to ptrace users".

I'd propose the following changes:

1) include some (if not all) of Denys's explanation in the actual text:

-If a thread group leader is traced and exits by calling _exit(2)...
+If a thread group leader is traced and exits for any reason (_exit,
exit_group, signal death, etc.), ...

2) include the bits about tracing other threads:

+If the other threads in the thread group are being traced, they will
not exit until they have been either waited for and restarted or
detached, thereby blocking the exit notification (WIFEXITED) of the
group leader to wait()/waitpid().

3) there's a typo in the original text:

-one of other threads
+one of the other threads

Feel free to rephrase any of the above.

Thoughts? We can also provide more details, including a reproducer, or
clarification if needed.

(PS: Please also credit Quentin Casasnovas with the report as we've both
spent more than a few hours tracking this down!)


Vegard
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: ptrace.2: BUGS (missing WIFEXITED notification)

[Michael Kerrisk (man-pages)]

Hi Denys,

Do you have any thoughts on the below?

Cheers,

Michael

On 05/12/2015 04:31 PM, Vegard Nossum wrote:
> [resend with Cc: linux-man]
> 
> Hi again :-)
> 
> We hit another edge case in the ptrace() interface and after several
> hours of chasing it down, we found that it was already described in the
> "BUGS" section:
> 
> "If a thread group leader is traced and exits by calling _exit(2), a
> PTRACE_EVENT_EXIT stop will happen for it (if requested), but the
> subsequent WIFEXITED notification will not be delivered until all other
> threads exit. As explained above, if one of other threads calls
> execve(2), the death of the thread group leader will never be reported.
> If the execed thread is not traced by this tracer, the tracer will never
> know that execve(2) happened. One possible workaround is to
> PTRACE_DETACH the thread group leader instead of restarting it in this
> case. Last confirmed on 2.6.38.6."
> 
> I wanted to write that we've also noticed the same thing not only for
> _exit() but also for terminating signals, however we also came across
> this bit in the manual source:
> 
> .\" Note from Denys Vlasenko:
> .\" Here "exits" means any kind of death - _exit, exit_group,
> .\" signal death. Signal death and exit_group cases are trivial,
> .\" though: since signal death and exit_group kill all other threads
> .\" too, "until all other threads exit" thing happens rather soon
> .\" in these cases. Therefore, only _exit presents observably
> .\" puzzling behavior to ptrace users: thread leader _exit's,
> .\" but WIFEXITED isn't reported! We are trying to explain here
> .\" why it is so.
> 
> There is a difference, however -- this behaviour can also be observed
> for the other types of death if you are currently tracing the other
> threads too!
> 
> In other words, when multiple threads are being traced and the group
> leader exits, waitpid() on this group leader will hang indefinitely
> (because the other threads won't exit until we wait for and CONT/DETACH
> them, and we don't receive the exit notification for the group leader
> until the other threads have really exited).
> 
> To me, this means that not only _exit() but also other types of death
> present "observably puzzling behavior to ptrace users".
> 
> I'd propose the following changes:
> 
> 1) include some (if not all) of Denys's explanation in the actual text:
> 
> -If a thread group leader is traced and exits by calling _exit(2)...
> +If a thread group leader is traced and exits for any reason (_exit,
> exit_group, signal death, etc.), ...
> 
> 2) include the bits about tracing other threads:
> 
> +If the other threads in the thread group are being traced, they will
> not exit until they have been either waited for and restarted or
> detached, thereby blocking the exit notification (WIFEXITED) of the
> group leader to wait()/waitpid().
> 
> 3) there's a typo in the original text:
> 
> -one of other threads
> +one of the other threads
> 
> Feel free to rephrase any of the above.
> 
> Thoughts? We can also provide more details, including a reproducer, or
> clarification if needed.
> 
> (PS: Please also credit Quentin Casasnovas with the report as we've both
> spent more than a few hours tracking this down!)
> 
> 
> Vegard
> 


-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: ptrace.2: BUGS (missing WIFEXITED notification)

[Denys Vlasenko]

On 05/14/2015 03:44 PM, Michael Kerrisk (man-pages) wrote:
> Hi Denys,
> 
> Do you have any thoughts on the below?

Yes, the poster is right: this part needs fixing, the behavior is
the same on any kind of process termination.


> On 05/12/2015 04:31 PM, Vegard Nossum wrote:
>> We hit another edge case in the ptrace() interface and after several
>> hours of chasing it down, we found that it was already described in the
>> "BUGS" section:
>>
>> "If a thread group leader is traced and exits by calling _exit(2), a

I think a possible fix is just to replace "exits by calling _exit(2)" part
of the above text with "terminates".

>> PTRACE_EVENT_EXIT stop will happen for it (if requested), but the
>> subsequent WIFEXITED notification will not be delivered until all other
>> threads exit. As explained above, if one of other threads calls
>> execve(2), the death of the thread group leader will never be reported.
>> If the execed thread is not traced by this tracer, the tracer will never
>> know that execve(2) happened. One possible workaround is to
>> PTRACE_DETACH the thread group leader instead of restarting it in this
>> case. Last confirmed on 2.6.38.6."
>>
>> I wanted to write that we've also noticed the same thing not only for
>> _exit() but also for terminating signals, however we also came across
>> this bit in the manual source:
>>
>> .\" Note from Denys Vlasenko:
>> .\" Here "exits" means any kind of death - _exit, exit_group,
>> .\" signal death. Signal death and exit_group cases are trivial,
>> .\" though: since signal death and exit_group kill all other threads
>> .\" too, "until all other threads exit" thing happens rather soon
>> .\" in these cases. Therefore, only _exit presents observably
>> .\" puzzling behavior to ptrace users: thread leader _exit's,
>> .\" but WIFEXITED isn't reported! We are trying to explain here
>> .\" why it is so.
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: ptrace.2: BUGS (missing WIFEXITED notification)

[Quentin Casasnovas]

On Thu, May 14, 2015 at 03:52:36PM +0200, Denys Vlasenko wrote:
> On 05/14/2015 03:44 PM, Michael Kerrisk (man-pages) wrote:
> > Hi Denys,
> > 
> > Do you have any thoughts on the below?
> 
> Yes, the poster is right: this part needs fixing, the behavior is
> the same on any kind of process termination.
> 
> 
> > On 05/12/2015 04:31 PM, Vegard Nossum wrote:
> >> We hit another edge case in the ptrace() interface and after several
> >> hours of chasing it down, we found that it was already described in the
> >> "BUGS" section:
> >>
> >> "If a thread group leader is traced and exits by calling _exit(2), a
> 
> I think a possible fix is just to replace "exits by calling _exit(2)" part
> of the above text with "terminates".
> 

Should we also add a little paragraph detailing that waitpid() would hang
indefinitely if one thread terminates while the others are in ptrace-stop?
This behaviour was quite puzzling as during our experiments,
waitpid(<thread_pid>, ...) was hanging on a zombie thread, and even using
waitpid(-1, ...)  wasn't helping.

The only way for us to work-around this behaviour was to have
PTRACE_O_TRACEEXIT and detach from the threads when we were notified of a
thread termination, without PTRACE_O_TRACEEXIT I don't think one can be
notified in way, leading to hangs almost certainly when tracing multiple
threads and some are in ptrace-stop.

Or maybe that's obvious from the following sentence:

"If a thread group leader is traced and exits by calling _exit(2), a
PTRACE_EVENT_EXIT stop will happen for it (if requested), but the
subsequent WIFEXITED notification will not be delivered until all other
threads exit."

I know we didn't realize the other threads would not exit (because they
were in ptrace-stop), blocking the delivery.

Quentin
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: ptrace.2: BUGS (missing WIFEXITED notification)

[Denys Vlasenko]

On 05/14/2015 06:28 PM, Quentin Casasnovas wrote:
> On Thu, May 14, 2015 at 03:52:36PM +0200, Denys Vlasenko wrote:
>> On 05/14/2015 03:44 PM, Michael Kerrisk (man-pages) wrote:
>>> Hi Denys,
>>>
>>> Do you have any thoughts on the below?
>>
>> Yes, the poster is right: this part needs fixing, the behavior is
>> the same on any kind of process termination.
>>
>>
>>> On 05/12/2015 04:31 PM, Vegard Nossum wrote:
>>>> We hit another edge case in the ptrace() interface and after several
>>>> hours of chasing it down, we found that it was already described in the
>>>> "BUGS" section:
>>>>
>>>> "If a thread group leader is traced and exits by calling _exit(2), a
>>
>> I think a possible fix is just to replace "exits by calling _exit(2)" part
>> of the above text with "terminates".
>>
> 
> Should we also add a little paragraph detailing that waitpid() would hang
> indefinitely if one thread terminates while the others are in ptrace-stop?

It implies this by saying "but the subsequent WIFEXITED notification
will not be delivered until all other threads exit".

If another thread is in ptrace-stop, it did not exit yet. Therefore,
WIFEXITED notification to the thread group leader will not be delivered.
Therefore, waitpid() on it would hang.

> The only way for us to work-around this behaviour was to have
> PTRACE_O_TRACEEXIT and detach from the threads when we were notified of a
> thread termination, without PTRACE_O_TRACEEXIT I don't think one can be
> notified in way, leading to hangs almost certainly when tracing multiple
> threads and some are in ptrace-stop.

Why do you do that waitpid on thread group leader?

strace is the user of ptrace, and it has no problem with this behavior.
It simply waits for all tracees via waitpid(-1). Whichever sends the next
notification is processed.


> Or maybe that's obvious from the following sentence:
> 
> "If a thread group leader is traced and exits by calling _exit(2), a
> PTRACE_EVENT_EXIT stop will happen for it (if requested), but the
> subsequent WIFEXITED notification will not be delivered until all other
> threads exit."

Exactly.
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: ptrace.2: BUGS (missing WIFEXITED notification)

[Quentin Casasnovas]

On Thu, May 14, 2015 at 06:39:59PM +0200, Denys Vlasenko wrote:
> On 05/14/2015 06:28 PM, Quentin Casasnovas wrote:
> > Should we also add a little paragraph detailing that waitpid() would hang
> > indefinitely if one thread terminates while the others are in ptrace-stop?
> 
> It implies this by saying "but the subsequent WIFEXITED notification
> will not be delivered until all other threads exit".
>
> If another thread is in ptrace-stop, it did not exit yet. Therefore,
> WIFEXITED notification to the thread group leader will not be delivered.
> Therefore, waitpid() on it would hang.
> 

Right, I guess it just wasn't obvious to us (or just me) that the threads
would not exit while in ptrace-stop.

> > The only way for us to work-around this behaviour was to have
> > PTRACE_O_TRACEEXIT and detach from the threads when we were notified of a
> > thread termination, without PTRACE_O_TRACEEXIT I don't think one can be
> > notified in way, leading to hangs almost certainly when tracing multiple
> > threads and some are in ptrace-stop.
> 
> Why do you do that waitpid on thread group leader?
> 
> strace is the user of ptrace, and it has no problem with this behavior.
> It simply waits for all tracees via waitpid(-1). Whichever sends the next
> notification is processed.
>

That's because strace has all threads running; in our use-case, we attach
to all threads of a group and restart a single thread to do some work.  In
this particular case, waitpid(-1, ...) does not help because to exit, the
other threads need to be PTRACE_CONT'ed.

Quentin
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: ptrace.2: BUGS (missing WIFEXITED notification)

[Denys Vlasenko]

On 05/14/2015 06:50 PM, Quentin Casasnovas wrote:
>>> The only way for us to work-around this behaviour was to have
>>> PTRACE_O_TRACEEXIT and detach from the threads when we were notified of a
>>> thread termination, without PTRACE_O_TRACEEXIT I don't think one can be
>>> notified in way, leading to hangs almost certainly when tracing multiple
>>> threads and some are in ptrace-stop.
>>
>> Why do you do that waitpid on thread group leader?
>>
>> strace is the user of ptrace, and it has no problem with this behavior.
>> It simply waits for all tracees via waitpid(-1). Whichever sends the next
>> notification is processed.
>>
> 
> That's because strace has all threads running; in our use-case, we attach
> to all threads of a group and restart a single thread to do some work.  In
> this particular case, waitpid(-1, ...) does not help because to exit, the
> other threads need to be PTRACE_CONT'ed.

Can you be more specific?
IF you have thread(s) which "do some work", then waitpid(-1) would work
for you - presumably, you wait for them to finish doing work,
or otherwise terminate, get a signal, whatever?
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: ptrace.2: BUGS (missing WIFEXITED notification)

[Quentin Casasnovas]

On Thu, May 14, 2015 at 07:06:00PM +0200, Denys Vlasenko wrote:
> On 05/14/2015 06:50 PM, Quentin Casasnovas wrote:
> >>> The only way for us to work-around this behaviour was to have
> >>> PTRACE_O_TRACEEXIT and detach from the threads when we were notified of a
> >>> thread termination, without PTRACE_O_TRACEEXIT I don't think one can be
> >>> notified in way, leading to hangs almost certainly when tracing multiple
> >>> threads and some are in ptrace-stop.
> >>
> >> Why do you do that waitpid on thread group leader?
> >>
> >> strace is the user of ptrace, and it has no problem with this behavior.
> >> It simply waits for all tracees via waitpid(-1). Whichever sends the next
> >> notification is processed.
> >>
> > 
> > That's because strace has all threads running; in our use-case, we attach
> > to all threads of a group and restart a single thread to do some work.  In
> > this particular case, waitpid(-1, ...) does not help because to exit, the
> > other threads need to be PTRACE_CONT'ed.
> 
> Can you be more specific?
> IF you have thread(s) which "do some work", then waitpid(-1) would work
> for you - presumably, you wait for them to finish doing work,
> or otherwise terminate, get a signal, whatever?

Oh you are right, PEBCAK!  It would appear I did not test this properly, I
cannot reproduce this behaviour...

So, if we attach to all threads of a group, call them thread A and B, then
CONT thread A, which then terminates for any reason, doing a waitpid(-1)
will indeed notify us for the termination even though thread B is in
ptrace-stop.

Sorry for the noise.

Quentin
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: ptrace.2: BUGS (missing WIFEXITED notification)

[Vegard Nossum]

On 05/14/2015 06:39 PM, Denys Vlasenko wrote:
> On 05/14/2015 06:28 PM, Quentin Casasnovas wrote:
>> On Thu, May 14, 2015 at 03:52:36PM +0200, Denys Vlasenko wrote:
>>> On 05/14/2015 03:44 PM, Michael Kerrisk (man-pages) wrote:
>>>> Hi Denys,
>>>>
>>>> Do you have any thoughts on the below?
>>>
>>> Yes, the poster is right: this part needs fixing, the behavior is
>>> the same on any kind of process termination.
>>>
>>>
>>>> On 05/12/2015 04:31 PM, Vegard Nossum wrote:
>>>>> We hit another edge case in the ptrace() interface and after several
>>>>> hours of chasing it down, we found that it was already described in the
>>>>> "BUGS" section:
>>>>>
>>>>> "If a thread group leader is traced and exits by calling _exit(2), a
>>>
>>> I think a possible fix is just to replace "exits by calling _exit(2)" part
>>> of the above text with "terminates".
>>>
>>
>> Should we also add a little paragraph detailing that waitpid() would hang
>> indefinitely if one thread terminates while the others are in ptrace-stop?
>
> It implies this by saying "but the subsequent WIFEXITED notification
> will not be delivered until all other threads exit".
>
> If another thread is in ptrace-stop, it did not exit yet. Therefore,
> WIFEXITED notification to the thread group leader will not be delivered.
> Therefore, waitpid() on it would hang.

While I agree that the information in the current man page is strictly
speaking sufficient, I personally still think it would be an improvement
to mention it explicitly (i.e. my proposed change #2 in the original
e-mail). Just because I think it's a sort of non-obvious pitfall; out of
hand, you don't expect a call to waitpid() on a process that has exited
to hang. That's just my opinion, though.


Vegard
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: ptrace.2: BUGS (missing WIFEXITED notification)

[Michael Kerrisk (man-pages)]

Hi Vegard,

On 15 May 2015 at 12:12, Vegard Nossum <vegard.nossum-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org> wrote:
> On 05/14/2015 06:39 PM, Denys Vlasenko wrote:
>>
>> On 05/14/2015 06:28 PM, Quentin Casasnovas wrote:
>>>
>>> On Thu, May 14, 2015 at 03:52:36PM +0200, Denys Vlasenko wrote:
>>>>
>>>> On 05/14/2015 03:44 PM, Michael Kerrisk (man-pages) wrote:
>>>>>
>>>>> Hi Denys,
>>>>>
>>>>> Do you have any thoughts on the below?
>>>>
>>>>
>>>> Yes, the poster is right: this part needs fixing, the behavior is
>>>> the same on any kind of process termination.
>>>>
>>>>
>>>>> On 05/12/2015 04:31 PM, Vegard Nossum wrote:
>>>>>>
>>>>>> We hit another edge case in the ptrace() interface and after several
>>>>>> hours of chasing it down, we found that it was already described in
>>>>>> the
>>>>>> "BUGS" section:
>>>>>>
>>>>>> "If a thread group leader is traced and exits by calling _exit(2), a
>>>>
>>>>
>>>> I think a possible fix is just to replace "exits by calling _exit(2)"
>>>> part
>>>> of the above text with "terminates".
>>>>
>>>
>>> Should we also add a little paragraph detailing that waitpid() would hang
>>> indefinitely if one thread terminates while the others are in
>>> ptrace-stop?
>>
>>
>> It implies this by saying "but the subsequent WIFEXITED notification
>> will not be delivered until all other threads exit".
>>
>> If another thread is in ptrace-stop, it did not exit yet. Therefore,
>> WIFEXITED notification to the thread group leader will not be delivered.
>> Therefore, waitpid() on it would hang.
>
>
> While I agree that the information in the current man page is strictly
> speaking sufficient, I personally still think it would be an improvement
> to mention it explicitly (i.e. my proposed change #2 in the original
> e-mail). Just because I think it's a sort of non-obvious pitfall; out of
> hand, you don't expect a call to waitpid() on a process that has exited
> to hang. That's just my opinion, though.

That sounds okay to me. Would you and/or Quentin be willing to put
together a patch to the man page?

Thanks,

Michael



-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: ptrace.2: BUGS (missing WIFEXITED notification)

[Michael Kerrisk (man-pages)]

Vegard (and Quentin): Ping!

On 05/15/2015 02:05 PM, Michael Kerrisk (man-pages) wrote:
> Hi Vegard,
> 
> On 15 May 2015 at 12:12, Vegard Nossum <vegard.nossum-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org> wrote:
>> On 05/14/2015 06:39 PM, Denys Vlasenko wrote:
>>>
>>> On 05/14/2015 06:28 PM, Quentin Casasnovas wrote:
>>>>
>>>> On Thu, May 14, 2015 at 03:52:36PM +0200, Denys Vlasenko wrote:
>>>>>
>>>>> On 05/14/2015 03:44 PM, Michael Kerrisk (man-pages) wrote:
>>>>>>
>>>>>> Hi Denys,
>>>>>>
>>>>>> Do you have any thoughts on the below?
>>>>>
>>>>>
>>>>> Yes, the poster is right: this part needs fixing, the behavior is
>>>>> the same on any kind of process termination.
>>>>>
>>>>>
>>>>>> On 05/12/2015 04:31 PM, Vegard Nossum wrote:
>>>>>>>
>>>>>>> We hit another edge case in the ptrace() interface and after several
>>>>>>> hours of chasing it down, we found that it was already described in
>>>>>>> the
>>>>>>> "BUGS" section:
>>>>>>>
>>>>>>> "If a thread group leader is traced and exits by calling _exit(2), a
>>>>>
>>>>>
>>>>> I think a possible fix is just to replace "exits by calling _exit(2)"
>>>>> part
>>>>> of the above text with "terminates".
>>>>>
>>>>
>>>> Should we also add a little paragraph detailing that waitpid() would hang
>>>> indefinitely if one thread terminates while the others are in
>>>> ptrace-stop?
>>>
>>>
>>> It implies this by saying "but the subsequent WIFEXITED notification
>>> will not be delivered until all other threads exit".
>>>
>>> If another thread is in ptrace-stop, it did not exit yet. Therefore,
>>> WIFEXITED notification to the thread group leader will not be delivered.
>>> Therefore, waitpid() on it would hang.
>>
>>
>> While I agree that the information in the current man page is strictly
>> speaking sufficient, I personally still think it would be an improvement
>> to mention it explicitly (i.e. my proposed change #2 in the original
>> e-mail). Just because I think it's a sort of non-obvious pitfall; out of
>> hand, you don't expect a call to waitpid() on a process that has exited
>> to hang. That's just my opinion, though.
> 
> That sounds okay to me. Would you and/or Quentin be willing to put
> together a patch to the man page?
> 
> Thanks,
> 
> Michael
> 
> 
> 


-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: ptrace.2: BUGS (missing WIFEXITED notification)

[Vegard Nossum]

[-- Attachment #1: Type: text/plain, Size: 2411 bytes --]

(including context for old thread)

On 06/18/2015 08:49 AM, Michael Kerrisk (man-pages) wrote:
> Vegard (and Quentin): Ping!
>
> On 05/15/2015 02:05 PM, Michael Kerrisk (man-pages) wrote:
>> On 15 May 2015 at 12:12, Vegard Nossum <vegard.nossum-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org> wrote:
>>> On 05/14/2015 06:39 PM, Denys Vlasenko wrote:
>>>> On 05/14/2015 06:28 PM, Quentin Casasnovas wrote:
>>>>> On Thu, May 14, 2015 at 03:52:36PM +0200, Denys Vlasenko wrote:
>>>>>> On 05/14/2015 03:44 PM, Michael Kerrisk (man-pages) wrote:
>>>>>>>
>>>>>>> Hi Denys,
>>>>>>>
>>>>>>> Do you have any thoughts on the below?
>>>>>>
>>>>>> Yes, the poster is right: this part needs fixing, the behavior is
>>>>>> the same on any kind of process termination.
>>>>>>
>>>>>>> On 05/12/2015 04:31 PM, Vegard Nossum wrote:
>>>>>>>>
>>>>>>>> We hit another edge case in the ptrace() interface and after several
>>>>>>>> hours of chasing it down, we found that it was already described in
>>>>>>>> the
>>>>>>>> "BUGS" section:
>>>>>>>>
>>>>>>>> "If a thread group leader is traced and exits by calling _exit(2), a
>>>>>>
>>>>>> I think a possible fix is just to replace "exits by calling _exit(2)"
>>>>>> part
>>>>>> of the above text with "terminates".
>>>>>
>>>>> Should we also add a little paragraph detailing that waitpid() would hang
>>>>> indefinitely if one thread terminates while the others are in
>>>>> ptrace-stop?
>>>>
>>>> It implies this by saying "but the subsequent WIFEXITED notification
>>>> will not be delivered until all other threads exit".
>>>>
>>>> If another thread is in ptrace-stop, it did not exit yet. Therefore,
>>>> WIFEXITED notification to the thread group leader will not be delivered.
>>>> Therefore, waitpid() on it would hang.
>>>
>>> While I agree that the information in the current man page is strictly
>>> speaking sufficient, I personally still think it would be an improvement
>>> to mention it explicitly (i.e. my proposed change #2 in the original
>>> e-mail). Just because I think it's a sort of non-obvious pitfall; out of
>>> hand, you don't expect a call to waitpid() on a process that has exited
>>> to hang. That's just my opinion, though.
>>
>> That sounds okay to me. Would you and/or Quentin be willing to put
>> together a patch to the man page?
>>

Hi,

Apologies for the delay. Here's a new patch, feel free to munge the
wording if you think it can be improved.


Vegard

[-- Attachment #2: 0001-ptrace.2-clarify-what-happens-when-group-leader-exit.patch --]
[-- Type: text/x-patch, Size: 2302 bytes --]

>From 91798cbc674787cf31b66e8cf52557495e659665 Mon Sep 17 00:00:00 2001
From: Vegard Nossum <vegard.nossum-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
Date: Fri, 12 Aug 2016 16:31:11 +0200
Subject: [PATCH] ptrace.2: clarify what happens when group leader exits

If both the group leader and other threads in the thread group are
being traced and the group leader exits, there will be no notification
of the group leader's exit to the tracer because it still has
unwaited-for children.

In other words, doing waitpid() on the group leader in this situation
will hang indefinitely.

The old wording makes it seem like this only happens when the group
leader calls _exit(), but in fact it happens for any sort of exit:
_exit, exit_group(), and signal death (as explained in the comment by
Denys Vlasenko).

Cc: Quentin Casasnovas <quentin.casasnovas-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
Cc: Denys Vlasenko <dvlasenk-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
Signed-off-by: Vegard Nossum <vegard.nossum-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
---
 man2/ptrace.2 | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git man2/ptrace.2 man2/ptrace.2
index 36d81b9..13bac29 100644
--- man2/ptrace.2
+++ man2/ptrace.2
@@ -2432,8 +2432,7 @@ if that is defined.
 Group-stop notifications are sent to the tracer, but not to real parent.
 Last confirmed on 2.6.38.6.
 .LP
-If a thread group leader is traced and exits by calling
-.BR _exit (2),
+If a thread group leader is traced and exits for any reason,
 .\" Note from Denys Vlasenko:
 .\"     Here "exits" means any kind of death - _exit, exit_group,
 .\"     signal death. Signal death and exit_group cases are trivial,
@@ -2448,7 +2447,14 @@ a
 stop will happen for it (if requested), but the subsequent
 .B WIFEXITED
 notification will not be delivered until all other threads exit.
-As explained above, if one of other threads calls
+(As a corollary, if the other threads in the thread group are being
+traced, they will not exit until they have been either waited for
+and restarted or detached, thereby blocking the exit notification
+of the group leader to
+.BR wait (2)
+and
+.BR waitpid (2).)
+As explained above, if one of the other threads calls
 .BR execve (2),
 the death of the thread group leader will
 .I never
-- 
1.9.1

Re: ptrace.2: BUGS (missing WIFEXITED notification)

[Denys Vlasenko]

On 08/12/2016 04:50 PM, Vegard Nossum wrote:
> (including context for old thread)
>
> On 06/18/2015 08:49 AM, Michael Kerrisk (man-pages) wrote:
>> Vegard (and Quentin): Ping!
>>
>> On 05/15/2015 02:05 PM, Michael Kerrisk (man-pages) wrote:
>>> On 15 May 2015 at 12:12, Vegard Nossum <vegard.nossum-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org> wrote:
>>>> On 05/14/2015 06:39 PM, Denys Vlasenko wrote:
>>>>> On 05/14/2015 06:28 PM, Quentin Casasnovas wrote:
>>>>>> On Thu, May 14, 2015 at 03:52:36PM +0200, Denys Vlasenko wrote:
>>>>>>> On 05/14/2015 03:44 PM, Michael Kerrisk (man-pages) wrote:
>>>>>>>>
>>>>>>>> Hi Denys,
>>>>>>>>
>>>>>>>> Do you have any thoughts on the below?
>>>>>>>
>>>>>>> Yes, the poster is right: this part needs fixing, the behavior is
>>>>>>> the same on any kind of process termination.
>>>>>>>
>>>>>>>> On 05/12/2015 04:31 PM, Vegard Nossum wrote:
>>>>>>>>>
>>>>>>>>> We hit another edge case in the ptrace() interface and after several
>>>>>>>>> hours of chasing it down, we found that it was already described in
>>>>>>>>> the
>>>>>>>>> "BUGS" section:
>>>>>>>>>
>>>>>>>>> "If a thread group leader is traced and exits by calling _exit(2), a
>>>>>>>
>>>>>>> I think a possible fix is just to replace "exits by calling _exit(2)"
>>>>>>> part
>>>>>>> of the above text with "terminates".
>>>>>>
>>>>>> Should we also add a little paragraph detailing that waitpid() would hang
>>>>>> indefinitely if one thread terminates while the others are in
>>>>>> ptrace-stop?
>>>>>
>>>>> It implies this by saying "but the subsequent WIFEXITED notification
>>>>> will not be delivered until all other threads exit".
>>>>>
>>>>> If another thread is in ptrace-stop, it did not exit yet. Therefore,
>>>>> WIFEXITED notification to the thread group leader will not be delivered.
>>>>> Therefore, waitpid() on it would hang.
>>>>
>>>> While I agree that the information in the current man page is strictly
>>>> speaking sufficient, I personally still think it would be an improvement
>>>> to mention it explicitly (i.e. my proposed change #2 in the original
>>>> e-mail). Just because I think it's a sort of non-obvious pitfall; out of
>>>> hand, you don't expect a call to waitpid() on a process that has exited
>>>> to hang. That's just my opinion, though.
>>>
>>> That sounds okay to me. Would you and/or Quentin be willing to put
>>> together a patch to the man page?
>>>
>
> Hi,
>
> Apologies for the delay. Here's a new patch, feel free to munge the
> wording if you think it can be improved.

-If a thread group leader is traced and exits by calling
-.BR _exit (2),
+If a thread group leader is traced and exits for any reason,
  .\" Note from Denys Vlasenko:


This looks good to me.


  stop will happen for it (if requested), but the subsequent WIFEXITED
  notification will not be delivered until all other threads exit.
+(As a corollary, if the other threads in the thread group are being
+traced, they will not exit until they have been either waited for
+and restarted or detached, thereby blocking the exit notification
+of the group leader to wait(2) and waitpid(2).)


Are you trying to prevent others falling into the same trap you fell into:
thinking that if you see PTRACE_EVENT_EXIT on a thread, it has exited?
I think your wording is not good at that. How about this?


  stop will happen for it (if requested), but the subsequent WIFEXITED
  notification will not be delivered until all other threads exit.
+(Note that other threads, if they are traced and are in
+PTRACE_EVENT_EXIT ptrace-stop, they did not exit yet.
+If you assume otherwise and wait on a thread leader to exit now,
+this may hang.
+You need to restart or detach them for them to exit. Generally,
+only seeing WIFEXITED or WIFSIGNALED status from the thread is
+a definite sign that it exited).
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: ptrace.2: BUGS (missing WIFEXITED notification)

[Vegard Nossum]

On 08/12/2016 06:15 PM, Denys Vlasenko wrote:
> On 08/12/2016 04:50 PM, Vegard Nossum wrote:
>   stop will happen for it (if requested), but the subsequent WIFEXITED
>   notification will not be delivered until all other threads exit.
> +(As a corollary, if the other threads in the thread group are being
> +traced, they will not exit until they have been either waited for
> +and restarted or detached, thereby blocking the exit notification
> +of the group leader to wait(2) and waitpid(2).)
>
>
> Are you trying to prevent others falling into the same trap you fell into:
> thinking that if you see PTRACE_EVENT_EXIT on a thread, it has exited?

Yes, or not exactly; more precisely, the trap we fell into was thinking
that if we see PTRACE_EVENT_EXIT, that means waitpid() will return
immediately. I know, the distinction is kind of subtle...

> I think your wording is not good at that. How about this?
>
>
>   stop will happen for it (if requested), but the subsequent WIFEXITED
>   notification will not be delivered until all other threads exit.
> +(Note that other threads, if they are traced and are in
> +PTRACE_EVENT_EXIT ptrace-stop, they did not exit yet.
> +If you assume otherwise and wait on a thread leader to exit now,
> +this may hang.
> +You need to restart or detach them for them to exit. Generally,
> +only seeing WIFEXITED or WIFSIGNALED status from the thread is
> +a definite sign that it exited).

Alright, another iteration on your version gives the following:

"""
Keep in mind that other threads in the thread group which are in a
PTRACE_EVENT_EXIT ptrace-stop have not actually exited yet. This means
that attempting to wait for the group leader with waitpid() will hang
indefinitely. In order prevent this, you must make sure that the other
threads exit properly by either restarting them or detaching from them.
In general, only WIFEXITED and WIFSIGNALED indicate that the child
has definitely exited.
"""


Vegard
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: ptrace.2: BUGS (missing WIFEXITED notification)

[Denys Vlasenko]

On 08/12/2016 08:37 PM, Vegard Nossum wrote:
> On 08/12/2016 06:15 PM, Denys Vlasenko wrote:
>> On 08/12/2016 04:50 PM, Vegard Nossum wrote:
>>   stop will happen for it (if requested), but the subsequent WIFEXITED
>>   notification will not be delivered until all other threads exit.
>> +(As a corollary, if the other threads in the thread group are being
>> +traced, they will not exit until they have been either waited for
>> +and restarted or detached, thereby blocking the exit notification
>> +of the group leader to wait(2) and waitpid(2).)
>>
>>
>> Are you trying to prevent others falling into the same trap you fell into:
>> thinking that if you see PTRACE_EVENT_EXIT on a thread, it has exited?
>
> Yes, or not exactly; more precisely, the trap we fell into was thinking
> that if we see PTRACE_EVENT_EXIT, that means waitpid() will return
> immediately. I know, the distinction is kind of subtle...
>
>> I think your wording is not good at that. How about this?
>>
>>
>>   stop will happen for it (if requested), but the subsequent WIFEXITED
>>   notification will not be delivered until all other threads exit.
>> +(Note that other threads, if they are traced and are in
>> +PTRACE_EVENT_EXIT ptrace-stop, they did not exit yet.
>> +If you assume otherwise and wait on a thread leader to exit now,
>> +this may hang.
>> +You need to restart or detach them for them to exit. Generally,
>> +only seeing WIFEXITED or WIFSIGNALED status from the thread is
>> +a definite sign that it exited).
>
> Alright, another iteration on your version gives the following:
>
> """
> Keep in mind that other threads in the thread group which are in a
> PTRACE_EVENT_EXIT ptrace-stop have not actually exited yet. This means
> that attempting to wait for the group leader with waitpid() will hang
> indefinitely. In order prevent this, you must make sure that the other
> threads exit properly by either restarting them or detaching from them.
> In general, only WIFEXITED and WIFSIGNALED indicate that the child
> has definitely exited.
> """

Looks good. Let's shorten it a bit:

Keep in mind that other threads in the thread group which are in a
PTRACE_EVENT_EXIT ptrace-stop have not exited yet.
Attempting to wait for the group leader with waitpid() will hang
indefinitely. You should make sure that all other threads
exit properly by restarting them or detaching from them.
In general, only WIFEXITED and WIFSIGNALED status indicate that
a thread has definitely exited.

This looks ok to me.
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html