More waitpid issues with CLONE_DETACHED/CLONE_THREAD

More waitpid issues with CLONE_DETACHED/CLONE_THREAD

[Daniel Jacobowitz]

This may be related to the python bug reported today...

I've been playing around with gdbserver support for NPTL threading all day
today.  Right now it works, except that when I say "kill" in the GDB client,
gdbserver hangs.  The problem is that we kill the child, and wait for it,
but wait never returns it.

write(2, "Killing inferior\n", 17)      = 17
ptrace(PTRACE_CONT, 8454, 0, SIG_0)     = 0
tkill(8454, SIGKILL)                    = 0
--- SIGCHLD (Child exited) @ 0 (0) ---
waitpid(8454, 0xbfffec04, WNOHANG)      = 0
waitpid(8454, 0xbfffec04, WNOHANG|__WCLONE) = -1 ECHILD (No child processes)
nanosleep({0, 1000000}, 0)              = ? ERESTART_RESTARTBLOCK (To be restarted)
--- SIGCHLD (Child exited) @ 0 (0) ---
setup()                                 = 0
waitpid(8454, 0xbfffec04, WNOHANG)      = 0
waitpid(8454, 0xbfffec04, WNOHANG|__WCLONE) = -1 ECHILD (No child processes)

and so on (looping on waitpid).  At this point, process 8454 is marked as a
zombie, and nothing can reap it.  After gdbserver is killed, it reparents to
init:

Name:   linux-dp
State:  Z (zombie)
SleepAVG:       91%
Tgid:   8454
Pid:    8454
PPid:   1
TracerPid:      0

but init can't reap it either.

8454 was the original (i.e. non-CLONE_DETACHED) thread.  Same behavior if I
use ptrace_kill.

GDB doesn't suffer from the same problem.  A little time with strace and I
found out why: GDB PTRACE_KILL's the detached threads, PTRACE_KILL's the
parent thread, waitpid's the detached threads, and then waitpid's the parent
thread.  No design, just different order of items on the linked list.

If I change gdbserver to do "kill thread; wait for thread; kill next thread;
wait for next thread; kill parent last; wait for parent last" then it
terminates and I don't get an unkillable zombie.

ptrace(PTRACE_KILL, 18348, 0, 0)        = 0
waitpid(18348, 0xbfffec04, WNOHANG)     = -1 ECHILD (No child processes)
--- SIGCHLD (Child exited) @ 0 (0) ---
waitpid(18348, [WIFSIGNALED(s) && WTERMSIG(s) == SIGKILL], WNOHANG|__WCLONE) = 18348
ptrace(PTRACE_KILL, 18349, 0, 0)        = 0
waitpid(18349, 0xbfffec04, WNOHANG)     = -1 ECHILD (No child processes)
waitpid(18349, [WIFSIGNALED(s) && WTERMSIG(s) == SIGKILL], WNOHANG|__WCLONE) = 18349
--- SIGCHLD (Child exited) @ 0 (0) ---
ptrace(PTRACE_KILL, 18350, 0, 0)        = 0
waitpid(18350, 0xbfffec04, WNOHANG)     = -1 ECHILD (No child processes)
waitpid(18350, [WIFSIGNALED(s) && WTERMSIG(s) == SIGKILL], WNOHANG|__WCLONE) = 18350
ptrace(PTRACE_KILL, 18351, 0, 0)        = 0
waitpid(18351, 0xbfffec04, WNOHANG)     = -1 ECHILD (No child processes)
waitpid(18351, [WIFSIGNALED(s) && WTERMSIG(s) == SIGKILL], WNOHANG|__WCLONE) = 18351
ptrace(PTRACE_KILL, 18352, 0, 0)        = 0
waitpid(18352, 0xbfffec04, WNOHANG)     = -1 ECHILD (No child processes)
waitpid(18352, [WIFSIGNALED(s) && WTERMSIG(s) == SIGKILL], WNOHANG|__WCLONE) = 18352
--- SIGCHLD (Child exited) @ 0 (0) ---
ptrace(PTRACE_KILL, 18329, 0, 0)        = 0
waitpid(18329, [WIFSIGNALED(s) && WTERMSIG(s) == SIGKILL], WNOHANG) = 18329
exit_group(0)                           = ?

So it looks like something gets very confused if the parent is SIGKILLed
before the children.  What should happen?

-- 
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer

Re: More waitpid issues with CLONE_DETACHED/CLONE_THREAD

[Linus Torvalds]

On Sat, 31 Jan 2004, Daniel Jacobowitz wrote:
>
> This may be related to the python bug reported today...

Indeed.

Having a "waitpid(x, .., WNOHANG)" return 0 is a very interesting 
condition. That condition basically guarantees that:

 - the kernel did find the child
 - but the kernel decided that the child cannot be reaped right then.

If you see the process as a Zombie in a "ps" listing, then we know that 
that isn't the reason why it couldn't be reaped. Can you verify that 
/proc/<pid>/status shows it as "Z (zombie)"?

In fact, if we see it as "Z (zombie)", we know even more: it means that 
wait_task_zombie() was never called, because that would have started out 
with changing the process state to "X (dead)".

And that in turn implies that "eligible_child()" would have returned 2.

Which is a normal occurrence: it happens when a process group leader still 
has threads attached to it. At that point it may be a Zombie, but we can't 
reap it yet. The threads have to go away before the thing can be reaped.

Can you verify that that process doesn't have any sub-threads? (Again, 
that should be easily visible in /proc/<pid>/task/).

Another alternative is that the process is a zombie, but it is being
traced. When that happens, it shows up on the "ptrace_children" list, and
we'll see in in wait4(), but we won't be able to reap it. 

Roland, Ingo - have you followed the discussion on linux-kernel? Something 
strange does seem to be going on..

		Linus

Re: More waitpid issues with CLONE_DETACHED/CLONE_THREAD

[Daniel Jacobowitz]

On Sat, Jan 31, 2004 at 08:38:43PM -0800, Linus Torvalds wrote:
> 
> 
> On Sat, 31 Jan 2004, Daniel Jacobowitz wrote:
> >
> > This may be related to the python bug reported today...
> 
> Indeed.
> 
> Having a "waitpid(x, .., WNOHANG)" return 0 is a very interesting 
> condition. That condition basically guarantees that:
> 
>  - the kernel did find the child
>  - but the kernel decided that the child cannot be reaped right then.
> 
> If you see the process as a Zombie in a "ps" listing, then we know that 
> that isn't the reason why it couldn't be reaped. Can you verify that 
> /proc/<pid>/status shows it as "Z (zombie)"?

Yes, it is a zombie when the waitpid is executed.

> In fact, if we see it as "Z (zombie)", we know even more: it means that 
> wait_task_zombie() was never called, because that would have started out 
> with changing the process state to "X (dead)".
> 
> And that in turn implies that "eligible_child()" would have returned 2.
> 
> Which is a normal occurrence: it happens when a process group leader still 
> has threads attached to it. At that point it may be a Zombie, but we can't 
> reap it yet. The threads have to go away before the thing can be reaped.
> 
> Can you verify that that process doesn't have any sub-threads? (Again, 
> that should be easily visible in /proc/<pid>/task/).

It is quite easily visible - in fact, it's hilarious.

 8454 pts/8    Z      0:00 [linux-dp] <defunct>

drow@nevyn:~% ls /proc/8454
auxv  cmdline  cwd@  environ  exe@  fd/  maps  mem  mounts  root@  stat

drow@nevyn:~% ls /proc/8454/task
ls: /proc/8454/task: No such file or directory

What that means I'm not entirely sure.

> Another alternative is that the process is a zombie, but it is being
> traced. When that happens, it shows up on the "ptrace_children" list, and
> we'll see in in wait4(), but we won't be able to reap it. 

At this point it is being traced - this is gdbserver.  But the same
process that is tracing it is calling waitpid.  And the problem
persists after the tracer dies; the process above is actually from a
couple of hours ago, and its tracer was killed.

> Roland, Ingo - have you followed the discussion on linux-kernel? Something 
> strange does seem to be going on..

The only two kernels I've tried were 2.6.0-test7 and 2.6.2-rc3, by the
way - same behavior in both.  I'll try to write a single program
testcase for this.

-- 
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer

Re: More waitpid issues with CLONE_DETACHED/CLONE_THREAD

[Daniel Jacobowitz]

On Sat, Jan 31, 2004 at 11:43:31PM -0500, Daniel Jacobowitz wrote:
> The only two kernels I've tried were 2.6.0-test7 and 2.6.2-rc3, by the
> way - same behavior in both.  I'll try to write a single program
> testcase for this.

Here you go.  The bug turns out not to be related directly to
CLONE_DETACHED.  Compile testcase with -DNOTHREAD to use fork (well,
clone, but without the fancy flags), without -DNOTHREAD to use
CLONE_DETACHED | CLONE_THREAD.

In either mode, compile with -DBUG to kill and reap the parent before
the child.  I get a variety of unkillable processes, hangs, and
unkilled parents; sometimes in the -DNOTHREAD case the parent simply is
not stopped by the PTRACE_KILL (remains in S state).  If you kill -9 it
from a terminal it becomes a zombie despite the fact that its parent is
in waitpid for it.

You may need to fiddle a little bit to get this to compile depending on
your distribution of choice, it assumes some constants from
<linux/ptrace.h> that require 2.6 headers... should work on any 2.6
kernel.

/* -DBUG to kill the parent before the child -> hang.  */
/* -DNOTHREAD to us fork instead of clone.  */

#include <stdio.h>
#include <string.h>
#include <unistd.h>
#include <sys/syscall.h>
#include <signal.h>
#include <sys/ptrace.h>
#include <sched.h>
#include <sys/wait.h>
#include <linux/ptrace.h>

int stack_one[8192], stack_two[8192];

int thread_func_two()
{
  write (1, "Thread 2\n", 9);
  while (1)
    sleep (1);
}

int thread_func_one()
{
  int ret;
  char retstr[10];

  write (1, "Thread 1\n", 9);

  ptrace (PTRACE_TRACEME, 0, 0, 0);

  write (1, "Thread 1 signalling\n", 20);

  syscall (SYS_tkill, getpid (), SIGUSR1);

  write (1, "Thread 1 cloning\n", 17);

  ret = clone (thread_func_two, stack_two + 8192,
#ifdef NOTHREAD
	SIGCHLD,
#else
	CLONE_DETACHED | CLONE_THREAD | CLONE_SIGHAND | CLONE_VM | CLONE_FS,
#endif
	NULL);
  sprintf (retstr, "= %d\n", ret);
  write (1, retstr, strlen (retstr));

  write (1, "Thread 1 sleeping\n", 18);

  while (1)
    sleep (1);
}

int main()
{
  int ret, wstat;
  int child = fork(), child2 = 0;

  if (child == 0)
    return thread_func_one();

  ptrace (PTRACE_SETOPTIONS, child, 0, PTRACE_O_TRACECLONE | PTRACE_O_TRACEFORK);
  ret = waitpid (child, &wstat, __WALL);

  ptrace (PTRACE_CONT, child, 0, 0);
  ret = waitpid (child, &wstat, __WALL);
  ptrace (PTRACE_GETEVENTMSG, child, 0, &child2);
  ret = waitpid (child2, &wstat, __WALL);

  ptrace (PTRACE_CONT, child, 0, 0);
  ptrace (PTRACE_CONT, child2, 0, 0);

#ifndef BUG
  ptrace (PTRACE_KILL, child2, 0, 0);
  ret = waitpid (child2, &wstat, __WALL);
#endif

  ptrace (PTRACE_KILL, child, 0, 0);
  ret = waitpid (child, &wstat, __WALL);

#ifdef BUG
  ptrace (PTRACE_KILL, child2, 0, 0);
  ret = waitpid (child2, &wstat, __WALL);
#endif

  return 0;
}

-- 
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer

Re: More waitpid issues with CLONE_DETACHED/CLONE_THREAD

[Linus Torvalds]

On Sun, 1 Feb 2004, Daniel Jacobowitz wrote:
> 
> Here you go.  The bug turns out not to be related directly to
> CLONE_DETACHED.  Compile testcase with -DNOTHREAD to use fork (well,
> clone, but without the fancy flags), without -DNOTHREAD to use
> CLONE_DETACHED | CLONE_THREAD.

I don't think this bug has anything to do with anything else.

This program seems to show that PTRACE_KILL simply doesn't work.

What ptrace_kill() does is effectively:

 - stage 1: ptrace_check_attach(): wait for child stopped

 - stage 2: PTRACE_KILL does:

	child->exit_code = SIGKILL;
	wake_up_process(child);

and the thing is, it looks like the signal handling changes have totally
made the child ignore the "exit_code" thing, unless I'm seriously
misreading something.

Roland, you know this code better than I do. Any comments?

I suspect the PTRACE_KILL logic should also do a

	spin_lock_irqsave(child->sighand->siglock, flags);
	sigaddset(&child->pending->signal, SIGKILL);
	set_tsk_thread_flag(child, TIF_SIGPENDING);
	spin_unlock_irqrestore(child->sighand->siglock, flags);

	ptrace_detach(child);

which would set the SIGKILL thing properly, but I suspect we had a good
reason not to do it that way originally. 

Daniel?

		Linus

Re: More waitpid issues with CLONE_DETACHED/CLONE_THREAD

[Roland McGrath]

> and the thing is, it looks like the signal handling changes have totally
> made the child ignore the "exit_code" thing, unless I'm seriously
> misreading something.

The only time that exit_code has ever been consulted is in
get_signal_to_deliver after the thread has stopped to notify its ptracer
and then been resumed.  That is still the case.  So it's only being ignored
if the child is stopped in some other place and not there.  I don't see the
scenario where that could be happening without the explicit involvement of
user-level stop signals (i.e. not the implicit stops done by ptrace).

Your changes would be equivalent to an atomic combination of
"kill(child_pid, SIGKILL); PTRACE_DETACH(pid);".  That is different from
what PTRACE_KILL normally does, which leaves the dying child attached so
that it reports its death to the ptracer before the original parent.

I haven't really looked into the problem with Dan's test case yet (didn't
seem to reproduce, but I haven't tried a current and cruft-free kernel yet).  
But I don't see any problem with the implementation of PTRACE_KILL.


Thanks,
Roland

Re: More waitpid issues with CLONE_DETACHED/CLONE_THREAD

[Linus Torvalds]

On Sun, 1 Feb 2004, Roland McGrath wrote:
> 
> I haven't really looked into the problem with Dan's test case yet (didn't
> seem to reproduce, but I haven't tried a current and cruft-free kernel yet).  
> But I don't see any problem with the implementation of PTRACE_KILL.

Hmm.. For me, Dan's program (with "-DBUG -DNOTHREAD") results in a 
sleeping parent, and both children are likewise just sleeping. Despite the 
fact that the parent just did the PTRACE_KILL on child2.

I didn't trace it through the kernel, it just looked like PTRACE_KILL 
didn't do anything.

		Linus

Re: More waitpid issues with CLONE_DETACHED/CLONE_THREAD

[Andries Brouwer]

On Sun, Feb 01, 2004 at 04:55:11PM -0800, Linus Torvalds wrote:

> On Sun, 1 Feb 2004, Roland McGrath wrote:

> > I haven't really looked into the problem with Dan's test case yet (didn't
> > seem to reproduce, but I haven't tried a current and cruft-free kernel yet).  
> > But I don't see any problem with the implementation of PTRACE_KILL.
> 
> Hmm.. For me, Dan's program (with "-DBUG -DNOTHREAD") results in a 
> sleeping parent, and both children are likewise just sleeping. Despite the 
> fact that the parent just did the PTRACE_KILL on child2.
> 
> I didn't trace it through the kernel, it just looked like PTRACE_KILL 
> didn't do anything.

I agree with both of you.

So, I think what happens is that PTRACE_KILL immediately after the PTRACE_CONT
works because there is no schedule in between, so the effect of PTRACE_KILL
is still seen by the (grand)child.
Once the grandchild has returned it has become immune for PTRACE_KILL.
Maybe there is no bug.

Andries

Re: More waitpid issues with CLONE_DETACHED/CLONE_THREAD

[Linus Torvalds]

On Mon, 2 Feb 2004, Andries Brouwer wrote:
> 
> So, I think what happens is that PTRACE_KILL immediately after the PTRACE_CONT
> works because there is no schedule in between, so the effect of PTRACE_KILL
> is still seen by the (grand)child.

Well, Duh! You'r eobviously right.

PTRACE_KILL is a special case, since it's supposed to work _regardless_ of 
whether the process being traced is actually stopped for tracing or not.

And Roland is correct that PTRACE_KILL works fine _if_ it is stopped. 

But for the case where it isn't (and Daniel's program isn't, since it did 
the PTRACE_CONT), PTRACE_KILL does nothing.

> Maybe there is no bug.

No, I do believe that PTRACE_KILL is supposed to kill the child even if it 
wasn't synchronized. See the special case for "ptrace_check_attach()", 
which allows a PTRACE_KILL to happen even for a nonsynchronized target.

		Linus

Re: More waitpid issues with CLONE_DETACHED/CLONE_THREAD

[Daniel Jacobowitz]

On Sun, Feb 01, 2004 at 01:41:48PM -0800, Linus Torvalds wrote:
> 
> 
> On Sun, 1 Feb 2004, Daniel Jacobowitz wrote:
> > 
> > Here you go.  The bug turns out not to be related directly to
> > CLONE_DETACHED.  Compile testcase with -DNOTHREAD to use fork (well,
> > clone, but without the fancy flags), without -DNOTHREAD to use
> > CLONE_DETACHED | CLONE_THREAD.
> 
> I don't think this bug has anything to do with anything else.
> 
> This program seems to show that PTRACE_KILL simply doesn't work.

> and the thing is, it looks like the signal handling changes have totally
> made the child ignore the "exit_code" thing, unless I'm seriously
> misreading something.

That may be (though I don't think so) but it reproduces without
PTRACE_KILL too.  Try the attached, which just replaced PTRACE_KILL
with PTRACE_CONT/tkill(pid, SIGKILL).  Still get zombies.  I haven't
tried reproducing entirely without ptrace yet.

> Roland, you know this code better than I do. Any comments?
> 
> I suspect the PTRACE_KILL logic should also do a
> 
> 	spin_lock_irqsave(child->sighand->siglock, flags);
> 	sigaddset(&child->pending->signal, SIGKILL);
> 	set_tsk_thread_flag(child, TIF_SIGPENDING);
> 	spin_unlock_irqrestore(child->sighand->siglock, flags);
> 
> 	ptrace_detach(child);
> 
> which would set the SIGKILL thing properly, but I suspect we had a good
> reason not to do it that way originally. 
> 
> Daniel?

I doubt there was a good reason.  This code hasn't changed in a hell of
a long time - it probably predates everything up there except sigaddset
:).

/* -DBUG to kill the parent before the child -> hang.  */
/* -DNOTHREAD to us fork instead of clone.  */

#include <stdio.h>
#include <string.h>
#include <unistd.h>
#include <sys/syscall.h>
#include <signal.h>
#include <sys/ptrace.h>
#include <sched.h>
#include <sys/wait.h>
#include <linux/ptrace.h>

int stack_one[8192], stack_two[8192];

int thread_func_two()
{
  write (1, "Thread 2\n", 9);
  while (1)
    sleep (1);
}

int thread_func_one()
{
  int ret;
  char retstr[10];

  write (1, "Thread 1\n", 9);

  ptrace (PTRACE_TRACEME, 0, 0, 0);

  write (1, "Thread 1 signalling\n", 20);

  syscall (SYS_tkill, getpid (), SIGUSR1);

  write (1, "Thread 1 cloning\n", 17);

  ret = clone (thread_func_two, stack_two + 8192,
#ifdef NOTHREAD
	SIGCHLD,
#else
	CLONE_DETACHED | CLONE_THREAD | CLONE_SIGHAND | CLONE_VM | CLONE_FS,
#endif
	NULL);
  sprintf (retstr, "= %d\n", ret);
  write (1, retstr, strlen (retstr));

  write (1, "Thread 1 sleeping\n", 18);

  while (1)
    sleep (1);
}

int main()
{
  int ret, wstat;
  int child = fork(), child2 = 0;

  if (child == 0)
    return thread_func_one();

  ptrace (PTRACE_SETOPTIONS, child, 0, PTRACE_O_TRACECLONE | PTRACE_O_TRACEFORK);
  ret = waitpid (child, &wstat, __WALL);

  ptrace (PTRACE_CONT, child, 0, 0);
  ret = waitpid (child, &wstat, __WALL);
  ptrace (PTRACE_GETEVENTMSG, child, 0, &child2);
  ret = waitpid (child2, &wstat, __WALL);

  ptrace (PTRACE_CONT, child, 0, 0);
  ptrace (PTRACE_CONT, child2, 0, 0);

#ifndef BUG
  ptrace (PTRACE_CONT, child2, 0, 0);
  syscall (SYS_tkill, child2, SIGKILL);
  ret = waitpid (child2, &wstat, __WALL);
#endif

  ptrace (PTRACE_CONT, child, 0, 0);
  syscall (SYS_tkill, child, SIGKILL);
  ret = waitpid (child, &wstat, __WALL);

#ifdef BUG
  ptrace (PTRACE_CONT, child2, 0, 0);
  syscall (SYS_tkill, child2, SIGKILL);
  ret = waitpid (child2, &wstat, __WALL);
#endif

  return 0;
}


-- 
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer

Re: More waitpid issues with CLONE_DETACHED/CLONE_THREAD

[Davide Libenzi]

On Sun, 1 Feb 2004, Daniel Jacobowitz wrote:

> On Sun, Feb 01, 2004 at 01:41:48PM -0800, Linus Torvalds wrote:
> > 
> > 
> > On Sun, 1 Feb 2004, Daniel Jacobowitz wrote:
> > > 
> > > Here you go.  The bug turns out not to be related directly to
> > > CLONE_DETACHED.  Compile testcase with -DNOTHREAD to use fork (well,
> > > clone, but without the fancy flags), without -DNOTHREAD to use
> > > CLONE_DETACHED | CLONE_THREAD.
> > 
> > I don't think this bug has anything to do with anything else.
> > 
> > This program seems to show that PTRACE_KILL simply doesn't work.
> 
> > and the thing is, it looks like the signal handling changes have totally
> > made the child ignore the "exit_code" thing, unless I'm seriously
> > misreading something.
> 
> That may be (though I don't think so) but it reproduces without
> PTRACE_KILL too.  Try the attached, which just replaced PTRACE_KILL
> with PTRACE_CONT/tkill(pid, SIGKILL).  Still get zombies.  I haven't
> tried reproducing entirely without ptrace yet.

Here w/out ptrace works as advertised.



- Davide




/* -DBUG to kill the parent before the child -> hang.  */
/* -DNOTHREAD to us fork instead of clone.  */

#include <stdio.h>
#include <string.h>
#include <unistd.h>
#include <sys/syscall.h>
#include <signal.h>
#include <sched.h>
#include <errno.h>
#include <sys/wait.h>


int stack_one[8192], stack_two[8192];
int fds1[2], fds2[2];


int thread_func_two()
{

	fprintf(stdout, "Thread 2: ppid = %d\n", getppid());
	while (1) {
		sleep (1);
		fprintf(stdout, "Thread 2: ppid = %d\n", getppid());
	}
}

int thread_func_one()
{
	int ret;

	fprintf(stdout, "Thread 1\n");

	read(fds1[0], &ret, sizeof(int));

	fprintf(stdout, "Thread 1 cloning\n");

	ret = clone (thread_func_two, stack_two + 8192,
#ifdef NOTHREAD
		     SIGCHLD,
#else
		     CLONE_DETACHED | CLONE_THREAD | CLONE_SIGHAND | CLONE_VM | CLONE_FS,
#endif
		     NULL);

	fprintf(stdout, "child2 = %d\n", ret);

	write(fds2[1], &ret, sizeof(int));

	fprintf(stdout, "Thread 1 sleeping\n");
	while (1)
		sleep (1);
}

int main()
{
	int ret, wstat, child, child2;

	if (pipe(fds1) < 0 || pipe(fds2)) {
		perror("pipe");
		return 1;
	}

	child = fork();
	if (child == 0)
		return thread_func_one();

	fprintf(stdout, "child = %d\n", child);

	write(fds1[1], &child, sizeof(int));
	read(fds2[0], &child2, sizeof(int));

	fprintf(stdout, "parent got child2 = %d\n", child2);

#ifndef BUG
	kill(child2, SIGKILL);
	ret = waitpid (child2, &wstat, __WALL);
	fprintf(stdout, "waitpid(%d) = %d (%s)\n", child2, ret, strerror(errno));
#endif

	kill(child, SIGKILL);
	ret = waitpid (child, &wstat, __WALL);
	fprintf(stdout, "waitpid(%d) = %d (%s)\n", child, ret, strerror(errno));

#ifdef BUG
	sleep(2);

	kill(child2, SIGKILL);
	ret = waitpid (child2, &wstat, __WALL);
	fprintf(stdout, "waitpid(%d) = %d (%s)\n", child2, ret, strerror(errno));
#endif

	return 0;
}

Re: More waitpid issues with CLONE_DETACHED/CLONE_THREAD

[Davide Libenzi]

On Sun, 1 Feb 2004, Davide Libenzi wrote:

> > That may be (though I don't think so) but it reproduces without
> > PTRACE_KILL too.  Try the attached, which just replaced PTRACE_KILL
> > with PTRACE_CONT/tkill(pid, SIGKILL).  Still get zombies.  I haven't
> > tried reproducing entirely without ptrace yet.
> 
> Here w/out ptrace works as advertised.

Yes, even for something that uses tkill() :-)


- Davide




/* -DBUG to kill the parent before the child -> hang.  */
/* -DNOTHREAD to us fork instead of clone.  */

#include <stdio.h>
#include <string.h>
#include <unistd.h>
#include <sys/syscall.h>
#include <signal.h>
#include <sched.h>
#include <errno.h>
#include <sys/wait.h>


int stack_one[8192], stack_two[8192];
int fds1[2], fds2[2];


int thread_func_two()
{

	fprintf(stdout, "Thread 2: ppid = %d\n", getppid());
	while (1) {
		sleep (1);
		fprintf(stdout, "Thread 2: ppid = %d\n", getppid());
	}
}

int thread_func_one()
{
	int ret;

	fprintf(stdout, "Thread 1\n");

	read(fds1[0], &ret, sizeof(int));

	fprintf(stdout, "Thread 1 cloning\n");

	ret = clone (thread_func_two, stack_two + 8192,
#ifdef NOTHREAD
		     SIGCHLD,
#else
		     CLONE_DETACHED | CLONE_THREAD | CLONE_SIGHAND | CLONE_VM | CLONE_FS,
#endif
		     NULL);

	fprintf(stdout, "child2 = %d\n", ret);

	write(fds2[1], &ret, sizeof(int));

	fprintf(stdout, "Thread 1 sleeping\n");
	while (1)
		sleep (1);
}

int main()
{
	int ret, wstat, child, child2;

	if (pipe(fds1) < 0 || pipe(fds2)) {
		perror("pipe");
		return 1;
	}

	child = fork();
	if (child == 0)
		return thread_func_one();

	fprintf(stdout, "child = %d\n", child);

	write(fds1[1], &child, sizeof(int));
	read(fds2[0], &child2, sizeof(int));

	fprintf(stdout, "parent got child2 = %d\n", child2);

#ifndef BUG
	syscall(SYS_tkill, child2, SIGKILL);
	ret = waitpid (child2, &wstat, __WALL);
	fprintf(stdout, "waitpid(%d) = %d (%s)\n", child2, ret,
		ret < 0 ? strerror(errno): "Success");
#endif

	syscall(SYS_tkill, child, SIGKILL);
	ret = waitpid (child, &wstat, __WALL);
	fprintf(stdout, "waitpid(%d) = %d (%s)\n", child, ret,
		ret < 0 ? strerror(errno): "Success");

#ifdef BUG
	sleep(2);

	syscall(SYS_tkill, child2, SIGKILL);
	ret = waitpid (child2, &wstat, __WALL);
	fprintf(stdout, "waitpid(%d) = %d (%s)\n", child2, ret,
		ret < 0 ? strerror(errno): "Success");
#endif

	return 0;
}

fs/eventpoll : reduce sizeof(struct epitem)

[dada1]

Hi Davide

On a x86_64 platform, I noticed that the size of (struct epitem) is 136 
bytes, rounded to 192 (because of  L1_CACHE alignment of 64 bytes)

If you reorder some fields of this structure, we can reduce the size to 
128 bytes.

The rationale is to group nwait & fd integer (32bits) fields, instead of 
mixing them with pointers (64 bits)

See you

Eric Dumazet

diff -u  linux-2.6.2/fs/eventpoll.c.save linux-2.6.2/fs/eventpoll.c
--- eventpoll.c.save    2004-02-04 14:59:49.000000000 +0100
+++ eventpoll.c 2004-02-04 15:02:20.000000000 +0100
@@ -241,14 +241,15 @@
        /* Number of active wait queue attached to poll operations */
        int nwait;

+       /* The file descriptor this item refers to */
+       int fd;
+
        /* List containing poll wait queues */
        struct list_head pwqlist;

        /* The "container" of this item */
        struct eventpoll *ep;

-       /* The file descriptor this item refers to */
-       int fd;

        /* The file this item refers to */
        struct file *file;

Re: fs/eventpoll : reduce sizeof(struct epitem)

[Davide Libenzi]

On Wed, 4 Feb 2004, dada1 wrote:

> Hi Davide
> 
> On a x86_64 platform, I noticed that the size of (struct epitem) is 136 
> bytes, rounded to 192 (because of  L1_CACHE alignment of 64 bytes)
> 
> If you reorder some fields of this structure, we can reduce the size to 
> 128 bytes.
> 
> The rationale is to group nwait & fd integer (32bits) fields, instead of 
> mixing them with pointers (64 bits)

Good catch. Thx I will.



- Davide

Re: More waitpid issues with CLONE_DETACHED/CLONE_THREAD

[Linus Torvalds]

On Sat, 31 Jan 2004, Daniel Jacobowitz wrote:
> > 
> > Can you verify that that process doesn't have any sub-threads? (Again, 
> > that should be easily visible in /proc/<pid>/task/).
> 
> It is quite easily visible - in fact, it's hilarious.
> 
>  8454 pts/8    Z      0:00 [linux-dp] <defunct>
> 
> drow@nevyn:~% ls /proc/8454
> auxv  cmdline  cwd@  environ  exe@  fd/  maps  mem  mounts  root@  stat
> 
> drow@nevyn:~% ls /proc/8454/task
> ls: /proc/8454/task: No such file or directory

Oh damn. Yeah, we don't allow you to even see the threads in this case (it
checks "pid_alive(task)". We don't want you to try to confuse things by 
opening files of processes that we think are dead.

For the case of trying to figure out sub-threads of a dead thread group
leader, that may actually be a bug.

Just for testing this, you might remove the check for

        if (!pid_alive(task))
                goto out;

in proc_pident_lookup() in fs/proc/base.c.

Ingo - comments? We really want the sysadmin to be able to find threads 
that have a dead group leader, and right now that seems to be impossible.

		Linus

Re: More waitpid issues with CLONE_DETACHED/CLONE_THREAD

[Daniel Jacobowitz]

On Sat, Jan 31, 2004 at 09:12:33PM -0800, Linus Torvalds wrote:
> 
> 
> On Sat, 31 Jan 2004, Daniel Jacobowitz wrote:
> > > 
> > > Can you verify that that process doesn't have any sub-threads? (Again, 
> > > that should be easily visible in /proc/<pid>/task/).
> > 
> > It is quite easily visible - in fact, it's hilarious.
> > 
> >  8454 pts/8    Z      0:00 [linux-dp] <defunct>
> > 
> > drow@nevyn:~% ls /proc/8454
> > auxv  cmdline  cwd@  environ  exe@  fd/  maps  mem  mounts  root@  stat
> > 
> > drow@nevyn:~% ls /proc/8454/task
> > ls: /proc/8454/task: No such file or directory
> 
> Oh damn. Yeah, we don't allow you to even see the threads in this case (it
> checks "pid_alive(task)". We don't want you to try to confuse things by 
> opening files of processes that we think are dead.
> 
> For the case of trying to figure out sub-threads of a dead thread group
> leader, that may actually be a bug.
> 
> Just for testing this, you might remove the check for
> 
>         if (!pid_alive(task))
>                 goto out;
> 
> in proc_pident_lookup() in fs/proc/base.c.
> 
> Ingo - comments? We really want the sysadmin to be able to find threads 
> that have a dead group leader, and right now that seems to be impossible.

I thought that a new group leader would be swapped in to that TID?  But
I was always confused by the mechanics of that.

Note that processes going invisible this way is, um, a bit of a
security problem.

-- 
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer

Re: More waitpid issues with CLONE_DETACHED/CLONE_THREAD

[Roland McGrath]

> I thought that a new group leader would be swapped in to that TID?  But
> I was always confused by the mechanics of that.

The group leader never changes.  The zombie group leader just sticks around
until there are no other threads in the group.  (The only exception here is
exec.)

Re: More waitpid issues with CLONE_DETACHED/CLONE_THREAD

[Daniel Jacobowitz]

On Sat, Jan 31, 2004 at 09:42:52PM -0800, Roland McGrath wrote:
> > I thought that a new group leader would be swapped in to that TID?  But
> > I was always confused by the mechanics of that.
> 
> The group leader never changes.  The zombie group leader just sticks around
> until there are no other threads in the group.  (The only exception here is
> exec.)

Ah, thanks, it was exec that I was thinking of.

-- 
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer