inconsistent behavior with ptrace(TRACEME) and fork/exec

[Mike Frysinger <vapier@gentoo.org>]

[-- Attachment #1.1: Type: text/plain, Size: 1533 bytes --]

i'm hoping my understanding of ptrace is correct and that the attached test 
case (reduced from LTP) isnt just completely broken ... my understanding is 
that if a parent forks and the child does a ptrace(TRACEME) right before 
doing an exec(), the kernel should always halt it and wait indefinitely for 
the parent to start ptracing it.

unfortunately, this behavior seems to be unreliable.  most of the time it 
works, but sometimes the kernel does not halt the child and it gladly does 
the exec() that it set out to do.  i dont believe this to be a race in the 
user space parent component as forcing it to delay via judicious usage of 
sleep() shows the same behavior.

if  all goes well, we should only ever see "SUCCESS!" from the test case:
$ gcc -Wall ptrace-vfork-traceme.c -o ptrace-vfork-traceme
$ while ./ptrace-vfork-traceme ; do :; done
SUCCESS! :D
SUCCESS! :D
SUCCESS! :D
SUCCESS! :D
SUCCESS! :D
SUCCESS! :D
SUCCESS! :D
SUCCESS! :D
SUCCESS! :D
failure, child exited with 17: Child exited
wait() = 12275
status = 1407
        WIFEXITED = 0
        WEXITSTATUS = 5
        WIFSIGNALED = 0
        WTERMSIG = 127 (Unknown signal 127)

i'm testing 2.6.26/2.6.27 atm.  both x86_64 and ppc64 seem to behave the same.  
while gcc-4.3.2 / glibc-2.8 is in use in both places, i dont think that 
matters.

also, while the attached test uses vfork(), same behavior can be observed with 
fork().  vfork() is used because i like my test cases to work on both MMU and 
no-MMU systems.
-mike

[-- Attachment #1.2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 835 bytes --]

[-- Attachment #2: ptrace-vfork-traceme.c --]
[-- Type: text/x-csrc, Size: 1568 bytes --]

#define _GNU_SOURCE
#include <errno.h>
#include <signal.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <sys/ptrace.h>
#include <sys/wait.h>

#define fail(msg, args...) \
do { \
	fprintf(stderr, "FAIL:%i: " msg "\n", __LINE__, ## args); \
	exit(1); \
} while (0)

static void child_exit(int sig)
{
	int status;
	printf("failure, child exited with %i: %s\n", sig, strsignal(sig));
	printf("wait() = %i\n", wait(&status));
	printf("status = %i\n", status);
	printf("\tWIFEXITED = %i\n", WIFEXITED(status));
	printf("\tWEXITSTATUS = %i\n", WEXITSTATUS(status));
	printf("\tWIFSIGNALED = %i\n", WIFSIGNALED(status));
	printf("\tWTERMSIG = %i (%s)\n", WTERMSIG(status), strsignal(WTERMSIG(status)));
	exit(1);
}

int main(int argc, char *argv[])
{
	long pret;
	pid_t pid;

	/* child process ... shouldnt be executed, but just in case ... */
	if (argc > 1 && !strcmp(argv[1], "child"))
		fail("kernel should have halted me...");

	pid = vfork();
	if (pid == -1)
		fail("vfork() didnt work: %m");
	else if (!pid) {
		/* do the child stuff here */
		errno = 0;
		pret = ptrace(PTRACE_TRACEME, 0, NULL, NULL);
		if (pret && errno)
			fail("ptrace(PTRACE_TRACEME) = %li: %m", pret);

		int eret = execlp(argv[0], argv[0], "child", NULL);
		fail("execlp() = %i", eret);
	}

	/* do the parent stuff here */
	signal(SIGCHLD, child_exit);

	errno = 0;
	pret = ptrace(PTRACE_PEEKUSER, pid, NULL, NULL);
	if (pret && errno)
		fail("ptrace(PTRACE_PEEKUSER, %i) = %li: %m", pid, pret);

	puts("SUCCESS! :D");

	return 0;
}