ptrace unexpected SIGTRAP (trace bit) on x86, x86_64 kernel 2.6.13.2

ptrace unexpected SIGTRAP (trace bit) on x86, x86_64 kernel 2.6.13.2

[John Reiser]

Ptrace is giving unexpected SIGTRAP (trace bit) in kernel 2.6.13.2
on both x86 and x86_64.

The 8-instruction program below just execve()s itself over and over.
When run under gdb, the first user-visible SIGTRAP is expected due to
the 'int3'.  But the second user-visible SIGTRAP is unexpected, as
there is no reason to trap.

Changing the line "nop; int3" to "nop; nop" gives a program that
just spins merrily when run under /bin/bash.  But gdb sees a SIGTRAP,
with the $pc pointing after the second 'nop'.  When run under strace
(strace gdb ./execve; (gdb) run), the process spins merrily with
no unexpected SIGTRAP.


-----execve.S
#include <asm/unistd.h>

/*
gcc -o execve -nostartfiles -nostdlib execve.S
gdb ./execve
run
p/x $ps
   # 0x202
c
p/x $ps
   # 0x302  TF (0x100) set, but should not be
*/

_start: .globl _start
        nop; int3
        popl %ebp  # argc
        movl (%esp),%ebx  # same filename from argv[0]
        movl %esp,%ecx    # same argv
        lea 4(%esp,%ebp,4),%edx  # same envp
        movl $__NR_execve,%eax   # here we go 'round the mulberry bush, ...
        int $0x80
-----end of execve.S

Previous history, and translation for x86_64 are at:
https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=144805#c23

-- 
John Reiser, jreiser@BitWagon.com

Re: ptrace unexpected SIGTRAP (trace bit) on x86, x86_64 kernel 2.6.13.2

[Daniel Jacobowitz]

On Thu, Sep 29, 2005 at 08:58:25AM -0700, John Reiser wrote:
> Ptrace is giving unexpected SIGTRAP (trace bit) in kernel 2.6.13.2
> on both x86 and x86_64.
> 
> The 8-instruction program below just execve()s itself over and over.
> When run under gdb, the first user-visible SIGTRAP is expected due to
> the 'int3'.  But the second user-visible SIGTRAP is unexpected, as
> there is no reason to trap.
> 
> Changing the line "nop; int3" to "nop; nop" gives a program that
> just spins merrily when run under /bin/bash.  But gdb sees a SIGTRAP,
> with the $pc pointing after the second 'nop'.  When run under strace
> (strace gdb ./execve; (gdb) run), the process spins merrily with
> no unexpected SIGTRAP.

Most likely GDB is receiving the SIGTRAP generated by execve().  I
don't know why it would appear to be after the int3 rather than before.
GDB has strictly limited support for programs which exec, mostly due to
thorny user interface issues.

-- 
Daniel Jacobowitz
CodeSourcery, LLC