Vger broken w.r.t. gdb

Vger broken w.r.t. gdb

[Daniel Jacobowitz]

After compiling the current vger 2.3.10 (gcc 2.95
-fno-strict-aliasing), I discovered an odd problem.  Strace works just
fine, but gdb doesn't.  Simply 'gdb ls', run it, quit.  No processes
will die but every new process will recieve a SIGTRAP immediately (not
quite sure if it's before or after the exec() yet).  Ignoring SIGTRAP
seems to make no difference.  We're reaching the function
ProgramCheckException in arch/ppc/kernel/traps.c:

Jul 28 21:07:32 drow kernel: Program check exception at PC: 3000c1a8, SR: 2d032, vector=700

This function includes the code:

        if (regs->msr & 0x100000) {
                /* IEEE FP exception */
                _exception(SIGFPE, regs);
        } else if (regs->msr & 0x20000) {
                /* trap exception */
#if defined(CONFIG_XMON) || defined(CONFIG_KGDB)
                if (debugger_bpt(regs))
                        return;
#endif
printk("Program check exception at PC: %lx, SR: %lx, vector=%lx\n",
  regs->nip, regs->msr, regs->trap);
                _exception(SIGTRAP, regs);
        } else {
                _exception(SIGILL, regs);
        }

I don't understand what this is supposed to do.  When are program check
exceptions generated?  And most of all, what are those MSR bits?  The
first one is (1 << 20), or bit 11; this is labeled as Reserved in my
reference.  The second is (1 << 17), or bit 14; this is marked
Implementation Dependent, and a note adds it is unused on the 601.

I would guess that the first one is supposed to check bit 20, or
(1 << 11); that's FE0 and controls recoverable and precise floating
point interrupts, so that would make more sense.  And my hunch is that
the second should be checking MSR_SE (1 << 10, bit 21).

Of course, since this file hasn't changed in an age, that leaves me
completely baffled as to why this was never a problem before.


Dan

/--------------------------------\  /--------------------------------\
|       Daniel Jacobowitz        |__|        SCS Class of 2002       |
|   Debian GNU/Linux Developer    __    Carnegie Mellon University   |
|         dan@debian.org         |  |       dmj+@andrew.cmu.edu      |
\--------------------------------/  \--------------------------------/

[[ This message was sent via the linuxppc-dev mailing list.  Replies are ]]
[[ not  forced  back  to the list, so be sure to Cc linuxppc-dev if your ]]
[[ reply is of general interest. Please check http://lists.linuxppc.org/ ]]
[[ and http://www.linuxppc.org/ for useful information before posting.   ]]

Re: Vger broken w.r.t. gdb

[Paul Mackerras]

Daniel Jacobowitz <drow@false.org> wrote:

> After compiling the current vger 2.3.10 (gcc 2.95
> -fno-strict-aliasing), I discovered an odd problem.  Strace works just
> fine, but gdb doesn't.  Simply 'gdb ls', run it, quit.  No processes
> will die but every new process will recieve a SIGTRAP immediately (not
> quite sure if it's before or after the exec() yet).  Ignoring SIGTRAP
> seems to make no difference.  We're reaching the function
> ProgramCheckException in arch/ppc/kernel/traps.c:
> 
> Jul 28 21:07:32 drow kernel: Program check exception at PC: 3000c1a8, SR: 2d032, vector=700

That address looks to be in some shared library.  I haven't been using
2.3.x lately (it dies somewhere near `starting crond' on my SMP
powermac), but I would suspect that the ppc ptrace code needs some
work.  It looks like gdb is inserting a trap instruction in a shared
library which is going into the in-memory copy of the page rather than
into a copy of the page.

> I don't understand what this is supposed to do.  When are program check
> exceptions generated?  And most of all, what are those MSR bits?  The

The program check exception is generated for various conditions such
as floating-point exceptions, illegal instructions, privileged
instructions in user mode, or trap instructions.  On a program check
exception, the CPU copies the MSR into the SRR1 register and then sets
one or more of the high bits of SRR1 to indicate the cause of the
exception.  The regs->msr value actually comes from the SRR1 register.

> first one is (1 << 20), or bit 11; this is labeled as Reserved in my
> reference.  The second is (1 << 17), or bit 14; this is marked
> Implementation Dependent, and a note adds it is unused on the 601.

You need to look at the SRR1 bit settings for the program check
exception.

Paul.

[[ This message was sent via the linuxppc-dev mailing list.  Replies are ]]
[[ not  forced  back  to the list, so be sure to Cc linuxppc-dev if your ]]
[[ reply is of general interest. Please check http://lists.linuxppc.org/ ]]
[[ and http://www.linuxppc.org/ for useful information before posting.   ]]

Re: Vger broken w.r.t. gdb

[Daniel Jacobowitz]

On Thu, Jul 29, 1999 at 02:00:39PM +1000, Paul Mackerras wrote:
> 
> Daniel Jacobowitz <drow@false.org> wrote:
> 
> > After compiling the current vger 2.3.10 (gcc 2.95
> > -fno-strict-aliasing), I discovered an odd problem.  Strace works just
> > fine, but gdb doesn't.  Simply 'gdb ls', run it, quit.  No processes
> > will die but every new process will recieve a SIGTRAP immediately (not
> > quite sure if it's before or after the exec() yet).  Ignoring SIGTRAP
> > seems to make no difference.  We're reaching the function
> > ProgramCheckException in arch/ppc/kernel/traps.c:
> > 
> > Jul 28 21:07:32 drow kernel: Program check exception at PC: 3000c1a8, SR: 2d032, vector=700
> 
> That address looks to be in some shared library.  I haven't been using
> 2.3.x lately (it dies somewhere near `starting crond' on my SMP
> powermac), but I would suspect that the ppc ptrace code needs some
> work.  It looks like gdb is inserting a trap instruction in a shared
> library which is going into the in-memory copy of the page rather than
> into a copy of the page.

Well, that came from running /bin/ls:
drow:/usr/src/kernel/linux/arch/ppc/kernel$ ldd /bin/ls
        libc.so.6 => /lib/libc.so.6 (0x016ba000)
        /lib/ld.so.1 => /lib/ld.so.1 (0x08000000)

And ls itself loads at 0x01800000 from what I can tell/remember.

0x30000000 seems to be where things are being mmap()'d.  Well, more or
less.  Relevant snippets:

execve("/bin/ls", ["ls"], [/* 21 vars */]) = 0
brk(0)                                  = 0x184d7cc
open("/etc/ld.so.cache", O_RDONLY)      = 3
mmap(0, 30308, PROT_READ, MAP_PRIVATE, 3, 0) = 0x30014000
close(3)                                = 0

Ahah, the first mmap was above the address we died at.

open("/lib/libc.so.6", O_RDONLY)        = 3
fstat(3, {st_mode=0200000, st_size=933225093, ...}) = 0
read(3, "\177ELF\1\2\1\0\0\0\0\0\0\0\0\0\0\3\0\24\0\0\0\1\0\2 \30"..., 4096) = 4096
mmap(0x16ba000, 1268792, PROT_READ|PROT_EXEC, MAP_PRIVATE, 3, 0) = 0x16ba000
mprotect(0x1799000, 355384, PROT_NONE)  = 0
close(3)                                = 0
munmap(0x30014000, 30308)               = 0

libc itself gets mapped in lower.

So my question is, what is at 0x3000c1a8?  It would appear, if I am not
misreading binfmt_elf.c, to be the program itself in its original
mmap'd location.  I'm not at all confident of that conclusion, though.

> 
> > I don't understand what this is supposed to do.  When are program check
> > exceptions generated?  And most of all, what are those MSR bits?  The
> 
> The program check exception is generated for various conditions such
> as floating-point exceptions, illegal instructions, privileged
> instructions in user mode, or trap instructions.  On a program check
> exception, the CPU copies the MSR into the SRR1 register and then sets
> one or more of the high bits of SRR1 to indicate the cause of the
> exception.  The regs->msr value actually comes from the SRR1 register.

So regs->msr is actually from SRR1 of the running program?  Does it
generally get stuffed back into the MSR when that task is running?

> > first one is (1 << 20), or bit 11; this is labeled as Reserved in my
> > reference.  The second is (1 << 17), or bit 14; this is marked
> > Implementation Dependent, and a note adds it is unused on the 601.
> 
> You need to look at the SRR1 bit settings for the program check
> exception.

... which I don't have a reference to at the moment.  Is this sort of
thing in the Motorola documentation for the processor?  I would hope
so.  I'll look through that tomorrow.


Dan

/--------------------------------\  /--------------------------------\
|       Daniel Jacobowitz        |__|        SCS Class of 2002       |
|   Debian GNU/Linux Developer    __    Carnegie Mellon University   |
|         dan@debian.org         |  |       dmj+@andrew.cmu.edu      |
\--------------------------------/  \--------------------------------/

[[ This message was sent via the linuxppc-dev mailing list.  Replies are ]]
[[ not  forced  back  to the list, so be sure to Cc linuxppc-dev if your ]]
[[ reply is of general interest. Please check http://lists.linuxppc.org/ ]]
[[ and http://www.linuxppc.org/ for useful information before posting.   ]]

Re: Vger broken w.r.t. gdb

[Paul Mackerras]

Daniel Jacobowitz <drow@false.org> wrote:

> So my question is, what is at 0x3000c1a8?  It would appear, if I am not
> misreading binfmt_elf.c, to be the program itself in its original
> mmap'd location.  I'm not at all confident of that conclusion, though.

Cat /proc/<pid>/maps while you have the ls process stopped at an
appropriate point.

> So regs->msr is actually from SRR1 of the running program?  Does it
> generally get stuffed back into the MSR when that task is running?

Yep, but the high bits get ignored.

Paul.

[[ This message was sent via the linuxppc-dev mailing list.  Replies are ]]
[[ not  forced  back  to the list, so be sure to Cc linuxppc-dev if your ]]
[[ reply is of general interest. Please check http://lists.linuxppc.org/ ]]
[[ and http://www.linuxppc.org/ for useful information before posting.   ]]

Re: Vger broken w.r.t. gdb

[Daniel Jacobowitz]

On Thu, Jul 29, 1999 at 03:48:59PM +1000, Paul Mackerras wrote:
> Daniel Jacobowitz <drow@false.org> wrote:
> 
> > So my question is, what is at 0x3000c1a8?  It would appear, if I am not
> > misreading binfmt_elf.c, to be the program itself in its original
> > mmap'd location.  I'm not at all confident of that conclusion, though.
> 
> Cat /proc/<pid>/maps while you have the ls process stopped at an
> appropriate point.
> 
> > So regs->msr is actually from SRR1 of the running program?  Does it
> > generally get stuffed back into the MSR when that task is running?
> 
> Yep, but the high bits get ignored.

Now I'm really confused.

>From my readings it appears that we should only reach this point if a
trap instruction was encountered.  But from what I can see no trap
instruction exists at that address.  At
<http://www.them.org/~drow/check-core> is a core dump I obtained while
my system was in this confusing state; check-test in the same directory
is the program responsible.  The instruction appears to be in
_dl_debug_state from what I can tell, but no trap instruction was
present there.

Perhaps something having to do with instruction caching by the
processor?  This is a completely wild guess, but if a trap instruction
was encountered, and then gdb cleared it, and the instruction cache was
not flushed...

Dan

/--------------------------------\  /--------------------------------\
|       Daniel Jacobowitz        |__|        SCS Class of 2002       |
|   Debian GNU/Linux Developer    __    Carnegie Mellon University   |
|         dan@debian.org         |  |       dmj+@andrew.cmu.edu      |
\--------------------------------/  \--------------------------------/

[[ This message was sent via the linuxppc-dev mailing list.  Replies are ]]
[[ not  forced  back  to the list, so be sure to Cc linuxppc-dev if your ]]
[[ reply is of general interest. Please check http://lists.linuxppc.org/ ]]
[[ and http://www.linuxppc.org/ for useful information before posting.   ]]

Re: Vger broken w.r.t. gdb

[Paul Mackerras]

Daniel Jacobowitz <drow@false.org> wrote:

> Now I'm really confused.
> 
> >From my readings it appears that we should only reach this point if a
> trap instruction was encountered.  But from what I can see no trap
> instruction exists at that address.  At

A common thing that gdb does is to put trap instructions into the
memory image of a process being debugged.  It sounds to me as though,
with all the page cache and memory management changes that have gone
on in 2.3, we now have the undesirable situation that a trap
instruction put in by gdb hasn't stayed confined to the process being
debugged but has leaked into the copy of the page that is being used
by other processes.

> Perhaps something having to do with instruction caching by the
> processor?  This is a completely wild guess, but if a trap instruction
> was encountered, and then gdb cleared it, and the instruction cache was
> not flushed...

Well, whatever the problem is, it's something new in 2.3, it's not a
long-standing omission of some cache flushing stuff.  It's far more
likely to be something in the ptrace syscall code which hasn't been
correctly updated to cope with the recent page cache and VM changes.

Paul.

[[ This message was sent via the linuxppc-dev mailing list.  Replies are ]]
[[ not  forced  back  to the list, so be sure to Cc linuxppc-dev if your ]]
[[ reply is of general interest. Please check http://lists.linuxppc.org/ ]]
[[ and http://www.linuxppc.org/ for useful information before posting.   ]]

Re: Vger broken w.r.t. gdb

[Daniel Jacobowitz]

On Fri, Jul 30, 1999 at 03:24:26PM +1000, Paul Mackerras wrote:
> Daniel Jacobowitz <drow@false.org> wrote:
> 
> > Now I'm really confused.
> > 
> > >From my readings it appears that we should only reach this point if a
> > trap instruction was encountered.  But from what I can see no trap
> > instruction exists at that address.  At
> 
> A common thing that gdb does is to put trap instructions into the
> memory image of a process being debugged.  It sounds to me as though,
> with all the page cache and memory management changes that have gone
> on in 2.3, we now have the undesirable situation that a trap
> instruction put in by gdb hasn't stayed confined to the process being
> debugged but has leaked into the copy of the page that is being used
> by other processes.

That makes sense, but wouldn't it have shown up in the core dump if
that were the case?



Dan

/--------------------------------\  /--------------------------------\
|       Daniel Jacobowitz        |__|        SCS Class of 2002       |
|   Debian GNU/Linux Developer    __    Carnegie Mellon University   |
|         dan@debian.org         |  |       dmj+@andrew.cmu.edu      |
\--------------------------------/  \--------------------------------/

[[ This message was sent via the linuxppc-dev mailing list.  Replies are ]]
[[ not  forced  back  to the list, so be sure to Cc linuxppc-dev if your ]]
[[ reply is of general interest. Please check http://lists.linuxppc.org/ ]]
[[ and http://www.linuxppc.org/ for useful information before posting.   ]]

nasty powerpc mmap problems (was: Re: Vger broken w.r.t. gdb)

[Daniel Jacobowitz]

[CC me in replies, I'm not on linux-kernel]

Background from the thread on linuxppc-dev: with the current vger tree,
gdb's breakpoints are apparently being set in the wrong copy of a
shared library, and not properly removed.  After running gdb over any
program, even with no extra breakpoints set, every dynamically linked
program run inside or outside of gdb dies with SIGTRAP.  The SIGTRAP
appears to be in _dl_debug_state or thereabouts in ld.so.

However, it now appears that this is symptomizing some much larger
problems with mmap().

On Fri, Jul 30, 1999 at 03:24:26PM +1000, Paul Mackerras wrote:
> Daniel Jacobowitz <drow@false.org> wrote:
> 
> > Now I'm really confused.
> > 
> > >From my readings it appears that we should only reach this point if a
> > trap instruction was encountered.  But from what I can see no trap
> > instruction exists at that address.  At
> 
> A common thing that gdb does is to put trap instructions into the
> memory image of a process being debugged.  It sounds to me as though,
> with all the page cache and memory management changes that have gone
> on in 2.3, we now have the undesirable situation that a trap
> instruction put in by gdb hasn't stayed confined to the process being
> debugged but has leaked into the copy of the page that is being used
> by other processes.

Woah!

I've found a much simpler test case for this problem.  It seems to me
to be more than ptrace being broken.  I wrote two test programs; mmap1
merely opens a file, mmaps it, closes it, prints some debugging info,
sleeps, and prints a little more.  Notice the permissions:
  int fd = open("testfile", O_RDONLY);
  ptr = mmap(0, 0x4000, PROT_READ|PROT_EXEC, MAP_SHARED, fd, 0);
  close(fd);

'attach' attaches to mmap1 and changes one value via PTRACE_POKEDATA.

Regard this:

drow:~/mmap$ dd if=/dev/zero of=testfile bs=4096 count=4
4+0 records in
4+0 records out
drow:~/mmap$ od testfile
0000000 000000 000000 000000 000000 000000 000000 000000 000000
*
0040000
drow:~/mmap$ ./mmap1
ptr[0x3000] is 0.
ptr + 0x3000 is 0x30017000.
And I am 1446

(in another window)
drow:~/mmap$ ./attach 1446 0x3001700
Value read was 0.
Value reread was 1.

(in the first window)
ptr[0x3000] is 1.
drow:~/mmap$ od testfile
0000000 000000 000000 000000 000000 000000 000000 000000 000000
*
0030000 000000 000001 000000 000000 000000 000000 000000 000000
0030020 000000 000000 000000 000000 000000 000000 000000 000000
*
0040000




Now hold on one second here!  We just changed the testfile that we'd
opened readonly and mmaped read|exec.  Something is definitely messed
up here.  Worse yet, this happens even if I have no permission to write
to the testfile.

Hopefully this is just on powerpc; could anyone running vger unstable
on another architecture test this?  The test programs are at
<http://www.them.org/~drow/mmap/>.  Can anyone offer insight?




Dan

/--------------------------------\  /--------------------------------\
|       Daniel Jacobowitz        |__|        SCS Class of 2002       |
|   Debian GNU/Linux Developer    __    Carnegie Mellon University   |
|         dan@debian.org         |  |       dmj+@andrew.cmu.edu      |
\--------------------------------/  \--------------------------------/

[[ This message was sent via the linuxppc-dev mailing list.  Replies are ]]
[[ not  forced  back  to the list, so be sure to Cc linuxppc-dev if your ]]
[[ reply is of general interest. Please check http://lists.linuxppc.org/ ]]
[[ and http://www.linuxppc.org/ for useful information before posting.   ]]

Re: nasty powerpc mmap problems (was: Re: Vger broken w.r.t. gdb)

[Daniel Jacobowitz]

[-- Attachment #1: Type: text/plain, Size: 1506 bytes --]

I think I've found the problem.  access_process_vm() would trigger a
page fault for write if the page was unmapped, but not if it was
already mapped for read.  The attached patch fixed all my problems.


On Fri, Jul 30, 1999 at 03:07:39AM -0400, Daniel Jacobowitz wrote:
> [CC me in replies, I'm not on linux-kernel]
> 
> Background from the thread on linuxppc-dev: with the current vger tree,
> gdb's breakpoints are apparently being set in the wrong copy of a
> shared library, and not properly removed.  After running gdb over any
> program, even with no extra breakpoints set, every dynamically linked
> program run inside or outside of gdb dies with SIGTRAP.  The SIGTRAP
> appears to be in _dl_debug_state or thereabouts in ld.so.

...

> I've found a much simpler test case for this problem.  It seems to me
> to be more than ptrace being broken.  I wrote two test programs; mmap1
> merely opens a file, mmaps it, closes it, prints some debugging info,
> sleeps, and prints a little more.  Notice the permissions:
>   int fd = open("testfile", O_RDONLY);
>   ptr = mmap(0, 0x4000, PROT_READ|PROT_EXEC, MAP_SHARED, fd, 0);
>   close(fd);


Dan

/--------------------------------\  /--------------------------------\
|       Daniel Jacobowitz        |__|        SCS Class of 2002       |
|   Debian GNU/Linux Developer    __    Carnegie Mellon University   |
|         dan@debian.org         |  |       dmj+@andrew.cmu.edu      |
\--------------------------------/  \--------------------------------/

[-- Attachment #2: fix-ptrace.diff --]
[-- Type: text/plain, Size: 539 bytes --]

Index: ptrace.c
===================================================================
RCS file: /cvs/linux/linux/kernel/ptrace.c,v
retrieving revision 1.2
diff -c -p -r1.2 ptrace.c
*** ptrace.c	1999/07/23 02:04:32	1.2
--- ptrace.c	1999/08/02 02:23:45
*************** repeat:
*** 38,43 ****
--- 38,45 ----
  	pgtable = pte_offset(pgmiddle, addr);
  	if (!pte_present(*pgtable))
  		goto fault_in_page;
+ 	if (write && !pte_write(*pgtable))
+ 		goto fault_in_page;
  	page = pte_page(*pgtable);
  	if (MAP_NR(page) >= max_mapnr)
  		return 0;