finding origin of dTLB miss

finding origin of dTLB miss

[Ed L Cashin]

Hi.  I'm trying to learn more about a dTLB fault that's occurring on
my system.  Inside do_sparc64_fault, these things are true:

  address:              0xefffe000   (bottom of the user stack region)
  fault code:           0x2          (dTLB miss)
  insn:                 0xc2060000
  tstate:               0x11009606
  tstate & TSTATE_PRIV: 1

Since tstate & TSTATE_PRIV is true, it seems like insn should be in
the kernel, but it doesn't look like a kernel address, since the
userland memory maps look like this:

00010000-00012000 r-xp 00000000 03:04 585266   /home/ecashin/ra/macc-switch/a.out
00020000-00022000 rwxp 00000000 03:04 585266   /home/ecashin/ra/macc-switch/a.out
70000000-7001c000 r-xp 00000000 03:04 341709   /lib/ld-2.3.3.so
7002a000-7002c000 rwxp 0001a000 03:04 341709   /lib/ld-2.3.3.so
7002c000-70174000 r-xp 00000000 03:04 341855   /lib/libc-2.3.3.so
70174000-7017c000 ---p 00148000 03:04 341855   /lib/libc-2.3.3.so
7017c000-7018a000 rwxp 00140000 03:04 341855   /lib/libc-2.3.3.so
7018a000-7018c000 rwxp 00000000 00:00 0 
efffe000-f0000000 rwxp 00000000 00:00 0     

Ksymoops doesn't say anything helpful when I input the line, "TSTATE:
0000000011009606 TPC: c2060000".  

I get the feeling I'm going about this in an entirely wrong way.  What
is the proper way to find out where this dTLB miss is occurring?

-- 
--Ed L Cashin            |   PGP public key:
  ecashin@uga.edu        |   http://noserose.net/e/pgp/

Re: finding origin of dTLB miss

[David S. Miller]

On Thu, 15 Jan 2004 17:42:20 -0500
Ed L Cashin <ecashin@uga.edu> wrote:

> Hi.  I'm trying to learn more about a dTLB fault that's occurring on
> my system.  Inside do_sparc64_fault, these things are true:
> 
>   address:              0xefffe000   (bottom of the user stack region)
>   fault code:           0x2          (dTLB miss)
>   insn:                 0xc2060000
>   tstate:               0x11009606
>   tstate & TSTATE_PRIV: 1

The 'insn' is "ld [%i0], %g1"  Presumably the compute stack address
causing the fault is in register %i0.

> Since tstate & TSTATE_PRIV is true, it seems like insn should be in
> the kernel, but it doesn't look like a kernel address, since the
> userland memory maps look like this:

The userland memory maps have nothing to do with where this fault
is occuring.  If TSTATE_PRIV is true, then absolutely the fault is occuring
in the kernel somewhere.

Also, the kernel and the user live in two seperate address spaces on sparc64.
Therefore, virtual address "X" means something totally different when in kernel
mode than the same virtual address "X" means in user mode.  Comparing raw
virtual addresses does not tell you if something is in the kernel or not, you must
combine the state of TSTATE_PRIV and the virtual address to determine where
something really is.

> Ksymoops doesn't say anything helpful when I input the line, "TSTATE:
> 0000000011009606 TPC: c2060000".  

That number after "TPC" is not the program counter, it is the instruction
itself.

Get the real TPC value from regs->tpc at the time of the fault, the look
up that instruction in the 'vmlinux' kernel image either using 'objdump'
or 'gdb'.

Re: finding origin of dTLB miss

[Ed L Cashin]

Thanks very much for the helpful response.

"David S. Miller" <davem@redhat.com> writes:

...
> The userland memory maps have nothing to do with where this fault
> is occuring.  If TSTATE_PRIV is true, then absolutely the fault is occuring
> in the kernel somewhere.
>
> Also, the kernel and the user live in two seperate address spaces on sparc64.

AH!  I may have known that at some point, but I had definitely
regressed into i386 assumptions.

> Therefore, virtual address "X" means something totally different when in kernel
> mode than the same virtual address "X" means in user mode.  Comparing raw
> virtual addresses does not tell you if something is in the kernel or not, you must
> combine the state of TSTATE_PRIV and the virtual address to determine where
> something really is.
>
>> Ksymoops doesn't say anything helpful when I input the line, "TSTATE:
>> 0000000011009606 TPC: c2060000".  
>
> That number after "TPC" is not the program counter, it is the instruction
> itself.
>
> Get the real TPC value from regs->tpc at the time of the fault, the look
> up that instruction in the 'vmlinux' kernel image either using 'objdump'
> or 'gdb'.

Oh, thanks.  I didn't notice that get_fault_insn dereferences
regs->tpc.  

I was able to find the kernel code that is inducing the dTLB miss
using gdb.  :)

-- 
--Ed L Cashin            |   PGP public key:
  ecashin@uga.edu        |   http://noserose.net/e/pgp/