[kvm-ppc-devel] some TLB tracing

[kvm-ppc-devel] some TLB tracing

[Hollis Blanchard]

I've updated the patches at http://penguinppc.org/~hollisb/kvm/ . (By
the way, I can post the patches separately and/or as part of a gitweb
tree if that would be more convenient for reviewers.)

The significant addition is a relay channel to debug TLB operations. I
would like to write a visualizer for this to illustrate the relationship
between the guest TLB and shadow TLB, but for now I've placed a
text-only analyzer (decode.py) in that directory.

There is also a sysrq hack to enable verbose printks (not intending to
send that one upstream).

To use the relay stuff, on the host:
        % gcc -Wall -O2 44x_tlb.c -o 44x_tlb
        % ./44x_tlb > tlblog.bin &
        % qemu-system-ppcemb ...
        % python decoder.py tlblog.bin # can be run on any system
        
Currently we're running into a problem that sometimes manifests itself
like this:

Freeing unused kernel memory: 340k init
Unable to handle kernel paging request for instruction fetch
Faulting instruction address: 0xc0000ac0
Oops: Kernel access of bad area, sig: 11 [#1]
Bamboo
Modules linked in:
NIP: c0000ac0 LR: 10009e8c CTR: 100088a0
REGS: bfedea50 TRAP: 0400   Not tainted  (2.6.25-rc3-hg1858cec8eb87-dirty)
MSR: 00021000 <ME>  CR: 40020022  XER: 20000000
TASK = c8810400[1] 'init' THREAD: c881e000
GPR00: 00000000 bfedeb00 1005ba50 00000000 10071fcc 10072018 00000019 00000000 
GPR08: 00000004 10070000 00000000 10072434 40000044 10079a64 00000000 00000000 
GPR16: 00000000 00000000 c881ff60 10071fcc 10071fcc 10071fcc 00000000 10072018 
GPR24: 00000000 00000004 10072018 00000018 00000000 00000fff 10071fcc 00021000 
NIP [c0000ac0] Decrementer+0x0/0xc0
LR [10009e8c] 0x10009e8c
Call Trace:
Instruction dump:
914b00b0 3d400002 614a1002 512a0420 4800c6ad c000b374 c000d91c 60000000 
60000000 60000000 60000000 60000000 <7d5043a6> 7d7143a6 7c3443a6 7d400026 

When you replay the TLB log from that run, you get the following output:

% python decoder.py tlblog.bin
idx tid      epn      rpn     attr
00:  00 00000000 00000000 00000007
01:  00 00000001 ef600000 ef600703
02:  00 00000002 ef400000 ef400703
03:  00 00000003 eec00000 eec00703
04:  00 00000004 ef600000 ef600703
05:  00 00000005 eec00000 eec00703
06:  00 00000006 0887f000 08870307
07:  00 00000007 ef600000 ef600703
08:  00 00000008 ef600000 ef600703
09:  00 00000009 ef600000 ef600703
10:  00 0000000a ef600000 ef600703
11:  00 0000000b ef600000 ef600703
12:  00 0000000c eec00000 eec00703
13:  00 0000000d e8001000 e8000703
14:  00 0000000e 00000000 00000000
15:  00 0000000f 00000000 00000000
16:  00 00000010 00000000 00000000
17:  01 00000011 08898000 08890149
18:  00 00000012 00000000 00000000
19:  00 00000013 00000000 00000000
20:  00 00000014 00000000 00000000
21:  01 00000015 088a2000 088a0149
22:  01 1000a210 088a2000 088a016d
23:  00 00000017 00000000 00000000
24:  00 00000018 00000000 00000000
25:  00 00000019 00000000 00000000
26:  00 0000001a 00000000 00000000
27:  01 0000001b 088a0000 088a0149
28:  00 0000001c 00000000 00000000
29:  00 0000001d 00000000 00000000
30:  00 0000001e 00000000 00000000
31:  00 0000001f 00000000 00000000
32:  00 00000020 00000000 00000000
33:  00 00000021 00000000 00000000
34:  01 10004210 0889c000 08890149
35:  00 00000023 00000000 00000000
36:  00 00000024 00000000 00000000
37:  00 00000025 00000000 00000000
38:  00 00000026 00000000 00000000
39:  00 00000027 00000000 00000000
40:  00 00000028 00000000 00000000
41:  01 10007210 0889f000 08890149
42:  00 0000002a 00000000 00000000
43:  00 0000002b 00000000 00000000
44:  00 0000002c 00000000 00000000
45:  00 0000002d 00000000 00000000
46:  00 0000002e 00000000 00000000
47:  00 0000002f 00000000 00000000
48:  00 00000030 00000000 00000000
49:  00 00000031 00000000 00000000
50:  00 00000032 00000000 00000000
51:  00 00000033 00000000 00000000
52:  00 00000034 00000000 00000000
53:  00 00000035 00000000 00000000
54:  00 00000036 00000000 00000000
55:  00 00000037 00000000 00000000
56:  00 00000038 00000000 00000000
57:  00 00000039 00000000 00000000
58:  00 0000003a 00000000 00000000
59:  00 0000003b 00000000 00000000
60:  00 0000003c 00000000 00000000
61:  00 0000003d 00000000 00000000
62:  00 0000003e 00000000 00000000
63:  00 c0000290 00000000 00000107

idx tid      epn      rpn     attr ref
00:  00 00000000 00000000 00000000 00
01:  00 00000000 00000000 00000000 00
02:  00 00000000 00000000 00000000 00
03:  00 00000000 00000000 00000000 00
04:  00 00000000 00000000 00000000 00
05:  00 00000000 00000000 00000000 00
06:  00 00000000 00000000 00000000 00
07:  00 00000000 00000000 00000000 00
08:  00 00000000 00000000 00000000 00
09:  00 00000000 00000000 00000000 00
10:  00 00000000 00000000 00000000 00
11:  00 00000000 00000000 00000000 00
12:  00 00000000 00000000 00000000 00
13:  00 00000000 00000000 00000000 00
14:  00 00000000 00000000 00000000 00
15:  00 00000000 00000000 00000000 00
16:  00 00000000 00000000 00000000 00
17:  00 00000000 00000000 00000000 00
18:  00 c020c310 0dfab000 0000003f 63
19:  00 c02d8310 0dc37000 0000003f 63
20:  00 c012a310 0eac9000 0000003f 63
21:  00 c000a310 0d829000 0000003f 63
22:  00 c012c310 0eacb000 0000003f 63
23:  00 c02ec310 0e3cf000 0000003f 63
24:  00 c0254310 0deb3000 0000003f 63
25:  00 c0237310 0de96000 0000003f 63
26:  00 c023c310 0de9b000 0000003f 63
27:  00 c02f5310 0e3d8000 0000003f 63
28:  00 c0244310 0dea3000 0000003f 63
29:  00 c0006310 0d825000 0000003f 63
30:  00 c0235310 0de94000 0000003f 63
31:  00 c0238310 0de97000 0000003f 63
32:  00 c02c9310 0dc28000 0000003f 63
33:  00 c0240310 0de9f000 0000003f 63
34:  00 c0246310 0dea5000 0000003f 63
35:  00 c021c310 0dfbb000 0000003f 63
36:  00 c0217310 0dfb6000 0000003f 63
37:  00 c0215310 0dfb4000 0000003f 63
38:  00 c0214310 0dfb3000 0000003f 63
39:  00 c0213310 0dfb2000 0000003f 63
40:  00 c0233310 0de92000 0000003f 63
41:  00 c0234310 0de93000 0000003f 63
42:  01 bfede310 0dc24000 0000001f 16
43:  00 c0005310 0d824000 0000003f 63
44:  00 c0049310 0dee8000 0000003f 63
45:  00 c0250310 0deaf000 0000003f 63
46:  00 c0026310 0dfe5000 0000003f 63
47:  00 c0027310 0dfe6000 0000003f 63
48:  00 c0032310 0dff1000 0000003f 63
49:  00 c8922310 0e360000 0000003f 63
50:  00 c8810310 0d8a0000 0000003f 63
51:  00 c8818310 0d8ac000 0000003f 63
52:  00 c02e7310 0e3ca000 0000003f 63
53:  00 c000d310 0d82c000 0000003f 63
54:  00 c003f310 0dffe000 0000003f 63
55:  00 00000000 00000000 00000000 00
56:  00 00000000 00000000 00000000 00
57:  00 00000000 00000000 00000000 00
58:  00 00000000 00000000 00000000 00
59:  00 00000000 00000000 00000000 00
60:  00 00000000 00000000 00000000 00
61:  00 00000000 00000000 00000000 00
62:  00 00000000 00000000 00000000 00
63:   -        -        -        -  -

No smoking guns, but shadow TLB entry 42 strikes me as odd: it refers to
guest TLB entry 16, but that guest entry is invalid.

There is indeed no shadow mapping for 0xc0000ac0, but if the host saw
that fault we should have created one from guest entry 63...

-- 
Hollis Blanchard
IBM Linux Technology Center


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
kvm-ppc-devel mailing list
kvm-ppc-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/kvm-ppc-devel