Unexpected data TLB miss happens when guest OS executing a "bl" instruction

Fei K Chen
2012-07-02T15:34:23Z

We are debuging kvm on IBM poweren chip by RSICWatch tool. An unexpected data TLB miss happened and we can not explain why. Any one have met this before?

1. Guest OS executes a "bl" instruction with PC=0xC0000000005A49CC. According to the guest linux kernel objdump file, the next instruction will be "mflr r0" with PC=0xC000000000599CC0.

2. By single-step execution in RISCWatch, guest OS does jump to an instruction with PC=0xC000000000599CC0. At this time, RISCWatch tool can not display what the instruction is. We guess this is because there is no instruction TLB entry in hardware TLB for PC=0xC000000000599CC0. Thus an instruction TLB miss is expected if we press the "Asmstep" to execute the next instruction.

3. Unfortunately, poweren jumps an instruction with PC=0xC000000000051FF4 which is the beginning of data TLB miss entry in kvm. We read the values in spr SRR0 and DEAR. Both of them are 0xC000000000599CC0. We even can not imagine why this happens.

4. As external interrupt will happen during single-step debugging, we set a hardware breakpoint at PC=0xC000000000599CC0, and let poweren directly run to that point.

5. When poweren stops at PC=0xC000000000599CC0, from the output of RISCWatch, a "trap" instruction is placed at PC=0xC000000000599CC0. It is different with what should be according to the kernel objdump file. The only explanation we can imagine is that our kvm code set a wrong TLB entry for PC=0xC000000000599CC0 (it may be brought by that unexpected data TLB miss).