[Qemu-devel] Re: software breakpoints disappearing/reappearing in KVM/qemu

[Jan Kiszka <jan.kiszka@web.de>]

[-- Attachment #1: Type: text/plain, Size: 3025 bytes --]

On 2011-04-08 20:50, Craig Brozefsky wrote:
> First, thank you very much for your work on KVM/QEMU and in particular
> the GDB support.  I have been a heavy user of it while implementing a
> programmable debugger that monitors/traces guest operating systems.  I
> need more breakpoints than hardware BPs available, and I need the
> performance of KVM acceleration.
> 
> I am running into a problem with software breakpoints disappearing and
> re-appearing.  I set them via the GDB stub.  If I examine the location
> in the qemu monitor I will see them there, but on occasion (and with
> greater frequency the more breakpoints I have) a given breakpoint will
> disappear according to the monitor, and then re-appear.
> 
> I also have processes executing the instructions at the breakpoint without
> stopping.  This makes my results quite unpredictable, and it has led too a
> few days of learning all about Windows system working set paging.

KVM injects software breakpoints into the guest pages - and that
intransparently. If the guest starts to read those breakpoints or even
manipulate the code pages, all kinds of weird things can happen.

This particularly includes swapping. QEMU does not track if a guest
reads and then replaces the page. There is some basic check when
removing a software breakpoint that triggers if the page was manipulated
in the meantime. But when that check triggers, it's too late: the guest
likely already saved the patched page elsewhere and can stumble and fall
the next time it restores it to some other linear address and then
executes it. BTW, that's also not what gdb is designed to handle.

> 
> My first approach was to try and re-install the breakpoints after a
> page was faulted in, thinking that the INT3 was disappearing when the
> page was read back into memory.  Setting a hbreak, I was trapping
> MMAccessFault, and reinstalling all by breakpoints when it hit its
> return address (after the IO Page Read Calls).  That didn't solve
> the problem, and left me wondering if I need to take into consideration the way
> KVM/qemu handle guest memory.  And that brings me to my question.

This kind of swapping is guest business, specifically with EPT/NPT.

> 
> Assuming I handle the guest kernel paging issue properly, are you
> aware of issues with the debugging support in KVM/qemu that would
> result in the INT3 blinking in and out of existence?
> 
> In the meantime, I've been reading source code, but it'll be a few
> days before I figure out the details of memory mgmt under KVM.  Any
> hints would be greatly appreciated.

QEMU's gdbstub in KVM mode is simply not designed to account for guests
swapping out code pages that contain breakpoints. Due to the fact that
the Linux kernel does not do these weird things to its own code, at
least I never seriously thought about potential workarounds. As your
approach indicates, those might become fairly guest-specific - if they
can be implemented reliably at all.

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 259 bytes --]