[RFC] Reworking KVM_DEBUG_GUEST

[RFC] Reworking KVM_DEBUG_GUEST

[Jan Kiszka]

[-- Attachment #1.1: Type: text/plain, Size: 2271 bytes --]

Hi,

before going wild with my idea, I would like to collect some comments on
this approach:

While doing first kernel debugging with my debug register patches for
kvm, I quickly ran into the 4-breakpoints-only limitation that comes
from the fact that we blindly map software to hardware breakpoints.
Unhandy, simply suboptimal. Also, having 4 breakpoint slots hard-coded
in the generic interface is not fair to arch that may support more.
Moreover, we do not support watchpoints although this would easily be
feasible. But if we supported watchpoints (via debug registers on x86),
we would need the break out of the 4 slots limitations even earlier. In
short, I came to the conclusion that a rewrite of the KVM_DEBUG_GUEST
interface is required.

Why do we set breakpoints in the kernel? Why not simply catching all
debug traps, inserting software breakpoint ops into the guest code, and
handling all this stuff as normal debuggers do? And the hardware
breakpoints should just be pushed through the kernel interface like
ptrace does.

The new KVM_DEBUG_GUEST interface I currently have in mind would look
like this:

#define KVM_DBGGUEST_ENABLE		0x01
#define KVM_DBGGUEST_SINGLESTEP		0x02

struct kvm_debug_guest {
	__u32 control;
	struct kvm_debug_guest_arch arch;
}

Setting KVM_DBGGUEST_ENABLE would forward all debug-related traps to
userspace first, which can then decide to handle or re-inject them.
KVM_DBGGUEST_SINGLESTEP would work as before. And the extension for x86
would look like this:

struct kvm_debug_guest_arch {
	__u32 use_hw_breakpoints;
	__u64 debugreg[8];
}

If use_hw_breakpoints is non-zero, KVM would completely overwrite the
guest's debug registers with the content of debugreg, giving full
control of this feature to the host-side debugger (faking the content of
debug registers, effectively disabling them for the guest - as we now do
all the time).

Questions:
- Does anyone see traps and pitfalls in this approach?
- May I replace the existing interface with this one, or am I overseeing
  some use case that already worked with the current code so that ABI
  compatibility is required (most debug stuff should have been simply
  broken so far, also due to bugs in userland)?

Jan


[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 254 bytes --]

[-- Attachment #2: Type: text/plain, Size: 320 bytes --]

-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone

[-- Attachment #3: Type: text/plain, Size: 158 bytes --]

_______________________________________________
kvm-devel mailing list
kvm-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/kvm-devel

Re: [RFC] Reworking KVM_DEBUG_GUEST

[Jerone Young]

On Mon, 2008-05-12 at 13:34 +0200, Jan Kiszka wrote:
> Hi,
> 
> before going wild with my idea, I would like to collect some comments on
> this approach:
> 
> While doing first kernel debugging with my debug register patches for
> kvm, I quickly ran into the 4-breakpoints-only limitation that comes
> from the fact that we blindly map software to hardware breakpoints.
> Unhandy, simply suboptimal. Also, having 4 breakpoint slots hard-coded
> in the generic interface is not fair to arch that may support more.
> Moreover, we do not support watchpoints although this would easily be
> feasible. But if we supported watchpoints (via debug registers on x86),
> we would need the break out of the 4 slots limitations even earlier. In
> short, I came to the conclusion that a rewrite of the KVM_DEBUG_GUEST
> interface is required.
So embedded power is also limited to 4 hardware registers for break
points. But there are 2 sepreate registers fro watch points. The reason
to use the registers is the hardware does the work for you and (at least
on Power) will throw an exception or trap. Then you deal with it.

But you still face the fact that you can only have a small number of
breakpoints & watch points. Also you cannot use gdb in the guest at the
sametime while using the gdb stub on the guest itself (as there is only
one set of registers).


> 
> Why do we set breakpoints in the kernel? Why not simply catching all
> debug traps, inserting software breakpoint ops into the guest code, and
> handling all this stuff as normal debuggers do? And the hardware
> breakpoints should just be pushed through the kernel interface like
> ptrace does.

See above...But the cpu basically does the work for you. So you don't
have to try and go through and first insert a trap into the code in
memory. But then you have to remember the code that you replaced with
the trap and execute it after you handle the trap. This can get a little
hairy. 

Currently I'm actually implementing breakpoint support now in Power. But
you do have to create some mappings to handle traps and see if you put
the trap there, and execute the code you replaced. Also what if the
breakpoint is removed. Then you have to go back through and actually
replace the trap code. Doesn't sound hard, but I'm not sure of all the
pitfalls.

> 
> The new KVM_DEBUG_GUEST interface I currently have in mind would look
> like this:
> 
> #define KVM_DBGGUEST_ENABLE		0x01
> #define KVM_DBGGUEST_SINGLESTEP		0x02
> 
> struct kvm_debug_guest {
> 	__u32 control;
> 	struct kvm_debug_guest_arch arch;
> }


> Setting KVM_DBGGUEST_ENABLE would forward all debug-related traps to
> userspace first, which can then decide to handle or re-inject them.
> KVM_DBGGUEST_SINGLESTEP would work as before. And the extension for x86
> would look like this:
> 
> struct kvm_debug_guest_arch {
> 	__u32 use_hw_breakpoints;
> 	__u64 debugreg[8];
> }
> 
> If use_hw_breakpoints is non-zero, KVM would completely overwrite the
> guest's debug registers with the content of debugreg, giving full
> control of this feature to the host-side debugger (faking the content of
> debug registers, effectively disabling them for the guest - as we now do
> all the time).

Hmmm...so today at least the gdbstub in qemu does not inject traps and
track code that it trapped (I could be mistaken). This whould all need
to be implemented as well.

> 
> Questions:
> - Does anyone see traps and pitfalls in this approach?
> - May I replace the existing interface with this one, or am I overseeing
>   some use case that already worked with the current code so that ABI
>   compatibility is required (most debug stuff should have been simply
>   broken so far, also due to bugs in userland)?

> 
> Jan
> 
> -------------------------------------------------------------------------
> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
> Don't miss this year's exciting event. There's still time to save $100. 
> Use priority code J8TL2D2. 
> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
> _______________________________________________ kvm-devel mailing list kvm-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kvm-devel


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft 
Defy all challenges. Microsoft(R) Visual Studio 2008. 
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

Re: [RFC] Reworking KVM_DEBUG_GUEST

[Jan Kiszka]

[-- Attachment #1.1: Type: text/plain, Size: 4674 bytes --]

Jerone Young wrote:
> On Mon, 2008-05-12 at 13:34 +0200, Jan Kiszka wrote:
>> Hi,
>>
>> before going wild with my idea, I would like to collect some comments on
>> this approach:
>>
>> While doing first kernel debugging with my debug register patches for
>> kvm, I quickly ran into the 4-breakpoints-only limitation that comes
>> from the fact that we blindly map software to hardware breakpoints.
>> Unhandy, simply suboptimal. Also, having 4 breakpoint slots hard-coded
>> in the generic interface is not fair to arch that may support more.
>> Moreover, we do not support watchpoints although this would easily be
>> feasible. But if we supported watchpoints (via debug registers on x86),
>> we would need the break out of the 4 slots limitations even earlier. In
>> short, I came to the conclusion that a rewrite of the KVM_DEBUG_GUEST
>> interface is required.
> So embedded power is also limited to 4 hardware registers for break
> points. But there are 2 sepreate registers fro watch points. The reason
> to use the registers is the hardware does the work for you and (at least
> on Power) will throw an exception or trap. Then you deal with it.
> 
> But you still face the fact that you can only have a small number of
> breakpoints & watch points. Also you cannot use gdb in the guest at the
> sametime while using the gdb stub on the guest itself (as there is only
> one set of registers).

So gdb on power relies only on those few hw-breakpoints? With x86 you
can perfectly run gdb (with soft BPs) in parallel with the gdbstub
(currently based on hw-BPs, but the same would be true for soft-BPs
inserted by the gdbstub).

> 
> 
>> Why do we set breakpoints in the kernel? Why not simply catching all
>> debug traps, inserting software breakpoint ops into the guest code, and
>> handling all this stuff as normal debuggers do? And the hardware
>> breakpoints should just be pushed through the kernel interface like
>> ptrace does.
> 
> See above...But the cpu basically does the work for you. So you don't
> have to try and go through and first insert a trap into the code in
> memory. But then you have to remember the code that you replaced with
> the trap and execute it after you handle the trap. This can get a little
> hairy. 

I cannot imaging that this is so hairy. It is basically daily (x86-)
debugger business. Maybe we need to handle it differently if other
arches prefer their own way. But for x86 I don't see a need to restrict
our self to use hw-BPs _only_.

> 
> Currently I'm actually implementing breakpoint support now in Power. But
> you do have to create some mappings to handle traps and see if you put
> the trap there, and execute the code you replaced. Also what if the
> breakpoint is removed. Then you have to go back through and actually
> replace the trap code. Doesn't sound hard, but I'm not sure of all the
> pitfalls.

Again, this /should/ not be different from what gdb does to applications
or kgdb does to the kernel. (Looks like I need to get my feet wet soon. :) )

> 
>> The new KVM_DEBUG_GUEST interface I currently have in mind would look
>> like this:
>>
>> #define KVM_DBGGUEST_ENABLE		0x01
>> #define KVM_DBGGUEST_SINGLESTEP		0x02
>>
>> struct kvm_debug_guest {
>> 	__u32 control;
>> 	struct kvm_debug_guest_arch arch;
>> }
> 
> 
>> Setting KVM_DBGGUEST_ENABLE would forward all debug-related traps to
>> userspace first, which can then decide to handle or re-inject them.
>> KVM_DBGGUEST_SINGLESTEP would work as before. And the extension for x86
>> would look like this:
>>
>> struct kvm_debug_guest_arch {
>> 	__u32 use_hw_breakpoints;
>> 	__u64 debugreg[8];
>> }
>>
>> If use_hw_breakpoints is non-zero, KVM would completely overwrite the
>> guest's debug registers with the content of debugreg, giving full
>> control of this feature to the host-side debugger (faking the content of
>> debug registers, effectively disabling them for the guest - as we now do
>> all the time).
> 
> Hmmm...so today at least the gdbstub in qemu does not inject traps and
> track code that it trapped (I could be mistaken). This whould all need
> to be implemented as well.

gdbstub inserts "virtual" traps today, ie. a call from the translated
guest code to a helper which signals the breakpoint to the stub. And I
don't want to change this. I want to add the BP injection/removal to
qemu-kvm as it already takes over breakpoint (and soon also watchpoint)
maintenance from qemu.

However, would the proposed interface for KVM_DEBUG_GUEST (with an
appropriate kvm_debug_guest_arch for power) restrict your plans in any way?

Jan


[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 254 bytes --]

[-- Attachment #2: Type: text/plain, Size: 230 bytes --]

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft 
Defy all challenges. Microsoft(R) Visual Studio 2008. 
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

[-- Attachment #3: Type: text/plain, Size: 158 bytes --]

_______________________________________________
kvm-devel mailing list
kvm-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/kvm-devel

Re: [RFC] Reworking KVM_DEBUG_GUEST

[Jerone Young]

On Wed, 2008-05-14 at 17:28 +0200, Jan Kiszka wrote:
> Jerone Young wrote:
> > On Mon, 2008-05-12 at 13:34 +0200, Jan Kiszka wrote:
> >> Hi,
> >>
> >> before going wild with my idea, I would like to collect some comments on
> >> this approach:
> >>
> >> While doing first kernel debugging with my debug register patches for
> >> kvm, I quickly ran into the 4-breakpoints-only limitation that comes
> >> from the fact that we blindly map software to hardware breakpoints.
> >> Unhandy, simply suboptimal. Also, having 4 breakpoint slots hard-coded
> >> in the generic interface is not fair to arch that may support more.
> >> Moreover, we do not support watchpoints although this would easily be
> >> feasible. But if we supported watchpoints (via debug registers on x86),
> >> we would need the break out of the 4 slots limitations even earlier. In
> >> short, I came to the conclusion that a rewrite of the KVM_DEBUG_GUEST
> >> interface is required.
> > So embedded power is also limited to 4 hardware registers for break
> > points. But there are 2 sepreate registers fro watch points. The reason
> > to use the registers is the hardware does the work for you and (at least
> > on Power) will throw an exception or trap. Then you deal with it.
> > 
> > But you still face the fact that you can only have a small number of
> > breakpoints & watch points. Also you cannot use gdb in the guest at the
> > sametime while using the gdb stub on the guest itself (as there is only
> > one set of registers).
> 
> So gdb on power relies only on those few hw-breakpoints? With x86 you
> can perfectly run gdb (with soft BPs) in parallel with the gdbstub
> (currently based on hw-BPs, but the same would be true for soft-BPs
> inserted by the gdbstub).
> 
> > 
> > 
> >> Why do we set breakpoints in the kernel? Why not simply catching all
> >> debug traps, inserting software breakpoint ops into the guest code, and
> >> handling all this stuff as normal debuggers do? And the hardware
> >> breakpoints should just be pushed through the kernel interface like
> >> ptrace does.
> > 
> > See above...But the cpu basically does the work for you. So you don't
> > have to try and go through and first insert a trap into the code in
> > memory. But then you have to remember the code that you replaced with
> > the trap and execute it after you handle the trap. This can get a little
> > hairy. 
> 
> I cannot imaging that this is so hairy. It is basically daily (x86-)
> debugger business. Maybe we need to handle it differently if other
> arches prefer their own way. But for x86 I don't see a need to restrict
> our self to use hw-BPs _only_.
> 
> > 
> > Currently I'm actually implementing breakpoint support now in Power. But
> > you do have to create some mappings to handle traps and see if you put
> > the trap there, and execute the code you replaced. Also what if the
> > breakpoint is removed. Then you have to go back through and actually
> > replace the trap code. Doesn't sound hard, but I'm not sure of all the
> > pitfalls.
> 
> Again, this /should/ not be different from what gdb does to applications
> or kgdb does to the kernel. (Looks like I need to get my feet wet soon. :) )
> 
> > 
> >> The new KVM_DEBUG_GUEST interface I currently have in mind would look
> >> like this:
> >>
> >> #define KVM_DBGGUEST_ENABLE		0x01
> >> #define KVM_DBGGUEST_SINGLESTEP		0x02
> >>
> >> struct kvm_debug_guest {
> >> 	__u32 control;
> >> 	struct kvm_debug_guest_arch arch;
> >> }
> > 
> > 
> >> Setting KVM_DBGGUEST_ENABLE would forward all debug-related traps to
> >> userspace first, which can then decide to handle or re-inject them.
> >> KVM_DBGGUEST_SINGLESTEP would work as before. And the extension for x86
> >> would look like this:
> >>
> >> struct kvm_debug_guest_arch {
> >> 	__u32 use_hw_breakpoints;
> >> 	__u64 debugreg[8];
> >> }
> >>
> >> If use_hw_breakpoints is non-zero, KVM would completely overwrite the
> >> guest's debug registers with the content of debugreg, giving full
> >> control of this feature to the host-side debugger (faking the content of
> >> debug registers, effectively disabling them for the guest - as we now do
> >> all the time).
> > 
> > Hmmm...so today at least the gdbstub in qemu does not inject traps and
> > track code that it trapped (I could be mistaken). This whould all need
> > to be implemented as well.
> 
> gdbstub inserts "virtual" traps today, ie. a call from the translated
> guest code to a helper which signals the breakpoint to the stub. And I
> don't want to change this. I want to add the BP injection/removal to
> qemu-kvm as it already takes over breakpoint (and soon also watchpoint)
> maintenance from qemu.
> 
> However, would the proposed interface for KVM_DEBUG_GUEST (with an
> appropriate kvm_debug_guest_arch for power) restrict your plans in any way?

I think you should go ahead and go for it. I will be required to make
changes around it for the use of hardware breakpoints if it goes in. But
honestly it would be better to only use software breakpoints if they
work, as opposed to hardware breakpoints .. due to the limits. 

> 
> Jan
> 


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft 
Defy all challenges. Microsoft(R) Visual Studio 2008. 
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

Re: [RFC] Reworking KVM_DEBUG_GUEST

[Hollis Blanchard]

On Wednesday 14 May 2008 10:28:51 Jan Kiszka wrote:
> So gdb on power relies only on those few hw-breakpoints? With x86 you
> can perfectly run gdb (with soft BPs) in parallel with the gdbstub
> (currently based on hw-BPs, but the same would be true for soft-BPs
> inserted by the gdbstub).

GDB on Power inserts trap instructions, i.e. standard "soft" breakpoints. It 
does not rely on the hardware breakpoints.

It gets a little more complicated when you involve a GDB stub. IIRC, GDB will 
ask the stub to set the breakpoint, and if the stub doesn't support it, GDB 
will fall back to overwriting the instructions in memory. Does the Qemu GDB 
stub advertise breakpoint support?

If not, the only support needed in KVM would be to send all debug interrupts 
to qemu, and allow qemu to send them back down for in-guest breakpoints.

-- 
Hollis Blanchard
IBM Linux Technology Center

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft 
Defy all challenges. Microsoft(R) Visual Studio 2008. 
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

Re: [RFC] Reworking KVM_DEBUG_GUEST

[Jan Kiszka]

[-- Attachment #1.1: Type: text/plain, Size: 1778 bytes --]

Hollis Blanchard wrote:
> On Wednesday 14 May 2008 10:28:51 Jan Kiszka wrote:
>> So gdb on power relies only on those few hw-breakpoints? With x86 you
>> can perfectly run gdb (with soft BPs) in parallel with the gdbstub
>> (currently based on hw-BPs, but the same would be true for soft-BPs
>> inserted by the gdbstub).
> 
> GDB on Power inserts trap instructions, i.e. standard "soft" breakpoints. It 
> does not rely on the hardware breakpoints.
> 
> It gets a little more complicated when you involve a GDB stub. IIRC, GDB will 
> ask the stub to set the breakpoint, and if the stub doesn't support it, GDB 
> will fall back to overwriting the instructions in memory. Does the Qemu GDB 
> stub advertise breakpoint support?

Yes, QEMU reacts on both Z0 (soft-BP) and Z1 (hard-BP). That's something
even gdbserver does not do! It just handles watchpoints (Z2..4).

> 
> If not, the only support needed in KVM would be to send all debug interrupts 
> to qemu, and allow qemu to send them back down for in-guest breakpoints.
> 

Simply returning "unsupported" on Z0 is not yet enough for x86, KVM's
kernel side should not yet inform QEMU about soft-BP exceptions. But in
theory, this should be easily fixable (or is already the case for other
archs). And it would safe us from keeping track of N software
breakpoints, where N could even become larger than 32, the current
hardcoded limit for plain QEMU. :)

Meanwhile I realized that the proposed KVM_DEBUG_GUEST API is
insufficient: We need a return channel for the debug register state
(specifically to figure out details about hit watchpoints). I'm now
favoring KVM_SET_DEBUG and KVM_GET_DEBUG as two new IOCTLs, enabling us
to write _and_ read-back the suggested data structure.

Jan


[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 254 bytes --]

[-- Attachment #2: Type: text/plain, Size: 230 bytes --]

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft 
Defy all challenges. Microsoft(R) Visual Studio 2008. 
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

[-- Attachment #3: Type: text/plain, Size: 158 bytes --]

_______________________________________________
kvm-devel mailing list
kvm-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/kvm-devel

Re: [RFC] Reworking KVM_DEBUG_GUEST

[Hollis Blanchard]

On Wednesday 14 May 2008 14:10:06 Jan Kiszka wrote:
> Hollis Blanchard wrote:
> > On Wednesday 14 May 2008 10:28:51 Jan Kiszka wrote:
> >> So gdb on power relies only on those few hw-breakpoints? With x86 you
> >> can perfectly run gdb (with soft BPs) in parallel with the gdbstub
> >> (currently based on hw-BPs, but the same would be true for soft-BPs
> >> inserted by the gdbstub).
> > 
> > GDB on Power inserts trap instructions, i.e. standard "soft" breakpoints. 
It 
> > does not rely on the hardware breakpoints.
> > 
> > It gets a little more complicated when you involve a GDB stub. IIRC, GDB 
will 
> > ask the stub to set the breakpoint, and if the stub doesn't support it, 
GDB 
> > will fall back to overwriting the instructions in memory. Does the Qemu 
GDB 
> > stub advertise breakpoint support?
> 
> Yes, QEMU reacts on both Z0 (soft-BP) and Z1 (hard-BP). That's something
> even gdbserver does not do! It just handles watchpoints (Z2..4).
> 
> > 
> > If not, the only support needed in KVM would be to send all debug 
interrupts 
> > to qemu, and allow qemu to send them back down for in-guest breakpoints.
> > 
> 
> Simply returning "unsupported" on Z0 is not yet enough for x86, KVM's
> kernel side should not yet inform QEMU about soft-BP exceptions. But in
> theory, this should be easily fixable (or is already the case for other
> archs). And it would safe us from keeping track of N software
> breakpoints, where N could even become larger than 32, the current
> hardcoded limit for plain QEMU. :)
> 
> Meanwhile I realized that the proposed KVM_DEBUG_GUEST API is
> insufficient: We need a return channel for the debug register state
> (specifically to figure out details about hit watchpoints). I'm now
> favoring KVM_SET_DEBUG and KVM_GET_DEBUG as two new IOCTLs, enabling us
> to write _and_ read-back the suggested data structure.

How about simply extending kvm_exit.debug to contain the virtual address of 
the breakpoint hit? In Qemu, when exit_reason == KVM_EXIT_DEBUG, it would 
just need to see if that address is for a breakpoint Qemu set or not. If so, 
it's happy. If not, (commence handwaving) tell KVM to forward the debug 
interrupt to the guest. This way, the list of breakpoints is maintained in 
userspace (in the qemu gdb stub), which is nice because it could be 
arbitrarily large.

Also, this is not specific to hardware debug registers: soft and hard 
breakpoint interrupts would follow the same path. There's still a question of 
whether the GDB stub should set the breakpoint itself (Z0/Z1) or force GDB to 
modify memory, but either way the KVM code is simple.

-- 
Hollis Blanchard
IBM Linux Technology Center

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft 
Defy all challenges. Microsoft(R) Visual Studio 2008. 
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

Re: [RFC] Reworking KVM_DEBUG_GUEST

[Jan Kiszka]

[-- Attachment #1.1: Type: text/plain, Size: 3294 bytes --]

Hollis Blanchard wrote:
> On Wednesday 14 May 2008 14:10:06 Jan Kiszka wrote:
>> Hollis Blanchard wrote:
>>> On Wednesday 14 May 2008 10:28:51 Jan Kiszka wrote:
>>>> So gdb on power relies only on those few hw-breakpoints? With x86 you
>>>> can perfectly run gdb (with soft BPs) in parallel with the gdbstub
>>>> (currently based on hw-BPs, but the same would be true for soft-BPs
>>>> inserted by the gdbstub).
>>> GDB on Power inserts trap instructions, i.e. standard "soft" breakpoints. 
> It 
>>> does not rely on the hardware breakpoints.
>>>
>>> It gets a little more complicated when you involve a GDB stub. IIRC, GDB 
> will 
>>> ask the stub to set the breakpoint, and if the stub doesn't support it, 
> GDB 
>>> will fall back to overwriting the instructions in memory. Does the Qemu 
> GDB 
>>> stub advertise breakpoint support?
>> Yes, QEMU reacts on both Z0 (soft-BP) and Z1 (hard-BP). That's something
>> even gdbserver does not do! It just handles watchpoints (Z2..4).
>>
>>> If not, the only support needed in KVM would be to send all debug 
> interrupts 
>>> to qemu, and allow qemu to send them back down for in-guest breakpoints.
>>>
>> Simply returning "unsupported" on Z0 is not yet enough for x86, KVM's
>> kernel side should not yet inform QEMU about soft-BP exceptions. But in
>> theory, this should be easily fixable (or is already the case for other
>> archs). And it would safe us from keeping track of N software
>> breakpoints, where N could even become larger than 32, the current
>> hardcoded limit for plain QEMU. :)
>>
>> Meanwhile I realized that the proposed KVM_DEBUG_GUEST API is
>> insufficient: We need a return channel for the debug register state
>> (specifically to figure out details about hit watchpoints). I'm now
>> favoring KVM_SET_DEBUG and KVM_GET_DEBUG as two new IOCTLs, enabling us
>> to write _and_ read-back the suggested data structure.
> 
> How about simply extending kvm_exit.debug to contain the virtual address of 
> the breakpoint hit?

Ah, there is an interface for such stuff already! And it can even take
quite some payload...

> In Qemu, when exit_reason == KVM_EXIT_DEBUG, it would 
> just need to see if that address is for a breakpoint Qemu set or not. If so, 
> it's happy. If not, (commence handwaving) tell KVM to forward the debug 
> interrupt to the guest. This way, the list of breakpoints is maintained in 
> userspace (in the qemu gdb stub), which is nice because it could be 
> arbitrarily large.

Yes, but I would rather pass the debug registers (more general: some
arch dependent state set) back in this slot. Those contain everything
the gdbstub needs to know to catch relevant hardware-BP/watchpoint
events (and report them to the gdb frontend).

> 
> Also, this is not specific to hardware debug registers: soft and hard 
> breakpoint interrupts would follow the same path. There's still a question of 
> whether the GDB stub should set the breakpoint itself (Z0/Z1) or force GDB to 
> modify memory, but either way the KVM code is simple.

Only rejecting Z0 will enable us to avoid any soft-BP tracking in
qemu-kvm, and that is definitely my plan. Z1 may become an option to add
later on and would be answered as "unsupported" for now.

Jan


[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 254 bytes --]

[-- Attachment #2: Type: text/plain, Size: 230 bytes --]

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft 
Defy all challenges. Microsoft(R) Visual Studio 2008. 
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

[-- Attachment #3: Type: text/plain, Size: 158 bytes --]

_______________________________________________
kvm-devel mailing list
kvm-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/kvm-devel

Re: [RFC] Reworking KVM_DEBUG_GUEST

[Hollis Blanchard]

On Wednesday 14 May 2008 14:49:02 Jan Kiszka wrote:
> > In Qemu, when exit_reason == KVM_EXIT_DEBUG, it would 
> > just need to see if that address is for a breakpoint Qemu set or not. If 
so, 
> > it's happy. If not, (commence handwaving) tell KVM to forward the debug 
> > interrupt to the guest. This way, the list of breakpoints is maintained in 
> > userspace (in the qemu gdb stub), which is nice because it could be 
> > arbitrarily large.
> 
> Yes, but I would rather pass the debug registers (more general: some
> arch dependent state set) back in this slot. Those contain everything
> the gdbstub needs to know to catch relevant hardware-BP/watchpoint
> events (and report them to the gdb frontend).

But what would the stub *do* with the contents of the debug registers? The 
only reason they were set is on behalf of the stub in the first place. In 
fact, in the case of soft breakpoints, KVM doesn't even know where all the 
set breakpoints are. The only thing KVM needs to report is the address of the 
breakpoint that was just hit.

Sorry if this gets formatted badly:

gdb					qemu stub				KVM
break *0xf00
sends Z0 packet 0xf00
					0xf00 -> BP list
					ioctl(KVM_DEBUG, 0xf00)
continue
					ioctl(KVM_RUN)
											running...
											breakpoint hit
											exit_reason = KVM_EXIT_DEBUG
											kvm_run.debug.address = current PC value
					search BP list for address
bp hit			<---	present        not present --->	send debug interrupt to guest

Notes:
- KVM_DEBUG in this case will set a hardware breakpoint. The alternative is to 
write an int3 into guest memory.
- The stub doesn't care how the hardware registers were configured. All it 
needs to know is a) that a breakpoint was hit, and b) at what address.

Does this make sense?

-- 
Hollis Blanchard
IBM Linux Technology Center

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft 
Defy all challenges. Microsoft(R) Visual Studio 2008. 
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

Re: [kvm-ppc-devel] [RFC] Reworking KVM_DEBUG_GUEST

[Hollis Blanchard]

On Wednesday 14 May 2008 16:06:00 Hollis Blanchard wrote:
> In 
> fact, in the case of soft breakpoints, KVM doesn't even know where all the 
> set breakpoints are.

Side note: I'm retract this sentence: I wrote it before I sketched out the 
pseudocode, and forgot to remove it. :)

-- 
Hollis Blanchard
IBM Linux Technology Center

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft 
Defy all challenges. Microsoft(R) Visual Studio 2008. 
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

Re: [kvm-ppc-devel] [RFC] Reworking KVM_DEBUG_GUEST

[Hollis Blanchard]

On Wednesday 14 May 2008 16:11:39 Hollis Blanchard wrote:
> On Wednesday 14 May 2008 16:06:00 Hollis Blanchard wrote:
> > In 
> > fact, in the case of soft breakpoints, KVM doesn't even know where all the 
> > set breakpoints are.
> 
> Side note: I'm retract this sentence: I wrote it before I sketched out the 
> pseudocode, and forgot to remove it. :)

Er no, let me try that again:

In my proposed design, the stub needs to know where all breakpoints are, HW or 
SW. (That means it must implement Z0/Z1.)

However, KVM itself doesn't need to know any of that: all breakpoints are 
referred to the stub for handling, and the stub will notify KVM if further 
action is needed in-kernel.

-- 
Hollis Blanchard
IBM Linux Technology Center

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft 
Defy all challenges. Microsoft(R) Visual Studio 2008. 
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

Re: [RFC] Reworking KVM_DEBUG_GUEST

[Avi Kivity]

Jan Kiszka wrote:
> Hi,
>
> before going wild with my idea, I would like to collect some comments on
> this approach:
>
> While doing first kernel debugging with my debug register patches for
> kvm, I quickly ran into the 4-breakpoints-only limitation that comes
> from the fact that we blindly map software to hardware breakpoints.
> Unhandy, simply suboptimal. Also, having 4 breakpoint slots hard-coded
> in the generic interface is not fair to arch that may support more.
> Moreover, we do not support watchpoints although this would easily be
> feasible. But if we supported watchpoints (via debug registers on x86),
> we would need the break out of the 4 slots limitations even earlier. In
> short, I came to the conclusion that a rewrite of the KVM_DEBUG_GUEST
> interface is required.
>   

The current interface is limited, yes.


> Why do we set breakpoints in the kernel? Why not simply catching all
> debug traps, inserting software breakpoint ops into the guest code, and
> handling all this stuff as normal debuggers do? And the hardware
> breakpoints should just be pushed through the kernel interface like
> ptrace does.
>   

The problem is that the breakpoints are visible to the guest.  If the 
guest swaps a page, the breakpoint will be swapped with it.  If it 
reallocates a page to a different use it will overwrite the breakpoint.  
It's very brittle.

For Linux kernel debugging these issues don't show up in practice, but 
other kernels are able to swap their own memory.

> The new KVM_DEBUG_GUEST interface I currently have in mind would look
> like this:
>
> #define KVM_DBGGUEST_ENABLE		0x01
> #define KVM_DBGGUEST_SINGLESTEP		0x02
>
> struct kvm_debug_guest {
> 	__u32 control;
>   

[pad]

> 	struct kvm_debug_guest_arch arch;
> }
>   

The guest debug inteface can probablty be 100% arch specific.

> Setting KVM_DBGGUEST_ENABLE would forward all debug-related traps to
> userspace first, which can then decide to handle or re-inject them.
> KVM_DBGGUEST_SINGLESTEP would work as before. And the extension for x86
> would look like this:
>
> struct kvm_debug_guest_arch {
> 	__u32 use_hw_breakpoints;
>   

[pad]

> 	__u64 debugreg[8];
> }
>
> If use_hw_breakpoints is non-zero, KVM would completely overwrite the
> guest's debug registers with the content of debugreg, giving full
> control of this feature to the host-side debugger (faking the content of
> debug registers, effectively disabling them for the guest - as we now do
> all the time).
>   

There's much more that can be done. Branch stepping, last branch 
recording, etc.

> Questions:
> - Does anyone see traps and pitfalls in this approach?
>   

It seems workable, modulo the non-transparency of the debugger.

> - May I replace the existing interface with this one, or am I overseeing
>   some use case that already worked with the current code so that ABI
>   compatibility is required (most debug stuff should have been simply
>   broken so far, also due to bugs in userland)?
>   

This will break compilation of older userspace, so a new interface is 
preferred, complete with KVM_CAP_...

-- 
Do not meddle in the internals of kernels, for they are subtle and quick to panic.


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft 
Defy all challenges. Microsoft(R) Visual Studio 2008. 
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/