State of debug register emulation

State of debug register emulation

[Jan Kiszka]

Hi,

looks like we are getting better and better here in hitting yet
unsupported corner-case features of KVM :). This time our guest fiddles
with hardware debugging registers, but quickly gets unhappy as they do
not yet have the expected effect.

Joerg, I found you SVM-related patch series in the archive which does
not seem to have raised much responses. Is this general direction OK?
Does it allow self-debugging of guests? But how are conflicts resolved
if both guest and host need the physical registers (host debugging the
guest which is debugging itself)?

I would try to dig into the VMX side if the general architecture is
-mostly- clear. [ Sorry, Joerg, someone put the latter type of HW on my
desk :->. Hope I can once check our stuff against SVM as well! ]

Thanks,
Jan

-- 
Siemens AG, Corporate Technology, CT SE 2
Corporate Competence Center Embedded Linux

-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone

Re: State of debug register emulation

[Joerg Roedel]

On Tue, Apr 29, 2008 at 03:07:25PM +0200, Jan Kiszka wrote:
> Hi,
> 
> looks like we are getting better and better here in hitting yet
> unsupported corner-case features of KVM :). This time our guest fiddles
> with hardware debugging registers, but quickly gets unhappy as they do
> not yet have the expected effect.

KVM is mostly tested with guests that run with paging. So a 16 bit
protected mode guest is not tested very well :)

> Joerg, I found you SVM-related patch series in the archive which does
> not seem to have raised much responses. Is this general direction OK?
> Does it allow self-debugging of guests? But how are conflicts resolved
> if both guest and host need the physical registers (host debugging the
> guest which is debugging itself)?

I sent a patchset in the past to enable guest debugging for SVM which
means debugging the guest from outside using gdb. But I was not able to
test these patches because the userspace side of guest debugging is
broken in the kvm-qemu.
Debugging in the guest should work without problems. The debug registers
are switched between guest and host if the guest uses them. So there
should be no problems when the guest and the host using the debug
registers.

> I would try to dig into the VMX side if the general architecture is
> -mostly- clear. [ Sorry, Joerg, someone put the latter type of HW on my
> desk :->. Hope I can once check our stuff against SVM as well! ]

With some debug output from SVM I can better help to  debug your
problems ;-)

Joerg

-- 
           |           AMD Saxony Limited Liability Company & Co. KG
 Operating |         Wilschdorfer Landstr. 101, 01109 Dresden, Germany
 System    |                  Register Court Dresden: HRA 4896
 Research  |              General Partner authorized to represent:
 Center    |             AMD Saxony LLC (Wilmington, Delaware, US)
           | General Manager of AMD Saxony LLC: Dr. Hans-R. Deppe, Thomas McCoy


-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone

Re: State of debug register emulation

[Jan Kiszka]

Joerg Roedel wrote:
> On Tue, Apr 29, 2008 at 03:07:25PM +0200, Jan Kiszka wrote:
>> Hi,
>>
>> looks like we are getting better and better here in hitting yet
>> unsupported corner-case features of KVM :). This time our guest fiddles
>> with hardware debugging registers, but quickly gets unhappy as they do
>> not yet have the expected effect.
> 
> KVM is mostly tested with guests that run with paging. So a 16 bit
> protected mode guest is not tested very well :)

Yes, we know (we also had a bit fun with stock QEMU in corner cases).
But that may change now... :)

> 
>> Joerg, I found you SVM-related patch series in the archive which does
>> not seem to have raised much responses. Is this general direction OK?
>> Does it allow self-debugging of guests? But how are conflicts resolved
>> if both guest and host need the physical registers (host debugging the
>> guest which is debugging itself)?
> 
> I sent a patchset in the past to enable guest debugging for SVM which
> means debugging the guest from outside using gdb. But I was not able to
> test these patches because the userspace side of guest debugging is
> broken in the kvm-qemu.
> Debugging in the guest should work without problems. The debug registers
> are switched between guest and host if the guest uses them. So there
> should be no problems when the guest and the host using the debug
> registers.

I'm currently digging my way through the current VMX code, but I cannot
confirm this. Not sure what SVM does, but as far as I understood the VMX
side, only DR7 is saved/restored in hardware. The rest is KVM's job.
Unfortunately the access to the real debug registers only happens "if
(vcpu->guest_debug.enabled)". And as all DR accesses of the guest are
trapped, but the desired transfers to/from guest registers are nops,
this cannot work yet, at least on VMX.

This still leaves me with the question how to handle the case when the
host sets and arms some debug registers to debug the guest and the
latter does the same to debug itself. Guest access will be trapped, OK,
but KVM will then have to decide which value should actually be
transfered into the registers. Hmm, does SVM virtualizes all debug
registers, leaving the real ones to the host?

> 
>> I would try to dig into the VMX side if the general architecture is
>> -mostly- clear. [ Sorry, Joerg, someone put the latter type of HW on my
>> desk :->. Hope I can once check our stuff against SVM as well! ]
> 
> With some debug output from SVM I can better help to  debug your
> problems ;-)

I'm sure :). But I guess this topic has a few common aspects to be
solved, too. So we may ideally end up with a single series of
debug-enabling patches for KVM (maybe even fixing userland - we are not
totally unfamiliar with the gdbstub here).

Jan

-- 
Siemens AG, Corporate Technology, CT SE 2
Corporate Competence Center Embedded Linux

-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone

Re: State of debug register emulation

[Avi Kivity]

Jan Kiszka wrote:
> This still leaves me with the question how to handle the case when the
> host sets and arms some debug registers to debug the guest and the
> latter does the same to debug itself. Guest access will be trapped, OK,
> but KVM will then have to decide which value should actually be
> transfered into the registers. Hmm, does SVM virtualizes all debug
> registers, leaving the real ones to the host?
>   

There's no way this can work.  There are still only four debug 
registers, and the guest and host together can ask for eight different 
addresses.  It is theoretically doable by hiding all mappings to pages 
that are debug targets, but it would probably double the kvm code size.

A good short-term compomise is to abort if the guest starts using 
enabling a debug address register.  A better solution might be to place 
host debug addresses into unused guest debug registers, so that as long 
as nr_guest_debug + nr_host_debug <= 4, we can still proceed.


-- 
Any sufficiently difficult bug is indistinguishable from a feature.


-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone

Re: State of debug register emulation

[Jan Kiszka]

Avi Kivity wrote:
> Jan Kiszka wrote:
>> This still leaves me with the question how to handle the case when the
>> host sets and arms some debug registers to debug the guest and the
>> latter does the same to debug itself. Guest access will be trapped, OK,
>> but KVM will then have to decide which value should actually be
>> transfered into the registers. Hmm, does SVM virtualizes all debug
>> registers, leaving the real ones to the host?
>>   
> 
> There's no way this can work.  There are still only four debug
> registers, and the guest and host together can ask for eight different
> addresses.  It is theoretically doable by hiding all mappings to pages
> that are debug targets, but it would probably double the kvm code size.
> 
> A good short-term compomise is to abort if the guest starts using
> enabling a debug address register.  A better solution might be to place
> host debug addresses into unused guest debug registers, so that as long
> as nr_guest_debug + nr_host_debug <= 4, we can still proceed.

I tried the latter, but we cannot cleanly share DR7 between both users.
Thus I'm now going for a prioritized approach: debug register will stop
to have any effect for the guest as soon as the host starts to use them.
That's far simpler the implement and also easier to understand for the user.

A bit work remains, though, to clean up and enhance the DRx support in
KVM. And to test the changes (will contact you, Joerg, regarding SVM
tests). Stay tuned.

Jan

-- 
Siemens AG, Corporate Technology, CT SE 2
Corporate Competence Center Embedded Linux

-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone

Re: State of debug register emulation

[Avi Kivity]

Jan Kiszka wrote:
> Avi Kivity wrote:
>   
>> Jan Kiszka wrote:
>>     
>>> This still leaves me with the question how to handle the case when the
>>> host sets and arms some debug registers to debug the guest and the
>>> latter does the same to debug itself. Guest access will be trapped, OK,
>>> but KVM will then have to decide which value should actually be
>>> transfered into the registers. Hmm, does SVM virtualizes all debug
>>> registers, leaving the real ones to the host?
>>>   
>>>       
>> There's no way this can work.  There are still only four debug
>> registers, and the guest and host together can ask for eight different
>> addresses.  It is theoretically doable by hiding all mappings to pages
>> that are debug targets, but it would probably double the kvm code size.
>>
>> A good short-term compomise is to abort if the guest starts using
>> enabling a debug address register.  A better solution might be to place
>> host debug addresses into unused guest debug registers, so that as long
>> as nr_guest_debug + nr_host_debug <= 4, we can still proceed.
>>     
>
> I tried the latter, but we cannot cleanly share DR7 between both users.
>   

I actually think we can, but...

> Thus I'm now going for a prioritized approach: debug register will stop
> to have any effect for the guest as soon as the host starts to use them.
> That's far simpler the implement and also easier to understand for the user.
>
>   

Agreed, having a simple model is preferred here, both from the user's 
point of view and from a code complexity point of view. If you're 
debugging a debugger use plain qemu.

-- 
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.


-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone