vmx real mode emulated mmio?

vmx real mode emulated mmio?

[Alex Williamson]

I don't really know what I'm doing messing around with realmode
exception handling, but are we missing something like this:

--- a/arch/x86/kvm/vmx.c
+++ b/arch/x86/kvm/vmx.c
@@ -4332,7 +4332,8 @@ static int handle_exception(struct kvm_vcpu *vcpu)
                if (vcpu->arch.halt_request) {
                        vcpu->arch.halt_request = 0;
                        return kvm_emulate_halt(vcpu);
-               }
+               } else if (vcpu->mmio_needed)
+                       return 0;
                return 1;
        }
 

The background is that I was playing around with when the MMIO regions
of PCI assigned devices get direct mapped through KVM and start using
memory slots and I found that the boot ROM on an e1000e card that I
often use for testing causes the VM to hang when not directly mapped
through a memory slot.  This happens with both pci-assign and vfio-pci.
Debugging further, I see the kernel instruction emulator decodes this
and sets all the necessary flags and fragments for userspace to complete
the access, but since it happened in real mode, vcpu_enter_guest always
returns 1 and therefore KVM_RUN never exits to userspace and KVM gets
stuck in an infinite loop re-handling this instruction.

The above is pretty trivial and allows the ROM to work again.  Is it
correct?  What other exceptions are we missing here that should allow an
exit?  This particular access generated a GP_VECTOR exception if we want
to key on that somewhere.  Thanks,

Alex

Re: vmx real mode emulated mmio?

[Gleb Natapov]

On Fri, Dec 14, 2012 at 04:42:36PM -0700, Alex Williamson wrote:
> 
> I don't really know what I'm doing messing around with realmode
> exception handling, but are we missing something like this:
> 
> --- a/arch/x86/kvm/vmx.c
> +++ b/arch/x86/kvm/vmx.c
> @@ -4332,7 +4332,8 @@ static int handle_exception(struct kvm_vcpu *vcpu)
>                 if (vcpu->arch.halt_request) {
>                         vcpu->arch.halt_request = 0;
>                         return kvm_emulate_halt(vcpu);
> -               }
> +               } else if (vcpu->mmio_needed)
> +                       return 0;
>                 return 1;
>         }
>  
> 
> The background is that I was playing around with when the MMIO regions
> of PCI assigned devices get direct mapped through KVM and start using
> memory slots and I found that the boot ROM on an e1000e card that I
> often use for testing causes the VM to hang when not directly mapped
> through a memory slot.  This happens with both pci-assign and vfio-pci.
> Debugging further, I see the kernel instruction emulator decodes this
> and sets all the necessary flags and fragments for userspace to complete
> the access, but since it happened in real mode, vcpu_enter_guest always
> returns 1 and therefore KVM_RUN never exits to userspace and KVM gets
> stuck in an infinite loop re-handling this instruction.
> 
> The above is pretty trivial and allows the ROM to work again.  Is it
> correct?  What other exceptions are we missing here that should allow an
> exit?  This particular access generated a GP_VECTOR exception if we want
> to key on that somewhere.  Thanks,
> 
As far as I see the patch is nop. If handle_rmode_exception() returns 1
mmio_needed should be 0. Also if handle_rmode_exception() makes IO
according to the code qemu should exit with "KVM: exception 13 exit
(error code 0x0)" or something. Can you provide a ftrace for the hang?

--
			Gleb.

Re: vmx real mode emulated mmio?

[Alex Williamson]

On Sat, 2012-12-15 at 10:07 +0200, Gleb Natapov wrote:
> On Fri, Dec 14, 2012 at 04:42:36PM -0700, Alex Williamson wrote:
> > 
> > I don't really know what I'm doing messing around with realmode
> > exception handling, but are we missing something like this:
> > 
> > --- a/arch/x86/kvm/vmx.c
> > +++ b/arch/x86/kvm/vmx.c
> > @@ -4332,7 +4332,8 @@ static int handle_exception(struct kvm_vcpu *vcpu)
> >                 if (vcpu->arch.halt_request) {
> >                         vcpu->arch.halt_request = 0;
> >                         return kvm_emulate_halt(vcpu);
> > -               }
> > +               } else if (vcpu->mmio_needed)
> > +                       return 0;
> >                 return 1;
> >         }
> >  
> > 
> > The background is that I was playing around with when the MMIO regions
> > of PCI assigned devices get direct mapped through KVM and start using
> > memory slots and I found that the boot ROM on an e1000e card that I
> > often use for testing causes the VM to hang when not directly mapped
> > through a memory slot.  This happens with both pci-assign and vfio-pci.
> > Debugging further, I see the kernel instruction emulator decodes this
> > and sets all the necessary flags and fragments for userspace to complete
> > the access, but since it happened in real mode, vcpu_enter_guest always
> > returns 1 and therefore KVM_RUN never exits to userspace and KVM gets
> > stuck in an infinite loop re-handling this instruction.
> > 
> > The above is pretty trivial and allows the ROM to work again.  Is it
> > correct?  What other exceptions are we missing here that should allow an
> > exit?  This particular access generated a GP_VECTOR exception if we want
> > to key on that somewhere.  Thanks,
> > 
> As far as I see the patch is nop. If handle_rmode_exception() returns 1
> mmio_needed should be 0. Also if handle_rmode_exception() makes IO
> according to the code qemu should exit with "KVM: exception 13 exit
> (error code 0x0)" or something. Can you provide a ftrace for the hang?

The real mode vector is GP_VECTOR, so handle_rmode_exception returns 1
even though emulate_instruction returns EMULATE_DO_MMIO.  Maybe the
point at which handle_rmode_exception gets EMULATE_DO_MMIO from
emulate_instruction is the point at which we should return 0.  I'll run
an ftrace if it's still not clear how we're getting here.  Thanks,

Alex

Re: vmx real mode emulated mmio?

[Gleb Natapov]

On Sat, Dec 15, 2012 at 08:54:22AM -0700, Alex Williamson wrote:
> On Sat, 2012-12-15 at 10:07 +0200, Gleb Natapov wrote:
> > On Fri, Dec 14, 2012 at 04:42:36PM -0700, Alex Williamson wrote:
> > > 
> > > I don't really know what I'm doing messing around with realmode
> > > exception handling, but are we missing something like this:
> > > 
> > > --- a/arch/x86/kvm/vmx.c
> > > +++ b/arch/x86/kvm/vmx.c
> > > @@ -4332,7 +4332,8 @@ static int handle_exception(struct kvm_vcpu *vcpu)
> > >                 if (vcpu->arch.halt_request) {
> > >                         vcpu->arch.halt_request = 0;
> > >                         return kvm_emulate_halt(vcpu);
> > > -               }
> > > +               } else if (vcpu->mmio_needed)
> > > +                       return 0;
> > >                 return 1;
> > >         }
> > >  
> > > 
> > > The background is that I was playing around with when the MMIO regions
> > > of PCI assigned devices get direct mapped through KVM and start using
> > > memory slots and I found that the boot ROM on an e1000e card that I
> > > often use for testing causes the VM to hang when not directly mapped
> > > through a memory slot.  This happens with both pci-assign and vfio-pci.
> > > Debugging further, I see the kernel instruction emulator decodes this
> > > and sets all the necessary flags and fragments for userspace to complete
> > > the access, but since it happened in real mode, vcpu_enter_guest always
> > > returns 1 and therefore KVM_RUN never exits to userspace and KVM gets
> > > stuck in an infinite loop re-handling this instruction.
> > > 
> > > The above is pretty trivial and allows the ROM to work again.  Is it
> > > correct?  What other exceptions are we missing here that should allow an
> > > exit?  This particular access generated a GP_VECTOR exception if we want
> > > to key on that somewhere.  Thanks,
> > > 
> > As far as I see the patch is nop. If handle_rmode_exception() returns 1
> > mmio_needed should be 0. Also if handle_rmode_exception() makes IO
> > according to the code qemu should exit with "KVM: exception 13 exit
> > (error code 0x0)" or something. Can you provide a ftrace for the hang?
> 
> The real mode vector is GP_VECTOR, so handle_rmode_exception returns 1
> even though emulate_instruction returns EMULATE_DO_MMIO.  Maybe the
Then #GP should be injected into a guest, which is incorrect of course.

> point at which handle_rmode_exception gets EMULATE_DO_MMIO from
> emulate_instruction is the point at which we should return 0.
Yes, this is likely what should be done here.

>                                                               I'll run
> an ftrace if it's still not clear how we're getting here.  Thanks,
> 
Yes please, I wonder what instruction is this. The code in
handle_exception() looks incorrect, but it is not clear to me me how to
fix it yet.

--
			Gleb.