Re: [PATCH kvm-unit-tests 0/8]: x86: vmx: Test INIT processing in various CPU VMX states

[Sean Christopherson <sean.j.christopherson@intel.com>]

On Mon, Sep 30, 2019 at 06:29:52PM -0700, Nadav Amit wrote:
> > On Sep 30, 2019, at 6:23 PM, Liran Alon <liran.alon@oracle.com> wrote:
> > 
> > If INIT signal won’t be kept pending until exiting VMX operation, target
> > CPU which was sent with INIT signal when it was running guest, basically
> > lost INIT signal forever and just received an exit-reason it cannot do much
> > with.  That’s why I thought Intel designed this mechanism like I specified
> > above.
> 
> Well, the host can always issue an INIT using an IPI.

And conversely, if the INIT persisted then the host would be forced to do
VMXOFF, otherwise it effectively couldn't do VM-Enter on that logical CPU.

> > I also remembered to verify this behaviour against some discussions made online:
> > 1) https://software.intel.com/en-us/forums/virtualization-software-development/topic/355484
> > * "When the 16-bit guest issues an INIT IPI to itself using the APIC, I run into an infinite VMExit situation that my hypervisor cannot seem to recover from.”
> > * "In response to the VMExit with a reason of 3 (which is expected), the hypervisor resets the 16-bit guest's registers, limits, access rights, etc. to simulate starting execution from a known initialization point.  However, it seems that as soon as the hypervisor resumes guest execution, the VMExit occurs again, repeatedly.”
> > 2) https://patchwork.kernel.org/patch/2244311/
> > "I actually find it very useful. On INIT vmexit hypervisor may call vmxoff and do proper reset."
> > 
> > Anyway, Sean, can you assist verifying inside Intel what should be the expected behaviour?
> 
> It might always be (yet) another kvm-unit-tests bug that is only apparent on
> bare-metal. But if Sean can confirm what the expected behavior is, it would
> save time.
> 
> I do not have an ITP, so debugging on bare-metal is not fun at all...

My understanding of the architecture is that the INIT should be consumed
on VM-Exit.  The only scenario where an event is not consumed/acknowledge
is when a vanilla interrupt occurs without VM_EXIT_ACK_INTR_ON_EXIT set,
in which case the VM-Exit is technically considered a "pending" interrupt.
For all other cases (NMI, SMI, INIT, and INTR w/ ACK-ON-EXIT), the VM-Exit
is the end result of delivering the event.

INITs are indeed blocked and not dropped in VMX root mode.  But entering
non-root (guest) mode should unblock INITs and cause a VM-Exit, and thus
clear the INIT that was pended while in VMX root mode.  This behavior does
not conflict with the whitepaper[*] referenced by link (2) above, and in
fact the whitepaper explicitly covers guest mode behavior in a footnote:

  When the processor is in VMX guest mode, delivery of INIT causes a
  normal VMEXIT, of course.

The INIT attack described uses "VMX mode" to refer to VMX root mode, and
other than the footnote, doesn't mention VMX guest mode.  My reading of it
is that they're showing a proof of concept of based on getting the OS into
VMX root mode but not actually running a guest, e.g. this can be done
in KVM by creating a VM (KVM_CREATE_VM) but not running it (KVM_RUN).

Anyways, I'll double check that the INIT should indeed be consumed as part
of the VM-Exit.

[*] https://invisiblethingslab.com/resources/2011/Software%20Attacks%20on%20Intel%20VT-d.pdf