Re: [PATCH V2] xen: vmx: Use an INT 2 call to process real NMI's instead of self_nmi() in VMEXIT handler

[Tim Deegan <tim@xen.org>]

At 08:07 +0000 on 16 Nov (1353053247), Jan Beulich wrote:
> I continue to not be in favor of special casing this in VMX code,
> considering that the problem is generic (i.e. similarly affects PV).
> I.e. either we handle the other case similarly (special code added
> also to the PV code path), or we deal with this in a single place,
> keeping the NMIs masked for an extended period of time.

Affects PV because PV might use SYSRET to return from the NMI handler?
Right.

> > All of this would be moot except for the risk that we might take an MCE
> > while in the NMI handler.  The IRET from the MCE handler re-enables NMIs
> > while we're still in the NMI handler, and a second NMI arriving could
> > break the NMI handler.  In the PV case, it will also clobber the NMI
> > handler's stack.
> 
> No - the entry code switches away from the dedicated stacks when
> the origin was in guest context (see handle_ist_exception in
> xen/arch/x86/x86_64/entry.S).

I see, thanks.  So it's only if we take the NMI while in the hypervisor
that we stay on the NMI stack and risk getting the stack clobbered.

> > We could potentially solve the problem by having the MCE handler check
> > whether it's returning to the NMI stack, and do a normal return in that
> > case.  It's a bit of extra code but only in the MCE handler, which is
> > not performance-critical. 
> 
> Yes, that could solve that nesting case (again not very difficult
> to implement).

How about we just have the MCE handler return without IRET in _all_
cases where it's returning to ring 0?  I think that entirely solves the
MCE-in-NMI problem, without all the extra mechanism meeded for the
linux-style solution.  (Unless we want to allow other traps in either
the NMI or MCE handlers).

[And it occurs to me that the linux-style solution is tricky because
detecting the case where you've taken an NMI and not yet set the
nmi-in-progress flag is hard in both SVM (in the NMI handler but on the
normal stack) and VMX (in the _vmexit_ handler and on the normal
stack).]

So I guess now I'm suggesting:
 - MCE never returns to Xen with IRET;
 - NMI handling in handle_vmexit() moves to beside the MCE handling;
 - Explicit IRET-to-self at the end of do_nmi() to unmask NMIs; and
 - no int $2. :)

How's that?  I feel sure I must have missed a case - itt sounds too
easy.

Tim.