NMI delivery not correctly working

NMI delivery not correctly working

[Christoph Egger]

Hi!

The NMI delivery does not work correctly.

In the section process_nmi: in
xen/arch/x86/x86_(32|64)/entry.S you set a bit in VCPU_nmi_masked,
deliver the NMI but not clear VCPU_nmi_masked.

I know, there is a line "current->nmi_masked = 0;" in do_iret() in
xen/arch/x86/x86_(32|64)/traps.c, but this is actually never called
after NMI delivery.

For short: The *first* NMI is actually delivered, but all following NMIs
are NEVER delivered!!

I don't know if this change is correct for 64 bit:

@@ -218,6 +220,7 @@ process_nmi:
         movq %rax,TRAPBOUNCE_eip(%rdx)
         movb $TBF_INTERRUPT,TRAPBOUNCE_flags(%rdx)
         call create_bounce_frame
+        movb  $0, VCPU_nmi_masked(%rbx)
         jmp  test_all_events


and for 32 bit:

@@ -261,6 +262,7 @@ process_nmi:
         movw $FLAT_KERNEL_CS,TRAPBOUNCE_cs(%edx)
         movb $TBF_INTERRUPT,TRAPBOUNCE_flags(%edx)
         call create_bounce_frame
+        movb $0,VCPU_nmi_masked(%ebx)
         jmp  test_all_events

but NMI delivery seems to work with these changes.


Christoph



-- 
AMD Saxony, Dresden, Germany
Operating System Research Center

Legal Information:
AMD Saxony Limited Liability Company & Co. KG
Sitz (Geschäftsanschrift):
   Wilschdorfer Landstr. 101, 01109 Dresden, Deutschland
Registergericht Dresden: HRA 4896
vertretungsberechtigter Komplementär:
   AMD Saxony LLC (Sitz Wilmington, Delaware, USA)
Geschäftsführer der AMD Saxony LLC:
   Dr. Hans-R. Deppe, Thomas McCoy

Re: NMI delivery not correctly working

[Keir Fraser]

On 24/7/07 16:47, "Christoph Egger" <Christoph.Egger@amd.com> wrote:

> I know, there is a line "current->nmi_masked = 0;" in do_iret() in
> xen/arch/x86/x86_(32|64)/traps.c, but this is actually never called
> after NMI delivery.

Guests *must* use the iret hypercall when returning from their NMI handler.
This is giving a virtualised equivalent of the 'NMI-blocking' latch
implemented in x86 processors, which is reset by the IRET instruction.

 -- Keir

RE: NMI delivery not correctly working

[Kaushik Barde]

Keir:

Processor disables calls to subsequent NMIs till it receives next IRET
instruction (via masking of maskable interrupts).
I guess, what I am not clear about is how a fully-virtualized guest
whould issue a IRET hypercall?

-Kaushik

-----Original Message-----
From: xen-devel-bounces@lists.xensource.com
[mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Keir Fraser
Sent: Tuesday, July 24, 2007 10:08 AM
To: Christoph Egger; xen-devel@lists.xensource.com
Subject: Re: [Xen-devel] NMI delivery not correctly working

On 24/7/07 16:47, "Christoph Egger" <Christoph.Egger@amd.com> wrote:

> I know, there is a line "current->nmi_masked = 0;" in do_iret() in
> xen/arch/x86/x86_(32|64)/traps.c, but this is actually never called
> after NMI delivery.

Guests *must* use the iret hypercall when returning from their NMI
handler.
This is giving a virtualised equivalent of the 'NMI-blocking' latch
implemented in x86 processors, which is reset by the IRET instruction.

 -- Keir


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: NMI delivery not correctly working

[Keir Fraser]

On 24/7/07 19:26, "Kaushik Barde" <Kaushik_Barde@Phoenix.com> wrote:

> Processor disables calls to subsequent NMIs till it receives next IRET
> instruction

Yes.

> (via masking of maskable interrupts).

No, since NMIs are, of course, not maskable by conventional means.

> I guess, what I am not clear about is how a fully-virtualized guest
> whould issue a IRET hypercall?

It's used just like any other hypercall. Guest jumps at the correct offset
into its hypercall transfer page.

 -- Keir

> -Kaushik
> 
> -----Original Message-----
> From: xen-devel-bounces@lists.xensource.com
> [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Keir Fraser
> Sent: Tuesday, July 24, 2007 10:08 AM
> To: Christoph Egger; xen-devel@lists.xensource.com
> Subject: Re: [Xen-devel] NMI delivery not correctly working
> 
> On 24/7/07 16:47, "Christoph Egger" <Christoph.Egger@amd.com> wrote:
> 
>> I know, there is a line "current->nmi_masked = 0;" in do_iret() in
>> xen/arch/x86/x86_(32|64)/traps.c, but this is actually never called
>> after NMI delivery.
> 
> Guests *must* use the iret hypercall when returning from their NMI
> handler.
> This is giving a virtualised equivalent of the 'NMI-blocking' latch
> implemented in x86 processors, which is reset by the IRET instruction.
> 
>  -- Keir
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

RE: NMI delivery not correctly working

[Kaushik Barde]

Thanks. 
 
>>
via masking of maskable interrupts
<<
What I'd meant there was all other interrupts are masked (by processor during NMI handling) except for NMI.
 
-Kaushik

________________________________

From: Keir Fraser [mailto:keir@xensource.com]
Sent: Tue 7/24/2007 11:22 PM
To: Kaushik Barde; Christoph Egger; xen-devel@lists.xensource.com
Subject: Re: [Xen-devel] NMI delivery not correctly working



On 24/7/07 19:26, "Kaushik Barde" <Kaushik_Barde@Phoenix.com> wrote:

> Processor disables calls to subsequent NMIs till it receives next IRET
> instruction

Yes.

> (via masking of maskable interrupts).

No, since NMIs are, of course, not maskable by conventional means.

> I guess, what I am not clear about is how a fully-virtualized guest
> whould issue a IRET hypercall?

It's used just like any other hypercall. Guest jumps at the correct offset
into its hypercall transfer page.

 -- Keir

> -Kaushik
>
> -----Original Message-----
> From: xen-devel-bounces@lists.xensource.com
> [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Keir Fraser
> Sent: Tuesday, July 24, 2007 10:08 AM
> To: Christoph Egger; xen-devel@lists.xensource.com
> Subject: Re: [Xen-devel] NMI delivery not correctly working
>
> On 24/7/07 16:47, "Christoph Egger" <Christoph.Egger@amd.com> wrote:
>
>> I know, there is a line "current->nmi_masked = 0;" in do_iret() in
>> xen/arch/x86/x86_(32|64)/traps.c, but this is actually never called
>> after NMI delivery.
>
> Guests *must* use the iret hypercall when returning from their NMI
> handler.
> This is giving a virtualised equivalent of the 'NMI-blocking' latch
> implemented in x86 processors, which is reset by the IRET instruction.
>
>  -- Keir
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

Re: NMI delivery not correctly working

[Christoph Egger]

On Tuesday 24 July 2007 19:07:53 Keir Fraser wrote:
> On 24/7/07 16:47, "Christoph Egger" <Christoph.Egger@amd.com> wrote:
> > I know, there is a line "current->nmi_masked = 0;" in do_iret() in
> > xen/arch/x86/x86_(32|64)/traps.c, but this is actually never called
> > after NMI delivery.
>
> Guests *must* use the iret hypercall when returning from their NMI handler.
> This is giving a virtualised equivalent of the 'NMI-blocking' latch
> implemented in x86 processors, which is reset by the IRET instruction.
>

Ah, I see. Tnx. It would be really nice to have
some documentation about hypercalls, though.

Christoph




-- 
AMD Saxony, Dresden, Germany
Operating System Research Center

Legal Information:
AMD Saxony Limited Liability Company & Co. KG
Sitz (Geschäftsanschrift):
   Wilschdorfer Landstr. 101, 01109 Dresden, Deutschland
Registergericht Dresden: HRA 4896
vertretungsberechtigter Komplementär:
   AMD Saxony LLC (Sitz Wilmington, Delaware, USA)
Geschäftsführer der AMD Saxony LLC:
   Dr. Hans-R. Deppe, Thomas McCoy

Re: NMI delivery not correctly working

[Keir Fraser]

On 25/7/07 07:43, "Kaushik Barde" <Kaushik_Barde@Phoenix.com> wrote:

> Thanks. 
>  
>>> 
> via masking of maskable interrupts
> <<
> What I'd meant there was all other interrupts are masked (by processor during
> NMI handling) except for NMI.

NMIs *are* masked by x86 CPU from when an NMI is delivered to the CPU until
after that CPU next executes an IRET. The CPU has a special NMI mask for
this purpose (not directly architecturally visible). It's unrelated to
EFLAGS.IF.

 -- Keir

Re: NMI delivery not correctly working

[Keir Fraser]

On 25/7/07 07:48, "Christoph Egger" <Christoph.Egger@amd.com> wrote:

>> Guests *must* use the iret hypercall when returning from their NMI handler.
>> This is giving a virtualised equivalent of the 'NMI-blocking' latch
>> implemented in x86 processors, which is reset by the IRET instruction.
>> 
> 
> Ah, I see. Tnx. It would be really nice to have
> some documentation about hypercalls, though.

Yes. :-)

Actually I try to put reasonable amounts of detail in the public header
files, and would like patches to beef up that self-documentation where it's
lacking.

Separate and more detailed docs would be great too, if someone has the time.

 -- Keir