[RFC PATCH] x86: Barf when faults happen in NMI

[RFC PATCH] x86: Barf when faults happen in NMI

[Frederic Weisbecker]

In x86, faults exit by executing the iret instruction, which then
reenables NMIs if we faulted in NMI context. Then if a fault
happens in NMI, another NMI can nest after the fault exits.

But we don't yet support nested NMIs because we have only one NMI
stack. To prevent that, trigger a bug when a fault happens in NMI
context.

Signed-off-by: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
---

I first thought about putting it in the vmalloc fault path only.
But then I saw more occasions for the kernel to fault (kmemcheck
or so), and so I thought it should be better put in the all in one
path. But I suspect you won't like that conditional in the big
x86 fault path.


 arch/x86/mm/fault.c |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)

diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c
index 4c4508e..80c997e 100644
--- a/arch/x86/mm/fault.c
+++ b/arch/x86/mm/fault.c
@@ -955,6 +955,8 @@ do_page_fault(struct pt_regs *regs, unsigned long error_code)
 	int write;
 	int fault;
 
+	BUG_ON(in_nmi());
+
 	tsk = current;
 	mm = tsk->mm;
 
-- 
1.6.2.3

Re: [RFC PATCH] x86: Barf when faults happen in NMI

[Mathieu Desnoyers]

* Frederic Weisbecker (fweisbec@gmail.com) wrote:
> In x86, faults exit by executing the iret instruction, which then
> reenables NMIs if we faulted in NMI context. Then if a fault
> happens in NMI, another NMI can nest after the fault exits.
> 
> But we don't yet support nested NMIs because we have only one NMI
> stack. To prevent that, trigger a bug when a fault happens in NMI
> context.
> 
> Signed-off-by: Frederic Weisbecker <fweisbec@gmail.com>
> Cc: Ingo Molnar <mingo@elte.hu>
> Cc: Thomas Gleixner <tglx@linutronix.de>
> Cc: H. Peter Anvin <hpa@zytor.com>
> Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
> ---
> 
> I first thought about putting it in the vmalloc fault path only.
> But then I saw more occasions for the kernel to fault (kmemcheck
> or so), and so I thought it should be better put in the all in one
> path. But I suspect you won't like that conditional in the big
> x86 fault path.
> 
> 
>  arch/x86/mm/fault.c |    2 ++
>  1 files changed, 2 insertions(+), 0 deletions(-)
> 
> diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c
> index 4c4508e..80c997e 100644
> --- a/arch/x86/mm/fault.c
> +++ b/arch/x86/mm/fault.c
> @@ -955,6 +955,8 @@ do_page_fault(struct pt_regs *regs, unsigned long error_code)
>  	int write;
>  	int fault;
>  
> +	BUG_ON(in_nmi());

Alternative idea: we could put the test at the beginning of the NMI handler, so
if a NMI handler nests over a processor already "in_nmi", then we bug. I agree
that this will trigger less easily than bugging in the fault handler (because we
need to hit the actual nmi-coming-in-because-iret-reenabled-them-too-early
scenario, but it's far less intrusive.

Thoughts ?

Mathieu

> +
>  	tsk = current;
>  	mm = tsk->mm;
>  
> -- 
> 1.6.2.3
> 

-- 
Mathieu Desnoyers
Operating System Efficiency R&D Consultant
EfficiOS Inc.
http://www.efficios.com

Re: [RFC PATCH] x86: Barf when faults happen in NMI

[Frederic Weisbecker]

On Mon, Sep 27, 2010 at 05:14:01PM -0400, Mathieu Desnoyers wrote:
> * Frederic Weisbecker (fweisbec@gmail.com) wrote:
> > In x86, faults exit by executing the iret instruction, which then
> > reenables NMIs if we faulted in NMI context. Then if a fault
> > happens in NMI, another NMI can nest after the fault exits.
> > 
> > But we don't yet support nested NMIs because we have only one NMI
> > stack. To prevent that, trigger a bug when a fault happens in NMI
> > context.
> > 
> > Signed-off-by: Frederic Weisbecker <fweisbec@gmail.com>
> > Cc: Ingo Molnar <mingo@elte.hu>
> > Cc: Thomas Gleixner <tglx@linutronix.de>
> > Cc: H. Peter Anvin <hpa@zytor.com>
> > Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
> > Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
> > ---
> > 
> > I first thought about putting it in the vmalloc fault path only.
> > But then I saw more occasions for the kernel to fault (kmemcheck
> > or so), and so I thought it should be better put in the all in one
> > path. But I suspect you won't like that conditional in the big
> > x86 fault path.
> > 
> > 
> >  arch/x86/mm/fault.c |    2 ++
> >  1 files changed, 2 insertions(+), 0 deletions(-)
> > 
> > diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c
> > index 4c4508e..80c997e 100644
> > --- a/arch/x86/mm/fault.c
> > +++ b/arch/x86/mm/fault.c
> > @@ -955,6 +955,8 @@ do_page_fault(struct pt_regs *regs, unsigned long error_code)
> >  	int write;
> >  	int fault;
> >  
> > +	BUG_ON(in_nmi());
> 
> Alternative idea: we could put the test at the beginning of the NMI handler, so
> if a NMI handler nests over a processor already "in_nmi", then we bug. I agree
> that this will trigger less easily than bugging in the fault handler (because we
> need to hit the actual nmi-coming-in-because-iret-reenabled-them-too-early
> scenario, but it's far less intrusive.
> 
> Thoughts ?


In fact we have that already in nmi_enter(). Now as you said that alone is probably
too light to find the reason of a nested NMI or to prevent it.

Re: [RFC PATCH] x86: Barf when faults happen in NMI

[Ingo Molnar]

* Frederic Weisbecker <fweisbec@gmail.com> wrote:

> In x86, faults exit by executing the iret instruction, which then 
> reenables NMIs if we faulted in NMI context. Then if a fault happens 
> in NMI, another NMI can nest after the fault exits.
> 
> But we don't yet support nested NMIs because we have only one NMI 
> stack. To prevent that, trigger a bug when a fault happens in NMI 
> context.
> 
> Signed-off-by: Frederic Weisbecker <fweisbec@gmail.com>
> Cc: Ingo Molnar <mingo@elte.hu>
> Cc: Thomas Gleixner <tglx@linutronix.de>
> Cc: H. Peter Anvin <hpa@zytor.com>
> Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
> ---
> 
> I first thought about putting it in the vmalloc fault path only. But 
> then I saw more occasions for the kernel to fault (kmemcheck or so), 
> and so I thought it should be better put in the all in one path. But I 
> suspect you won't like that conditional in the big x86 fault path.
> 
> 
>  arch/x86/mm/fault.c |    2 ++
>  1 files changed, 2 insertions(+), 0 deletions(-)
> 
> diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c
> index 4c4508e..80c997e 100644
> --- a/arch/x86/mm/fault.c
> +++ b/arch/x86/mm/fault.c
> @@ -955,6 +955,8 @@ do_page_fault(struct pt_regs *regs, unsigned long error_code)
>  	int write;
>  	int fault;
>  
> +	BUG_ON(in_nmi());
> +

Hm, this is a fastpath, so it would be nice to put this into the vmalloc 
and kmemcheck paths (even though it's less clean that way).

Also, a WARN_ON_ONCE() would probably help more in getting bug reports 
out of such systems, than a BUG_ON() crashed kernel.

Thanks,

	Ingo

Re: [RFC PATCH] x86: Barf when faults happen in NMI

[Frederic Weisbecker]

On Tue, Sep 28, 2010 at 12:36:18PM +0200, Ingo Molnar wrote:
> 
> * Frederic Weisbecker <fweisbec@gmail.com> wrote:
> 
> > In x86, faults exit by executing the iret instruction, which then 
> > reenables NMIs if we faulted in NMI context. Then if a fault happens 
> > in NMI, another NMI can nest after the fault exits.
> > 
> > But we don't yet support nested NMIs because we have only one NMI 
> > stack. To prevent that, trigger a bug when a fault happens in NMI 
> > context.
> > 
> > Signed-off-by: Frederic Weisbecker <fweisbec@gmail.com>
> > Cc: Ingo Molnar <mingo@elte.hu>
> > Cc: Thomas Gleixner <tglx@linutronix.de>
> > Cc: H. Peter Anvin <hpa@zytor.com>
> > Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
> > Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
> > ---
> > 
> > I first thought about putting it in the vmalloc fault path only. But 
> > then I saw more occasions for the kernel to fault (kmemcheck or so), 
> > and so I thought it should be better put in the all in one path. But I 
> > suspect you won't like that conditional in the big x86 fault path.
> > 
> > 
> >  arch/x86/mm/fault.c |    2 ++
> >  1 files changed, 2 insertions(+), 0 deletions(-)
> > 
> > diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c
> > index 4c4508e..80c997e 100644
> > --- a/arch/x86/mm/fault.c
> > +++ b/arch/x86/mm/fault.c
> > @@ -955,6 +955,8 @@ do_page_fault(struct pt_regs *regs, unsigned long error_code)
> >  	int write;
> >  	int fault;
> >  
> > +	BUG_ON(in_nmi());
> > +
> 
> Hm, this is a fastpath, so it would be nice to put this into the vmalloc 
> and kmemcheck paths (even though it's less clean that way).
> 
> Also, a WARN_ON_ONCE() would probably help more in getting bug reports 
> out of such systems, than a BUG_ON() crashed kernel.
> 
> Thanks,
> 
> 	Ingo


Ok, will resend then.

Thanks.