Re: [PATCH 0/3] ring-buffer: less locking and only disable preemption

[Ingo Molnar <mingo@elte.hu>]

* Ingo Molnar <mingo@elte.hu> wrote:

> * Steven Rostedt <rostedt@goodmis.org> wrote:
> 
> > The dynamic function tracer is another issue. The problem with NMIs 
> > has nothing to do with locking, or corrupting the buffers. It has to 
> > do with the dynamic code modification.  Whenever we modify code, we 
> > must guarantee that it will not be executed on another CPU.
> > 
> > Kstop_machine serves this purpose rather well. We can modify code 
> > without worrying it will be executed on another CPU, except for NMIs. 
> > The problem now comes where an NMI can come in and execute the code 
> > being modified. That's why I put in all the notrace, lines. But it 
> > gets difficult because of nmi_notifier can call all over the kernel.  
> > Perhaps, we can simply disable the nmi-notifier when we are doing the 
> > kstop_machine call?
> 
> that would definitely be one way to reduce the cross section, but not 
> enough i'm afraid. For example in the nmi_watchdog=2 case we call into 
> various lapic functions and paravirt lapic handlers which makes it all 
> spread to 3-4 paravirtualization flavors ...
> 
> sched_clock()'s notrace aspects were pretty manageable, but this in 
> its current form is not.

there's a relatively simple method that would solve all these 
impact-size problems.

We cannot stop NMIs (and MCEs, etc.), but we can make kernel code 
modifications atomic, by adding the following thin layer ontop of it:

   #define MAX_CODE_SIZE 10

   int redo_len;
   u8 *redo_vaddr;

   u8 redo_buffer[MAX_CODE_SIZE];

   atomic_t __read_mostly redo_pending;

and use it in do_nmi():

   if (unlikely(atomic_read(&redo_pending)))
	modify_code_redo();

i.e. when we modify code, we first fill in the redo_buffer[], redo_vaddr 
and redo_len[], then we set redo_pending flag. Then we modify the kernel 
code, and clear the redo_pending flag.

If an NMI (or MCE) handler intervenes, it will notice the pending 
'transaction' and will copy redo_buffer[] to the (redo_vaddr,len) 
location and will continue.

So as far as non-maskable contexts are concerned, kernel code patching 
becomes an atomic operation. do_nmi() has to be marked notrace but 
that's all and easy to maintain.

Hm?

	Ingo