Re: [PATCH 0/3] ring-buffer: less locking and only disable preemption

[Mathieu Desnoyers <compudj@krystal.dyndns.org>]

* Ingo Molnar (mingo@elte.hu) wrote:
> 
> * Mathieu Desnoyers <compudj@krystal.dyndns.org> wrote:
> 
> > explains that code modification on x86 SMP systems is not only a 
> > matter of atomicity, but also a matter of not changing the code 
> > underneath a running CPU which is making assumptions that it won't 
> > change underneath without issuing a synchronizing instruction before 
> > the new code is used by the CPU. The scheme you propose here takes 
> > care of atomicity, but does not take care of the synchronization 
> > problem. A sync_core() would probably be required when such 
> > modification is detected.
> 
> that's wrong, my scheme protects against these cases: before _any_ code 
> is modified we set the redo_pending atomic flag, and make sure that 
> previous NMI handlers have stopped executing. (easy enough)
> 

Hi Ingo,

Hrm, how will this take care of the following race ?

CPU A                                CPU B
- NMI fires
- NMI handler checks for
    redo_pending flag, == 0
- NMI handler runs code              - set redo_pending
    about to be modified
                                     - NMI fires
                                     - NMI handler checks redo_pending,
                                       == 1, executes modify_code_redo()
           -- race : NMI on A executes code modified by B --
- NMI handler finished running
    code about to be modified

Mathieu

> then the atomic update of redo_pending should be a sufficient barrier 
> for another CPU to notice the pending transaction.
> 
> Note that the cross-CPU modification can still be 'half done' when the 
> NMI hits, that's why we execute modify_code_redo() to 'redo' the full 
> modification before executing further NMI code. That is executed _on the 
> CPU_ that triggers an NMI, and the CPU itself is self-consistent.
> 
> ( The modify_code_redo() will have to do a sync_cores() of course, like 
>   all self-modifying code, to flush speculative execution. )
> 
> > Also, speaking of plain atomicity, you scheme does not seem to protect 
> > against NMIs running on a different CPU, because the non-atomic change 
> > could race with such NMI.
> 
> That's wrong too. Another CPU will notice that redo_pending is set and 
> will execute modify_code_redo() from its NMI handler _before_ calling 
> all the notifiers and other 'wide' code paths.
> 
> the only item that needs to be marked 'notrace' is only the highlevel 
> do_nmi() handler itself. (as that executes before we have a chance to 
> execute modify_code_redo())
> 
> So we trade a large, fragile, and unmapped set of NMI-implicated 
> codepaths for a very tight and well controlled an easy to maintain 
> codepath.
> 
> 	Ingo
> 

-- 
Mathieu Desnoyers
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68