Re: [PATCH 0/3] ring-buffer: less locking and only disable preemption

Mathieu Desnoyers
2008-10-06T13:53:35Z

* Ingo Molnar (mingo@elte.hu) wrote:
>
> * Mathieu Desnoyers <compudj@krystal.dyndns.org> wrote:
>
> > explains that code modification on x86 SMP systems is not only a
> > matter of atomicity, but also a matter of not changing the code
> > underneath a running CPU which is making assumptions that it won't
> > change underneath without issuing a synchronizing instruction before
> > the new code is used by the CPU. The scheme you propose here takes
> > care of atomicity, but does not take care of the synchronization
> > problem. A sync_core() would probably be required when such
> > modification is detected.
>
> that's wrong, my scheme protects against these cases: before _any_ code
> is modified we set the redo_pending atomic flag, and make sure that
> previous NMI handlers have stopped executing. (easy enough)
>

Hi Ingo,

Hrm, how will this take care of the following race ?

CPU A CPU B
- NMI fires
- NMI handler checks for
redo_pending flag, == 0
- NMI handler runs code - set redo_pending
about to be modified
- NMI fires
- NMI handler checks redo_pending,
== 1, executes modify_code_redo()
-- race : NMI on A executes code modified by B --
- NMI handler finished running
code about to be modified

Mathieu

> then the atomic update of redo_pending should be a sufficient barrier
> for another CPU to notice the pending transaction.
>
> Note that the cross-CPU modification can still be 'half done' when the
> NMI hits, that's why we execute modify_code_redo() to 'redo' the full
> modification before executing further NMI code. That is executed _on the
> CPU_ that triggers an NMI, and the CPU itself is self-consistent.
>
> ( The modify_code_redo() will have to do a sync_cores() of course, like
> all self-modifying code, to flush speculative execution. )
>
> > Also, speaking of plain atomicity, you scheme does not seem to protect
> > against NMIs running on a different CPU, because the non-atomic change
> > could race with such NMI.
>
> That's wrong too. Another CPU will notice that redo_pending is set and
> will execute modify_code_redo() from its NMI handler _before_ calling
> all the notifiers and other 'wide' code paths.
>
> the only item that needs to be marked 'notrace' is only the highlevel
> do_nmi() handler itself. (as that executes before we have a chance to
> execute modify_code_redo())
>
> So we trade a large, fragile, and unmapped set of NMI-implicated
> codepaths for a very tight and well controlled an easy to maintain
> codepath.
>
> Ingo
>

--
Mathieu Desnoyers
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F BA06 3F25 A8FE 3BAE 9A68