Re: [PATCH 0/3] ring-buffer: less locking and only disable preemption

Ingo Molnar
2008-10-05T10:14:06Z

* Mathieu Desnoyers <compudj@krystal.dyndns.org> wrote:

> explains that code modification on x86 SMP systems is not only a
> matter of atomicity, but also a matter of not changing the code
> underneath a running CPU which is making assumptions that it won't
> change underneath without issuing a synchronizing instruction before
> the new code is used by the CPU. The scheme you propose here takes
> care of atomicity, but does not take care of the synchronization
> problem. A sync_core() would probably be required when such
> modification is detected.

that's wrong, my scheme protects against these cases: before _any_ code
is modified we set the redo_pending atomic flag, and make sure that
previous NMI handlers have stopped executing. (easy enough)

then the atomic update of redo_pending should be a sufficient barrier
for another CPU to notice the pending transaction.

Note that the cross-CPU modification can still be 'half done' when the
NMI hits, that's why we execute modify_code_redo() to 'redo' the full
modification before executing further NMI code. That is executed _on the
CPU_ that triggers an NMI, and the CPU itself is self-consistent.

( The modify_code_redo() will have to do a sync_cores() of course, like
all self-modifying code, to flush speculative execution. )

> Also, speaking of plain atomicity, you scheme does not seem to protect
> against NMIs running on a different CPU, because the non-atomic change
> could race with such NMI.

That's wrong too. Another CPU will notice that redo_pending is set and
will execute modify_code_redo() from its NMI handler _before_ calling
all the notifiers and other 'wide' code paths.

the only item that needs to be marked 'notrace' is only the highlevel
do_nmi() handler itself. (as that executes before we have a chance to
execute modify_code_redo())

So we trade a large, fragile, and unmapped set of NMI-implicated
codepaths for a very tight and well controlled an easy to maintain
codepath.

Ingo