Re: [PATCH -tip] introduce sys_membarrier(): process-wide memory barrier (v9)

[Mathieu Desnoyers <mathieu.desnoyers@efficios.com>]

* Josh Triplett (josh@joshtriplett.org) wrote:
> On Thu, Feb 25, 2010 at 06:23:16PM -0500, Mathieu Desnoyers wrote:
> > I am proposing this patch for the 2.6.34 merge window, as I think it is ready
> > for inclusion.
> > 
> > Here is an implementation of a new system call, sys_membarrier(), which
> > executes a memory barrier on all threads of the current process.
> [...] 
> 
> > Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
> > Acked-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
> > Acked-by: Steven Rostedt <rostedt@goodmis.org>
> > Acked-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
> > CC: Nicholas Miell <nmiell@comcast.net>
> > CC: Linus Torvalds <torvalds@linux-foundation.org>
> > CC: mingo@elte.hu
> > CC: laijs@cn.fujitsu.com
> > CC: dipankar@in.ibm.com
> > CC: akpm@linux-foundation.org
> > CC: josh@joshtriplett.org
> 
> Acked-by: Josh Triplett <josh@joshtriplett.org>
> 
> I agree that v9 seems ready for inclusion.

Thanks!

> 
> Out of curiosity, do you have any benchmarks for the case of not
> detecting sys_membarrier dynamically?  Detecting it at library
> initialization time, for instance, or even just compiling to assume its
> presence?  I'd like to know how much that would improve the numbers.

Citing the patch changelog:

Results in liburcu:

Operations in 10s, 6 readers, 2 writers:

(what we previously had)
memory barriers in reader: 973494744 reads, 892368 writes
signal-based scheme:      6289946025 reads,   1251 writes

(what we have now, with dynamic sys_membarrier check, expedited scheme)
memory barriers in reader: 907693804 reads, 817793 writes
sys_membarrier scheme:    4316818891 reads, 503790 writes

So basically, yes, there is a significant overhead on the read-side if we
compare the dynamic check (0.39 ns/read per reader) to the signal-based scheme
(0.26 ns/read per reader) (which only needs the barrier()). On the update-side,
we cannot care less though.

> 
> If significant, it might make sense to try to have a mechanism similar
> to SMP alternatives, to have different code in either case.  dlopen,
> function pointers, runtime code patching (nop out the rmb), or similar.

Yes, definitely. It could also be useful to switch between UP and SMP primitives
dynamically when spawning the second thread in a process. We should be careful
when sharing memory maps between processes though.

Thanks,

Mathieu

> 
> - Josh Triplett

-- 
Mathieu Desnoyers
Operating System Efficiency Consultant
EfficiOS Inc.
http://www.efficios.com