Re: [RFC patch] introduce sys_membarrier(): process-wide memory barrier (v9)

[Mathieu Desnoyers <mathieu.desnoyers@efficios.com>]

* Nick Piggin (npiggin@suse.de) wrote:
> On Mon, Feb 22, 2010 at 04:23:21PM -0500, Mathieu Desnoyers wrote:
> > * Chris Friesen (cfriesen@nortel.com) wrote:
> > > On 02/12/2010 04:46 PM, Mathieu Desnoyers wrote:
> > > 
> > > > Editorial question: 
> > > > 
> > > > This synchronization only takes care of threads using the current process memory
> > > > map. It should not be used to synchronize accesses performed on memory maps
> > > > shared between different processes. Is that a limitation we can live with ?
> > > 
> > > It makes sense for an initial version.  It would be unfortunate if this
> > > were a permanent limitation, since using separate processes with
> > > explicit shared memory is a useful way to mitigate memory trampler issues.
> > > 
> > > If we were going to allow that, it might make sense to add an address
> > > range such that only those processes which have mapped that range would
> > > execute the barrier.  Come to think of it, it might be possible to use
> > > this somehow to avoid having to execute the barrier on *all* threads
> > > within a process.
> > 
> > The extensible system call mandatory and optional flags will allow this kind of
> > improvement later on if this appears to be needed. It will also allow user-space
> > to detect if later kernels support these new features or not. But meanwhile I
> > think it's good to start with this implementation that covers 99.99% of
> > use-cases I can currently think of (ok, well, maybe I'm just unimaginative) ;)
> 
> It's a good point, I think having at least the ability to do
> process-shared or process-private in the first version of the API might
> be a good idea. That matches glibc's synchronisation routines so it
> would probably be a desirable feature even if you don't implement it in
> your library initially.

I am tempted to say that we should probably wait for users of this API feature
to manifest themselves before we go on and implement it. This will ensure that
we don't end up maintaining an unused feature and this provides a minimum
testability. For now, returning -EINVAL seems like an appropriate response for
this system call feature.

As I said above, given the exensible nature of the sys_membarrier flags, we can
assign a MEMBARRIER_SHARED_MEM or something like that to a mandatory flag bit
later on. So when userspace start using this flag on old kernels that do not
support it, -EINVAL will be returned, and then the application will know it must
use a fallback. So, basically, we don't even need to define this flag now.

> 
> When writing multiprocessor scalable software, threads should often be
> avoided. They share so much state that it is easy to run into
> scalability issues in the kernel. So yes it would be really nice to
> have userspace RCU available in a process-shared mode.
> 

Agreed, although some major modifications would also be needed in the userspace
RCU library to do that, because it currently rely on being able to access other
thread's TLS.

Thanks,

Mathieu