Re: [RFC patch] introduce sys_membarrier(): process-wide memory barrier (v9)

[Nick Piggin <npiggin@suse.de>]

On Wed, Feb 24, 2010 at 10:22:52AM -0500, Mathieu Desnoyers wrote:
> * Nick Piggin (npiggin@suse.de) wrote:
> > On Mon, Feb 22, 2010 at 04:23:21PM -0500, Mathieu Desnoyers wrote:
> > > * Chris Friesen (cfriesen@nortel.com) wrote:
> > > > On 02/12/2010 04:46 PM, Mathieu Desnoyers wrote:
> > > > 
> > > > > Editorial question: 
> > > > > 
> > > > > This synchronization only takes care of threads using the current process memory
> > > > > map. It should not be used to synchronize accesses performed on memory maps
> > > > > shared between different processes. Is that a limitation we can live with ?
> > > > 
> > > > It makes sense for an initial version.  It would be unfortunate if this
> > > > were a permanent limitation, since using separate processes with
> > > > explicit shared memory is a useful way to mitigate memory trampler issues.
> > > > 
> > > > If we were going to allow that, it might make sense to add an address
> > > > range such that only those processes which have mapped that range would
> > > > execute the barrier.  Come to think of it, it might be possible to use
> > > > this somehow to avoid having to execute the barrier on *all* threads
> > > > within a process.
> > > 
> > > The extensible system call mandatory and optional flags will allow this kind of
> > > improvement later on if this appears to be needed. It will also allow user-space
> > > to detect if later kernels support these new features or not. But meanwhile I
> > > think it's good to start with this implementation that covers 99.99% of
> > > use-cases I can currently think of (ok, well, maybe I'm just unimaginative) ;)
> > 
> > It's a good point, I think having at least the ability to do
> > process-shared or process-private in the first version of the API might
> > be a good idea. That matches glibc's synchronisation routines so it
> > would probably be a desirable feature even if you don't implement it in
> > your library initially.
> 
> I am tempted to say that we should probably wait for users of this API feature
> to manifest themselves before we go on and implement it. This will ensure that
> we don't end up maintaining an unused feature and this provides a minimum
> testability. For now, returning -EINVAL seems like an appropriate response for
> this system call feature.

It would be very trivial compared to the process-private case. Just IPI
all CPUs. It would allow older kernels to work with newer process based
apps as they get implemented. But... not a really big deal I suppose.


> As I said above, given the exensible nature of the sys_membarrier flags, we can
> assign a MEMBARRIER_SHARED_MEM or something like that to a mandatory flag bit
> later on. So when userspace start using this flag on old kernels that do not
> support it, -EINVAL will be returned, and then the application will know it must
> use a fallback. So, basically, we don't even need to define this flag now.
> 
> > 
> > When writing multiprocessor scalable software, threads should often be
> > avoided. They share so much state that it is easy to run into
> > scalability issues in the kernel. So yes it would be really nice to
> > have userspace RCU available in a process-shared mode.
> > 
> 
> Agreed, although some major modifications would also be needed in the userspace
> RCU library to do that, because it currently rely on being able to access other
> thread's TLS.

OK. It would be a good feature to keep in mind, I believe.