Re: Rough notes from sys_membarrier() lightning BoF

[Mathieu Desnoyers <mathieu.desnoyers@efficios.com>]

----- On Sep 18, 2017, at 3:29 PM, Paul E. McKenney paulmck@linux.vnet.ibm.com wrote:

> On Mon, Sep 18, 2017 at 03:04:21PM -0400, Alan Stern wrote:
>> On Sun, 17 Sep 2017, Paul E. McKenney wrote:
>> 
>> > Hello!
>> > 
>> > Rough notes from our discussion last Thursday.  Please reply to the
>> > group with any needed elaborations or corrections.
>> > 
>> > Adding Andy and Michael on CC since this most closely affects their
>> > architectures.  Also adding Dave Watson and Maged Michael because
>> > the preferred approach requires that processes wanting to use the
>> > lightweight sys_membarrier() do a registration step.
>> > 
>> > 							Thanx, Paul
>> > 
>> > ------------------------------------------------------------------------
>> > 
>> > Problem:
>> > 
>> > 1.	The current sys_membarrier() introduces an smp_mb() that
>> > 	is not otherwise required on powerpc.
>> > 
>> > 2.	The envisioned JIT variant of sys_membarrier() assumes that
>> > 	the return-to-user instruction sequence handling any change
>> > 	to the usermode instruction stream, and Andy Lutomirski's
>> > 	upcoming changes invalidate this assumption.  It is believed
>> > 	that powerpc has a similar issue.
>> 
>> > E.	Require that threads register before using sys_membarrier() for
>> > 	private or JIT usage.  (The historical implementation using
>> > 	synchronize_sched() would continue to -not- require registration,
>> > 	both for compatibility and because there is no need to do so.)
>> > 
>> > 	For x86 and powerpc, this registration would set a TIF flag
>> > 	on all of the current process's threads.  This flag would be
>> > 	inherited by any later thread creation within that process, and
>> > 	would be cleared by fork() and exec().	When this TIF flag is set,
>> 
>> Why a TIF flag, and why clear it during fork()?  If a process registers
>> to use private expedited sys_membarrier, shouldn't that apply to
>> threads it will create in the future just as much as to threads it has
>> already created?
> 
> The reason for a TIF flag is to keep this per-architecture, as only
> powerpc and x86 need it.
> 
> The reason for clearing it during fork() is that fork() creates a new
> process initially having but a single thread, which might or might
> not use sys_membarrier().  Usually not, as most instances of fork()
> are quickly followed by exec().  In addition, if we give an error for
> unregistered use of private sys_membarrier(), clearing on fork() gets an
> unambiguous error instead of a silent likely failure (due to libraries
> being confused by the fork()).

I think clearing that state on fork() would be unexpected. The child process
inherits from the parent flag in my current implementation. Clearing the
flag is only provided through exec().

Libraries don't get re-initialized on fork, only on exec(). Therefore, it
makes sense for the child process to inherit the state from its parent.

Thanks,

Mathieu

-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com