From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751385AbdH0Ufh (ORCPT ); Sun, 27 Aug 2017 16:35:37 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:38928 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1751277AbdH0Ufg (ORCPT ); Sun, 27 Aug 2017 16:35:36 -0400 Date: Sun, 27 Aug 2017 13:35:43 -0700 From: "Paul E. McKenney" To: Mathieu Desnoyers Cc: Peter Zijlstra , linux-kernel@vger.kernel.org, Boqun Feng , Andrew Hunter , Maged Michael , gromer@google.com, Avi Kivity , Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Dave Watson , Andy Lutomirski , Will Deacon , Hans Boehm Subject: Re: [RFC PATCH 2/2] membarrier: provide register sync core cmd Reply-To: paulmck@linux.vnet.ibm.com References: <20170827195404.22171-1-mathieu.desnoyers@efficios.com> <20170827195404.22171-2-mathieu.desnoyers@efficios.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20170827195404.22171-2-mathieu.desnoyers@efficios.com> User-Agent: Mutt/1.5.21 (2010-09-15) X-TM-AS-GCONF: 00 x-cbid: 17082720-2213-0000-0000-000002100A12 X-IBM-SpamModules-Scores: X-IBM-SpamModules-Versions: BY=3.00007623; HX=3.00000241; KW=3.00000007; PH=3.00000004; SC=3.00000225; SDB=6.00908487; UDB=6.00455517; IPR=6.00688725; BA=6.00005555; NDR=6.00000001; ZLA=6.00000005; ZF=6.00000009; ZB=6.00000000; ZP=6.00000000; ZH=6.00000000; ZU=6.00000002; MB=3.00016890; XFM=3.00000015; UTC=2017-08-27 20:35:34 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 17082720-2214-0000-0000-0000575AB947 Message-Id: <20170827203543.GH11320@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2017-08-27_07:,, signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=2 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1707230000 definitions=main-1708270314 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Sun, Aug 27, 2017 at 12:54:04PM -0700, Mathieu Desnoyers wrote: > Add a new MEMBARRIER_CMD_REGISTER_SYNC_CORE command to the membarrier > system call. It allows processes to register their intent to have their > threads issue core serializing barriers in addition to memory barriers > whenever a membarrier command is performed. > > It is relevant for reclaim of JIT code, which requires to issue core > serializing barriers on all threads running on behalf of a process > after ensuring the old code is not visible anymore, before re-using > memory for new code. > > When a processes returns from a MEMBARRIER_CMD_REGISTER_SYNC_CORE > command, it is guaranteed that all following MEMBARRIER_CMD_SHARED and > MEMBARRIER_CMD_PRIVATE_EXPEDITED issue core serializing barriers, in > addition to the memory barriers, on all of its running threads. I have queued both of these for testing and review: git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu.git f54cf5123eb9 ("membarrier: Speed up single-threaded private cmd") 0d6eb99818da ("membarrier: Provide register sync core cmd") Could the people who asked for this please test it? The second commit is non-obvious enough to need some careful review and intensive testing before I can in good conscious pass it upstream. Thanx, Paul > Signed-off-by: Mathieu Desnoyers > CC: Peter Zijlstra > CC: Paul E. McKenney > CC: Boqun Feng > CC: Andrew Hunter > CC: Maged Michael > CC: gromer@google.com > CC: Avi Kivity > CC: Benjamin Herrenschmidt > CC: Paul Mackerras > CC: Michael Ellerman > CC: Dave Watson > CC: Andy Lutomirski > CC: Will Deacon > CC: Hans Boehm > --- > fs/exec.c | 1 + > include/linux/sched.h | 52 +++++++++++++++++++++++++++++++++++++++++ > include/uapi/linux/membarrier.h | 1 + > kernel/fork.c | 2 ++ > kernel/sched/core.c | 3 +++ > kernel/sched/membarrier.c | 37 ++++++++++++++++++++++++++--- > 6 files changed, 93 insertions(+), 3 deletions(-) > > diff --git a/fs/exec.c b/fs/exec.c > index 62175cbcc801..a4ab3253bac7 100644 > --- a/fs/exec.c > +++ b/fs/exec.c > @@ -1794,6 +1794,7 @@ static int do_execveat_common(int fd, struct filename *filename, > /* execve succeeded */ > current->fs->in_exec = 0; > current->in_execve = 0; > + membarrier_execve(current); > acct_update_integrals(current); > task_numa_free(current); > free_bprm(bprm); > diff --git a/include/linux/sched.h b/include/linux/sched.h > index 8337e2db0bb2..b1ecdc4e8b84 100644 > --- a/include/linux/sched.h > +++ b/include/linux/sched.h > @@ -1086,6 +1086,9 @@ struct task_struct { > /* Used by LSM modules for access restriction: */ > void *security; > #endif > +#ifdef CONFIG_MEMBARRIER > + int membarrier_sync_core; > +#endif > > /* > * New fields for task_struct should be added above here, so that > @@ -1623,4 +1626,53 @@ extern long sched_getaffinity(pid_t pid, struct cpumask *mask); > #define TASK_SIZE_OF(tsk) TASK_SIZE > #endif > > +#ifdef CONFIG_MEMBARRIER > +static inline void membarrier_fork(struct task_struct *t, > + unsigned long clone_flags) > +{ > + /* > + * Coherence of membarrier_sync_core against thread fork is > + * protected by siglock. membarrier_fork is called with siglock > + * held. > + */ > + t->membarrier_sync_core = current->membarrier_sync_core; > +} > +static inline void membarrier_execve(struct task_struct *t) > +{ > + t->membarrier_sync_core = 0; > +} > +static inline void membarrier_sched_out(struct task_struct *t) > +{ > + /* > + * Core serialization is performed before the memory barrier > + * preceding the store to rq->curr. > + */ > + if (unlikely(READ_ONCE(t->membarrier_sync_core))) > + sync_core(); > +} > +static inline void membarrier_sched_in(struct task_struct *t) > +{ > + /* > + * Core serialization is performed after the memory barrier > + * following the store to rq->curr. > + */ > + if (unlikely(READ_ONCE(t->membarrier_sync_core))) > + sync_core(); > +} > +#else > +static inline void membarrier_fork(struct task_struct *t, > + unsigned long clone_flags) > +{ > +} > +static inline void membarrier_execve(struct task_struct *t) > +{ > +} > +static inline void membarrier_sched_out(struct task_struct *t) > +{ > +} > +static inline void membarrier_sched_in(struct task_struct *t) > +{ > +} > +#endif > + > #endif > diff --git a/include/uapi/linux/membarrier.h b/include/uapi/linux/membarrier.h > index 6d47b3249d8a..bc26fcf8fb7b 100644 > --- a/include/uapi/linux/membarrier.h > +++ b/include/uapi/linux/membarrier.h > @@ -67,6 +67,7 @@ enum membarrier_cmd { > /* reserved for MEMBARRIER_CMD_SHARED_EXPEDITED (1 << 1) */ > /* reserved for MEMBARRIER_CMD_PRIVATE (1 << 2) */ > MEMBARRIER_CMD_PRIVATE_EXPEDITED = (1 << 3), > + MEMBARRIER_CMD_REGISTER_SYNC_CORE = (1 << 4), > }; > > #endif /* _UAPI_LINUX_MEMBARRIER_H */ > diff --git a/kernel/fork.c b/kernel/fork.c > index 17921b0390b4..713b3c932671 100644 > --- a/kernel/fork.c > +++ b/kernel/fork.c > @@ -1840,6 +1840,8 @@ static __latent_entropy struct task_struct *copy_process( > */ > copy_seccomp(p); > > + membarrier_fork(p, clone_flags); > + > /* > * Process group and session signals need to be delivered to just the > * parent before the fork or both the parent and the child after the > diff --git a/kernel/sched/core.c b/kernel/sched/core.c > index 0e36d9960d91..3ca27d066950 100644 > --- a/kernel/sched/core.c > +++ b/kernel/sched/core.c > @@ -3292,6 +3292,8 @@ static void __sched notrace __schedule(bool preempt) > local_irq_disable(); > rcu_note_context_switch(preempt); > > + membarrier_sched_out(prev); > + > /* > * Make sure that signal_pending_state()->signal_pending() below > * can't be reordered with __set_current_state(TASK_INTERRUPTIBLE) > @@ -3364,6 +3366,7 @@ static void __sched notrace __schedule(bool preempt) > > /* Also unlocks the rq: */ > rq = context_switch(rq, prev, next, &rf); > + membarrier_sched_in(next); > } else { > rq->clock_update_flags &= ~(RQCF_ACT_SKIP|RQCF_REQ_SKIP); > rq_unlock_irq(rq, &rf); > diff --git a/kernel/sched/membarrier.c b/kernel/sched/membarrier.c > index 7eec6914d2d2..f128caf64b49 100644 > --- a/kernel/sched/membarrier.c > +++ b/kernel/sched/membarrier.c > @@ -25,12 +25,16 @@ > * Bitmask made from a "or" of all commands within enum membarrier_cmd, > * except MEMBARRIER_CMD_QUERY. > */ > -#define MEMBARRIER_CMD_BITMASK \ > - (MEMBARRIER_CMD_SHARED | MEMBARRIER_CMD_PRIVATE_EXPEDITED) > +#define MEMBARRIER_CMD_BITMASK \ > + (MEMBARRIER_CMD_SHARED \ > + | MEMBARRIER_CMD_PRIVATE_EXPEDITED \ > + | MEMBARRIER_CMD_REGISTER_SYNC_CORE) > > static void ipi_mb(void *info) > { > - smp_mb(); /* IPIs should be serializing but paranoid. */ > + /* IPIs should be serializing but paranoid. */ > + smp_mb(); > + sync_core(); > } > > static void membarrier_private_expedited(void) > @@ -96,6 +100,30 @@ static void membarrier_private_expedited(void) > smp_mb(); /* exit from system call is not a mb */ > } > > +static void membarrier_register_sync_core(void) > +{ > + struct task_struct *p = current, *t; > + > + if (get_nr_threads(p) == 1) { > + p->membarrier_sync_core = 1; > + return; > + } > + > + /* > + * Coherence of membarrier_sync_core against thread fork is > + * protected by siglock. > + */ > + spin_lock(&p->sighand->siglock); > + for_each_thread(p, t) > + WRITE_ONCE(t->membarrier_sync_core, 1); > + spin_unlock(&p->sighand->siglock); > + /* > + * Ensure all future scheduler execution will observe the new > + * membarrier_sync_core state for this process. > + */ > + synchronize_sched(); > +} > + > /** > * sys_membarrier - issue memory barriers on a set of threads > * @cmd: Takes command values defined in enum membarrier_cmd. > @@ -146,6 +174,9 @@ SYSCALL_DEFINE2(membarrier, int, cmd, int, flags) > case MEMBARRIER_CMD_PRIVATE_EXPEDITED: > membarrier_private_expedited(); > return 0; > + case MEMBARRIER_CMD_REGISTER_SYNC_CORE: > + membarrier_register_sync_core(); > + return 0; > default: > return -EINVAL; > } > -- > 2.11.0 >