From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755555AbdGXV6a (ORCPT ); Mon, 24 Jul 2017 17:58:30 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:60398 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1751482AbdGXV6V (ORCPT ); Mon, 24 Jul 2017 17:58:21 -0400 From: "Paul E. McKenney" To: linux-kernel@vger.kernel.org Cc: mingo@kernel.org, jiangshanlai@gmail.com, dipankar@in.ibm.com, akpm@linux-foundation.org, mathieu.desnoyers@efficios.com, josh@joshtriplett.org, tglx@linutronix.de, peterz@infradead.org, rostedt@goodmis.org, dhowells@redhat.com, edumazet@google.com, fweisbec@gmail.com, oleg@redhat.com, "Paul E. McKenney" Subject: [PATCH tip/core/rcu 4/5] sys_membarrier: Add expedited option Date: Mon, 24 Jul 2017 14:58:16 -0700 X-Mailer: git-send-email 2.5.2 In-Reply-To: <20170724215758.GA12075@linux.vnet.ibm.com> References: <20170724215758.GA12075@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 x-cbid: 17072421-0036-0000-0000-0000024E0F9D X-IBM-SpamModules-Scores: X-IBM-SpamModules-Versions: BY=3.00007419; HX=3.00000241; KW=3.00000007; PH=3.00000004; SC=3.00000214; SDB=6.00892377; UDB=6.00446024; IPR=6.00672541; BA=6.00005489; NDR=6.00000001; ZLA=6.00000005; ZF=6.00000009; ZB=6.00000000; ZP=6.00000000; ZH=6.00000000; ZU=6.00000002; MB=3.00016356; XFM=3.00000015; UTC=2017-07-24 21:58:19 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 17072421-0037-0000-0000-00004134A649 Message-Id: <1500933497-12612-4-git-send-email-paulmck@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2017-07-24_13:,, signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=1 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1706020000 definitions=main-1707240335 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The sys_membarrier() system call has proven too slow for some use cases, which has prompted users to instead rely on TLB shootdown. Although TLB shootdown is much faster, it has the slight disadvantage of not working at all on arm and arm64. This commit therefore adds an expedited option to the sys_membarrier() system call. Signed-off-by: Paul E. McKenney --- include/uapi/linux/membarrier.h | 11 +++++++++++ kernel/membarrier.c | 7 ++++++- 2 files changed, 17 insertions(+), 1 deletion(-) diff --git a/include/uapi/linux/membarrier.h b/include/uapi/linux/membarrier.h index e0b108bd2624..ba36d8a6be61 100644 --- a/include/uapi/linux/membarrier.h +++ b/include/uapi/linux/membarrier.h @@ -40,6 +40,16 @@ * (non-running threads are de facto in such a * state). This covers threads from all processes * running on the system. This command returns 0. + * @MEMBARRIER_CMD_SHARED_EXPEDITED: Execute a memory barrier on all + * running threads, but in an expedited fashion. + * Upon return from system call, the caller thread + * is ensured that all running threads have passed + * through a state where all memory accesses to + * user-space addresses match program order between + * entry to and return from the system call + * (non-running threads are de facto in such a + * state). This covers threads from all processes + * running on the system. This command returns 0. * * Command to be passed to the membarrier system call. The commands need to * be a single bit each, except for MEMBARRIER_CMD_QUERY which is assigned to @@ -48,6 +58,7 @@ enum membarrier_cmd { MEMBARRIER_CMD_QUERY = 0, MEMBARRIER_CMD_SHARED = (1 << 0), + MEMBARRIER_CMD_SHARED_EXPEDITED = (2 << 0), }; #endif /* _UAPI_LINUX_MEMBARRIER_H */ diff --git a/kernel/membarrier.c b/kernel/membarrier.c index 9f9284f37f8d..b749c39bb219 100644 --- a/kernel/membarrier.c +++ b/kernel/membarrier.c @@ -22,7 +22,8 @@ * Bitmask made from a "or" of all commands within enum membarrier_cmd, * except MEMBARRIER_CMD_QUERY. */ -#define MEMBARRIER_CMD_BITMASK (MEMBARRIER_CMD_SHARED) +#define MEMBARRIER_CMD_BITMASK (MEMBARRIER_CMD_SHARED | \ + MEMBARRIER_CMD_SHARED_EXPEDITED) /** * sys_membarrier - issue memory barriers on a set of threads @@ -64,6 +65,10 @@ SYSCALL_DEFINE2(membarrier, int, cmd, int, flags) if (num_online_cpus() > 1) synchronize_sched(); return 0; + case MEMBARRIER_CMD_SHARED_EXPEDITED: + if (num_online_cpus() > 1) + synchronize_sched_expedited(); + return 0; default: return -EINVAL; } -- 2.5.2