From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:55076 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754448AbcKQRBW (ORCPT ); Thu, 17 Nov 2016 12:01:22 -0500 Received: from pps.filterd (m0098396.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.17/8.16.0.17) with SMTP id uAHE9BG1026569 for ; Thu, 17 Nov 2016 09:09:19 -0500 Received: from e37.co.us.ibm.com (e37.co.us.ibm.com [32.97.110.158]) by mx0a-001b2d01.pphosted.com with ESMTP id 26s8fqsqrq-1 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT) for ; Thu, 17 Nov 2016 09:09:18 -0500 Received: from localhost by e37.co.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Thu, 17 Nov 2016 07:09:18 -0700 Date: Thu, 17 Nov 2016 06:09:15 -0800 From: "Paul E. McKenney" To: Mathieu Desnoyers Cc: Lai Jiangshan , linux-kernel , Josh Triplett , rostedt , stable Subject: Re: [PATCH] Fix: disable sys_membarrier when nohz_full is enabled Reply-To: paulmck@linux.vnet.ibm.com References: <1478190568-5829-1-git-send-email-mathieu.desnoyers@efficios.com> <604945667.4957.1479383194368.JavaMail.zimbra@efficios.com> <20161117134015.GT3612@linux.vnet.ibm.com> <11301812.5114.1479390867218.JavaMail.zimbra@efficios.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <11301812.5114.1479390867218.JavaMail.zimbra@efficios.com> Message-Id: <20161117140915.GX3612@linux.vnet.ibm.com> Sender: stable-owner@vger.kernel.org List-ID: On Thu, Nov 17, 2016 at 01:54:27PM +0000, Mathieu Desnoyers wrote: > ----- On Nov 17, 2016, at 8:40 AM, Paul E. McKenney paulmck@linux.vnet.ibm.com wrote: > > > On Thu, Nov 17, 2016 at 11:46:34AM +0000, Mathieu Desnoyers wrote: > >> ----- On Nov 17, 2016, at 1:51 AM, Lai Jiangshan jiangshanlai@gmail.com wrote: > >> > >> > On Fri, Nov 4, 2016 at 12:29 AM, Mathieu Desnoyers > >> > wrote: > >> >> Userspace applications should be allowed to expect the membarrier system > >> >> call with MEMBARRIER_CMD_SHARED command to issue memory barriers on > >> >> nohz_full CPUs, but synchronize_sched() does not take those into > >> >> account. > >> >> > >> >> Given that we do not want unrelated processes to be able to affect > >> >> real-time sensitive nohz_full CPUs, simply return ENOSYS when membarrier > >> >> is invoked on a kernel with enabled nohz_full CPUs. > >> >> > >> >> Signed-off-by: Mathieu Desnoyers > >> >> CC: "Paul E. McKenney" > >> >> CC: Josh Triplett > >> >> CC: Steven Rostedt > >> >> CC: Lai Jiangshan > >> >> CC: [3.10+] > >> >> --- > >> >> kernel/membarrier.c | 4 ++++ > >> >> 1 file changed, 4 insertions(+) > >> >> > >> >> diff --git a/kernel/membarrier.c b/kernel/membarrier.c > >> >> index 536c727..9f9284f 100644 > >> >> --- a/kernel/membarrier.c > >> >> +++ b/kernel/membarrier.c > >> >> @@ -16,6 +16,7 @@ > >> >> > >> >> #include > >> >> #include > >> >> +#include > >> >> > >> >> /* > >> >> * Bitmask made from a "or" of all commands within enum membarrier_cmd, > >> >> @@ -51,6 +52,9 @@ > >> >> */ > >> >> SYSCALL_DEFINE2(membarrier, int, cmd, int, flags) > >> >> { > >> >> + /* MEMBARRIER_CMD_SHARED is not compatible with nohz_full. */ > >> >> + if (tick_nohz_full_enabled()) > >> >> + return -ENOSYS; > >> > > >> > I guess this code needs to be moved down into the branch of > >> > "case MEMBARRIER_CMD_SHARED" to match its comment. > >> > >> No, that would be unexpected from user-space. Either a system > >> call is implemented or not, not "implemented for some parameters". > >> > >> We also want MEMBARRIER_CMD_QUERY to return -ENOSYS in this case, > >> and all other parameter values to also return -ENOSYS (rather than > >> -EINVAL). > >> > >> If a system call that returns successfully on CMD_QUERY or EINVAL, > >> user-space may assume it will not have to handle ENOSYS in the > >> next calls. > >> > >> > >> > > >> > Acked-by: Lai Jiangshan > >> > > >> > But I'm afraid, in the future, tick_nohz_full will become a default y > >> > feature. thus it makes sys_membarrier() always disabled. we might > >> > need a new MEMBARRIER_CMD_XXX to handle it? > >> > >> This may require that we send an IPI to nohz_full CPUs, which will > >> disturb them real-time wise. Any better ideas ? > > > > Restrict the IPIs to CPUs running the process executing the > > sys_membarrier() system call. This would mean that CPUs only > > are interrupted by their own application's request. > > This would break use-cases of cross-process shared memory. :-( Good point -- getting this working does look to be good clean fun... Thanx, Paul