From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752149AbcKQT1g (ORCPT ); Thu, 17 Nov 2016 14:27:36 -0500 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:53169 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1750772AbcKQT1e (ORCPT ); Thu, 17 Nov 2016 14:27:34 -0500 Date: Thu, 17 Nov 2016 05:40:15 -0800 From: "Paul E. McKenney" To: Mathieu Desnoyers Cc: Lai Jiangshan , linux-kernel , Josh Triplett , rostedt , stable Subject: Re: [PATCH] Fix: disable sys_membarrier when nohz_full is enabled Reply-To: paulmck@linux.vnet.ibm.com References: <1478190568-5829-1-git-send-email-mathieu.desnoyers@efficios.com> <604945667.4957.1479383194368.JavaMail.zimbra@efficios.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <604945667.4957.1479383194368.JavaMail.zimbra@efficios.com> User-Agent: Mutt/1.5.21 (2010-09-15) X-TM-AS-GCONF: 00 X-Content-Scanned: Fidelis XPS MAILER x-cbid: 16111713-0004-0000-0000-000010DFDC92 X-IBM-SpamModules-Scores: X-IBM-SpamModules-Versions: BY=3.00006094; HX=3.00000240; KW=3.00000007; PH=3.00000004; SC=3.00000189; SDB=6.00781942; UDB=6.00377261; IPR=6.00559446; BA=6.00004889; NDR=6.00000001; ZLA=6.00000005; ZF=6.00000009; ZB=6.00000000; ZP=6.00000000; ZH=6.00000000; ZU=6.00000002; MB=3.00013361; XFM=3.00000011; UTC=2016-11-17 13:40:17 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 16111713-0005-0000-0000-00007AA8FFD1 Message-Id: <20161117134015.GT3612@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2016-11-17_06:,, signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=0 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1609300000 definitions=main-1611170247 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, Nov 17, 2016 at 11:46:34AM +0000, Mathieu Desnoyers wrote: > ----- On Nov 17, 2016, at 1:51 AM, Lai Jiangshan jiangshanlai@gmail.com wrote: > > > On Fri, Nov 4, 2016 at 12:29 AM, Mathieu Desnoyers > > wrote: > >> Userspace applications should be allowed to expect the membarrier system > >> call with MEMBARRIER_CMD_SHARED command to issue memory barriers on > >> nohz_full CPUs, but synchronize_sched() does not take those into > >> account. > >> > >> Given that we do not want unrelated processes to be able to affect > >> real-time sensitive nohz_full CPUs, simply return ENOSYS when membarrier > >> is invoked on a kernel with enabled nohz_full CPUs. > >> > >> Signed-off-by: Mathieu Desnoyers > >> CC: "Paul E. McKenney" > >> CC: Josh Triplett > >> CC: Steven Rostedt > >> CC: Lai Jiangshan > >> CC: [3.10+] > >> --- > >> kernel/membarrier.c | 4 ++++ > >> 1 file changed, 4 insertions(+) > >> > >> diff --git a/kernel/membarrier.c b/kernel/membarrier.c > >> index 536c727..9f9284f 100644 > >> --- a/kernel/membarrier.c > >> +++ b/kernel/membarrier.c > >> @@ -16,6 +16,7 @@ > >> > >> #include > >> #include > >> +#include > >> > >> /* > >> * Bitmask made from a "or" of all commands within enum membarrier_cmd, > >> @@ -51,6 +52,9 @@ > >> */ > >> SYSCALL_DEFINE2(membarrier, int, cmd, int, flags) > >> { > >> + /* MEMBARRIER_CMD_SHARED is not compatible with nohz_full. */ > >> + if (tick_nohz_full_enabled()) > >> + return -ENOSYS; > > > > I guess this code needs to be moved down into the branch of > > "case MEMBARRIER_CMD_SHARED" to match its comment. > > No, that would be unexpected from user-space. Either a system > call is implemented or not, not "implemented for some parameters". > > We also want MEMBARRIER_CMD_QUERY to return -ENOSYS in this case, > and all other parameter values to also return -ENOSYS (rather than > -EINVAL). > > If a system call that returns successfully on CMD_QUERY or EINVAL, > user-space may assume it will not have to handle ENOSYS in the > next calls. > > > > > > Acked-by: Lai Jiangshan > > > > But I'm afraid, in the future, tick_nohz_full will become a default y > > feature. thus it makes sys_membarrier() always disabled. we might > > need a new MEMBARRIER_CMD_XXX to handle it? > > This may require that we send an IPI to nohz_full CPUs, which will > disturb them real-time wise. Any better ideas ? Restrict the IPIs to CPUs running the process executing the sys_membarrier() system call. This would mean that CPUs only are interrupted by their own application's request. Thanx, Paul