From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932867AbcKQOdQ (ORCPT ); Thu, 17 Nov 2016 09:33:16 -0500 Received: from mail.efficios.com ([167.114.142.141]:51858 "EHLO mail.efficios.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753531AbcKQOdN (ORCPT ); Thu, 17 Nov 2016 09:33:13 -0500 Date: Thu, 17 Nov 2016 13:54:27 +0000 (UTC) From: Mathieu Desnoyers To: "Paul E. McKenney" Cc: Lai Jiangshan , linux-kernel , Josh Triplett , rostedt , stable Message-ID: <11301812.5114.1479390867218.JavaMail.zimbra@efficios.com> In-Reply-To: <20161117134015.GT3612@linux.vnet.ibm.com> References: <1478190568-5829-1-git-send-email-mathieu.desnoyers@efficios.com> <604945667.4957.1479383194368.JavaMail.zimbra@efficios.com> <20161117134015.GT3612@linux.vnet.ibm.com> Subject: Re: [PATCH] Fix: disable sys_membarrier when nohz_full is enabled MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [167.114.142.141] X-Mailer: Zimbra 8.7.1_GA_1670 (ZimbraWebClient - FF45 (Linux)/8.7.1_GA_1670) Thread-Topic: disable sys_membarrier when nohz_full is enabled Thread-Index: jRuBS/Jl2rC2ZejBsZhHsgw31twxiQ== Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org ----- On Nov 17, 2016, at 8:40 AM, Paul E. McKenney paulmck@linux.vnet.ibm.com wrote: > On Thu, Nov 17, 2016 at 11:46:34AM +0000, Mathieu Desnoyers wrote: >> ----- On Nov 17, 2016, at 1:51 AM, Lai Jiangshan jiangshanlai@gmail.com wrote: >> >> > On Fri, Nov 4, 2016 at 12:29 AM, Mathieu Desnoyers >> > wrote: >> >> Userspace applications should be allowed to expect the membarrier system >> >> call with MEMBARRIER_CMD_SHARED command to issue memory barriers on >> >> nohz_full CPUs, but synchronize_sched() does not take those into >> >> account. >> >> >> >> Given that we do not want unrelated processes to be able to affect >> >> real-time sensitive nohz_full CPUs, simply return ENOSYS when membarrier >> >> is invoked on a kernel with enabled nohz_full CPUs. >> >> >> >> Signed-off-by: Mathieu Desnoyers >> >> CC: "Paul E. McKenney" >> >> CC: Josh Triplett >> >> CC: Steven Rostedt >> >> CC: Lai Jiangshan >> >> CC: [3.10+] >> >> --- >> >> kernel/membarrier.c | 4 ++++ >> >> 1 file changed, 4 insertions(+) >> >> >> >> diff --git a/kernel/membarrier.c b/kernel/membarrier.c >> >> index 536c727..9f9284f 100644 >> >> --- a/kernel/membarrier.c >> >> +++ b/kernel/membarrier.c >> >> @@ -16,6 +16,7 @@ >> >> >> >> #include >> >> #include >> >> +#include >> >> >> >> /* >> >> * Bitmask made from a "or" of all commands within enum membarrier_cmd, >> >> @@ -51,6 +52,9 @@ >> >> */ >> >> SYSCALL_DEFINE2(membarrier, int, cmd, int, flags) >> >> { >> >> + /* MEMBARRIER_CMD_SHARED is not compatible with nohz_full. */ >> >> + if (tick_nohz_full_enabled()) >> >> + return -ENOSYS; >> > >> > I guess this code needs to be moved down into the branch of >> > "case MEMBARRIER_CMD_SHARED" to match its comment. >> >> No, that would be unexpected from user-space. Either a system >> call is implemented or not, not "implemented for some parameters". >> >> We also want MEMBARRIER_CMD_QUERY to return -ENOSYS in this case, >> and all other parameter values to also return -ENOSYS (rather than >> -EINVAL). >> >> If a system call that returns successfully on CMD_QUERY or EINVAL, >> user-space may assume it will not have to handle ENOSYS in the >> next calls. >> >> >> > >> > Acked-by: Lai Jiangshan >> > >> > But I'm afraid, in the future, tick_nohz_full will become a default y >> > feature. thus it makes sys_membarrier() always disabled. we might >> > need a new MEMBARRIER_CMD_XXX to handle it? >> >> This may require that we send an IPI to nohz_full CPUs, which will >> disturb them real-time wise. Any better ideas ? > > Restrict the IPIs to CPUs running the process executing the > sys_membarrier() system call. This would mean that CPUs only > are interrupted by their own application's request. This would break use-cases of cross-process shared memory. :-( Mathieu > > Thanx, Paul -- Mathieu Desnoyers EfficiOS Inc. http://www.efficios.com