From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S941179AbcKQR2P (ORCPT ); Thu, 17 Nov 2016 12:28:15 -0500 Received: from mail.efficios.com ([167.114.142.141]:53909 "EHLO mail.efficios.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932650AbcKQR2M (ORCPT ); Thu, 17 Nov 2016 12:28:12 -0500 Date: Thu, 17 Nov 2016 15:02:18 +0000 (UTC) From: Mathieu Desnoyers To: rostedt Cc: "Paul E. McKenney" , Lai Jiangshan , linux-kernel , Josh Triplett , stable Message-ID: <1466810760.5394.1479394938124.JavaMail.zimbra@efficios.com> In-Reply-To: <20161117095011.1857bca5@gandalf.local.home> References: <1478190568-5829-1-git-send-email-mathieu.desnoyers@efficios.com> <604945667.4957.1479383194368.JavaMail.zimbra@efficios.com> <20161117134015.GT3612@linux.vnet.ibm.com> <11301812.5114.1479390867218.JavaMail.zimbra@efficios.com> <20161117095011.1857bca5@gandalf.local.home> Subject: Re: [PATCH] Fix: disable sys_membarrier when nohz_full is enabled MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [167.114.142.141] X-Mailer: Zimbra 8.7.1_GA_1670 (ZimbraWebClient - FF45 (Linux)/8.7.1_GA_1670) Thread-Topic: disable sys_membarrier when nohz_full is enabled Thread-Index: 7X5BxRN1kXzRzvVCRyQnrWTou2b//A== Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org ----- On Nov 17, 2016, at 9:50 AM, rostedt rostedt@goodmis.org wrote: > On Thu, 17 Nov 2016 13:54:27 +0000 (UTC) > Mathieu Desnoyers wrote: > > >> >> > >> >> > Acked-by: Lai Jiangshan >> >> > >> >> > But I'm afraid, in the future, tick_nohz_full will become a default y >> >> > feature. thus it makes sys_membarrier() always disabled. we might >> >> > need a new MEMBARRIER_CMD_XXX to handle it? >> >> >> >> This may require that we send an IPI to nohz_full CPUs, which will >> >> disturb them real-time wise. Any better ideas ? >> > >> > Restrict the IPIs to CPUs running the process executing the >> > sys_membarrier() system call. This would mean that CPUs only >> > are interrupted by their own application's request. >> >> This would break use-cases of cross-process shared memory. :-( > > Perhaps make this an opt in. That is, all processes that want to be > affected by this can call this function with some flag that sets a flag > in tasks struct. And have that process get an IPI even in no-hz-full > mode if it asked to do it. That's an interesting approach. I would be tempted to give it a per-thread (rather than per-process) scope. E.g., a thread could do the following to ask to be interrupted by IPIs: membarrier(MEMBARRIER_CMD_REGISTER_EXPEDITED, 0) and could unregister with: membarrier(MEMBARRIER_CMD_UNREGISTER_EXPEDITED, 0) We can then keep a per-thread refcount internally. (not sure the "EXPEDITED" is the right word there... do we want it to be "NOHZ_FULL" instead ?) Then in membarrier(MEMBARRIER_CMD_SHARED, 0), for each nohz_full cpu, we grab the rq lock, and only send an IPI if the running thread is registered as "expedited". Thoughts ? Thanks, Mathieu -- Mathieu Desnoyers EfficiOS Inc. http://www.efficios.com