From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751562AbdGZVSJ (ORCPT ); Wed, 26 Jul 2017 17:18:09 -0400 Received: from smtprelay0109.hostedemail.com ([216.40.44.109]:46894 "EHLO smtprelay.hostedemail.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1751458AbdGZVSI (ORCPT ); Wed, 26 Jul 2017 17:18:08 -0400 X-Session-Marker: 726F737465647440676F6F646D69732E6F7267 X-Spam-Summary: 2,0,0,,d41d8cd98f00b204,rostedt@goodmis.org,:::::::::::::::::::::::::::,RULES_HIT:41:355:379:541:599:800:960:973:982:988:989:1260:1277:1311:1313:1314:1345:1359:1437:1515:1516:1518:1534:1543:1593:1594:1605:1711:1730:1747:1777:1792:2393:2553:2559:2562:2691:2693:2904:3138:3139:3140:3141:3142:3622:3865:3866:3867:3868:3870:3871:3872:3873:3874:4321:5007:6119:6261:6742:7875:7903:8660:8957:9040:10004:10394:10400:10848:10967:11026:11232:11658:11914:12294:12296:12740:12760:12895:13148:13161:13180:13229:13230:13255:13439:13868:14096:14097:14181:14659:14721:21063:21080:21611:21627:30012:30054:30090:30091,0,RBL:none,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:,MSBL:0,DNSBL:none,Custom_rules:0:0:0,LFtime:4,LUA_SUMMARY:none X-HE-Tag: touch31_80c40c6587d07 X-Filterd-Recvd-Size: 4609 Date: Wed, 26 Jul 2017 17:18:01 -0400 From: Steven Rostedt To: "Paul E. McKenney" Cc: linux-kernel@vger.kernel.org, mingo@kernel.org, jiangshanlai@gmail.com, dipankar@in.ibm.com, akpm@linux-foundation.org, mathieu.desnoyers@efficios.com, josh@joshtriplett.org, tglx@linutronix.de, peterz@infradead.org, dhowells@redhat.com, edumazet@google.com, fweisbec@gmail.com, oleg@redhat.com Subject: Re: [PATCH tip/core/rcu 02/15] rcu: Use timer as backstop for NOCB deferred wakeups Message-ID: <20170726171801.5da044c3@vmware.local.home> In-Reply-To: <20170726000540.GE3730@linux.vnet.ibm.com> References: <20170724214425.GA9665@linux.vnet.ibm.com> <1500932684-10469-2-git-send-email-paulmck@linux.vnet.ibm.com> <20170725141220.4d2d968e@vmware.local.home> <20170725191814.GU3730@linux.vnet.ibm.com> <20170725181710.44cd1002@vmware.local.home> <20170726000540.GE3730@linux.vnet.ibm.com> X-Mailer: Claws Mail 3.15.0-dirty (GTK+ 2.24.31; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, 25 Jul 2017 17:05:40 -0700 "Paul E. McKenney" wrote: > On Tue, Jul 25, 2017 at 06:17:10PM -0400, Steven Rostedt wrote: > > On Tue, 25 Jul 2017 12:18:14 -0700 > > "Paul E. McKenney" wrote: > > > > > On Tue, Jul 25, 2017 at 02:12:20PM -0400, Steven Rostedt wrote: > > > > On Mon, 24 Jul 2017 14:44:31 -0700 > > > > "Paul E. McKenney" wrote: > > > > > > > > > The handling of RCU's no-CBs CPUs has a maintenance headache, namely > > > > > that if call_rcu() is invoked with interrupts disabled, the rcuo kthread > > > > > wakeup must be defered to a point where we can be sure that scheduler > > > > > locks are not held. Of course, there are a lot of code paths leading > > > > > from an interrupts-disabled invocation of call_rcu(), and missing any > > > > > one of these can result in excessive callback-invocation latency, and > > > > > potentially even system hangs. > > > > > > > > What about using irq_work? That's what perf and ftrace use for such a > > > > case. > > > > > > I hadn't looked at irq_work before, thank you for the pointer! > > > > > > I nevertheless believe that timers work better in this particular case > > > because they can be cancelled (which appears to be the common case), they > > > > Is the common case here that it doesn't trigger? That is, the > > del_timer() will be called? > > If you have lots of call_rcu() invocations, many of them will be invoked > with interrupts enabled, and a later one with interrupts enabled will > take care of things for the earlier ones. So there can be workloads > where this is the case. Note, only the first irq_work called will take action. The other callers will see that a irq_work is pending and will not reivoke one. > > > > normally are not at all time-critical, and because running in softirq > > > is just fine -- no need to run out of the scheduling-clock interrupt. > > > > irq_work doesn't always use the scheduling clock. IIRC, it will simply > > trigger a interrupt (if the arch supports it), and the work will be > > done when interrupts are enabled (the interrupt that will do the work > > will trigger) > > Ah, OK, so scheduling clock is just the backstop. Still, softirq > is a bit nicer to manage than hardirq. Still requires a hard interrupt (timer) (thinking of NOHZ FULL where this does matter). > > > > Seem reasonable? > > > > Don't know. With irq_work, you just call it and forget about it. No > > need to mod or del timers. > > But I could have a series of call_rcu() invocations with interrupts > disabled, so I would need to interact somehow with the irq_work handler. > Either that or dynamically allocate the needed data structure. > > Or am I missing something here? You treat it just like you are with the timer code. You have a irq_work struct attached to your rdp descriptor. And call irq_work_run() when interrupts are disabled. If it hasn't already been invoked it will invoke one. Then the irq_work handler will look at the rdp attached to the irq_work (container_of()), and then wake the associated thread. It is much lighter weight than a timer setup. -- Steve