From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S936364AbdDZPot (ORCPT ); Wed, 26 Apr 2017 11:44:49 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:57411 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1760320AbdDZPoJ (ORCPT ); Wed, 26 Apr 2017 11:44:09 -0400 Date: Wed, 26 Apr 2017 08:44:02 -0700 From: "Paul E. McKenney" To: Mike Galbraith Cc: LKML Subject: Re: TREE_SRCU slows hotplug by factor ~16 Reply-To: paulmck@linux.vnet.ibm.com References: <1493002089.4145.7.camel@gmx.de> <20170424033250.GX3956@linux.vnet.ibm.com> <1493011484.4166.13.camel@gmx.de> <20170424062220.GZ3956@linux.vnet.ibm.com> <1493019303.4617.6.camel@gmx.de> <20170424162442.GI3956@linux.vnet.ibm.com> <20170425223642.GA10931@linux.vnet.ibm.com> <20170426143145.GA5683@linux.vnet.ibm.com> <1493220380.6176.2.camel@gmx.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1493220380.6176.2.camel@gmx.de> User-Agent: Mutt/1.5.21 (2010-09-15) X-TM-AS-GCONF: 00 x-cbid: 17042615-0008-0000-0000-00000205DBE6 X-IBM-SpamModules-Scores: X-IBM-SpamModules-Versions: BY=3.00006977; HX=3.00000240; KW=3.00000007; PH=3.00000004; SC=3.00000208; SDB=6.00852783; UDB=6.00421567; IPR=6.00631621; BA=6.00005313; NDR=6.00000001; ZLA=6.00000005; ZF=6.00000009; ZB=6.00000000; ZP=6.00000000; ZH=6.00000000; ZU=6.00000002; MB=3.00015188; XFM=3.00000013; UTC=2017-04-26 15:44:04 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 17042615-0009-0000-0000-000034F4C3DE Message-Id: <20170426154402.GT3956@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2017-04-26_10:,, signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=0 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1703280000 definitions=main-1704260269 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, Apr 26, 2017 at 05:26:20PM +0200, Mike Galbraith wrote: > On Wed, 2017-04-26 at 07:31 -0700, Paul E. McKenney wrote: > > > And a sneak preview, semi-tested. If you get a chance to run this, please > > let me know now it goes. > > That took 'time stress-cpu-hotplug.sh' down to 48s, close to classic. Woo-hoo!!! ;-) And thank you for your testing efforts! Should I be comparing this with the 55s number from your initial email, or to the 39s number? Either way, given the unusual nature of Steven's hotplug stress test, I believe that I am good enough for this merge window. But if we are talking 48s for Tree SRCU vs. 39s with Classic SRCU, it would be good to at least understand where the remaining slowdown is. Here are a couple of possible causes: o My holdoff is too long. I set it to 50 microseconds based on your trace, which shows a minimum grace-period separation of 118 microseconds. But perhaps the trace was too short to show the full variation. One way to check this is to run with srcutree.exp_holdoff=25000 or some such. (Please note that srcutree.exp_holdoff is in nanoseconds, -not- microseconds.) o My expedited throttling is too aggressive. This is controlled by the following lines of code in srcu_gp_end() in the file kernel/rcu/srcutree.c: /* Throttle expedited grace periods: Should be rare! */ srcu_reschedule(sp, rcu_seq_ctr(gpseq) & 0x3ff ? 0 : SRCU_INTERVAL); The "0x3ff" says that one in 1024 grace periods should be forced to be at least partially non-expedited, regardless of anything else. If making this be (say) "0xfff" gets you three-quarters of the way to the 39s, that indicates that this is the controlling factor. o Of course, another question is how much variation there is in the timing of that stress test. If further reduction is needed, and none of these help, could you please send me a trace of the full run of the same form as the last one you sent me, covering calls to and returns from call_srcu(), synchronize_srcu(), and synchronize_srcu_expedited()? Thanx, paul