From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751677AbdCDGpV (ORCPT ); Sat, 4 Mar 2017 01:45:21 -0500 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:59708 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750751AbdCDGpT (ORCPT ); Sat, 4 Mar 2017 01:45:19 -0500 Date: Fri, 3 Mar 2017 16:17:15 -0800 From: "Paul E. McKenney" To: Tejun Heo Cc: jiangshanlai@gmail.com, linux-kernel@vger.kernel.org Subject: Re: Is it really safe to use workqueues to drive expedited grace periods? Reply-To: paulmck@linux.vnet.ibm.com References: <20170210212158.GA20183@linux.vnet.ibm.com> <20170211023541.GB19050@mtj.duckdns.org> <20170214001600.GZ30506@linux.vnet.ibm.com> <20170303193049.GC22962@wtj.duckdns.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20170303193049.GC22962@wtj.duckdns.org> User-Agent: Mutt/1.5.21 (2010-09-15) X-TM-AS-GCONF: 00 X-Content-Scanned: Fidelis XPS MAILER x-cbid: 17030406-0044-0000-0000-000002B77885 X-IBM-SpamModules-Scores: X-IBM-SpamModules-Versions: BY=3.00006719; HX=3.00000240; KW=3.00000007; PH=3.00000004; SC=3.00000206; SDB=6.00829846; UDB=6.00406894; IPR=6.00607361; BA=6.00005186; NDR=6.00000001; ZLA=6.00000005; ZF=6.00000009; ZB=6.00000000; ZP=6.00000000; ZH=6.00000000; ZU=6.00000002; MB=3.00014515; XFM=3.00000012; UTC=2017-03-04 06:45:15 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 17030406-0045-0000-0000-000006E57D81 Message-Id: <20170304001715.GA30506@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2017-03-04_04:,, signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=0 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1702020001 definitions=main-1703040052 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, Mar 03, 2017 at 11:30:49AM -0800, Tejun Heo wrote: > Hello, Paul. > > Sorry about the long delay. Travel + sickness. Just starting to > catch up with things now. No problem, I have been distracted by other projects in any case. ;-) > On Mon, Feb 13, 2017 at 04:16:00PM -0800, Paul E. McKenney wrote: > > Thank you for the information! So if I am to continue using workqueues > > for expedited RCU grace periods, I believe that need to do the following: > > > > 1. Use alloc_workqueue() to create my own WQ_MEM_RECLAIM > > workqueue. > > This is only necessary if RCU can be in a dependency chain in the > memory reclaim path - e.g. somebody doing synchronize_expedited_rcu() > in some obscure writeback path; however, given how wildly RCU is used, > I don't think this is a bad idea. The only extra overhead which comes > from it is memory used for an extra workqueue and a rescuer thread > after all. I suspect that SLAB_DESTROY_BY_RCU qualifies -- such a slab allocator could have a great many slabs waiting for an RCU grace period. > > 2. Rework my workqueue handler to avoid blocking waiting for > > the expedited grace period to complete. I should be able > > to do a small number of timed wait, but if I actually > > wait for the grace period to complete, I might end up > > hogging the reserved items. (Or does my workqueue supply > > them for me? If so, so much the better!) > > So, what the dedicated workqueue w/ WQ_MEMRECLAIM guarantees is that > there will always be at least one worker thread which is executing > work items from the workqueue. > > As long as your workqueue usage guarantees forward progress - that is, > as long as one work item in the workqueue won't block indefinitely on > another work item on the same workqueue, you shouldn't need to reword > the workqueue handler. > > If there can be dependency chain among work items of the same > WQ_MEMRECLAIM workqueue, often the best approach is breaking up the > chain and putting them into their own WQ_MEMRECLAIM workqueues. Thank you, good to know! > > 3. Concurrency would not be a problem -- there can be no more > > four work elements in flight across both possible flavors > > of expedited grace periods. > > You usually don't have to worry about concurrency all that much with > workqueues. They'll provide the maximum the system can as long as > that's possible. > > If the four work items can depend on each other, it'd be best to put > them in separate workqueues. If not, there's nothing to worry about. For a given pair, one can be acquiring a mutext that the other holds. Ah, so if only one task was running, that would fail to make forward progress. That said, splitting would be a bit weird in this case, because alternating grace periods for a given RCU flavor would need to schedule work on a different workqueue. Might be easier to do mutex_trylock() and reschedule... And thank you for the info, very helpful! Thanx, Paul