From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S965491AbcBCWH5 (ORCPT ); Wed, 3 Feb 2016 17:07:57 -0500 Received: from g1t6223.austin.hp.com ([15.73.96.124]:58788 "EHLO g1t6223.austin.hp.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S965324AbcBCWH4 (ORCPT ); Wed, 3 Feb 2016 17:07:56 -0500 Message-ID: <56B27A37.6070105@hpe.com> Date: Wed, 03 Feb 2016 17:07:51 -0500 From: Waiman Long User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0.12) Gecko/20130109 Thunderbird/10.0.12 MIME-Version: 1.0 To: Davidlohr Bueso CC: Peter Zijlstra , Ding Tianhong , Jason Low , Ingo Molnar , "linux-kernel@vger.kernel.org" , Linus Torvalds , "Paul E. McKenney" , Thomas Gleixner , Will Deacon , Tim Chen , Waiman Long Subject: Re: [PATCH] locking/mutex: Avoid spinner vs waiter starvation References: <20160122085422.GO6357@twins.programming.kicks-ass.net> <1453458019.9727.8.camel@j-VirtualBox> <20160122105312.GQ6357@twins.programming.kicks-ass.net> <20160122105652.GE6375@twins.programming.kicks-ass.net> <20160122110653.GF6375@twins.programming.kicks-ass.net> <56A235DC.2060900@hpe.com> <56A48566.9040206@huawei.com> <20160129095347.GA6356@twins.programming.kicks-ass.net> <56AC0F74.2040808@huawei.com> <20160201100824.GO6357@twins.programming.kicks-ass.net> <20160202211906.GD16147@linux-uzut.site> In-Reply-To: <20160202211906.GD16147@linux-uzut.site> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 02/02/2016 04:19 PM, Davidlohr Bueso wrote: > On Mon, 01 Feb 2016, Peter Zijlstra wrote: > >> Subject: locking/mutex: Avoid spinner vs waiter starvation >> From: Peter Zijlstra >> Date: Fri, 22 Jan 2016 12:06:53 +0100 >> >> Ding Tianhong reported that under his load the optimistic spinners >> would totally starve a task that ended up on the wait list. >> >> Fix this by ensuring the top waiter also partakes in the optimistic >> spin queue. >> >> There are a few subtle differences between the assumed state of >> regular optimistic spinners and those already on the wait list, which >> result in the @acquired complication of the acquire path. >> >> Most notable are: >> >> - waiters are on the wait list and need to be taken off >> - mutex_optimistic_spin() sets the lock->count to 0 on acquire >> even though there might be more tasks on the wait list. > > Right, the main impact I see with these complications are that the > window of when a waiter takes the lock via spinning and then acquires > the wait_lock to remove itself from the list, will allow an unlock > thread to set the lock as available in the fastpath which could in > turn allow a third thread the steal the lock. With high contention, > this window will be come obviously larger as we contend for the > wait_lock. > > CPU-0 CPU-1 CPU-3 > __mutex_lock_common mutex_optimistic_spin > (->count now 0) > __mutex_fastpath_unlock > (->count now 1) __mutex_fastpath_lock > (stolen) > > spin_lock_mutex(&lock->wait_lock, flags); > > But we've always been bad when it comes to counter and waiters. > > Thanks, > Davidlohr I don't quite get how that can happen. CPU0 cannot change the count to 0 unless CPU1, the lock holder, does the unlock first. Once CPU0 sees a count of 1 and change it to 0, it is the lock holder and there can be no other CPU that can do the unlock. Cheers, Longman