From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756679AbcBDBhU (ORCPT ); Wed, 3 Feb 2016 20:37:20 -0500 Received: from g1t6225.austin.hp.com ([15.73.96.126]:42973 "EHLO g1t6225.austin.hp.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752260AbcBDBhT (ORCPT ); Wed, 3 Feb 2016 20:37:19 -0500 Message-ID: <1454549736.2072.15.camel@j-VirtualBox> Subject: Re: [PATCH] locking/mutex: Avoid spinner vs waiter starvation From: Jason Low To: Peter Zijlstra Cc: Ding Tianhong , Waiman Long , Ingo Molnar , "linux-kernel@vger.kernel.org" , Davidlohr Bueso , Linus Torvalds , "Paul E. McKenney" , Thomas Gleixner , Will Deacon , Tim Chen , Waiman Long , jason.low2@hpe.com Date: Wed, 03 Feb 2016 17:35:36 -0800 In-Reply-To: <20160201100824.GO6357@twins.programming.kicks-ass.net> References: <56A1638A.7050202@hpe.com> <20160122085422.GO6357@twins.programming.kicks-ass.net> <1453458019.9727.8.camel@j-VirtualBox> <20160122105312.GQ6357@twins.programming.kicks-ass.net> <20160122105652.GE6375@twins.programming.kicks-ass.net> <20160122110653.GF6375@twins.programming.kicks-ass.net> <56A235DC.2060900@hpe.com> <56A48566.9040206@huawei.com> <20160129095347.GA6356@twins.programming.kicks-ass.net> <56AC0F74.2040808@huawei.com> <20160201100824.GO6357@twins.programming.kicks-ass.net> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.10.4-0ubuntu2 Mime-Version: 1.0 Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, 2016-02-01 at 11:08 +0100, Peter Zijlstra wrote: > On Sat, Jan 30, 2016 at 09:18:44AM +0800, Ding Tianhong wrote: > > On 2016/1/29 17:53, Peter Zijlstra wrote: > > > On Sun, Jan 24, 2016 at 04:03:50PM +0800, Ding Tianhong wrote: > > > > > >> looks good to me, I will try this solution and report the result, thanks everyone. > > > > > > Did you get a change to run with this? > > > > > > . > > > > > > > I backport this patch to 3.10 lts kernel, and didn't change any logic, > > Till now, the patch works fine to me, and no need to change anything, > > So I think this patch is no problem, could you formal release this > > patch to the latest kernel? :) > > Thanks for testing, I've queued the below patch. > > --- > Subject: locking/mutex: Avoid spinner vs waiter starvation > From: Peter Zijlstra > Date: Fri, 22 Jan 2016 12:06:53 +0100 > > Ding Tianhong reported that under his load the optimistic spinners > would totally starve a task that ended up on the wait list. > > Fix this by ensuring the top waiter also partakes in the optimistic > spin queue. > > There are a few subtle differences between the assumed state of > regular optimistic spinners and those already on the wait list, which > result in the @acquired complication of the acquire path. > > Most notable are: > > - waiters are on the wait list and need to be taken off > - mutex_optimistic_spin() sets the lock->count to 0 on acquire > even though there might be more tasks on the wait list. > > Cc: Jason Low > Cc: Ingo Molnar > Cc: Tim Chen > Cc: Linus Torvalds > Cc: Waiman Long > Cc: Thomas Gleixner > Cc: "Paul E. McKenney" > Cc: Davidlohr Bueso > Cc: Will Deacon > Reported-by: Ding Tianhong > Tested-by: Ding Tianhong > Tested-by: "Huang, Ying" > Suggested-by: Waiman Long > Signed-off-by: Peter Zijlstra (Intel) > Link: http://lkml.kernel.org/r/20160122110653.GF6375@twins.programming.kicks-ass.net I've done some testing with this patch with some of the AIM7 workloads and found that this reduced throughput by about 10%. The reduction in throughput is expected since spinning as a waiter is less efficient. Another observation I made is that the top waiter spinners would often times require needing to reschedule before being able to acquire the lock from spinning when there was high contention. A waiter can go into the cycle of spin -> reschedule -> spin -> reschedule. So although the chance of starvation is reduced, this patch doesn't fully address the issue of waiter starvation. Jason