From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754022AbcHXTuW (ORCPT ); Wed, 24 Aug 2016 15:50:22 -0400 Received: from mail-cys01nam02on0131.outbound.protection.outlook.com ([104.47.37.131]:24224 "EHLO NAM02-CY1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1752725AbcHXTuT (ORCPT ); Wed, 24 Aug 2016 15:50:19 -0400 Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=waiman.long@hpe.com; Message-ID: <57BDFA72.4050700@hpe.com> Date: Wed, 24 Aug 2016 15:50:10 -0400 From: Waiman Long User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0.12) Gecko/20130109 Thunderbird/10.0.12 MIME-Version: 1.0 To: Peter Zijlstra CC: Linus Torvalds , Jason Low , Ding Tianhong , Thomas Gleixner , Will Deacon , Ingo Molnar , Imre Deak , Linux Kernel Mailing List , Davidlohr Bueso , Tim Chen , Terry Rudd , "Paul E. McKenney" , Jason Low Subject: Re: [RFC][PATCH 3/3] locking/mutex: Add lock handoff to avoid starvation References: <20160823124617.015645861@infradead.org> <20160823124856.898171453@infradead.org> <57BCA869.1050501@hpe.com> <20160823203204.GU10153@twins.programming.kicks-ass.net> In-Reply-To: <20160823203204.GU10153@twins.programming.kicks-ass.net> Content-Type: text/plain; charset="ISO-8859-1"; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [72.71.243.96] X-ClientProxiedBy: CY1PR14CA0010.namprd14.prod.outlook.com (10.163.13.148) To CS1PR84MB0311.NAMPRD84.PROD.OUTLOOK.COM (10.162.190.29) X-MS-Office365-Filtering-Correlation-Id: a00c1fdc-d1ba-4293-451f-08d3cc57df06 X-Microsoft-Exchange-Diagnostics: 1;CS1PR84MB0311;2:iUi5LFgjdsNIqeXRiB9njJSUW8mq4xVxuHrvM1jRrKOmnzQr3g9C0bNHAZj9kEdMPpNPrUeV7v5QKy4ix06EiaPIFTzMFLk/oPy9IedxNMoJR49/wtzgw7zHWtcDuo5FCVOmv46pkyJNymBbpROhG5JMrqSHURJoH+hcptr865NXlP/OPpu83pANPt1IvvqF;3:5+GbE12LslMa37u0GnPvlA2uMJjFBDlfpFPHWvYY2FlgwSchzetV4emCS5FF3aahYbPxXC8p1JUKqV9lJY/hOLT0//4WfxzZoU133u2+cssicoOknw5FFVYdxfPXk5IN;25:ZKAk/wphO8B/BW6cjIeXIVc6k00DmqXTIxKwRhXmPcW1VNOhR0TU+YOkX+aNsVzVF1337/t7RAnuLrgbX56pgYHIiCoSsdb7P9K3L2luNA17pg/f6VtAXnafxAcK14Lyi9DHV2fcdUi7y8jD8ztWze6MMBYYUHG1avg3x2Ui6mg3O2a0ZDancblQ1R0MuVKZmcmTp6VHxYf+s1xUbEoOA3Z/YB1mg2UDbQ2ZKFKjEwuA8Yn8Ez2FGFrWKUR3Zp+6hdpUxhWW9nh3xKoXR8Fwcg0P713sziF2VzZ+Lou8zNG4tRZLhVKus7SqjV4n7doDTjmlmK1in7n4uLtEcVGbxdclX9UAAy4yQASS2GcYkAo6SCdAE2XbpKZnQI5xHOhMD+C93rNEipmeIVO9eZrX10fCHEr2LBziRQAEz+24kN8= X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:CS1PR84MB0311; X-Microsoft-Exchange-Diagnostics: 1;CS1PR84MB0311;31:6G/6d6yuZuIhfB0ojntY2Jwau2u4hIGqchmjcyHZPCkghapxLvGIvKScSs5ztoqaju/5rK7VM69HBjug1yscr0zbnoXaKW1kIGzXhQRMp5TGwvs4ououDlvy9x6Q0xMWHuzXDXuNL3HO25pC3OeMe1Zs6FFxCLo+9ZNB8TIp+peHHmaWOzuOwTby21aMr7Uw8ARhsEF6lVgAdTgD/ckU9o4DXBagfjtU3YYEu7sTs1o=;20:xULk3zZXAuHEckMKqKTFP+3TJPyjouwL3jKT7w9XEs3b3NAKLgi8xp9mS7wM+0OfM88Q+hbAM00E8W/xCQoxREXJb5s/y7fVYwfnbwLY+vok6bOlYLSEAkr/Cgeb8+ERsWVX1RbWpM/HklSbiau6LpupHqWSxDKCufkyh8TRwEY3d5pMDna46RKx4wh9vX2falOaecIXYF3PSkyb7Bdf3Mle+gJnsC0Ocbtkz7uuYJ8156cBtO5st/9Rj5Ozs/jmKgCinM1oWFjcAPt7JR1W8cDIxiQvGd2fpn1tDaj5UbkAys+F4jiR1uX42KqvKCKb+EcWOTbnpMczynuUgj3LjNpLf8Er6oz08hFx6zXCAMuWlEayzDINapqkdvp9dm4AZ7AtO6IEPx6KCDOWopkFv5VIWwVlQGsJRLWlDU3JXrqmb44T8e5QsWw/8Oh7XIn9y7DRWZ1XIDYB5xPIB+C3lb/38M0d2puvt7FbN51uWtIQgbLMmaWh19rHUemF8CaP X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(6040176)(601004)(2401047)(8121501046)(5005006)(3002001)(10201501046)(6055026);SRVR:CS1PR84MB0311;BCL:0;PCL:0;RULEID:;SRVR:CS1PR84MB0311; X-Microsoft-Exchange-Diagnostics: 1;CS1PR84MB0311;4:qNNycpyc/lVfr7VZ54/GfvLRUU1m3P7qIMTeU90tDUADyYjPfgJEQjCmzQMqA8UwBrMXEgFOnDl8wyOHWSoTk8oYiBrgC2wh5jvI0u8NuOREs3MTAOFvK3gatc1A1jc79Ns6RWp2KdM/uYalUPhzSul78ndzgNbFmhjUiBfszAngCClLQyTHWnp3U3aWjz30hk1pllBusRI1cLCECy+RjeO9LHvnJ2QnoO6MKOtQj2gFKm1bwzEOvhyuN2O6pnJ7Z5v/m2Kzl8bFHFJwCP1Ot2mzlkx35sYj3PQ2D/m0THFFEXx2c0FRMAeiIEyVtR8hpVR2WVF838OuDSiOPTzX+DebzVXY0nRVmkuyX10N5ey3PuGxB2VJMyE9pTcOwJKU1Ija1QuBOq0O06iuzNVmPQ== X-Forefront-PRVS: 0044C17179 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10019020)(4630300001)(6049001)(6009001)(7916002)(199003)(189002)(24454002)(377454003)(8676002)(23756003)(4001350100001)(68736007)(586003)(230700001)(33656002)(189998001)(7846002)(97736004)(64126003)(7736002)(81156014)(59896002)(36756003)(65816999)(81166006)(83506001)(101416001)(2950100001)(110136002)(76176999)(54356999)(8666005)(2906002)(4326007)(305945005)(77096005)(87266999)(50466002)(42186005)(106356001)(47776003)(7416002)(80316001)(50986999)(65806001)(66066001)(65956001)(3846002)(5660300001)(117156001)(6116002)(105586002)(92566002)(86362001)(93886004)(7059030);DIR:OUT;SFP:1102;SCL:1;SRVR:CS1PR84MB0311;H:[192.168.142.191];FPR:;SPF:None;PTR:InfoNoRecords;A:1;MX:1;LANG:en; X-Microsoft-Exchange-Diagnostics: =?iso-8859-1?Q?1;CS1PR84MB0311;23:oQhf3rM0gfr0KMWmXjXiFL8/Q1YWzoyUE3UJ1Pj?= =?iso-8859-1?Q?qv6aXnQHn1miOj+GKSJUK7l9gaDfQr/LKAarX48g3YCqSfxGQ9sXNZKi/T?= =?iso-8859-1?Q?79Lj/Mt+MmcijaC51TBRANKtBUAx7LPyhfBPxiul331u4aq6MD5wYRe7C7?= =?iso-8859-1?Q?in+VfOTz8VTDxfPr2NsvWLyhfM+xTltJNo0E95o7suo1N/yuB4NkZvqZdG?= =?iso-8859-1?Q?e2+KoAMSS8UspmbkoxZdOK+/Yv3SVAIAM9rb1JrBhGr5WuCxxm/n55KLO+?= =?iso-8859-1?Q?d+LoRsWtsdG4/6Dls1+Na4q7CbVsKE66F/RGF/zwzEjwSF26ho5Fg1pi7P?= =?iso-8859-1?Q?ceKWfhKA6pO3VfIkcbEEwBk60BI09VlQhTYGfh9Oad8iOcJQ2cDKBPRcHB?= =?iso-8859-1?Q?vVQdOQWFvZTTKgNozzM0dadZznpjE6aLCCHwVXjTo56bpGp81xGJl3oZAQ?= =?iso-8859-1?Q?9eNTIMjHJlDQ+o2EimCCORTIEcSX9PW12w7PTfA8OlUO9NW6dsnz9fWbBM?= =?iso-8859-1?Q?c4xb5Kn/lnzYERH1aqXgX74v/sWRHpGluaLvh/zvtFWDhKX4eLcyn4iRAq?= =?iso-8859-1?Q?G/FcrbPAcG9He8aqkKE5XjbYZuQaoql79o5RsUE//cGHwKDkkJTLYzNNXG?= =?iso-8859-1?Q?Rxqm8OyAxfgthP8kHCff24RJi7KUBh7X0kh9PeQSLTum6tPP73NJuAEHTn?= =?iso-8859-1?Q?X5CaABh5CNxMlaTgWQaOMpz7gmwdxQ1Kycyju8QtYcyYD6fdjTrViStbYS?= =?iso-8859-1?Q?fj8+t0AFlyFnVfFjRwLp2y8mUiAv7DLH8HKKGEUiyTrvRQr+bPgddlypdG?= =?iso-8859-1?Q?/7w9nyhBdER/hVznEXyKDd3kJ30FKNA3LgCVQBJq04PjzTeLJIONhZrFk+?= =?iso-8859-1?Q?otQsoAHjephhYLQixo0bANsrLpdEW+YQfPLIxpX5UJ5dcSbcijzd1u6Uur?= =?iso-8859-1?Q?xnzDb9pzRU0tph8PTls1eWu6TM2gz1ZwY6psjWQPd9z/E3b/uBw2CJ5wa8?= =?iso-8859-1?Q?a97UTwArn+IaNT9cG4KU6pCsAt/wgV2xEhdA/Lmaxx8OPG7w7V7H/OZ46Q?= =?iso-8859-1?Q?jqf1KqtM6p7QoUtLRmjk5fN5cZsLzFn0ISketmWPD4wNvg6noUGlppiA+I?= =?iso-8859-1?Q?CID0JTo2/sthTABBp43A9yXL57mGdnOl0MwknWI4P4Qob/zj1Yl/2K0Jv4?= =?iso-8859-1?Q?WEJKPKubXj0fk8gMjATUYFrlaiix2MTds6lzRlZbGXxYkCtaUpUUk8dVlf?= =?iso-8859-1?Q?NrJTQnqTOREjaICoqqWZWCNc4QKCYJAlLKWv0SCZjDo2Kr/WKFwN/EZnJQ?= =?iso-8859-1?Q?LRh3Q13n60AcLOQJdbig4EyifiHKu8IUdw31kIr9PUKhEA94zL88LHaoJS?= =?iso-8859-1?Q?psPVe6UzA6ZnNMFZqIDzYw/7qyDf4WK/tAtzDXxNGUOFfPlskBA=3D=3D?= X-Microsoft-Exchange-Diagnostics: 1;CS1PR84MB0311;6:0C1C0cEnfYArokrPM/0B+CvVC3iK+G6FIQRQTrXHbROouNlTREgQhfQfMkoMcAFn+zB2Oxi1GaCQ3krpr1W6JLXNhobPYZkZktVNvnC3QcWCQ0u6CQxB8PT6yE79Cc64hfJ2fxkZ4189djr3pvoT+5oxBtYFVi2xZigCWO0O782h+UnsidwnDDfALTtSThD0FOwVzuSsN2kp83t2JbXSp4pqf8FtEuGtCmuB7R4TobUIr+krN03lQN3i0xcjbpUb4Gx4I4Lg5pNOQvuoFcRd7Z1P47YA9aJmppWUizfV6dRhPBv397qUUBd4rxmZb+cxPswSg0O9dWcsvj8YrERcQw==;5:YPpVEgGW0dExRC7NktN60UKB7nmCouSsfojupHpqjXY3Dwim4tXkNUeJyGUaEVDNGtkzr7bRLtY8QszcL2Djr4gpzcw75eYP4VyReZunmQfXesPc62HglnvQvIahAKGsigmMIxdWN2pVhlib23TTKA==;24:uNFkZjCNoI7tgxbaHS52firTi+63oLVJdVwbdVX2Y94iHCXgVgBz9a06M3xltCTRdAouRfRt5lS2aWdmTiQt7Fkudu0+vpQzQIVkQVgcySE=;7:sEIlQ3j1Jv78EpgBQVbkBtaY6H8/c6gFe6LBy07wh2AzZFR3hxrqeZOUh7tVh1fb3wkHSgGzKp0pwPHMJNvWqt2NRYby5eRZaXY3cwJnnWdDD/qJm809MFNOp7Ud+7osjR7vmWp0OJfSLYpFZCnmxh00/Av3VCAgl17kAAWQoJnZxjPkEDpitdpsSoCTChrR+lEe4fnoaJJKWkNrouBCzhyPMGrOLbNbyFjq86ODUKNvh2uAAh0Y5irXOptsN4L+ SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: hpe.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Aug 2016 19:50:15.2372 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: CS1PR84MB0311 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 08/23/2016 04:32 PM, Peter Zijlstra wrote: > On Tue, Aug 23, 2016 at 03:47:53PM -0400, Waiman Long wrote: >> On 08/23/2016 08:46 AM, Peter Zijlstra wrote: >>> N >>> @@ -573,8 +600,14 @@ __mutex_lock_common(struct mutex *lock, >>> schedule_preempt_disabled(); >>> spin_lock_mutex(&lock->wait_lock, flags); >>> >>> + if (__mutex_owner(lock) == current) >>> + break; >>> + >>> if (__mutex_trylock(lock)) >>> break; >>> + >>> + if (__mutex_waiter_is_first(lock,&waiter)) >>> + __mutex_set_flag(lock, MUTEX_FLAG_HANDOFF); >>> } >>> __set_task_state(task, TASK_RUNNING); >>> >>> >> You may want to think about doing some spinning while the owner is active >> instead of going back to sleep again here. > For sure; I just didn't bother pulling in your patches. I didn't want to > sink in more time in case people really hated on 1/3 ;-) I think there is race in how the handoff is being done. CPU 0 CPU 1 CPU 2 ----- ----- ----- __mutex_lock_common: mutex_optimistic_spin: __mutex_trylock() mutex_unlock: if (owner& MUTEX_FLAG_HANDOFF) owner&= 0x3; __mutex_trylock(); owner = CPU2; __mutex_set_flag(lock, MUTEX_FLAG_HANDOFF) __mutex_unlock_slowpath: __mutex_handoff: owner = CPU0; Now both CPUs 1 and 2 think they have the lock. One way to fix that is to check if the owner is still the original lock holder (CPU 0) before doing the handoff, like: --- a/kernel/locking/mutex.c +++ b/kernel/locking/mutex.c @@ -97,6 +97,8 @@ static void __mutex_handoff(struct mutex *lock, struct task_st for (;;) { unsigned long old, new; + if ((owner & ~MUTEX_FLAG_ALL) != current) + break; new = (owner & MUTEX_FLAG_WAITERS); new |= (unsigned long)task; I also think that the MUTEX_FLAG_HANDOFF bit needs to be cleared if the list is empty. @@ -614,7 +633,7 @@ __mutex_lock_common(struct mutex *lock, long state, unsigned mutex_remove_waiter(lock, &waiter, task); /* set it to 0 if there are no waiters left: */ if (likely(list_empty(&lock->wait_list))) - __mutex_clear_flag(lock, MUTEX_FLAG_WAITERS); + __mutex_clear_flag(lock, MUTEX_FLAG_WAITERS|MUTEX_FLAG_HANDOFF); Or we should try to reset the handoff bit after the while loop exit if the bit is still set. Cheers, Longman