From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751743AbcGRUul (ORCPT ); Mon, 18 Jul 2016 16:50:41 -0400 Received: from mail-bl2nam02on0125.outbound.protection.outlook.com ([104.47.38.125]:61460 "EHLO NAM02-BL2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1751013AbcGRUuh (ORCPT ); Mon, 18 Jul 2016 16:50:37 -0400 Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=waiman.long@hpe.com; Message-ID: <578D4110.70706@hpe.com> Date: Mon, 18 Jul 2016 16:50:24 -0400 From: Waiman Long User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0.12) Gecko/20130109 Thunderbird/10.0.12 MIME-Version: 1.0 To: Jason Low CC: Peter Zijlstra , Ingo Molnar , , Davidlohr Bueso , Subject: Re: [PATCH v3 0/3] locking/mutex: Enable optimistic spinning of lock waiter References: <1458668804-10138-1-git-send-email-Waiman.Long@hpe.com> <1468864758.2367.27.camel@j-VirtualBox> In-Reply-To: <1468864758.2367.27.camel@j-VirtualBox> Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [72.71.243.105] X-ClientProxiedBy: BY1PR14CA0018.namprd14.prod.outlook.com (10.161.91.28) To CS1PR84MB0310.NAMPRD84.PROD.OUTLOOK.COM (10.162.190.28) X-MS-Office365-Filtering-Correlation-Id: d6a9e1b8-82cc-4b49-df7b-08d3af4d2a59 X-Microsoft-Exchange-Diagnostics: 1;CS1PR84MB0310;2:/vh0qkuhqUnQ6lCgu1GByp+5Vm8g2FQxOFlgXL2xcJmSuYO7LdASMk1o2NsbsD9zgs+KRZc3Bp9pOqQIb78CI3ycGfIDRtewoy+NfNGP0hSXQGSzq5X8lbhEHQuXFI78s8O8RzyyMe4+x0wtoG8JbkRzBTb/I3EEJeOTH2McoJWv7wW0blp7rCaNg/t0Mekt;3:l+USqKBKqv1yEgqSSx0Fjd/cNkBgBX1P94g/Aj/i+q48zD5HkkiFQgJ0vT/MzutKXscDFSaJyWfConzzOBud6QJ1BkqHEdPuJmLC9xVd9LT64TzY9/1dGsb8/TujZ95A X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:CS1PR84MB0310; X-Microsoft-Exchange-Diagnostics: 1;CS1PR84MB0310;25:FTqEvmrpeyQ9uPLl4x1s7GlCLvLCvUNvpVICYJ2KGZFy41hq5W/0n4RnhexXxahbHQ4fdBjhkElgThqDzcAmC7Kl9wrTzy9SMG6ppb0fUEgN2jZ3O7DfPq81MhKBwSFdyV7Ixufyv/Vn2mjWsqLT7OMnaNeWY7I/nLP+2bOKQ9gLeWZAU8I0BDEKkbvQlMqKeNl+yD6F8/atDIo61Cg47ygss1ePWU2ITWiE+EJ9DjjjA4s7hKItptmSCcMtfL7aOErxxcRGwfTz6fJ8F/YIuQBe9q4HO4ON4kMQFgcSvqSDfGt8xLAC7+VdWW8zMh77acwV8Ufhh1JTJigZNooCse+/Vobv1LYjf8HawnNe3S9Y1gPjIePVx/WwQvn7V8MyvUnWvUIjHF3rTTdmrt+7O7L6cIDuuYB529TNfpexUb9hyWvUOh89OZbyszNHMszDoOwirMWlFhTXgddi1OB9PQKQ5r3uG0Q5gEgngXTOislir/n1pSjc02cc9cVZ9nFdLsU9usnjKWH6aY8gUU07vHN7zQkWAbtCqG/oZUPjISWr6ci36Enw009C3XRzT8TMjVHdDew75xX2xBS0Ku4Mz9KQYIrh0w1F0++kKz6IpFSrf3gGQE9fRQVyFEFI2nXt0EiRUBWGdTej6NP701T4nThzDzmBRJWchoVShjkaBVaStBWVryrijd1C2bQSr38iB019iyyTpElDoFfFxfE/pA==;31:5l9Q1SSpY6vo9vio8OLVNeNOY8AfzRBLSnJ+ZBzrcYdJFNNUEbJ/ijeBjghLpd7HFgCh0zbwNoA5jiYY8be9xjGf80ER8ymh3RLU4v6luaWLwqFGReXjJpFFGtYP3HWD45OUtH8oMI3GysDcqDTl7LUKcS5AERHrUD9E6pr0fLkCS/bg9zMhOU4lDNqgUnhqAUGaj771sN6tXBuuCqmvIA== X-Microsoft-Exchange-Diagnostics: 1;CS1PR84MB0310;20:ItZa8W5kNVyIlsT4apUevUnWSK9clZBQ2Xj5APGH78ubYdabeS502Nq6uJsU95JLlT6cN+PQ07GnLiX809EEkHlhZE9uX0Nf1WE291gql41OqaShfN5iog+t7T8f2PiNbz7vAlQxZ7S5Li0l2UywLB6ynlI4CYewGPicvccqk4o6lwAHBM6i8jsdwboefpT93xyhT53SUAoOaMjBzrI7PmgtsaimT/ly9DlqJT8lFr3Z4GVjbn1vP+xM6t2Yd0oxiAY94/dBhQ7HLWKKXRJT97HPI3Oz0s5gep+ehznNBQ/kKaxSFrYeN9acdvC7oebYYdp4hvWclFbPSOL0Z9NwYBp2hQi+m8uTR1Oaof9DFxW1486iiWhwOOcdWd1lCnCNAgn2eSHf55xmFw0SRWbLWiuCa2/QV6uqUO/ws92UbOmQdCV3AVByA+lfjv23NXsPOgZleieqSVocVs9mKxqfa+abUeBKYSpQSF1bwydmiuixAwlnHUAW+5OgMg1Wr7SS;4:db8FK2gwIKdGjJXf3CK+Dl310MfDY1NgMb5c1DNS4i7LziYrFNX9zr+nr5n/icAnMyYYBVWvBOCIvLNn0UY6Lvstjt/3Ho2ZU+tqI3GjWR/sYbxMHanpAeKSbP4ghWF1mE2eYD/ALo5Wdvt9QLOHCe3D6z81bc9oSIb5++JNQ4AcIDDeKLFIqGtrTtUw7WjFdnyPRRBb4JSUOI2gebS8E3UyvEEuQmSAB65dCi7pcb+3tNU6qAkoJCXUWk6q27Nu4lM7SAXPKlNESsuDzej+bmBLjRJmqZ6WAJwOr7+PRuVqzuX0qidoq86XDIg7pGmjWjmEolHUHvO8GJtR8hKY5phDDZaxRwOBoA+yKXNo4BTDpaVdILXiviNtL2eVOCKLH1a3LwT5cZ8gW0lZbQY8CgzQsjljbCPGYYtFOQp3Px21fZZoUYO/3exX9y9kHvDC X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(158342451672863); X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(601004)(2401047)(8121501046)(5005006)(10201501046)(3002001)(6055026);SRVR:CS1PR84MB0310;BCL:0;PCL:0;RULEID:;SRVR:CS1PR84MB0310; X-Forefront-PRVS: 00073DB75F X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10019020)(4630300001)(6009001)(6049001)(7916002)(377454003)(199003)(377424004)(189002)(24454002)(65956001)(65806001)(23676002)(66066001)(76176999)(54356999)(47776003)(65816999)(50986999)(305945005)(230700001)(3846002)(36756003)(2906002)(7846002)(4326007)(8676002)(68736007)(8666005)(2950100001)(110136002)(42186005)(189998001)(117156001)(6116002)(86362001)(81166006)(83506001)(77096005)(64126003)(33656002)(97736004)(105586002)(92566002)(50466002)(4001350100001)(106356001)(586003)(7736002)(101416001)(81156014)(7059030);DIR:OUT;SFP:1102;SCL:1;SRVR:CS1PR84MB0310;H:[192.168.142.171];FPR:;SPF:None;PTR:InfoNoRecords;A:1;MX:1;LANG:en; X-Microsoft-Exchange-Diagnostics: =?utf-8?B?MTtDUzFQUjg0TUIwMzEwOzIzOlRiT3Y2TWRqR1kvd3NzQkdoQjBGenNydFox?= =?utf-8?B?VHpmY3J2QWpGS3FTYTU3OVNLcHlLVkdpQ3FQUnFLTTR6NWpsRVdwMlZEMCsr?= =?utf-8?B?K1VxUlUxaVJybkpSZnpvTXRwV0ZteFRjYVhndWRpZDcrS01DUUxJUFNONldR?= =?utf-8?B?aG1JU3dTTFF4K0ZSQWJqd2o1SnhQSENLV2xCVExkVkdQVXV4ZCtJeG85VjlT?= =?utf-8?B?OXpJa0UwZzNGOFozcVNUa3ZyRlR4Y2VsbkhzRmFZeWNUUkxoQXBnQXg1Snk0?= =?utf-8?B?eC8xdXR2V3MzRkcrcE1QdmQrTkM1TUsyMlFqK1hPY2xnNG1Mb2FmWUJuZE53?= =?utf-8?B?ekhWaXR3NWw2YjlWMXRSNllyT0ZSVGY3VUhXNDZGMnhHTVBZby8yWncrS1NE?= =?utf-8?B?ZTcwcDdUZVNGNzR5Ty90bFJiL0d4SGgyUFFzYUUvRG9EVGJ6bytDTDk2MGp3?= =?utf-8?B?RjhNbDhyZnlHWkdTQUV0dXNmM1hTNEJQdVRqSWJxRXZ0NndoazQ0bVZicDA0?= =?utf-8?B?VnJ5VHFMNmhxWGl4SmF0bTBGMUZ3aHVBMlA1MjlHMUFJbERYa2Z3TUFqMWJQ?= =?utf-8?B?MkpoZm11Y0ZSc244UE9pSFdhK0xKYXhKOU5FZE01bklWM1drUXE3VDhydkhr?= =?utf-8?B?OVVCWFhuUHowK3R1eUl1Z243MWZqbENaUnpndGlLd3BEd08rMjlzRk1RaklD?= =?utf-8?B?OVVjbXpFUXFJWTYrcU4xQk5BMjR6YUNmUktuTmZ6YSt5ZTZPZENzQ2QrM0VI?= =?utf-8?B?dG8yRU83bzVpcFJDR0hGQmpQWW01dzVSOUJpaG94aVlHWGk3SG4wT0xLT2py?= =?utf-8?B?ZXJTRXVhWHVlbmgyUkpicVRGbGNiSlB3NjZURFB5aXdHRFdDd0V1K3R0N0pi?= =?utf-8?B?QXgrWlJkK2oxMTN3aEFrYUY1NG1EVU1iNTVYOWZvRnQwaEYxdWI2aWI2RW11?= =?utf-8?B?cW1LcXhzZUJSNFpVYVNUU0xQM3Job2JobmlxQm0xT2V2akpSa0k5RHpFK3hF?= =?utf-8?B?bzRUMklmYkIwWmt6MjluTTcrUXFlSWgraHdqbFRXaXJZSUIyeWJ4d1JwaG1n?= =?utf-8?B?WnlrT2xrWEJObkMzWkZ3aFBLZFNFa09mZ0cyaUY5MzFEbkZmVzdDNFJ2U055?= =?utf-8?B?N0lwY1paNkxzeW0zZWFLdUFwK212aE5adFJNRFZuSDhjSnpCckd1cnNOK1k0?= =?utf-8?B?RFczTkNodW5YQThjTDVodlhUenE3V0R0SmdBQkVuell2SlE1Z0RCRTlycDBl?= =?utf-8?B?ZTVaeUJlU2VmNjc1bjBNRWV5SWM0NXV3OVdNWEtDWEpsQzZZd24vVncwa0ZR?= =?utf-8?B?NGZvelFmSGVlRWhiSC9OdUg2bjhqaVl4VmFkc0xsK0NyNndNU2FJRENRamVx?= =?utf-8?B?Ty9lNDFiQlZHKzJIMllvd212UGlqMnNrNm5XVFpiVmhJL1VOKzQ1L0o2Vnlr?= =?utf-8?B?aHRYTlNyTitDaGV3a2tSdU84TmNrd2lBeHFBVjRta1FxWmFFWlU1d0JKMk9u?= =?utf-8?B?eWhLeDJtU0dleGJaQmZIdHdKRGxWNEdXbkhPQnVsNHlXemZwYnVIdWNGbFRs?= =?utf-8?B?Y0J3Ulc2M0pPK29MNEMwOENBZFl5dTlaNGVQSFdMbEFiYmZycFJ0eU9ZR3N2?= =?utf-8?B?YnZMd3BMRmFqdFFlS1EwQlZ3TkZ3M3JVa1hUS1BqNzJ1NU1FRys3dzU3MzEx?= =?utf-8?Q?Lkl3Es+3GX9r+LrF/w=3D?= X-Microsoft-Exchange-Diagnostics: 1;CS1PR84MB0310;6:Yr47Lt1dma8nCwXgEvqg6Ugzy+r3+H8y1CueGkGWSYE/R+sVf8/qHi3RSnanfy/+OzrG9toGbccw91Id4PRipiQvDutcqe0PGaf3EfYCmb1qnY8aWMyaoT7nzXSbnwnqQdXSXuU1dDUVrSl9W0XH6kmZXpTUCkKtVp8PDxlszZTt7fZvXzkb/FprfMSWNyOelBTVehfZ0LEo4B0W+dTrVEYEdSEDrJzLbcDZul2DGV6Vsufv3eOOZjSBdCZ+8bSQRaIiOsbpcPgXN7Uq2x5/Mzv87uCdYj8jSaB0wplEFZccjzjKZfqRL6BkWWDeKTdlijU5FeGowS5xaRfAAzfBFA==;5:oMTxKlONj86eaGnF352am+C+h0KTiZ6z9T9aqJUrgQSm3Tq68r+vRjPmJo5H/3whQYkOUHSxbdYZJ5LsCJTsBbEOehaG0RfNT9RpruxDnnyppH2m4rp1hboVJqXxy/g/Kj/UiDyoJ35RdgqsHhxISQ==;24:pAt4KPiUoIU4E0sHWdlXsobcVZw9kkof9a7IHTCce8gQQvwpX/i4T4xR9AVCTWla0dgK3fq0zzFWG7COsD+A14kK3sc3GC7/w5JIs3KMbpE=;7:Cf8nGbKQWSOivaGetbEQgAwkGq8j1HIj/aY5Pv6D2Q29CcHALfnqFEGAjzdsC8+7x3BiRVZGBlqCufTVmOhLJ5hUMrAIeSZqRANI/Tcz+Cr0+cvQSnSQYJOneNMhOpeNK6Um/xogPY6hVu78ipNRleobA5jRiYHUiazpWY06vLrh5G2zr2G1wRINIOMKx/l+6F7mwFOcyaQ0dRFQ0K6U9tIhOjZKmB5j8zHnins3hcVXDat1Ms6AYjvgcWpdhXhd SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: hpe.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Jul 2016 20:50:33.0662 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: CS1PR84MB0310 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 07/18/2016 01:59 PM, Jason Low wrote: > On Tue, 2016-03-22 at 13:46 -0400, Waiman Long wrote: >> v2->v3: >> - Remove patch 4 as it is not useful. >> - Allow need_resched() check for waiter& add more comments about >> changes to address issues raised by PeterZ. >> >> v1->v2: >> - Set task state to running before doing optimistic spinning. >> - Add 2 more patches to handle possible missed wakeups and wasteful >> spinning in try_to_wake_up() function. >> >> This patchset is a variant of PeterZ's "locking/mutex: Avoid spinner >> vs waiter starvation" patch. The major difference is that the >> waiter-spinner won't enter into the OSQ used by the spinners. Instead, >> it will spin directly on the lock in parallel with the queue head >> of the OSQ. So there will be a bit more cacheline contention on the >> lock cacheline, but that shouldn't cause noticeable impact on system >> performance. >> >> This patchset tries to address 2 issues with Peter's patch: >> >> 1) Ding Tianhong still find that hanging task could happen in some cases. >> 2) Jason Low found that there was performance regression for some AIM7 >> workloads. >> >> By making the waiter-spinner to spin directly on the mutex, it will >> increase the chance for the waiter-spinner to get the lock instead >> of waiting in the OSQ for its turn. >> >> Patch 1 modifies the mutex_optimistic_spin() function to enable it >> to be called by a waiter-spinner that doesn't need to go into the OSQ. >> >> Patch 2 modifies the mutex locking slowpath to make the waiter call >> mutex_optimistic_spin() to do spinning after being waken up. > Just in case, here was the Waiman's patch that I was referring to for > addressing some of the starvation issues with mutex by allowing the > waiter to do optimistic spinning after being woken up. > >> Patch 3 reverses the sequence of setting task state and changing >> mutex count to -1 to prevent the possibility of missed wakeup. >> >> Patch 4 modifies the wakeup code to abandon the wakeup operation >> while spinning on the on_cpu flag if the task has changed back to a >> non-sleeping state. >> >> My own test on a 4-socket E7-4820 v3 system showed a regression of >> about 4% in the high_systime workload with Peter's patch which this >> new patch effectively eliminates. >> >> Testing on an 8-socket Westmere-EX server, however, has performance >> change from -9% to than +140% on the fserver workload of AIM7 >> depending on how the system was set up. >> >> Waiman Long (3): >> locking/mutex: Add waiter parameter to mutex_optimistic_spin() >> locking/mutex: Enable optimistic spinning of woken task in wait queue >> locking/mutex: Avoid missed wakeup of mutex waiter >> >> kernel/locking/mutex.c | 126 ++++++++++++++++++++++++++++++++++-------------- >> 1 files changed, 90 insertions(+), 36 deletions(-) I have just sent out an updated patchset to address this issue. Cheers, Longman