From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S965339AbcBQRmK (ORCPT ); Wed, 17 Feb 2016 12:42:10 -0500 Received: from mail-bn1bbn0105.outbound.protection.outlook.com ([157.56.111.105]:64336 "EHLO na01-bn1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S965298AbcBQRmI (ORCPT ); Wed, 17 Feb 2016 12:42:08 -0500 Authentication-Results: infradead.org; dkim=none (message not signed) header.d=none;infradead.org; dmarc=none action=none header.from=hpe.com; Message-ID: <56C4B0E1.4090902@hpe.com> Date: Wed, 17 Feb 2016 12:41:53 -0500 From: Waiman Long User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0.12) Gecko/20130109 Thunderbird/10.0.12 MIME-Version: 1.0 To: Peter Zijlstra CC: Christoph Lameter , Dave Chinner , Alexander Viro , Jan Kara , Jeff Layton , "J. Bruce Fields" , Tejun Heo , , , Ingo Molnar , Andi Kleen , Dave Chinner , Scott J Norton , Douglas Hatch Subject: Re: [RFC PATCH 1/2] lib/percpu-list: Per-cpu list with associated per-cpu locks References: <1455672680-7153-1-git-send-email-Waiman.Long@hpe.com> <1455672680-7153-2-git-send-email-Waiman.Long@hpe.com> <20160217095318.GO14668@dastard> <20160217110040.GB6357@twins.programming.kicks-ass.net> <20160217110520.GN6375@twins.programming.kicks-ass.net> <56C49CCA.7090805@hpe.com> <56C4AA19.1080907@hpe.com> <20160217171845.GK6357@twins.programming.kicks-ass.net> In-Reply-To: <20160217171845.GK6357@twins.programming.kicks-ass.net> Content-Type: text/plain; charset="ISO-8859-1"; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [71.168.64.11] X-ClientProxiedBy: CY1PR0801CA0027.namprd08.prod.outlook.com (25.163.136.165) To CS1PR84MB0135.NAMPRD84.PROD.OUTLOOK.COM (25.162.189.26) X-Microsoft-Exchange-Diagnostics: 1;CS1PR84MB0135;2:2QVgMqX04gSfZwImMCrSkakNFASJIjK+nwpwjkYFMuUMP0zJoHe56GciJ+va3qOS1dUjGUq4DQwHnpFge7UzirxmEv1g3ozRrtGap2FpMPdqEOmG/GNRzech/ugb9r+qAt8qb2akgKSI1CHNI1h5HA==;3:6XYduQMEu/ExlWM1LwtYstIVuUiRRJ2fmGyirlZWGz/ecA3Duh9LIRkdk8zcS6a9WyqUAcy8ZwOqxUVu5qMAtx1CLEYrrdVI5Ta+qFtSpwUE+wGFK/0Z9OCDZT2fe9fz;25:gpdRxmuilvpdvQHxW3D40mYkXGfOG8EhqVkluNPZDVz+muUZBbCMrouGG9hZE+JOd9hAVe7v2ANnfgXcz1VvUUZC1vzAjSIvS+qhUoq2F5p+thLkm2aCiKYVoMcA8OKtN9547kboHp8sInAn2hlMdRvWubsWSztoUL/tRmlwLVsZ4WdezG4ZWRo2nlrkJmZEhplNs5yAK+X0E8VSox5DyErkA+Fpp+Aug/EO911Wu/te0zTPUQ38CgutKRhJlnOR5yo1uaJ3Jk6V9b2LNH1BchKwFFPQsK7r2B62WxgFMUnmhymRjMEklbbL9cpniqyV;20:UGOe+cqMgXZaXo88pkETk8eDV+e+PvrQU/pY3/HMeUR4wDIxBdMR+HCZm6iznUblLS0aL3IgYyWOzAlekqzcYT6yA7aLb2bogZh6Ay3iTNAva8HrdZnLywnxUbWcz7OCRkxTjJW3Rp7uyKkzzyNZ8Uc+wrKDLjzTuIa1Rz8QfuyqZFLWyjigOC5my848Anb8jg6kOZu720SDy6QlVFpznEb7t71A3NTHoAAxO+Mk8lWVCclVEOKm4gX/9wX9wU7F X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:CS1PR84MB0135; X-MS-Office365-Filtering-Correlation-Id: 2a2cbf1d-0568-4fca-067f-08d337c1a709 X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(601004)(2401047)(8121501046)(5005006)(3002001)(10201501046);SRVR:CS1PR84MB0135;BCL:0;PCL:0;RULEID:;SRVR:CS1PR84MB0135; X-Microsoft-Exchange-Diagnostics: 1;CS1PR84MB0135;4:5Z0TxsWNX+wfEyYUh6p28wD4k/jiDdyE0LYU7bVh56aRdEEeQGFNhvkAKCxrxVqIJ5oVWtLcODS2ZoDR7Ioiq2UIVUk20zinUwdUkorCR58ZDC5EGQRVnUux65odwsHxfmBOGwit2blSzhlfsFNBxa94BvSBvxxN0MBwDtSrlHgDYeV4KYBjsLej5GVFO+tms+Q+m4ZiA1Qj1OiazRM6fmmOU3q6k4AnOK8DzmZMLw09y3na1nWvcfFst/iHz8+a0vSgUVNL0mTVYtwPdIcV+H85kPgju3T1SofeZuLB6yVvGJBiEfp6tNOziUvy6TTjjfUDWF1w03emL50J9g1hQ6B5NMdGwBiTv7LvdRs8MxIw5JsW/klhYhz74A+u7aNo X-Forefront-PRVS: 085551F5A8 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10019020)(4630300001)(6009001)(6049001)(377454003)(479174004)(24454002)(80316001)(189998001)(54356999)(86362001)(65816999)(87266999)(117156001)(33656002)(50986999)(87976001)(65806001)(66066001)(65956001)(76176999)(83506001)(47776003)(230700001)(110136002)(42186005)(5004730100002)(4326007)(36756003)(5001960100002)(23756003)(4001350100001)(92566002)(586003)(6116002)(3846002)(1096002)(77096005)(2906002)(2950100001)(40100003)(5008740100001)(7059030)(217873001);DIR:OUT;SFP:1102;SCL:1;SRVR:CS1PR84MB0135;H:[192.168.142.188];FPR:;SPF:None;MLV:sfv;LANG:en; X-Microsoft-Exchange-Diagnostics: =?iso-8859-1?Q?1;CS1PR84MB0135;23:o9WoIS90Dv5U1MF/5wflTjVef73mRGWMlJ/3+e1?= =?iso-8859-1?Q?zaaCpX9Q3iqTh9b3JIDrY4Gup6IKIJQm4+Gh62xznp95X+mFYy/8dheB5Y?= =?iso-8859-1?Q?US6nisu4rxhb0kaEgAh0xHdhInyv0SpONvScmz7QoFgYM+EyL9ahWQFCe1?= =?iso-8859-1?Q?I/izDfiG0KFEfO2QcHnhxePOwsPPXBA0ivrvv0Wo0pHaWKfncbQ8CPJI8z?= =?iso-8859-1?Q?V+vlL0NWp4q3Gf6ylnXMkZM1gsPXf0NrH96vgo/imekzNfoWAsUkS5bGI8?= =?iso-8859-1?Q?CHyo8FHVXfJty95vw4DwsAnDtue8qZ43WDtq+qKMkGbdNmOUfGPTejx3iL?= =?iso-8859-1?Q?36rXUDSrTOrhhd0RDZkmtrFqGcUZYJTam1U2NaklX2GT8RdvVbBSKdMQ2z?= =?iso-8859-1?Q?vlKWdukII9FwufP1JrJrpoVUG94pqEIRs0Fnmomc50xSAPU3OnYgTtYuuo?= =?iso-8859-1?Q?mKe+KeIWvyxNPP6NMPeizbizQnq0eKEs6bXEWkORosA1fg+hf2xGKKa279?= =?iso-8859-1?Q?16cqkQHkyeZWReIxh4jx0QVPg5A44RF24RcSOCF4CqYrATfncwtrn1sIq+?= =?iso-8859-1?Q?M/K1KzSh//IhqZN3Ykh5R6G6aPxdgo/Bovn4ve3uEempB3e7Zv9mh6Ktr/?= =?iso-8859-1?Q?4uGbo2rz3CsRjgayLd3ub2D0lPqiHUJWzTNemuyWRr6pJmOZGERKhnfFEg?= =?iso-8859-1?Q?ByD7nge330NiNjrasNCFrN7BukCEAWTtmPfS/wqN9TRl1TqH5rL6X/qei4?= =?iso-8859-1?Q?okGXkJ7LJ/N90/fIGVRbnKEhQNIgm/hk5W5babS3US1/BOlqdERvaZOruI?= =?iso-8859-1?Q?Stvtqp3hTixjz1BoJMfQMGRF6WSQFvVnrT/tiNZyhW92t9zYb3ivw3gdz9?= =?iso-8859-1?Q?T4hX5dQ944Muj86d/lHJ+RG/r+ngAjmMb7ef2JbKWQuVf1eR8RFa6TDFMh?= =?iso-8859-1?Q?Tw7npy6n9JKw4H5H2zEt+BvX5INVIFt6aM21P82lTv17e6mFPfLFVKOP+/?= =?iso-8859-1?Q?jxLhYilXrsh3UMDMqnHQNnfUMMbcWlU9zDO3peMiHLzyfF3EcNozmkP4HH?= =?iso-8859-1?Q?4XMUKaW0ccsPDG7v2RnqIbvh26qC5/SzwLpt4Jh5CMtQ/NSpVRpQ1c/PRc?= =?iso-8859-1?Q?7UHivPl63uibpk7XUa4N5jFH7YUQOtguKadEh1xEnbQ+ZVAM=3D?= X-Microsoft-Exchange-Diagnostics: 1;CS1PR84MB0135;5:m0AtyIwFAk4jCXhS6Kt9qEfID0Cxl8etFTq0btPpHF24eoVJ4rUn0E8skr6MuHyzt1eRKWV8UKKdu4MUvq3qOshloHHIgU/KwDh9LqjixO41NDZxkxui6IJTiB7vEF1W0cGO8V6SIqMTU5I/EOpcMQ==;24:Seqz5FCz+06CsEYui6vcI+qxCdwcICc7mop3ay9TQN6IoPGEs1zZUOqFex5/mbxdQlSyRJdZMhpTGKgTY5F5npop+XRWXDzm62EUUQeYbbg= SpamDiagnosticOutput: 1:23 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: hpe.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Feb 2016 17:42:04.6692 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: CS1PR84MB0135 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 02/17/2016 12:18 PM, Peter Zijlstra wrote: > On Wed, Feb 17, 2016 at 12:12:57PM -0500, Waiman Long wrote: >> On 02/17/2016 11:27 AM, Christoph Lameter wrote: >>> On Wed, 17 Feb 2016, Waiman Long wrote: >>> >>>> I know we can use RCU for singly linked list, but I don't think we can use >>>> that for doubly linked list as there is no easy way to make atomic changes to >>>> both prev and next pointers simultaneously unless you are taking about 16b >>>> cmpxchg which is only supported in some architecture. >>> But its supported in the most important architecutes. You can fall back to >>> spinlocks on the ones that do not support it. >>> >> I guess with some limitations on how the lists can be traversed, we may be >> able to do that with RCU without lock. However, that will make the code more >> complex and harder to verify. Given that in both my and Dave's testing that >> contentions with list insertion and deletion are almost gone from the perf >> profile when they used to be a bottleneck, is it really worth the effort to >> do such a conversion? > My initial concern was the preempt disable delay introduced by holding > the spinlock over the entire iteration. > > There is no saying how many elements are on that list and there is no > lock break. But preempt_disable() is called at the beginning of the spin_lock() call. So the additional preempt_disable() in percpu_list_add() is just to cover the this_cpu_ptr() call to make sure that the cpu number doesn't change. So we are talking about a few ns at most here. Actually, I think I can remove the preempt_disable() and preempt_enable() calls as we just need to put list entry in one of the per-cpu lists. It doesn't need to be the same CPU of the current task. Cheers, Longman