From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755593AbcGKRhz (ORCPT ); Mon, 11 Jul 2016 13:37:55 -0400 Received: from mail-co1nam03on0136.outbound.protection.outlook.com ([104.47.40.136]:7227 "EHLO NAM03-CO1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1753054AbcGKRhx (ORCPT ); Mon, 11 Jul 2016 13:37:53 -0400 X-Greylist: delayed 3267 seconds by postgrey-1.27 at vger.kernel.org; Mon, 11 Jul 2016 13:37:53 EDT Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=waiman.long@hpe.com; Message-ID: <5783D965.2030506@hpe.com> Date: Mon, 11 Jul 2016 13:37:41 -0400 From: Waiman Long User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0.12) Gecko/20130109 Thunderbird/10.0.12 MIME-Version: 1.0 To: Andi Kleen CC: Alexander Viro , Jan Kara , Jeff Layton , "J. Bruce Fields" , Tejun Heo , Christoph Lameter , , , Ingo Molnar , Peter Zijlstra , Dave Chinner , Boqun Feng , Scott J Norton , Douglas Hatch Subject: Re: [RESEND PATCH 1/5] lib/dlock-list: Distributed and lock-protected lists References: <1465328155-56754-1-git-send-email-Waiman.Long@hpe.com> <1465328155-56754-2-git-send-email-Waiman.Long@hpe.com> <20160607201340.GL13997@two.firstfloor.org> In-Reply-To: <20160607201340.GL13997@two.firstfloor.org> Content-Type: text/plain; charset="ISO-8859-1"; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [72.71.243.90] X-ClientProxiedBy: BY2PR06CA062.namprd06.prod.outlook.com (10.141.250.180) To DF4PR84MB0315.NAMPRD84.PROD.OUTLOOK.COM (10.162.193.29) X-MS-Office365-Filtering-Correlation-Id: f1b63934-9d02-4b1a-7a61-08d3a9b214fa X-Microsoft-Exchange-Diagnostics: 1;DF4PR84MB0315;2:Ghehrp2Idr8kPMNaBNcsHUptU/aKaT7V6XrUe8SBoenMqJIlantoEbOQoIsi9jzHiSAhaSSLZUbdV/JPlNLxQBejF7yaYe9EhhzOERbh71FZpl7xq3/1X0riqKP9WdGuHmXoahOE8E1P0drWUvTnd9iCOhORD0kePHVrHsNYEi61tk+JL8JFhKpACCZTdWOt;3:CqrPE7rCg9/zMe6CKyxXxBfR7RC63Q8SRUVo1psPebHNIMQeLC6kmy3RjBuRH0KcxsQAcEXbOVatLzFNBntrTS5mm0IK9i79S5HHm/xV3eQqU3aBS8NVvncx5GtuRIPz;25:7YDtHgiksoCOakm5haKtWnpBwj5xQRlQAku8asc48ot4yFEioy1SaLY3I2T84iT9xWcG2zg21tlFDS8DVWVVZ+t0pnI+rsQ6Z1KVI+pQbOWyaBVDmBPk7zDLoNMXrcUc33Ak5BtgoOS89Yax3qklNJMfdjIATTcQGfah0q2LWMQXs6UP7bU7yLFUh0tYIHfmHLpNAA/b5pPew4Wu+ab9NkJnv0ejrlSAw2rNp5ts2AhUwkZy1nl19zC36RA4FGrLrUH4O3VC7v87nbUm4t0bCxYdeNp9A0VhtD7Wejpi31xAW3RrDsbfspmOo2pJT69jumllAIOL5hScTeHPfU1DYrfNgithIAtSEM+o/38tybU/7PzEYRuGWa7zIi11tIFS5cbsuHKgAVVlS24/xDKNbKXQ20Bz+Huig6h01pvmIMc=;31:jvBDCotri2QBvAhhtxj/JxaEWtNSCMxK7y84te26pI3Vbzv4j/OdyQ1dpCxBc6szxU/BCxFKnpQ1GIbfwMdTxux9BHxxfLrEGVLYxkP0dhYNNEVCMivrLOIDR43MuJ1b9xWMyxBvKtwmgGCjOEmg9rVwl5+rLAFN2qvbWe2yb76kHcwFcuewZKYxWIh3zh1vQ947mYP6ccynWFWmcyXIWg== X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:DF4PR84MB0315; X-Microsoft-Exchange-Diagnostics: 1;DF4PR84MB0315;20:PBHkIID2Lhn445Ub64L+CdvM0uOwigYNetCvP8qnPkamiR3WR5jXsg/KG9hzQiOrVo9yi5UH22mkguzD/39pymczoshCaU0NPRGiIXs1NpaZN6nj43GQ4qnhU8vZK0qofJZU+nBZTwPb5KiNvbFmy9X6xLIlW7aiJxOEhWjM0JCLsK/SxID21tdSMZZsyMyZPL8f5AxwLyVwZDkZOgjxfxm6IMp4V6v4emZUq5mgWC9R4VkbzgZfEkgpMRo4A4NJ0wiNgo+DbPwdJPAsG7dT88PPNUPoi0Ruavz3DFt0KjsmCbp7mWa5utWjEFDH2ITahY7LiabkuaSM7tt1DYdMYL2SxxCx0wmFYmIpsYuexjEVf5Vu7lXHW3PQEmRCJQUFn6jGcyuXAzGhp7hha/87ZNpURElo1Fe8x/gL3+gMAWYBYN9qceT+Uwf/b23UWoYSXZiYShrdj5tKDtYFImrGm1TPmoMLPSf8Ef5Bzm9Dx40rEHHV2NB3lsUHCdPQwOD1;4:Ffmdhi+T/UCfGJjdThVde0kKcSQEREZIQhnZEf7VMM8xRlBcm3X0QtpmqaVd4cL5gSR9ev9yAcjcSLQ4jl6q7F9dyiXwTCVnUpViGiO5gY/0KAxk0uH3ZnuTfN8TJk+YpfQPFscQJ/SDos5rBjnvuoLqJ4E9QQAuTFmQxIPGM6EjJiMHfyY7LllJ26SShdC/vGar0H9DQ4aOtR7qFkwmUTNgPQ4ek1vzx3Yy3k6d6vUSVgDfRtdMpRfC8gbh19s1qbuuvUW4e2cvZFBgGDnxk+EkgGRBgqF5mQczs47fC/DU0OLSZoObcEH72LDKHeMsBIYHTEKbXRT9jchuOZi3SQhDQ13S1zo4r8Q+9/x8x5WE84rUqnaTWB7d7xKXKuDa X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(601004)(2401047)(8121501046)(5005006)(3002001)(10201501046);SRVR:DF4PR84MB0315;BCL:0;PCL:0;RULEID:;SRVR:DF4PR84MB0315; X-Forefront-PRVS: 00003DBFE7 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10019020)(4630300001)(6009001)(6049001)(7916002)(377454003)(189002)(199003)(24454002)(106356001)(4326007)(2906002)(81166006)(81156014)(68736007)(117156001)(59896002)(50466002)(7736002)(105586002)(7846002)(36756003)(76176999)(87266999)(54356999)(189998001)(92566002)(4001350100001)(64126003)(23756003)(586003)(6116002)(3846002)(33656002)(65956001)(66066001)(42186005)(86362001)(47776003)(65806001)(80316001)(65816999)(2950100001)(83506001)(50986999)(101416001)(230700001)(8676002)(305945005)(97736004)(77096005)(110136002)(217873001);DIR:OUT;SFP:1102;SCL:1;SRVR:DF4PR84MB0315;H:[192.168.142.165];FPR:;SPF:None;PTR:InfoNoRecords;MX:1;A:1;LANG:en; X-Microsoft-Exchange-Diagnostics: =?iso-8859-1?Q?1;DF4PR84MB0315;23:RbtXD2zGj3N5C/np+NvSM/Sx94h459QD/ao62hf?= =?iso-8859-1?Q?K5FJp7xI5QijVKPHT44WFeG5yAYlQvGOXYwlaBuTCVHrq0/ujFS8yEgRq8?= =?iso-8859-1?Q?8iSdBWgtQSpDgX93RyeLbUcfbspZQAHQFRquusbCIfUouBlphryO5vxIvX?= =?iso-8859-1?Q?4m3r9y/5Mtlda8tdMr3+Nvq8cH72GrlipFNP0jE5mIsdR8oxU/uu94zC4w?= =?iso-8859-1?Q?Tqtcv1dSJXwRPAuEKwbpi5lyiV64V74dqKp/LZLe6krOr3wuEptJIXLLsY?= =?iso-8859-1?Q?G5WMGXE0jm+vdEoGUXBxKD1M+yyMhxYJis6w2FC8s77KfZ7f1mfJjRGTLY?= =?iso-8859-1?Q?A11f9VOm4gHvBPMTvgsn5sCivhqBqNUBwShz+A4JT6AU+5sMI/jMeXWsGt?= =?iso-8859-1?Q?RliCnXX1dl0OVVuTRtLqhw6REaHtcWfpcoayuvzaUg/N7r31OVdowFrCAP?= =?iso-8859-1?Q?uDuWs7bbIfzC14IUeTP1lq+AYL+3asRETBnZbYxCOFwm2IVNZZ/MnUtmZR?= =?iso-8859-1?Q?Vj5Tr17gWQvb0Izs6LXDJ+criQdJAQ20tXLhZcowuK0IlOliVAsR9IJWWP?= =?iso-8859-1?Q?/8uRaVZN+nuk+k1R2Y3Y8AZbRS9ThDhKnPpOnd9HXf+rg5vEbJQMqijbx1?= =?iso-8859-1?Q?rLu9Jgb7rK0COBu4rH+loQOTN8CIiD7Yxibm+JjlYAPG5HpWmjo0/1YgEy?= =?iso-8859-1?Q?dUSipPY1dHBe3tLqUo+U7B9RyS+phbmBxbVIe0a/hmCzM7w4Z50mjwWOoI?= =?iso-8859-1?Q?VoCh13sJxqQk7TozJwRhbhlVvhCmideF2NLv4p3POTWCXuA8xmrRa/cmxT?= =?iso-8859-1?Q?kxJ6F3JIdUO2R+v3n1OqEP6AHF19+X603Dss1CX6jlnIThmeg7OVskFKqK?= =?iso-8859-1?Q?gbjcCtuFVYXV2oQtzh0zpAwCw5cNTttFU+2QEQyTcei3sgYObK/ZyWnbPR?= =?iso-8859-1?Q?zcFaf438HFt+AL7NmfjEuPgoTeCbnt7yJW+fG8WsFIjMMbT5uwz9puqGAY?= =?iso-8859-1?Q?3ATr/YVzQmzhd5NYHLz+yBYlFWN5cFtBhjhXtmxXlAtTV6cseMXku+Pnaq?= =?iso-8859-1?Q?C8VPsLm1bxZLcz7+Asb25QEqa9x0JuRGSpwomLpn9cn4o/R2hlWeCSHm8q?= =?iso-8859-1?Q?dXzS+zGNVsrRdBXmf3mqhVgG8WudVt1RkeU62Nw73ozxNdl/PwNUdue815?= =?iso-8859-1?Q?wKVRpmKk2N7zFqxy3xUvEbqDVcut00Ik/iswZqqNCfA4pxPdmiHLNcyzzp?= =?iso-8859-1?Q?5WnQ9kpRTNgrqGpzNPSEYBj6u4CO8GbrRtAJhq8kZq0dZPSM7qbmgbQOcJ?= =?iso-8859-1?Q?esUTtQDctE+iGe5T6O8oRMq?= X-Microsoft-Exchange-Diagnostics: 1;DF4PR84MB0315;6:mUNvKvc9vBluOXCNQ+ylpsXVy+t35iUyvnQcSrIRuVhgrva7Z/Hq9/pjCJObtrEkHIGAtzaq4uQuCNk8PbClF6h7l/ApT6pxCpPotiLWSpCjTUfRd+6rrjOShoyJDhH2b9q6kTQKyYokOIchUXc8b0io+n0Wi3B8TTYBg2JwZ/1V5/Z0T63P8Lwbabe8A8008x7fBWdjmc6T1vDeR3Ul5eZ4ueTPs3E842oZckI/govOBkuFIRo3sfM4EiaJNs/1ghnSF7UM51sjnIffkPSAsPQRxCCm4C5D4WOhajSHYLk=;5:7ZhI2+4erAgDO+pFNcHzF59SbKvzinqAICkveqg28JlO5a5pxxIVLaFHyufR4QPT/GujDF0nx859aGPvO4qjgqF8DhdXbCAg7snnW/hYpw4AxIjPYmpXyXOc+176I2hiVz7+4wAWyx2sMLpxPR4lIw==;24:qL2ADIr/ZEFzFAcHIuwo0ttwnaUcH5dGKmlftMU3pN2MXE5A6F9Tysow2F0JNDGEcAuTFSOJCl56gwY91/0Xo7G5JpgJA9ZzNUUdyJP4Zh0=;7:l9WI/BgrmgPyYfnPd8qkd21qKsl0M3e6APexEhYuEHEF5Z0vcbYU9uOAVVSkRl2vpLy5SJwfClLXmRHKvOEeSdSuwXT9dYOzrqH2CoPXD/TAnIb+1u5Z3HdY0MbKZG15ZK8SAsxj9l0Z7oYOwcga0tA5dw1WhAYNzRqTWhzwMiL3pcKaZigSuTRWITVArjkNjuvkwSNUAvU64XVfmeq+IFQ58f9nFI1bJnFy62/07IDc0V9FsULMA9MOz/ONlgYq SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: hpe.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Jul 2016 17:37:48.6632 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: DF4PR84MB0315 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 06/07/2016 04:13 PM, Andi Kleen wrote: > On Tue, Jun 07, 2016 at 03:35:51PM -0400, Waiman Long wrote: >> Linked list is used everywhere in the Linux kernel. However, if many >> threads are trying to add or delete entries into the same linked list, >> it can create a performance bottleneck. >> >> This patch introduces a new list APIs that provide a set of distributed >> lists (one per CPU), each of which is protected by its own spinlock. > One thing I don't like is that it is per CPU. One per CPU is almost > certainly overkill and not needed for true scalability, especially > on systems using SMT. Also it makes the case where everything has to > be walked more and more expensive, because all these locks have to > be taken. Even when not contended this will add up. When iterating the lists, the lock shouldn't be taken when a list is empty. > It would be better to do this per every Nth CPU. Now I don't have > a clear answer what the best N is, but I'm pretty sure it's> 1. > For example at least on SMT systems only per core instead of per > thread. Likely even more coarse grained, although per socket > may be not good enough. > > -Andi I have just sent out an updated patch to mapped 2 cores to each list. Maybe you can take a look to see if that is good enough from your point of view. Cheers, Longman