From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932987AbcFGXyA (ORCPT ); Tue, 7 Jun 2016 19:54:00 -0400 Received: from mail-bn1bon0115.outbound.protection.outlook.com ([157.56.111.115]:53248 "EHLO na01-bn1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S932962AbcFGXx4 (ORCPT ); Tue, 7 Jun 2016 19:53:56 -0400 X-Greylist: delayed 15836 seconds by postgrey-1.27 at vger.kernel.org; Tue, 07 Jun 2016 19:53:56 EDT Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=waiman.long@hpe.com; Message-ID: <57575E87.1070905@hpe.com> Date: Tue, 7 Jun 2016 19:53:43 -0400 From: Waiman Long User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0.12) Gecko/20130109 Thunderbird/10.0.12 MIME-Version: 1.0 To: Andi Kleen CC: Alexander Viro , Jan Kara , Jeff Layton , "J. Bruce Fields" , Tejun Heo , Christoph Lameter , , , Ingo Molnar , Peter Zijlstra , Dave Chinner , Boqun Feng , Scott J Norton , Douglas Hatch Subject: Re: [RESEND PATCH 1/5] lib/dlock-list: Distributed and lock-protected lists References: <1465328155-56754-1-git-send-email-Waiman.Long@hpe.com> <1465328155-56754-2-git-send-email-Waiman.Long@hpe.com> <20160607201340.GL13997@two.firstfloor.org> In-Reply-To: <20160607201340.GL13997@two.firstfloor.org> Content-Type: text/plain; charset="ISO-8859-1"; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [71.168.64.186] X-ClientProxiedBy: CO2PR11CA0025.namprd11.prod.outlook.com (10.141.242.163) To TU4PR84MB0317.NAMPRD84.PROD.OUTLOOK.COM (10.162.186.27) X-MS-Office365-Filtering-Correlation-Id: 1a78f047-13d3-480b-fe56-08d38f2efad9 X-Microsoft-Exchange-Diagnostics: 1;TU4PR84MB0317;2:fYL1bR+NLzcZ/IEHX1I0yHnr/xQoVzQtLd57KfD5+yPMkYk4HqbDdzKsPcedm0VskR2gkyJdpbrI5Szc5f/FfFh0Ci9TKOVruIAnaccjLEAaoO4PI6900nT/YpakGcGsYVqbVida4N9CCd3w6bRnC5FwgJojGCZonWKlspoV7yci/jrq6QyeMTVNiujHcf0L;3:xydCpd1cAPoRozRcahjjumgiTai7cuPgRA0rokRQmiGkoq6VlHqMl/UbGRLa3+K5hDu6YZfhDf43shCts6bk0ymCTaCxRyyk2X9qSZm3MnMR8ZaSjr5uSRyhsjWJGko/;25:fkuMXs6h3yuRHoNYke4Vx9THujDI15fjkKP+Dp4XNk7/WnOT12ufeAq1WYE+WCkOnE9Y7OCbI8lfnBjrAYY3ipoiqk55kgNOZ8rgTBP7Aa7+y04Iyp/Di/6g8yHprwaat4fl/ZtsdK1h7FkEDIc8EI3TBY0XRrW5+Bqq0BOgXYY389NG/7q17BFn/5BrDi//kux+6trq/8Ppr+IRuNnWEPsze//TGjxTnMfIVTsVw5MOhs1OD35g6D/sFyATzHHcP47LSpNN7XmvhvCqLEVMFpUfdo1z9QjdaE+fzSI+zBj4Hv5SNvI2HRfpLVig0egA/YowiXLuEMA2frbkHZ25ULcCfcOuVGKAptD4sGPM6n9To4hzp7PWGHPVit7LrvfHO8GhAwWDp8CiLTqkOGXih2wryX4l4odrSVX4fIwF67k= X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:TU4PR84MB0317; X-LD-Processed: 105b2061-b669-4b31-92ac-24d304d195dc,ExtAddr X-Microsoft-Exchange-Diagnostics: 1;TU4PR84MB0317;20:L8Nu1w3e10rC7A00W3GNhY58AWcbAmaCPQFcKagGIMpc3itDefxU7STYhG113LnvLiGRMutdWiARbU0nqNTOM2Ys3b9MTWE5yHli9UFMC1C6yJRttHHE4D+vm/l2LbLhFxR+L3tBU8oeAD+OvYMio7YLmcCG++R9LYTEyoNoNKEHySxIjQAFO82hd0cMBvFVBnoDWiO+Epn95JpT47M4dU1Ep8d0Iyuy4neLXptxky8QJjFf0Mfqlx+nccGXvkT8GElR4jPsVlmfXIjTjM1YmAbuzhBrTwtzN8hgWSPmPwu7SFaFQcQMvuqwZhsEQBPQ3pyZkK9ajMdNn9HJjZXNkkcfoQF+FE1OgeGQEDqR71gCY0by6uZ7xzTMLL4wBshJKIiNnWxmkFMMeIhLrnHhKesKFvJAj9IlT7dG2LkeXo3QEEJZiXwdJ3CTOOgKiZxDgpyNyi2y0MW34tbdTAGbyDSf2pXL6DcKp63txya3BQlKrOMhjsb0Dvr8wPVw26c5;4:lE+i0Z5m2vMDm3bVHwBNs9LVqTUjtEhynZFFO7N+BoKMd7sq3qYH0XY48ohWBKnbFvRJURXfzpFTa7pgOw4iix6WhK2/NZuRV8FLzNI8tOi2TFE9XN1B6CUx9mQjl1CSaNYWAjyQLvjBoqdkv0+g85zXlUiHjPqg2Uv9h5eNWUZXg9+0gejEWPVltEbY5UCNN7FDDAvpvD+cCdD1Zd9yCpLJCc+s1P8ZzErQYBBK+XQKunACasLBhXN8viiiiRzxzD3455KsVLQs9TJ7G0wnduw5d0duMDVMVROnMOlaAswOXLRxxOPAtqDJvadI7BNmBzetZFKBlHScrZm4/5PzOhG7iYywfp4iEDd+g6hF1sNKq90QnfrX2kAs0Bs5zM7w X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(601004)(2401047)(5005006)(8121501046)(10201501046)(3002001);SRVR:TU4PR84MB0317;BCL:0;PCL:0;RULEID:;SRVR:TU4PR84MB0317; X-Forefront-PRVS: 09669DB681 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10019020)(4630300001)(6049001)(6009001)(377454003)(51914003)(189002)(24454002)(199003)(59896002)(5008740100001)(50466002)(33656002)(230700001)(586003)(106356001)(3846002)(6116002)(105586002)(86362001)(101416001)(117156001)(87266999)(189998001)(65806001)(65956001)(99136001)(2950100001)(54356999)(50986999)(76176999)(42186005)(110136002)(4001350100001)(97736004)(66066001)(65816999)(77096005)(47776003)(2906002)(4326007)(92566002)(80316001)(81166006)(81156014)(5004730100002)(64126003)(23756003)(8676002)(36756003)(68736007)(83506001)(217873001);DIR:OUT;SFP:1102;SCL:1;SRVR:TU4PR84MB0317;H:[192.168.142.147];FPR:;SPF:None;PTR:InfoNoRecords;MX:1;A:1;LANG:en; X-Microsoft-Exchange-Diagnostics: =?iso-8859-1?Q?1;TU4PR84MB0317;23:yGIIzyhKwJChokU07TuQZAeaKUF70stNKmLfCm6?= =?iso-8859-1?Q?BB1vtY80xQsbCNj1BFGCnIKB2tSNzgHhpD19oV1gBD5XyVjmwU5Czf/9/F?= =?iso-8859-1?Q?MDf1jOZOMK5qOfMmYfEG/8OIZteXu+40zPN0Hbc3hr3vXFWf7Wj+inIWgS?= =?iso-8859-1?Q?Dq2tI2mQqCoeAAkTsg8PEaglsVmG5PYxeyz+aAF6mu/kPtadvJmRnT38q/?= =?iso-8859-1?Q?/esdmgdbrVyhCeiXKTXyM2Pm+O+JafEZdZcCfR2hy7M2Ab8RtB/qSH/Rtm?= =?iso-8859-1?Q?llr30qwQNdL+0IGFdCsroe/LoSZwnYK5VfwHpQT/L9VbYZMhxgjiXdHED2?= =?iso-8859-1?Q?4sdFRzHjZEeyqtz0H2yY66ACnZTh6cQBYsDsAi5fKhaFdId7WGA1K5P07p?= =?iso-8859-1?Q?IAXvqsfnxJrpaZDPjuX6q5tbarNDU50h/+jrDj90LFh8RIPsQJUT9Qdk6O?= =?iso-8859-1?Q?T377hrIkMrkCXwMUSv/iR/2o109FxOCwAl/9BdNWZnLQwgiMHf1iYrb2kN?= =?iso-8859-1?Q?O/A4cvd6IHhMItW0vAS6SQysNF/EVe//bbks31eej7F2wgFJTZJRvK01Up?= =?iso-8859-1?Q?Dvcwg2g/QUtu1SDF3E9g6MhIngkLGV+tL/lTHSHxlfazN7HlRKYatUCN0L?= =?iso-8859-1?Q?I1qtMha7+GvqH8B+WBuA3AgOFSQK/mOGyS3y5Vy50Zos+koTiEmnSdT7VX?= =?iso-8859-1?Q?hcn4EomHHJtOmrBQBpi3v2D9w6HZhvtJMae5xghptQpR724scLWSjgcwvB?= =?iso-8859-1?Q?ytNAR4MPGqtcJaNjh9mO3BB2nN1VPwZfVhT/dlB9Y/h7f//ghCoYrh3zuU?= =?iso-8859-1?Q?lRfiAwsMgy+kq6hdK6aD0he54VMgXy5yD4o4sGHFIXYBCizEZjagfAvPB+?= =?iso-8859-1?Q?HEZ9C+kgyWQIjBuVTnxJlNfRpga1yFPxfrY7aO7R5UxlxYO92ivP37QtgU?= =?iso-8859-1?Q?byMATls07nXG2K9vwg4G9Sg3zYyfUhkHip7CA1FEmb6nkDyy3DC4sVRSdn?= =?iso-8859-1?Q?zYrv6KF+IfiHBoqjV0GFnSpIWgx7SCZYbskP50TU4Oqdfl22/7CWX7ymor?= =?iso-8859-1?Q?8aQdr713z5lWoxxO9+yVjNpgiY5y1/LYZYe9LOUC5lV8C6a66dcveQXO/u?= =?iso-8859-1?Q?0TQQ70HBLhtzuuxk1HqIFWDuzMJzU30U5zONsYTN4BXy3v3ymxk4RYx7GS?= =?iso-8859-1?Q?Ow4hsAIX0BT8lgZsDqzJ5S3zXeVg5CXStHVZGpSGA81bHZBVdsLXSOj5WO?= =?iso-8859-1?Q?fz49OcYQDbiQn47ddyTsy+A6HT65aSWZVfny/BthjKv5YoMsS9j2MmscMi?= =?iso-8859-1?Q?OjabP6ApFMUwYxPDr4IbI9hA5JzBNbLA8sb8t91sC6Afy0OAUEyUQBuris?= =?iso-8859-1?Q?rJstASHw=3D?= X-Microsoft-Exchange-Diagnostics: 1;TU4PR84MB0317;5:8quHNNR37O30mS9Xbi7/X8Bl65zD+Y/R1hkBOFgYPH2XrnZfEZTouiTD8vEr5Oy5QCw5I0y+JBhFH3ccuEe7bHIt6slsOoD5CrZ7ZTVvP7wVuBDWreLw6UIbda4QLDwVKxa727vS5cFcimeE1Iu+BA==;24:WMU3msaluNqu8oJpTnNeu+rsCO6lNfg6FiCNyG2gef7T+b+xI1hs2l6jyK3eQfpD0lDX+vCVfxlzXbhT3KfDIjgGMTJFRZNEuq03o/wOCzQ=;7:7JsABgZl2g3GE9l69U9eaWCsOKHbiY28J0G11ABMtsBNhn/SwxaDUsm9GH8iTSIu6ctj520F+4up5u4+1bbrjFbalkV3f98Bna+rGHMXhgHUMrkWNexGbj6lsZhjcckj96+pnDUWcrXW91PZBGiIuyIw6/EPx8svlOWJTQMcucbPFfYMrrSNquTyKraR4Wdjt0qnv67ZBCdMbIIVqsvRkkk7xXXjRJDW6S/6cMYyb7I= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: hpe.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Jun 2016 23:53:50.3134 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: TU4PR84MB0317 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 06/07/2016 04:13 PM, Andi Kleen wrote: > On Tue, Jun 07, 2016 at 03:35:51PM -0400, Waiman Long wrote: >> Linked list is used everywhere in the Linux kernel. However, if many >> threads are trying to add or delete entries into the same linked list, >> it can create a performance bottleneck. >> >> This patch introduces a new list APIs that provide a set of distributed >> lists (one per CPU), each of which is protected by its own spinlock. > One thing I don't like is that it is per CPU. One per CPU is almost > certainly overkill and not needed for true scalability, especially > on systems using SMT. Also it makes the case where everything has to > be walked more and more expensive, because all these locks have to > be taken. Even when not contended this will add up. > > It would be better to do this per every Nth CPU. Now I don't have > a clear answer what the best N is, but I'm pretty sure it's> 1. > For example at least on SMT systems only per core instead of per > thread. Likely even more coarse grained, although per socket > may be not good enough. > > -Andi Thanks for the comment. That will need a new per group of cpus construct somewhere between per-cpu and per-node. I will think about this a bit to see how to move forward. Cheers, Longman