From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-fsdevel-owner@vger.kernel.org>
Received: from mail-bn1bon0115.outbound.protection.outlook.com ([157.56.111.115]:53248
	"EHLO na01-bn1-obe.outbound.protection.outlook.com"
	rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP
	id S932962AbcFGXx4 (ORCPT <rfc822;linux-fsdevel@vger.kernel.org>);
	Tue, 7 Jun 2016 19:53:56 -0400
Message-ID: <57575E87.1070905@hpe.com>
Date: Tue, 7 Jun 2016 19:53:43 -0400
From: Waiman Long <waiman.long@hpe.com>
MIME-Version: 1.0
To: Andi Kleen <andi@firstfloor.org>
CC: Alexander Viro <viro@zeniv.linux.org.uk>, Jan Kara <jack@suse.com>,
	Jeff Layton <jlayton@poochiereds.net>,
	"J. Bruce Fields" <bfields@fieldses.org>,
	Tejun Heo <tj@kernel.org>,
	Christoph Lameter <cl@linux-foundation.org>,
	<linux-fsdevel@vger.kernel.org>, <linux-kernel@vger.kernel.org>,
	Ingo Molnar <mingo@redhat.com>,
	Peter Zijlstra <peterz@infradead.org>,
	Dave Chinner <dchinner@redhat.com>,
	Boqun Feng <boqun.feng@gmail.com>,
	Scott J Norton <scott.norton@hpe.com>,
	Douglas Hatch <doug.hatch@hpe.com>
Subject: Re: [RESEND PATCH 1/5] lib/dlock-list: Distributed and lock-protected
 lists
References: <1465328155-56754-1-git-send-email-Waiman.Long@hpe.com> <1465328155-56754-2-git-send-email-Waiman.Long@hpe.com> <20160607201340.GL13997@two.firstfloor.org>
In-Reply-To: <20160607201340.GL13997@two.firstfloor.org>
Content-Type: text/plain; charset="ISO-8859-1"; format=flowed
Content-Transfer-Encoding: 7bit
Sender: linux-fsdevel-owner@vger.kernel.org
List-ID: <linux-fsdevel.vger.kernel.org>

On 06/07/2016 04:13 PM, Andi Kleen wrote:
> On Tue, Jun 07, 2016 at 03:35:51PM -0400, Waiman Long wrote:
>> Linked list is used everywhere in the Linux kernel. However, if many
>> threads are trying to add or delete entries into the same linked list,
>> it can create a performance bottleneck.
>>
>> This patch introduces a new list APIs that provide a set of distributed
>> lists (one per CPU), each of which is protected by its own spinlock.
> One thing I don't like is that it is per CPU. One per CPU is almost
> certainly overkill and not needed for true scalability, especially
> on systems using SMT. Also it makes the case where everything has to
> be walked more and more expensive, because all these locks have to
> be taken. Even when not contended this will add up.
>
> It would be better to do this per every Nth CPU. Now I don't have
> a clear answer what the best N is, but I'm pretty sure it's>  1.
> For example at least on SMT systems only per core instead of per
> thread. Likely even more coarse grained, although per socket
> may be not good enough.
>
> -Andi

Thanks for the comment. That will need a new per group of cpus construct 
somewhere between per-cpu and per-node. I will think about this a bit to 
see how to move forward.

Cheers,
Longman