From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Message-ID: <5783D965.2030506@hpe.com>
Date: Mon, 11 Jul 2016 13:37:41 -0400
From: Waiman Long <waiman.long@hpe.com>
MIME-Version: 1.0
To: Andi Kleen <andi@firstfloor.org>
CC: Alexander Viro <viro@zeniv.linux.org.uk>, Jan Kara <jack@suse.com>,
	Jeff Layton <jlayton@poochiereds.net>,
	"J. Bruce Fields" <bfields@fieldses.org>,
	Tejun Heo <tj@kernel.org>,
	Christoph Lameter <cl@linux-foundation.org>,
	<linux-fsdevel@vger.kernel.org>, <linux-kernel@vger.kernel.org>,
	Ingo Molnar <mingo@redhat.com>,
	Peter Zijlstra <peterz@infradead.org>,
	Dave Chinner <dchinner@redhat.com>,
	Boqun Feng <boqun.feng@gmail.com>,
	Scott J Norton <scott.norton@hpe.com>,
	Douglas Hatch <doug.hatch@hpe.com>
Subject: Re: [RESEND PATCH 1/5] lib/dlock-list: Distributed and lock-protected
 lists
References: <1465328155-56754-1-git-send-email-Waiman.Long@hpe.com> <1465328155-56754-2-git-send-email-Waiman.Long@hpe.com> <20160607201340.GL13997@two.firstfloor.org>
In-Reply-To: <20160607201340.GL13997@two.firstfloor.org>
Content-Type: text/plain; charset="ISO-8859-1"; format=flowed
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-fsdevel.vger.kernel.org>

On 06/07/2016 04:13 PM, Andi Kleen wrote:
> On Tue, Jun 07, 2016 at 03:35:51PM -0400, Waiman Long wrote:
>> Linked list is used everywhere in the Linux kernel. However, if many
>> threads are trying to add or delete entries into the same linked list,
>> it can create a performance bottleneck.
>>
>> This patch introduces a new list APIs that provide a set of distributed
>> lists (one per CPU), each of which is protected by its own spinlock.
> One thing I don't like is that it is per CPU. One per CPU is almost
> certainly overkill and not needed for true scalability, especially
> on systems using SMT. Also it makes the case where everything has to
> be walked more and more expensive, because all these locks have to
> be taken. Even when not contended this will add up.

When iterating the lists, the lock shouldn't be taken when a list is empty.

> It would be better to do this per every Nth CPU. Now I don't have
> a clear answer what the best N is, but I'm pretty sure it's>  1.
> For example at least on SMT systems only per core instead of per
> thread. Likely even more coarse grained, although per socket
> may be not good enough.
>
> -Andi

I have just sent out an updated patch to mapped 2 cores to each list. 
Maybe you can take a look to see if that is good enough from your point 
of view.

Cheers,
Longman