From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1758014AbbIVO2N (ORCPT ); Tue, 22 Sep 2015 10:28:13 -0400 Received: from terminus.zytor.com ([198.137.202.10]:50131 "EHLO terminus.zytor.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1757882AbbIVO2L (ORCPT ); Tue, 22 Sep 2015 10:28:11 -0400 Date: Tue, 22 Sep 2015 07:27:33 -0700 From: tip-bot for Rasmus Villemoes Message-ID: Cc: mingo@kernel.org, tglx@linutronix.de, fengguang.wu@intel.com, dave@stgolabs.net, linux-kernel@vger.kernel.org, bigeasy@linutronix.de, peterz@infradead.org, linux@rasmusvillemoes.dk, hpa@zytor.com Reply-To: linux-kernel@vger.kernel.org, dave@stgolabs.net, fengguang.wu@intel.com, tglx@linutronix.de, mingo@kernel.org, hpa@zytor.com, linux@rasmusvillemoes.dk, peterz@infradead.org, bigeasy@linutronix.de In-Reply-To: <1441834601-13633-1-git-send-email-linux@rasmusvillemoes.dk> References: <1441834601-13633-1-git-send-email-linux@rasmusvillemoes.dk> To: linux-tip-commits@vger.kernel.org Subject: [tip:locking/core] futex: Force hot variables into a single cache line Git-Commit-ID: ac742d37180bee83bc433be087b66a17af2883b9 X-Mailer: tip-git-log-daemon Robot-ID: Robot-Unsubscribe: Contact to get blacklisted from these emails MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=UTF-8 Content-Disposition: inline Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Commit-ID: ac742d37180bee83bc433be087b66a17af2883b9 Gitweb: http://git.kernel.org/tip/ac742d37180bee83bc433be087b66a17af2883b9 Author: Rasmus Villemoes AuthorDate: Wed, 9 Sep 2015 23:36:40 +0200 Committer: Thomas Gleixner CommitDate: Tue, 22 Sep 2015 16:23:15 +0200 futex: Force hot variables into a single cache line futex_hash() references two global variables: the base pointer futex_queues and the size of the array futex_hashsize. The latter is marked __read_mostly, while the former is not, so they are likely to end up very far from each other. This means that futex_hash() is likely to encounter two cache misses. We could mark futex_queues as __read_mostly as well, but that doesn't guarantee they'll end up next to each other (and even if they do, they may still end up in different cache lines). So put the two variables in a small singleton struct with sufficient alignment and mark that as __read_mostly. Signed-off-by: Rasmus Villemoes Cc: Peter Zijlstra Cc: Davidlohr Bueso Cc: kbuild test robot Cc: Sebastian Andrzej Siewior Link: http://lkml.kernel.org/r/1441834601-13633-1-git-send-email-linux@rasmusvillemoes.dk Signed-off-by: Thomas Gleixner --- kernel/futex.c | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/kernel/futex.c b/kernel/futex.c index 6e443ef..dfc86e9 100644 --- a/kernel/futex.c +++ b/kernel/futex.c @@ -255,9 +255,18 @@ struct futex_hash_bucket { struct plist_head chain; } ____cacheline_aligned_in_smp; -static unsigned long __read_mostly futex_hashsize; +/* + * The base of the bucket array and its size are always used together + * (after initialization only in hash_futex()), so ensure that they + * reside in the same cacheline. + */ +static struct { + struct futex_hash_bucket *queues; + unsigned long hashsize; +} __futex_data __read_mostly __aligned(2*sizeof(long)); +#define futex_queues (__futex_data.queues) +#define futex_hashsize (__futex_data.hashsize) -static struct futex_hash_bucket *futex_queues; /* * Fault injections for futexes.