From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8D98BC27C53 for ; Wed, 12 Jun 2024 17:23:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 10C1C6B0099; Wed, 12 Jun 2024 13:23:04 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0BBFB6B009A; Wed, 12 Jun 2024 13:23:04 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EECB56B009B; Wed, 12 Jun 2024 13:23:03 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id D120E6B0099 for ; Wed, 12 Jun 2024 13:23:03 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 54926A2B11 for ; Wed, 12 Jun 2024 17:23:03 +0000 (UTC) X-FDA: 82222907046.18.302DDFB Received: from gentwo.org (gentwo.org [62.72.0.81]) by imf26.hostedemail.com (Postfix) with ESMTP id B262014001F for ; Wed, 12 Jun 2024 17:23:01 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=none; spf=softfail (imf26.hostedemail.com: 62.72.0.81 is neither permitted nor denied by domain of cl@linux.com) smtp.mailfrom=cl@linux.com; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=linux.com (policy=none) ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1718212981; a=rsa-sha256; cv=none; b=honaIokD/u/4uxAOhZqnVlEeWVYzd2IR+HXRfeAeZ7/F5re4KjYgfbZ9aXXMy7xTrFkrjs +bHFNHO9zcyjQL949JXOeroPUT4GJRMp28mY6HizCEd2gYfVCMEdP7Llu+U6OrKtb5vzs6 jmBE7ocDepVJjBegHzOkvxWExImyYrw= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=none; spf=softfail (imf26.hostedemail.com: 62.72.0.81 is neither permitted nor denied by domain of cl@linux.com) smtp.mailfrom=cl@linux.com; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=linux.com (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1718212981; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tZqaZrMbD5lZm822m/Xi5In+i2tsYxPwOp0FfWNDk14=; b=eE7HRdhE8KM8gcgTdtV6IEytdsUgmumIPqVQkY3p+uQf+9C0Zx9761qLVIv9sWRtwCbGeY ijg8NKl2bQPpJSNOC3RLfyAdX/fEo/b/kAFJPVfOwUGtfmpfB+ZLzTEUH1St7xfj9RP9+4 d08mT8oPJDrDCffunbRLDwORerlF79Y= Received: by gentwo.org (Postfix, from userid 1003) id 80E4040B11; Wed, 12 Jun 2024 10:23:00 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by gentwo.org (Postfix) with ESMTP id 7FB9440A8C; Wed, 12 Jun 2024 10:23:00 -0700 (PDT) Date: Wed, 12 Jun 2024 10:23:00 -0700 (PDT) From: "Christoph Lameter (Ampere)" To: Peter Zijlstra cc: tglx@linutronix.de, axboe@kernel.dk, linux-kernel@vger.kernel.org, mingo@redhat.com, dvhart@infradead.org, dave@stgolabs.net, andrealmeid@igalia.com, Andrew Morton , urezki@gmail.com, hch@infradead.org, lstoakes@gmail.com, Arnd Bergmann , linux-api@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, malteskarupke@web.de Subject: Re: [PATCH v1 11/14] futex: Implement FUTEX2_NUMA In-Reply-To: <20230721105744.434742902@infradead.org> Message-ID: <9dc04e4c-2adc-5084-4ea1-b200d82be29f@linux.com> References: <20230721102237.268073801@infradead.org> <20230721105744.434742902@infradead.org> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed X-Stat-Signature: 1azhzg1ayuu9qnresmjufzi44jdsisi7 X-Rspamd-Queue-Id: B262014001F X-Rspamd-Server: rspam04 X-Rspam-User: X-HE-Tag: 1718212981-386166 X-HE-Meta: U2FsdGVkX186gioWsg3CIUO9J8f7O+qq5PdDEdB604skOdAV9n3bT+UEikzRTS0hdgHkUVZWsjg8ZXgKydy6dkprCn6ZrguSBTYFsRc+Xiac5ZbAiIXL/UeTuSLzrl9MdHeQGZl24VECi/Ye/juhCnsvoB4jYkx+wsyhqjSLCEKMJmmCEIc2B9Id2shuMOz+T5hCPvFEeI56I0t9tj21ptRmbK/qKRFDfkP7u6NjAjNXqVGRgjvU8aD57y78gJu85j3i+Iqnw4HcMf+FE8wEck2apzjAat+89mgnWI3J24zPyPrzSnJqxwvEUKbT2XohG4kRlT6Xq7085f21ou8zuJyLy6ne7I/WBORiNCJN5LY4z5NpWJXWbY6haBksKO9mo9LqQ5Dl2ad0tFA0IycM9VPAiIVMwSVsrY4LABWbWohPp2hdoQ4Yr8+euu7X5KC05XvOs/lrmpyHqwlAiLpyq00pVo/TKDaUaJ3PydUE2cTappwAWx9UlZanXE2mGBx89zbbvqe6AsdrxBI1MDZcyoD3Ra8/PUXCPpViFozcyaTtLO6RtFz1hspM1ypZWE70/PHlY3aFIV1KeccsJTuFgq5ZqSsnRdK2VfgwjXEo1yUcPwnDLn8jzy8RJaCOiTP7cO2rBUHxxuFQXv2sewwjmeC0/+E7O4wjZ2XMphBMe7o8sru/Hqtoez++fh/M2P6dw/xm1rsBfHsWOtzKivqVev0ltIv1j//wB2cc+IRHhSF1JsH/WnoMGCTNM7tWdpXstYaMYZwqKBjMWCmnAiHM+QwtOcY6FnA1O9rp22yKwEy4FgNVl8fvyW6md82iIXRlMw9mJQ2MYFeookXalOW11YEgZiV/tkaj9U2SvmP/c/bD4ms7CkVr3YX4HJJHXgOeKFyyOO7kC3aQoSCgZ8fAFzGW+TRED9I8Jvgk3/mJi+twTo1rkGrffilGza10Uf1C2ntNP5aNjvDpOyL6jwo Jiomp+lg +YKPVwQZfWrNQf8jbl/5l/Lll4ati5qsD6mNS6E6Rqlc3d2h+0H8vltgX2i2io4jG9xqN/zUzkqYNH8YsY4h32KwSCZrd/AM5GPUlgYnjs+O2D/9mnOFXozUSqjcWM2n/lIkGBn5CCXNYXCtra2nRiH26oabhssKrAPQldP1CYA54Joy330FEjWQYnA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, 21 Jul 2023, Peter Zijlstra wrote: > Extend the futex2 interface to be numa aware. Sorry to be chiming in that late but it seems that this is useful to mitigate NUMA issues also for our platform. > When FUTEX2_NUMA is not set, the node is simply an extention of the > hash, such that traditional futexes are still interleaved over the > nodes. Could we follow NUMA policies like with other metadata allocations during systen call processing? If there is no NUMA task policy then the futex should be placed on the local NUMA node. That way the placement of the futex can be controlled by the tasks memory policy. We could skip the FUTEX2_NUMA option. > @@ -114,10 +137,29 @@ late_initcall(fail_futex_debugfs); > */ > struct futex_hash_bucket *futex_hash(union futex_key *key) > { > - u32 hash = jhash2((u32 *)key, offsetof(typeof(*key), both.offset) / 4, > + u32 hash = jhash2((u32 *)key, > + offsetof(typeof(*key), both.offset) / sizeof(u32), > key->both.offset); > + int node = key->both.node; > > - return &futex_queues[hash & (futex_hashsize - 1)]; > + if (node == -1) { > + /* > + * In case of !FLAGS_NUMA, use some unused hash bits to pick a > + * node -- this ensures regular futexes are interleaved across > + * the nodes and avoids having to allocate multiple > + * hash-tables. > + * > + * NOTE: this isn't perfectly uniform, but it is fast and > + * handles sparse node masks. > + */ > + node = (hash >> futex_hashshift) % nr_node_ids; > + if (!node_possible(node)) { > + node = find_next_bit_wrap(node_possible_map.bits, > + nr_node_ids, node); > + } Use memory allocation policies here instead?