From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from DM5PR21CU001.outbound.protection.outlook.com (mail-centralusazon11011053.outbound.protection.outlook.com [52.101.62.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 83B2C28E0 for ; Thu, 9 Apr 2026 05:17:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.62.53 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775711834; cv=fail; b=d2nQX7XRbJU0M4Tq4fba8TG85tZBel+lrmeePPlFTwhb6bRvahqmipJZwnkAPlWhYDM864JCbD7Pid0DZK40uBNkk/EzFG5FEwZUD6v0Z3egGga14jkbn0+PYIZT9pBzXCKNPUSb8RIEKySjn5QG1QXrmlbEEyQaoZvm1sc/UDU= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775711834; c=relaxed/simple; bh=OfnsO4aifF/XQMRuLZF6XKY4zaO484H0NswxoXB1ybg=; h=Message-ID:Date:MIME-Version:Subject:From:To:CC:References: In-Reply-To:Content-Type; b=PGreU2uKkaS0Hfzj1raYIA0zc7fAYLA6p6ezoo/t9ipty3b7JFl/ZVKKuWngYdWdUeMDUXpSwlpNpe/cHKtPPkjF1I60uYuzexIg0HGp46f/tb4xQE/j/lAEiDszx1SjMblk+JNF7+YYatkDs5V53NJ/+alu57w3nfGDKD6zX4Q= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=en2hk5mK; arc=fail smtp.client-ip=52.101.62.53 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="en2hk5mK" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=uVLTsm9mtm9EsZZcFFHvqxqV++Yy2lWCXNmQ9cMt4twEgnPG2DPXy0iMvh/yCM00ytId/PEO+mYsoacUc6rqNZO0GwVh7aDWkYoVzlxn4CVzYxiIK32upickQNwPzEQei3CPJnpTmNAsO3G88/PBL48k4nFieEx5KWgd0OohWWdexYjw51g2kIlPGPcN8YQIy9cGaYX5CjtzpKhyUl/HWyJlOMi4Y/8XgITjE/c0MTnz4KSggErNzxd2GhFKZxT/L5ZsbrvCyfxgbeMew/hySK5+1bLbNWt8yb8eE+4ibmJersvDQOA6l95oFgdj3MjgElE2OMsgEF2GsayR4JHQxw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=tnV8lBW0AMdEkd2gcy0njRG/Re82ckW+RWUK8HlTJOM=; b=uHPsSz8X8tkIP6myXUaCePT6aUeuwd6RoN23+sUFP4ib+N0aHjNeyBd2osqkGdNhLjim1MW7scrX15r5d6daltDryvldB3sd+BdInkQYNWCRjJZXWSl+rH7BrSo9VVK+OrQX+6X/H0e2NGfERLW2Me083oh3CBy0sC9h3xVCrm1z54VhsXUnk6cZxIEy9ui1vS9mqY1n2XqjuQmwfMO23qJiAOm5wKyXHndOCN5ozNI37IdhAB+5Jx0ZEHfHFtq1JL//Mdc/+VbyxSgI3zkggZqZXK+cCAMnyQZiUQWpUmXI5lkKmgRnzwLleAdOWyLv24bGAYhDCjhyz6lxi97jZw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=intel.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=tnV8lBW0AMdEkd2gcy0njRG/Re82ckW+RWUK8HlTJOM=; b=en2hk5mK8Z7xVpD0zeB46dp0E+3E7OdELhlVqC7n6rYWGNV5uqPVZSs7/dUQytDGuuQY7m5gEupBbYLLLGBbjW4KP1S32c8jjbTR4exCa25IM9OEYBQ8cZP4CiXcgfzFSfc2x5H4nCRi7ybXvP7lI6mgsNFpMcI3wPexId4ENr0= Received: from MN0PR02CA0026.namprd02.prod.outlook.com (2603:10b6:208:530::7) by BY5PR12MB4292.namprd12.prod.outlook.com (2603:10b6:a03:212::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9769.15; Thu, 9 Apr 2026 05:17:09 +0000 Received: from BL02EPF0001A100.namprd03.prod.outlook.com (2603:10b6:208:530:cafe::6c) by MN0PR02CA0026.outlook.office365.com (2603:10b6:208:530::7) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9769.37 via Frontend Transport; Thu, 9 Apr 2026 05:16:59 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=satlexmb07.amd.com; pr=C Received: from satlexmb07.amd.com (165.204.84.17) by BL02EPF0001A100.mail.protection.outlook.com (10.167.242.107) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9769.17 via Frontend Transport; Thu, 9 Apr 2026 05:17:09 +0000 Received: from satlexmb07.amd.com (10.181.42.216) by satlexmb07.amd.com (10.181.42.216) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.17; Thu, 9 Apr 2026 00:17:09 -0500 Received: from [10.136.36.199] (10.180.168.240) by satlexmb07.amd.com (10.181.42.216) with Microsoft SMTP Server id 15.2.2562.17 via Frontend Transport; Thu, 9 Apr 2026 00:17:07 -0500 Message-ID: Date: Thu, 9 Apr 2026 10:47:06 +0530 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v2 4/4] sched/rt: Split cpupri_vec->cpumask to per NUMA node to reduce contention From: K Prateek Nayak To: "Chen, Yu C" , Tim Chen , Peter Zijlstra CC: Pan Deng , , , References: <20260320124003.GU3738786@noisy.programming.kicks-ass.net> <63a095f02428700a7ff2623b8ea81e524a406834.camel@linux.intel.com> <20260324120008.GB3738010@noisy.programming.kicks-ass.net> <138c3f9d-309f-41e6-aa72-a3f6bd713bf0@intel.com> <22072ef8-5aec-49ac-9cc4-8a80bec14261@amd.com> <64649c85-29ab-4f70-a0c4-3c83cbdae2fc@intel.com> <20260402105530.GA3738786@noisy.programming.kicks-ass.net> <93d7eb33-c3a5-4498-bc26-57806b73d9e0@amd.com> <3b66e8e8-07e0-4f3e-a3ba-d97133af5162@intel.com> <1c742a1d8ecd8e314d704d46a44e2b8893479e50.camel@linux.intel.com> <2881a07f-ff14-4faa-9da7-3fbe206a463d@amd.com> <14eda829-fc6b-407a-93a3-0794ab521177@intel.com> <2dcbf93d-030d-466c-8b1c-8387513e9eb9@amd.com> Content-Language: en-US In-Reply-To: <2dcbf93d-030d-466c-8b1c-8387513e9eb9@amd.com> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL02EPF0001A100:EE_|BY5PR12MB4292:EE_ X-MS-Office365-Filtering-Correlation-Id: 3f30b67d-5832-455a-6f61-08de95f73fee X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|1800799024|376014|36860700016|18002099003|56012099003|22082099003; X-Microsoft-Antispam-Message-Info: 1lFBLh8+tpF3eIcINKI0uOVhGrmF7i6oQEP+YQkR9myW28c3NFMEIfEfj0f/btW0lV9kuVRjJsw1UQkteuV86w+eEmkGVNldG69VFSVKdj04e/Rt5jAWb7YBwKa0S2exs7ownIeGADLsrMkcpRZFw4ata7f9jk7dxAlQ7ZiWzqMDQeA1AavwmQOBi0PEkDcePuVpPqSb0vOzzj7hOBu1RtpyoqvksckDpry5ebFn4EKr5QcnwOtrD6xZd2VSMpoiTOJhbk5lU3p+gOlA4EDC741F1Br/p59hwqzOa1MDnJiIVY8XxO1BzdcQdJkrBaJG2cRwyoqy9TUietTOBiQQaKN4Er7oCW8Wr4obMCMzi/vDah8rAExnf6JDV2Am/IW+adzsr7mvuHyM96Zk30Gt1SDyeMrNRafL0mgwejP4ij7XQP1p6S8MOPW1F1iStfm4qyCCy4/bjlJgTcefQMKR99KhKZcktLZXkVEUBexqmuuvOjIr9OuBqTsmiJU2dJsKbRX7kxoU0V7u9jmoU8EKmwv94Y0rV7Zkw/ydLv8JEhgqMTJtz+hm7jxkeyhwnQgRnTZbvfuRSXSvoTxqLi9/1ywm+WO+ZMhHuTDVLOeDGMDv6lECPJd2Pvyg1dPjN9Wb2vuavb2pZb3Zo5mLRPJg5fF6Cbc9LHxgYuzBixoO3Q7jYX2zrnZAvzupxa4a1f6L1DhVGurCKTsOnyuDIc5mEPUSoWn6FAXFeE2JQbn3Hm5KG3bS3F11ylEUrUZSsI/XBDUVgpGSnNqoFP4MBkWpnA== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:satlexmb07.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(82310400026)(1800799024)(376014)(36860700016)(18002099003)(56012099003)(22082099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 8AKSFS52327m1Z1s/W/gHJFcfFd8y/zTABZ1m/UpX4pF3N+/BlNJJF8E2qmsnxufYfVuFpCYfJLAxS/AXJg5Z7vrwnskmJ87wWNpqqEH+rN3NF9t0DnruW+L0QLIq/4PSYRet4wQ7d52cwr0MPJwHhSEAhNeK6DtgAIEAzDIf525l/DYFnbaiP3oMS2mUIcL/mvTcFwzjPwO4vw5cC5AZcccneSnRaaVUDayULl9MfY/kx32ASMBm6tQZIxQfPbGglG/STzVWF7Yyxgga0Vpml1hKDJganJ5iJir6DPJ0kwijdQfl+2p/DsC9lYVNGPQWC8EZQsU71xjqPdfK8+81UJpWZ9zImDy6dkOjOQGdHjKrT+8xsS+BoBHIoeBGLj6FXtuiFoIgYiROVEFmI132VM6BtIHnTMTxn5KGXIaWManhfE3ZqL30BO4mO04BwQ+ X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Apr 2026 05:17:09.4761 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 3f30b67d-5832-455a-6f61-08de95f73fee X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[satlexmb07.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BL02EPF0001A100.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR12MB4292 Hello Chenyu, Tim, On 4/8/2026 9:22 PM, K Prateek Nayak wrote: > Hello Chenyu, > > On 4/8/2026 5:05 PM, Chen, Yu C wrote: >> We haven't tried breaking it down further. One possible approach >> is to partition it at L2 scope, the benefit of which may depend on >> the workload. > > I fear at that point we'll have too many cachelines and too much > cache pollution when the CPU starts reading this at tick to schedule > a newidle balance. > > A 128 core system would bring in 128 * 64B = 8kB worth of data to > traverse the mask and at that point it becomes a trade off between > how fast you want reads vs writes and does it even speed up writes > after a certain point? > > Sorry I got distracted by some other stuff today but I'll share the > results from my experiments tomorrow. Here is some data from an experiments I ran on a 3rd Generation EPYC system (2 socket x 64C/128T (8LLCs per socket)): Experiment: Two threads pinned per-CPU on all CPUs yielding to each other and are operating on some cpumask - one setting the current CPU on the mask and other clearing the current CPU: Just an estimate of worst case scenario is we have to do one modification per sched-switch. I'm measuring total cycles taken for cpumask operations with following variants: %cycles vs global mask operation global mask : 100.0000% (var: 3.28%) per-NUMA mask : 32.9209% (var: 7.77%) per-LLC mask : 1.2977% (var: 4.85%) per-LLC mask (u8 operation; no LOCK prefix) : 0.4930% (var: 0.83%) per-NUMA split is 3X faster, per-LLC on this 16LLC machine is 77x faster and since there is enough space in the cacheline we can use a u8 to set and clear the CPu atomically without LOCK prefix and then do a >> 3 to get the CPU index from set bit which is 202x faster. If we use the u8 operations, we can only read 8CPUs per 8-byte load on 64-bit system but with per-LLC mask, we can scan all 16CPUs on the LLC with one 8-byte read and and per-NUMA one requires two 8-byte reads to scan the 128CPUs per socket. I think per-LLC mask (or, as Tim suggested, 64CPUs per cacheline) is a good tradeoff between the speedup vs amount of loads required to piece together the full cpumask. Thoughts? -- Thanks and Regards, Prateek