From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F3258241C8C for ; Thu, 2 Apr 2026 03:15:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=198.175.65.16 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775099725; cv=fail; b=GQ71WsBhP/nB8ndy1Y55KqvujHlo8HkDqpREb/A/PqVFYGisXW9u0skEXz44eCcd05H8wM7oyUXjlOhZXk6mwhOnuNlfUFmRs4v5rwKR6FeaGpRoIc/i25fiSfbxNjoN3Ad7skKQUtOFTV1orTEp/nOnefBlU1q/3yekDUXDLdo= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775099725; c=relaxed/simple; bh=rW5/A1wOWb/YmBsiujWr1Gk7GbGceezPXR6sUUs1yrk=; h=Message-ID:Date:From:Subject:To:CC:References:In-Reply-To: Content-Type:MIME-Version; b=s5ZisblVvoXdlBiLLe9iCEA6S2Ggv8WmCJmpjvqPr85f+ZKJ0hctSaPCq55Gv2xA977uNYJATIhH4am4ABNP5mecPtiGPczKfN4RnRiUu6TFXeqeaUPZX6LhrgZNzFpJ8wEyqG7j8p7RiKM/4SfiODyBz/+5VOAr6hB9v/UCGHw= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=B1w1tiEI; arc=fail smtp.client-ip=198.175.65.16 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="B1w1tiEI" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1775099724; x=1806635724; h=message-id:date:from:subject:to:cc:references: in-reply-to:content-transfer-encoding:mime-version; bh=rW5/A1wOWb/YmBsiujWr1Gk7GbGceezPXR6sUUs1yrk=; b=B1w1tiEIJz1assoAmP5ML0WRAJf6MvGNxlljQu6itE5hoxukqd4h4lCf 6QOmfAvgdMIXUhQOH2iWQ7hAKnsrNVj490yImcTlGzm1AkZs/B7MGr3l/ 35GTwMFEy0Hdl96Vn9AX5s58B/FxClYn6jb8IUsDUctSEGCEskAGjCCd6 zzLkyhAGhYfzl0GfrBEwvC5Ul4OT4dNTf6P9Bong2m+mBbaaVLUX1MCWS viAbPblkg1U2QFZ8iAByc2T8yx8LR4KPrqyG0fdfJiOxEHmp1PvfoNR9H 9b5Gjbq1crC7mdMwGx4jpNw7cMUOddnTLT/P93uqdcNIhhJACC8IHZOVv A==; X-CSE-ConnectionGUID: N16RFGrwRXyj1DQLyrPy2A== X-CSE-MsgGUID: +IISuiI3S5KBpG89lUvdUQ== X-IronPort-AV: E=McAfee;i="6800,10657,11746"; a="76341153" X-IronPort-AV: E=Sophos;i="6.23,155,1770624000"; d="scan'208";a="76341153" Received: from fmviesa010.fm.intel.com ([10.60.135.150]) by orvoesa108.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Apr 2026 20:15:24 -0700 X-CSE-ConnectionGUID: RUqspxiLSUy3N4mAl349uA== X-CSE-MsgGUID: ng8AyiwLT26AGacpCWcoyQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.23,155,1770624000"; d="scan'208";a="222497561" Received: from orsmsx903.amr.corp.intel.com ([10.22.229.25]) by fmviesa010.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Apr 2026 20:15:23 -0700 Received: from ORSMSX903.amr.corp.intel.com (10.22.229.25) by ORSMSX903.amr.corp.intel.com (10.22.229.25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.37; Wed, 1 Apr 2026 20:15:22 -0700 Received: from ORSEDG901.ED.cps.intel.com (10.7.248.11) by ORSMSX903.amr.corp.intel.com (10.22.229.25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.37 via Frontend Transport; Wed, 1 Apr 2026 20:15:22 -0700 Received: from PH8PR06CU001.outbound.protection.outlook.com (40.107.209.65) by edgegateway.intel.com (134.134.137.111) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.37; Wed, 1 Apr 2026 20:15:22 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Z7FJBNMBpq/EqFteYBSSibgvkKo35HNhz/ybeCnX7Nmc9ueKJp8THtTIcTlGBdg684xl8ww1T6NBN/G+N7AkoD7B0ullKhpcUJt9vlJYyXwyuW1VGezCTLurxCwWtj8CArXH6Hqofji4Z7Q5PD1nJXiaJiK122VXSjENhBZn2tjD6WOit/Z/ZJD8J/e+ZeazxJiMwWrKnd5L3F8vkImTtiUi6RmW9f0JCV6xvwqcFEu5a8CaBRmvOGULrhe7GOyUBWigYGcdzdgcwoUN2BkzAcp+TLMktqHD2kM+VgyNvBF+yqODjbi3/AgJeFtShX+wZkLvlj0IDMq6s8MYWm+atg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=T9ASMyvuRPNFjlBI937+LrRdhw/iqiRMQRK60QphKjU=; b=WtUJ+njOZpSc7YOBx/HxkowWhE5AmGBpTSQJMVGyF9S/2dl3OBhv/Q7j19bklhHB9rjqkllqP1l30u/yWLavv0If/XBn0avZM5Nq5i9xHcZpI4oN9Kk+QWDcbcrIbdvftSuF+5HTxeb5/WcD2pNvWt1fpkNp210tdwQ4PFf74mi6RKGVpmWpI62haELLjGRb8xDQFXz2SrqvcGAkqqH9R1HgJNM4/33y/ndfJB8lyAleBtYhQXl+7T/SYcwJDHrxMdmPVhVfFd6xFYxMLVovDZlk5/EPcqyxcxXgkjYmv0/pAfkXUV9Z9McuuUaBnRMID0gxGtGxddQlGhqVBrqmRQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from DM4PR11MB6020.namprd11.prod.outlook.com (2603:10b6:8:61::19) by BY1PR11MB7983.namprd11.prod.outlook.com (2603:10b6:a03:52b::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9769.15; Thu, 2 Apr 2026 03:15:14 +0000 Received: from DM4PR11MB6020.namprd11.prod.outlook.com ([fe80::3058:1480:e4ac:5765]) by DM4PR11MB6020.namprd11.prod.outlook.com ([fe80::3058:1480:e4ac:5765%6]) with mapi id 15.20.9769.016; Thu, 2 Apr 2026 03:15:13 +0000 Message-ID: <64649c85-29ab-4f70-a0c4-3c83cbdae2fc@intel.com> Date: Thu, 2 Apr 2026 11:15:03 +0800 User-Agent: Mozilla Thunderbird From: "Chen, Yu C" Subject: Re: [PATCH v2 4/4] sched/rt: Split cpupri_vec->cpumask to per NUMA node to reduce contention To: K Prateek Nayak , Peter Zijlstra , Tim Chen CC: Pan Deng , , , References: <20260320124003.GU3738786@noisy.programming.kicks-ass.net> <63a095f02428700a7ff2623b8ea81e524a406834.camel@linux.intel.com> <20260324120008.GB3738010@noisy.programming.kicks-ass.net> <138c3f9d-309f-41e6-aa72-a3f6bd713bf0@intel.com> <22072ef8-5aec-49ac-9cc4-8a80bec14261@amd.com> Content-Language: en-US In-Reply-To: <22072ef8-5aec-49ac-9cc4-8a80bec14261@amd.com> Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 8bit X-ClientProxiedBy: SI2PR02CA0029.apcprd02.prod.outlook.com (2603:1096:4:195::6) To DM4PR11MB6020.namprd11.prod.outlook.com (2603:10b6:8:61::19) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR11MB6020:EE_|BY1PR11MB7983:EE_ X-MS-Office365-Filtering-Correlation-Id: 6d0b96fe-d6a8-47ab-6ff2-08de90660e0c X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|1800799024|376014|56012099003|22082099003|18002099003; X-Microsoft-Antispam-Message-Info: JH6XoPuYHfv8I+q4TKIA32MOoOXFE2ov/jzSHI2+LOI40/I7esEF0yg9KC3Sv7tSYEQLMHH282nx+7Gq3kF2OBfSXTm+92/FkAFfncCiYv/un6wY/qlfSNpFHo4UwXO63aSugaYRmQ7i5Pyo2RD/u4sNzy4VudZx75d9fgit9u3EVQOZlw2C0+N05jyA/YpOkxr1+dRBMFF1Bu5gSa6ccxy99ReFuQmuFXLpA7hxDT73ksDkra0LnSCJvDyOucdOzFKnQDKe5VnAo976g+OzEpr9cZXd+qsXm/n/Q8f98kyteyCpQrk5m+yXrIhBKCCDcOXjvMwSnufiDWCA/VxdCKxP/a4x4lufW874XxhR0k/85o3AloYKOUJc55gRKgFhRBkF8U4dNY5XwLaPpDUGaLS8EwUZevWWjIeFMXahq+lbMl8hm2cViIunU3nmkOs7D+HlGDHViqxzk09CWv0jsydq+NwP5LoTb4n+mL68fOX7z6Kq00yh6sxZUuL1ha091XPCOQg9IHt4x/VYHt/lzOSmgvYplfVOMNsT5TyoRw7rdliN0QnfycIIJtVn8hnl7J6BmW1mlzzkdr7hk3Fxh3x4xzkxumaXF9f1NZXOdjM2rVCfRY9vIyhPSVstv7+egYtpUUlivm5oYpjWKHtZ4Oxqb8XSVqjOvVK4GhQzZ/ZS2yN79BdP8h8n5MeEW1riHNRrJ/kUlrCx8TYWjU1xEhhseYDO+a1wRVzD2CD8RhM= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM4PR11MB6020.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(1800799024)(376014)(56012099003)(22082099003)(18002099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?RnhxTWc2ZlJkRWxrTlFiaFRUZ1ZWbWlSWVRSOFYvUHpuSW94MTJuaVNOZlRG?= =?utf-8?B?VC9Fd0R0dURUSCsvYWVRR25TV3l1azY2KytHT2kvazJLSTZ1ZHZSY1I4MUg4?= =?utf-8?B?Wkx2cElLZ01PeUJxYlFIWEdhK0wwTzErM05XWEdjZzNTZzRUL0l1ZG15RWli?= =?utf-8?B?Q1ArTzl3YkJpcU5xLzMzZFZhODgrUjdMbFBpU3A5SlMrMWx0cCtHVUhtZENh?= =?utf-8?B?UE9QTXZBdkxCM3d2WXV2NXM4OXJ2MXArVFg1am5QOGRKZm9ueExSaE5uZm1J?= =?utf-8?B?cDlBSGw5UHd0S3ZhWWdWYTA0TTJESHprKzlJZzhaNmo1QmNNNkFCNExPbll3?= =?utf-8?B?QlZnU1lQOVZQMVI2SytNZWYvZExWNVNLUTIrczZ1SHN4ek4ySlFSMzNwZ0N3?= =?utf-8?B?d2ljcWtMckxoNVhXUUtDUnpDNHY5ZFlNeTA4LzQyb28zeU9KQlp0eHhDYXVh?= =?utf-8?B?QUNnMlFyNkpMRWhhYkhYaXpyYXhnaWdTR1RaLzAzVnhIaDhFSzQ3TldsbFVN?= =?utf-8?B?NTA0WTk4a3AxWW9KQktoZThsdGVEYmw4R3RJTzNBdXAxbEtFbUV3cW1ieTMz?= =?utf-8?B?OHFTSG5GVEIyU2tXN3p0cTRXWE9XSnZ6VWZNcCtwZHpYeENZVUx5NitrN3RL?= =?utf-8?B?VENIVG9obVlaMzJPUkQzSEtlRURua0puM0RMZ1hyS2NQZjAxR3VNNmRqZVR2?= =?utf-8?B?OEtvbVl4cTZ3RTRTVm0vNGIwVW9qS1NiQ3BNaWFXaUVIVEM3LzV3V3ZsOERq?= =?utf-8?B?WERsNCswUkgzWVpWRklhcVRuV0dBTndVUHRlWkoyckRFMms5enh2MWlPUFU4?= =?utf-8?B?QU9jYjE0Q0JuRFd0ZnFCa1RDbHJoT1l6OXU0UUdIcWJYWEMxeWhDU2JUeldp?= =?utf-8?B?T0VUVHVENGRRaFpLK01IYjIyQW05NXF6cHA4aCtrcEVlR29NZERYMFhlMFNz?= =?utf-8?B?REw5S2t5NGtpK1MxMlVLeHQvL1VIVUhvOVlKSm5XYTRERTNDRXhPbjVZQjJI?= =?utf-8?B?WW5vOWgvaE9rMHdEcm5RMkVoQzQ5UlNJL2lpV2s4YUhhNWFya1BvY09YbWRv?= =?utf-8?B?Zm0zaitQSlRrWTJEMm9JamdNNnV5aWdpb3cxQk1oMXNXeFEreVdZN3lpTGlG?= =?utf-8?B?NTFVYXFoOWxwUWRWOGJaazhaTDJiMThEaTV6NkNvMDJULzM3ajNPQzZzdXhh?= =?utf-8?B?MmNiVHNEQUZiMWJtQm8rdmZXYUdBcFNzOU50d3Q1c1BZeFQzMDlPbUtmYlM5?= =?utf-8?B?NVhpcEdwQVp5WndEajZXbWpjZldjUTA4SWpyVjNnYkFkQjk5U2RkbHJtQzho?= =?utf-8?B?eWJsMzlMckdvbUphekNycms5bW5sc2lCSGUvdEdBRGpQekNYckFYck5JTldM?= =?utf-8?B?YTZwSmw3MElHMElMeGl2N0NQTkx2aldMdkx6dnhGS1cyYXRzeEpodmJCQjFq?= =?utf-8?B?WWxuUFlmVld3SG5RaUZNOHAxbTRyUXBsbGdZS283Mk0rQ0pMZk5oV1c4azRr?= =?utf-8?B?QzhOL1pKOFFVSU5zZTFVc3B5WjRGM1gwNTQzNmJSbjlOS1p5SXJVNG4reGpv?= =?utf-8?B?Rk1wRjRiZ3MzSGNsNCtDc1ZBZDFHZVEwcTdKZ0RsSlZUaGV3dFJWT2VKZE4x?= =?utf-8?B?dnMrVVhLZ1BXM0JvdDRCYnl4S3NzU0pSWEM1RWg4T3A5Z2JYeEtLU3p1TUFi?= =?utf-8?B?UnJ3TXZjNUh0b1RsYTVmTWdLa1AxWDRoT040N0xxazBkT3N1OVV6bU9peGhC?= =?utf-8?B?SmRpTWdRcEJZZXBCTFJBL1BxOU1pa0lNS2p1bDI4SklzTWEyQ09oeFFHMnpC?= =?utf-8?B?M2owczFqS2tWOVI3N014QVdlWEw1UW8zV2N1RmJMa3NYVERpR1FlRjdjWm41?= =?utf-8?B?cFB6OVBhd1h3SnJCRU1zb05LT0EyWFovbnhtZURMczBuSkVvSk9qcitOUEVF?= =?utf-8?B?Y2lMT3VheGlhU1dDZXo2eFc5bGtNQ3dZUGpETDM2YWQwQ1Nod29rZmtKSTM0?= =?utf-8?B?dEFoKzk0MU5GWjdFUy9xTTVoKzFpRWIvQ0NmSTFrR1Rja3dUSWVoSWJQVWZx?= =?utf-8?B?dlRQYitPR1loSGN3anVsdGtrQzgwa1dmSzZHYzhEMWdyUmNTSUpWUTFZVTVs?= =?utf-8?B?eGZwT3BhVDlGNmZDb0ZOak9tK1gvcTNOcTRpR3ZES2J3RHV2T0FCSzh1ekFM?= =?utf-8?B?WG9zWHlIWFpZK1lkaWhYQlhaOGVvaFErdU8xYUpvcEV6WGR5T0I2ZmZTaW5G?= =?utf-8?B?YzR6b0lrNk1WRlhmdHNZWSsxSWFzc25hQkdXQVJ2SlZIb1NWazFtWHRvNk1E?= =?utf-8?B?L1poem5LVkVxRTYxSzdlV0gzNXhPU29uaXVKcE9NU2dYVG95UW9zdz09?= X-Exchange-RoutingPolicyChecked: lWt2SjDZbTlfqJ0KUar8An4TQWSjOJXB4vfLSHF0B1K88SHFTNzd8AIUQ+BLmHy9Sg3iBYzfSeRXk2jxE+xXEt7/igEBcZC1C1VBTYaztB26jCtVvtVWQd62CE7FPdlntDs3O13VLRMcAmUqZMmL42T7rTQV1FCRcxAd0h8BNEwyfA87C1xhKEGdcYxMhY/iYicntkCbL9snWFs0qAHz8pSB64HI31VjKFdcfAJR/EfkmFdpXxmh5v+1Pj70zhgssBZsBkqRab8Q6w1W9cVpo9tdvJecpb5v9UfDRno9Vxbs2WZzBpdqM2eJxFIETN4c9Xx+T0nVkkIWZDSaZyuAZQ== X-MS-Exchange-CrossTenant-Network-Message-Id: 6d0b96fe-d6a8-47ab-6ff2-08de90660e0c X-MS-Exchange-CrossTenant-AuthSource: DM4PR11MB6020.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Apr 2026 03:15:13.2622 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: qtWV2wzS7t+5d229T7YSZKchLKjBxq3dXwhBFD6DwZ7o/FT9gJhOwOEhnEI71tautj91xre9cDVL2wN7j861sw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY1PR11MB7983 X-OriginatorOrg: intel.com Hello Prateek, On 3/31/2026 6:19 PM, K Prateek Nayak wrote: > Hello Chenyu, > > On 3/31/2026 11:07 AM, Chen, Yu C wrote: >> update of the test: >> With above change, I did a simple hackbench test on >> a system with multiple LLCs within 1 node, so the benefit >> is significant(+12%~+30%) when system is under-loaded, while >> some regression when overloaded(-10%)(need to figure out) > > Could it be because of how we are traversing the CPUs now for idle load > balancing? Since we use the first set bit for ilb_cpu and also staring > balancing from that very CPu, we might just stop after a successful > balance on the ilb_cpu. > > Would something like below on top of Peter's suggestion + your fix help? > > (lightly tested; Has survived sched messaging on baremetal) > > diff --git a/include/linux/sbm.h b/include/linux/sbm.h > index 8beade6c0585..98c4c1866534 100644 > --- a/include/linux/sbm.h > +++ b/include/linux/sbm.h > @@ -76,8 +76,45 @@ static inline bool sbm_cpu_test(struct sbm *sbm, int cpu) > return __sbm_op(sbm, test_bit); > } > > +static __always_inline > +unsigned int sbm_find_next_bit_wrap(struct sbm *sbm, int start) > +{ > + int bit = sbm_find_next_bit(sbm, start); > + > + if (bit >= 0 || start == 0) > + return bit; > + > + bit = sbm_find_next_bit(sbm, 0); > + return bit < start ? bit : -1; > +} > + > +static __always_inline > +unsigned int __sbm_for_each_wrap(struct sbm *sbm, int start, int n) > +{ > + int bit; > + > + /* If not wrapped around */ > + if (n > start) { > + /* and have a bit, just return it. */ > + bit = sbm_find_next_bit(sbm, n); > + if (bit >= 0) > + return bit; > + > + /* Otherwise, wrap around and ... */ > + n = 0; > + } > + > + /* Search the other part. */ > + bit = sbm_find_next_bit(sbm, n); > + return bit < start ? bit : -1; > +} > + > #define sbm_for_each_set_bit(sbm, idx) \ > for (int idx = sbm_find_next_bit(sbm, 0); \ > idx >= 0; idx = sbm_find_next_bit(sbm, idx+1)) > > +#define sbm_for_each_set_bit_wrap(sbm, idx, start) \ > + for (int idx = sbm_find_next_bit_wrap(sbm, start); \ > + idx >= 0; idx = __sbm_for_each_wrap(sbm, start, idx+1)) > + > #endif /* _LINUX_SBM_H */ > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c > index a3a423c4706e..f485afb6286d 100644 > --- a/kernel/sched/fair.c > +++ b/kernel/sched/fair.c > @@ -12916,6 +12916,7 @@ static void _nohz_idle_balance(struct rq *this_rq, unsigned int flags) > int this_cpu = this_rq->cpu; > int balance_cpu; > struct rq *rq; > + u32 start; > > WARN_ON_ONCE((flags & NOHZ_KICK_MASK) == NOHZ_BALANCE_KICK); > > @@ -12944,7 +12945,8 @@ static void _nohz_idle_balance(struct rq *this_rq, unsigned int flags) > * Start with the next CPU after this_cpu so we will end with this_cpu and let a > * chance for other idle cpu to pull load. > */ > - sbm_for_each_set_bit(nohz.sbm, idx) { > + start = arch_sbm_cpu_to_idx((this_cpu + 1) % nr_cpu_ids); > + sbm_for_each_set_bit_wrap(nohz.sbm, idx, start) { > balance_cpu = arch_sbm_idx_to_cpu(idx); > > if (!idle_cpu(balance_cpu)) > --- > > This is pretty much giving me similar performance as tip for sched > messaging runs under heavy load but your mileage may vary :-) > Thanks very much for providing this optimization. It should help more nohz idle CPUs-beyond just the currently selected ilb_cpu to assist in offloading work. When I applied this patch and reran the test, it appeared to introduce some regressions (underload and overload) compared to the baseline without Peter’s sbm applied. One suspicion is that with sbm enabled(without your patch), more tasks are "aggregated" onto the first CPU(or maybe the front part) in nohz.sbm, because sbm_for_each_set_bit() always picks the first idle CPU to pull work. As we already know, hackbench on our platform strongly prefers being aggregated rather than being spread across different LLCs. So with the spreading fix, the hackbench might be put to different CPUs. Anyway, I'll run more rounds of testing to check whether this is consistent or merely due to run-to-run variance. And I'll try other workloads besides hackbench. Or do you have suggestion on what workload we can try, which is sensitive to nohz cpumask access(I chose hackbench because I found Shrikanth was using hackbench for nohz evaluation in commit 5d86d542f6) thanks, Chenyu CPUs