From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9B7C63A9015 for ; Tue, 12 May 2026 09:29:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=192.198.163.12 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778578199; cv=fail; b=m1wRsiFBEdzBCUCoHL8JXO7JRwfJkzdpBDrDWWcrd5VVpBa+snEn2Rj0NIvW0PZQyN6Pb7roTMQJ01vYwCEH+6UyB8KH8T0I7LMFQInFGK+K1vQIe8FpJi7wAZAwdpOUSrEJMi4kUKuA5rlDvNzYp6a7NQik/EQGWYn+UYCXUdw= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778578199; c=relaxed/simple; bh=814sIsqc1TcY1FMI1NHv7uuk/+Hie1YUsKe58yG+g5A=; h=Message-ID:Date:Subject:To:CC:References:From:In-Reply-To: Content-Type:MIME-Version; b=MH9GlNtS2Ov9G1y4BI8qQpiskHGBY/t9DiKiuxgnSWS1z7RNxZ2Z88TmA+wzKi2OoBvnnQ5zEWeVZv449/4dZjlF/iNgeyfDc73EHVLIO0ZIlW3vLyW1ZeNAAokEPfDqUsdSmY4YtJgLwjMyVJQcxsE4eYtfprEU0xT7+MBRaGo= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=F2mNjtnD; arc=fail smtp.client-ip=192.198.163.12 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="F2mNjtnD" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1778578197; x=1810114197; h=message-id:date:subject:to:cc:references:from: in-reply-to:content-transfer-encoding:mime-version; bh=814sIsqc1TcY1FMI1NHv7uuk/+Hie1YUsKe58yG+g5A=; b=F2mNjtnDf8Mlrm9GIksrMfYVDWdJFmnokvxJy0S8oC3WuZad6941nxIp 5BEDkYfXf9ZnqutWomqKHJb1OyBgG4Yfog/NTCa9HSKOMx/09LSo5Hoos z+cz6q4PKGmEm8UWuc6hlGX6MCe8EdZ8D5ZvqTHT7JUA8aGsLjuijwO3W EBGM8A4bmegbYTZdWG/3QprhLnMjmgkm4vRdCQyInyZWgurU3CiIxwN1m NIZ1b+J+SCSUROIWhYuIW/4VYL7PJUrffHQz3n3J2+EFKUGBxFLjcal/o n90MqIHagjRIECqr1nZQbLs7bKKm+7dm7EI/8rnsuwrwJyLJRQKpf89RN w==; X-CSE-ConnectionGUID: FvcKCvmZSOaUUphKsovDxA== X-CSE-MsgGUID: sZhLU94CTbiHxUIGfu/HMw== X-IronPort-AV: E=McAfee;i="6800,10657,11783"; a="83335203" X-IronPort-AV: E=Sophos;i="6.23,230,1770624000"; d="scan'208";a="83335203" Received: from fmviesa010.fm.intel.com ([10.60.135.150]) by fmvoesa106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 May 2026 02:29:55 -0700 X-CSE-ConnectionGUID: e6WPbr+fRDG9+dJxamxd3A== X-CSE-MsgGUID: g8fspZD/TIGw7v5eqSfbyQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.23,230,1770624000"; d="scan'208";a="233407358" Received: from fmsmsx901.amr.corp.intel.com ([10.18.126.90]) by fmviesa010.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 May 2026 02:29:55 -0700 Received: from FMSMSX903.amr.corp.intel.com (10.18.126.92) by fmsmsx901.amr.corp.intel.com (10.18.126.90) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.37; Tue, 12 May 2026 02:29:54 -0700 Received: from fmsedg902.ED.cps.intel.com (10.1.192.144) by FMSMSX903.amr.corp.intel.com (10.18.126.92) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.37 via Frontend Transport; Tue, 12 May 2026 02:29:54 -0700 Received: from SJ2PR03CU001.outbound.protection.outlook.com (52.101.43.22) by edgegateway.intel.com (192.55.55.82) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.37; Tue, 12 May 2026 02:29:54 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=XpJWITnQeRXxPVYVrmlfCjzPYAUy3Gq73/YvWnA6MOg848p4k9qpA13qVLUu/6DantvkhqzyJS9juL7qX47EtCqrf4USZLljuobPfAwhzvL7G7MYhOiwW6UYdka6qh64LnfniuRZDe9k29kb+mR0M66UNZyEhl3hpbdxuNww2t3hNwt85wOL/Nq1f9X0Gl4sqFbd+SoOfuyobRImLxAoWn4Pf+BvX8Du8fEHxUUxFEpj6VxTl163lHiCboXUdODU34PLYs2FwmteoAKihC028mjHBA7tyMLvUKiKkDgmkMSjmG6ecIF1FCIMeD8KTLNvhzSdoIyUGvpK4owXWpV2Vw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=yl8A9YQwBKR5PptbFvlv7hJ3lOErkuylSzNQj50BGGo=; b=eVlL8NcxiRufXB5E5KOpJuAOdRafD0IB8EH6HFC2gIJTjkKXcMRBTozbY40XCZ5mL9KNlQarXI5E5ZSTQjAw52/34o+NYCrf7YB7o+pkVyESN00QGy18hqqUtgjzJ24mfMM0+IDDHgEgwQXEsuNwThefU5kXDSdwJqYlOzuq3En8x6dia/c4Z2D+EuJYY/VLEAzUz65O19XGrblAUZzf306qE7MJaGm7KKMO55wR2Fl+0yWVbITcyjVV9eaxwtIMq8Kceha+MdpponLeT3wkiw0lvWo8wvY5oIvt4HEkwgwwEQkEHq/P8rzTmWEcxNwlMH4KL4hE0Mi+edhtkuUijA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from DM4PR11MB6020.namprd11.prod.outlook.com (2603:10b6:8:61::19) by SJ1PR11MB6300.namprd11.prod.outlook.com (2603:10b6:a03:455::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9891.15; Tue, 12 May 2026 09:29:47 +0000 Received: from DM4PR11MB6020.namprd11.prod.outlook.com ([fe80::3058:1480:e4ac:5765]) by DM4PR11MB6020.namprd11.prod.outlook.com ([fe80::3058:1480:e4ac:5765%6]) with mapi id 15.20.9913.009; Tue, 12 May 2026 09:29:47 +0000 Message-ID: <1dab1e94-43d1-4528-986d-0bed73e84bae@intel.com> Date: Tue, 12 May 2026 17:29:40 +0800 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH 3/3] x86/sbm: Derive leaf granularity from LLC cacheinfo instead of topology domain To: K Prateek Nayak CC: , , , , , References: <729726b9-c669-41e2-887d-bdf9da703034@amd.com> <20260510155920.2587431-1-yu.c.chen@intel.com> <20260510155920.2587431-4-yu.c.chen@intel.com> Content-Language: en-US From: "Chen, Yu C" In-Reply-To: Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 7bit X-ClientProxiedBy: TPYP295CA0044.TWNP295.PROD.OUTLOOK.COM (2603:1096:7d0:7::18) To DM4PR11MB6020.namprd11.prod.outlook.com (2603:10b6:8:61::19) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR11MB6020:EE_|SJ1PR11MB6300:EE_ X-MS-Office365-Filtering-Correlation-Id: b0e15ce8-e813-49a4-1e7a-08deb009025c X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|376014|1800799024|18002099003|56012099003|22082099003|11063799003; X-Microsoft-Antispam-Message-Info: mtx4lGuB01C+1+WNOQMcy6VW+sn6Qf9r5K6PdStgJJgSeqRoGE+krGvBkILeNXVRFTCKx0APAUoEUDecQvWcAz8h2f7KyKURk6u2z9c0Qj4G6cThhhWtPsG6Zf5grDIq/0fe6hc9/ji6wPZsYHH0HXhIIEIUiJK+ci690O6YbsR6NIo3a6Cn779DcqDw9/l+rzRt6a+8mhWNavwOgmwzY4hijtzW6lvOKbQcMRB5wADmCdL/p5M9OQzPhvR++ncuZ4Ep+pjOgBEGzDynPGFDsAKE9ggYjjmU2gtppp6GJUieknUQetqYcqDKbgdnnbciHp0L7f5gER/wTwJxcM/sTxqRObYwcVmI3HluXBtDlN6fX+ROboTac+d2DccE7RM/I+IjsNgB3yQbeZNfAbq9CYdYkt0xkTzCirNmlQW5YTLm6q9vJa1uoUJrW6JapqKPUFZOMcIUu9GFg0CJVWjXp36tq350bRZYPmOP38ab67YVWN8PjawLsAv+CMRXux7zZD4md3Cq0fEth3FGdHXJQq16B3jFLPdDlgpQyP/cdGBXYiLEYkHFkwA8ph9+/KlI7/K477jK9Ew/Jj0O6aWgzpMUmSRR7YWVax6oD7WbpKa0tUIljT3jkjV4rFmxqK1NqbP2dAQfirxpUzWuxHMhI7EW2pHG5TKbfFte4v3GH3e63KJutK1vohIDTjhBEtqZ X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM4PR11MB6020.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(376014)(1800799024)(18002099003)(56012099003)(22082099003)(11063799003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?a3dtY1lGVUtwQWVuN3hUNGR2S0M2bmdzVTM3QXdiRGpCblhOazMrYjBxdEFy?= =?utf-8?B?dGloQzdZV0Fnc1prMWllSFRmc2kwY29uOXBZcmYxanU3TzZxRC81OThHSVdL?= =?utf-8?B?V3lGMHd3VkRuN3BYYStVUTB5c01kUHFCSTE2SzhYeWsveC9VQWVFQnYreFdh?= =?utf-8?B?amhSbWdhMklBKzRxSStESzF1OXhSQVBPcnNvcFFXUS80cm9QZ0JuTXNxOFRS?= =?utf-8?B?Ym9vY3lIeEFibU5waVlOb2FXSkptOU9oNGFZd3N5WHByWmd6eW1hZC9BaHdx?= =?utf-8?B?cHJ3TCtoODlhbnMyL3VtRVlNZFF2UjlBVGs3QUR6Q0hxWnU4TW5FV0xZTHht?= =?utf-8?B?dXFsVUNZUEhFejF4VFRhb25kaHZ0V2ZNZnArZE85amNMdmt0c2lXWUoxcW1T?= =?utf-8?B?WWx1RThmeDlqZGR0em5ZSlpiTzRVb2Z6eHQ4SlZhcVV4UEFmVkZQZ2VPVjZW?= =?utf-8?B?dDhtRWNQdG0zTmJvcjFISEtacGJYa2ZzeTFrY05kSDg3T1dJU0l0dXpUUlhM?= =?utf-8?B?YXFzNDdMblJOZ1FLeHo2Y0daN3ZJd1FjbHNaNXI5SEFETHlzZkErOEI5bnA3?= =?utf-8?B?ek4xNmt4WGN2K2lGb2YzOUFTK1FobWN6ei9SOTdTb3RvZEZudnpiaUsweGVQ?= =?utf-8?B?MFdxSzRXd1c4Nis0QTBTZmpZUm5wb0NsenpXU3g4cDB4VFdrTVNkVWJHRG90?= =?utf-8?B?NG1jeWJHdW9GNXY2NmZMWGl3SllmRExxU1VnMzQzbEJUTSsrNnNraUxlbGdr?= =?utf-8?B?Ym82QkJwNUZNVHEvSU1GR01KYVY0WGs1RXJkcVhkTFRyeERGTjQ2b2E0TDdP?= =?utf-8?B?VGxDZ3JLcng0K2ZsRmtnN0EwYjZlaXZ3SVhxa2IwK296VmFRRDlDVlJyeCt5?= =?utf-8?B?MnB3akxIS2Nkc0NHV004dWtlMk1ReTBaempBM0EraWNmb0pkVCttaS9zT2RV?= =?utf-8?B?ZGNtZ3Qyb245YkpCaWpoZFRzOVVZOEsrb3orV2ZQenhDMDd6QklzVE9UMEl6?= =?utf-8?B?bDBJaXFTbzQ3SmJuTEorSC82VUg4TTFFUm9wVTVkUnRVbkNRSnRDeDhvcSt3?= =?utf-8?B?MThqSVNlSjhnSE0vVFc5aXNWOUwxQzYydXo4QnZlSm1YWVRIaDhqWHlyYnNP?= =?utf-8?B?c2kwSzBZbkFkTCtSUXN3ZkNMaTh2bm1ORlorRFFnOURMajE4MXB6ekpFa1VH?= =?utf-8?B?U1dSYjZqVWZmUWNOOFBSS0RUWVZVMW5QUzVFZ1QreDVPUFkzSjdTakpyTkVD?= =?utf-8?B?SzZGT0RTTmQvbHZjMHp4MEhBdE8wNC9uUS9xTkJRS0lOUGl4RHh4QmYwc2lK?= =?utf-8?B?ZzF1alRLbHdaaDdXNHE4ZFlkMVdnYWJSY2dFMHFsTnBTNStVVDUyYnN6MzNM?= =?utf-8?B?dFZhMDIwNElaaXRlUDRTbjhCUkdYRnpuYkFXV0dQT1pqZTBRVk1LTTRjZWRH?= =?utf-8?B?YWtzNWNyMU9vMTZ0aDMrbys1VktuZGVqeURWUHlnelZjUVI2WDdTREo1NXBx?= =?utf-8?B?T1M1ZUpNN1VuVWlhZFQzVTF4dDY5b3QyWk1CRVFmOXpMWkNDTEJKR3g4YkxW?= =?utf-8?B?Q1NiOFRyMjhKUjhRTS9uN1dIV1dlSjJ4Z0ZoVTdYUDZJdUNSYzJkSFpNNHBK?= =?utf-8?B?ZkRZSGRxdVQ2WU5rSGlmN3NJZEJGNzRDVC9ROFM4eHFjVlpLUURTSVZXdnRl?= =?utf-8?B?YmQydlYrbjJqK2dROVF3bGdLb0I0eE16MnFZVnAxZWxMczdKLzRVK0lhNnB0?= =?utf-8?B?Q0V6WTZIU2x6QmdXTGNwTllDZEcyMUFoWWZPRHhsenlrWTM5NEJNWmUxSkJG?= =?utf-8?B?V1NSSGxaSmg0a0RENzVqUm5vZGRoMGhXTkp1MndIUm5kVGV3NG1aWnM4N3ZK?= =?utf-8?B?aUhyOFR1d29keW1qcDRreENxb09SbkxLUU5icVJNNzdGUUhyc3hkZ0VLWlpM?= =?utf-8?B?NklzOFUyclhnYjVyWTJNOFBUbUpwd0FQbitINWNPOFFYR0lORlluMXBEKzlX?= =?utf-8?B?OVNDSkZnNzQyNjNkdzVGS1I3Qy9ueHc0OGZITDBiV3dnZGpFdW4zbkE3c2w2?= =?utf-8?B?cG1YRTNLcHYwZE9wUjBCWjlyYXY2QkszNTNKTXZUeGdYK1R2UjJPcmcxWEQv?= =?utf-8?B?OXk0OUd6Q0xVd1E1bUMyM3FML2ZtQ2VuWnlyRWRoakhGSFNhNFRZYjRzSC9p?= =?utf-8?B?Z2FIcHpXcGQ4cktKVWZhVnlPVFFiK0l0NllOeS9kN3pUWkdydHZCV2lzZGk1?= =?utf-8?B?R3ozeW8rcThjTFFreWpBV2Vub2FjQW9JMTRnaThjS2pqZHRqcTVHaVBQWnRE?= =?utf-8?B?bVdjQkpiYVpVNVZGY1o5OXA3WHlxbkRCVFdtUFRZaHIzWjkvN0lEUT09?= X-Exchange-RoutingPolicyChecked: MB9852ztQed2cC7dzxS3XN5m9G4YswMQ3cU1AJIV4rL+HRXZr77OQ2rvx2PLV0o8RC0cSTFhbQw9Jjvx9l0ye4ZdBWmCpHLa4Z1cbNiLQW+Dng8usja7T31ncYUpYCDWGwotKsOidKwUUaAI+4v+zbgeh1Y5xLKUF2sDmRpjQaZl/C5F9Ku+VEeVJ6ovs0gSpUC3WYfDD4y7QgEP91eXQWiWLUfx7wFdJfcd77Ybq8OQ8uFTK0Scr9KKJjBBcxD+Fb6hM8/Y6AUkemge4psNjCFRG2drwvPRHIexH+yKETJBKfLMHZezGU7B33bwG6SVAhUg5ej4lmgz55RsFMyvtw== X-MS-Exchange-CrossTenant-Network-Message-Id: b0e15ce8-e813-49a4-1e7a-08deb009025c X-MS-Exchange-CrossTenant-AuthSource: DM4PR11MB6020.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 May 2026 09:29:47.5040 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: YuDy4DDsLQpiwES0+ngMnWzjj/i6orbzcWW0MuC2wMGFiMwkiWig8Imj+vNGWD5umavwM6fmCj3zuamq1SZdrg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ1PR11MB6300 X-OriginatorOrg: intel.com Hi Prateek, On 5/11/2026 3:48 PM, K Prateek Nayak wrote: >> - if (!amd_fill_cpuid4_info(llc_index, &id4)) >> + if (!amd_fill_cpuid4_info(llc_index, &id4)) { >> c->topo.llc_id = get_cache_id(c->topo.apicid, &id4); >> + if (c == &boot_cpu_data) >> + arch_sbm_shift = get_count_order(1 + id4.eax.split.num_threads_sharing); >> + } > > So I'm slightly skeptical on AMD's heterogenous processors based systems > getting this right but I have to get my hands on one to confirm. > > Either ways, it seems like an AMD specific problem that I'll chase down > if it exists but this should be fine from testing perspective on your > system. > Right, this reminds me that Intel's heterogeneous (hybrid) processors might also need to be accounted for if the platform has multiple LLCs. >> struct sbm *sbm_alloc(void) >> { >> - unsigned int nr = arch_sbm_leafs; >> - unsigned int nbits = 1U << arch_sbm_shift; >> - unsigned int nlongs = BITS_TO_LONGS(nbits); >> - struct sbm_root *root = kzalloc_flex(*root, leafs, nr); >> + unsigned int nr; >> + unsigned int nbits; >> + unsigned int nlongs; >> + struct sbm_root *root; >> struct sbm_leaf *leaf; >> + >> + if (!arch_sbm_shift) { >> + unsigned int max_idx = num_possible_cpus(); >> + >> + /* >> + * unsigned long is the base unit for bitmap in sbm_leaf. >> + * Use that for default bitmap size for compact bitmap >> + * without unused bits. >> + */ >> + arch_sbm_shift = BYTES_TO_BITS(sizeof(unsigned long)); >> + arch_sbm_leafs = 1 + (max_idx >> arch_sbm_shift); >> + arch_sbm_mask = (1 << arch_sbm_shift) - 1; >> + arch_sbm_bits = arch_sbm_shift; > > Side note: > > So while chasing sbitmap, I realized there are some users of sbitmap out Thanks for pointing it out. I took a look at sbitmap; it seems to provide cache-friendly bit allocation strategy for different CPUs. This seems to be a bit different usage model from sbm, which aims to provide a 1:1 mapping between a CPU and its corresponding bit in a mask in a cache-friendly manner. That said, the allocation logic could be reused between sbitmap and sbm IMO. > there that are essentially using its minimal functionality that smb > provides and can be converted over to save an extra cacheline worth of > overhead. > > Does it make sense to keep the arch_sbm_* stuff specific to the > scheduler and allow wider use of sbm for any sparse bitmap usage? > Yes, I think this is feasible. We can introduce struct sbm *sbm_alloc(unsigned int max_idx, unsigned int leaf_bits) to allow reuse by other non-scheduler components. >> + } >> + >> + nr = arch_sbm_leafs; >> + nbits = 1U << arch_sbm_shift; >> + nlongs = BITS_TO_LONGS(nbits); >> + root = kzalloc_flex(*root, leafs, nr); >> if (!root) >> return NULL; >> > > My QEMU has suddenly refused to boot after the conversion to cache > properties leaf changes so I'll try to see why that is the case. > Thanks, I haven't tested on VM yet, let me take a look. thanks, Chenyu