From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from DM1PR04CU001.outbound.protection.outlook.com (mail-centralusazon11010030.outbound.protection.outlook.com [52.101.61.30]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B3F243A6EE1 for ; Mon, 11 May 2026 07:48:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.61.30 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778485746; cv=fail; b=H7QN/q/LgBUUL3xXy8kSdBpFnV/8uxvKewhnCzVOLXw4zCarvXsdFeX9SMjQxEqcbjMRGiczLkalS8Fm/6vBn3o+8PS4eLJ0opTlirHZ4zlAknlKWlv8w+l+tpBHPtjV87eXBnFn1mi9U4y40g/+9K/KiXjbgBKBp10ehaOUN1Q= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778485746; c=relaxed/simple; bh=/pcjD7GH6yDYuJai/WJsGTQGJUSxA+yzmbyyYZfjPig=; h=Message-ID:Date:MIME-Version:Subject:To:CC:References:From: In-Reply-To:Content-Type; b=gB4OVWcDi9ARzUh8EGhws4y7XW+l/C+mgvlld8+ClajhXt/ZDi1jQRrHnN7s4bPNsJTzG4KSzFXZi9f4yKClnl1oFgLec2pM9iAfHcWwQBZKyCQppmZveZp0vsLHEcCyBJ2d4aZ4qDcu3QY89Yzd0gM//Z3wV9bmhgS2SI+xIVA= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=3dKh/Ked; arc=fail smtp.client-ip=52.101.61.30 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="3dKh/Ked" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=qS8nzJOJxZNAjuYS40zs1hDjpjybORVUAGPDVaUj48oK6KvK2neUH6qs2Pj8E330tNLvYR/H8VzlRzH/7g4xw7abR8XGItqpefMgbki1KitA3Lh3y+E12XnzscOK8zY2PnM5YEgkKGbAfjNxQ4fYjXoOEsKvKfC/HNFybjMa9bJfsLPurixXoBnVIeBH/Xrgm6MapbKot+36ljlo+NwdMgkl+ya+Z0DdiJXnsFa/YtWHeHdJF4I9tglmZ+gvYbU4bJ9GpdxGmMz7w2+7LvOqaw8JIn4o1JGkFgjg5BiEKa6hGFBCb7A+2sU6AIM+1/sHrzo/1ZYzYjs4hYYuF/h72A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=k/uY9ip4DF4cQuggNfGbSmQyW1Wh5ss/O1L/3ZFBdMc=; b=qPgHS+zP5aleH74ExCSWQn/G6ST07eoJMaeL42tZjpSujOdbj3dm+O4hKhqYJayU8/aa0ApRu2GQbVo9RwWWx6ykSKsBG9+/OYMuP8sCECUZrXla8T7+gB24P4547uZ9Mxe99ZQGWAb6F1UGph2cA6D0s0YJwmMZ59HyqqM1sBM13vdOvfiEiTdBz02Ol8GbB8or2yD2kiRNACfTFniiw4WEusPGMuUKXVlMIXNhyPoH4J0pva8/bQSVE0BqpY+lS8Kvrp8ROZ/Qi3d3c0xjldjiU92jUACRRsV7ivriP6c9iypfb03/lwQcEaqIDwzy3025VVqGCqkhWrwYN8OpNQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=intel.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=k/uY9ip4DF4cQuggNfGbSmQyW1Wh5ss/O1L/3ZFBdMc=; b=3dKh/KedBFlf6iJVSi9/OBw5aI35SrSzYWozSiOsCldjq92Zq6b2E37kBSNXGDRM1hPGxNSirM7XcdM7//6FTTYiBZ+ToNi2jXk8xECtO8mTX7M6yyt+d+x0NkDLQjaXSDoqnVbNHG0jOLNcJLkqmq1kXwJeLnegeKrHVHZshCs= Received: from DSSP220CA0013.NAMP220.PROD.OUTLOOK.COM (2603:10b6:8:3d3::8) by SJ5PPF6375781D1.namprd12.prod.outlook.com (2603:10b6:a0f:fc02::995) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9891.22; Mon, 11 May 2026 07:48:45 +0000 Received: from DS3PEPF0000C37C.namprd04.prod.outlook.com (2603:10b6:8:3d3:cafe::bd) by DSSP220CA0013.outlook.office365.com (2603:10b6:8:3d3::8) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9891.23 via Frontend Transport; Mon, 11 May 2026 07:48:45 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=satlexmb08.amd.com; pr=C Received: from satlexmb08.amd.com (165.204.84.17) by DS3PEPF0000C37C.mail.protection.outlook.com (10.167.23.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.25.13 via Frontend Transport; Mon, 11 May 2026 07:48:45 +0000 Received: from SATLEXMB03.amd.com (10.181.40.144) by satlexmb08.amd.com (10.181.42.217) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.2.2562.41; Mon, 11 May 2026 02:48:44 -0500 Received: from satlexmb07.amd.com (10.181.42.216) by SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Mon, 11 May 2026 02:48:44 -0500 Received: from [10.136.36.106] (10.180.168.240) by satlexmb07.amd.com (10.181.42.216) with Microsoft SMTP Server id 15.2.2562.41 via Frontend Transport; Mon, 11 May 2026 02:48:42 -0500 Message-ID: Date: Mon, 11 May 2026 13:18:41 +0530 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH 3/3] x86/sbm: Derive leaf granularity from LLC cacheinfo instead of topology domain To: Chen Yu , , CC: , , , References: <729726b9-c669-41e2-887d-bdf9da703034@amd.com> <20260510155920.2587431-1-yu.c.chen@intel.com> <20260510155920.2587431-4-yu.c.chen@intel.com> Content-Language: en-US From: K Prateek Nayak In-Reply-To: <20260510155920.2587431-4-yu.c.chen@intel.com> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit Received-SPF: None (SATLEXMB03.amd.com: kprateek.nayak@amd.com does not designate permitted sender hosts) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS3PEPF0000C37C:EE_|SJ5PPF6375781D1:EE_ X-MS-Office365-Filtering-Correlation-Id: fef3a816-79ed-4bc6-f559-08deaf31ba9e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|36860700016|82310400026|18002099003|56012099003|22082099003; X-Microsoft-Antispam-Message-Info: iGkEmNjY+aq73I6t+wNcdaN4NcRMKBMUq7cvNX6m4E4uLaOsnrN6A0Wz1JLbL+kATIe/OVhmx8hwkZpdBwSOmH8xo7jAg2lGV0MrPGf+KzkFwzInsHfuspA2DAJp1ps2a+TTc7+YPQWIoOiCnBECIiX1fMp7buHCuNcT2JAqukJ8heUg7qKCdazLwmeCbZ8pIpDvku/bFxqQM2wv9Ukkj3kPVxhwz0KaED23Xfcowcz0n//dCkNiaAtUkXsQsEN1ua8Q6Ar57s/DdEbNNg98ouS47XVVIdSYHrf2SLgEJIKLzPTyZAcV8f6juxXdAo+A325ucmkkxkugSzyr21Akff+OTfiBgzHDPrxVeCLehVPiWCVHNkb5s9P1M4ShheHJyiqvhELmVJZxgS7pCl22/hSNUj99YNPLgJaGvd3vPkx7ojImabTCuW1SgoGdAWtAKIbFhx/erAN0DfJjNz/Irq6O9bzpi1IMdOoHjvgqROM1yyKWf1VtM5zTbBUzmWCOCu74l457vroAhI+CCwM9cZS6kkwFxuHbFeqRaxvQmV2oO1f+qu6jcPYDvOYeizIYFOua3PEg7Gw7WW4WrxHr0MczRIq0aHdz4M5G6jZyOha/mslUFWRsJVyVA2xZt1YN8Tx34xH9vpraVMiAdUS8faMIcl/rSP/fPrmmPSNs81Dv4soQuKW1jnnueK6xZKcVdXwrQaofmxyY8ST+8Eer5KtlPwAiCQlvvRXKeQm5hSI= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:satlexmb08.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(376014)(1800799024)(36860700016)(82310400026)(18002099003)(56012099003)(22082099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: w2WjzaEEt0msOy1Z+z2n8Qun00y5B9dxJH4YonSnXpwQSiKAwjfwpQtTZdTufzu9OHjENgwXXBAjOYVyD+NwrESwiA2R79yluSm0sOBlwYfEBUh1pl104vgC6Ap+IeeXGL525dqpKCCmrNTpCZgRiRhI44HbCqmtP05qhCU0e3DL5B+LyTfXSQmzY8V1xNVmeK1q28i0Z/IY+4pPzqpocdP0dZELBnvlGeOsUjGq5+C1LTev2cWMs8xrYDf1bKfe/jEWAj8SbEav1MYJoQHotuq3v8EzbGb8x29TmByC6AgWUH6b8geYeFPr4nhYbHHsN+6NhX8EEesLj279x0lTz+flE65Fh+at1VYkpMRoal22EixzcDRaauWzKkzQUy7zVLQhJRyEdPPQ85TBeiyAIeNZFs2kl7qiLhewFRot9HEt7CciWBB/ptg80EvO1opC X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 May 2026 07:48:45.1839 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: fef3a816-79ed-4bc6-f559-08deaf31ba9e X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[satlexmb08.amd.com] X-MS-Exchange-CrossTenant-AuthSource: DS3PEPF0000C37C.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ5PPF6375781D1 Hello Chenyu, Tim, Thank you for working on it! I got distracted seeing if there was a way to make the sbitmap work here with minimal overhead but that data structure is too over engineered for block layer specific use cases (separate cacheline to track clearing with delayed clearing, alloc_hints, and other fancy bells and whistles) and it took me a while to get something going with that. Perhaps keeping the minimal sbm functionality out separate is a good way forward. On 5/10/2026 9:29 PM, Chen Yu wrote: > @@ -317,12 +319,16 @@ void cacheinfo_amd_init_llc_id(struct cpuinfo_x86 *c, u16 die_id) > if (c->x86 < 0x17) { > /* Pre-Zen: LLC is at the node level */ > c->topo.llc_id = die_id; > + if (c == &boot_cpu_data) > + arch_sbm_shift = topology_get_domain_shift(TOPO_DIE_DOMAIN); > } else if (c->x86 == 0x17 && c->x86_model <= 0x1F) { > /* > * Family 17h up to 1F models: LLC is at the core > * complex level. Core complex ID is ApicId[3]. > */ > c->topo.llc_id = c->topo.apicid >> 3; > + if (c == &boot_cpu_data) > + arch_sbm_shift = 3; > } else { > /* > * Newer families: LLC ID is calculated from the number > @@ -331,8 +337,11 @@ void cacheinfo_amd_init_llc_id(struct cpuinfo_x86 *c, u16 die_id) > u32 llc_index = find_num_cache_leaves(c) - 1; > struct _cpuid4_info id4 = {}; > > - if (!amd_fill_cpuid4_info(llc_index, &id4)) > + if (!amd_fill_cpuid4_info(llc_index, &id4)) { > c->topo.llc_id = get_cache_id(c->topo.apicid, &id4); > + if (c == &boot_cpu_data) > + arch_sbm_shift = get_count_order(1 + id4.eax.split.num_threads_sharing); > + } So I'm slightly skeptical on AMD's heterogenous processors based systems getting this right but I have to get my hands on one to confirm. Either ways, it seems like an AMD specific problem that I'll chase down if it exists but this should be fine from testing perspective on your system. > struct sbm *sbm_alloc(void) > { > - unsigned int nr = arch_sbm_leafs; > - unsigned int nbits = 1U << arch_sbm_shift; > - unsigned int nlongs = BITS_TO_LONGS(nbits); > - struct sbm_root *root = kzalloc_flex(*root, leafs, nr); > + unsigned int nr; > + unsigned int nbits; > + unsigned int nlongs; > + struct sbm_root *root; > struct sbm_leaf *leaf; > + > + if (!arch_sbm_shift) { > + unsigned int max_idx = num_possible_cpus(); > + > + /* > + * unsigned long is the base unit for bitmap in sbm_leaf. > + * Use that for default bitmap size for compact bitmap > + * without unused bits. > + */ > + arch_sbm_shift = BYTES_TO_BITS(sizeof(unsigned long)); > + arch_sbm_leafs = 1 + (max_idx >> arch_sbm_shift); > + arch_sbm_mask = (1 << arch_sbm_shift) - 1; > + arch_sbm_bits = arch_sbm_shift; Side note: So while chasing sbitmap, I realized there are some users of sbitmap out there that are essentially using its minimal functionality that smb provides and can be converted over to save an extra cacheline worth of overhead. Does it make sense to keep the arch_sbm_* stuff specific to the scheduler and allow wider use of sbm for any sparse bitmap usage? > + } > + > + nr = arch_sbm_leafs; > + nbits = 1U << arch_sbm_shift; > + nlongs = BITS_TO_LONGS(nbits); > + root = kzalloc_flex(*root, leafs, nr); > if (!root) > return NULL; > My QEMU has suddenly refused to boot after the conversion to cache properties leaf changes so I'll try to see why that is the case. -- Thanks and Regards, Prateek