From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from CH5PR02CU005.outbound.protection.outlook.com (mail-northcentralusazon11012036.outbound.protection.outlook.com [40.107.200.36]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 841502550AF; Sat, 21 Mar 2026 08:59:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.200.36 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774083561; cv=fail; b=KYFYRM78WvHcjQpJ1LscS/t1wkX2Ain+244mxgLBDQDW+G/ozABTGfXt0hZA8hb396HdIzvZ1iXzmQbp8uWSWDoCArqaCHJe5YurtQjlP52kYJbMNy7roeECu7T6fUwkUdf2Mli5Cq4MiAAScL8L2HqxTWb/Nv1A28wlTSPiY/4= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774083561; c=relaxed/simple; bh=fa9RkFVFYdlh2UDQUlpYl/RS7o7lnnJZRSLwqRZTMEM=; h=Message-ID:Date:MIME-Version:Subject:To:CC:References:From: In-Reply-To:Content-Type; b=cH0i3ET5QyU8ZzhIHGI0oW70uu9ndCx4GTUZA3I9ENd7wUCKPDs/M0x7j6zE+QJIccqeTUXqmr1qXXGtXSP8atWMuV/0p7fk9CpB6qa6ZOAoIkfpDDTTlf+3HWwxGWb/a0fYYrwt5QNg2lQf0tTPnh9TvLxhztHZYkGvxqWoGeE= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=bqNHND25; arc=fail smtp.client-ip=40.107.200.36 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="bqNHND25" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=VaKc7hPfBMO8eL4wtTazFO099e+crGfU5i3hsLG0WCPX7Nh5PGKZeCUx6pY0gAl5hvEryIOdE+jdF7pcwRDwECe2u/ftXlDeQ7muTd4Gus1MLtDd0CSItMagCv+oaBOStuIrTj1uqGoEREiK+Tb0Brpuei4MJecvel+qL5MAcjqJUt3c4kzr2UfpUnMppe68DHqBUAt2ngcdQQcwfqWneUcMhus88ylldwjJw5nhVIOGxmh6Oq698hSaNTwEEop/5QGJekZ3ivTCv19vd0URKb2pWA119B2+seaJBW5OMdEJ0GXvTXe+E6cHjGK3m14sL5Tu+9mjYtXnRyqXpOitAA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=tB6AP4nQtdlF0zjDfv5zaDbBnedc3SxgNMo08TDL0rg=; b=c3X3GwHNqqHVm1MDCEUOI0EOGbq/6ANhdyL6JVJ2W8lJPMz6sgZynrI90crG31iKQL2lsYrvznyQiQFxYlSxY3NB8CcEqpX4FzB9hDLALI8+1tayx/WrLvqNVxfDdLpesjt+Ct107uET/rNYFn9qIn2QC64QXWyQ0iKeyJmQ8fN8/HnU1/Szh+nvmYvm3Tt1/Gl9QzgfehYKYg5VLpn18SmRAszlq7+G5dHIb6v8E/fpgwN86ZSbvEmvamNN3PorsZdTld7LgNFIIsKcZmwiA6vXclvEmE05O5xVbybuaxE1s8lp33pO0PK0PmyMbzrkE0KfWaNANEeI8BQ6HGvV+g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=intel.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=tB6AP4nQtdlF0zjDfv5zaDbBnedc3SxgNMo08TDL0rg=; b=bqNHND25LRxShIULgUKcW1MwVG9z/bQwmZnlhbmcpfYqn6funuAeBx8trtTug6c518xj+8pkJFkcwhE7SrO9CbsrnM9e0OrNMEdMkwQiMDn+4m3GcBEiitJ/8aTeJzJ29RZdjeZZ6TXJMqkcK9Iv/YIAIrmux+yDxwAANqBuXuc= Received: from BY3PR05CA0027.namprd05.prod.outlook.com (2603:10b6:a03:254::32) by SA1PR12MB9489.namprd12.prod.outlook.com (2603:10b6:806:45c::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9745.9; Sat, 21 Mar 2026 08:59:15 +0000 Received: from SJ5PEPF000001C9.namprd05.prod.outlook.com (2603:10b6:a03:254:cafe::de) by BY3PR05CA0027.outlook.office365.com (2603:10b6:a03:254::32) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9723.25 via Frontend Transport; Sat, 21 Mar 2026 08:59:13 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=satlexmb07.amd.com; pr=C Received: from satlexmb07.amd.com (165.204.84.17) by SJ5PEPF000001C9.mail.protection.outlook.com (10.167.242.37) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9723.19 via Frontend Transport; Sat, 21 Mar 2026 08:59:15 +0000 Received: from Satlexmb09.amd.com (10.181.42.218) by satlexmb07.amd.com (10.181.42.216) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.17; Sat, 21 Mar 2026 03:59:14 -0500 Received: from satlexmb07.amd.com (10.181.42.216) by satlexmb09.amd.com (10.181.42.218) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.17; Sat, 21 Mar 2026 01:59:14 -0700 Received: from [172.31.184.125] (10.180.168.240) by satlexmb07.amd.com (10.181.42.216) with Microsoft SMTP Server id 15.2.2562.17 via Frontend Transport; Sat, 21 Mar 2026 03:59:12 -0500 Message-ID: <7fad91ea-e6cd-43c8-abe3-16d7843247ed@amd.com> Date: Sat, 21 Mar 2026 14:29:05 +0530 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [tip: sched/core] sched/topology: Compute sd_weight considering cpuset partitions To: "Chen, Yu C" CC: , , Shrikanth Hegde , Valentin Schneider , Dietmar Eggemann , , Nathan Chancellor , Peter Zijlstra References: <20260312044434.1974-2-kprateek.nayak@amd.com> <177382132440.1647592.1849180094328011054.tip-bot2@tip-bot2> <20260320235824.GA1176840@ax162> <60406550-d90e-4efa-a4d9-f901421887f8@intel.com> <470ce693-ee2e-414e-930b-d6581d649110@intel.com> Content-Language: en-US From: K Prateek Nayak In-Reply-To: <470ce693-ee2e-414e-930b-d6581d649110@intel.com> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 8bit X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ5PEPF000001C9:EE_|SA1PR12MB9489:EE_ X-MS-Office365-Filtering-Correlation-Id: ec1928c7-5d29-4abe-f47e-08de872820f2 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|36860700016|376014|1800799024|56012099003|22082099003|18002099003; X-Microsoft-Antispam-Message-Info: ES9AF8M+1UC2cYIn4xSyEU+1VOUiPFRxpENGUpPYkBKMBPoJCqpt4FABHHJSaXgeEPFOKm7WuMBETmQdj793sZ0d24A+PAYqnuTtu4SE/gWRxt7iIW47n5pXxF8OARpsQdMVpH+S2lcyGbEZ38ETrcbHCS53NfAvOYxO+9S2miJnnklWSIqfWm7yr8P2IAVTI1R1xRFJPU7LUAXbDo2lb194OrBeXwefbX8X0vZVLErL0zdvGPMlIoFJkzCjUeqJN8ieHR6GKmDrQXx1Od41mctOTa94QgdTPycKCXQwPwiKMlcv2D6sMgtz9ocaelCn2DhW+rYPCTCMvOGPz77x+5zlOH9btmKODSWgXz3XCDoxSmrlzL80r98XByZqNlFAG8lAM6gOAQUB3uhhfL71Q5VevkX9lb4qgOFaMhj53ns1xZLSYoYMDOpjGwV02/YPA7Nx0oL6SSoYEkXDUOU+0LcYiCO8APYwChx8ejqjzf5QbOQffnl3Jle15uo3LOq/WUkBCBFnNn+SCtXgzUClui3MzLFALsDaWg0KzVUmBSdkEmmO725kO+/VdrAi6d3Nt3uPPbS5eE8CX4gsSO7zNTCmpkeuy+Fk+hB0hIq4EqRYnAfbW8TCmJci5tVzSKgVAibGQAW7XIGHmRoZmXCCEnbNEmyiDS1ixyY/eVmCPqDvLT+XUeluZ/mT3yNtIMkIvNPYWjaYnmMgh1vhLpXMicF1FM+UmV9HZz/pTIkpzvHH+4o1WsjeGsvGCDj+/+o//k3j9MM8sKrzrHh0rtb3jw== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:satlexmb07.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(82310400026)(36860700016)(376014)(1800799024)(56012099003)(22082099003)(18002099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: CaM5wRbwe5+XrIP4DvFO2LKtrxvbRd3rlmUcFMFSeYfHY53NjffsJvw71WG6OHmozFatoT65a4LB3EQ/zhgxAePtZloLDjtsaxjFMzWdFtmh0wegPx9X2SbxClvTpuH/I4lSkd4TmCCY43qjNiJiiQhYwuH3WONCwdqkxCiE5uxkJ9Eg/zxTkIWw+Kqeo/TNfHjE9v2RfguNTybU1d6dJ7JeGHF2Mcy5eFr2BtbmRsEbvb96JGEoNT/1WrR51HPR8Gwus2OQTI2l1YOG1JbMbP/iWXfp7tiCsN1RE50E7TtdHaXY6cnx0JRkNfn+UMD417lLNp9UEoj/3Hjk+RRogvjqxaNHQsQ/1/5g0NKKsNg6Snke9GjUDQfGTd2atd/CrO5ekwZ7amontpgU/K4X0IIKGxPnSE2mBg91PBQyulGeTmtfYHgg8zc+k0icBrDf X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Mar 2026 08:59:15.3675 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: ec1928c7-5d29-4abe-f47e-08de872820f2 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[satlexmb07.amd.com] X-MS-Exchange-CrossTenant-AuthSource: SJ5PEPF000001C9.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB9489 Hello Chenyu, On 3/21/2026 1:17 PM, Chen, Yu C wrote: > On 3/21/2026 3:33 PM, Chen, Yu C wrote: >> On 3/21/2026 11:36 AM, K Prateek Nayak wrote: >>>    sd->span_weight = cpumask_weight(sched_domain_span(sd)); >>> >>> which should have crashed too if we had a NULL pointer in the >>> cpumask range. So I'm at a loss. Maybe the pc points to a >>> different location in your build? >>> >> >> A wild guess, the major change is that we access sd->span, before >> initializing  the sd structure with *sd = { ... }. The sd is allocated >> via alloc_percpu() uninitialized, the span at the end of the sd structure >> remain uninitialized. It is unclear how cpumask_weight(sd->span) might be >> affected by this uninitialized state. Before this patch, after *sd = { ... } >> is executed, the contents of  sd->span are explicitly set to 0, which might >> be safer? >> > > I replied too fast, please ignore above comments, the sd->span should have been > set via cpumask_and(sd_span, cpu_map, tl->mask(tl, cpu)) So I managed to reproduce the crash and it is actually crashing at: last->next = first; in build_sched_groups(). If I print the span befora nd after we do the *sd = { ... }, I see: [ 0.056301] span before: 0 [ 0.056559] span after: [ 0.056686] span double check: double check does a cpumask_pr_args(sched_domain_span(sd)). This solves the crash on top of this patch: diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c index 79bab80af8f2..b347ae5d2786 100644 --- a/kernel/sched/topology.c +++ b/kernel/sched/topology.c @@ -1693,6 +1693,8 @@ sd_init(struct sched_domain_topology_level *tl, .name = tl->name, }; + cpumask_and(sd_span, cpu_map, tl->mask(tl, cpu)); + WARN_ONCE((sd->flags & (SD_SHARE_CPUCAPACITY | SD_ASYM_CPUCAPACITY)) == (SD_SHARE_CPUCAPACITY | SD_ASYM_CPUCAPACITY), "CPU capacity asymmetry not supported on SMT\n"); --- And I see: [ 0.056479] span before: 0 [ 0.056749] span after: 0 [ 0.056881] span double check: 0 But since span[] is a variable array at the end of sched_domain struct, doing a *sd = { ... } shouldn't modify it since the size isn't known at compile time and the compiler will only overwrite the fixed fields. Is there a compiler angle I'm missing here? The cpumask_and() that comes first looks like: @ kernel/sched/topology.c:1649: cpumask_and(sd_span, cpu_map, tl->mask(tl, cpu)); ldr r3, [r9] @ MEM[(const struct cpumask * (*) (struct sched_domain_topology_level *, int) *)tl_317], MEM[(const struct cpumask * (*) (struct sched_domain_topology_level *, int) *)tl_317] @ kernel/sched/topology.c:1646: u64 now = sched_clock(); strd r0, [sp, #28] @,, @ kernel/sched/topology.c:1649: cpumask_and(sd_span, cpu_map, tl->mask(tl, cpu)); mov r1, r6 @, i mov r0, r9 @, ivtmp.1798 @ ./include/linux/bitmap.h:329: return (*dst = *src1 & *src2 & BITMAP_LAST_WORD_MASK(nbits)) != 0; mov r4, fp @ tmp740, sd @ kernel/sched/topology.c:1649: cpumask_and(sd_span, cpu_map, tl->mask(tl, cpu)); blx r3 @ MEM[(const struct cpumask * (*) (struct sched_domain_topology_level *, int) *)tl_317] @ ./include/linux/bitmap.h:329: return (*dst = *src1 & *src2 & BITMAP_LAST_WORD_MASK(nbits)) != 0; ldr r3, [r0] @ MEM[(const long unsigned int *)_356], MEM[(const long unsigned int *)_356] ldr r0, [r7] @ MEM[(const long unsigned int *)cpu_map_104(D)], MEM[(const long unsigned int *)cpu_map_104(D)] and r0, r0, r3 @ tmp736, MEM[(const long unsigned int *)cpu_map_104(D)], MEM[(const long unsigned int *)_356] @ ./include/linux/bitmap.h:329: return (*dst = *src1 & *src2 & BITMAP_LAST_WORD_MASK(nbits)) != 0; uxth r0, r0 @ _360, tmp736 @ ./include/linux/bitmap.h:329: return (*dst = *src1 & *src2 & BITMAP_LAST_WORD_MASK(nbits)) != 0; str r0, [r4, #292]! @ _360, MEM[(long unsigned int *)sd_352 + 292B] --- *sd assignment looks as follows in my disassembly: .L1867: @ kernel/sched/topology.c:1660: *sd = (struct sched_domain){ ldr ip, [sp, #48] @ tmp1203, %sfp mov r2, #296 @, mov r0, fp @, sd mov r1, #0 @, ldr r3, [ip] @ jiffies.324_453, jiffies str r3, [sp, #36] @ jiffies.324_453, %sfp ldr ip, [ip] @ jiffies.326_454, jiffies @ kernel/sched/topology.c:1693: .name = tl->name, ldr r3, [r9, #28] @ _455, MEM[(char * *)tl_317 + 28B] str r3, [sp, #16] @ _455, %sfp @ kernel/sched/topology.c:1660: *sd = (struct sched_domain){ str ip, [sp, #8] @ jiffies.326_454, %sfp bl memset @ ldr r3, [sp, #36] @ jiffies.324_453, %sfp ldr r2, [sp, #28] @ now, %sfp str r3, [fp, #48] @ jiffies.324_453, sd_352->last_balance ldr r3, [sp, #16] @ _455, %sfp ldr ip, [sp, #8] @ jiffies.326_454, %sfp str r2, [fp, #72] @ now, sd_352->newidle_stamp str r3, [fp, #272] @ _455, sd_352->name mov r3, #16 @ tmp1502, ldr r2, [sp, #32] @ now, %sfp str r3, [fp, #20] @ tmp1502, sd_352->busy_factor @ kernel/sched/topology.c:1678: | sd_flags orr r3, r4, #4096 @ _452, sd_flags, @ kernel/sched/topology.c:1696: WARN_ONCE((sd->flags & (SD_SHARE_CPUCAPACITY | SD_ASYM_CPUCAPACITY)) == and r4, r4, #160 @ tmp779, sd_flags, @ kernel/sched/topology.c:1678: | sd_flags orr r3, r3, #23 @ _452, _452, @ kernel/sched/topology.c:1660: *sd = (struct sched_domain){ str r2, [fp, #76] @ now, sd_352->newidle_stamp @ kernel/sched/topology.c:1696: WARN_ONCE((sd->flags & (SD_SHARE_CPUCAPACITY | SD_ASYM_CPUCAPACITY)) == cmp r4, #160 @ tmp779, @ kernel/sched/topology.c:1660: *sd = (struct sched_domain){ mov r2, #512 @ tmp776, str ip, [fp, #88] @ jiffies.326_454, sd_352->last_decay_max_lb_cost str r2, [fp, #60] @ tmp776, sd_352->newidle_call str r2, [fp, #68] @ tmp776, sd_352->newidle_ratio @ kernel/sched/topology.c:1662: .max_interval = 2*sd_weight, lsl r2, r10, #1 @ tmp773, _484, @ kernel/sched/topology.c:1660: *sd = (struct sched_domain){ str r5, [fp, #4] @ sd, sd_352->child str r2, [fp, #16] @ tmp773, sd_352->max_interval mov r2, #117 @ tmp775, str r10, [fp, #12] @ _484, sd_352->min_interval str r2, [fp, #24] @ tmp775, sd_352->imbalance_pct mov r2, #256 @ tmp777, str r10, [fp, #52] @ _484, sd_352->balance_interval str r3, [fp, #40] @ _452, sd_352->flags str r2, [fp, #64] @ tmp777, sd_352->newidle_success --- If I add the new cpumask_and() I get the following after *sd assignment: @ kernel/sched/topology.c:1696: cpumask_and(sd_span, cpu_map, tl->mask(tl, cpu)); ldr r3, [r9] @ MEM[(const struct cpumask * (*) (struct sched_domain_topology_level *, int) *)tl_317], MEM[(const struct cpumask * (*) (struct sched_domain_topology_level *, int) *)tl_317] blx r3 @ MEM[(const struct cpumask * (*) (struct sched_domain_topology_level *, int) *)tl_317] @ ./include/linux/bitmap.h:329: return (*dst = *src1 & *src2 & BITMAP_LAST_WORD_MASK(nbits)) != 0; ldr r3, [r7] @ MEM[(const long unsigned int *)cpu_map_104(D)], MEM[(const long unsigned int *)cpu_map_104(D)] ldr r2, [r0] @ MEM[(const long unsigned int *)_457], MEM[(const long unsigned int *)_457] and r3, r3, r2 @ tmp788, MEM[(const long unsigned int *)cpu_map_104(D)], MEM[(const long unsigned int *)_457] @ ./include/linux/bitmap.h:329: return (*dst = *src1 & *src2 & BITMAP_LAST_WORD_MASK(nbits)) != 0; uxth r3, r3 @ tmp791, tmp788 @ ./include/linux/bitmap.h:329: return (*dst = *src1 & *src2 & BITMAP_LAST_WORD_MASK(nbits)) != 0; str r3, [fp, #292] @ tmp791, MEM[(long unsigned int *)sd_352 + 292B] --- Both cpumask_and() seems to store to: MEM[(long unsigned int *)sd_352 + 292B] So I'm at a loss why this happens. Let me dig little more. -- Thanks and Regards, Prateek