From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2049.outbound.protection.outlook.com [40.107.236.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0BA7218DF62 for ; Wed, 18 Dec 2024 10:23:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.236.49 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734517429; cv=fail; b=I81z30wQkYyw94WXWi73lpGBrhWkAzP1nuqydNd4ZGLT4hPYvXuqtrTukVBWaG80ZspQleD+shpgNDoOoS0gcyVOF+pyOHOI5mzevcDCNl1pqlmCuS5Gd3BgeaKN6OV3+paGTDjcV1Dt1jHAsnyERy5MDM2lQKE2YjB0C2T1s4w= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734517429; c=relaxed/simple; bh=wOvoJMNeYeCV7pFNChtNviuSxzc8lqYj0oxwn5kUREk=; h=Date:From:To:Cc:Subject:Message-ID:References:Content-Type: Content-Disposition:In-Reply-To:MIME-Version; b=PiqXZnvLiiUxhz1Y7AzBaePWuW5crTM94Gknmyztn3AQgIW7s8p/JVjwNh1yMvhhFfcQBkBC1BrlxaWdaM3viH8CJ/vsNww56VGsLog2O0evsfZfBJUh5kM18ZM/oDrcfBu+7juPnxgHt0wXXz7+hGcl9c9VXA/3ydyMMSaCiUM= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=eEA0NCEG; arc=fail smtp.client-ip=40.107.236.49 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="eEA0NCEG" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=INiizDpSt3L5zCHvWBjfVqRyYt2A/2kQ/+wA1cAiNlrWTAbp6f2WqmgJRuLsqVfPYdHK/eK3iCD5RWmr9TGN/n2WqEpGT2KUh/x4f6dRLFdGE7BYVrimkU49i80qutE+G/lv6rBd7sTabytnuOTYb43EOVcJru/p+LgRy0Cwcs8cH5HaKgQholdnvOtwGy7UiQr2WqlmsHF4Bw3UeRBMa8xS4NqDrRMbl7WAFev9flbPba4Z/vNI99F/Zpeceoo8VfvuX+qt4J/11uhC8KB73LyjG/quMNpPc9cPJY+a2QpJEo9cdme+qcyq84mhgepWqEECIMh7t+GrqUpRYraixQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=3fMN4WeEDM45uWfNsNIxJcC5+FvfCPYGTu9DCP4wkZg=; b=e1B77lUoJZkP3A32su3dbU68qJGJtSd2y8XXUFlVFW78Jb/1TLvBd2B0m9F2vUOk+YMn2Kn9GqEYroO4X8ySP0Nx3hOjhMjVOTStkJs9Zo2vw8KS5KjZAbj3v8oMTl8MpXNN9pptalEAUcLbulPx/HciU/uA0IRxZI4NNzy9wBj3O3CotBdN5R2Kz509clGNk06BtwkN9cfy93fuu8Tm6NIRlNXV9ywqWpGCUX2uek15KhwqbfX6gB4LeTMuF4xwP9CnKLvi0QdCvQZTHBV2TcB9Wb/OnnjIQvXZnYYmWQ1Dcd6mH6mO6BWFD/3evORGmbhxFhQ+/4KcioMudEfyJA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=3fMN4WeEDM45uWfNsNIxJcC5+FvfCPYGTu9DCP4wkZg=; b=eEA0NCEGRWrTz74EMhK0LX8BsnCAjNy0I0J8xvKlEYCM1kX/O3wMTjzxKSWE8blpuxGjV2Xrw0NZxqgbnWzcMn4aLIK7g7Ganx+MIp9diDOxrR06pFr3mJ/OYe2Q6ZT5uyCHVUWMpbAklzlM3+cyIBECMSrE4dZbaH4ll+Tp9ned7roOSLquePio2A4GluHEVcNZpBRIZnWciqVxXezWJgiJVvqCLKhJow1ThieQ58OVDYmS3cqHbuZTtt6lKEdeZYcoRxmVpS1+nbHunEgw1sVB26Icp4wa6qBtl0bLOK5SuYKG4BqCq8ua/nJjzne9dIG9koS6kz4gOmgwm60ZxQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from CY5PR12MB6405.namprd12.prod.outlook.com (2603:10b6:930:3e::17) by BY5PR12MB4291.namprd12.prod.outlook.com (2603:10b6:a03:20c::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8251.16; Wed, 18 Dec 2024 10:23:44 +0000 Received: from CY5PR12MB6405.namprd12.prod.outlook.com ([fe80::2119:c96c:b455:53b5]) by CY5PR12MB6405.namprd12.prod.outlook.com ([fe80::2119:c96c:b455:53b5%3]) with mapi id 15.20.8251.015; Wed, 18 Dec 2024 10:23:44 +0000 Date: Wed, 18 Dec 2024 11:23:40 +0100 From: Andrea Righi To: Tejun Heo Cc: David Vernet , Changwoo Min , Yury Norov , Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , linux-kernel@vger.kernel.org Subject: Re: [PATCH 1/6] sched/topology: introduce for_each_numa_hop_node() / sched_numa_hop_node() Message-ID: References: <20241217094156.577262-1-arighi@nvidia.com> <20241217094156.577262-2-arighi@nvidia.com> Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-ClientProxiedBy: FR4P281CA0167.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:b7::8) To CY5PR12MB6405.namprd12.prod.outlook.com (2603:10b6:930:3e::17) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CY5PR12MB6405:EE_|BY5PR12MB4291:EE_ X-MS-Office365-Filtering-Correlation-Id: be0336ab-7b3d-4e7f-faa2-08dd1f4e0d57 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|366016|7416014; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?KNiGPtS1XTJazUxkjbwv7ILq8/qwEKJoi8l2cEZP+VvLWuI7j2hBuhIvAvtN?= =?us-ascii?Q?pCAVxY+F95DJKzztNZnIb0wiiI8EbsX6A47iNQDzoQsZn7G/18gCJwbCXMVv?= =?us-ascii?Q?lCVUMPrL88TMXenszhWXUzRxnFz21rPF3PkVfIzWR4mCFV/TRXNRMv3kzX0+?= =?us-ascii?Q?xLjkuo/xkKuggEsqaJi5iWVgscePRKkxYvOXR6WjfQIrmaiEqS7/n1LtfxO8?= =?us-ascii?Q?n66CpvHizX0LKoipui1ldr+sb1QiL96o+bdyfAzqSytNU48y0lQ3d6bDOqKp?= =?us-ascii?Q?i1yv+RcgVYUynUW/+aGO5S/q6R8tnvXaibcMJDmoClRPDnHnPOZgRfFRzwpx?= =?us-ascii?Q?HpyXEMZ8RnKOVJT48LPprmOZfkrIk1uwoBvdxVBMb/2RIZVoeNELLYdMm2br?= =?us-ascii?Q?5st5xoZcavHKgXQ/koktlYDE2gJ5fIZ+7VNLpoVbSovEf4HKNceYF07yGtsj?= =?us-ascii?Q?wFHSDvizfKXKjPAMdRf5eIgjvJL3bAaOYwUFrMEYAYsg9/hMj63f36vW3qVR?= =?us-ascii?Q?2HOC7eXhRdf59RjZgu/G0+AydJVuwzD8VSaccc9D/xxXrM9MyvhSxLUOAmzL?= =?us-ascii?Q?L8IrKFYnBXSWfLwVTaoSkWJqMmKhXS2ussizR4Svwxzi7piGiYSnERX3q/G3?= =?us-ascii?Q?PKhlouvtH9gsaLxloW8rtt4ty2D9hMEdariNS3VOeWqXDm4m0jHMk25lIkjp?= =?us-ascii?Q?IhPbDiW0RecVXGgrLtlrMaLcRO+0g7HBLC+sT64q7N/i0YZ1ip2qfYHe7/4+?= =?us-ascii?Q?W7lBPJdWWNpnWvd+hy7ABaG2RgSuvNc8gVgM2/TP+xHEFTlchJSSbGCpcId/?= =?us-ascii?Q?7FOPlOZNPdyw2TWs5fItyL4oieVXPJDnufklOhM7ZHebX6GFOId5TU0fYz4/?= =?us-ascii?Q?3H+iJRqbh6XYY9xHqyRbAcXhKITeXUKzuSFhL7ezKa7gkchf68zw1BhLtej8?= =?us-ascii?Q?/TAEjM6iD0Ti/wZhqFNNMC32fn06IB+TEtaTJea2uidBbEJyH0VuzkdSBZkS?= =?us-ascii?Q?r63d+kD088OM/sFSmmtrNjPw0oVqjmLhqNTVCRGuH3CduaFtkpHO8r/4HHXN?= =?us-ascii?Q?CJ+wKRoWjnU6bAwttz+WvZBHBW6oh/fLh5jBu22uqjhKAOzUc2Le05TZkNjO?= =?us-ascii?Q?GbN52vpFceS3QMwhqS/yhlLmRy/Qa5UAFcg2B4eVyIOh9+6berVb2Ru0TiXD?= =?us-ascii?Q?CHJ9iLZQKJcTWW2YaMDu/8VAJOhn1/O82l7Qh7IQtccbQRDECLdvXgvzhJFU?= =?us-ascii?Q?0Kz1fWWL+TwOewr5bWKLNeHQwXrMTXWo2BWoJwnvQ418x3IcCGt7uWym9bdd?= =?us-ascii?Q?ILbJlcrNc2MvJn1jx/nnb2/gL3PrAou0IvorlmfLwaGursQJ1bB6+4YyIkun?= =?us-ascii?Q?ROomLZvNoaqHQUb87Yb4H4WE2rAq?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CY5PR12MB6405.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(1800799024)(366016)(7416014);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?kUyf9WR5Y3Kjz8R9FxYuur1nGPJMcbWc8JAupOLrC8bTWnbVeFIuKEn5mD9P?= =?us-ascii?Q?TotYRs8gY1evBy9xCD+CxpPXuWkMe2RXIZbQR2pUO3TdQrQENJWNufCSdlIa?= =?us-ascii?Q?JMvDV3zjLLuYwDBEmc4AGvii7NGBVm1gWN+iQ9bK5oLwD7sHPxk54fvNj5SF?= =?us-ascii?Q?KoqpqsPRlGEQr8x5Kg9ztYlqHu/XXrpSI8dG9yZpr5KLBkQkdvWEQiqowhe3?= =?us-ascii?Q?1kxVnR7xgnWauRSbFOqplx4E1Bzh5Q7mWfcU+Sbx4kx1EH1011vgAIGRCUKU?= =?us-ascii?Q?OTP2Pp59xi3e8EasQsFJEdjOiCx0R1oKW+s9SVJMZXULgVTz7HnadTN7hj4D?= =?us-ascii?Q?axlC2ZKL0UZO2ndfIMPypf7JztdUyRaO/C0bHJoDE+mlEiP5px1mIRZXY7J1?= =?us-ascii?Q?4o7JTjb7re+Fklglv6cJdfLtTED7HQJ8Yp7VNJ8mTcaem6WCnMLF6sSNlvfG?= =?us-ascii?Q?hguySKzOe3qmrKM2RNwnCJO1nT2qtIK6TXmKQjm2k/idZhMhHHq0e925/2Ua?= =?us-ascii?Q?L4eaXHiM+EH9IgX7aTT3/P2igflUTyjb9c89uJoWC6JOgBeAHDV2se5VHgVn?= =?us-ascii?Q?+QS1rTyH5DAhc6Rj6b9gELU158MRTTg6ABz2vQnV8MG4olhPNKIbvuwUeniB?= =?us-ascii?Q?CiV9D8xQVRL9i81VdWgN3VVBI8vxJSYSQz4WML2TNWoitcpWGFyBbIcA9XAb?= =?us-ascii?Q?rTzdqWyR0Xb3IoxaTSHRe0wxNZ/ZixxUMGR1YDyKeDwP8rEdugemn9qKbo9p?= =?us-ascii?Q?b1JXSlHwjeHpuPehI3OS/MXb/PuN2XljQtU9gM0C3zl6EgH6Js0ivjyqakk9?= =?us-ascii?Q?NDEmQPHG+kfGTlJveFNs02G+Pvnv9g+Q/4jo6k6msGorFb7nClwvJaYGn6CT?= =?us-ascii?Q?FuQgHnmJH42nZT+RlR4tJjMsWVYsesSfNfFaImQx/imYOU/snnXqonQ+oITo?= =?us-ascii?Q?RLQHIdk5/8CmWckqy8XHr6Af7Twn4Xl8sCeCrtVTDz4n87M6cF+jnhkZ64ix?= =?us-ascii?Q?1WuAe80s6GYpcTdrvnWLdkh11sOc7bjTzIM2ntcI3BqeJ/Uw012cGun8EBtX?= =?us-ascii?Q?iDbZI/uIR8cBv28jaTlDWutYozzmibNpfOzrD0fDwYPmzCsAcGrA3LCOt//h?= =?us-ascii?Q?0tbXI358KIfvgt0jnsfSAKH5sLx41lZ1v2L/ssU2SEKVO1w+8TzwwxVHhlNs?= =?us-ascii?Q?qcpONJoS6xJrrtPTg7Hgh8p4iptXIKn+QYG4UFBylHaCKUM3SVx4fzj3URoj?= =?us-ascii?Q?bSW0MxDbsL8R3p3sacTW4OyvOA7pT3x4Ob0ZFn03RwtxIoEh0UV6Gs7NCm0i?= =?us-ascii?Q?Ia1kE7wGK3TCVotbtzHKHD9sQb19Vy1O/6Ye5rnbVR2hyZ4yt64WHm6H6YV8?= =?us-ascii?Q?rHH+irB2U7b3ykIZCs/X2oYRqsKn7j5GnXMPQIa+xEeRfnGGLxxSNyAKHt2L?= =?us-ascii?Q?vL6bqg1Bu4R5TL/mZUT3gEV7a+20NvM6yUpfTCqhYrh4Sp/HLuUWNSe1QwOX?= =?us-ascii?Q?a2fpTGJMjHHmIsmKF2K2P1oYCGYb4tnG+Huv8sJrE7kjUwJ4bUZOZZcr5PK/?= =?us-ascii?Q?k1AM2O1gIYqntGD4NzaCJhVL4xSXNQN5J/GflOHs?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: be0336ab-7b3d-4e7f-faa2-08dd1f4e0d57 X-MS-Exchange-CrossTenant-AuthSource: CY5PR12MB6405.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Dec 2024 10:23:44.9066 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: tJG7KIAcR6HI0eGb43N3CVHAFJ+vChgDsB8eAm1Tzr352EHAOYTxjMUkfFYS8b1Jl+K0V6ulwrQUOBojhAgyeg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR12MB4291 On Tue, Dec 17, 2024 at 11:57:44AM -1000, Tejun Heo wrote: > Hello, > > On Tue, Dec 17, 2024 at 10:32:26AM +0100, Andrea Righi wrote: > ... > > +int sched_numa_hop_node(nodemask_t *hop_nodes, int start, unsigned int state) > > +{ > > + int dist, n, min_node, min_dist; > > + > > + if (state >= NR_NODE_STATES) > > + return NUMA_NO_NODE; > > + > > + min_node = NUMA_NO_NODE; > > + min_dist = INT_MAX; > > + > > + for_each_node_state(n, state) { > > + if (n == start || node_isset(n, *hop_nodes)) > > + continue; > > + dist = node_distance(start, n); > > + if (dist < min_dist) { > > + min_dist = dist; > > + min_node = n; > > + } > > + } > > + if (min_node != NUMA_NO_NODE) > > + node_set(min_node, *hop_nodes); > > + > > + return min_node; > > +} > > So, this would work but given that there is nothing dynamic about this > ordering, would it make more sense to build the ordering and store it > per-node? Then, the iteration just becomes walking that array. I've also considered doing that. I don't know if it'd work with offline nodes, but maybe we can just check node_online(node) at each iteration and skip those that are not online. -Andrea