From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2088.outbound.protection.outlook.com [40.107.94.88]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 38B5A1C760D for ; Fri, 3 Jan 2025 08:55:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.94.88 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735894529; cv=fail; b=WIxXxh48aKsypOw3u1o8Qs2FGZ4xy9rCjEdNdTh5QUK5fXJ0YDW7HBEacLRFJnX8pcM904rWvqYdKDhn3Hhckwnpv4iKM/d+2CZlR9gpQ41aPjYnOoy/iwpDpKHMiFhBkL3i0gamDDTNgQ/HZUxkidbR0wlq0o+fDVGMJhM6sG0= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735894529; c=relaxed/simple; bh=lR4M5nxyfkewQPbwBwaP7latv4tAjS4eNqvmH7LsEFw=; h=Date:From:To:Cc:Subject:Message-ID:References:Content-Type: Content-Disposition:In-Reply-To:MIME-Version; b=ov2kHVTlexwacLVJOfDqMf1LzJA1aJuJo+efHLjDvLDa7PSWcKXgS+yrSirxEtLVn+YXhjmI/e9e9Ajmiilokz6rHbAOWWH5yZGAE5dp4bVjPVtq4v18+eEooJNlLjiGPdGKhyLFfIq/jp7xFKX2NO9pT1/osaUvDFA5gQhT+SU= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=fTMv+OU1; arc=fail smtp.client-ip=40.107.94.88 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="fTMv+OU1" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=kQ19xgUdtsX7ywy5rGkYD5SiCLqDe6/ku0Z4tvej0eH5AKkkzrW6bTyAFYE+RS3bUEaKQnnkQ42I3uSHnFOXl3/NspkVLI+HRmSbdLnpNNVPvsaYNFckr2xNGQOOr+loS3C34q6A/rK8bJbOg5ZzK8btZilrSIL1Kv26hD7opYs8MGXq2eEYfA4hIl9UMGeV0/GX9zf/Kk5bEpU6aXYqRb5LBvW6v5L6iIcjPnuJz1pt32qjkoeL6PIZ1+S4gypF5h2GKlgjeWXg5BZIL7ZIAEBYeT7D1Jqm54Lb3CRnH2t3z+zIpD0bPCTm8ziGcF18Jd0EGgD9A1Q2hYIdq8LKGA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=XB1nPLy+loDCQ4xEXoLXo3ukLZCBiy4AKt2IcyKcNUg=; b=RmVRr/cFsGpca9VPhCIPlix2tsXPvguNMqFf2jDTFx6Rg82R1FQNZf1k0SBOAax6YS2ZVJ4OPL2hf/L2U9kG4QfInPpnbw1eb8UG5lemaPGW4w16mVBJOaoM4SQ+r5rnQ2vl5LMWlvnbdtMgzVctP6n6J1ybi31wA/LCJcHPoZK+Nt/k/HO/LcCo7EGo8WKZfgSOt6iPIDOsvqJI7QhxZ04kXe9Ix+g/tY/kIJtjhebxFBT3qPd4vjUor7K3ymzSSW0s7ZJ3IlNg5JUvU7nc/JFPDEJ+1ATZxxr9nxFHUBp8YLgm+U1CzDgrtTwUzHitJIjPHfh5ymja5SGGog4bQw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XB1nPLy+loDCQ4xEXoLXo3ukLZCBiy4AKt2IcyKcNUg=; b=fTMv+OU1hryoZ+SG9Vb3o1sJuoTeleoqjT/OFyxXo+Zq/zm1m9EnB7s4mQoov2m0SSgU1fTiaMsyV6OmVFbxbFqzjYPHkDqn+e7erL5M8Omybgx970aXgNCb5Yi/HkVUhtFOqzEcT+cAcEUfGKW0AJ0y6LdN4lxSC6Tq2kgrmFBqh+Sz9OXChL8bHNlekGxMcvj0PK0RwmBg3ujFST2c4DhXOsMDhUUHsX6KdI3U+f0SHjP7kalyUfOodwzLDQtvDYFmj4RZKL21rnVtg3qb8uCVLaZ6Nat7LolnqtNP7GhFFbkpCDU7j37X3cAqbFSI5spCt5zkAw+VPG/6CZR1Gw== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from CY5PR12MB6405.namprd12.prod.outlook.com (2603:10b6:930:3e::17) by PH8PR12MB7327.namprd12.prod.outlook.com (2603:10b6:510:215::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8314.12; Fri, 3 Jan 2025 08:55:19 +0000 Received: from CY5PR12MB6405.namprd12.prod.outlook.com ([fe80::2119:c96c:b455:53b5]) by CY5PR12MB6405.namprd12.prod.outlook.com ([fe80::2119:c96c:b455:53b5%3]) with mapi id 15.20.8314.013; Fri, 3 Jan 2025 08:55:19 +0000 Date: Fri, 3 Jan 2025 09:55:14 +0100 From: Andrea Righi To: Tejun Heo Cc: David Vernet , Changwoo Min , linux-kernel@vger.kernel.org Subject: Re: [PATCH] sched_ext: Refresh idle state when kicking CPUs Message-ID: References: <20250101182449.21517-1-arighi@nvidia.com> Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-ClientProxiedBy: FR4P281CA0173.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:b7::16) To CY5PR12MB6405.namprd12.prod.outlook.com (2603:10b6:930:3e::17) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CY5PR12MB6405:EE_|PH8PR12MB7327:EE_ X-MS-Office365-Filtering-Correlation-Id: b67ffd60-6480-4422-dc4b-08dd2bd45987 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|376014|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?ApLUlbe1RE1rIKcbKxT/a4y9OU4k8ZV+P5KQDsqjzKFdcSD9MG/m9rOs+DTa?= =?us-ascii?Q?TB6fCtBO0lzzCvoF3Rnu9kun2SAW+wtElpwauQfrZckP3OhixKudJV+yrxv/?= =?us-ascii?Q?kwklImdgYsXXTUwHp8mnq/JS9ffaG8O9qKqQeI9+qMexhZB3N6O/YesmZi3K?= =?us-ascii?Q?seI4IU3jUMnKKwcT7+rslYsjy+tpuHxhdJM/BNveqX/7ZbChs8u3eC61OQYZ?= =?us-ascii?Q?xndY87Tkm6hzLRd5YSFh+iHa4VoFpv2kwRltXhLpRe/bTC+femgtqUpjNfW+?= =?us-ascii?Q?Lrbo3DChaZKIynWKiLUnp6Tp6zz9W7zfAEXgLdxddqFid/Sj9jYo3Nh61vR1?= =?us-ascii?Q?cVN7NK9ok07TiOFAOOWWLMDRsQzyndnFpFOcxGtFGuxWEn+IGvUoCRF0Nj2W?= =?us-ascii?Q?5+PMIslj2G8yCHaHWbOMcaEeEPUAbRbbDlBgcB6AbHpO1ak3yI9C+BafCf3O?= =?us-ascii?Q?w/kLHxQvW/nJ3nA6kIW8e7lN2vGDUvcfvVpRThgkwRKMtc+m365f6KDF9jnI?= =?us-ascii?Q?8fO+2KqcB+Rir9hR0z6RdlmQBh3KL2hLkbdpwMSDKfVofpWaLQw/3br9+64h?= =?us-ascii?Q?73v9rvOv2ETnZqWV7gg8R5X1syTFuOeu+/73pdcdr15wFj5br8EHE7i9AKMm?= =?us-ascii?Q?+QCPSNRBSJheMXrxmHUkO/urBt5hv0z1hYMNuHMjxfJ9VXhw/jX2Z61TMl5F?= =?us-ascii?Q?oVPXI1Ao4RTq7Y5ZMACdFIoVgI6ItrZro9W+F+qLWoZZOBM2UBnssJszDpxA?= =?us-ascii?Q?WD4Riv7cf7fy5uxpY3njT/mWheW8rrDxQNn6WA9EMv5QT0aYXqamYxr65oM8?= =?us-ascii?Q?TVUyxvYHsTz+e7dSI/LH3zwAuj57bPNCtTsBzOmifMncXCqtqBto5GEF31I2?= =?us-ascii?Q?Idlf10cN+u2EtVLRGWoCdE2A/x+TYUMea2v1JvPTior0MiJ8WPnTPKz7ivjs?= =?us-ascii?Q?ts55TfTFmTrQ30KTkp8s5HM4TkUt7UPGsTvGcJ7rJqWUCzDz67Okl4g3Ar/Y?= =?us-ascii?Q?adtCoF5pfOL+Vt7GWpO4nlXVk/2tAfB6bDiUKwEAIsmlRXhmR8cz3xgZ14T+?= =?us-ascii?Q?Mo5U/M5jajX6QsVZhcSjdvlb8OZq+ldD51MKpFiqQj8hYlRQKDo7Xga4gZyB?= =?us-ascii?Q?n1eOcjm6WI+unMRbd81BRBetRggJFnRSa2XUKkw+c1EJ6oHR59CHQwR0C/J8?= =?us-ascii?Q?6welR60OB2lw67zV6SzH5G6u19LIE1I98VbeRFpBwWE38CwB9M6ovBEvXhpL?= =?us-ascii?Q?cc4hLOvND0Fh1x6oe/iNgBJKk2TdlvKjchOEvKPa9yacg1wTPkxDy5ybgsF+?= =?us-ascii?Q?l6UQYpCRdhjYsnjJuqkLQp4KECne0Mye7HZQ78lABFliYVGD00EJOeS9uL49?= =?us-ascii?Q?xbtV8noKod+08e42jEx/WUEIy9CC?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CY5PR12MB6405.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(376014)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?iuMzTc2ZtMUXbyyL8F3mnEpeV9dBzWFyYvaFMDS8/WNl5Zj2MmQ/hdkbaiIp?= =?us-ascii?Q?cRXXzmqQSsWai+fYVWJWkjjF17IzbHaf5AsB4RF8jEchf5AfWXyBoWBOIXs+?= =?us-ascii?Q?SjxEB09Q8H7g5Rq6c3Kc/Oya4dSdIt55Q715mjq5zovnNGeJ/rxkpEePvOsY?= =?us-ascii?Q?CIviLgKoChxq2BiQIxzzNdF3mCAIQpqqYNHuv2RaRwyeBDJxyzlfoFIb7Kfw?= =?us-ascii?Q?8JWPlNspTmI+MrOK99MfjQYNoQJEc2JraixDSxd7syE1gcH45Myxnb3sCuqT?= =?us-ascii?Q?3479zk+AwlUCoOkn/Fa4F/miUBJFCdfRxdZK3z75umPAK3wxLkckh8Nq8cRi?= =?us-ascii?Q?LUcS5SQHKZUmcO0vgP5TPOq2cobMs/uniIQpoCGuLs+WSQ9qktrdqS/t89zQ?= =?us-ascii?Q?k0U5dAoCIU+GEE+0LEIrhllxJlDrtxqyCWL1hVlBSsxzTy6sbyPOLXqXFmNE?= =?us-ascii?Q?7CAVIdR8VAAQEP8fNkfhZbZNLR5TVNTOncTSEmQ1ZLul2fV2kZ0vR4j7VYtQ?= =?us-ascii?Q?5JNhanUNLBGEY6unrnUhlDXh5slbFj7C7lTaOWga5mIe/JoRY0RoGMfk7y6K?= =?us-ascii?Q?c60tjs4H5yuLgr9ftLQhoSMazmr6MAaf6dvP1WJleNLELQFOaZ6ZBCM9fL+d?= =?us-ascii?Q?cbxxDY0iqNnNrCwZuusp+T512RbGpyUKTcfNcsdTbUe9mLUZylzbB3ZB7BYl?= =?us-ascii?Q?DYwwMaDRFAF+8izEYp3t6YQJ5Q+DSKu09cmzZmqh5VpqS25hyl1w5PhK01xo?= =?us-ascii?Q?7JyebHLsmOR40uRctCiYKuE2xJ3vrAmBdXPweGNP8T7PP+TPDXVLer3eZ41L?= =?us-ascii?Q?9uZ1wwgdkjOP0+PD5Cq2IRonPF7oSXzvEL/RpiihWzzX7Q3ULwiez5xDw8Bs?= =?us-ascii?Q?e9JWGfRVrKW9rMVLBRV7+uKBK2U9/+4Aa1Len2QLqefZ/L+mgoxikeE1yWVY?= =?us-ascii?Q?lXlOplPkzyzSnmEgrhZM6XW8xMqmYFu6QZIUWCyYrcY/p2N1vkByjjf7MtQD?= =?us-ascii?Q?Ha16NIGNbGQffDG6HC7vnR+DSV80nxyd9FgrHB44sdxjfbZQ1ZSQdTZjvlEm?= =?us-ascii?Q?33w5TsmlrzIXDS9Hthj1wdcFo2JGiDreza1ZPcN8V4E0ZIKQLEJ72E3r/Ri0?= =?us-ascii?Q?bQxq8SmEuBqdZWbvEbDWaJaqX1OX57XMKOxGQEnUKp1MQwYqelZNM5Los4SV?= =?us-ascii?Q?e8Pyt5TKLc3aIaveVIAcXB+K5k7B9q+0t19UtEH2x5EhFR0KycoATuM8nhiz?= =?us-ascii?Q?BolhEBGhAbb9HPss+I96tP/CeZvVdhI41Jc9+tuR5sAkh1mZ/OvP4IqKLK9u?= =?us-ascii?Q?qiwCwE7zohZt80R9QZzqGRpAMMhrlg8e6PFflsycED5q1OUm1iVKpq7EYqcq?= =?us-ascii?Q?IiLS6PSwv4BhjW5Mjg646poLRl/U8eXJAUvH+r3XK2Va6kSFbfZo8dguKZbK?= =?us-ascii?Q?48ARRGWX0V68TW5xKIaW8Ey8MdigFwGit8gZ3RiiYXQxTlNyDQcGsErTYPys?= =?us-ascii?Q?Z8dzVTVcPBtCIFFsio/6ABoPLKiL+NSaa3kWJ1ZobgsCR4qrglWnrr4ffAVV?= =?us-ascii?Q?g5iZXsHJALJyOHGouUXkGxwsazTewob0gwPv7cXw?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: b67ffd60-6480-4422-dc4b-08dd2bd45987 X-MS-Exchange-CrossTenant-AuthSource: CY5PR12MB6405.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Jan 2025 08:55:19.3015 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: SGPd6AL3i+7ehzU4NGDs2tZR65cyAlcfHrAv/FMMx71wZGxXibv25f9FOf7I5VW6YMunmrhJ851hfg5bZg9WKg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH8PR12MB7327 On Thu, Jan 02, 2025 at 12:33:06PM -1000, Tejun Heo wrote: > Hello, > > On Wed, Jan 01, 2025 at 07:24:49PM +0100, Andrea Righi wrote: > ... > > @@ -6104,6 +6117,7 @@ static bool kick_one_cpu(s32 cpu, struct rq *this_rq, unsigned long *pseqs) > > should_wait = true; > > } > > > > + refresh_idle_state_on_kick(rq); > > resched_curr(rq); > > I'm not sure this is quite correct. e.g. This can cause multiple > back-to-back busy->busy transitions and can incorrectly assert idle when the > CPU ends up running non-idle tasks afterwards. Right, we solve one issue, but it might result in some CPUs being marked as idle even though they're actively running a task. > > When the put_prev/set_next paths were reorganized, we lost the signal on the > CPU re-entering idle from idle. However, that signal is still available if > we hook into idle_class->pick_task(), right? So, if we move > update_idle(true) call there and make sure that we don't generate an event > on busy->busy transitions, we should be able to restore the previous > behavior? Which is basically what I did here: https://lore.kernel.org/lkml/20241015111539.12136-1-andrea.righi@linux.dev/ We didn't fully like this, because it'd introduce unbalanced transitions, as update_idle(cpu, true) can be generated multiple times. But it's probably fine, at the end we would just restore the original behavior and it'd allow to solve both the "pick_idle + kick CPU" and the "kick from update_idle()" scenarios. If we like this approach I can send a new patch updating the comment to better clarify the scenarios that we are trying to solve. What do you think? Thanks, -Andrea