From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from SN4PR0501CU005.outbound.protection.outlook.com (mail-southcentralusazon11011070.outbound.protection.outlook.com [40.93.194.70]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C602737DAD5 for ; Thu, 4 Jun 2026 06:59:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.93.194.70 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780556351; cv=fail; b=pxkNG2MjrOLP/l9aWNpQTsfRm06u1zD38xl7qMvb+ccMAhh0DlCQtSCaAQ/tPS/lWIgKoOWQJ/KX0R133fkGYpcGyJ7CKkvp1fMPMpgv3upQO+q0TK4hTbMHCGTkwe7QXK/R+xqOOVAqdvp5Zrzu++t81sxRXmTWcA5cxZddVzQ= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780556351; c=relaxed/simple; bh=28+NG+eicLDo+berVo5G0OhVIMWL9y6iT8IXnAjsC6Q=; h=Message-ID:Date:MIME-Version:Subject:To:CC:References:From: In-Reply-To:Content-Type; b=j6eClS7xhE1adoCqyTXF3u6KCmL3IrFSioAQdhQFSAIurOs6txJVqFVhqvxzmEOXm+yQz+j4HXLCl5dpZzaafdXvAWTxsWjibaNhVJjRN69t7nGy8UJsBgwnYqWU+XQhc+uYG1NlZ1PlVjnBcrC80g39TyuTTXBuERguhdZeTnA= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=VvpMi11A; arc=fail smtp.client-ip=40.93.194.70 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="VvpMi11A" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Ao6q3PDsEakimstW1lYZpXFQch8ZFiCybtXMWqgHT7SmSG7Qz+oP38f2r4pU3Bu4MSg0bTMqNVeEfhdMR7HzdVhbgddXnX7NBYIPpDnryVY55PPMKU+x5q6UYvz96mlw/daooKBU8i0KPP8sJ2Rr5uuOnvBdsnYh1fc0eCEbVl3nGzYYfrX7/vduUJU8DxUM3NjFMAKBE/kKtAgG9Rz6M70/wSWkKLvTje3kfqtJQFTk49113lEfOdMdKMnHwt0K844MOGkGbIQU+8QREFB5DbwAHTWMC/dD3UVxmjr0JMtcrJ7yijqCTt0n35XcIpo6v88wgYjW1XNsTIhyYFBo9w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=CLdP4BoWLPW82uegHcsknsGr50W7EkckTCkpDpRnm5I=; b=gs7sqNrl0nS9R+W5NO9rRGtNP3He09ApQJvMO4eAXlURPVwhG3pCWqmHT9MvJuXPiHNylx2fNXlPQnfAr5ZKxyAO7G/fVQ3RfKlPG9MQ2NAhvCj1WI8C/5BXUBPB8XhMHt8flYwft/SXYMInpG2DTyk+h9jvbCHmPCwW6/mYpI0mxMkqvEqkNaCvZutqTo5pJydCpHD+Rd9R4VZtnqZ/2op5irtwsyuThAPhtQHq9xUtMp0cgw3RBXXpJ3QAZDcAzkvnR0M2ZHgMTEW/JI02mR2M1AC4zAjAzqqvVbiFgi8mXOugLw99g3ZbDjvXXztmhJqTVpHo9SUAAvXHsK/tkg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=linux.ibm.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=CLdP4BoWLPW82uegHcsknsGr50W7EkckTCkpDpRnm5I=; b=VvpMi11AGp8IUg+QtbySqNR0/4cREtj1HIGH304T5eMwikbYfdXYFA79RBSPRfSbu+1TgMi6C30q+wjpIQRnMOFmIufvVjiX4O6P/H5opTGrtNlNIDyzRzxzZ2DtLrypqZEJI6zX79Uz/cLBJzM/NTeE9eIhADGQj1L64GJIOyY= Received: from SN7P222CA0008.NAMP222.PROD.OUTLOOK.COM (2603:10b6:806:124::12) by IA1PR12MB6137.namprd12.prod.outlook.com (2603:10b6:208:3eb::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.92.7; Thu, 4 Jun 2026 06:59:01 +0000 Received: from SN1PEPF0002529F.namprd05.prod.outlook.com (2603:10b6:806:124:cafe::99) by SN7P222CA0008.outlook.office365.com (2603:10b6:806:124::12) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.92.7 via Frontend Transport; Thu, 4 Jun 2026 06:59:01 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=satlexmb08.amd.com; pr=C Received: from satlexmb08.amd.com (165.204.84.17) by SN1PEPF0002529F.mail.protection.outlook.com (10.167.242.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.92.5 via Frontend Transport; Thu, 4 Jun 2026 06:59:01 +0000 Received: from satlexmb10.amd.com (10.181.42.219) by satlexmb08.amd.com (10.181.42.217) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.41; Thu, 4 Jun 2026 01:59:00 -0500 Received: from satlexmb08.amd.com (10.181.42.217) by satlexmb10.amd.com (10.181.42.219) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.41; Thu, 4 Jun 2026 01:59:00 -0500 Received: from [10.136.33.4] (10.180.168.240) by satlexmb08.amd.com (10.181.42.217) with Microsoft SMTP Server id 15.2.2562.41 via Frontend Transport; Thu, 4 Jun 2026 01:58:53 -0500 Message-ID: <83032c54-0cdf-429c-bcff-bba4e5b2002b@amd.com> Date: Thu, 4 Jun 2026 12:28:47 +0530 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v3 11/20] sched/core: Push current task from non preferred CPU To: Shrikanth Hegde , , , , , , , CC: , , , , , , , , , , , , , , , , , , , , , , References: <20260514152204.481115-1-sshegde@linux.ibm.com> <20260514152204.481115-12-sshegde@linux.ibm.com> Content-Language: en-US From: K Prateek Nayak In-Reply-To: <20260514152204.481115-12-sshegde@linux.ibm.com> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN1PEPF0002529F:EE_|IA1PR12MB6137:EE_ X-MS-Office365-Filtering-Correlation-Id: c075c811-87f0-4289-f682-08dec206c1f0 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|36860700016|1800799024|7416014|376014|11063799006|4143699003|3023799007|56012099006|18002099003|22082099003; X-Microsoft-Antispam-Message-Info: bnChVEzkdyGzep5wCgNOw5VTKRkeWc1hsi9vWJNGZpZ9/RvqpeuX3OWn962pL6QSwNqaBJKBOqJpr1L36VWiQfs7rSISk9dUE8anbbiVgyXDLmZesKYDU0WxIaahSNMzHiCwShzpvovcbvav6urByIOKDT59HiJGmuBXf5U5vhrea3qsvg/cssyn8uHmhb6TSj9Ofh50S5eT9tF9urNItvVk/4+oqQr9Wohdsrf1Il7YwMEOOIQgKbWHwvDCB1Z+ATfML7baQ+goiBIcNCwcEgP6wtrHFR/u9Hbbv9lDRTFJMEkOjjjRfwWBR9ws1YVWdNWZkOFDLk6Ey1bLCUzJSWgu1yJaeWxK4qIZbba9k9BXM8PAsTvuQ3wyA4/YKlZjlROsnIeef2px/ymTngSojBTF7znrmlWhqtCbYLypIlhPhDNKS02J3sMq4OKQ8+kRneT209gJuDcKIlgNXIhU+eSNayloLJDA6a5iw92arN5n7f6+F52Wf+pIAohtU99vtL6Ohb+6sc1k5617FeotxeGQ8aGsI/eV3fi0W2fjh3xhU/Mw+a04Sd8BXtF1X1Bud5tzKnWF9bfMZKgexkj2zaYpMh/Jx46K3PfPinLxHGTj0ewDRNzBFKbd9PIJOdSdmuMlw1DRGqaJn0DItO3DVZO5xmMcx2QMa2Gh93GR3EQqPYwSkIsz8QoIsBELzWpgF/pZPDDyjaxO3zGWzqBQfaOQj8G4TiNgYHsDrLwwD9s= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:satlexmb08.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(82310400026)(36860700016)(1800799024)(7416014)(376014)(11063799006)(4143699003)(3023799007)(56012099006)(18002099003)(22082099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: g2P1K9A2ZZQCPNr+RBN0vl3X2zyFBVTrlNLH+YZvGaAJXpV7tQKyqXG3dRrtoGQ3+rLq/jIQoDXxBeBzD2cg1Au2UgBuLUYYmXrzVY9EkVgWeypRt6jcwb2iP1ovwEjcZVq2/KUzuqi8/w2xU5AKeSLfoQzcj4YOOD4rihc8g77h0msLShi1HxNPX4e03QKIJHk6gdwkzgDTH555yVtP1GmC+ZgkYXgT3N7NGqn1fcEK054ZL4ZoupRJNcqQS5OvIZvWKQXubNR+jitYqZwrpvjuPekyhiRqSZZBNG5MRVf+nPPQiZsgNkQOlpkj0IGLkLOQW+KYiiuCRxsIQSh1aZRNoR7RfmhbY0Ba5leVUfG0zN1MeOE8nmYGSO0ocQ36xoKTGbannPZZydm0yAqAzguxxCCZ9IGEYr02Gxc5QCEa5r7F5c9b4OIX4KEcXem6 X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Jun 2026 06:59:01.1967 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: c075c811-87f0-4289-f682-08dec206c1f0 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[satlexmb08.amd.com] X-MS-Exchange-CrossTenant-AuthSource: SN1PEPF0002529F.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB6137 Hello Shrikanth, On 5/14/2026 8:51 PM, Shrikanth Hegde wrote: > diff --git a/kernel/sched/core.c b/kernel/sched/core.c > index 86fa4bfaead0..508773e71929 100644 > --- a/kernel/sched/core.c > +++ b/kernel/sched/core.c > @@ -5678,6 +5678,9 @@ void sched_tick(void) > unsigned long hw_pressure; > u64 resched_latency; > > + if (!cpu_preferred(cpu)) > + sched_push_current_non_preferred_cpu(rq); > + Is there a reason why we don't do a bulk move of all the queued tasks at once? Is it simply for the latency reason or could it lead to too many task movements if the CPU frequently transitions in and out of the preferred mask ? > if (housekeeping_cpu(cpu, HK_TYPE_KERNEL_NOISE)) > arch_scale_freq_tick(); > > @@ -11263,3 +11266,87 @@ void sched_change_end(struct sched_change_ctx *ctx) > p->sched_class->prio_changed(rq, p, ctx->prio); > } > } > + > +#ifdef CONFIG_PREFERRED_CPU > +/* npc - non preferred CPU */ > +static DEFINE_PER_CPU(struct cpu_stop_work, npc_push_task_work); > + > +static int sched_non_preferred_cpu_push_stop(void *arg) > +{ > + struct task_struct *p = arg; > + struct rq *rq = this_rq(); > + struct rq_flags rf; > + int cpu; I think we should do that sanity check for cpu_preferred() here instead of doing it back to back when sched_push_current_non_preferred_cpu() is called from the tick handler. > + > + raw_spin_lock_irq(&p->pi_lock); > + rq_lock(rq, &rf); > + rq->push_task_work_done = 0; > + > + update_rq_clock(rq); > + > + if (task_rq(p) == rq && task_on_rq_queued(p)) { > + cpu = select_fallback_rq(rq->cpu, p); > + rq = __migrate_task(rq, &rf, p, cpu); > + } > + > + rq_unlock(rq, &rf); > + raw_spin_unlock_irq(&p->pi_lock); > + put_task_struct(p); > + > + return 0; > +} > + > +/* > + * Push the current task running on non-preferred CPU. > + * Using this non preferred CPU will lead to more vCPU preemptions > + * in the host. So it is better not to use this CPU. > + * > + * Since task is running, call a stopper to push the task out. This is > + * similar to how task moves during hotplug. In select_fallback_rq a > + * preferred CPU will be chosen and henceforth task shouldn't come back to > + * this CPU again. > + * > + * Works for FAIR/RT class only > + * > + * If task is affined only non-preferred CPUs, it can't be moved out > + */ > +void sched_push_current_non_preferred_cpu(struct rq *rq) > +{ > + struct task_struct *push_task = rq->curr; > + unsigned long flags; > + struct rq_flags rf; > + > + /* sanity check */ > + if (cpu_preferred(rq->cpu)) > + return; > + > + /* Push only if it is FAIR/RT class */ > + if (push_task->sched_class != &fair_sched_class && > + push_task->sched_class != &rt_sched_class) > + return; > + > + if (kthread_is_per_cpu(push_task) || > + is_migration_disabled(push_task)) > + return; > + > + /* Is there any preferred CPU in the affinity list */ > + if (!task_has_preferred_cpus(push_task)) > + return; I think there is some value to teach the __migrate_enable() path about the preferred_cpu - that way, we don't end up with __set_cpus_allowed_ptr_locked() putting a task on non-preferred CPU by selecting it from: cpumask_any_and_distribute(cpu_valid_mask, ctx->new_mask); and affine_move_task() can handle cases where the task does a migrate_enable() from a non-preferred CPU / changes affinity and can be pushed out readily. I'll let Peter comment since that bit already very complicated and this would be adding more to that already complicated machinery. > + > + /* There is already a stopper thread for this. Dont race with it */ > + if (rq->push_task_work_done == 1) > + return; > + > + local_irq_save(flags); > + > + get_task_struct(push_task); > + > + rq_lock(rq, &rf); nit. the irqsave and rq_lock can use guards. > + rq->push_task_work_done = 1; nit. Apart from task_has_preferred_cpus(), all the checks above have no dependency on CONFIG_PREFERRED_CPU. Can we set some local indicator in sched_tick() within the rq_lock() section to then schedule the stopper once we drop the rq_lock there? That way we don't have to grab the rq_lock thrice in the worst case scenario where we have to schedule the stopper. > + rq_unlock(rq, &rf); > + > + stop_one_cpu_nowait(rq->cpu, sched_non_preferred_cpu_push_stop, > + push_task, this_cpu_ptr(&npc_push_task_work)); > + local_irq_restore(flags); > +} > +#endif -- Thanks and Regards, Prateek