From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-ed1-f44.google.com (mail-ed1-f44.google.com [209.85.208.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A5F9640EBA8 for ; Mon, 24 Aug 2026 12:52:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.44 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787575978; cv=none; b=tgXCPa5qT9Ho25c4zZ82Fb0ZJ38BHnJewY8oocJTaUNeyIZacPJ7xgkPJHO5D/L0Ujoxmam5HxHP+LnyhzVbOI+mQlJMMoMVh+tl42uNn5HntjgHvN1SnovXQ33gp1kPnJqiYwlAnGpvwayEQmRX7F4BFeG0VJGvdOIswEMxj3g= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787575978; c=relaxed/simple; bh=rAIsWQt3GEyagZyWMotzhnfEHx5bYGbDe8frnVL23j0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=fHm93g7fUtEHCO9v7gWTRLyR8F7RBZ7Oa8VGagDjgx4p0BU8u/6SEZtcgZhKBj6LEmF6lKpuH+hJ7v7nOHiUwq8H3D3Y0e9j7+/+eAVfk/KpqJUSTktv6nBmQzXyegLT33xUlD6aaTisTXwlMUdXN943odyNdnCzm8z8uREWWRQ= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=nT318qdy; arc=none smtp.client-ip=209.85.208.44 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="nT318qdy" Received: by mail-ed1-f44.google.com with SMTP id 4fb4d7f45d1cf-69f4acf7047so616621a12.3 for ; Mon, 24 Aug 2026 05:52:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1787575975; x=1788180775; darn=vger.kernel.org; h=content-transfer-encoding:content-type:mime-version:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to:content-type; bh=wWeKAezujR9DfI/TM1hhon1MqctOcFMwrHpI++UJFXw=; b=nT318qdy1mbmrZuWF763V/QDY/IlYMqc3JQoKntqRo2DHRUKKlrq0oJGl+IpgK7Xyo 7gErtlh/h1QuP0P0ShnMLPUwzLh+qrNJBAGlMGN0GbkMDkL3bzMeGDEVHgS9fMHTOFNb 7H1JR0TIqZ0ieJGhD7HYQd0z6EZUzevmXf53rvqvwMxhCYaKY2AYYTVsDlTeFef5DH5B Las0LBanqW9Sw87cn/ppJE1uaRWRj2UWCxkuQfYk+8fRykPXcaHx7hVLwveQmOYiW3cU PhVpArkb5iB4VEGdRTxlKEBUldUz+zepXtCmuv2t7K/p93OEM3ALVet3O9sNdEMy5i3J onCA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1787575975; x=1788180775; h=content-transfer-encoding:content-type:mime-version:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=wWeKAezujR9DfI/TM1hhon1MqctOcFMwrHpI++UJFXw=; b=ReMtfaSjfzhwPgc8YYNZW2j0Xv4C6Qz0P7EhQYaUprUd43P2k8YPaw5HN1CE9bRzhW gSDCuK0Tnnz/QXIRq5VhJQWliOCrAmSqfEASPdDpvx/5jwZr1uV+Y0ZStHTvK1pMopJb zpXrfjJ7qRxXMlIAwOEXAGwxAVUYjLgyFYVShP1qrSlkP3gHWzkqyUp4NMnuPT1e/QOB 3VNsIU6jagx0ZbKsLPrTQIvOJjACoOTUabneAiNsIA/aBfaDC25hOYmhJG4s+7m2DKWs E8moBcLnweuSfMgbw243SjJn0OazPZ5MLjsTN/uxqNtAYgUiELkkuIjKPLfBKXYdZCyA GK2Q== X-Forwarded-Encrypted: i=1; AHgh+RqwAHEsKGEhyYDgAQ3h24I9cKDXTVG9hxi5PQrlmTeCHOpuj+1dooUBo3jT+Pv7v2zLHkcRJ+nF/GjE5ek=@vger.kernel.org X-Gm-Message-State: AFuF++l5YQTOrFu0ya+WMU5Fs9wMWTo/wsipr31mbve2X+wlUrPqWuFd m0DML0iX03PBuFr0FowmR6V90Fpp2zNHc/xe6s8CQhjRImWRISnV04+NzW7qx+Ua X-Gm-Gg: AR+sD10w/lmwNn3fmCZG8QTGjPBilgDAxXlfZCUfMhuGe/RBaRPatcrFY/xf2ocMjWF zNolRtOYHgb5+iwfudJJZNQVlmj5qUKzFz53vN0x78e8cK/Ls3B6cqj8bGU9X5jHOIoB8ug7+5j VzM3yJUN9PwqyPR/XaCj2YUbRUYMLDYLqT1xkV5mdv92dvyHidSquG/3l4g/J3dtAyX3dDo005y xoLB5h+hdM1r+RIlDPaQmyqXDU4k+MMBK92mZ4zNGymQ7i3KQFmnhUR+8XGRQWosqimdKvp/si6 JnB7JQ/qUAjeyog5XsJScvqryqLRhhNFgI4aTAneE4Ywvz1/HBN0sh3L6GezKJcbtw4zKVn1/LI iPELwpu6GVH9FKxiSRwtxjHxqCxTq+SyBghd5gCqRkuNZgAVC4Ukjk4tWEJoVpNzNeTG4RGPm9B h7we5YjlIDp/11z7TN6vY8/ctIS7gwFg4oDPn/OtE2zV8VurC0/nbclIHLUMSVb4SHdkZfJpR1W 9NRiaNoawC/vDEivvHoQw== X-Received: by 2002:a17:907:f497:b0:c16:604a:b2bd with SMTP id a640c23a62f3a-c246a61c431mr1461836566b.3.1787575974734; Mon, 24 Aug 2026 05:52:54 -0700 (PDT) Received: from lima-kdev.local ([85.100.66.184]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-c249685c573sm1573839466b.51.2026.08.24.05.52.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Aug 2026 05:52:54 -0700 (PDT) From: Kayra Cizmeci To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , K Prateek Nayak Cc: Kayra Cizmeci , linux-kernel@vger.kernel.org Subject: [PATCH 2/2] sched/fair: avoid recalculating curr status in place_entity() and requeue_delayed_entity() Date: Mon, 24 Aug 2026 15:52:21 +0300 Message-ID: <20260824125223.508178-2-kayracizmeci@gmail.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260824125223.508178-1-kayracizmeci@gmail.com> References: <20260824125223.508178-1-kayracizmeci@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit In enqueue_task_fair() a bool is calculated by cfs_rq->curr == se. But this information gets recalculated on requeue_delayed_entity() and requeue_delayed_entity() only gets called in enqueue_task_fair(). And on place_entity() if se == curr we call the avg_vruntime_weight() twice with the same input. place_entity() only gets called in enqueue_task_fair() and requeue_delayed_entity(). Use the information on enqueue_task_fair() in requeue_delayed_entity(). And place_entity() to avoid calling avg_vruntime_weight() twice. Signed-off-by: Kayra Cizmeci --- The patch needs curr to be invariant so I added the same  (cfs_rq->curr == se) to places in requeue_delayed_entity() and and enqueue_task_fair(). Like after the place_entity() call in requeue_delayed_entity(), or like before if (curr) the old place of the calculation. And added WARN_ON_ONCE(curr != new_calc_curr) or something like that I don't know how to say it normally. Then I booted these changes on x86 (Zen 3) called perf bench sched messaging with 200 groups and 5000 loops. And then I make sure if the requeue_delayed_entity() was really working with ftrace. The results were good but I left the computer on 2 more hours and then checked the results again. It was still good, but considering that I booted the kernel with busybox I don't think much happened on that 2 hour window. kernel/sched/fair.c | 36 ++++++++++++++++++++++-------------- 1 file changed, 22 insertions(+), 14 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index b411384125ec..304ef70685d2 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -6175,7 +6175,7 @@ void __setparam_fair(struct task_struct *p, const struct sched_attr *attr) } static void -place_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int flags) +place_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int flags, bool is_curr) { u64 vslice, vruntime = avg_vruntime(cfs_rq); unsigned int nr_queued = cfs_rq->h_nr_queued; @@ -6199,7 +6199,7 @@ place_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int flags) */ if (sched_feat(PLACE_LAG) && nr_queued && se->vlag) { struct sched_entity *curr = cfs_rq->curr; - long load, weight; + long load, weight, curr_weight; lag = se->vlag; @@ -6256,10 +6256,17 @@ place_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int flags) * vl_i = (W + w_i)*vl'_i / W */ load = cfs_rq->sum_weight; - if (curr && curr->on_rq) - load += avg_vruntime_weight(cfs_rq, curr->h_load.weight); + if (curr) { + curr_weight = avg_vruntime_weight(cfs_rq, curr->h_load.weight); + if (curr->on_rq) + load += curr_weight; + } + + if (is_curr) + weight = curr_weight; + else + weight = avg_vruntime_weight(cfs_rq, se->h_load.weight); - weight = avg_vruntime_weight(cfs_rq, se->h_load.weight); lag *= load + weight; if (WARN_ON_ONCE(!load)) load = 1; @@ -7900,7 +7907,7 @@ static int choose_idle_cpu(int cpu, struct task_struct *p) } static void -requeue_delayed_entity(struct cfs_rq *cfs_rq, struct sched_entity *se) +requeue_delayed_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, bool curr) { /* * se->sched_delayed should imply: se->on_rq == 1. @@ -7912,10 +7919,10 @@ requeue_delayed_entity(struct cfs_rq *cfs_rq, struct sched_entity *se) if (update_entity_lag(cfs_rq, se)) { cfs_rq->h_nr_queued--; - if (se != cfs_rq->curr) + if (!curr) __dequeue_entity(cfs_rq, se); - place_entity(cfs_rq, se, 0); - if (se != cfs_rq->curr) + place_entity(cfs_rq, se, 0, curr); + if (!curr) __enqueue_entity(cfs_rq, se); cfs_rq->h_nr_queued++; } @@ -8000,9 +8007,10 @@ enqueue_task_fair(struct rq *rq, struct task_struct *p, int flags) util_est_enqueue(cfs_rq, p); update_curr_eevdf(cfs_rq); + curr = (cfs_rq->curr == se); if (delayed) { - requeue_delayed_entity(cfs_rq, se); + requeue_delayed_entity(cfs_rq, se, curr); return; } @@ -8017,18 +8025,18 @@ enqueue_task_fair(struct rq *rq, struct task_struct *p, int flags) /* * XXX comment on the curr thing */ - curr = (cfs_rq->curr == se); + if (curr) - place_entity(cfs_rq, se, flags); + place_entity(cfs_rq, se, flags, curr); if (se->on_rq && se->sched_delayed) - requeue_delayed_entity(cfs_rq, se); + requeue_delayed_entity(cfs_rq, se, curr); weight = enqueue_hierarchy(p, flags); if (!curr) { reweight_eevdf(cfs_rq, se, weight, false); - place_entity(cfs_rq, se, flags | ENQUEUE_QUEUED); + place_entity(cfs_rq, se, flags | ENQUEUE_QUEUED, curr); __enqueue_entity(cfs_rq, se); } -- 2.53.0