All of lore.kernel.org
 help / color / mirror / Atom feed
From: Cristian Prundeanu <cpru@amazon.com>
To: <linux-tip-commits@vger.kernel.org>
Cc: <linux-kernel@vger.kernel.org>,
	Peter Zijlstra <peterz@infradead.org>,
	Ingo Molnar <mingo@redhat.com>, <x86@kernel.org>,
	<linux-arm-kernel@lists.infradead.org>,
	Bjoern Doebel <doebel@amazon.com>,
	Hazem Mohamed Abuelfotoh <abuehaze@amazon.com>,
	Geoff Blake <blakgeof@amazon.com>,
	Ali Saidi <alisaidi@amazon.com>, Csaba Csoma <csabac@amazon.com>,
	Cristian Prundeanu <cpru@amazon.com>
Subject: [PATCH 0/2] [tip: sched/core] sched: Disable PLACE_LAG and RUN_TO_PARITY and move them to sysctl
Date: Thu, 17 Oct 2024 00:19:58 -0500	[thread overview]
Message-ID: <20241017052000.99200-1-cpru@amazon.com> (raw)

This patchset disables the scheduler features PLACE_LAG and RUN_TO_PARITY 
and moves them to sysctl.

Replacing CFS with the EEVDF scheduler in kernel 6.6 introduced 
significant performance degradation in multiple database-oriented 
workloads. This degradation manifests in all kernel versions using EEVDF, 
across multiple Linux distributions, hardware architectures (x86_64, 
aarm64, amd64), and CPU generations.

For example, running mysql+hammerdb results in a 12-17% throughput 
reduction and 12-18% latency increase compared to kernel 6.5 (using 
default scheduler settings everywhere). The magnitude of this performance 
impact is comparable to the average performance difference of a CPU 
generation over its predecessor.

Testing combinations of available scheduler features showed that the 
largest improvement (short of disabling all EEVDF features) came from 
disabling both PLACE_LAG and RUN_TO_PARITY:

Kernel   | default  | NO_PLACE_LAG and
aarm64   | config   | NO_RUN_TO_PARITY
---------+----------+-----------------
6.5      | baseline |  N/A
6.6      | -13.2%   | -6.8%
6.7      | -13.1%   | -6.0%
6.8      | -12.3%   | -6.5%
6.9      | -12.7%   | -6.9%
6.10     | -13.5%   | -5.8%
6.11     | -12.6%   | -5.8%
6.12-rc2 | -12.2%   | -8.9%
---------+----------+-----------------

Kernel   | default  | NO_PLACE_LAG and
x86_64   | config   | NO_RUN_TO_PARITY
---------+----------+-----------------
6.5      | baseline |  N/A
6.6      | -16.8%   | -10.8%
6.7      | -16.4%   |  -9.9%
6.8      | -17.2%   |  -9.5%
6.9      | -17.4%   |  -9.7%
6.10     | -16.5%   |  -9.0%
6.11     | -15.0%   |  -8.5%
6.12-rc2 | -12.7%   | -10.9%
---------+----------+-----------------

While the long term approach is debugging and fixing the scheduler 
behavior, algorithm changes to address performance issues of this nature 
are specialized (and likely prolonged or open-ended) research. Until a 
change is identified which fixes the performance degradation, in the 
interest of a better out-of-the-box performance: (1) disable these 
features by default, and (2) expose these values in sysctl instead of 
debugfs, so they can be more easily persisted across reboots.

Cristian Prundeanu (2):
  sched: Disable PLACE_LAG and RUN_TO_PARITY
  sched: Move PLACE_LAG and RUN_TO_PARITY to sysctl

 include/linux/sched/sysctl.h |  8 ++++++++
 kernel/sched/core.c          | 13 +++++++++++++
 kernel/sched/fair.c          |  5 +++--
 kernel/sched/features.h      | 10 ----------
 kernel/sysctl.c              | 20 ++++++++++++++++++++
 5 files changed, 44 insertions(+), 12 deletions(-)

-- 
2.40.1



             reply	other threads:[~2024-10-17  5:22 UTC|newest]

Thread overview: 26+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-10-17  5:19 Cristian Prundeanu [this message]
2024-10-17  5:19 ` [PATCH 1/2] [tip: sched/core] sched: Disable PLACE_LAG and RUN_TO_PARITY Cristian Prundeanu
2024-10-17  5:20 ` [PATCH 2/2] [tip: sched/core] sched: Move PLACE_LAG and RUN_TO_PARITY to sysctl Cristian Prundeanu
2024-10-17  9:10 ` [PATCH 0/2] [tip: sched/core] sched: Disable PLACE_LAG and RUN_TO_PARITY and move them " Peter Zijlstra
2024-10-17 18:19   ` Prundeanu, Cristian
2024-10-18  7:07     ` K Prateek Nayak
2024-10-18  9:54     ` Mohamed Abuelfotoh, Hazem
2024-11-14 20:10 ` Joseph Salisbury
2024-11-19 10:29   ` Dietmar Eggemann
2024-11-25 11:35 ` Cristian Prundeanu
2024-11-26  3:58   ` K Prateek Nayak
2024-11-26 15:12   ` Dietmar Eggemann
2024-11-28 10:32   ` Cristian Prundeanu
2024-11-29 10:12     ` Dietmar Eggemann
  -- strict thread matches above, loose matches on Subject: below --
2024-10-19  2:30 Prundeanu, Cristian
2024-10-24  8:12 ` Benjamin Herrenschmidt
2024-10-25 14:43   ` Gautham R. Shenoy
2024-10-29  4:57     ` Cristian Prundeanu
2024-10-30 10:21       ` Dietmar Eggemann
2024-11-01 13:05       ` Peter Zijlstra
2024-11-04 10:19       ` Gautham R. Shenoy
2024-11-04 10:34         ` K Prateek Nayak
2025-01-19 11:04 [GIT PULL] sched/urgent for v6.13 Borislav Petkov
2025-01-28 23:09 [PATCH 0/2] [tip: sched/core] sched: Disable PLACE_LAG and RUN_TO_PARITY and move them to sysctl Cristian Prundeanu
2025-02-11  3:27 ` K Prateek Nayak
2025-02-12  5:41   ` Cristian Prundeanu
2025-02-12  9:43     ` Peter Zijlstra

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20241017052000.99200-1-cpru@amazon.com \
    --to=cpru@amazon.com \
    --cc=abuehaze@amazon.com \
    --cc=alisaidi@amazon.com \
    --cc=blakgeof@amazon.com \
    --cc=csabac@amazon.com \
    --cc=doebel@amazon.com \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-tip-commits@vger.kernel.org \
    --cc=mingo@redhat.com \
    --cc=peterz@infradead.org \
    --cc=x86@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.