From: Juri Lelli <juri.lelli@redhat.com>
To: Tejun Heo <tj@kernel.org>
Cc: Yuri Andriaccio <yurand2000@gmail.com>,
bsegall@google.com, cgroups@vger.kernel.org,
dietmar.eggemann@arm.com, hannes@cmpxchg.org,
kprateek.nayak@amd.com, linux-kernel@vger.kernel.org,
luca.abeni@santannapisa.it, mgorman@suse.de, mingo@redhat.com,
mkoutny@suse.com, peterz@infradead.org, rostedt@goodmis.org,
vincent.guittot@linaro.org, vschneid@redhat.com,
yuri.andriaccio@santannapisa.it
Subject: Re: [RFC PATCH v7 00/29] Hierarchical Constant Bandwidth Server
Date: Thu, 19 Nov 2026 12:23:48 +0100 [thread overview]
Message-ID: <av7cRIyAMoiIR5k-@jlelli-thinkpadt14gen4.remote.csb> (raw)
In-Reply-To: <58cb2cbaf82c0f671213eaca46465cd5@kernel.org>
Hello,
On 17/11/26 22:07, Tejun Heo wrote:
> Hello, Yuri.
>
> The summary sounds fine for the most part. Some comments below.
Sorry to interject again, but I had a thought that I would like to
share. Also, the very similar names probably don't help keeping track of
who said what, but anyway. :P
> On Wed, Nov 11, 2026 at 06:56:57PM +0100, Yuri Andriaccio wrote:
> > - `cpu.rt.max` only accepts per-CPU settings, so the global "default" is
> > removed. Removing all overrides means the cgroup is a "max" cgroup, while
> > just a single override means that the cgroup has its own budget.
> > + This new interface disallows zero-runtime overrides, as a zero override
> > is equal to an empty override when there is at least one non-zero override.
> > As an example: [<CPU0 10ms 100ms>, <CPU1 0ms 100ms>] is equal to just
> > having [<CPU0 10ms 100ms>]. This seems a bit odd, how to make this better?
>
> It's odd because a missing entry means "max" when there are no entries
> and zero when there are some. Can a missing entry always mean "max",
> per CPU? On a CPU that a cgroup has no entry for, its tasks would be
> served by the nearest ancestor which has one. Zero then becomes a
> regular value which is shown on read, and the two configs in your
> example are different. The former can't run RT tasks on CPU1. The
> latter runs them on the ancestor's budget there.
>
> This means that a cgroup can have its own budget on some CPUs and be
> "max" on the others. That is a fundamental part of the interface, so I
> think it's best to get it right from the get-go rather than starting
> with a restriction.
>
Agree on making zero a regular value and on a missing entry always
meaning "max". There is one case where "the ancestor's budget" doesn't
exist though, and that is when the walk ends at the root.
The root doesn't have a server. Its cpu.rt.max is the pool of
SCHED_DEADLINE bandwidth that child servers are carved from, and
FIFO/RR tasks served by the root just run in the root runqueue, outside
HCBS (as with !RT_GROUP_SCHED). So take a top level group with
0 runtime=10000 period=100000
no entry for CPU1, and a few while(1) FIFO tasks allowed on CPUs 0-1.
On CPU0 they are throttled by the group's server. Once they get to
CPU1 (wakeup placement, push/pull, affinity changes) they run in the
root runqueue and nothing throttles them, only DL tasks and the
fair/ext servers. The group reads as 10% of CPU0, root reads as
runtime=100000 everywhere, and the admin reasonably thinks the group is
contained. It isn't.
Making a missing entry mean zero at the top would bring back the
ambiguity you pointed out, so what about keeping your semantics and
adding a validity rule instead:
A cgroup can be served by the root either on all of its CPUs or on
none of them.
In practice, a cgroup whose parent is entirely root served (e.g. a
top level group) has entries for all possible CPUs or for none. A
write that would leave some CPUs served by a server (or zero) and
others by the root fails with -EINVAL, like the existing admission
checks. Further down, a missing entry is fine and means the parent's
context on that CPU, which is always a real server or an explicit
zero.
With that, per CPU:
- no entry: served by the parent's context on that CPU
- runtime=0: no RT on that CPU, shown on read
- runtime=N period=P: own server on that CPU
A missing entry never changes meaning; the rule only rejects the
configurations where it would escape HCBS. For the example above, the
group must say "1 runtime=0 ..." (no RT on CPU1) or give CPU1 a
budget. The same config under a parent with servers on CPUs 0-1 is
accepted, and CPU1 runs on the parent's server.
What do you think?
> The root has no ancestor to fall back to, so a missing entry can't mean
> anything there. It has to always carry an entry for every possible CPU,
> which defaults to zero, and reject "max":
>
> 0-127 runtime=0 period=1000000
Also, related to the above: on the root these numbers mean something
else (how much DL bandwidth cgroups may reserve, not a server for the
root's tasks), and showing them in the same file and format is part of
what makes the example misleading. Would you be OK with dropping
cpu.rt.max on the root and exposing the parameters through a separately
named root-only file (or the existing sysctls)? That way, having no
entry all the way up consistently means unmanaged FIFO/RR, and nothing
on the root looks like a budget those tasks share.
Thanks!
Juri
next prev parent reply other threads:[~2026-11-19 11:23 UTC|newest]
Thread overview: 37+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-09 16:35 [RFC PATCH v7 00/29] Hierarchical Constant Bandwidth Server Yuri Andriaccio
2026-11-09 16:35 ` [RFC PATCH v7 01/29] sched/deadline: Introduce dl_freq_invariant flag for fair/ext servers Yuri Andriaccio
2026-11-09 16:35 ` [RFC PATCH v7 02/29] sched/deadline: Do not access dl_se->rq directly Yuri Andriaccio
2026-11-09 16:35 ` [RFC PATCH v7 03/29] sched/deadline: Distinguish between dl_rq and my_q Yuri Andriaccio
2026-11-09 16:35 ` [RFC PATCH v7 04/29] sched/rt: Pass an rt_rq instead of an rq where needed Yuri Andriaccio
2026-11-09 16:35 ` [RFC PATCH v7 05/29] sched/rt: Move functions from rt.c to sched.h Yuri Andriaccio
2026-11-09 16:35 ` [RFC PATCH v7 06/29] sched/rt: Disable RT_GROUP_SCHED Yuri Andriaccio
2026-11-09 16:35 ` [RFC PATCH v7 07/29] sched/rt: Remove unnecessary runqueue pointer in struct rt_rq Yuri Andriaccio
2026-11-09 16:35 ` [RFC PATCH v7 08/29] sched/rt: Introduce HCBS specific structs in task_group Yuri Andriaccio
2026-11-09 16:35 ` [RFC PATCH v7 09/29] sched/core: Initialize HCBS specific structures Yuri Andriaccio
2026-11-09 16:35 ` [RFC PATCH v7 10/29] sched/deadline: Add dl_init_tg Yuri Andriaccio
2026-11-09 16:35 ` [RFC PATCH v7 11/29] sched/rt: Add {alloc/unregister/free}_rt_sched_group Yuri Andriaccio
2026-11-09 16:35 ` [RFC PATCH v7 12/29] sched/deadline: Account rt-cgroups bandwidth in deadline tasks schedulability tests Yuri Andriaccio
2026-11-09 16:35 ` [RFC PATCH v7 13/29] sched/rt: Implement dl-server operations for rt-cgroups Yuri Andriaccio
2026-11-09 16:36 ` [RFC PATCH v7 14/29] sched/rt: Update task event callbacks for HCBS scheduling Yuri Andriaccio
2026-11-09 16:36 ` [RFC PATCH v7 15/29] sched/rt: Remove support for cgroups-v1 Yuri Andriaccio
2026-11-09 16:36 ` [RFC PATCH v7 16/29] sched/rt: Update rt-cgroup schedulability checks Yuri Andriaccio
2026-11-09 16:36 ` [RFC PATCH v7 17/29] sched/rt: Update task's RT runqueue when switching scheduling class Yuri Andriaccio
2026-11-09 16:36 ` [RFC PATCH v7 18/29] sched/rt: Remove old RT_GROUP_SCHED data structures Yuri Andriaccio
2026-11-09 16:36 ` [RFC PATCH v7 19/29] sched/rt: Add HCBS migration code to related functions Yuri Andriaccio
2026-11-09 16:36 ` [RFC PATCH v7 20/29] sched/rt: Hook HCBS migration functions Yuri Andriaccio
2026-11-09 16:36 ` [RFC PATCH v7 21/29] sched/core: Execute enqueued balance callbacks when changing allowed CPUs Yuri Andriaccio
2026-11-09 16:36 ` [RFC PATCH v7 22/29] sched/rt: Try pull task on empty server pick Yuri Andriaccio
2026-11-09 16:36 ` [RFC PATCH v7 23/29] sched/core: Execute enqueued balance callbacks after migrate_disable_switch Yuri Andriaccio
2026-11-09 16:36 ` [RFC PATCH v7 24/29] sched/syscalls: Allow unpriviledged tasks to set RT priority inside cgroups Yuri Andriaccio
2026-11-09 16:36 ` [RFC PATCH v7 25/29] sched/rt: Enable HRTICK_DL for HCBS servers Yuri Andriaccio
2026-11-09 16:36 ` [RFC PATCH v7 26/29] Documentation: Update documentation for real-time cgroups Yuri Andriaccio
2026-11-09 16:36 ` [RFC PATCH v7 27/29] sched/rt: Integrate cpuset and cpu controllers Yuri Andriaccio
2026-11-09 16:36 ` [RFC PATCH v7 28/29] sched/rt: Prevent CPU offline operation on active rt-cgroups Yuri Andriaccio
2026-11-09 16:36 ` [RFC PATCH v7 29/29] sched/rt: Introduce multi-cpu support Yuri Andriaccio
2026-11-10 21:43 ` [RFC PATCH v7 00/29] Hierarchical Constant Bandwidth Server Tejun Heo
2026-11-11 17:56 ` Yuri Andriaccio
2026-11-17 15:49 ` Juri Lelli
2026-11-18 8:07 ` Tejun Heo
2026-11-19 11:23 ` Juri Lelli [this message]
2026-11-16 14:25 ` Juri Lelli
2026-11-17 13:00 ` Yuri Andriaccio
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=av7cRIyAMoiIR5k-@jlelli-thinkpadt14gen4.remote.csb \
--to=juri.lelli@redhat.com \
--cc=bsegall@google.com \
--cc=cgroups@vger.kernel.org \
--cc=dietmar.eggemann@arm.com \
--cc=hannes@cmpxchg.org \
--cc=kprateek.nayak@amd.com \
--cc=linux-kernel@vger.kernel.org \
--cc=luca.abeni@santannapisa.it \
--cc=mgorman@suse.de \
--cc=mingo@redhat.com \
--cc=mkoutny@suse.com \
--cc=peterz@infradead.org \
--cc=rostedt@goodmis.org \
--cc=tj@kernel.org \
--cc=vincent.guittot@linaro.org \
--cc=vschneid@redhat.com \
--cc=yurand2000@gmail.com \
--cc=yuri.andriaccio@santannapisa.it \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox