From: Juri Lelli <juri.lelli@redhat.com>
To: luca abeni <luca.abeni@santannapisa.it>
Cc: Tejun Heo <tj@kernel.org>, Yuri Andriaccio <yurand2000@gmail.com>,
bsegall@google.com, cgroups@vger.kernel.org,
dietmar.eggemann@arm.com, hannes@cmpxchg.org,
kprateek.nayak@amd.com, linux-kernel@vger.kernel.org,
mgorman@suse.de, mingo@redhat.com, mkoutny@suse.com,
peterz@infradead.org, rostedt@goodmis.org,
vincent.guittot@linaro.org, vschneid@redhat.com,
yuri.andriaccio@santannapisa.it
Subject: Re: [RFC PATCH v7 00/29] Hierarchical Constant Bandwidth Server
Date: Thu, 19 Nov 2026 15:36:01 +0100 [thread overview]
Message-ID: <av8JUXtjhd3PoWL7@jlelli-thinkpadt14gen4.remote.csb> (raw)
In-Reply-To: <20261119152744.3a2f7aed@linux.fritz.box>
On 19/11/26 15:27, luca abeni wrote:
> Hi Juri,
>
> On Thu, 19 Nov 2026 12:23:48 +0100
> Juri Lelli <juri.lelli@redhat.com> wrote:
> [...]
> > > On Wed, Nov 11, 2026 at 06:56:57PM +0100, Yuri Andriaccio wrote:
> > > > - `cpu.rt.max` only accepts per-CPU settings, so the global
> > > > "default" is removed. Removing all overrides means the cgroup is
> > > > a "max" cgroup, while just a single override means that the
> > > > cgroup has its own budget.
> > > > + This new interface disallows zero-runtime overrides, as a
> > > > zero override is equal to an empty override when there is at
> > > > least one non-zero override. As an example: [<CPU0 10ms 100ms>,
> > > > <CPU1 0ms 100ms>] is equal to just having [<CPU0 10ms 100ms>].
> > > > This seems a bit odd, how to make this better?
> > >
> > > It's odd because a missing entry means "max" when there are no
> > > entries and zero when there are some. Can a missing entry always
> > > mean "max", per CPU? On a CPU that a cgroup has no entry for, its
> > > tasks would be served by the nearest ancestor which has one. Zero
> > > then becomes a regular value which is shown on read, and the two
> > > configs in your example are different. The former can't run RT
> > > tasks on CPU1. The latter runs them on the ancestor's budget there.
> > >
> > > This means that a cgroup can have its own budget on some CPUs and be
> > > "max" on the others. That is a fundamental part of the interface,
> > > so I think it's best to get it right from the get-go rather than
> > > starting with a restriction.
> > >
> >
> > Agree on making zero a regular value and on a missing entry always
> > meaning "max".
>
> I think "0" as a regular value is agreed upon by everyone :)
> But I would avoid "missing entries meaning max"... See my previous
> email.
> Basically, I think a cgroup should either inherit from the parent for
> every core, or define a deadline entity (even one with 0 runtime) on
> every core. Mixing the two is, in my opinion, problematic (for example,
> it would change the serving reservation when a task migrates from one
> core to another)).
> I hope this "all-or-nothing" approach is acceptable...
>
>
> [...]
> > Making a missing entry mean zero at the top would bring back the
> > ambiguity you pointed out, so what about keeping your semantics and
> > adding a validity rule instead:
> >
> > A cgroup can be served by the root either on all of its CPUs or on
> > none of them.
>
> As mentioned above, I would be even more drastic:
> A cgroup can be served by an ancestor either on all of its CPUs or on
> none of them.
>
> What do you think? Can this approach be acceptable?
Yes, works for me. The example I gave is just the extreme case of the
problem you describe (the "other reservation" being no reservation at
all), and your argument covers the nested case I still allowed.
Tejun, I think this still addresses the original concern: zero is a
regular value with a single meaning ("no RT on this CPU"), and no
entry only exists for a cgroup that is "max" as a whole. It's not a
restriction to relax later, though: a task changing reservation on
migration is what we want to rule out.
> > In practice, a cgroup whose parent is entirely root served (e.g. a
> > top level group) has entries for all possible CPUs or for none. A
> > write that would leave some CPUs served by a server (or zero) and
> > others by the root fails with -EINVAL, like the existing admission
> > checks. Further down, a missing entry is fine and means the parent's
> > context on that CPU, which is always a real server or an explicit
> > zero.
> >
> > With that, per CPU:
> >
> > - no entry: served by the parent's context on that CPU
> > - runtime=0: no RT on that CPU, shown on read
> > - runtime=N period=P: own server on that CPU
>
> If what I wrote above is OK, I would just avoid the "no entry" case,
> and I would define the interface so that missing entries are not
> possible (for example, see my previous email: we first set a (runtime,
> period) for all core, then we can override some of them... But not with
> "max").
OK.
> > > The root has no ancestor to fall back to, so a missing entry can't
> > > mean anything there. It has to always carry an entry for every
> > > possible CPU, which defaults to zero, and reject "max":
> > >
> > > 0-127 runtime=0 period=1000000
> >
> > Also, related to the above: on the root these numbers mean something
> > else (how much DL bandwidth cgroups may reserve, not a server for the
> > root's tasks), and showing them in the same file and format is part of
> > what makes the example misleading. Would you be OK with dropping
> > cpu.rt.max on the root and exposing the parameters through a
> > separately named root-only file (or the existing sysctls)? That way,
> > having no entry all the way up consistently means unmanaged FIFO/RR,
> > and nothing on the root looks like a budget those tasks share.
>
> Regardless of the filenames we decide, the defaults for these settings
> indicate the fraction of CPU time that can be consumed by all real-time
> tasks that are not in the root cgroup, right?
Not quite, I think? They cap the bandwidth that can be reserved for
cgroup servers, so they only cover RT tasks that are served by one.
RT tasks in a non-root "max" cgroup (with no configured ancestor) are
not in the root cgroup, but run in the root runqueue and aren't limited
by those values at all (sched_rt_can_attach() lets them in
unconditionally). Which is the kind of confusion I'd like the root
interface to avoid.
Thanks,
Juri
next prev parent reply other threads:[~2026-11-19 14:36 UTC|newest]
Thread overview: 45+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-09 16:35 [RFC PATCH v7 00/29] Hierarchical Constant Bandwidth Server Yuri Andriaccio
2026-11-09 16:35 ` [RFC PATCH v7 01/29] sched/deadline: Introduce dl_freq_invariant flag for fair/ext servers Yuri Andriaccio
2026-11-09 16:35 ` [RFC PATCH v7 02/29] sched/deadline: Do not access dl_se->rq directly Yuri Andriaccio
2026-11-09 16:35 ` [RFC PATCH v7 03/29] sched/deadline: Distinguish between dl_rq and my_q Yuri Andriaccio
2026-11-09 16:35 ` [RFC PATCH v7 04/29] sched/rt: Pass an rt_rq instead of an rq where needed Yuri Andriaccio
2026-11-09 16:35 ` [RFC PATCH v7 05/29] sched/rt: Move functions from rt.c to sched.h Yuri Andriaccio
2026-11-09 16:35 ` [RFC PATCH v7 06/29] sched/rt: Disable RT_GROUP_SCHED Yuri Andriaccio
2026-11-09 16:35 ` [RFC PATCH v7 07/29] sched/rt: Remove unnecessary runqueue pointer in struct rt_rq Yuri Andriaccio
2026-11-09 16:35 ` [RFC PATCH v7 08/29] sched/rt: Introduce HCBS specific structs in task_group Yuri Andriaccio
2026-11-09 16:35 ` [RFC PATCH v7 09/29] sched/core: Initialize HCBS specific structures Yuri Andriaccio
2026-11-09 16:35 ` [RFC PATCH v7 10/29] sched/deadline: Add dl_init_tg Yuri Andriaccio
2026-11-09 16:35 ` [RFC PATCH v7 11/29] sched/rt: Add {alloc/unregister/free}_rt_sched_group Yuri Andriaccio
2026-11-09 16:35 ` [RFC PATCH v7 12/29] sched/deadline: Account rt-cgroups bandwidth in deadline tasks schedulability tests Yuri Andriaccio
2026-11-09 16:35 ` [RFC PATCH v7 13/29] sched/rt: Implement dl-server operations for rt-cgroups Yuri Andriaccio
2026-11-09 16:36 ` [RFC PATCH v7 14/29] sched/rt: Update task event callbacks for HCBS scheduling Yuri Andriaccio
2026-11-09 16:36 ` [RFC PATCH v7 15/29] sched/rt: Remove support for cgroups-v1 Yuri Andriaccio
2026-11-09 16:36 ` [RFC PATCH v7 16/29] sched/rt: Update rt-cgroup schedulability checks Yuri Andriaccio
2026-11-09 16:36 ` [RFC PATCH v7 17/29] sched/rt: Update task's RT runqueue when switching scheduling class Yuri Andriaccio
2026-11-09 16:36 ` [RFC PATCH v7 18/29] sched/rt: Remove old RT_GROUP_SCHED data structures Yuri Andriaccio
2026-11-09 16:36 ` [RFC PATCH v7 19/29] sched/rt: Add HCBS migration code to related functions Yuri Andriaccio
2026-11-09 16:36 ` [RFC PATCH v7 20/29] sched/rt: Hook HCBS migration functions Yuri Andriaccio
2026-11-09 16:36 ` [RFC PATCH v7 21/29] sched/core: Execute enqueued balance callbacks when changing allowed CPUs Yuri Andriaccio
2026-11-09 16:36 ` [RFC PATCH v7 22/29] sched/rt: Try pull task on empty server pick Yuri Andriaccio
2026-11-09 16:36 ` [RFC PATCH v7 23/29] sched/core: Execute enqueued balance callbacks after migrate_disable_switch Yuri Andriaccio
2026-11-09 16:36 ` [RFC PATCH v7 24/29] sched/syscalls: Allow unpriviledged tasks to set RT priority inside cgroups Yuri Andriaccio
2026-11-09 16:36 ` [RFC PATCH v7 25/29] sched/rt: Enable HRTICK_DL for HCBS servers Yuri Andriaccio
2026-11-09 16:36 ` [RFC PATCH v7 26/29] Documentation: Update documentation for real-time cgroups Yuri Andriaccio
2026-11-09 16:36 ` [RFC PATCH v7 27/29] sched/rt: Integrate cpuset and cpu controllers Yuri Andriaccio
2026-11-09 16:36 ` [RFC PATCH v7 28/29] sched/rt: Prevent CPU offline operation on active rt-cgroups Yuri Andriaccio
2026-11-09 16:36 ` [RFC PATCH v7 29/29] sched/rt: Introduce multi-cpu support Yuri Andriaccio
2026-11-10 21:43 ` [RFC PATCH v7 00/29] Hierarchical Constant Bandwidth Server Tejun Heo
2026-11-11 17:56 ` Yuri Andriaccio
2026-11-17 15:49 ` Juri Lelli
2026-11-18 8:07 ` Tejun Heo
2026-11-19 11:23 ` Juri Lelli
2026-11-19 14:27 ` luca abeni
2026-11-19 14:36 ` Juri Lelli [this message]
2026-11-19 14:42 ` luca abeni
2026-11-19 14:14 ` luca abeni
2026-11-20 6:46 ` Tejun Heo
2026-11-20 11:06 ` luca abeni
2026-11-20 11:55 ` Tejun Heo
2026-11-20 20:24 ` luca abeni
2026-11-16 14:25 ` Juri Lelli
2026-11-17 13:00 ` Yuri Andriaccio
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=av8JUXtjhd3PoWL7@jlelli-thinkpadt14gen4.remote.csb \
--to=juri.lelli@redhat.com \
--cc=bsegall@google.com \
--cc=cgroups@vger.kernel.org \
--cc=dietmar.eggemann@arm.com \
--cc=hannes@cmpxchg.org \
--cc=kprateek.nayak@amd.com \
--cc=linux-kernel@vger.kernel.org \
--cc=luca.abeni@santannapisa.it \
--cc=mgorman@suse.de \
--cc=mingo@redhat.com \
--cc=mkoutny@suse.com \
--cc=peterz@infradead.org \
--cc=rostedt@goodmis.org \
--cc=tj@kernel.org \
--cc=vincent.guittot@linaro.org \
--cc=vschneid@redhat.com \
--cc=yurand2000@gmail.com \
--cc=yuri.andriaccio@santannapisa.it \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.