Re: [LSF/MM/BPF TOPIC] Discuss more features + use cases for sched_ext

[Joel Fernandes <joel@joelfernandes.org>]

On 1/29/2024 8:50 PM, Tejun Heo wrote:> On Mon, Jan 29, 2024 at 05:42:54PM
-0500, Joel Fernandes wrote:
>>> This is a great topic. I think integrating/merging such mechanism with the NEST
>>> scheduler could be useful too? You mentioned there is sched_ext implementation
>>> of NEST already? One reason that's interesting to me is the task-packing and
>>> less-spreading may have power benefits, this is exactly what EAS on ARM does,
>>> but it also uses an energy model to know when packing is a bad idea. Since we
>>> don't have fine grained control of frequency on Intel, I wonder what else can we
>>> do to know when the scheduler should pack and when to spread. Maybe something
>>> simple which does not need an energy model but packs based on some other
>>> signal/heuristic would be great in the short term.
>>>
>>> Maybe a signal can be the "Quality of service (QoS)" approach where tasks with
>>> lower QoS are packed more aggressively and higher QoS are spread more (?).
> 
> This was done for a different purpose (improving tail latencies on latency
> critical workload) but it uses soft-affinity based packing which maybe can
> translate to power-aware scheduling:
> 
>   https://github.com/sched-ext/scx/blob/case-studies/case-studies/scx_layered.md

Thanks! I am looking more into this (scx_layered) for the latency benefits as
well. David kindly gave me an introduction to it last week. It seems quite
similar to our approach with using RT (round-robin) for the higher tier (that is
have a higher tier of tasks that are fair scheduled over a lower one). There is
the issue of starvation though (a higher tier/layer starves a lower one), so
we're incorporating the DL server to help with that:
https://lore.kernel.org/all/cover.1699095159.git.bristot@kernel.org/
https://lore.kernel.org/all/20240216183108.1564958-1-joel@joelfernandes.org/

Interesting on the soft-affinity feature and yeah that help save power and might
be a better approach than say our usage of RT.

> I have a raptor lake-H laptop which has E and P cores and by default the
> threads are being spread across all CPUs which probably isn't best for power
> consumption. I was thinking about writing a scheduler which uses a similar
> strategy as scx_layered - pack the cores one by one overflowing to the next
> core from E to P when the average utilization crosses a set threshold. Most
> of the logic is already in scx_layered, so maybe it can just be a part of
> that. I'm curious whether whether and how much power can be saved with a
> generic approach like that.

Can the scx NEST scheduler be reused for this? AFAIR, it does similar task
packing. Though that is to keep more cores idle than to pack tasks to a certain
type of core, if I remember Julia's presentation.

thanks,

 - Joel