Re: [REPORT] sched_ext: sub-scheduler tasks are severely under-scheduled vs root-owned tasks

[Tejun Heo <tj@kernel.org>]

Hello, Tao.

The following is a Claude-generated analysis.

On Fri, Oct 09, 2026 at 09:51:35AM +0800, Tao Cui wrote:
> The scheduler is a single binary (scx_k2) with parent/root and
> child/sub modes. The parent grants SCX_CAP_PERF | SCX_CAP_ENQ_IMMED
> to the child over a cgroup, while the child only writes cpuperf
> target 1 from ops.tick(). Both schedulers print their counters every
> 2s.

Reproduced on for-7.4 with your sources in a 4 cpu VM. The starvation
comes from the two schedulers, not the kernel.

The child's ops.enqueue() inserts every task into its own user DSQ. The
only consumer of that DSQ is the child's ops.dispatch(), and a child's
dispatch runs only when its parent calls scx_bpf_sub_dispatch() from its
own ops.dispatch(). scx_k2's parent never does. See the comment above
enum scx_cap_flags in kernel/sched/ext/internal.h and how scx_qmap
dispatches its children. The SysRq-D dump during the stall shows the
spinner in the child's DSQ 0 with a full slice while cpu0 runs root
tasks.

> In an A/B test, having the parent kick the task's CPU from
> ops.enqueue() with scx_bpf_kick_cid() and hand its dispatch turn to
> the child with scx_bpf_sub_dispatch() prevents the starvation.

scx_bpf_sub_dispatch() is only callable from ops.dispatch(). With the
parent calling it there, the child's dispatch does run, but every
scx_bpf_dsq_move_to_local() is rejected because the grant lacks
SCX_CAP_ENQ. SCX_CAP_ENQ_IMMED only covers IMMED inserts. The task goes
to the reject DSQ and is reenqueued, SCX_EV_REENQ_REPEAT climbs on the
child, and it still starves. With the sub dispatch call and SCX_CAP_ENQ
granted as well, the spinner runs at full duty under the child.

> Actual: the child claims the task, and its ops.tick() initially
> services it, but setcnt then stops increasing and the task receives
> little or no CPU time - 0-91 ticks per 6s window across our runs,
> most rounds 0.

The initial burst is the bypass DSQ drained during attach. In the
move-after-attach order the task keeps running through dispatch's
keep-last path until something else wakes on cpu0 and pushes it through
the child's ops.enqueue(), which is why some windows look healthy. The
stall watchdog catches it past your 6s window:

  sched_ext: BPF sub-scheduler "k2" disabled (runnable task stall)
  sched_ext: k2: sh[1401] failed to run for 30.696s

> One possibility is that, on a NO_HZ idle CPU, no balance is
> triggered for the child's DSQ, so the child's ops.dispatch() is
> never called.

The parent's dispatch on cid 0 ran several times a second throughout
the stall. It only consumed the parent's own DSQ.

Thanks.

-- 
tejun