Re: [PATCH sched_ext/for-7.1] sched_ext: Documentation: Add missing calls to quiescent(), runnable()

Kuba Piecuch
2026-04-09T08:46:05Z

On Wed Apr 8, 2026 at 2:54 PM UTC, Andrea Righi wrote:
...
>>
>> Another inaccuracy not related to direct dispatch: property changes can occur
>> while a task is running, while the psedocode only allows for property changes
>> while a task is queued.
>
> Sure... but again, modelling all the possible scenarios would make the
> pseudocode completely unreadable.

I'm not arguing we should cover all scenarios.

I'm ok with omitting scenarios whose existence depends on a configuration flag
or presence/absence of a callback, because:

a) Using the right configuration, one can actually write a scheduler where the
pseudocode is an accurate representation of the task lifecycle;

b) The assumptions about the configuration can be clearly stated next to the
pseudocode.

I'm less ok with omitting specific scenarios that can't be simply "turned off"
because they are triggered by the scheduled tasks themselves. A task's property
being changed while it's running is one example of such a scenario -- one can't
just prevent it from happening by setting a configuration flag, and sched_ext
schedulers implementing dequeue/quiescent/runnable/enqueue should be aware of
it.

What I especially don't like is giving the reader a partial picture that looks
like a complete one, as is the case with property changes here. We're letting
the reader know that it can happen, but the pseudocode makes it look like it
can only happen while a task is queued and not while it's running, giving the
reader a false impression that they can assume property changes apply only to
queued tasks.

>
> IMHO it'd be better to give an overview of the most common use cases here and
> clarify in the description that the diagram doesn't cover all the possible
> scenarios. This one is a special use case that, personally, I wouldn't cover in
> the pseudocode.
>
>>
>> There's also preemption by a higher sched class, which is not covered in the
>> loop condition (task_is_runnable(task) && task->scx.slice > 0), unless we take
>> task_is_runnable() to return false if there's a higher-priority sched class
>> with runnable tasks on the CPU, though that would be in conflict with the
>> actual implementation of task_is_runnable() in include/linux/sched.h.
>
> Ditto.
>
>>
>> >
>> >>
>> >> A more general comment about the pseudocode: I think it can be useful to
>> >> introduce someone new to the general flow of the callbacks in sched_ext,
>> >> but the documentation should be clear that this is a simplified view that
>> >> makes assumptions about the behavior of the BPF scheduler itself (flags like
>> >> SCX_OPS_ENQ_LAST, whether the scheduler uses direct dispatch), as well as
>> >> the overall system (Can sched_ext be preempted by a higher-priority sched
>> >> class? Can scheduling properties of a task be changed while it's running?)
>> >> Without stating these assumptions clearly, we risk leaving the reader falsely
>> >> believing they have a complete understanding.
>> >
>> > Of course this schema is not a complete representation of the entire sched_ext
>> > state machine, if we put everything it'd become too big and complex. I think we
>> > should just cover the most common use cases here. Maybe we can clarify this in
>> > the description before this diagram.
>>
>> Let's agree on what inaccuracies need to be fixed and I'll send a v2 with fixes
>> and attach an appropriate disclaimer to the pseudocode.
>
> If we move ops.dispatch() + ops.dequeue() inside the ops.enqueue() block I think
> the pseudocode becomes "fairly" accurate. At least more accurate than what we
> have right now. It won't be perfect, but it can help newer sched_ext devs having
> an overview the task lifecycle without going too much into implementation
> details.
>
> So, to recap, what do you think about this?
>
> ops.init_task(); /* A new task is created */
> ops.enable(); /* Enable BPF scheduling for the task */
>
> while (task in SCHED_EXT) {
> if (task can migrate)
> ops.select_cpu(); /* Called on wakeup (optimization) */
>
> ops.runnable(); /* Task becomes ready to run */
>
> while (task_is_runnable(task)) {
> if (task is not in a DSQ || task->scx.slice == 0) {
> ops.enqueue(); /* Task can be added to a DSQ */
>
> /* Task property change (i.e., affinity, nice, etc.)? */
> if (sched_change(task)) {
> ops.dequeue(); /* Exiting BPF scheduler custody */
> ops.quiescent();
>
> /* Property change callback, e.g. ops.set_weight() */
>
> ops.runnable();
> continue;
> }
>
> /* Any usable CPU becomes available */
>
> ops.dispatch(); /* Task is moved to a local DSQ */
> ops.dequeue(); /* Exiting BPF scheduler custody */
> }
>
> ops.running(); /* Task starts running on its assigned CPU */
>
> while (task_is_runnable(task) && task->scx.slice > 0) {
> ops.tick(); /* Called every 1/HZ seconds */
>
> if (task->scx.slice == 0)
> ops.dispatch(); /* task->scx.slice can be refilled */
> }
>
> ops.stopping(); /* Task stops running (time slice expires or wait) */
> }
>
> ops.quiescent(); /* Task releases its assigned CPU (wait) */
> }
>
> ops.disable(); /* Disable BPF scheduling for the task */
> ops.exit_task(); /* Task is destroyed */

I don't love it (and I probably never will), but I agree it's the best so far.
I'll send a v2 with the updated pseudocode and I'll put a bit of a disclaimer
before it.

Thanks,
Kuba