Re: [PATCH v3 3/3] sched_ext: Allow scx_bpf_reenqueue_local() to be called from anywhere

[Andrea Righi <arighi@nvidia.com>]

On Thu, Dec 11, 2025 at 06:20:22AM -1000, Tejun Heo wrote:
> On Thu, Dec 11, 2025 at 06:17:03AM -1000, Tejun Heo wrote:
> > Hello, Kuba.
> > 
> > On Thu, Dec 11, 2025 at 02:24:04PM +0000, Kuba Piecuch wrote:
> > > Since we're doing a direct dispatch, the user scheduler task will be
> > > inserted into the dispatch queue in enable_task_scx(), without dropping the rq
> > > lock between example_enqueue() and the insertion, which means reenq_local()
> > > will run afterwards (since it's deferred using irq_work), removing all tasks
> > > from the DSQ, including the userspace scheduler.
> > > 
> > > A similar problem arises even if we don't do direct dispatch and drop the rq
> > > lock after example_enqueue(): since dispatching and reenq_local() are deferred
> > > using different irq_work entries, and irq_work_run() processes entries from
> > > newest to oldest, dispatching will be handled before reenq_local(), yielding
> > > the same result.
> > 
> > Oh yeah, the asynchronity can become pretty confusing.
> > 
> > > The user may be unaware of this behavior (it's not mentioned anywhere) and
> > > expect the reenqueue to happen before dispatching the new task.
> > > 
> > > I think at the very least we should make users aware of this in the comment
> > > for scx_bpf_reenqueue_local___v2().
> > 
> > Documentation is always helpful but I wonder whether this can be improved by
> > the reenqueue function capturing the dsq seq number and re-enqueueing only
> > the ones that were enqueued before the reenqueue was called.
> 
> That doesn't fix the ordering problem in the other direction, but the
> reordering in the other direction seems inherently less useful at least.
> Maybe this can also be solved with seq - ie. make enqueue record the current
> seq, mark the DSQ with the latest reenqueue seq and at enqueue commit time
> if the captured seq is alreaddy reenqueued, trigger reenqueue.

How about making it even more explicit renaming the kfunc to something like
scx_bpf_async_reenqueue_local() (and documenting it)?

Thanks,
-Andrea