Re: [PATCH 1/3] sched: add sched_task_call()

Josh Poimboeuf
2015-02-16T22:05:36Z

On Mon, Feb 16, 2015 at 09:44:36PM +0100, Peter Zijlstra wrote:
> On Mon, Feb 16, 2015 at 12:52:34PM -0600, Josh Poimboeuf wrote:
> > +++ b/kernel/sched/core.c
> > @@ -1338,6 +1338,23 @@ void kick_process(struct task_struct *p)
> > EXPORT_SYMBOL_GPL(kick_process);
> > #endif /* CONFIG_SMP */
> >
> > +/***
> > + * sched_task_call - call a function with a task's state locked
> > + *
> > + * The task is guaranteed to remain either active or inactive during the
> > + * function call.
> > + */
> > +void sched_task_call(sched_task_call_func_t func, struct task_struct *p,
> > + void *data)
> > +{
> > + unsigned long flags;
> > + struct rq *rq;
> > +
> > + rq = task_rq_lock(p, &flags);
> > + func(p, data);
> > + task_rq_unlock(rq, p, &flags);
> > +}
>
> Yeah, I think not. We're so not going to allow running random code under
> rq->lock and p->pi_lock.

Yeah, I can understand that. I definitely want to avoid touching the
scheduler code. Basically I'm trying to find a way to atomically do the
following:

if (task is sleeping) {
walk the stack
if (certain set of functions isn't on the stack)
set (or clear) a thread flag for the task
}

Any ideas on how I can achieve that? So far my ideas are:

1. Use task_rq_lock() -- but rq_lock is internal to sched code.

2. Use wait_task_inactive() -- I could call it twice, with the stack
checking in between, and use ncsw to ensure that it didn't reschedule
in the mean time. But this still seems racy. i.e., I think the task
could start running after the second call to wait_task_inactive()
returns but before setting the thread flag. Not sure if that's a
realistic race condition or not.

3. Use set_cpus_allowed() to temporarily pin the task to its current
CPU, and then call smp_call_function_single() to run the above
critical section on that CPU. I'm not sure if there's a race-free
way to do it but it's a lot more disruptive than I'd like...

Any ideas or guidance would be greatly appreciated!

--
Josh