cpusets and kthreads, inconsistent behaviour

[Max Krasnyansky <maxk@qualcomm.com>]

I pointed this out in the email thread about PF_THREAD_BOUND patch and wanted
to restart the thread to make sure that people pay attention :).
I was going to cook up a patch for this and wanted to get some early feedback
to avoid time waste.

Basically the issue is that current behaviour of the cpusets is inconsistent
with regards to kthreads. Kthreads inherit cpuset from a parent properly but
they simply ignore cpuset.cpus when their cpu affinity is set/updated.
I think the behaviour must be consistent across the board. cpuset.cpus must
apply to _all_ the tasks in the set, not just some of the tasks. If kthread
must run on the cpus other than current_cpuset.cpus then it should detach from
the cpuset.

To give you an example kthreads like scsi_eh, kswapd, kacpid, pdflush,
kseriod, etc are all started with cpus_allows=ALL_CPUS even though they
inherit a cpuset from kthreadd. Yes they can moved manually (with
sched_setaffinity) but the behaviour is not consistent, and for no good
reason. kthreads can be stopped/started at any time (module load for example)
which means that the user will have to keep moving them.

To sum it up here is what I'm suggesting:
kthread_bind(task, cpu)
{
   // Set PF_THREAD_BOUND
   // Move into root cpuset
   // Bind to the cpu
}

kthread_setaffinity(task, cpumask)
{
   // Enforce cpuset.cpus_allowed
   // Updated affinity mask and migrate kthread (if needed)
}

Kthreads that do not require strict cpu binding will be calling
kthread_setaffinity() instead of set_cpus_allowed_ptr() and such.

Kthreads that require strict cpu binding will be calling kthread_bind() and
detach from the cpuset they inherit from their parent.

That way the behaviour is consistent across the board.

Comments ?

Max

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/