Re: [PATCH 1/7] work_on_cpu: helper for doing task on a CPU.

[Oleg Nesterov <oleg@redhat.com>]

On 10/24, Gautham R Shenoy wrote:
>
> Having a rule that we shouldn't use work_on_cpu() in cpu-hotplug path
> is a good thing. But maintaining it can be difficult.
>
> We've seen that in the past with the cpucontrol mutex.
> We had clear rules that functions which get called in
> cpu-hotplug callback paths, shouldn't take this mutex. But with
> functions that were called in the cpu-hotplug notifier
> path as well as normal paths, it created a whole locking mess,
> and took quite some time to fix.
>
> Similarly, right now, we can have a BUG_ON() which notifies us whenever
> someone ends up calling a function that invokes work_on_cpu() from the
> CPU-Hotplug callpath. But we will fix it only when the BUG_ON() is hit.
>
> On the other hand, if we have a mechanism that's guaranteed to work
> irrespective of the callpaths, why not use that ?

If we add another wq for work_on_cpu(), then we add another hard-to-maintain
rule: get_online_cpus() must not be used by any work which can be queued
on that wq. And, yet another per-cpu thread...

Personally I don't even think we need a BUG_ON() in work_on_cpu(), because
I don't think cpu-hotplug path is so special.

Not that I have a strong opinion though.

Oleg.