Re: Is stopmachine() preempt safe?

[Keith Owens <kaos@ocs.com.au>]

Rusty Russell (on Mon, 28 Aug 2006 09:38:55 +1000) wrote:
>On Sun, 2006-08-27 at 19:42 +1000, Keith Owens wrote:
>> I cannot convince myself that stopmachine() is preempt safe.  What
>> prevents this race with CONFIG_PREEMPT=y?
>
>Nothing.  Read side is preempt_disable.  Write side is stopmachine.

That is very worrying.  The whole point of stopmachine is to get all
cpus to a known state with no locally cached global data, so the caller
of stopmachine can safely fiddle with some global data (like updating
the module lists).  But CONFIG_PREEMPT defeats this and turns any code
that relies on stopmachine into a race.

What we need is either a scheduler flag or a new task state to be
assigned to the kstopmachine threads.  That indicator says

  If the current state is not preempt active then schedule me.

  If the current state is preempt active then put me back in the active
  queue.

  While the runqueue contains at least one task with this flag then
  ignore reschedule on irq and prempt_enable.

That will ensure that the kstopmachine threads get scheduled as soon as
possible but only when all the preempted tasks have got to a clean stop
point, e.g. sleep or yield.  At which point they have no locally cached
global data, making stopmachine race safe again.