Re: [PATCH] improve stop_machine performance

[Tejun Heo <tj@kernel.org>]

Hello,

On 03/05/2010 06:20 AM, Dimitri Sivanich wrote:
> On systems with large cpu counts, we've been seeing long bootup times
> associated with stop_machine operations.  I've noticed that by simply
> removing the creation of the workqueue and associated percpu variables
> in subsequent stop_machine calls, we can reduce boot times on a
> 1024 processor SGI UV system from 25-30 (or more) minutes down to 12
> minutes.
> 
> The attached patch does this in a simple way by removing the
> stop_machine_destroy interface, thereby by leaving the workqueues and
> percpu variables for later use once they are created.
> 
> If people are against having these areas around after boot, maybe there
> are some alternatives that will still allow for this optimization:
> 
>  - Set a timer to go off after a configurable number of minutes, at
>    which point the workqueue areas will be deleted.
> 
>  - Keep the stop_machine_destroy function, but somehow run it at the tail
>    end of boot (after modules have loaded), rather than running it at
>    every stop_machine call.

Yeah, I can indeed imagine that creating and destroying all those
workers on every module load during boot would be very costly if there
are lots of CPUs.  How about sharing the migration thread so that it
serves as one-per-cpu uninterruptible RT simple thread pool?  It's not
like these things can run taking their turns anyway.  I'll go ahead
and make something up.

Thanks.

-- 
tejun