Re: [PATCH, RFC] reimplement flush_workqueue()

[Oleg Nesterov <oleg@tv-sign.ru>]

On 01/04, Andrew Morton wrote:
>
> On Thu, 4 Jan 2007 17:29:36 +0300
> Oleg Nesterov <oleg@tv-sign.ru> wrote:
> 
> > > In brief:
> > > 
> > > keventd thread					hotplug thread
> > > --------------					--------------
> > > 
> > >   run_workqueue()
> > > 	|
> > >      work_fn()
> > > 	 |
> > > 	flush_workqueue()
> > > 	     |	
> > > 	   flush_cpu_workqueue
> > > 		|				cpu_down()
> > > 	     mutex_unlock(wq_mutex);		     |
> > > 	(above opens window for hotplug)	   mutex_lock(wq_mutex);
> > >     		|				   /* bring down cpu */	
> > > 	     wait_for_completition();		     notifier(CPU_DEAD, ..)
> > > 		| 				       workqueue_cpu_callback
> > > 		| 				        cleanup_workqueue_thread
> > > 		|					  kthread_stop()
> > > 		|
> > > 		|
> > > 	     mutex_lock(wq_mutex); <- Can deadlock
> > > 
> > > 
> > > The kthread_stop() will wait for keventd() thread to exit, but keventd()
> > > is blocked on mutex_lock(wq_mutex) leading to a deadlock.
> 
> This?
> 
> 
> --- a/kernel/workqueue.c~flush_workqueue-use-preempt_disable-to-hold-off-cpu-hotplug
> +++ a/kernel/workqueue.c
> @@ -419,18 +419,22 @@ static void flush_cpu_workqueue(struct c
>  		 * Probably keventd trying to flush its own queue. So simply run
>  		 * it by hand rather than deadlocking.
>  		 */
> -		mutex_unlock(&workqueue_mutex);
> +		preempt_enable();

Ah, (looking at _cpu_down()->stop_machine()), so preempt_disable() not only "pins"
the current CPU, it blocks cpu_down(), yes ???

I guess this should work then. I'll try to re-check this code on weekend.

Oleg.