Re: [RFC PATCH 1/4] Refcount Based Cpu-Hotplug Implementation

[Oleg Nesterov <oleg@tv-sign.ru>]

On 10/17, Gautham R Shenoy wrote:
>
> On Wed, Oct 17, 2007 at 10:47:41AM +1000, Rusty Russell wrote:
> > 
> > 	I can't see where you re-initialize the completion.  
> 
> The cpu_hotplug.readers_done is a global variable which has been
> initialized in cpu_hotplug_init. 
> 
> So I am wondering is the re-initialization required ? 

I don't understand why should we re-initialize the completion too,
but see below.

> > > +static void cpu_hotplug_begin(void)
> > > +{
> > > +	mutex_lock(&cpu_hotplug.lock);
> > > +	cpu_hotplug.active_writer = current;
> > > +	while (cpu_hotplug.refcount) {
> > > +		mutex_unlock(&cpu_hotplug.lock);
> > > +		wait_for_completion(&cpu_hotplug.readers_done);
> > > +		mutex_lock(&cpu_hotplug.lock);
> > > +	}
> > 
> > AFAICT this will busy-wait on the second CPU hotplug.

Why?

> Well when the first cpu_hotplug comes out of wait_for_completion, it
> would have decremented the ->done count, so it's as good as new 
> for the second CPU hotplug, no?

No, because we decrement the ->done count only once, but there is no
guarantee that ->done == 1 when we get CPU after wakeup. Another reader
can do lock_cpu_hotplug/unlock_cpu_hotplug in between, so we have a race.

But I disagree with "Yet once a completion is completed, it needs to be
re-initialized to be reused: it's "complete" and wait_for_completion
will return immediately thereafter".

Rusty, could you please clarify?

Side note, we don't block the new readers while cpu_hotplug_begin() waits
for the completion. I don't think this is a problem, but perhaps it makes
sense to document the possible livelock.

Oleg.