Re: BUG: spinlock bad magic on CPU#0, migration/0/9

[Oleg Nesterov <oleg@redhat.com>]

Nicholas, sorry, I sent the patch but forgot to CC you.
See https://lkml.org/lkml/2015/2/12/587

And please note that "completion" was specially designed to guarantee
that complete() can't play with this memory after wait_for_completion/etc
returns.

On 02/12, Oleg Nesterov wrote:
>
> On 02/12, Nicholas Mc Guire wrote:
> >
> > On Thu, 12 Feb 2015, Oleg Nesterov wrote:
> >
> > > No, sorry, only the 2nd one.
> > >
> > > > Unless at least document how
> > > > you can use these helpers.
> > > >
> > > > Consider this code:
> > > >
> > > > 	void xxx(void)
> > > > 	{
> > > > 		struct completion c;
> > > >
> > > > 		init_completion(&c);
> > > >
> > > > 		expose_this_completion(&c);
> > > >
> > > > 		while (!completion_done(&c)
> > > > 			schedule_timeout_uninterruptible(1);
> >
> > But that would not break due to the change - even if completion_done() had a
> > problem - complete_done() is not consuming x->done it is only checking it?
> 
> Nicholas, looks like you didn't read the text below:
> 
> > > > Before that change this code was correct, now it is not. Hmm and note that
> > > > this is what stop_machine_from_inactive_cpu() does although I do not know
> > > > if this is related or not.
> > > >
> > > > Because completion_done() can now race with complete(), the final
> > > > spin_unlock() can write to the memory after it was freed/reused. In this
> > > > case it can write to the stack after return.
> 
> Or I misunderstood you.
> 
> > >  bool completion_done(struct completion *x)
> > >  {
> > > -	return !!ACCESS_ONCE(x->done);
> > > +	if (!READ_ONCE(x->done))
> > > +		return false;
> > > +
> > > +	smp_rmb();
> > > +	spin_unlock_wait(&x->wait.lock);
> > > +	return true;
> >
> > what would be the sense of the spin_unlock_wait here ?
> > all you are interested in here is the state of x->done
> 
> No. Please see above.
> 
> > regarding the smp_rmb() where would the counterpart to that be ?
> 
> to avoid the reordering, we should not read ->wait.lock before ->done.
> 
> Oleg.