Re: [PATCH] kernel: fix data race in put_pid

[Oleg Nesterov <oleg@redhat.com>]

On 09/18, Peter Zijlstra wrote:
>
> On Fri, Sep 18, 2015 at 03:28:44PM +0200, Oleg Nesterov wrote:
> > On 09/18, Peter Zijlstra wrote:
> > >
> > > 	ns = pid->numbers[pid->level].ns;
> > > 	if ((atomic_read(&pid->count) == 1) ||
> > > 	     atomic_dec_and_test(&pid->count)) {
> > >
> > > +		smp_read_barrier_depends(); /* ctrl-dep */
> >
> > Not sure... Firstly it is not clear what this barrier pairs with. And I
> > have to admit that I can not understand if _CTRL() logic applies here.
> > The same for atomic_read_ctrl().
>
> The control dependency barrier pairs with the full barrier of
> atomic_dec_and_test.

Yes thanks. I already got it. I hope ;)

> > OK, please forget about put_pid() for the moment. Suppose we have
> >
> > 	X = 1;
> > 	synchronize_sched();
> > 	Y = 1;
> >
> > Or
> > 	X = 1;
> > 	call_rcu_sched( func => { Y = 1; } );
> >
> >
> >
> > Now. In theory this this code is wrong:
> >
> > 	if (Y) {
> > 		BUG_ON(X == 0);
> > 	}
> >
> > But this is correct:
> >
> > 	if (Y) {
> > 		rcu_read_lock_sched();
> > 		rcu_read_unlock_sched();
> > 		BUG_ON(X == 0);
> > 	}
> >
> > So perhaps something like this
> >
> > 	/*
> > 	 * Comment to explain it is eq to read_lock + read_unlock,
> > 	 * in a sense that this guarantees a full barrier wrt to
> > 	 * the previous synchronize_sched().
> > 	 */
> > 	#define rcu_read_barrier_sched()	barrier()
> >
> > make sense?
> >
> >
> > And again, I simply can't understand if this code
> >
> > 	if (READ_ONCE_CTRL(Y))
> > 		BUG_ON(X == 0);
> >
> > to me it does _not_ look correct in theory.
>
> So control dependencies provide a load-store barrier. Your examples
> above rely on a load-load barrier; BUG_ON(X == 0) is a load.

Yes, yes...

What I tried to say is that we could fix it another way. And even look
at this problem from another angle. No, it is not that I think it would
be better in this particular case, but still...

put_pid() could do

	if (atomic_read(&pid->count) == 1) {

		rcu_read_lock();
		rcu_read_unlock();
		
		kmem_cache_free(pid);
	}

if we observe atomic_read() == 1, we know that we have at least one
gp pass after all other writes to this memory (namely hlist_del_rcu()
which removes it from rcu-list). Because we can see atomic_read() == 1
until delayed_put_pid() (called by RCU) drops its reference.

and perhaps this lock + unlock pair (which is nop at least for _sched)
makes some sense in general...

Oleg.