Re: [RFC, PATCH 1/2] qrcu: "quick" srcu implementation

[Oleg Nesterov <oleg@tv-sign.ru>]

On 11/29, Paul E. McKenney wrote:
>
> On Thu, Nov 30, 2006 at 04:57:14AM +0300, Oleg Nesterov wrote:
> > (the same patch + comments from Paul)
> > 
> With the addition of a comment for the smp_mb() at the beginning of
> synchronize_qrcu(), shown below:
> 
> Acked-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>

Thanks!

> 	/*
> 	 * The following memory barrier is needed to ensure that
> 	 * and subsequent freeing of data elements previously
> 	 * removed is seen by other CPUs after the wait completes.
> 	 */

I think we have another reason for mb(), but I can't suggest a clear
comment.

	struct data {
		...
		int in_use;
		...
	}

	void free_data(struct data *p)
	{
		BUG_ON(p->in_use);
		kfree(p);
	}

	struct data *DATA;

Reader:

	qrcu_read_lock();
	data = rcu_dereference(DATA);

	data->in_use = 1;
	do_something(data);
	data->in_use = 0;

	qrcu_read_unlock();

Writer:

	old = DATA;
	DATA = alloc_new_data();
	
	synchronize_qrcu();
	free_data(old);

qrcu_read_unlock() does (implicit) mb() on reader's side, but we must pair
it on our side, otherwise we can't be sure (of course, _only_ in theory) we
are seeing all the changes (->in_use == 0) made by the reader.

> Hmmm...  Now I am wondering if the memory barriers inherent in the
> __wait_event() suffice for this last barrier...  :-/  Thoughts?
> 
> > +	smp_mb();

Fastpath skips __wait_event(), and it is possible that the reader does
lock/unlock between the first 'mb()' and 'if (atomic_read() == 1)'.

Oleg.