Re: memory barrier question

[David Howells <dhowells@redhat.com>]

Miklos Szeredi <miklos@szeredi.hu> wrote:

> Consider the following example:
> 
> Start:
> 	p = NULL;
> 	x = 0;
> 
> CPU1:
> 	atomic_inc(&x);
> 	p = &x;
> 
> CPU2:
> 	if (p)
> 		z = atomic_read(p);
> 
> Is it possible to end up with z == 0?

I think so.  I'm not sure that you can assume that CPU1 does its two
'operations' in the same order.  You can guarantee that the read of x,
increment, and write of x will be done in an order, and that no one else will
see an intermediate state, but you can't guarantee that CPU2 will see x
changed before p is changed.

In Documentation/memory-barriers.txt, it says:

	The following also do _not_ imply memory barriers, and so may require
	explicit memory barriers under some circumstances
	(smp_mb__before_atomic_dec() for instance):

		atomic_add();
		atomic_sub();
		atomic_inc();
		atomic_dec();

so you need _two_ memory barriers, e.g.:

	CPU1:
		atomic_inc(&x);
		smp_mb__after_atomic_inc()
		p = &x;

	CPU2:
		q = p;
		smp_rmb();
		if (q)
			z = atomic_read(q);

Note that atomic_inc() may imply a suitable memory barrier on some arches, and
so has special variant barrier functions of its own.

> What if there's a lock/unlock before setting "p"?

If there's a lock+unlock between, then this counts as a full memory barrier:

	CPU1:
		atomic_inc(&x);
		spin_lock(&foo);
		spin_unlock(&foo);
		p = &x;

but you still need the matching smp_rmb() on CPU2.

> What if there's a write barrier before setting "p"?

That's fine, but you still need the matching smp_rmb() on CPU2.

David