Re: [PATCH] Document Linux's memory barriers [try #4]

David Howells
2006-03-15T01:19:21Z

Linus Torvalds <torvalds@osdl.org> wrote:

> That's not that different from doing
>
> ptr = read a
> data = read [ptr]
>
> and speculating the result of the first read.

But that would lead to the situation I suggested (q == &b and d == a), not the
one Paul suggested (q == &b and d == old b) because we'd speculate on the old
value of the pointer, and so see it before it's updated, and thus still
pointing to a.

> The cache is fully coherent, but the coherency isn't _ordered_.
>
> Remember: the smp_wmb() only orders on the _writer_ side. Not on the
> reader side. The writer may send out the stuff in a particular order, but
> the reader might see them in a different order because _it_ might queue
> the bus events internally for its caches (in particular, it could end up
> delaying updating a particular way in the cache because it's busy).

Ummm... So whilst smp_wmb() commits writes to the mercy of the cache coherency
system in a particular order, the updates can be passed over from one cache to
another and committed to the reader's cache in any order, and can even be
delayed:

CPU 1 CPU 2 COMMENT
=============== =============== =======================================
a == 0, b == 1 and p == &a, q == &a
b = 2;
smp_wmb(); Make sure b is changed before p
<post b=2>
<queue b=2>
p = &b; q = p;
<post p=&b>
<queue p=&b>
d = *q;
<commit p=&b>
<post q=p>
<read *q> Reads from b before b updated in cache
<post d=*q>
<commit b=2>

I presume the Alpha MB instruction forces cache queue completion in addition
to a partial ordering on memory accesses:

CPU 1 CPU 2 COMMENT
=============== =============== =======================================
a == 0, b == 1 and p == &a, q == &a
b = 2;
smp_wmb(); Make sure b is changed before p
<post b=2>
<queue b=2>
p = &b; q = p;
<post p=&b>
<queue p=&b>
smp_read_barrier_depends();
<commit b=2>
<commit p=&b>
d = *q;
<post q=p>
<read *q> Reads new value of b
<post d=*q>

David