What does atomic_read actually do?

What does atomic_read actually do?

[Joseph Seigh]

It doesn't do anything that would actually guarantee that the fetch from
memory would be atomic as far as I can see, at least in the x86 version.
The C standard has nothing to say about atomicity w.r.t. multithreading or
multiprocessing.  Is this a gcc compiler thing?  If so, does gcc guarantee
that it will fetch aligned ints with a single instruction on all platforms
or just x86?   And what's with volatile since if the C standard implies
nothing about multithreading then it follows that volatile has no meaning
with respect to multithreading either?  Also a gcc thing?  Are volatile
semantics well defined enough that you can use it to make the compiler
synchronize memory state as far as it is concerned?

Joe Seigh

Re: What does atomic_read actually do?

[Paolo Ornati]

On Sat, 18 Dec 2004 11:23:37 -0500
"Joseph Seigh" <jseigh_02@xemaps.com> wrote:

> It doesn't do anything that would actually guarantee that the fetch
> from memory would be atomic as far as I can see, at least in the x86
> version. The C standard has nothing to say about atomicity w.r.t.
> multithreading or multiprocessing.  Is this a gcc compiler thing?  If
> so, does gcc guarantee that it will fetch aligned ints with a single
> instruction on all platforms or just x86?   And what's with volatile
> since if the C standard implies nothing about multithreading then it
> follows that volatile has no meaning with respect to multithreading
> either?  Also a gcc thing?  Are volatile semantics well defined enough
> that you can use it to make the compiler synchronize memory state as
> far as it is concerned?

http://www.gnu.org/software/libc/manual/html_node/Atomic-Data-Access.html#Atomic%20Data%20Access

http://www.gnu.org/software/libc/manual/html_node/Atomic-Types.html#Atomic%20Types

http://gcc.gnu.org/onlinedocs/gcc/Volatiles.html

-- 
	Paolo Ornati
	Gentoo Linux (kernel 2.6.9-gentoo-r6)

Re: What does atomic_read actually do?

[Joseph Seigh]

On Sat, 18 Dec 2004 18:11:02 +0100, Paolo Ornati <ornati@fastwebnet.it>  
wrote:

> On Sat, 18 Dec 2004 11:23:37 -0500
> "Joseph Seigh" <jseigh_02@xemaps.com> wrote:
>
>> It doesn't do anything that would actually guarantee that the fetch
>> from memory would be atomic as far as I can see, at least in the x86
>> version. The C standard has nothing to say about atomicity w.r.t.
>> multithreading or multiprocessing.  Is this a gcc compiler thing?  If
>> so, does gcc guarantee that it will fetch aligned ints with a single
>> instruction on all platforms or just x86?   And what's with volatile
>> since if the C standard implies nothing about multithreading then it
>> follows that volatile has no meaning with respect to multithreading
>> either?  Also a gcc thing?  Are volatile semantics well defined enough
>> that you can use it to make the compiler synchronize memory state as
>> far as it is concerned?
>
> http://www.gnu.org/software/libc/manual/html_node/Atomic-Data-Access.html#Atomic%20Data%20Access
>
> http://www.gnu.org/software/libc/manual/html_node/Atomic-Types.html#Atomic%20Types

sig_atomic_t is only meaningful with respect to unix signals and is  
meaningless with
respect to threads.  And sig_atomic_t isn't atomic_t anyway.  And it has  
to useful
size, it could be in int on some platforms but only a byte on others.   
Signals and
threading are difficult enough as separate subjects.  Combining them is  
insanely
difficult but that's another topic.

>
> http://gcc.gnu.org/onlinedocs/gcc/Volatiles.html

"The C standard leaves it implementation defined as to what constitutes a  
volatile access."
Plus, I'm not sure sequence points are defined in C, and IIRC, C++ can  
combine and/or
reorder volatile accesses between sequence points.

Joe Seigh

Re: What does atomic_read actually do?

[Arjan van de Ven]

On Sat, 2004-12-18 at 11:23 -0500, Joseph Seigh wrote:
> It doesn't do anything that would actually guarantee that the fetch from
> memory would be atomic as far as I can see, at least in the x86 version.

define atomic....

what linux atomics guarantee you is that you either "see" the old or the
new value if you use atomic_* as the sole accessor API, with the
footnote that this only holds if you don't forcefully misalign the
atomic_t.

if you want ordering guarantees on top... you need to use explicit
bariers for that (wmb/rmb and friends).

For the "no inbetween" rule, doing the read the way x86 does works on
x86, since x86 makes sure that on the write side, no intermediate
results become visible.

Re: What does atomic_read actually do?

[Joseph Seigh]

On Sat, 18 Dec 2004 19:34:27 +0100, Arjan van de Ven <arjan@infradead.org>  
wrote:

> On Sat, 2004-12-18 at 11:23 -0500, Joseph Seigh wrote:
>> It doesn't do anything that would actually guarantee that the fetch from
>> memory would be atomic as far as I can see, at least in the x86 version.
>
> define atomic....
>
> what linux atomics guarantee you is that you either "see" the old or the
> new value if you use atomic_* as the sole accessor API, with the
> footnote that this only holds if you don't forcefully misalign the
> atomic_t.
>
> if you want ordering guarantees on top... you need to use explicit
> bariers for that (wmb/rmb and friends).
>
> For the "no inbetween" rule, doing the read the way x86 does works on
> x86, since x86 makes sure that on the write side, no intermediate
> results become visible.

I mean atomic in the either old or new sense.  I'm wondering what  
guarantees
the atomicity.  Not the C standard.  I can see the gcc compiler uses a MOV
instruction to load the atomic_t from memory which is guaranteed atomic by
the architecture if aligned properly.  But gcc does that for any old int
as far as I can see, so why use atomic_read?

Joe Seigh

Re: What does atomic_read actually do?

[Joe Korty]

On Sat, Dec 18, 2004 at 02:20:44PM -0500, Joseph Seigh wrote:
> so why use atomic_read?

Because atomic_t is opaque and atomic_read lets you get at
the value when that is all you want to do.

Regards,
Joe

Re: What does atomic_read actually do?

[Arjan van de Ven]

On Sat, 2004-12-18 at 14:20 -0500, Joseph Seigh wrote:
> I mean atomic in the either old or new sense.  I'm wondering what  
> guarantees
> the atomicity.  Not the C standard.  I can see the gcc compiler uses a MOV
> instruction to load the atomic_t from memory which is guaranteed atomic by
> the architecture if aligned properly.  But gcc does that for any old int
> as far as I can see, so why use atomic_read?

it does so on *x86*.
On other architectures, something else might be needed..

also it makes the api more symetric as well.

Re: What does atomic_read actually do?

[Joseph Seigh]

On Sat, 18 Dec 2004 20:54:40 +0100, Arjan van de Ven <arjan@infradead.org>  
wrote:

> On Sat, 2004-12-18 at 14:20 -0500, Joseph Seigh wrote:
>> I mean atomic in the either old or new sense.  I'm wondering what
>> guarantees
>> the atomicity.  Not the C standard.  I can see the gcc compiler uses a  
>> MOV
>> instruction to load the atomic_t from memory which is guaranteed atomic  
>> by
>> the architecture if aligned properly.  But gcc does that for any old int
>> as far as I can see, so why use atomic_read?
>
> it does so on *x86

Is this documented for gcc anywhere?  Just because it does so doesn't
mean it's guaranteed.

Joe Seigh

Re: What does atomic_read actually do?

[Brian Gerst]

Joseph Seigh wrote:
> It doesn't do anything that would actually guarantee that the fetch from
> memory would be atomic as far as I can see, at least in the x86 version.
> The C standard has nothing to say about atomicity w.r.t. multithreading or
> multiprocessing.  Is this a gcc compiler thing?  If so, does gcc guarantee
> that it will fetch aligned ints with a single instruction on all platforms
> or just x86?   And what's with volatile since if the C standard implies
> nothing about multithreading then it follows that volatile has no meaning
> with respect to multithreading either?  Also a gcc thing?  Are volatile
> semantics well defined enough that you can use it to make the compiler
> synchronize memory state as far as it is concerned?
> 
> Joe Seigh

For x86, the processor guarantees atomicity for simple aligned reads or 
writes.  Read-modify-write instructions need a lock prefix in order to 
become atomic.  The volatile is there so gcc doesn't miss the value 
changing from within an interrupt.

--
				Brian Gerst

Re: What does atomic_read actually do?

[Brian Gerst]

Joseph Seigh wrote:
> On Sat, 18 Dec 2004 20:54:40 +0100, Arjan van de Ven 
> <arjan@infradead.org>  wrote:
> 
>> On Sat, 2004-12-18 at 14:20 -0500, Joseph Seigh wrote:
>>
>>> I mean atomic in the either old or new sense.  I'm wondering what
>>> guarantees
>>> the atomicity.  Not the C standard.  I can see the gcc compiler uses 
>>> a  MOV
>>> instruction to load the atomic_t from memory which is guaranteed 
>>> atomic  by
>>> the architecture if aligned properly.  But gcc does that for any old int
>>> as far as I can see, so why use atomic_read?
>>
>>
>> it does so on *x86
> 
> 
> Is this documented for gcc anywhere?  Just because it does so doesn't
> mean it's guaranteed.
> 
> Joe Seigh

ftp://download.intel.com/design/Pentium4/manuals/25366814.pdf

Chapter 7, section 1

--
				Brian Gerst

Re: What does atomic_read actually do?

[Robert Love]

On Sat, 2004-12-18 at 15:43 -0500, Joseph Seigh wrote:

> > it does so on *x86
> 
> Is this documented for gcc anywhere?  Just because it does so doesn't
> mean it's guaranteed.

Listen to what Arjan is saying: It is not a compiler feature.  x86
already guarantees that an aligned word-size read is atomic in the
nothing-can-interleave sense.

	Robert Love

Re: What does atomic_read actually do?

[Joseph Seigh]

On Sun, 19 Dec 2004 17:21:06 -0500, Robert Love <rml@novell.com> wrote:

> On Sat, 2004-12-18 at 15:43 -0500, Joseph Seigh wrote:
>
>> > it does so on *x86
>>
>> Is this documented for gcc anywhere?  Just because it does so doesn't
>> mean it's guaranteed.
>
> Listen to what Arjan is saying: It is not a compiler feature.  x86
> already guarantees that an aligned word-size read is atomic in the
> nothing-can-interleave sense.
>
I'm aware of that.  I'm not asking a question about x86 architecture.  I'm
asking what guarantees that the compiler will load the int using one MOV
instruction since there's nothing in the C standard that requires that,  
even
for volatile.   I think it's unlikely the compiler would use multiple loads
a byte at a time but it really requires a compiler person to  
authoritatively
make that statement.

It's a big problem getting support for threaded programming in C since the
C standard doesn't acknowlege threads.  For Posix threads, Posix had to  
come
up with a separate compliance certification for C compilers.  So when you  
use
posix pthreads, you have to use a posix compliant compiler to ensure your
program will work correctly.

It's the same issue here.  The atomic functions are another thread api.   
What's
the assurance that gcc supports this api correctly?   There was the  
possibility
since the C standard leaves it implementation dependent what constitutes
volatile access, that gcc did something special there.  But the gcc  
documentation
says this for volatile, "There is no guarantee that these reads and writes  
are atomic,
especially for objects larger than int."

http://gcc.gnu.org/onlinedocs/gcc-3.4.3/gcc/Volatiles.html#Volatiles

Joe Seigh

Re: What does atomic_read actually do?

[Arjan van de Ven]

> I'm aware of that.  I'm not asking a question about x86 architecture.  I'm
> asking what guarantees that the compiler will load the int using one MOV
> instruction since there's nothing in the C standard that requires that,  
> even
> for volatile.   I think it's unlikely the compiler would use multiple loads
> a byte at a time but it really requires a compiler person to  
> authoritatively
> make that statement.

well... nothing really guarantees it other than that it's rather really
hard to NOT do it with one mov. And the gcc people care about their
quality of implementation enough that they will never do this...

Re: What does atomic_read actually do?

[Andrea Arcangeli]

On Sun, Dec 19, 2004 at 06:50:54PM -0500, Joseph Seigh wrote:
> What's the assurance that gcc supports this api correctly?   There was
> the  possibility since the C standard leaves it implementation
> dependent what constitutes volatile access, that gcc did something
> special there.  But the gcc  documentation says this for volatile,
> "There is no guarantee that these reads and writes  are atomic,
> especially for objects larger than int."

set_pte_atomic also requires atomicity in
asm-generic/pgtable.h:ptep_establish, but it's not even using volatile
and it's a 64bit pointer that we're writing to.  We're relaying on the
compiler to do the right thing for us. I don't think it's a good idea
for the long run, but Benjamin on ppc64 rejected my suggestion to
rewrite set_pte_atomic in asm, so I doubt you'll be able to rewrite
atomic_read with asm either (because at least atomic_read is an int and
not a long pointer, and at least atomic_read is a volatile unlike
set_pte).

My point is that even before worrying about the theoretical correctness
of atomic_read, I would suggest to worry about set_pte_atomic first,
which is a lot more likely to break if something. The compiler may truly
execute two separate writes if power of 2 bitshifts are involved, as the
optimal compilation of the C source.

Re: What does atomic_read actually do?

[Joseph Seigh]

On Mon, 20 Dec 2004 13:52:23 +0100, Andrea Arcangeli <andrea@suse.de>  
wrote:
>
> set_pte_atomic also requires atomicity in
> asm-generic/pgtable.h:ptep_establish, but it's not even using volatile
> and it's a 64bit pointer that we're writing to.  We're relaying on the
> compiler to do the right thing for us. I don't think it's a good idea
> for the long run, but Benjamin on ppc64 rejected my suggestion to
> rewrite set_pte_atomic in asm, so I doubt you'll be able to rewrite
> atomic_read with asm either (because at least atomic_read is an int and
> not a long pointer, and at least atomic_read is a volatile unlike
> set_pte).
>
> My point is that even before worrying about the theoretical correctness
> of atomic_read, I would suggest to worry about set_pte_atomic first,
> which is a lot more likely to break if something. The compiler may truly
> execute two separate writes if power of 2 bitshifts are involved, as the
> optimal compilation of the C source.

Actually, I'm programming out in user space.  I was looking at the atomic.h
stuff to see how much would be usable.  But I have to look at reading
and writing pointers atomically also since that is heavily used in  
lock-free
programming, e.g. RCU for preemptive user threads which I have a couple of
implementations for.  I remember a discussion a while back about dependent
load memory ordering (alpha doesn't have it) for RCU but I don't recall any
discussion of the requirement for gcc to do atomic loads and stores.  I  
guess
it's just assumed.

It would be nice to have macros tied to the linux ones, ie. check the  
current
linux versions when porting them.  But I can always use asm if I have to.   
I'll
probably have to modify the conditions of use to say "no warranty is  
implied
and we really do mean it" and add a compiler flag to select between the  
safe asm
version and the "Do you feel lucky?" C macro version.

Joe Seigh