atomic operations in user space

atomic operations in user space

[Xupei Liang]

Hi,

On the PPC platform, can I use the atomic operations
defined in the kernel, e.g. atomic_add(),
atomic_dec(), etc. in the user space programs? 

I compiled the following program:

#include <asm-ppc/atomic.h>

int main()
{
	atomic_t atom;

	atomic_add(1, &atom);
	return (0);
}

I am able to compile it and the atomic_add seems to be
translated into the correct instructions. Can anybody
please confirm? Thanks.

Regards,

Terry Liang

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

Re: atomic operations in user space

[Arnd Bergmann]

Am Tuesday 22 August 2006 22:50 schrieb Xupei Liang:
> #include <asm-ppc/atomic.h>
>
> int main()
> {
> =A0=A0=A0=A0=A0=A0=A0=A0atomic_t atom;
>
> =A0=A0=A0=A0=A0=A0=A0=A0atomic_add(1, &atom);
> =A0=A0=A0=A0=A0=A0=A0=A0return (0);
> }
>
> I am able to compile it and the atomic_add seems to be
> translated into the correct instructions. Can anybody
> please confirm? Thanks.

No, you can't do that, the kernel headers are not generally
working in user space, recent kernel versions protect you
from doing this.

If you don't have a library providing you with atomic operations,
you can copy the inline functions from the kernel, which will
work fine, but require that your app is GPL licensed.

	Arnd <><

atomic operations in user space

[Terry Liang]

[-- Attachment #1: Type: text/plain, Size: 350 bytes --]

Thanks. Arnd. My main concern is whether the operations are really atomic as they are in the kernel space. I have read some discussion in another forum that on other platforms, even if you are able to compile the atomic_add(), atomic_set(), etc. from an user space application,  they don't guarantee to be atomic. Thanks.

Regards,

Terry Liang

[-- Attachment #2: Type: text/html, Size: 820 bytes --]

RE: atomic operations in user space

[Li Yang-r58472]

Why do you need atomic operations in user land?  IPC will be sufficient
to deal with race conditions between processes.=20

Best Regards,
Leo
> -----Original Message-----
> From: linuxppc-embedded-bounces+leoli=3Dfreescale.com@ozlabs.org
> [mailto:linuxppc-embedded-bounces+leoli=3Dfreescale.com@ozlabs.org] On
Behalf Of
> Terry Liang
> Sent: Thursday, August 24, 2006 3:04 AM
> To: linuxppc-embedded@ozlabs.org
> Subject: atomic operations in user space
>=20
> Thanks. Arnd. My main concern is whether the operations are really
atomic as they
> are in the kernel space. I have read some discussion in another forum
that on other
> platforms, even if you are able to compile the atomic_add(),
atomic_set(), etc.
> from an user space application,  they don't guarantee to be atomic.
Thanks.
>=20
> Regards,
>=20
> Terry Liang

Re: atomic operations in user space

[Brent Cook]

[-- Attachment #1: Type: text/plain, Size: 2202 bytes --]

On Thursday 24 August 2006 05:39, Li Yang-r58472 wrote:
> Why do you need atomic operations in user land?  IPC will be sufficient
> to deal with race conditions between processes.
>
> Best Regards,
> Leo

What about multiple threads within a process updating a counter?

Of course, if you look at these functions in the kernel header, they're just 2 or 3 inline assembly calls - you could easily rewrite them. Google for 'PowerPC atomic increment' and grab one of the unencumbered implementations if you need to use it in a non-GPL program.

On the other hand, I see no license at the top of my /usr/include/asm-i386/atomic.h file at all, same for PowerPC - are Linux header files actually GPL or are they more like the glibc headers, with exceptions made for userspace programs?

The atomic operations on x86 were accidentally exported early on, so they have to hang around apparently for compatibility (there are some mailing list threads out there to this effect.) Currently, you just have to assume in Linux that if you include something from /usr/include/linux or asm that it will not necessarily be cross-version or cross-architecture compatible. Not every arch in Linux even has atomic operations of this nature, which I guess is the main reason why they are not exported in general.

 - Brent

> > -----Original Message-----
> > From: linuxppc-embedded-bounces+leoli=freescale.com@ozlabs.org
> > [mailto:linuxppc-embedded-bounces+leoli=freescale.com@ozlabs.org] On
>
> Behalf Of
>
> > Terry Liang
> > Sent: Thursday, August 24, 2006 3:04 AM
> > To: linuxppc-embedded@ozlabs.org
> > Subject: atomic operations in user space
> >
> > Thanks. Arnd. My main concern is whether the operations are really
>
> atomic as they
>
> > are in the kernel space. I have read some discussion in another forum
>
> that on other
>
> > platforms, even if you are able to compile the atomic_add(),
>
> atomic_set(), etc.
>
> > from an user space application,  they don't guarantee to be atomic.
>
> Thanks.
>
> > Regards,
> >
> > Terry Liang
>
> _______________________________________________
> Linuxppc-embedded mailing list
> Linuxppc-embedded@ozlabs.org
> https://ozlabs.org/mailman/listinfo/linuxppc-embedded

[-- Attachment #2: Type: text/html, Size: 2875 bytes --]

RE: atomic operations in user space

[Li Yang-r58472]

> -----Original Message-----
> From: Brent Cook [mailto:bcook@bpointsys.com]
> Sent: Thursday, August 24, 2006 10:18 PM
> To: linuxppc-embedded@ozlabs.org
> Cc: Li Yang-r58472; Terry Liang
> Subject: Re: atomic operations in user space
>=20
> On Thursday 24 August 2006 05:39, Li Yang-r58472 wrote:
>=20
> > Why do you need atomic operations in user land? IPC will be
sufficient
>=20
> > to deal with race conditions between processes.
>=20
> >
>=20
> > Best Regards,
>=20
> > Leo
>=20
> What about multiple threads within a process updating a counter?

Is there anything preventing semaphore to be used in threads?
>=20
> Of course, if you look at these functions in the kernel header,
they're just 2 or
> 3 inline assembly calls - you could easily rewrite them. Google for
'PowerPC atomic
> increment' and grab one of the unencumbered implementations if you
need to use it
> in a non-GPL program.
>=20
> On the other hand, I see no license at the top of my
/usr/include/asm-i386/atomic.h
> file at all, same for PowerPC - are Linux header files actually GPL or
are they
> more like the glibc headers, with exceptions made for userspace
programs?
>=20
> The atomic operations on x86 were accidentally exported early on, so
they have to
> hang around apparently for compatibility (there are some mailing list
threads out
> there to this effect.) Currently, you just have to assume in Linux
that if you
> include something from /usr/include/linux or asm that it will not
necessarily be
> cross-version or cross-architecture compatible. Not every arch in
Linux even has
> atomic operations of this nature, which I guess is the main reason why
they are
> not exported in general.
>=20
> - Brent

RE: atomic operations in user space

[Xupei Liang]

I think it is less expensive using atomic operation
sometimes in the user space, e.g. when updating a
counter. If this counter is to be updated by a lot of
processes, using semaphore can potentially cause a lot
of task switching. 

Regards,

Terry Liang


> -----Original Message-----
> From: Brent Cook [mailto:bcook at bpointsys.com]
> Sent: Thursday, August 24, 2006 10:18 PM
> To: linuxppc-embedded at ozlabs.org
> Cc: Li Yang-r58472; Terry Liang
> Subject: Re: atomic operations in user space
> 
> On Thursday 24 August 2006 05:39, Li Yang-r58472
wrote:
> 
> > Why do you need atomic operations in user land?
IPC will be
sufficient
> 
> > to deal with race conditions between processes.
> 
> >
> 
> > Best Regards,
> 
> > Leo
> 
> What about multiple threads within a process
updating a counter?

Is there anything preventing semaphore to be used in
threads?
> 
> Of course, if you look at these functions in the
kernel header,
they're just 2 or
> 3 inline assembly calls - you could easily rewrite
them. Google for
'PowerPC atomic
> increment' and grab one of the unencumbered
implementations if you
need to use it
> in a non-GPL program.
> 
> On the other hand, I see no license at the top of my
/usr/include/asm-i386/atomic.h
> file at all, same for PowerPC - are Linux header
files actually GPL or
are they
> more like the glibc headers, with exceptions made
for userspace
programs?
> 
> The atomic operations on x86 were accidentally
exported early on, so
they have to
> hang around apparently for compatibility (there are
some mailing list
threads out
> there to this effect.) Currently, you just have to
assume in Linux
that if you
> include something from /usr/include/linux or asm
that it will not
necessarily be
> cross-version or cross-architecture compatible. Not
every arch in
Linux even has
> atomic operations of this nature, which I guess is
the main reason why
they are
> not exported in general.
> 
> - Brent



__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

RE: atomic operations in user space

[Li Yang-r58472]

> -----Original Message-----
> From: linuxppc-embedded-bounces+leoli=3Dfreescale.com@ozlabs.org
> [mailto:linuxppc-embedded-bounces+leoli=3Dfreescale.com@ozlabs.org] On
Behalf Of
> Xupei Liang
> Sent: Tuesday, August 29, 2006 8:44 AM
> To: linuxppc-embedded@ozlabs.org
> Subject: RE: atomic operations in user space
>=20
> I think it is less expensive using atomic operation
> sometimes in the user space, e.g. when updating a

Atomic operations are working under strict restrictions.  Generally they
won't work for user space.  Your best bet is to use semaphore instead.

> counter. If this counter is to be updated by a lot of
> processes, using semaphore can potentially cause a lot
> of task switching.
>=20
> Regards,
>=20
> Terry Liang
>=20
>=20
> > -----Original Message-----
> > From: Brent Cook [mailto:bcook at bpointsys.com]
> > Sent: Thursday, August 24, 2006 10:18 PM
> > To: linuxppc-embedded at ozlabs.org
> > Cc: Li Yang-r58472; Terry Liang
> > Subject: Re: atomic operations in user space
> >
> > On Thursday 24 August 2006 05:39, Li Yang-r58472
> wrote:
> >
> > > Why do you need atomic operations in user land?
> IPC will be
> sufficient
> >
> > > to deal with race conditions between processes.
> >
> > >
> >
> > > Best Regards,
> >
> > > Leo
> >
> > What about multiple threads within a process
> updating a counter?
>=20
> Is there anything preventing semaphore to be used in
> threads?
> >
> > Of course, if you look at these functions in the
> kernel header,
> they're just 2 or
> > 3 inline assembly calls - you could easily rewrite
> them. Google for
> 'PowerPC atomic
> > increment' and grab one of the unencumbered
> implementations if you
> need to use it
> > in a non-GPL program.
> >
> > On the other hand, I see no license at the top of my
> /usr/include/asm-i386/atomic.h
> > file at all, same for PowerPC - are Linux header
> files actually GPL or
> are they
> > more like the glibc headers, with exceptions made
> for userspace
> programs?
> >
> > The atomic operations on x86 were accidentally
> exported early on, so
> they have to
> > hang around apparently for compatibility (there are
> some mailing list
> threads out
> > there to this effect.) Currently, you just have to
> assume in Linux
> that if you
> > include something from /usr/include/linux or asm
> that it will not
> necessarily be
> > cross-version or cross-architecture compatible. Not
> every arch in
> Linux even has
> > atomic operations of this nature, which I guess is
> the main reason why
> they are
> > not exported in general.
> >
> > - Brent
>=20
>=20
>=20
> __________________________________________________
> Do You Yahoo!?
> Tired of spam?  Yahoo! Mail has the best spam protection around
> http://mail.yahoo.com
> _______________________________________________
> Linuxppc-embedded mailing list
> Linuxppc-embedded@ozlabs.org
> https://ozlabs.org/mailman/listinfo/linuxppc-embedded

RE: atomic operations in user space

[Esben Nielsen]

On Tue, 29 Aug 2006, Li Yang-r58472 wrote:

>> -----Original Message-----
>> From: linuxppc-embedded-bounces+leoli=freescale.com@ozlabs.org
>> [mailto:linuxppc-embedded-bounces+leoli=freescale.com@ozlabs.org] On
> Behalf Of
>> Xupei Liang
>> Sent: Tuesday, August 29, 2006 8:44 AM
>> To: linuxppc-embedded@ozlabs.org
>> Subject: RE: atomic operations in user space
>>
>> I think it is less expensive using atomic operation
>> sometimes in the user space, e.g. when updating a
>
> Atomic operations are working under strict restrictions.  Generally they
> won't work for user space.  Your best bet is to use semaphore instead.
>

Wrong.
1) Semaphores give priority inversions. Use a mutex with priority 
inheritance instead. This comes in 2.6.18.
2) These mutexes are based on futexes which requires atomic operations in 
userspace. These are available on most architectures. Look at the glibc code
in nptl/sysdeps/unix/sysv/linux/powerpc/lowlevellock.h forinstance. 
Use that and your PPC manual to implement your atomic operations.

Esben

>> counter. If this counter is to be updated by a lot of
>> processes, using semaphore can potentially cause a lot
>> of task switching.
>>
>> Regards,
>>
>> Terry Liang
>>
>>
>>> -----Original Message-----
>>> From: Brent Cook [mailto:bcook at bpointsys.com]
>>> Sent: Thursday, August 24, 2006 10:18 PM
>>> To: linuxppc-embedded at ozlabs.org
>>> Cc: Li Yang-r58472; Terry Liang
>>> Subject: Re: atomic operations in user space
>>>
>>> On Thursday 24 August 2006 05:39, Li Yang-r58472
>> wrote:
>>>
>>>> Why do you need atomic operations in user land?
>> IPC will be
>> sufficient
>>>
>>>> to deal with race conditions between processes.
>>>
>>>>
>>>
>>>> Best Regards,
>>>
>>>> Leo
>>>
>>> What about multiple threads within a process
>> updating a counter?
>>
>> Is there anything preventing semaphore to be used in
>> threads?
>>>
>>> Of course, if you look at these functions in the
>> kernel header,
>> they're just 2 or
>>> 3 inline assembly calls - you could easily rewrite
>> them. Google for
>> 'PowerPC atomic
>>> increment' and grab one of the unencumbered
>> implementations if you
>> need to use it
>>> in a non-GPL program.
>>>
>>> On the other hand, I see no license at the top of my
>> /usr/include/asm-i386/atomic.h
>>> file at all, same for PowerPC - are Linux header
>> files actually GPL or
>> are they
>>> more like the glibc headers, with exceptions made
>> for userspace
>> programs?
>>>
>>> The atomic operations on x86 were accidentally
>> exported early on, so
>> they have to
>>> hang around apparently for compatibility (there are
>> some mailing list
>> threads out
>>> there to this effect.) Currently, you just have to
>> assume in Linux
>> that if you
>>> include something from /usr/include/linux or asm
>> that it will not
>> necessarily be
>>> cross-version or cross-architecture compatible. Not
>> every arch in
>> Linux even has
>>> atomic operations of this nature, which I guess is
>> the main reason why
>> they are
>>> not exported in general.
>>>
>>> - Brent
>>
>>
>>
>> __________________________________________________
>> Do You Yahoo!?
>> Tired of spam?  Yahoo! Mail has the best spam protection around
>> http://mail.yahoo.com
>> _______________________________________________
>> Linuxppc-embedded mailing list
>> Linuxppc-embedded@ozlabs.org
>> https://ozlabs.org/mailman/listinfo/linuxppc-embedded
> _______________________________________________
> Linuxppc-embedded mailing list
> Linuxppc-embedded@ozlabs.org
> https://ozlabs.org/mailman/listinfo/linuxppc-embedded
>

RE: atomic operations in user space

[Li Yang-r58472]

> -----Original Message-----
> From: Esben Nielsen [mailto:nielsen.esben@gogglemail.com]
> Sent: Tuesday, August 29, 2006 4:33 PM
> To: Li Yang-r58472
> Cc: Xupei Liang; linuxppc-embedded@ozlabs.org
> Subject: RE: atomic operations in user space
>=20
>=20
>=20
> On Tue, 29 Aug 2006, Li Yang-r58472 wrote:
>=20
> >> -----Original Message-----
> >> From: linuxppc-embedded-bounces+leoli=3Dfreescale.com@ozlabs.org
> >> [mailto:linuxppc-embedded-bounces+leoli=3Dfreescale.com@ozlabs.org]
On
> > Behalf Of
> >> Xupei Liang
> >> Sent: Tuesday, August 29, 2006 8:44 AM
> >> To: linuxppc-embedded@ozlabs.org
> >> Subject: RE: atomic operations in user space
> >>
> >> I think it is less expensive using atomic operation
> >> sometimes in the user space, e.g. when updating a
> >
> > Atomic operations are working under strict restrictions.  Generally
they
> > won't work for user space.  Your best bet is to use semaphore
instead.
> >
>=20
> Wrong.
> 1) Semaphores give priority inversions. Use a mutex with priority
> inheritance instead. This comes in 2.6.18.
> 2) These mutexes are based on futexes which requires atomic operations
in
> userspace. These are available on most architectures. Look at the
glibc code
> in nptl/sysdeps/unix/sysv/linux/powerpc/lowlevellock.h forinstance.
> Use that and your PPC manual to implement your atomic operations.

No matter semaphore or futex, it uses system calls to kernel.  And the
true atomic operation is in kernel not user space.  Maybe it's feasible
for other architectures to do atomic operations directly in user space.
IMHO, not for powerpc.

>=20
> Esben
>=20
> >> counter. If this counter is to be updated by a lot of
> >> processes, using semaphore can potentially cause a lot
> >> of task switching.
> >>
> >> Regards,
> >>
> >> Terry Liang
> >>
> >>
> >>> -----Original Message-----
> >>> From: Brent Cook [mailto:bcook at bpointsys.com]
> >>> Sent: Thursday, August 24, 2006 10:18 PM
> >>> To: linuxppc-embedded at ozlabs.org
> >>> Cc: Li Yang-r58472; Terry Liang
> >>> Subject: Re: atomic operations in user space
> >>>
> >>> On Thursday 24 August 2006 05:39, Li Yang-r58472
> >> wrote:
> >>>
> >>>> Why do you need atomic operations in user land?
> >> IPC will be
> >> sufficient
> >>>
> >>>> to deal with race conditions between processes.
> >>>
> >>>>
> >>>
> >>>> Best Regards,
> >>>
> >>>> Leo
> >>>
> >>> What about multiple threads within a process
> >> updating a counter?
> >>
> >> Is there anything preventing semaphore to be used in
> >> threads?
> >>>
> >>> Of course, if you look at these functions in the
> >> kernel header,
> >> they're just 2 or
> >>> 3 inline assembly calls - you could easily rewrite
> >> them. Google for
> >> 'PowerPC atomic
> >>> increment' and grab one of the unencumbered
> >> implementations if you
> >> need to use it
> >>> in a non-GPL program.
> >>>
> >>> On the other hand, I see no license at the top of my
> >> /usr/include/asm-i386/atomic.h
> >>> file at all, same for PowerPC - are Linux header
> >> files actually GPL or
> >> are they
> >>> more like the glibc headers, with exceptions made
> >> for userspace
> >> programs?
> >>>
> >>> The atomic operations on x86 were accidentally
> >> exported early on, so
> >> they have to
> >>> hang around apparently for compatibility (there are
> >> some mailing list
> >> threads out
> >>> there to this effect.) Currently, you just have to
> >> assume in Linux
> >> that if you
> >>> include something from /usr/include/linux or asm
> >> that it will not
> >> necessarily be
> >>> cross-version or cross-architecture compatible. Not
> >> every arch in
> >> Linux even has
> >>> atomic operations of this nature, which I guess is
> >> the main reason why
> >> they are
> >>> not exported in general.
> >>>
> >>> - Brent
> >>
> >>
> >>
> >> __________________________________________________
> >> Do You Yahoo!?
> >> Tired of spam?  Yahoo! Mail has the best spam protection around
> >> http://mail.yahoo.com
> >> _______________________________________________
> >> Linuxppc-embedded mailing list
> >> Linuxppc-embedded@ozlabs.org
> >> https://ozlabs.org/mailman/listinfo/linuxppc-embedded
> > _______________________________________________
> > Linuxppc-embedded mailing list
> > Linuxppc-embedded@ozlabs.org
> > https://ozlabs.org/mailman/listinfo/linuxppc-embedded
> >

RE: atomic operations in user space

[Liu Dave-r63238]

> > 2) These mutexes are based on futexes which requires atomic=20
> > operations in userspace. These are available on most architectures.
Look at
> > the glibc code in
> > nptl/sysdeps/unix/sysv/linux/powerpc/lowlevellock.h for instance.
> > Use that and your PPC manual to implement your atomic operations.
>=20
> No matter semaphore or futex, it uses system calls to kernel.  And the
> true atomic operation is in kernel not user space.  Maybe=20
> it's feasible
> for other architectures to do atomic operations directly in=20
> user space.
> IMHO, not for powerpc.

Are you meaning that we didn't do atomic operations directly in user
space
on powerpc platform ?

-DAve

RE: atomic operations in user space

[Esben Nielsen]

On Tue, 29 Aug 2006, Liu Dave-r63238 wrote:

>>> 2) These mutexes are based on futexes which requires atomic
>>> operations in userspace. These are available on most architectures.
> Look at
>>> the glibc code in
>>> nptl/sysdeps/unix/sysv/linux/powerpc/lowlevellock.h for instance.
>>> Use that and your PPC manual to implement your atomic operations.
>>
>> No matter semaphore or futex, it uses system calls to kernel.

There is only a system call if there is congestion - that is the whole idea
behind the futex.

>> And the
>> true atomic operation is in kernel not user space.

"True" atomic operations are available in user space on most architectures.

>> Maybe
>> it's feasible
>> for other architectures to do atomic operations directly in
>> user space.
>> IMHO, not for powerpc.

It is available for PowerPC, but not in POWER and POWER2 instructionsets 
according to http://www.nersc.gov/vendor_docs/ibm/asm/lwarx.htm#idx607
It is the same in the ARM world: Atomic instructions was introduced in 
ARMv6 I believe. Older ARM processors don't have them.

>
> Are you meaning that we didn't do atomic operations directly in user
> space
> on powerpc platform ?
>

Well, that is not the conclusion I get either when reading the glibc code.
Try to look at glibc-2.3.5/sysdeps/powerpc/bits/atomic.h.

This is by the way probably what the original post in this thread wanted 
in the first place!

Esben


> -DAve
> _______________________________________________
> Linuxppc-embedded mailing list
> Linuxppc-embedded@ozlabs.org
> https://ozlabs.org/mailman/listinfo/linuxppc-embedded
>

RE: atomic operations in user space

[Liu Dave-r63238]

> >>> 2) These mutexes are based on futexes which requires atomic=20
> >>> operations in userspace. These are available on most=20
> architectures.
> > Look at
> >>> the glibc code in
> >>> nptl/sysdeps/unix/sysv/linux/powerpc/lowlevellock.h for instance.
> >>> Use that and your PPC manual to implement your atomic operations.
> >>
> >> No matter semaphore or futex, it uses system calls to kernel.
>=20
> There is only a system call if there is congestion - that is=20
> the whole idea behind the futex.
>=20
> >> And the
> >> true atomic operation is in kernel not user space.
>=20
> "True" atomic operations are available in user space on most=20
> architectures.
>=20
> >> Maybe
> >> it's feasible
> >> for other architectures to do atomic operations directly in user=20
> >> space.
> >> IMHO, not for powerpc.
>=20
> It is available for PowerPC, but not in POWER and POWER2=20
> instructionsets according to=20
> http://www.nersc.gov/vendor_docs/ibm/asm/lwarx.htm#idx607
> It is the same in the ARM world: Atomic instructions was introduced in
> ARMv6 I believe. Older ARM processors don't have them.
>=20
> >
> > Are you meaning that we didn't do atomic operations directly in user
> > space
> > on powerpc platform ?
> >
>=20
> Well, that is not the conclusion I get either when reading=20
> the glibc code.
> Try to look at glibc-2.3.5/sysdeps/powerpc/bits/atomic.h.
>=20
> This is by the way probably what the original post in this=20
> thread wanted=20
> in the first place!
>=20
> Esben

I totally agree with you and Brant about it. I don't believe we cannot
do the atomic operation in user space on powerpc.

-DAve=20

RE: atomic operations in user space

[Li Yang-r58472]

> -----Original Message-----
> From: Esben Nielsen [mailto:nielsen.esben@gogglemail.com]
> Sent: Tuesday, August 29, 2006 5:57 PM
> To: Liu Dave-r63238
> Cc: Li Yang-r58472; Esben Nielsen; Xupei Liang;
linuxppc-embedded@ozlabs.org
> Subject: RE: atomic operations in user space
>=20
>=20
>=20
> On Tue, 29 Aug 2006, Liu Dave-r63238 wrote:
>=20
> >>> 2) These mutexes are based on futexes which requires atomic
> >>> operations in userspace. These are available on most
architectures.
> > Look at
> >>> the glibc code in
> >>> nptl/sysdeps/unix/sysv/linux/powerpc/lowlevellock.h for instance.
> >>> Use that and your PPC manual to implement your atomic operations.
> >>
> >> No matter semaphore or futex, it uses system calls to kernel.
>=20
> There is only a system call if there is congestion - that is the whole
idea
> behind the futex.
>=20
> >> And the
> >> true atomic operation is in kernel not user space.
>=20
> "True" atomic operations are available in user space on most
architectures.
>=20
> >> Maybe
> >> it's feasible
> >> for other architectures to do atomic operations directly in
> >> user space.
> >> IMHO, not for powerpc.
>=20
> It is available for PowerPC, but not in POWER and POWER2
instructionsets
> according to http://www.nersc.gov/vendor_docs/ibm/asm/lwarx.htm#idx607
> It is the same in the ARM world: Atomic instructions was introduced in
> ARMv6 I believe. Older ARM processors don't have them.

Yes, I do know there are lwarx and stwrx instructions.  But there is
only one reservation per CPU and reservation can be re-established with
no difference.
So there are possibilities reservation is broken and reserved again in
one atomic block.

Task A			Task B
lwarx
				......
				lwarx
				stwrx

				.....
				.....
				lwarx
stwrx
.....
				stwrx

The addresses of above operations are the same.

In this case Thread A thinks that it is atomic as it holds the same
reservation, but it is actually broken.  Such control flow can't be
prevented in user space.

>=20
> >
> > Are you meaning that we didn't do atomic operations directly in user
> > space
> > on powerpc platform ?
> >
>=20
> Well, that is not the conclusion I get either when reading the glibc
code.
> Try to look at glibc-2.3.5/sysdeps/powerpc/bits/atomic.h.
>=20
> This is by the way probably what the original post in this thread
wanted
> in the first place!
>=20
> Esben
>=20
>=20
> > -DAve

RE: atomic operations in user space

[Esben Nielsen]

On Tue, 29 Aug 2006, Li Yang-r58472 wrote:

>
>> -----Original Message-----
>> From: Esben Nielsen [mailto:nielsen.esben@gogglemail.com]
>> Sent: Tuesday, August 29, 2006 5:57 PM
>> To: Liu Dave-r63238
>> Cc: Li Yang-r58472; Esben Nielsen; Xupei Liang;
> linuxppc-embedded@ozlabs.org
>> Subject: RE: atomic operations in user space
>>
>>
>>
>> On Tue, 29 Aug 2006, Liu Dave-r63238 wrote:
>>
>>>>> 2) These mutexes are based on futexes which requires atomic
>>>>> operations in userspace. These are available on most
> architectures.
>>> Look at
>>>>> the glibc code in
>>>>> nptl/sysdeps/unix/sysv/linux/powerpc/lowlevellock.h for instance.
>>>>> Use that and your PPC manual to implement your atomic operations.
>>>>
>>>> No matter semaphore or futex, it uses system calls to kernel.
>>
>> There is only a system call if there is congestion - that is the whole
> idea
>> behind the futex.
>>
>>>> And the
>>>> true atomic operation is in kernel not user space.
>>
>> "True" atomic operations are available in user space on most
> architectures.
>>
>>>> Maybe
>>>> it's feasible
>>>> for other architectures to do atomic operations directly in
>>>> user space.
>>>> IMHO, not for powerpc.
>>
>> It is available for PowerPC, but not in POWER and POWER2
> instructionsets
>> according to http://www.nersc.gov/vendor_docs/ibm/asm/lwarx.htm#idx607
>> It is the same in the ARM world: Atomic instructions was introduced in
>> ARMv6 I believe. Older ARM processors don't have them.
>
> Yes, I do know there are lwarx and stwrx instructions.  But there is
> only one reservation per CPU and reservation can be re-established with
> no difference.
> So there are possibilities reservation is broken and reserved again in
> one atomic block.
>
> Task A			Task B
> lwarx
> 				......
> 				lwarx
> 				stwrx
>
> 				.....
> 				.....
> 				lwarx
> stwrx
> .....
> 				stwrx
>
> The addresses of above operations are the same.
>
> In this case Thread A thinks that it is atomic as it holds the same
> reservation, but it is actually broken.  Such control flow can't be
> prevented in user space.
>

So you are saying that futexes on powerpc are broken?

Esben

>>
>>>
>>> Are you meaning that we didn't do atomic operations directly in user
>>> space
>>> on powerpc platform ?
>>>
>>
>> Well, that is not the conclusion I get either when reading the glibc
> code.
>> Try to look at glibc-2.3.5/sysdeps/powerpc/bits/atomic.h.
>>
>> This is by the way probably what the original post in this thread
> wanted
>> in the first place!
>>
>> Esben
>>
>>
>>> -DAve
>
> _______________________________________________
> Linuxppc-embedded mailing list
> Linuxppc-embedded@ozlabs.org
> https://ozlabs.org/mailman/listinfo/linuxppc-embedded
>

RE: atomic operations in user space

[Esben Nielsen]

On Tue, 29 Aug 2006, Esben Nielsen wrote:

>
>
> On Tue, 29 Aug 2006, Li Yang-r58472 wrote:
>
>> 
>> >  -----Original Message-----
>> >  From: Esben Nielsen [mailto:nielsen.esben@gogglemail.com]
>> >  Sent: Tuesday, August 29, 2006 5:57 PM
>> >  To: Liu Dave-r63238
>> >  Cc: Li Yang-r58472; Esben Nielsen; Xupei Liang;
>>  linuxppc-embedded@ozlabs.org
>> >  Subject: RE: atomic operations in user space
>> > 
>> > 
>> > 
>> >  On Tue, 29 Aug 2006, Liu Dave-r63238 wrote:
>> > 
>> > > > >  2) These mutexes are based on futexes which requires atomic
>> > > > >  operations in userspace. These are available on most
>>  architectures.
>> > >  Look at
>> > > > >  the glibc code in
>> > > > >  nptl/sysdeps/unix/sysv/linux/powerpc/lowlevellock.h for instance.
>> > > > >  Use that and your PPC manual to implement your atomic operations.
>> > > > 
>> > > >  No matter semaphore or futex, it uses system calls to kernel.
>> > 
>> >  There is only a system call if there is congestion - that is the whole
>>  idea
>> >  behind the futex.
>> > 
>> > > >  And the
>> > > >  true atomic operation is in kernel not user space.
>> > 
>> >  "True" atomic operations are available in user space on most
>>  architectures.
>> > 
>> > > >  Maybe
>> > > >  it's feasible
>> > > >  for other architectures to do atomic operations directly in
>> > > >  user space.
>> > > >  IMHO, not for powerpc.
>> > 
>> >  It is available for PowerPC, but not in POWER and POWER2
>>  instructionsets
>> >  according to http://www.nersc.gov/vendor_docs/ibm/asm/lwarx.htm#idx607
>> >  It is the same in the ARM world: Atomic instructions was introduced in
>> >  ARMv6 I believe. Older ARM processors don't have them.
>>
>>  Yes, I do know there are lwarx and stwrx instructions.  But there is
>>  only one reservation per CPU and reservation can be re-established with
>>  no difference.
>>  So there are possibilities reservation is broken and reserved again in
>>  one atomic block.
>>
>>  Task A			Task B
>>  lwarx
>>      ......
>>      lwarx
>>      stwrx
>>
>>      .....
>>      .....
>>      lwarx
>>  stwrx
>>  .....
>>      stwrx
>>
>>  The addresses of above operations are the same.
>>
>>  In this case Thread A thinks that it is atomic as it holds the same
>>  reservation, but it is actually broken.  Such control flow can't be
>>  prevented in user space.
>> 
>
> So you are saying that futexes on powerpc are broken?

Are you sure the reservation isn't automaticly broken due to the task 
switch?

>
> Esben
>

Esben

>> > 
>> > > 
>> > >  Are you meaning that we didn't do atomic operations directly in user
>> > >  space
>> > >  on powerpc platform ?
>> > > 
>> > 
>> >  Well, that is not the conclusion I get either when reading the glibc
>>  code.
>> >  Try to look at glibc-2.3.5/sysdeps/powerpc/bits/atomic.h.
>> > 
>> >  This is by the way probably what the original post in this thread
>>  wanted
>> >  in the first place!
>> > 
>> >  Esben
>> > 
>> > 
>> > >  -DAve
>>
>>  _______________________________________________
>>  Linuxppc-embedded mailing list
>>  Linuxppc-embedded@ozlabs.org
>>  https://ozlabs.org/mailman/listinfo/linuxppc-embedded
>> 
>
>

RE: atomic operations in user space

[Brent Cook]

> -----Original Message-----
> From:=20
> linuxppc-embedded-bounces+bcook=3Dbpointsys.com@ozlabs.org=20
> [mailto:linuxppc-embedded-bounces+bcook=3Dbpointsys.com@ozlabs.o
rg] On Behalf Of Li Yang-r58472
> Sent: Tuesday, August 29, 2006 5:53 AM
> To: Esben Nielsen; Liu Dave-r63238
> Cc: Xupei Liang; linuxppc-embedded@ozlabs.org
> Subject: RE: atomic operations in user space
>=20
> > It is available for PowerPC, but not in POWER and POWER2
> instructionsets
> > according to=20
> http://www.nersc.gov/vendor_docs/ibm/asm/lwarx.htm#idx607
> > It is the same in the ARM world: Atomic instructions was=20
> introduced in
> > ARMv6 I believe. Older ARM processors don't have them.
>=20
> Yes, I do know there are lwarx and stwrx instructions.  But=20
> there is only one reservation per CPU and reservation can be=20
> re-established with no difference.
> So there are possibilities reservation is broken and reserved=20
> again in one atomic block.
>=20
> Task A			Task B
> lwarx
> 				......
> 				lwarx
> 				stwrx
>=20
> 				.....
> 				.....
> 				lwarx
> stwrx
> .....
> 				stwrx
>=20
> The addresses of above operations are the same.
>=20
> In this case Thread A thinks that it is atomic as it holds=20
> the same reservation, but it is actually broken.  Such=20
> control flow can't be prevented in user space.
>=20

This is exactly how it is supposed to work! That's why there is a loop
in the atomic increment - you check if you still had the reservation
after the transaction by checking the result from the stwcx, and if not,
retry. This works perfectly well no matter if it is from userspace or
the kernel - it is a CPU function. It even works between CPUs on some
later PowerPCs - I've had it working perfectly from userspace on a dual
7448 setup.

PowerPC atomic instructions are not atomic in the sense that they _lock_
anything. The only thing these instructions give you is a way to see if
another thread used the instructions while your thread was using them,
and if you see a conflict, you retry until your code finishes
uninterrupted. This makes it fine for short transactions, like atomic
increment. You probably would not want to use these instructions for
protecting longer transactions of more than a few instructions.

If you look at how, for instance, futexes work in Linux, they rely on
being able to do atomic increment and decrement from _userspace_.

Read this article to understand how it works:
http://www-128.ibm.com/developerworks/library/pa-atom/

 - Brent

RE: atomic operations in user space

[Li Yang]

> This is exactly how it is supposed to work! That's why there is a loop
> in the atomic increment - you check if you still had the reservation
> after the transaction by checking the result from the stwcx, and if not,
> retry.

I surely know all the theories you mentioned clearly.  But please do
look at the case I gave.  Correct me if I missed anything.  Thanks

All the lwarx and stwcx operate on the same address.

> Task A		Task B
> lwarx				
// Get RESERVATION
> 			......
> 			lwarx
> 			stwcx

// RESERVATION cleared
>
> 			.....
> 			.....
> 			lwarx

// Get RESERVATION again
> stwcx				

//Note here: RESERVATION is valid, address is the same.
So the result is commited, no retry for task A

> .....
> 			stwcx
//RESERVATION is cleared, retry atomic op for task B

Please be noted that reservation is only identified by reservation bit
and address operated on.  So different lwarx's on the same address,
may be considered as the same reservation.

RE: atomic operations in user space

[Esben Nielsen]

On Tue, 29 Aug 2006, Li Yang wrote:

>> This is exactly how it is supposed to work! That's why there is a loop
>> in the atomic increment - you check if you still had the reservation
>> after the transaction by checking the result from the stwcx, and if not,
>> retry.
>
> I surely know all the theories you mentioned clearly.  But please do
> look at the case I gave.  Correct me if I missed anything.  Thanks
>
> All the lwarx and stwcx operate on the same address.
>
>> Task A		Task B
>> lwarx
> // Get RESERVATION
>> 			......
>> 			lwarx
>> 			stwcx
>
> // RESERVATION cleared
>>
>> 			.....
>> 			.....
>> 			lwarx
>
> // Get RESERVATION again

Now we do a task switch involving atomic operations, and thus an 
reservation on another address.

>> stwcx
>
> //Note here: RESERVATION is valid, address is the same.
> So the result is commited, no retry for task A
>
>> .....
>> 			stwcx
> //RESERVATION is cleared, retry atomic op for task B
>
> Please be noted that reservation is only identified by reservation bit
> and address operated on.  So different lwarx's on the same address,
> may be considered as the same reservation.
> _______________________________________________
> Linuxppc-embedded mailing list
> Linuxppc-embedded@ozlabs.org
> https://ozlabs.org/mailman/listinfo/linuxppc-embedded
>

Re: atomic operations in user space

[Li Yang]

On 8/30/06, Esben Nielsen <nielsen.esben@gogglemail.com> wrote:
>
>
> On Tue, 29 Aug 2006, Li Yang wrote:
>
> >> This is exactly how it is supposed to work! That's why there is a loop
> >> in the atomic increment - you check if you still had the reservation
> >> after the transaction by checking the result from the stwcx, and if not,
> >> retry.
> >
> > I surely know all the theories you mentioned clearly.  But please do
> > look at the case I gave.  Correct me if I missed anything.  Thanks
> >
> > All the lwarx and stwcx operate on the same address.
> >
> >> Task A               Task B
> >> lwarx
> > // Get RESERVATION
> >>                      ......
> >>                      lwarx
> >>                      stwcx
> >
> > // RESERVATION cleared
> >>
> >>                      .....
> >>                      .....
> >>                      lwarx
> >
> > // Get RESERVATION again
>
> Now we do a task switch involving atomic operations, and thus an
> reservation on another address.
>

This makes sense for me.

RE: atomic operations in user space

[Liu Dave-r63238]

[snip]
> I surely know all the theories you mentioned clearly.  But=20
> please do look at the case I gave.  Correct me if I missed=20
> anything.  Thanks
>=20
> All the lwarx and stwcx operate on the same address.
>=20
> > Task A		Task B
> > lwarx			=09
> // Get RESERVATION
> > 			......
> > 			lwarx
> > 			stwcx
>=20
> // RESERVATION cleared
> >
> > 			.....
> > 			.....
> > 			lwarx
>=20
> // Get RESERVATION again
> > stwcx			=09
>=20
> //Note here: RESERVATION is valid, address is the same.
> So the result is commited, no retry for task A
>=20
> > .....
> > 			stwcx
> //RESERVATION is cleared, retry atomic op for task B
>=20
> Please be noted that reservation is only identified by=20
> reservation bit and address operated on.  So different=20
> lwarx's on the same address, may be considered as the same=20
> reservation.

Is this your reason we cannot do atomic operation in user space?

How about the kernel space? You can image it.
The context switching as above also happen in kernel space,
Why we can do atomic operation in kernel space, not do in user space?

You are assuming the context switching cause the reservation broken.
but we can do atomic operation in kernel space.  The context switching
really is the execption of processor, If we can clear the wrong
RESERVATION
before exception return, I think we can solve this problem. We can dummy
stwcx. before exception return or the processor automaticly clear the
reservation in exception.=20

Are you missing these important things?

-DAve

RE: atomic operations in user space

[Liu Dave-r63238]

> [snip]
> > I surely know all the theories you mentioned clearly.  But=20
> please do=20
> > look at the case I gave.  Correct me if I missed anything.  Thanks
> >=20
> > All the lwarx and stwcx operate on the same address.
> >=20
> > > Task A		Task B
> > > lwarx			=09
> > // Get RESERVATION
> > > 			......
> > > 			lwarx
> > > 			stwcx
> >=20
> > // RESERVATION cleared
> > >
> > > 			.....
> > > 			.....
> > > 			lwarx
> >=20
> > // Get RESERVATION again
> > > stwcx			=09
> >=20
> > //Note here: RESERVATION is valid, address is the same.
> > So the result is commited, no retry for task A
> >=20
> > > .....
> > > 			stwcx
> > //RESERVATION is cleared, retry atomic op for task B
> >=20
> > Please be noted that reservation is only identified by=20
> reservation bit=20
> > and address operated on.  So different lwarx's on the same address,=20
> > may be considered as the same reservation.
>=20
> Is this your reason we cannot do atomic operation in user space?
>=20
> How about the kernel space? You can image it.
> The context switching as above also happen in kernel space,=20
> Why we can do atomic operation in kernel space, not do in user space?
>=20
> You are assuming the context switching cause the reservation broken.
> but we can do atomic operation in kernel space.  The context=20
> switching really is the execption of processor, If we can=20
> clear the wrong RESERVATION before exception return, I think=20
> we can solve this problem. We can dummy stwcx. before=20
> exception return or the processor automaticly clear the=20
> reservation in exception.=20
>=20
> Are you missing these important things?
>=20
> -DAve

I got it. I noticed that all of execption return in kernel did stwcx.
to clear the wrong reserved bit. See the source code.

        .globl  ret_from_except_full
ret_from_except_full:
        REST_NVGPRS(r1)
        /* fall through */

        .globl  ret_from_except
ret_from_except:
           ......

restore:
        lwz     r0,GPR0(r1)
        lwz     r2,GPR2(r1)
        REST_4GPRS(3, r1)
        REST_2GPRS(7, r1)

        lwz     r10,_XER(r1)
        lwz     r11,_CTR(r1)
        mtspr   SPRN_XER,r10
        mtctr   r11

        PPC405_ERR77(0,r1)
        stwcx.  r0,0,r1                 /* to clear the reservation */

          ......

         rfi

-DAve

 =20

RE: atomic operations in user space

[Li Yang-r58472]

> -----Original Message-----
> From: Liu Dave-r63238
> Sent: Wednesday, August 30, 2006 10:27 AM
> To: Liu Dave-r63238; Li Yang-r58472; linuxppc-embedded@ozlabs.org;
> linuxppc-dev@ozlabs.org
> Subject: RE: atomic operations in user space
>=20
> > [snip]
> > > I surely know all the theories you mentioned clearly.  But
> > please do
> > > look at the case I gave.  Correct me if I missed anything.  Thanks
> > >
> > > All the lwarx and stwcx operate on the same address.
> > >
> > > > Task A		Task B
> > > > lwarx
> > > // Get RESERVATION
> > > > 			......
> > > > 			lwarx
> > > > 			stwcx
> > >
> > > // RESERVATION cleared
> > > >
> > > > 			.....
> > > > 			.....
> > > > 			lwarx
> > >
> > > // Get RESERVATION again
> > > > stwcx
> > >
> > > //Note here: RESERVATION is valid, address is the same.
> > > So the result is commited, no retry for task A
> > >
> > > > .....
> > > > 			stwcx
> > > //RESERVATION is cleared, retry atomic op for task B
> > >
> > > Please be noted that reservation is only identified by
> > reservation bit
> > > and address operated on.  So different lwarx's on the same
address,
> > > may be considered as the same reservation.
> >
> > Is this your reason we cannot do atomic operation in user space?
> >
> > How about the kernel space? You can image it.
> > The context switching as above also happen in kernel space,
> > Why we can do atomic operation in kernel space, not do in user
space?
> >
> > You are assuming the context switching cause the reservation broken.
> > but we can do atomic operation in kernel space.  The context
> > switching really is the execption of processor, If we can
> > clear the wrong RESERVATION before exception return, I think
> > we can solve this problem. We can dummy stwcx. before
> > exception return or the processor automaticly clear the
> > reservation in exception.
> >
> > Are you missing these important things?
> >
> > -DAve
>=20
> I got it. I noticed that all of execption return in kernel did stwcx.
> to clear the wrong reserved bit. See the source code.
>=20
>         .globl  ret_from_except_full
> ret_from_except_full:
>         REST_NVGPRS(r1)
>         /* fall through */
>=20
>         .globl  ret_from_except
> ret_from_except:
>            ......
>=20
> restore:
>         lwz     r0,GPR0(r1)
>         lwz     r2,GPR2(r1)
>         REST_4GPRS(3, r1)
>         REST_2GPRS(7, r1)
>=20
>         lwz     r10,_XER(r1)
>         lwz     r11,_CTR(r1)
>         mtspr   SPRN_XER,r10
>         mtctr   r11
>=20
>         PPC405_ERR77(0,r1)
>         stwcx.  r0,0,r1                 /* to clear the reservation */

Ya, you found the point.  There is no problem for me about this
question.

- Leo