PowerPC reservations

PowerPC reservations

[Kalle Pokki]

Hi,

Can someone please help me understand how the memory reservations in 
PowerPC actually work. Let's just assume uniprocessor with a pre-emptive 
scheduler, and take a text-book example of an atomic increment case, 
which is also frequently used in e.g. the Linux kernel. With two atomic 
operations, everything seems to be just fine. But how about with three 
concurrent threads of execution?

 From the following code, assume r3 contains the same address for each 
incrementing operation. If the first atomic increment is pre-empted  in 
the middle, execution then jumps to the second increment (by the 
scheduler). The second increment runs through and succeeds, and 
continues straight to the third increment. Then it is again pre-empted 
in the middle, execution returning to the first increment. Now the 
processor has the reservation with the correct address, and the first 
increment succeeds when still holding the original input value. The 
first and the second increment thus write the same value in memory. 
After the first increment, the scheduler again continues the third 
increment, which doesn't succeed a first, but the second round succeeds. 
However, the value in the address pointed by r3 was not increased by 
three, but by two.

Am I just not getting how this is really supposed to work? Are there 
still some other constructs in use to prevent this, e.g. extra stwcx. 
instructions when changing the thread of execution?

I'm also wondering why the architecture specifically defines the stwcx. 
instruction to have, well, undefined behavior in case the reservation 
address differs from the address of the previous lwarx...

1:      lwarx r6, r0, r3
        addi r6, r6, 1
        stwcx. r6, r0, r3
        bne- 1b

.....

2:      lwarx r7, r0, r3
        addi r7, r7, 1
        stwcx. r7, r0, r3
        bne- 2b

3:      lwarx r8, r0, r3
        addi r8, r8, 1
        stwcx. r8, r0, r3
        bne- 3b

Re: PowerPC reservations

[Michael R. Zucca]

Kalle Pokki wrote:

> Am I just not getting how this is really supposed to work? Are there 
> still some other constructs in use to prevent this, e.g. extra stwcx. 
> instructions when changing the thread of execution?

Yes. When a context switch occurs, there's a stcwx. to clear any 
reservations from the previous thread. When you enter another thread 
from a context switch, you enter it with no reservations pending.

So you might have something that looks like this:

thread 1 -> lwarx w/ successful reservation
context switch -> stwcx.. to garbage location to clear any reservations
thread 2 -> stwcx. fails since there's no reservation pending
context switch -> stwcx. to garbage location to clear any reservations
thread 1 -> stwcx. fails since there's no reservation pending
thread 1 -> loops to try lwarx again

So the context switcher protects a reservation from "leaking" into 
another thread. In that sense, you can act as though lwarx/stwcx. pairs 
belong to a single thread.

The other important thing to remember is that there can only be one 
outstanding reservation. Thus, lwarx and stcwx. instructions have to be 
"paired" or you're going to get unexpected results.