LDREX/STREX and pre-emption on SMP hardware

[jamie@shareable.org (Jamie Lokier)]

Catalin Marinas wrote:
> With interrupts (I1, I2 interrupt handlers)
> 
> I1			I2
> LDREX
> 			LDREX
> 			STREX (succeeds)
> STREX (fails)
> 
> In the interrupt case, they are nested so the STREX in I2 is always
> executed before STREX in I1 (you can extrapolate with several nested
> interrupts).

This assumes LDREX/STREX are always called in pairs.  But this is in
fact _not_ the case.  Take a look at atomic_cmpxchg:

	do {
		__asm__ __volatile__("@ atomic_cmpxchg\n"
		"ldrex	%1, [%2]\n"
		"mov	%0, #0\n"
		"teq	%1, %3\n"
		"strexeq %0, %4, [%2]\n"
		    : "=&r" (res), "=&r" (oldval)
		    : "r" (&ptr->counter), "Ir" (old), "r" (new)
		    : "cc");
	} while (res);

In the case where ptr->counter != old, STREX is not executed, and the
do{...}while loop does not loop.  Thus LDREX/STREX aren't paired.

It may be that atomic_cmpxchg() is always called in a loop, but if so
I don't think that's a documented requirement for all callers.

A simple solution is to either call CLREXNE after STREXEQ, or change
STREXEQ to STREX and change the logic so that if ptr->counter != old,
the value it tries to store is what it read.  The latter uses fewer
instructions - and even eliminates the do...while, making it really
compact, but may cause more cache line dirtying.

If you think I'm right and tell me what you prefer I'll prepare a
patch.

-- Jamie