Re: [gcv v3 27/35] arm: Replace __get_cpu_var uses

[Will Deacon <will.deacon@arm.com>]

On Wed, Sep 04, 2013 at 03:17:09PM +0100, Christoph Lameter wrote:
> On Wed, 4 Sep 2013, Will Deacon wrote:
> > God knows! You're completely right, and we simply disable interrupts which I
> > somehow misread as taking a lock. However, is it guaranteed that mixing
> > an atomic64_* access with a this_cpu_inc_return will retain atomicity
> > between the two? E.g. if you get interrupted during an atomic64_xchg
> > operation, the interrupt handler issues this_cpu_inc_return, then on return
> > to the xchg operation it must reissue any reads that had been executed
> > prior to the interrupt. This should work on ARM/ARM64 (returning from the
> > interrupt will clear the exclusive monitor) but I don't know about other
> > architectures.
> 
> You cannot get interrupted during an atomic64_xchg operation. atomic and
> this_cpu operations are stricly serialzed since both should be behaving
> like single instructions. __this_cpu ops relax that requirement in case
> the arch code incurs significant overhead to make that happen. In cases
> where we know that preemption/interrupt disable etc takes care of things
> __this_cpu ops come into play.

Hmm, why can't you get interrupted during atomic64_xchg? On ARM, we have the
following sequence:

  static inline u64 atomic64_xchg(atomic64_t *ptr, u64 new)
  {
  	u64 result;
  	unsigned long tmp;
  
  	smp_mb();
  
  	__asm__ __volatile__("@ atomic64_xchg\n"
  "1:	ldrexd	%0, %H0, [%3]\n"
  "	strexd	%1, %4, %H4, [%3]\n"
  "	teq	%1, #0\n"
  "	bne	1b"
  	: "=&r" (result), "=&r" (tmp), "+Qo" (ptr->counter)
  	: "r" (&ptr->counter), "r" (new)
  	: "cc");
  
  	smp_mb();
  
  	return result;
  }

which relies on interrupts clearing the exclusive monitor to force us back
around the loop in the inline asm. I could imagine other architectures doing
similar, but only detecting the other writer if it used the same
instructions.

Will