Re: spinaphore conceptual draft

[Andi Kleen <ak@muc.de>]

Chris Friesen <cfriesen@nortel.com> writes:

> Andi Kleen wrote:
>> On Mon, May 30, 2005 at 08:52:13AM -0600, Chris Friesen wrote:
>
>>>What about rdtsc?
>
>> It fails the reliable and monotonic test on AMD;
>
> tsc goes backwards on AMD?  Under what circumstances (I'm curious,
> since I'm running one...)

It is not synchronized between CPUs and slowly drifts and can even run
at completely different frequencies there when you use powernow! on a
MP system. 

It can be used reliably when you only do deltas on same CPU
and correct for the changing frequency. However then you run
into other problems, like it being quite slow on Intel.

I suspect any attempt to use time stamps in locks is a bad
idea because of this.

Note that at least on modern Intel systems you can just use
MONITOR/MWAIT which is much more efficient, if you are willing to eat
the fake wakeups due to the cache line padding they use.

My impression is that the aggressive bus access avoidance the
original poster was trying to implement is not that useful
on modern systems anyways which have fast busses. Also 
it is not even clear it even saves anything; after all the
CPU will always snoop cache accesses for all cache lines 
and polling for the EXCLUSIVE transition of the local cache line
is probably either free or very cheap.

Optimizing for the congested case is always a bad idea anyways; in
case lock congestion is a problem it is better to fix the lock.  And
when you have a really congested lock that for some reason cannot be
fixed then using some kind of queued lock (which also gives you
fairness on NUMA) is probably a better idea. But again you should
really fix the the lock instead, everything else is the wrong answer.

BTW Getting the fairness on the backoff scheme right would have been
probably a major problem.

The thing that seems to tickle CPU vendors much more is to avoid
wasting CPU time on SMT or on Hypervisors while spinning.  That can be
all done with much simpler hints (like rep ; nop).

-Andi