Re: AMD X2 unsynced TSC fix?

[Willy Tarreau <w@1wt.eu>]

On Sat, Oct 28, 2006 at 11:37:22AM -0700, Andi Kleen wrote:
> On Friday 27 October 2006 22:28, Willy Tarreau wrote:
> > On Fri, Oct 27, 2006 at 11:28:00PM -0400, Lee Revell wrote:
> > > On Fri, 2006-10-27 at 18:04 -0700, Andi Kleen wrote:
> > > > I don't think it makes too much sense to hack on pure RDTSC when
> > > > gtod is fast enough -- RDTSC will be always icky and hard to use.
> > >
> > > I agree FWIW, our application would be happy to just use gtod if it
> > > wasn't so slow on these machines.
> >
> > Agreed, I had to turn about 20 dual-core servers to single core because
> > the only way to get a monotonic gtod made it so slow that it was not
> > worth using a dual-core. 
> 
> Curious - what workload was that? 

Two different but related workloads :
  - load balancer doing between 10 and 100k gtod per second on a sun
    x2100 under RHEL 3. HPET was not available and the only way I found
    to get monotonic clock was to use the APIC timer IIRC (it was more
    than 6 months ago, so sorry if I don't remember about all the details).

  - network sniffer that I tried to tune to get the highest possible packet
    rates on gigabit ethernet.

> While gtod is time critical and often appears high on profile lists it is 
> normally not as time critical as you're claiming it is; especially not
> time critical enough to warrant such radical action.

Yes it was, because the small gain of using a dual core with such
a workload was clearly lost by that change. IIRC, I reached 25000
sessions/s on dual core with TSC if I didn't care about the clock,
20000 without TSC, and 18000 on single core+TSC. But with the sniffer,
it was even worse : I had 500 kpps in dual-core+TSC, 70kpps without
TSC and 300 kpps with single-core+TSC. Since I had to buy the same
machines for both uses, this last argument was enough for me to stick
to a single core.

> > I initially considered buying one dual-core 
> > AMD for my own use, but after seeing this, I'm definitely sure I won't
> > ever buy one as long as this problem is not fixed, as it causes too
> > many problems.
> 
> It's somewhat slower, but I'm not sure what "too many problems" you're
> refering to.

Anticipated or delayed timeouts on the proxy, time measurement errors
(when the logs show that a session finishes before it begins, there's
a real problem, particularly because we use those logs for troubleshooting).
And for the sniffer, getting wrong times by about 2s was a real problem too.
I would have preferred to get something monotonic with little accuracy than
out of order packets !

This is definitely a design problem on those chips, probably because
marketting targets gamers only. And that's very sad, because they are
excellent processors !

> -Andi

Regards,
Willy