high res timestamps and SMP

high res timestamps and SMP

[Chris Friesen]

We have a requirement to have high-res timestamps available on SMP systems.

Assuming that we are running identical cpus, is a sync-up at boot time 
enough to give usable time values, or do I need to do force periodic 
re-syncs?

We're currently looking at MIPS, x86 (Xeons), and PPC.

Thanks,

Chris


-- 
Chris Friesen                    | MailStop: 043/33/F10
Nortel Networks                  | work: (613) 765-0557
3500 Carling Avenue              | fax:  (613) 765-2986
Nepean, ON K2H 8E9 Canada        | email: cfriesen@nortelnetworks.com

Re: high res timestamps and SMP

[john stultz]

On Wed, 2003-11-19 at 14:40, Chris Friesen wrote:
> We have a requirement to have high-res timestamps available on SMP systems.
> 
> Assuming that we are running identical cpus, is a sync-up at boot time 
> enough to give usable time values, or do I need to do force periodic 
> re-syncs?

If the cpus (or their time stamp counter) are all driven by the same
signal and you do not suffer from NUMA effects, then syncing them should
be enough. 

However, if you suffer from NUMA effects, or if the counters are not
driven off the same signal, its likely you could run into problems. 

> We're currently looking at MIPS, x86 (Xeons), and PPC.

o No clue on MIPS.

o The x86 TSC is a horrible time source, but may work well enough on
simple SMP systems. 

o PPC has a nice in-cpu time-base register (ppc folks, feel free to
smack or correct me on this) which is driven off the bus-clock and is
synced in hardware. 


good luck!
-john

Re: high res timestamps and SMP

[Mikael Pettersson]

john stultz writes:
 > On Wed, 2003-11-19 at 14:40, Chris Friesen wrote:
 > > We have a requirement to have high-res timestamps available on SMP systems.
...
 > o PPC has a nice in-cpu time-base register (ppc folks, feel free to
 > smack or correct me on this) which is driven off the bus-clock and is
 > synced in hardware. 

Last time I checked, all 32-bit PowerPC chips ran that clock at 1/4
of the bus clock speed.

That may be adequate for time-of-day and I/O delays and such, but
it's worthless for timestamps or performance measurements.

It's possible to count core clocks via a performance counter, but
I don't know if they are synced between CPUs.

Re: high res timestamps and SMP

[Anton Blanchard]

Hi,

> Last time I checked, all 32-bit PowerPC chips ran that clock at 1/4
> of the bus clock speed.

It depends on the chip, on recent ppc64 boxes its 1/8 of the processor
clock speed.

> That may be adequate for time-of-day and I/O delays and such, but
> it's worthless for timestamps or performance measurements.

Can you show me the application that depends on timestamps being that
accurate?

Running the multiplier at a set fraction of the processor speed
is a good idea I think. Go look at any large x86 box (and possibly ia64
box) and you will find the timebases are not synced. Even our biggest
box has the timebase synced, its easy to do when the timebase is running
at a reasonable rate.

In a trade off between unsynced timebases and timebase running at a
fraction of the cpu speed, ill take the latter :) (Ask the x86 guys what
a pain unsynced timebases are)

Anton

Re: high res timestamps and SMP

[Chris Friesen]

Anton Blanchard wrote:

> Running the multiplier at a set fraction of the processor speed
> is a good idea I think. Go look at any large x86 box (and possibly ia64
> box) and you will find the timebases are not synced.

By "large", you mean NUMA, right?  I was under the impression that the 
kernel did sync up the timebases for SMP.

Chris


-- 
Chris Friesen                    | MailStop: 043/33/F10
Nortel Networks                  | work: (613) 765-0557
3500 Carling Avenue              | fax:  (613) 765-2986
Nepean, ON K2H 8E9 Canada        | email: cfriesen@nortelnetworks.com