Re: Whats the purpose of get_cycles_sync()

[Vojtech Pavlik <vojtech@suse.cz>]

On Tue, Oct 30, 2007 at 09:21:02PM +0100, Andi Kleen wrote:
> "Joerg Roedel" <joerg.roedel@amd.com> writes:
> 
> > I would like to answer what the special purpose of the get_cycles_sync()
> > function is in the x86 architecture. In special I ask myself why
> > this function has to be *sync*?
> 
> Vojtech had one test that tested time monotonicity over CPUs 
> and it constantly failed until we added the CPUID on K8 C stepping. 
> He can give details on the test.
> 
> I suspect the reason was because the CPU reordered the RDTSCs so that
> a later RDTSC could return a value before an earlier one. This can
> happen because gettimeofday() is so fast that a tight loop calling it can
> fit more than one iteration into the CPU's reordering window.

The K8's still guarantee that subsequent RDTSCs return increasing
values, even if the processor reorders them.

What could have been happening then was that the RDTSC instruction might
have been reordered by the CPU out of the seqlock, causing trouble in
the calculation.

Anyway, adding the CPUID didn't solve all the problems we've seen back
then, and so far none of the approaches for using TSC without acquiring
a spinlock on multi-socket AMD boxes worked 100% correctly.

> That is why newer kernels use RDTSCP if available which doesn't need
> to be intercepted and is synchronous.  And since all AMD SVM systems
> have RDTSCP they are fine.
> 
> On Intel Core2 without RDTSCP the CPUID can be still intercepted right
> now, but the real fix there is to readd FEATURE_SYNC_TSC for Core2 --
> the RDTSC there is always monotonic per CPU and the patch that changed
> that (f3d73707a1e84f0687a05144b70b660441e999c7) was bogus and must be
> reverted. I didn't catch that in time unfortunately.

-- 
Vojtech Pavlik
Director SuSE Labs