Re: rdtsc in userspace

[Jeremy Fitzhardinge <jeremy@goop.org>]

On 10/23/09 15:51, Dan Magenheimer wrote:
> In measuring rdtsc usage in the kernel, both Jeremy
> and I noticed that compiling the kernel causes a
> large number of userland rdtsc's.  At first I thought
> that this meant that gcc was using rdtsc, but gcc
> sources do not show any use of rdtsc.  Next I suspected
> bash, but it doesn't either.

I think the dynamic linker may use rdtsc as a piece of randomness for
randomizing the addresses of libraries and other mappings.

>   Finally, it appears that
> the calls are occurring from glibc, and searching for
> uses of rdtsc, I found that glibc has its
> own implementation of clock_gettime that uses rdtsc
> directly!
>   

When I run clock_gettime on my system it uses the vsyscall version
(which is either a syscall or, with my recent patches, all in usermode).

Using strace on your test program should show whether its really
bypassing the syscall or not.

> The manpage for clock_gettime ("man 3 clock_gettime")
> has a lengthy caveat about using clock_gettime on
> an SMP system BUT provides a mechanism to test to
> see if your SMP system is safe, clock_getcpuclockid(0).
>   

My manpages don't have anything like that for clock_gettime().

> See for example (near the end):
>
> http://linux.die.net/man/3/clock_gettime
>   

That caveat is for CLOCK_PROCESS_CPUTIME_ID which doesn't seem
reasonable to count on for monotonicity (my manpages
CLOCK_PROCESS_CPUTIME_ID just refer to as "High resolution per-process
timer.").   CLOCK_REALTIME or CLOCK_MONOTONIC is a different matter.

> But testing this on a system I know is unsafe, the
> test says my system is safe!  Looking at the source
> code, I can see why... the test is a compile-time
> test, not a run-time test, and on x86 it appears that
> it will always pass.
>
> So I wrote a simple test program that uses clock_gettime().
> It does indeed do userland rdtsc's.
>   

Using what CLOCK_?

> So... any program that uses clock_gettime to approximate
> the passage of time is subject to break on Xen across
> a migration.  Even if the programmer uses the prescribed
> method for testing "safeness", it is subject to break.
>   

I always see clock_gettime(CLOCK_MONOTONIC or CLOCK_REALTIME, ...) make
a syscall or vsyscall.  If you're testing any of the thread-local times
then I think its less interesting.

    J