VDSO gettimeofday() x86_64 linux 3.2

VDSO gettimeofday() x86_64 linux 3.2

[wilson self]

I noticed recently that my get time of day calls are quite a bit
slower than I would have expected; and it would see that a likely
cause is that they are not going via vdso, but rather a system call.

In older kernels there was kernel.vsyscall64 parameter to enable them,
but this seems to be gone in later kernel releases.  What config
option or parameter might I be missing that could cause gettimeofday()
to use the fallback methods?

Thanks
wilson

Re: VDSO gettimeofday() x86_64 linux 3.2

[Andy Lutomirski]

On 10/16/2012 11:09 AM, wilson self wrote:
> I noticed recently that my get time of day calls are quite a bit
> slower than I would have expected; and it would see that a likely
> cause is that they are not going via vdso, but rather a system call.
> 
> In older kernels there was kernel.vsyscall64 parameter to enable them,
> but this seems to be gone in later kernel releases.  What config
> option or parameter might I be missing that could cause gettimeofday()
> to use the fallback methods?

That option is gone because it's treated as unconditionally true.  What
does /sys/devices/system/clocksource/clocksource0/current_clocksource
say?  The usual cause of this problem is a clocksource other than tsc or
hpet.  hpet is amazingly slow, though, and arguably the hpet vdso code
could be deleted because the syscall overhead is the least of your
worries if you're using the hpet for timing.

(Also, if you're doing the call yourself instead of using libc, make
sure you're using the vdso and not the old vsyscall.  The vsyscall (call
direct to 0xffffffffff600000) is the slowest possible way to read the
time -- it's several times slower than the syscall on my box.  This is
mostly intentional.

--Andy

Re: VDSO gettimeofday() x86_64 linux 3.2

[wilson self]

current_clocksource is tsc.

the entire source of the test application:
---
#include <stdio.h>
#include <stdlib.h>
#include <sys/time.h>
#include <time.h>

int main() {
      struct timeval tim;
      gettimeofday(&tim, NULL);
      printf("%.6lf seconds\n", tim.tv_sec+tim.tv_usec/1000000.0);
}
---
nothing fancy going on here.

thanks for the help.

On Tue, Oct 16, 2012 at 2:24 PM, Andy Lutomirski <luto@amacapital.net> wrote:
> On 10/16/2012 11:09 AM, wilson self wrote:
>> I noticed recently that my get time of day calls are quite a bit
>> slower than I would have expected; and it would see that a likely
>> cause is that they are not going via vdso, but rather a system call.
>>
>> In older kernels there was kernel.vsyscall64 parameter to enable them,
>> but this seems to be gone in later kernel releases.  What config
>> option or parameter might I be missing that could cause gettimeofday()
>> to use the fallback methods?
>
> That option is gone because it's treated as unconditionally true.  What
> does /sys/devices/system/clocksource/clocksource0/current_clocksource
> say?  The usual cause of this problem is a clocksource other than tsc or
> hpet.  hpet is amazingly slow, though, and arguably the hpet vdso code
> could be deleted because the syscall overhead is the least of your
> worries if you're using the hpet for timing.
>
> (Also, if you're doing the call yourself instead of using libc, make
> sure you're using the vdso and not the old vsyscall.  The vsyscall (call
> direct to 0xffffffffff600000) is the slowest possible way to read the
> time -- it's several times slower than the syscall on my box.  This is
> mostly intentional.
>
> --Andy

Re: VDSO gettimeofday() x86_64 linux 3.2

[Andy Lutomirski]

On Tue, Oct 16, 2012 at 12:33 PM, wilson self <wself00@gmail.com> wrote:
> current_clocksource is tsc.
>
> the entire source of the test application:
> ---
> #include <stdio.h>
> #include <stdlib.h>
> #include <sys/time.h>
> #include <time.h>
>
> int main() {
>       struct timeval tim;
>       gettimeofday(&tim, NULL);
>       printf("%.6lf seconds\n", tim.tv_sec+tim.tv_usec/1000000.0);
> }
> ---
> nothing fancy going on here.
>
> thanks for the help.

There's lots of hidden fanciness.  What glibc version are you using
and how are you linking to glibc?  All but the newest glibc versions
use the old vsyscall when statically linked.  The newest versions use
the syscall instead.  If you're dynamically linking, then gettimeofday
should be okay even with fairly old glibc versions.

(FWIW, glibc maintainership has changed since this was decided.  Feel
free to bug the new maintainers if you care about statically-linked
performance.  The only other runtime library I know of that uses the
vdso or vsyscalls is Go, and the newest versions do the right thing.)

--Andy

Re: VDSO gettimeofday() x86_64 linux 3.2

[wilson self]

I am just using gcc timetest.c -o timetest

Should be dynamically linked. glibc is 2.5, which is quite old, but I
think this should still work with it, no?

On Tue, Oct 16, 2012 at 2:39 PM, Andy Lutomirski <luto@amacapital.net> wrote:
> On Tue, Oct 16, 2012 at 12:33 PM, wilson self <wself00@gmail.com> wrote:
>> current_clocksource is tsc.
>>
>> the entire source of the test application:
>> ---
>> #include <stdio.h>
>> #include <stdlib.h>
>> #include <sys/time.h>
>> #include <time.h>
>>
>> int main() {
>>       struct timeval tim;
>>       gettimeofday(&tim, NULL);
>>       printf("%.6lf seconds\n", tim.tv_sec+tim.tv_usec/1000000.0);
>> }
>> ---
>> nothing fancy going on here.
>>
>> thanks for the help.
>
> There's lots of hidden fanciness.  What glibc version are you using
> and how are you linking to glibc?  All but the newest glibc versions
> use the old vsyscall when statically linked.  The newest versions use
> the syscall instead.  If you're dynamically linking, then gettimeofday
> should be okay even with fairly old glibc versions.
>
> (FWIW, glibc maintainership has changed since this was decided.  Feel
> free to bug the new maintainers if you care about statically-linked
> performance.  The only other runtime library I know of that uses the
> vdso or vsyscalls is Go, and the newest versions do the right thing.)
>
> --Andy

Re: VDSO gettimeofday() x86_64 linux 3.2

[Andy Lutomirski]

On Tue, Oct 16, 2012 at 12:43 PM, wilson self <wself00@gmail.com> wrote:
> I am just using gcc timetest.c -o timetest
>
> Should be dynamically linked. glibc is 2.5, which is quite old, but I
> think this should still work with it, no?

You need at least glibc 2.7 for this to work well on modern kernels.
FWIW, glibc 2.7 is five years old.

--Andy