From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jeremy Fitzhardinge Subject: Re: rdtsc in userspace Date: Fri, 23 Oct 2009 17:59:17 -0700 Message-ID: <4AE25165.9030904@goop.org> References: Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Dan Magenheimer Cc: kurt.hackel@oracle.com, Avi Kivity , "Xen-Devel (E-mail)" , Keir Fraser , Tim Deegan List-Id: xen-devel@lists.xenproject.org On 10/23/09 15:51, Dan Magenheimer wrote: > In measuring rdtsc usage in the kernel, both Jeremy > and I noticed that compiling the kernel causes a > large number of userland rdtsc's. At first I thought > that this meant that gcc was using rdtsc, but gcc > sources do not show any use of rdtsc. Next I suspected > bash, but it doesn't either. I think the dynamic linker may use rdtsc as a piece of randomness for randomizing the addresses of libraries and other mappings. > Finally, it appears that > the calls are occurring from glibc, and searching for > uses of rdtsc, I found that glibc has its > own implementation of clock_gettime that uses rdtsc > directly! > When I run clock_gettime on my system it uses the vsyscall version (which is either a syscall or, with my recent patches, all in usermode). Using strace on your test program should show whether its really bypassing the syscall or not. > The manpage for clock_gettime ("man 3 clock_gettime") > has a lengthy caveat about using clock_gettime on > an SMP system BUT provides a mechanism to test to > see if your SMP system is safe, clock_getcpuclockid(0). > My manpages don't have anything like that for clock_gettime(). > See for example (near the end): > > http://linux.die.net/man/3/clock_gettime > That caveat is for CLOCK_PROCESS_CPUTIME_ID which doesn't seem reasonable to count on for monotonicity (my manpages CLOCK_PROCESS_CPUTIME_ID just refer to as "High resolution per-process timer."). CLOCK_REALTIME or CLOCK_MONOTONIC is a different matter. > But testing this on a system I know is unsafe, the > test says my system is safe! Looking at the source > code, I can see why... the test is a compile-time > test, not a run-time test, and on x86 it appears that > it will always pass. > > So I wrote a simple test program that uses clock_gettime(). > It does indeed do userland rdtsc's. > Using what CLOCK_? > So... any program that uses clock_gettime to approximate > the passage of time is subject to break on Xen across > a migration. Even if the programmer uses the prescribed > method for testing "safeness", it is subject to break. > I always see clock_gettime(CLOCK_MONOTONIC or CLOCK_REALTIME, ...) make a syscall or vsyscall. If you're testing any of the thread-local times then I think its less interesting. J