gettimeofday() vsyscall for kvm-clock?

gettimeofday() vsyscall for kvm-clock?

[Andrew Theurer]

Wondering if a user-space gettimofday() for kvm-clock has been 
considered before.  I am seeing a pretty large difference in performance 
between tsc and kvm-clock.  I have to assume at least some of this is 
due to the mode switch for kvm-clock.  Here are the results:

(this is a 16 vCPU VM on a 16 thread 2S Nehalem-EP host, looped 
gettimeofday() calls on all vCPUs)

tsc:		.0645 usec per call
kvm-clock: 	.4222 usec per call (6.54x)


-Andrew Theurer

Re: gettimeofday() vsyscall for kvm-clock?

[Marcelo Tosatti]

On Mon, May 21, 2012 at 03:26:54PM -0500, Andrew Theurer wrote:
> Wondering if a user-space gettimofday() for kvm-clock has been
> considered before.  I am seeing a pretty large difference in
> performance between tsc and kvm-clock.  I have to assume at least
> some of this is due to the mode switch for kvm-clock.  Here are the
> results:
> 
> (this is a 16 vCPU VM on a 16 thread 2S Nehalem-EP host, looped
> gettimeofday() calls on all vCPUs)
> 
> tsc:		.0645 usec per call
> kvm-clock: 	.4222 usec per call (6.54x)
> 
> 
> -Andrew Theurer

https://bugzilla.redhat.com/show_bug.cgi?id=679207

"model name : Intel(R) Xeon(R) CPU           E5540  @ 2.53GHz

native, gettimeofday (vsyscall): 45ns
guest, kvmclock (syscall): 198ns"

But this was before 

commit 489fb490dbf8dab0249ad82b56688ae3842a79e8
Author: Glauber Costa <glommer@redhat.com>
Date:   Tue May 11 12:17:40 2010 -0400

    x86, paravirt: Add a global synchronization point for pvclock

(see the full changelog for details).

Can you try disabling the global variable, to see if that makes 
a difference (should not be enabled in production)? Untested patch
(against guest kernel) below 

diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c
index 086eb58..7347a31 100644
--- a/arch/x86/kernel/kvmclock.c
+++ b/arch/x86/kernel/kvmclock.c
@@ -235,6 +235,6 @@ void __init kvmclock_init(void)
 	pv_info.paravirt_enabled = 1;
 	pv_info.name = "KVM";
 
-	if (kvm_para_has_feature(KVM_FEATURE_CLOCKSOURCE_STABLE_BIT))
+	//if (kvm_para_has_feature(KVM_FEATURE_CLOCKSOURCE_STABLE_BIT))
 		pvclock_set_flags(PVCLOCK_TSC_STABLE_BIT);
 }

Also, how is UP gettimeofday performance?

Re: gettimeofday() vsyscall for kvm-clock?

[Andrew Theurer]

On 05/21/2012 03:36 PM, Marcelo Tosatti wrote:
> On Mon, May 21, 2012 at 03:26:54PM -0500, Andrew Theurer wrote:
>> Wondering if a user-space gettimofday() for kvm-clock has been
>> considered before.  I am seeing a pretty large difference in
>> performance between tsc and kvm-clock.  I have to assume at least
>> some of this is due to the mode switch for kvm-clock.  Here are the
>> results:
>>
>> (this is a 16 vCPU VM on a 16 thread 2S Nehalem-EP host, looped
>> gettimeofday() calls on all vCPUs)
>>
>> tsc:		.0645 usec per call
>> kvm-clock: 	.4222 usec per call (6.54x)
>>
>>
>> -Andrew Theurer
>
> https://bugzilla.redhat.com/show_bug.cgi?id=679207
>
> "model name : Intel(R) Xeon(R) CPU           E5540  @ 2.53GHz
>
> native, gettimeofday (vsyscall): 45ns
> guest, kvmclock (syscall): 198ns"
>
> But this was before
>
> commit 489fb490dbf8dab0249ad82b56688ae3842a79e8
> Author: Glauber Costa<glommer@redhat.com>
> Date:   Tue May 11 12:17:40 2010 -0400
>
>      x86, paravirt: Add a global synchronization point for pvclock
>
> (see the full changelog for details).
>
> Can you try disabling the global variable, to see if that makes
> a difference (should not be enabled in production)? Untested patch
> (against guest kernel) below

The following was re-done on a 3.4 guest kernel (previously RHEL kernel):

1-way:
   tsc:		.0315
   kvm-clock:	.2112 (6.7x)

16-way:
   tsc:		.0432
   kvm-clock: 	.4825 (11.1x)

Now with global var disabled:

16-way:
   kvm-clock:	.4628

Does not look like much of a difference.

-Andrew