[PATCH 0/4] kvmclock fixes

[PATCH 0/4] kvmclock fixes

[Glauber Costa]

Hi,

With this series, I'm introducing the ability to turn off the kvmclock.
It is done by writting any thing to the first three bits of the address
passed as a parameter to the msr.

For that to work properly, we should make absolutely sure the guest is aligned.
If we really care about compatibility here, we can set the KVM_CAP_CLOCKSOURCE 
to 0, and define a new feature flag to mean that the clock data structures
should be aligned. However, as the guest part has not yet made its way to linus,
we're in a privileged position of having no users other than the experimental users,
and can change it withouth bothering so much.

/* Comments? */



-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

[PATCH 1/4] [PATCH] kvmclock: release time_page if msr is rewritten

[Glauber Costa]

If the calling cpu rewrites the system clock msr for any reason,
release the page we allocated in the last time

Signed-off-by: Glauber Costa <gcosta@redhat.com>
---
 arch/x86/kvm/x86.c |    3 +++
 1 files changed, 3 insertions(+), 0 deletions(-)

diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index e65a9d6..6abd784 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -588,6 +588,9 @@ int kvm_set_msr_common(struct kvm_vcpu *
 		kvm_write_wall_clock(vcpu->kvm, data);
 		break;
 	case MSR_KVM_SYSTEM_TIME: {
+		if (vcpu->arch.time_page)
+			kvm_release_page_dirty(vcpu->arch.time_page);
+
 		vcpu->arch.time = data & PAGE_MASK;
 		vcpu->arch.time_offset = data & ~PAGE_MASK;
 
-- 
1.4.2


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

[PATCH 2/4] [PATCH] cacheline-align kvmclock structures

[Glauber Costa]

Align the kvm_vcpu_time array to the size of a cacheline.

Signed-off-by: Glauber Costa <gcosta@redhat.com>
---
 arch/x86/kernel/kvmclock.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c
index b8da3bf..d82406a 100644
--- a/arch/x86/kernel/kvmclock.c
+++ b/arch/x86/kernel/kvmclock.c
@@ -36,7 +36,7 @@ early_param("no-kvmclock", parse_no_kvmc
 struct shared_info shared_info __attribute__((__aligned__(PAGE_SIZE)));
 
 /* The hypervisor will put information about time periodically here */
-static struct kvm_vcpu_time_info hv_clock[NR_CPUS];
+static struct kvm_vcpu_time_info hv_clock[NR_CPUS] __cacheline_aligned;
 #define get_clock(cpu, field) hv_clock[cpu].field
 
 static inline u64 kvm_get_delta(u64 last_tsc)
-- 
1.4.2


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

[PATCH 3/4] [PATCH] kvmclock: allow it to be turned off

[Glauber Costa]

Use the lower 3 lower bits of the system time msr to turn off the clock.
This means that all clock registration has to be aligned in a 4-byte boundary

Signed-off-by: Glauber Costa <gcosta@redhat.com>
---
 arch/x86/kvm/x86.c |    5 +++++
 1 files changed, 5 insertions(+), 0 deletions(-)

diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 6abd784..7ce14ce 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -591,6 +591,11 @@ int kvm_set_msr_common(struct kvm_vcpu *
 		if (vcpu->arch.time_page)
 			kvm_release_page_dirty(vcpu->arch.time_page);
 
+		/* 4-byte unaligned accesses are invalid */
+		if (data & 0x7) {
+			vcpu->arch.time_page = NULL;
+			break;
+		}
 		vcpu->arch.time = data & PAGE_MASK;
 		vcpu->arch.time_offset = data & ~PAGE_MASK;
 
-- 
1.4.2


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

[PATCH 4/4] [PATCH] cleanup leftovers

[Glauber Costa]

clean this leftover in kvmclock.c

Signed-off-by: Glauber Costa <gcosta@redhat.com>
---
 arch/x86/kernel/kvmclock.c |    3 ---
 1 files changed, 0 insertions(+), 3 deletions(-)

diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c
index d82406a..8ff12b6 100644
--- a/arch/x86/kernel/kvmclock.c
+++ b/arch/x86/kernel/kvmclock.c
@@ -20,7 +20,6 @@ #include <linux/clocksource.h>
 #include <linux/kvm_para.h>
 #include <asm/arch_hooks.h>
 #include <asm/msr.h>
-#include <xen/interface/xen.h>
 
 #define KVM_SCALE 22
 
@@ -33,8 +32,6 @@ static int parse_no_kvmclock(char *arg)
 }
 early_param("no-kvmclock", parse_no_kvmclock);
 
-struct shared_info shared_info __attribute__((__aligned__(PAGE_SIZE)));
-
 /* The hypervisor will put information about time periodically here */
 static struct kvm_vcpu_time_info hv_clock[NR_CPUS] __cacheline_aligned;
 #define get_clock(cpu, field) hv_clock[cpu].field
-- 
1.4.2


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

Re: [PATCH 4/4] [PATCH] cleanup leftovers

[Avi Kivity]

Glauber Costa wrote:
> clean this leftover in kvmclock.c
>
>   

Applied, thanks.

-- 
Do not meddle in the internals of kernels, for they are subtle and quick to panic.


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

Re: [PATCH 3/4] [PATCH] kvmclock: allow it to be turned off

[Avi Kivity]

Glauber Costa wrote:
> Use the lower 3 lower bits of the system time msr to turn off the clock.
> This means that all clock registration has to be aligned in a 4-byte boundary
>
>   

3 bits -> 8 bytes.

How about just using just bit 0 as an enable bit (not a disable bit).  
That means the default value of zero means the clock is disabled, and 
that we have a couple of more bits to enable future features.

> Signed-off-by: Glauber Costa <gcosta@redhat.com>
> ---
>  arch/x86/kvm/x86.c |    5 +++++
>  1 files changed, 5 insertions(+), 0 deletions(-)
>
> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
> index 6abd784..7ce14ce 100644
> --- a/arch/x86/kvm/x86.c
> +++ b/arch/x86/kvm/x86.c
> @@ -591,6 +591,11 @@ int kvm_set_msr_common(struct kvm_vcpu *
>  		if (vcpu->arch.time_page)
>  			kvm_release_page_dirty(vcpu->arch.time_page);
>  
> +		/* 4-byte unaligned accesses are invalid */
> +		if (data & 0x7) {
> +			vcpu->arch.time_page = NULL;
> +			break;
> +		}
>  		vcpu->arch.time = data & PAGE_MASK;
>  		vcpu->arch.time_offset = data & ~PAGE_MASK;
>  
>   


-- 
Do not meddle in the internals of kernels, for they are subtle and quick to panic.


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

Re: [PATCH 3/4] [PATCH] kvmclock: allow it to be turned off

[Glauber Costa]

Avi Kivity wrote:
> Glauber Costa wrote:
>> Use the lower 3 lower bits of the system time msr to turn off the clock.
>> This means that all clock registration has to be aligned in a 4-byte 
>> boundary
>>
>>   
> 
> 3 bits -> 8 bytes.
dohh!! true
/me ashamed.

> How about just using just bit 0 as an enable bit (not a disable bit).  
> That means the default value of zero means the clock is disabled, and 
> that we have a couple of more bits to enable future features.

Apart from the fact that it will break every single guest out there, 
that's ok. As I said: these things are so early, that maybe we can pay 
this price. Your call.

>> Signed-off-by: Glauber Costa <gcosta@redhat.com>
>> ---
>>  arch/x86/kvm/x86.c |    5 +++++
>>  1 files changed, 5 insertions(+), 0 deletions(-)
>>
>> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
>> index 6abd784..7ce14ce 100644
>> --- a/arch/x86/kvm/x86.c
>> +++ b/arch/x86/kvm/x86.c
>> @@ -591,6 +591,11 @@ int kvm_set_msr_common(struct kvm_vcpu *
>>          if (vcpu->arch.time_page)
>>              kvm_release_page_dirty(vcpu->arch.time_page);
>>  
>> +        /* 4-byte unaligned accesses are invalid */
>> +        if (data & 0x7) {
>> +            vcpu->arch.time_page = NULL;
>> +            break;
>> +        }
>>          vcpu->arch.time = data & PAGE_MASK;
>>          vcpu->arch.time_offset = data & ~PAGE_MASK;
>>  
>>   
> 
> 


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

Re: [PATCH 3/4] [PATCH] kvmclock: allow it to be turned off

[Avi Kivity]

Glauber Costa wrote:
> Apart from the fact that it will break every single guest out there, 
> that's ok. As I said: these things are so early, that maybe we can pay 
> this price. Your call.
>   

Which guests?  kvmclock is only in kvm.git, and I don't think any distro 
is based on that.

-- 
Do not meddle in the internals of kernels, for they are subtle and quick to panic.


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

Re: [PATCH 3/4] [PATCH] kvmclock: allow it to be turned off

[Glauber Costa]

Avi Kivity wrote:
> Glauber Costa wrote:
>> Apart from the fact that it will break every single guest out there, 
>> that's ok. As I said: these things are so early, that maybe we can pay 
>> this price. Your call.
>>   
> 
> Which guests?  kvmclock is only in kvm.git, and I don't think any distro 
> is based on that.
> 
My guests at home? ;-)

That's exactly what I meant by "early". Just wanted to make sure you're 
all ok with that.

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

Re: [PATCH 2/4] [PATCH] cacheline-align kvmclock structures

[Avi Kivity]

Glauber Costa wrote:
> Align the kvm_vcpu_time array to the size of a cacheline.
>
> Signed-off-by: Glauber Costa <gcosta@redhat.com>
> ---
>  arch/x86/kernel/kvmclock.c |    2 +-
>  1 files changed, 1 insertions(+), 1 deletions(-)
>
> diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c
> index b8da3bf..d82406a 100644
> --- a/arch/x86/kernel/kvmclock.c
> +++ b/arch/x86/kernel/kvmclock.c
> @@ -36,7 +36,7 @@ early_param("no-kvmclock", parse_no_kvmc
>  struct shared_info shared_info __attribute__((__aligned__(PAGE_SIZE)));
>  
>  /* The hypervisor will put information about time periodically here */
> -static struct kvm_vcpu_time_info hv_clock[NR_CPUS];
> +static struct kvm_vcpu_time_info hv_clock[NR_CPUS] __cacheline_aligned;
>  #define get_clock(cpu, field) hv_clock[cpu].field
>  
>  static inline u64 kvm_get_delta(u64 last_tsc)
>   

I think this will align only the array itself, not members, so any write 
will (and the following reads) will cause a cacheline bounce.

Switching to per_cpu() both clarifies the intent and fixes the issue.  
Still need the 8-byte alignment.  Might be best to stick that on the 
structure declaration, so it applies to all guests.

-- 
Do not meddle in the internals of kernels, for they are subtle and quick to panic.


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

Re: [PATCH 1/4] [PATCH] kvmclock: release time_page if msr is rewritten

[Avi Kivity]

Glauber Costa wrote:
> If the calling cpu rewrites the system clock msr for any reason,
> release the page we allocated in the last time
>   

Applied, thanks.

-- 
Do not meddle in the internals of kernels, for they are subtle and quick to panic.


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/