Re: [RFC PATCH 1/2] Hyper-H reference counter

Gleb Natapov
2013-05-16T08:10:30Z

On Tue, May 14, 2013 at 04:14:11PM +0200, Peter Lieven wrote:
> On 13.05.2013 13:45, Vadim Rozenfeld wrote:
> >Signed-off: Peter Lieven <pl@dlh.net>
> >Signed-off: Gleb Natapov <gleb@redhat.com>
> >Signed-off: Vadim Rozenfeld <vrozenfe@redhat.com>
> >
> >The following patch allows to activate Hyper-V
> >reference time counter
> >---
> > arch/x86/include/asm/kvm_host.h | 2 ++
> > arch/x86/include/uapi/asm/hyperv.h | 3 +++
> > arch/x86/kvm/x86.c | 25 ++++++++++++++++++++++++-
> > 3 files changed, 29 insertions(+), 1 deletion(-)
> >
> >diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
> >index 3741c65..f0fee35 100644
> >--- a/arch/x86/include/asm/kvm_host.h
> >+++ b/arch/x86/include/asm/kvm_host.h
> >@@ -575,6 +575,8 @@ struct kvm_arch {
> > /* fields used by HYPER-V emulation */
> > u64 hv_guest_os_id;
> > u64 hv_hypercall;
> >+ u64 hv_ref_count;
> >+ u64 hv_tsc_page;
> >
> > #ifdef CONFIG_KVM_MMU_AUDIT
> > int audit_point;
> >diff --git a/arch/x86/include/uapi/asm/hyperv.h b/arch/x86/include/uapi/asm/hyperv.h
> >index b80420b..9711819 100644
> >--- a/arch/x86/include/uapi/asm/hyperv.h
> >+++ b/arch/x86/include/uapi/asm/hyperv.h
> >@@ -136,6 +136,9 @@
> > /* MSR used to read the per-partition time reference counter */
> > #define HV_X64_MSR_TIME_REF_COUNT 0x40000020
> >
> >+/* A partition's reference time stamp counter (TSC) page */
> >+#define HV_X64_MSR_REFERENCE_TSC 0x40000021
> >+
> > /* Define the virtual APIC registers */
> > #define HV_X64_MSR_EOI 0x40000070
> > #define HV_X64_MSR_ICR 0x40000071
> >diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
> >index 094b5d9..1a4036d 100644
> >--- a/arch/x86/kvm/x86.c
> >+++ b/arch/x86/kvm/x86.c
> >@@ -843,7 +843,7 @@ EXPORT_SYMBOL_GPL(kvm_rdpmc);
> > static u32 msrs_to_save[] = {
> > MSR_KVM_SYSTEM_TIME, MSR_KVM_WALL_CLOCK,
> > MSR_KVM_SYSTEM_TIME_NEW, MSR_KVM_WALL_CLOCK_NEW,
> >- HV_X64_MSR_GUEST_OS_ID, HV_X64_MSR_HYPERCALL,
> >+ HV_X64_MSR_GUEST_OS_ID, HV_X64_MSR_HYPERCALL,HV_X64_MSR_TIME_REF_COUNT,
> > HV_X64_MSR_APIC_ASSIST_PAGE, MSR_KVM_ASYNC_PF_EN, MSR_KVM_STEAL_TIME,
> > MSR_KVM_PV_EOI_EN,
> > MSR_IA32_SYSENTER_CS, MSR_IA32_SYSENTER_ESP, MSR_IA32_SYSENTER_EIP,
> >@@ -1764,6 +1764,8 @@ static bool kvm_hv_msr_partition_wide(u32 msr)
> > switch (msr) {
> > case HV_X64_MSR_GUEST_OS_ID:
> > case HV_X64_MSR_HYPERCALL:
> >+ case HV_X64_MSR_REFERENCE_TSC:
> >+ case HV_X64_MSR_TIME_REF_COUNT:
> > r = true;
> > break;
> > }
> >@@ -1803,6 +1805,21 @@ static int set_msr_hyperv_pw(struct kvm_vcpu *vcpu, u32 msr, u64 data)
> > if (__copy_to_user((void __user *)addr, instructions, 4))
> > return 1;
> > kvm->arch.hv_hypercall = data;
> >+ kvm->arch.hv_ref_count = get_kernel_ns();
> >+ break;
> >+ }
> >+ case HV_X64_MSR_REFERENCE_TSC: {
> >+ u64 gfn;
> >+ unsigned long addr;
> >+ u32 tsc_ref;
> >+ gfn = data >> HV_X64_MSR_HYPERCALL_PAGE_ADDRESS_SHIFT;
> >+ addr = gfn_to_hva(kvm, gfn);
> >+ if (kvm_is_error_hva(addr))
> >+ return 1;
> >+ tsc_ref = 0;
> >+ if(__copy_to_user((void __user *)addr, &tsc_ref, sizeof(tsc_ref)))
> >+ return 1;
> >+ kvm->arch.hv_tsc_page = data;
> > break;
> > }
> > default:
> >@@ -2229,6 +2246,12 @@ static int get_msr_hyperv_pw(struct kvm_vcpu *vcpu, u32 msr, u64 *pdata)
> > case HV_X64_MSR_HYPERCALL:
> > data = kvm->arch.hv_hypercall;
> > break;
> >+ case HV_X64_MSR_TIME_REF_COUNT:
> >+ data = div_u64(get_kernel_ns() - kvm->arch.hv_ref_count,100);
> >+ break;
>
>
> in an earlier version of this patch I have the following:
>
> + case HV_X64_MSR_TIME_REF_COUNT: {
> + u64 now_ns;
> + local_irq_disable();
> + now_ns = get_kernel_ns();
> + data = div_u64(now_ns + kvm->arch.kvmclock_offset - kvm->arch.hv_ref_count,100);
> + local_irq_enable();
> + break;
> + }
>
> I do not know if this is right, but I can report that this one is working without any flaws
> since approx. 1.5 years.
>
get_kernel_ns() should be called at least with preemption disabled and
all other callers disable irq around it too, also without
kvmclock_offset the counter will be incorrect after migration. Marcelo,
is kvmclock_offset guaranties migration correctness or more is required?

--
Gleb.