Re: [Qemu-devel] [PATCH v2] kvmclock: Ensure time in migration never goes backward

[Marcelo Tosatti <mtosatti@redhat.com>]

On Mon, Jun 02, 2014 at 10:31:44PM +0200, Marcin Gibuła wrote:
> >+    cpu_physical_memory_read(kvmclock_struct_pa, &time, sizeof(time));
> >+
> >+    delta = migration_tsc - time.tsc_timestamp;
> 
> Hi,
> 
> when I was testing live storage migration with libvirt I found out
> that this patch can cause virtual machine to hang when completing
> mirror job.
> 
> This is (probably) because kvmclock_current_nsec() is called twice
> in a row and on second call time.tsc_timestamp is larger than
> migration_tsc. This causes delta to be huge and sets timer to
> invalid value.
> 
> The double call happens when switching from old to new disk
> (pivoting in libvirt's nomenclature).
> 
> Example values:
> 
> First call: migration_tsc: 12052203518652476, time_tsc:
> 12052203301565676, delta 108543400
> 
> Second call: migration_tsc: 12052203518652476, time_tsc:
> 12052204478600322, delta 9223372036374801885
> 
> Perhaps it is worth adding:
> 
> if (time.tsc_timestamp > migration_tsc) {
>     return 0;
> }
> 
> there? Untested though...

Hi Marcin,

Can you give this patch a try?  Should read the guest TSC values after
stopping the VM.


diff --git a/hw/i386/kvm/clock.c b/hw/i386/kvm/clock.c
index 6f4ed28a..bef2504 100644
--- a/hw/i386/kvm/clock.c
+++ b/hw/i386/kvm/clock.c
@@ -17,6 +17,7 @@
 #include "qemu/host-utils.h"
 #include "sysemu/sysemu.h"
 #include "sysemu/kvm.h"
+#include "sysemu/cpus.h"
 #include "hw/sysbus.h"
 #include "hw/kvm/clock.h"
 
@@ -65,6 +66,7 @@ static uint64_t kvmclock_current_nsec(KVMClockState *s)
 
     cpu_physical_memory_read(kvmclock_struct_pa, &time, sizeof(time));
 
+    assert(time.tsc_timestamp <= migration_tsc);
     delta = migration_tsc - time.tsc_timestamp;
     if (time.tsc_shift < 0) {
         delta >>= -time.tsc_shift;
@@ -123,6 +125,8 @@ static void kvmclock_vm_state_change(void *opaque, int running,
         if (s->clock_valid) {
             return;
         }
+
+        cpu_synchronize_all_states();
         ret = kvm_vm_ioctl(kvm_state, KVM_GET_CLOCK, &data);
         if (ret < 0) {
             fprintf(stderr, "KVM_GET_CLOCK failed: %s\n", strerror(ret));