[Qemu-devel] [PULL] migration patches

[Qemu-devel] [PULL] migration patches

[Amit Shah]

Hi Juan,

Here's a compilation of migration-related patches from the list that
I've reviewed.

For the first patch, I picked Markus's over yours, just because it's
been on the list longer.


The following changes since commit 9fbee91a131a05e443d7108d7fbdf3ca91020290:

  Merge remote-tracking branch 'remotes/kvm/uq/master' into staging (2014-02-27 16:00:31 +0000)

are available in the git repository at:


  git://git.kernel.org/pub/scm/virt/qemu/amit/migration.git for-quintela

for you to fetch changes up to 6bb5d0981b1331df2150ae48ed9fabafea33d1d2:

  migration: add more traces (2014-03-07 01:23:50 +0530)

----------------------------------------------------------------

Alexey Kardashevskiy (3):
  vl: add system_wakeup_request tracepoint
  migration: extend section_start/end traces
  migration: add more traces

Markus Armbruster (1):
  qemu_file: Fix mismerge of "use fwrite() correctly"

 migration.c  | 30 ++++++------------------------
 qemu-file.c  |  4 +++-
 savevm.c     | 22 ++++++++++++++++------
 trace-events | 21 +++++++++++++++++++--
 vl.c         |  2 ++
 vmstate.c    |  2 ++
 6 files changed, 48 insertions(+), 33 deletions(-)

-- 
1.8.5.3

[Qemu-devel] [PATCH 1/4] qemu_file: Fix mismerge of "use fwrite() correctly"

[Amit Shah]

From: Markus Armbruster <armbru@redhat.com>

Reviewers accepted v2 of the patch, but what got committed was v1,
with the R-bys for v2.  This is the v1->v2 followup fix.

[Amit:
 This fixes commit aded6539d983280212e08d09f14157b1cb4d58cc
]

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Amit Shah <amit.shah@redhat.com>
Signed-off-by: Amit Shah <amit.shah@redhat.com>
---
 qemu-file.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/qemu-file.c b/qemu-file.c
index f074af1..e5ec798 100644
--- a/qemu-file.c
+++ b/qemu-file.c
@@ -105,7 +105,7 @@ static int stdio_put_buffer(void *opaque, const uint8_t *buf, int64_t pos,
     res = fwrite(buf, 1, size, s->stdio_file);
 
     if (res != size) {
-        return -EIO;	/* fake errno value */
+        return -errno;
     }
     return res;
 }
-- 
1.8.5.3

[Qemu-devel] [PATCH 2/4] vl: add system_wakeup_request tracepoint

[Amit Shah]

From: Alexey Kardashevskiy <aik@ozlabs.ru>

It might be useful for tracing migration.

Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
Signed-off-by: Amit Shah <amit.shah@redhat.com>
---
 trace-events | 1 +
 vl.c         | 2 ++
 2 files changed, 3 insertions(+)

diff --git a/trace-events b/trace-events
index 580281d..91a2edc 100644
--- a/trace-events
+++ b/trace-events
@@ -486,6 +486,7 @@ runstate_set(int new_state) "new state %d"
 g_malloc(size_t size, void *ptr) "size %zu ptr %p"
 g_realloc(void *ptr, size_t size, void *newptr) "ptr %p size %zu newptr %p"
 g_free(void *ptr) "ptr %p"
+system_wakeup_request(int reason) "reason=%d"
 
 # block/qcow2.c
 qcow2_writev_start_req(void *co, int64_t sector, int nb_sectors) "co %p sector %" PRIx64 " nb_sectors %d"
diff --git a/vl.c b/vl.c
index 1d27b34..179f14e 100644
--- a/vl.c
+++ b/vl.c
@@ -1837,6 +1837,8 @@ void qemu_register_suspend_notifier(Notifier *notifier)
 
 void qemu_system_wakeup_request(WakeupReason reason)
 {
+    trace_system_wakeup_request(reason);
+
     if (!runstate_check(RUN_STATE_SUSPENDED)) {
         return;
     }
-- 
1.8.5.3

[Qemu-devel] [PATCH 3/4] migration: extend section_start/end traces

[Amit Shah]

From: Alexey Kardashevskiy <aik@ozlabs.ru>

This adds @idstr to savevm_section_start and savevm_section_end
tracepoints.

Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
Signed-off-by: Amit Shah <amit.shah@redhat.com>
---
 savevm.c     | 12 ++++++------
 trace-events |  4 ++--
 2 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/savevm.c b/savevm.c
index 7329fc5..d094fbb 100644
--- a/savevm.c
+++ b/savevm.c
@@ -527,13 +527,13 @@ int qemu_savevm_state_iterate(QEMUFile *f)
         if (qemu_file_rate_limit(f)) {
             return 0;
         }
-        trace_savevm_section_start();
+        trace_savevm_section_start(se->idstr, se->section_id);
         /* Section type */
         qemu_put_byte(f, QEMU_VM_SECTION_PART);
         qemu_put_be32(f, se->section_id);
 
         ret = se->ops->save_live_iterate(f, se->opaque);
-        trace_savevm_section_end(se->section_id);
+        trace_savevm_section_end(se->idstr, se->section_id);
 
         if (ret < 0) {
             qemu_file_set_error(f, ret);
@@ -565,13 +565,13 @@ void qemu_savevm_state_complete(QEMUFile *f)
                 continue;
             }
         }
-        trace_savevm_section_start();
+        trace_savevm_section_start(se->idstr, se->section_id);
         /* Section type */
         qemu_put_byte(f, QEMU_VM_SECTION_END);
         qemu_put_be32(f, se->section_id);
 
         ret = se->ops->save_live_complete(f, se->opaque);
-        trace_savevm_section_end(se->section_id);
+        trace_savevm_section_end(se->idstr, se->section_id);
         if (ret < 0) {
             qemu_file_set_error(f, ret);
             return;
@@ -584,7 +584,7 @@ void qemu_savevm_state_complete(QEMUFile *f)
         if ((!se->ops || !se->ops->save_state) && !se->vmsd) {
             continue;
         }
-        trace_savevm_section_start();
+        trace_savevm_section_start(se->idstr, se->section_id);
         /* Section type */
         qemu_put_byte(f, QEMU_VM_SECTION_FULL);
         qemu_put_be32(f, se->section_id);
@@ -598,7 +598,7 @@ void qemu_savevm_state_complete(QEMUFile *f)
         qemu_put_be32(f, se->version_id);
 
         vmstate_save(f, se);
-        trace_savevm_section_end(se->section_id);
+        trace_savevm_section_end(se->idstr, se->section_id);
     }
 
     qemu_put_byte(f, QEMU_VM_EOF);
diff --git a/trace-events b/trace-events
index 91a2edc..828a3e9 100644
--- a/trace-events
+++ b/trace-events
@@ -1031,8 +1031,8 @@ vmware_scratch_write(uint32_t index, uint32_t value) "index %d, value 0x%x"
 vmware_setmode(uint32_t w, uint32_t h, uint32_t bpp) "%dx%d @ %d bpp"
 
 # savevm.c
-savevm_section_start(void) ""
-savevm_section_end(unsigned int section_id) "section_id %u"
+savevm_section_start(const char *id, unsigned int section_id) "%s, section_id %u"
+savevm_section_end(const char *id, unsigned int section_id) "%s, section_id %u"
 
 # arch_init.c
 migration_bitmap_sync_start(void) ""
-- 
1.8.5.3

[Qemu-devel] [PATCH 4/4] migration: add more traces

[Amit Shah]

From: Alexey Kardashevskiy <aik@ozlabs.ru>

This replaces DPRINTF macro with tracepoints.

This moves some messages from migration.c to savevm.c.

This adds tracepoint to signal about fields failed to migrate.

Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
Signed-off-by: Amit Shah <amit.shah@redhat.com>
---
 migration.c  | 30 ++++++------------------------
 qemu-file.c  |  2 ++
 savevm.c     | 10 ++++++++++
 trace-events | 16 ++++++++++++++++
 vmstate.c    |  2 ++
 5 files changed, 36 insertions(+), 24 deletions(-)

diff --git a/migration.c b/migration.c
index 14235b2..2497c5d 100644
--- a/migration.c
+++ b/migration.c
@@ -26,16 +26,6 @@
 #include "qmp-commands.h"
 #include "trace.h"
 
-//#define DEBUG_MIGRATION
-
-#ifdef DEBUG_MIGRATION
-#define DPRINTF(fmt, ...) \
-    do { printf("migration: " fmt, ## __VA_ARGS__); } while (0)
-#else
-#define DPRINTF(fmt, ...) \
-    do { } while (0)
-#endif
-
 enum {
     MIG_STATE_ERROR = -1,
     MIG_STATE_NONE,
@@ -111,7 +101,6 @@ static void process_incoming_migration_co(void *opaque)
         exit(EXIT_FAILURE);
     }
     qemu_announce_self();
-    DPRINTF("successfully loaded vm state\n");
 
     bdrv_clear_incoming_migration_all();
     /* Make sure all file formats flush their mutable metadata */
@@ -300,7 +289,7 @@ static void migrate_fd_cleanup(void *opaque)
     s->cleanup_bh = NULL;
 
     if (s->file) {
-        DPRINTF("closing file\n");
+        trace_migrate_fd_cleanup();
         qemu_mutex_unlock_iothread();
         qemu_thread_join(&s->thread);
         qemu_mutex_lock_iothread();
@@ -323,7 +312,7 @@ static void migrate_fd_cleanup(void *opaque)
 
 void migrate_fd_error(MigrationState *s)
 {
-    DPRINTF("setting error state\n");
+    trace_migrate_fd_error();
     assert(s->file == NULL);
     s->state = MIG_STATE_ERROR;
     trace_migrate_set_state(MIG_STATE_ERROR);
@@ -333,7 +322,7 @@ void migrate_fd_error(MigrationState *s)
 static void migrate_fd_cancel(MigrationState *s)
 {
     int old_state ;
-    DPRINTF("cancelling migration\n");
+    trace_migrate_fd_cancel();
 
     do {
         old_state = s->state;
@@ -583,29 +572,23 @@ static void *migration_thread(void *opaque)
     int64_t start_time = initial_time;
     bool old_vm_running = false;
 
-    DPRINTF("beginning savevm\n");
     qemu_savevm_state_begin(s->file, &s->params);
 
     s->setup_time = qemu_clock_get_ms(QEMU_CLOCK_HOST) - setup_start;
     migrate_set_state(s, MIG_STATE_SETUP, MIG_STATE_ACTIVE);
 
-    DPRINTF("setup complete\n");
-
     while (s->state == MIG_STATE_ACTIVE) {
         int64_t current_time;
         uint64_t pending_size;
 
         if (!qemu_file_rate_limit(s->file)) {
-            DPRINTF("iterate\n");
             pending_size = qemu_savevm_state_pending(s->file, max_size);
-            DPRINTF("pending size %" PRIu64 " max %" PRIu64 "\n",
-                    pending_size, max_size);
+            trace_migrate_pending(pending_size, max_size);
             if (pending_size && pending_size >= max_size) {
                 qemu_savevm_state_iterate(s->file);
             } else {
                 int ret;
 
-                DPRINTF("done iterating\n");
                 qemu_mutex_lock_iothread();
                 start_time = qemu_clock_get_ms(QEMU_CLOCK_REALTIME);
                 qemu_system_wakeup_request(QEMU_WAKEUP_REASON_OTHER);
@@ -644,9 +627,8 @@ static void *migration_thread(void *opaque)
             s->mbps = time_spent ? (((double) transferred_bytes * 8.0) /
                     ((double) time_spent / 1000.0)) / 1000.0 / 1000.0 : -1;
 
-            DPRINTF("transferred %" PRIu64 " time_spent %" PRIu64
-                    " bandwidth %g max_size %" PRId64 "\n",
-                    transferred_bytes, time_spent, bandwidth, max_size);
+            trace_migrate_transferred(transferred_bytes, time_spent,
+                                      bandwidth, max_size);
             /* if we haven't sent anything, we don't want to recalculate
                10000 is a small enough number for our purposes */
             if (s->dirty_bytes_rate && transferred_bytes > 10000) {
diff --git a/qemu-file.c b/qemu-file.c
index e5ec798..8d5f45d 100644
--- a/qemu-file.c
+++ b/qemu-file.c
@@ -4,6 +4,7 @@
 #include "block/coroutine.h"
 #include "migration/migration.h"
 #include "migration/qemu-file.h"
+#include "trace.h"
 
 #define IO_BUF_SIZE 32768
 #define MAX_IOV_SIZE MIN(IOV_MAX, 64)
@@ -595,6 +596,7 @@ int qemu_fclose(QEMUFile *f)
         ret = f->last_error;
     }
     g_free(f);
+    trace_qemu_file_fclose();
     return ret;
 }
 
diff --git a/savevm.c b/savevm.c
index d094fbb..ef7d9ce 100644
--- a/savevm.c
+++ b/savevm.c
@@ -41,6 +41,7 @@
 #include "qemu/iov.h"
 #include "block/snapshot.h"
 #include "block/qapi.h"
+#include <netinet/ether.h>
 
 #define SELF_ANNOUNCE_ROUNDS 5
 
@@ -81,6 +82,8 @@ static void qemu_announce_self_iter(NICState *nic, void *opaque)
     uint8_t buf[60];
     int len;
 
+    trace_qemu_announce_self_iter(ether_ntoa((struct ether_addr *)
+                                             &nic->conf->macaddr));
     len = announce_self_create(buf, nic->conf->macaddr.a);
 
     qemu_send_packet_raw(qemu_get_queue(nic), buf, len);
@@ -429,6 +432,7 @@ void vmstate_unregister(DeviceState *dev, const VMStateDescription *vmsd,
 
 static int vmstate_load(QEMUFile *f, SaveStateEntry *se, int version_id)
 {
+    trace_vmstate_load(se->idstr, se->vmsd ? se->vmsd->name : "(old)");
     if (!se->vmsd) {         /* Old style */
         return se->ops->load_state(f, se->opaque, version_id);
     }
@@ -437,6 +441,7 @@ static int vmstate_load(QEMUFile *f, SaveStateEntry *se, int version_id)
 
 static void vmstate_save(QEMUFile *f, SaveStateEntry *se)
 {
+    trace_vmstate_save(se->idstr, se->vmsd ? se->vmsd->name : "(old)");
     if (!se->vmsd) {         /* Old style */
         se->ops->save_state(f, se->opaque);
         return;
@@ -463,6 +468,7 @@ void qemu_savevm_state_begin(QEMUFile *f,
     SaveStateEntry *se;
     int ret;
 
+    trace_savevm_state_begin();
     QTAILQ_FOREACH(se, &savevm_handlers, entry) {
         if (!se->ops || !se->ops->set_params) {
             continue;
@@ -515,6 +521,7 @@ int qemu_savevm_state_iterate(QEMUFile *f)
     SaveStateEntry *se;
     int ret = 1;
 
+    trace_savevm_state_iterate();
     QTAILQ_FOREACH(se, &savevm_handlers, entry) {
         if (!se->ops || !se->ops->save_live_iterate) {
             continue;
@@ -554,6 +561,8 @@ void qemu_savevm_state_complete(QEMUFile *f)
     SaveStateEntry *se;
     int ret;
 
+    trace_savevm_state_complete();
+
     cpu_synchronize_all_states();
 
     QTAILQ_FOREACH(se, &savevm_handlers, entry) {
@@ -628,6 +637,7 @@ void qemu_savevm_state_cancel(void)
 {
     SaveStateEntry *se;
 
+    trace_savevm_state_cancel();
     QTAILQ_FOREACH(se, &savevm_handlers, entry) {
         if (se->ops && se->ops->cancel) {
             se->ops->cancel(se->opaque);
diff --git a/trace-events b/trace-events
index 828a3e9..395da76 100644
--- a/trace-events
+++ b/trace-events
@@ -1033,6 +1033,17 @@ vmware_setmode(uint32_t w, uint32_t h, uint32_t bpp) "%dx%d @ %d bpp"
 # savevm.c
 savevm_section_start(const char *id, unsigned int section_id) "%s, section_id %u"
 savevm_section_end(const char *id, unsigned int section_id) "%s, section_id %u"
+savevm_state_begin(void) ""
+savevm_state_iterate(void) ""
+savevm_state_complete(void) ""
+savevm_state_cancel(void) ""
+vmstate_save(const char *idstr, const char *vmsd_name) "%s, %s"
+vmstate_load(const char *idstr, const char *vmsd_name) "%s, %s"
+vmstate_load_field_error(const char *field, int ret) "field \"%s\" load failed, ret = %d"
+qemu_announce_self_iter(const char *mac) "%s"
+
+# qemu-file.c
+qemu_file_fclose(void) ""
 
 # arch_init.c
 migration_bitmap_sync_start(void) ""
@@ -1165,6 +1176,11 @@ virtio_ccw_new_device(int cssid, int ssid, int schid, int devno, const char *dev
 
 # migration.c
 migrate_set_state(int new_state) "new state %d"
+migrate_fd_cleanup(void) ""
+migrate_fd_error(void) ""
+migrate_fd_cancel(void) ""
+migrate_pending(uint64_t size, uint64_t max) "pending size %" PRIu64 " max %" PRIu64
+migrate_transferred(uint64_t tranferred, uint64_t time_spent, double bandwidth, uint64_t size) "transferred %" PRIu64 " time_spent %" PRIu64 " bandwidth %g max_size %" PRId64
 
 # kvm-all.c
 kvm_ioctl(int type, void *arg) "type 0x%x, arg %p"
diff --git a/vmstate.c b/vmstate.c
index d1f5eb0..b689f2f 100644
--- a/vmstate.c
+++ b/vmstate.c
@@ -3,6 +3,7 @@
 #include "migration/qemu-file.h"
 #include "migration/vmstate.h"
 #include "qemu/bitops.h"
+#include "trace.h"
 
 static void vmstate_subsection_save(QEMUFile *f, const VMStateDescription *vmsd,
                                     void *opaque);
@@ -73,6 +74,7 @@ int vmstate_load_state(QEMUFile *f, const VMStateDescription *vmsd,
 
                 }
                 if (ret < 0) {
+                    trace_vmstate_load_field_error(field->name, ret);
                     return ret;
                 }
             }
-- 
1.8.5.3

Re: [Qemu-devel] [PULL] migration patches

[Juan Quintela]

Amit Shah <amit.shah@redhat.com> wrote:
> Hi Juan,
>
> Here's a compilation of migration-related patches from the list that
> I've reviewed.
>
> For the first patch, I picked Markus's over yours, just because it's
> been on the list longer.
>
>
> The following changes since commit 9fbee91a131a05e443d7108d7fbdf3ca91020290:
>
>   Merge remote-tracking branch 'remotes/kvm/uq/master' into staging (2014-02-27 16:00:31 +0000)
>
> are available in the git repository at:
>
>
>   git://git.kernel.org/pub/scm/virt/qemu/amit/migration.git for-quintela
>
> for you to fetch changes up to 6bb5d0981b1331df2150ae48ed9fabafea33d1d2:
>
>   migration: add more traces (2014-03-07 01:23:50 +0530)
>
> ----------------------------------------------------------------
>
> Alexey Kardashevskiy (3):
>   vl: add system_wakeup_request tracepoint
>   migration: extend section_start/end traces
>   migration: add more traces
>
> Markus Armbruster (1):
>   qemu_file: Fix mismerge of "use fwrite() correctly"
>
>  migration.c  | 30 ++++++------------------------
>  qemu-file.c  |  4 +++-
>  savevm.c     | 22 ++++++++++++++++------
>  trace-events | 21 +++++++++++++++++++--
>  vl.c         |  2 ++
>  vmstate.c    |  2 ++
>  6 files changed, 48 insertions(+), 33 deletions(-)

Reviewed-by: Juan Quintela <quintela@redhat.com>

Re: [Qemu-devel] [PULL] migration patches

[Peter Maydell]

On 6 March 2014 20:03, Amit Shah <amit.shah@redhat.com> wrote:
> Hi Juan,
>
> Here's a compilation of migration-related patches from the list that
> I've reviewed.
>
> For the first patch, I picked Markus's over yours, just because it's
> been on the list longer.
>
>
> The following changes since commit 9fbee91a131a05e443d7108d7fbdf3ca91020290:
>
>   Merge remote-tracking branch 'remotes/kvm/uq/master' into staging (2014-02-27 16:00:31 +0000)
>
> are available in the git repository at:
>
>
>   git://git.kernel.org/pub/scm/virt/qemu/amit/migration.git for-quintela

Just for clarity: I'm assuming from the salutation and the branch name
that you're expecting Juan to merge these into his tree; I'm not
going to apply them to master.

thanks
-- PMM

Re: [Qemu-devel] [PULL] migration patches

[Juan Quintela]

Peter Maydell <peter.maydell@linaro.org> wrote:
> On 6 March 2014 20:03, Amit Shah <amit.shah@redhat.com> wrote:
>> Hi Juan,
>>
>> Here's a compilation of migration-related patches from the list that
>> I've reviewed.
>>
>> For the first patch, I picked Markus's over yours, just because it's
>> been on the list longer.
>>
>>
>> The following changes since commit 9fbee91a131a05e443d7108d7fbdf3ca91020290:
>>
>>   Merge remote-tracking branch 'remotes/kvm/uq/master' into staging
>> (2014-02-27 16:00:31 +0000)
>>
>> are available in the git repository at:
>>
>>
>>   git://git.kernel.org/pub/scm/virt/qemu/amit/migration.git for-quintela
>
> Just for clarity: I'm assuming from the salutation and the branch name
> that you're expecting Juan to merge these into his tree; I'm not
> going to apply them to master.

They are already on my last pull request.

Thanks to both, JUan.

Re: [Qemu-devel] [PULL] migration patches

[Amit Shah]

Hi Peter,

On (Sat) 08 Mar 2014 [12:19:10], Peter Maydell wrote:
> On 6 March 2014 20:03, Amit Shah <amit.shah@redhat.com> wrote:
> > Hi Juan,
> >
> > Here's a compilation of migration-related patches from the list that
> > I've reviewed.
> >
> > For the first patch, I picked Markus's over yours, just because it's
> > been on the list longer.
> >
> >
> > The following changes since commit 9fbee91a131a05e443d7108d7fbdf3ca91020290:
> >
> >   Merge remote-tracking branch 'remotes/kvm/uq/master' into staging (2014-02-27 16:00:31 +0000)
> >
> > are available in the git repository at:
> >
> >
> >   git://git.kernel.org/pub/scm/virt/qemu/amit/migration.git for-quintela
> 
> Just for clarity: I'm assuming from the salutation and the branch name
> that you're expecting Juan to merge these into his tree; I'm not
> going to apply them to master.

Yes, indeed, that was the case for this pull request.

I intend to review a few migration patches on the list and send them
on to Juan for review and later merging.  If they're trivial, I'll
pass them on to you directly.

Should I mark the subject line as [migration PULL], so you don't have
to bother with the pull req?  Something else?

		Amit

Re: [Qemu-devel] [PULL] migration patches

[Peter Maydell]

On 10 March 2014 04:39, Amit Shah <amit.shah@redhat.com> wrote:
> Should I mark the subject line as [migration PULL], so you don't have
> to bother with the pull req?  Something else?

That seems a reasonable convention, yes, or [PULL migration] maybe.

thanks
-- PMM