[Qemu-devel] [RFC] QEMU Live Migration

[Qemu-devel] [RFC] QEMU Live Migration

[Anthony Liguori]

[-- Attachment #1: Type: text/plain, Size: 977 bytes --]

Howdy,

I wanted to post a draft of my QEMU live migration patch.  I think 
there's a lingering memory allocation problem.  I also need to implement 
ARP forwarding and a TCP transport stream.

However, I thought I'd post an initial version in case anyone wants to 
play around with it.

Once you apply the patch, make sure you've got it on both machines.  For 
my setup, I've got a common NFS share mounted in /mnt.  Then I run:

qemu -hda /mnt/win2k.img -monitor telnet:0.0.0.0:1025,server,nowait -vnc :2

Then, I connect to the monitor and execute:

(qemu) migrate ssh://woolly

When the migration completes, the monitor command will finish and then I 
can connect to the machine on woolly:2.

The ssh command will be derived from argc/argv so it's important to 
ensure that the -hda path is valid on both machines.

All three patches are required and should be applied in order of: 
qemu-apic-save-restore.diff qemu-file.diff qemu-migration.diff

Regards,

Anthony Liguori

[-- Attachment #2: qemu-migration.diff --]
[-- Type: text/x-patch, Size: 23794 bytes --]

diff -r 8ad5450b580b Makefile.target
--- a/Makefile.target	Tue Jan 16 17:04:55 2007 -0600
+++ b/Makefile.target	Tue Jan 16 17:04:56 2007 -0600
@@ -300,7 +300,7 @@ endif
 
 # must use static linking to avoid leaving stuff in virtual address space
 VL_OBJS=vl.o osdep.o readline.o monitor.o pci.o console.o loader.o isa_mmio.o
-VL_OBJS+=cutils.o
+VL_OBJS+=cutils.o migration.o
 VL_OBJS+=block.o block-raw.o
 VL_OBJS+=block-cow.o block-qcow.o aes.o block-vmdk.o block-cloop.o block-dmg.o block-bochs.o block-vpc.o block-vvfat.o block-qcow2.o
 ifdef CONFIG_WIN32
diff -r 8ad5450b580b cpu-all.h
--- a/cpu-all.h	Tue Jan 16 17:04:55 2007 -0600
+++ b/cpu-all.h	Sun Jan 21 17:39:13 2007 -0600
@@ -884,6 +884,7 @@ int cpu_memory_rw_debug(CPUState *env, t
 
 #define VGA_DIRTY_FLAG  0x01
 #define CODE_DIRTY_FLAG 0x02
+#define MIGRATION_DIRTY_FLAG 0x04
 
 /* read dirty bit (return 0 or 1) */
 static inline int cpu_physical_memory_is_dirty(ram_addr_t addr)
@@ -905,6 +906,10 @@ void cpu_physical_memory_reset_dirty(ram
 void cpu_physical_memory_reset_dirty(ram_addr_t start, ram_addr_t end,
                                      int dirty_flags);
 void cpu_tlb_update_dirty(CPUState *env);
+
+void cpu_physical_memory_set_dirty_tracking(int enable);
+
+int cpu_physical_memory_get_dirty_tracking(void);
 
 void dump_exec_info(FILE *f,
                     int (*cpu_fprintf)(FILE *f, const char *fmt, ...));
diff -r 8ad5450b580b cutils.c
--- a/cutils.c	Tue Jan 16 17:04:55 2007 -0600
+++ b/cutils.c	Sun Jan 21 16:58:07 2007 -0600
@@ -81,3 +81,43 @@ int stristart(const char *str, const cha
         *ptr = p;
     return 1;
 }
+
+int hex2bin(char ch)
+{
+    if (ch >= '0' && ch <= '9')
+	return ch - '0';
+    else if (ch >= 'A' && ch <= 'Z')
+	return 10 + ch - 'A';
+    else if (ch >= 'a' && ch <= 'z')
+	return 10 + ch - 'a';
+
+    return -1;
+}
+
+char *urldecode(const char *ptr)
+{
+    char *ret;
+    int i;
+
+    ret = qemu_mallocz(strlen(ptr) + 1);
+    if (ret == NULL)
+	return NULL;
+
+    for (i = 0; *ptr; ptr++, i++) {
+	switch (*ptr) {
+	case '%':
+	    if (ptr[1] == 0 || ptr[2] == 0)
+		break;
+	    ret[i] = hex2bin(ptr[1]) << 4 | hex2bin(ptr[2]);
+	    ptr += 2;
+	    break;
+	default:
+	   ret[i] = *ptr;
+	   break;
+	}
+    }
+    ret[i] = 0;
+
+    return ret;
+}
+
diff -r 8ad5450b580b exec.c
--- a/exec.c	Tue Jan 16 17:04:55 2007 -0600
+++ b/exec.c	Sun Jan 21 17:40:00 2007 -0600
@@ -82,6 +82,7 @@ int phys_ram_fd;
 int phys_ram_fd;
 uint8_t *phys_ram_base;
 uint8_t *phys_ram_dirty;
+static int in_migration;
 
 CPUState *first_cpu;
 /* current CPU in the current thread. It is only valid inside
@@ -1406,6 +1407,16 @@ void cpu_physical_memory_reset_dirty(ram
         }
     }
 #endif
+}
+
+void cpu_physical_memory_set_dirty_tracking(int enable)
+{
+    in_migration = enable;
+}
+
+int cpu_physical_memory_get_dirty_tracking(void)
+{
+    return in_migration;
 }
 
 static inline void tlb_update_dirty(CPUTLBEntry *tlb_entry)
@@ -2231,6 +2242,14 @@ uint32_t lduw_phys(target_phys_addr_t ad
     return tswap16(val);
 }
 
+#ifdef __GNUC__
+#define likely(x) __builtin_expect(!!(x), 1)
+#define unlikely(x) __builtin_expect(!!(x), 0)
+#else
+#define likely(x) x
+#define unlikely(x) x
+#endif
+
 /* warning: addr must be aligned. The ram page is not masked as dirty
    and the code inside is not invalidated. It is useful if the dirty
    bits are used to track modified PTEs */
@@ -2252,9 +2271,21 @@ void stl_phys_notdirty(target_phys_addr_
         io_index = (pd >> IO_MEM_SHIFT) & (IO_MEM_NB_ENTRIES - 1);
         io_mem_write[io_index][2](io_mem_opaque[io_index], addr, val);
     } else {
-        ptr = phys_ram_base + (pd & TARGET_PAGE_MASK) + 
-            (addr & ~TARGET_PAGE_MASK);
+        unsigned long addr1;
+        addr1 = (pd & TARGET_PAGE_MASK) + (addr & ~TARGET_PAGE_MASK);
+
+        ptr = phys_ram_base + addr1;
         stl_p(ptr, val);
+
+	if (unlikely(in_migration)) {
+	    if (!cpu_physical_memory_is_dirty(addr1)) {
+		/* invalidate code */
+		tb_invalidate_phys_page_range(addr1, addr1 + 4, 0);
+		/* set dirty bit */
+		phys_ram_dirty[addr1 >> TARGET_PAGE_BITS] |=
+		    (0xff & ~CODE_DIRTY_FLAG);
+	    }
+        }
     }
 }
 
diff -r 8ad5450b580b monitor.c
--- a/monitor.c	Tue Jan 16 17:04:55 2007 -0600
+++ b/monitor.c	Sun Jan 21 16:57:09 2007 -0600
@@ -1253,6 +1253,8 @@ static term_cmd_t term_cmds[] = {
        "capture index", "stop capture" },
     { "memsave", "lis", do_memory_save, 
       "addr size file", "save to disk virtual memory dump starting at 'addr' of size 'size'", },
+    { "migrate", "-ds", do_migrate,
+      "[-d] command", "migrate the VM using command (use -d to not wait for command to complete)" },
     { NULL, NULL, }, 
 };
 
@@ -2428,12 +2430,26 @@ static void term_read(void *opaque, cons
         readline_handle_byte(buf[i]);
 }
 
+static int monitor_suspended;
+
+void monitor_suspend(void)
+{
+    monitor_suspended = 1;
+}
+
+void monitor_resume(void)
+{
+    monitor_suspended = 0;
+    monitor_start_input();
+}
+
 static void monitor_start_input(void);
 
 static void monitor_handle_command1(void *opaque, const char *cmdline)
 {
     monitor_handle_command(cmdline);
-    monitor_start_input();
+    if (!monitor_suspended)
+	monitor_start_input();
 }
 
 static void monitor_start_input(void)
diff -r 8ad5450b580b vl.c
--- a/vl.c	Tue Jan 16 17:04:55 2007 -0600
+++ b/vl.c	Sun Jan 21 20:13:13 2007 -0600
@@ -169,6 +169,7 @@ int fd_bootchk = 1;
 int fd_bootchk = 1;
 int no_reboot = 0;
 int daemonize = 0;
+int incoming = 0;
 const char *option_rom[MAX_OPTION_ROMS];
 int nb_option_roms;
 
@@ -4764,6 +4765,73 @@ int qemu_loadvm_state(QEMUFile *f)
         }
         /* always seek to exact end of record */
         qemu_fseek(f, cur_pos + record_len, SEEK_SET);
+     }
+    ret = 0;
+ the_end:
+    return ret;
+}
+
+int qemu_live_savevm_state(QEMUFile *f)
+{
+    SaveStateEntry *se;
+    int len, ret;
+
+    qemu_put_be32(f, QEMU_VM_FILE_MAGIC);
+    qemu_put_be32(f, QEMU_VM_FILE_VERSION);
+
+    for(se = first_se; se != NULL; se = se->next) {
+	len = strlen(se->idstr);
+
+	qemu_put_byte(f, len);
+	qemu_put_buffer(f, se->idstr, len);
+	qemu_put_be32(f, se->instance_id);
+	qemu_put_be32(f, se->version_id);
+
+	se->save_state(f, se->opaque);
+    }
+
+    qemu_put_byte(f, 0);
+
+    ret = 0;
+    return ret;
+}
+
+int qemu_live_loadvm_state(QEMUFile *f)
+{
+    SaveStateEntry *se;
+    int len, ret, instance_id, version_id;
+    unsigned int v;
+    char idstr[256];
+    
+    v = qemu_get_be32(f);
+    if (v != QEMU_VM_FILE_MAGIC)
+        goto fail;
+    v = qemu_get_be32(f);
+    if (v != QEMU_VM_FILE_VERSION) {
+    fail:
+        ret = -1;
+        goto the_end;
+    }
+
+    for(;;) {
+        len = qemu_get_byte(f);
+	if (len == 0)
+	    break;
+        qemu_get_buffer(f, idstr, len);
+        idstr[len] = '\0';
+        instance_id = qemu_get_be32(f);
+        version_id = qemu_get_be32(f);
+        se = find_se(idstr, instance_id);
+        if (!se) {
+            fprintf(stderr, "qemu: warning: instance 0x%x of device '%s' not present in current VM\n", 
+                    instance_id, idstr);
+        } else {
+            ret = se->load_state(f, se->opaque, version_id);
+            if (ret < 0) {
+                fprintf(stderr, "qemu: warning: error while loading state for instance 0x%x of device '%s'\n", 
+                        instance_id, idstr);
+            }
+        }
     }
     ret = 0;
  the_end:
@@ -5589,7 +5657,20 @@ static void ram_decompress_close(RamDeco
     inflateEnd(&s->zstream);
 }
 
-static void ram_save(QEMUFile *f, void *opaque)
+static void ram_save_live(QEMUFile *f, void *opaque)
+{
+    target_ulong addr;
+
+    for (addr = 0; addr < phys_ram_size; addr += TARGET_PAGE_SIZE) {
+	if (cpu_physical_memory_get_dirty(addr, MIGRATION_DIRTY_FLAG)) {
+	    qemu_put_be32(f, addr);
+	    qemu_put_buffer(f, phys_ram_base + addr, TARGET_PAGE_SIZE);
+	}
+    }
+    qemu_put_be32(f, 1);
+}
+
+static void ram_save_static(QEMUFile *f, void *opaque)
 {
     int i;
     RamCompressState s1, *s = &s1;
@@ -5633,16 +5714,39 @@ static void ram_save(QEMUFile *f, void *
     ram_compress_close(s);
 }
 
-static int ram_load(QEMUFile *f, void *opaque, int version_id)
+static void ram_save(QEMUFile *f, void *opaque)
+{
+    int in_migration = cpu_physical_memory_get_dirty_tracking();
+
+    qemu_put_byte(f, in_migration);
+
+    if (in_migration)
+	ram_save_live(f, opaque);
+    else
+	ram_save_static(f, opaque);
+}
+
+static int ram_load_live(QEMUFile *f, void *opaque)
+{
+    target_ulong addr;
+
+    do {
+	addr = qemu_get_be32(f);
+	if (addr == 1)
+	    break;
+
+	qemu_get_buffer(f, phys_ram_base + addr, TARGET_PAGE_SIZE);
+    } while (1);
+
+    return 0;
+}
+
+static int ram_load_static(QEMUFile *f, void *opaque)
 {
     RamDecompressState s1, *s = &s1;
     uint8_t buf[10];
     int i;
 
-    if (version_id == 1)
-        return ram_load_v1(f, opaque);
-    if (version_id != 2)
-        return -EINVAL;
     if (qemu_get_be32(f) != phys_ram_size)
         return -EINVAL;
     if (ram_decompress_open(s, f) < 0)
@@ -5686,6 +5790,30 @@ static int ram_load(QEMUFile *f, void *o
     }
     ram_decompress_close(s);
     return 0;
+}
+
+static int ram_load(QEMUFile *f, void *opaque, int version_id)
+{
+    int ret;
+
+    switch (version_id) {
+    case 1:
+        ret = ram_load_v1(f, opaque);
+	break;
+    case 3:
+	if (qemu_get_byte(f)) {
+	    ret = ram_load_live(f, opaque);
+	    break;
+	}
+    case 2:
+	ret = ram_load_static(f, opaque);
+	break;
+    default:
+	ret = -EINVAL;
+	break;
+    }
+
+    return ret;
 }
 
 /***********************************************************/
@@ -6309,6 +6437,7 @@ enum {
     QEMU_OPTION_no_reboot,
     QEMU_OPTION_daemonize,
     QEMU_OPTION_option_rom,
+    QEMU_OPTION_incoming,
 };
 
 typedef struct QEMUOption {
@@ -6374,6 +6503,7 @@ const QEMUOption qemu_options[] = {
     { "serial", 1, QEMU_OPTION_serial },
     { "parallel", 1, QEMU_OPTION_parallel },
     { "loadvm", HAS_ARG, QEMU_OPTION_loadvm },
+    { "incoming", 0, QEMU_OPTION_incoming },
     { "full-screen", 0, QEMU_OPTION_full_screen },
 #ifdef CONFIG_SDL
     { "no-quit", 0, QEMU_OPTION_no_quit },
@@ -6594,6 +6724,17 @@ static BOOL WINAPI qemu_ctrl_handler(DWO
 #endif
 
 #define MAX_NET_CLIENTS 32
+
+static int saved_argc;
+static char **saved_argv;
+
+void qemu_get_launch_info(int *argc, char ***argv, int *opt_daemonize, int *opt_incoming)
+{
+    *argc = saved_argc;
+    *argv = saved_argv;
+    *opt_daemonize = daemonize;
+    *opt_incoming = incoming;
+}
 
 int main(int argc, char **argv)
 {
@@ -6623,6 +6764,9 @@ int main(int argc, char **argv)
     char usb_devices[MAX_USB_CMDLINE][128];
     int usb_devices_index;
     int fds[2];
+
+    saved_argc = argc;
+    saved_argv = argv;
 
     LIST_INIT (&vm_change_state_head);
 #ifndef _WIN32
@@ -6987,6 +7131,9 @@ int main(int argc, char **argv)
 	    case QEMU_OPTION_loadvm:
 		loadvm = optarg;
 		break;
+	    case QEMU_OPTION_incoming:
+		incoming = 1;
+		break;
             case QEMU_OPTION_full_screen:
                 full_screen = 1;
                 break;
@@ -7057,11 +7204,6 @@ int main(int argc, char **argv)
     }
 
 #ifndef _WIN32
-    if (daemonize && !nographic && vnc_display == NULL) {
-	fprintf(stderr, "Can only daemonize if using -nographic or -vnc\n");
-	daemonize = 0;
-    }
-
     if (daemonize) {
 	pid_t pid;
 
@@ -7096,7 +7238,6 @@ int main(int argc, char **argv)
 	    exit(1);
 
 	umask(027);
-	chdir("/");
 
         signal(SIGTSTP, SIG_IGN);
         signal(SIGTTOU, SIG_IGN);
@@ -7240,7 +7381,7 @@ int main(int argc, char **argv)
     }
 
     register_savevm("timer", 0, 2, timer_save, timer_load, NULL);
-    register_savevm("ram", 0, 2, ram_save, ram_load, NULL);
+    register_savevm("ram", 0, 3, ram_save, ram_load, NULL);
 
     init_ioports();
 
@@ -7322,8 +7463,16 @@ int main(int argc, char **argv)
         }
     } else 
 #endif
-    if (loadvm)
-        do_loadvm(loadvm);
+    if (loadvm) {
+	do_loadvm(loadvm);
+    }
+
+    if (incoming) {
+	if (migrate_incoming(STDIN_FILENO) == -1) {
+	    printf("Migration failed\n");
+	    exit(1);
+	}
+    }
 
     {
         /* XXX: simplify init */
@@ -7346,6 +7495,7 @@ int main(int argc, char **argv)
 	if (len != 1)
 	    exit(1);
 
+	chdir("/");
 	fd = open("/dev/null", O_RDWR);
 	if (fd == -1)
 	    exit(1);
diff -r 8ad5450b580b vl.h
--- a/vl.h	Tue Jan 16 17:04:55 2007 -0600
+++ b/vl.h	Sun Jan 21 18:06:27 2007 -0600
@@ -107,8 +107,12 @@ char *pstrcat(char *buf, int buf_size, c
 char *pstrcat(char *buf, int buf_size, const char *s);
 int strstart(const char *str, const char *val, const char **ptr);
 int stristart(const char *str, const char *val, const char **ptr);
+int hex2bin(char ch);
+char *urldecode(const char *ptr);
 
 /* vl.c */
+void qemu_get_launch_info(int *argc, char ***argv, int *opt_daemonize, int *opt_incoming);
+
 uint64_t muldiv64(uint64_t a, uint32_t b, uint32_t c);
 
 void hw_error(const char *fmt, ...);
@@ -436,8 +440,8 @@ typedef void (QEMUFileCloseFunc)(void *o
 
 QEMUFile *qemu_fopen(void *opaque, QEMUFilePutBufferFunc *put_buffer,
 		     QEMUFileGetBufferFunc *get_buffer, QEMUFileCloseFunc *close);
-
 QEMUFile *qemu_fopen_file(const char *filename, const char *mode);
+QEMUFile *qemu_fopen_fd(int fd);
 void qemu_fflush(QEMUFile *f);
 void qemu_fclose(QEMUFile *f);
 void qemu_put_buffer(QEMUFile *f, const uint8_t *buf, int size);
@@ -525,6 +529,9 @@ void do_loadvm(const char *name);
 void do_loadvm(const char *name);
 void do_delvm(const char *name);
 void do_info_snapshots(void);
+
+int qemu_live_savevm_state(QEMUFile *f);
+int qemu_live_loadvm_state(QEMUFile *f);
 
 /* bottom halves */
 typedef void QEMUBHFunc(void *opaque);
@@ -1362,6 +1369,10 @@ pflash_t *pflash_register (target_ulong 
 
 #endif /* defined(QEMU_TOOL) */
 
+/* migration.c */
+void do_migrate(int detach, const char *command);
+int migrate_incoming(int fd);
+
 /* monitor.c */
 void monitor_init(CharDriverState *hd, int show_banner);
 void term_puts(const char *str);
@@ -1372,6 +1383,8 @@ void term_print_help(void);
 void term_print_help(void);
 void monitor_readline(const char *prompt, int is_password,
                       char *buf, int buf_size);
+void monitor_suspend(void);
+void monitor_resume(void);
 
 /* readline.c */
 typedef void ReadLineFunc(void *opaque, const char *str);
diff -r 8ad5450b580b migration.c
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/migration.c	Sun Jan 21 20:52:26 2007 -0600
@@ -0,0 +1,392 @@
+/*
+ * QEMU migration support
+ * 
+ * Copyright (C) 2007 Anthony Liguori <anthony@codemonkey.ws>
+ * 
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+
+#include "vl.h"
+
+#include <sys/wait.h>
+
+#define MAX_THROTTLE		(32 << 20)
+#define MIN_FINALIZE_SIZE	(200 << 10)
+
+typedef struct MigrationState
+{
+    int fd;
+    pid_t pid;
+    int detach;
+    int throttle_count;
+    int n_buffer;
+    int *has_error;
+    char buffer[TARGET_PAGE_SIZE + 4];
+    target_ulong addr;
+    int updated_pages;
+    QEMUTimer *timer;
+} MigrationState;
+
+/* QEMUFile migration implementation */
+
+static void migrate_put_buffer(void *opaque, const uint8_t *buf, int64_t pos, int size)
+{
+    MigrationState *s = opaque;
+    int offset = 0;
+
+    if (*s->has_error)
+	return;
+
+    while (offset < size) {
+	ssize_t len;
+
+	len = write(s->fd, buf + offset, size - offset);
+	if (len == -1) {
+	    if (errno == EAGAIN || errno == EINTR)
+		continue;
+	    *s->has_error = 1;
+	    break;
+	} else if (len == 0) {
+	    *s->has_error = 1;
+	    break;
+	}
+
+	offset += len;
+    }
+}
+
+static void migrate_close(void *opaque)
+{
+    MigrationState *s = opaque;
+    int status = 0;
+
+    close(s->fd);
+again:
+    if (!s->detach && waitpid(s->pid, &status, 0) == -1) {
+	if (errno == EINTR || errno == EAGAIN)
+	    goto again;
+    }
+
+    if (status != 0)
+	*s->has_error = 1;
+
+    qemu_free(s);
+}
+
+/* Outgoing migration routines */
+
+static void migrate_finish(MigrationState *s)
+{
+    QEMUFile *f;
+    int ret;
+    int *has_error = s->has_error;
+
+    fcntl(s->fd, F_SETFL, 0);
+
+    f = qemu_fopen(s, migrate_put_buffer, NULL, migrate_close);
+    qemu_aio_flush();
+    vm_stop(0);
+    qemu_put_be32(f, 1);
+    ret = qemu_live_savevm_state(f);
+    qemu_fclose(f);
+    if (ret != 0 || *has_error) {
+	term_printf("Migration failed!\n");
+	vm_start();
+    }
+    if (!s->detach)
+	monitor_resume();
+    qemu_free(has_error);
+    cpu_physical_memory_set_dirty_tracking(0);
+}
+
+static int migrate_write_buffer(MigrationState *s)
+{
+    if (*s->has_error)
+	return 0;
+
+    if (s->n_buffer != sizeof(s->buffer)) {
+	ssize_t len;
+    again:
+	len = write(s->fd, s->buffer + s->n_buffer, sizeof(s->buffer) - s->n_buffer);
+	if (len == -1) {
+	    if (errno == EINTR)
+		goto again;
+	    if (errno == EAGAIN)
+		return 1;
+	    *s->has_error = 1;
+	    return 0;
+	}
+	if (len == 0) {
+	    *s->has_error = 1;
+	    return 0;
+	}
+
+	s->throttle_count += len;
+	s->n_buffer += len;
+	if (s->n_buffer != sizeof(s->buffer))
+	    goto again;
+    }
+
+    if (s->throttle_count > MAX_THROTTLE) {
+	printf("exceeded throttle count\n");
+	qemu_set_fd_handler2(s->fd, NULL, NULL, NULL, NULL);
+	return 1;
+    }
+
+    return 0;
+}
+
+static int migrate_check_convergence(MigrationState *s)
+{
+    target_ulong addr;
+    int dirty_count = 0;
+
+    for (addr = 0; addr < phys_ram_size; addr += TARGET_PAGE_SIZE) {
+	if (cpu_physical_memory_get_dirty(addr, MIGRATION_DIRTY_FLAG))
+	    dirty_count++;
+    }
+
+    return ((dirty_count * TARGET_PAGE_SIZE) < MIN_FINALIZE_SIZE);
+}
+
+static void migrate_write(void *opaque)
+{
+    MigrationState *s = opaque;
+
+    if (migrate_write_buffer(s))
+	return;
+
+    if (migrate_check_convergence(s)) {
+	qemu_del_timer(s->timer);
+	qemu_free_timer(s->timer);
+	qemu_set_fd_handler2(s->fd, NULL, NULL, NULL, NULL);
+	migrate_finish(s);
+	return;
+    }	
+
+    while (s->addr < phys_ram_size) {
+	if (cpu_physical_memory_get_dirty(s->addr, MIGRATION_DIRTY_FLAG)) {
+	    uint32_t value = cpu_to_be32(s->addr);
+
+	    memcpy(s->buffer, &value, 4);
+	    memcpy(s->buffer + 4, phys_ram_base + s->addr, TARGET_PAGE_SIZE);
+	    s->n_buffer = 0;
+
+	    cpu_physical_memory_reset_dirty(s->addr, s->addr + TARGET_PAGE_SIZE, MIGRATION_DIRTY_FLAG);
+
+	    s->addr += TARGET_PAGE_SIZE;
+
+	    if (migrate_write_buffer(s))
+		return;
+	} else
+	    s->addr += TARGET_PAGE_SIZE;
+    }
+
+    s->updated_pages = 0;
+    s->addr = 0;
+}
+
+static void migrate_reset_throttle(void *opaque)
+{
+    MigrationState *s = opaque;
+
+    if (s->throttle_count > MAX_THROTTLE) {
+	qemu_set_fd_handler2(s->fd, NULL, NULL, migrate_write, s);
+    }
+    s->throttle_count = 0;
+    qemu_mod_timer(s->timer, qemu_get_clock(rt_clock) + 1000);
+}
+
+static int start_migration(MigrationState *s)
+{
+    uint32_t value = cpu_to_be32(phys_ram_size);
+    target_phys_addr_t addr;
+    size_t offset = 0;
+	
+    while (offset != 4) {
+	ssize_t len = write(s->fd, ((char *)&value) + offset, 4 - offset);
+	if (len == -1 && errno == EINTR)
+	    continue;
+
+	if (len < 1)
+	    return -EIO;
+
+	offset += len;
+    }
+
+    fcntl(s->fd, F_SETFL, O_NONBLOCK);
+
+    for (addr = 0; addr < phys_ram_size; addr += TARGET_PAGE_SIZE) {
+	if (!cpu_physical_memory_get_dirty(addr, MIGRATION_DIRTY_FLAG))
+	    cpu_physical_memory_set_dirty(addr);
+    }
+
+    cpu_physical_memory_set_dirty_tracking(1);
+
+    s->updated_pages = 0;
+    s->addr = 0;
+    s->n_buffer = sizeof(s->buffer);
+    s->timer = qemu_new_timer(rt_clock, migrate_reset_throttle, s);
+
+    qemu_mod_timer(s->timer, qemu_get_clock(rt_clock));
+    qemu_set_fd_handler2(s->fd, NULL, NULL, migrate_write, s);
+}
+
+/* Incoming migration */
+
+int migrate_incoming(int fd)
+{
+    int ret;
+    QEMUFile *f = qemu_fopen_fd(STDIN_FILENO);
+    uint32_t addr;
+
+    if (qemu_get_be32(f) != phys_ram_size)
+	return -1;
+	    
+    do {
+	int l;
+	addr = qemu_get_be32(f);
+	if (addr == 1)
+	    break;
+	l = qemu_get_buffer(f, phys_ram_base + addr, TARGET_PAGE_SIZE);
+	if (l != TARGET_PAGE_SIZE)
+	    return -1;
+    } while (1);
+
+    qemu_aio_flush();
+    vm_stop(0);
+    ret = qemu_live_loadvm_state(f);
+    qemu_fclose(f);
+
+    return ret;
+}
+
+/* Migration monitor command */
+
+void do_migrate(int detach, const char *uri)
+{
+    MigrationState *s;
+    int fds[2];
+    const char *ptr;
+    char *command = NULL;
+    char **argv = NULL;
+    int i;
+
+    if (strstart(uri, "exec:", &ptr)) {
+	char *subcommand = urldecode(ptr);
+	if (subcommand == NULL) {
+	    term_printf("Allocation error\n");
+	    return;
+	}
+
+	command = strdup("/bin/sh");
+	argv = qemu_mallocz(sizeof(char *) * 4);
+	argv[0] = strdup("sh");
+	argv[1] = strdup("-c");
+	argv[2] = subcommand;
+	argv[3] = NULL;
+    } else if (strstart(uri, "ssh:", &ptr)) {
+	char *host, *end;
+	int qemu_argc, daemonize = 0, incoming = 0, argc, i;
+	char **qemu_argv;
+
+	if (ptr[0] != '/' && ptr[1] != '/') {
+	    term_printf("Malformed ssh uri\n");
+	    return;
+	}
+
+	qemu_get_launch_info(&qemu_argc, &qemu_argv, &daemonize, &incoming);
+
+	argc = 3 + qemu_argc;
+	if (!daemonize)
+	    argc++;
+	if (!incoming)
+	    argc++;
+
+	host = strdup(ptr + 2);
+	end = strchr(host, '/');
+	if (end) *end = 0;
+
+	command = strdup("ssh");
+	argv = qemu_mallocz(sizeof(char *) * (argc + 1));
+	argv[0] = strdup("ssh");
+	argv[1] = strdup("-XC");
+	argv[2] = host;
+
+	for (i = 0; i < qemu_argc; i++)
+	    argv[3 + i] = strdup(qemu_argv[i]);
+
+	if (!daemonize)
+	    argv[3 + i++] = strdup("-daemonize");
+	if (!incoming)
+	    argv[3 + i++] = strdup("-incoming");
+	argv[3 + i] = NULL;
+    } else {
+	term_printf("Unknown migration protocol '%s'\n", uri);
+	return;
+    }
+
+    if (pipe(fds) == -1) {
+	term_printf("pipe()\n");
+	return;
+    }
+
+#if 0
+    term_printf("execlp('%s'", command);
+    for (i = 0; argv[i]; i++) {
+	term_printf(", '%s'", argv[i]);
+    }
+    term_printf(", NULL);\n");
+    return;
+#endif
+
+    s = qemu_mallocz(sizeof(MigrationState));
+    if (s == NULL) {
+	term_printf("Allocation error\n");
+	return;
+    }
+
+    s->fd = fds[1];
+    s->pid = fork();
+    if (s->pid == -1) {
+	term_printf("fork error\n");
+	return;
+    }
+    if (s->pid == 0) {
+	close(fds[1]);
+	dup2(fds[0], STDIN_FILENO);
+	execvp(command, argv);
+	exit(1);
+    } else
+	close(fds[0]);
+
+    s->detach = detach;
+    s->has_error = qemu_mallocz(sizeof(int));
+#if 1
+    qemu_free(command);
+    for (i = 0; argv[i]; i++)
+	qemu_free(argv[i]);
+    qemu_free(argv);
+#endif
+
+    if (start_migration(s) == -1)
+	term_printf("Could not start migration\n");
+    else if (!s->detach)
+	monitor_suspend();
+}

[-- Attachment #3: qemu-apic-save-restore.diff --]
[-- Type: text/x-patch, Size: 438 bytes --]

diff -r 533ed6dbb849 hw/apic.c
--- a/hw/apic.c	Mon Jan 08 13:05:52 2007 -0600
+++ b/hw/apic.c	Sun Jan 14 18:58:42 2007 -0600
@@ -831,7 +831,7 @@ int apic_init(CPUState *env)
     }
     s->timer = qemu_new_timer(vm_clock, apic_timer, s);
 
-    register_savevm("apic", 0, 1, apic_save, apic_load, s);
+    register_savevm("apic", 0, 2, apic_save, apic_load, s);
     qemu_register_reset(apic_reset, s);
     
     local_apics[s->id] = s;

[-- Attachment #4: qemu-file.diff --]
[-- Type: text/x-patch, Size: 8161 bytes --]

diff -r eb6f93d3d545 audio/wavaudio.c
--- a/audio/wavaudio.c	Sun Jan 14 18:58:45 2007 -0600
+++ b/audio/wavaudio.c	Sun Jan 14 19:00:22 2007 -0600
@@ -151,7 +151,7 @@ static int wav_init_out (HWVoiceOut *hw,
     le_store (hdr + 28, hw->info.freq << (bits16 + stereo), 4);
     le_store (hdr + 32, 1 << (bits16 + stereo), 2);
 
-    wav->f = qemu_fopen (conf.wav_path, "wb");
+    wav->f = qemu_fopen_file (conf.wav_path, "wb");
     if (!wav->f) {
         dolog ("Failed to open wave file `%s'\nReason: %s\n",
                conf.wav_path, strerror (errno));
diff -r eb6f93d3d545 audio/wavcapture.c
--- a/audio/wavcapture.c	Sun Jan 14 18:58:45 2007 -0600
+++ b/audio/wavcapture.c	Sun Jan 14 19:00:22 2007 -0600
@@ -132,7 +132,7 @@ int wav_start_capture (CaptureState *s, 
     le_store (hdr + 28, freq << shift, 4);
     le_store (hdr + 32, 1 << shift, 2);
 
-    wav->f = qemu_fopen (path, "wb");
+    wav->f = qemu_fopen_file (path, "wb");
     if (!wav->f) {
         term_printf ("Failed to open wave file `%s'\nReason: %s\n",
                      path, strerror (errno));
diff -r eb6f93d3d545 vl.c
--- a/vl.c	Sun Jan 14 18:58:45 2007 -0600
+++ b/vl.c	Sun Jan 14 19:06:50 2007 -0600
@@ -4319,11 +4319,11 @@ void qemu_del_wait_object(HANDLE handle,
 #define IO_BUF_SIZE 32768
 
 struct QEMUFile {
-    FILE *outfile;
-    BlockDriverState *bs;
-    int is_file;
-    int is_writable;
-    int64_t base_offset;
+    QEMUFilePutBufferFunc *put_buffer;
+    QEMUFileGetBufferFunc *get_buffer;
+    QEMUFileCloseFunc *close;
+    void *opaque;
+
     int64_t buf_offset; /* start of buffer when writing, end of buffer
                            when reading */
     int buf_index;
@@ -4331,58 +4331,148 @@ struct QEMUFile {
     uint8_t buf[IO_BUF_SIZE];
 };
 
-QEMUFile *qemu_fopen(const char *filename, const char *mode)
+typedef struct QEMUFileFD
+{
+    int fd;
+} QEMUFileFD;
+
+static int fd_get_buffer(void *opaque, uint8_t *buf, int64_t pos, int size)
+{
+    QEMUFileFD *s = opaque;
+    int offset = 0;
+
+    while (offset < size) {
+	ssize_t len;
+
+	len = read(s->fd, buf + offset, size - offset);
+	if (len == -1) {
+	    if (errno == EINTR || errno == EAGAIN)
+		continue;
+	    return -1;
+	} else if (len == 0)
+	    break;
+	offset += len;
+    }
+
+    return offset;
+}
+
+QEMUFile *qemu_fopen_fd(int fd)
+{
+    QEMUFileFD *s = qemu_mallocz(sizeof(QEMUFileFD));
+    s->fd = fd;
+    return qemu_fopen(s, NULL, fd_get_buffer, qemu_free);
+}
+
+typedef struct QEMUFileUnix
+{
+    FILE *outfile;
+} QEMUFileUnix;
+
+static void file_put_buffer(void *opaque, const uint8_t *buf, int64_t pos, int size)
+{
+    QEMUFileUnix *s = opaque;
+    fseek(s->outfile, pos, SEEK_SET);
+    fwrite(buf, 1, size, s->outfile);
+}
+
+static int file_get_buffer(void *opaque, uint8_t *buf, int64_t pos, int size)
+{
+    QEMUFileUnix *s = opaque;
+    fseek(s->outfile, pos, SEEK_SET);
+    return fread(buf, 1, size, s->outfile);
+}
+
+static void file_close(void *opaque)
+{
+    QEMUFileUnix *s = opaque;
+    fclose(s->outfile);
+    qemu_free(s);
+}
+
+QEMUFile *qemu_fopen_file(const char *filename, const char *mode)
+{
+    QEMUFileUnix *s;
+
+    s = qemu_mallocz(sizeof(QEMUFileUnix));
+    if (!s)
+        return NULL;
+
+    s->outfile = fopen(filename, mode);
+    if (!s->outfile)
+        goto fail;
+
+    if (!strcmp(mode, "wb"))
+	return qemu_fopen(s, file_put_buffer, NULL, file_close);
+    else if (!strcmp(mode, "rb"))
+	return qemu_fopen(s, NULL, file_get_buffer, file_close);
+
+fail:
+    if (s->outfile)
+        fclose(s->outfile);
+    qemu_free(s);
+    return NULL;
+}
+
+typedef struct QEMUFileBdrv
+{
+    BlockDriverState *bs;
+    int64_t base_offset;
+} QEMUFileBdrv;
+
+static void bdrv_put_buffer(void *opaque, const uint8_t *buf, int64_t pos, int size)
+{
+    QEMUFileBdrv *s = opaque;
+    bdrv_pwrite(s->bs, s->base_offset + pos, buf, size);
+}
+
+static int bdrv_get_buffer(void *opaque, uint8_t *buf, int64_t pos, int size)
+{
+    QEMUFileBdrv *s = opaque;
+    return bdrv_pread(s->bs, s->base_offset + pos, buf, size);
+}
+
+QEMUFile *qemu_fopen_bdrv(BlockDriverState *bs, int64_t offset, int is_writable)
+{
+    QEMUFileBdrv *s;
+
+    s = qemu_mallocz(sizeof(QEMUFileBdrv));
+    if (!s)
+        return NULL;
+
+    s->bs = bs;
+    s->base_offset = offset;
+
+    if (is_writable)
+	return qemu_fopen(s, bdrv_put_buffer, NULL, qemu_free);
+
+    return qemu_fopen(s, NULL, bdrv_get_buffer, qemu_free);
+}
+
+QEMUFile *qemu_fopen(void *opaque, QEMUFilePutBufferFunc *put_buffer,
+		     QEMUFileGetBufferFunc *get_buffer, QEMUFileCloseFunc *close)
 {
     QEMUFile *f;
 
     f = qemu_mallocz(sizeof(QEMUFile));
     if (!f)
-        return NULL;
-    if (!strcmp(mode, "wb")) {
-        f->is_writable = 1;
-    } else if (!strcmp(mode, "rb")) {
-        f->is_writable = 0;
-    } else {
-        goto fail;
-    }
-    f->outfile = fopen(filename, mode);
-    if (!f->outfile)
-        goto fail;
-    f->is_file = 1;
+	return NULL;
+
+    f->opaque = opaque;
+    f->put_buffer = put_buffer;
+    f->get_buffer = get_buffer;
+    f->close = close;
+
     return f;
- fail:
-    if (f->outfile)
-        fclose(f->outfile);
-    qemu_free(f);
-    return NULL;
-}
-
-QEMUFile *qemu_fopen_bdrv(BlockDriverState *bs, int64_t offset, int is_writable)
-{
-    QEMUFile *f;
-
-    f = qemu_mallocz(sizeof(QEMUFile));
-    if (!f)
-        return NULL;
-    f->is_file = 0;
-    f->bs = bs;
-    f->is_writable = is_writable;
-    f->base_offset = offset;
-    return f;
 }
 
 void qemu_fflush(QEMUFile *f)
 {
-    if (!f->is_writable)
+    if (!f->put_buffer)
         return;
+
     if (f->buf_index > 0) {
-        if (f->is_file) {
-            fseek(f->outfile, f->buf_offset, SEEK_SET);
-            fwrite(f->buf, 1, f->buf_index, f->outfile);
-        } else {
-            bdrv_pwrite(f->bs, f->base_offset + f->buf_offset, 
-                        f->buf, f->buf_index);
-        }
+	f->put_buffer(f->opaque, f->buf, f->buf_offset, f->buf_index);
         f->buf_offset += f->buf_index;
         f->buf_index = 0;
     }
@@ -4392,19 +4482,13 @@ static void qemu_fill_buffer(QEMUFile *f
 {
     int len;
 
-    if (f->is_writable)
+    if (!f->get_buffer)
         return;
-    if (f->is_file) {
-        fseek(f->outfile, f->buf_offset, SEEK_SET);
-        len = fread(f->buf, 1, IO_BUF_SIZE, f->outfile);
-        if (len < 0)
-            len = 0;
-    } else {
-        len = bdrv_pread(f->bs, f->base_offset + f->buf_offset, 
-                         f->buf, IO_BUF_SIZE);
-        if (len < 0)
-            len = 0;
-    }
+
+    len = f->get_buffer(f->opaque, f->buf, f->buf_offset, IO_BUF_SIZE);
+    if (len < 0)
+	len = 0;
+
     f->buf_index = 0;
     f->buf_size = len;
     f->buf_offset += len;
@@ -4412,11 +4496,9 @@ static void qemu_fill_buffer(QEMUFile *f
 
 void qemu_fclose(QEMUFile *f)
 {
-    if (f->is_writable)
-        qemu_fflush(f);
-    if (f->is_file) {
-        fclose(f->outfile);
-    }
+    qemu_fflush(f);
+    if (f->close)
+	f->close(f->opaque);
     qemu_free(f);
 }
 
@@ -4491,7 +4573,7 @@ int64_t qemu_fseek(QEMUFile *f, int64_t 
         /* SEEK_END not supported */
         return -1;
     }
-    if (f->is_writable) {
+    if (f->put_buffer) {
         qemu_fflush(f);
         f->buf_offset = pos;
     } else {
diff -r eb6f93d3d545 vl.h
--- a/vl.h	Sun Jan 14 18:58:45 2007 -0600
+++ b/vl.h	Sun Jan 14 19:06:50 2007 -0600
@@ -430,7 +430,14 @@ void cpu_disable_ticks(void);
 
 typedef struct QEMUFile QEMUFile;
 
-QEMUFile *qemu_fopen(const char *filename, const char *mode);
+typedef void (QEMUFilePutBufferFunc)(void *opaque, const uint8_t *buf, int64_t pos, int size);
+typedef int (QEMUFileGetBufferFunc)(void *opaque, uint8_t *buf, int64_t pos, int size);
+typedef void (QEMUFileCloseFunc)(void *opaque);
+
+QEMUFile *qemu_fopen(void *opaque, QEMUFilePutBufferFunc *put_buffer,
+		     QEMUFileGetBufferFunc *get_buffer, QEMUFileCloseFunc *close);
+
+QEMUFile *qemu_fopen_file(const char *filename, const char *mode);
 void qemu_fflush(QEMUFile *f);
 void qemu_fclose(QEMUFile *f);
 void qemu_put_buffer(QEMUFile *f, const uint8_t *buf, int size);

[Qemu-devel] QEMU Live Migration: arp-like packet patch

[Uri Lublin]

[-- Attachment #1.1: Type: text/plain, Size: 1962 bytes --]

Hello,

We, kvm developers at Qumranet, have been developing a Live Migration solution
    for qemu (and kvm) too (http://kvm.sourceforge.net/migration.html).
We are working with qemu-0.8.2 and are planning to upgrade to the current 
    qemu CVS.
Anthony Liguori sent patches implementing Live Migration 
    (http://lists.gnu.org/archive/html/qemu-devel/2007-01/msg00245.html).
Some differences are:
  - Anthony's solution uses ssh our solution uses TCP sockets.
  - Anthony's interface is a single qemu-monitor command accepting a uri,
        our interface is a single qemu command accepting a few subcommands.
  - Anthony's solution supports migration start without any preparations,
        our solution requires starting a guest (qemu) on the destination host, and
        establish connection before starting the migration.

Naturally, Anthony's solution can be easily enhanced to support most of our interface.

Since there is no need for two different migration solutions, we'd like to
    cooperate with Anthony (and with you) and contribute some patches which
    are additions to his solution.
Those patches would be based on Anthony's solution (and have to be applied after 
    his patches are applied).

The first patch (attached) is an arp-like packet sent (broadcast) when a 
    migration successfully completes.
The purpose of this packet is to inform network switches that the guest moved.
Although the most known unsolicited packet is the famous "gratuitous arp",  
    we are not sending an arp packet since we do not know the guest's ip address,
    and we do not want to receive replies.
Instead we use an ethernet "experimental" packet.
The packet is sent only through tap interfaces.
Our implementation currently supports only linux hosts (adding support for 
    other operating systems is probably/hopefully as easy as changing the include
    preprocessor directive).

Regards,
    Uri Lublin.

[-- Attachment #1.2: Type: text/html, Size: 2955 bytes --]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: migration_introduce_self_packet.diff --]
[-- Type: text/x-patch; name="migration_introduce_self_packet.diff", Size: 1980 bytes --]

--- qemu/vl.c	2007-01-23 10:37:50.000000000 +0200
+++ qemu-new/vl.c	2007-01-23 10:26:49.000000000 +0200
@@ -51,6 +51,7 @@
 #ifndef __sun__
 #include <linux/if.h>
 #include <linux/if_tun.h>
+#include <linux/if_ether.h>
 #include <pty.h>
 #include <malloc.h>
 #include <linux/rtc.h>
@@ -3940,6 +3941,50 @@
     }
 }
 
+#if defined(__linux__)
+#define SELF_ANNOUNCE_ROUNDS 5
+#define ETH_P_EXPERIMENTAL 0x01F1 /* just a number in experimental range */
+#define EXPERIMENTAL_MAGIC 0xf1f23f4f
+int announce_self_create(unsigned char *buf, 
+                         unsigned char mac_addr[ETH_ALEN])
+{
+    struct ethhdr *eh = (struct ethhdr*)buf;
+    uint32_t *magic   = (uint32_t*)(eh+1);
+    unsigned char *p  = (unsigned char*)(magic + 1);
+
+    /* ethernet header */
+    memset(eh->h_dest,   0xff,     ETH_ALEN);
+    memcpy(eh->h_source, mac_addr, ETH_ALEN);
+    eh->h_proto = htons(ETH_P_EXPERIMENTAL);
+
+    /* magic data */
+    *magic = EXPERIMENTAL_MAGIC;
+
+    return p - buf; /* sizeof(*eh) + sizeof(*magic) */
+}
+
+void qemu_tap_announce_self(void)
+{
+    int i, j, len;
+    VLANState *vlan;
+    VLANClientState *vc;
+    uint8_t buf[256];
+
+    for (i=0; i<nb_nics; i++) {     /* for all nics */
+        len = announce_self_create(buf, nd_table[i].macaddr);
+        vlan = nd_table[i].vlan;
+        for(vc = vlan->first_client; vc != NULL; vc = vc->next) {
+            if (vc->fd_read == tap_receive) { /* send only if tap */
+                for (j=0; j<SELF_ANNOUNCE_ROUNDS ; j++) {
+                    vc->fd_read(vc->opaque, buf, len);
+                }
+            }
+        } /* for vc -- look for tap_receive */
+    } /* for i -- all nics */
+}
+#else
+void qemu_tap_announce_self(void) {}
+#endif
 /***********************************************************/
 /* USB devices */
 
@@ -7480,6 +7525,9 @@
 	    printf("Migration failed\n");
 	    exit(1);
 	}
+        else {
+            qemu_tap_announce_self();
+        }
     }
 
     {

Re: [Qemu-devel] [RFC] QEMU Live Migration

[Fabrice Bellard]

I looked at your patch and it is OK for me. The 'ssh:' implementation is 
a nice trick. In order to be usable on win32, it is still needed to 
implement a more portable system with a TCP connection.

The dirty flag 0x04 is unfortunately used by kqemu - I should have 
documented it ! You can use 0x08 for the live migration.

Regards,

Fabrice.

Anthony Liguori wrote:
> Howdy,
> 
> I wanted to post a draft of my QEMU live migration patch.  I think 
> there's a lingering memory allocation problem.  I also need to implement 
> ARP forwarding and a TCP transport stream.
> 
> However, I thought I'd post an initial version in case anyone wants to 
> play around with it.
> 
> Once you apply the patch, make sure you've got it on both machines.  For 
> my setup, I've got a common NFS share mounted in /mnt.  Then I run:
> 
> qemu -hda /mnt/win2k.img -monitor telnet:0.0.0.0:1025,server,nowait -vnc :2
> 
> Then, I connect to the monitor and execute:
> 
> (qemu) migrate ssh://woolly
> 
> When the migration completes, the monitor command will finish and then I 
> can connect to the machine on woolly:2.
> 
> The ssh command will be derived from argc/argv so it's important to 
> ensure that the -hda path is valid on both machines.
> 
> All three patches are required and should be applied in order of: 
> qemu-apic-save-restore.diff qemu-file.diff qemu-migration.diff
> 
> Regards,
> 
> Anthony Liguori

RE: [Qemu-devel] QEMU Live Migration: arp-like packet patch

[Uri Lublin]

[-- Attachment #1: Type: text/plain, Size: 2418 bytes --]


By now (more than two months after I sent it) this patch is less relevant, as it is already included in Anthony's updated qemu-live-migration patch.
Is live migration going to be accepted soon ?

Regards,
    Uri.

-----Original Message-----
From: qemu-devel-bounces+uril=qumranet.com@nongnu.org on behalf of Uri Lublin
Sent: Wed 24/01/2007 18:36
To: qemu-devel@nongnu.org
Subject: [Qemu-devel] QEMU Live Migration: arp-like packet patch
 
Hello,

We, kvm developers at Qumranet, have been developing a Live Migration solution
    for qemu (and kvm) too (http://kvm.sourceforge.net/migration.html).
We are working with qemu-0.8.2 and are planning to upgrade to the current 
    qemu CVS.
Anthony Liguori sent patches implementing Live Migration 
    (http://lists.gnu.org/archive/html/qemu-devel/2007-01/msg00245.html).
Some differences are:
  - Anthony's solution uses ssh our solution uses TCP sockets.
  - Anthony's interface is a single qemu-monitor command accepting a uri,
        our interface is a single qemu command accepting a few subcommands.
  - Anthony's solution supports migration start without any preparations,
        our solution requires starting a guest (qemu) on the destination host, and
        establish connection before starting the migration.

Naturally, Anthony's solution can be easily enhanced to support most of our interface.

Since there is no need for two different migration solutions, we'd like to
    cooperate with Anthony (and with you) and contribute some patches which
    are additions to his solution.
Those patches would be based on Anthony's solution (and have to be applied after 
    his patches are applied).

The first patch (attached) is an arp-like packet sent (broadcast) when a 
    migration successfully completes.
The purpose of this packet is to inform network switches that the guest moved.
Although the most known unsolicited packet is the famous "gratuitous arp",  
    we are not sending an arp packet since we do not know the guest's ip address,
    and we do not want to receive replies.
Instead we use an ethernet "experimental" packet.
The packet is sent only through tap interfaces.
Our implementation currently supports only linux hosts (adding support for 
    other operating systems is probably/hopefully as easy as changing the include
    preprocessor directive).

Regards,
    Uri Lublin.


[-- Attachment #2: winmail.dat --]
[-- Type: application/ms-tnef, Size: 4156 bytes --]