From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1ManSj-0007sp-9y for qemu-devel@nongnu.org; Tue, 11 Aug 2009 05:16:09 -0400 Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1ManSd-0007pF-Jk for qemu-devel@nongnu.org; Tue, 11 Aug 2009 05:16:07 -0400 Received: from [199.232.76.173] (port=52691 helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1ManSd-0007p5-7J for qemu-devel@nongnu.org; Tue, 11 Aug 2009 05:16:03 -0400 Received: from mx2.redhat.com ([66.187.237.31]:34755) by monty-python.gnu.org with esmtp (Exim 4.60) (envelope-from ) id 1ManSc-00039S-LC for qemu-devel@nongnu.org; Tue, 11 Aug 2009 05:16:03 -0400 Received: from int-mx2.corp.redhat.com (int-mx2.corp.redhat.com [172.16.27.26]) by mx2.redhat.com (8.13.8/8.13.8) with ESMTP id n7B9G1PI009203 for ; Tue, 11 Aug 2009 05:16:01 -0400 Message-ID: <4A8136CF.5040304@redhat.com> Date: Tue, 11 Aug 2009 11:15:59 +0200 From: Chris Lalancette MIME-Version: 1.0 Subject: Re: [Qemu-devel] [PATCH] Migration via unix sockets. References: <1249485869-23590-1-git-send-email-clalance@redhat.com> <4A7FF53F.2000106@redhat.com> <4A8005B4.9030600@redhat.com> In-Reply-To: <4A8005B4.9030600@redhat.com> Content-Type: multipart/mixed; boundary="------------080208060709080200050102" List-Id: qemu-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Avi Kivity Cc: qemu-devel@nongnu.org This is a multi-part message in MIME format. --------------080208060709080200050102 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Avi Kivity wrote: > On 08/10/2009 01:23 PM, Chris Lalancette wrote: >> Chris Lalancette wrote: >> >>> Implement migration via unix sockets. While you can fake this using >>> exec and netcat, this involves forking another process and is >>> generally not very nice. By doing this directly in qemu, we can avoid >>> the copy through the external nc command. This is useful for >>> implementations (such as libvirt) that want to do "secure" migration; >>> we pipe the data on the sending side into the unix socket, libvirt >>> picks it up, encrypts it, and transports it, and then on the remote >>> side libvirt decrypts it, dumps it to another unix socket, and >>> feeds it into qemu. >>> >>> The implementation is straightforward and looks very similar to >>> migration-exec.c and migration-tcp.c >>> >> ping? >> > > It would be nice to support migration via arbitrary fd using the recent > SCM_RIGHTS support. A possible implementation of that is attached. I have to say, though, that it is way more clumsy to use than the unix implementation I posted earlier. On the outgoing side, you have to use the "getfd" monitor command to pass the fd using SCM_RIGHTS, then you have to issue another monitor command to start the migration: (qemu) getfd migration # passes opened fd via SCM_RIGHTS (qemu) migrate -d fd:migration On the incoming side, you have to pass the fd via the command-line. That means that you have to first arrange for it not to be closed on exec, and it also means that qemu is now depending on the external program to correctly set up the incoming socket so that qemu can just do the accept() on it. None of these problems are insurmountable, but they do make it cumbersome to use in general, and very difficult to use from the command-line. The other option is that I've misunderstood your intent, and if that is the case, please correct me where I'm wrong :). -- Chris Lalancette --------------080208060709080200050102 Content-Type: text/x-patch; name="qemu-migration-fd.patch" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="qemu-migration-fd.patch" diff --git a/Makefile b/Makefile index c5763b7..a8cd0c9 100644 --- a/Makefile +++ b/Makefile @@ -110,7 +110,7 @@ obj-$(CONFIG_BRLAPI) += baum.o LIBS+=$(BRLAPI_LIBS) obj-$(CONFIG_WIN32) += tap-win32.o -obj-$(CONFIG_POSIX) += migration-exec.o +obj-$(CONFIG_POSIX) += migration-exec.o migration-fd.o ifdef CONFIG_COREAUDIO AUDIO_PT = y diff --git a/migration-fd.c b/migration-fd.c new file mode 100644 index 0000000..fea5a43 --- /dev/null +++ b/migration-fd.c @@ -0,0 +1,146 @@ +/* + * QEMU live migration via generic fd + * + * Copyright Red Hat, Inc. 2009 + * + * Authors: + * Chris Lalancette + * + * This work is licensed under the terms of the GNU GPL, version 2. See + * the COPYING file in the top-level directory. + * + */ + +#include "qemu-common.h" +#include "qemu_socket.h" +#include "migration.h" +#include "monitor.h" +#include "qemu-char.h" +#include "sysemu.h" +#include "buffered_file.h" +#include "block.h" +#include "qemu_socket.h" + +//#define DEBUG_MIGRATION_FD + +#ifdef DEBUG_MIGRATION_FD +#define dprintf(fmt, ...) \ + do { printf("migration-fd: " fmt, ## __VA_ARGS__); } while (0) +#else +#define dprintf(fmt, ...) \ + do { } while (0) +#endif + +static int fd_errno(FdMigrationState *s) +{ + return errno; +} + +static int fd_write(FdMigrationState *s, const void * buf, size_t size) +{ + return write(s->fd, buf, size); +} + +static int fd_close(FdMigrationState *s) +{ + dprintf("fd_close\n"); + if (s->fd != -1) { + close(s->fd); + s->fd = -1; + } + return 0; +} + +MigrationState *fd_start_outgoing_migration(Monitor *mon, + const char *fdname, + int64_t bandwidth_limit, + int detach) +{ + FdMigrationState *s; + + s = qemu_mallocz(sizeof(*s)); + + s->fd = monitor_get_fd(mon, fdname); + if (s->fd == -1) { + dprintf("fd_migration: no file descriptor supplied via SCM_RIGHTS\n"); + qemu_free(s); + return NULL; + } + + socket_set_nonblock(s->fd); + + s->get_error = fd_errno; + s->write = fd_write; + s->close = fd_close; + s->mig_state.cancel = migrate_fd_cancel; + s->mig_state.get_status = migrate_fd_get_status; + s->mig_state.release = migrate_fd_release; + + s->state = MIG_STATE_ACTIVE; + s->mon_resume = NULL; + s->bandwidth_limit = bandwidth_limit; + + if (!detach) + migrate_fd_monitor_suspend(s); + + migrate_fd_connect(s); + + return &s->mig_state; +} + +static void fd_accept_incoming_migration(void *opaque) +{ + struct sockaddr addr; + socklen_t addrlen = sizeof(addr); + int s = (unsigned long)opaque; + QEMUFile *f; + int c, ret; + + do { + c = accept(s, &addr, &addrlen); + } while (c == -1 && socket_error() == EINTR); + + if (c == -1) { + fprintf(stderr, "could not accept migration connection\n"); + return; + } + + dprintf("accepted migration\n"); + + f = qemu_fopen_socket(c); + if (f == NULL) { + fprintf(stderr, "could not qemu_fopen socket\n"); + goto out; + } + + ret = qemu_loadvm_state(f); + if (ret < 0) { + fprintf(stderr, "load of migration failed\n"); + goto out_fopen; + } + qemu_announce_self(); + dprintf("successfully loaded vm state\n"); + + /* we've successfully migrated, close the server socket */ + qemu_set_fd_handler2(s, NULL, NULL, NULL, NULL); + close(s); + +out_fopen: + qemu_fclose(f); +out: + close(c); +} + +int fd_start_incoming_migration(const char *infd) +{ + int fd; + + dprintf("Attempting to start an fd incoming migration\n"); + + fd = strtol(infd, NULL, 0); + + qemu_set_fd_handler2(fd, NULL, fd_accept_incoming_migration, NULL, + (void *)(unsigned long)fd); + + return 0; +} diff --git a/migration.c b/migration.c index 34e2bc1..c18d595 100644 --- a/migration.c +++ b/migration.c @@ -45,4 +45,6 @@ void qemu_start_incoming_migration(const char *uri) exec_start_incoming_migration(p); + else if (strstart(uri, "fd:", &p)) + fd_start_incoming_migration(p); #endif else fprintf(stderr, "unknown migration protocol: %s\n", uri); @@ -62,4 +64,6 @@ void do_migrate(Monitor *mon, int detach, const char *uri) s = exec_start_outgoing_migration(p, max_throttle, detach); + else if (strstart(uri, "fd:", &p)) + s = fd_start_outgoing_migration(mon, p, max_throttle, detach); #endif else monitor_printf(mon, "unknown migration protocol: %s\n", uri); diff --git a/migration.h b/migration.h index 0ed1fcb..96dad38 100644 --- a/migration.h +++ b/migration.h @@ -79,6 +79,13 @@ MigrationState *unix_start_outgoing_migration(const char *path, int64_t bandwidth_limit, int detach); +int fd_start_incoming_migration(const char *path); + +MigrationState *fd_start_outgoing_migration(Monitor *mon, + const char *fdname, + int64_t bandwidth_limit, + int detach); + void migrate_fd_monitor_suspend(FdMigrationState *s); void migrate_fd_error(FdMigrationState *s); --------------080208060709080200050102--