xen-devel.lists.xenproject.org archive mirror
 help / color / mirror / Atom feed
* Suggestion for merging xl save/restore/migrate/migrate-receive
@ 2013-09-13 16:04 Zhigang Wang
  2013-09-16 10:04 ` George Dunlap
  0 siblings, 1 reply; 22+ messages in thread
From: Zhigang Wang @ 2013-09-13 16:04 UTC (permalink / raw)
  To: xen-devel, Ian Jackson

[-- Attachment #1: Type: text/plain, Size: 424 bytes --]

Hi,

As we talked in
http://lists.xen.org/archives/html/xen-devel/2013-09/msg00211.html , I have a
suggestion: what about merging xl save/restore/migrate/migrate-receive?

Here is the description: xl-migrate.rst

Also there is a workaround for the xl migration daemon example: xl-migrate-socat.rst

Patch for the example: xen-xl-migrate-socat.patch

(I don't have enough knowledge to implement the merge.)

Thanks,

Zhigang

[-- Attachment #2: xen-xl-migrate-socat.patch --]
[-- Type: text/x-patch, Size: 7739 bytes --]

diff --git a/tools/hotplug/Linux/Makefile b/tools/hotplug/Linux/Makefile
index 47655f6..586728b 100644
--- a/tools/hotplug/Linux/Makefile
+++ b/tools/hotplug/Linux/Makefile
@@ -55,6 +55,8 @@ endif
 	$(INSTALL_PROG) $(XENCOMMONS_INITD) $(DESTDIR)$(INITD_DIR)
 	$(INSTALL_DATA) $(XENCOMMONS_SYSCONFIG) $(DESTDIR)$(SYSCONFIG_DIR)/xencommons
 	$(INSTALL_PROG) init.d/xen-watchdog $(DESTDIR)$(INITD_DIR)
+	$(INSTALL_PROG) init.d/xl-migrate-received $(DESTDIR)$(INITD_DIR)
+	$(INSTALL_PROG) init.d/xl-ssl-migrate-received $(DESTDIR)$(INITD_DIR)
 
 .PHONY: install-scripts
 install-scripts:
diff --git a/tools/hotplug/Linux/init.d/xl-migrate-received b/tools/hotplug/Linux/init.d/xl-migrate-received
new file mode 100644
index 0000000..d67cf7d
--- /dev/null
+++ b/tools/hotplug/Linux/init.d/xl-migrate-received
@@ -0,0 +1,106 @@
+#!/bin/bash
+#
+# xl-migrate-received: xl migrate receive daemon
+#
+# chkconfig: 2345 80 20
+# description: xl migrate receive daemon
+
+. /etc/rc.d/init.d/functions
+
+start() {
+    socat -ly TCP-LISTEN:8004,reuseaddr,fork EXEC:"xl migrate-receive" >/dev/null 2>&1 </dev/null &
+    pid=$(ps -eo pid,args | grep "socat -ly TCP-LISTEN:8004" | grep -v grep | awk '{print $1}')
+    if [ -n "$pid" ]; then
+        action $"Starting xl migrate receive daemon: " true
+        touch /var/lock/subsys/xl-migrate-received
+    else
+        action $"Starting xl migrate receive daemon: " false
+    fi
+}
+
+stop() {
+    pid=$(ps -eo pid,args | grep "socat -ly TCP-LISTEN:8004" | grep -v grep | awk '{print $1}')
+    action $"Stopping xl migrate receive daemon: " kill -9 $pid
+    [ $? -eq 0 ] && rm -f /var/lock/subsys/xl-migrate-received
+}
+
+status() {
+    pid=$(ps -eo pid,args | grep "socat -ly TCP-LISTEN:8004" | grep -v grep | awk '{print $1}')
+    if [ -n "$pid" ]; then
+        echo $"xl migrate receive daemon is running"
+    else
+        echo $"xl migrate receive daemon is stopped"
+    fi
+}
+
+case "$1" in
+    start)
+        start
+        ;;
+    stop)
+        stop
+        ;;
+    restart)
+        stop
+        start
+        ;;
+    status)
+        status
+        ;;
+    *)
+        echo $"Usage: $0 {start|stop|restart|status}"
+        exit 1
+esac
+#!/bin/bash
+#
+# xl-migrate-received: xl migrate receive daemon
+#
+# chkconfig: 2345 80 20
+# description: xl migrate receive daemon
+
+. /etc/rc.d/init.d/functions
+
+start() {
+    socat -ly TCP-LISTEN:8004,reuseaddr,fork EXEC:"xl migrate-receive" >/dev/null 2>&1 </dev/null &
+    pid=$(ps -eo pid,args | grep "socat -ly TCP-LISTEN:8004" | grep -v grep | awk '{print $1}')
+    if [ -n "$pid" ]; then
+        action $"Starting xl migrate receive daemon: " true
+        touch /var/lock/subsys/xl-migrate-received
+    else
+        action $"Starting xl migrate receive daemon: " false
+    fi
+}
+
+stop() {
+    pid=$(ps -eo pid,args | grep "socat -ly TCP-LISTEN:8004" | grep -v grep | awk '{print $1}')
+    action $"Stopping xl migrate receive daemon: " kill -9 $pid
+    [ $? -eq 0 ] && rm -f /var/lock/subsys/xl-migrate-received
+}
+
+status() {
+    pid=$(ps -eo pid,args | grep "socat -ly TCP-LISTEN:8004" | grep -v grep | awk '{print $1}')
+    if [ -n "$pid" ]; then
+        echo $"xl migrate receive daemon is running"
+    else
+        echo $"xl migrate receive daemon is stopped"
+    fi
+}
+
+case "$1" in
+    start)
+        start
+        ;;
+    stop)
+        stop
+        ;;
+    restart)
+        stop
+        start
+        ;;
+    status)
+        status
+        ;;
+    *)
+        echo $"Usage: $0 {start|stop|restart|status}"
+        exit 1
+esac
diff --git a/tools/hotplug/Linux/init.d/xl-ssl-migrate-received b/tools/hotplug/Linux/init.d/xl-ssl-migrate-received
new file mode 100644
index 0000000..272ed9c
--- /dev/null
+++ b/tools/hotplug/Linux/init.d/xl-ssl-migrate-received
@@ -0,0 +1,106 @@
+#!/bin/bash
+#
+# xl-ssl-migrate-received: xl ssl migrate receive daemon
+#
+# chkconfig: 2345 80 20
+# description: xl ssl migrate receive daemon
+
+. /etc/rc.d/init.d/functions
+
+start() {
+    socat -ly OPENSSL-LISTEN:8005,reuseaddr,fork,key=/etc/ovs-agent/cert/key.pem,cert=/etc/ovs-agent/cert/certificate.pem,verify=0 EXEC:"xl migrate-receive" >/dev/null 2>&1 </dev/null &
+    pid=$(ps -eo pid,args | grep "socat -ly OPENSSL-LISTEN:8005" | grep -v grep | awk '{print $1}')
+    if [ -n "$pid" ]; then
+        action $"Starting xl ssl migrate receive daemon: " true
+        touch /var/lock/subsys/xl-migrate-received
+    else
+        action $"Starting xl ssl migrate receive daemon: " false
+    fi
+}
+
+stop() {
+    pid=$(ps -eo pid,args | grep "socat -ly OPENSSL-LISTEN:8005" | grep -v grep | awk '{print $1}')
+    action $"Stopping xl ssl migrate receive daemon: " kill -9 $pid
+    [ $? -eq 0 ] && rm -f /var/lock/subsys/xl-migrate-received
+}
+
+status() {
+    pid=$(ps -eo pid,args | grep "socat -ly OPENSSL-LISTEN:8005" | grep -v grep | awk '{print $1}')
+    if [ -n "$pid" ]; then
+        echo $"xl ssl migrate receive daemon is running"
+    else
+        echo $"xl ssl migrate receive daemon is stopped"
+    fi
+}
+
+case "$1" in
+    start)
+        start
+        ;;
+    stop)
+        stop
+        ;;
+    restart)
+        stop
+        start
+        ;;
+    status)
+        status
+        ;;
+    *)
+        echo $"Usage: $0 {start|stop|restart|status}"
+        exit 1
+esac
+#!/bin/bash
+#
+# xl-ssl-migrate-received: xl ssl migrate receive daemon
+#
+# chkconfig: 2345 80 20
+# description: xl ssl migrate receive daemon
+
+. /etc/rc.d/init.d/functions
+
+start() {
+    socat -ly OPENSSL-LISTEN:8005,reuseaddr,fork,key=/etc/ovs-agent/cert/key.pem,cert=/etc/ovs-agent/cert/certificate.pem,verify=0 EXEC:"xl migrate-receive" >/dev/null 2>&1 </dev/null &
+    pid=$(ps -eo pid,args | grep "socat -ly OPENSSL-LISTEN:8005" | grep -v grep | awk '{print $1}')
+    if [ -n "$pid" ]; then
+        action $"Starting xl ssl migrate receive daemon: " true
+        touch /var/lock/subsys/xl-migrate-received
+    else
+        action $"Starting xl ssl migrate receive daemon: " false
+    fi
+}
+
+stop() {
+    pid=$(ps -eo pid,args | grep "socat -ly OPENSSL-LISTEN:8005" | grep -v grep | awk '{print $1}')
+    action $"Stopping xl ssl migrate receive daemon: " kill -9 $pid
+    [ $? -eq 0 ] && rm -f /var/lock/subsys/xl-migrate-received
+}
+
+status() {
+    pid=$(ps -eo pid,args | grep "socat -ly OPENSSL-LISTEN:8005" | grep -v grep | awk '{print $1}')
+    if [ -n "$pid" ]; then
+        echo $"xl ssl migrate receive daemon is running"
+    else
+        echo $"xl ssl migrate receive daemon is stopped"
+    fi
+}
+
+case "$1" in
+    start)
+        start
+        ;;
+    stop)
+        stop
+        ;;
+    restart)
+        stop
+        start
+        ;;
+    status)
+        status
+        ;;
+    *)
+        echo $"Usage: $0 {start|stop|restart|status}"
+        exit 1
+esac
diff --git a/tools/misc/Makefile b/tools/misc/Makefile
index 59def7a..88aaf20 100644
--- a/tools/misc/Makefile
+++ b/tools/misc/Makefile
@@ -16,7 +16,7 @@ TARGETS := $(TARGETS-y)
 
 SUBDIRS := $(SUBDIRS-y)
 
-INSTALL_BIN-y := xencons xencov_split
+INSTALL_BIN-y := xencons xencov_split xl-migrate-command xl-ssl-migrate-command
 INSTALL_BIN-$(CONFIG_X86) += xen-detect
 INSTALL_BIN := $(INSTALL_BIN-y)
 
diff --git a/tools/misc/xl-migrate-command b/tools/misc/xl-migrate-command
new file mode 100644
index 0000000..0558ff9
--- /dev/null
+++ b/tools/misc/xl-migrate-command
@@ -0,0 +1,3 @@
+#!/bin/sh
+
+socat - TCP:$1:8004
diff --git a/tools/misc/xl-ssl-migrate-command b/tools/misc/xl-ssl-migrate-command
new file mode 100644
index 0000000..628c554
--- /dev/null
+++ b/tools/misc/xl-ssl-migrate-command
@@ -0,0 +1,3 @@
+#!/bin/sh
+
+socat - OPENSSL:$1:8005,verify=0

[-- Attachment #3: xl-migrate-socat.rst --]
[-- Type: text/prs.fallenstein.rst, Size: 1133 bytes --]

==========
XL Migrate
==========

:Author: Zhigang Wang
:Contact: zhigang.x.wang@oracle.com
:Date: 2013-03-05

Current Status
==============

* xl migrate leverages ssh/sshd::

      xl migrate <domain> <host>

* In order to migrate a VM without user interactive, we have to configure ssh
  keys for all Servers in a pool. Key management with dynamic Server Pools is
  error prone.

* In certain cases, customers need non-ssl migrate, which greatly improves the
  migration speed. There's no way to do it with ssh.


Proposal
========

* Implement dedicated daemons for ssl and non-ssl migration receive.

* `socat <http://www.dest-unreach.org/socat/>`_ can be used.

* Patch: xen-xl-migrate-socat.patch


Testing
=======

* Install the new Xen.

* Start the services::

      service xl-migrate-received start
      service xl-ssl-migrate-received start

* Start a VM::

      xl create vm.cfg

* Migrate::

      xl migrate -s xl-migrate-command <domid> localhost

* SSL migrate::

      xl migrate -s xl-ssl-migrate-command <domid> localhost

* Speed for PV guest with 1024M memory:

  - ssh: 45s
  - socat: 9s
  - socat ssl: 57s

[-- Attachment #4: xl-migrate.rst --]
[-- Type: text/prs.fallenstein.rst, Size: 2509 bytes --]

==========
XL Migrate
==========

:Author: Zhigang Wang
:Contact: zhigang.x.wang@oracle.com
:Date: 2013-03-08

Current Status
==============

* xl migrate leverages ssh/sshd.

* In order to migrate a VM without user interactive, we have to configure ssh
  keys for all Servers in a pool. Key management brings complexity.

* In certain cases, customers need non-ssl migrate, which greatly improves the
  migration speed. There's no way to do it with ssh.

* Current xl migrate command is not intuitive, especially the `-s` option::

      # xl migrate
      Usage: xl [-v] migrate [options] <Domain> <host>
      
      Save a domain state to restore later.
      
      Options:
      
      -h              Print this help.
      -C <config>     Send <config> instead of config file from creation.
      -s <sshcommand> Use <sshcommand> instead of ssh.  String will be passed
                      to sh. If empty, run <host> instead of ssh <host> xl
                      migrate-receive [-d -e]
      -e              Do not wait in the background (on <host>) for the death
                      of the domain.

  It's a little hard to adapt other tools as transport.

* We have differnt implementation for `xl save/restore` and `xl migrate/migrate-receive`. Can we merge them?


Proposal
========

* Implement dedicated daemons for ssl and non-ssl migration receive (`socat <http://www.dest-unreach.org/socat/>`_ can be used).

  Example patch for dedicated migrate receive daemon: xen-xl-migrate-socat.patch

  `socat` will call `xl restore` instead of `xl migrate-receive` after the following change.

* Merge `xl migrate/migrate-receive` to `xl save/restore`:

  - To save a VM::

      # xl save [-c] <domain> -f vm.chk

    Or::

      # xl save [-c] <domain> >vm.chk

  - To restore a VM::

      # xl restore -f vm.chk

    Or::

      # cat vm.chk | xl restore

  - To migrate a VM using ssh/sshd::

      # xl save -c <domain> | ssh root@<remote-host> xl restore

    We can implement a wrapper to make `xl migrate <domain> <remote-host>` to call the above command. 

  - To migrate a VM using dedicated migrate receive daemon::

      # xl save -c <domain> | socat - TCP:<remote-host>:8004"

    Or with SSL::

      # xl save -c <domain> | socat - OPENSSL:<remote-host>:8005,verify=0

  - Localhost migration::

      # xl save <domain> | xl restrore

  - Localhost live migration::

      # xl save -c <domain> | xl restrore

  Patch: I don't have the capability and time to implement it yet.

[-- Attachment #5: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply related	[flat|nested] 22+ messages in thread
end of thread, other threads:[~2013-10-03 13:34 UTC | newest]

Thread overview: 22+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2013-09-13 16:04 Suggestion for merging xl save/restore/migrate/migrate-receive Zhigang Wang
2013-09-16 10:04 ` George Dunlap
2013-09-16 15:51   ` Zhigang Wang
2013-09-16 16:05     ` George Dunlap
2013-09-16 16:07       ` George Dunlap
2013-09-16 16:20     ` Ian Jackson
2013-09-16 16:40       ` George Dunlap
2013-09-16 17:06         ` Ian Jackson
2013-09-16 17:21         ` Zhigang Wang
2013-09-16 17:41       ` Zhigang Wang
2013-09-16 20:42         ` Ian Campbell
2013-09-16 20:51           ` Zhigang Wang
2013-09-17  8:25         ` George Dunlap
2013-09-17  9:26           ` Ian Jackson
2013-09-17 10:07             ` George Dunlap
2013-09-17 13:44             ` Zhigang Wang
2013-09-24 16:46           ` Konrad Rzeszutek Wilk
2013-09-25 10:06             ` George Dunlap
2013-10-03  2:19               ` Matt Wilson
2013-10-03 13:34                 ` Zhigang Wang
2013-09-17 10:28     ` George Dunlap
2013-09-17 10:45       ` Processed: " xen

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).