From: Wen Congyang <wency@cn.fujitsu.com>
To: xen devel <xen-devel@lists.xen.org>
Cc: Ian Campbell <Ian.Campbell@citrix.com>,
Wen Congyang <wency@cn.fujitsu.com>,
Ian Jackson <Ian.Jackson@eu.citrix.com>,
Jiang Yunhong <yunhong.jiang@intel.com>,
Dong Eddie <eddie.dong@intel.com>,
Yang Hongyang <yanghy@cn.fujitsu.com>,
Lai Jiangshan <laijs@cn.fujitsu.com>
Subject: [RFC Patch v3 00/22] COarse-grain LOck-stepping Virtual Machines for Non-stop Service
Date: Fri, 5 Sep 2014 17:25:35 +0800 [thread overview]
Message-ID: <1409909158-19243-1-git-send-email-wency@cn.fujitsu.com> (raw)
Virtual machine (VM) replication is a well known technique for providing
application-agnostic software-implemented hardware fault tolerance -
"non-stop service". Currently, remus provides this function, but it buffers
all output packets, and the latency is unacceptable.
In xen summit 2012, We introduce a new VM replication solution: colo
(COarse-grain LOck-stepping virtual machine). The presentation is in
the following URL:
http://www.slideshare.net/xen_com_mgr/colo-coarsegrain-lockstepping-virtual-machines-for-nonstop-service
Here is the summary of the solution:
>From the client's point of view, as long as the client observes identical
responses from the primary and secondary VMs, according to the service
semantics, then the secondary vm is a valid replica of the primary
vm, and can successfully take over when a hardware failure of the
primary vm is detected.
This patchset is RFC, and implements the framework and disk replication of COLO:
1. Both primary vm and secondary vm are running
2. do checkoint
3. disk replication(use blktap2)
3. nic replication(use colo-agent)
This patchset is based on remus-v19, bugfix, and colo-prepare patchset, and use migration v1.
Only supports hvm guest now. The codes are also hosted on github:
https://github.com/wencongyang/xen/tree/colo-v3
TODO list:
1. Use migration v2 to implement COLO
2. support pvm
Known bugs:
1. qemu may segment fault
2. secondary vm is running, but don't response.
Usage:
1. update the vm's configfile:
disk:
disk = [ 'format=colo,devtype=disk,access=w,vdev=hda,backendtype=tap,target=192.168.3.1:9000|aio:/root/images/hvm/hvm_nopv/hvm.img' ]
nic:
vif = [ 'mac=00:16:4f:00:00:11, bridge=br0, model=e1000, forwarddev=eth0' ]
Note: the forwarddev of primary and secondary host should be connected directly,
and no other app uses it. If you don't have such nic, you can use vlan to make it.
2. build colo-agent:
You can get colo-agent from github:
https://github.com/wencongyang/colo-agent
3. run:
xl remus -c <domname> <secondary host IP>
Patch 1-4 : update remus to reuse remus device codes
Patch 5-12 : COLO framework related codes
Patch 13-16: move some block-remus's codes to block-replication.c. These codes will
be reused by COLO.
Patch 17 : implement block-colo
Patch 18 : update libxl to support blktap2
Patch 19 : implement disk replication
Patch 20-22: implement nic replication
Patch 23 : A patch for qemu-xen
Changlog from v2 to v3:
1. rebase to newest remus
2. add nic replication support
Changlog from v1 to v2:
1. rebase to newest remus
2. add disk replication support
Wen Congyang (22):
move remus related codes to libxl_remus.c
rename remus device to checkpoint device
adjust the indentation
don't touch remus in checkpoint_device
Update libxl_save_msgs_gen.pl to support return data from xl to xc
Allow slave sends data to master
secondary vm suspend/resume/checkpoint code
primary vm suspend/get_dirty_pfn/resume/checkpoint code
xc_domain_save: flush cache before calling callbacks->postcopy() in
colo mode
COLO: xc related codes
send store mfn and console mfn to xl before resuming secondary vm
implement the cmdline for COLO
blktap2: connect to backup asynchronously
switch to unprotected mode before closing
blktap2: move async connect related codes to block-replication.c
blktap2: move ramdisk related codes to block-replication.c
block-colo: implement colo disk replication
support blktap COLO in xl:
libxl/colo: setup and control disk replication for blktap2 backends
setup and control colo-agent for primary vm
setup and control colo-agent for secondary vm
colo: cmdline switches and config vars to control colo-agent
docs/man/xl.conf.pod.5 | 6 +
docs/man/xl.pod.1 | 11 +-
tools/blktap2/drivers/Makefile | 5 +-
tools/blktap2/drivers/block-colo.c | 1151 +++++++++++++++++++
tools/blktap2/drivers/block-remus.c | 1174 +++++---------------
tools/blktap2/drivers/block-replication.c | 1116 +++++++++++++++++++
tools/blktap2/drivers/block-replication.h | 217 ++++
tools/blktap2/drivers/tapdisk-control.c | 6 +
tools/blktap2/drivers/tapdisk-disktype.c | 9 +
tools/blktap2/drivers/tapdisk-disktype.h | 3 +-
tools/blktap2/drivers/tapdisk-interface.c | 18 +
tools/blktap2/drivers/tapdisk-interface.h | 1 +
tools/blktap2/drivers/tapdisk-vbd.c | 9 +
tools/blktap2/drivers/tapdisk-vbd.h | 1 +
tools/blktap2/drivers/tapdisk.h | 1 +
tools/hotplug/Linux/Makefile | 2 +
tools/hotplug/Linux/colo-agent-setup | 198 ++++
tools/hotplug/Linux/remus-netbuf-setup | 45 +-
tools/hotplug/Linux/xen-network-ft.sh | 102 ++
tools/libxc/xc_domain_restore.c | 44 +-
tools/libxc/xc_domain_save.c | 66 +-
tools/libxc/xenguest.h | 40 +
tools/libxl/Makefile | 11 +-
tools/libxl/colo-tc.c | 589 ++++++++++
tools/libxl/libxl.c | 78 +-
tools/libxl/libxl.h | 3 +-
tools/libxl/libxl_blktap2.c | 6 +-
...xl_remus_device.c => libxl_checkpoint_device.c} | 229 ++--
tools/libxl/libxl_colo.h | 48 +
tools/libxl/libxl_colo_nic.c | 309 ++++++
tools/libxl/libxl_colo_restore.c | 990 +++++++++++++++++
tools/libxl/libxl_colo_save.c | 809 ++++++++++++++
tools/libxl/libxl_colo_save_disk_blktap2.c | 214 ++++
tools/libxl/libxl_create.c | 151 ++-
tools/libxl/libxl_device.c | 4 +-
tools/libxl/libxl_dm.c | 3 +-
tools/libxl/libxl_dom.c | 234 +---
tools/libxl/libxl_internal.h | 218 ++--
tools/libxl/libxl_netbuffer.c | 127 ++-
tools/libxl/libxl_noblktap2.c | 29 +
tools/libxl/libxl_nonetbuffer.c | 14 +-
tools/libxl/libxl_remus.c | 377 +++++++
tools/libxl/libxl_remus.h | 27 +
tools/libxl/libxl_remus_disk_drbd.c | 67 +-
tools/libxl/libxl_save_callout.c | 37 +-
tools/libxl/libxl_save_helper.c | 17 +
tools/libxl/libxl_save_msgs_gen.pl | 74 +-
tools/libxl/libxl_types.idl | 15 +-
tools/libxl/libxl_types_internal.idl | 4 +-
tools/libxl/libxl_utils.c | 2 +
tools/libxl/libxlu_disk_l.l | 1 +
tools/libxl/xl.c | 3 +
tools/libxl/xl.h | 1 +
tools/libxl/xl_cmdimpl.c | 95 +-
tools/libxl/xl_cmdtable.c | 3 +-
55 files changed, 7433 insertions(+), 1581 deletions(-)
create mode 100644 tools/blktap2/drivers/block-colo.c
create mode 100644 tools/blktap2/drivers/block-replication.c
create mode 100644 tools/blktap2/drivers/block-replication.h
create mode 100755 tools/hotplug/Linux/colo-agent-setup
create mode 100644 tools/hotplug/Linux/xen-network-ft.sh
create mode 100644 tools/libxl/colo-tc.c
rename tools/libxl/{libxl_remus_device.c => libxl_checkpoint_device.c} (40%)
create mode 100644 tools/libxl/libxl_colo.h
create mode 100644 tools/libxl/libxl_colo_nic.c
create mode 100644 tools/libxl/libxl_colo_restore.c
create mode 100644 tools/libxl/libxl_colo_save.c
create mode 100644 tools/libxl/libxl_colo_save_disk_blktap2.c
create mode 100644 tools/libxl/libxl_remus.c
create mode 100644 tools/libxl/libxl_remus.h
--
1.9.3
next reply other threads:[~2014-09-05 9:25 UTC|newest]
Thread overview: 33+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-09-05 9:25 Wen Congyang [this message]
2014-09-05 9:25 ` [RFC Patch v3 01/22] move remus related codes to libxl_remus.c Wen Congyang
2014-09-05 9:25 ` [RFC Patch v3 02/22] rename remus device to checkpoint device Wen Congyang
2014-09-05 9:25 ` [RFC Patch v3 03/22] adjust the indentation Wen Congyang
2014-09-05 9:25 ` [RFC Patch v3 04/22] don't touch remus in checkpoint_device Wen Congyang
2014-09-05 9:25 ` [RFC Patch v3 05/22] Update libxl_save_msgs_gen.pl to support return data from xl to xc Wen Congyang
2014-09-05 9:25 ` [RFC Patch v3 06/22] Allow slave sends data to master Wen Congyang
2014-09-05 9:25 ` [RFC Patch v3 07/22] secondary vm suspend/resume/checkpoint code Wen Congyang
2014-09-05 9:25 ` [RFC Patch v3 08/22] primary vm suspend/get_dirty_pfn/resume/checkpoint code Wen Congyang
2014-09-05 9:25 ` [RFC Patch v3 09/22] xc_domain_save: flush cache before calling callbacks->postcopy() in colo mode Wen Congyang
2014-09-05 9:25 ` [RFC Patch v3 10/22] COLO: xc related codes Wen Congyang
2014-09-05 9:25 ` [RFC Patch v3 11/22] send store mfn and console mfn to xl before resuming secondary vm Wen Congyang
2014-09-05 9:25 ` [RFC Patch v3 12/22] implement the cmdline for COLO Wen Congyang
2014-09-05 9:25 ` [RFC Patch v3 13/22] blktap2: connect to backup asynchronously Wen Congyang
2014-09-24 19:11 ` Shriram Rajagopalan
2014-09-25 5:40 ` Wen Congyang
2014-09-05 9:25 ` [RFC Patch v3 14/22] switch to unprotected mode before closing Wen Congyang
2014-09-05 9:25 ` [RFC Patch v3 15/22] blktap2: move async connect related codes to block-replication.c Wen Congyang
2014-09-24 18:48 ` Shriram Rajagopalan
2014-09-05 9:25 ` [RFC Patch v3 16/22] blktap2: move ramdisk " Wen Congyang
2014-09-24 18:44 ` Shriram Rajagopalan
2014-09-26 5:18 ` Wen Congyang
2014-09-05 9:25 ` [RFC Patch v3 17/22] block-colo: implement colo disk replication Wen Congyang
2014-09-05 9:25 ` [RFC Patch v3 18/22] support blktap COLO in xl: Wen Congyang
2014-09-05 9:25 ` [RFC Patch v3 19/22] libxl/colo: setup and control disk replication for blktap2 backends Wen Congyang
2014-09-05 9:25 ` [RFC Patch v3 20/22] setup and control colo-agent for primary vm Wen Congyang
2014-09-05 9:25 ` [RFC Patch v3 21/22] setup and control colo-agent for secondary vm Wen Congyang
2014-09-05 9:25 ` [RFC Patch v3 22/22] colo: cmdline switches and config vars to control colo-agent Wen Congyang
2014-09-05 9:25 ` [RFC Patch v3 23/22] Introduce "xen-load-devices-state" Wen Congyang
2014-09-05 21:57 ` Stefano Stabellini
[not found] ` <alpine.DEB.2.02.1409052229550.2334@kaball.uk.xensource.com>
2014-09-09 2:47 ` Wen Congyang
[not found] ` <540E6A44.8090507@cn.fujitsu.com>
2014-09-10 19:15 ` Stefano Stabellini
[not found] ` <alpine.DEB.2.02.1409102005450.8137@kaball.uk.xensource.com>
2014-09-11 5:03 ` Wen Congyang
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1409909158-19243-1-git-send-email-wency@cn.fujitsu.com \
--to=wency@cn.fujitsu.com \
--cc=Ian.Campbell@citrix.com \
--cc=Ian.Jackson@eu.citrix.com \
--cc=eddie.dong@intel.com \
--cc=laijs@cn.fujitsu.com \
--cc=xen-devel@lists.xen.org \
--cc=yanghy@cn.fujitsu.com \
--cc=yunhong.jiang@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).