* [PATCH 1 of 3] Let xend choose to do hard or soft domain resumption depending on
2007-02-27 2:36 [PATCH 0 of 3] Enable domain checkpointing via xm save --checkpoint Brendan Cully
@ 2007-02-27 2:36 ` Brendan Cully
2007-02-27 2:36 ` [PATCH 2 of 3] Add xm save -c/--checkpoint option Brendan Cully
` (2 subsequent siblings)
3 siblings, 0 replies; 6+ messages in thread
From: Brendan Cully @ 2007-02-27 2:36 UTC (permalink / raw)
To: xen-devel
# HG changeset patch
# User Brendan Cully <brendan@cs.ubc.ca>
# Date 1172543764 28800
# Node ID b2edb7f3393255f7b51af7fe688de687233b525e
# Parent e7b2a282c9e70fbadd38ccc7cc7d1fcb6a32b886
Let xend choose to do hard or soft domain resumption depending on
whether the domain advertises support for soft resumption in its
elfnotes.
Signed-off-by: Brendan Cully <brendan@kublai.com>
diff -r e7b2a282c9e7 -r b2edb7f33932 tools/libxc/xc_resume.c
--- a/tools/libxc/xc_resume.c Mon Feb 26 17:20:36 2007 +0000
+++ b/tools/libxc/xc_resume.c Mon Feb 26 18:36:04 2007 -0800
@@ -169,13 +169,9 @@ static int xc_domain_resume_any(int xc_h
* (2) should be used only for guests which cannot handle the special
* new return code. (1) is always safe (but slower).
*/
-int xc_domain_resume(int xc_handle, uint32_t domid)
+int xc_domain_resume(int xc_handle, uint32_t domid, int fast)
{
- /*
- * XXX: Implement a way to select between options (1) and (2).
- * Or expose the options as two different methods to Python.
- */
- return (0
+ return (fast
? xc_domain_resume_cooperative(xc_handle, domid)
: xc_domain_resume_any(xc_handle, domid));
}
diff -r e7b2a282c9e7 -r b2edb7f33932 tools/libxc/xenctrl.h
--- a/tools/libxc/xenctrl.h Mon Feb 26 17:20:36 2007 +0000
+++ b/tools/libxc/xenctrl.h Mon Feb 26 18:36:04 2007 -0800
@@ -243,10 +243,12 @@ int xc_domain_destroy(int xc_handle,
*
* @parm xc_handle a handle to an open hypervisor interface
* @parm domid the domain id to resume
+ * @parm fast use cooperative resume (guest must support this)
* return 0 on success, -1 on failure
*/
int xc_domain_resume(int xc_handle,
- uint32_t domid);
+ uint32_t domid,
+ int fast);
/**
* This function will shutdown a domain. This is intended for use in
diff -r e7b2a282c9e7 -r b2edb7f33932 tools/python/xen/lowlevel/xc/xc.c
--- a/tools/python/xen/lowlevel/xc/xc.c Mon Feb 26 17:20:36 2007 +0000
+++ b/tools/python/xen/lowlevel/xc/xc.c Mon Feb 26 18:36:04 2007 -0800
@@ -178,7 +178,17 @@ static PyObject *pyxc_domain_shutdown(Xc
static PyObject *pyxc_domain_resume(XcObject *self, PyObject *args)
{
- return dom_op(self, args, xc_domain_resume);
+ uint32_t dom;
+ int fast;
+
+ if (!PyArg_ParseTuple(args, "ii", &dom, &fast))
+ return NULL;
+
+ if (xc_domain_resume(self->xc_handle, dom, fast) != 0)
+ return pyxc_error_to_exception();
+
+ Py_INCREF(zero);
+ return zero;
}
static PyObject *pyxc_vcpu_setaffinity(XcObject *self,
@@ -1124,7 +1134,8 @@ static PyMethodDef pyxc_methods[] = {
(PyCFunction)pyxc_domain_resume,
METH_VARARGS, "\n"
"Resume execution of a suspended domain.\n"
- " dom [int]: Identifier of domain to be resumed.\n\n"
+ " dom [int]: Identifier of domain to be resumed.\n"
+ " fast [int]: Use cooperative resume.\n\n"
"Returns: [int] 0 on success; -1 on error.\n" },
{ "domain_shutdown",
diff -r e7b2a282c9e7 -r b2edb7f33932 tools/python/xen/xend/XendCheckpoint.py
--- a/tools/python/xen/xend/XendCheckpoint.py Mon Feb 26 17:20:36 2007 +0000
+++ b/tools/python/xen/xend/XendCheckpoint.py Mon Feb 26 18:36:04 2007 -0800
@@ -137,23 +137,6 @@ def save(fd, dominfo, network, live, dst
log.exception("Save failed on domain %s (%s).", domain_name,
dominfo.getDomid())
- dominfo._releaseDevices()
- dominfo.testDeviceComplete()
- dominfo.testvifsComplete()
- log.debug("XendCheckpoint.save: devices released")
-
- dominfo._resetChannels()
-
- dominfo._removeDom('control/shutdown')
- dominfo._removeDom('device-misc/vif/nextDeviceID')
-
- dominfo._createChannels()
- dominfo._introduceDomain()
- dominfo._storeDomDetails()
-
- dominfo._createDevices()
- log.debug("XendCheckpoint.save: devices created")
-
dominfo.resumeDomain()
log.debug("XendCheckpoint.save: resumeDomain")
diff -r e7b2a282c9e7 -r b2edb7f33932 tools/python/xen/xend/XendDomainInfo.py
--- a/tools/python/xen/xend/XendDomainInfo.py Mon Feb 26 17:20:36 2007 +0000
+++ b/tools/python/xen/xend/XendDomainInfo.py Mon Feb 26 18:36:04 2007 -0800
@@ -1662,10 +1662,31 @@ class XendDomainInfo:
def resumeDomain(self):
log.debug("XendDomainInfo.resumeDomain(%s)", str(self.domid))
+ if self.domid is None:
+ return
try:
- if self.domid is not None:
- xc.domain_resume(self.domid)
- ResumeDomain(self.domid)
+ # could also fetch a parsed key from xenstore
+ fast = 'suspend_cancel' in self.info.get_notes().get('FEATURES', '').split('|')
+ if not fast:
+ self._releaseDevices()
+ self.testDeviceComplete()
+ self.testvifsComplete()
+ log.debug("XendDomainInfo.resumeDomain: devices released")
+
+ self._resetChannels()
+
+ self._removeDom('control/shutdown')
+ self._removeDom('device-misc/vif/nextDeviceID')
+
+ self._createChannels()
+ self._introduceDomain()
+ self._storeDomDetails()
+
+ self._createDevices()
+ log.debug("XendDomainInfo.resumeDomain: devices created")
+
+ xc.domain_resume(self.domid, fast)
+ ResumeDomain(self.domid)
except:
log.exception("XendDomainInfo.resume: xc.domain_resume failed on domain %s." % (str(self.domid)))
^ permalink raw reply [flat|nested] 6+ messages in thread* [PATCH 2 of 3] Add xm save -c/--checkpoint option
2007-02-27 2:36 [PATCH 0 of 3] Enable domain checkpointing via xm save --checkpoint Brendan Cully
2007-02-27 2:36 ` [PATCH 1 of 3] Let xend choose to do hard or soft domain resumption depending on Brendan Cully
@ 2007-02-27 2:36 ` Brendan Cully
2007-02-27 2:36 ` [PATCH 3 of 3] Add suspend_cancel flag to linux elf notes Brendan Cully
2007-02-27 16:10 ` [PATCH 0 of 3] Enable domain checkpointing via xm save --checkpoint Keir Fraser
3 siblings, 0 replies; 6+ messages in thread
From: Brendan Cully @ 2007-02-27 2:36 UTC (permalink / raw)
To: xen-devel
# HG changeset patch
# User Brendan Cully <brendan@cs.ubc.ca>
# Date 1172543764 28800
# Node ID b207ed258dcf3a6aee865ca82bc170f68e589342
# Parent b2edb7f3393255f7b51af7fe688de687233b525e
Add xm save -c/--checkpoint option
xm save --checkpoint leaves the domain running after creating the
snapshot.
Signed-off-by: Brendan Cully <brendan@cs.ubc.ca>
diff -r b2edb7f33932 -r b207ed258dcf tools/python/xen/xend/XendCheckpoint.py
--- a/tools/python/xen/xend/XendCheckpoint.py Mon Feb 26 18:36:04 2007 -0800
+++ b/tools/python/xen/xend/XendCheckpoint.py Mon Feb 26 18:36:04 2007 -0800
@@ -54,7 +54,7 @@ def read_exact(fd, size, errmsg):
return buf
-def save(fd, dominfo, network, live, dst):
+def save(fd, dominfo, network, live, dst, checkpoint=False):
write_exact(fd, SIGNATURE, "could not write guest state file: signature")
config = sxp.to_string(dominfo.sxpr())
@@ -121,9 +121,11 @@ def save(fd, dominfo, network, live, dst
os.close(qemu_fd)
os.remove("/tmp/xen.qemu-dm.%d" % dominfo.getDomid())
- dominfo.destroyDomain()
- dominfo.testDeviceComplete()
-
+ if checkpoint:
+ dominfo.resumeDomain()
+ else:
+ dominfo.destroyDomain()
+ dominfo.testDeviceComplete()
try:
dominfo.setName(domain_name)
except VmError:
diff -r b2edb7f33932 -r b207ed258dcf tools/python/xen/xend/XendDomain.py
--- a/tools/python/xen/xend/XendDomain.py Mon Feb 26 18:36:04 2007 -0800
+++ b/tools/python/xen/xend/XendDomain.py Mon Feb 26 18:36:04 2007 -0800
@@ -1172,7 +1172,7 @@ class XendDomain:
XendCheckpoint.save(sock.fileno(), dominfo, True, live, dst)
sock.close()
- def domain_save(self, domid, dst):
+ def domain_save(self, domid, dst, checkpoint):
"""Start saving a domain to file.
@param domid: Domain ID or Name
@@ -1196,8 +1196,8 @@ class XendDomain:
oflags |= os.O_LARGEFILE
fd = os.open(dst, oflags)
try:
- # For now we don't support 'live checkpoint'
- XendCheckpoint.save(fd, dominfo, False, False, dst)
+ XendCheckpoint.save(fd, dominfo, False, False, dst,
+ checkpoint=checkpoint)
finally:
os.close(fd)
except OSError, ex:
diff -r b2edb7f33932 -r b207ed258dcf tools/python/xen/xm/main.py
--- a/tools/python/xen/xm/main.py Mon Feb 26 18:36:04 2007 -0800
+++ b/tools/python/xen/xm/main.py Mon Feb 26 18:36:04 2007 -0800
@@ -102,7 +102,7 @@ SUBCOMMAND_HELP = {
'reboot' : ('<Domain> [-wa]', 'Reboot a domain.'),
'restore' : ('<CheckpointFile> [-p]',
'Restore a domain from a saved state.'),
- 'save' : ('<Domain> <CheckpointFile>',
+ 'save' : ('[-c] <Domain> <CheckpointFile>',
'Save a domain state to restore later.'),
'shutdown' : ('<Domain> [-waRH]', 'Shutdown a domain.'),
'top' : ('', 'Monitor a host and the domains in real time.'),
@@ -232,6 +232,9 @@ SUBCOMMAND_OPTIONS = {
'resume': (
('-p', '--paused', 'Do not unpause domain after resuming it'),
),
+ 'save': (
+ ('-c', '--checkpoint', 'Leave domain running after creating snapshot'),
+ ),
'restore': (
('-p', '--paused', 'Do not unpause domain after restoring it'),
),
@@ -604,21 +607,37 @@ def xm_shell(args):
#########################################################################
def xm_save(args):
- arg_check(args, "save", 2)
-
- try:
- dominfo = parse_doms_info(server.xend.domain(args[0]))
+ arg_check(args, "save", 2, 3)
+
+ try:
+ (options, params) = getopt.gnu_getopt(args, 'c', ['checkpoint'])
+ except getopt.GetoptError, opterr:
+ err(opterr)
+ sys.exit(1)
+
+ checkpoint = False
+ for (k, v) in options:
+ if k in ['-c', '--checkpoint']:
+ checkpoint = True
+
+ if len(params) != 2:
+ err("Wrong number of parameters")
+ usage('save')
+ sys.exit(1)
+
+ try:
+ dominfo = parse_doms_info(server.xend.domain(params[0]))
except xmlrpclib.Fault, ex:
raise ex
domid = dominfo['domid']
- savefile = os.path.abspath(args[1])
+ savefile = os.path.abspath(params[1])
if not os.access(os.path.dirname(savefile), os.W_OK):
err("xm save: Unable to create file %s" % savefile)
sys.exit(1)
- server.xend.domain.save(domid, savefile)
+ server.xend.domain.save(domid, savefile, checkpoint)
def xm_restore(args):
arg_check(args, "restore", 1, 2)
^ permalink raw reply [flat|nested] 6+ messages in thread* [PATCH 3 of 3] Add suspend_cancel flag to linux elf notes
2007-02-27 2:36 [PATCH 0 of 3] Enable domain checkpointing via xm save --checkpoint Brendan Cully
2007-02-27 2:36 ` [PATCH 1 of 3] Let xend choose to do hard or soft domain resumption depending on Brendan Cully
2007-02-27 2:36 ` [PATCH 2 of 3] Add xm save -c/--checkpoint option Brendan Cully
@ 2007-02-27 2:36 ` Brendan Cully
2007-02-27 16:10 ` [PATCH 0 of 3] Enable domain checkpointing via xm save --checkpoint Keir Fraser
3 siblings, 0 replies; 6+ messages in thread
From: Brendan Cully @ 2007-02-27 2:36 UTC (permalink / raw)
To: xen-devel
# HG changeset patch
# User Brendan Cully <brendan@cs.ubc.ca>
# Date 1172543764 28800
# Node ID 728b370eaca01f5d7dd3e1ec59b34a0860372645
# Parent b207ed258dcf3a6aee865ca82bc170f68e589342
Add suspend_cancel flag to linux elf notes.
Signed-off-by: Brendan Cully <brendan@cs.ubc.ca>
diff -r b207ed258dcf -r 728b370eaca0 linux-2.6-xen-sparse/arch/i386/kernel/head-xen.S
--- a/linux-2.6-xen-sparse/arch/i386/kernel/head-xen.S Mon Feb 26 18:36:04 2007 -0800
+++ b/linux-2.6-xen-sparse/arch/i386/kernel/head-xen.S Mon Feb 26 18:36:04 2007 -0800
@@ -175,6 +175,7 @@ ENTRY(cpu_gdt_table)
.ascii "|auto_translated_physmap"
.ascii "|pae_pgdir_above_4gb"
.ascii "|supervisor_mode_kernel"
+ .ascii "|suspend_cancel"
#ifdef CONFIG_X86_PAE
.ascii ",PAE=yes[extended-cr3]"
#else
@@ -197,7 +198,7 @@ ENTRY(cpu_gdt_table)
ELFNOTE(Xen, XEN_ELFNOTE_ENTRY, .long, startup_32)
ELFNOTE(Xen, XEN_ELFNOTE_HYPERCALL_PAGE, .long, hypercall_page)
ELFNOTE(Xen, XEN_ELFNOTE_HV_START_LOW, .long, HYPERVISOR_VIRT_START)
- ELFNOTE(Xen, XEN_ELFNOTE_FEATURES, .asciz, "writable_page_tables|writable_descriptor_tables|auto_translated_physmap|pae_pgdir_above_4gb|supervisor_mode_kernel")
+ ELFNOTE(Xen, XEN_ELFNOTE_FEATURES, .asciz, "writable_page_tables|writable_descriptor_tables|auto_translated_physmap|pae_pgdir_above_4gb|supervisor_mode_kernel|suspend_cancel")
#ifdef CONFIG_X86_PAE
ELFNOTE(Xen, XEN_ELFNOTE_PAE_MODE, .asciz, "yes")
ELFNOTE(Xen, XEN_ELFNOTE_L1_MFN_VALID, .quad, _PAGE_PRESENT,_PAGE_PRESENT)
diff -r b207ed258dcf -r 728b370eaca0 xen/common/libelf/libelf-dominfo.c
--- a/xen/common/libelf/libelf-dominfo.c Mon Feb 26 18:36:04 2007 -0800
+++ b/xen/common/libelf/libelf-dominfo.c Mon Feb 26 18:36:04 2007 -0800
@@ -12,7 +12,8 @@ const char *elf_xen_feature_names[] = {
[XENFEAT_writable_descriptor_tables] = "writable_descriptor_tables",
[XENFEAT_auto_translated_physmap] = "auto_translated_physmap",
[XENFEAT_supervisor_mode_kernel] = "supervisor_mode_kernel",
- [XENFEAT_pae_pgdir_above_4gb] = "pae_pgdir_above_4gb"
+ [XENFEAT_pae_pgdir_above_4gb] = "pae_pgdir_above_4gb",
+ [XENFEAT_suspend_cancel] = "suspend_cancel"
};
const int elf_xen_features =
sizeof(elf_xen_feature_names) / sizeof(elf_xen_feature_names[0]);
diff -r b207ed258dcf -r 728b370eaca0 xen/include/public/features.h
--- a/xen/include/public/features.h Mon Feb 26 18:36:04 2007 -0800
+++ b/xen/include/public/features.h Mon Feb 26 18:36:04 2007 -0800
@@ -56,6 +56,12 @@
*/
#define XENFEAT_pae_pgdir_above_4gb 4
+/*
+ * If set, the guest is checkpointable. If the suspend hypercall returns
+ * 1, the domain will undo its suspend prep work and continue.
+ */
+#define XENFEAT_suspend_cancel 5
+
#define XENFEAT_NR_SUBMAPS 1
#endif /* __XEN_PUBLIC_FEATURES_H__ */
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [PATCH 0 of 3] Enable domain checkpointing via xm save --checkpoint
2007-02-27 2:36 [PATCH 0 of 3] Enable domain checkpointing via xm save --checkpoint Brendan Cully
` (2 preceding siblings ...)
2007-02-27 2:36 ` [PATCH 3 of 3] Add suspend_cancel flag to linux elf notes Brendan Cully
@ 2007-02-27 16:10 ` Keir Fraser
2007-02-28 6:42 ` Brendan Cully
3 siblings, 1 reply; 6+ messages in thread
From: Keir Fraser @ 2007-02-27 16:10 UTC (permalink / raw)
To: Brendan Cully, xen-devel
Basically okay, but please create a new elfnote rather than adding a
feature. The feature stuff has kind of got a bit confused, but is meant to
indicate features provided by Xen to a particular guest. The feature maps in
the guest header indicate which Xen features it requires and/or supports.
This new flag has nothing to do with the hypervisor itself so I think it
belongs in its own elfnote (XEN_ELFNOTE_SUSPEND_CANCEL?). This can either be
a boolean (so no elfnote description bytes) or perhaps we should map it to a
long with value 1 for now (so we can add further save/restore revisions
later if we like).
Oh, also remember to add the elfnote to the x86/64 head-xen.S.
-- Keir
On 27/2/07 02:36, "Brendan Cully" <brendan@cs.ubc.ca> wrote:
> The attached patches create a --checkpoint flag for xm save:
>
> xc_domain_resume lets the caller alert the guest that it is being
> resumed. Callers should not do this unless they know the guest
> supports the operation.
>
> When xend wants to resume a domain, it checks for the suspend_cancel
> feature in the elfnotes it has recorded from domain build or
> resume. If the flag is present, it simply lets the guest handle
> resuming, otherwise it tears down and rebuilds the domain and lets the
> guest act as if it is resuming in a new domain.
>
> A new suspend_cancel feature is added to the kernel 'features'
> elfnote.
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [PATCH 0 of 3] Enable domain checkpointing via xm save --checkpoint
2007-02-27 16:10 ` [PATCH 0 of 3] Enable domain checkpointing via xm save --checkpoint Keir Fraser
@ 2007-02-28 6:42 ` Brendan Cully
0 siblings, 0 replies; 6+ messages in thread
From: Brendan Cully @ 2007-02-28 6:42 UTC (permalink / raw)
To: Keir Fraser; +Cc: xen-devel
On Tuesday, 27 February 2007 at 16:10, Keir Fraser wrote:
> Basically okay, but please create a new elfnote rather than adding a
> feature. The feature stuff has kind of got a bit confused, but is meant to
> indicate features provided by Xen to a particular guest. The feature maps in
> the guest header indicate which Xen features it requires and/or supports.
> This new flag has nothing to do with the hypervisor itself so I think it
> belongs in its own elfnote (XEN_ELFNOTE_SUSPEND_CANCEL?). This can either be
> a boolean (so no elfnote description bytes) or perhaps we should map it to a
> long with value 1 for now (so we can add further save/restore revisions
> later if we like).
Ok, I've resubmitted with this approach, making the note a long.
> Oh, also remember to add the elfnote to the x86/64 head-xen.S.
Done, thanks for the heads up.
^ permalink raw reply [flat|nested] 6+ messages in thread