[PATCH 0/2] reset xenstore watches to fix kexec in Xen PVonHVM guests

[PATCH 0/2] reset xenstore watches to fix kexec in Xen PVonHVM guests

[Olaf Hering]

The following series improves kexec in a Xen PVonHVM guest.

It is available via git:

 git://github.com/olafhering/linux.git xen-kexec-XS_RESET_WATCHES-3.0

A new xenstored command XS_RESET_WATCHES has been added in
xen-unstable.hg changeset 23839:42a45baf037d. The command removes all
watches and transactions for the guest. The following patches make use
of the new command to wipe all existing watches during startup.

Olaf

Olaf Hering (2):
  xen/pv-on-hvm kexec: update xs_wire.h:xsd_sockmsg_type from
    xen-unstable
  xen/pv-on-hvm kexec: add xs_reset_watches to shutdown watches from
    old kernel

 drivers/xen/xenbus/xenbus_xs.c     |   13 +++++++++++++
 include/xen/interface/io/xs_wire.h |    6 +++++-
 2 files changed, 18 insertions(+), 1 deletions(-)

-- 
1.7.3.4

[PATCH 1/2] xen/pv-on-hvm kexec: update xs_wire.h:xsd_sockmsg_type from xen-unstable

[Olaf Hering]

Update include/xen/interface/io/xs_wire.h from xen-unstable.
Now entries in xsd_sockmsg_type were added.

Signed-off-by: Olaf Hering <olaf@aepfle.de>
---
 include/xen/interface/io/xs_wire.h |    5 ++++-
 1 files changed, 4 insertions(+), 1 deletions(-)

diff --git a/include/xen/interface/io/xs_wire.h b/include/xen/interface/io/xs_wire.h
index 99fcffb..f6f07aa 100644
--- a/include/xen/interface/io/xs_wire.h
+++ b/include/xen/interface/io/xs_wire.h
@@ -26,7 +26,10 @@ enum xsd_sockmsg_type
     XS_SET_PERMS,
     XS_WATCH_EVENT,
     XS_ERROR,
-    XS_IS_DOMAIN_INTRODUCED
+    XS_IS_DOMAIN_INTRODUCED,
+    XS_RESUME,
+    XS_SET_TARGET,
+    XS_RESTRICT
 };
 
 #define XS_WRITE_NONE "NONE"
-- 
1.7.3.4

[PATCH 2/2] xen/pv-on-hvm kexec: add xs_reset_watches to shutdown watches from old kernel

[Olaf Hering]

Add new xs_reset_watches function to shutdown watches from old kernel after
kexec boot.  The old kernel does not unregister all watches in the
shutdown path.  They are still active, the double registration can not
be detected by the new kernel.  When the watches fire, unexpected events
will arrive and the xenwatch thread will crash (jumps to NULL).  An
orderly reboot of a hvm guest will destroy the entire guest with all its
resources (including the watches) before it is rebuilt from scratch, so
the missing unregister is not an issue in that case.

With this change the xenstored is instructed to wipe all active watches
for the guest.  However, a patch for xenstored is required so that it
accepts the XS_RESET_WATCHES request from a client (see changeset
23839:42a45baf037d in xen-unstable.hg). Without the patch for xenstored
the registration of watches will fail and some features of a PVonHVM
guest are not available. The guest is still able to boot, but repeated
kexec boots will fail.

v5:
  - use xs_single instead of passing a dummy string to xs_talkv

v4:
  - ignore -EEXIST in xs_reset_watches

v3:
  - use XS_RESET_WATCHES instead of XS_INTRODUCE

v2:
  - move all code which deals with XS_INTRODUCE into xs_introduce()
    (based on feedback from Ian Campbell)
  - remove casts from kvec assignment

Signed-off-by: Olaf Hering <olaf@aepfle.de>
---
 drivers/xen/xenbus/xenbus_xs.c     |   13 +++++++++++++
 include/xen/interface/io/xs_wire.h |    3 ++-
 2 files changed, 15 insertions(+), 1 deletions(-)

diff --git a/drivers/xen/xenbus/xenbus_xs.c b/drivers/xen/xenbus/xenbus_xs.c
index 5534690..01bdb98 100644
--- a/drivers/xen/xenbus/xenbus_xs.c
+++ b/drivers/xen/xenbus/xenbus_xs.c
@@ -620,6 +620,15 @@ static struct xenbus_watch *find_watch(const char *token)
 	return NULL;
 }
 
+static void xs_reset_watches(void)
+{
+	int err;
+
+	err = xs_error(xs_single(XBT_NIL, XS_RESET_WATCHES, "", NULL));
+	if (err && err != -EEXIST)
+		printk(KERN_WARNING "xs_reset_watches failed: %d\n", err);
+}
+
 /* Register callback to watch this node. */
 int register_xenbus_watch(struct xenbus_watch *watch)
 {
@@ -897,5 +906,9 @@ int xs_init(void)
 	if (IS_ERR(task))
 		return PTR_ERR(task);
 
+	/* shutdown watches for kexec boot */
+	if (xen_hvm_domain())
+		xs_reset_watches();
+
 	return 0;
 }
diff --git a/include/xen/interface/io/xs_wire.h b/include/xen/interface/io/xs_wire.h
index f6f07aa..f0b6890 100644
--- a/include/xen/interface/io/xs_wire.h
+++ b/include/xen/interface/io/xs_wire.h
@@ -29,7 +29,8 @@ enum xsd_sockmsg_type
     XS_IS_DOMAIN_INTRODUCED,
     XS_RESUME,
     XS_SET_TARGET,
-    XS_RESTRICT
+    XS_RESTRICT,
+    XS_RESET_WATCHES
 };
 
 #define XS_WRITE_NONE "NONE"
-- 
1.7.3.4

Re: [PATCH 2/2] xen/pv-on-hvm kexec: add xs_reset_watches to shutdown watches from old kernel

[Ian Campbell]

On Thu, 2011-09-22 at 15:14 +0100, Olaf Hering wrote:
> Add new xs_reset_watches function to shutdown watches from old kernel after
> kexec boot.  The old kernel does not unregister all watches in the
> shutdown path.  They are still active, the double registration can not
> be detected by the new kernel.  When the watches fire, unexpected events
> will arrive and the xenwatch thread will crash (jumps to NULL).  An
> orderly reboot of a hvm guest will destroy the entire guest with all its
> resources (including the watches) before it is rebuilt from scratch, so
> the missing unregister is not an issue in that case.
> 
> With this change the xenstored is instructed to wipe all active watches
> for the guest.  However, a patch for xenstored is required so that it
> accepts the XS_RESET_WATCHES request from a client (see changeset
> 23839:42a45baf037d in xen-unstable.hg). Without the patch for xenstored
> the registration of watches will fail and some features of a PVonHVM
> guest are not available. The guest is still able to boot, but repeated
> kexec boots will fail.

This appears to break with oxenstored. It just hangs waiting for a
response from the daemon.

I suspect it is a bug in the daemon if it doesn't respond with an
appropriate error for an unknown command. I'll see if I can figure out
what is going wrong.

Ian.

Re: [PATCH 2/2] xen/pv-on-hvm kexec: add xs_reset_watches to shutdown watches from old kernel

[Olaf Hering]

On Fri, Dec 09, Ian Campbell wrote:

> I suspect it is a bug in the daemon if it doesn't respond with an
> appropriate error for an unknown command. I'll see if I can figure out
> what is going wrong.

Yes, please.

Olaf