[PATCH v2 00/16] Save/Restore Support for mini-OS PVH

[PATCH v2 00/16] Save/Restore Support for mini-OS PVH

[Bruno Alvisio]

Hi all,

I am sending the second revision for supporting save/restore in Mini-OS PVH. The 
branch can be found at: 

https://github.com/balvisio/mini-os/tree/feature/mini-os-suspend-support-submission-2

Feedback would be greatly appreciated.

Cheers,

Bruno

Signed-off-by: Bruno Alvisio <bruno.alvisio@gmail.com>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 01/17] Save/Restore Support: Refactor HYPERVISOR_suspend hypercall

[Bruno Alvisio]

Directly using the SHUTDOWN_suspend macro as a parameter for the schedop
hypercall causes an error in the Xen hypercall handler. Also for consistency,
the SHUTDOWN_suspend param is wrapped in the sched_shutdown struct.

Signed-off-by: Bruno Alvisio <bruno.alvisio@gmail.com>
Reviewed-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
---
 include/x86/x86_32/hypercall-x86_32.h | 4 ++--
 include/x86/x86_64/hypercall-x86_64.h | 4 ++--
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/include/x86/x86_32/hypercall-x86_32.h b/include/x86/x86_32/hypercall-x86_32.h
index 5c93464..70505a4 100644
--- a/include/x86/x86_32/hypercall-x86_32.h
+++ b/include/x86/x86_32/hypercall-x86_32.h
@@ -298,8 +298,8 @@ static inline int
 HYPERVISOR_suspend(
 	unsigned long srec)
 {
-	return _hypercall3(int, sched_op, SCHEDOP_shutdown,
-			   SHUTDOWN_suspend, srec);
+	struct sched_shutdown shutdown = { .reason = SHUTDOWN_suspend };
+	return _hypercall3(int, sched_op, SCHEDOP_shutdown, &shutdown, srec);
 }
 
 static inline int
diff --git a/include/x86/x86_64/hypercall-x86_64.h b/include/x86/x86_64/hypercall-x86_64.h
index 6171812..95f8ade 100644
--- a/include/x86/x86_64/hypercall-x86_64.h
+++ b/include/x86/x86_64/hypercall-x86_64.h
@@ -305,8 +305,8 @@ static inline int
 HYPERVISOR_suspend(
 	unsigned long srec)
 {
-	return _hypercall3(int, sched_op, SCHEDOP_shutdown,
-			   SHUTDOWN_suspend, srec);
+	struct sched_shutdown shutdown = { .reason = SHUTDOWN_suspend };
+	return _hypercall3(int, sched_op, SCHEDOP_shutdown, &shutdown, srec);
 }
 
 static inline int
-- 
2.3.2 (Apple Git-55)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 02/17] Save/Restore Support: Refactor trap_init() and setup vector callbacks

[Bruno Alvisio]

Currently the setup of the IDT and the request to set the HVM vector callbacks
are performed both in the trap_init function.

As part of the post-suspend operation, the HVM vector callback needs to be setup
again while the IDT does not. Thus, the trap_init function is split into two
separate functions: trap_init (sets up IDT) and xen_callback_vector (sets the
HVM vector callback). During the post-suspend operations the xen_callback_vector
function will be invoked.

Signed-off-by: Bruno Alvisio <bruno.alvisio@gmail.com>
Reviewed-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
---
 arch/x86/traps.c | 17 +++++++++++------
 include/x86/os.h |  3 +++
 2 files changed, 14 insertions(+), 6 deletions(-)

diff --git a/arch/x86/traps.c b/arch/x86/traps.c
index aa17da3..a7388a5 100644
--- a/arch/x86/traps.c
+++ b/arch/x86/traps.c
@@ -389,6 +389,16 @@ static void setup_gate(unsigned int entry, void *addr, unsigned int dpl)
 #endif
 }
 
+void xen_callback_vector(void)
+{
+    if (hvm_set_parameter(HVM_PARAM_CALLBACK_IRQ,
+                         (2ULL << 56) | TRAP_xen_callback))
+    {
+        xprintk("Request for Xen HVM callback vector failed\n");
+        do_exit();
+    }
+}
+
 void trap_init(void)
 {
     setup_gate(TRAP_divide_error, &divide_error, 0);
@@ -415,12 +425,7 @@ void trap_init(void)
     gdt[GDTE_TSS] = (typeof(*gdt))INIT_GDTE((unsigned long)&tss, 0x67, 0x89);
     asm volatile ("ltr %w0" :: "rm" (GDTE_TSS * 8));
 
-    if ( hvm_set_parameter(HVM_PARAM_CALLBACK_IRQ,
-                           (2ULL << 56) | TRAP_xen_callback) )
-    {
-        xprintk("Request for Xen HVM callback vector failed\n");
-        do_exit();
-    }
+    xen_callback_vector();
 }
 
 void trap_fini(void)
diff --git a/include/x86/os.h b/include/x86/os.h
index fbc2eeb..d155914 100644
--- a/include/x86/os.h
+++ b/include/x86/os.h
@@ -67,6 +67,9 @@ extern shared_info_t *HYPERVISOR_shared_info;
 
 void trap_init(void);
 void trap_fini(void);
+#ifndef CONFIG_PARAVIRT
+void xen_callback_vector(void);
+#endif
 
 void arch_fini(void);
 
-- 
2.3.2 (Apple Git-55)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 03/17] Save/Restore Support: Declare kernel and arch pre/post suspend functions

[Bruno Alvisio]

For mini-OS to support suspend and restore, the kernel will have to suspend
different modules such as xenbus, console, irq, etc. During save/restore the
kernel and arch pre_suspend and post_suspend functions will be invoked to
suspend/resume each of the modules.

Signed-off-by: Bruno Alvisio <bruno.alvisio@gmail.com>
Reviewed-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
---
 arch/x86/setup.c | 10 ++++++++++
 include/kernel.h |  2 ++
 include/x86/os.h |  4 ++--
 kernel.c         | 10 ++++++++++
 4 files changed, 24 insertions(+), 2 deletions(-)

diff --git a/arch/x86/setup.c b/arch/x86/setup.c
index 5278227..3dd86f9 100644
--- a/arch/x86/setup.c
+++ b/arch/x86/setup.c
@@ -204,6 +204,16 @@ arch_init(void *par)
 	start_kernel();
 }
 
+void arch_pre_suspend(void)
+{
+
+}
+
+void arch_post_suspend(int canceled)
+{
+
+}
+
 void
 arch_fini(void)
 {
diff --git a/include/kernel.h b/include/kernel.h
index d37ddda..161d757 100644
--- a/include/kernel.h
+++ b/include/kernel.h
@@ -5,6 +5,8 @@
 extern char cmdline[MAX_CMDLINE_SIZE];
 
 void start_kernel(void);
+void pre_suspend(void);
+void post_suspend(int canceled);
 void do_exit(void) __attribute__((noreturn));
 void arch_do_exit(void);
 void stop_kernel(void);
diff --git a/include/x86/os.h b/include/x86/os.h
index d155914..a73b63e 100644
--- a/include/x86/os.h
+++ b/include/x86/os.h
@@ -71,10 +71,10 @@ void trap_fini(void);
 void xen_callback_vector(void);
 #endif
 
+void arch_pre_suspend(void);
+void arch_post_suspend(int canceled);
 void arch_fini(void);
 
-
-
 #ifdef CONFIG_PARAVIRT
 
 /* 
diff --git a/kernel.c b/kernel.c
index 0d84a9b..90c865a 100644
--- a/kernel.c
+++ b/kernel.c
@@ -155,6 +155,16 @@ void start_kernel(void)
     run_idle_thread();
 }
 
+void pre_suspend(void)
+{
+
+}
+
+void post_suspend(int canceled)
+{
+
+}
+
 void stop_kernel(void)
 {
     /* TODO: fs import */
-- 
2.3.2 (Apple Git-55)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 04/17] Save/Restore Support: Add xenbus_release_wait_for_watch

[Bruno Alvisio]

xenbus_release_wait_for_watch generates a fake event to trigger make
xenbus_wait_for_watch return. This is necessary to wake up waiting threads.

release_xenbus_id additionally checks if the number of requests == 0 to wake
up the 'waiting' suspend xenbus thread.

Signed-off-by: Bruno Alvisio <bruno.alvisio@gmail.com>
Reviewed-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
---
Changed since v1:
  * Added doc for change in release_xenbus_id
---
 include/xenbus.h |  1 +
 xenbus/xenbus.c  | 10 +++++++++-
 2 files changed, 10 insertions(+), 1 deletion(-)

diff --git a/include/xenbus.h b/include/xenbus.h
index 12391b9..b2d5072 100644
--- a/include/xenbus.h
+++ b/include/xenbus.h
@@ -42,6 +42,7 @@ char *xenbus_unwatch_path_token(xenbus_transaction_t xbt, const char *path, cons
 extern struct wait_queue_head xenbus_watch_queue;
 void xenbus_wait_for_watch(xenbus_event_queue *queue);
 char **xenbus_wait_for_watch_return(xenbus_event_queue *queue);
+void xenbus_release_wait_for_watch(xenbus_event_queue *queue);
 char* xenbus_wait_for_value(const char *path, const char *value, xenbus_event_queue *queue);
 char *xenbus_wait_for_state_change(const char* path, XenbusState *state, xenbus_event_queue *queue);
 char *xenbus_switch_state(xenbus_transaction_t xbt, const char* path, XenbusState state);
diff --git a/xenbus/xenbus.c b/xenbus/xenbus.c
index 636786c..c2d2bd1 100644
--- a/xenbus/xenbus.c
+++ b/xenbus/xenbus.c
@@ -129,6 +129,14 @@ void xenbus_wait_for_watch(xenbus_event_queue *queue)
         printk("unexpected path returned by watch\n");
 }
 
+void xenbus_release_wait_for_watch(xenbus_event_queue *queue)
+{
+    struct xenbus_event *event = malloc(sizeof(*event));
+    event->next = *queue;
+    *queue = event;
+    wake_up(&xenbus_watch_queue);
+}
+
 char* xenbus_wait_for_value(const char* path, const char* value, xenbus_event_queue *queue)
 {
     if (!queue)
@@ -318,7 +326,7 @@ static void release_xenbus_id(int id)
     req_info[id].in_use = 0;
     nr_live_reqs--;
     req_info[id].in_use = 0;
-    if (nr_live_reqs == NR_REQS - 1)
+    if (nr_live_reqs == 0 || nr_live_reqs == NR_REQS - 1)
         wake_up(&req_wq);
     spin_unlock(&req_lock);
 }
-- 
2.3.2 (Apple Git-55)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 05/17] Save/Restore Support: Add kernel shutdown logic to shutdown.c

[Bruno Alvisio]

Created shutdown.c for the shutdown thread and all the shutdown related
functions.

Signed-off-by: Bruno Alvisio <bruno.alvisio@gmail.com>
---
Changesd since v1:
   * Updated license to a BSD 3-clause. This license was taken
from the updated original file. (Repo: sysml/mini-os)
---
 Makefile           |   1 +
 include/shutdown.h |  11 ++++
 shutdown.c         | 188 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 200 insertions(+)
 create mode 100644 include/shutdown.h
 create mode 100644 shutdown.c

diff --git a/Makefile b/Makefile
index 88315c4..6a05de6 100644
--- a/Makefile
+++ b/Makefile
@@ -53,6 +53,7 @@ src-y += mm.c
 src-$(CONFIG_NETFRONT) += netfront.c
 src-$(CONFIG_PCIFRONT) += pcifront.c
 src-y += sched.c
+src-y += shutdown.c
 src-$(CONFIG_TEST) += test.c
 src-$(CONFIG_BALLOON) += balloon.c
 
diff --git a/include/shutdown.h b/include/shutdown.h
new file mode 100644
index 0000000..a5ec019
--- /dev/null
+++ b/include/shutdown.h
@@ -0,0 +1,11 @@
+#ifndef _SHUTDOWN_H_
+#define _SHUTDOWN_H_
+
+#include <mini-os/hypervisor.h>
+
+void init_shutdown(start_info_t *si);
+
+void kernel_shutdown(int reason) __attribute__((noreturn));
+void kernel_suspend(void);
+
+#endif
diff --git a/shutdown.c b/shutdown.c
new file mode 100644
index 0000000..aba146e
--- /dev/null
+++ b/shutdown.c
@@ -0,0 +1,188 @@
+/*
+ *          MiniOS
+ *
+ *   file: fromdevice.cc
+ *
+ * Authors: Joao Martins <joao.martins@neclab.eu>
+ *
+ *
+ * Copyright (c) 2014, NEC Europe Ltd., NEC Corporation. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ * 3. Neither the name of the copyright holder nor the names of its
+ *    contributors may be used to endorse or promote products derived from
+ *    this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
+ * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+ * POSSIBILITY OF SUCH DAMAGE.
+ *
+ * THIS HEADER MAY NOT BE EXTRACTED OR MODIFIED IN ANY WAY.
+ */
+
+#include <mini-os/os.h>
+#include <mini-os/events.h>
+#include <mini-os/kernel.h>
+#include <mini-os/sched.h>
+#include <mini-os/shutdown.h>
+#include <mini-os/lib.h>
+#include <mini-os/xenbus.h>
+#include <mini-os/xmalloc.h>
+
+
+static start_info_t *start_info_ptr;
+
+static const char *path = "control/shutdown";
+static const char *token = "control/shutdown";
+static xenbus_event_queue events = NULL;
+static int end_shutdown_thread = 0;
+
+#ifdef CONFIG_XENBUS
+/* This should be overridden by the application we are linked against. */
+__attribute__((weak)) void app_shutdown(unsigned reason)
+{
+    printk("Shutdown requested: %d\n", reason);
+    if (reason == SHUTDOWN_suspend) {
+        kernel_suspend();
+    } else {
+        struct sched_shutdown sched_shutdown = { .reason = reason };
+        HYPERVISOR_sched_op(SCHEDOP_shutdown, &sched_shutdown);
+    }
+}
+
+static void shutdown_thread(void *p)
+{
+    char *shutdown, *err;
+    unsigned int shutdown_reason;
+
+    xenbus_watch_path_token(XBT_NIL, path, token, &events);
+
+    for ( ;; ) {
+        xenbus_wait_for_watch(&events);
+        if ((err = xenbus_read(XBT_NIL, path, &shutdown))) {
+            free(err);
+            do_exit();
+        }
+
+        if (end_shutdown_thread)
+            break;
+
+        if (!strcmp(shutdown, "")) {
+            /* Avoid spurious event on xenbus */
+            /* FIXME: investigate the reason of the spurious event */
+            free(shutdown);
+            continue;
+        } else if (!strcmp(shutdown, "poweroff")) {
+            shutdown_reason = SHUTDOWN_poweroff;
+        } else if (!strcmp(shutdown, "reboot")) {
+            shutdown_reason = SHUTDOWN_reboot;
+        } else if (!strcmp(shutdown, "suspend")) {
+            shutdown_reason = SHUTDOWN_suspend;
+        } else {
+            shutdown_reason = SHUTDOWN_crash;
+        }
+        free(shutdown);
+
+        /* Acknowledge shutdown request */
+        if ((err = xenbus_write(XBT_NIL, path, ""))) {
+            free(err);
+            do_exit();
+        }
+
+        app_shutdown(shutdown_reason);
+    }
+}
+#endif
+
+static void fini_shutdown(void)
+{
+    char *err;
+
+    end_shutdown_thread = 1;
+    xenbus_release_wait_for_watch(&events);
+    err = xenbus_unwatch_path_token(XBT_NIL, path, token);
+    if (err) {
+        free(err);
+        do_exit();
+    }
+}
+
+void init_shutdown(start_info_t *si)
+{
+    start_info_ptr = si;
+
+    end_shutdown_thread = 0;
+    create_thread("shutdown", shutdown_thread, NULL);
+}
+
+void kernel_shutdown(int reason)
+{
+    char* reason_str = NULL;
+
+    switch(reason) {
+        case SHUTDOWN_poweroff:
+            reason_str = "poweroff";
+            break;
+        case SHUTDOWN_reboot:
+            reason_str = "reboot";
+            break;
+        case SHUTDOWN_crash:
+            reason_str = "crash";
+            break;
+        default:
+            do_exit();
+            break;
+    }
+
+    printk("MiniOS will shutdown (reason = %s) ...\n", reason_str);
+
+    fini_shutdown();
+
+    stop_kernel();
+
+    for ( ;; ) {
+        struct sched_shutdown sched_shutdown = { .reason = reason };
+        HYPERVISOR_sched_op(SCHEDOP_shutdown, &sched_shutdown);
+    }
+}
+
+void kernel_suspend(void)
+{
+    int rc;
+
+    printk("MiniOS will suspend ...\n");
+
+    pre_suspend();
+    arch_pre_suspend();
+
+    /*
+     * This hypercall returns 1 if the suspend
+     * was cancelled and 0 if resuming in a new domain
+     */
+    rc = HYPERVISOR_suspend(virt_to_mfn(start_info_ptr));
+
+    arch_post_suspend(rc);
+    post_suspend(rc);
+
+    if (rc) {
+        printk("MiniOS suspend canceled!");
+    } else {
+        printk("MiniOS resumed from suspend!\n");
+    }
+}
-- 
2.3.2 (Apple Git-55)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 06/17] Save/Restore Support: Moved shutdown thread to shutdown.c

[Bruno Alvisio]

The shutdown thread present in kernel.c was removed and now the thread in
shutdown.c is created instead.

Signed-off-by: Bruno Alvisio <bruno.alvisio@gmail.com>
Reviewed-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
---
 arch/x86/setup.c |  2 +-
 include/kernel.h |  2 +-
 kernel.c         | 50 ++++++--------------------------------------------
 3 files changed, 8 insertions(+), 46 deletions(-)

diff --git a/arch/x86/setup.c b/arch/x86/setup.c
index 3dd86f9..31fa2c6 100644
--- a/arch/x86/setup.c
+++ b/arch/x86/setup.c
@@ -201,7 +201,7 @@ arch_init(void *par)
 	memcpy(&start_info, par, sizeof(start_info));
 #endif
 
-	start_kernel();
+	start_kernel((start_info_t *)par);
 }
 
 void arch_pre_suspend(void)
diff --git a/include/kernel.h b/include/kernel.h
index 161d757..742abf5 100644
--- a/include/kernel.h
+++ b/include/kernel.h
@@ -4,7 +4,7 @@
 #define MAX_CMDLINE_SIZE 1024
 extern char cmdline[MAX_CMDLINE_SIZE];
 
-void start_kernel(void);
+void start_kernel(void* par);
 void pre_suspend(void);
 void post_suspend(int canceled);
 void do_exit(void) __attribute__((noreturn));
diff --git a/kernel.c b/kernel.c
index 90c865a..1cd40e8 100644
--- a/kernel.c
+++ b/kernel.c
@@ -42,6 +42,9 @@
 #include <mini-os/blkfront.h>
 #include <mini-os/fbfront.h>
 #include <mini-os/pcifront.h>
+#ifdef CONFIG_XENBUS
+#include <mini-os/shutdown.h>
+#endif
 #include <mini-os/xmalloc.h>
 #include <fcntl.h>
 #include <xen/features.h>
@@ -66,48 +69,6 @@ void setup_xen_features(void)
     }
 }
 
-#ifdef CONFIG_XENBUS
-/* This should be overridden by the application we are linked against. */
-__attribute__((weak)) void app_shutdown(unsigned reason)
-{
-    struct sched_shutdown sched_shutdown = { .reason = reason };
-    printk("Shutdown requested: %d\n", reason);
-    HYPERVISOR_sched_op(SCHEDOP_shutdown, &sched_shutdown);
-}
-
-static void shutdown_thread(void *p)
-{
-    const char *path = "control/shutdown";
-    const char *token = path;
-    xenbus_event_queue events = NULL;
-    char *shutdown = NULL, *err;
-    unsigned int shutdown_reason;
-    xenbus_watch_path_token(XBT_NIL, path, token, &events);
-    while ((err = xenbus_read(XBT_NIL, path, &shutdown)) != NULL || !strcmp(shutdown, ""))
-    {
-        free(err);
-        free(shutdown);
-        shutdown = NULL;
-        xenbus_wait_for_watch(&events);
-    }
-    err = xenbus_unwatch_path_token(XBT_NIL, path, token);
-    free(err);
-    err = xenbus_write(XBT_NIL, path, "");
-    free(err);
-    printk("Shutting down (%s)\n", shutdown);
-
-    if (!strcmp(shutdown, "poweroff"))
-        shutdown_reason = SHUTDOWN_poweroff;
-    else if (!strcmp(shutdown, "reboot"))
-        shutdown_reason = SHUTDOWN_reboot;
-    else
-        /* Unknown */
-        shutdown_reason = SHUTDOWN_crash;
-    app_shutdown(shutdown_reason);
-    free(shutdown);
-}
-#endif
-
 
 /* This should be overridden by the application we are linked against. */
 __attribute__((weak)) int app_main(void *p)
@@ -116,7 +77,7 @@ __attribute__((weak)) int app_main(void *p)
     return 0;
 }
 
-void start_kernel(void)
+void start_kernel(void* par)
 {
     /* Set up events. */
     init_events();
@@ -145,7 +106,8 @@ void start_kernel(void)
     init_xenbus();
 
 #ifdef CONFIG_XENBUS
-    create_thread("shutdown", shutdown_thread, NULL);
+    /* Init shutdown thread */
+    init_shutdown((start_info_t *)par);
 #endif
 
     /* Call (possibly overridden) app_main() */
-- 
2.3.2 (Apple Git-55)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 07/17] Save/Restore Support: Add unmap_shared_info

[Bruno Alvisio]

This function is necessary as part of the pre-suspend operation.

Signed-off-by: Bruno Alvisio <bruno.alvisio@gmail.com>
Reviewed-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
---
Changed since v1:
 * Changed HYPERVISOR_shared_info for shared_info
---
 arch/x86/setup.c     | 12 ++++++++++++
 hypervisor.c         | 12 ++++++++++++
 include/hypervisor.h |  1 +
 3 files changed, 25 insertions(+)

diff --git a/arch/x86/setup.c b/arch/x86/setup.c
index 31fa2c6..b6e0541 100644
--- a/arch/x86/setup.c
+++ b/arch/x86/setup.c
@@ -93,6 +93,18 @@ shared_info_t *map_shared_info(void *p)
     return (shared_info_t *)shared_info;
 }
 
+void unmap_shared_info(void)
+{
+    int rc;
+
+    if ( (rc = HYPERVISOR_update_va_mapping((unsigned long)shared_info,
+            __pte((virt_to_mfn(shared_info)<<L1_PAGETABLE_SHIFT)| L1_PROT), UVMF_INVLPG)) )
+    {
+        printk("Failed to unmap shared_info page!! rc=%d\n", rc);
+        do_exit();
+    }
+}
+
 static void get_cmdline(void *p)
 {
     start_info_t *si = p;
diff --git a/hypervisor.c b/hypervisor.c
index 1647121..d3857e7 100644
--- a/hypervisor.c
+++ b/hypervisor.c
@@ -78,6 +78,18 @@ shared_info_t *map_shared_info(void *p)
 
     return &shared_info;
 }
+
+void unmap_shared_info(void)
+{
+    struct xen_remove_from_physmap xrtp;
+
+    xrtp.domid = DOMID_SELF;
+    xrtp.gpfn = virt_to_pfn(&shared_info);
+    if ( HYPERVISOR_memory_op(XENMEM_remove_from_physmap, &xrtp) != 0 )
+        BUG();
+
+    return;
+}
 #endif
 
 void do_hypervisor_callback(struct pt_regs *regs)
diff --git a/include/hypervisor.h b/include/hypervisor.h
index f3b1f3c..1d09271 100644
--- a/include/hypervisor.h
+++ b/include/hypervisor.h
@@ -43,6 +43,7 @@ int hvm_get_parameter(int idx, uint64_t *value);
 int hvm_set_parameter(int idx, uint64_t value);
 #endif
 shared_info_t *map_shared_info(void *p);
+void unmap_shared_info(void);
 void force_evtchn_callback(void);
 void do_hypervisor_callback(struct pt_regs *regs);
 void mask_evtchn(uint32_t port);
-- 
2.3.2 (Apple Git-55)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 08/17] Save/Restore Support: Add arch_mm_pre|post_suspend

[Bruno Alvisio]

For PV guests the pagetables reference the real MFNs rather than PFNs, so when
the guest is resumed into a different area of a hosts memory, these will need to
be rewritten. Thus for PV guests the MFNs need to be replaced with PFNs:
canonicalization.

PVH guests are auto-translated so no memory operation is needed.

Signed-off-by: Bruno Alvisio <bruno.alvisio@gmail.com>
Reviewed-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
---
 arch/x86/mm.c         | 14 ++++++++++++++
 include/x86/arch_mm.h |  3 +++
 2 files changed, 17 insertions(+)

diff --git a/arch/x86/mm.c b/arch/x86/mm.c
index 05ad029..1b163ac 100644
--- a/arch/x86/mm.c
+++ b/arch/x86/mm.c
@@ -848,6 +848,20 @@ void arch_init_p2m(unsigned long max_pfn)
 
     arch_remap_p2m(max_pfn);
 }
+
+void arch_mm_pre_suspend(void)
+{
+    //TODO: Canonicalize pagetables
+}
+
+void arch_mm_post_suspend(int canceled)
+{
+    //TODO: Locate pagetables and 'uncanonicalize' them
+}
+#else
+void arch_mm_pre_suspend(void){ }
+
+void arch_mm_post_suspend(int canceled){ }
 #endif
 
 void arch_init_mm(unsigned long* start_pfn_p, unsigned long* max_pfn_p)
diff --git a/include/x86/arch_mm.h b/include/x86/arch_mm.h
index ab8a53e..cbbeb21 100644
--- a/include/x86/arch_mm.h
+++ b/include/x86/arch_mm.h
@@ -279,6 +279,9 @@ pgentry_t *need_pgt(unsigned long addr);
 void arch_mm_preinit(void *p);
 unsigned long alloc_virt_kernel(unsigned n_pages);
 
+void arch_mm_pre_suspend(void);
+void arch_mm_post_suspend(int canceled);
+
 #ifndef CONFIG_PARAVIRT
 void arch_print_memmap(void);
 #endif
-- 
2.3.2 (Apple Git-55)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 09/17] Save/Restore Support: Disable/enable IRQs during suspend/restore

[Bruno Alvisio]

Signed-off-by: Bruno Alvisio <bruno.alvisio@gmail.com>
Reviewed-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
---
 kernel.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/kernel.c b/kernel.c
index 1cd40e8..782eb79 100644
--- a/kernel.c
+++ b/kernel.c
@@ -119,12 +119,12 @@ void start_kernel(void* par)
 
 void pre_suspend(void)
 {
-
+    local_irq_disable();
 }
 
 void post_suspend(int canceled)
 {
-
+    local_irq_enable();
 }
 
 void stop_kernel(void)
-- 
2.3.2 (Apple Git-55)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 10/17] Save/Restore Support: Add suspend/resume support for timers

[Bruno Alvisio]

Signed-off-by: Bruno Alvisio <bruno.alvisio@gmail.com>
---
Changed since v1:
   * Removed resume/suspend_time() and used init/fini_time() instead
---
 arch/x86/time.c | 1 -
 kernel.c        | 4 ++++
 2 files changed, 4 insertions(+), 1 deletion(-)

diff --git a/arch/x86/time.c b/arch/x86/time.c
index 3658142..8077c80 100644
--- a/arch/x86/time.c
+++ b/arch/x86/time.c
@@ -233,7 +233,6 @@ static void timer_handler(evtchn_port_t ev, struct pt_regs *regs, void *ign)
 static evtchn_port_t port;
 void init_time(void)
 {
-    printk("Initialising timer interface\n");
     port = bind_virq(VIRQ_TIMER, &timer_handler, NULL);
     unmask_evtchn(port);
 }
diff --git a/kernel.c b/kernel.c
index 782eb79..3564af3 100644
--- a/kernel.c
+++ b/kernel.c
@@ -120,10 +120,14 @@ void start_kernel(void* par)
 void pre_suspend(void)
 {
     local_irq_disable();
+
+    fini_time();
 }
 
 void post_suspend(int canceled)
 {
+    init_time();
+
     local_irq_enable();
 }
 
-- 
2.3.2 (Apple Git-55)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 11/17] Save/Restore Support: Add suspend/restore support for console

[Bruno Alvisio]

Signed-off-by: Bruno Alvisio <bruno.alvisio@gmail.com>
Reviewed-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
---
 console/console.c      | 15 ++++++++-
 console/xenbus.c       |  3 +-
 console/xencons_ring.c | 83 +++++++++++++++++++++++++++++++-------------------
 include/console.h      |  6 +++-
 kernel.c               |  4 +++
 lib/sys.c              |  2 +-
 6 files changed, 77 insertions(+), 36 deletions(-)

diff --git a/console/console.c b/console/console.c
index 2e04552..9814506 100644
--- a/console/console.c
+++ b/console/console.c
@@ -52,6 +52,7 @@
 
 /* If console not initialised the printk will be sent to xen serial line 
    NOTE: you need to enable verbose in xen/Rules.mk for it to work. */
+static struct consfront_dev* xen_console = NULL;
 static int console_initialised = 0;
 
 __attribute__((weak)) void console_input(char * buf, unsigned len)
@@ -162,8 +163,20 @@ void xprintk(const char *fmt, ...)
 void init_console(void)
 {   
     printk("Initialising console ... ");
-    xencons_ring_init();    
+    xen_console = xencons_ring_init();
     console_initialised = 1;
     /* This is also required to notify the daemon */
     printk("done.\n");
 }
+
+void suspend_console(void)
+{
+    console_initialised = 0;
+    xencons_ring_fini(xen_console);
+}
+
+void resume_console(void)
+{
+    xencons_ring_resume(xen_console);
+    console_initialised = 1;
+}
\ No newline at end of file
diff --git a/console/xenbus.c b/console/xenbus.c
index 1c9a590..654b469 100644
--- a/console/xenbus.c
+++ b/console/xenbus.c
@@ -188,8 +188,7 @@ error:
     return NULL;
 }
 
-void fini_console(struct consfront_dev *dev)
+void fini_consfront(struct consfront_dev *dev)
 {
     if (dev) free_consfront(dev);
 }
-
diff --git a/console/xencons_ring.c b/console/xencons_ring.c
index dd64a41..b6db74e 100644
--- a/console/xencons_ring.c
+++ b/console/xencons_ring.c
@@ -19,6 +19,8 @@ DECLARE_WAIT_QUEUE_HEAD(console_queue);
 static struct xencons_interface *console_ring;
 uint32_t console_evtchn;
 
+static struct consfront_dev* resume_xen_console(struct consfront_dev* dev);
+
 #ifdef CONFIG_PARAVIRT
 void get_console(void *p)
 {
@@ -32,10 +34,12 @@ void get_console(void *p)
 {
     uint64_t v = -1;
 
-    hvm_get_parameter(HVM_PARAM_CONSOLE_EVTCHN, &v);
+    if (hvm_get_parameter(HVM_PARAM_CONSOLE_EVTCHN, &v))
+        BUG();
     console_evtchn = v;
 
-    hvm_get_parameter(HVM_PARAM_CONSOLE_PFN, &v);
+    if (hvm_get_parameter(HVM_PARAM_CONSOLE_PFN, &v))
+        BUG();
     console_ring = (struct xencons_interface *)map_frame_virt(v);
 }
 #endif
@@ -89,9 +93,7 @@ int xencons_ring_send(struct consfront_dev *dev, const char *data, unsigned len)
     notify_daemon(dev);
 
     return sent;
-}	
-
-
+}
 
 void console_handle_input(evtchn_port_t port, struct pt_regs *regs, void *data)
 {
@@ -177,41 +179,60 @@ int xencons_ring_recv(struct consfront_dev *dev, char *data, unsigned len)
 
 struct consfront_dev *xencons_ring_init(void)
 {
-	int err;
-	struct consfront_dev *dev;
+    struct consfront_dev *dev;
 
-	if (!console_evtchn)
-		return 0;
+    if (!console_evtchn)
+        return 0;
 
-	dev = malloc(sizeof(struct consfront_dev));
-	memset(dev, 0, sizeof(struct consfront_dev));
-	dev->nodename = "device/console";
-	dev->dom = 0;
-	dev->backend = 0;
-	dev->ring_ref = 0;
+    dev = malloc(sizeof(struct consfront_dev));
+    memset(dev, 0, sizeof(struct consfront_dev));
+    dev->nodename = "device/console";
+    dev->dom = 0;
+    dev->backend = 0;
+    dev->ring_ref = 0;
 
 #ifdef HAVE_LIBC
-	dev->fd = -1;
+    dev->fd = -1;
 #endif
-	dev->evtchn = console_evtchn;
-	dev->ring = xencons_interface();
-
-	err = bind_evtchn(dev->evtchn, console_handle_input, dev);
-	if (err <= 0) {
-		printk("XEN console request chn bind failed %i\n", err);
-                free(dev);
-		return NULL;
-	}
-        unmask_evtchn(dev->evtchn);
 
-	/* In case we have in-flight data after save/restore... */
-	notify_daemon(dev);
+    return resume_xen_console(dev);
+}
+
+static struct consfront_dev* resume_xen_console(struct consfront_dev* dev)
+{
+    int err;
 
-	return dev;
+    dev->evtchn = console_evtchn;
+    dev->ring = xencons_interface();
+
+    err = bind_evtchn(dev->evtchn, console_handle_input, dev);
+    if (err <= 0) {
+        printk("XEN console request chn bind failed %i\n", err);
+        free(dev);
+        return NULL;
+    }
+    unmask_evtchn(dev->evtchn);
+
+    /* In case we have in-flight data after save/restore... */
+    notify_daemon(dev);
+
+    return dev;
 }
 
-void xencons_resume(void)
+void xencons_ring_fini(struct consfront_dev* dev)
 {
-	(void)xencons_ring_init();
+    if (dev)
+        mask_evtchn(dev->evtchn);
 }
 
+void xencons_ring_resume(struct consfront_dev* dev)
+{
+    if (dev) {
+#if CONFIG_PARAVIRT
+        get_console(&start_info);
+#else
+        get_console(0);
+#endif
+        resume_xen_console(dev);
+    }
+}
diff --git a/include/console.h b/include/console.h
index 539cccd..0d7bf07 100644
--- a/include/console.h
+++ b/include/console.h
@@ -78,11 +78,15 @@ void xencons_tx(void);
 void get_console(void *p);
 void init_console(void);
 void console_print(struct consfront_dev *dev, char *data, int length);
-void fini_console(struct consfront_dev *dev);
+void fini_consfront(struct consfront_dev *dev);
+void suspend_console(void);
+void resume_console(void);
 
 /* Low level functions defined in xencons_ring.c */
 extern struct wait_queue_head console_queue;
 struct consfront_dev *xencons_ring_init(void);
+void xencons_ring_fini(struct consfront_dev* dev);
+void xencons_ring_resume(struct consfront_dev* dev);
 struct consfront_dev *init_consfront(char *_nodename);
 int xencons_ring_send(struct consfront_dev *dev, const char *data, unsigned len);
 int xencons_ring_send_no_notify(struct consfront_dev *dev, const char *data, unsigned len);
diff --git a/kernel.c b/kernel.c
index 3564af3..2fb69bf 100644
--- a/kernel.c
+++ b/kernel.c
@@ -122,10 +122,14 @@ void pre_suspend(void)
     local_irq_disable();
 
     fini_time();
+
+    suspend_console();
 }
 
 void post_suspend(int canceled)
 {
+    resume_console();
+
     init_time();
 
     local_irq_enable();
diff --git a/lib/sys.c b/lib/sys.c
index 23dc2a5..da434fc 100644
--- a/lib/sys.c
+++ b/lib/sys.c
@@ -487,7 +487,7 @@ int close(int fd)
 #ifdef CONFIG_CONSFRONT
         case FTYPE_SAVEFILE:
         case FTYPE_CONSOLE:
-            fini_console(files[fd].cons.dev);
+            fini_consfront(files[fd].cons.dev);
             files[fd].type = FTYPE_NONE;
             return 0;
 #endif
-- 
2.3.2 (Apple Git-55)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 00/16] Save/Restore Support for mini-OS PVH

[Bruno Alvisio]

Hi all,

I am sending the second revision for supporting save/restore in Mini-OS PVH. The 
branch can be found at: 

https://github.com/balvisio/mini-os/tree/feature/mini-os-suspend-support-submission-2

Feedback would be greatly appreciated.

Cheers,

Bruno

Signed-off-by: Bruno Alvisio <bruno.alvisio@gmail.com>


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel