[Qemu-devel] [PATCH v6 0/2] qemu-ga: add hook to quiesce the guest on fsfreeze-freeze/thaw

[Qemu-devel] [PATCH v6 0/2] qemu-ga: add hook to quiesce the guest on fsfreeze-freeze/thaw

[Tomoki Sekiyama]

Hi,

This is version 6 of the qemu-ga fsfreeze hook patchset.

*Changes from v5:
 patch 1/3 and 3/3 are squashed.

*Changes from v4: ( http://patchwork.ozlabs.org/patch/200924/ )
 1/2: Improve error messages based on Luiz Capitulino's patchset.
      Make fsfreeze-hook disabled by default, and enabled only if
      --fsfreeze-hook option is specified.
 2/2: Add exit 0 at the last of the sample script.
      Add check for presence of mysql in mysql-flush.sh.

This patchset depends on Luiz Capitulino's patchset to improve error
handling, which is available at:
https://lists.gnu.org/archive/html/qemu-devel/2012-11/msg03016.html

---

Tomoki Sekiyama (2):
      qemu-ga: execute hook to quiesce the guest on fsfreeze-freeze/thaw
      qemu-ga: sample fsfreeze hooks


 docs/qemu-guest-agent/fsfreeze-hook                |   33 +++++++++
 .../fsfreeze-hook.d.sample/mysql-flush.sh          |   55 ++++++++++++++++
 qemu-ga.c                                          |   42 ++++++++++++
 qga/commands-posix.c                               |   70 ++++++++++++++++++++
 qga/guest-agent-core.h                             |    1 
 5 files changed, 200 insertions(+), 1 deletion(-)
 create mode 100755 docs/qemu-guest-agent/fsfreeze-hook
 create mode 100755 docs/qemu-guest-agent/fsfreeze-hook.d.sample/mysql-flush.sh


Thanks,
-- 
Tomoki Sekiyama <tomoki.sekiyama.qu@hitachi.com>
Linux Technology Center
Hitachi, Ltd., Yokohama Research Laboratory

Re: [Qemu-devel] [PATCH v6 1/2] qemu-ga: execute hook to quiesce the guest on fsfreeze-freeze/thaw

[Tomoki Sekiyama]

To use the online disk snapshot for online-backup, application-level
consistency of the snapshot image is required. However, currently the
guest agent can provide only filesystem-level consistency, and the
snapshot may contain dirty data, for example, incomplete transactions.
This patch provides the opportunity to quiesce applications before
snapshot is taken.

If --fsfreeze-hook option is specified, the hook is executed with
"freeze" argument before the filesystem is frozen by fsfreeze-freeze
command. As for fsfreeze-thaw command, the hook is executed with "thaw"
argument after the filesystem is thawed.

This patch depends on patchset to improve error reporting by Luiz Capitulino:
  http://lists.gnu.org/archive/html/qemu-devel/2012-11/msg03016.html

Signed-off-by: Tomoki Sekiyama <tomoki.sekiyama.qu@hitachi.com>
---
 qemu-ga.c              |   42 ++++++++++++++++++++++++++++-
 qga/commands-posix.c   |   70 ++++++++++++++++++++++++++++++++++++++++++++++++
 qga/guest-agent-core.h |    1 +
 3 files changed, 112 insertions(+), 1 deletion(-)

diff --git a/qemu-ga.c b/qemu-ga.c
index 9b59a52..53ce462 100644
--- a/qemu-ga.c
+++ b/qemu-ga.c
@@ -34,6 +34,12 @@
 #include "qga/service-win32.h"
 #include <windows.h>
 #endif
+#ifdef __linux__
+#include <linux/fs.h>
+#ifdef FIFREEZE
+#define CONFIG_FSFREEZE
+#endif
+#endif
 
 #ifndef _WIN32
 #define QGA_VIRTIO_PATH_DEFAULT "/dev/virtio-ports/org.qemu.guest_agent.0"
@@ -42,6 +48,9 @@
 #endif
 #define QGA_STATEDIR_DEFAULT CONFIG_QEMU_LOCALSTATEDIR "/run"
 #define QGA_PIDFILE_DEFAULT QGA_STATEDIR_DEFAULT "/qemu-ga.pid"
+#ifdef CONFIG_FSFREEZE
+#define QGA_FSFREEZE_HOOK_DEFAULT CONFIG_QEMU_CONFDIR "/fsfreeze-hook"
+#endif
 #define QGA_SENTINEL_BYTE 0xFF
 
 struct GAState {
@@ -64,6 +73,9 @@ struct GAState {
         const char *log_filepath;
         const char *pid_filepath;
     } deferred_options;
+#ifdef CONFIG_FSFREEZE
+    const char *fsfreeze_hook;
+#endif
 };
 
 struct GAState *ga_state;
@@ -153,6 +165,10 @@ static void usage(const char *cmd)
 "                    %s)\n"
 "  -l, --logfile     set logfile path, logs to stderr by default\n"
 "  -f, --pidfile     specify pidfile (default is %s)\n"
+#ifdef CONFIG_FSFREEZE
+"  -F, --fsfreeze-hook\n"
+"                    enable fsfreeze hook (default is %s)\n"
+#endif
 "  -t, --statedir    specify dir to store state information (absolute paths\n"
 "                    only, default is %s)\n"
 "  -v, --verbose     log extra debugging information\n"
@@ -167,6 +183,9 @@ static void usage(const char *cmd)
 "\n"
 "Report bugs to <mdroth@linux.vnet.ibm.com>\n"
     , cmd, QEMU_VERSION, QGA_VIRTIO_PATH_DEFAULT, QGA_PIDFILE_DEFAULT,
+#ifdef CONFIG_FSFREEZE
+    QGA_FSFREEZE_HOOK_DEFAULT,
+#endif
     QGA_STATEDIR_DEFAULT);
 }
 
@@ -401,6 +420,13 @@ void ga_unset_frozen(GAState *s)
     }
 }
 
+#ifdef CONFIG_FSFREEZE
+const char *ga_fsfreeze_hook(GAState *s)
+{
+    return s->fsfreeze_hook;
+}
+#endif
+
 static void become_daemon(const char *pidfile)
 {
 #ifndef _WIN32
@@ -678,10 +704,13 @@ VOID WINAPI service_main(DWORD argc, TCHAR *argv[])
 
 int main(int argc, char **argv)
 {
-    const char *sopt = "hVvdm:p:l:f:b:s:t:";
+    const char *sopt = "hVvdm:p:l:f:F::b:s:t:";
     const char *method = NULL, *path = NULL;
     const char *log_filepath = NULL;
     const char *pid_filepath = QGA_PIDFILE_DEFAULT;
+#ifdef CONFIG_FSFREEZE
+    const char *fsfreeze_hook = NULL;
+#endif
     const char *state_dir = QGA_STATEDIR_DEFAULT;
 #ifdef _WIN32
     const char *service = NULL;
@@ -691,6 +720,9 @@ int main(int argc, char **argv)
         { "version", 0, NULL, 'V' },
         { "logfile", 1, NULL, 'l' },
         { "pidfile", 1, NULL, 'f' },
+#ifdef CONFIG_FSFREEZE
+        { "fsfreeze-hook", 2, NULL, 'F' },
+#endif
         { "verbose", 0, NULL, 'v' },
         { "method", 1, NULL, 'm' },
         { "path", 1, NULL, 'p' },
@@ -723,6 +755,11 @@ int main(int argc, char **argv)
         case 'f':
             pid_filepath = optarg;
             break;
+#ifdef CONFIG_FSFREEZE
+        case 'F':
+            fsfreeze_hook = optarg ? optarg : QGA_FSFREEZE_HOOK_DEFAULT;
+            break;
+#endif
         case 't':
              state_dir = optarg;
              break;
@@ -786,6 +823,9 @@ int main(int argc, char **argv)
     s = g_malloc0(sizeof(GAState));
     s->log_level = log_level;
     s->log_file = stderr;
+#ifdef CONFIG_FSFREEZE
+    s->fsfreeze_hook = fsfreeze_hook;
+#endif
     g_log_set_default_handler(ga_log, s);
     g_log_set_fatal_mask(NULL, G_LOG_LEVEL_ERROR);
     ga_enable_logging(s);
diff --git a/qga/commands-posix.c b/qga/commands-posix.c
index ce058b5..dbfcbdf 100644
--- a/qga/commands-posix.c
+++ b/qga/commands-posix.c
@@ -410,6 +410,67 @@ static void build_fs_mount_list(FsMountList *mounts, Error **err)
 
 #if defined(CONFIG_FSFREEZE)
 
+typedef enum {
+    FSFREEZE_HOOK_THAW = 0,
+    FSFREEZE_HOOK_FREEZE,
+} FsfreezeHookArg;
+
+const char *fsfreeze_hook_arg_string[] = {
+    "thaw",
+    "freeze",
+};
+
+static void execute_fsfreeze_hook(FsfreezeHookArg arg, Error **err)
+{
+    int status;
+    pid_t pid;
+    const char *hook;
+    const char *arg_str = fsfreeze_hook_arg_string[arg];
+    Error *local_err = NULL;
+
+    hook = ga_fsfreeze_hook(ga_state);
+    if (!hook) {
+        return;
+    }
+    if (access(hook, X_OK) != 0) {
+        error_setg_errno(err, errno,
+                         "fsfreeze hook '%s' is not executable", hook);
+        return;
+    }
+
+    slog("executing fsfreeze hook with arg `%s'", arg_str);
+    pid = fork();
+    if (pid == 0) {
+        setsid();
+        reopen_fd_to_null(0);
+        reopen_fd_to_null(1);
+        reopen_fd_to_null(2);
+
+        execle(hook, hook, arg_str, NULL, environ);
+        _exit(EXIT_FAILURE);
+    } else if (pid < 0) {
+        error_setg_errno(err, errno, "failed to create child process");
+        return;
+    }
+
+    ga_wait_child(pid, &status, &local_err);
+    if (error_is_set(&local_err)) {
+        error_propagate(err, local_err);
+        return;
+    }
+
+    if (!WIFEXITED(status)) {
+        error_setg(err, "fsfreeze hook has terminated abnormally");
+        return;
+    }
+
+    status = WEXITSTATUS(status);
+    if (status) {
+        error_setg(err, "fsfreeze hook has failed with status %d", status);
+        return;
+    }
+}
+
 /*
  * Return status of freeze/thaw
  */
@@ -436,6 +497,12 @@ int64_t qmp_guest_fsfreeze_freeze(Error **err)
 
     slog("guest-fsfreeze called");
 
+    execute_fsfreeze_hook(FSFREEZE_HOOK_FREEZE, &local_err);
+    if (error_is_set(&local_err)) {
+        error_propagate(err, local_err);
+        return -1;
+    }
+
     QTAILQ_INIT(&mounts);
     build_fs_mount_list(&mounts, &local_err);
     if (error_is_set(&local_err)) {
@@ -537,6 +604,9 @@ int64_t qmp_guest_fsfreeze_thaw(Error **err)
 
     ga_unset_frozen(ga_state);
     free_fs_mount_list(&mounts);
+
+    execute_fsfreeze_hook(FSFREEZE_HOOK_THAW, err);
+
     return i;
 }
 
diff --git a/qga/guest-agent-core.h b/qga/guest-agent-core.h
index 49a7abe..c6e8de0 100644
--- a/qga/guest-agent-core.h
+++ b/qga/guest-agent-core.h
@@ -34,6 +34,7 @@ void ga_set_response_delimited(GAState *s);
 bool ga_is_frozen(GAState *s);
 void ga_set_frozen(GAState *s);
 void ga_unset_frozen(GAState *s);
+const char *ga_fsfreeze_hook(GAState *s);
 
 #ifndef _WIN32
 void reopen_fd_to_null(int fd);

Re: [Qemu-devel] [PATCH v6 1/2] qemu-ga: execute hook to quiesce the guest on fsfreeze-freeze/thaw

[Luiz Capitulino]

On Fri, 30 Nov 2012 22:33:54 +0900
Tomoki Sekiyama <tomoki.sekiyama.qu@hitachi.com> wrote:

> To use the online disk snapshot for online-backup, application-level
> consistency of the snapshot image is required. However, currently the
> guest agent can provide only filesystem-level consistency, and the
> snapshot may contain dirty data, for example, incomplete transactions.
> This patch provides the opportunity to quiesce applications before
> snapshot is taken.
> 
> If --fsfreeze-hook option is specified, the hook is executed with
> "freeze" argument before the filesystem is frozen by fsfreeze-freeze
> command. As for fsfreeze-thaw command, the hook is executed with "thaw"
> argument after the filesystem is thawed.
> 
> This patch depends on patchset to improve error reporting by Luiz Capitulino:
>   http://lists.gnu.org/archive/html/qemu-devel/2012-11/msg03016.html
> 
> Signed-off-by: Tomoki Sekiyama <tomoki.sekiyama.qu@hitachi.com>
> ---
>  qemu-ga.c              |   42 ++++++++++++++++++++++++++++-
>  qga/commands-posix.c   |   70 ++++++++++++++++++++++++++++++++++++++++++++++++
>  qga/guest-agent-core.h |    1 +
>  3 files changed, 112 insertions(+), 1 deletion(-)
> 
> diff --git a/qemu-ga.c b/qemu-ga.c
> index 9b59a52..53ce462 100644
> --- a/qemu-ga.c
> +++ b/qemu-ga.c
> @@ -34,6 +34,12 @@
>  #include "qga/service-win32.h"
>  #include <windows.h>
>  #endif
> +#ifdef __linux__
> +#include <linux/fs.h>
> +#ifdef FIFREEZE
> +#define CONFIG_FSFREEZE
> +#endif
> +#endif
>  
>  #ifndef _WIN32
>  #define QGA_VIRTIO_PATH_DEFAULT "/dev/virtio-ports/org.qemu.guest_agent.0"
> @@ -42,6 +48,9 @@
>  #endif
>  #define QGA_STATEDIR_DEFAULT CONFIG_QEMU_LOCALSTATEDIR "/run"
>  #define QGA_PIDFILE_DEFAULT QGA_STATEDIR_DEFAULT "/qemu-ga.pid"
> +#ifdef CONFIG_FSFREEZE
> +#define QGA_FSFREEZE_HOOK_DEFAULT CONFIG_QEMU_CONFDIR "/fsfreeze-hook"
> +#endif
>  #define QGA_SENTINEL_BYTE 0xFF
>  
>  struct GAState {
> @@ -64,6 +73,9 @@ struct GAState {
>          const char *log_filepath;
>          const char *pid_filepath;
>      } deferred_options;
> +#ifdef CONFIG_FSFREEZE
> +    const char *fsfreeze_hook;
> +#endif
>  };
>  
>  struct GAState *ga_state;
> @@ -153,6 +165,10 @@ static void usage(const char *cmd)
>  "                    %s)\n"
>  "  -l, --logfile     set logfile path, logs to stderr by default\n"
>  "  -f, --pidfile     specify pidfile (default is %s)\n"
> +#ifdef CONFIG_FSFREEZE
> +"  -F, --fsfreeze-hook\n"
> +"                    enable fsfreeze hook (default is %s)\n"
> +#endif
>  "  -t, --statedir    specify dir to store state information (absolute paths\n"
>  "                    only, default is %s)\n"
>  "  -v, --verbose     log extra debugging information\n"
> @@ -167,6 +183,9 @@ static void usage(const char *cmd)
>  "\n"
>  "Report bugs to <mdroth@linux.vnet.ibm.com>\n"
>      , cmd, QEMU_VERSION, QGA_VIRTIO_PATH_DEFAULT, QGA_PIDFILE_DEFAULT,
> +#ifdef CONFIG_FSFREEZE
> +    QGA_FSFREEZE_HOOK_DEFAULT,
> +#endif
>      QGA_STATEDIR_DEFAULT);
>  }
>  
> @@ -401,6 +420,13 @@ void ga_unset_frozen(GAState *s)
>      }
>  }
>  
> +#ifdef CONFIG_FSFREEZE
> +const char *ga_fsfreeze_hook(GAState *s)
> +{
> +    return s->fsfreeze_hook;
> +}
> +#endif
> +
>  static void become_daemon(const char *pidfile)
>  {
>  #ifndef _WIN32
> @@ -678,10 +704,13 @@ VOID WINAPI service_main(DWORD argc, TCHAR *argv[])
>  
>  int main(int argc, char **argv)
>  {
> -    const char *sopt = "hVvdm:p:l:f:b:s:t:";
> +    const char *sopt = "hVvdm:p:l:f:F::b:s:t:";
>      const char *method = NULL, *path = NULL;
>      const char *log_filepath = NULL;
>      const char *pid_filepath = QGA_PIDFILE_DEFAULT;
> +#ifdef CONFIG_FSFREEZE
> +    const char *fsfreeze_hook = NULL;
> +#endif
>      const char *state_dir = QGA_STATEDIR_DEFAULT;
>  #ifdef _WIN32
>      const char *service = NULL;
> @@ -691,6 +720,9 @@ int main(int argc, char **argv)
>          { "version", 0, NULL, 'V' },
>          { "logfile", 1, NULL, 'l' },
>          { "pidfile", 1, NULL, 'f' },
> +#ifdef CONFIG_FSFREEZE
> +        { "fsfreeze-hook", 2, NULL, 'F' },
> +#endif
>          { "verbose", 0, NULL, 'v' },
>          { "method", 1, NULL, 'm' },
>          { "path", 1, NULL, 'p' },
> @@ -723,6 +755,11 @@ int main(int argc, char **argv)
>          case 'f':
>              pid_filepath = optarg;
>              break;
> +#ifdef CONFIG_FSFREEZE
> +        case 'F':
> +            fsfreeze_hook = optarg ? optarg : QGA_FSFREEZE_HOOK_DEFAULT;
> +            break;
> +#endif
>          case 't':
>               state_dir = optarg;
>               break;
> @@ -786,6 +823,9 @@ int main(int argc, char **argv)
>      s = g_malloc0(sizeof(GAState));
>      s->log_level = log_level;
>      s->log_file = stderr;
> +#ifdef CONFIG_FSFREEZE
> +    s->fsfreeze_hook = fsfreeze_hook;
> +#endif
>      g_log_set_default_handler(ga_log, s);
>      g_log_set_fatal_mask(NULL, G_LOG_LEVEL_ERROR);
>      ga_enable_logging(s);
> diff --git a/qga/commands-posix.c b/qga/commands-posix.c
> index ce058b5..dbfcbdf 100644
> --- a/qga/commands-posix.c
> +++ b/qga/commands-posix.c
> @@ -410,6 +410,67 @@ static void build_fs_mount_list(FsMountList *mounts, Error **err)
>  
>  #if defined(CONFIG_FSFREEZE)
>  
> +typedef enum {
> +    FSFREEZE_HOOK_THAW = 0,
> +    FSFREEZE_HOOK_FREEZE,
> +} FsfreezeHookArg;
> +
> +const char *fsfreeze_hook_arg_string[] = {
> +    "thaw",
> +    "freeze",
> +};
> +
> +static void execute_fsfreeze_hook(FsfreezeHookArg arg, Error **err)
> +{
> +    int status;
> +    pid_t pid;
> +    const char *hook;
> +    const char *arg_str = fsfreeze_hook_arg_string[arg];
> +    Error *local_err = NULL;
> +
> +    hook = ga_fsfreeze_hook(ga_state);
> +    if (!hook) {
> +        return;
> +    }
> +    if (access(hook, X_OK) != 0) {
> +        error_setg_errno(err, errno,
> +                         "fsfreeze hook '%s' is not executable", hook);

Please, change this to something like:

  error_setg_errno(err, errno, "can't access hook script '%s'", hook);

Otherwise we'll get error messages like:

 { "error": {
   "class": "GenericError",
   "desc": "fsfreeze hook '/usr/local/etc/qemu/fsfreeze-hook' is not executable: No such file or directory" } }

Otherwise this patch looks good now.

> +        return;
> +    }
> +
> +    slog("executing fsfreeze hook with arg `%s'", arg_str);
> +    pid = fork();
> +    if (pid == 0) {
> +        setsid();
> +        reopen_fd_to_null(0);
> +        reopen_fd_to_null(1);
> +        reopen_fd_to_null(2);
> +
> +        execle(hook, hook, arg_str, NULL, environ);
> +        _exit(EXIT_FAILURE);
> +    } else if (pid < 0) {
> +        error_setg_errno(err, errno, "failed to create child process");
> +        return;
> +    }
> +
> +    ga_wait_child(pid, &status, &local_err);
> +    if (error_is_set(&local_err)) {
> +        error_propagate(err, local_err);
> +        return;
> +    }
> +
> +    if (!WIFEXITED(status)) {
> +        error_setg(err, "fsfreeze hook has terminated abnormally");
> +        return;
> +    }
> +
> +    status = WEXITSTATUS(status);
> +    if (status) {
> +        error_setg(err, "fsfreeze hook has failed with status %d", status);
> +        return;
> +    }
> +}
> +
>  /*
>   * Return status of freeze/thaw
>   */
> @@ -436,6 +497,12 @@ int64_t qmp_guest_fsfreeze_freeze(Error **err)
>  
>      slog("guest-fsfreeze called");
>  
> +    execute_fsfreeze_hook(FSFREEZE_HOOK_FREEZE, &local_err);
> +    if (error_is_set(&local_err)) {
> +        error_propagate(err, local_err);
> +        return -1;
> +    }
> +
>      QTAILQ_INIT(&mounts);
>      build_fs_mount_list(&mounts, &local_err);
>      if (error_is_set(&local_err)) {
> @@ -537,6 +604,9 @@ int64_t qmp_guest_fsfreeze_thaw(Error **err)
>  
>      ga_unset_frozen(ga_state);
>      free_fs_mount_list(&mounts);
> +
> +    execute_fsfreeze_hook(FSFREEZE_HOOK_THAW, err);
> +
>      return i;
>  }
>  
> diff --git a/qga/guest-agent-core.h b/qga/guest-agent-core.h
> index 49a7abe..c6e8de0 100644
> --- a/qga/guest-agent-core.h
> +++ b/qga/guest-agent-core.h
> @@ -34,6 +34,7 @@ void ga_set_response_delimited(GAState *s);
>  bool ga_is_frozen(GAState *s);
>  void ga_set_frozen(GAState *s);
>  void ga_unset_frozen(GAState *s);
> +const char *ga_fsfreeze_hook(GAState *s);
>  
>  #ifndef _WIN32
>  void reopen_fd_to_null(int fd);
> 

Re: [Qemu-devel] [PATCH v6 2/2] qemu-ga: sample fsfreeze hooks

[Tomoki Sekiyama]

Adds sample hook scripts for --fsfreeze-hook option of qemu-ga.
  - fsfreeze-hook : execute scripts in fsfreeze-hook.d/
  - fsfreeze-hook.d.sample/mysql-flush.sh : quiesce MySQL before snapshot

Signed-off-by: Tomoki Sekiyama <tomoki.sekiyama.qu@hitachi.com>
---
 docs/qemu-guest-agent/fsfreeze-hook                |   33 ++++++++++++
 .../fsfreeze-hook.d.sample/mysql-flush.sh          |   55 ++++++++++++++++++++
 2 files changed, 88 insertions(+)
 create mode 100755 docs/qemu-guest-agent/fsfreeze-hook
 create mode 100755 docs/qemu-guest-agent/fsfreeze-hook.d.sample/mysql-flush.sh

diff --git a/docs/qemu-guest-agent/fsfreeze-hook b/docs/qemu-guest-agent/fsfreeze-hook
new file mode 100755
index 0000000..ed7d86d
--- /dev/null
+++ b/docs/qemu-guest-agent/fsfreeze-hook
@@ -0,0 +1,33 @@
+#!/bin/sh
+
+# This script is executed when a guest agent receives fsfreeze-freeze and
+# fsfreeze-thaw command, if it is specified in --fsfreeze-hook (-F)
+# option of qemu-ga or placed in default path (/etc/qemu/fsfreeze-hook).
+# When the agent receives fsfreeze-freeze request, this script is issued with
+# "freeze" argument before the filesystem is freezed. And for fsfreeze-thaw
+# request, it is issued with "thaw" argument after filesystem is thawed.
+
+LOGFILE=/var/log/qga-fsfreeze-hook.log
+FSFREEZE_D=$(dirname -- "$0")/fsfreeze-hook.d
+
+# Check whether file $1 is a backup or rpm-generated file and should be ignored
+is_ignored_file() {
+    case "$1" in
+        *~ | *.bak | *.orig | *.rpmnew | *.rpmorig | *.rpmsave)
+            return 0 ;;
+    esac
+    return 1
+}
+
+# Iterate executables in directory "fsfreeze-hook.d" with the specified args
+[ ! -d "$FSFREEZE_D" ] && exit 1
+for file in "$FSFREEZE_D"/* ; do
+    is_ignored_file "$file" && continue
+    [ -x "$file" ] || continue
+    echo "$(date): execute $file $@" >>$LOGFILE
+    "$file" "$@" >>$LOGFILE 2>&1
+    STATUS=$?
+    echo "$(date): $file finished with status=$STATUS" >>$LOGFILE
+done
+
+exit 0
diff --git a/docs/qemu-guest-agent/fsfreeze-hook.d.sample/mysql-flush.sh b/docs/qemu-guest-agent/fsfreeze-hook.d.sample/mysql-flush.sh
new file mode 100755
index 0000000..611215e
--- /dev/null
+++ b/docs/qemu-guest-agent/fsfreeze-hook.d.sample/mysql-flush.sh
@@ -0,0 +1,55 @@
+#!/bin/sh
+
+# Flush MySQL tables to the disk before the filesystem is freezed.
+# At the same time, this keeps a read lock while the filesystem is freezed
+# in order to avoid write accesses by the other clients.
+
+MYSQL="/usr/bin/mysql"
+MYSQL_OPTS="-uroot" #"-prootpassword"
+FIFO=/tmp/mysql-flush.fifo
+MYSQL_CMD="$MYSQL $MYSQL_OPTS"
+
+# Check mysql is installed and the server running
+[ -x $MYSQL ] && $MYSQL_CMD < /dev/null || exit 0
+
+flush_and_wait() {
+    printf "FLUSH TABLES WITH READ LOCK \\G\n"
+    read < $FIFO
+    printf "UNLOCK TABLES \\G\n"
+}
+
+case "$1" in
+    freeze)
+        mkfifo $FIFO || exit 1
+        flush_and_wait | $MYSQL_CMD &
+        # wait until every block is flushed
+        while [ "$(echo 'SHOW STATUS LIKE "Key_blocks_not_flushed"' |\
+                 $MYSQL_CMD | tail -1 | cut -f 2)" -gt 0 ]; do
+            sleep 1
+        done
+        # for InnoDB, wait until every log is flushed
+        INNODB_STATUS=$(mktemp /tmp/mysql-flush.XXXXXX)
+        [ $? -ne 0 ] && exit 2
+        trap "rm -f $INNODB_STATUS" SIGINT
+        while :; do
+            printf "SHOW ENGINE INNODB STATUS \\G" | $MYSQL_CMD > $INNODB_STATUS
+            LOG_CURRENT=$(grep 'Log sequence number' $INNODB_STATUS |\
+                          tr -s ' ' | cut -d' ' -f4)
+            LOG_FLUSHED=$(grep 'Log flushed up to' $INNODB_STATUS |\
+                          tr -s ' ' | cut -d' ' -f5)
+            [ "$LOG_CURRENT" = "$LOG_FLUSHED" ] && break
+            sleep 1
+        done
+        rm -f $INNODB_STATUS
+        ;;
+
+    thaw)
+        [ ! -p $FIFO ] && exit 1
+        echo > $FIFO
+        rm -f $FIFO
+        ;;
+
+    *)
+        exit 1
+        ;;
+esac

Re: [Qemu-devel] [PATCH v6 2/2] qemu-ga: sample fsfreeze hooks

[Luiz Capitulino]

On Fri, 30 Nov 2012 22:33:57 +0900
Tomoki Sekiyama <tomoki.sekiyama.qu@hitachi.com> wrote:

> Adds sample hook scripts for --fsfreeze-hook option of qemu-ga.
>   - fsfreeze-hook : execute scripts in fsfreeze-hook.d/
>   - fsfreeze-hook.d.sample/mysql-flush.sh : quiesce MySQL before snapshot
> 
> Signed-off-by: Tomoki Sekiyama <tomoki.sekiyama.qu@hitachi.com>
> ---
>  docs/qemu-guest-agent/fsfreeze-hook                |   33 ++++++++++++
>  .../fsfreeze-hook.d.sample/mysql-flush.sh          |   55 ++++++++++++++++++++
>  2 files changed, 88 insertions(+)
>  create mode 100755 docs/qemu-guest-agent/fsfreeze-hook
>  create mode 100755 docs/qemu-guest-agent/fsfreeze-hook.d.sample/mysql-flush.sh
> 
> diff --git a/docs/qemu-guest-agent/fsfreeze-hook b/docs/qemu-guest-agent/fsfreeze-hook
> new file mode 100755
> index 0000000..ed7d86d
> --- /dev/null
> +++ b/docs/qemu-guest-agent/fsfreeze-hook
> @@ -0,0 +1,33 @@
> +#!/bin/sh
> +
> +# This script is executed when a guest agent receives fsfreeze-freeze and
> +# fsfreeze-thaw command, if it is specified in --fsfreeze-hook (-F)
> +# option of qemu-ga or placed in default path (/etc/qemu/fsfreeze-hook).
> +# When the agent receives fsfreeze-freeze request, this script is issued with
> +# "freeze" argument before the filesystem is freezed. And for fsfreeze-thaw
> +# request, it is issued with "thaw" argument after filesystem is thawed.
> +
> +LOGFILE=/var/log/qga-fsfreeze-hook.log
> +FSFREEZE_D=$(dirname -- "$0")/fsfreeze-hook.d

The script you introduce is called fsfreeze-hook.d.sample. If it's expected
that people really use these two scripts (vs. doc purposes only) I suggest
the following:

 1. Move these files to scripts/
 2. Drop the .sample suffix

> +
> +# Check whether file $1 is a backup or rpm-generated file and should be ignored
> +is_ignored_file() {
> +    case "$1" in
> +        *~ | *.bak | *.orig | *.rpmnew | *.rpmorig | *.rpmsave)
> +            return 0 ;;
> +    esac
> +    return 1
> +}
> +
> +# Iterate executables in directory "fsfreeze-hook.d" with the specified args
> +[ ! -d "$FSFREEZE_D" ] && exit 1
> +for file in "$FSFREEZE_D"/* ; do
> +    is_ignored_file "$file" && continue
> +    [ -x "$file" ] || continue
> +    echo "$(date): execute $file $@" >>$LOGFILE
> +    "$file" "$@" >>$LOGFILE 2>&1
> +    STATUS=$?
> +    echo "$(date): $file finished with status=$STATUS" >>$LOGFILE
> +done
> +
> +exit 0
> diff --git a/docs/qemu-guest-agent/fsfreeze-hook.d.sample/mysql-flush.sh b/docs/qemu-guest-agent/fsfreeze-hook.d.sample/mysql-flush.sh
> new file mode 100755
> index 0000000..611215e
> --- /dev/null
> +++ b/docs/qemu-guest-agent/fsfreeze-hook.d.sample/mysql-flush.sh
> @@ -0,0 +1,55 @@
> +#!/bin/sh
> +
> +# Flush MySQL tables to the disk before the filesystem is freezed.
> +# At the same time, this keeps a read lock while the filesystem is freezed
> +# in order to avoid write accesses by the other clients.
> +
> +MYSQL="/usr/bin/mysql"
> +MYSQL_OPTS="-uroot" #"-prootpassword"
> +FIFO=/tmp/mysql-flush.fifo
> +MYSQL_CMD="$MYSQL $MYSQL_OPTS"
> +
> +# Check mysql is installed and the server running
> +[ -x $MYSQL ] && $MYSQL_CMD < /dev/null || exit 0
> +
> +flush_and_wait() {
> +    printf "FLUSH TABLES WITH READ LOCK \\G\n"
> +    read < $FIFO
> +    printf "UNLOCK TABLES \\G\n"
> +}
> +
> +case "$1" in
> +    freeze)
> +        mkfifo $FIFO || exit 1
> +        flush_and_wait | $MYSQL_CMD &
> +        # wait until every block is flushed
> +        while [ "$(echo 'SHOW STATUS LIKE "Key_blocks_not_flushed"' |\
> +                 $MYSQL_CMD | tail -1 | cut -f 2)" -gt 0 ]; do
> +            sleep 1
> +        done
> +        # for InnoDB, wait until every log is flushed
> +        INNODB_STATUS=$(mktemp /tmp/mysql-flush.XXXXXX)
> +        [ $? -ne 0 ] && exit 2
> +        trap "rm -f $INNODB_STATUS" SIGINT
> +        while :; do
> +            printf "SHOW ENGINE INNODB STATUS \\G" | $MYSQL_CMD > $INNODB_STATUS
> +            LOG_CURRENT=$(grep 'Log sequence number' $INNODB_STATUS |\
> +                          tr -s ' ' | cut -d' ' -f4)
> +            LOG_FLUSHED=$(grep 'Log flushed up to' $INNODB_STATUS |\
> +                          tr -s ' ' | cut -d' ' -f5)
> +            [ "$LOG_CURRENT" = "$LOG_FLUSHED" ] && break
> +            sleep 1
> +        done
> +        rm -f $INNODB_STATUS
> +        ;;
> +
> +    thaw)
> +        [ ! -p $FIFO ] && exit 1
> +        echo > $FIFO
> +        rm -f $FIFO
> +        ;;
> +
> +    *)
> +        exit 1
> +        ;;
> +esac
> 

Re: [Qemu-devel] [PATCH v6 2/2] qemu-ga: sample fsfreeze hooks

[Tomoki Sekiyama]

On 2012/12/05 1:51, Luiz Capitulino wrote:
> On Fri, 30 Nov 2012 22:33:57 +0900
> Tomoki Sekiyama <tomoki.sekiyama.qu@hitachi.com> wrote:
> 
>> Adds sample hook scripts for --fsfreeze-hook option of qemu-ga.
>>   - fsfreeze-hook : execute scripts in fsfreeze-hook.d/
>>   - fsfreeze-hook.d.sample/mysql-flush.sh : quiesce MySQL before snapshot
>>
>> Signed-off-by: Tomoki Sekiyama <tomoki.sekiyama.qu@hitachi.com>
>> ---
>>  docs/qemu-guest-agent/fsfreeze-hook                |   33 ++++++++++++
>>  .../fsfreeze-hook.d.sample/mysql-flush.sh          |   55 ++++++++++++++++++++
>>  2 files changed, 88 insertions(+)
>>  create mode 100755 docs/qemu-guest-agent/fsfreeze-hook
>>  create mode 100755 docs/qemu-guest-agent/fsfreeze-hook.d.sample/mysql-flush.sh
>>
>> diff --git a/docs/qemu-guest-agent/fsfreeze-hook b/docs/qemu-guest-agent/fsfreeze-hook
>> new file mode 100755
>> index 0000000..ed7d86d
>> --- /dev/null
>> +++ b/docs/qemu-guest-agent/fsfreeze-hook
>> @@ -0,0 +1,33 @@
>> +#!/bin/sh
>> +
>> +# This script is executed when a guest agent receives fsfreeze-freeze and
>> +# fsfreeze-thaw command, if it is specified in --fsfreeze-hook (-F)
>> +# option of qemu-ga or placed in default path (/etc/qemu/fsfreeze-hook).
>> +# When the agent receives fsfreeze-freeze request, this script is issued with
>> +# "freeze" argument before the filesystem is freezed. And for fsfreeze-thaw
>> +# request, it is issued with "thaw" argument after filesystem is thawed.
>> +
>> +LOGFILE=/var/log/qga-fsfreeze-hook.log
>> +FSFREEZE_D=$(dirname -- "$0")/fsfreeze-hook.d
> 
> The script you introduce is called fsfreeze-hook.d.sample. If it's expected
> that people really use these two scripts (vs. doc purposes only) I suggest
> the following:
> 
>  1. Move these files to scripts/
>  2. Drop the .sample suffix

It is good for "fsfreeze-hook". But as for "mysql-flush.sh", users may
need to configure authentication etc. if their environments are with
non-default setup (by edditing either the script or ~/.my.cnf file).

So, I'd like to (1)move these files to scripts/, but (2)reamin .sample
suffix to indicate that users may need to modify the script.
How do you think?

Thanks,
-- 
Tomoki Sekiyama <tomoki.sekiyama.qu@hitachi.com>
Linux Technology Center
Hitachi, Ltd., Yokohama Research Laboratory

Re: [Qemu-devel] [PATCH v6 2/2] qemu-ga: sample fsfreeze hooks

[Luiz Capitulino]

On Wed, 05 Dec 2012 17:41:08 +0900
Tomoki Sekiyama <tomoki.sekiyama.qu@hitachi.com> wrote:

> On 2012/12/05 1:51, Luiz Capitulino wrote:
> > On Fri, 30 Nov 2012 22:33:57 +0900
> > Tomoki Sekiyama <tomoki.sekiyama.qu@hitachi.com> wrote:
> > 
> >> Adds sample hook scripts for --fsfreeze-hook option of qemu-ga.
> >>   - fsfreeze-hook : execute scripts in fsfreeze-hook.d/
> >>   - fsfreeze-hook.d.sample/mysql-flush.sh : quiesce MySQL before snapshot
> >>
> >> Signed-off-by: Tomoki Sekiyama <tomoki.sekiyama.qu@hitachi.com>
> >> ---
> >>  docs/qemu-guest-agent/fsfreeze-hook                |   33 ++++++++++++
> >>  .../fsfreeze-hook.d.sample/mysql-flush.sh          |   55 ++++++++++++++++++++
> >>  2 files changed, 88 insertions(+)
> >>  create mode 100755 docs/qemu-guest-agent/fsfreeze-hook
> >>  create mode 100755 docs/qemu-guest-agent/fsfreeze-hook.d.sample/mysql-flush.sh
> >>
> >> diff --git a/docs/qemu-guest-agent/fsfreeze-hook b/docs/qemu-guest-agent/fsfreeze-hook
> >> new file mode 100755
> >> index 0000000..ed7d86d
> >> --- /dev/null
> >> +++ b/docs/qemu-guest-agent/fsfreeze-hook
> >> @@ -0,0 +1,33 @@
> >> +#!/bin/sh
> >> +
> >> +# This script is executed when a guest agent receives fsfreeze-freeze and
> >> +# fsfreeze-thaw command, if it is specified in --fsfreeze-hook (-F)
> >> +# option of qemu-ga or placed in default path (/etc/qemu/fsfreeze-hook).
> >> +# When the agent receives fsfreeze-freeze request, this script is issued with
> >> +# "freeze" argument before the filesystem is freezed. And for fsfreeze-thaw
> >> +# request, it is issued with "thaw" argument after filesystem is thawed.
> >> +
> >> +LOGFILE=/var/log/qga-fsfreeze-hook.log
> >> +FSFREEZE_D=$(dirname -- "$0")/fsfreeze-hook.d
> > 
> > The script you introduce is called fsfreeze-hook.d.sample. If it's expected
> > that people really use these two scripts (vs. doc purposes only) I suggest
> > the following:
> > 
> >  1. Move these files to scripts/
> >  2. Drop the .sample suffix
> 
> It is good for "fsfreeze-hook". But as for "mysql-flush.sh", users may
> need to configure authentication etc. if their environments are with
> non-default setup (by edditing either the script or ~/.my.cnf file).
> 
> So, I'd like to (1)move these files to scripts/, but (2)reamin .sample
> suffix to indicate that users may need to modify the script.
> How do you think?

You could add the '.sample' suffix to the mysql script and add that
suffix to is_ignored_file().

Re: [Qemu-devel] [PATCH v6 2/2] qemu-ga: sample fsfreeze hooks

[Tomoki Sekiyama]

On 2012/12/05 20:49, Luiz Capitulino wrote:
> On Wed, 05 Dec 2012 17:41:08 +0900
> Tomoki Sekiyama <tomoki.sekiyama.qu@hitachi.com> wrote:
> 
>> On 2012/12/05 1:51, Luiz Capitulino wrote:
>>> On Fri, 30 Nov 2012 22:33:57 +0900
>>> Tomoki Sekiyama <tomoki.sekiyama.qu@hitachi.com> wrote:
>>>
>>>> Adds sample hook scripts for --fsfreeze-hook option of qemu-ga.
>>>>   - fsfreeze-hook : execute scripts in fsfreeze-hook.d/
>>>>   - fsfreeze-hook.d.sample/mysql-flush.sh : quiesce MySQL before snapshot
>>>>
>>>> Signed-off-by: Tomoki Sekiyama <tomoki.sekiyama.qu@hitachi.com>
>>>> ---
>>>>  docs/qemu-guest-agent/fsfreeze-hook                |   33 ++++++++++++
>>>>  .../fsfreeze-hook.d.sample/mysql-flush.sh          |   55 ++++++++++++++++++++
>>>>  2 files changed, 88 insertions(+)
>>>>  create mode 100755 docs/qemu-guest-agent/fsfreeze-hook
>>>>  create mode 100755 docs/qemu-guest-agent/fsfreeze-hook.d.sample/mysql-flush.sh
>>>>
>>>> diff --git a/docs/qemu-guest-agent/fsfreeze-hook b/docs/qemu-guest-agent/fsfreeze-hook
>>>> new file mode 100755
>>>> index 0000000..ed7d86d
>>>> --- /dev/null
>>>> +++ b/docs/qemu-guest-agent/fsfreeze-hook
>>>> @@ -0,0 +1,33 @@
>>>> +#!/bin/sh
>>>> +
>>>> +# This script is executed when a guest agent receives fsfreeze-freeze and
>>>> +# fsfreeze-thaw command, if it is specified in --fsfreeze-hook (-F)
>>>> +# option of qemu-ga or placed in default path (/etc/qemu/fsfreeze-hook).
>>>> +# When the agent receives fsfreeze-freeze request, this script is issued with
>>>> +# "freeze" argument before the filesystem is freezed. And for fsfreeze-thaw
>>>> +# request, it is issued with "thaw" argument after filesystem is thawed.
>>>> +
>>>> +LOGFILE=/var/log/qga-fsfreeze-hook.log
>>>> +FSFREEZE_D=$(dirname -- "$0")/fsfreeze-hook.d
>>>
>>> The script you introduce is called fsfreeze-hook.d.sample. If it's expected
>>> that people really use these two scripts (vs. doc purposes only) I suggest
>>> the following:
>>>
>>>  1. Move these files to scripts/
>>>  2. Drop the .sample suffix
>>
>> It is good for "fsfreeze-hook". But as for "mysql-flush.sh", users may
>> need to configure authentication etc. if their environments are with
>> non-default setup (by edditing either the script or ~/.my.cnf file).
>>
>> So, I'd like to (1)move these files to scripts/, but (2)reamin .sample
>> suffix to indicate that users may need to modify the script.
>> How do you think?
> 
> You could add the '.sample' suffix to the mysql script and add that
> suffix to is_ignored_file().
> 

Okey, I will try this in the next version.
(It seems Makefile also needs a patch to treat directry named "*.d".)

Thanks,
-- 
Tomoki Sekiyama <tomoki.sekiyama.qu@hitachi.com>
Linux Technology Center
Hitachi, Ltd., Yokohama Research Laboratory

Re: [Qemu-devel] [PATCH v6 0/2] qemu-ga: add hook to quiesce the guest on fsfreeze-freeze/thaw

[Luiz Capitulino]

On Fri, 30 Nov 2012 22:33:49 +0900
Tomoki Sekiyama <tomoki.sekiyama.qu@hitachi.com> wrote:

> Hi,
> 
> This is version 6 of the qemu-ga fsfreeze hook patchset.

Made two small comments, otherwise series looks good to me now.