[Qemu-devel] [PATCH v5 0/3] qemu-ga: add hook to quiesce the guest on fsfreeze-freeze/thaw

[Qemu-devel] [PATCH v5 0/3] qemu-ga: add hook to quiesce the guest on fsfreeze-freeze/thaw

[Tomoki Sekiyama]

Hi,

This is version 5 of the qemu-ga fsfreeze hook patchset.

*Changes from v4: ( http://patchwork.ozlabs.org/patch/200924/ )
 1/3: Improve error messages when fsfreeze hook is not executable.
      Make fsfreeze-hook disabled by default, and enabled only if
      --fsfreeze-hook option is specified.
 2/3: Add exit 0 at the last of the sample script.
      Add check for presence of mysql in mysql-flush.sh.
 3/3: For 1.4: Improve error message based on Luiz Capitulino's patchset.

Patch 3/3 depends on Luiz Capitulino's patchset to improve error handling,
which is available at
https://lists.gnu.org/archive/html/qemu-devel/2012-11/msg03016.html.
The other patches are applicable without them (for 1.3).

---

Tomoki Sekiyama (3):
      qemu-ga: execute hook to quiesce the guest on fsfreeze-freeze/thaw
      qemu-ga: sample fsfreeze hooks
      [for 1.4] qemu-ga: Improve error reporting in execute_fsfreeze_hook()



Thanks,
-- 
Tomoki Sekiyama <tomoki.sekiyama.qu@hitachi.com>
Linux Technology Center
Hitachi, Ltd., Yokohama Research Laboratory

Re: [Qemu-devel] [PATCH v5 1/3] qemu-ga: execute hook to quiesce the guest on fsfreeze-freeze/thaw

[Tomoki Sekiyama]

To use the online disk snapshot for online-backup, application-level
consistency of the snapshot image is required. However, currently the
guest agent can provide only filesystem-level consistency, and the
snapshot may contain dirty data, for example, incomplete transactions.
This patch provides the opportunity to quiesce applications before
snapshot is taken.

If --fsfreeze-hook option is specified, the hook is executed with
"freeze" argument before the filesystem is frozen by fsfreeze-freeze
command. As for fsfreeze-thaw command, the hook is executed with "thaw"
argument after the filesystem is thawed.

Signed-off-by: Tomoki Sekiyama <tomoki.sekiyama.qu@hitachi.com>
---
 qemu-ga.c              |   42 ++++++++++++++++++++++++++-
 qga/commands-posix.c   |   76 ++++++++++++++++++++++++++++++++++++++++++++++++
 qga/guest-agent-core.h |    1 +
 3 files changed, 118 insertions(+), 1 deletion(-)

diff --git a/qemu-ga.c b/qemu-ga.c
index 9b59a52..53ce462 100644
--- a/qemu-ga.c
+++ b/qemu-ga.c
@@ -34,6 +34,12 @@
 #include "qga/service-win32.h"
 #include <windows.h>
 #endif
+#ifdef __linux__
+#include <linux/fs.h>
+#ifdef FIFREEZE
+#define CONFIG_FSFREEZE
+#endif
+#endif
 
 #ifndef _WIN32
 #define QGA_VIRTIO_PATH_DEFAULT "/dev/virtio-ports/org.qemu.guest_agent.0"
@@ -42,6 +48,9 @@
 #endif
 #define QGA_STATEDIR_DEFAULT CONFIG_QEMU_LOCALSTATEDIR "/run"
 #define QGA_PIDFILE_DEFAULT QGA_STATEDIR_DEFAULT "/qemu-ga.pid"
+#ifdef CONFIG_FSFREEZE
+#define QGA_FSFREEZE_HOOK_DEFAULT CONFIG_QEMU_CONFDIR "/fsfreeze-hook"
+#endif
 #define QGA_SENTINEL_BYTE 0xFF
 
 struct GAState {
@@ -64,6 +73,9 @@ struct GAState {
         const char *log_filepath;
         const char *pid_filepath;
     } deferred_options;
+#ifdef CONFIG_FSFREEZE
+    const char *fsfreeze_hook;
+#endif
 };
 
 struct GAState *ga_state;
@@ -153,6 +165,10 @@ static void usage(const char *cmd)
 "                    %s)\n"
 "  -l, --logfile     set logfile path, logs to stderr by default\n"
 "  -f, --pidfile     specify pidfile (default is %s)\n"
+#ifdef CONFIG_FSFREEZE
+"  -F, --fsfreeze-hook\n"
+"                    enable fsfreeze hook (default is %s)\n"
+#endif
 "  -t, --statedir    specify dir to store state information (absolute paths\n"
 "                    only, default is %s)\n"
 "  -v, --verbose     log extra debugging information\n"
@@ -167,6 +183,9 @@ static void usage(const char *cmd)
 "\n"
 "Report bugs to <mdroth@linux.vnet.ibm.com>\n"
     , cmd, QEMU_VERSION, QGA_VIRTIO_PATH_DEFAULT, QGA_PIDFILE_DEFAULT,
+#ifdef CONFIG_FSFREEZE
+    QGA_FSFREEZE_HOOK_DEFAULT,
+#endif
     QGA_STATEDIR_DEFAULT);
 }
 
@@ -401,6 +420,13 @@ void ga_unset_frozen(GAState *s)
     }
 }
 
+#ifdef CONFIG_FSFREEZE
+const char *ga_fsfreeze_hook(GAState *s)
+{
+    return s->fsfreeze_hook;
+}
+#endif
+
 static void become_daemon(const char *pidfile)
 {
 #ifndef _WIN32
@@ -678,10 +704,13 @@ VOID WINAPI service_main(DWORD argc, TCHAR *argv[])
 
 int main(int argc, char **argv)
 {
-    const char *sopt = "hVvdm:p:l:f:b:s:t:";
+    const char *sopt = "hVvdm:p:l:f:F::b:s:t:";
     const char *method = NULL, *path = NULL;
     const char *log_filepath = NULL;
     const char *pid_filepath = QGA_PIDFILE_DEFAULT;
+#ifdef CONFIG_FSFREEZE
+    const char *fsfreeze_hook = NULL;
+#endif
     const char *state_dir = QGA_STATEDIR_DEFAULT;
 #ifdef _WIN32
     const char *service = NULL;
@@ -691,6 +720,9 @@ int main(int argc, char **argv)
         { "version", 0, NULL, 'V' },
         { "logfile", 1, NULL, 'l' },
         { "pidfile", 1, NULL, 'f' },
+#ifdef CONFIG_FSFREEZE
+        { "fsfreeze-hook", 2, NULL, 'F' },
+#endif
         { "verbose", 0, NULL, 'v' },
         { "method", 1, NULL, 'm' },
         { "path", 1, NULL, 'p' },
@@ -723,6 +755,11 @@ int main(int argc, char **argv)
         case 'f':
             pid_filepath = optarg;
             break;
+#ifdef CONFIG_FSFREEZE
+        case 'F':
+            fsfreeze_hook = optarg ? optarg : QGA_FSFREEZE_HOOK_DEFAULT;
+            break;
+#endif
         case 't':
              state_dir = optarg;
              break;
@@ -786,6 +823,9 @@ int main(int argc, char **argv)
     s = g_malloc0(sizeof(GAState));
     s->log_level = log_level;
     s->log_file = stderr;
+#ifdef CONFIG_FSFREEZE
+    s->fsfreeze_hook = fsfreeze_hook;
+#endif
     g_log_set_default_handler(ga_log, s);
     g_log_set_fatal_mask(NULL, G_LOG_LEVEL_ERROR);
     ga_enable_logging(s);
diff --git a/qga/commands-posix.c b/qga/commands-posix.c
index 726930a..d33fe9a 100644
--- a/qga/commands-posix.c
+++ b/qga/commands-posix.c
@@ -384,6 +384,72 @@ static int build_fs_mount_list(FsMountList *mounts)
 
 #if defined(CONFIG_FSFREEZE)
 
+typedef enum {
+    FSFREEZE_HOOK_THAW = 0,
+    FSFREEZE_HOOK_FREEZE,
+} FsfreezeHookArg;
+
+const char *fsfreeze_hook_arg_string[] = {
+    "thaw",
+    "freeze",
+};
+
+static void execute_fsfreeze_hook(FsfreezeHookArg arg, Error **err)
+{
+    int status;
+    pid_t pid, rpid;
+    const char *hook;
+    const char *arg_str = fsfreeze_hook_arg_string[arg];
+    char err_msg[512];
+
+    hook = ga_fsfreeze_hook(ga_state);
+    if (!hook) {
+        return;
+    }
+    if (access(hook, X_OK) != 0) {
+        error_set_errno(err, errno, QERR_QGA_COMMAND_FAILED,
+                        "fsfreeze hook is not executable");
+        return;
+    }
+
+    slog("executing fsfreeze hook with arg `%s'", arg_str);
+    pid = fork();
+    if (pid == 0) {
+        setsid();
+        reopen_fd_to_null(0);
+        reopen_fd_to_null(1);
+        reopen_fd_to_null(2);
+
+        execle(hook, hook, arg_str, NULL, environ);
+        _exit(EXIT_FAILURE);
+    } else if (pid < 0) {
+        error_setg_errno(err, errno, "failed to create child process");
+        return;
+    }
+
+    do {
+        rpid = waitpid(pid, &status, 0);
+    } while (rpid == -1 && errno == EINTR);
+    if (rpid < 0) {
+        sprintf(err_msg, "failed to wait for fsfreeze hook(pid: %d)", pid);
+        error_set_errno(err, errno, QERR_QGA_COMMAND_FAILED, err_msg);
+        return;
+    }
+
+    if (!WIFEXITED(status)) {
+        error_set(err, QERR_QGA_COMMAND_FAILED,
+                  "fsfreeze hook has terminated abnormally");
+        return;
+    }
+
+    status = WEXITSTATUS(status);
+    if (status) {
+        sprintf(err_msg, "fsfreeze hook has failed with signal %d", status);
+        error_set(err, QERR_QGA_COMMAND_FAILED, err_msg);
+        return;
+    }
+}
+
 /*
  * Return status of freeze/thaw
  */
@@ -405,11 +471,18 @@ int64_t qmp_guest_fsfreeze_freeze(Error **err)
     int ret = 0, i = 0;
     FsMountList mounts;
     struct FsMount *mount;
+    Error *local_err = NULL;
     int fd;
     char err_msg[512];
 
     slog("guest-fsfreeze called");
 
+    execute_fsfreeze_hook(FSFREEZE_HOOK_FREEZE, &local_err);
+    if (error_is_set(&local_err)) {
+        error_propagate(err, local_err);
+        return -1;
+    }
+
     QTAILQ_INIT(&mounts);
     ret = build_fs_mount_list(&mounts);
     if (ret < 0) {
@@ -513,6 +586,9 @@ int64_t qmp_guest_fsfreeze_thaw(Error **err)
 
     ga_unset_frozen(ga_state);
     free_fs_mount_list(&mounts);
+
+    execute_fsfreeze_hook(FSFREEZE_HOOK_THAW, err);
+
     return i;
 }
 
diff --git a/qga/guest-agent-core.h b/qga/guest-agent-core.h
index 49a7abe..c6e8de0 100644
--- a/qga/guest-agent-core.h
+++ b/qga/guest-agent-core.h
@@ -34,6 +34,7 @@ void ga_set_response_delimited(GAState *s);
 bool ga_is_frozen(GAState *s);
 void ga_set_frozen(GAState *s);
 void ga_unset_frozen(GAState *s);
+const char *ga_fsfreeze_hook(GAState *s);
 
 #ifndef _WIN32
 void reopen_fd_to_null(int fd);

Re: [Qemu-devel] [PATCH v5 2/3] qemu-ga: sample fsfreeze hooks

[Tomoki Sekiyama]

Adds sample hook scripts for --fsfreeze-hook option of qemu-ga.
  - fsfreeze-hook : execute scripts in fsfreeze-hook.d/
  - fsfreeze-hook.d.sample/mysql-flush.sh : quiesce MySQL before snapshot

Signed-off-by: Tomoki Sekiyama <tomoki.sekiyama.qu@hitachi.com>
---
 docs/qemu-guest-agent/fsfreeze-hook                |   33 ++++++++++++
 .../fsfreeze-hook.d.sample/mysql-flush.sh          |   55 ++++++++++++++++++++
 2 files changed, 88 insertions(+)
 create mode 100755 docs/qemu-guest-agent/fsfreeze-hook
 create mode 100755 docs/qemu-guest-agent/fsfreeze-hook.d.sample/mysql-flush.sh

diff --git a/docs/qemu-guest-agent/fsfreeze-hook b/docs/qemu-guest-agent/fsfreeze-hook
new file mode 100755
index 0000000..ed7d86d
--- /dev/null
+++ b/docs/qemu-guest-agent/fsfreeze-hook
@@ -0,0 +1,33 @@
+#!/bin/sh
+
+# This script is executed when a guest agent receives fsfreeze-freeze and
+# fsfreeze-thaw command, if it is specified in --fsfreeze-hook (-F)
+# option of qemu-ga or placed in default path (/etc/qemu/fsfreeze-hook).
+# When the agent receives fsfreeze-freeze request, this script is issued with
+# "freeze" argument before the filesystem is freezed. And for fsfreeze-thaw
+# request, it is issued with "thaw" argument after filesystem is thawed.
+
+LOGFILE=/var/log/qga-fsfreeze-hook.log
+FSFREEZE_D=$(dirname -- "$0")/fsfreeze-hook.d
+
+# Check whether file $1 is a backup or rpm-generated file and should be ignored
+is_ignored_file() {
+    case "$1" in
+        *~ | *.bak | *.orig | *.rpmnew | *.rpmorig | *.rpmsave)
+            return 0 ;;
+    esac
+    return 1
+}
+
+# Iterate executables in directory "fsfreeze-hook.d" with the specified args
+[ ! -d "$FSFREEZE_D" ] && exit 1
+for file in "$FSFREEZE_D"/* ; do
+    is_ignored_file "$file" && continue
+    [ -x "$file" ] || continue
+    echo "$(date): execute $file $@" >>$LOGFILE
+    "$file" "$@" >>$LOGFILE 2>&1
+    STATUS=$?
+    echo "$(date): $file finished with status=$STATUS" >>$LOGFILE
+done
+
+exit 0
diff --git a/docs/qemu-guest-agent/fsfreeze-hook.d.sample/mysql-flush.sh b/docs/qemu-guest-agent/fsfreeze-hook.d.sample/mysql-flush.sh
new file mode 100755
index 0000000..611215e
--- /dev/null
+++ b/docs/qemu-guest-agent/fsfreeze-hook.d.sample/mysql-flush.sh
@@ -0,0 +1,55 @@
+#!/bin/sh
+
+# Flush MySQL tables to the disk before the filesystem is freezed.
+# At the same time, this keeps a read lock while the filesystem is freezed
+# in order to avoid write accesses by the other clients.
+
+MYSQL="/usr/bin/mysql"
+MYSQL_OPTS="-uroot" #"-prootpassword"
+FIFO=/tmp/mysql-flush.fifo
+MYSQL_CMD="$MYSQL $MYSQL_OPTS"
+
+# Check mysql is installed and the server running
+[ -x $MYSQL ] && $MYSQL_CMD < /dev/null || exit 0
+
+flush_and_wait() {
+    printf "FLUSH TABLES WITH READ LOCK \\G\n"
+    read < $FIFO
+    printf "UNLOCK TABLES \\G\n"
+}
+
+case "$1" in
+    freeze)
+        mkfifo $FIFO || exit 1
+        flush_and_wait | $MYSQL_CMD &
+        # wait until every block is flushed
+        while [ "$(echo 'SHOW STATUS LIKE "Key_blocks_not_flushed"' |\
+                 $MYSQL_CMD | tail -1 | cut -f 2)" -gt 0 ]; do
+            sleep 1
+        done
+        # for InnoDB, wait until every log is flushed
+        INNODB_STATUS=$(mktemp /tmp/mysql-flush.XXXXXX)
+        [ $? -ne 0 ] && exit 2
+        trap "rm -f $INNODB_STATUS" SIGINT
+        while :; do
+            printf "SHOW ENGINE INNODB STATUS \\G" | $MYSQL_CMD > $INNODB_STATUS
+            LOG_CURRENT=$(grep 'Log sequence number' $INNODB_STATUS |\
+                          tr -s ' ' | cut -d' ' -f4)
+            LOG_FLUSHED=$(grep 'Log flushed up to' $INNODB_STATUS |\
+                          tr -s ' ' | cut -d' ' -f5)
+            [ "$LOG_CURRENT" = "$LOG_FLUSHED" ] && break
+            sleep 1
+        done
+        rm -f $INNODB_STATUS
+        ;;
+
+    thaw)
+        [ ! -p $FIFO ] && exit 1
+        echo > $FIFO
+        rm -f $FIFO
+        ;;
+
+    *)
+        exit 1
+        ;;
+esac

Re: [Qemu-devel] [PATCH v5 3/3] [for 1.4] qemu-ga: Improve error reporting in execute_fsfreeze_hook()

[Tomoki Sekiyama]

Improve error reporting in execute_fsfreeze_hook() using
ga_wait_child().

This patch depends on patchset to improve error reporting by Luiz
Capitulino, especially on http://patchwork.ozlabs.org/patch/202208/ .

Signed-off-by: Tomoki Sekiyama <tomoki.sekiyama.qu@hitachi.com>
---
 qga/commands-posix.c |   23 +++++++++--------------
 1 file changed, 9 insertions(+), 14 deletions(-)

diff --git a/qga/commands-posix.c b/qga/commands-posix.c
index bf22a58..dbfcbdf 100644
--- a/qga/commands-posix.c
+++ b/qga/commands-posix.c
@@ -423,18 +423,18 @@ const char *fsfreeze_hook_arg_string[] = {
 static void execute_fsfreeze_hook(FsfreezeHookArg arg, Error **err)
 {
     int status;
-    pid_t pid, rpid;
+    pid_t pid;
     const char *hook;
     const char *arg_str = fsfreeze_hook_arg_string[arg];
-    char err_msg[512];
+    Error *local_err = NULL;
 
     hook = ga_fsfreeze_hook(ga_state);
     if (!hook) {
         return;
     }
     if (access(hook, X_OK) != 0) {
-        error_set_errno(err, errno, QERR_QGA_COMMAND_FAILED,
-                        "fsfreeze hook is not executable");
+        error_setg_errno(err, errno,
+                         "fsfreeze hook '%s' is not executable", hook);
         return;
     }
 
@@ -453,25 +453,20 @@ static void execute_fsfreeze_hook(FsfreezeHookArg arg, Error **err)
         return;
     }
 
-    do {
-        rpid = waitpid(pid, &status, 0);
-    } while (rpid == -1 && errno == EINTR);
-    if (rpid < 0) {
-        sprintf(err_msg, "failed to wait for fsfreeze hook(pid: %d)", pid);
-        error_set_errno(err, errno, QERR_QGA_COMMAND_FAILED, err_msg);
+    ga_wait_child(pid, &status, &local_err);
+    if (error_is_set(&local_err)) {
+        error_propagate(err, local_err);
         return;
     }
 
     if (!WIFEXITED(status)) {
-        error_set(err, QERR_QGA_COMMAND_FAILED,
-                  "fsfreeze hook has terminated abnormally");
+        error_setg(err, "fsfreeze hook has terminated abnormally");
         return;
     }
 
     status = WEXITSTATUS(status);
     if (status) {
-        sprintf(err_msg, "fsfreeze hook has failed with signal %d", status);
-        error_set(err, QERR_QGA_COMMAND_FAILED, err_msg);
+        error_setg(err, "fsfreeze hook has failed with status %d", status);
         return;
     }
 }

Re: [Qemu-devel] [PATCH v5 0/3] qemu-ga: add hook to quiesce the guest on fsfreeze-freeze/thaw

[Luiz Capitulino]

On Fri, 30 Nov 2012 15:25:57 +0900
Tomoki Sekiyama <tomoki.sekiyama.qu@hitachi.com> wrote:

> Hi,
> 
> This is version 5 of the qemu-ga fsfreeze hook patchset.

This series split is not good. You should squash patches 3/3 and 1/3 and
rebase this series on top of:

 http://lists.gnu.org/archive/html/qemu-devel/2012-11/msg03016.html

Also, please look at qmp_shutdown() and bios_supports_mode() for examples
of command execution with improved error reporting.

> *Changes from v4: ( http://patchwork.ozlabs.org/patch/200924/ )
>  1/3: Improve error messages when fsfreeze hook is not executable.
>       Make fsfreeze-hook disabled by default, and enabled only if
>       --fsfreeze-hook option is specified.

Let me clarify that there are two ways of doing this. One is the way above,
the other way is to enable it by default but allow something like
"--fsfreeze-hook no" to disable the hook execution.

Both ways are fine with me.

Re: [Qemu-devel] [PATCH v5 0/3] qemu-ga: add hook to quiesce the guest on fsfreeze-freeze/thaw

[Tomoki Sekiyama]

On 2012/11/30 20:42, Luiz Capitulino wrote:
> On Fri, 30 Nov 2012 15:25:57 +0900
> Tomoki Sekiyama <tomoki.sekiyama.qu@hitachi.com> wrote:
> 
>> Hi,
>>
>> This is version 5 of the qemu-ga fsfreeze hook patchset.
> 
> This series split is not good. You should squash patches 3/3 and 1/3 and
> rebase this series on top of:
> 
>  http://lists.gnu.org/archive/html/qemu-devel/2012-11/msg03016.html

I see, I will resend a patchset squashed and rebased on this.
BTW, is there a git branch for 1.4 anywhere?

> Also, please look at qmp_shutdown() and bios_supports_mode() for examples
> of command execution with improved error reporting.
> 
>> *Changes from v4: ( http://patchwork.ozlabs.org/patch/200924/ )
>>  1/3: Improve error messages when fsfreeze hook is not executable.
>>       Make fsfreeze-hook disabled by default, and enabled only if
>>       --fsfreeze-hook option is specified.
> 
> Let me clarify that there are two ways of doing this. One is the way above,
> the other way is to enable it by default but allow something like
> "--fsfreeze-hook no" to disable the hook execution.

I chose the way to disable it by default because it's more precise.
If we enable the hook by default, we need to guess whether the hook is
executed by its presence, and then the hook placed on wrong path will
not cause an error.

> Both ways are fine with me.
> 

Thanks,
-- 
Tomoki Sekiyama <tomoki.sekiyama.qu@hitachi.com>
Linux Technology Center
Hitachi, Ltd., Yokohama Research Laboratory

Re: [Qemu-devel] [PATCH v5 0/3] qemu-ga: add hook to quiesce the guest on fsfreeze-freeze/thaw

[Luiz Capitulino]

On Fri, 30 Nov 2012 21:40:04 +0900
Tomoki Sekiyama <tomoki.sekiyama.qu@hitachi.com> wrote:

> On 2012/11/30 20:42, Luiz Capitulino wrote:
> > On Fri, 30 Nov 2012 15:25:57 +0900
> > Tomoki Sekiyama <tomoki.sekiyama.qu@hitachi.com> wrote:
> > 
> >> Hi,
> >>
> >> This is version 5 of the qemu-ga fsfreeze hook patchset.
> > 
> > This series split is not good. You should squash patches 3/3 and 1/3 and
> > rebase this series on top of:
> > 
> >  http://lists.gnu.org/archive/html/qemu-devel/2012-11/msg03016.html
> 
> I see, I will resend a patchset squashed and rebased on this.
> BTW, is there a git branch for 1.4 anywhere?

You mean, a development branch? As master is in hard-freeze, each maintainer
is responsible for queueing patches for 1.4.