[PATCH v1 0/2] Implement -run-with exit-with-parent=on

[PATCH v1 0/2] Implement -run-with exit-with-parent=on

[Richard W.M. Jones]

An earlier WIP version was posted here:
https://lists.gnu.org/archive/html/qemu-devel/2025-10/msg00309.html

In this version I have incorporated all of Daniel Berrange's feedback,
and I also tested the change on macOS.

The new second patch modifies the test suite to use the new flag
unconditionally, instead of open coding setting PR_SET_PDEATHSIG etc.
The test suite fails for me both before and after this patch in the
same way, so I assume this is nothing to do with the patch itself?

  4/405 qemu:func-quick+func-x86_64 / func-x86_64-bad_vmstate               ERROR            0.15s   exit status 1

Some points of note:

 - There's no way to find out if the qemu binary supports
   exit-with-parent=on except to try it.  Maybe this should be exposed
   somehow?

 - On macOS I wasn't able to find a satisfactory way to force
   shutdown, except calling 'qemu_system_killed' and pretending we'd
   been killed by SIGTERM (which does at least emulate what Linux &
   FreeBSD do).  I suppose it'd be nice if there was a "killed by
   parent" reason which also forced shutdown.

Rich.

[PATCH v1 1/2] Implement -run-with exit-with-parent=on

[Richard W.M. Jones]

Libguestfs wants to use qemu to run a captive appliance.  When the
program linked to libguestfs exits, we want qemu to be cleaned up.
Libguestfs goes to great lengths to do this at the moment: it either
forks a separate process to ensure clean-up is done, or it asks
libvirt to clean up the qemu process.  However this is complicated and
not totally reliable.

On Linux, FreeBSD and macOS, there are mechanisms to ensure a signal
or message is delivered to a process when its parent process goes
away.  The qemu test suite even uses this mechanism on Linux (see
PR_SET_PDEATHSIG in tests/qtest/libqtest.c).

In nbdkit we have long had the concept of running nbdkit captively,
and we have the nbdkit --exit-with-parent flag to help
(https://libguestfs.org/nbdkit-captive.1.html#EXIT-WITH-PARENT)

This commit adds the same mechanism.  The syntax is:

  qemu -run-with exit-with-parent=on [...]

This is not a feature that most typical users of qemu (for running
general purpose, long-lived VMs) should use, so it defaults to off.

The exit-with-parent.[ch] files are copied from nbdkit, where they
have a 3-clause BSD license which is compatible with qemu:

https://gitlab.com/nbdkit/nbdkit/-/tree/master/common/utils?ref_type=heads

Thanks: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Richard W.M. Jones <rjones@redhat.com>
---
 include/qemu/exit-with-parent.h |  47 +++++++++
 qemu-options.hx                 |  13 ++-
 system/exit-with-parent.c       | 169 ++++++++++++++++++++++++++++++++
 system/meson.build              |   1 +
 system/vl.c                     |  13 +++
 5 files changed, 241 insertions(+), 2 deletions(-)

diff --git a/include/qemu/exit-with-parent.h b/include/qemu/exit-with-parent.h
new file mode 100644
index 0000000000..141f68d53b
--- /dev/null
+++ b/include/qemu/exit-with-parent.h
@@ -0,0 +1,47 @@
+/*
+ * SPDX-License-Identifier: BSD-3-Clause
+ * Originally derived from nbdkit common/utils/exit-with-parent.h
+ * Copyright Red Hat
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are
+ * met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ *
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * * Neither the name of Red Hat nor the names of its contributors may be
+ * used to endorse or promote products derived from this software without
+ * specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
+ * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
+ * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+ * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+ * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+ * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+#ifndef NBDKIT_EXIT_WITH_PARENT_H
+#define NBDKIT_EXIT_WITH_PARENT_H
+
+/* Test if the feature is available on the platform. */
+extern bool can_exit_with_parent(void);
+
+/*
+ * --exit-with-parent: kill the current process if the parent exits.
+ * This may return -1 on error.
+ */
+extern int set_exit_with_parent(void);
+
+#endif /* NBDKIT_EXIT_WITH_PARENT_H */
diff --git a/qemu-options.hx b/qemu-options.hx
index 075f4be2e3..9ff0346afe 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -5437,15 +5437,18 @@ DEF("qtest-log", HAS_ARG, QEMU_OPTION_qtest_log, "", QEMU_ARCH_ALL)
 
 #if defined(CONFIG_POSIX) && !defined(EMSCRIPTEN)
 DEF("run-with", HAS_ARG, QEMU_OPTION_run_with,
-    "-run-with [async-teardown=on|off][,chroot=dir][user=username|uid:gid]\n"
+    "-run-with [async-teardown=on|off][,chroot=dir]\n" \
+    "          [,exit-with-parent=on|off][,user=username|uid:gid]\n"
     "                Set miscellaneous QEMU process lifecycle options:\n"
     "                async-teardown=on enables asynchronous teardown (Linux only)\n"
+    "                exit-with-parent=on causes QEMU to exit if the parent\n"
+    "                  process of QEMU exits (Linux, FreeBSD, macOS only)\n"
     "                chroot=dir chroot to dir just before starting the VM\n"
     "                user=username switch to the specified user before starting the VM\n"
     "                user=uid:gid ditto, but use specified user-ID and group-ID instead\n",
     QEMU_ARCH_ALL)
 SRST
-``-run-with [async-teardown=on|off][,chroot=dir][user=username|uid:gid]``
+``-run-with [async-teardown=on|off][,chroot=dir][,exit-with-parent=on|off][,user=username|uid:gid]``
     Set QEMU process lifecycle options.
 
     ``async-teardown=on`` enables asynchronous teardown. A new process called
@@ -5463,6 +5466,12 @@ SRST
     immediately before starting the guest execution. This is especially useful
     in combination with ``user=...``.
 
+    ``exit-with-parent=on`` causes QEMU to exit if the parent process of
+    QEMU exits.  This can be used when QEMU runs a captive appliance,
+    where the lifetime of the appliance is scoped to the parent process.
+    In case the parent process crashes, QEMU is still cleaned up.
+    This only works on Linux, FreeBSD and macOS platforms.
+
     ``user=username`` or ``user=uid:gid`` can be used to drop root privileges
     before starting guest execution. QEMU will use the ``setuid`` and ``setgid``
     system calls to switch to the specified identity.  Note that the
diff --git a/system/exit-with-parent.c b/system/exit-with-parent.c
new file mode 100644
index 0000000000..913439d553
--- /dev/null
+++ b/system/exit-with-parent.c
@@ -0,0 +1,169 @@
+/*
+ * SPDX-License-Identifier: BSD-3-Clause
+ * Originally derived from nbdkit common/utils/exit-with-parent.c
+ * Copyright Red Hat
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are
+ * met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ *
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * * Neither the name of Red Hat nor the names of its contributors may be
+ * used to endorse or promote products derived from this software without
+ * specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
+ * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
+ * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+ * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+ * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+ * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+/*
+ * Implement the --exit-with-parent feature on operating systems which
+ * support it.
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/exit-with-parent.h"
+
+#ifdef CONFIG_LINUX
+#include <sys/prctl.h>
+#endif
+
+#ifdef __FreeBSD__
+#include <sys/procctl.h>
+#endif
+
+#if defined(CONFIG_LINUX)
+
+/*
+ * Send SIGTERM to self when the parent exits.  This will cause
+ * qemu_system_killed() to be called.
+ *
+ * PR_SET_PDEATHSIG has been defined since Linux 2.1.57.
+ */
+int
+set_exit_with_parent(void)
+{
+    return prctl(PR_SET_PDEATHSIG, SIGTERM);
+}
+
+bool
+can_exit_with_parent(void)
+{
+    return true;
+}
+
+#elif defined(__FreeBSD__)
+
+/*
+ * Send SIGTERM to self when the parent exits.  This will cause
+ * qemu_system_killed() to be called.
+ *
+ * PROC_PDEATHSIG_CTL has been defined since FreeBSD 11.2.
+ */
+int
+set_exit_with_parent(void)
+{
+    const int sig = SIGTERM;
+    return procctl(P_PID, 0, PROC_PDEATHSIG_CTL, (void *) &sig);
+}
+
+bool
+can_exit_with_parent(void)
+{
+    return true;
+}
+
+#elif defined(__APPLE__)
+
+/* For macOS. */
+
+#include "qemu/thread.h"
+#include "qemu/error-report.h"
+#include "system/runstate.h"
+#include <sys/event.h>
+
+static void *
+exit_with_parent_loop(void *vp)
+{
+    const pid_t ppid = getppid();
+    int fd;
+    struct kevent kev, res[1];
+    int r;
+
+    /* Register the kevent to wait for ppid to exit. */
+    fd = kqueue();
+    if (fd == -1) {
+        error_report("exit_with_parent_loop: kqueue: %m");
+        return NULL;
+    }
+    EV_SET(&kev, ppid, EVFILT_PROC, EV_ADD | EV_ENABLE, NOTE_EXIT, 0, NULL);
+    if (kevent(fd, &kev, 1, NULL, 0, NULL) == -1) {
+        error_report("exit_with_parent_loop: kevent: %m");
+        close(fd);
+        return NULL;
+    }
+
+    /* Wait for the kevent to happen. */
+    r = kevent(fd, 0, 0, res, 1, NULL);
+    if (r == 1 && res[0].ident == ppid) {
+        /* Behave like Linux and FreeBSD above, as if SIGTERM was sent */
+        qemu_system_killed(SIGTERM, ppid);
+    }
+
+    return NULL;
+}
+
+int
+set_exit_with_parent(void)
+{
+    int r;
+    QemuThread exit_with_parent_thread;
+
+    /*
+     * We have to block waiting for kevent, so that requires that we
+     * start a background thread.
+     */
+    qemu_thread_create(&exit_with_parent_thread,
+                       "exit-parent",
+                       exit_with_parent_loop, NULL,
+                       QEMU_THREAD_DETACHED);
+    return 0;
+}
+
+bool
+can_exit_with_parent(void)
+{
+    return true;
+}
+
+#else /* any platform that doesn't support this function */
+
+int
+set_exit_with_parent(void)
+{
+    g_assert_not_reached();
+}
+
+bool
+can_exit_with_parent(void)
+{
+    return false;
+}
+
+#endif
diff --git a/system/meson.build b/system/meson.build
index 6d21ff9faa..4b69ef0f5f 100644
--- a/system/meson.build
+++ b/system/meson.build
@@ -15,6 +15,7 @@ system_ss.add(files(
   'datadir.c',
   'dirtylimit.c',
   'dma-helpers.c',
+  'exit-with-parent.c',
   'globals.c',
   'ioport.c',
   'ram-block-attributes.c',
diff --git a/system/vl.c b/system/vl.c
index 00f3694725..47b6e60b94 100644
--- a/system/vl.c
+++ b/system/vl.c
@@ -53,6 +53,7 @@
 #include "qemu/sockets.h"
 #include "qemu/accel.h"
 #include "qemu/async-teardown.h"
+#include "qemu/exit-with-parent.h"
 #include "hw/usb.h"
 #include "hw/isa/isa.h"
 #include "hw/scsi/scsi.h"
@@ -783,6 +784,10 @@ static QemuOptsList qemu_run_with_opts = {
             .name = "chroot",
             .type = QEMU_OPT_STRING,
         },
+        {
+            .name = "exit-with-parent",
+            .type = QEMU_OPT_BOOL,
+        },
         {
             .name = "user",
             .type = QEMU_OPT_STRING,
@@ -3690,6 +3695,14 @@ void qemu_init(int argc, char **argv)
                 if (str) {
                     os_set_chroot(str);
                 }
+                if (qemu_opt_get_bool(opts, "exit-with-parent", false)) {
+                    if (!can_exit_with_parent()) {
+                        error_report("exit-with-parent is not available"
+                                     " on this platform");
+                        exit(1);
+                    }
+                    set_exit_with_parent();
+                }
                 str = qemu_opt_get(opts, "user");
                 if (str) {
                     if (!os_set_runas(str)) {
-- 
2.50.1

[PATCH v1 2/2] tests/qtest: Use exit-with-parent=on in qtest invocations

[Richard W.M. Jones]

Previously libqtest.c set PR_SET_PDEATHSIG (or the equivalent on
FreeBSD) after forking the qemu subprocess.  However we can get the
same behaviour now by using the new -run-with exit-with-parent=on
flag, on platforms that support it.
---
 tests/qtest/libqtest.c | 21 +++------------------
 1 file changed, 3 insertions(+), 18 deletions(-)

diff --git a/tests/qtest/libqtest.c b/tests/qtest/libqtest.c
index 933d085869..12f865b805 100644
--- a/tests/qtest/libqtest.c
+++ b/tests/qtest/libqtest.c
@@ -433,24 +433,6 @@ static QTestState *qtest_spawn_qemu(const char *qemu_bin, const char *args,
 #ifndef _WIN32
     pid = fork();
     if (pid == 0) {
-#ifdef __linux__
-        /*
-         * Although we register a ABRT handler to kill off QEMU
-         * when g_assert() triggers, we want an extra safety
-         * net. The QEMU process might be non-functional and
-         * thus not have responded to SIGTERM. The test script
-         * might also have crashed with SEGV, in which case the
-         * cleanup handlers won't ever run.
-         *
-         * This PR_SET_PDEATHSIG setup will ensure any remaining
-         * QEMU will get terminated with SIGKILL in these cases.
-         */
-        prctl(PR_SET_PDEATHSIG, SIGKILL, 0, 0, 0);
-#endif /* __linux__ */
-#ifdef __FreeBSD__
-        int sig = SIGKILL;
-        procctl(P_PID, getpid(), PROC_PDEATHSIG_CTL, &sig);
-#endif /* __FreeBSD__ */
         execlp("/bin/sh", "sh", "-c", command->str, NULL);
         exit(1);
     }
@@ -481,6 +463,9 @@ gchar *qtest_qemu_args(const char *extra_args)
                       "-mon chardev=char0,mode=control "
                       "-display none "
                       "-audio none "
+#if defined(__linux__) || defined(__FreeBSD__) || defined(__APPLE__)
+                      "-run-with exit-with-parent=on "
+#endif
                       "%s"
                       " -accel qtest",
 
-- 
2.50.1

Re: [PATCH v1 0/2] Implement -run-with exit-with-parent=on

[Richard W.M. Jones]

On Fri, Oct 03, 2025 at 02:24:36PM +0100, Richard W.M. Jones wrote:
> An earlier WIP version was posted here:
> https://lists.gnu.org/archive/html/qemu-devel/2025-10/msg00309.html
> 
> In this version I have incorporated all of Daniel Berrange's feedback,
> and I also tested the change on macOS.
> 
> The new second patch modifies the test suite to use the new flag
> unconditionally, instead of open coding setting PR_SET_PDEATHSIG etc.
> The test suite fails for me both before and after this patch in the
> same way, so I assume this is nothing to do with the patch itself?
> 
>   4/405 qemu:func-quick+func-x86_64 / func-x86_64-bad_vmstate               ERROR            0.15s   exit status 1
> 
> Some points of note:
> 
>  - There's no way to find out if the qemu binary supports
>    exit-with-parent=on except to try it.  Maybe this should be exposed
>    somehow?

.. or exit-with-parent=best, but then it would no longer be a straight
boolean option.

>  - On macOS I wasn't able to find a satisfactory way to force
>    shutdown, except calling 'qemu_system_killed' and pretending we'd
>    been killed by SIGTERM (which does at least emulate what Linux &
>    FreeBSD do).  I suppose it'd be nice if there was a "killed by
>    parent" reason which also forced shutdown.
> 
> Rich.
> 

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org

Re: [PATCH v1 1/2] Implement -run-with exit-with-parent=on

[Richard W.M. Jones]

On Fri, Oct 03, 2025 at 02:24:37PM +0100, Richard W.M. Jones wrote:
[...]
> +int
> +set_exit_with_parent(void)
> +{
> +    int r;
> +    QemuThread exit_with_parent_thread;
> +
> +    /*
> +     * We have to block waiting for kevent, so that requires that we
> +     * start a background thread.
> +     */
> +    qemu_thread_create(&exit_with_parent_thread,
> +                       "exit-parent",
> +                       exit_with_parent_loop, NULL,
> +                       QEMU_THREAD_DETACHED);
> +    return 0;
> +}

'r' is unused.  Fixed in my local copy.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top