[PATCH 00/21] system: introduce a live data recorder facility

[PATCH 00/21] system: introduce a live data recorder facility

[Daniel P. Berrangé]

This series introduces a new facility called 'tldr' which is a
live data recorder. This is intended to aid support of production
QEMU deployments, by providing a data collection facility that
can be left "always on", such that in the event of a QEMU crash
or guest OS crash, certain important information will be recorded.

The data recorder has to be enabled by adding a "tldr-config"
object instance with -object / object_add.

With this series, the data recorder collects

 * error_report/warn_report/error_printf messages
 * QMP events
 * QMP errors
 * QMP commands, which are not query-XXX/qom-list,
   or any command associated with a QMP error.
 * qemu_log output

The last item is intended primarily to be used in combination
with tracing and its 'log' backend, to have continuous
recording of certain interesting probes.

Each type of data is collected in a distinct circular
buffer, such that high volumes of one type of data, does
not result in overwriting other types of low volume data.

The data recorder is set to trigger on SIGBUS, SIGSEGV,
SIGILL, SIGFPE and SIGABRT, or manually via QMP 'tldr-dump'.

There is a open question in one of the patches about SIGBUS
handling, since that is already used for MCE capture and I'm
unclear if I'm breaking something about KVM MCE reporting.

This series proposes using sigaltstack() to register an
alternative signal handling stack. This would mainly be
useful if we have a crash from a stack overflow, but the
cost is that we have 8k extra per thread, and I'm not
entirely convinced that's a beneficial tradeoff. Might
be better to just accept stack overflows as a risk.

This series only captures text based info, but as a followup
I would like to extend this to capture statistics data from
relevant subsystems. For example, the BlockBackend has its
own "always on" stats accounting code, which can be queried
with QMP. Ideally I would like to figure out a way to dump
all the BlockBackend stats in the data recorder output.
Likewise perhaps capture NIC stats and/or KVM stats in an
"always on" manner for some interesting baseline dataset.
The challenge with that is making it safe to access the
stats data without taking any mutex locks, since this must
run from a signal handler.

A typical recommended usage would be:

 $QEMU  -object tldr-config,id=tldr
        -d tldr
        -trace events=/var/lib/libvirt/qemu/trace.txt

This leaves dumps to go to QEMU's stderr on crash, but
to send them to a file

   -object tldr-config,id=tldr,crash-target=/some/path

The '-d tldr' flag redirects qemu_log() to the TLDR buffer
instead of stdout, and the '-trace events' arg defines
the "always on" tracepoints we want to capture.

An example crash dump seen if I send SIGBUS to a libvirt
managed QEMU is, with 'object*' tracepoints collected:

🚨🚨🚨 QEMU Data Recorder Start 🚨🚨🚨

🔍 Trigger:
  - Signal: SIGBUS (7)
  - Code: User process kill (0)
  - Address: 0x1e5a9e

🎥  Message recording 'command':
2026-10-15T10:59:11.318427Z {"execute": "qmp_capabilities", "id": "libvirt-1"}
2026-10-15T10:59:11.322047Z {"execute": "migrate-set-capabilities", "arguments": {"capabilities": [{"state": true, "capability": "events"}]}, "id": "libvirt-3"}
2026-10-15T10:59:11.347566Z {"execute": "balloon", "arguments": {"value": 4294967296}, "id": "libvirt-9"}
2026-10-15T10:59:11.354570Z {"execute": "set-action", "arguments": {"panic": "pause"}, "id": "libvirt-12"}
2026-10-15T10:59:11.358046Z {"execute": "cont", "id": "libvirt-14"}

🎥  Message recording 'event':
2026-10-15T10:59:11.358108Z {"timestamp": {"seconds": 1792061951, "microseconds": 358094}, "event": "RESUME"}
2026-10-15T10:59:20.757392Z {"timestamp": {"seconds": 1792061960, "microseconds": 757371}, "event": "VSERPORT_CHANGE", "data": {"open": true, "id": "console1"}}
2026-10-15T10:59:24.431398Z {"timestamp": {"seconds": 1792061964, "microseconds": 431379}, "event": "NIC_RX_FILTER_CHANGED", "data": {"name": "net0", "path": "/machine/peripheral/net0/virtio-backend"}}
2026-10-15T10:59:32.586658Z {"timestamp": {"seconds": 1792061972, "microseconds": 586603}, "event": "RTC_CHANGE", "data": {"offset": 0, "qom-path": "/machine/unattached/device[12]/rtc"}}

🎥  Message recording 'log':
28:QIO_CHANNEL_FILE)
2026-10-15T10:59:31.753125Z object_dynamic_cast_assert obj=0x557d5c828780 type=chardev-pty->chardev-pty (../chardev/char-pty.c:50:PTY_CHARDEV)
2026-10-15T10:59:31.753128Z object_dynamic_cast_assert obj=0x557d5c828780 type=chardev-pty->chardev-pty (../chardev/char-pty.c:50:PTY_CHARDEV)
2026-10-15T10:59:31.796114Z object_dynamic_cast_assert obj=0x557d5ca7d3a0 type=host-x86_64-cpu->x86_64-cpu (../target/i386/cpu-qom.h:31:X86_CPU)
2026-10-15T10:59:31.796148Z object_class_dynamic_cast_assert type=kvm-apic->apic-common (/home/berrange/src/virt/qemu/include/hw/i386/apic_internal.h:131:APIC_COMMON_GET_CLASS)

...snip...

2026-10-15T10:59:33.713641Z object_dynamic_cast_assert obj=0x557d5cc20b10 type=PCIE->PCI (/home/berrange/src/virt/qemu/include/hw/pci/pci.h:289:PCI_BUS)
2026-10-15T10:59:33.713643Z object_class_dynamic_cast_assert type=PCIE->PCI (/home/berrange/src/virt/qemu/include/hw/pci/pci.h:289:PCI_BUS_GET_CLASS)
2026-10-15T10:59:33.713648Z object_dynamic_cast_assert obj=0x557d5ca03700 type=host-x86_64-cpu->x86_64-cpu (../target/i386/cpu-qom.h:31:X86_CPU)
2026-10-15T10:59:33.713664Z object_dynamic_cast_assert obj=0x557d5eadd320 type=virtio-gpu-device->virtio-device (/home/berrange/src/virt/qemu/include/hw/virtio/virtio.h:91:VIRTIO_DEVICE)

🚨🚨🚨 QEMU Data Recorder End 🚨🚨🚨

"object_*" is not a particularly good choice for 'always on'
data collection as it is high volume and mostly noise. The
job of identifying the "interesting" low volume tracepoints
is outstanding. We can provide guidance upstream but downstream
should control what they're interested in collecting. We might
want to add further tracepoints in the code at interesting
places that we useful for post-crash debugging.

Daniel P. Berrangé (21):
  osdep: add 'ignore_value(x)' macro
  system: move SIGBUS signal handling to signal.c
  system: rename qemu_init_sigbus to qemu_init_crash_signals
  system: run SIGBUS handler on alternate signal stack
  util: expand alternate signal stack to all threads
  osdep: provide dummy siginfo_t definition
  util: introduce 'tldr', a system data recorder for crash handling
  qapi: add 'tldr-dump' command for on-demand data recorder output
  system: trigger TLDR crash dump on unhandled SIGBUS
  system: simplify signal handler logic
  main: allow SIGBUS in the main loop
  system: enable SIGBUS handling on non-Linux platforms
  system: enable crash dumps on SIGSEGV, SIGILL, SIGFPE and SIGABRT
  util: introduce capture of textual messages in data recorder
  util: capture warnings and errors in the data recorder
  monitor: remove obsolete/redundant exports
  monitor: capture monitor commands and events in data recorder
  log: enable directing log messages to data recorder
  tests/qtest: add API to declare an expected crash signal
  qtest: add 'crash' command to generate 'authentic' process crashes
  tests/qtest: add test to validate data recorder on crashes

 MAINTAINERS                 |  12 +
 include/qemu/log.h          |   1 +
 include/qemu/osdep.h        |  16 +
 include/qemu/tldr.h         |  76 ++++
 include/system/signal.h     |  25 ++
 meson.build                 |   4 +
 monitor/monitor-internal.h  |   2 -
 monitor/qmp.c               |  66 +++-
 qapi/meson.build            |   1 +
 qapi/qapi-schema.json       |   1 +
 qapi/qom.json               |  18 +
 qapi/tldr.json              |  62 +++
 qemu-options.hx             |  24 ++
 stubs/meson.build           |   1 +
 stubs/monitor-puts.c        |   9 +
 system/cpus.c               |  78 +---
 system/meson.build          |   1 +
 system/qtest.c              |  50 +++
 system/signal.c             | 149 +++++++
 tests/qtest/libqtest.c      |  62 ++-
 tests/qtest/libqtest.h      |  27 ++
 tests/qtest/meson.build     |   3 +
 tests/qtest/tldr-test.c     | 124 ++++++
 tests/unit/meson.build      |   1 +
 tests/unit/test-util-tldr.c | 377 ++++++++++++++++++
 util/error-report.c         |  43 ++-
 util/log.c                  |  42 +-
 util/main-loop.c            |   1 -
 util/meson.build            |   1 +
 util/qemu-thread-posix.c    |  20 +
 util/tldr-internal.h        |  52 +++
 util/tldr.c                 | 750 ++++++++++++++++++++++++++++++++++++
 32 files changed, 1976 insertions(+), 123 deletions(-)
 create mode 100644 include/qemu/tldr.h
 create mode 100644 include/system/signal.h
 create mode 100644 qapi/tldr.json
 create mode 100644 stubs/monitor-puts.c
 create mode 100644 system/signal.c
 create mode 100644 tests/qtest/tldr-test.c
 create mode 100644 tests/unit/test-util-tldr.c
 create mode 100644 util/tldr-internal.h
 create mode 100644 util/tldr.c

-- 
2.56.0

[PATCH 01/21] osdep: add 'ignore_value(x)' macro

[Daniel P. Berrangé]

Functions annotated with 'warn_unused_result' cannot be cast
to (void) to silence the compiler. Add the ignore_value()
macro which tricks the compiler into thinking the result is
genuinely used, avoiding the need for manual workarounds where
the return value is intentionally ignored.

This clever macro is derived from one originating with the
GNULIB project:

  https://gitweb.git.savannah.gnu.org/gitweb/?p=gnulib.git;a=blob;f=lib/ignore-value.h;hb=HEAD

but simplified since we don't need compat with old GCC and
optimizing on clang is overkill.

Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 include/qemu/osdep.h | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/include/qemu/osdep.h b/include/qemu/osdep.h
index 1ec5b42230b..c1ca2c127b6 100644
--- a/include/qemu/osdep.h
+++ b/include/qemu/osdep.h
@@ -251,6 +251,14 @@ extern "C" {
 #endif
 
 
+/*
+ * Merely casting to (void) is not sufficient for functions
+ * annotated with the "warn_unused_result" attribute
+ */
+#define ignore_value(x) \
+    (__extension__ ({ __typeof__(x) __x = (x); (void) __x; }))
+
+
 /*
  * For mingw, as of v6.0.0, the function implementing the assert macro is
  * not marked as noreturn, so the compiler cannot delete code following an
-- 
2.56.0

[PATCH 02/21] system: move SIGBUS signal handling to signal.c

[Daniel P. Berrangé]

Introduce a new file 'system/signal.c' to hold the SIGBUS signal
handler, since subsequent commits are going to expand the signal
handling logic somewhat.

The existing copyright header/license boilerplate is retained
instead of a SPDX tag, to respect historic context.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 MAINTAINERS             |   2 +
 include/system/signal.h |  25 ++++++++++
 system/cpus.c           |  76 +---------------------------
 system/meson.build      |   1 +
 system/signal.c         | 107 ++++++++++++++++++++++++++++++++++++++++
 5 files changed, 136 insertions(+), 75 deletions(-)
 create mode 100644 include/system/signal.h
 create mode 100644 system/signal.c

diff --git a/MAINTAINERS b/MAINTAINERS
index fc60193aa3f..6b5aafe43b1 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -553,6 +553,7 @@ F: include/exec/target_long.h
 F: include/qemu/accel.h
 F: include/system/accel-*.h
 F: include/system/cpus.h
+F: include/system/signal.h
 F: include/accel/accel-*.h
 F: accel/accel-*.?
 F: accel/dummy-cpus.?
@@ -562,6 +563,7 @@ F: cpu-common.c
 F: cpu-target.c
 F: qapi/accelerator.json
 F: system/cpus.c
+F: system/signal.c
 
 Apple Silicon HVF CPUs
 M: Alexander Graf <agraf@csgraf.de>
diff --git a/include/system/signal.h b/include/system/signal.h
new file mode 100644
index 00000000000..f8d00221bcf
--- /dev/null
+++ b/include/system/signal.h
@@ -0,0 +1,25 @@
+/*
+ * QEMU System Emulator
+ *
+ * Copyright (c) 2003-2008 Fabrice Bellard
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+
+void qemu_init_sigbus(void);
diff --git a/system/cpus.c b/system/cpus.c
index e11a5aab6a6..b231ca4c4ac 100644
--- a/system/cpus.c
+++ b/system/cpus.c
@@ -39,28 +39,11 @@
 #include "system/runstate.h"
 #include "system/cpu-timers.h"
 #include "system/whpx.h"
+#include "system/signal.h"
 #include "hw/core/boards.h"
 #include "hw/core/hw-error.h"
 #include "trace.h"
 
-#ifdef CONFIG_LINUX
-
-#include <sys/prctl.h>
-
-#ifndef PR_MCE_KILL
-#define PR_MCE_KILL 33
-#endif
-
-#ifndef PR_MCE_KILL_SET
-#define PR_MCE_KILL_SET 1
-#endif
-
-#ifndef PR_MCE_KILL_EARLY
-#define PR_MCE_KILL_EARLY 1
-#endif
-
-#endif /* CONFIG_LINUX */
-
 /* The Big QEMU Lock (BQL) */
 static QemuMutex bql;
 
@@ -302,63 +285,6 @@ void cpu_handle_guest_debug(CPUState *cpu)
     }
 }
 
-#ifdef CONFIG_LINUX
-static void sigbus_reraise(void)
-{
-    sigset_t set;
-    struct sigaction action;
-
-    memset(&action, 0, sizeof(action));
-    action.sa_handler = SIG_DFL;
-    if (!sigaction(SIGBUS, &action, NULL)) {
-        raise(SIGBUS);
-        sigemptyset(&set);
-        sigaddset(&set, SIGBUS);
-        pthread_sigmask(SIG_UNBLOCK, &set, NULL);
-    }
-    perror("Failed to re-raise SIGBUS!");
-    abort();
-}
-
-static void sigbus_handler(int n, siginfo_t *siginfo, void *ctx)
-{
-    if (siginfo->si_code != BUS_MCEERR_AO && siginfo->si_code != BUS_MCEERR_AR) {
-        sigbus_reraise();
-    }
-
-    if (current_cpu) {
-        /* Called asynchronously in VCPU thread.  */
-        if (kvm_on_sigbus_vcpu(current_cpu, siginfo->si_code, siginfo->si_addr)) {
-            sigbus_reraise();
-        }
-    } else {
-        /* Called synchronously (via signalfd) in main thread.  */
-        if (kvm_on_sigbus(siginfo->si_code, siginfo->si_addr)) {
-            sigbus_reraise();
-        }
-    }
-}
-
-static void qemu_init_sigbus(void)
-{
-    struct sigaction action;
-
-    /*
-     * ALERT: when modifying this, take care that SIGBUS forwarding in
-     * qemu_prealloc_mem() will continue working as expected.
-     */
-    memset(&action, 0, sizeof(action));
-    action.sa_flags = SA_SIGINFO;
-    action.sa_sigaction = sigbus_handler;
-    sigaction(SIGBUS, &action, NULL);
-
-    prctl(PR_MCE_KILL, PR_MCE_KILL_SET, PR_MCE_KILL_EARLY, 0, 0);
-}
-#else /* !CONFIG_LINUX */
-static void qemu_init_sigbus(void)
-{
-}
-#endif /* !CONFIG_LINUX */
 
 static QemuThread io_thread;
 
diff --git a/system/meson.build b/system/meson.build
index a6ba7a04f7d..1829de3d25b 100644
--- a/system/meson.build
+++ b/system/meson.build
@@ -24,6 +24,7 @@ system_ss.add(files(
   'rtc.c',
   'runstate-action.c',
   'runstate.c',
+  'signal.c',
   'watchpoint.c',
 ))
 
diff --git a/system/signal.c b/system/signal.c
new file mode 100644
index 00000000000..6d69db2bc79
--- /dev/null
+++ b/system/signal.c
@@ -0,0 +1,107 @@
+/*
+ * QEMU System Emulator
+ *
+ * Copyright (c) 2003-2008 Fabrice Bellard
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+
+#include "qemu/osdep.h"
+#include "system/signal.h"
+#include "hw/core/cpu.h"
+#include "system/kvm.h"
+
+#ifdef CONFIG_LINUX
+
+#include <sys/prctl.h>
+
+#ifndef PR_MCE_KILL
+#define PR_MCE_KILL 33
+#endif
+
+#ifndef PR_MCE_KILL_SET
+#define PR_MCE_KILL_SET 1
+#endif
+
+#ifndef PR_MCE_KILL_EARLY
+#define PR_MCE_KILL_EARLY 1
+#endif
+
+#endif /* CONFIG_LINUX */
+
+
+#ifdef CONFIG_LINUX
+static void sigbus_reraise(void)
+{
+    sigset_t set;
+    struct sigaction action;
+
+    memset(&action, 0, sizeof(action));
+    action.sa_handler = SIG_DFL;
+    if (!sigaction(SIGBUS, &action, NULL)) {
+        raise(SIGBUS);
+        sigemptyset(&set);
+        sigaddset(&set, SIGBUS);
+        pthread_sigmask(SIG_UNBLOCK, &set, NULL);
+    }
+    perror("Failed to re-raise SIGBUS!");
+    abort();
+}
+
+static void sigbus_handler(int n, siginfo_t *siginfo, void *ctx)
+{
+    if (siginfo->si_code != BUS_MCEERR_AO &&
+        siginfo->si_code != BUS_MCEERR_AR) {
+        sigbus_reraise();
+    }
+
+    if (current_cpu) {
+        /* Called asynchronously in VCPU thread.  */
+        if (kvm_on_sigbus_vcpu(current_cpu, siginfo->si_code,
+                               siginfo->si_addr)) {
+            sigbus_reraise();
+        }
+    } else {
+        /* Called synchronously (via signalfd) in main thread.  */
+        if (kvm_on_sigbus(siginfo->si_code, siginfo->si_addr)) {
+            sigbus_reraise();
+        }
+    }
+}
+
+void qemu_init_sigbus(void)
+{
+    struct sigaction action;
+
+    /*
+     * ALERT: when modifying this, take care that SIGBUS forwarding in
+     * qemu_prealloc_mem() will continue working as expected.
+     */
+    memset(&action, 0, sizeof(action));
+    action.sa_flags = SA_SIGINFO;
+    action.sa_sigaction = sigbus_handler;
+    sigaction(SIGBUS, &action, NULL);
+
+    prctl(PR_MCE_KILL, PR_MCE_KILL_SET, PR_MCE_KILL_EARLY, 0, 0);
+}
+#else /* !CONFIG_LINUX */
+void qemu_init_sigbus(void)
+{
+}
+#endif /* !CONFIG_LINUX */
-- 
2.56.0

[PATCH 03/21] system: rename qemu_init_sigbus to qemu_init_crash_signals

[Daniel P. Berrangé]

Support for handling further crash signals will be introduced
in subsequent commits, so remove 'sigbus' from the initializer
function name.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 include/system/signal.h | 2 +-
 system/cpus.c           | 2 +-
 system/signal.c         | 4 ++--
 3 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/include/system/signal.h b/include/system/signal.h
index f8d00221bcf..bb9b77428b1 100644
--- a/include/system/signal.h
+++ b/include/system/signal.h
@@ -22,4 +22,4 @@
  * THE SOFTWARE.
  */
 
-void qemu_init_sigbus(void);
+void qemu_init_crash_signals(void);
diff --git a/system/cpus.c b/system/cpus.c
index b231ca4c4ac..870d4cfaaa3 100644
--- a/system/cpus.c
+++ b/system/cpus.c
@@ -295,7 +295,7 @@ static QemuCond qemu_pause_cond;
 
 void qemu_init_cpu_loop(void)
 {
-    qemu_init_sigbus();
+    qemu_init_crash_signals();
     qemu_cond_init(&qemu_cpu_cond);
     qemu_cond_init(&qemu_pause_cond);
     qemu_mutex_init(&bql);
diff --git a/system/signal.c b/system/signal.c
index 6d69db2bc79..423325c9a1a 100644
--- a/system/signal.c
+++ b/system/signal.c
@@ -85,7 +85,7 @@ static void sigbus_handler(int n, siginfo_t *siginfo, void *ctx)
     }
 }
 
-void qemu_init_sigbus(void)
+void qemu_init_crash_signals(void)
 {
     struct sigaction action;
 
@@ -101,7 +101,7 @@ void qemu_init_sigbus(void)
     prctl(PR_MCE_KILL, PR_MCE_KILL_SET, PR_MCE_KILL_EARLY, 0, 0);
 }
 #else /* !CONFIG_LINUX */
-void qemu_init_sigbus(void)
+void qemu_init_crash_signals(void)
 {
 }
 #endif /* !CONFIG_LINUX */
-- 
2.56.0

[PATCH 04/21] system: run SIGBUS handler on alternate signal stack

[Daniel P. Berrangé]

In preparation for introducing data recorder dumps during handling of
of crash signals (BUS, SEGV, ILL, FPE, ABRT), introduce an alternative
signal stack. This gives a known safe stack for code to operate under.

This adds overhead of SIGSTKSZ bytes, which is OS-dependent, but on
Linux is 8kb.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 meson.build     |  1 +
 system/signal.c | 14 +++++++++++++-
 2 files changed, 14 insertions(+), 1 deletion(-)

diff --git a/meson.build b/meson.build
index 255c887e8e6..b383c7fceef 100644
--- a/meson.build
+++ b/meson.build
@@ -2647,6 +2647,7 @@ config_host_data.set('HAVE_GLIB_WITH_SLICE_ALLOCATOR', glib_has_gslice)
 config_host_data.set('HAVE_OPENPTY', cc.has_function('openpty', dependencies: util))
 config_host_data.set('HAVE_STRCHRNUL', cc.has_function('strchrnul', prefix: osdep_prefix))
 config_host_data.set('HAVE_SYSTEM_FUNCTION', cc.has_function('system', prefix: '#include <stdlib.h>'))
+config_host_data.set('CONFIG_SIGALTSTACK', cc.has_function('sigaltstack'))
 if rbd.found()
   config_host_data.set('HAVE_RBD_NAMESPACE_EXISTS',
                        cc.has_function('rbd_namespace_exists',
diff --git a/system/signal.c b/system/signal.c
index 423325c9a1a..dc5a75e2816 100644
--- a/system/signal.c
+++ b/system/signal.c
@@ -26,6 +26,7 @@
 #include "system/signal.h"
 #include "hw/core/cpu.h"
 #include "system/kvm.h"
+#include "qemu/error-report.h"
 
 #ifdef CONFIG_LINUX
 
@@ -89,12 +90,23 @@ void qemu_init_crash_signals(void)
 {
     struct sigaction action;
 
+#ifdef CONFIG_SIGALTSTACK
+    static stack_t signal_stack;
+    signal_stack.ss_sp = (void *)g_new0(char, SIGSTKSZ);
+    signal_stack.ss_size = SIGSTKSZ;
+    if (sigaltstack(&signal_stack, NULL) < 0) {
+        error_printf("unable to enable alternate signal stack: %s",
+                     strerror(errno));
+        abort();
+    }
+#endif
+
     /*
      * ALERT: when modifying this, take care that SIGBUS forwarding in
      * qemu_prealloc_mem() will continue working as expected.
      */
     memset(&action, 0, sizeof(action));
-    action.sa_flags = SA_SIGINFO;
+    action.sa_flags = SA_SIGINFO | SA_ONSTACK;
     action.sa_sigaction = sigbus_handler;
     sigaction(SIGBUS, &action, NULL);
 
-- 
2.56.0

[PATCH 05/21] util: expand alternate signal stack to all threads

[Daniel P. Berrangé]

The sigaltstack() function is per thread, so setting SA_ONSTACK has
no effect unless all threads have an alternate signal stack registered.

This adds overhead of SIGSTKSZ bytes per thread, which is OS-dependent,
but on Linux is 8kb.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 util/qemu-thread-posix.c | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/util/qemu-thread-posix.c b/util/qemu-thread-posix.c
index 089606c93fb..66858018057 100644
--- a/util/qemu-thread-posix.c
+++ b/util/qemu-thread-posix.c
@@ -381,12 +381,28 @@ typedef struct {
     char *name;
 } QemuThreadArgs;
 
+static void qemu_thread_sigstack_free(void *data)
+{
+    stack_t sigstack = {
+        .ss_sp = NULL,
+        .ss_size = 0,
+        .ss_flags = SS_DISABLE,
+    };
+    sigaltstack(&sigstack, NULL);
+    g_free(data);
+}
+
 static void *qemu_thread_start(void *args)
 {
     QemuThreadArgs *qemu_thread_args = args;
     void *(*start_routine)(void *) = qemu_thread_args->start_routine;
     void *arg = qemu_thread_args->arg;
     void *r;
+    stack_t sigstack = {
+        .ss_sp = g_new0(char, SIGSTKSZ),
+        .ss_size = SIGSTKSZ,
+        .ss_flags = 0,
+    };
 
     if (qemu_thread_args->name) {
         qemu_thread_set_name(qemu_thread_args->name);
@@ -395,6 +411,9 @@ static void *qemu_thread_start(void *args)
     g_free(qemu_thread_args->name);
     g_free(qemu_thread_args);
 
+    sigaltstack(&sigstack, NULL);
+    pthread_cleanup_push(qemu_thread_sigstack_free, sigstack.ss_sp);
+
     /*
      * GCC 11 with glibc 2.17 on PowerPC reports
      *
@@ -413,6 +432,7 @@ static void *qemu_thread_start(void *args)
     pthread_cleanup_push(qemu_thread_atexit_notify, NULL);
     r = start_routine(arg);
     pthread_cleanup_pop(1);
+    pthread_cleanup_pop(1);
 
 #pragma GCC diagnostic pop
 
-- 
2.56.0

[PATCH 06/21] osdep: provide dummy siginfo_t definition

[Daniel P. Berrangé]

Windows platforms lack sigaction and thus also lack siginfo_t. For
the convenience of later code, provide a dummy siginfo_t struct
definition when it is missing.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 include/qemu/osdep.h | 8 ++++++++
 meson.build          | 2 ++
 2 files changed, 10 insertions(+)

diff --git a/include/qemu/osdep.h b/include/qemu/osdep.h
index c1ca2c127b6..fdb61d5d51e 100644
--- a/include/qemu/osdep.h
+++ b/include/qemu/osdep.h
@@ -635,6 +635,14 @@ void sigaction_invoke(struct sigaction *action,
                       struct qemu_signalfd_siginfo *info);
 #endif
 
+#ifndef HAVE_SIGINFO_T
+typedef struct {
+    int si_signo;
+    int si_code;
+    void *si_addr;
+} siginfo_t;
+#endif
+
 /*
  * Don't introduce new usage of this function, prefer the following
  * qemu_open/qemu_create that take an "Error **errp"
diff --git a/meson.build b/meson.build
index b383c7fceef..c6207443365 100644
--- a/meson.build
+++ b/meson.build
@@ -2709,6 +2709,8 @@ config_host_data.set('HAVE_OPTRESET',
                      cc.has_header_symbol('getopt.h', 'optreset'))
 config_host_data.set('HAVE_IPPROTO_MPTCP',
                      cc.has_header_symbol('netinet/in.h', 'IPPROTO_MPTCP'))
+config_host_data.set('HAVE_SIGINFO_T',
+                     cc.has_header_symbol('signal.h', 'siginfo_t'))
 if libaio.found()
   config_host_data.set('HAVE_IO_PREP_PWRITEV2',
                        cc.has_header_symbol('libaio.h', 'io_prep_pwritev2'))
-- 
2.56.0

[PATCH 07/21] util: introduce 'tldr', a system data recorder for crash handling

[Daniel P. Berrangé]

Introduce a new subsystem "tldr" aka "thrifty live data recorder"
that will be able to continuously collect certain information at
runtime and dump that in a crash report when QEMU exits abnormally.
As a second use case, it will support manual triggering on interesting
events such as guest OS panic.

Initially the framework does nothing except print the signal
information. Later commits will introduce broader data collection.
This relies on the dummy siginfo_t struct for portability to Windows.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 MAINTAINERS                 |   8 +
 include/qemu/tldr.h         |  24 ++
 qapi/qom.json               |  15 ++
 qemu-options.hx             |  14 ++
 tests/unit/meson.build      |   1 +
 tests/unit/test-util-tldr.c | 105 +++++++++
 util/meson.build            |   1 +
 util/tldr-internal.h        |  21 ++
 util/tldr.c                 | 438 ++++++++++++++++++++++++++++++++++++
 9 files changed, 627 insertions(+)
 create mode 100644 include/qemu/tldr.h
 create mode 100644 tests/unit/test-util-tldr.c
 create mode 100644 util/tldr-internal.h
 create mode 100644 util/tldr.c

diff --git a/MAINTAINERS b/MAINTAINERS
index 6b5aafe43b1..4fa68810c1e 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -4772,3 +4772,11 @@ Machine development tool
 M: Maksim Davydov <davydov-max@yandex-team.ru>
 S: Supported
 F: scripts/compare-machine-types.py
+
+Data Recorder
+M: Daniel P. Berrangé <berrange@redhat.com>
+S: Maintained
+F: util/tldr.c
+F: util/tldr-internal.h
+F: include/qemu/tldr.h
+F: tests/unit/test-util-tldr.c
diff --git a/include/qemu/tldr.h b/include/qemu/tldr.h
new file mode 100644
index 00000000000..67152b136b3
--- /dev/null
+++ b/include/qemu/tldr.h
@@ -0,0 +1,24 @@
+/*
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ *
+ * TL;DR: Thrifty Live Data Recorder
+ *
+ * Copyright (C) 2026 Red Hat Inc.
+ */
+
+#ifndef QEMU_TLDR_H__
+#define QEMU_TLDR_H__
+
+/**
+ * qtldr_crash_dump_signal:
+ * @si: signal information
+ *
+ * Print a crash dump in response to a signal.
+ *
+ * The crash will be printed to the configured crash
+ * file, if set. If opening the crash file fails, or
+ * no path is configured, STDERR will be used.
+ */
+void qtldr_crash_dump_signal(siginfo_t *si);
+
+#endif /* QEMU_TLDR_H__ */
diff --git a/qapi/qom.json b/qapi/qom.json
index 49f222796ab..1c9031fba9a 100644
--- a/qapi/qom.json
+++ b/qapi/qom.json
@@ -1233,6 +1233,19 @@
   'data': { '*pretty': 'bool',
             '*close-action': 'MonitorQMPCloseAction' } }
 
+##
+# @QTLDRConfigProperties:
+#     Configuration for data recorder
+#
+# @crash-target: fully qualified path to perserve data recorder
+#                contents on QEMU crash
+#
+# Since: 11.2.0
+##
+{ 'struct': 'QTLDRConfigProperties',
+  'data': { '*crash-target': 'str' }
+}
+
 ##
 # @ObjectType:
 #
@@ -1302,6 +1315,7 @@
     's390-pv-guest',
     'tdx-guest',
     'throttle-group',
+    'tldr-config',
     'tls-creds-anon',
     'tls-creds-psk',
     'tls-creds-x509',
@@ -1379,6 +1393,7 @@
       'tdx-guest':                  'TdxGuestProperties',
       'thread-context':             'ThreadContextProperties',
       'throttle-group':             'ThrottleGroupProperties',
+      'tldr-config':                'QTLDRConfigProperties',
       'tls-creds-anon':             'TlsCredsAnonProperties',
       'tls-creds-psk':              'TlsCredsPskProperties',
       'tls-creds-x509':             'TlsCredsX509Properties',
diff --git a/qemu-options.hx b/qemu-options.hx
index 2f6863180b0..2f63d86b635 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -6682,6 +6682,20 @@ SRST
         ::
 
             (qemu) qom-set /objects/iothread1 poll-max-ns 100000
+
+
+    ``-object tldr-config,id=id,crash-target=path``
+        Creating the ``tldr-config`` object enables the thrifty live data
+        recorder, which continuously captures important information about
+        the operation of the QEMU process. When a SIGSEGV, SIGBUS,
+        SIGILL, SIGFPE or SIGABRT are seen the contents of the data
+        recorder will be dumped to the stderr stream. If the
+        ``crash-target`` parameter was given, that specifies a fully
+        qualified file path to write the dump to instead of stderr.
+
+        The ``tldr-config`` object can be added at initial startup
+        or via QMP at runtime. Once created, however, it is not
+        permitted to disable it.
 ERST
 
 
diff --git a/tests/unit/meson.build b/tests/unit/meson.build
index d17435c1877..aa02895e767 100644
--- a/tests/unit/meson.build
+++ b/tests/unit/meson.build
@@ -136,6 +136,7 @@ if have_system
     'test-qmp-cmds': [testqapi],
     'test-xbzrle': [migration],
     'test-util-sockets': ['socket-helpers.c'],
+    'test-util-tldr': [],
     'test-base64': [],
     'test-bufferiszero': [],
     'test-smp-parse': [qom, meson.project_source_root() / 'hw/core/machine-smp.c'],
diff --git a/tests/unit/test-util-tldr.c b/tests/unit/test-util-tldr.c
new file mode 100644
index 00000000000..4384e10cc18
--- /dev/null
+++ b/tests/unit/test-util-tldr.c
@@ -0,0 +1,105 @@
+/*
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ *
+ * Tests for util/tldr.c
+ *
+ * Copyright 2026 Red Hat, Inc.
+ */
+
+#include "qemu/osdep.h"
+#include "util/tldr-internal.h"
+
+typedef struct {
+    bool siginfo;
+    const char *data;
+} QTLDRTestData;
+
+#define TEST_SIGINFO_NULL \
+    QTLDR_DUMP_TRIGGER_PREFIX "manual\n"
+
+#define TEST_SIGINFO_SEGV \
+    QTLDR_DUMP_TRIGGER_PREFIX "\n" \
+    "  - Signal: SIGSEGV (11)\n" \
+    "  - Code: Address not mapped (1)\n" \
+    "  - Address: 0x87539319\n"
+
+
+static void
+test_tldr_one(QTLDRTestData *data)
+{
+    g_autofree char *tmpname = NULL;
+    int tmpfd;
+    char buf[1024];
+    ssize_t dumplen;
+
+    tmpfd = g_file_open_tmp("test-util-tldr-XXXXXX",
+                            &tmpname, NULL);
+    unlink(tmpname);
+
+    if (data->siginfo) {
+        siginfo_t si = {};
+        si.si_signo = 11; /*
+                           * SIGSEGV on Linux, but don't use constant
+                           * since expected output string hardcodes 11
+                           */
+        si.si_code = 1;
+        si.si_addr = (void *)0x87539319;
+        qtldr_crash_dump_internal(tmpfd, &si);
+    } else {
+        qtldr_crash_dump_internal(tmpfd, NULL);
+    }
+
+    lseek(tmpfd, 0, SEEK_SET);
+
+    dumplen = read(tmpfd, buf, sizeof(buf));
+
+    assert(dumplen > 0);
+    buf[dumplen] = '\0';
+
+    close(tmpfd);
+
+    g_assert_cmpstr(buf, ==, data->data);
+}
+
+
+static void
+test_tldr_minimal_default(void)
+{
+    QTLDRTestData data = {
+        .siginfo = false,
+        .data =
+            QTLDR_DUMP_LEADER
+            TEST_SIGINFO_NULL
+            QTLDR_DUMP_TRAILER
+    };
+
+    test_tldr_one(&data);
+}
+
+
+static void
+test_tldr_minimal_siginfo(void)
+{
+    QTLDRTestData data = {
+        .siginfo = true,
+        .data =
+            QTLDR_DUMP_LEADER
+            TEST_SIGINFO_SEGV
+            QTLDR_DUMP_TRAILER
+    };
+
+    test_tldr_one(&data);
+}
+
+
+int main(int argc, char **argv)
+{
+    g_test_init(&argc, &argv, NULL);
+
+    g_test_add_func("/util/tldr/minimal/default",
+                    test_tldr_minimal_default);
+    g_test_add_func("/util/tldr/minimal/siginfo",
+                    test_tldr_minimal_siginfo);
+
+    return g_test_run();
+}
diff --git a/util/meson.build b/util/meson.build
index fa174c07a57..00c2350c87e 100644
--- a/util/meson.build
+++ b/util/meson.build
@@ -66,6 +66,7 @@ util_ss.add(files('int128.c'))
 util_ss.add(files('memalign.c'))
 util_ss.add(files('interval-tree.c'))
 util_ss.add(files('lockcnt.c'))
+util_ss.add(files('tldr.c'))
 
 if have_system
   util_ss.add(files('crc-ccitt.c'))
diff --git a/util/tldr-internal.h b/util/tldr-internal.h
new file mode 100644
index 00000000000..193d8425eee
--- /dev/null
+++ b/util/tldr-internal.h
@@ -0,0 +1,21 @@
+/*
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ *
+ * TL;DR: Thrifty Live Data Recorder
+ *
+ * Copyright (C) 2026 Red Hat Inc.
+ */
+
+#ifndef QEMU_TLDR_INTERNAL_H__
+#define QEMU_TLDR_INTERNAL_H__
+
+#include "qemu/tldr.h"
+
+#define QTLDR_DUMP_LEADER "\n🚨🚨🚨 QEMU Data Recorder Start 🚨🚨🚨\n"
+#define QTLDR_DUMP_TRAILER "\n🚨🚨🚨 QEMU Data Recorder End 🚨🚨🚨\n\n"
+#define QTLDR_DUMP_TRIGGER_PREFIX "\n🔍 Trigger:"
+#define QTLDR_DUMP_OPEN_ERROR "🔥 Unable to dump to '"
+
+void qtldr_crash_dump_internal(int fd, siginfo_t *si);
+
+#endif /* QEMU_TLDR_INTERNAL_H__ */
diff --git a/util/tldr.c b/util/tldr.c
new file mode 100644
index 00000000000..13e9881bc9b
--- /dev/null
+++ b/util/tldr.c
@@ -0,0 +1,438 @@
+/*
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ *
+ * TL;DR: Thrifty Live Data Recorder
+ *
+ * Copyright (C) 2026 Red Hat Inc.
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/atomic.h"
+#include "util/tldr-internal.h"
+#include "qom/object.h"
+#include "qom/object_interfaces.h"
+#include "qapi/error.h"
+
+struct QTLDRConfig {
+    Object parent;
+
+    char *crash_target;
+};
+
+static int qtldr_active;
+static char *qtldr_crash_target;
+
+#define TYPE_QTLDR_CONFIG "tldr-config"
+
+OBJECT_DECLARE_SIMPLE_TYPE(QTLDRConfig, QTLDR_CONFIG);
+
+OBJECT_DEFINE_SIMPLE_TYPE_WITH_INTERFACES(QTLDRConfig,
+                                          qtldr_config,
+                                          QTLDR_CONFIG,
+                                          OBJECT,
+                                          { TYPE_USER_CREATABLE },
+                                          { NULL });
+
+
+static void
+qtldr_config_finalize(Object *obj)
+{
+    QTLDRConfig *cfg = QTLDR_CONFIG(obj);
+
+    g_free(cfg->crash_target);
+}
+
+
+static char *
+qtldr_config_get_crash_target(Object *obj, Error **errp)
+{
+    QTLDRConfig *cfg = QTLDR_CONFIG(obj);
+    return g_strdup(cfg->crash_target);
+}
+
+
+static void
+qtldr_config_set_crash_target(Object *obj, const char *path, Error **errp)
+{
+    QTLDRConfig *cfg = QTLDR_CONFIG(obj);
+    g_free(cfg->crash_target);
+    cfg->crash_target = g_strdup(path);
+}
+
+
+static void
+qtldr_config_complete(UserCreatable *uc, Error **errp)
+{
+    QTLDRConfig *cfg = QTLDR_CONFIG(uc);
+
+    /*
+     * Forbid changes, since once configured, we can't safely
+     * change it due to signal handlers running without ability
+     * to hold mutex locks
+     */
+    if (qtldr_crash_target) {
+        error_setg(errp, "TLDR configuration cannot be changed");
+        return;
+    }
+
+    qatomic_store_release(&qtldr_crash_target, g_strdup(cfg->crash_target));
+    qatomic_inc(&qtldr_active);
+}
+
+
+static bool
+qtldr_config_prepare_delete(UserCreatable *uc, Error **errp)
+{
+    /*
+     * Forbid deletion, since once configured, we can't safely
+     * change it due to signal handlers running without ability
+     * to hold mutex locks
+     */
+    error_setg(errp, "Deleting the 'tldr-config' object is not supported");
+    return false;
+}
+
+
+static void
+qtldr_config_class_init(ObjectClass *klass, const void *opaque)
+{
+    UserCreatableClass *ucc = USER_CREATABLE_CLASS(klass);
+    object_class_property_add_str(klass, "crash-target",
+                                  qtldr_config_get_crash_target,
+                                  qtldr_config_set_crash_target);
+
+    ucc->complete = qtldr_config_complete;
+    ucc->prepare_delete = qtldr_config_prepare_delete;
+}
+
+
+static void
+qtldr_config_init(Object *obj)
+{
+}
+
+
+/*
+ * WARNING: Code below this marker is used when dumping
+ * WARNING: the data recorder buffers. It is subject to
+ * WARNING: execution directly in signal handler context
+ * WARNING: on a limited stack (SIGSTKSZ).
+ *
+ * WARNING: Thus from here onwards must only use async
+ * WARNING: signal safe POSIX functions.
+ */
+
+
+static void
+qtldr_write_int_buf(int fd, uint64_t v,
+                    const char *digits, size_t digitslen,
+                    char *target, size_t targetlen)
+{
+    size_t offset = targetlen;
+    do {
+        uint64_t r = (v % digitslen);
+        target[--offset] = digits[r];
+        v /= digitslen;
+    } while (v && offset);
+    ignore_value(write(fd, target + offset, targetlen - offset));
+}
+
+
+static void
+qtldr_write_int_dec(int fd, uint64_t v)
+{
+    static char digits[] = "0123456789";
+    char target[20];
+    qtldr_write_int_buf(fd, v, digits, strlen(digits),
+                        target, G_N_ELEMENTS(target));
+}
+
+
+static void
+qtldr_write_int_hex(int fd, uint64_t v)
+{
+    static const char digits[] = "0123456789abcdef";
+    char target[16];
+    qtldr_write_int_buf(fd, v, digits, strlen(digits),
+                        target, G_N_ELEMENTS(target));
+}
+
+
+static const char *
+qtldr_crash_dump_signal_str(int signo)
+{
+    switch (signo) {
+    case SIGSEGV:
+        return "SIGSEGV";
+#ifndef WIN32
+    case SIGBUS:
+        return "SIGBUS";
+#endif
+    case SIGABRT:
+        return "SIGABRT";
+    case SIGILL:
+        return "SIGILL";
+    case SIGFPE:
+        return "SIGFPE";
+    }
+    return NULL;
+}
+
+
+static const char *
+qtldr_crash_dump_code_str(int signo, int code)
+{
+    switch (code) {
+#ifdef SI_USER
+    case SI_USER:
+        return "User process kill";
+#endif
+#ifdef SI_KERNEL
+    case SI_KERNEL:
+        return "Kernel process kill";
+#endif
+#ifdef SI_QUEUE
+    case SI_QUEUE:
+        return "Signal queue";
+#endif
+#ifdef SI_TIMER
+    case SI_TIMER:
+        return "Timer expired";
+#endif
+#ifdef SI_MESGQ
+    case SI_MESGQ:
+        return "Message queue state changed";
+#endif
+#ifdef SI_ASYNCIO
+    case SI_ASYNCIO:
+        return "AIO completed";
+#endif
+#ifdef SI_TKILL
+    case SI_TKILL:
+        return "User thread kill";
+#endif
+    }
+    switch (signo) {
+    case SIGSEGV:
+        switch (code) {
+#ifdef SEGV_MAPERR
+        case SEGV_MAPERR:
+            return "Address not mapped";
+#endif
+#ifdef SEGV_ACCERR
+        case SEGV_ACCERR:
+            return "Access permissions";
+#endif
+#ifdef SEGV_BNDERR
+        case SEGV_BNDERR:
+            return "Address out of bounds";
+#endif
+#ifdef SEGV_PKUERR
+        case SEGV_PKUERR:
+            return "Memory protection keys";
+#endif
+        }
+        break;
+
+#ifndef WIN32
+    case SIGBUS:
+        switch (code) {
+#ifdef BUS_ADRALN
+        case BUS_ADRALN:
+            return "Address alignment";
+#endif
+#ifdef BUS_ADDRER
+        case BUS_ADRERR:
+            return "Physical address error";
+#endif
+#ifdef BUS_OBJERR
+        case BUS_OBJERR:
+            return "Object hardware error";
+#endif
+#ifdef BUS_MCEERR_AR
+        case BUS_MCEERR_AR:
+            return "Memory error, machine check";
+#endif
+#ifdef BUS_MCEERR_AO
+        case BUS_MCEERR_AO:
+            return "Memory error, not consumed";
+#endif
+        }
+        break;
+#endif
+
+    case SIGILL:
+        switch (code) {
+#ifdef ILL_ILLOPC
+        case ILL_ILLOPC:
+            return "Illegal opcode";
+#endif
+#ifdef ILL_ILLOPN
+        case ILL_ILLOPN:
+            return "Illegal operand";
+#endif
+#ifdef ILL_ILLADR
+        case ILL_ILLADR:
+            return "Illegal addressing mode";
+#endif
+#ifdef ILL_ILLTRP
+        case ILL_ILLTRP:
+            return "Illegal trap";
+#endif
+#ifdef ILL_PRVOPC
+        case ILL_PRVOPC:
+            return "Privileged opcode";
+#endif
+#ifdef ILL_PRVREG
+        case ILL_PRVREG:
+            return "Privileged register";
+#endif
+#ifdef ILL_COPROC
+        case ILL_COPROC:
+            return "Coprocessor error";
+#endif
+#ifdef ILL_BADSTK
+        case ILL_BADSTK:
+            return "Internal stack error";
+#endif
+        }
+        break;
+
+    case SIGFPE:
+        switch (code) {
+#ifdef FPE_INTDIV
+        case FPE_INTDIV:
+            return "Integer divide by zero";
+#endif
+#ifdef FPE_INTOVF
+        case FPE_INTOVF:
+            return "Integer overflow";
+#endif
+#ifdef FPE_FLTDIV
+        case FPE_FLTDIV:
+            return "Floating-point divide by zero";
+#endif
+#ifdef FPE_FLTOVF
+        case FPE_FLTOVF:
+            return "Floating-point overflow";
+#endif
+#ifdef FPE_FLTUND
+        case FPE_FLTUND:
+            return "Floating-point underflow";
+#endif
+#ifdef FPE_FLTRES
+        case FPE_FLTRES:
+            return "Floating-point inexact result";
+#endif
+#ifdef FPE_FLTINV
+        case FPE_FLTINV:
+            return "Floating-point invalid operation";
+#endif
+#ifdef FPE_FLTSUB
+        case FPE_FLTSUB:
+            return "Subscript out of range";
+#endif
+        }
+        break;
+    }
+    return NULL;
+}
+
+
+static void
+qtldr_crash_dump_print_field(int fd, const char *label,
+                             const char *valstr, int val)
+{
+    ignore_value(write(fd, label, strlen(label)));
+    if (valstr) {
+        ignore_value(write(fd, valstr, strlen(valstr)));
+        ignore_value(write(fd, " (", 2));
+    }
+    qtldr_write_int_dec(fd, val);
+    if (valstr) {
+        ignore_value(write(fd, ")", 1));
+    }
+    ignore_value(write(fd, "\n", 1));
+}
+
+
+static void
+qtldr_crash_dump_print_siginfo(int fd, siginfo_t *si)
+{
+    const char *signostr = qtldr_crash_dump_signal_str(si->si_signo);
+    const char *codestr = qtldr_crash_dump_code_str(si->si_signo, si->si_code);
+    ignore_value(write(fd, QTLDR_DUMP_TRIGGER_PREFIX,
+                       strlen(QTLDR_DUMP_TRIGGER_PREFIX)));
+    ignore_value(write(fd, "\n", 1));
+
+    qtldr_crash_dump_print_field(fd, "  - Signal: ", signostr, si->si_signo);
+    qtldr_crash_dump_print_field(fd, "  - Code: ", codestr, si->si_code);
+
+    ignore_value(write(fd, "  - Address: 0x", 15));
+    qtldr_write_int_hex(fd, (uint64_t)si->si_addr);
+    ignore_value(write(fd, "\n", 1));
+}
+
+
+void
+qtldr_crash_dump_internal(int fd, siginfo_t *si)
+{
+    if (fd == -1) {
+        fd = STDERR_FILENO;
+    }
+
+    ignore_value(write(fd, QTLDR_DUMP_LEADER, strlen(QTLDR_DUMP_LEADER)));
+    if (si) {
+        qtldr_crash_dump_print_siginfo(fd, si);
+    } else {
+        ignore_value(write(fd, QTLDR_DUMP_TRIGGER_PREFIX,
+                           strlen(QTLDR_DUMP_TRIGGER_PREFIX)));
+        ignore_value(write(fd, "manual\n", 7));
+    }
+
+    ignore_value(write(fd, QTLDR_DUMP_TRAILER, strlen(QTLDR_DUMP_TRAILER)));
+}
+
+
+void
+qtldr_crash_dump_signal(siginfo_t *si)
+{
+    int fd = STDERR_FILENO;
+    const char *tgt;
+
+    if (!qatomic_read(&qtldr_active)) {
+        return;
+    }
+
+    tgt = qatomic_load_acquire(&qtldr_crash_target);
+    if (tgt) {
+        /*
+         * Must NOT use qemu_create, as it is not safe from
+         * signal handler context due to using monitor APIs
+         */
+        fd = open(tgt, O_WRONLY | O_CREAT | O_TRUNC, 0600);
+        if (fd < 0) {
+            /*
+             * Couldn't open our designated crash report
+             * file location, so fallback to STDERR as
+             * the least worst next best option, and
+             * warn the user of the failure
+             */
+            fd = STDERR_FILENO;
+            int saved_errno = errno;
+            ignore_value(write(fd, QTLDR_DUMP_OPEN_ERROR,
+                               strlen(QTLDR_DUMP_OPEN_ERROR)));
+            ignore_value(write(fd, tgt, strlen(tgt)));
+            ignore_value(write(fd, "': ", 3));
+            qtldr_write_int_dec(fd, saved_errno);
+            ignore_value(write(fd, "\n", 1));
+        }
+    }
+
+    qtldr_crash_dump_internal(fd, si);
+
+    if (fd != STDERR_FILENO) {
+        qemu_fdatasync(fd);
+        close(fd);
+    }
+}
-- 
2.56.0

[PATCH 08/21] qapi: add 'tldr-dump' command for on-demand data recorder output

[Daniel P. Berrangé]

While the data recorder is designed to dump its contents in
response to various signals that result in a QEMU crash, it
is also interesting to explicitly request dump.

This could be simply as an adhoc debugging activity, or could
be in response to a specific event like a guest OS panic
notification.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 MAINTAINERS           |  1 +
 qapi/meson.build      |  1 +
 qapi/qapi-schema.json |  1 +
 qapi/tldr.json        | 28 ++++++++++++++++++++++++++++
 qemu-options.hx       |  2 ++
 util/tldr.c           | 32 ++++++++++++++++++++++++++++++++
 6 files changed, 65 insertions(+)
 create mode 100644 qapi/tldr.json

diff --git a/MAINTAINERS b/MAINTAINERS
index 4fa68810c1e..54265d987cb 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -4780,3 +4780,4 @@ F: util/tldr.c
 F: util/tldr-internal.h
 F: include/qemu/tldr.h
 F: tests/unit/test-util-tldr.c
+F: qapi/tldr.json
diff --git a/qapi/meson.build b/qapi/meson.build
index a46269b5a0c..f8aecb61284 100644
--- a/qapi/meson.build
+++ b/qapi/meson.build
@@ -49,6 +49,7 @@ qapi_all_modules = [
   'run-state',
   'sockets',
   'stats',
+  'tldr',
   'trace',
   'transaction',
   'vfio',
diff --git a/qapi/qapi-schema.json b/qapi/qapi-schema.json
index b93dd68d94c..b36777970d1 100644
--- a/qapi/qapi-schema.json
+++ b/qapi/qapi-schema.json
@@ -76,3 +76,4 @@
 { 'include': 'cryptodev.json' }
 { 'include': 'cxl.json' }
 { 'include': 'uefi.json' }
+{ 'include': 'tldr.json' }
diff --git a/qapi/tldr.json b/qapi/tldr.json
new file mode 100644
index 00000000000..8e2b7cda324
--- /dev/null
+++ b/qapi/tldr.json
@@ -0,0 +1,28 @@
+# -*- Mode: Python -*-
+# vim: filetype=python
+# SPDX-License-Identifier: GPL-2.0-or-later
+
+##
+# @QTLDRDumpOptions:
+#     Options for `tldr-dump`.
+#
+# @target: the fully qualified file path to dump
+#     the data recorder to
+#
+# Since: 11.2.0
+##
+{ 'struct': 'QTLDRDumpOptions',
+  'data': { 'target': 'str' } }
+
+##
+# @tldr-dump:
+#     Dump the data recorder to a file
+#
+# Since: 11.2.0
+#
+# .. qmp-example::
+#
+#     -> { "execute": "tldr-dump", "arguments": { "target": "/tmp/qemu.txt" } }
+#     <- { "return": {} }
+##
+{ 'command': 'tldr-dump', 'data': 'QTLDRDumpOptions' }
diff --git a/qemu-options.hx b/qemu-options.hx
index 2f63d86b635..5f96257cef0 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -6692,6 +6692,8 @@ SRST
         recorder will be dumped to the stderr stream. If the
         ``crash-target`` parameter was given, that specifies a fully
         qualified file path to write the dump to instead of stderr.
+        The ``tldr-dump`` QMP command can also be used at any time to
+        save the captured data to an arbitrary file.
 
         The ``tldr-config`` object can be added at initial startup
         or via QMP at runtime. Once created, however, it is not
diff --git a/util/tldr.c b/util/tldr.c
index 13e9881bc9b..ea7c26b8e9b 100644
--- a/util/tldr.c
+++ b/util/tldr.c
@@ -12,6 +12,7 @@
 #include "qom/object.h"
 #include "qom/object_interfaces.h"
 #include "qapi/error.h"
+#include "qapi/qapi-commands-tldr.h"
 
 struct QTLDRConfig {
     Object parent;
@@ -436,3 +437,34 @@ qtldr_crash_dump_signal(siginfo_t *si)
         close(fd);
     }
 }
+
+
+/*
+ * WARNING: Code below this marker provides a QMP handler
+ * WARNING: for use in normal execution. It calls into
+ * WARNING: the common crash report outputting APIs, but
+ * WARNING: this specific method is not restricted to async
+ * WARNING: signal safe POSIX functions.
+ */
+
+
+void
+qmp_tldr_dump(const char *target, Error **errp)
+{
+    int fd;
+
+    if (!qatomic_read(&qtldr_active)) {
+        error_setg(errp, "No 'tldr-config' object has been creted");
+        return;
+    }
+
+    fd = qemu_create(target, O_WRONLY | O_TRUNC, 0600, errp);
+    if (fd < 0) {
+        return;
+    }
+
+    qtldr_crash_dump_internal(fd, NULL);
+
+    qemu_fdatasync(fd);
+    close(fd);
+}
-- 
2.56.0

[PATCH 09/21] system: trigger TLDR crash dump on unhandled SIGBUS

[Daniel P. Berrangé]

QEMU currently installs a SIGBUS handler which either forwards MCEs
to KVM, or re-raises the SIGBUS. Leave MCE handling untouched, but
wire up other to trigger the TLDR crash dump in the re-raise scenario.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 system/signal.c | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/system/signal.c b/system/signal.c
index dc5a75e2816..f667b56255d 100644
--- a/system/signal.c
+++ b/system/signal.c
@@ -27,6 +27,7 @@
 #include "hw/core/cpu.h"
 #include "system/kvm.h"
 #include "qemu/error-report.h"
+#include "qemu/tldr.h"
 
 #ifdef CONFIG_LINUX
 
@@ -48,7 +49,7 @@
 
 
 #ifdef CONFIG_LINUX
-static void sigbus_reraise(void)
+static void sigbus_reraise(siginfo_t *siginfo)
 {
     sigset_t set;
     struct sigaction action;
@@ -56,6 +57,7 @@ static void sigbus_reraise(void)
     memset(&action, 0, sizeof(action));
     action.sa_handler = SIG_DFL;
     if (!sigaction(SIGBUS, &action, NULL)) {
+        qtldr_crash_dump_signal(siginfo);
         raise(SIGBUS);
         sigemptyset(&set);
         sigaddset(&set, SIGBUS);
@@ -69,19 +71,19 @@ static void sigbus_handler(int n, siginfo_t *siginfo, void *ctx)
 {
     if (siginfo->si_code != BUS_MCEERR_AO &&
         siginfo->si_code != BUS_MCEERR_AR) {
-        sigbus_reraise();
+        sigbus_reraise(siginfo);
     }
 
     if (current_cpu) {
         /* Called asynchronously in VCPU thread.  */
         if (kvm_on_sigbus_vcpu(current_cpu, siginfo->si_code,
                                siginfo->si_addr)) {
-            sigbus_reraise();
+            sigbus_reraise(siginfo);
         }
     } else {
         /* Called synchronously (via signalfd) in main thread.  */
         if (kvm_on_sigbus(siginfo->si_code, siginfo->si_addr)) {
-            sigbus_reraise();
+            sigbus_reraise(siginfo);
         }
     }
 }
-- 
2.56.0

[PATCH 10/21] system: simplify signal handler logic

[Daniel P. Berrangé]

sigaction() can fail if

 * The signal number is invalid
 * The signal number is KILL / STOP
 * The sigaction struct points to invalid memory

None of these scenarios can arise with our use of sigaction(), so
rather than try to handle failures, assert that the call passed.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 system/signal.c | 15 ++++++---------
 1 file changed, 6 insertions(+), 9 deletions(-)

diff --git a/system/signal.c b/system/signal.c
index f667b56255d..88316089fc0 100644
--- a/system/signal.c
+++ b/system/signal.c
@@ -56,15 +56,12 @@ static void sigbus_reraise(siginfo_t *siginfo)
 
     memset(&action, 0, sizeof(action));
     action.sa_handler = SIG_DFL;
-    if (!sigaction(SIGBUS, &action, NULL)) {
-        qtldr_crash_dump_signal(siginfo);
-        raise(SIGBUS);
-        sigemptyset(&set);
-        sigaddset(&set, SIGBUS);
-        pthread_sigmask(SIG_UNBLOCK, &set, NULL);
-    }
-    perror("Failed to re-raise SIGBUS!");
-    abort();
+    assert(sigaction(SIGBUS, &action, NULL) == 0);
+    qtldr_crash_dump_signal(siginfo);
+    raise(SIGBUS);
+    sigemptyset(&set);
+    sigaddset(&set, SIGBUS);
+    pthread_sigmask(SIG_UNBLOCK, &set, NULL);
 }
 
 static void sigbus_handler(int n, siginfo_t *siginfo, void *ctx)
-- 
2.56.0

[PATCH 11/21] main: allow SIGBUS in the main loop

[Daniel P. Berrangé]

The data recorder wants to dump in response to SIGBUS in any thread,
so it must be unblocked in the main loop.

XXX: this restriction appears to have been something historical
related to the MCE handling to force MCEs out of the main thread,
however, the signal handler appears to be able to matchup to vCPU
threads regardless. Find someone with more knowledge to sanity
check this change, as we really want to catch BUS errors from all
threads, not merely vCPU threads.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 util/main-loop.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/util/main-loop.c b/util/main-loop.c
index 65e5dbcc941..8d531f5ec90 100644
--- a/util/main-loop.c
+++ b/util/main-loop.c
@@ -99,7 +99,6 @@ static int qemu_signal_init(Error **errp)
     sigaddset(&set, SIG_IPI);
     sigaddset(&set, SIGIO);
     sigaddset(&set, SIGALRM);
-    sigaddset(&set, SIGBUS);
     /* SIGINT cannot be handled via signalfd, so that ^C can be used
      * to interrupt QEMU when it is being run under gdb.  SIGHUP and
      * SIGTERM are also handled asynchronously, even though it is not
-- 
2.56.0

[PATCH 12/21] system: enable SIGBUS handling on non-Linux platforms

[Daniel P. Berrangé]

The MCE handling code is Linux/KVM specific, however, the handling
of SIGBUS and crash dumps can work on any platform supporting the
sigaction() function. The current function for re-raising SIGBUS
can be used as the sigation handler on non-Linux by just adding
the extra args.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 meson.build     |  1 +
 system/signal.c | 22 +++++++++++++++-------
 2 files changed, 16 insertions(+), 7 deletions(-)

diff --git a/meson.build b/meson.build
index c6207443365..c96c8884f51 100644
--- a/meson.build
+++ b/meson.build
@@ -2648,6 +2648,7 @@ config_host_data.set('HAVE_OPENPTY', cc.has_function('openpty', dependencies: ut
 config_host_data.set('HAVE_STRCHRNUL', cc.has_function('strchrnul', prefix: osdep_prefix))
 config_host_data.set('HAVE_SYSTEM_FUNCTION', cc.has_function('system', prefix: '#include <stdlib.h>'))
 config_host_data.set('CONFIG_SIGALTSTACK', cc.has_function('sigaltstack'))
+config_host_data.set('CONFIG_SIGACTION', cc.has_function('sigaction'))
 if rbd.found()
   config_host_data.set('HAVE_RBD_NAMESPACE_EXISTS',
                        cc.has_function('rbd_namespace_exists',
diff --git a/system/signal.c b/system/signal.c
index 88316089fc0..615704900bc 100644
--- a/system/signal.c
+++ b/system/signal.c
@@ -48,8 +48,8 @@
 #endif /* CONFIG_LINUX */
 
 
-#ifdef CONFIG_LINUX
-static void sigbus_reraise(siginfo_t *siginfo)
+#ifdef CONFIG_SIGACTION
+static void sig_handler(int n, siginfo_t *siginfo, void *ctx)
 {
     sigset_t set;
     struct sigaction action;
@@ -64,26 +64,28 @@ static void sigbus_reraise(siginfo_t *siginfo)
     pthread_sigmask(SIG_UNBLOCK, &set, NULL);
 }
 
+#ifdef CONFIG_LINUX
 static void sigbus_handler(int n, siginfo_t *siginfo, void *ctx)
 {
     if (siginfo->si_code != BUS_MCEERR_AO &&
         siginfo->si_code != BUS_MCEERR_AR) {
-        sigbus_reraise(siginfo);
+        sig_handler(n, siginfo, ctx);
     }
 
     if (current_cpu) {
         /* Called asynchronously in VCPU thread.  */
         if (kvm_on_sigbus_vcpu(current_cpu, siginfo->si_code,
                                siginfo->si_addr)) {
-            sigbus_reraise(siginfo);
+            sig_handler(n, siginfo, ctx);
         }
     } else {
         /* Called synchronously (via signalfd) in main thread.  */
         if (kvm_on_sigbus(siginfo->si_code, siginfo->si_addr)) {
-            sigbus_reraise(siginfo);
+            sig_handler(n, siginfo, ctx);
         }
     }
 }
+#endif
 
 void qemu_init_crash_signals(void)
 {
@@ -106,13 +108,19 @@ void qemu_init_crash_signals(void)
      */
     memset(&action, 0, sizeof(action));
     action.sa_flags = SA_SIGINFO | SA_ONSTACK;
+#ifdef CONFIG_LINUX
     action.sa_sigaction = sigbus_handler;
+#else
+    action.sa_sigaction = sig_handler;
+#endif
     sigaction(SIGBUS, &action, NULL);
 
+#ifdef CONFIG_LINUX
     prctl(PR_MCE_KILL, PR_MCE_KILL_SET, PR_MCE_KILL_EARLY, 0, 0);
+#endif
 }
-#else /* !CONFIG_LINUX */
+#else /* !CONFIG_SIGACTION */
 void qemu_init_crash_signals(void)
 {
 }
-#endif /* !CONFIG_LINUX */
+#endif /* !CONFIG_SIGACTION */
-- 
2.56.0

[PATCH 13/21] system: enable crash dumps on SIGSEGV, SIGILL, SIGFPE and SIGABRT

[Daniel P. Berrangé]

Expand the current SIGBUS crash dumps onto further signals that
represent program crashes from bad code or guest OS behaviour.
This change covers Windows platforms too, by using the dummy
siginfo struct definition from osdeup.h.

Note that the signal handler gets unregistered before trying the
crash dump, so in the unlikely event that crash dump itself crashes,
QEMU will exit, rather than getting into a loop.

In a debugger we'll still see the full stack history of the original
flaw, just with one extra frame present for the signal handler.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 system/signal.c | 33 ++++++++++++++++++++++++++++-----
 1 file changed, 28 insertions(+), 5 deletions(-)

diff --git a/system/signal.c b/system/signal.c
index 615704900bc..4568ded4cdd 100644
--- a/system/signal.c
+++ b/system/signal.c
@@ -56,11 +56,11 @@ static void sig_handler(int n, siginfo_t *siginfo, void *ctx)
 
     memset(&action, 0, sizeof(action));
     action.sa_handler = SIG_DFL;
-    assert(sigaction(SIGBUS, &action, NULL) == 0);
+    assert(sigaction(siginfo->si_signo, &action, NULL) == 0);
     qtldr_crash_dump_signal(siginfo);
-    raise(SIGBUS);
+    raise(siginfo->si_signo);
     sigemptyset(&set);
-    sigaddset(&set, SIGBUS);
+    sigaddset(&set, siginfo->si_signo);
     pthread_sigmask(SIG_UNBLOCK, &set, NULL);
 }
 
@@ -108,10 +108,13 @@ void qemu_init_crash_signals(void)
      */
     memset(&action, 0, sizeof(action));
     action.sa_flags = SA_SIGINFO | SA_ONSTACK;
+    action.sa_sigaction = sig_handler;
+    sigaction(SIGSEGV, &action, NULL);
+    sigaction(SIGABRT, &action, NULL);
+    sigaction(SIGILL, &action, NULL);
+    sigaction(SIGFPE, &action, NULL);
 #ifdef CONFIG_LINUX
     action.sa_sigaction = sigbus_handler;
-#else
-    action.sa_sigaction = sig_handler;
 #endif
     sigaction(SIGBUS, &action, NULL);
 
@@ -120,7 +123,27 @@ void qemu_init_crash_signals(void)
 #endif
 }
 #else /* !CONFIG_SIGACTION */
+
+static void sig_handler(int signo)
+{
+    siginfo_t si = {
+        .si_signo = signo,
+        .si_code = 0,
+        .si_addr = 0x0,
+    };
+    assert(signal(signo, SIG_DFL) == 0);
+    qtldr_crash_dump_signal(&si);
+    raise(signo);
+}
+
 void qemu_init_crash_signals(void)
 {
+#ifndef WIN32
+    signal(SIGBUS, sig_handler);
+#endif
+    signal(SIGSEGV, sig_handler);
+    signal(SIGABRT, sig_handler);
+    signal(SIGILL, sig_handler);
+    signal(SIGFPE, sig_handler);
 }
 #endif /* !CONFIG_SIGACTION */
-- 
2.56.0

[PATCH 14/21] util: introduce capture of textual messages in data recorder

[Daniel P. Berrangé]

Introduce a circular buffer concept to the data recorder that can be
used to capture text based messages from subsystems. Support for
multiple distinct "buckets" is provided, one per type of information
to be captured. This ensures that data from subsystems which is
captured at a high rate, does not prematurely wrap the buffer and
overwrite (still relevant) messages from a subsystem that records
at a low rate.

Each bucket has a max capacity of 64 KB by default (though it is
configurable), but grows on demand. IOW the bucket capacity puts
an upper limit on memory usage, actual usage will be zero until
data is recorded. Every captured message will be prefixed by a
timestamp and suffixed by a newline, if one is not already present
at the end of the message.

While writing into the bufers is synchronized with mutex locks,
when dumping from a signal handler the buffers are accessed
without any locking to avoid deadlocks. Atomic loads/stores are
used in certain places so the signal handler can dump data with
acceptable accuracy even when racing with other threads still
writing new messages.

When the circular buffer wraps, old messages will be overwritten
without attention to message boundaries. IOW, when later dumping
the data recorder, the first printed message is likely to have
its leading text and timestamp missing. This is considered an
acceptable tradeoff to avoid tracking message boundaries which
greatly adds complexity and is hard todo safely without locks
when dumping.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 include/qemu/tldr.h         |  52 +++++++
 qapi/qom.json               |   5 +-
 qapi/tldr.json              |  34 +++++
 qemu-options.hx             |   8 +
 tests/unit/test-util-tldr.c | 272 ++++++++++++++++++++++++++++++++++
 util/tldr-internal.h        |  31 ++++
 util/tldr.c                 | 282 +++++++++++++++++++++++++++++++++++-
 7 files changed, 682 insertions(+), 2 deletions(-)

diff --git a/include/qemu/tldr.h b/include/qemu/tldr.h
index 67152b136b3..5766acecb64 100644
--- a/include/qemu/tldr.h
+++ b/include/qemu/tldr.h
@@ -9,6 +9,58 @@
 #ifndef QEMU_TLDR_H__
 #define QEMU_TLDR_H__
 
+#include "qapi/qapi-types-tldr.h"
+
+extern unsigned int qtldr_msg_enabled_mask;
+
+/**
+ * qtldr_msg_enabled:
+ * @bucket: bucket to check status for
+ *
+ * Returns: true if @bucket is enabled for recording
+ */
+#define qtldr_msg_enabled(bucket) \
+    (qtldr_msg_enabled_mask & (1 << bucket))
+
+/**
+ * qtldr_msg_record:
+ * @bucket: bucket to record into
+ * @msg: the message text to record
+ *
+ * Records @msg into @bucket, potentially overwriting
+ * old messages if the capacity is at its maximum
+ */
+void qtldr_msg_record(QTLDRMsgBucketType bucket,
+                      const char *msg);
+
+/**
+ * qtldr_msg_record_printf:
+ * @bucket: bucket to record into
+ * @fmt: the message format string to record
+ * @...: the arguments for @fmt
+ *
+ * Formats @fmt as a printf string and records the
+ * result into @bucket, potentially overwriting
+ * old messages if the capacity is at its maximum
+ */
+void qtldr_msg_record_printf(QTLDRMsgBucketType bucket,
+                             const char *fmt, ...)
+    G_GNUC_PRINTF(2, 3);
+
+/**
+ * qtldr_msg_record_vprintf:
+ * @bucket: bucket to record into
+ * @fmt: the message format string to record
+ * @ap: the arguments for @fmt
+ *
+ * Formats @fmt as a printf string and records the
+ * result into @bucket, potentially overwriting
+ * old messages if the capacity is at its maximum
+ */
+void qtldr_msg_record_vprintf(QTLDRMsgBucketType bucket,
+                              const char *fmt, va_list ap)
+    G_GNUC_PRINTF(2, 0);
+
 /**
  * qtldr_crash_dump_signal:
  * @si: signal information
diff --git a/qapi/qom.json b/qapi/qom.json
index 1c9031fba9a..f1c37bce6c2 100644
--- a/qapi/qom.json
+++ b/qapi/qom.json
@@ -8,6 +8,7 @@
 { 'include': 'block-core.json' }
 { 'include': 'common.json' }
 { 'include': 'crypto.json' }
+{ 'include': 'tldr.json' }
 
 ##
 # ***********************
@@ -1239,11 +1240,13 @@
 #
 # @crash-target: fully qualified path to perserve data recorder
 #                contents on QEMU crash
+# @messages: custom configuration for message buckets
 #
 # Since: 11.2.0
 ##
 { 'struct': 'QTLDRConfigProperties',
-  'data': { '*crash-target': 'str' }
+  'data': { '*crash-target': 'str',
+            '*messages': ['QTLDRMsgBucketConfig'] }
 }
 
 ##
diff --git a/qapi/tldr.json b/qapi/tldr.json
index 8e2b7cda324..987d36b91f8 100644
--- a/qapi/tldr.json
+++ b/qapi/tldr.json
@@ -26,3 +26,37 @@
 #     <- { "return": {} }
 ##
 { 'command': 'tldr-dump', 'data': 'QTLDRDumpOptions' }
+
+##
+# @QTLDRMsgBucketType:
+#     List of message bucket types
+#
+# @error: error messages and warnings
+# @command: QMP commands which change state and error responses
+# @event: QMP events reflecting state changes
+# @log: log messages from any subsystems
+#
+# Since: 11.2.0
+##
+{ 'enum': 'QTLDRMsgBucketType',
+  'data': ['error', 'command', 'event', 'log' ] }
+
+
+##
+# @QTLDRMsgBucketConfig:
+#     Configuration for message buckets
+#
+# @type: the message bucket to enable
+# @capacity: override the default bucket size, in bytes (64kb)
+#
+# Each bucket allocates memory on demand, so capacity merely
+# puts an upper limit on the memory usage in a bucket.  The
+# capacity must be larger than the per-message size limit,
+# which is 2048 bytes.
+#
+# Since: 11.2.0
+##
+{ 'struct': 'QTLDRMsgBucketConfig',
+  'data': { 'type': 'QTLDRMsgBucketType',
+            '*capacity': 'size' }
+}
diff --git a/qemu-options.hx b/qemu-options.hx
index 5f96257cef0..da9d1e2b7bd 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -6685,6 +6685,8 @@ SRST
 
 
     ``-object tldr-config,id=id,crash-target=path``
+        \
+    ``-object "{'qom-type':'tldr-config','id':'tldr','crash-target':'foo','messages':[{'type':'BUCKET','capacity':BYTES}]}"``
         Creating the ``tldr-config`` object enables the thrifty live data
         recorder, which continuously captures important information about
         the operation of the QEMU process. When a SIGSEGV, SIGBUS,
@@ -6695,6 +6697,12 @@ SRST
         The ``tldr-dump`` QMP command can also be used at any time to
         save the captured data to an arbitrary file.
 
+        The data recorder will capture upto 64kb of data by default in
+        each bucket, but when using the JSON syntax for ``-object``, this
+        can be tailored with the ``messages`` parameter. ``BUCKET`` takes
+        one of ``error``, ``command``, ``event``, or ``log`` and the
+        capacity value is in bytes.
+
         The ``tldr-config`` object can be added at initial startup
         or via QMP at runtime. Once created, however, it is not
         permitted to disable it.
diff --git a/tests/unit/test-util-tldr.c b/tests/unit/test-util-tldr.c
index 4384e10cc18..42b4e350540 100644
--- a/tests/unit/test-util-tldr.c
+++ b/tests/unit/test-util-tldr.c
@@ -9,11 +9,20 @@
 #include "qemu/osdep.h"
 #include "util/tldr-internal.h"
 
+typedef struct {
+    size_t next;
+    const char *msg;
+} QTLDRTestStep;
+
 typedef struct {
     bool siginfo;
+    size_t capacity;
+    QTLDRTestStep *steps;
     const char *data;
 } QTLDRTestData;
 
+static int timestampIndex;
+
 #define TEST_SIGINFO_NULL \
     QTLDR_DUMP_TRIGGER_PREFIX "manual\n"
 
@@ -23,15 +32,65 @@ typedef struct {
     "  - Code: Address not mapped (1)\n" \
     "  - Address: 0x87539319\n"
 
+/*
+ * Total message length including timestamp, \f, record len,
+ * message and extra newline will be 58 bytes
+ */
+#define TEST_MSG_1_TRUNC "mnopqrstuvwxyz\n"
+#define TEST_MSG_1 "abcdefghijkl" TEST_MSG_1_TRUNC
+#define TEST_MSG_2 "0123456789helloworld!!@@##"
+#define TEST_MSG_3 ">$><%><@@@>[*&^()]!!<<<<()\n"
+#define TEST_TIMESTAMP_1_TRUNC "026-10-05T16:35:40.601961Z"
+#define TEST_TIMESTAMP_1 "2" TEST_TIMESTAMP_1_TRUNC
+#define TEST_TIMESTAMP_2_TRUNC "T16:38:40.601961Z"
+#define TEST_TIMESTAMP_2 "2026-10-05" TEST_TIMESTAMP_2_TRUNC
+#define TEST_TIMESTAMP_3 "2026-10-05T16:40:40.601961Z"
+
+
+static char *
+test_tldr_now_str_func(void)
+{
+    static const char *timestamps[] = {
+        TEST_TIMESTAMP_1,
+        TEST_TIMESTAMP_2,
+        TEST_TIMESTAMP_3,
+    };
+    timestampIndex = timestampIndex % 3;
+    return g_strdup(timestamps[timestampIndex++]);
+}
+
 
 static void
 test_tldr_one(QTLDRTestData *data)
 {
+    QTLDRMsgBucket *bucket = &(qtldr_msg_buckets[QTLDR_MSG_BUCKET_TYPE_ERROR]);
+    size_t i;
     g_autofree char *tmpname = NULL;
     int tmpfd;
     char buf[1024];
     ssize_t dumplen;
 
+    qtldr_msg_enabled_mask = (1 << QTLDR_MSG_BUCKET_TYPE_ERROR);
+
+    if (data->capacity && data->steps) {
+        timestampIndex = 0;
+
+        g_clear_pointer(&bucket->buf, g_free);
+        bucket->allocated = 0;
+        bucket->capacity = data->capacity;
+        bucket->next = 0;
+
+        for (i = 0; data->steps[i].msg; i++) {
+            qtldr_msg_record(QTLDR_MSG_BUCKET_TYPE_ERROR,
+                             data->steps[i].msg);
+
+            assert(bucket->next == data->steps[i].next);
+            assert(bucket->allocated == data->capacity);
+            assert(bucket->capacity == data->capacity);
+            assert(bucket->buf != NULL);
+        }
+    }
+
     tmpfd = g_file_open_tmp("test-util-tldr-XXXXXX",
                             &tmpname, NULL);
     unlink(tmpname);
@@ -92,14 +151,227 @@ test_tldr_minimal_siginfo(void)
 }
 
 
+/*
+ * A single message which easily fits in the buffer
+ * to sanity check minimal functionality
+ */
+static void
+test_tldr_msgs_simple(void)
+{
+    QTLDRTestData data = {
+        .capacity = 70,
+        .steps = (QTLDRTestStep[]){
+            { .next = 55, .msg = TEST_MSG_1 },
+            { .next = -1, .msg = NULL },
+        },
+        .data =
+            QTLDR_DUMP_LEADER
+            TEST_SIGINFO_NULL
+            QTLDR_DUMP_MSG_BUCKET_PREFIX "error':\n"
+            TEST_TIMESTAMP_1 " " TEST_MSG_1
+            QTLDR_DUMP_TRAILER
+    };
+
+    test_tldr_one(&data);
+}
+
+
+/*
+ * A pair of messages which exactly hit the buffer
+ * size to sanity check we wrap back to start on
+ * the limit.
+ */
+static void
+test_tldr_msgs_exact(void)
+{
+    QTLDRTestData data = {
+        .capacity = 110,
+        .steps = (QTLDRTestStep[]){
+            { .next = 55, .msg = TEST_MSG_1 },
+            { .next = 0, .msg = TEST_MSG_2 },
+            { .next = -1, .msg = NULL },
+        },
+        .data =
+            QTLDR_DUMP_LEADER
+            TEST_SIGINFO_NULL
+            QTLDR_DUMP_MSG_BUCKET_PREFIX "error':\n"
+            TEST_TIMESTAMP_1 " " TEST_MSG_1
+            TEST_TIMESTAMP_2 " " TEST_MSG_2 "\n"
+            QTLDR_DUMP_TRAILER
+    };
+
+    test_tldr_one(&data);
+}
+
+
+/*
+ * A pair of messages which easily exceed the buffer
+ * size to sanity check we wrap and overwrite existing
+ * old data
+ */
+static void
+test_tldr_msgs_wrap_simple(void)
+{
+    QTLDRTestData data = {
+        .capacity = 70,
+        .steps = (QTLDRTestStep[]){
+            { .next = 55, .msg = TEST_MSG_1 },
+            { .next = 40, .msg = TEST_MSG_2 },
+            { .next = -1, .msg = NULL },
+        },
+        .data =
+            QTLDR_DUMP_LEADER
+            TEST_SIGINFO_NULL
+            QTLDR_DUMP_MSG_BUCKET_PREFIX "error':\n"
+            TEST_MSG_1_TRUNC
+            TEST_TIMESTAMP_2 " " TEST_MSG_2 "\n"
+            QTLDR_DUMP_TRAILER
+    };
+
+    test_tldr_one(&data);
+}
+
+
+/*
+ * A pair of messages which both exactly match the
+ * buffer size to further sanity check wrapping
+ * behaviour
+ */
+static void
+test_tldr_msgs_wrap_exact(void)
+{
+    QTLDRTestData data = {
+        .capacity = 55,
+        .steps = (QTLDRTestStep[]){
+            { .next = 0, .msg = TEST_MSG_1 },
+            { .next = 0, .msg = TEST_MSG_2 },
+            { .next = -1, .msg = NULL },
+        },
+        .data =
+            QTLDR_DUMP_LEADER
+            TEST_SIGINFO_NULL
+            QTLDR_DUMP_MSG_BUCKET_PREFIX "error':\n"
+            TEST_TIMESTAMP_2 " " TEST_MSG_2 "\n"
+            QTLDR_DUMP_TRAILER
+    };
+
+    test_tldr_one(&data);
+}
+
+
+/*
+ * A set of messages where first two are one less
+ * than the buffer size, to further sanity check
+ * wrapping behaviour
+ */
+static void
+test_tldr_msgs_wrap_oneextra(void)
+{
+    QTLDRTestData data = {
+        .capacity = 111,
+        .steps = (QTLDRTestStep[]){
+            { .next = 55, .msg = TEST_MSG_1 },
+            { .next = 110, .msg = TEST_MSG_2 },
+            { .next = 54, .msg = TEST_MSG_3 },
+            { .next = -1, .msg = NULL },
+        },
+        .data =
+            QTLDR_DUMP_LEADER
+            TEST_SIGINFO_NULL
+            QTLDR_DUMP_MSG_BUCKET_PREFIX "error':\n"
+            "\n"
+            TEST_TIMESTAMP_2 " " TEST_MSG_2 "\n"
+            TEST_TIMESTAMP_3 " " TEST_MSG_3
+            QTLDR_DUMP_TRAILER
+    };
+
+    test_tldr_one(&data);
+}
+
+
+/*
+ * A set of messages where first two are one larger
+ * than the buffer size, to further sanity check
+ * wrapping behaviour
+ */
+static void
+test_tldr_msgs_wrap_oneunder(void)
+{
+    QTLDRTestData data = {
+        .capacity = 109,
+        .steps = (QTLDRTestStep[]){
+            { .next = 55, .msg = TEST_MSG_1 },
+            { .next = 1, .msg = TEST_MSG_2 },
+            { .next = -1, .msg = NULL },
+        },
+        .data =
+            QTLDR_DUMP_LEADER
+            TEST_SIGINFO_NULL
+            QTLDR_DUMP_MSG_BUCKET_PREFIX "error':\n"
+            TEST_TIMESTAMP_1_TRUNC " " TEST_MSG_1
+            TEST_TIMESTAMP_2 " " TEST_MSG_2 "\n"
+            QTLDR_DUMP_TRAILER
+    };
+
+    test_tldr_one(&data);
+}
+
+
+/*
+ * A set of messages that make us wrap around
+ * many times over.
+ */
+static void
+test_tldr_msgs_wrap_repeat(void)
+{
+    QTLDRTestData data = {
+        .capacity = 100,
+        .steps = (QTLDRTestStep[]){
+            { .next = 55, .msg = TEST_MSG_1 },
+            { .next = 10, .msg = TEST_MSG_2 },
+            { .next = 65, .msg = TEST_MSG_3 },
+            { .next = 20, .msg = TEST_MSG_1 },
+            { .next = 75, .msg = TEST_MSG_2 },
+            { .next = 30, .msg = TEST_MSG_3 },
+            { .next = -1, .msg = NULL },
+        },
+        .data =
+            QTLDR_DUMP_LEADER
+            TEST_SIGINFO_NULL
+            QTLDR_DUMP_MSG_BUCKET_PREFIX "error':\n"
+            TEST_TIMESTAMP_2_TRUNC " " TEST_MSG_2 "\n"
+            TEST_TIMESTAMP_3 " " TEST_MSG_3
+            QTLDR_DUMP_TRAILER
+    };
+
+    test_tldr_one(&data);
+}
+
+
 int main(int argc, char **argv)
 {
     g_test_init(&argc, &argv, NULL);
 
+    qtldr_now_str_func = test_tldr_now_str_func;
+
     g_test_add_func("/util/tldr/minimal/default",
                     test_tldr_minimal_default);
     g_test_add_func("/util/tldr/minimal/siginfo",
                     test_tldr_minimal_siginfo);
+    g_test_add_func("/util/tldr/msgs/simple",
+                    test_tldr_msgs_simple);
+    g_test_add_func("/util/tldr/msgs/exact",
+                    test_tldr_msgs_exact);
+    g_test_add_func("/util/tldr/msgs/wrap/simple",
+                    test_tldr_msgs_wrap_simple);
+    g_test_add_func("/util/tldr/msgs/wrap/exact",
+                    test_tldr_msgs_wrap_exact);
+    g_test_add_func("/util/tldr/msgs/wrap/oneextra",
+                    test_tldr_msgs_wrap_oneextra);
+    g_test_add_func("/util/tldr/msgs/wrap/oneunder",
+                    test_tldr_msgs_wrap_oneunder);
+    g_test_add_func("/util/tldr/msgs/wrap/repeat",
+                    test_tldr_msgs_wrap_repeat);
 
     return g_test_run();
 }
diff --git a/util/tldr-internal.h b/util/tldr-internal.h
index 193d8425eee..f523565db85 100644
--- a/util/tldr-internal.h
+++ b/util/tldr-internal.h
@@ -14,8 +14,39 @@
 #define QTLDR_DUMP_LEADER "\n🚨🚨🚨 QEMU Data Recorder Start 🚨🚨🚨\n"
 #define QTLDR_DUMP_TRAILER "\n🚨🚨🚨 QEMU Data Recorder End 🚨🚨🚨\n\n"
 #define QTLDR_DUMP_TRIGGER_PREFIX "\n🔍 Trigger:"
+#define QTLDR_DUMP_MSG_BUCKET_PREFIX "\n🎥  Message recording '"
 #define QTLDR_DUMP_OPEN_ERROR "🔥 Unable to dump to '"
 
+/*
+ * All writes to this struct must be protected by
+ * the qtldr_lock. Reads and writes of 'allocated'
+ * and 'next' must also use atomics, since they
+ * are accessed without locks during signal handler
+ * dumps. Reads of 'buf' may be done without locks
+ * or atomics, since we're happy to dump whatever
+ * incomplete writes may be in progress.
+ */
+typedef struct {
+    char *buf;
+    size_t allocated; /* Currently allocated */
+    size_t capacity; /* Max permitted allocation */
+    size_t next; /* Offset to write next byte */
+} QTLDRMsgBucket;
+
+#define QTLDR_MSG_ENTRY_MAX_BYTES (2 * 1024)
+#define QTLDR_MSG_BUCKET_SIZE_BYTES (64 * 1024)
+
+/*
+ * MUST always be greater than entry size, otherwise
+ * wrapping logic doesn't work
+ */
+G_STATIC_ASSERT(QTLDR_MSG_BUCKET_SIZE_BYTES >
+                QTLDR_MSG_ENTRY_MAX_BYTES);
+
+extern QTLDRMsgBucket qtldr_msg_buckets[];
+
+extern char *(*qtldr_now_str_func)(void);
+
 void qtldr_crash_dump_internal(int fd, siginfo_t *si);
 
 #endif /* QEMU_TLDR_INTERNAL_H__ */
diff --git a/util/tldr.c b/util/tldr.c
index ea7c26b8e9b..34922d82ee4 100644
--- a/util/tldr.c
+++ b/util/tldr.c
@@ -8,20 +8,26 @@
 
 #include "qemu/osdep.h"
 #include "qemu/atomic.h"
+#include "qemu/thread.h"
 #include "util/tldr-internal.h"
 #include "qom/object.h"
 #include "qom/object_interfaces.h"
 #include "qapi/error.h"
 #include "qapi/qapi-commands-tldr.h"
+#include "qapi/qapi-visit-tldr.h"
 
 struct QTLDRConfig {
     Object parent;
 
     char *crash_target;
+    QTLDRMsgBucketConfigList *messages;
 };
 
+static QemuMutex qtldr_lock;
 static int qtldr_active;
 static char *qtldr_crash_target;
+QTLDRMsgBucket qtldr_msg_buckets[QTLDR_MSG_BUCKET_TYPE__MAX];
+unsigned int qtldr_msg_enabled_mask;
 
 #define TYPE_QTLDR_CONFIG "tldr-config"
 
@@ -34,6 +40,22 @@ OBJECT_DEFINE_SIMPLE_TYPE_WITH_INTERFACES(QTLDRConfig,
                                           { TYPE_USER_CREATABLE },
                                           { NULL });
 
+static char *
+qtldr_now_str_func_default(void)
+{
+    g_autoptr(GDateTime) dt = g_date_time_new_now_utc();
+    return g_date_time_format_iso8601(dt);
+}
+
+char *(*qtldr_now_str_func)(void) = qtldr_now_str_func_default;
+
+
+static void __attribute__((constructor))
+qtldr_global_init(void)
+{
+    qemu_mutex_init(&qtldr_lock);
+}
+
 
 static void
 qtldr_config_finalize(Object *obj)
@@ -41,6 +63,7 @@ qtldr_config_finalize(Object *obj)
     QTLDRConfig *cfg = QTLDR_CONFIG(obj);
 
     g_free(cfg->crash_target);
+    qapi_free_QTLDRMsgBucketConfigList(cfg->messages);
 }
 
 
@@ -60,24 +83,93 @@ qtldr_config_set_crash_target(Object *obj, const char *path, Error **errp)
     cfg->crash_target = g_strdup(path);
 }
 
+static void qtldr_config_get_messages(Object *obj, Visitor *v, const char *name,
+                                      void *opaque, Error **errp)
+{
+    QTLDRConfig *cfg = QTLDR_CONFIG(obj);
+
+    visit_type_QTLDRMsgBucketConfigList(v, name, &cfg->messages, errp);
+}
+
+static void qtldr_config_set_messages(Object *obj, Visitor *v, const char *name,
+                                      void *opaque, Error **errp)
+{
+    QTLDRConfig *cfg = QTLDR_CONFIG(obj);
+    QTLDRMsgBucketConfigList *oldmsgs;
+
+    oldmsgs = cfg->messages;
+    visit_type_QTLDRMsgBucketConfigList(v, name, &cfg->messages, errp);
+
+    qapi_free_QTLDRMsgBucketConfigList(oldmsgs);
+}
 
 static void
 qtldr_config_complete(UserCreatable *uc, Error **errp)
 {
     QTLDRConfig *cfg = QTLDR_CONFIG(uc);
+    QTLDRMsgBucketConfigList *messages = cfg->messages;
 
     /*
      * Forbid changes, since once configured, we can't safely
      * change it due to signal handlers running without ability
      * to hold mutex locks
      */
-    if (qtldr_crash_target) {
+    if (qtldr_crash_target || qtldr_msg_enabled_mask) {
         error_setg(errp, "TLDR configuration cannot be changed");
         return;
     }
 
+    /*
+     * Sanity check before changing anything, so errors don't
+     * leave us in a partially configured state
+     */
+    while (messages) {
+        QTLDRMsgBucketConfig *msgcfg = messages->value;
+        assert(msgcfg->type < QTLDR_MSG_BUCKET_TYPE__MAX);
+
+        if (msgcfg->has_capacity &&
+            msgcfg->capacity <= QTLDR_MSG_ENTRY_MAX_BYTES) {
+            error_setg(errp, "Message bucket capacity must be greater than %d",
+                       QTLDR_MSG_ENTRY_MAX_BYTES);
+            return;
+        }
+        messages = messages->next;
+    }
+
+    qemu_mutex_lock(&qtldr_lock);
     qatomic_store_release(&qtldr_crash_target, g_strdup(cfg->crash_target));
+
+    messages = cfg->messages;
+    while (messages) {
+        QTLDRMsgBucketConfig *msgcfg = messages->value;
+        assert(msgcfg->type < QTLDR_MSG_BUCKET_TYPE__MAX);
+
+        if (msgcfg->has_capacity) {
+            qtldr_msg_buckets[msgcfg->type].capacity = msgcfg->capacity;
+        } else {
+            qtldr_msg_buckets[msgcfg->type].capacity =
+                QTLDR_MSG_BUCKET_SIZE_BYTES;
+        }
+
+        qtldr_msg_enabled_mask |= (1 << msgcfg->type);
+
+        messages = messages->next;
+    }
+
+    if (!cfg->messages) {
+        size_t i;
+        qtldr_msg_enabled_mask =
+            (1 << QTLDR_MSG_BUCKET_TYPE_ERROR) |
+            (1 << QTLDR_MSG_BUCKET_TYPE_COMMAND) |
+            (1 << QTLDR_MSG_BUCKET_TYPE_EVENT) |
+            (1 << QTLDR_MSG_BUCKET_TYPE_LOG);
+        for (i = 0; i < QTLDR_MSG_BUCKET_TYPE__MAX; i++) {
+            qtldr_msg_buckets[i].capacity = QTLDR_MSG_BUCKET_SIZE_BYTES;
+        }
+    }
+
     qatomic_inc(&qtldr_active);
+    qemu_mutex_unlock(&qtldr_lock);
 }
 
 
@@ -101,6 +193,10 @@ qtldr_config_class_init(ObjectClass *klass, const void *opaque)
     object_class_property_add_str(klass, "crash-target",
                                   qtldr_config_get_crash_target,
                                   qtldr_config_set_crash_target);
+    object_class_property_add(klass, "messages", "QTLDRMsgBucketConfigList",
+                              qtldr_config_get_messages,
+                              qtldr_config_set_messages,
+                              NULL, NULL);
 
     ucc->complete = qtldr_config_complete;
     ucc->prepare_delete = qtldr_config_prepare_delete;
@@ -113,6 +209,145 @@ qtldr_config_init(Object *obj)
 }
 
 
+/*
+ * WARNING: The data recorder is intended to be called from
+ * WARNING: many places within QEMU, including tracing, error
+ * WARNING: reporting and logging code.
+ *
+ * WARNING: The code following this marker is used in the call
+ * WARNING: chain from qtldr_msg_record() and must be self
+ * WARNING: contained and performance oriented.
+ */
+
+static void
+qtldr_msg_record_ensure_size(QTLDRMsgBucket *bucket,
+                             size_t need)
+{
+    /*
+     * If bucket is not yet at max size then we can't have
+     * wrapped the buffer and so 'head' must reflect the
+     * amount of written data so far.
+     */
+    if (bucket->allocated < bucket->capacity) {
+        size_t space = bucket->allocated - bucket->next;
+        if (space < need) {
+            size_t newlen = MIN(bucket->capacity,
+                                bucket->allocated + MAX(4096, need - space));
+            bucket->buf = g_realloc(bucket->buf, newlen);
+            /*
+             * Fill with zeros, so the dumping method
+             * knows what is unused so far.
+             */
+            memset(bucket->buf + bucket->allocated, '\0',
+                   newlen - bucket->allocated);
+            qatomic_store_release(&bucket->allocated, newlen);
+        }
+    }
+}
+
+
+static size_t
+qtldr_msg_record_append_buf(QTLDRMsgBucket *bucket,
+                            size_t next,
+                            const void *buf,
+                            size_t len)
+{
+    size_t forend = MIN(bucket->allocated - next, len);
+    size_t forfront = len - forend;
+
+    if (forend) {
+        memcpy(bucket->buf + next, buf, forend);
+    }
+    if (forfront) {
+        memcpy(bucket->buf, buf + forend, forfront);
+        return forfront;
+    } else {
+        return (next + forend) % bucket->allocated;
+    }
+}
+
+
+void
+qtldr_msg_record(QTLDRMsgBucketType type,
+                 const char *msg)
+{
+    QTLDRMsgBucket *bucket;
+    size_t msglen, timelen;
+    g_autofree char *timestr = qtldr_now_str_func();
+    bool addnl = false;
+    uint16_t recordlen;
+    size_t next;
+
+    assert(type < QTLDR_MSG_BUCKET_TYPE__MAX);
+    if (!qtldr_msg_enabled(type)) {
+        return;
+    }
+
+    qemu_mutex_lock(&qtldr_lock);
+
+    bucket = &qtldr_msg_buckets[type];
+    timelen = strlen(timestr);
+    msglen = strlen(msg);
+
+    recordlen = timelen + 1 /* " " */ + msglen;
+    if (recordlen > (QTLDR_MSG_ENTRY_MAX_BYTES - 1)) {
+        size_t cut = (QTLDR_MSG_ENTRY_MAX_BYTES - 1)  - recordlen;
+        msglen -= cut;
+        recordlen -= cut;
+    }
+
+    if (msglen && msg[msglen - 1] != '\n') {
+        addnl = true;
+        recordlen++;
+    }
+
+
+    qtldr_msg_record_ensure_size(bucket, recordlen);
+    next = qtldr_msg_record_append_buf(bucket, bucket->next, timestr, timelen);
+    next = qtldr_msg_record_append_buf(bucket, next, " ", 1);
+    next = qtldr_msg_record_append_buf(bucket, next, msg, msglen);
+    if (addnl) {
+        next = qtldr_msg_record_append_buf(bucket, next, "\n", 1);
+    }
+
+    /*
+     * We could be racing with a signal handler dumping text
+     * outside the mutex lock, so while we write text without
+     * any sync, we atomically change the "next" pointer.
+     *
+     * Worst case, some text the signal handler dumps will be
+     * "new" instead of old part of the buffer.
+     */
+    qatomic_store_release(&bucket->next, next);
+    qemu_mutex_unlock(&qtldr_lock);
+}
+
+
+void
+qtldr_msg_record_printf(QTLDRMsgBucketType type,
+                        const char *fmt, ...)
+{
+    char msg[1024];
+    va_list ap;
+
+    va_start(ap, fmt);
+    vsnprintf(msg, G_N_ELEMENTS(msg), fmt, ap);
+    va_end(ap);
+
+    qtldr_msg_record(type, msg);
+}
+
+
+void
+qtldr_msg_record_vprintf(QTLDRMsgBucketType type,
+                         const char *fmt, va_list ap)
+{
+    char msg[1024];
+    vsnprintf(msg, G_N_ELEMENTS(msg), fmt, ap);
+    qtldr_msg_record(type, msg);
+}
+
+
 /*
  * WARNING: Code below this marker is used when dumping
  * WARNING: the data recorder buffers. It is subject to
@@ -159,6 +394,43 @@ qtldr_write_int_hex(int fd, uint64_t v)
 }
 
 
+static void
+qtldr_crash_dump_msg(int fd, QTLDRMsgBucketType type, QTLDRMsgBucket *bucket)
+{
+    size_t allocated, head;
+
+    if (!bucket->allocated) {
+        return;
+    }
+    ignore_value(write(fd, QTLDR_DUMP_MSG_BUCKET_PREFIX,
+                       strlen(QTLDR_DUMP_MSG_BUCKET_PREFIX)));
+    ignore_value(write(fd, QTLDRMsgBucketType_str(type),
+                       strlen(QTLDRMsgBucketType_str(type))));
+    ignore_value(write(fd, "':\n", 3));
+
+    /*
+     * We're potentially racing with a thread appending.
+     * Load 'head' first, so we're guaranteed it does
+     * not point beyond the 'allocated' region, should
+     * we be racing with buffer enlargement
+     */
+    head = qatomic_load_acquire(&bucket->next);
+    allocated = qatomic_load_acquire(&bucket->allocated);
+
+    /*
+     * If we haven't wrapped around yet, the allocated but
+     * unused space is filled with NULs. We can assume
+     * that no valid messages contain NULs, since we used
+     * "strlen" when writing messages originally.
+     */
+    if (bucket->buf[head] != '\0') {
+        ignore_value(write(fd, bucket->buf + head,
+                           allocated - head));
+    }
+    ignore_value(write(fd, bucket->buf, head));
+}
+
+
 static const char *
 qtldr_crash_dump_signal_str(int signo)
 {
@@ -378,6 +650,8 @@ qtldr_crash_dump_print_siginfo(int fd, siginfo_t *si)
 void
 qtldr_crash_dump_internal(int fd, siginfo_t *si)
 {
+    size_t i;
+
     if (fd == -1) {
         fd = STDERR_FILENO;
     }
@@ -391,6 +665,10 @@ qtldr_crash_dump_internal(int fd, siginfo_t *si)
         ignore_value(write(fd, "manual\n", 7));
     }
 
+    for (i = 0; i < QTLDR_MSG_BUCKET_TYPE__MAX; i++) {
+        qtldr_crash_dump_msg(fd, i, &qtldr_msg_buckets[i]);
+    }
+
     ignore_value(write(fd, QTLDR_DUMP_TRAILER, strlen(QTLDR_DUMP_TRAILER)));
 }
 
@@ -463,7 +741,9 @@ qmp_tldr_dump(const char *target, Error **errp)
         return;
     }
 
+    qemu_mutex_lock(&qtldr_lock);
     qtldr_crash_dump_internal(fd, NULL);
+    qemu_mutex_unlock(&qtldr_lock);
 
     qemu_fdatasync(fd);
     close(fd);
-- 
2.56.0

[PATCH 15/21] util: capture warnings and errors in the data recorder

[Daniel P. Berrangé]

Any messages triggered at runtime for warn_report and error_report
are potentially interesting, so should be captured for the data
recorder. The most common expected scenario is use of any deprecated
features which are liable to trigger warn_report.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 MAINTAINERS          |  1 +
 stubs/meson.build    |  1 +
 stubs/monitor-puts.c |  9 +++++++++
 util/error-report.c  | 43 ++++++++++++++++++++++++-------------------
 4 files changed, 35 insertions(+), 19 deletions(-)
 create mode 100644 stubs/monitor-puts.c

diff --git a/MAINTAINERS b/MAINTAINERS
index 54265d987cb..3a7cd827e66 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -3566,6 +3566,7 @@ F: include/monitor/hmp-completion.h
 F: tests/qtest/test-hmp.c
 F: include/qemu/qemu-print.h
 F: util/qemu-print.c
+F: stubs/monitor-puts.c
 
 Network device backends
 M: Jason Wang <jasowangio@gmail.com>
diff --git a/stubs/meson.build b/stubs/meson.build
index 29b6644b254..52c733e32a9 100644
--- a/stubs/meson.build
+++ b/stubs/meson.build
@@ -9,6 +9,7 @@ stub_ss.add(files('is-daemonized.c'))
 stub_ss.add(files('monitor-core.c'))
 stub_ss.add(files('monitor-cur.c'))
 stub_ss.add(files('monitor-hmp.c'))
+stub_ss.add(files('monitor-puts.c'))
 stub_ss.add(files('qapi-event-emit.c'))
 stub_ss.add(files('replay-mode.c'))
 stub_ss.add(files('trace-control.c'))
diff --git a/stubs/monitor-puts.c b/stubs/monitor-puts.c
new file mode 100644
index 00000000000..bb9b726dd5e
--- /dev/null
+++ b/stubs/monitor-puts.c
@@ -0,0 +1,9 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+#include "qemu/osdep.h"
+#include "monitor/monitor.h"
+
+int monitor_puts(Monitor *mon, const char *str)
+{
+    g_assert_not_reached();
+}
diff --git a/util/error-report.c b/util/error-report.c
index 8932beebfb9..251d8b7d16e 100644
--- a/util/error-report.c
+++ b/util/error-report.c
@@ -13,6 +13,7 @@
 #include "qemu/osdep.h"
 #include "monitor/hmp.h"
 #include "qemu/error-report.h"
+#include "qemu/tldr.h"
 
 /*
  * @report_type is the type of message: error, warning or
@@ -32,28 +33,32 @@ const char *error_guest_name;
 /*
  * Print to the @hmp monitor, else to stderr.
  */
-static int G_GNUC_PRINTF(2, 0)
-error_vprintf_hmp(MonitorHMP *hmp, const char *fmt, va_list ap)
+static int G_GNUC_PRINTF(3, 0)
+error_vprintf_hmp(MonitorHMP *hmp, bool tldr, const char *fmt, va_list ap)
 {
+    g_autofree char *str = g_strdup_vprintf(fmt, ap);
+    if (qtldr_msg_enabled(QTLDR_MSG_BUCKET_TYPE_ERROR) && tldr) {
+        qtldr_msg_record(QTLDR_MSG_BUCKET_TYPE_ERROR, str);
+    }
 #ifdef CONFIG_HMP
     if (hmp) {
-        return monitor_hmp_vprintf(hmp, fmt, ap);
+        return monitor_puts(MONITOR(hmp), str);
     }
 #endif
-    return vfprintf(stderr, fmt, ap);
+    return fwrite(str, 1, strlen(str), stderr);
 }
 
 /*
  * Print to the @hmp monitor, else to stderr.
  */
-static int G_GNUC_PRINTF(2, 3)
-error_printf_hmp(MonitorHMP *hmp, const char *fmt, ...)
+static int G_GNUC_PRINTF(3, 4)
+error_printf_hmp(MonitorHMP *hmp, bool tldr, const char *fmt, ...)
 {
     va_list ap;
     int ret;
 
     va_start(ap, fmt);
-    ret = error_vprintf_hmp(hmp, fmt, ap);
+    ret = error_vprintf_hmp(hmp, tldr, fmt, ap);
     va_end(ap);
     return ret;
 }
@@ -63,7 +68,7 @@ error_printf_hmp(MonitorHMP *hmp, const char *fmt, ...)
  */
 int error_vprintf(const char *fmt, va_list ap)
 {
-    return error_vprintf_hmp(monitor_cur_hmp(), fmt, ap);
+    return error_vprintf_hmp(monitor_cur_hmp(), true, fmt, ap);
 }
 
 /*
@@ -75,7 +80,7 @@ int error_printf(const char *fmt, ...)
     int ret;
 
     va_start(ap, fmt);
-    ret = error_vprintf_hmp(monitor_cur_hmp(), fmt, ap);
+    ret = error_vprintf_hmp(monitor_cur_hmp(), true, fmt, ap);
     va_end(ap);
     return ret;
 }
@@ -193,20 +198,20 @@ static void print_loc(MonitorHMP *hmp)
     case LOC_CMDLINE:
         argp = cur_loc->ptr;
         for (i = 0; i < cur_loc->num; i++) {
-            error_printf_hmp(hmp, "%s%s", sep, argp[i]);
+            error_printf_hmp(hmp, false, "%s%s", sep, argp[i]);
             sep = " ";
         }
-        error_printf_hmp(hmp, ": ");
+        error_printf_hmp(hmp, false, ": ");
         break;
     case LOC_FILE:
-        error_printf_hmp(hmp, "%s:", (const char *)cur_loc->ptr);
+        error_printf_hmp(hmp, false, "%s:", (const char *)cur_loc->ptr);
         if (cur_loc->num) {
-            error_printf_hmp(hmp, "%d:", cur_loc->num);
+            error_printf_hmp(hmp, false, "%d:", cur_loc->num);
         }
-        error_printf_hmp(hmp, " ");
+        error_printf_hmp(hmp, false, " ");
         break;
     default:
-        error_printf_hmp(hmp, "%s", sep);
+        error_printf_hmp(hmp, false, "%s", sep);
     }
 }
 
@@ -251,15 +256,15 @@ static void vreport(report_type type, const char *fmt, va_list ap)
     case REPORT_TYPE_ERROR:
         break;
     case REPORT_TYPE_WARNING:
-        error_printf_hmp(hmp, "warning: ");
+        error_printf_hmp(hmp, false, "warning: ");
         break;
     case REPORT_TYPE_INFO:
-        error_printf_hmp(hmp, "info: ");
+        error_printf_hmp(hmp, false, "info: ");
         break;
     }
 
-    error_vprintf_hmp(hmp, fmt, ap);
-    error_printf_hmp(hmp, "\n");
+    error_vprintf_hmp(hmp, true, fmt, ap);
+    error_printf_hmp(hmp, false, "\n");
 
     if (!hmp) {
         qemu_funlockfile(stderr);
-- 
2.56.0

[PATCH 16/21] monitor: remove obsolete/redundant exports

[Daniel P. Berrangé]

qmp_send_response is only used within qmp.c so can be static
and monitor_data_destroy_qmp no longer exists.

Fixes: b723eeb815 (monitor: move monitor_data_(init|destroy) into QOM init/finalize)
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 monitor/monitor-internal.h | 2 --
 monitor/qmp.c              | 3 ++-
 2 files changed, 2 insertions(+), 3 deletions(-)

diff --git a/monitor/monitor-internal.h b/monitor/monitor-internal.h
index 822a66d06d0..dfdb8fb8caa 100644
--- a/monitor/monitor-internal.h
+++ b/monitor/monitor-internal.h
@@ -122,8 +122,6 @@ void monitor_cancel_out_watch(Monitor *mon);
 void monitor_list_append(Monitor *mon);
 void monitor_fdsets_cleanup(void);
 
-void qmp_send_response(MonitorQMP *mon, const QDict *rsp);
-void monitor_data_destroy_qmp(MonitorQMP *mon);
 void coroutine_fn monitor_qmp_dispatcher_co(void *data);
 void qmp_dispatcher_co_wake(void);
 
diff --git a/monitor/qmp.c b/monitor/qmp.c
index aec03157750..ccb39a48d4c 100644
--- a/monitor/qmp.c
+++ b/monitor/qmp.c
@@ -80,6 +80,7 @@ static MonitorQMP *qmp_dispatcher_current_mon;
 OBJECT_DEFINE_TYPE(MonitorQMP, monitor_qmp, MONITOR_QMP, MONITOR);
 
 static void monitor_qmp_cleanup_req_queue_locked(MonitorQMP *mon);
+static void qmp_send_response(MonitorQMP *mon, const QDict *rsp);
 
 static void monitor_qmp_finalize(Object *obj)
 {
@@ -242,7 +243,7 @@ static void monitor_qmp_cleanup_queue_and_resume(MonitorQMP *mon)
 
 }
 
-void qmp_send_response(MonitorQMP *mon, const QDict *rsp)
+static void qmp_send_response(MonitorQMP *mon, const QDict *rsp)
 {
     const QObject *data = QOBJECT(rsp);
     GString *json;
-- 
2.56.0

[PATCH 17/21] monitor: capture monitor commands and events in data recorder

[Daniel P. Berrangé]

QMP monitor commands which change system state (ie not query-XXX
commands) are relevant to the data recorder, as they reflect what
has been done to QEMU at runtime by the mgmt application.

QMP monitor events are relevant to the data recorder, as they
reflect a combination of what has been done to QEMU at runtime by
the mgmt application, as well as changes triggered by the guest OS.

QMP error responses are relevant either to reveal mistakes in mgmt
apps, or to reveal problems in QEMU. If the associated QMP command
was not already recorded, it will be recorded for any error
response to provide context to the failure.

QMP normal responses are skipped since the data can be volumous
and thus result in poor signal/noise ratio.

The QMP commands are never captured to a string buffer as input
is streamed into the JSON parser and the result is spat out as
a QObject. Thus, the commands need to be formatted back into JSON
string format for the data recorder. This is a mild but acceptable
performance overhead, in the overall context of handling a QMP
command.

The QMP events are captured immediately before they are written
back on the wire where already in string format.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 monitor/qmp.c | 67 +++++++++++++++++++++++++++++++++++++++++++++------
 1 file changed, 59 insertions(+), 8 deletions(-)

diff --git a/monitor/qmp.c b/monitor/qmp.c
index ccb39a48d4c..659ad23d832 100644
--- a/monitor/qmp.c
+++ b/monitor/qmp.c
@@ -34,6 +34,7 @@
 #include "qobject/qjson.h"
 #include "qobject/qlist.h"
 #include "qom/object_interfaces.h"
+#include "qemu/tldr.h"
 #include "trace.h"
 
 /*
@@ -80,7 +81,8 @@ static MonitorQMP *qmp_dispatcher_current_mon;
 OBJECT_DEFINE_TYPE(MonitorQMP, monitor_qmp, MONITOR_QMP, MONITOR);
 
 static void monitor_qmp_cleanup_req_queue_locked(MonitorQMP *mon);
-static void qmp_send_response(MonitorQMP *mon, const QDict *rsp);
+static void qmp_send_response(MonitorQMP *mon, const QDict *rsp,
+                              const QObject *ctx);
 
 static void monitor_qmp_finalize(Object *obj)
 {
@@ -169,7 +171,7 @@ static void monitor_qmp_emit_event(Monitor *mon, QAPIEvent event, QDict *qdict)
         }
     }
 
-    qmp_send_response(qmp, qdict);
+    qmp_send_response(qmp, qdict, NULL);
 }
 
 static bool monitor_qmp_requires_iothread(const Monitor *mon)
@@ -243,7 +245,8 @@ static void monitor_qmp_cleanup_queue_and_resume(MonitorQMP *mon)
 
 }
 
-static void qmp_send_response(MonitorQMP *mon, const QDict *rsp)
+static void qmp_send_response(MonitorQMP *mon, const QDict *rsp,
+                              const QObject *ctx)
 {
     const QObject *data = QOBJECT(rsp);
     GString *json;
@@ -252,6 +255,30 @@ static void qmp_send_response(MonitorQMP *mon, const QDict *rsp)
     assert(json != NULL);
     trace_monitor_qmp_respond(mon, json->str);
 
+    /*
+     * Errors are always useful to record. We recorded the
+     * command in the caller in many cases, but if not, then
+     * 'ctx' will be non-NULL and we should record it now
+     * for context.
+     */
+    if (qtldr_msg_enabled(QTLDR_MSG_BUCKET_TYPE_COMMAND) &&
+        qdict_haskey(rsp, "error")) {
+        if (ctx) {
+            g_autoptr(GString) ctxstr = qobject_to_json(ctx);
+            qtldr_msg_record(QTLDR_MSG_BUCKET_TYPE_COMMAND, ctxstr->str);
+        }
+        qtldr_msg_record(QTLDR_MSG_BUCKET_TYPE_COMMAND, json->str);
+    }
+
+    /*
+     * Capture events, since they represent changes
+     * in state or behaviour
+     */
+    if (qtldr_msg_enabled(QTLDR_MSG_BUCKET_TYPE_EVENT) &&
+        qdict_haskey(rsp, "event")) {
+        qtldr_msg_record(QTLDR_MSG_BUCKET_TYPE_EVENT, json->str);
+    }
+
     g_string_append_c(json, '\n');
     monitor_puts(&mon->parent_obj, json->str);
 
@@ -263,10 +290,10 @@ static void qmp_send_response(MonitorQMP *mon, const QDict *rsp)
  * Null @rsp can only happen for commands with QCO_NO_SUCCESS_RESP.
  * Nothing is emitted then.
  */
-static void monitor_qmp_respond(MonitorQMP *mon, QDict *rsp)
+static void monitor_qmp_respond(MonitorQMP *mon, QDict *rsp, QObject *ctx)
 {
     if (rsp) {
-        qmp_send_response(mon, rsp);
+        qmp_send_response(mon, rsp, ctx);
     }
 }
 
@@ -276,9 +303,33 @@ static void monitor_qmp_respond(MonitorQMP *mon, QDict *rsp)
  */
 static void monitor_qmp_dispatch(MonitorQMP *mon, QObject *req)
 {
+    QObject *ctx = req;
     QDict *rsp;
     QDict *error;
 
+    /*
+     * Capture commands which are liable to change state,
+     * i.e. exclude purely information query comamnds
+     */
+    if (qtldr_msg_enabled(QTLDR_MSG_BUCKET_TYPE_COMMAND)) {
+        QDict *cmd = qobject_to(QDict, req);
+        if (cmd != NULL) {
+            const char *cmdname = qdict_get_try_str(cmd, "execute")
+                ?: qdict_get_try_str(cmd, "exec-oob");
+            if (cmdname != NULL &&
+                !g_str_has_prefix(cmdname, "query-") &&
+                !g_str_has_prefix(cmdname, "qom-list")) {
+                g_autoptr(GString) cmdstr = qobject_to_json(req);
+                qtldr_msg_record(QTLDR_MSG_BUCKET_TYPE_COMMAND, cmdstr->str);
+                /*
+                 * Prevent recording cmd twice if the later response
+                 * is an error
+                 */
+                ctx = NULL;
+            }
+        }
+    }
+
     rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon),
                        &mon->parent_obj);
 
@@ -294,7 +345,7 @@ static void monitor_qmp_dispatch(MonitorQMP *mon, QObject *req)
         }
     }
 
-    monitor_qmp_respond(mon, rsp);
+    monitor_qmp_respond(mon, rsp, ctx);
     qobject_unref(rsp);
 }
 
@@ -455,7 +506,7 @@ void coroutine_fn monitor_qmp_dispatcher_co(void *data)
             trace_monitor_qmp_err_in_band(error_get_pretty(req_obj->err));
             rsp = qmp_error_response(req_obj->err);
             req_obj->err = NULL;
-            monitor_qmp_respond(mon, rsp);
+            monitor_qmp_respond(mon, rsp, NULL);
             qobject_unref(rsp);
         }
 
@@ -623,7 +674,7 @@ static void monitor_qmp_event(void *opaque, QEMUChrEvent event)
             monitor_qmp_caps_reset(mon);
         }
         data = qmp_greeting(mon);
-        qmp_send_response(mon, data);
+        qmp_send_response(mon, data, NULL);
         qobject_unref(data);
         break;
     case CHR_EVENT_CLOSED:
-- 
2.56.0

[PATCH 18/21] log: enable directing log messages to data recorder

[Daniel P. Berrangé]

Currently the log APIs can output messages to stderr, a named
loge file, or discarded to /dev/null.  Introduce a "tldr" log
mask category which causes log messages to be directed to the
data recorder.

This this output is mutually exclusive with file based logging
and only captures single line logs emitted with qemu_log() or
qemu_log_mask(). Log callers using direct FILE* access will be
discarded. This is sufficient for the primary intended use case
which is to collect output from the 'log' trace backend.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 include/qemu/log.h |  1 +
 util/log.c         | 42 +++++++++++++++++++++++++++++++++---------
 2 files changed, 34 insertions(+), 9 deletions(-)

diff --git a/include/qemu/log.h b/include/qemu/log.h
index 92956e5d0f5..47d89681d8e 100644
--- a/include/qemu/log.h
+++ b/include/qemu/log.h
@@ -38,6 +38,7 @@ bool qemu_log_separate(void);
 #define CPU_LOG_TB_VPU     (1u << 21)
 #define LOG_TB_OP_PLUGIN   (1u << 22)
 #define LOG_INVALID_MEM    (1u << 23)
+#define LOG_TLDR           (1u << 24)
 
 /* Lock/unlock output. */
 
diff --git a/util/log.c b/util/log.c
index 7cffbc1bf89..df7499012f6 100644
--- a/util/log.c
+++ b/util/log.c
@@ -27,6 +27,7 @@
 #include "qemu/thread.h"
 #include "qemu/lockable.h"
 #include "qemu/rcu.h"
+#include "qemu/tldr.h"
 #ifdef CONFIG_LINUX
 #include <sys/syscall.h>
 #endif
@@ -172,12 +173,20 @@ void qemu_log_unlock(FILE *logfile)
 
 void qemu_log(const char *fmt, ...)
 {
-    FILE *f = qemu_log_trylock_with_context();
-    if (f) {
-        va_list ap;
-        va_start(ap, fmt);
-        vfprintf(f, fmt, ap);
-        va_end(ap);
+    va_list ap;
+    if (qemu_loglevel & LOG_TLDR) {
+        if (qtldr_msg_enabled(QTLDR_MSG_BUCKET_TYPE_LOG)) {
+            va_start(ap, fmt);
+            qtldr_msg_record_vprintf(QTLDR_MSG_BUCKET_TYPE_LOG, fmt, ap);
+            va_end(ap);
+        }
+    } else {
+        FILE *f = qemu_log_trylock_with_context();
+        if (f) {
+            va_start(ap, fmt);
+            vfprintf(f, fmt, ap);
+            va_end(ap);
+        }
         qemu_log_unlock(f);
     }
 }
@@ -236,6 +245,7 @@ static bool qemu_set_log_internal(const char *filename, bool changed_name,
     bool need_to_open_file;
     bool daemonized;
     bool per_thread;
+    bool tldr;
     FILE *logfile;
 
     QEMU_LOCK_GUARD(&global_mutex);
@@ -251,8 +261,20 @@ static bool qemu_set_log_internal(const char *filename, bool changed_name,
     }
 
     per_thread = log_flags & LOG_PER_THREAD;
+    tldr = log_flags & LOG_TLDR;
 
-    if (changed_name) {
+    if (tldr) {
+        if (filename) {
+            error_setg(errp, "Cannot enable TLDR with a log file");
+            return false;
+        }
+        if (per_thread) {
+            error_setg(errp, "Cannot enable TLDR with per-thread log files");
+            return false;
+        }
+    }
+
+    if (changed_name && !tldr) {
         char *newname = NULL;
 
         /*
@@ -282,7 +304,7 @@ static bool qemu_set_log_internal(const char *filename, bool changed_name,
         g_free(global_filename);
         global_filename = newname;
         filename = newname;
-    } else {
+    } else if (!tldr) {
         filename = global_filename;
         if (per_thread &&
             valid_filename_template(filename, true, errp) == vft_error) {
@@ -309,7 +331,7 @@ static bool qemu_set_log_internal(const char *filename, bool changed_name,
          * stderr or to a file (if there is a filename).
          * If per-thread, open the file for each thread in qemu_log_trylock().
          */
-        need_to_open_file = qemu_loglevel && !log_per_thread;
+        need_to_open_file = qemu_loglevel && !log_per_thread && !tldr;
     } else {
         /*
          * If we are daemonized, we will only log if there is a filename.
@@ -531,6 +553,8 @@ const QEMULogItem qemu_log_items[] = {
       "include VPU registers in the 'cpu' logging" },
     { LOG_INVALID_MEM, "invalid_mem",
       "log invalid memory accesses" },
+    { LOG_TLDR, "tldr",
+      "log to QEMU data recorder instead ofa file" },
     { 0, NULL, NULL },
 };
 
-- 
2.56.0

[PATCH 19/21] tests/qtest: add API to declare an expected crash signal

[Daniel P. Berrangé]

Most qtests are validating positive behaviour, but some might
wish to validate that QEMU crashes. Add an API for libqtest
that allows declaring an expected crash signal.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 tests/qtest/libqtest.c | 36 +++++++++++++++++++++++++++---------
 tests/qtest/libqtest.h | 11 +++++++++++
 2 files changed, 38 insertions(+), 9 deletions(-)

diff --git a/tests/qtest/libqtest.c b/tests/qtest/libqtest.c
index 05d88476154..e65451171e4 100644
--- a/tests/qtest/libqtest.c
+++ b/tests/qtest/libqtest.c
@@ -86,6 +86,9 @@ struct QTestState
     DWORD exit_code;
 #endif
     int expected_status;
+#ifndef _WIN32
+    int expected_signal;
+#endif
     bool big_endian;
     bool irq_level[MAX_IRQ];
     GString *rx;
@@ -185,6 +188,13 @@ void qtest_set_expected_status(QTestState *s, int status)
     s->expected_status = status;
 }
 
+#ifndef _WIN32
+void qtest_set_expected_signal(QTestState *s, int signal)
+{
+    s->expected_signal = signal;
+}
+#endif
+
 static void qtest_check_status(QTestState *s)
 {
     assert(s->qemu_pid == -1);
@@ -195,20 +205,28 @@ static void qtest_check_status(QTestState *s)
      */
 #ifndef _WIN32
     int wstatus = s->wstatus;
-    if (WIFEXITED(wstatus) && WEXITSTATUS(wstatus) != s->expected_status) {
-        fprintf(stderr, "%s:%d: kill_qemu() tried to terminate QEMU "
-                "process but encountered exit status %d (expected %d)\n",
-                __FILE__, __LINE__, WEXITSTATUS(wstatus), s->expected_status);
-        abort();
+    if (WIFEXITED(wstatus)) {
+        if (s->expected_signal) {
+            fprintf(stderr, "%s:%d: encountered exit status %d but expected signal %d\n",
+                    __FILE__, __LINE__, WEXITSTATUS(wstatus), s->expected_signal);
+            abort();
+        } else if (WEXITSTATUS(wstatus) != s->expected_status) {
+            fprintf(stderr, "%s:%d: kill_qemu() tried to terminate QEMU "
+                    "process but encountered exit status %d (expected %d)\n",
+                    __FILE__, __LINE__, WEXITSTATUS(wstatus), s->expected_status);
+            abort();
+        }
     } else if (WIFSIGNALED(wstatus)) {
         int sig = WTERMSIG(wstatus);
         const char *signame = strsignal(sig) ?: "unknown ???";
         const char *dump = WCOREDUMP(wstatus) ? " (core dumped)" : "";
 
-        fprintf(stderr, "%s:%d: kill_qemu() detected QEMU death "
-                "from signal %d (%s)%s\n",
-                __FILE__, __LINE__, sig, signame, dump);
-        abort();
+        if (sig != s->expected_signal) {
+            fprintf(stderr, "%s:%d: kill_qemu() detected QEMU death "
+                    "from signal %d (%s)%s\n",
+                    __FILE__, __LINE__, sig, signame, dump);
+            abort();
+        }
     }
 #else
     if (s->exit_code != s->expected_status) {
diff --git a/tests/qtest/libqtest.h b/tests/qtest/libqtest.h
index 20982d3fe8d..ddcf075e33a 100644
--- a/tests/qtest/libqtest.h
+++ b/tests/qtest/libqtest.h
@@ -1155,6 +1155,17 @@ bool qtest_probe_child(QTestState *s);
  */
 void qtest_set_expected_status(QTestState *s, int status);
 
+#ifndef _WIN32
+/**
+ * qtest_set_expected_signal:
+ * @s: QTestState instance to operate on.
+ * @signal: an expected crash signal.
+ *
+ * Set expected crash signal of the child.
+ */
+void qtest_set_expected_signal(QTestState *s, int signal);
+#endif
+
 QTestState *qtest_inproc_init(QTestState **s, bool log, const char* arch,
                     void (*send)(void*, const char*));
 
-- 
2.56.0

[PATCH 20/21] qtest: add 'crash' command to generate 'authentic' process crashes

[Daniel P. Berrangé]

The 'crash' command provides a mechanism to generate process crashes,
which serve to test the signal catching and handling facilities in
a more authentic manner than simply running 'raise(SIG..)' against
ourself.

Available usage is

  crash null-pointer  -> SIGSEGV on null pointer
  crash invalid-addr -> SIGSEGV on derefence invalid address
  crash mapping-not-backed -> SIGBUS on writing to map with no storage
  crash bad-alignment -> SIGBUS on unaligned data write
  crash fpe-div-zero -> SIGFPE on floating point divide by zero
  crash assert -> SIGABRT on assertion failure
  crash illegal-inst -> SIGILL on illegal instruction

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 system/qtest.c         | 50 ++++++++++++++++++++++++++++++++++++++++++
 tests/qtest/libqtest.c | 26 ++++++++++++++++++++++
 tests/qtest/libqtest.h | 16 ++++++++++++++
 3 files changed, 92 insertions(+)

diff --git a/system/qtest.c b/system/qtest.c
index 47c954838dd..0ff04cf24e8 100644
--- a/system/qtest.c
+++ b/system/qtest.c
@@ -811,6 +811,56 @@ static void qtest_process_command(CharFrontend *chr, gchar **words)
         qtest_sendf(chr, err ? "FAIL %s\n" : "OK\n",
                     err ? error_get_pretty(err) : NULL);
         error_free(err);
+    } else if (strcmp(words[0], "crash") == 0) {
+        if (strcmp(words[1], "null-pointer") == 0) {
+            char *c = (void *)0x0;
+            *c = 'a'; /* SIGSEGV */
+        } else if (strcmp(words[1], "invalid-addr") == 0) {
+            /* Hopefully invalid addr on all supported targets */
+            char *c = (void *)0x12345;
+            *c = 'b'; /* SIGSEGV */
+#ifndef WIN32
+        } else if (strcmp(words[1], "mapping-not-backed") == 0) {
+            char *m;
+            g_autofree char *tmpname = NULL;
+            int tmpfd = g_file_open_tmp("qtestcrash-XXXXXX", &tmpname, NULL);
+            m = (char *)mmap(0, 4, PROT_WRITE, MAP_PRIVATE, tmpfd, 0);
+            unlink(tmpname);
+            assert(m != MAP_FAILED);
+            /*
+             * Since no writes have been made to 'tmpfd', the backing
+             * store is still zero-length, and thus writes should
+             * trigger a bus error
+             */
+            m[0] = 0; /* SIGBUS */
+#if defined(__x86_64__) || defined(__aarch64__)
+        } else if (strcmp(words[1], "bad-alignment") == 0) {
+            /*
+             * XXX: can we assume any other arches will generate
+             * bus errors on unaligned writes ?
+             */
+            char *cptr = g_malloc(sizeof(int) + 1) + 1;
+            int *iptr = (int *)cptr;
+#ifdef __x86_64__
+            /* Turn on alignment checks */
+            __asm__("pushf\norl $0x40000,(%rsp)\npopf");
+#endif
+            *iptr = 42; /* SIGBUS */
+#endif
+#endif
+        } else if (strcmp(words[1], "fpe-div-zero") == 0) {
+            volatile int foo = 1729; /* volatile to stop optimizing */
+            volatile int bar = 0; /* volatile to stop optimizing */
+
+            foo = foo / bar; /* SIGFPE */
+            (void)foo;
+        } else if (strcmp(words[1], "assert") == 0) {
+            assert(1 == 2); /* SIGABRT */
+        } else if (strcmp(words[1], "illegal-inst") == 0) {
+            __builtin_trap(); /* SIGILL */
+        } else {
+            qtest_sendf(chr, "FAIL unsupported crash action '%s'", words[1]);
+        }
     } else if (process_command_cb && process_command_cb(chr, words)) {
         /* Command got consumed by the callback handler */
     } else {
diff --git a/tests/qtest/libqtest.c b/tests/qtest/libqtest.c
index e65451171e4..d6ca97b1c11 100644
--- a/tests/qtest/libqtest.c
+++ b/tests/qtest/libqtest.c
@@ -828,6 +828,32 @@ static int qtest_query_target_endianness(QTestState *s)
     return big_endian;
 }
 
+static const char *crashtypestr[] = {
+    "null-pointer",
+    "invalid-addr",
+    "mapping-not-backed",
+#if defined(__x86_64__) || defined(__aarch64__)
+    "bad-alignment",
+#endif
+    "fpe-div-zero",
+    "assert",
+    "illegal-inst",
+};
+G_STATIC_ASSERT(G_N_ELEMENTS(crashtypestr) == QTEST_CRASH__MAX);
+
+void qtest_crash(QTestState *s, QTestCrashType type)
+{
+    char buf[1024];
+    ssize_t len;
+    assert(type < QTEST_CRASH__MAX);
+    qtest_sendf(s, "crash %s\n", crashtypestr[type]);
+    len = recv(s->fd, buf, sizeof(buf), 0);
+    if (len != 0) {
+        g_printerr("Expected crash from QEMU but saw qtest reply");
+        abort();
+    }
+}
+
 QDict *qtest_qmp_receive(QTestState *s)
 {
     while (true) {
diff --git a/tests/qtest/libqtest.h b/tests/qtest/libqtest.h
index ddcf075e33a..4988e627592 100644
--- a/tests/qtest/libqtest.h
+++ b/tests/qtest/libqtest.h
@@ -1232,4 +1232,20 @@ bool mkimg(const char *file, const char *fmt, unsigned size_mb);
  */
 bool qtest_verbose(const char *domain);
 
+typedef enum {
+    QTEST_CRASH_NULL_POINTER,
+    QTEST_CRASH_INVALID_ADDR,
+    QTEST_CRASH_MAPPING_NOT_BACKED,
+#if defined(__x86_64__) || defined(__aarch64__)
+    QTEST_CRASH_BAD_ALIGNMENT,
+#endif
+    QTEST_CRASH_FPE_DIV_ZERO,
+    QTEST_CRASH_ASSERT,
+    QTEST_CRASH_ILLEGAL_INST,
+
+    QTEST_CRASH__MAX,
+} QTestCrashType;
+
+void qtest_crash(QTestState *s, QTestCrashType type);
+
 #endif
-- 
2.56.0

[PATCH 21/21] tests/qtest: add test to validate data recorder on crashes

[Daniel P. Berrangé]

Use qtest to trigger a QEMU crash and validate that a data
recorder crash dump is written to disk.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 tests/qtest/meson.build |   3 +
 tests/qtest/tldr-test.c | 124 ++++++++++++++++++++++++++++++++++++++++
 2 files changed, 127 insertions(+)
 create mode 100644 tests/qtest/tldr-test.c

diff --git a/tests/qtest/meson.build b/tests/qtest/meson.build
index 200abf6f36d..c7bba9ea828 100644
--- a/tests/qtest/meson.build
+++ b/tests/qtest/meson.build
@@ -27,6 +27,9 @@ qtests_generic = [
   'qmp-cmd-test',
   'qom-test',
 ]
+if host_os != 'windows'
+  qtests_generic += ['tldr-test']
+endif
 if have_hmp
   qtests_generic += [
     'test-hmp',
diff --git a/tests/qtest/tldr-test.c b/tests/qtest/tldr-test.c
new file mode 100644
index 00000000000..b20d44577af
--- /dev/null
+++ b/tests/qtest/tldr-test.c
@@ -0,0 +1,124 @@
+/*
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ *
+ * QTest testcase for the data recorder (tldr)
+ */
+
+#include "qemu/osdep.h"
+#include "libqtest.h"
+#include "util/tldr-internal.h"
+
+
+static QTestState *qtldr_test_start(const char *crashfile)
+{
+    return qtest_initf("-object tldr-config,id=tldr,crash-target=%s", crashfile);
+}
+
+static void test_crash(QTestCrashType type, int signo)
+{
+    QTestState *qts = NULL;
+    g_autofree char *crashfile = NULL;
+    g_autofree char *crashreport = NULL;
+    size_t crashlen = 0;
+    int fd = g_file_open_tmp("tldr-test-XXXXXX",
+                             &crashfile, NULL);
+    assert(fd != -1);
+    qts = qtldr_test_start(crashfile);
+
+    qtest_crash(qts, type);
+
+    assert(g_file_get_contents(crashfile, &crashreport, &crashlen, NULL));
+    close(fd);
+    unlink(crashfile);
+    qtest_set_expected_signal(qts, signo);
+    qtest_quit(qts);
+
+    assert(crashlen != 0);
+    assert(strstr(crashreport, QTLDR_DUMP_LEADER));
+    assert(strstr(crashreport, QTLDR_DUMP_TRIGGER_PREFIX));
+    assert(strstr(crashreport, QTLDR_DUMP_MSG_BUCKET_PREFIX));
+    assert(strstr(crashreport, QTLDR_DUMP_TRAILER));
+}
+
+static void test_crash_null_pointer(void)
+{
+    test_crash(QTEST_CRASH_NULL_POINTER, SIGSEGV);
+}
+
+static void test_crash_invalid_addr(void)
+{
+    test_crash(QTEST_CRASH_INVALID_ADDR, SIGSEGV);
+}
+
+static void test_crash_mapping_not_backed(void)
+{
+    test_crash(QTEST_CRASH_MAPPING_NOT_BACKED, SIGBUS);
+}
+
+#if defined(__x86_64__) || defined(__aarch64__)
+static void test_crash_bad_alignment(void)
+{
+    test_crash(QTEST_CRASH_BAD_ALIGNMENT, SIGBUS);
+}
+#endif
+
+static void test_crash_fpe_div_zero(void)
+{
+    test_crash(QTEST_CRASH_FPE_DIV_ZERO, SIGFPE);
+}
+
+
+static void test_crash_assert(void)
+{
+    test_crash(QTEST_CRASH_ASSERT, SIGABRT);
+}
+
+static void test_crash_illegal_inst(void)
+{
+    test_crash(QTEST_CRASH_ILLEGAL_INST, SIGILL);
+}
+
+static void test_qmp(void)
+{
+    QTestState *qts = NULL;
+    g_autofree char *crashfile = NULL;
+    g_autofree char *crashreport = NULL;
+    size_t crashlen = 0;
+    int fd = g_file_open_tmp("tldr-test-XXXXXX",
+                             &crashfile, NULL);
+    assert(fd != -1);
+    qts = qtldr_test_start(crashfile);
+
+    qtest_qmp_assert_success(
+        qts, "{'execute':'tldr-dump','arguments':{'target': %s}}",
+        crashfile);
+
+    assert(g_file_get_contents(crashfile, &crashreport, &crashlen, NULL));
+    close(fd);
+    unlink(crashfile);
+    qtest_quit(qts);
+
+    assert(crashlen != 0);
+    assert(strstr(crashreport, QTLDR_DUMP_LEADER));
+    assert(strstr(crashreport, QTLDR_DUMP_TRIGGER_PREFIX));
+    assert(strstr(crashreport, QTLDR_DUMP_MSG_BUCKET_PREFIX));
+    assert(strstr(crashreport, QTLDR_DUMP_TRAILER));
+}
+
+int main(int argc, char **argv)
+{
+    g_test_init(&argc, &argv, NULL);
+
+    qtest_add_func("/tldr/null-pointer", test_crash_null_pointer);
+    qtest_add_func("/tldr/invalid-addr", test_crash_invalid_addr);
+    qtest_add_func("/tldr/mapping-not-backed", test_crash_mapping_not_backed);
+#if defined(__x86_64__) || defined(__aarch64__)
+    qtest_add_func("/tldr/bad-alignment", test_crash_bad_alignment);
+#endif
+    qtest_add_func("/tldr/fpe-div-zero", test_crash_fpe_div_zero);
+    qtest_add_func("/tldr/assert", test_crash_assert);
+    qtest_add_func("/tldr/illegal-inst", test_crash_illegal_inst);
+    qtest_add_func("/tldr/qmp", test_qmp);
+
+    return g_test_run();
+}
-- 
2.56.0

Re: [PATCH 00/21] system: introduce a live data recorder facility

[marcandre.lureau]

> This series introduces a new facility called 'tldr' which is a
> live data recorder. This is intended to aid support of production
> QEMU deployments, by providing a data collection facility that
> can be left "always on", such that in the event of a QEMU crash
> or guest OS crash, certain important information will be recorded.
> 
> The data recorder has to be enabled by adding a "tldr-config"
> object instance with -object / object_add.
> 
> With this series, the data recorder collects
> 
>  * error_report/warn_report/error_printf messages
>  * QMP events

It should probably be recorded at emission, not per monitor

>  * QMP errors
>  * QMP commands, which are not query-XXX/qom-list,
>    or any command associated with a QMP error.
>  * qemu_log output
> 
> The last item is intended primarily to be used in combination
> with tracing and its 'log' backend, to have continuous
> recording of certain interesting probes.
> 
> Each type of data is collected in a distinct circular
> buffer, such that high volumes of one type of data, does
> not result in overwriting other types of low volume data.
> 
> The data recorder is set to trigger on SIGBUS, SIGSEGV,
> SIGILL, SIGFPE and SIGABRT, or manually via QMP 'tldr-dump'.

Instead of allocating and dumping by a signal handler, why not use a file-backed
mmap or a memfd? We would need to define a small fixed header for the file
layout/buckets.  This would avoid all the issues with signal handlers.

There would be a small window for consistency on crash, but that's probably
acceptable.

Plain ASCII section markers would probably be easier to deal with than emojis

-- 
Marc-André Lureau <marcandre.lureau@redhat.com>

Re: [PATCH 03/21] system: rename qemu_init_sigbus to qemu_init_crash_signals

[marcandre.lureau]

On Thu, 15 Oct 2026 12:20:26 +0100, Daniel P. Berrangé <berrange@redhat.com> wrote:
> Support for handling further crash signals will be introduced
> in subsequent commits, so remove 'sigbus' from the initializer
> function name.

Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>

-- 
Marc-André Lureau <marcandre.lureau@redhat.com>

Re: [PATCH 02/21] system: move SIGBUS signal handling to signal.c

[marcandre.lureau]

On Thu, 15 Oct 2026 12:20:25 +0100, Daniel P. Berrangé <berrange@redhat.com> wrote:
> Introduce a new file 'system/signal.c' to hold the SIGBUS signal
> handler, since subsequent commits are going to expand the signal
> handling logic somewhat.
> 
> The existing copyright header/license boilerplate is retained
> instead of a SPDX tag, to respect historic context.
> 
> [...]

Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>

-- 
Marc-André Lureau <marcandre.lureau@redhat.com>

Re: [PATCH 00/21] system: introduce a live data recorder facility

[Daniel P. Berrangé]

On Thu, Oct 15, 2026 at 04:15:52PM +0400, marcandre.lureau@redhat.com wrote:
> > This series introduces a new facility called 'tldr' which is a
> > live data recorder. This is intended to aid support of production
> > QEMU deployments, by providing a data collection facility that
> > can be left "always on", such that in the event of a QEMU crash
> > or guest OS crash, certain important information will be recorded.
> > 
> > The data recorder has to be enabled by adding a "tldr-config"
> > object instance with -object / object_add.
> > 
> > With this series, the data recorder collects
> > 
> >  * error_report/warn_report/error_printf messages
> >  * QMP events
> 
> It should probably be recorded at emission, not per monitor

Opps.

> >  * QMP errors
> >  * QMP commands, which are not query-XXX/qom-list,
> >    or any command associated with a QMP error.
> >  * qemu_log output
> > 
> > The last item is intended primarily to be used in combination
> > with tracing and its 'log' backend, to have continuous
> > recording of certain interesting probes.
> > 
> > Each type of data is collected in a distinct circular
> > buffer, such that high volumes of one type of data, does
> > not result in overwriting other types of low volume data.
> > 
> > The data recorder is set to trigger on SIGBUS, SIGSEGV,
> > SIGILL, SIGFPE and SIGABRT, or manually via QMP 'tldr-dump'.
> 
> Instead of allocating and dumping by a signal handler, why not use a file-backed
> mmap or a memfd? We would need to define a small fixed header for the file
> layout/buckets.  This would avoid all the issues with signal handlers.

That would impose a continuous I/O workload on data recording, which
is something I wanted to explicitly avoid, until the time the data is
needed to be preserved. This only persists the final data, not any
intermediate data that is overwritten as the circular buffer wraps.
Coupling the in-memory layout to the on-disk layout also means that
dumps would always be the max size, since we would not have grow-on
demand buffers.

File backed would be even worse for the follow up goal of including
statistics, since it'd be putting disk I/O into the stats gathering
path.

> There would be a small window for consistency on crash, but that's probably
> acceptable.
> 
> Plain ASCII section markers would probably be easier to deal with than emojis



With regards,
Daniel
-- 
|: https://berrange.com       ~~        https://hachyderm.io/@berrange :|
|: https://libvirt.org          ~~          https://entangle-photo.org :|
|: https://pixelfed.art/berrange   ~~    https://fstop138.berrange.com :|

Re: [PATCH 17/21] monitor: capture monitor commands and events in data recorder

[Dr. David Alan Gilbert]

* Daniel P. Berrangé (berrange@redhat.com) wrote:
> QMP monitor commands which change system state (ie not query-XXX
> commands) are relevant to the data recorder, as they reflect what
> has been done to QEMU at runtime by the mgmt application.

Do you have any filters for avoiding capturing passwords/keys
(which admitedly they should be doing by files, but hmm).

Dave

> QMP monitor events are relevant to the data recorder, as they
> reflect a combination of what has been done to QEMU at runtime by
> the mgmt application, as well as changes triggered by the guest OS.
> 
> QMP error responses are relevant either to reveal mistakes in mgmt
> apps, or to reveal problems in QEMU. If the associated QMP command
> was not already recorded, it will be recorded for any error
> response to provide context to the failure.
> 
> QMP normal responses are skipped since the data can be volumous
> and thus result in poor signal/noise ratio.
> 
> The QMP commands are never captured to a string buffer as input
> is streamed into the JSON parser and the result is spat out as
> a QObject. Thus, the commands need to be formatted back into JSON
> string format for the data recorder. This is a mild but acceptable
> performance overhead, in the overall context of handling a QMP
> command.
> 
> The QMP events are captured immediately before they are written
> back on the wire where already in string format.
> 
> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> ---
>  monitor/qmp.c | 67 +++++++++++++++++++++++++++++++++++++++++++++------
>  1 file changed, 59 insertions(+), 8 deletions(-)
> 
> diff --git a/monitor/qmp.c b/monitor/qmp.c
> index ccb39a48d4c..659ad23d832 100644
> --- a/monitor/qmp.c
> +++ b/monitor/qmp.c
> @@ -34,6 +34,7 @@
>  #include "qobject/qjson.h"
>  #include "qobject/qlist.h"
>  #include "qom/object_interfaces.h"
> +#include "qemu/tldr.h"
>  #include "trace.h"
>  
>  /*
> @@ -80,7 +81,8 @@ static MonitorQMP *qmp_dispatcher_current_mon;
>  OBJECT_DEFINE_TYPE(MonitorQMP, monitor_qmp, MONITOR_QMP, MONITOR);
>  
>  static void monitor_qmp_cleanup_req_queue_locked(MonitorQMP *mon);
> -static void qmp_send_response(MonitorQMP *mon, const QDict *rsp);
> +static void qmp_send_response(MonitorQMP *mon, const QDict *rsp,
> +                              const QObject *ctx);
>  
>  static void monitor_qmp_finalize(Object *obj)
>  {
> @@ -169,7 +171,7 @@ static void monitor_qmp_emit_event(Monitor *mon, QAPIEvent event, QDict *qdict)
>          }
>      }
>  
> -    qmp_send_response(qmp, qdict);
> +    qmp_send_response(qmp, qdict, NULL);
>  }
>  
>  static bool monitor_qmp_requires_iothread(const Monitor *mon)
> @@ -243,7 +245,8 @@ static void monitor_qmp_cleanup_queue_and_resume(MonitorQMP *mon)
>  
>  }
>  
> -static void qmp_send_response(MonitorQMP *mon, const QDict *rsp)
> +static void qmp_send_response(MonitorQMP *mon, const QDict *rsp,
> +                              const QObject *ctx)
>  {
>      const QObject *data = QOBJECT(rsp);
>      GString *json;
> @@ -252,6 +255,30 @@ static void qmp_send_response(MonitorQMP *mon, const QDict *rsp)
>      assert(json != NULL);
>      trace_monitor_qmp_respond(mon, json->str);
>  
> +    /*
> +     * Errors are always useful to record. We recorded the
> +     * command in the caller in many cases, but if not, then
> +     * 'ctx' will be non-NULL and we should record it now
> +     * for context.
> +     */
> +    if (qtldr_msg_enabled(QTLDR_MSG_BUCKET_TYPE_COMMAND) &&
> +        qdict_haskey(rsp, "error")) {
> +        if (ctx) {
> +            g_autoptr(GString) ctxstr = qobject_to_json(ctx);
> +            qtldr_msg_record(QTLDR_MSG_BUCKET_TYPE_COMMAND, ctxstr->str);
> +        }
> +        qtldr_msg_record(QTLDR_MSG_BUCKET_TYPE_COMMAND, json->str);
> +    }
> +
> +    /*
> +     * Capture events, since they represent changes
> +     * in state or behaviour
> +     */
> +    if (qtldr_msg_enabled(QTLDR_MSG_BUCKET_TYPE_EVENT) &&
> +        qdict_haskey(rsp, "event")) {
> +        qtldr_msg_record(QTLDR_MSG_BUCKET_TYPE_EVENT, json->str);
> +    }
> +
>      g_string_append_c(json, '\n');
>      monitor_puts(&mon->parent_obj, json->str);
>  
> @@ -263,10 +290,10 @@ static void qmp_send_response(MonitorQMP *mon, const QDict *rsp)
>   * Null @rsp can only happen for commands with QCO_NO_SUCCESS_RESP.
>   * Nothing is emitted then.
>   */
> -static void monitor_qmp_respond(MonitorQMP *mon, QDict *rsp)
> +static void monitor_qmp_respond(MonitorQMP *mon, QDict *rsp, QObject *ctx)
>  {
>      if (rsp) {
> -        qmp_send_response(mon, rsp);
> +        qmp_send_response(mon, rsp, ctx);
>      }
>  }
>  
> @@ -276,9 +303,33 @@ static void monitor_qmp_respond(MonitorQMP *mon, QDict *rsp)
>   */
>  static void monitor_qmp_dispatch(MonitorQMP *mon, QObject *req)
>  {
> +    QObject *ctx = req;
>      QDict *rsp;
>      QDict *error;
>  
> +    /*
> +     * Capture commands which are liable to change state,
> +     * i.e. exclude purely information query comamnds
> +     */
> +    if (qtldr_msg_enabled(QTLDR_MSG_BUCKET_TYPE_COMMAND)) {
> +        QDict *cmd = qobject_to(QDict, req);
> +        if (cmd != NULL) {
> +            const char *cmdname = qdict_get_try_str(cmd, "execute")
> +                ?: qdict_get_try_str(cmd, "exec-oob");
> +            if (cmdname != NULL &&
> +                !g_str_has_prefix(cmdname, "query-") &&
> +                !g_str_has_prefix(cmdname, "qom-list")) {
> +                g_autoptr(GString) cmdstr = qobject_to_json(req);
> +                qtldr_msg_record(QTLDR_MSG_BUCKET_TYPE_COMMAND, cmdstr->str);
> +                /*
> +                 * Prevent recording cmd twice if the later response
> +                 * is an error
> +                 */
> +                ctx = NULL;
> +            }
> +        }
> +    }
> +
>      rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon),
>                         &mon->parent_obj);
>  
> @@ -294,7 +345,7 @@ static void monitor_qmp_dispatch(MonitorQMP *mon, QObject *req)
>          }
>      }
>  
> -    monitor_qmp_respond(mon, rsp);
> +    monitor_qmp_respond(mon, rsp, ctx);
>      qobject_unref(rsp);
>  }
>  
> @@ -455,7 +506,7 @@ void coroutine_fn monitor_qmp_dispatcher_co(void *data)
>              trace_monitor_qmp_err_in_band(error_get_pretty(req_obj->err));
>              rsp = qmp_error_response(req_obj->err);
>              req_obj->err = NULL;
> -            monitor_qmp_respond(mon, rsp);
> +            monitor_qmp_respond(mon, rsp, NULL);
>              qobject_unref(rsp);
>          }
>  
> @@ -623,7 +674,7 @@ static void monitor_qmp_event(void *opaque, QEMUChrEvent event)
>              monitor_qmp_caps_reset(mon);
>          }
>          data = qmp_greeting(mon);
> -        qmp_send_response(mon, data);
> +        qmp_send_response(mon, data, NULL);
>          qobject_unref(data);
>          break;
>      case CHR_EVENT_CLOSED:
> -- 
> 2.56.0
> 
-- 
 -----Open up your eyes, open up your mind, open up your code -------   
/ Dr. David Alan Gilbert    |       Running GNU/Linux       | Happy  \ 
\        dave @ treblig.org |                               | In Hex /
 \ _________________________|_____ http://www.treblig.org   |_______/

Re: [PATCH 17/21] monitor: capture monitor commands and events in data recorder

[Daniel P. Berrangé]

On Thu, Oct 15, 2026 at 03:57:09PM +0000, Dr. David Alan Gilbert wrote:
> * Daniel P. Berrangé (berrange@redhat.com) wrote:
> > QMP monitor commands which change system state (ie not query-XXX
> > commands) are relevant to the data recorder, as they reflect what
> > has been done to QEMU at runtime by the mgmt application.
> 
> Do you have any filters for avoiding capturing passwords/keys
> (which admitedly they should be doing by files, but hmm).

The "secret" object type supports this. You create a primary
key which you pass to QEMU in a file on disk at startup, so
it is invisible in process listing or QMP logs. Then any
further secrets created at runtime should be encrypted with
this primary key, so they are not compromised if capturing
QMP logs

  https://www.qemu.org/docs/master/system/secrets.html#best-practice


With regards,
Daniel
-- 
|: https://berrange.com       ~~        https://hachyderm.io/@berrange :|
|: https://libvirt.org          ~~          https://entangle-photo.org :|
|: https://pixelfed.art/berrange   ~~    https://fstop138.berrange.com :|

Re: [PATCH 00/21] system: introduce a live data recorder facility

[Pierrick Bouvier]

Hi Daniel,

On 10/15/2026 4:20 AM, Daniel P. Berrangé wrote:
> This series introduces a new facility called 'tldr' which is a
> live data recorder. This is intended to aid support of production
> QEMU deployments, by providing a data collection facility that
> can be left "always on", such that in the event of a QEMU crash
> or guest OS crash, certain important information will be recorded.
> 
> The data recorder has to be enabled by adding a "tldr-config"
> object instance with -object / object_add.
> 
> With this series, the data recorder collects
> 
>  * error_report/warn_report/error_printf messages
>  * QMP events
>  * QMP errors
>  * QMP commands, which are not query-XXX/qom-list,
>    or any command associated with a QMP error.
>  * qemu_log output
> 
> The last item is intended primarily to be used in combination
> with tracing and its 'log' backend, to have continuous
> recording of certain interesting probes.
> 
> Each type of data is collected in a distinct circular
> buffer, such that high volumes of one type of data, does
> not result in overwriting other types of low volume data.
> 
> The data recorder is set to trigger on SIGBUS, SIGSEGV,
> SIGILL, SIGFPE and SIGABRT, or manually via QMP 'tldr-dump'.
> 
> There is a open question in one of the patches about SIGBUS
> handling, since that is already used for MCE capture and I'm
> unclear if I'm breaking something about KVM MCE reporting.
> 
> This series proposes using sigaltstack() to register an
> alternative signal handling stack. This would mainly be
> useful if we have a crash from a stack overflow, but the
> cost is that we have 8k extra per thread, and I'm not
> entirely convinced that's a beneficial tradeoff. Might
> be better to just accept stack overflows as a risk.
> 
> This series only captures text based info, but as a followup
> I would like to extend this to capture statistics data from
> relevant subsystems. For example, the BlockBackend has its
> own "always on" stats accounting code, which can be queried
> with QMP. Ideally I would like to figure out a way to dump
> all the BlockBackend stats in the data recorder output.
> Likewise perhaps capture NIC stats and/or KVM stats in an
> "always on" manner for some interesting baseline dataset.
> The challenge with that is making it safe to access the
> stats data without taking any mutex locks, since this must
> run from a signal handler.
> 
> A typical recommended usage would be:
> 
>  $QEMU  -object tldr-config,id=tldr
>         -d tldr
>         -trace events=/var/lib/libvirt/qemu/trace.txt
> 
> This leaves dumps to go to QEMU's stderr on crash, but
> to send them to a file
> 
>    -object tldr-config,id=tldr,crash-target=/some/path
> 
> The '-d tldr' flag redirects qemu_log() to the TLDR buffer
> instead of stdout, and the '-trace events' arg defines
> the "always on" tracepoints we want to capture.
>

if I understand correctly, the goal of this new object is to provide a
per-topic rotating log in memory that will be flushed on disk only when
we have a specific event (exit, or any signal). That's sounds like a
good idea!

Why not refactor exisiting trace/log and provide the same feature
directly there? This would be transparent, and user could decide to
either dump the full file (same than before) or benefit from rotating
memory buffered, with any custom size.

$QEMU -d category,size=4M -trace events=out,max-size=16M

Regards,
Pierrick