Re: [Qemu-devel] [PATCH v8 0/5] hypertrace: Lightweight guest-to-QEMU trace channel

["Lluís Vilanova" <vilanova@ac.upc.edu>]

Stefan Hajnoczi writes:

> On Sun, Jul 30, 2017 at 05:08:18PM +0300, Lluís Vilanova wrote:
>> The hypertrace channel allows guest code to emit events in QEMU (the host) using
>> its tracing infrastructure (see "docs/trace.txt"). This works in both 'system'
>> and 'user' modes, is architecture-agnostic and introduces minimal noise on the
>> guest.
>> 
>> See first commit for a full description, use-cases and an example.
>> 
>> Signed-off-by: Lluís Vilanova <vilanova@ac.upc.edu>
>> ---
>> 
>> Changes in v8
>> =============
>> 
>> * Do not use 'seq' when there's no extra hypertrace arguments (BSD behaves
>> differently for "seq 0").
>> * Fix compilation for bsd-user.

> Hi Lluís,
> Any changes regarding the fundamental approach since September 2016?

> Back then I had the following concerns about hypertrace for full-system
> virtualization:

> Going to QEMU for every guest trace event has high overhead under
> virtualization.  The alternative approach is recording separate traces
> and merging them for analysis.  This is possible with trace-cmd [1] and
> LTTng [2].

> Merging traces eliminates the performance bottleneck and does not
> require new paravirt interfaces or guest tracing libraries.  I think it
> it would be a distraction to support hypertrace for the virtualization
> use case because it fundamentally has a high overhead.

> I see promise in using hypertrace for TCG because it is low-overhead
> when running inside the QEMU process.  I'll review the patches again
> with this in mind and not focus on virtualization.

> [1] https://lists.nongnu.org/archive/html/qemu-devel/2016-03/msg00887.html
> [2] http://archive.eclipse.org/tracecompass/doc/stable/org.eclipse.tracecompass.doc.user/Virtual-Machine-Analysis.html
>     and also generic trace synchronization
>     http://archive.eclipse.org/tracecompass/doc/stable/org.eclipse.tracecompass.doc.user/Trace-synchronization.html#Trace_synchronization

There's been no fundamental changes since then (just the few bits listed in the
v5-v8 changelog).

But I'm kind of split on this one.

If you want high-performance trace correlation, this will work much better for
TCG than virtualization (where [1] will probably be more efficient).

If you want a hook to trigger other operations (like in the docs example), I
think this is the right approach. In fact, my initial interest in hypertrace was
for instrumentation, so maybe this should be subsumed into the proposal of a
stable instrumentation API.

What do you think?

Cheers,
  Lluis


> Stefan

>> Changes in v7
>> =============
>> 
>> * Use 'expr' instead of assuming 'bash' when generating the "emit.c" file.
>> * Restore generation of trace-events-all.
>> 
>> 
>> Changes in v6
>> =============
>> 
>> * Fix compilation errors.
>> 
>> 
>> Changes in v5
>> =============
>> 
>> * Rebase on 5a477a7806.
>> * Fix typo in "bsd-user/main.c" [Stephan Hajnoczi].
>> * Replace abort() with exit() in command-line errors [Stephan Hajnoczi].
>> * Fix alignment of data and control channels [Stephan Hajnoczi].
>> * Fix signal reflection in user-mode (SIGINT, SIGABRT, SIGSEGV) [Stephan Hajnoczi].
>> * Clarify semantics of hypertrace_guest_mmap_check() [Stephan Hajnoczi].
>> * Use uintptr_t instead of unsigned long in SEGV handler [Stephan Hajnoczi].
>> * Emit hypertrace's event with host-endian arguments [Stephan Hajnoczi].
>> * Enable true concurrency between user-mode guest threads by using a spearate control channel page per client [Stephan Hajnoczi].
>> * Remove unused PAGE_SIZE define [Stephan Hajnoczi].
>> * Submit linux kernel API module separately to Linux upstream [Stephan Hajnoczi].
>> * Assume guest code events are always enabled.
>> 
>> 
>> Changes in v4
>> =============
>> 
>> * Fix typo in stap script example.
>> * Fix compilation instructions in doc/hypertrace.txt.
>> * Rebase on 0737f32daf.
>> 
>> 
>> Changes in v3
>> =============
>> 
>> * Rebase on 4a58f35.
>> * Remove debugging printf's.
>> * Fix style issues identified by checkpatch.
>> * Fix control channel mapping in guest linux module.
>> * Add a short event description in "trace-events".
>> * Polish documentation in 1st patch.
>> 
>> 
>> Changes in v2
>> =============
>> 
>> * Remove unnecessary casts for g2h() [Eric Blake].
>> * Use perror() [Eric Blake].
>> * Avoid expansions in application example [Eric Blake].
>> * Add copyright in document "hypertrace.txt" [Eric Blake].
>> * Make the user-mode hypertrace invocations thread-safe [Stefan Hajnoczi].
>> * Split dynamic hypertrace configuration into a separate "config" channel.
>> 
>> Lluís Vilanova (5):
>> hypertrace: Add documentation
>> hypertrace: Add tracing event "guest_hypertrace"
>> hypertrace: [*-user] Add QEMU-side proxy to "guest_hypertrace" event
>> hypertrace: [softmmu] Add QEMU-side proxy to "guest_hypertrace" event
>> hypertrace: Add guest-side user-level library
>> 
>> 
>> Makefile                           |   11 +
>> Makefile.objs                      |    6 +
>> bsd-user/main.c                    |   17 +
>> bsd-user/mmap.c                    |   15 +
>> bsd-user/qemu.h                    |    3 
>> bsd-user/syscall.c                 |   34 ++-
>> configure                          |   36 +++
>> docs/devel/tracing.txt             |    3 
>> docs/hypertrace.txt                |  225 ++++++++++++++++++++
>> hypertrace/Makefile.objs           |   25 ++
>> hypertrace/common.c                |   55 +++++
>> hypertrace/common.h                |   25 ++
>> hypertrace/guest/Makefile          |   30 +++
>> hypertrace/guest/common.c          |  301 ++++++++++++++++++++++++++
>> hypertrace/guest/qemu-hypertrace.h |   80 +++++++
>> hypertrace/softmmu.c               |  237 +++++++++++++++++++++
>> hypertrace/user.c                  |  415 ++++++++++++++++++++++++++++++++++++
>> hypertrace/user.h                  |   71 ++++++
>> include/hw/pci/pci.h               |    2 
>> include/qom/cpu.h                  |    4 
>> linux-user/main.c                  |   19 ++
>> linux-user/mmap.c                  |   16 +
>> linux-user/qemu.h                  |    3 
>> linux-user/signal.c                |   12 +
>> linux-user/syscall.c               |   31 ++-
>> rules.mak                          |    2 
>> trace-events                       |   11 +
>> 27 files changed, 1660 insertions(+), 29 deletions(-)
>> create mode 100644 docs/hypertrace.txt
>> create mode 100644 hypertrace/Makefile.objs
>> create mode 100644 hypertrace/common.c
>> create mode 100644 hypertrace/common.h
>> create mode 100644 hypertrace/guest/Makefile
>> create mode 100644 hypertrace/guest/common.c
>> create mode 100644 hypertrace/guest/qemu-hypertrace.h
>> create mode 100644 hypertrace/softmmu.c
>> create mode 100644 hypertrace/user.c
>> create mode 100644 hypertrace/user.h
>> 
>> 
>> To: qemu-devel@nongnu.org
>> Cc: Stefan Hajnoczi <stefanha@redhat.com>
>> Cc: Eric Blake <eblake@redhat.com>
>> Cc: Luiz Capitulino <lcapitulino@redhat.com>
>> Cc: Daniel P Berrange <berrange@redhat.com>