From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([208.118.235.92]:38255) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UQboJ-0005Q3-F4 for qemu-devel@nongnu.org; Fri, 12 Apr 2013 07:06:28 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1UQboH-0007l8-Ks for qemu-devel@nongnu.org; Fri, 12 Apr 2013 07:06:27 -0400 Received: from mail-wi0-x22a.google.com ([2a00:1450:400c:c05::22a]:45404) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UQboH-0007km-ES for qemu-devel@nongnu.org; Fri, 12 Apr 2013 07:06:25 -0400 Received: by mail-wi0-f170.google.com with SMTP id hm11so1517955wib.3 for ; Fri, 12 Apr 2013 04:06:24 -0700 (PDT) Date: Fri, 12 Apr 2013 13:06:21 +0200 From: Stefan Hajnoczi Message-ID: <20130412110621.GA5102@stefanha-thinkpad.redhat.com> References: <1365679516-12675-1-git-send-email-eiichi.tsukata.xh@hitachi.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1365679516-12675-1-git-send-email-eiichi.tsukata.xh@hitachi.com> Subject: Re: [Qemu-devel] [PATCH v2 0/2] New trace backend: ftrace List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Eiichi Tsukata Cc: qemu-devel@nongnu.org, stefanha@redhat.com, yrl.pp-manager.tt@hitachi.com On Thu, Apr 11, 2013 at 08:25:14PM +0900, Eiichi Tsukata wrote: > This patchset introduces a new tracing backend "ftrace". > > Currently, QEMU tracing backends do not support userspace tracing with ftrace. > Collecting QEMU trace data and kernel trace data simultaneouly is useful for > latency analysis and debugging especially when using KVM. > > With ftrace backend, you can easily collect QEMU-kernel merged trace data > using existing ftrace event-based tracer. If you use KVM, you can effectively > compare VM_EXIT and QEMU userspace handler. > > To try it out, compile QEMU with tracing backend ftrace, then enable KVM events > in ftrace: > > # echo 1 > /sys/kernel/debug/tracing/events/kvm/enable > > After running qemu by root user, you can get the trace: > > # cat /sys/kernel/debug/tracing/trace > > Example: > # tracer: nop > # > # entries-in-buffer/entries-written: 8434/345512 #P:4 > # > # _-----=> irqs-off > # / _----=> need-resched > # | / _---=> hardirq/softirq > # || / _--=> preempt-depth > # ||| / delay > # TASK-PID CPU# |||| TIMESTAMP FUNCTION > # | | | |||| | | > > qemu-system-x86-31930 [000] d... 23580.595951: kvm_exit: reason IO_INSTRUCTION rip 0xc45d info 710048 0 > qemu-system-x86-31930 [000] .... 23580.595954: kvm_emulate_insn: f0000:c45d:e4 71 (real) > qemu-system-x86-31930 [000] .... 23580.595955: kvm_pio: pio_read at 0x71 size 1 count 1 > qemu-system-x86-31930 [000] .... 23580.595956: kvm_userspace_exit: reason KVM_EXIT_IO (2) > qemu-system-x86-31930 [000] ...1 23580.595959: tracing_mark_write: cpu_set_apic_base 00000000fee00900 > qemu-system-x86-31930 [000] ...1 23580.595961: tracing_mark_write: cpu_in addr 0x71 value 0 > qemu-system-x86-31930 [000] d... 23580.595964: kvm_entry: vcpu 0 > > > "tracing_mark_write: cpu_set_apic_base 00000000fee00900" and > "tracing_mark_write: cpu_in addr 0x71 value 0" are QEMU trace data. > Others are Kernel trace data. > > Furthermore, the ftrace backend overhead is smaller than uprobe-based event > tracer or SystemTap. My microbenchmark shows that ftrace tracing backend > overhead is about 0.8us per tracepoint, whereas uprobe-based event tracer > or SystemTap overhead is about 2.0us. > > Changes in v2: > * fix Stefan's mail address. > * use snprintf return value not to waste trace buffer. > > Eiichi Tsukata (2): > trace: Add ftrace tracing backend > trace: document ftrace backend > > configure | 8 +++ > docs/tracing.txt | 16 ++++++ > scripts/tracetool/backend/ftrace.py | 54 +++++++++++++++++++ > trace/Makefile.objs | 1 + > trace/ftrace.c | 102 ++++++++++++++++++++++++++++++++++++ > trace/ftrace.h | 10 ++++ > 6 files changed, 191 insertions(+) > create mode 100644 scripts/tracetool/backend/ftrace.py > create mode 100644 trace/ftrace.c > create mode 100644 trace/ftrace.h > > -- > 1.8.1.4 > > Thanks, applied to my tracing tree: https://github.com/stefanha/qemu/commits/tracing Stefan