From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([208.118.235.92]:33175) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UPCoS-0004qv-UX for qemu-devel@nongnu.org; Mon, 08 Apr 2013 10:12:56 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1UPCoM-00044P-7o for qemu-devel@nongnu.org; Mon, 08 Apr 2013 10:12:48 -0400 Received: from mail-we0-x234.google.com ([2a00:1450:400c:c03::234]:62502) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UPCoM-00042b-2P for qemu-devel@nongnu.org; Mon, 08 Apr 2013 10:12:42 -0400 Received: by mail-we0-f180.google.com with SMTP id r5so4424856wey.39 for ; Mon, 08 Apr 2013 07:12:41 -0700 (PDT) Date: Mon, 8 Apr 2013 16:12:34 +0200 From: Stefan Hajnoczi Message-ID: <20130408141234.GF4429@stefanha-thinkpad.redhat.com> References: <1364970659-5715-1-git-send-email-eiichi.tsukata.xh@hitachi.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1364970659-5715-1-git-send-email-eiichi.tsukata.xh@hitachi.com> Subject: Re: [Qemu-devel] [PATCH 0/2] New trace backend: ftrace List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Eiichi Tsukata Cc: qemu-devel@nongnu.org On Wed, Apr 03, 2013 at 03:30:57PM +0900, Eiichi Tsukata wrote: > This patchset introduces a new tracing backend "ftrace". > > Currently, QEMU tracing backends do not support userspace tracing with ftrace. > Collecting QEMU trace data and kernel trace data simultaniouly is useful for > latency analysis and debugging especially when using KVM. > > With ftrace backend, you can easily collect QEMU-kernel merged trace data > using existing ftrace event-based tracer. If you use KVM, you can effectively > compare VM_EXIT and QEMU userspace handler. > > To try it out, compile QEMU with tracing backend ftrace, then enable KVM events > in ftrace: > > # echo 1 > /sys/kernel/debug/tracing/events/kvm/enable > > After running qemu by root user, you can get the trace: > > # cat /sys/kernel/debug/tracing/trace > > Example: > # tracer: nop > # > # entries-in-buffer/entries-written: 8434/345512 #P:4 > # > # _-----=> irqs-off > # / _----=> need-resched > # | / _---=> hardirq/softirq > # || / _--=> preempt-depth > # ||| / delay > # TASK-PID CPU# |||| TIMESTAMP FUNCTION > # | | | |||| | | > > qemu-system-x86-31930 [000] d... 23580.595951: kvm_exit: reason IO_INSTRUCTION rip 0xc45d info 710048 0 > qemu-system-x86-31930 [000] .... 23580.595954: kvm_emulate_insn: f0000:c45d:e4 71 (real) > qemu-system-x86-31930 [000] .... 23580.595955: kvm_pio: pio_read at 0x71 size 1 count 1 > qemu-system-x86-31930 [000] .... 23580.595956: kvm_userspace_exit: reason KVM_EXIT_IO (2) > qemu-system-x86-31930 [000] ...1 23580.595959: tracing_mark_write: cpu_set_apic_base 00000000fee00900 > qemu-system-x86-31930 [000] ...1 23580.595961: tracing_mark_write: cpu_in addr 0x71 value 0 > qemu-system-x86-31930 [000] d... 23580.595964: kvm_entry: vcpu 0 > > > "tracing_mark_write: cpu_set_apic_base 00000000fee00900" and > "tracing_mark_write: cpu_in addr 0x71 value 0" are QEMU trace data. > Others are Kernel trace data. > > Furthermore, the ftrace backend overhead is smaller than uprobe-based event > tracer or SystemTap. My microbenchmark shows that ftrace tracing backend > overhead is about 0.8us per tracepoint, whereas uprobe-based event tracer > or SystemTap overhead is about 2.0us. > > Eiichi Tsukata (2): > trace: Add ftrace tracing backend > trace: document frace backend > > configure | 8 +++ > docs/tracing.txt | 16 ++++++ > scripts/tracetool/backend/ftrace.py | 53 +++++++++++++++++++ > trace/Makefile.objs | 1 + > trace/ftrace.c | 102 ++++++++++++++++++++++++++++++++++++ > trace/ftrace.h | 10 ++++ > 6 files changed, 190 insertions(+) > create mode 100644 scripts/tracetool/backend/ftrace.py > create mode 100644 trace/ftrace.c > create mode 100644 trace/ftrace.h Looks nice. I posted two comments. Stefan