From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([208.118.235.92]:42179) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1S65tG-0001f8-Vl for qemu-devel@nongnu.org; Fri, 09 Mar 2012 14:54:16 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1S65tE-0000g1-3f for qemu-devel@nongnu.org; Fri, 09 Mar 2012 14:54:14 -0500 Received: from e34.co.us.ibm.com ([32.97.110.152]:52138) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1S65tD-0000Z8-R5 for qemu-devel@nongnu.org; Fri, 09 Mar 2012 14:54:12 -0500 Received: from /spool/local by e34.co.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Fri, 9 Mar 2012 12:54:07 -0700 Received: from d03relay04.boulder.ibm.com (d03relay04.boulder.ibm.com [9.17.195.106]) by d03dlp02.boulder.ibm.com (Postfix) with ESMTP id C20E53E40050 for ; Fri, 9 Mar 2012 12:53:33 -0700 (MST) Received: from d03av06.boulder.ibm.com (d03av06.boulder.ibm.com [9.17.195.245]) by d03relay04.boulder.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id q29JrOI1095338 for ; Fri, 9 Mar 2012 12:53:24 -0700 Received: from d03av06.boulder.ibm.com (loopback [127.0.0.1]) by d03av06.boulder.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id q29JreZE012600 for ; Fri, 9 Mar 2012 12:53:40 -0700 Message-ID: <4F5A5FB2.9080701@us.ibm.com> Date: Fri, 09 Mar 2012 13:53:22 -0600 From: Anthony Liguori MIME-Version: 1.0 References: <1331302420-29737-1-git-send-email-stefanha@linux.vnet.ibm.com> In-Reply-To: <1331302420-29737-1-git-send-email-stefanha@linux.vnet.ibm.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [PATCH] kvm: add flightrecorder script List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Stefan Hajnoczi Cc: qemu-devel@nongnu.org, kvm@vger.kernel.org, Avi Kivity On 03/09/2012 08:13 AM, Stefan Hajnoczi wrote: > The kvm kernel module includes a number of trace events which can be > useful when debugging system behavior. Even on production systems these > trace events can be used to observe guest behavior and identify the > source of problems. > > The kvm_flightrecorder script is a command-line wrapper for the > /sys/kernel/debug/tracing interface. Kernel symbols do not need to be > installed. > > This script captures a fixed-size buffer of KVM trace events. Recent > events overwrite the oldest events when the buffer size is exceeded and > it is possible to leave KVM tracing enabled for any period of time with > just a fixed-size buffer. If the buffer is large enough this script is > a useful tool for collecting detailed information after an issue occurs > with a guest. Hence the name "flight recorder". > > The script can also be used in 'tail' mode to simply view KVM trace > events as they occur. This is handy for development and to ensure that > the guest is indeed running. > > Signed-off-by: Stefan Hajnoczi > --- > scripts/kvm/kvm_flightrecorder | 126 ++++++++++++++++++++++++++++++++++++++++ > 1 files changed, 126 insertions(+), 0 deletions(-) > create mode 100755 scripts/kvm/kvm_flightrecorder I assume this should go through uq/master... Thanks for sending this out. Regards, Anthony Liguori > > diff --git a/scripts/kvm/kvm_flightrecorder b/scripts/kvm/kvm_flightrecorder > new file mode 100755 > index 0000000..7fb1c2d > --- /dev/null > +++ b/scripts/kvm/kvm_flightrecorder > @@ -0,0 +1,126 @@ > +#!/usr/bin/env python > +# > +# KVM Flight Recorder - ring buffer tracing script > +# > +# Copyright (C) 2012 IBM Corp > +# > +# Author: Stefan Hajnoczi > +# > +# This script provides a command-line interface to kvm ftrace and is designed > +# to be used as a flight recorder that is always running. To start in-memory > +# recording: > +# > +# sudo kvm_flightrecorder start 8192 # 8 MB per-cpu ring buffers > +# > +# The per-cpu ring buffer size can be given in KB as an optional argument to > +# the 'start' subcommand. > +# > +# To stop the flight recorder: > +# > +# sudo kvm_flightrecorder stop > +# > +# To dump the contents of the flight recorder (this can be done when the > +# recorder is stopped or while it is running): > +# > +# sudo kvm_flightrecorder dump>/path/to/dump.txt > +# > +# To observe the trace while it is running, use the 'tail' subcommand: > +# > +# sudo kvm_flightrecorder tail > +# > +# Note that the flight recorder may impact overall system performance by > +# consuming CPU cycles. No disk I/O is performed since the ring buffer holds a > +# fixed-size in-memory trace. > + > +import sys > +import os > + > +tracing_dir = '/sys/kernel/debug/tracing' > + > +def trace_path(*args): > + return os.path.join(tracing_dir, *args) > + > +def write_file(path, data): > + open(path, 'wb').write(data) > + > +def enable_event(subsystem, event, enable): > + write_file(trace_path('events', subsystem, event, 'enable'), '1' if enable else '0') > + > +def enable_subsystem(subsystem, enable): > + write_file(trace_path('events', subsystem, 'enable'), '1' if enable else '0') > + > +def start_tracing(): > + enable_subsystem('kvm', True) > + write_file(trace_path('tracing_on'), '1') > + > +def stop_tracing(): > + write_file(trace_path('tracing_on'), '0') > + enable_subsystem('kvm', False) > + write_file(trace_path('events', 'enable'), '0') > + write_file(trace_path('current_tracer'), 'nop') > + > +def dump_trace(): > + tracefile = open(trace_path('trace'), 'r') > + try: > + lines = True > + while lines: > + lines = tracefile.readlines(64 * 1024) > + sys.stdout.writelines(lines) > + except KeyboardInterrupt: > + pass > + > +def tail_trace(): > + try: > + for line in open(trace_path('trace_pipe'), 'r'): > + sys.stdout.write(line) > + except KeyboardInterrupt: > + pass > + > +def usage(): > + print 'Usage: %s start [buffer_size_kb] | stop | dump | tail' % sys.argv[0] > + print 'Control the KVM flight recorder tracing.' > + sys.exit(0) > + > +def main(): > + if len(sys.argv)< 2: > + usage() > + > + cmd = sys.argv[1] > + if cmd == '--version': > + print 'kvm_flightrecorder version 1.0' > + sys.exit(0) > + > + if not os.path.isdir(tracing_dir): > + print 'Unable to tracing debugfs directory, try:' > + print 'mount -t debugfs none /sys/kernel/debug' > + sys.exit(1) > + if not os.access(tracing_dir, os.W_OK): > + print 'Unable to write to tracing debugfs directory, please run as root' > + sys.exit(1) > + > + if cmd == 'start': > + stop_tracing() # clean up first > + > + if len(sys.argv) == 3: > + try: > + buffer_size_kb = int(sys.argv[2]) > + except ValueError: > + print 'Invalid per-cpu trace buffer size in KB' > + sys.exit(1) > + write_file(trace_path('buffer_size_kb'), str(buffer_size_kb)) > + print 'Per-CPU ring buffer size set to %d KB' % buffer_size_kb > + > + start_tracing() > + print 'KVM flight recorder enabled' > + elif cmd == 'stop': > + stop_tracing() > + print 'KVM flight recorder disabled' > + elif cmd == 'dump': > + dump_trace() > + elif cmd == 'tail': > + tail_trace() > + else: > + usage() > + > +if __name__ == '__main__': > + sys.exit(main())