* [Qemu-devel] [RFC PATCH v2 1/4] Converting tracetool.sh to tracetool.py
2012-01-09 15:46 [Qemu-devel] [RFC PATCH v2 0/4] simpletrace : support var num of args and strings Harsh Prateek Bora
@ 2012-01-09 15:46 ` Harsh Prateek Bora
2012-01-09 21:06 ` Andreas Färber
2012-01-09 15:46 ` [Qemu-devel] [RFC PATCH v2 2/4] Makefile and configure changes for tracetool.py Harsh Prateek Bora
` (3 subsequent siblings)
4 siblings, 1 reply; 18+ messages in thread
From: Harsh Prateek Bora @ 2012-01-09 15:46 UTC (permalink / raw)
To: qemu-devel; +Cc: aneesh.kumar, mathieu.desnoyers, vilanova, stefanha
Note: Upstream Qemu have build issues with LTTng ust backend, as the trace
event APIs are not yet stable in ust and therefore ust backend is not
supported in this script as of now. Once the ust API stablises, this script
can be updated for ust backend also.
Signed-off-by: Harsh Prateek Bora <harsh@linux.vnet.ibm.com>
---
scripts/tracetool | 643 --------------------------------------------------
scripts/tracetool.py | 505 +++++++++++++++++++++++++++++++++++++++
2 files changed, 505 insertions(+), 643 deletions(-)
delete mode 100755 scripts/tracetool
create mode 100755 scripts/tracetool.py
diff --git a/scripts/tracetool b/scripts/tracetool
deleted file mode 100755
index 4c9951d..0000000
--- a/scripts/tracetool
+++ /dev/null
@@ -1,643 +0,0 @@
-#!/bin/sh
-#
-# Code generator for trace events
-#
-# Copyright IBM, Corp. 2010
-#
-# This work is licensed under the terms of the GNU GPL, version 2. See
-# the COPYING file in the top-level directory.
-
-# Disable pathname expansion, makes processing text with '*' characters simpler
-set -f
-
-usage()
-{
- cat >&2 <<EOF
-usage: $0 [--nop | --simple | --stderr | --ust | --dtrace] [-h | -c]
-Generate tracing code for a file on stdin.
-
-Backends:
- --nop Tracing disabled
- --simple Simple built-in backend
- --stderr Stderr built-in backend
- --ust LTTng User Space Tracing backend
- --dtrace DTrace/SystemTAP backend
-
-Output formats:
- -h Generate .h file
- -c Generate .c file
- -d Generate .d file (DTrace only)
- --stap Generate .stp file (DTrace with SystemTAP only)
-
-Options:
- --binary [path] Full path to QEMU binary
- --target-arch [arch] QEMU emulator target arch
- --target-type [type] QEMU emulator target type ('system' or 'user')
- --probe-prefix [prefix] Prefix for dtrace probe names
- (default: qemu-\$targettype-\$targetarch)
-
-EOF
- exit 1
-}
-
-# Print a line without interpreting backslash escapes
-#
-# The built-in echo command may interpret backslash escapes without an option
-# to disable this behavior.
-puts()
-{
- printf "%s\n" "$1"
-}
-
-# Get the name of a trace event
-get_name()
-{
- local name
- name=${1%%\(*}
- echo "${name##* }"
-}
-
-# Get the given property of a trace event
-# 1: trace-events line
-# 2: property name
-# -> return 0 if property is present, or 1 otherwise
-has_property()
-{
- local props prop
- props=${1%%\(*}
- props=${props% *}
- for prop in $props; do
- if [ "$prop" = "$2" ]; then
- return 0
- fi
- done
- return 1
-}
-
-# Get the argument list of a trace event, including types and names
-get_args()
-{
- local args
- args=${1#*\(}
- args=${args%%\)*}
- echo "$args"
-}
-
-# Get the argument name list of a trace event
-get_argnames()
-{
- local nfields field name sep
- nfields=0
- sep="$2"
- for field in $(get_args "$1"); do
- nfields=$((nfields + 1))
-
- # Drop pointer star
- field=${field#\*}
-
- # Only argument names have commas at the end
- name=${field%,}
- test "$field" = "$name" && continue
-
- printf "%s%s " $name $sep
- done
-
- # Last argument name
- if [ "$nfields" -gt 1 ]
- then
- printf "%s" "$name"
- fi
-}
-
-# Get the number of arguments to a trace event
-get_argc()
-{
- local name argc
- argc=0
- for name in $(get_argnames "$1", ","); do
- argc=$((argc + 1))
- done
- echo $argc
-}
-
-# Get the format string including double quotes for a trace event
-get_fmt()
-{
- puts "${1#*)}"
-}
-
-linetoh_begin_nop()
-{
- return
-}
-
-linetoh_nop()
-{
- local name args
- name=$(get_name "$1")
- args=$(get_args "$1")
-
- # Define an empty function for the trace event
- cat <<EOF
-static inline void trace_$name($args)
-{
-}
-EOF
-}
-
-linetoh_end_nop()
-{
- return
-}
-
-linetoc_begin_nop()
-{
- return
-}
-
-linetoc_nop()
-{
- # No need for function definitions in nop backend
- return
-}
-
-linetoc_end_nop()
-{
- return
-}
-
-linetoh_begin_simple()
-{
- cat <<EOF
-#include "trace/simple.h"
-EOF
-
- simple_event_num=0
-}
-
-cast_args_to_uint64_t()
-{
- local arg
- for arg in $(get_argnames "$1", ","); do
- printf "%s" "(uint64_t)(uintptr_t)$arg"
- done
-}
-
-linetoh_simple()
-{
- local name args argc trace_args
- name=$(get_name "$1")
- args=$(get_args "$1")
- argc=$(get_argc "$1")
-
- trace_args="$simple_event_num"
- if [ "$argc" -gt 0 ]
- then
- trace_args="$trace_args, $(cast_args_to_uint64_t "$1")"
- fi
-
- cat <<EOF
-static inline void trace_$name($args)
-{
- trace$argc($trace_args);
-}
-EOF
-
- simple_event_num=$((simple_event_num + 1))
-}
-
-linetoh_end_simple()
-{
- cat <<EOF
-#define NR_TRACE_EVENTS $simple_event_num
-extern TraceEvent trace_list[NR_TRACE_EVENTS];
-EOF
-}
-
-linetoc_begin_simple()
-{
- cat <<EOF
-#include "trace.h"
-
-TraceEvent trace_list[] = {
-EOF
- simple_event_num=0
-
-}
-
-linetoc_simple()
-{
- local name
- name=$(get_name "$1")
- cat <<EOF
-{.tp_name = "$name", .state=0},
-EOF
- simple_event_num=$((simple_event_num + 1))
-}
-
-linetoc_end_simple()
-{
- cat <<EOF
-};
-EOF
-}
-
-#STDERR
-linetoh_begin_stderr()
-{
- cat <<EOF
-#include <stdio.h>
-#include "trace/stderr.h"
-
-extern TraceEvent trace_list[];
-EOF
-
- stderr_event_num=0
-}
-
-linetoh_stderr()
-{
- local name args argnames argc fmt
- name=$(get_name "$1")
- args=$(get_args "$1")
- argnames=$(get_argnames "$1" ",")
- argc=$(get_argc "$1")
- fmt=$(get_fmt "$1")
-
- if [ "$argc" -gt 0 ]; then
- argnames=", $argnames"
- fi
-
- cat <<EOF
-static inline void trace_$name($args)
-{
- if (trace_list[$stderr_event_num].state != 0) {
- fprintf(stderr, "$name " $fmt "\n" $argnames);
- }
-}
-EOF
- stderr_event_num=$((stderr_event_num + 1))
-
-}
-
-linetoh_end_stderr()
-{
- cat <<EOF
-#define NR_TRACE_EVENTS $stderr_event_num
-EOF
-}
-
-linetoc_begin_stderr()
-{
- cat <<EOF
-#include "trace.h"
-
-TraceEvent trace_list[] = {
-EOF
- stderr_event_num=0
-}
-
-linetoc_stderr()
-{
- local name
- name=$(get_name "$1")
- cat <<EOF
-{.tp_name = "$name", .state=0},
-EOF
- stderr_event_num=$(($stderr_event_num + 1))
-}
-
-linetoc_end_stderr()
-{
- cat <<EOF
-};
-EOF
-}
-#END OF STDERR
-
-# Clean up after UST headers which pollute the namespace
-ust_clean_namespace() {
- cat <<EOF
-#undef mutex_lock
-#undef mutex_unlock
-#undef inline
-#undef wmb
-EOF
-}
-
-linetoh_begin_ust()
-{
- echo "#include <ust/tracepoint.h>"
- ust_clean_namespace
-}
-
-linetoh_ust()
-{
- local name args argnames
- name=$(get_name "$1")
- args=$(get_args "$1")
- argnames=$(get_argnames "$1", ",")
-
- cat <<EOF
-DECLARE_TRACE(ust_$name, TP_PROTO($args), TP_ARGS($argnames));
-#define trace_$name trace_ust_$name
-EOF
-}
-
-linetoh_end_ust()
-{
- return
-}
-
-linetoc_begin_ust()
-{
- cat <<EOF
-#include <ust/marker.h>
-$(ust_clean_namespace)
-#include "trace.h"
-EOF
-}
-
-linetoc_ust()
-{
- local name args argnames fmt
- name=$(get_name "$1")
- args=$(get_args "$1")
- argnames=$(get_argnames "$1", ",")
- [ -z "$argnames" ] || argnames=", $argnames"
- fmt=$(get_fmt "$1")
-
- cat <<EOF
-DEFINE_TRACE(ust_$name);
-
-static void ust_${name}_probe($args)
-{
- trace_mark(ust, $name, $fmt$argnames);
-}
-EOF
-
- # Collect names for later
- names="$names $name"
-}
-
-linetoc_end_ust()
-{
- cat <<EOF
-static void __attribute__((constructor)) trace_init(void)
-{
-EOF
-
- for name in $names; do
- cat <<EOF
- register_trace_ust_$name(ust_${name}_probe);
-EOF
- done
-
- echo "}"
-}
-
-linetoh_begin_dtrace()
-{
- cat <<EOF
-#include "trace-dtrace.h"
-EOF
-}
-
-linetoh_dtrace()
-{
- local name args argnames nameupper
- name=$(get_name "$1")
- args=$(get_args "$1")
- argnames=$(get_argnames "$1", ",")
-
- nameupper=`echo $name | tr '[:lower:]' '[:upper:]'`
-
- # Define an empty function for the trace event
- cat <<EOF
-static inline void trace_$name($args) {
- if (QEMU_${nameupper}_ENABLED()) {
- QEMU_${nameupper}($argnames);
- }
-}
-EOF
-}
-
-linetoh_end_dtrace()
-{
- return
-}
-
-linetoc_begin_dtrace()
-{
- return
-}
-
-linetoc_dtrace()
-{
- # No need for function definitions in dtrace backend
- return
-}
-
-linetoc_end_dtrace()
-{
- return
-}
-
-linetod_begin_dtrace()
-{
- cat <<EOF
-provider qemu {
-EOF
-}
-
-linetod_dtrace()
-{
- local name args
- name=$(get_name "$1")
- args=$(get_args "$1")
-
- # DTrace provider syntax expects foo() for empty
- # params, not foo(void)
- if [ "$args" = "void" ]; then
- args=""
- fi
-
- # Define prototype for probe arguments
- cat <<EOF
- probe $name($args);
-EOF
-}
-
-linetod_end_dtrace()
-{
- cat <<EOF
-};
-EOF
-}
-
-linetostap_begin_dtrace()
-{
- return
-}
-
-linetostap_dtrace()
-{
- local i arg name args arglist
- name=$(get_name "$1")
- args=$(get_args "$1")
- arglist=$(get_argnames "$1", "")
-
- # Define prototype for probe arguments
- cat <<EOF
-probe $probeprefix.$name = process("$binary").mark("$name")
-{
-EOF
-
- i=1
- for arg in $arglist
- do
- # 'limit' is a reserved keyword
- if [ "$arg" = "limit" ]; then
- arg="_limit"
- fi
- cat <<EOF
- $arg = \$arg$i;
-EOF
- i="$((i+1))"
- done
-
- cat <<EOF
-}
-EOF
-}
-
-linetostap_end_dtrace()
-{
- return
-}
-
-# Process stdin by calling begin, line, and end functions for the backend
-convert()
-{
- local begin process_line end str disable
- begin="lineto$1_begin_$backend"
- process_line="lineto$1_$backend"
- end="lineto$1_end_$backend"
-
- "$begin"
-
- while read -r str; do
- # Skip comments and empty lines
- test -z "${str%%#*}" && continue
-
- echo
- # Process the line. The nop backend handles disabled lines.
- if has_property "$str" "disable"; then
- "lineto$1_nop" "$str"
- else
- "$process_line" "$str"
- fi
- done
-
- echo
- "$end"
-}
-
-tracetoh()
-{
- cat <<EOF
-#ifndef TRACE_H
-#define TRACE_H
-
-/* This file is autogenerated by tracetool, do not edit. */
-
-#include "qemu-common.h"
-EOF
- convert h
- echo "#endif /* TRACE_H */"
-}
-
-tracetoc()
-{
- echo "/* This file is autogenerated by tracetool, do not edit. */"
- convert c
-}
-
-tracetod()
-{
- if [ $backend != "dtrace" ]; then
- echo "DTrace probe generator not applicable to $backend backend"
- exit 1
- fi
- echo "/* This file is autogenerated by tracetool, do not edit. */"
- convert d
-}
-
-tracetostap()
-{
- if [ $backend != "dtrace" ]; then
- echo "SystemTAP tapset generator not applicable to $backend backend"
- exit 1
- fi
- if [ -z "$binary" ]; then
- echo "--binary is required for SystemTAP tapset generator"
- exit 1
- fi
- if [ -z "$probeprefix" -a -z "$targettype" ]; then
- echo "--target-type is required for SystemTAP tapset generator"
- exit 1
- fi
- if [ -z "$probeprefix" -a -z "$targetarch" ]; then
- echo "--target-arch is required for SystemTAP tapset generator"
- exit 1
- fi
- if [ -z "$probeprefix" ]; then
- probeprefix="qemu.$targettype.$targetarch";
- fi
- echo "/* This file is autogenerated by tracetool, do not edit. */"
- convert stap
-}
-
-
-backend=
-output=
-binary=
-targettype=
-targetarch=
-probeprefix=
-
-
-until [ -z "$1" ]
-do
- case "$1" in
- "--nop" | "--simple" | "--stderr" | "--ust" | "--dtrace") backend="${1#--}" ;;
-
- "--binary") shift ; binary="$1" ;;
- "--target-arch") shift ; targetarch="$1" ;;
- "--target-type") shift ; targettype="$1" ;;
- "--probe-prefix") shift ; probeprefix="$1" ;;
-
- "-h" | "-c" | "-d") output="${1#-}" ;;
- "--stap") output="${1#--}" ;;
-
- "--check-backend") exit 0 ;; # used by ./configure to test for backend
-
- "--list-backends") # used by ./configure to list available backends
- echo "nop simple stderr ust dtrace"
- exit 0
- ;;
-
- *)
- usage;;
- esac
- shift
-done
-
-if [ "$backend" = "" -o "$output" = "" ]; then
- usage
-fi
-
-gen="traceto$output"
-"$gen"
-
-exit 0
diff --git a/scripts/tracetool.py b/scripts/tracetool.py
new file mode 100755
index 0000000..408d6ed
--- /dev/null
+++ b/scripts/tracetool.py
@@ -0,0 +1,505 @@
+#!/usr/bin/env python
+# Python based tracetool script (Code generator for trace events)
+#
+# Copyright IBM, Corp. 2011
+#
+# This work is licensed under the terms of the GNU GPL, version 2. See
+# the COPYING file in the top-level directory.
+#
+#
+
+import sys
+import getopt
+
+def usage():
+ print "Tracetool: Generate tracing code for trace events file on stdin"
+ print "Usage:"
+ print sys.argv[0], " --backend=[nop|simple|stderr|dtrace] [-h|-c|-d|--stap]"
+ print '''
+Backends:
+ --nop Tracing disabled
+ --simple Simple built-in backend
+ --stderr Stderr built-in backend
+ --dtrace DTrace/SystemTAP backend
+
+Output formats:
+ -h Generate .h file
+ -c Generate .c file
+ -d Generate .d file (DTrace only)
+ --stap Generate .stp file (DTrace with SystemTAP only)
+
+Options:
+ --binary [path] Full path to QEMU binary
+ --target-arch [arch] QEMU emulator target arch
+ --target-type [type] QEMU emulator target type ('system' or 'user')
+ --probe-prefix [prefix] Prefix for dtrace probe names
+ (default: qemu-targettype-targetarch)
+'''
+ sys.exit(1)
+
+def get_name(line, sep='('):
+ head, sep, tail = line.partition(sep)
+ return head
+
+def get_args(line, sep1='(', sep2=')'):
+ head, sep1, tail = line.partition(sep1)
+ args, sep2, fmt_str = tail.partition(sep2)
+ return args
+
+def get_argnames(line, sep=','):
+ nfields = 0
+ str = []
+ args = get_args(line)
+ for field in args.split():
+ nfields = nfields + 1
+ # Drop pointer star
+ type, ptr, tail = field.partition('*')
+ if type != field:
+ field = tail
+
+ name, sep, tail = field.partition(',')
+
+ if name == field:
+ continue
+ str.append(name)
+ str.append(", ")
+
+ if nfields > 1:
+ str.append(name)
+ return ''.join(str)
+ else:
+ return ''
+
+def get_argc(line):
+ argc = 0
+ argnames = get_argnames(line)
+ if argnames:
+ for name in argnames.split(','):
+ argc = argc + 1
+ return argc
+
+def get_fmt(line, sep=')'):
+ event, sep, fmt = line.partition(sep)
+ return fmt
+
+def calc_sizeofargs(line):
+ args = get_args(line)
+ argc = get_argc(line)
+ strtype = ('const char*', 'char*', 'const char *', 'char *')
+ str = []
+ newstr = ""
+ if argc > 0:
+ str = args.split(',')
+ for elem in str:
+ if elem.lstrip().startswith(strtype): #strings
+ type, sep, var = elem.rpartition('*')
+ newstr = newstr+"4 + strlen("+var.lstrip()+") + "
+ #elif '*' in elem:
+ # newstr = newstr + "4 + " # pointer vars
+ else:
+ #type, sep, var = elem.rpartition(' ')
+ #newstr = newstr+"sizeof("+type.lstrip()+") + "
+ newstr = newstr + '8 + '
+ newstr = newstr + '0' # for last +
+ return newstr
+
+
+def trace_h_begin():
+ print '''#ifndef TRACE_H
+#define TRACE_H
+
+/* This file is autogenerated by tracetool, do not edit. */
+
+#include "qemu-common.h"'''
+ return
+
+def trace_h_end():
+ print '#endif /* TRACE_H */'
+ return
+
+def trace_c_begin():
+ print '/* This file is autogenerated by tracetool, do not edit. */'
+ return
+
+def trace_c_end():
+ # nop, required for trace_gen
+ return
+
+def nop_h(events):
+ print
+ for event in events:
+ print '''static inline void trace_%(name)s(%(args)s)
+{
+}
+''' % {
+ 'name': event.name,
+ 'args': event.args
+}
+ return
+
+def nop_c(events):
+ # nop, reqd for converters
+ return
+
+
+def simple_h(events):
+ print '#include "trace/simple.h"'
+ print
+ for event in events:
+ print 'void trace_%(name)s(%(args)s);' % {
+ 'name': event.name,
+ 'args': event.args
+}
+ print
+ print '#define NR_TRACE_EVENTS %d' % (event.num + 1)
+ print 'extern TraceEvent trace_list[NR_TRACE_EVENTS];'
+
+ return
+
+def is_string(arg):
+ strtype = ('const char*', 'char*', 'const char *', 'char *')
+ if arg.lstrip().startswith(strtype):
+ return True
+ else:
+ return False
+
+def simple_c(events):
+ rec_off = 0
+ print '#include "trace.h"'
+ print '#include "trace/simple.h"'
+ print
+ print 'TraceEvent trace_list[] = {'
+ print
+ eventlist = list(events)
+ for event in eventlist:
+ print '{.tp_name = "%(name)s", .state=0},' % {
+ 'name': event.name
+}
+ print
+ print '};'
+ print
+ for event in eventlist:
+ argc = event.argc
+ print '''void trace_%(name)s(%(args)s)
+{
+ unsigned int tbuf_idx, rec_off;
+ uint64_t var64 __attribute__ ((unused));
+ uint64_t pvar64 __attribute__ ((unused));
+ uint32_t slen __attribute__ ((unused));
+
+ if (!trace_list[%(event_id)s].state) {
+ return;
+ }
+''' % {
+ 'name': event.name,
+ 'args': event.args,
+ 'event_id': event.num,
+}
+ print '''
+ tbuf_idx = trace_alloc_record(%(event_id)s, %(sizestr)s);
+ rec_off = (tbuf_idx + ST_V2_REC_HDR_LEN) %% TRACE_BUF_LEN; /* seek record header */
+''' % {'event_id': event.num, 'sizestr': event.sizestr,}
+
+ if argc > 0:
+ str = event.arglist
+ for elem in str:
+ if is_string(elem): # if string
+ type, sep, var = elem.rpartition('*')
+ print '''
+ slen = strlen(%(var)s);
+ write_to_buffer(rec_off, (uint8_t*)&slen, sizeof(slen));
+ rec_off += sizeof(slen);''' % {
+ 'var': var.lstrip()
+}
+ print '''
+ write_to_buffer(rec_off, (uint8_t*)%(var)s, slen);
+ rec_off += slen;''' % {
+ 'var': var.lstrip()
+}
+ elif '*' in elem: # pointer var (not string)
+ type, sep, var = elem.rpartition('*')
+ print '''
+ pvar64 = (uint64_t)(uint64_t*)%(var)s;
+ write_to_buffer(rec_off, (uint8_t*)&pvar64, sizeof(uint64_t));
+ rec_off += sizeof(uint64_t);''' % {
+ 'var': var.lstrip()
+}
+ else: # primitive data type
+ type, sep, var = elem.rpartition(' ')
+ print '''
+ var64 = (uint64_t)%(var)s;
+ write_to_buffer(rec_off, (uint8_t*)&var64, sizeof(uint64_t));
+ rec_off += sizeof(uint64_t);''' % {
+ 'var': var.lstrip()
+}
+ print '''
+ trace_mark_record_complete(tbuf_idx);'''
+ print '}'
+ print
+
+ return
+
+def stderr_h(events):
+ print '''#include <stdio.h>
+#include "trace/stderr.h"
+
+extern TraceEvent trace_list[];'''
+ for event in events:
+ argnames = event.argnames
+ if event.argc > 0:
+ argnames = ', ' + event.argnames
+ else:
+ argnames = ''
+ print '''
+static inline void trace_%(name)s(%(args)s)
+{
+ if (trace_list[%(event_num)s].state != 0) {
+ fprintf(stderr, "%(name)s " %(fmt)s "\\n" %(argnames)s);
+ }
+}''' % {
+ 'name': event.name,
+ 'args': event.args,
+ 'event_num': event.num,
+ 'fmt': event.fmt.rstrip('\n'),
+ 'argnames': argnames
+}
+ print
+ print '#define NR_TRACE_EVENTS %d' % (event.num + 1)
+
+def stderr_c(events):
+ print '''#include "trace.h"
+
+TraceEvent trace_list[] = {
+'''
+ for event in events:
+ print '{.tp_name = "%(name)s", .state=0},' % {
+ 'name': event.name
+}
+ print
+ print '};'
+
+def dtrace_h(events):
+ print '#include "trace-dtrace.h"'
+ print
+ for event in events:
+ print '''static inline void trace_%(name)s(%(args)s) {
+ if (QEMU_%(uppername)s_ENABLED()) {
+ QEMU_%(uppername)s(%(argnames)s);
+ }
+}
+''' % {
+ 'name': event.name,
+ 'args': event.args,
+ 'uppername': event.name.upper(),
+ 'argnames': event.argnames,
+}
+
+def dtrace_c(events):
+ return # No need for function definitions in dtrace backend
+
+def dtrace_d(events):
+ print 'provider qemu {'
+ for event in events:
+ args = event.args
+
+ # DTrace provider syntax expects foo() for empty
+ # params, not foo(void)
+ if args == 'void':
+ args = ''
+
+ # Define prototype for probe arguments
+ print '''
+ probe %(name)s(%(args)s);''' % {
+ 'name': event.name,
+ 'args': args
+}
+ print
+ print '};'
+ return
+
+def dtrace_stp(events):
+ for event in events:
+ # Define prototype for probe arguments
+ print '''
+probe %(probeprefix)s.%(name)s = process("%(binary)s").mark("%(name)s")
+{''' % {
+ 'probeprefix': probeprefix,
+ 'name': event.name,
+ 'binary': binary
+}
+ i = 1
+ if event.argc > 0:
+ for arg in event.argnames.split(','):
+ # 'limit' is a reserved keyword
+ if arg == 'limit':
+ arg = '_limit'
+ print ' %s = $arg%d;' % (arg.lstrip(), i)
+ i += 1
+ print '}'
+ print
+ return
+
+def trace_stap_begin():
+ global probeprefix
+ if backend != "dtrace":
+ print 'SystemTAP tapset generator not applicable to %s backend' % backend
+ sys.exit(1)
+ if binary == "":
+ print '--binary is required for SystemTAP tapset generator'
+ sys.exit(1)
+ if ((probeprefix == "") and (targettype == "")):
+ print '--target-type is required for SystemTAP tapset generator'
+ sys.exit(1)
+ if ((probeprefix == "") and (targetarch == "")):
+ print '--target-arch is required for SystemTAP tapset generator'
+ sys.exit(1)
+ if probeprefix == "":
+ probeprefix = 'qemu.' + targettype + '.' + targetarch
+ print '/* This file is autogenerated by tracetool, do not edit. */'
+ return
+
+def trace_stap_end():
+ return #nop, reqd for trace_gen
+
+def trace_d_begin():
+ if backend != 'dtrace':
+ print 'DTrace probe generator not applicable to %s backend' % backend
+ sys.exit(1)
+ print '/* This file is autogenerated by tracetool, do not edit. */'
+
+def trace_d_end():
+ return #nop, reqd for trace_gen
+
+
+# Registry of backends and their converter functions
+converters = {
+ 'simple': {
+ 'h': simple_h,
+ 'c': simple_c,
+ },
+
+ 'nop': {
+ 'h': nop_h,
+ 'c': nop_c,
+ },
+
+ 'stderr': {
+ 'h': stderr_h,
+ 'c': stderr_c,
+ },
+
+ 'dtrace': {
+ 'h': dtrace_h,
+ 'c': dtrace_c,
+ 'd': dtrace_d,
+ 'stap': dtrace_stp
+ },
+}
+
+# Trace file header and footer code generators
+trace_gen = {
+ 'h': {
+ 'begin': trace_h_begin,
+ 'end': trace_h_end,
+ },
+ 'c': {
+ 'begin': trace_c_begin,
+ 'end': trace_c_end,
+ },
+ 'd': {
+ 'begin': trace_d_begin,
+ 'end': trace_d_end,
+ },
+ 'stap': {
+ 'begin': trace_stap_begin,
+ 'end': trace_stap_end,
+ },
+}
+
+# A trace event
+class Event(object):
+ def __init__(self, num, line):
+ self.num = num
+ self.args = get_args(line)
+ self.arglist = self.args.split(',')
+ self.name = get_name(line)
+ self.argc = get_argc(line)
+ self.argnames = get_argnames(line)
+ self.sizestr = calc_sizeofargs(line)
+ self.fmt = get_fmt(line)
+
+# Generator that yields Event objects given a trace-events file object
+def read_events(fobj):
+ event_num = 0
+ for line in fobj:
+ if not line.strip():
+ continue
+ if line.lstrip().startswith('#'):
+ continue
+ yield Event(event_num, line)
+ event_num += 1
+
+backend = ""
+output = ""
+binary = ""
+targettype = ""
+targetarch = ""
+probeprefix = ""
+
+def main():
+ global backend, output, binary, targettype, targetarch, probeprefix
+ supported_backends = ["simple", "nop", "stderr", "dtrace"]
+ short_options = "hcd"
+ long_options = ["stap", "backend=", "binary=", "target-arch=", "target-type=", "probe-prefix=", "list-backends", "check-backend"]
+ try:
+ opts, args = getopt.getopt(sys.argv[1:], short_options, long_options)
+ except getopt.GetoptError, err:
+ # print help information and exit:
+ print str(err) # will print something like "option -a not recognized"
+ usage()
+ sys.exit(2)
+ for opt, arg in opts:
+ if opt == '-h':
+ output = 'h'
+ elif opt == '-c':
+ output = 'c'
+ elif opt == '-d':
+ output = 'd'
+ elif opt == '--stap':
+ output = 'stap'
+ elif opt == '--backend':
+ backend = arg
+ elif opt == '--binary':
+ binary = arg
+ elif opt == '--target-arch':
+ targetarch = arg
+ elif opt == '--target-type':
+ targettype = arg
+ elif opt == '--probe-prefix':
+ probeprefix = arg
+ elif opt == '--list-backends':
+ print 'simple, nop, stderr, dtrace'
+ sys.exit(0)
+ elif opt == "--check-backend":
+ if any(backend in s for s in supported_backends):
+ sys.exit(0)
+ else:
+ sys.exit(1)
+ else:
+ #assert False, "unhandled option"
+ print "unhandled option: ", opt
+ usage()
+
+ if backend == "" or output == "":
+ usage()
+ sys.exit(0)
+
+ events = read_events(sys.stdin)
+ trace_gen[output]['begin']()
+ converters[backend][output](events)
+ trace_gen[output]['end']()
+ return
+
+if __name__ == "__main__":
+ main()
+
--
1.7.1.1
^ permalink raw reply related [flat|nested] 18+ messages in thread* Re: [Qemu-devel] [RFC PATCH v2 1/4] Converting tracetool.sh to tracetool.py
2012-01-09 15:46 ` [Qemu-devel] [RFC PATCH v2 1/4] Converting tracetool.sh to tracetool.py Harsh Prateek Bora
@ 2012-01-09 21:06 ` Andreas Färber
2012-01-10 6:12 ` Harsh Bora
0 siblings, 1 reply; 18+ messages in thread
From: Andreas Färber @ 2012-01-09 21:06 UTC (permalink / raw)
To: Harsh Prateek Bora
Cc: vilanova, mathieu.desnoyers, qemu-devel, stefanha, aneesh.kumar
Am 09.01.2012 16:46, schrieb Harsh Prateek Bora:
> Note: Upstream Qemu have build issues with LTTng ust backend, as the trace
> event APIs are not yet stable in ust and therefore ust backend is not
> supported in this script as of now. Once the ust API stablises, this script
> can be updated for ust backend also.
>
> Signed-off-by: Harsh Prateek Bora <harsh@linux.vnet.ibm.com>
> ---
> scripts/tracetool | 643 --------------------------------------------------
> scripts/tracetool.py | 505 +++++++++++++++++++++++++++++++++++++++
> 2 files changed, 505 insertions(+), 643 deletions(-)
> delete mode 100755 scripts/tracetool
> create mode 100755 scripts/tracetool.py
This will break the build. You need to squash 2/4 into this one.
What does "ust backend is not supported" mean? Did you leave it out
completely, or did not test it, or tested it and it's not working?
Andreas
--
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v2 1/4] Converting tracetool.sh to tracetool.py
2012-01-09 21:06 ` Andreas Färber
@ 2012-01-10 6:12 ` Harsh Bora
0 siblings, 0 replies; 18+ messages in thread
From: Harsh Bora @ 2012-01-10 6:12 UTC (permalink / raw)
To: Andreas Färber
Cc: vilanova, mathieu.desnoyers, qemu-devel, stefanha, aneesh.kumar
On 01/10/2012 02:36 AM, Andreas Färber wrote:
> Am 09.01.2012 16:46, schrieb Harsh Prateek Bora:
>> Note: Upstream Qemu have build issues with LTTng ust backend, as the trace
>> event APIs are not yet stable in ust and therefore ust backend is not
>> supported in this script as of now. Once the ust API stablises, this script
>> can be updated for ust backend also.
>>
>> Signed-off-by: Harsh Prateek Bora<harsh@linux.vnet.ibm.com>
>> ---
>> scripts/tracetool | 643 --------------------------------------------------
>> scripts/tracetool.py | 505 +++++++++++++++++++++++++++++++++++++++
>> 2 files changed, 505 insertions(+), 643 deletions(-)
>> delete mode 100755 scripts/tracetool
>> create mode 100755 scripts/tracetool.py
>
> This will break the build. You need to squash 2/4 into this one.
Yes, 2/4 will be squashed with it.
>
> What does "ust backend is not supported" mean? Did you leave it out
> completely, or did not test it, or tested it and it's not working?
Please see discussion on 0/4 patch thread. Existing tracetool needs a
fix for ust, and I was not sure about the fix, therefore not converted yet.
- Harsh
>
> Andreas
>
^ permalink raw reply [flat|nested] 18+ messages in thread
* [Qemu-devel] [RFC PATCH v2 2/4] Makefile and configure changes for tracetool.py
2012-01-09 15:46 [Qemu-devel] [RFC PATCH v2 0/4] simpletrace : support var num of args and strings Harsh Prateek Bora
2012-01-09 15:46 ` [Qemu-devel] [RFC PATCH v2 1/4] Converting tracetool.sh to tracetool.py Harsh Prateek Bora
@ 2012-01-09 15:46 ` Harsh Prateek Bora
2012-01-09 15:46 ` [Qemu-devel] [RFC PATCH v2 3/4] simpletrace-v2: Handle variable number/size of elements per trace record Harsh Prateek Bora
` (2 subsequent siblings)
4 siblings, 0 replies; 18+ messages in thread
From: Harsh Prateek Bora @ 2012-01-09 15:46 UTC (permalink / raw)
To: qemu-devel; +Cc: aneesh.kumar, mathieu.desnoyers, vilanova, stefanha
Use tracetool.py to generate tracing code for requested backend.
Signed-off-by: Harsh Prateek Bora <harsh@linux.vnet.ibm.com>
---
Makefile.objs | 6 +++---
Makefile.target | 10 +++++-----
configure | 4 ++--
3 files changed, 10 insertions(+), 10 deletions(-)
diff --git a/Makefile.objs b/Makefile.objs
index 804bc3c..984034a 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -344,12 +344,12 @@ else
trace.h: trace.h-timestamp
endif
trace.h-timestamp: $(SRC_PATH)/trace-events $(BUILD_DIR)/config-host.mak
- $(call quiet-command,sh $(SRC_PATH)/scripts/tracetool --$(TRACE_BACKEND) -h < $< > $@," GEN trace.h")
+ $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/tracetool.py --backend=$(TRACE_BACKEND) -h < $< > $@," GEN trace.h")
@cmp -s $@ trace.h || cp $@ trace.h
trace.c: trace.c-timestamp
trace.c-timestamp: $(SRC_PATH)/trace-events $(BUILD_DIR)/config-host.mak
- $(call quiet-command,sh $(SRC_PATH)/scripts/tracetool --$(TRACE_BACKEND) -c < $< > $@," GEN trace.c")
+ $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/tracetool.py --backend=$(TRACE_BACKEND) -c < $< > $@," GEN trace.c")
@cmp -s $@ trace.c || cp $@ trace.c
trace.o: trace.c $(GENERATED_HEADERS)
@@ -362,7 +362,7 @@ trace-dtrace.h: trace-dtrace.dtrace
# rule file. So we use '.dtrace' instead
trace-dtrace.dtrace: trace-dtrace.dtrace-timestamp
trace-dtrace.dtrace-timestamp: $(SRC_PATH)/trace-events $(BUILD_DIR)/config-host.mak
- $(call quiet-command,sh $(SRC_PATH)/scripts/tracetool --$(TRACE_BACKEND) -d < $< > $@," GEN trace-dtrace.dtrace")
+ $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/tracetool.py --backend=$(TRACE_BACKEND) -d < $< > $@," GEN trace-dtrace.dtrace")
@cmp -s $@ trace-dtrace.dtrace || cp $@ trace-dtrace.dtrace
trace-dtrace.o: trace-dtrace.dtrace $(GENERATED_HEADERS)
diff --git a/Makefile.target b/Makefile.target
index 1e90df7..07b21d1 100644
--- a/Makefile.target
+++ b/Makefile.target
@@ -50,11 +50,11 @@ TARGET_TYPE=system
endif
$(QEMU_PROG).stp:
- $(call quiet-command,sh $(SRC_PATH)/scripts/tracetool \
- --$(TRACE_BACKEND) \
- --binary $(bindir)/$(QEMU_PROG) \
- --target-arch $(TARGET_ARCH) \
- --target-type $(TARGET_TYPE) \
+ $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/tracetool.py \
+ --backend=$(TRACE_BACKEND) \
+ --binary=$(bindir)/$(QEMU_PROG) \
+ --target-arch=$(TARGET_ARCH) \
+ --target-type=$(TARGET_TYPE) \
--stap < $(SRC_PATH)/trace-events > $(QEMU_PROG).stp," GEN $(QEMU_PROG).stp")
else
stap:
diff --git a/configure b/configure
index 19e8394..0ea8634 100755
--- a/configure
+++ b/configure
@@ -1040,7 +1040,7 @@ echo " --disable-docs disable documentation build"
echo " --disable-vhost-net disable vhost-net acceleration support"
echo " --enable-vhost-net enable vhost-net acceleration support"
echo " --enable-trace-backend=B Set trace backend"
-echo " Available backends:" $("$source_path"/scripts/tracetool --list-backends)
+echo " Available backends:" $("$source_path"/scripts/tracetool.py --list-backends)
echo " --with-trace-file=NAME Full PATH,NAME of file to store traces"
echo " Default:trace-<pid>"
echo " --disable-spice disable spice"
@@ -2477,7 +2477,7 @@ fi
##########################################
# check if trace backend exists
-sh "$source_path/scripts/tracetool" "--$trace_backend" --check-backend > /dev/null 2> /dev/null
+$python "$source_path/scripts/tracetool.py" "--backend=$trace_backend" --check-backend > /dev/null 2> /dev/null
if test "$?" -ne 0 ; then
echo
echo "Error: invalid trace backend"
--
1.7.1.1
^ permalink raw reply related [flat|nested] 18+ messages in thread* [Qemu-devel] [RFC PATCH v2 3/4] simpletrace-v2: Handle variable number/size of elements per trace record.
2012-01-09 15:46 [Qemu-devel] [RFC PATCH v2 0/4] simpletrace : support var num of args and strings Harsh Prateek Bora
2012-01-09 15:46 ` [Qemu-devel] [RFC PATCH v2 1/4] Converting tracetool.sh to tracetool.py Harsh Prateek Bora
2012-01-09 15:46 ` [Qemu-devel] [RFC PATCH v2 2/4] Makefile and configure changes for tracetool.py Harsh Prateek Bora
@ 2012-01-09 15:46 ` Harsh Prateek Bora
2012-01-09 15:46 ` [Qemu-devel] [RFC PATCH v2 4/4] simpletrace.py: updated log reader script to handle new log format Harsh Prateek Bora
2012-01-09 16:01 ` [Qemu-devel] [RFC PATCH v2 0/4] simpletrace : support var num of args and strings Mathieu Desnoyers
4 siblings, 0 replies; 18+ messages in thread
From: Harsh Prateek Bora @ 2012-01-09 15:46 UTC (permalink / raw)
To: qemu-devel; +Cc: aneesh.kumar, mathieu.desnoyers, vilanova, stefanha
Advantages over existing simpletrace backend:
- More than 6 elements (vitually unlimited) arguments can be traced.
- This allows to trace strings (variable size element) as well.
Signed-off-by: Harsh Prateek Bora <harsh@linux.vnet.ibm.com>
---
monitor.c | 2 +-
trace/simple.c | 178 ++++++++++++++++++++------------------------------------
trace/simple.h | 31 ++++++++--
3 files changed, 88 insertions(+), 123 deletions(-)
diff --git a/monitor.c b/monitor.c
index ffda0fe..b6f85d1 100644
--- a/monitor.c
+++ b/monitor.c
@@ -945,7 +945,7 @@ static void do_info_cpu_stats(Monitor *mon)
#if defined(CONFIG_TRACE_SIMPLE)
static void do_info_trace(Monitor *mon)
{
- st_print_trace((FILE *)mon, &monitor_fprintf);
+ /* FIXME: st_print_trace((FILE *)mon, &monitor_fprintf); */
}
#endif
diff --git a/trace/simple.c b/trace/simple.c
index b639dda..65cafe2 100644
--- a/trace/simple.c
+++ b/trace/simple.c
@@ -27,7 +27,7 @@
#define HEADER_MAGIC 0xf2b177cb0aa429b4ULL
/** Trace file version number, bump if format changes */
-#define HEADER_VERSION 0
+#define HEADER_VERSION 2
/** Records were dropped event ID */
#define DROPPED_EVENT_ID (~(uint64_t)0 - 1)
@@ -35,23 +35,6 @@
/** Trace record is valid */
#define TRACE_RECORD_VALID ((uint64_t)1 << 63)
-/** Trace buffer entry */
-typedef struct {
- uint64_t event;
- uint64_t timestamp_ns;
- uint64_t x1;
- uint64_t x2;
- uint64_t x3;
- uint64_t x4;
- uint64_t x5;
- uint64_t x6;
-} TraceRecord;
-
-enum {
- TRACE_BUF_LEN = 4096,
- TRACE_BUF_FLUSH_THRESHOLD = TRACE_BUF_LEN / 4,
-};
-
/*
* Trace records are written out by a dedicated thread. The thread waits for
* records to become available, writes them out, and then waits again.
@@ -62,11 +45,12 @@ static GCond *trace_empty_cond;
static bool trace_available;
static bool trace_writeout_enabled;
-static TraceRecord trace_buf[TRACE_BUF_LEN];
+uint8_t trace_buf[TRACE_BUF_LEN];
static unsigned int trace_idx;
static FILE *trace_fp;
static char *trace_file_name = NULL;
+
/**
* Read a trace record from the trace buffer
*
@@ -75,16 +59,19 @@ static char *trace_file_name = NULL;
*
* Returns false if the record is not valid.
*/
-static bool get_trace_record(unsigned int idx, TraceRecord *record)
+static bool get_trace_record(unsigned int idx, TraceRecord **recordptr)
{
- if (!(trace_buf[idx].event & TRACE_RECORD_VALID)) {
+ TraceRecord *record = (TraceRecord *) &trace_buf[idx];
+ if (!(record->event & TRACE_RECORD_VALID)) {
return false;
}
__sync_synchronize(); /* read memory barrier before accessing record */
- *record = trace_buf[idx];
- record->event &= ~TRACE_RECORD_VALID;
+ *recordptr = g_malloc(record->length);
+ /* make a copy of record to avoid being overwritten */
+ memcpy(*recordptr, record, record->length);
+ (*recordptr)->event &= ~TRACE_RECORD_VALID;
return true;
}
@@ -106,6 +93,47 @@ static void flush_trace_file(bool wait)
g_static_mutex_unlock(&trace_lock);
}
+unsigned int trace_alloc_record(TraceEventID event, uint32_t datasize)
+{
+ unsigned int idx, rec_off;
+ uint32_t rec_len = ST_V2_REC_HDR_LEN + datasize;
+ uint64_t timestamp_ns = get_clock();
+
+ idx = g_atomic_int_exchange_and_add((gint *)&trace_idx, rec_len) % TRACE_BUF_LEN;
+ rec_off = idx;
+ write_to_buffer(rec_off, (uint8_t*)&event, sizeof(event));
+ rec_off += sizeof(event);
+ write_to_buffer(rec_off, (uint8_t*)×tamp_ns, sizeof(timestamp_ns));
+ rec_off += sizeof(timestamp_ns);
+ write_to_buffer(rec_off, (uint8_t*)&rec_len, sizeof(rec_len));
+ rec_off += sizeof(rec_len);
+ return idx;
+}
+
+void write_to_buffer(unsigned int idx, uint8_t *dataptr, uint32_t size)
+{
+ uint32_t x = 0;
+ while (x < size)
+ {
+ if (idx >= TRACE_BUF_LEN) {
+ idx = idx % TRACE_BUF_LEN;
+ }
+ trace_buf[idx++] = dataptr[x++];
+ }
+}
+
+void trace_mark_record_complete(unsigned int idx)
+{
+ TraceRecord *record = (TraceRecord*) &trace_buf[idx];
+
+ __sync_synchronize(); /* write barrier before marking as valid */
+ record->event |= TRACE_RECORD_VALID;
+
+ if (idx > TRACE_BUF_FLUSH_THRESHOLD) {
+ flush_trace_file(false);
+ }
+}
+
static void wait_for_trace_records_available(void)
{
g_static_mutex_lock(&trace_lock);
@@ -120,7 +148,7 @@ static void wait_for_trace_records_available(void)
static gpointer writeout_thread(gpointer opaque)
{
- TraceRecord record;
+ TraceRecord record, *recordptr;
unsigned int writeout_idx = 0;
unsigned int num_available, idx;
size_t unused __attribute__ ((unused));
@@ -130,19 +158,20 @@ static gpointer writeout_thread(gpointer opaque)
num_available = trace_idx - writeout_idx;
if (num_available > TRACE_BUF_LEN) {
- record = (TraceRecord){
- .event = DROPPED_EVENT_ID,
- .x1 = num_available,
- };
- unused = fwrite(&record, sizeof(record), 1, trace_fp);
+ record.event = DROPPED_EVENT_ID,
+ record.length = num_available,
+ unused = fwrite(&record, ST_V2_REC_HDR_LEN, 1, trace_fp);
writeout_idx += num_available;
}
idx = writeout_idx % TRACE_BUF_LEN;
- while (get_trace_record(idx, &record)) {
- trace_buf[idx].event = 0; /* clear valid bit */
- unused = fwrite(&record, sizeof(record), 1, trace_fp);
- idx = ++writeout_idx % TRACE_BUF_LEN;
+ while (get_trace_record(idx, &recordptr)) {
+ unused = fwrite(recordptr, recordptr->length, 1, trace_fp);
+ writeout_idx += recordptr->length;
+ g_free(recordptr);
+ recordptr = (TraceRecord *) &trace_buf[idx];
+ recordptr->event = 0;
+ idx = writeout_idx % TRACE_BUF_LEN;
}
fflush(trace_fp);
@@ -150,71 +179,7 @@ static gpointer writeout_thread(gpointer opaque)
return NULL;
}
-static void trace(TraceEventID event, uint64_t x1, uint64_t x2, uint64_t x3,
- uint64_t x4, uint64_t x5, uint64_t x6)
-{
- unsigned int idx;
- uint64_t timestamp;
-
- if (!trace_list[event].state) {
- return;
- }
-
- timestamp = get_clock();
-
- idx = g_atomic_int_exchange_and_add((gint *)&trace_idx, 1) % TRACE_BUF_LEN;
- trace_buf[idx] = (TraceRecord){
- .event = event,
- .timestamp_ns = timestamp,
- .x1 = x1,
- .x2 = x2,
- .x3 = x3,
- .x4 = x4,
- .x5 = x5,
- .x6 = x6,
- };
- __sync_synchronize(); /* write barrier before marking as valid */
- trace_buf[idx].event |= TRACE_RECORD_VALID;
-
- if ((idx + 1) % TRACE_BUF_FLUSH_THRESHOLD == 0) {
- flush_trace_file(false);
- }
-}
-
-void trace0(TraceEventID event)
-{
- trace(event, 0, 0, 0, 0, 0, 0);
-}
-
-void trace1(TraceEventID event, uint64_t x1)
-{
- trace(event, x1, 0, 0, 0, 0, 0);
-}
-
-void trace2(TraceEventID event, uint64_t x1, uint64_t x2)
-{
- trace(event, x1, x2, 0, 0, 0, 0);
-}
-void trace3(TraceEventID event, uint64_t x1, uint64_t x2, uint64_t x3)
-{
- trace(event, x1, x2, x3, 0, 0, 0);
-}
-
-void trace4(TraceEventID event, uint64_t x1, uint64_t x2, uint64_t x3, uint64_t x4)
-{
- trace(event, x1, x2, x3, x4, 0, 0);
-}
-
-void trace5(TraceEventID event, uint64_t x1, uint64_t x2, uint64_t x3, uint64_t x4, uint64_t x5)
-{
- trace(event, x1, x2, x3, x4, x5, 0);
-}
-
-void trace6(TraceEventID event, uint64_t x1, uint64_t x2, uint64_t x3, uint64_t x4, uint64_t x5, uint64_t x6)
-{
- trace(event, x1, x2, x3, x4, x5, x6);
-}
void st_set_trace_file_enabled(bool enable)
{
@@ -231,7 +196,7 @@ void st_set_trace_file_enabled(bool enable)
static const TraceRecord header = {
.event = HEADER_EVENT_ID,
.timestamp_ns = HEADER_MAGIC,
- .x1 = HEADER_VERSION,
+ .length = HEADER_VERSION,
};
trace_fp = fopen(trace_file_name, "wb");
@@ -288,23 +253,6 @@ void st_print_trace_file_status(FILE *stream, int (*stream_printf)(FILE *stream,
trace_file_name, trace_fp ? "on" : "off");
}
-void st_print_trace(FILE *stream, int (*stream_printf)(FILE *stream, const char *fmt, ...))
-{
- unsigned int i;
-
- for (i = 0; i < TRACE_BUF_LEN; i++) {
- TraceRecord record;
-
- if (!get_trace_record(i, &record)) {
- continue;
- }
- stream_printf(stream, "Event %" PRIu64 " : %" PRIx64 " %" PRIx64
- " %" PRIx64 " %" PRIx64 " %" PRIx64 " %" PRIx64 "\n",
- record.event, record.x1, record.x2,
- record.x3, record.x4, record.x5,
- record.x6);
- }
-}
void st_flush_trace_buffer(void)
{
diff --git a/trace/simple.h b/trace/simple.h
index 466e75b..671cbeb 100644
--- a/trace/simple.h
+++ b/trace/simple.h
@@ -22,17 +22,34 @@ typedef struct {
bool state;
} TraceEvent;
-void trace0(TraceEventID event);
-void trace1(TraceEventID event, uint64_t x1);
-void trace2(TraceEventID event, uint64_t x1, uint64_t x2);
-void trace3(TraceEventID event, uint64_t x1, uint64_t x2, uint64_t x3);
-void trace4(TraceEventID event, uint64_t x1, uint64_t x2, uint64_t x3, uint64_t x4);
-void trace5(TraceEventID event, uint64_t x1, uint64_t x2, uint64_t x3, uint64_t x4, uint64_t x5);
-void trace6(TraceEventID event, uint64_t x1, uint64_t x2, uint64_t x3, uint64_t x4, uint64_t x5, uint64_t x6);
void st_print_trace(FILE *stream, fprintf_function stream_printf);
void st_print_trace_file_status(FILE *stream, fprintf_function stream_printf);
void st_set_trace_file_enabled(bool enable);
bool st_set_trace_file(const char *file);
void st_flush_trace_buffer(void);
+/* Interfaces for simpletrace v2 */
+
+/** Trace buffer entry */
+typedef struct {
+ uint64_t event; /* TraceEventID */
+ uint64_t timestamp_ns;
+ uint32_t length; /* in bytes */
+ uint32_t reserved; /* unused */
+ uint8_t arguments[]; /* arguments position affects ST_V2_REC_HDR_LEN */
+} TraceRecord;
+
+#define ST_V2_REC_HDR_LEN 24
+
+enum {
+ TRACE_BUF_LEN = 4096 * 64,
+ TRACE_BUF_FLUSH_THRESHOLD = TRACE_BUF_LEN / 4,
+};
+
+extern uint8_t trace_buf[TRACE_BUF_LEN];
+
+unsigned int trace_alloc_record(TraceEventID event, uint32_t datasize);
+void write_to_buffer(unsigned int idx, uint8_t *dataptr, uint32_t size);
+void trace_mark_record_complete(unsigned int idx);
+
#endif /* TRACE_SIMPLE_H */
--
1.7.1.1
^ permalink raw reply related [flat|nested] 18+ messages in thread* [Qemu-devel] [RFC PATCH v2 4/4] simpletrace.py: updated log reader script to handle new log format
2012-01-09 15:46 [Qemu-devel] [RFC PATCH v2 0/4] simpletrace : support var num of args and strings Harsh Prateek Bora
` (2 preceding siblings ...)
2012-01-09 15:46 ` [Qemu-devel] [RFC PATCH v2 3/4] simpletrace-v2: Handle variable number/size of elements per trace record Harsh Prateek Bora
@ 2012-01-09 15:46 ` Harsh Prateek Bora
2012-01-09 16:01 ` [Qemu-devel] [RFC PATCH v2 0/4] simpletrace : support var num of args and strings Mathieu Desnoyers
4 siblings, 0 replies; 18+ messages in thread
From: Harsh Prateek Bora @ 2012-01-09 15:46 UTC (permalink / raw)
To: qemu-devel; +Cc: aneesh.kumar, mathieu.desnoyers, vilanova, stefanha
Note: This script has been updated with minimal changes required to observe
the new trace log format in action and therefore can be improved for a
better design. It can still read the logs from older log format as well.
Signed-off-by: Harsh Prateek Bora <harsh@linux.vnet.ibm.com>
---
scripts/simpletrace.py | 110 +++++++++++++++++++++++++++++++++++++++++++-----
1 files changed, 99 insertions(+), 11 deletions(-)
diff --git a/scripts/simpletrace.py b/scripts/simpletrace.py
index f55e5e6..69829dd 100755
--- a/scripts/simpletrace.py
+++ b/scripts/simpletrace.py
@@ -12,16 +12,37 @@
import struct
import re
import inspect
+import sys
+from tracetool import *
header_event_id = 0xffffffffffffffff
header_magic = 0xf2b177cb0aa429b4
header_version = 0
+log_header_ver = 2
dropped_event_id = 0xfffffffffffffffe
trace_fmt = '=QQQQQQQQ'
+header_fmt = '=QQII'
+header_len = struct.calcsize(header_fmt)
trace_len = struct.calcsize(trace_fmt)
event_re = re.compile(r'(disable\s+)?([a-zA-Z0-9_]+)\(([^)]*)\).*')
+def get_event_argtypes(fobj):
+ """Parse a trace-events file into {event_num: (name, type arg, ...)}."""
+
+ events = {dropped_event_id: ('name', 'args')}
+ event_num = 0
+ for line in fobj:
+ m = event_re.match(line.strip())
+ if m is None:
+ continue
+
+ disable, name, args = m.groups()
+ events[event_num] = tuple(args.split(','))
+ event_num += 1
+ return events
+
+
def parse_events(fobj):
"""Parse a trace-events file into {event_num: (name, arg1, ...)}."""
@@ -48,15 +69,16 @@ def read_record(fobj):
return None
return struct.unpack(trace_fmt, s)
+def read_header(fobj):
+ '''Read a trace record header'''
+ s = fobj.read(header_len)
+ if len(s) != header_len:
+ return None
+ return struct.unpack(header_fmt, s)
+
+
def read_trace_file(fobj):
"""Deserialize trace records from a file, yielding record tuples (event_num, timestamp, arg1, ..., arg6)."""
- header = read_record(fobj)
- if header is None or \
- header[0] != header_event_id or \
- header[1] != header_magic or \
- header[2] != header_version:
- raise ValueError('not a trace file or incompatible version')
-
while True:
rec = read_record(fobj)
if rec is None:
@@ -64,6 +86,59 @@ def read_trace_file(fobj):
yield rec
+def process_event(event_id, fobj):
+ params = etypes[event_id]
+ for elem in params:
+ if is_string(elem):
+ l = fobj.read(4)
+ (len,) = struct.unpack('=L', l)
+ s = fobj.read(len)
+ sfmt = `len`+'s'
+ (str,) = struct.unpack_from(sfmt, s)
+ type, sep, var = elem.rpartition('*')
+ print '%(arg)s=%(val)s' % { 'arg': var.lstrip(), 'val': str },
+ elif '*' in elem:
+ (value,) = struct.unpack('=Q', fobj.read(8))
+ type, sep, var = elem.rpartition('*')
+ print '%(arg)s=0x%(val)u' % { 'arg': var.lstrip(), 'val': value },
+ else:
+ (value,) = struct.unpack('=Q', fobj.read(8))
+ type, sep, var = elem.rpartition(' ')
+ print '%(arg)s=%(val)u' % { 'arg': var.lstrip(), 'val': value },
+ print
+ return
+
+etypes = get_event_argtypes(open(sys.argv[1], 'r'))
+def processv2(fobj):
+ '''Process simpletrace v2 log trace records'''
+ events = parse_events(open(sys.argv[1], 'r'))
+ #print events
+ max_events = len(events) - 1
+ last_timestamp = None
+ while True:
+ # read record header
+ rechdr = read_header(fobj)
+ if rechdr is None:
+ print "No more records"
+ break
+
+ if rechdr[0] >= max_events:
+ print "Invalid Event ID found, Trace Log may be corrupted !!!"
+ sys.exit(1)
+
+ if last_timestamp is None:
+ last_timestamp = rechdr[1]
+ delta_ns = rechdr[1] - last_timestamp
+ last_timestamp = rechdr[1]
+
+ print events[rechdr[0]][0], '%0.3f' % (delta_ns / 1000.0),
+ # read data
+ # process_event(rec[0], fobj)
+ # rechdr[2] is record length (including header)
+ #print etypes[rechdr[0]]
+ #data = fobj.read(rechdr[2] - header_len)
+ process_event(rechdr[0], fobj)
+
class Analyzer(object):
"""A trace file analyzer which processes trace records.
@@ -90,8 +165,6 @@ def process(events, log, analyzer):
"""Invoke an analyzer on each event in a log."""
if isinstance(events, str):
events = parse_events(open(events, 'r'))
- if isinstance(log, str):
- log = open(log, 'rb')
def build_fn(analyzer, event):
fn = getattr(analyzer, event[0], None)
@@ -129,8 +202,10 @@ def run(analyzer):
sys.exit(1)
events = parse_events(open(sys.argv[1], 'r'))
- process(events, sys.argv[2], analyzer)
+ process(events, log, analyzer)
+
+log = open(sys.argv[2], 'rb')
if __name__ == '__main__':
class Formatter(Analyzer):
def __init__(self):
@@ -148,4 +223,17 @@ if __name__ == '__main__':
fields.append('%s=0x%x' % (event[i], rec[i + 1]))
print ' '.join(fields)
- run(Formatter())
+ #run(Formatter())
+ header = read_header(log)
+ if header is None or \
+ header[0] != header_event_id or \
+ header[1] != header_magic:
+ raise ValueError('not a valid trace file')
+ if header[2] == 0: # version 1
+ # ver 1 tracelog header contains few more unused bytes
+ temp = log.read(40) # read those bytes and continue
+ run(Formatter())
+ elif header[2] == 2: # version 2
+ processv2(log)
+ else:
+ raise ValueError('trace file version not supported')
--
1.7.1.1
^ permalink raw reply related [flat|nested] 18+ messages in thread* Re: [Qemu-devel] [RFC PATCH v2 0/4] simpletrace : support var num of args and strings.
2012-01-09 15:46 [Qemu-devel] [RFC PATCH v2 0/4] simpletrace : support var num of args and strings Harsh Prateek Bora
` (3 preceding siblings ...)
2012-01-09 15:46 ` [Qemu-devel] [RFC PATCH v2 4/4] simpletrace.py: updated log reader script to handle new log format Harsh Prateek Bora
@ 2012-01-09 16:01 ` Mathieu Desnoyers
2012-01-09 19:20 ` Harsh Bora
4 siblings, 1 reply; 18+ messages in thread
From: Mathieu Desnoyers @ 2012-01-09 16:01 UTC (permalink / raw)
To: Harsh Prateek Bora
Cc: aneesh.kumar, lttng-dev, qemu-devel, stefanha, vilanova
* Harsh Prateek Bora (harsh@linux.vnet.ibm.com) wrote:
> Existing simple trace can log upto 6 args per trace event and does not
> support strings in trace record format. Introducing new trace format as
> discussed earlier on list to support variable number/size of arguments.
> (Ref: http://lists.gnu.org/archive/html/qemu-devel/2011-11/msg03426.html)
>
> Basic testing of this patch is successful. Stress testing not yet done.
>
> Apply patches, then run:
>
> make distclean
> ./configure with --enable-trace-backend=simple
> make
> sudo make install
>
> Sample tracelog showing strings support:
> [harsh@harshbora v9fs]$ scripts/simpletrace.py trace-events trace-23261
> v9fs_version 0.000 tag=65535 id=100 msize=8192 version=9P2000.L
> v9fs_version_return 17.530 tag=65535 id=100 msize=8192 version=9P2000.L
> v9fs_attach 180.121 tag=1 id=104 fid=0 afid=18446744073709551615
> uname=nobody aname=
>
>
> Note: LTTng ust backend is broken in upstream qemu, therefore tracetool.py
> doesnt support ust backend as of now. IIUC, ust's trace event APIs are under
> development and not yet stable.
Hi,
FYI, the LTTng-UST TRACEPOINT_EVENT API is very much stable as of now.
Even though we are still in LTTng-UST 2.0 prereleases, the fact that we
started the round of discussions on this API last summer makes us
confident that from this point on we should not have to change it.
Moreover, I would like to know if the old UST 0.x (0.16 is the latest)
is broken wrt qemu, or if this is just for LTTng-2.0 UST support ?
UST 0.x instrumentation is not supposed to have broken wrt qemu.
Best regards,
Mathieu
>
> Version History:
>
> v2:
> - Updated tracetool.py to support nop, stderr, dtrace backend
>
> v1:
> - Working protoype with tracetool.py converted only for simpletrace backend
>
> Harsh Prateek Bora (4):
> Converting tracetool.sh to tracetool.py
> Makefile and configure changes for tracetool.py
> simpletrace-v2: Handle variable number/size of elements per trace
> record.
> simpletrace.py: updated log reader script to handle new log format
>
> Makefile.objs | 6 +-
> Makefile.target | 10 +-
> configure | 4 +-
> monitor.c | 2 +-
> scripts/simpletrace.py | 110 ++++++++++-
> scripts/tracetool.py | 505 ++++++++++++++++++++++++++++++++++++++++++++++++
> trace/simple.c | 178 ++++++-----------
> trace/simple.h | 31 +++-
> 8 files changed, 702 insertions(+), 144 deletions(-)
> create mode 100755 scripts/tracetool.py
>
--
Mathieu Desnoyers
Operating System Efficiency R&D Consultant
EfficiOS Inc.
http://www.efficios.com
^ permalink raw reply [flat|nested] 18+ messages in thread* Re: [Qemu-devel] [RFC PATCH v2 0/4] simpletrace : support var num of args and strings.
2012-01-09 16:01 ` [Qemu-devel] [RFC PATCH v2 0/4] simpletrace : support var num of args and strings Mathieu Desnoyers
@ 2012-01-09 19:20 ` Harsh Bora
2012-01-10 0:14 ` Mathieu Desnoyers
0 siblings, 1 reply; 18+ messages in thread
From: Harsh Bora @ 2012-01-09 19:20 UTC (permalink / raw)
To: Mathieu Desnoyers; +Cc: aneesh.kumar, lttng-dev, qemu-devel, stefanha, vilanova
On 01/09/2012 09:31 PM, Mathieu Desnoyers wrote:
> * Harsh Prateek Bora (harsh@linux.vnet.ibm.com) wrote:
>> Existing simple trace can log upto 6 args per trace event and does not
>> support strings in trace record format. Introducing new trace format as
>> discussed earlier on list to support variable number/size of arguments.
>> (Ref: http://lists.gnu.org/archive/html/qemu-devel/2011-11/msg03426.html)
>>
>> Basic testing of this patch is successful. Stress testing not yet done.
>>
>> Apply patches, then run:
>>
>> make distclean
>> ./configure with --enable-trace-backend=simple
>> make
>> sudo make install
>>
>> Sample tracelog showing strings support:
>> [harsh@harshbora v9fs]$ scripts/simpletrace.py trace-events trace-23261
>> v9fs_version 0.000 tag=65535 id=100 msize=8192 version=9P2000.L
>> v9fs_version_return 17.530 tag=65535 id=100 msize=8192 version=9P2000.L
>> v9fs_attach 180.121 tag=1 id=104 fid=0 afid=18446744073709551615
>> uname=nobody aname=
>>
>>
>> Note: LTTng ust backend is broken in upstream qemu, therefore tracetool.py
>> doesnt support ust backend as of now. IIUC, ust's trace event APIs are under
>> development and not yet stable.
>
> Hi,
>
> FYI, the LTTng-UST TRACEPOINT_EVENT API is very much stable as of now.
> Even though we are still in LTTng-UST 2.0 prereleases, the fact that we
> started the round of discussions on this API last summer makes us
> confident that from this point on we should not have to change it.
>
> Moreover, I would like to know if the old UST 0.x (0.16 is the latest)
> is broken wrt qemu, or if this is just for LTTng-2.0 UST support ?
> UST 0.x instrumentation is not supposed to have broken wrt qemu.
>
Hi,
Thanks for an early response. I had tried building with ust 0.16 and it
gives compilation errors, specially for trace events with 'void' argument:
CC osdep.o
In file included from osdep.c:49:
trace.h: In function ‘__trace_ust_slavio_misc_update_irq_raise’:
trace.h:277: error: ‘void’ must be the only parameter
trace.h:277: error: expected expression before ‘)’ token
trace.h:277: error: too many arguments to function ‘(void (*)(void
*))__tp_it_func’
trace.h: At top level:
trace.h:277: error: ‘void’ must be the only parameter
trace.h:277: error: ‘void’ must be the only parameter
In file included from osdep.c:49:
trace.h: In function ‘__trace_ust_slavio_misc_update_irq_lower’:
trace.h:280: error: ‘void’ must be the only parameter
trace.h:280: error: expected expression before ‘)’ token
trace.h:280: error: too many arguments to function ‘(void (*)(void
*))__tp_it_func’
I am not sure which interface is supposed to be used for void
arguments in ust 0.16. Moreover, if ust 2.0 uses different interfaces,
we might want to use the latest one.
regards,
Harsh
> Best regards,
>
> Mathieu
>
>>
>> Version History:
>>
>> v2:
>> - Updated tracetool.py to support nop, stderr, dtrace backend
>>
>> v1:
>> - Working protoype with tracetool.py converted only for simpletrace backend
>>
>> Harsh Prateek Bora (4):
>> Converting tracetool.sh to tracetool.py
>> Makefile and configure changes for tracetool.py
>> simpletrace-v2: Handle variable number/size of elements per trace
>> record.
>> simpletrace.py: updated log reader script to handle new log format
>>
>> Makefile.objs | 6 +-
>> Makefile.target | 10 +-
>> configure | 4 +-
>> monitor.c | 2 +-
>> scripts/simpletrace.py | 110 ++++++++++-
>> scripts/tracetool.py | 505 ++++++++++++++++++++++++++++++++++++++++++++++++
>> trace/simple.c | 178 ++++++-----------
>> trace/simple.h | 31 +++-
>> 8 files changed, 702 insertions(+), 144 deletions(-)
>> create mode 100755 scripts/tracetool.py
>>
>
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Qemu-devel] [RFC PATCH v2 0/4] simpletrace : support var num of args and strings.
2012-01-09 19:20 ` Harsh Bora
@ 2012-01-10 0:14 ` Mathieu Desnoyers
2012-01-10 6:54 ` Harsh Bora
2012-01-10 14:58 ` Stefan Hajnoczi
0 siblings, 2 replies; 18+ messages in thread
From: Mathieu Desnoyers @ 2012-01-10 0:14 UTC (permalink / raw)
To: Harsh Bora; +Cc: aneesh.kumar, lttng-dev, qemu-devel, stefanha, vilanova
* Harsh Bora (harsh@linux.vnet.ibm.com) wrote:
> On 01/09/2012 09:31 PM, Mathieu Desnoyers wrote:
>> * Harsh Prateek Bora (harsh@linux.vnet.ibm.com) wrote:
>>> Existing simple trace can log upto 6 args per trace event and does not
>>> support strings in trace record format. Introducing new trace format as
>>> discussed earlier on list to support variable number/size of arguments.
>>> (Ref: http://lists.gnu.org/archive/html/qemu-devel/2011-11/msg03426.html)
>>>
>>> Basic testing of this patch is successful. Stress testing not yet done.
>>>
>>> Apply patches, then run:
>>>
>>> make distclean
>>> ./configure with --enable-trace-backend=simple
>>> make
>>> sudo make install
>>>
>>> Sample tracelog showing strings support:
>>> [harsh@harshbora v9fs]$ scripts/simpletrace.py trace-events trace-23261
>>> v9fs_version 0.000 tag=65535 id=100 msize=8192 version=9P2000.L
>>> v9fs_version_return 17.530 tag=65535 id=100 msize=8192 version=9P2000.L
>>> v9fs_attach 180.121 tag=1 id=104 fid=0 afid=18446744073709551615
>>> uname=nobody aname=
>>>
>>>
>>> Note: LTTng ust backend is broken in upstream qemu, therefore tracetool.py
>>> doesnt support ust backend as of now. IIUC, ust's trace event APIs are under
>>> development and not yet stable.
>>
>> Hi,
>>
>> FYI, the LTTng-UST TRACEPOINT_EVENT API is very much stable as of now.
>> Even though we are still in LTTng-UST 2.0 prereleases, the fact that we
>> started the round of discussions on this API last summer makes us
>> confident that from this point on we should not have to change it.
>>
>> Moreover, I would like to know if the old UST 0.x (0.16 is the latest)
>> is broken wrt qemu, or if this is just for LTTng-2.0 UST support ?
>> UST 0.x instrumentation is not supposed to have broken wrt qemu.
>>
>
> Hi,
> Thanks for an early response. I had tried building with ust 0.16 and it
> gives compilation errors, specially for trace events with 'void'
> argument:
>
> CC osdep.o
> In file included from osdep.c:49:
> trace.h: In function ‘__trace_ust_slavio_misc_update_irq_raise’:
> trace.h:277: error: ‘void’ must be the only parameter
> trace.h:277: error: expected expression before ‘)’ token
> trace.h:277: error: too many arguments to function ‘(void (*)(void
> *))__tp_it_func’
> trace.h: At top level:
> trace.h:277: error: ‘void’ must be the only parameter
> trace.h:277: error: ‘void’ must be the only parameter
> In file included from osdep.c:49:
> trace.h: In function ‘__trace_ust_slavio_misc_update_irq_lower’:
> trace.h:280: error: ‘void’ must be the only parameter
> trace.h:280: error: expected expression before ‘)’ token
> trace.h:280: error: too many arguments to function ‘(void (*)(void
> *))__tp_it_func’
>
>
> I am not sure which interface is supposed to be used for void arguments
> in ust 0.16.
Looking at scripts/tracetool:
linetoh_ust()
{
local name args argnames
name=$(get_name "$1")
args=$(get_args "$1")
argnames=$(get_argnames "$1", ",")
cat <<EOF
DECLARE_TRACE(ust_$name, TP_PROTO($args), TP_ARGS($argnames));
#define trace_$name trace_ust_$name
EOF
}
for those tracepoints with argument "void", DECLARE_TRACE_NOARGS should
be used for UST 0.16. Similar for:
DEFINE_TRACE(ust_$name); -> DEFINE_TRACE_NOARGS(ust_$name);
> Moreover, if ust 2.0 uses different interfaces, we might
> want to use the latest one.
Note that this kind of special-case won't be needed with LTTng-UST 2.0
TRACEPOINT_EVENT. In place of DECLARE_TRACE, one would use:
TRACEPOINT_EVENT(qemu_kvm, $name,
TP_ARGS($args),
TP_FIELDS()
)
Note that I notice that some care will need to be taken to generate the
TP_FIELDS() from your existing trace-events file, an example:
g_realloc(void *ptr, size_t size, void *newptr)
would have to be translated to:
TRACE_EVENT(qemu_kvm, g_realloc,
TP_ARGS(void *, ptr, size_t, size, void *, newptr),
TP_FIELDS(
ctf_integer_hex(void *, ptr, ptr)
ctf_integer(size_t, size, size)
ctf_integer_hex(void *, newptr, newptr)
)
)
Note that the bright side is that the tracepoint probe does not need to
be hand-coded anymore, and there is no need to use the markers anymore
neither, which makes the tracer much faster.
For most of your fields (using %p, %d style format strings), you should
use ctf_integer or ctf_integer_hex (the latter lets the trace viewer
know that the data should be printed as hexadecimal).
You will likely need to detect the %s format strings you have there and
translate them into ctf_string(field, field) too. You can have a look at
lttng-ust tests/hello/*.[ch] for examples.
The call which would have looked like trace_qemu_kvm_g_realloc() in UST
0.x should now be done with:
tracepoint(qemu_kvm, g_realloc, ptr, size, newptr);
This is needed to (very soon) add support for sdt.h in LTTng-UST 2.0, so
systemtap and gdb can hook into tracepoints declared by lttng-ust 2.0.
Best regards,
Mathieu
>
> regards,
> Harsh
>
>> Best regards,
>>
>> Mathieu
>>
>>>
>>> Version History:
>>>
>>> v2:
>>> - Updated tracetool.py to support nop, stderr, dtrace backend
>>>
>>> v1:
>>> - Working protoype with tracetool.py converted only for simpletrace backend
>>>
>>> Harsh Prateek Bora (4):
>>> Converting tracetool.sh to tracetool.py
>>> Makefile and configure changes for tracetool.py
>>> simpletrace-v2: Handle variable number/size of elements per trace
>>> record.
>>> simpletrace.py: updated log reader script to handle new log format
>>>
>>> Makefile.objs | 6 +-
>>> Makefile.target | 10 +-
>>> configure | 4 +-
>>> monitor.c | 2 +-
>>> scripts/simpletrace.py | 110 ++++++++++-
>>> scripts/tracetool.py | 505 ++++++++++++++++++++++++++++++++++++++++++++++++
>>> trace/simple.c | 178 ++++++-----------
>>> trace/simple.h | 31 +++-
>>> 8 files changed, 702 insertions(+), 144 deletions(-)
>>> create mode 100755 scripts/tracetool.py
>>>
>>
>
--
Mathieu Desnoyers
Operating System Efficiency R&D Consultant
EfficiOS Inc.
http://www.efficios.com
^ permalink raw reply [flat|nested] 18+ messages in thread* Re: [Qemu-devel] [RFC PATCH v2 0/4] simpletrace : support var num of args and strings.
2012-01-10 0:14 ` Mathieu Desnoyers
@ 2012-01-10 6:54 ` Harsh Bora
2012-01-10 7:17 ` [Qemu-devel] [lttng-dev] " Mathieu Desnoyers
2012-01-10 14:58 ` Stefan Hajnoczi
1 sibling, 1 reply; 18+ messages in thread
From: Harsh Bora @ 2012-01-10 6:54 UTC (permalink / raw)
To: Mathieu Desnoyers; +Cc: aneesh.kumar, lttng-dev, qemu-devel, stefanha, vilanova
On 01/10/2012 05:44 AM, Mathieu Desnoyers wrote:
> * Harsh Bora (harsh@linux.vnet.ibm.com) wrote:
>> On 01/09/2012 09:31 PM, Mathieu Desnoyers wrote:
>>> * Harsh Prateek Bora (harsh@linux.vnet.ibm.com) wrote:
>>>> Existing simple trace can log upto 6 args per trace event and does not
>>>> support strings in trace record format. Introducing new trace format as
>>>> discussed earlier on list to support variable number/size of arguments.
>>>> (Ref: http://lists.gnu.org/archive/html/qemu-devel/2011-11/msg03426.html)
>>>>
>>>> Basic testing of this patch is successful. Stress testing not yet done.
>>>>
>>>> Apply patches, then run:
>>>>
>>>> make distclean
>>>> ./configure with --enable-trace-backend=simple
>>>> make
>>>> sudo make install
>>>>
>>>> Sample tracelog showing strings support:
>>>> [harsh@harshbora v9fs]$ scripts/simpletrace.py trace-events trace-23261
>>>> v9fs_version 0.000 tag=65535 id=100 msize=8192 version=9P2000.L
>>>> v9fs_version_return 17.530 tag=65535 id=100 msize=8192 version=9P2000.L
>>>> v9fs_attach 180.121 tag=1 id=104 fid=0 afid=18446744073709551615
>>>> uname=nobody aname=
>>>>
>>>>
>>>> Note: LTTng ust backend is broken in upstream qemu, therefore tracetool.py
>>>> doesnt support ust backend as of now. IIUC, ust's trace event APIs are under
>>>> development and not yet stable.
>>>
>>> Hi,
>>>
>>> FYI, the LTTng-UST TRACEPOINT_EVENT API is very much stable as of now.
>>> Even though we are still in LTTng-UST 2.0 prereleases, the fact that we
>>> started the round of discussions on this API last summer makes us
>>> confident that from this point on we should not have to change it.
>>>
>>> Moreover, I would like to know if the old UST 0.x (0.16 is the latest)
>>> is broken wrt qemu, or if this is just for LTTng-2.0 UST support ?
>>> UST 0.x instrumentation is not supposed to have broken wrt qemu.
>>>
>>
>> Hi,
>> Thanks for an early response. I had tried building with ust 0.16 and it
>> gives compilation errors, specially for trace events with 'void'
>> argument:
>>
>> CC osdep.o
>> In file included from osdep.c:49:
>> trace.h: In function ‘__trace_ust_slavio_misc_update_irq_raise’:
>> trace.h:277: error: ‘void’ must be the only parameter
>> trace.h:277: error: expected expression before ‘)’ token
>> trace.h:277: error: too many arguments to function ‘(void (*)(void
>> *))__tp_it_func’
>> trace.h: At top level:
>> trace.h:277: error: ‘void’ must be the only parameter
>> trace.h:277: error: ‘void’ must be the only parameter
>> In file included from osdep.c:49:
>> trace.h: In function ‘__trace_ust_slavio_misc_update_irq_lower’:
>> trace.h:280: error: ‘void’ must be the only parameter
>> trace.h:280: error: expected expression before ‘)’ token
>> trace.h:280: error: too many arguments to function ‘(void (*)(void
>> *))__tp_it_func’
>>
>>
>> I am not sure which interface is supposed to be used for void arguments
>> in ust 0.16.
>
> Looking at scripts/tracetool:
>
> linetoh_ust()
> {
> local name args argnames
> name=$(get_name "$1")
> args=$(get_args "$1")
> argnames=$(get_argnames "$1", ",")
>
> cat<<EOF
> DECLARE_TRACE(ust_$name, TP_PROTO($args), TP_ARGS($argnames));
> #define trace_$name trace_ust_$name
> EOF
> }
>
> for those tracepoints with argument "void", DECLARE_TRACE_NOARGS should
> be used for UST 0.16. Similar for:
>
> DEFINE_TRACE(ust_$name); -> DEFINE_TRACE_NOARGS(ust_$name);
>
I had tried with _NOARGS variants initially by manually changing the
auto-generated code like this:
In trace.h:
DECLARE_TRACE_NOARGS(ust_slavio_misc_update_irq_raise);
#define trace_slavio_misc_update_irq_raise
trace_ust_slavio_misc_update_irq_raise
In trace.c:
DEFINE_TRACE_NOARGS(ust_slavio_misc_update_irq_raise);
static void ust_slavio_misc_update_irq_raise_probe()
{
trace_mark(ust, slavio_misc_update_irq_raise);
}
However, it still gave error like this:
[harsh@harshbora v9fs]$ make
CC osdep.o
cc1: warnings being treated as errors
In file included from osdep.c:49:
trace.h:277: error: data definition has no type or storage class
trace.h:277: error: type defaults to ‘int’ in declaration of
‘DECLARE_TRACE_NOARGS’
trace.h:277: error: parameter names (without types) in function declaration
It will be great if you can provide a sample code (to be auto-generated)
required for a trace-event with void argument.
- Harsh
>> Moreover, if ust 2.0 uses different interfaces, we might
>> want to use the latest one.
>
> Note that this kind of special-case won't be needed with LTTng-UST 2.0
> TRACEPOINT_EVENT. In place of DECLARE_TRACE, one would use:
>
> TRACEPOINT_EVENT(qemu_kvm, $name,
> TP_ARGS($args),
> TP_FIELDS()
> )
>
> Note that I notice that some care will need to be taken to generate the
> TP_FIELDS() from your existing trace-events file, an example:
>
> g_realloc(void *ptr, size_t size, void *newptr)
>
> would have to be translated to:
>
> TRACE_EVENT(qemu_kvm, g_realloc,
> TP_ARGS(void *, ptr, size_t, size, void *, newptr),
> TP_FIELDS(
> ctf_integer_hex(void *, ptr, ptr)
> ctf_integer(size_t, size, size)
> ctf_integer_hex(void *, newptr, newptr)
> )
> )
>
> Note that the bright side is that the tracepoint probe does not need to
> be hand-coded anymore, and there is no need to use the markers anymore
> neither, which makes the tracer much faster.
>
> For most of your fields (using %p, %d style format strings), you should
> use ctf_integer or ctf_integer_hex (the latter lets the trace viewer
> know that the data should be printed as hexadecimal).
> You will likely need to detect the %s format strings you have there and
> translate them into ctf_string(field, field) too. You can have a look at
> lttng-ust tests/hello/*.[ch] for examples.
Thanks for the info, I will look into it later.
>
> The call which would have looked like trace_qemu_kvm_g_realloc() in UST
> 0.x should now be done with:
>
> tracepoint(qemu_kvm, g_realloc, ptr, size, newptr);
>
> This is needed to (very soon) add support for sdt.h in LTTng-UST 2.0, so
> systemtap and gdb can hook into tracepoints declared by lttng-ust 2.0.
>
> Best regards,
>
> Mathieu
>
>>
>> regards,
>> Harsh
>>
>>> Best regards,
>>>
>>> Mathieu
>>>
>>>>
>>>> Version History:
>>>>
>>>> v2:
>>>> - Updated tracetool.py to support nop, stderr, dtrace backend
>>>>
>>>> v1:
>>>> - Working protoype with tracetool.py converted only for simpletrace backend
>>>>
>>>> Harsh Prateek Bora (4):
>>>> Converting tracetool.sh to tracetool.py
>>>> Makefile and configure changes for tracetool.py
>>>> simpletrace-v2: Handle variable number/size of elements per trace
>>>> record.
>>>> simpletrace.py: updated log reader script to handle new log format
>>>>
>>>> Makefile.objs | 6 +-
>>>> Makefile.target | 10 +-
>>>> configure | 4 +-
>>>> monitor.c | 2 +-
>>>> scripts/simpletrace.py | 110 ++++++++++-
>>>> scripts/tracetool.py | 505 ++++++++++++++++++++++++++++++++++++++++++++++++
>>>> trace/simple.c | 178 ++++++-----------
>>>> trace/simple.h | 31 +++-
>>>> 8 files changed, 702 insertions(+), 144 deletions(-)
>>>> create mode 100755 scripts/tracetool.py
>>>>
>>>
>>
>
^ permalink raw reply [flat|nested] 18+ messages in thread* Re: [Qemu-devel] [lttng-dev] [RFC PATCH v2 0/4] simpletrace : support var num of args and strings.
2012-01-10 6:54 ` Harsh Bora
@ 2012-01-10 7:17 ` Mathieu Desnoyers
2012-01-10 9:06 ` Harsh Bora
0 siblings, 1 reply; 18+ messages in thread
From: Mathieu Desnoyers @ 2012-01-10 7:17 UTC (permalink / raw)
To: Harsh Bora
Cc: stefanha, qemu-devel, lttng-dev, Mathieu Desnoyers, aneesh.kumar,
vilanova
* Harsh Bora (harsh@linux.vnet.ibm.com) wrote:
> On 01/10/2012 05:44 AM, Mathieu Desnoyers wrote:
>> * Harsh Bora (harsh@linux.vnet.ibm.com) wrote:
>>> On 01/09/2012 09:31 PM, Mathieu Desnoyers wrote:
>>>> * Harsh Prateek Bora (harsh@linux.vnet.ibm.com) wrote:
>>>>> Existing simple trace can log upto 6 args per trace event and does not
>>>>> support strings in trace record format. Introducing new trace format as
>>>>> discussed earlier on list to support variable number/size of arguments.
>>>>> (Ref: http://lists.gnu.org/archive/html/qemu-devel/2011-11/msg03426.html)
>>>>>
>>>>> Basic testing of this patch is successful. Stress testing not yet done.
>>>>>
>>>>> Apply patches, then run:
>>>>>
>>>>> make distclean
>>>>> ./configure with --enable-trace-backend=simple
>>>>> make
>>>>> sudo make install
>>>>>
>>>>> Sample tracelog showing strings support:
>>>>> [harsh@harshbora v9fs]$ scripts/simpletrace.py trace-events trace-23261
>>>>> v9fs_version 0.000 tag=65535 id=100 msize=8192 version=9P2000.L
>>>>> v9fs_version_return 17.530 tag=65535 id=100 msize=8192 version=9P2000.L
>>>>> v9fs_attach 180.121 tag=1 id=104 fid=0 afid=18446744073709551615
>>>>> uname=nobody aname=
>>>>>
>>>>>
>>>>> Note: LTTng ust backend is broken in upstream qemu, therefore tracetool.py
>>>>> doesnt support ust backend as of now. IIUC, ust's trace event APIs are under
>>>>> development and not yet stable.
>>>>
>>>> Hi,
>>>>
>>>> FYI, the LTTng-UST TRACEPOINT_EVENT API is very much stable as of now.
>>>> Even though we are still in LTTng-UST 2.0 prereleases, the fact that we
>>>> started the round of discussions on this API last summer makes us
>>>> confident that from this point on we should not have to change it.
>>>>
>>>> Moreover, I would like to know if the old UST 0.x (0.16 is the latest)
>>>> is broken wrt qemu, or if this is just for LTTng-2.0 UST support ?
>>>> UST 0.x instrumentation is not supposed to have broken wrt qemu.
>>>>
>>>
>>> Hi,
>>> Thanks for an early response. I had tried building with ust 0.16 and it
>>> gives compilation errors, specially for trace events with 'void'
>>> argument:
>>>
>>> CC osdep.o
>>> In file included from osdep.c:49:
>>> trace.h: In function ‘__trace_ust_slavio_misc_update_irq_raise’:
>>> trace.h:277: error: ‘void’ must be the only parameter
>>> trace.h:277: error: expected expression before ‘)’ token
>>> trace.h:277: error: too many arguments to function ‘(void (*)(void
>>> *))__tp_it_func’
>>> trace.h: At top level:
>>> trace.h:277: error: ‘void’ must be the only parameter
>>> trace.h:277: error: ‘void’ must be the only parameter
>>> In file included from osdep.c:49:
>>> trace.h: In function ‘__trace_ust_slavio_misc_update_irq_lower’:
>>> trace.h:280: error: ‘void’ must be the only parameter
>>> trace.h:280: error: expected expression before ‘)’ token
>>> trace.h:280: error: too many arguments to function ‘(void (*)(void
>>> *))__tp_it_func’
>>>
>>>
>>> I am not sure which interface is supposed to be used for void arguments
>>> in ust 0.16.
>>
>> Looking at scripts/tracetool:
>>
>> linetoh_ust()
>> {
>> local name args argnames
>> name=$(get_name "$1")
>> args=$(get_args "$1")
>> argnames=$(get_argnames "$1", ",")
>>
>> cat<<EOF
>> DECLARE_TRACE(ust_$name, TP_PROTO($args), TP_ARGS($argnames));
>> #define trace_$name trace_ust_$name
>> EOF
>> }
>>
>> for those tracepoints with argument "void", DECLARE_TRACE_NOARGS should
>> be used for UST 0.16. Similar for:
>>
>> DEFINE_TRACE(ust_$name); -> DEFINE_TRACE_NOARGS(ust_$name);
>>
>
> I had tried with _NOARGS variants initially by manually changing the
> auto-generated code like this:
>
> In trace.h:
>
> DECLARE_TRACE_NOARGS(ust_slavio_misc_update_irq_raise);
> #define trace_slavio_misc_update_irq_raise
> trace_ust_slavio_misc_update_irq_raise
>
> In trace.c:
>
> DEFINE_TRACE_NOARGS(ust_slavio_misc_update_irq_raise);
>
> static void ust_slavio_misc_update_irq_raise_probe()
> {
> trace_mark(ust, slavio_misc_update_irq_raise);
> }
>
>
> However, it still gave error like this:
>
> [harsh@harshbora v9fs]$ make
> CC osdep.o
> cc1: warnings being treated as errors
> In file included from osdep.c:49:
> trace.h:277: error: data definition has no type or storage class
> trace.h:277: error: type defaults to ‘int’ in declaration of
> ‘DECLARE_TRACE_NOARGS’
> trace.h:277: error: parameter names (without types) in function declaration
>
>
> It will be great if you can provide a sample code (to be auto-generated)
> required for a trace-event with void argument.
Ah, right, now I remember that I only left DEFINE_TRACE and DECLARE_TRACE
as backward compatibility in UST 0.x, and did only add the "_NOARGS" to
the backward compatibility macros with an added underscore prefix.
If you really just want to have compatibility with UST 0.16, changing the
minimum of things compared with what you currently have, I would
recommend to use:
in trace.h:
_DECLARE_TRACEPOINT_NOARGS(ust_slavio_misc_update_irq_raise)
#define trace_slavio_misc_update_irq_raise trace_ust_slavio_misc_update_irq_raise
in trace.c:
DEFINE_TRACE(ust_slavio_misc_update_irq_raise);
static void ust_slavio_misc_update_irq_raise_probe(void)
{
trace_mark(ust, slavio_misc_update_irq_raise, UST_MARKER_NOARGS);
}
Let me know how this works for you,
Best regards,
Mathieu
>
> - Harsh
>
>>> Moreover, if ust 2.0 uses different interfaces, we might
>>> want to use the latest one.
>>
>> Note that this kind of special-case won't be needed with LTTng-UST 2.0
>> TRACEPOINT_EVENT. In place of DECLARE_TRACE, one would use:
>>
>> TRACEPOINT_EVENT(qemu_kvm, $name,
>> TP_ARGS($args),
>> TP_FIELDS()
>> )
>>
>> Note that I notice that some care will need to be taken to generate the
>> TP_FIELDS() from your existing trace-events file, an example:
>>
>> g_realloc(void *ptr, size_t size, void *newptr)
>>
>> would have to be translated to:
>>
>> TRACE_EVENT(qemu_kvm, g_realloc,
>> TP_ARGS(void *, ptr, size_t, size, void *, newptr),
>> TP_FIELDS(
>> ctf_integer_hex(void *, ptr, ptr)
>> ctf_integer(size_t, size, size)
>> ctf_integer_hex(void *, newptr, newptr)
>> )
>> )
>>
>> Note that the bright side is that the tracepoint probe does not need to
>> be hand-coded anymore, and there is no need to use the markers anymore
>> neither, which makes the tracer much faster.
>>
>> For most of your fields (using %p, %d style format strings), you should
>> use ctf_integer or ctf_integer_hex (the latter lets the trace viewer
>> know that the data should be printed as hexadecimal).
>> You will likely need to detect the %s format strings you have there and
>> translate them into ctf_string(field, field) too. You can have a look at
>> lttng-ust tests/hello/*.[ch] for examples.
>
> Thanks for the info, I will look into it later.
>>
>> The call which would have looked like trace_qemu_kvm_g_realloc() in UST
>> 0.x should now be done with:
>>
>> tracepoint(qemu_kvm, g_realloc, ptr, size, newptr);
>>
>> This is needed to (very soon) add support for sdt.h in LTTng-UST 2.0, so
>> systemtap and gdb can hook into tracepoints declared by lttng-ust 2.0.
>>
>> Best regards,
>>
>> Mathieu
>>
>>>
>>> regards,
>>> Harsh
>>>
>>>> Best regards,
>>>>
>>>> Mathieu
>>>>
>>>>>
>>>>> Version History:
>>>>>
>>>>> v2:
>>>>> - Updated tracetool.py to support nop, stderr, dtrace backend
>>>>>
>>>>> v1:
>>>>> - Working protoype with tracetool.py converted only for simpletrace backend
>>>>>
>>>>> Harsh Prateek Bora (4):
>>>>> Converting tracetool.sh to tracetool.py
>>>>> Makefile and configure changes for tracetool.py
>>>>> simpletrace-v2: Handle variable number/size of elements per trace
>>>>> record.
>>>>> simpletrace.py: updated log reader script to handle new log format
>>>>>
>>>>> Makefile.objs | 6 +-
>>>>> Makefile.target | 10 +-
>>>>> configure | 4 +-
>>>>> monitor.c | 2 +-
>>>>> scripts/simpletrace.py | 110 ++++++++++-
>>>>> scripts/tracetool.py | 505 ++++++++++++++++++++++++++++++++++++++++++++++++
>>>>> trace/simple.c | 178 ++++++-----------
>>>>> trace/simple.h | 31 +++-
>>>>> 8 files changed, 702 insertions(+), 144 deletions(-)
>>>>> create mode 100755 scripts/tracetool.py
>>>>>
>>>>
>>>
>>
>
>
> _______________________________________________
> lttng-dev mailing list
> lttng-dev@lists.lttng.org
> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
--
Mathieu Desnoyers
Operating System Efficiency R&D Consultant
EfficiOS Inc.
http://www.efficios.com
^ permalink raw reply [flat|nested] 18+ messages in thread* Re: [Qemu-devel] [lttng-dev] [RFC PATCH v2 0/4] simpletrace : support var num of args and strings.
2012-01-10 7:17 ` [Qemu-devel] [lttng-dev] " Mathieu Desnoyers
@ 2012-01-10 9:06 ` Harsh Bora
2012-01-10 10:44 ` Harsh Bora
0 siblings, 1 reply; 18+ messages in thread
From: Harsh Bora @ 2012-01-10 9:06 UTC (permalink / raw)
To: Mathieu Desnoyers
Cc: stefanha, qemu-devel, lttng-dev, Mathieu Desnoyers, aneesh.kumar,
vilanova
On 01/10/2012 12:47 PM, Mathieu Desnoyers wrote:
> * Harsh Bora (harsh@linux.vnet.ibm.com) wrote:
>> On 01/10/2012 05:44 AM, Mathieu Desnoyers wrote:
>>> * Harsh Bora (harsh@linux.vnet.ibm.com) wrote:
>>>> On 01/09/2012 09:31 PM, Mathieu Desnoyers wrote:
>>>>> * Harsh Prateek Bora (harsh@linux.vnet.ibm.com) wrote:
>>>>>> Existing simple trace can log upto 6 args per trace event and does not
>>>>>> support strings in trace record format. Introducing new trace format as
>>>>>> discussed earlier on list to support variable number/size of arguments.
>>>>>> (Ref: http://lists.gnu.org/archive/html/qemu-devel/2011-11/msg03426.html)
>>>>>>
>>>>>> Basic testing of this patch is successful. Stress testing not yet done.
>>>>>>
>>>>>> Apply patches, then run:
>>>>>>
>>>>>> make distclean
>>>>>> ./configure with --enable-trace-backend=simple
>>>>>> make
>>>>>> sudo make install
>>>>>>
>>>>>> Sample tracelog showing strings support:
>>>>>> [harsh@harshbora v9fs]$ scripts/simpletrace.py trace-events trace-23261
>>>>>> v9fs_version 0.000 tag=65535 id=100 msize=8192 version=9P2000.L
>>>>>> v9fs_version_return 17.530 tag=65535 id=100 msize=8192 version=9P2000.L
>>>>>> v9fs_attach 180.121 tag=1 id=104 fid=0 afid=18446744073709551615
>>>>>> uname=nobody aname=
>>>>>>
>>>>>>
>>>>>> Note: LTTng ust backend is broken in upstream qemu, therefore tracetool.py
>>>>>> doesnt support ust backend as of now. IIUC, ust's trace event APIs are under
>>>>>> development and not yet stable.
>>>>>
>>>>> Hi,
>>>>>
>>>>> FYI, the LTTng-UST TRACEPOINT_EVENT API is very much stable as of now.
>>>>> Even though we are still in LTTng-UST 2.0 prereleases, the fact that we
>>>>> started the round of discussions on this API last summer makes us
>>>>> confident that from this point on we should not have to change it.
>>>>>
>>>>> Moreover, I would like to know if the old UST 0.x (0.16 is the latest)
>>>>> is broken wrt qemu, or if this is just for LTTng-2.0 UST support ?
>>>>> UST 0.x instrumentation is not supposed to have broken wrt qemu.
>>>>>
>>>>
>>>> Hi,
>>>> Thanks for an early response. I had tried building with ust 0.16 and it
>>>> gives compilation errors, specially for trace events with 'void'
>>>> argument:
>>>>
>>>> CC osdep.o
>>>> In file included from osdep.c:49:
>>>> trace.h: In function ‘__trace_ust_slavio_misc_update_irq_raise’:
>>>> trace.h:277: error: ‘void’ must be the only parameter
>>>> trace.h:277: error: expected expression before ‘)’ token
>>>> trace.h:277: error: too many arguments to function ‘(void (*)(void
>>>> *))__tp_it_func’
>>>> trace.h: At top level:
>>>> trace.h:277: error: ‘void’ must be the only parameter
>>>> trace.h:277: error: ‘void’ must be the only parameter
>>>> In file included from osdep.c:49:
>>>> trace.h: In function ‘__trace_ust_slavio_misc_update_irq_lower’:
>>>> trace.h:280: error: ‘void’ must be the only parameter
>>>> trace.h:280: error: expected expression before ‘)’ token
>>>> trace.h:280: error: too many arguments to function ‘(void (*)(void
>>>> *))__tp_it_func’
>>>>
>>>>
>>>> I am not sure which interface is supposed to be used for void arguments
>>>> in ust 0.16.
>>>
>>> Looking at scripts/tracetool:
>>>
>>> linetoh_ust()
>>> {
>>> local name args argnames
>>> name=$(get_name "$1")
>>> args=$(get_args "$1")
>>> argnames=$(get_argnames "$1", ",")
>>>
>>> cat<<EOF
>>> DECLARE_TRACE(ust_$name, TP_PROTO($args), TP_ARGS($argnames));
>>> #define trace_$name trace_ust_$name
>>> EOF
>>> }
>>>
>>> for those tracepoints with argument "void", DECLARE_TRACE_NOARGS should
>>> be used for UST 0.16. Similar for:
>>>
>>> DEFINE_TRACE(ust_$name); -> DEFINE_TRACE_NOARGS(ust_$name);
>>>
>>
>> I had tried with _NOARGS variants initially by manually changing the
>> auto-generated code like this:
>>
>> In trace.h:
>>
>> DECLARE_TRACE_NOARGS(ust_slavio_misc_update_irq_raise);
>> #define trace_slavio_misc_update_irq_raise
>> trace_ust_slavio_misc_update_irq_raise
>>
>> In trace.c:
>>
>> DEFINE_TRACE_NOARGS(ust_slavio_misc_update_irq_raise);
>>
>> static void ust_slavio_misc_update_irq_raise_probe()
>> {
>> trace_mark(ust, slavio_misc_update_irq_raise);
>> }
>>
>>
>> However, it still gave error like this:
>>
>> [harsh@harshbora v9fs]$ make
>> CC osdep.o
>> cc1: warnings being treated as errors
>> In file included from osdep.c:49:
>> trace.h:277: error: data definition has no type or storage class
>> trace.h:277: error: type defaults to ‘int’ in declaration of
>> ‘DECLARE_TRACE_NOARGS’
>> trace.h:277: error: parameter names (without types) in function declaration
>>
>>
>> It will be great if you can provide a sample code (to be auto-generated)
>> required for a trace-event with void argument.
>
> Ah, right, now I remember that I only left DEFINE_TRACE and DECLARE_TRACE
> as backward compatibility in UST 0.x, and did only add the "_NOARGS" to
> the backward compatibility macros with an added underscore prefix.
>
> If you really just want to have compatibility with UST 0.16, changing the
> minimum of things compared with what you currently have, I would
> recommend to use:
>
> in trace.h:
> _DECLARE_TRACEPOINT_NOARGS(ust_slavio_misc_update_irq_raise)
> #define trace_slavio_misc_update_irq_raise trace_ust_slavio_misc_update_irq_raise
>
> in trace.c:
> DEFINE_TRACE(ust_slavio_misc_update_irq_raise);
>
> static void ust_slavio_misc_update_irq_raise_probe(void)
> {
> trace_mark(ust, slavio_misc_update_irq_raise, UST_MARKER_NOARGS);
> }
>
> Let me know how this works for you,
Thanks for the help, I wrote the python code to auto-generate the same
for events with void arg. However, after doing make, it says:
trace.o:(__ust_markers+0x4a08): undefined reference to
`__ust_marker_empty_function'
trace.o:(__ust_markers+0x4a60): undefined reference to `ust_marker_probe_cb'
trace.o:(__ust_markers+0x4a68): undefined reference to
`__ust_marker_empty_function'
trace.o:(__ust_markers+0x4ac0): undefined reference to `ust_marker_probe_cb'
I have tested using both:
1) ust 0.15 with urcu 0.6.
2) ust 0.16 with urcu 0.6.6
Any idea, why this error ?
- Harsh
>
> Best regards,
>
> Mathieu
>
>>
>> - Harsh
>>
>>>> Moreover, if ust 2.0 uses different interfaces, we might
>>>> want to use the latest one.
>>>
>>> Note that this kind of special-case won't be needed with LTTng-UST 2.0
>>> TRACEPOINT_EVENT. In place of DECLARE_TRACE, one would use:
>>>
>>> TRACEPOINT_EVENT(qemu_kvm, $name,
>>> TP_ARGS($args),
>>> TP_FIELDS()
>>> )
>>>
>>> Note that I notice that some care will need to be taken to generate the
>>> TP_FIELDS() from your existing trace-events file, an example:
>>>
>>> g_realloc(void *ptr, size_t size, void *newptr)
>>>
>>> would have to be translated to:
>>>
>>> TRACE_EVENT(qemu_kvm, g_realloc,
>>> TP_ARGS(void *, ptr, size_t, size, void *, newptr),
>>> TP_FIELDS(
>>> ctf_integer_hex(void *, ptr, ptr)
>>> ctf_integer(size_t, size, size)
>>> ctf_integer_hex(void *, newptr, newptr)
>>> )
>>> )
>>>
>>> Note that the bright side is that the tracepoint probe does not need to
>>> be hand-coded anymore, and there is no need to use the markers anymore
>>> neither, which makes the tracer much faster.
>>>
>>> For most of your fields (using %p, %d style format strings), you should
>>> use ctf_integer or ctf_integer_hex (the latter lets the trace viewer
>>> know that the data should be printed as hexadecimal).
>>> You will likely need to detect the %s format strings you have there and
>>> translate them into ctf_string(field, field) too. You can have a look at
>>> lttng-ust tests/hello/*.[ch] for examples.
>>
>> Thanks for the info, I will look into it later.
>>>
>>> The call which would have looked like trace_qemu_kvm_g_realloc() in UST
>>> 0.x should now be done with:
>>>
>>> tracepoint(qemu_kvm, g_realloc, ptr, size, newptr);
>>>
>>> This is needed to (very soon) add support for sdt.h in LTTng-UST 2.0, so
>>> systemtap and gdb can hook into tracepoints declared by lttng-ust 2.0.
>>>
>>> Best regards,
>>>
>>> Mathieu
>>>
>>>>
>>>> regards,
>>>> Harsh
>>>>
>>>>> Best regards,
>>>>>
>>>>> Mathieu
>>>>>
>>>>>>
>>>>>> Version History:
>>>>>>
>>>>>> v2:
>>>>>> - Updated tracetool.py to support nop, stderr, dtrace backend
>>>>>>
>>>>>> v1:
>>>>>> - Working protoype with tracetool.py converted only for simpletrace backend
>>>>>>
>>>>>> Harsh Prateek Bora (4):
>>>>>> Converting tracetool.sh to tracetool.py
>>>>>> Makefile and configure changes for tracetool.py
>>>>>> simpletrace-v2: Handle variable number/size of elements per trace
>>>>>> record.
>>>>>> simpletrace.py: updated log reader script to handle new log format
>>>>>>
>>>>>> Makefile.objs | 6 +-
>>>>>> Makefile.target | 10 +-
>>>>>> configure | 4 +-
>>>>>> monitor.c | 2 +-
>>>>>> scripts/simpletrace.py | 110 ++++++++++-
>>>>>> scripts/tracetool.py | 505 ++++++++++++++++++++++++++++++++++++++++++++++++
>>>>>> trace/simple.c | 178 ++++++-----------
>>>>>> trace/simple.h | 31 +++-
>>>>>> 8 files changed, 702 insertions(+), 144 deletions(-)
>>>>>> create mode 100755 scripts/tracetool.py
>>>>>>
>>>>>
>>>>
>>>
>>
>>
>> _______________________________________________
>> lttng-dev mailing list
>> lttng-dev@lists.lttng.org
>> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>
^ permalink raw reply [flat|nested] 18+ messages in thread* Re: [Qemu-devel] [lttng-dev] [RFC PATCH v2 0/4] simpletrace : support var num of args and strings.
2012-01-10 9:06 ` Harsh Bora
@ 2012-01-10 10:44 ` Harsh Bora
2012-01-10 14:18 ` Mathieu Desnoyers
0 siblings, 1 reply; 18+ messages in thread
From: Harsh Bora @ 2012-01-10 10:44 UTC (permalink / raw)
To: Mathieu Desnoyers
Cc: stefanha, qemu-devel, lttng-dev, Mathieu Desnoyers, aneesh.kumar,
vilanova
On 01/10/2012 02:36 PM, Harsh Bora wrote:
> On 01/10/2012 12:47 PM, Mathieu Desnoyers wrote:
>> * Harsh Bora (harsh@linux.vnet.ibm.com) wrote:
>>> On 01/10/2012 05:44 AM, Mathieu Desnoyers wrote:
>>>> * Harsh Bora (harsh@linux.vnet.ibm.com) wrote:
>>>>> On 01/09/2012 09:31 PM, Mathieu Desnoyers wrote:
>>>>>> * Harsh Prateek Bora (harsh@linux.vnet.ibm.com) wrote:
>>>>>>> Existing simple trace can log upto 6 args per trace event and
>>>>>>> does not
>>>>>>> support strings in trace record format. Introducing new trace
>>>>>>> format as
>>>>>>> discussed earlier on list to support variable number/size of
>>>>>>> arguments.
>>>>>>> (Ref:
>>>>>>> http://lists.gnu.org/archive/html/qemu-devel/2011-11/msg03426.html)
>>>>>>>
>>>>>>> Basic testing of this patch is successful. Stress testing not yet
>>>>>>> done.
>>>>>>>
>>>>>>> Apply patches, then run:
>>>>>>>
>>>>>>> make distclean
>>>>>>> ./configure with --enable-trace-backend=simple
>>>>>>> make
>>>>>>> sudo make install
>>>>>>>
>>>>>>> Sample tracelog showing strings support:
>>>>>>> [harsh@harshbora v9fs]$ scripts/simpletrace.py trace-events
>>>>>>> trace-23261
>>>>>>> v9fs_version 0.000 tag=65535 id=100 msize=8192 version=9P2000.L
>>>>>>> v9fs_version_return 17.530 tag=65535 id=100 msize=8192
>>>>>>> version=9P2000.L
>>>>>>> v9fs_attach 180.121 tag=1 id=104 fid=0 afid=18446744073709551615
>>>>>>> uname=nobody aname=
>>>>>>>
>>>>>>>
>>>>>>> Note: LTTng ust backend is broken in upstream qemu, therefore
>>>>>>> tracetool.py
>>>>>>> doesnt support ust backend as of now. IIUC, ust's trace event
>>>>>>> APIs are under
>>>>>>> development and not yet stable.
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> FYI, the LTTng-UST TRACEPOINT_EVENT API is very much stable as of
>>>>>> now.
>>>>>> Even though we are still in LTTng-UST 2.0 prereleases, the fact
>>>>>> that we
>>>>>> started the round of discussions on this API last summer makes us
>>>>>> confident that from this point on we should not have to change it.
>>>>>>
>>>>>> Moreover, I would like to know if the old UST 0.x (0.16 is the
>>>>>> latest)
>>>>>> is broken wrt qemu, or if this is just for LTTng-2.0 UST support ?
>>>>>> UST 0.x instrumentation is not supposed to have broken wrt qemu.
>>>>>>
>>>>>
>>>>> Hi,
>>>>> Thanks for an early response. I had tried building with ust 0.16
>>>>> and it
>>>>> gives compilation errors, specially for trace events with 'void'
>>>>> argument:
>>>>>
>>>>> CC osdep.o
>>>>> In file included from osdep.c:49:
>>>>> trace.h: In function ‘__trace_ust_slavio_misc_update_irq_raise’:
>>>>> trace.h:277: error: ‘void’ must be the only parameter
>>>>> trace.h:277: error: expected expression before ‘)’ token
>>>>> trace.h:277: error: too many arguments to function ‘(void (*)(void
>>>>> *))__tp_it_func’
>>>>> trace.h: At top level:
>>>>> trace.h:277: error: ‘void’ must be the only parameter
>>>>> trace.h:277: error: ‘void’ must be the only parameter
>>>>> In file included from osdep.c:49:
>>>>> trace.h: In function ‘__trace_ust_slavio_misc_update_irq_lower’:
>>>>> trace.h:280: error: ‘void’ must be the only parameter
>>>>> trace.h:280: error: expected expression before ‘)’ token
>>>>> trace.h:280: error: too many arguments to function ‘(void (*)(void
>>>>> *))__tp_it_func’
>>>>>
>>>>>
>>>>> I am not sure which interface is supposed to be used for void
>>>>> arguments
>>>>> in ust 0.16.
>>>>
>>>> Looking at scripts/tracetool:
>>>>
>>>> linetoh_ust()
>>>> {
>>>> local name args argnames
>>>> name=$(get_name "$1")
>>>> args=$(get_args "$1")
>>>> argnames=$(get_argnames "$1", ",")
>>>>
>>>> cat<<EOF
>>>> DECLARE_TRACE(ust_$name, TP_PROTO($args), TP_ARGS($argnames));
>>>> #define trace_$name trace_ust_$name
>>>> EOF
>>>> }
>>>>
>>>> for those tracepoints with argument "void", DECLARE_TRACE_NOARGS should
>>>> be used for UST 0.16. Similar for:
>>>>
>>>> DEFINE_TRACE(ust_$name); -> DEFINE_TRACE_NOARGS(ust_$name);
>>>>
>>>
>>> I had tried with _NOARGS variants initially by manually changing the
>>> auto-generated code like this:
>>>
>>> In trace.h:
>>>
>>> DECLARE_TRACE_NOARGS(ust_slavio_misc_update_irq_raise);
>>> #define trace_slavio_misc_update_irq_raise
>>> trace_ust_slavio_misc_update_irq_raise
>>>
>>> In trace.c:
>>>
>>> DEFINE_TRACE_NOARGS(ust_slavio_misc_update_irq_raise);
>>>
>>> static void ust_slavio_misc_update_irq_raise_probe()
>>> {
>>> trace_mark(ust, slavio_misc_update_irq_raise);
>>> }
>>>
>>>
>>> However, it still gave error like this:
>>>
>>> [harsh@harshbora v9fs]$ make
>>> CC osdep.o
>>> cc1: warnings being treated as errors
>>> In file included from osdep.c:49:
>>> trace.h:277: error: data definition has no type or storage class
>>> trace.h:277: error: type defaults to ‘int’ in declaration of
>>> ‘DECLARE_TRACE_NOARGS’
>>> trace.h:277: error: parameter names (without types) in function
>>> declaration
>>>
>>>
>>> It will be great if you can provide a sample code (to be auto-generated)
>>> required for a trace-event with void argument.
>>
>> Ah, right, now I remember that I only left DEFINE_TRACE and DECLARE_TRACE
>> as backward compatibility in UST 0.x, and did only add the "_NOARGS" to
>> the backward compatibility macros with an added underscore prefix.
>>
>> If you really just want to have compatibility with UST 0.16, changing the
>> minimum of things compared with what you currently have, I would
>> recommend to use:
>>
>> in trace.h:
>> _DECLARE_TRACEPOINT_NOARGS(ust_slavio_misc_update_irq_raise)
>> #define trace_slavio_misc_update_irq_raise
>> trace_ust_slavio_misc_update_irq_raise
>>
>> in trace.c:
>> DEFINE_TRACE(ust_slavio_misc_update_irq_raise);
>>
>> static void ust_slavio_misc_update_irq_raise_probe(void)
>> {
>> trace_mark(ust, slavio_misc_update_irq_raise, UST_MARKER_NOARGS);
>> }
>>
>> Let me know how this works for you,
>
> Thanks for the help, I wrote the python code to auto-generate the same
> for events with void arg. However, after doing make, it says:
>
> trace.o:(__ust_markers+0x4a08): undefined reference to
> `__ust_marker_empty_function'
> trace.o:(__ust_markers+0x4a60): undefined reference to
> `ust_marker_probe_cb'
> trace.o:(__ust_markers+0x4a68): undefined reference to
> `__ust_marker_empty_function'
> trace.o:(__ust_markers+0x4ac0): undefined reference to
> `ust_marker_probe_cb'
>
> I have tested using both:
> 1) ust 0.15 with urcu 0.6.
> 2) ust 0.16 with urcu 0.6.6
>
> Any idea, why this error ?
Ok, I got it (Thanks Aneesh). It required this change in configure:
diff --git a/configure b/configure
index 0ea8634..40a18d8 100755
--- a/configure
+++ b/configure
@@ -2495,7 +2495,8 @@ if test "$trace_backend" = "ust"; then
int main(void) { return 0; }
EOF
if compile_prog "" "" ; then
- LIBS="-lust $LIBS"
+ LIBS="-lust -lurcu-bp $LIBS"
+ libs_qga+="-lust -lurcu-bp"
else
regards,
Harsh
>
> - Harsh
>
>>
>> Best regards,
>>
>> Mathieu
>>
>>>
>>> - Harsh
>>>
>>>>> Moreover, if ust 2.0 uses different interfaces, we might
>>>>> want to use the latest one.
>>>>
>>>> Note that this kind of special-case won't be needed with LTTng-UST 2.0
>>>> TRACEPOINT_EVENT. In place of DECLARE_TRACE, one would use:
>>>>
>>>> TRACEPOINT_EVENT(qemu_kvm, $name,
>>>> TP_ARGS($args),
>>>> TP_FIELDS()
>>>> )
>>>>
>>>> Note that I notice that some care will need to be taken to generate the
>>>> TP_FIELDS() from your existing trace-events file, an example:
>>>>
>>>> g_realloc(void *ptr, size_t size, void *newptr)
>>>>
>>>> would have to be translated to:
>>>>
>>>> TRACE_EVENT(qemu_kvm, g_realloc,
>>>> TP_ARGS(void *, ptr, size_t, size, void *, newptr),
>>>> TP_FIELDS(
>>>> ctf_integer_hex(void *, ptr, ptr)
>>>> ctf_integer(size_t, size, size)
>>>> ctf_integer_hex(void *, newptr, newptr)
>>>> )
>>>> )
>>>>
>>>> Note that the bright side is that the tracepoint probe does not need to
>>>> be hand-coded anymore, and there is no need to use the markers anymore
>>>> neither, which makes the tracer much faster.
>>>>
>>>> For most of your fields (using %p, %d style format strings), you should
>>>> use ctf_integer or ctf_integer_hex (the latter lets the trace viewer
>>>> know that the data should be printed as hexadecimal).
>>>> You will likely need to detect the %s format strings you have there and
>>>> translate them into ctf_string(field, field) too. You can have a
>>>> look at
>>>> lttng-ust tests/hello/*.[ch] for examples.
>>>
>>> Thanks for the info, I will look into it later.
>>>>
>>>> The call which would have looked like trace_qemu_kvm_g_realloc() in UST
>>>> 0.x should now be done with:
>>>>
>>>> tracepoint(qemu_kvm, g_realloc, ptr, size, newptr);
>>>>
>>>> This is needed to (very soon) add support for sdt.h in LTTng-UST
>>>> 2.0, so
>>>> systemtap and gdb can hook into tracepoints declared by lttng-ust 2.0.
>>>>
>>>> Best regards,
>>>>
>>>> Mathieu
>>>>
>>>>>
>>>>> regards,
>>>>> Harsh
>>>>>
>>>>>> Best regards,
>>>>>>
>>>>>> Mathieu
>>>>>>
>>>>>>>
>>>>>>> Version History:
>>>>>>>
>>>>>>> v2:
>>>>>>> - Updated tracetool.py to support nop, stderr, dtrace backend
>>>>>>>
>>>>>>> v1:
>>>>>>> - Working protoype with tracetool.py converted only for
>>>>>>> simpletrace backend
>>>>>>>
>>>>>>> Harsh Prateek Bora (4):
>>>>>>> Converting tracetool.sh to tracetool.py
>>>>>>> Makefile and configure changes for tracetool.py
>>>>>>> simpletrace-v2: Handle variable number/size of elements per trace
>>>>>>> record.
>>>>>>> simpletrace.py: updated log reader script to handle new log format
>>>>>>>
>>>>>>> Makefile.objs | 6 +-
>>>>>>> Makefile.target | 10 +-
>>>>>>> configure | 4 +-
>>>>>>> monitor.c | 2 +-
>>>>>>> scripts/simpletrace.py | 110 ++++++++++-
>>>>>>> scripts/tracetool.py | 505
>>>>>>> ++++++++++++++++++++++++++++++++++++++++++++++++
>>>>>>> trace/simple.c | 178 ++++++-----------
>>>>>>> trace/simple.h | 31 +++-
>>>>>>> 8 files changed, 702 insertions(+), 144 deletions(-)
>>>>>>> create mode 100755 scripts/tracetool.py
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>>
>>> _______________________________________________
>>> lttng-dev mailing list
>>> lttng-dev@lists.lttng.org
>>> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>>
>
^ permalink raw reply related [flat|nested] 18+ messages in thread* Re: [Qemu-devel] [lttng-dev] [RFC PATCH v2 0/4] simpletrace : support var num of args and strings.
2012-01-10 10:44 ` Harsh Bora
@ 2012-01-10 14:18 ` Mathieu Desnoyers
0 siblings, 0 replies; 18+ messages in thread
From: Mathieu Desnoyers @ 2012-01-10 14:18 UTC (permalink / raw)
To: Harsh Bora
Cc: stefanha, qemu-devel, lttng-dev, Mathieu Desnoyers, aneesh.kumar,
vilanova
* Harsh Bora (harsh@linux.vnet.ibm.com) wrote:
> On 01/10/2012 02:36 PM, Harsh Bora wrote:
>> On 01/10/2012 12:47 PM, Mathieu Desnoyers wrote:
>>> * Harsh Bora (harsh@linux.vnet.ibm.com) wrote:
>>>> On 01/10/2012 05:44 AM, Mathieu Desnoyers wrote:
>>>>> * Harsh Bora (harsh@linux.vnet.ibm.com) wrote:
>>>>>> On 01/09/2012 09:31 PM, Mathieu Desnoyers wrote:
>>>>>>> * Harsh Prateek Bora (harsh@linux.vnet.ibm.com) wrote:
>>>>>>>> Existing simple trace can log upto 6 args per trace event and
>>>>>>>> does not
>>>>>>>> support strings in trace record format. Introducing new trace
>>>>>>>> format as
>>>>>>>> discussed earlier on list to support variable number/size of
>>>>>>>> arguments.
>>>>>>>> (Ref:
>>>>>>>> http://lists.gnu.org/archive/html/qemu-devel/2011-11/msg03426.html)
>>>>>>>>
>>>>>>>> Basic testing of this patch is successful. Stress testing not yet
>>>>>>>> done.
>>>>>>>>
>>>>>>>> Apply patches, then run:
>>>>>>>>
>>>>>>>> make distclean
>>>>>>>> ./configure with --enable-trace-backend=simple
>>>>>>>> make
>>>>>>>> sudo make install
>>>>>>>>
>>>>>>>> Sample tracelog showing strings support:
>>>>>>>> [harsh@harshbora v9fs]$ scripts/simpletrace.py trace-events
>>>>>>>> trace-23261
>>>>>>>> v9fs_version 0.000 tag=65535 id=100 msize=8192 version=9P2000.L
>>>>>>>> v9fs_version_return 17.530 tag=65535 id=100 msize=8192
>>>>>>>> version=9P2000.L
>>>>>>>> v9fs_attach 180.121 tag=1 id=104 fid=0 afid=18446744073709551615
>>>>>>>> uname=nobody aname=
>>>>>>>>
>>>>>>>>
>>>>>>>> Note: LTTng ust backend is broken in upstream qemu, therefore
>>>>>>>> tracetool.py
>>>>>>>> doesnt support ust backend as of now. IIUC, ust's trace event
>>>>>>>> APIs are under
>>>>>>>> development and not yet stable.
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> FYI, the LTTng-UST TRACEPOINT_EVENT API is very much stable as of
>>>>>>> now.
>>>>>>> Even though we are still in LTTng-UST 2.0 prereleases, the fact
>>>>>>> that we
>>>>>>> started the round of discussions on this API last summer makes us
>>>>>>> confident that from this point on we should not have to change it.
>>>>>>>
>>>>>>> Moreover, I would like to know if the old UST 0.x (0.16 is the
>>>>>>> latest)
>>>>>>> is broken wrt qemu, or if this is just for LTTng-2.0 UST support ?
>>>>>>> UST 0.x instrumentation is not supposed to have broken wrt qemu.
>>>>>>>
>>>>>>
>>>>>> Hi,
>>>>>> Thanks for an early response. I had tried building with ust 0.16
>>>>>> and it
>>>>>> gives compilation errors, specially for trace events with 'void'
>>>>>> argument:
>>>>>>
>>>>>> CC osdep.o
>>>>>> In file included from osdep.c:49:
>>>>>> trace.h: In function ‘__trace_ust_slavio_misc_update_irq_raise’:
>>>>>> trace.h:277: error: ‘void’ must be the only parameter
>>>>>> trace.h:277: error: expected expression before ‘)’ token
>>>>>> trace.h:277: error: too many arguments to function ‘(void (*)(void
>>>>>> *))__tp_it_func’
>>>>>> trace.h: At top level:
>>>>>> trace.h:277: error: ‘void’ must be the only parameter
>>>>>> trace.h:277: error: ‘void’ must be the only parameter
>>>>>> In file included from osdep.c:49:
>>>>>> trace.h: In function ‘__trace_ust_slavio_misc_update_irq_lower’:
>>>>>> trace.h:280: error: ‘void’ must be the only parameter
>>>>>> trace.h:280: error: expected expression before ‘)’ token
>>>>>> trace.h:280: error: too many arguments to function ‘(void (*)(void
>>>>>> *))__tp_it_func’
>>>>>>
>>>>>>
>>>>>> I am not sure which interface is supposed to be used for void
>>>>>> arguments
>>>>>> in ust 0.16.
>>>>>
>>>>> Looking at scripts/tracetool:
>>>>>
>>>>> linetoh_ust()
>>>>> {
>>>>> local name args argnames
>>>>> name=$(get_name "$1")
>>>>> args=$(get_args "$1")
>>>>> argnames=$(get_argnames "$1", ",")
>>>>>
>>>>> cat<<EOF
>>>>> DECLARE_TRACE(ust_$name, TP_PROTO($args), TP_ARGS($argnames));
>>>>> #define trace_$name trace_ust_$name
>>>>> EOF
>>>>> }
>>>>>
>>>>> for those tracepoints with argument "void", DECLARE_TRACE_NOARGS should
>>>>> be used for UST 0.16. Similar for:
>>>>>
>>>>> DEFINE_TRACE(ust_$name); -> DEFINE_TRACE_NOARGS(ust_$name);
>>>>>
>>>>
>>>> I had tried with _NOARGS variants initially by manually changing the
>>>> auto-generated code like this:
>>>>
>>>> In trace.h:
>>>>
>>>> DECLARE_TRACE_NOARGS(ust_slavio_misc_update_irq_raise);
>>>> #define trace_slavio_misc_update_irq_raise
>>>> trace_ust_slavio_misc_update_irq_raise
>>>>
>>>> In trace.c:
>>>>
>>>> DEFINE_TRACE_NOARGS(ust_slavio_misc_update_irq_raise);
>>>>
>>>> static void ust_slavio_misc_update_irq_raise_probe()
>>>> {
>>>> trace_mark(ust, slavio_misc_update_irq_raise);
>>>> }
>>>>
>>>>
>>>> However, it still gave error like this:
>>>>
>>>> [harsh@harshbora v9fs]$ make
>>>> CC osdep.o
>>>> cc1: warnings being treated as errors
>>>> In file included from osdep.c:49:
>>>> trace.h:277: error: data definition has no type or storage class
>>>> trace.h:277: error: type defaults to ‘int’ in declaration of
>>>> ‘DECLARE_TRACE_NOARGS’
>>>> trace.h:277: error: parameter names (without types) in function
>>>> declaration
>>>>
>>>>
>>>> It will be great if you can provide a sample code (to be auto-generated)
>>>> required for a trace-event with void argument.
>>>
>>> Ah, right, now I remember that I only left DEFINE_TRACE and DECLARE_TRACE
>>> as backward compatibility in UST 0.x, and did only add the "_NOARGS" to
>>> the backward compatibility macros with an added underscore prefix.
>>>
>>> If you really just want to have compatibility with UST 0.16, changing the
>>> minimum of things compared with what you currently have, I would
>>> recommend to use:
>>>
>>> in trace.h:
>>> _DECLARE_TRACEPOINT_NOARGS(ust_slavio_misc_update_irq_raise)
>>> #define trace_slavio_misc_update_irq_raise
>>> trace_ust_slavio_misc_update_irq_raise
>>>
>>> in trace.c:
>>> DEFINE_TRACE(ust_slavio_misc_update_irq_raise);
>>>
>>> static void ust_slavio_misc_update_irq_raise_probe(void)
>>> {
>>> trace_mark(ust, slavio_misc_update_irq_raise, UST_MARKER_NOARGS);
>>> }
>>>
>>> Let me know how this works for you,
>>
>> Thanks for the help, I wrote the python code to auto-generate the same
>> for events with void arg. However, after doing make, it says:
>>
>> trace.o:(__ust_markers+0x4a08): undefined reference to
>> `__ust_marker_empty_function'
>> trace.o:(__ust_markers+0x4a60): undefined reference to
>> `ust_marker_probe_cb'
>> trace.o:(__ust_markers+0x4a68): undefined reference to
>> `__ust_marker_empty_function'
>> trace.o:(__ust_markers+0x4ac0): undefined reference to
>> `ust_marker_probe_cb'
>>
>> I have tested using both:
>> 1) ust 0.15 with urcu 0.6.
>> 2) ust 0.16 with urcu 0.6.6
>>
>> Any idea, why this error ?
>
> Ok, I got it (Thanks Aneesh). It required this change in configure:
>
> diff --git a/configure b/configure
> index 0ea8634..40a18d8 100755
> --- a/configure
> +++ b/configure
> @@ -2495,7 +2495,8 @@ if test "$trace_backend" = "ust"; then
> int main(void) { return 0; }
> EOF
> if compile_prog "" "" ; then
> - LIBS="-lust $LIBS"
> + LIBS="-lust -lurcu-bp $LIBS"
> + libs_qga+="-lust -lurcu-bp"
> else
I'm glad it works for you for 0.x. Please note that lttng-ust 2.0 will
not require that many tweaks: I've made sure the instrumentation is more
straightforward.
Thanks,
Mathieu
>
> regards,
> Harsh
>
>>
>> - Harsh
>>
>>>
>>> Best regards,
>>>
>>> Mathieu
>>>
>>>>
>>>> - Harsh
>>>>
>>>>>> Moreover, if ust 2.0 uses different interfaces, we might
>>>>>> want to use the latest one.
>>>>>
>>>>> Note that this kind of special-case won't be needed with LTTng-UST 2.0
>>>>> TRACEPOINT_EVENT. In place of DECLARE_TRACE, one would use:
>>>>>
>>>>> TRACEPOINT_EVENT(qemu_kvm, $name,
>>>>> TP_ARGS($args),
>>>>> TP_FIELDS()
>>>>> )
>>>>>
>>>>> Note that I notice that some care will need to be taken to generate the
>>>>> TP_FIELDS() from your existing trace-events file, an example:
>>>>>
>>>>> g_realloc(void *ptr, size_t size, void *newptr)
>>>>>
>>>>> would have to be translated to:
>>>>>
>>>>> TRACE_EVENT(qemu_kvm, g_realloc,
>>>>> TP_ARGS(void *, ptr, size_t, size, void *, newptr),
>>>>> TP_FIELDS(
>>>>> ctf_integer_hex(void *, ptr, ptr)
>>>>> ctf_integer(size_t, size, size)
>>>>> ctf_integer_hex(void *, newptr, newptr)
>>>>> )
>>>>> )
>>>>>
>>>>> Note that the bright side is that the tracepoint probe does not need to
>>>>> be hand-coded anymore, and there is no need to use the markers anymore
>>>>> neither, which makes the tracer much faster.
>>>>>
>>>>> For most of your fields (using %p, %d style format strings), you should
>>>>> use ctf_integer or ctf_integer_hex (the latter lets the trace viewer
>>>>> know that the data should be printed as hexadecimal).
>>>>> You will likely need to detect the %s format strings you have there and
>>>>> translate them into ctf_string(field, field) too. You can have a
>>>>> look at
>>>>> lttng-ust tests/hello/*.[ch] for examples.
>>>>
>>>> Thanks for the info, I will look into it later.
>>>>>
>>>>> The call which would have looked like trace_qemu_kvm_g_realloc() in UST
>>>>> 0.x should now be done with:
>>>>>
>>>>> tracepoint(qemu_kvm, g_realloc, ptr, size, newptr);
>>>>>
>>>>> This is needed to (very soon) add support for sdt.h in LTTng-UST
>>>>> 2.0, so
>>>>> systemtap and gdb can hook into tracepoints declared by lttng-ust 2.0.
>>>>>
>>>>> Best regards,
>>>>>
>>>>> Mathieu
>>>>>
>>>>>>
>>>>>> regards,
>>>>>> Harsh
>>>>>>
>>>>>>> Best regards,
>>>>>>>
>>>>>>> Mathieu
>>>>>>>
>>>>>>>>
>>>>>>>> Version History:
>>>>>>>>
>>>>>>>> v2:
>>>>>>>> - Updated tracetool.py to support nop, stderr, dtrace backend
>>>>>>>>
>>>>>>>> v1:
>>>>>>>> - Working protoype with tracetool.py converted only for
>>>>>>>> simpletrace backend
>>>>>>>>
>>>>>>>> Harsh Prateek Bora (4):
>>>>>>>> Converting tracetool.sh to tracetool.py
>>>>>>>> Makefile and configure changes for tracetool.py
>>>>>>>> simpletrace-v2: Handle variable number/size of elements per trace
>>>>>>>> record.
>>>>>>>> simpletrace.py: updated log reader script to handle new log format
>>>>>>>>
>>>>>>>> Makefile.objs | 6 +-
>>>>>>>> Makefile.target | 10 +-
>>>>>>>> configure | 4 +-
>>>>>>>> monitor.c | 2 +-
>>>>>>>> scripts/simpletrace.py | 110 ++++++++++-
>>>>>>>> scripts/tracetool.py | 505
>>>>>>>> ++++++++++++++++++++++++++++++++++++++++++++++++
>>>>>>>> trace/simple.c | 178 ++++++-----------
>>>>>>>> trace/simple.h | 31 +++-
>>>>>>>> 8 files changed, 702 insertions(+), 144 deletions(-)
>>>>>>>> create mode 100755 scripts/tracetool.py
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> lttng-dev mailing list
>>>> lttng-dev@lists.lttng.org
>>>> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>>>
>>
>
--
Mathieu Desnoyers
Operating System Efficiency R&D Consultant
EfficiOS Inc.
http://www.efficios.com
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Qemu-devel] [lttng-dev] [RFC PATCH v2 0/4] simpletrace : support var num of args and strings.
2012-01-10 0:14 ` Mathieu Desnoyers
2012-01-10 6:54 ` Harsh Bora
@ 2012-01-10 14:58 ` Stefan Hajnoczi
2012-01-10 17:29 ` Mathieu Desnoyers
1 sibling, 1 reply; 18+ messages in thread
From: Stefan Hajnoczi @ 2012-01-10 14:58 UTC (permalink / raw)
To: Mathieu Desnoyers
Cc: stefanha, Harsh Bora, qemu-devel, lttng-dev, aneesh.kumar,
vilanova
On Tue, Jan 10, 2012 at 12:14 AM, Mathieu Desnoyers
<mathieu.desnoyers@efficios.com> wrote:
> This is needed to (very soon) add support for sdt.h in LTTng-UST 2.0, so
> systemtap and gdb can hook into tracepoints declared by lttng-ust 2.0.
If lttng supports sdt.h in the future will it also provide a dtrace(1)
wrapper? I'm wondering if we can boil it down to the common DTrace
code that we already use in QEMU for SystemTap and that the
SmartOS/Illumos folks have used with DTrace.
Stefan
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Qemu-devel] [lttng-dev] [RFC PATCH v2 0/4] simpletrace : support var num of args and strings.
2012-01-10 14:58 ` Stefan Hajnoczi
@ 2012-01-10 17:29 ` Mathieu Desnoyers
2012-01-11 9:30 ` Stefan Hajnoczi
0 siblings, 1 reply; 18+ messages in thread
From: Mathieu Desnoyers @ 2012-01-10 17:29 UTC (permalink / raw)
To: Stefan Hajnoczi
Cc: stefanha, Harsh Bora, qemu-devel, lttng-dev, aneesh.kumar,
vilanova
Hi Stefan,
* Stefan Hajnoczi (stefanha@gmail.com) wrote:
> On Tue, Jan 10, 2012 at 12:14 AM, Mathieu Desnoyers
> <mathieu.desnoyers@efficios.com> wrote:
> > This is needed to (very soon) add support for sdt.h in LTTng-UST 2.0, so
> > systemtap and gdb can hook into tracepoints declared by lttng-ust 2.0.
>
> If lttng supports sdt.h in the future will it also provide a dtrace(1)
> wrapper? I'm wondering if we can boil it down to the common DTrace
> code that we already use in QEMU for SystemTap and that the
> SmartOS/Illumos folks have used with DTrace.
What we plan to provide is the other way around: because we want to keep
the TRACEPOINT_EVENT description, which specify the mapping between the
arguments passed to the tracepoint() and what is to be serialized into
the trace, as part of the application code, we require that the
instrumentation be specified in the form of TRACEPOINT_EVENT and called
with tracepoint(). The bonus here is that the tracepoint() macro can
embed a STAP_PROBEV() call, which should be compatible with systemtap
and gdb.
Another point worth noting is that sdt provider uses a breakpoint to
call the probes from the application instrumentation sites, which brings
an added overhead compared to tracepoints when it is activated. This is
another reason for staying with tracepoints and not use SDT as is within
lttng-ust.
One more point is that tracepoints put all side-effects of the
parameters passed to the tracepoint() macro inside the "instrumentation
enabled" code path (pointer dereferenced, offsets computation, functions
called...). Therefore, when tracepoints are disabled, they just cost a
branch (which we plan to optimize to a no-op, as permitted by gcc asm
goto).
In theory, it might not be impossible to create a translator from a
subset of the dtrace language to TRACEPOINT_EVENT declarations. It is
just not on our roadmap at this point, and I'm not sure it's worth the
effort, when a simple script could translate from qemu-kvm trace-event
file to TRACEPOINT_EVENT declarations.
Thoughts ?
Best regards,
Mathieu
--
Mathieu Desnoyers
Operating System Efficiency R&D Consultant
EfficiOS Inc.
http://www.efficios.com
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Qemu-devel] [lttng-dev] [RFC PATCH v2 0/4] simpletrace : support var num of args and strings.
2012-01-10 17:29 ` Mathieu Desnoyers
@ 2012-01-11 9:30 ` Stefan Hajnoczi
0 siblings, 0 replies; 18+ messages in thread
From: Stefan Hajnoczi @ 2012-01-11 9:30 UTC (permalink / raw)
To: Mathieu Desnoyers
Cc: stefanha, Harsh Bora, qemu-devel, lttng-dev, aneesh.kumar,
vilanova
On Tue, Jan 10, 2012 at 5:29 PM, Mathieu Desnoyers
<mathieu.desnoyers@efficios.com> wrote:
> Hi Stefan,
>
> * Stefan Hajnoczi (stefanha@gmail.com) wrote:
>> On Tue, Jan 10, 2012 at 12:14 AM, Mathieu Desnoyers
>> <mathieu.desnoyers@efficios.com> wrote:
>> > This is needed to (very soon) add support for sdt.h in LTTng-UST 2.0, so
>> > systemtap and gdb can hook into tracepoints declared by lttng-ust 2.0.
>>
>> If lttng supports sdt.h in the future will it also provide a dtrace(1)
>> wrapper? I'm wondering if we can boil it down to the common DTrace
>> code that we already use in QEMU for SystemTap and that the
>> SmartOS/Illumos folks have used with DTrace.
>
> What we plan to provide is the other way around: because we want to keep
> the TRACEPOINT_EVENT description, which specify the mapping between the
> arguments passed to the tracepoint() and what is to be serialized into
> the trace, as part of the application code, we require that the
> instrumentation be specified in the form of TRACEPOINT_EVENT and called
> with tracepoint(). The bonus here is that the tracepoint() macro can
> embed a STAP_PROBEV() call, which should be compatible with systemtap
> and gdb.
I see, thanks for explaining.
> One more point is that tracepoints put all side-effects of the
> parameters passed to the tracepoint() macro inside the "instrumentation
> enabled" code path (pointer dereferenced, offsets computation, functions
> called...). Therefore, when tracepoints are disabled, they just cost a
> branch (which we plan to optimize to a no-op, as permitted by gcc asm
> goto).
SystemTap has a solution for this problem where you wrap the
tracepoint with a guard:
if (QEMU_FOO_ENABLED()) {
/* ...expensive stuff here... */
QEMU_FOO(a, b, c);
}
In QEMU we don't really make use of this but I've heard applications
with expensive tracepoint arguments use this.
> In theory, it might not be impossible to create a translator from a
> subset of the dtrace language to TRACEPOINT_EVENT declarations. It is
> just not on our roadmap at this point, and I'm not sure it's worth the
> effort, when a simple script could translate from qemu-kvm trace-event
> file to TRACEPOINT_EVENT declarations.
Okay, QEMU can continue to emit LTTng UST tracepoints. I thought you
were planning to make UST source-compatible with DTrace sdt.h, which
would be convenient for applications that are already DTrace
instrumented but want a fast shared-memory mechanism instead of
breakpoints - they could use ust-dtrace(1) instead of dtrace(1) and
magically work with UST. In QEMU's case it doesn't matter because
we're already geared towards multiple tracing backends.
Stefan
^ permalink raw reply [flat|nested] 18+ messages in thread