[Qemu-devel] [PATCH 0/2] Add support for SystemTAP and DTrace tracing backends (v5)

[Qemu-devel] [PATCH 0/2] Add support for SystemTAP and DTrace tracing backends (v5)

[Daniel P. Berrange]

A repost of the SystemTAP/DTrace patches from

  http://lists.gnu.org/archive/html/qemu-devel/2010-11/msg00496.html

The patch is now split into two pieces. 

The first patch contains the generic DTrace tracing backend support.

The second patch contains additional pieces for SystemTAP to generate
a tapset for each of the qemu-system-XXX binaries, to simplify life
for admins wanting to use the tracing backend. This addresses the 
problem in previous versions of the patch where the tapset only worked 
for /usr/bin/qemu and no other binaries. Unfortunately SystemTAP does 
not allow for use of wildcards, so it was neccessary to go for a separate 
tapset file per emulator binary.

Changes in v5:

 - The Makefile merge error is now fixed

[Qemu-devel] [PATCH 1/2] Add a DTrace tracing backend targetted for SystemTAP compatability

[Daniel P. Berrange]

This introduces a new tracing backend that targets the SystemTAP
implementation of DTrace userspace tracing. The core functionality
should be applicable and standard across any DTrace implementation
on Solaris, OS-X, *BSD, but the Makefile rules will likely need
some small additional changes to cope with OS specific build
requirements.

This backend builds a little differently from the other tracing
backends. Specifically there is no 'trace.c' file, because the
'dtrace' command line tool generates a '.o' file directly from
the dtrace probe definition file. The probe definition is usually
named with a '.d' extension but QEMU uses '.d' files for its
external makefile dependancy tracking, so this uses '.dtrace' as
the extension for the probe definition file.

The 'tracetool' program gains the ability to generate a trace.h
file for DTrace, and also to generate the trace.d file containing
the dtrace probe definition.

Example usage of a dtrace probe in systemtap looks like:

  probe process("qemu").mark("qemu_malloc") {
    printf("Malloc %d %p\n", $arg1, $arg2);
  }

* .gitignore: Ignore trace-dtrace.*
* Makefile: Extra rules for generating DTrace files
* Makefile.obj: Don't build trace.o for DTrace, use
  trace-dtrace.o generated by 'dtrace' instead
* tracetool: Support for generating DTrace data files

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
---
 .gitignore    |    2 +
 Makefile      |   23 +++++++++++
 Makefile.objs |    4 ++
 configure     |   14 ++++++-
 tracetool     |  116 ++++++++++++++++++++++++++++++++++++++++++++++++++++-----
 5 files changed, 148 insertions(+), 11 deletions(-)

diff --git a/.gitignore b/.gitignore
index a43e4d1..3efb4ec 100644
--- a/.gitignore
+++ b/.gitignore
@@ -4,6 +4,8 @@ config-host.*
 config-target.*
 trace.h
 trace.c
+trace-dtrace.h
+trace-dtrace.dtrace
 *-timestamp
 *-softmmu
 *-darwin-user
diff --git a/Makefile b/Makefile
index 02698e9..554ad97 100644
--- a/Makefile
+++ b/Makefile
@@ -1,6 +1,9 @@
 # Makefile for QEMU.
 
 GENERATED_HEADERS = config-host.h trace.h qemu-options.def
+ifeq ($(TRACE_BACKEND),dtrace)
+GENERATED_HEADERS += trace-dtrace.h
+endif
 
 ifneq ($(wildcard config-host.mak),)
 # Put the all: rule here so that config-host.mak can contain dependencies.
@@ -108,7 +111,11 @@ ui/vnc.o: QEMU_CFLAGS += $(VNC_TLS_CFLAGS)
 
 bt-host.o: QEMU_CFLAGS += $(BLUEZ_CFLAGS)
 
+ifeq ($(TRACE_BACKEND),dtrace)
+trace.h: trace.h-timestamp trace-dtrace.h
+else
 trace.h: trace.h-timestamp
+endif
 trace.h-timestamp: $(SRC_PATH)/trace-events config-host.mak
 	$(call quiet-command,sh $(SRC_PATH)/tracetool --$(TRACE_BACKEND) -h < $< > $@,"  GEN   trace.h")
 	@cmp -s $@ trace.h || cp $@ trace.h
@@ -120,6 +127,20 @@ trace.c-timestamp: $(SRC_PATH)/trace-events config-host.mak
 
 trace.o: trace.c $(GENERATED_HEADERS)
 
+trace-dtrace.h: trace-dtrace.dtrace
+	$(call quiet-command,dtrace -o $@ -h -s $<, "  GEN   trace-dtrace.h")
+
+# Normal practice is to name DTrace probe file with a '.d' extension
+# but that gets picked up by QEMU's Makefile as an external dependancy
+# rule file. So we use '.dtrace' instead
+trace-dtrace.dtrace: trace-dtrace.dtrace-timestamp
+trace-dtrace.dtrace-timestamp: $(SRC_PATH)/trace-events config-host.mak
+	$(call quiet-command,sh $(SRC_PATH)/tracetool --$(TRACE_BACKEND) -d < $< > $@,"  GEN   trace-dtrace.dtrace")
+	@cmp -s $@ trace-dtrace.dtrace || cp $@ trace-dtrace.dtrace
+
+trace-dtrace.o: trace-dtrace.dtrace $(GENERATED_HEADERS)
+	$(call quiet-command,dtrace -o $@ -G -s $<, "  GEN trace-dtrace.o")
+
 simpletrace.o: simpletrace.c $(GENERATED_HEADERS)
 
 version.o: $(SRC_PATH)/version.rc config-host.mak
@@ -157,6 +178,8 @@ clean:
 	rm -f slirp/*.o slirp/*.d audio/*.o audio/*.d block/*.o block/*.d net/*.o net/*.d fsdev/*.o fsdev/*.d ui/*.o ui/*.d
 	rm -f qemu-img-cmds.h
 	rm -f trace.c trace.h trace.c-timestamp trace.h-timestamp
+	rm -f trace-dtrace.dtrace trace-dtrace.dtrace-timestamp
+	rm -f trace-dtrace.h trace-dtrace.h-timestamp
 	$(MAKE) -C tests clean
 	for d in $(ALL_SUBDIRS) libhw32 libhw64 libuser libdis libdis-user; do \
 	if test -d $$d; then $(MAKE) -C $$d $@ || exit 1; fi; \
diff --git a/Makefile.objs b/Makefile.objs
index faf485e..84fc80e 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -285,11 +285,15 @@ libdis-$(CONFIG_SPARC_DIS) += sparc-dis.o
 ######################################################################
 # trace
 
+ifeq ($(TRACE_BACKEND),dtrace)
+trace-obj-y = trace-dtrace.o
+else
 trace-obj-y = trace.o
 ifeq ($(TRACE_BACKEND),simple)
 trace-obj-y += simpletrace.o
 user-obj-y += qemu-timer-common.o
 endif
+endif
 
 vl.o: QEMU_CFLAGS+=$(GPROF_CFLAGS)
 
diff --git a/configure b/configure
index 7025d2b..f8dad3e 100755
--- a/configure
+++ b/configure
@@ -929,7 +929,7 @@ echo "  --enable-docs            enable documentation build"
 echo "  --disable-docs           disable documentation build"
 echo "  --disable-vhost-net      disable vhost-net acceleration support"
 echo "  --enable-vhost-net       enable vhost-net acceleration support"
-echo "  --trace-backend=B        Trace backend nop simple ust"
+echo "  --trace-backend=B        Trace backend nop simple ust dtrace"
 echo "  --trace-file=NAME        Full PATH,NAME of file to store traces"
 echo "                           Default:trace-<pid>"
 echo "  --disable-spice          disable spice"
@@ -2193,6 +2193,18 @@ EOF
     exit 1
   fi
 fi
+
+##########################################
+# For 'dtrace' backend, test if 'dtrace' command is present
+if test "$trace_backend" = "dtrace"; then
+  if ! has 'dtrace' ; then
+    echo
+    echo "Error: dtrace command is not found in PATH $PATH"
+    echo
+    exit 1
+  fi
+fi
+
 ##########################################
 # End of CC checks
 # After here, no more $cc or $ld runs
diff --git a/tracetool b/tracetool
index 7010858..5b6636a 100755
--- a/tracetool
+++ b/tracetool
@@ -20,10 +20,12 @@ Backends:
   --nop     Tracing disabled
   --simple  Simple built-in backend
   --ust     LTTng User Space Tracing backend
+  --dtrace  DTrace/SystemTAP backend
 
 Output formats:
   -h    Generate .h file
   -c    Generate .c file
+  -d    Generate .d file (DTrace only)
 EOF
     exit 1
 }
@@ -46,8 +48,9 @@ get_args()
 # Get the argument name list of a trace event
 get_argnames()
 {
-    local nfields field name
+    local nfields field name sep
     nfields=0
+    sep="$2"
     for field in $(get_args "$1"); do
         nfields=$((nfields + 1))
 
@@ -58,7 +61,7 @@ get_argnames()
         name=${field%,}
         test "$field" = "$name" && continue
 
-        printf "%s" "$name, "
+        printf "%s%s " $name $sep
     done
 
     # Last argument name
@@ -73,7 +76,7 @@ get_argc()
 {
     local name argc
     argc=0
-    for name in $(get_argnames "$1"); do
+    for name in $(get_argnames "$1", ","); do
         argc=$((argc + 1))
     done
     echo $argc
@@ -154,7 +157,7 @@ EOF
 cast_args_to_uint64_t()
 {
     local arg
-    for arg in $(get_argnames "$1"); do
+    for arg in $(get_argnames "$1", ","); do
         printf "%s" "(uint64_t)(uintptr_t)$arg"
     done
 }
@@ -247,7 +250,7 @@ linetoh_ust()
     local name args argnames
     name=$(get_name "$1")
     args=$(get_args "$1")
-    argnames=$(get_argnames "$1")
+    argnames=$(get_argnames "$1", ",")
 
     cat <<EOF
 DECLARE_TRACE(ust_$name, TP_PROTO($args), TP_ARGS($argnames));
@@ -274,7 +277,7 @@ linetoc_ust()
     local name args argnames fmt
     name=$(get_name "$1")
     args=$(get_args "$1")
-    argnames=$(get_argnames "$1")
+    argnames=$(get_argnames "$1", ",")
     fmt=$(get_fmt "$1")
 
     cat <<EOF
@@ -306,6 +309,87 @@ EOF
     echo "}"
 }
 
+linetoh_begin_dtrace()
+{
+    cat <<EOF
+#include "trace-dtrace.h"
+EOF
+}
+
+linetoh_dtrace()
+{
+    local name args argnames state nameupper
+    name=$(get_name "$1")
+    args=$(get_args "$1")
+    argnames=$(get_argnames "$1", ",")
+    state=$(get_state "$1")
+    if [ "$state" = "0" ] ; then
+        name=${name##disable }
+    fi
+
+    nameupper=`echo $name | tr '[:lower:]' '[:upper:]'`
+
+    # Define an empty function for the trace event
+    cat <<EOF
+static inline void trace_$name($args) {
+    if (QEMU_${nameupper}_ENABLED()) {
+        QEMU_${nameupper}($argnames);
+    }
+}
+EOF
+}
+
+linetoh_end_dtrace()
+{
+    return
+}
+
+linetoc_begin_dtrace()
+{
+    return
+}
+
+linetoc_dtrace()
+{
+    # No need for function definitions in dtrace backend
+    return
+}
+
+linetoc_end_dtrace()
+{
+    return
+}
+
+linetod_begin_dtrace()
+{
+    cat <<EOF
+provider qemu {
+EOF
+}
+
+linetod_dtrace()
+{
+    local name args state
+    name=$(get_name "$1")
+    args=$(get_args "$1")
+    state=$(get_state "$1")
+    if [ "$state" = "0" ] ; then
+        name=${name##disable }
+    fi
+
+    # Define prototype for probe arguments
+    cat <<EOF
+        probe $name($args);
+EOF
+}
+
+linetod_end_dtrace()
+{
+    cat <<EOF
+};
+EOF
+}
+
 # Process stdin by calling begin, line, and end functions for the backend
 convert()
 {
@@ -324,9 +408,10 @@ convert()
         disable=${str%%disable *}
         echo
         if test -z "$disable"; then
-            # Pass the disabled state as an arg to lineto$1_simple().
-            # For all other cases, call lineto$1_nop()
-            if [ $backend = "simple" ]; then
+            # Pass the disabled state as an arg for the simple
+            # or DTrace backends which handle it dynamically.
+            # For all other backends, call lineto$1_nop()
+            if [ $backend = "simple" -o "$backend" = "dtrace" ]; then
                 "$process_line" "$str"
             else
                 "lineto$1_nop" "${str##disable }"
@@ -360,9 +445,19 @@ tracetoc()
     convert c
 }
 
+tracetod()
+{
+    if [ $backend != "dtrace" ]; then
+       echo "DTrace probe generator not applicable to $backend backend"
+       exit 1
+    fi
+    echo "/* This file is autogenerated by tracetool, do not edit. */"
+    convert d
+}
+
 # Choose backend
 case "$1" in
-"--nop" | "--simple" | "--ust") backend="${1#--}" ;;
+"--nop" | "--simple" | "--ust" | "--dtrace") backend="${1#--}" ;;
 *) usage ;;
 esac
 shift
@@ -370,6 +465,7 @@ shift
 case "$1" in
 "-h") tracetoh ;;
 "-c") tracetoc ;;
+"-d") tracetod ;;
 "--check-backend") exit 0 ;; # used by ./configure to test for backend
 *) usage ;;
 esac
-- 
1.7.2.3

Re: [Qemu-devel] [PATCH 1/2] Add a DTrace tracing backend targetted for SystemTAP compatability

[Anthony Liguori]

On 11/08/2010 01:33 PM, Daniel P. Berrange wrote:
> This introduces a new tracing backend that targets the SystemTAP
> implementation of DTrace userspace tracing. The core functionality
> should be applicable and standard across any DTrace implementation
> on Solaris, OS-X, *BSD, but the Makefile rules will likely need
> some small additional changes to cope with OS specific build
> requirements.
>
> This backend builds a little differently from the other tracing
> backends. Specifically there is no 'trace.c' file, because the
> 'dtrace' command line tool generates a '.o' file directly from
> the dtrace probe definition file. The probe definition is usually
> named with a '.d' extension but QEMU uses '.d' files for its
> external makefile dependancy tracking, so this uses '.dtrace' as
> the extension for the probe definition file.
>
> The 'tracetool' program gains the ability to generate a trace.h
> file for DTrace, and also to generate the trace.d file containing
> the dtrace probe definition.
>
> Example usage of a dtrace probe in systemtap looks like:
>
>    probe process("qemu").mark("qemu_malloc") {
>      printf("Malloc %d %p\n", $arg1, $arg2);
>    }
>
> * .gitignore: Ignore trace-dtrace.*
> * Makefile: Extra rules for generating DTrace files
> * Makefile.obj: Don't build trace.o for DTrace, use
>    trace-dtrace.o generated by 'dtrace' instead
> * tracetool: Support for generating DTrace data files
>
> Signed-off-by: Daniel P. Berrange<berrange@redhat.com>
>    

Applied both.  Thanks.

Regards,

Anthony Liguori

> ---
>   .gitignore    |    2 +
>   Makefile      |   23 +++++++++++
>   Makefile.objs |    4 ++
>   configure     |   14 ++++++-
>   tracetool     |  116 ++++++++++++++++++++++++++++++++++++++++++++++++++++-----
>   5 files changed, 148 insertions(+), 11 deletions(-)
>
> diff --git a/.gitignore b/.gitignore
> index a43e4d1..3efb4ec 100644
> --- a/.gitignore
> +++ b/.gitignore
> @@ -4,6 +4,8 @@ config-host.*
>   config-target.*
>   trace.h
>   trace.c
> +trace-dtrace.h
> +trace-dtrace.dtrace
>   *-timestamp
>   *-softmmu
>   *-darwin-user
> diff --git a/Makefile b/Makefile
> index 02698e9..554ad97 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -1,6 +1,9 @@
>   # Makefile for QEMU.
>
>   GENERATED_HEADERS = config-host.h trace.h qemu-options.def
> +ifeq ($(TRACE_BACKEND),dtrace)
> +GENERATED_HEADERS += trace-dtrace.h
> +endif
>
>   ifneq ($(wildcard config-host.mak),)
>   # Put the all: rule here so that config-host.mak can contain dependencies.
> @@ -108,7 +111,11 @@ ui/vnc.o: QEMU_CFLAGS += $(VNC_TLS_CFLAGS)
>
>   bt-host.o: QEMU_CFLAGS += $(BLUEZ_CFLAGS)
>
> +ifeq ($(TRACE_BACKEND),dtrace)
> +trace.h: trace.h-timestamp trace-dtrace.h
> +else
>   trace.h: trace.h-timestamp
> +endif
>   trace.h-timestamp: $(SRC_PATH)/trace-events config-host.mak
>   	$(call quiet-command,sh $(SRC_PATH)/tracetool --$(TRACE_BACKEND) -h<  $<  >  $@,"  GEN   trace.h")
>   	@cmp -s $@ trace.h || cp $@ trace.h
> @@ -120,6 +127,20 @@ trace.c-timestamp: $(SRC_PATH)/trace-events config-host.mak
>
>   trace.o: trace.c $(GENERATED_HEADERS)
>
> +trace-dtrace.h: trace-dtrace.dtrace
> +	$(call quiet-command,dtrace -o $@ -h -s $<, "  GEN   trace-dtrace.h")
> +
> +# Normal practice is to name DTrace probe file with a '.d' extension
> +# but that gets picked up by QEMU's Makefile as an external dependancy
> +# rule file. So we use '.dtrace' instead
> +trace-dtrace.dtrace: trace-dtrace.dtrace-timestamp
> +trace-dtrace.dtrace-timestamp: $(SRC_PATH)/trace-events config-host.mak
> +	$(call quiet-command,sh $(SRC_PATH)/tracetool --$(TRACE_BACKEND) -d<  $<  >  $@,"  GEN   trace-dtrace.dtrace")
> +	@cmp -s $@ trace-dtrace.dtrace || cp $@ trace-dtrace.dtrace
> +
> +trace-dtrace.o: trace-dtrace.dtrace $(GENERATED_HEADERS)
> +	$(call quiet-command,dtrace -o $@ -G -s $<, "  GEN trace-dtrace.o")
> +
>   simpletrace.o: simpletrace.c $(GENERATED_HEADERS)
>
>   version.o: $(SRC_PATH)/version.rc config-host.mak
> @@ -157,6 +178,8 @@ clean:
>   	rm -f slirp/*.o slirp/*.d audio/*.o audio/*.d block/*.o block/*.d net/*.o net/*.d fsdev/*.o fsdev/*.d ui/*.o ui/*.d
>   	rm -f qemu-img-cmds.h
>   	rm -f trace.c trace.h trace.c-timestamp trace.h-timestamp
> +	rm -f trace-dtrace.dtrace trace-dtrace.dtrace-timestamp
> +	rm -f trace-dtrace.h trace-dtrace.h-timestamp
>   	$(MAKE) -C tests clean
>   	for d in $(ALL_SUBDIRS) libhw32 libhw64 libuser libdis libdis-user; do \
>   	if test -d $$d; then $(MAKE) -C $$d $@ || exit 1; fi; \
> diff --git a/Makefile.objs b/Makefile.objs
> index faf485e..84fc80e 100644
> --- a/Makefile.objs
> +++ b/Makefile.objs
> @@ -285,11 +285,15 @@ libdis-$(CONFIG_SPARC_DIS) += sparc-dis.o
>   ######################################################################
>   # trace
>
> +ifeq ($(TRACE_BACKEND),dtrace)
> +trace-obj-y = trace-dtrace.o
> +else
>   trace-obj-y = trace.o
>   ifeq ($(TRACE_BACKEND),simple)
>   trace-obj-y += simpletrace.o
>   user-obj-y += qemu-timer-common.o
>   endif
> +endif
>
>   vl.o: QEMU_CFLAGS+=$(GPROF_CFLAGS)
>
> diff --git a/configure b/configure
> index 7025d2b..f8dad3e 100755
> --- a/configure
> +++ b/configure
> @@ -929,7 +929,7 @@ echo "  --enable-docs            enable documentation build"
>   echo "  --disable-docs           disable documentation build"
>   echo "  --disable-vhost-net      disable vhost-net acceleration support"
>   echo "  --enable-vhost-net       enable vhost-net acceleration support"
> -echo "  --trace-backend=B        Trace backend nop simple ust"
> +echo "  --trace-backend=B        Trace backend nop simple ust dtrace"
>   echo "  --trace-file=NAME        Full PATH,NAME of file to store traces"
>   echo "                           Default:trace-<pid>"
>   echo "  --disable-spice          disable spice"
> @@ -2193,6 +2193,18 @@ EOF
>       exit 1
>     fi
>   fi
> +
> +##########################################
> +# For 'dtrace' backend, test if 'dtrace' command is present
> +if test "$trace_backend" = "dtrace"; then
> +  if ! has 'dtrace' ; then
> +    echo
> +    echo "Error: dtrace command is not found in PATH $PATH"
> +    echo
> +    exit 1
> +  fi
> +fi
> +
>   ##########################################
>   # End of CC checks
>   # After here, no more $cc or $ld runs
> diff --git a/tracetool b/tracetool
> index 7010858..5b6636a 100755
> --- a/tracetool
> +++ b/tracetool
> @@ -20,10 +20,12 @@ Backends:
>     --nop     Tracing disabled
>     --simple  Simple built-in backend
>     --ust     LTTng User Space Tracing backend
> +  --dtrace  DTrace/SystemTAP backend
>
>   Output formats:
>     -h    Generate .h file
>     -c    Generate .c file
> +  -d    Generate .d file (DTrace only)
>   EOF
>       exit 1
>   }
> @@ -46,8 +48,9 @@ get_args()
>   # Get the argument name list of a trace event
>   get_argnames()
>   {
> -    local nfields field name
> +    local nfields field name sep
>       nfields=0
> +    sep="$2"
>       for field in $(get_args "$1"); do
>           nfields=$((nfields + 1))
>
> @@ -58,7 +61,7 @@ get_argnames()
>           name=${field%,}
>           test "$field" = "$name"&&  continue
>
> -        printf "%s" "$name, "
> +        printf "%s%s " $name $sep
>       done
>
>       # Last argument name
> @@ -73,7 +76,7 @@ get_argc()
>   {
>       local name argc
>       argc=0
> -    for name in $(get_argnames "$1"); do
> +    for name in $(get_argnames "$1", ","); do
>           argc=$((argc + 1))
>       done
>       echo $argc
> @@ -154,7 +157,7 @@ EOF
>   cast_args_to_uint64_t()
>   {
>       local arg
> -    for arg in $(get_argnames "$1"); do
> +    for arg in $(get_argnames "$1", ","); do
>           printf "%s" "(uint64_t)(uintptr_t)$arg"
>       done
>   }
> @@ -247,7 +250,7 @@ linetoh_ust()
>       local name args argnames
>       name=$(get_name "$1")
>       args=$(get_args "$1")
> -    argnames=$(get_argnames "$1")
> +    argnames=$(get_argnames "$1", ",")
>
>       cat<<EOF
>   DECLARE_TRACE(ust_$name, TP_PROTO($args), TP_ARGS($argnames));
> @@ -274,7 +277,7 @@ linetoc_ust()
>       local name args argnames fmt
>       name=$(get_name "$1")
>       args=$(get_args "$1")
> -    argnames=$(get_argnames "$1")
> +    argnames=$(get_argnames "$1", ",")
>       fmt=$(get_fmt "$1")
>
>       cat<<EOF
> @@ -306,6 +309,87 @@ EOF
>       echo "}"
>   }
>
> +linetoh_begin_dtrace()
> +{
> +    cat<<EOF
> +#include "trace-dtrace.h"
> +EOF
> +}
> +
> +linetoh_dtrace()
> +{
> +    local name args argnames state nameupper
> +    name=$(get_name "$1")
> +    args=$(get_args "$1")
> +    argnames=$(get_argnames "$1", ",")
> +    state=$(get_state "$1")
> +    if [ "$state" = "0" ] ; then
> +        name=${name##disable }
> +    fi
> +
> +    nameupper=`echo $name | tr '[:lower:]' '[:upper:]'`
> +
> +    # Define an empty function for the trace event
> +    cat<<EOF
> +static inline void trace_$name($args) {
> +    if (QEMU_${nameupper}_ENABLED()) {
> +        QEMU_${nameupper}($argnames);
> +    }
> +}
> +EOF
> +}
> +
> +linetoh_end_dtrace()
> +{
> +    return
> +}
> +
> +linetoc_begin_dtrace()
> +{
> +    return
> +}
> +
> +linetoc_dtrace()
> +{
> +    # No need for function definitions in dtrace backend
> +    return
> +}
> +
> +linetoc_end_dtrace()
> +{
> +    return
> +}
> +
> +linetod_begin_dtrace()
> +{
> +    cat<<EOF
> +provider qemu {
> +EOF
> +}
> +
> +linetod_dtrace()
> +{
> +    local name args state
> +    name=$(get_name "$1")
> +    args=$(get_args "$1")
> +    state=$(get_state "$1")
> +    if [ "$state" = "0" ] ; then
> +        name=${name##disable }
> +    fi
> +
> +    # Define prototype for probe arguments
> +    cat<<EOF
> +        probe $name($args);
> +EOF
> +}
> +
> +linetod_end_dtrace()
> +{
> +    cat<<EOF
> +};
> +EOF
> +}
> +
>   # Process stdin by calling begin, line, and end functions for the backend
>   convert()
>   {
> @@ -324,9 +408,10 @@ convert()
>           disable=${str%%disable *}
>           echo
>           if test -z "$disable"; then
> -            # Pass the disabled state as an arg to lineto$1_simple().
> -            # For all other cases, call lineto$1_nop()
> -            if [ $backend = "simple" ]; then
> +            # Pass the disabled state as an arg for the simple
> +            # or DTrace backends which handle it dynamically.
> +            # For all other backends, call lineto$1_nop()
> +            if [ $backend = "simple" -o "$backend" = "dtrace" ]; then
>                   "$process_line" "$str"
>               else
>                   "lineto$1_nop" "${str##disable }"
> @@ -360,9 +445,19 @@ tracetoc()
>       convert c
>   }
>
> +tracetod()
> +{
> +    if [ $backend != "dtrace" ]; then
> +       echo "DTrace probe generator not applicable to $backend backend"
> +       exit 1
> +    fi
> +    echo "/* This file is autogenerated by tracetool, do not edit. */"
> +    convert d
> +}
> +
>   # Choose backend
>   case "$1" in
> -"--nop" | "--simple" | "--ust") backend="${1#--}" ;;
> +"--nop" | "--simple" | "--ust" | "--dtrace") backend="${1#--}" ;;
>   *) usage ;;
>   esac
>   shift
> @@ -370,6 +465,7 @@ shift
>   case "$1" in
>   "-h") tracetoh ;;
>   "-c") tracetoc ;;
> +"-d") tracetod ;;
>   "--check-backend") exit 0 ;; # used by ./configure to test for backend
>   *) usage ;;
>   esac
>    

Re: [Qemu-devel] [PATCH 1/2] Add a DTrace tracing backend targetted for SystemTAP compatability

[Peter Maydell]

On 16 November 2010 15:46, Anthony Liguori <anthony@codemonkey.ws> wrote:
> On 11/08/2010 01:33 PM, Daniel P. Berrange wrote:
>> This introduces a new tracing backend that targets the SystemTAP
>> implementation of DTrace userspace tracing.

> Applied both.  Thanks.

Unfortunately these commits:
2834c3e Add support for generating a systemtap tapset static probes
4addb11 Add a DTrace tracing backend targetted for SystemTAP compatability

seem to have broken building on x86:
 git clone git://git.qemu.org/qemu.git
 cd qemu
 ./configure
 make

fails with
  LINK  i386-softmmu/trace
/usr/lib/gcc/x86_64-linux-gnu/4.4.5/../../../../lib/crt1.o: In
function `_start':
(.text+0x20): undefined reference to `main'
collect2: ld returned 1 exit status

Incidentally, although trace.c is autogenerated, if you delete it and
then type make this does not cause it to be regenerated, which
seems wrong to me.

-- PMM

Re: [Qemu-devel] [PATCH 1/2] Add a DTrace tracing backend targetted for SystemTAP compatability

[Anthony Liguori]

On 11/16/2010 11:43 AM, Peter Maydell wrote:
> On 16 November 2010 15:46, Anthony Liguori<anthony@codemonkey.ws>  wrote:
>    
>> On 11/08/2010 01:33 PM, Daniel P. Berrange wrote:
>>      
>>> This introduces a new tracing backend that targets the SystemTAP
>>> implementation of DTrace userspace tracing.
>>>        
>    
>> Applied both.  Thanks.
>>      
> Unfortunately these commits:
> 2834c3e Add support for generating a systemtap tapset static probes
> 4addb11 Add a DTrace tracing backend targetted for SystemTAP compatability
>    

What's your configure output?

I don't have the right environment to build with systemtap support, but 
--trace-backend=nop should work regardless.

Regards,

Anthony Liguori

> seem to have broken building on x86:
>   git clone git://git.qemu.org/qemu.git
>   cd qemu
>   ./configure
>   make
>
> fails with
>    LINK  i386-softmmu/trace
> /usr/lib/gcc/x86_64-linux-gnu/4.4.5/../../../../lib/crt1.o: In
> function `_start':
> (.text+0x20): undefined reference to `main'
> collect2: ld returned 1 exit status
>
> Incidentally, although trace.c is autogenerated, if you delete it and
> then type make this does not cause it to be regenerated, which
> seems wrong to me.
>
> -- PMM
>    

Re: [Qemu-devel] [PATCH 1/2] Add a DTrace tracing backend targetted for SystemTAP compatability

[Peter Maydell]

[-- Attachment #1: Type: text/plain, Size: 2931 bytes --]

On 16 November 2010 18:10, Anthony Liguori <anthony@codemonkey.ws> wrote:
> On 11/16/2010 11:43 AM, Peter Maydell wrote:
>> Unfortunately these commits:
>> 2834c3e Add support for generating a systemtap tapset static probes
>> 4addb11 Add a DTrace tracing backend targetted for SystemTAP compatability
>
> What's your configure output?

I've attached it; the trace related bits are:
Trace backend     nop
Trace output file trace-<pid>

> I don't have the right environment to build with systemtap support, but
> --trace-backend=nop should work regardless.

I'm using the nop backend, yes.

I think the problem is that commit 2834c3e adds a target 'trace:' to the
Makefile.target which looks like it's intended to be a phony target. However
it isn't marked as such, so make actually tries to create a binary 'trace'
by falling back to its default rules (since there's a "trace.c" in the root
directory):

petmay01@LinaroE102767:~/qemu-test/qemu/i386-softmmu$ make -n trace
echo "  CC    trace.o" && gcc -I/home/petmay01/qemu-test/qemu/slirp
-Werror -m64 -I. -I/home/petmay01/qemu-test/qemu -D_FORTIFY_SOURCE=2
-D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE
-Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wendif-labels
-Wwrite-strings -Wmissing-prototypes -fno-strict-aliasing
-fstack-protector-all -Wempty-body -Wnested-externs -Wformat-security
-Wformat-y2k -Winit-self -Wignored-qualifiers -Wold-style-declaration
-Wold-style-definition -Wtype-limits -DHAS_AUDIO -DHAS_AUDIO_CHOICE
-I/home/petmay01/qemu-test/qemu/fpu
-I/home/petmay01/qemu-test/qemu/tcg
-I/home/petmay01/qemu-test/qemu/tcg/i386  -DTARGET_PHYS_ADDR_BITS=32
-I.. -I/home/petmay01/qemu-test/qemu/target-i386 -DNEED_CPU_H     -MMD
-MP -MT trace.o -MF ./trace.d -O2 -g  -c -o trace.o
/home/petmay01/qemu-test/qemu/trace.c
echo "  LINK  trace" && gcc -I/home/petmay01/qemu-test/qemu/slirp
-Werror -m64 -I. -I/home/petmay01/qemu-test/qemu -D_FORTIFY_SOURCE=2
-D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE
-Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wendif-labels
-Wwrite-strings -Wmissing-prototypes -fno-strict-aliasing
-fstack-protector-all -Wempty-body -Wnested-externs -Wformat-security
-Wformat-y2k -Winit-self -Wignored-qualifiers -Wold-style-declaration
-Wold-style-definition -Wtype-limits -DHAS_AUDIO -DHAS_AUDIO_CHOICE
-I/home/petmay01/qemu-test/qemu/fpu
-I/home/petmay01/qemu-test/qemu/tcg
-I/home/petmay01/qemu-test/qemu/tcg/i386  -DTARGET_PHYS_ADDR_BITS=32
-I.. -I/home/petmay01/qemu-test/qemu/target-i386 -DNEED_CPU_H     -O2
-g  -Wl,--warn-common -m64 -g   -o trace trace.o -lrt -lpthread
-lutil -lcurl   -lncurses  -luuid -lpng -lsasl2 -lgnutls   -lSDL
-lX11  -laio -lm -lz

...and linking only trace.o into a binary 'trace' fails because
trace.c doesn't have a main() (or indeed any functions at all).

If I add a ".PHONY: trace" or change the "trace" target
name to "tracexyzzy" then this fixes the problem.

-- PMM

[-- Attachment #2: configure.out --]
[-- Type: application/octet-stream, Size: 2468 bytes --]

Install prefix    /usr/local
BIOS directory    /usr/local/share/qemu
binary directory  /usr/local/bin
config directory  /usr/local/etc
Manual directory  /usr/local/share/man
ELF interp prefix /usr/gnemul/qemu-%M
Source path       /home/petmay01/qemu-test/qemu
C compiler        gcc
Host C compiler   gcc
CFLAGS            -O2 -g 
QEMU_CFLAGS       -Werror -m64 -I. -I$(SRC_PATH) -D_FORTIFY_SOURCE=2 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wendif-labels -Wwrite-strings -Wmissing-prototypes -fno-strict-aliasing  -fstack-protector-all -Wempty-body -Wnested-externs -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers -Wold-style-declaration -Wold-style-definition -Wtype-limits
LDFLAGS           -Wl,--warn-common -m64 -g 
make              make
install           install
host CPU          x86_64
host big endian   no
target list       i386-softmmu x86_64-softmmu arm-softmmu cris-softmmu m68k-softmmu microblaze-softmmu mips-softmmu mipsel-softmmu mips64-softmmu mips64el-softmmu ppc-softmmu ppcemb-softmmu ppc64-softmmu sh4-softmmu sh4eb-softmmu sparc-softmmu sparc64-softmmu i386-linux-user x86_64-linux-user alpha-linux-user arm-linux-user armeb-linux-user cris-linux-user m68k-linux-user microblaze-linux-user mips-linux-user mipsel-linux-user ppc-linux-user ppc64-linux-user ppc64abi32-linux-user sh4-linux-user sh4eb-linux-user sparc-linux-user sparc64-linux-user sparc32plus-linux-user 
tcg debug enabled no
Mon debug enabled no
gprof enabled     no
sparse enabled    no
strip binaries    yes
profiler          no
static build      no
-Werror enabled   yes
SDL support       yes
curses support    yes
curl support      yes
check support     no
mingw32 support   no
Audio drivers     oss
Extra audio cards ac97 es1370 sb16 hda
Block whitelist   
Mixer emulation   no
VNC TLS support   yes
VNC SASL support  yes
VNC JPEG support  no
VNC PNG support   yes
VNC thread        no
xen support       no
brlapi support    no
bluez  support    no
Documentation     no
NPTL support      yes
GUEST_BASE        yes
PIE user targets  no
vde support       no
IO thread         no
Linux AIO support yes
ATTR/XATTR support no
Install blobs     yes
KVM support       yes
fdt support       no
preadv support    yes
fdatasync         yes
madvise           yes
posix_madvise     yes
uuid support      yes
vhost-net support yes
Trace backend     nop
Trace output file trace-<pid>
spice support     no

Re: [Qemu-devel] [PATCH 1/2] Add a DTrace tracing backend targetted for SystemTAP compatability

[Anthony Liguori]

On 11/16/2010 12:54 PM, Peter Maydell wrote:
> On 16 November 2010 18:10, Anthony Liguori<anthony@codemonkey.ws>  wrote:
>    
>> On 11/16/2010 11:43 AM, Peter Maydell wrote:
>>      
>>> Unfortunately these commits:
>>> 2834c3e Add support for generating a systemtap tapset static probes
>>> 4addb11 Add a DTrace tracing backend targetted for SystemTAP compatability
>>>        
>> What's your configure output?
>>      
> I've attached it; the trace related bits are:
> Trace backend     nop
> Trace output file trace-<pid>
>
>    
>> I don't have the right environment to build with systemtap support, but
>> --trace-backend=nop should work regardless.
>>      
> I'm using the nop backend, yes.
>
> I think the problem is that commit 2834c3e adds a target 'trace:' to the
> Makefile.target which looks like it's intended to be a phony target. However
> it isn't marked as such, so make actually tries to create a binary 'trace'
> by falling back to its default rules (since there's a "trace.c" in the root
> directory):
>
> petmay01@LinaroE102767:~/qemu-test/qemu/i386-softmmu$ make -n trace
> echo "  CC    trace.o"&&  gcc -I/home/petmay01/qemu-test/qemu/slirp
> -Werror -m64 -I. -I/home/petmay01/qemu-test/qemu -D_FORTIFY_SOURCE=2
> -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE
> -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wendif-labels
> -Wwrite-strings -Wmissing-prototypes -fno-strict-aliasing
> -fstack-protector-all -Wempty-body -Wnested-externs -Wformat-security
> -Wformat-y2k -Winit-self -Wignored-qualifiers -Wold-style-declaration
> -Wold-style-definition -Wtype-limits -DHAS_AUDIO -DHAS_AUDIO_CHOICE
> -I/home/petmay01/qemu-test/qemu/fpu
> -I/home/petmay01/qemu-test/qemu/tcg
> -I/home/petmay01/qemu-test/qemu/tcg/i386  -DTARGET_PHYS_ADDR_BITS=32
> -I.. -I/home/petmay01/qemu-test/qemu/target-i386 -DNEED_CPU_H     -MMD
> -MP -MT trace.o -MF ./trace.d -O2 -g  -c -o trace.o
> /home/petmay01/qemu-test/qemu/trace.c
> echo "  LINK  trace"&&  gcc -I/home/petmay01/qemu-test/qemu/slirp
> -Werror -m64 -I. -I/home/petmay01/qemu-test/qemu -D_FORTIFY_SOURCE=2
> -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE
> -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wendif-labels
> -Wwrite-strings -Wmissing-prototypes -fno-strict-aliasing
> -fstack-protector-all -Wempty-body -Wnested-externs -Wformat-security
> -Wformat-y2k -Winit-self -Wignored-qualifiers -Wold-style-declaration
> -Wold-style-definition -Wtype-limits -DHAS_AUDIO -DHAS_AUDIO_CHOICE
> -I/home/petmay01/qemu-test/qemu/fpu
> -I/home/petmay01/qemu-test/qemu/tcg
> -I/home/petmay01/qemu-test/qemu/tcg/i386  -DTARGET_PHYS_ADDR_BITS=32
> -I.. -I/home/petmay01/qemu-test/qemu/target-i386 -DNEED_CPU_H     -O2
> -g  -Wl,--warn-common -m64 -g   -o trace trace.o -lrt -lpthread
> -lutil -lcurl   -lncurses  -luuid -lpng -lsasl2 -lgnutls   -lSDL
> -lX11  -laio -lm -lz
>
> ...and linking only trace.o into a binary 'trace' fails because
> trace.c doesn't have a main() (or indeed any functions at all).
>
> If I add a ".PHONY: trace" or change the "trace" target
> name to "tracexyzzy" then this fixes the problem.
>    

Curious, care to send a patch?  I think I'm not seeing this because I 
build with srcdir != objdir.

Regards,

Anthony Liguori

> -- PMM
>    

[Qemu-devel] [PATCH] Fix compilation failure with simple trace when srcdir==objdir

[Peter Maydell]

Fix a makefile error that meant that qemu would not compile if
the source and object directories were the same.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 Makefile.target |   11 +++++------
 1 files changed, 5 insertions(+), 6 deletions(-)

diff --git a/Makefile.target b/Makefile.target
index a5e6410..652c7d2 100644
--- a/Makefile.target
+++ b/Makefile.target
@@ -30,6 +30,7 @@ endif
 endif
 
 PROGS=$(QEMU_PROG)
+STPFILES=
 
 ifndef CONFIG_HAIKU
 LIBS+=-lm
@@ -41,19 +42,17 @@ config-target.h: config-target.h-timestamp
 config-target.h-timestamp: config-target.mak
 
 ifdef CONFIG_SYSTEMTAP_TRACE
-trace: $(QEMU_PROG).stp
+STPFILES+=$(QEMU_PROG).stp
 
 $(QEMU_PROG).stp:
 	$(call quiet-command,sh $(SRC_PATH)/tracetool \
 		--$(TRACE_BACKEND) \
 		--bindir $(bindir) \
 		--target $(TARGET_ARCH) \
-		-s < $(SRC_PATH)/trace-events > $(QEMU_PROG).stp,"  GEN   $(QEMU_PROG).stp")
-else
-trace:
+		-s < $(SRC_PATH)/trace-events > $@,"  GEN   $@")
 endif
 
-all: $(PROGS) trace
+all: $(PROGS) $(STPFILES)
 
 # Dummy command so that make thinks it has done something
 	@true
@@ -363,7 +362,7 @@ endif
 endif
 ifdef CONFIG_SYSTEMTAP_TRACE
 	$(INSTALL_DIR) "$(DESTDIR)$(datadir)/../systemtap/tapset"
-	$(INSTALL_DATA) $(QEMU_PROG).stp "$(DESTDIR)$(datadir)/../systemtap/tapset"
+	$(INSTALL_DATA) $(STPFILES) "$(DESTDIR)$(datadir)/../systemtap/tapset"
 endif
 
 # Include automatically generated dependency files
-- 
1.7.1

Re: [Qemu-devel] [PATCH] Fix compilation failure with simple trace when srcdir==objdir

[Stefan Hajnoczi]

On Tue, Nov 16, 2010 at 8:07 PM, Peter Maydell <peter.maydell@linaro.org> wrote:
> Fix a makefile error that meant that qemu would not compile if
> the source and object directories were the same.
>
> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
> ---
>  Makefile.target |   11 +++++------
>  1 files changed, 5 insertions(+), 6 deletions(-)

This patch fixes the build for me.

Stefan

Re: [Qemu-devel] [PATCH] Fix compilation failure with simple trace when srcdir==objdir

[Anthony Liguori]

On 11/16/2010 02:07 PM, Peter Maydell wrote:
> Fix a makefile error that meant that qemu would not compile if
> the source and object directories were the same.
>
> Signed-off-by: Peter Maydell<peter.maydell@linaro.org>
>    

Applied.  Thanks.

Regards,

Anthony Liguori
> ---
>   Makefile.target |   11 +++++------
>   1 files changed, 5 insertions(+), 6 deletions(-)
>
> diff --git a/Makefile.target b/Makefile.target
> index a5e6410..652c7d2 100644
> --- a/Makefile.target
> +++ b/Makefile.target
> @@ -30,6 +30,7 @@ endif
>   endif
>
>   PROGS=$(QEMU_PROG)
> +STPFILES=
>
>   ifndef CONFIG_HAIKU
>   LIBS+=-lm
> @@ -41,19 +42,17 @@ config-target.h: config-target.h-timestamp
>   config-target.h-timestamp: config-target.mak
>
>   ifdef CONFIG_SYSTEMTAP_TRACE
> -trace: $(QEMU_PROG).stp
> +STPFILES+=$(QEMU_PROG).stp
>
>   $(QEMU_PROG).stp:
>   	$(call quiet-command,sh $(SRC_PATH)/tracetool \
>   		--$(TRACE_BACKEND) \
>   		--bindir $(bindir) \
>   		--target $(TARGET_ARCH) \
> -		-s<  $(SRC_PATH)/trace-events>  $(QEMU_PROG).stp,"  GEN   $(QEMU_PROG).stp")
> -else
> -trace:
> +		-s<  $(SRC_PATH)/trace-events>  $@,"  GEN   $@")
>   endif
>
> -all: $(PROGS) trace
> +all: $(PROGS) $(STPFILES)
>
>   # Dummy command so that make thinks it has done something
>   	@true
> @@ -363,7 +362,7 @@ endif
>   endif
>   ifdef CONFIG_SYSTEMTAP_TRACE
>   	$(INSTALL_DIR) "$(DESTDIR)$(datadir)/../systemtap/tapset"
> -	$(INSTALL_DATA) $(QEMU_PROG).stp "$(DESTDIR)$(datadir)/../systemtap/tapset"
> +	$(INSTALL_DATA) $(STPFILES) "$(DESTDIR)$(datadir)/../systemtap/tapset"
>   endif
>
>   # Include automatically generated dependency files
>    

Re: [Qemu-devel] [PATCH 1/2] Add a DTrace tracing backend targetted for SystemTAP compatability

[Peter Maydell]

On 16 November 2010 18:58, Anthony Liguori <anthony@codemonkey.ws> wrote:
> On 11/16/2010 12:54 PM, Peter Maydell wrote:
>> If I add a ".PHONY: trace" or change the "trace" target
>> name to "tracexyzzy" then this fixes the problem.
>
> Curious, care to send a patch?  I think I'm not seeing this because I build
> with srcdir != objdir.

Ah, yes, if I build in a different directory to the sources I can build
without having to patch the makefiles. (That feature of configure
doesn't seem to be documented anywhere, unless I've missed
something.)

Anyway, patch sent. I opted to tidy up the makefile a little to avoid
either an ugly target name or a GNU-makeism. I fiddled configure
so I could test the CONFIG_SYSTEMTAP_TRACE bit despite
not having systemtap on this machine, but it would be good if
a real systemtap user could check it.

A couple of oddities I noticed along the way:

(1) How do you get configure to enable CONFIG_SYSTEMTAP_TRACE?
The condition is:
if test "$trace_backend" = "dtrace" -a "$trace_backend_stap" = "yes" ; then
  echo "CONFIG_SYSTEMTAP_TRACE=y" >> $config_host_mak
fi
but trace_backend_stap is only set inside code guarded by
if test "$trace_backend" = "ust"
...
so as far as I can tell the two halves of the -a will never
both be true.

(2) in Makefile.target, the blank line after the all: line:
> all: $(PROGS) $(STPFILES)
>
> # Dummy command so that make thinks it has done something
>         @true

is I think a bit misleading although it is valid makefile syntax.
I suspect it got added by accident.

-- PMM

Re: [Qemu-devel] [PATCH 1/2] Add a DTrace tracing backend targetted for SystemTAP compatability

[Daniel P. Berrange]

On Tue, Nov 16, 2010 at 06:54:57PM +0000, Peter Maydell wrote:
> On 16 November 2010 18:10, Anthony Liguori <anthony@codemonkey.ws> wrote:
> > On 11/16/2010 11:43 AM, Peter Maydell wrote:
> >> Unfortunately these commits:
> >> 2834c3e Add support for generating a systemtap tapset static probes
> >> 4addb11 Add a DTrace tracing backend targetted for SystemTAP compatability
> >
> > What's your configure output?
> 
> I've attached it; the trace related bits are:
> Trace backend     nop
> Trace output file trace-<pid>
> 
> > I don't have the right environment to build with systemtap support, but
> > --trace-backend=nop should work regardless.
> 
> I'm using the nop backend, yes.
> 
> I think the problem is that commit 2834c3e adds a target 'trace:' to the
> Makefile.target which looks like it's intended to be a phony target. However
> it isn't marked as such, so make actually tries to create a binary 'trace'
> by falling back to its default rules (since there's a "trace.c" in the root
> directory):

This Makefile problem was fixed in the v6 patches I posted on Nov 12th

http://lists.gnu.org/archive/html/qemu-devel/2010-11/msg01033.html

Unfortunately the v5 patches were applied by mistake.

Regards,
Daniel
-- 
|: Red Hat, Engineering, London    -o-   http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :|
|: http://autobuild.org        -o-         http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505  -o-   F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|

Re: [Qemu-devel] [PATCH 1/2] Add a DTrace tracing backend targetted for SystemTAP compatability

[Daniel P. Berrange]

On Tue, Nov 16, 2010 at 09:46:20AM -0600, Anthony Liguori wrote:
> On 11/08/2010 01:33 PM, Daniel P. Berrange wrote:
> >This introduces a new tracing backend that targets the SystemTAP
> >implementation of DTrace userspace tracing. The core functionality
> >should be applicable and standard across any DTrace implementation
> >on Solaris, OS-X, *BSD, but the Makefile rules will likely need
> >some small additional changes to cope with OS specific build
> >requirements.
> >
> >This backend builds a little differently from the other tracing
> >backends. Specifically there is no 'trace.c' file, because the
> >'dtrace' command line tool generates a '.o' file directly from
> >the dtrace probe definition file. The probe definition is usually
> >named with a '.d' extension but QEMU uses '.d' files for its
> >external makefile dependancy tracking, so this uses '.dtrace' as
> >the extension for the probe definition file.
> >
> >The 'tracetool' program gains the ability to generate a trace.h
> >file for DTrace, and also to generate the trace.d file containing
> >the dtrace probe definition.
> >
> >Example usage of a dtrace probe in systemtap looks like:
> >
> >   probe process("qemu").mark("qemu_malloc") {
> >     printf("Malloc %d %p\n", $arg1, $arg2);
> >   }
> >
> >* .gitignore: Ignore trace-dtrace.*
> >* Makefile: Extra rules for generating DTrace files
> >* Makefile.obj: Don't build trace.o for DTrace, use
> >   trace-dtrace.o generated by 'dtrace' instead
> >* tracetool: Support for generating DTrace data files
> >
> >Signed-off-by: Daniel P. Berrange<berrange@redhat.com>
> >   
> 
> Applied both.  Thanks.


I'm afraid you have applied an old version of the patches. Please
revert these, and apply version 6, from Nov 12th

http://lists.gnu.org/archive/html/qemu-devel/2010-11/msg01033.html

Regards,
Daniel
-- 
|: Red Hat, Engineering, London    -o-   http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :|
|: http://autobuild.org        -o-         http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505  -o-   F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|

Re: [Qemu-devel] [PATCH 1/2] Add a DTrace tracing backend targetted for SystemTAP compatability

[Anthony Liguori]

On 11/17/2010 05:35 AM, Daniel P. Berrange wrote:
> On Tue, Nov 16, 2010 at 09:46:20AM -0600, Anthony Liguori wrote:
>    
>> On 11/08/2010 01:33 PM, Daniel P. Berrange wrote:
>>      
>>> This introduces a new tracing backend that targets the SystemTAP
>>> implementation of DTrace userspace tracing. The core functionality
>>> should be applicable and standard across any DTrace implementation
>>> on Solaris, OS-X, *BSD, but the Makefile rules will likely need
>>> some small additional changes to cope with OS specific build
>>> requirements.
>>>
>>> This backend builds a little differently from the other tracing
>>> backends. Specifically there is no 'trace.c' file, because the
>>> 'dtrace' command line tool generates a '.o' file directly from
>>> the dtrace probe definition file. The probe definition is usually
>>> named with a '.d' extension but QEMU uses '.d' files for its
>>> external makefile dependancy tracking, so this uses '.dtrace' as
>>> the extension for the probe definition file.
>>>
>>> The 'tracetool' program gains the ability to generate a trace.h
>>> file for DTrace, and also to generate the trace.d file containing
>>> the dtrace probe definition.
>>>
>>> Example usage of a dtrace probe in systemtap looks like:
>>>
>>>    probe process("qemu").mark("qemu_malloc") {
>>>      printf("Malloc %d %p\n", $arg1, $arg2);
>>>    }
>>>
>>> * .gitignore: Ignore trace-dtrace.*
>>> * Makefile: Extra rules for generating DTrace files
>>> * Makefile.obj: Don't build trace.o for DTrace, use
>>>    trace-dtrace.o generated by 'dtrace' instead
>>> * tracetool: Support for generating DTrace data files
>>>
>>> Signed-off-by: Daniel P. Berrange<berrange@redhat.com>
>>>
>>>        
>> Applied both.  Thanks.
>>      
>
> I'm afraid you have applied an old version of the patches. Please
> revert these, and apply version 6, from Nov 12th
>    

Sure, sorry about that.

But in the future, please indicate the version somewhere in the patch 
(preferrably the header).

Regards,

Anthony Liguori

> http://lists.gnu.org/archive/html/qemu-devel/2010-11/msg01033.html
>
> Regards,
> Daniel
>    

Re: [Qemu-devel] [PATCH 1/2] Add a DTrace tracing backend targetted for SystemTAP compatability

[Stefan Hajnoczi]

On Thu, Nov 18, 2010 at 12:00 AM, Anthony Liguori <anthony@codemonkey.ws> wrote:
>>> Applied both.  Thanks.
>>>
>>
>> I'm afraid you have applied an old version of the patches. Please
>> revert these, and apply version 6, from Nov 12th
>>
>
> Sure, sorry about that.
>
> But in the future, please indicate the version somewhere in the patch
> (preferrably the header).

Here's the changelog from v5 to v6:

 - Fix handling of probes with no-args in DTrace provider
 - Generate one tapset per target, for both system and user
  emulators
 - Re-write command line arg parsing in tracetool so it
  doesn't assume a particular ordering of args
 - Change '-s' to '--stap' in tracetool to avoid confusion
 - Pass full binary name into tracetool

It would be nice to have v6.  Is it possible to revert v5 and apply v6 instead?

Stefan

[Qemu-devel] [PATCH 2/2] Add support for generating a systemtap tapset static probes

[Daniel P. Berrange]

This introduces generation of a qemu.stp/qemu-system-XXX.stp
files which provides tapsets with friendly names for static
probes & their arguments. Instead of

    probe process("qemu").mark("qemu_malloc") {
        printf("Malloc %d %p\n", $arg1, $arg2);
    }

It is now possible todo

    probe qemu.system.i386.qemu_malloc {
        printf("Malloc %d %p\n", size, ptr);
    }

There is one tapset defined per target arch.

* Makefile: Generate a qemu.stp file for systemtap
* tracetool: Support for generating systemtap tapsets

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
---
 Makefile.target |   19 +++++++++++-
 configure       |    7 ++++
 tracetool       |   92 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 117 insertions(+), 1 deletions(-)

diff --git a/Makefile.target b/Makefile.target
index 91e6e74..a5e6410 100644
--- a/Makefile.target
+++ b/Makefile.target
@@ -40,7 +40,20 @@ kvm.o kvm-all.o vhost.o vhost_net.o: QEMU_CFLAGS+=$(KVM_CFLAGS)
 config-target.h: config-target.h-timestamp
 config-target.h-timestamp: config-target.mak
 
-all: $(PROGS)
+ifdef CONFIG_SYSTEMTAP_TRACE
+trace: $(QEMU_PROG).stp
+
+$(QEMU_PROG).stp:
+	$(call quiet-command,sh $(SRC_PATH)/tracetool \
+		--$(TRACE_BACKEND) \
+		--bindir $(bindir) \
+		--target $(TARGET_ARCH) \
+		-s < $(SRC_PATH)/trace-events > $(QEMU_PROG).stp,"  GEN   $(QEMU_PROG).stp")
+else
+trace:
+endif
+
+all: $(PROGS) trace
 
 # Dummy command so that make thinks it has done something
 	@true
@@ -348,6 +361,10 @@ ifneq ($(STRIP),)
 	$(STRIP) $(patsubst %,"$(DESTDIR)$(bindir)/%",$(PROGS))
 endif
 endif
+ifdef CONFIG_SYSTEMTAP_TRACE
+	$(INSTALL_DIR) "$(DESTDIR)$(datadir)/../systemtap/tapset"
+	$(INSTALL_DATA) $(QEMU_PROG).stp "$(DESTDIR)$(datadir)/../systemtap/tapset"
+endif
 
 # Include automatically generated dependency files
 -include $(wildcard *.d */*.d)
diff --git a/configure b/configure
index f8dad3e..e560f87 100755
--- a/configure
+++ b/configure
@@ -2192,6 +2192,10 @@ EOF
     echo
     exit 1
   fi
+  trace_backend_stap="no"
+  if has 'stap' ; then
+    trace_backend_stap="yes"
+  fi
 fi
 
 ##########################################
@@ -2645,6 +2649,9 @@ fi
 if test "$trace_backend" = "simple"; then
   trace_file="\"$trace_file-%u\""
 fi
+if test "$trace_backend" = "dtrace" -a "$trace_backend_stap" = "yes" ; then
+  echo "CONFIG_SYSTEMTAP_TRACE=y" >> $config_host_mak
+fi
 echo "CONFIG_TRACE_FILE=$trace_file" >> $config_host_mak
 
 echo "TOOLS=$tools" >> $config_host_mak
diff --git a/tracetool b/tracetool
index 5b6636a..d797ab7 100755
--- a/tracetool
+++ b/tracetool
@@ -26,6 +26,12 @@ Output formats:
   -h    Generate .h file
   -c    Generate .c file
   -d    Generate .d file (DTrace only)
+  -s    Generate .stp file (DTrace with SystemTAP only)
+
+Options:
+  --bindir [bindir]  QEMU binary install location
+  --target [arch]    QEMU target architecture
+
 EOF
     exit 1
 }
@@ -390,6 +396,54 @@ linetod_end_dtrace()
 EOF
 }
 
+linetos_begin_dtrace()
+{
+    return
+}
+
+linetos_dtrace()
+{
+    local name args arglist state
+    name=$(get_name "$1")
+    args=$(get_args "$1")
+    arglist=$(get_argnames "$1", "")
+    state=$(get_state "$1")
+    if [ "$state" = "0" ] ; then
+        name=${name##disable }
+    fi
+
+    if [ "$target" = "i386" ]
+    then
+      binary="qemu"
+    else
+      binary="qemu-system-$target"
+    fi
+
+    # Define prototype for probe arguments
+    cat <<EOF
+probe qemu.system.$target.$name = process("$bindir/$binary").mark("$name")
+{
+EOF
+
+    i=1
+    for arg in $arglist
+    do
+        cat <<EOF
+  $arg = \$arg$i;
+EOF
+	i="$((i+1))"
+    done
+
+    cat <<EOF
+}
+EOF
+}
+
+linetos_end_dtrace()
+{
+    return
+}
+
 # Process stdin by calling begin, line, and end functions for the backend
 convert()
 {
@@ -455,6 +509,24 @@ tracetod()
     convert d
 }
 
+tracetos()
+{
+    if [ $backend != "dtrace" ]; then
+       echo "SystemTAP tapset generator not applicable to $backend backend"
+       exit 1
+    fi
+    if [ -z "$target" ]; then
+       echo "--target is required for SystemTAP tapset generator"
+       exit 1
+    fi
+    if [ -z "$bindir" ]; then
+       echo "--bindir is required for SystemTAP tapset generator"
+       exit 1
+    fi
+    echo "/* This file is autogenerated by tracetool, do not edit. */"
+    convert s
+}
+
 # Choose backend
 case "$1" in
 "--nop" | "--simple" | "--ust" | "--dtrace") backend="${1#--}" ;;
@@ -462,10 +534,30 @@ case "$1" in
 esac
 shift
 
+bindir=
+case "$1" in
+  "--bindir")
+    bindir=$2
+    shift
+    shift
+    ;;
+esac
+
+target=
+case "$1" in
+  "--target")
+    target=$2
+    shift
+    shift
+    ;;
+esac
+
+
 case "$1" in
 "-h") tracetoh ;;
 "-c") tracetoc ;;
 "-d") tracetod ;;
+"-s") tracetos ;;
 "--check-backend") exit 0 ;; # used by ./configure to test for backend
 *) usage ;;
 esac
-- 
1.7.2.3

[Qemu-devel] Add support for SystemTAP and DTrace tracing backends (v6)

[Daniel P. Berrange]

An update of the SystemTAP/DTrace patches from

  http://lists.gnu.org/archive/html/qemu-devel/2010-11/msg00563.html

The first patch contains the generic DTrace tracing backend support.
The second patch contains additional pieces for SystemTAP to generate
a tapset for each of the qemu-system-XXX binaries, to simplify life
for admins wanting to use the tracing backend.

Changed in v6:

 - Fix handling of probes with no-args in DTrace provider
 - Generate one tapset per target, for both system and user
   emulators
 - Re-write command line arg parsing in tracetool so it
   doesn't assume a particular ordering of args
 - Change '-s' to '--stap' in tracetool to avoid confusion
 - Pass full binary name into tracetool

[Qemu-devel] [PATCH 1/2] Add a DTrace tracing backend targetted for SystemTAP compatability

[Daniel P. Berrange]

This introduces a new tracing backend that targets the SystemTAP
implementation of DTrace userspace tracing. The core functionality
should be applicable and standard across any DTrace implementation
on Solaris, OS-X, *BSD, but the Makefile rules will likely need
some small additional changes to cope with OS specific build
requirements.

This backend builds a little differently from the other tracing
backends. Specifically there is no 'trace.c' file, because the
'dtrace' command line tool generates a '.o' file directly from
the dtrace probe definition file. The probe definition is usually
named with a '.d' extension but QEMU uses '.d' files for its
external makefile dependancy tracking, so this uses '.dtrace' as
the extension for the probe definition file.

The 'tracetool' program gains the ability to generate a trace.h
file for DTrace, and also to generate the trace.d file containing
the dtrace probe definition.

Example usage of a dtrace probe in systemtap looks like:

  probe process("qemu").mark("qemu_malloc") {
    printf("Malloc %d %p\n", $arg1, $arg2);
  }

* .gitignore: Ignore trace-dtrace.*
* Makefile: Extra rules for generating DTrace files
* Makefile.obj: Don't build trace.o for DTrace, use
  trace-dtrace.o generated by 'dtrace' instead
* tracetool: Support for generating DTrace data files

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
---
 .gitignore    |    2 +
 Makefile      |   23 +++++++++++
 Makefile.objs |    4 ++
 configure     |   14 ++++++-
 tracetool     |  122 ++++++++++++++++++++++++++++++++++++++++++++++++++++-----
 5 files changed, 154 insertions(+), 11 deletions(-)

diff --git a/.gitignore b/.gitignore
index a43e4d1..3efb4ec 100644
--- a/.gitignore
+++ b/.gitignore
@@ -4,6 +4,8 @@ config-host.*
 config-target.*
 trace.h
 trace.c
+trace-dtrace.h
+trace-dtrace.dtrace
 *-timestamp
 *-softmmu
 *-darwin-user
diff --git a/Makefile b/Makefile
index 02698e9..554ad97 100644
--- a/Makefile
+++ b/Makefile
@@ -1,6 +1,9 @@
 # Makefile for QEMU.
 
 GENERATED_HEADERS = config-host.h trace.h qemu-options.def
+ifeq ($(TRACE_BACKEND),dtrace)
+GENERATED_HEADERS += trace-dtrace.h
+endif
 
 ifneq ($(wildcard config-host.mak),)
 # Put the all: rule here so that config-host.mak can contain dependencies.
@@ -108,7 +111,11 @@ ui/vnc.o: QEMU_CFLAGS += $(VNC_TLS_CFLAGS)
 
 bt-host.o: QEMU_CFLAGS += $(BLUEZ_CFLAGS)
 
+ifeq ($(TRACE_BACKEND),dtrace)
+trace.h: trace.h-timestamp trace-dtrace.h
+else
 trace.h: trace.h-timestamp
+endif
 trace.h-timestamp: $(SRC_PATH)/trace-events config-host.mak
 	$(call quiet-command,sh $(SRC_PATH)/tracetool --$(TRACE_BACKEND) -h < $< > $@,"  GEN   trace.h")
 	@cmp -s $@ trace.h || cp $@ trace.h
@@ -120,6 +127,20 @@ trace.c-timestamp: $(SRC_PATH)/trace-events config-host.mak
 
 trace.o: trace.c $(GENERATED_HEADERS)
 
+trace-dtrace.h: trace-dtrace.dtrace
+	$(call quiet-command,dtrace -o $@ -h -s $<, "  GEN   trace-dtrace.h")
+
+# Normal practice is to name DTrace probe file with a '.d' extension
+# but that gets picked up by QEMU's Makefile as an external dependancy
+# rule file. So we use '.dtrace' instead
+trace-dtrace.dtrace: trace-dtrace.dtrace-timestamp
+trace-dtrace.dtrace-timestamp: $(SRC_PATH)/trace-events config-host.mak
+	$(call quiet-command,sh $(SRC_PATH)/tracetool --$(TRACE_BACKEND) -d < $< > $@,"  GEN   trace-dtrace.dtrace")
+	@cmp -s $@ trace-dtrace.dtrace || cp $@ trace-dtrace.dtrace
+
+trace-dtrace.o: trace-dtrace.dtrace $(GENERATED_HEADERS)
+	$(call quiet-command,dtrace -o $@ -G -s $<, "  GEN trace-dtrace.o")
+
 simpletrace.o: simpletrace.c $(GENERATED_HEADERS)
 
 version.o: $(SRC_PATH)/version.rc config-host.mak
@@ -157,6 +178,8 @@ clean:
 	rm -f slirp/*.o slirp/*.d audio/*.o audio/*.d block/*.o block/*.d net/*.o net/*.d fsdev/*.o fsdev/*.d ui/*.o ui/*.d
 	rm -f qemu-img-cmds.h
 	rm -f trace.c trace.h trace.c-timestamp trace.h-timestamp
+	rm -f trace-dtrace.dtrace trace-dtrace.dtrace-timestamp
+	rm -f trace-dtrace.h trace-dtrace.h-timestamp
 	$(MAKE) -C tests clean
 	for d in $(ALL_SUBDIRS) libhw32 libhw64 libuser libdis libdis-user; do \
 	if test -d $$d; then $(MAKE) -C $$d $@ || exit 1; fi; \
diff --git a/Makefile.objs b/Makefile.objs
index 15569af..23b17ce 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -286,11 +286,15 @@ libdis-$(CONFIG_SPARC_DIS) += sparc-dis.o
 ######################################################################
 # trace
 
+ifeq ($(TRACE_BACKEND),dtrace)
+trace-obj-y = trace-dtrace.o
+else
 trace-obj-y = trace.o
 ifeq ($(TRACE_BACKEND),simple)
 trace-obj-y += simpletrace.o
 user-obj-y += qemu-timer-common.o
 endif
+endif
 
 vl.o: QEMU_CFLAGS+=$(GPROF_CFLAGS)
 
diff --git a/configure b/configure
index 7025d2b..f8dad3e 100755
--- a/configure
+++ b/configure
@@ -929,7 +929,7 @@ echo "  --enable-docs            enable documentation build"
 echo "  --disable-docs           disable documentation build"
 echo "  --disable-vhost-net      disable vhost-net acceleration support"
 echo "  --enable-vhost-net       enable vhost-net acceleration support"
-echo "  --trace-backend=B        Trace backend nop simple ust"
+echo "  --trace-backend=B        Trace backend nop simple ust dtrace"
 echo "  --trace-file=NAME        Full PATH,NAME of file to store traces"
 echo "                           Default:trace-<pid>"
 echo "  --disable-spice          disable spice"
@@ -2193,6 +2193,18 @@ EOF
     exit 1
   fi
 fi
+
+##########################################
+# For 'dtrace' backend, test if 'dtrace' command is present
+if test "$trace_backend" = "dtrace"; then
+  if ! has 'dtrace' ; then
+    echo
+    echo "Error: dtrace command is not found in PATH $PATH"
+    echo
+    exit 1
+  fi
+fi
+
 ##########################################
 # End of CC checks
 # After here, no more $cc or $ld runs
diff --git a/tracetool b/tracetool
index 7010858..1ade103 100755
--- a/tracetool
+++ b/tracetool
@@ -20,10 +20,12 @@ Backends:
   --nop     Tracing disabled
   --simple  Simple built-in backend
   --ust     LTTng User Space Tracing backend
+  --dtrace  DTrace/SystemTAP backend
 
 Output formats:
   -h    Generate .h file
   -c    Generate .c file
+  -d    Generate .d file (DTrace only)
 EOF
     exit 1
 }
@@ -46,8 +48,9 @@ get_args()
 # Get the argument name list of a trace event
 get_argnames()
 {
-    local nfields field name
+    local nfields field name sep
     nfields=0
+    sep="$2"
     for field in $(get_args "$1"); do
         nfields=$((nfields + 1))
 
@@ -58,7 +61,7 @@ get_argnames()
         name=${field%,}
         test "$field" = "$name" && continue
 
-        printf "%s" "$name, "
+        printf "%s%s " $name $sep
     done
 
     # Last argument name
@@ -73,7 +76,7 @@ get_argc()
 {
     local name argc
     argc=0
-    for name in $(get_argnames "$1"); do
+    for name in $(get_argnames "$1", ","); do
         argc=$((argc + 1))
     done
     echo $argc
@@ -154,7 +157,7 @@ EOF
 cast_args_to_uint64_t()
 {
     local arg
-    for arg in $(get_argnames "$1"); do
+    for arg in $(get_argnames "$1", ","); do
         printf "%s" "(uint64_t)(uintptr_t)$arg"
     done
 }
@@ -247,7 +250,7 @@ linetoh_ust()
     local name args argnames
     name=$(get_name "$1")
     args=$(get_args "$1")
-    argnames=$(get_argnames "$1")
+    argnames=$(get_argnames "$1", ",")
 
     cat <<EOF
 DECLARE_TRACE(ust_$name, TP_PROTO($args), TP_ARGS($argnames));
@@ -274,7 +277,7 @@ linetoc_ust()
     local name args argnames fmt
     name=$(get_name "$1")
     args=$(get_args "$1")
-    argnames=$(get_argnames "$1")
+    argnames=$(get_argnames "$1", ",")
     fmt=$(get_fmt "$1")
 
     cat <<EOF
@@ -306,6 +309,93 @@ EOF
     echo "}"
 }
 
+linetoh_begin_dtrace()
+{
+    cat <<EOF
+#include "trace-dtrace.h"
+EOF
+}
+
+linetoh_dtrace()
+{
+    local name args argnames state nameupper
+    name=$(get_name "$1")
+    args=$(get_args "$1")
+    argnames=$(get_argnames "$1", ",")
+    state=$(get_state "$1")
+    if [ "$state" = "0" ] ; then
+        name=${name##disable }
+    fi
+
+    nameupper=`echo $name | tr '[:lower:]' '[:upper:]'`
+
+    # Define an empty function for the trace event
+    cat <<EOF
+static inline void trace_$name($args) {
+    if (QEMU_${nameupper}_ENABLED()) {
+        QEMU_${nameupper}($argnames);
+    }
+}
+EOF
+}
+
+linetoh_end_dtrace()
+{
+    return
+}
+
+linetoc_begin_dtrace()
+{
+    return
+}
+
+linetoc_dtrace()
+{
+    # No need for function definitions in dtrace backend
+    return
+}
+
+linetoc_end_dtrace()
+{
+    return
+}
+
+linetod_begin_dtrace()
+{
+    cat <<EOF
+provider qemu {
+EOF
+}
+
+linetod_dtrace()
+{
+    local name args state
+    name=$(get_name "$1")
+    args=$(get_args "$1")
+    state=$(get_state "$1")
+    if [ "$state" = "0" ] ; then
+        name=${name##disable }
+    fi
+
+    # DTrace provider syntax expects foo() for empty
+    # params, not foo(void)
+    if [ "$args" = "void" ]; then
+       args=""
+    fi
+
+    # Define prototype for probe arguments
+    cat <<EOF
+        probe $name($args);
+EOF
+}
+
+linetod_end_dtrace()
+{
+    cat <<EOF
+};
+EOF
+}
+
 # Process stdin by calling begin, line, and end functions for the backend
 convert()
 {
@@ -324,9 +414,10 @@ convert()
         disable=${str%%disable *}
         echo
         if test -z "$disable"; then
-            # Pass the disabled state as an arg to lineto$1_simple().
-            # For all other cases, call lineto$1_nop()
-            if [ $backend = "simple" ]; then
+            # Pass the disabled state as an arg for the simple
+            # or DTrace backends which handle it dynamically.
+            # For all other backends, call lineto$1_nop()
+            if [ $backend = "simple" -o "$backend" = "dtrace" ]; then
                 "$process_line" "$str"
             else
                 "lineto$1_nop" "${str##disable }"
@@ -360,9 +451,19 @@ tracetoc()
     convert c
 }
 
+tracetod()
+{
+    if [ $backend != "dtrace" ]; then
+       echo "DTrace probe generator not applicable to $backend backend"
+       exit 1
+    fi
+    echo "/* This file is autogenerated by tracetool, do not edit. */"
+    convert d
+}
+
 # Choose backend
 case "$1" in
-"--nop" | "--simple" | "--ust") backend="${1#--}" ;;
+"--nop" | "--simple" | "--ust" | "--dtrace") backend="${1#--}" ;;
 *) usage ;;
 esac
 shift
@@ -370,6 +471,7 @@ shift
 case "$1" in
 "-h") tracetoh ;;
 "-c") tracetoc ;;
+"-d") tracetod ;;
 "--check-backend") exit 0 ;; # used by ./configure to test for backend
 *) usage ;;
 esac
-- 
1.7.2.3

Re: [Qemu-devel] [PATCH 1/2] Add a DTrace tracing backend targetted for SystemTAP compatability

[Stefan Hajnoczi]

On Fri, Nov 12, 2010 at 1:20 PM, Daniel P. Berrange <berrange@redhat.com> wrote:
> This introduces a new tracing backend that targets the SystemTAP
> implementation of DTrace userspace tracing. The core functionality
> should be applicable and standard across any DTrace implementation
> on Solaris, OS-X, *BSD, but the Makefile rules will likely need
> some small additional changes to cope with OS specific build
> requirements.
>
> This backend builds a little differently from the other tracing
> backends. Specifically there is no 'trace.c' file, because the
> 'dtrace' command line tool generates a '.o' file directly from
> the dtrace probe definition file. The probe definition is usually
> named with a '.d' extension but QEMU uses '.d' files for its
> external makefile dependancy tracking, so this uses '.dtrace' as
> the extension for the probe definition file.
>
> The 'tracetool' program gains the ability to generate a trace.h
> file for DTrace, and also to generate the trace.d file containing
> the dtrace probe definition.
>
> Example usage of a dtrace probe in systemtap looks like:
>
>  probe process("qemu").mark("qemu_malloc") {
>    printf("Malloc %d %p\n", $arg1, $arg2);
>  }
>
> * .gitignore: Ignore trace-dtrace.*
> * Makefile: Extra rules for generating DTrace files
> * Makefile.obj: Don't build trace.o for DTrace, use
>  trace-dtrace.o generated by 'dtrace' instead
> * tracetool: Support for generating DTrace data files
>
> Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
> ---
>  .gitignore    |    2 +
>  Makefile      |   23 +++++++++++
>  Makefile.objs |    4 ++
>  configure     |   14 ++++++-
>  tracetool     |  122 ++++++++++++++++++++++++++++++++++++++++++++++++++++-----
>  5 files changed, 154 insertions(+), 11 deletions(-)

Reviewed-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>

[Qemu-devel] [PATCH 0/2] Add support for SystemTAP and DTrace tracing backends

[Daniel P. Berrange]

This is an update of the SystemTAP/DTrace patches previously sent

  http://lists.gnu.org/archive/html/qemu-devel/2010-10/msg01451.html

The patch is now split into two pieces. 

The first patch contains the generic DTrace tracing backend support.

The second patch contains additional pieces for SystemTAP to generate
a tapset for each of the qemu-system-XXX binaries, to simplify life
for admins wanting to use the tracing backend. This addresses the 
problem in previous versions of the patch where the tapset only worked 
for /usr/bin/qemu and no other binaries. Unfortunately SystemTAP does 
not allow for use of wildcards, so it was neccessary to go for a separate 
tapset file per emulator binary.

Regards,
Daniel

[Qemu-devel] [PATCH 1/2] Add a DTrace tracing backend targetted for SystemTAP compatability

[Daniel P. Berrange]

From: Daniel P. Berrange <berrange@redhat.com>

This introduces a new tracing backend that targets the SystemTAP
implementation of DTrace userspace tracing. The core functionality
should be applicable and standard across any DTrace implementation
on Solaris, OS-X, *BSD, but the Makefile rules will likely need
some small additional changes to cope with OS specific build
requirements.

This backend builds a little differently from the other tracing
backends. Specifically there is no 'trace.c' file, because the
'dtrace' command line tool generates a '.o' file directly from
the dtrace probe definition file. The probe definition is usually
named with a '.d' extension but QEMU uses '.d' files for its
external makefile dependancy tracking, so this uses '.dtrace' as
the extension for the probe definition file.

The 'tracetool' program gains the ability to generate a trace.h
file for DTrace, and also to generate the trace.d file containing
the dtrace probe definition.

Example usage of a dtrace probe in systemtap looks like:

  probe process("qemu").mark("qemu_malloc") {
    printf("Malloc %d %p\n", $arg1, $arg2);
  }

* .gitignore: Ignore trace-dtrace.*
* Makefile: Extra rules for generating DTrace files
* Makefile.obj: Don't build trace.o for DTrace, use
  trace-dtrace.o generated by 'dtrace' instead
* tracetool: Support for generating DTrace data files

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
---
 .gitignore    |    2 +
 Makefile      |   24 ++++++++++++
 Makefile.objs |    4 ++
 configure     |   14 ++++++-
 tracetool     |  116 ++++++++++++++++++++++++++++++++++++++++++++++++++++-----
 5 files changed, 149 insertions(+), 11 deletions(-)

diff --git a/.gitignore b/.gitignore
index a43e4d1..3efb4ec 100644
--- a/.gitignore
+++ b/.gitignore
@@ -4,6 +4,8 @@ config-host.*
 config-target.*
 trace.h
 trace.c
+trace-dtrace.h
+trace-dtrace.dtrace
 *-timestamp
 *-softmmu
 *-darwin-user
diff --git a/Makefile b/Makefile
index 02698e9..384bf72 100644
--- a/Makefile
+++ b/Makefile
@@ -1,6 +1,10 @@
 # Makefile for QEMU.
 
 GENERATED_HEADERS = config-host.h trace.h qemu-options.def
+GENERATED_HEADERS = config-host.h trace.h
+ifeq ($(TRACE_BACKEND),dtrace)
+GENERATED_HEADERS += trace-dtrace.h
+endif
 
 ifneq ($(wildcard config-host.mak),)
 # Put the all: rule here so that config-host.mak can contain dependencies.
@@ -108,7 +112,11 @@ ui/vnc.o: QEMU_CFLAGS += $(VNC_TLS_CFLAGS)
 
 bt-host.o: QEMU_CFLAGS += $(BLUEZ_CFLAGS)
 
+ifeq ($(TRACE_BACKEND),dtrace)
+trace.h: trace.h-timestamp trace-dtrace.h
+else
 trace.h: trace.h-timestamp
+endif
 trace.h-timestamp: $(SRC_PATH)/trace-events config-host.mak
 	$(call quiet-command,sh $(SRC_PATH)/tracetool --$(TRACE_BACKEND) -h < $< > $@,"  GEN   trace.h")
 	@cmp -s $@ trace.h || cp $@ trace.h
@@ -120,6 +128,20 @@ trace.c-timestamp: $(SRC_PATH)/trace-events config-host.mak
 
 trace.o: trace.c $(GENERATED_HEADERS)
 
+trace-dtrace.h: trace-dtrace.dtrace
+	$(call quiet-command,dtrace -o $@ -h -s $<, "  GEN   trace-dtrace.h")
+
+# Normal practice is to name DTrace probe file with a '.d' extension
+# but that gets picked up by QEMU's Makefile as an external dependancy
+# rule file. So we use '.dtrace' instead
+trace-dtrace.dtrace: trace-dtrace.dtrace-timestamp
+trace-dtrace.dtrace-timestamp: $(SRC_PATH)/trace-events config-host.mak
+	$(call quiet-command,sh $(SRC_PATH)/tracetool --$(TRACE_BACKEND) -d < $< > $@,"  GEN   trace-dtrace.dtrace")
+	@cmp -s $@ trace-dtrace.dtrace || cp $@ trace-dtrace.dtrace
+
+trace-dtrace.o: trace-dtrace.dtrace $(GENERATED_HEADERS)
+	$(call quiet-command,dtrace -o $@ -G -s $<, "  GEN trace-dtrace.o")
+
 simpletrace.o: simpletrace.c $(GENERATED_HEADERS)
 
 version.o: $(SRC_PATH)/version.rc config-host.mak
@@ -157,6 +179,8 @@ clean:
 	rm -f slirp/*.o slirp/*.d audio/*.o audio/*.d block/*.o block/*.d net/*.o net/*.d fsdev/*.o fsdev/*.d ui/*.o ui/*.d
 	rm -f qemu-img-cmds.h
 	rm -f trace.c trace.h trace.c-timestamp trace.h-timestamp
+	rm -f trace-dtrace.dtrace trace-dtrace.dtrace-timestamp
+	rm -f trace-dtrace.h trace-dtrace.h-timestamp
 	$(MAKE) -C tests clean
 	for d in $(ALL_SUBDIRS) libhw32 libhw64 libuser libdis libdis-user; do \
 	if test -d $$d; then $(MAKE) -C $$d $@ || exit 1; fi; \
diff --git a/Makefile.objs b/Makefile.objs
index faf485e..84fc80e 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -285,11 +285,15 @@ libdis-$(CONFIG_SPARC_DIS) += sparc-dis.o
 ######################################################################
 # trace
 
+ifeq ($(TRACE_BACKEND),dtrace)
+trace-obj-y = trace-dtrace.o
+else
 trace-obj-y = trace.o
 ifeq ($(TRACE_BACKEND),simple)
 trace-obj-y += simpletrace.o
 user-obj-y += qemu-timer-common.o
 endif
+endif
 
 vl.o: QEMU_CFLAGS+=$(GPROF_CFLAGS)
 
diff --git a/configure b/configure
index 7025d2b..f8dad3e 100755
--- a/configure
+++ b/configure
@@ -929,7 +929,7 @@ echo "  --enable-docs            enable documentation build"
 echo "  --disable-docs           disable documentation build"
 echo "  --disable-vhost-net      disable vhost-net acceleration support"
 echo "  --enable-vhost-net       enable vhost-net acceleration support"
-echo "  --trace-backend=B        Trace backend nop simple ust"
+echo "  --trace-backend=B        Trace backend nop simple ust dtrace"
 echo "  --trace-file=NAME        Full PATH,NAME of file to store traces"
 echo "                           Default:trace-<pid>"
 echo "  --disable-spice          disable spice"
@@ -2193,6 +2193,18 @@ EOF
     exit 1
   fi
 fi
+
+##########################################
+# For 'dtrace' backend, test if 'dtrace' command is present
+if test "$trace_backend" = "dtrace"; then
+  if ! has 'dtrace' ; then
+    echo
+    echo "Error: dtrace command is not found in PATH $PATH"
+    echo
+    exit 1
+  fi
+fi
+
 ##########################################
 # End of CC checks
 # After here, no more $cc or $ld runs
diff --git a/tracetool b/tracetool
index 7010858..5b6636a 100755
--- a/tracetool
+++ b/tracetool
@@ -20,10 +20,12 @@ Backends:
   --nop     Tracing disabled
   --simple  Simple built-in backend
   --ust     LTTng User Space Tracing backend
+  --dtrace  DTrace/SystemTAP backend
 
 Output formats:
   -h    Generate .h file
   -c    Generate .c file
+  -d    Generate .d file (DTrace only)
 EOF
     exit 1
 }
@@ -46,8 +48,9 @@ get_args()
 # Get the argument name list of a trace event
 get_argnames()
 {
-    local nfields field name
+    local nfields field name sep
     nfields=0
+    sep="$2"
     for field in $(get_args "$1"); do
         nfields=$((nfields + 1))
 
@@ -58,7 +61,7 @@ get_argnames()
         name=${field%,}
         test "$field" = "$name" && continue
 
-        printf "%s" "$name, "
+        printf "%s%s " $name $sep
     done
 
     # Last argument name
@@ -73,7 +76,7 @@ get_argc()
 {
     local name argc
     argc=0
-    for name in $(get_argnames "$1"); do
+    for name in $(get_argnames "$1", ","); do
         argc=$((argc + 1))
     done
     echo $argc
@@ -154,7 +157,7 @@ EOF
 cast_args_to_uint64_t()
 {
     local arg
-    for arg in $(get_argnames "$1"); do
+    for arg in $(get_argnames "$1", ","); do
         printf "%s" "(uint64_t)(uintptr_t)$arg"
     done
 }
@@ -247,7 +250,7 @@ linetoh_ust()
     local name args argnames
     name=$(get_name "$1")
     args=$(get_args "$1")
-    argnames=$(get_argnames "$1")
+    argnames=$(get_argnames "$1", ",")
 
     cat <<EOF
 DECLARE_TRACE(ust_$name, TP_PROTO($args), TP_ARGS($argnames));
@@ -274,7 +277,7 @@ linetoc_ust()
     local name args argnames fmt
     name=$(get_name "$1")
     args=$(get_args "$1")
-    argnames=$(get_argnames "$1")
+    argnames=$(get_argnames "$1", ",")
     fmt=$(get_fmt "$1")
 
     cat <<EOF
@@ -306,6 +309,87 @@ EOF
     echo "}"
 }
 
+linetoh_begin_dtrace()
+{
+    cat <<EOF
+#include "trace-dtrace.h"
+EOF
+}
+
+linetoh_dtrace()
+{
+    local name args argnames state nameupper
+    name=$(get_name "$1")
+    args=$(get_args "$1")
+    argnames=$(get_argnames "$1", ",")
+    state=$(get_state "$1")
+    if [ "$state" = "0" ] ; then
+        name=${name##disable }
+    fi
+
+    nameupper=`echo $name | tr '[:lower:]' '[:upper:]'`
+
+    # Define an empty function for the trace event
+    cat <<EOF
+static inline void trace_$name($args) {
+    if (QEMU_${nameupper}_ENABLED()) {
+        QEMU_${nameupper}($argnames);
+    }
+}
+EOF
+}
+
+linetoh_end_dtrace()
+{
+    return
+}
+
+linetoc_begin_dtrace()
+{
+    return
+}
+
+linetoc_dtrace()
+{
+    # No need for function definitions in dtrace backend
+    return
+}
+
+linetoc_end_dtrace()
+{
+    return
+}
+
+linetod_begin_dtrace()
+{
+    cat <<EOF
+provider qemu {
+EOF
+}
+
+linetod_dtrace()
+{
+    local name args state
+    name=$(get_name "$1")
+    args=$(get_args "$1")
+    state=$(get_state "$1")
+    if [ "$state" = "0" ] ; then
+        name=${name##disable }
+    fi
+
+    # Define prototype for probe arguments
+    cat <<EOF
+        probe $name($args);
+EOF
+}
+
+linetod_end_dtrace()
+{
+    cat <<EOF
+};
+EOF
+}
+
 # Process stdin by calling begin, line, and end functions for the backend
 convert()
 {
@@ -324,9 +408,10 @@ convert()
         disable=${str%%disable *}
         echo
         if test -z "$disable"; then
-            # Pass the disabled state as an arg to lineto$1_simple().
-            # For all other cases, call lineto$1_nop()
-            if [ $backend = "simple" ]; then
+            # Pass the disabled state as an arg for the simple
+            # or DTrace backends which handle it dynamically.
+            # For all other backends, call lineto$1_nop()
+            if [ $backend = "simple" -o "$backend" = "dtrace" ]; then
                 "$process_line" "$str"
             else
                 "lineto$1_nop" "${str##disable }"
@@ -360,9 +445,19 @@ tracetoc()
     convert c
 }
 
+tracetod()
+{
+    if [ $backend != "dtrace" ]; then
+       echo "DTrace probe generator not applicable to $backend backend"
+       exit 1
+    fi
+    echo "/* This file is autogenerated by tracetool, do not edit. */"
+    convert d
+}
+
 # Choose backend
 case "$1" in
-"--nop" | "--simple" | "--ust") backend="${1#--}" ;;
+"--nop" | "--simple" | "--ust" | "--dtrace") backend="${1#--}" ;;
 *) usage ;;
 esac
 shift
@@ -370,6 +465,7 @@ shift
 case "$1" in
 "-h") tracetoh ;;
 "-c") tracetoc ;;
+"-d") tracetod ;;
 "--check-backend") exit 0 ;; # used by ./configure to test for backend
 *) usage ;;
 esac
-- 
1.7.2.3

Re: [Qemu-devel] [PATCH 1/2] Add a DTrace tracing backend targetted for SystemTAP compatability

[Stefan Hajnoczi]

On Mon, Nov 8, 2010 at 11:33 AM, Daniel P. Berrange <dan@berrange.com> wrote:
> diff --git a/Makefile b/Makefile
> index 02698e9..384bf72 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -1,6 +1,10 @@
>  # Makefile for QEMU.
>
>  GENERATED_HEADERS = config-host.h trace.h qemu-options.def
> +GENERATED_HEADERS = config-host.h trace.h
> +ifeq ($(TRACE_BACKEND),dtrace)
> +GENERATED_HEADERS += trace-dtrace.h
> +endif

What happened to qemu-options.def?  GENERATED_HEADERS is defined
twice, the first line should probably be deleted.

Looks good otherwise.

Stefan

Re: [Qemu-devel] [PATCH 1/2] Add a DTrace tracing backend targetted for SystemTAP compatability

[Daniel P. Berrange]

On Mon, Nov 08, 2010 at 04:52:01PM +0000, Stefan Hajnoczi wrote:
> On Mon, Nov 8, 2010 at 11:33 AM, Daniel P. Berrange <dan@berrange.com> wrote:
> > diff --git a/Makefile b/Makefile
> > index 02698e9..384bf72 100644
> > --- a/Makefile
> > +++ b/Makefile
> > @@ -1,6 +1,10 @@
> >  # Makefile for QEMU.
> >
> >  GENERATED_HEADERS = config-host.h trace.h qemu-options.def
> > +GENERATED_HEADERS = config-host.h trace.h
> > +ifeq ($(TRACE_BACKEND),dtrace)
> > +GENERATED_HEADERS += trace-dtrace.h
> > +endif
> 
> What happened to qemu-options.def?  GENERATED_HEADERS is defined
> twice, the first line should probably be deleted.

Oh, the loss of qemu-options.def is a rebase/merge error. GENERATED_HEADERS
isn't defined twice, because the second line is using += to append to the
first definition.

Regards,
Daniel
-- 
|: Red Hat, Engineering, London    -o-   http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :|
|: http://autobuild.org        -o-         http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505  -o-   F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|

Re: [Qemu-devel] [PATCH 1/2] Add a DTrace tracing backend targetted for SystemTAP compatability

[Stefan Hajnoczi]

On Mon, Nov 8, 2010 at 4:57 PM, Daniel P. Berrange <berrange@redhat.com> wrote:
> On Mon, Nov 08, 2010 at 04:52:01PM +0000, Stefan Hajnoczi wrote:
>> On Mon, Nov 8, 2010 at 11:33 AM, Daniel P. Berrange <dan@berrange.com> wrote:
>> > diff --git a/Makefile b/Makefile
>> > index 02698e9..384bf72 100644
>> > --- a/Makefile
>> > +++ b/Makefile
>> > @@ -1,6 +1,10 @@
>> >  # Makefile for QEMU.
>> >
>> >  GENERATED_HEADERS = config-host.h trace.h qemu-options.def
>> > +GENERATED_HEADERS = config-host.h trace.h
>> > +ifeq ($(TRACE_BACKEND),dtrace)
>> > +GENERATED_HEADERS += trace-dtrace.h
>> > +endif
>>
>> What happened to qemu-options.def?  GENERATED_HEADERS is defined
>> twice, the first line should probably be deleted.
>
> Oh, the loss of qemu-options.def is a rebase/merge error. GENERATED_HEADERS
> isn't defined twice, because the second line is using += to append to the
> first definition.

Ah, it's the merge error I was referring to:

 @@ -1,6 +1,10 @@
 # Makefile for QEMU.

 GENERATED_HEADERS = config-host.h trace.h qemu-options.def
+GENERATED_HEADERS = config-host.h trace.h

Stefan