[Qemu-devel] [RFC PATCH 0/3] Fix UST backend for LTTng 2.x

[Qemu-devel] [RFC PATCH 0/3] Fix UST backend for LTTng 2.x

[Mohamad Gebai]

These patches fix the UST backend which got broken by LTTng 2.x.

An interesting new feature in LTTng 2.x is that tracepoints now need to belong
to a provider. A provider is a way to group tracepoints in a logical way, for
instance tracepoints that belong to a specific file or subsystem. This would
then allow users to enable all tracepoints of a specific provider at once.

In these patches, only one provider is defined, which is called "ust" (and
tracepoints are defined in generated-ust-provider.h). It would be interesting to
take advantage of this feature in the future, separating tracepoints into
multiple providers. Of course, this would require more modifications as well as
a way to specify to which provider each tracepoint belongs in file trace-events.

Mohamad

Mohamad Gebai (3):
  Fix configure script for LTTng 2.x
  Modified the tracetool framework for LTTng 2.x.
  Adapt Makefiles to the new LTTng ust interface.

 Makefile                                 |    5 ++
 configure                                |    9 ++--
 scripts/tracetool/backend/events.py      |   44 ++++++++++++++++
 scripts/tracetool/backend/ust.py         |   82 ++++++------------------------
 scripts/tracetool/format/ust_events_c.py |   30 +++++++++++
 scripts/tracetool/format/ust_events_h.py |   39 ++++++++++++++
 trace/Makefile.objs                      |   29 ++++++++++-
 7 files changed, 165 insertions(+), 73 deletions(-)
 create mode 100644 scripts/tracetool/format/ust_events_c.py
 create mode 100644 scripts/tracetool/format/ust_events_h.py

-- 
1.7.10.4

[Qemu-devel] [RFC PATCH 1/3] Fix configure script for LTTng 2.x

[Mohamad Gebai]

Signed-off-by: Mohamad Gebai <mohamad.gebai@polymtl.ca>
---
 configure |    9 ++++-----
 1 file changed, 4 insertions(+), 5 deletions(-)

diff --git a/configure b/configure
index 05e16da..b3c9b80 100755
--- a/configure
+++ b/configure
@@ -3283,15 +3283,14 @@ fi
 # For 'ust' backend, test if ust headers are present
 if test "$trace_backend" = "ust"; then
   cat > $TMPC << EOF
-#include <ust/tracepoint.h>
-#include <ust/marker.h>
+#include <lttng/tracepoint.h>
 int main(void) { return 0; }
 EOF
   if compile_prog "" "" ; then
-    LIBS="-lust -lurcu-bp $LIBS"
-    libs_qga="-lust -lurcu-bp $libs_qga"
+    LIBS="-llttng-ust -lurcu-bp -ldl $LIBS"
+    libs_qga="-llttng-ust -lurcu-bp -ldl $libs_qga"
   else
-    error_exit "Trace backend 'ust' missing libust header files"
+    error_exit "Trace backend 'ust' missing lttng-ust header files"
   fi
 fi
 
-- 
1.7.10.4

[Qemu-devel] [RFC PATCH 2/3] Modified the tracetool framework for LTTng 2.x.

[Mohamad Gebai]

* A new format is required to generate definitions for ust tracepoints.
  Files ust_events_h.py and ust_events_c.py define common macros, while
  new function ust_events_h in events.py does the actual definition of
  each tracepoint.
* ust.py generates the new interface for calling userspace tracepoints
  with LTTng 2.x, replacing trace_name(args) to tracepoint(name, args).
* As explained in ust_events_c.py, -Wredundant-decls gives a warning
  when compiling with gcc 4.7 or older. This is specific to lttng-ust so
  for now use a pragma clause to avoid getting a warning.

Signed-off-by: Mohamad Gebai <mohamad.gebai@polymtl.ca>
---
 scripts/tracetool/backend/events.py      |   44 ++++++++++++++++
 scripts/tracetool/backend/ust.py         |   82 ++++++------------------------
 scripts/tracetool/format/ust_events_c.py |   30 +++++++++++
 scripts/tracetool/format/ust_events_h.py |   39 ++++++++++++++
 4 files changed, 129 insertions(+), 66 deletions(-)
 create mode 100644 scripts/tracetool/format/ust_events_c.py
 create mode 100644 scripts/tracetool/format/ust_events_h.py

diff --git a/scripts/tracetool/backend/events.py b/scripts/tracetool/backend/events.py
index 5afce3e..49b7c65 100644
--- a/scripts/tracetool/backend/events.py
+++ b/scripts/tracetool/backend/events.py
@@ -6,6 +6,8 @@ Generic event description.
 
 This is a dummy backend to establish appropriate frontend/backend compatibility
 checks.
+
+Generates tracepoint definitions for LTTng userspace tracing when using ust as a backend.
 """
 
 __author__     = "Lluís Vilanova <vilanova@ac.upc.edu>"
@@ -16,8 +18,50 @@ __maintainer__ = "Stefan Hajnoczi"
 __email__      = "stefanha@linux.vnet.ibm.com"
 
 
+from tracetool import out
+
 def events_h(events):
     pass
 
 def events_c(events):
     pass
+
+def ust_events_c(events):
+    pass
+
+def ust_events_h(events):
+    for e in events:
+        if len(e.args) > 0:
+            out('TRACEPOINT_EVENT(',
+                '   ust,',
+                '   %(name)s,',
+                '   TP_ARGS(%(args)s),',
+                '   TP_FIELDS(',
+                name = e.name,
+                args = ", ".join(", ".join(i) for i in e.args),
+                )
+
+            for t,n in e.args:
+                if ('int' in t) or ('long' in t) or ('unsigned' in t) or ('size_t' in t):
+                    out('       ctf_integer(' + t + ', ' + n + ', ' + n + ')')
+                elif ('double' in t) or ('float' in t):
+                    out('       ctf_float(' + t + ', ' + n + ', ' + n + ')')
+                elif ('char *' in t) or ('char*' in t):
+                    out('       ctf_string(' + n + ', ' + n + ')')
+                elif ('void *' in t) or ('void*' in t):
+                    out('       ctf_integer_hex(unsigned long, ' + n + ', ' + n + ')')
+
+            out('   )',
+                ')',
+                '')
+
+        else:
+            out('TRACEPOINT_EVENT(',
+                '   ust,',
+                '   %(name)s,',
+                '   TP_ARGS(void),',
+                '   TP_FIELDS()',
+                ')',
+                '',
+                name = e.name,
+                )
diff --git a/scripts/tracetool/backend/ust.py b/scripts/tracetool/backend/ust.py
index ea36995..8695c74 100644
--- a/scripts/tracetool/backend/ust.py
+++ b/scripts/tracetool/backend/ust.py
@@ -18,76 +18,26 @@ from tracetool import out
 
 PUBLIC = True
 
-
 def c(events):
-    out('#include <ust/marker.h>',
-        '#undef mutex_lock',
-        '#undef mutex_unlock',
-        '#undef inline',
-        '#undef wmb',
-        '#include "trace.h"')
+    pass
+
 
+def h(events):
+    out('#include <lttng/tracepoint.h>',
+        '#include "trace/generated-ust-provider.h"',
+        '')
     for e in events:
         argnames = ", ".join(e.args.names())
         if len(e.args) > 0:
-            argnames = ', ' + argnames
-
-            out('DEFINE_TRACE(ust_%(name)s);',
-                '',
-                'static void ust_%(name)s_probe(%(args)s)',
-                '{',
-                '    trace_mark(ust, %(name)s, %(fmt)s%(argnames)s);',
-                '}',
-                name = e.name,
-                args = e.args,
-                fmt = e.fmt,
-                argnames = argnames,
-                )
-
-        else:
-            out('DEFINE_TRACE(ust_%(name)s);',
-                '',
-                'static void ust_%(name)s_probe(%(args)s)',
-                '{',
-                '    trace_mark(ust, %(name)s, UST_MARKER_NOARGS);',
-                '}',
-                name = e.name,
-                args = e.args,
-                )
-
-    # register probes
-    out('',
-        'static void __attribute__((constructor)) trace_init(void)',
-        '{')
-
-    for e in events:
-        out('    register_trace_ust_%(name)s(ust_%(name)s_probe);',
+            argnames = ", " + argnames
+
+        out('static inline void trace_%(name)s(%(args)s)',
+            '{',
+            '    tracepoint(ust, %(name)s, %(tp_args)s);'
+            '',
+            '}',
+            '',
             name = e.name,
+            args = e.args,
+            tp_args = ", ".join(e.args.names()),
             )
-
-    out('}')
-
-
-def h(events):
-    out('#include <ust/tracepoint.h>',
-        '#undef mutex_lock',
-        '#undef mutex_unlock',
-        '#undef inline',
-        '#undef wmb')
-
-    for e in events:
-        if len(e.args) > 0:
-            out('DECLARE_TRACE(ust_%(name)s, TP_PROTO(%(args)s), TP_ARGS(%(argnames)s));',
-                '#define trace_%(name)s trace_ust_%(name)s',
-                name = e.name,
-                args = e.args,
-                argnames = ", ".join(e.args.names()),
-                )
-
-        else:
-            out('_DECLARE_TRACEPOINT_NOARGS(ust_%(name)s);',
-                '#define trace_%(name)s trace_ust_%(name)s',
-                name = e.name,
-                )
-
-    out()
diff --git a/scripts/tracetool/format/ust_events_c.py b/scripts/tracetool/format/ust_events_c.py
new file mode 100644
index 0000000..71328f3
--- /dev/null
+++ b/scripts/tracetool/format/ust_events_c.py
@@ -0,0 +1,30 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+
+"""
+Generate .c for LTTng ust event description.
+"""
+
+__author__     = "Lluís Vilanova <vilanova@ac.upc.edu>"
+__copyright__  = "Copyright 2012, Lluís Vilanova <vilanova@ac.upc.edu>"
+__license__    = "GPL version 2 or (at your option) any later version"
+
+__maintainer__ = "Stefan Hajnoczi"
+__email__      = "stefanha@linux.vnet.ibm.com"
+
+
+from tracetool import out
+
+
+def begin(events):
+    out('/* This file is autogenerated by tracetool, do not edit. */',
+        '',
+        '#define TRACEPOINT_DEFINE',
+        '#define TRACEPOINT_CREATE_PROBES',
+        '',
+        '/* If gcc version 4.7 or older is used, LTTng ust gives a warning when compiling with',
+        '   -Wredundant-decls.',
+        ' */',
+        '#pragma GCC diagnostic ignored "-Wredundant-decls"',
+        '',
+        '#include "generated-ust-provider.h"')
diff --git a/scripts/tracetool/format/ust_events_h.py b/scripts/tracetool/format/ust_events_h.py
new file mode 100644
index 0000000..269576d
--- /dev/null
+++ b/scripts/tracetool/format/ust_events_h.py
@@ -0,0 +1,39 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+
+"""
+Generate .h for LTTng ust event description.
+"""
+
+__author__     = "Lluís Vilanova <vilanova@ac.upc.edu>"
+__copyright__  = "Copyright 2012, Lluís Vilanova <vilanova@ac.upc.edu>"
+__license__    = "GPL version 2 or (at your option) any later version"
+
+__maintainer__ = "Stefan Hajnoczi"
+__email__      = "stefanha@linux.vnet.ibm.com"
+
+
+from tracetool import out
+
+
+def begin(events):
+    out('/* This file is autogenerated by tracetool, do not edit. */',
+        '',
+        '#undef TRACEPOINT_PROVIDER',
+        '#define TRACEPOINT_PROVIDER ust',
+        '',
+        '#undef TRACEPOINT_INCLUDE_FILE',
+        '#define TRACEPOINT_INCLUDE_FILE ./generated-ust-provider.h',
+        '',
+        '#if !defined (TRACE__GENERATED_UST_H) || defined(TRACEPOINT_HEADER_MULTI_READ)',
+        '#define TRACE__GENERATED_UST_H',
+        '',
+        '#include "qemu-common.h"',
+        '#include <lttng/tracepoint.h>',
+        '')
+
+def end(events):
+    out('#endif /* TRACE__GENERATED_UST_H */',
+        '',
+        '/* This part must be outside ifdef protection */',
+        '#include <lttng/tracepoint-event.h>')
-- 
1.7.10.4

[Qemu-devel] [RFC PATCH 3/3] Adapt Makefiles to the new LTTng ust interface.

[Mohamad Gebai]

Add generation of new files for LTTng ust.

Signed-off-by: Mohamad Gebai <mohamad.gebai@polymtl.ca>
---
 Makefile            |    5 +++++
 trace/Makefile.objs |   29 +++++++++++++++++++++++++++--
 2 files changed, 32 insertions(+), 2 deletions(-)

diff --git a/Makefile b/Makefile
index 60fb87e..0c797c5 100644
--- a/Makefile
+++ b/Makefile
@@ -50,6 +50,11 @@ GENERATED_HEADERS += trace/generated-tracers-dtrace.h
 endif
 GENERATED_SOURCES += trace/generated-tracers.c
 
+ifeq ($(TRACE_BACKEND),ust)
+GENERATED_HEADERS += trace/generated-ust-provider.h
+GENERATED_SOURCES += trace/generated-ust.c
+endif
+
 # Don't try to regenerate Makefile or configure
 # We don't generate any of them
 Makefile: ;
diff --git a/trace/Makefile.objs b/trace/Makefile.objs
index 3b88e49..00880b3 100644
--- a/trace/Makefile.objs
+++ b/trace/Makefile.objs
@@ -1,9 +1,33 @@
 # -*- mode: makefile -*-
 
 ######################################################################
+# Auto-generated event descriptions for LTTng ust code
+
+ifeq ($(TRACE_BACKEND),ust)
+$(obj)/generated-ust-provider.h: $(obj)/generated-ust-provider.h-timestamp
+$(obj)/generated-ust-provider.h-timestamp: $(SRC_PATH)/trace-events
+	$(call quiet-command,$(TRACETOOL) \
+		--format=ust-events-h \
+		--backend=events \
+		< $< > $@,"  GEN   $(patsubst %-timestamp,%,$@)")
+	@cmp -s $@ $(patsubst %-timestamp,%,$@) || cp $@ $(patsubst %-timestamp,%,$@)
+
+$(obj)/generated-ust.c: $(obj)/generated-ust.c-timestamp $(BUILD_DIR)/config-host.mak
+$(obj)/generated-ust.c-timestamp: $(SRC_PATH)/trace-events
+	$(call quiet-command,$(TRACETOOL) \
+		--format=ust-events-c \
+		--backend=events \
+		< $< > $@,"  GEN   $(patsubst %-timestamp,%,$@)")
+	@cmp -s $@ $(patsubst %-timestamp,%,$@) || cp $@ $(patsubst %-timestamp,%,$@)
+
+ust_provider=$(obj)/generated-ust-provider.h
+ust=$(obj)/generated-ust.c
+endif
+
+######################################################################
 # Auto-generated event descriptions
 
-$(obj)/generated-events.h: $(obj)/generated-events.h-timestamp
+$(obj)/generated-events.h: $(obj)/generated-events.h-timestamp $(ust_provider)
 $(obj)/generated-events.h-timestamp: $(SRC_PATH)/trace-events
 	$(call quiet-command,$(TRACETOOL) \
 		--format=events-h \
@@ -11,7 +35,7 @@ $(obj)/generated-events.h-timestamp: $(SRC_PATH)/trace-events
 		< $< > $@,"  GEN   $(patsubst %-timestamp,%,$@)")
 	@cmp -s $@ $(patsubst %-timestamp,%,$@) || cp $@ $(patsubst %-timestamp,%,$@)
 
-$(obj)/generated-events.c: $(obj)/generated-events.c-timestamp $(BUILD_DIR)/config-host.mak
+$(obj)/generated-events.c: $(obj)/generated-events.c-timestamp $(BUILD_DIR)/config-host.mak $(ust)
 $(obj)/generated-events.c-timestamp: $(SRC_PATH)/trace-events
 	$(call quiet-command,$(TRACETOOL) \
 		--format=events-c \
@@ -77,5 +101,6 @@ util-obj-$(CONFIG_TRACE_DEFAULT) += default.o
 util-obj-$(CONFIG_TRACE_SIMPLE) += simple.o
 util-obj-$(CONFIG_TRACE_STDERR) += stderr.o
 util-obj-$(CONFIG_TRACE_FTRACE) += ftrace.o
+util-obj-$(CONFIG_TRACE_UST) += generated-ust.o
 util-obj-y += control.o
 util-obj-y += generated-tracers.o
-- 
1.7.10.4

Re: [Qemu-devel] [RFC PATCH 0/3] Fix UST backend for LTTng 2.x

[Alex Bennée]

mohamad.gebai@gmail.com writes:

> These patches fix the UST backend which got broken by LTTng 2.x.
>
> An interesting new feature in LTTng 2.x is that tracepoints now need to belong
> to a provider. A provider is a way to group tracepoints in a logical way, for
> instance tracepoints that belong to a specific file or subsystem. This would
> then allow users to enable all tracepoints of a specific provider at once.
>
> In these patches, only one provider is defined, which is called "ust" (and
> tracepoints are defined in generated-ust-provider.h). It would be interesting to
> take advantage of this feature in the future, separating tracepoints into
> multiple providers. Of course, this would require more modifications as well as
> a way to specify to which provider each tracepoint belongs in file trace-events.
>
I'm getting loads of compile failures like:

./trace/generated-tracers.h: In function ‘trace_esp_dma_enable’:
./trace/generated-tracers.h:3355:1: error: expected expression before ‘)’ token
./trace/generated-tracers.h:3355:1: error: expected expression before ‘)’ token
./trace/generated-tracers.h:3355:1: error: expected expression before ‘)’ token
./trace/generated-tracers.h:3355:1: error: implicit declaration of function ‘__tracepoint_cb_ust___esp_dma_enable’ [-Werror=implicit-function-declaration]
./trace/generated-tracers.h:3355:1: error: nested extern declaration of ‘__tracepoint_cb_ust___esp_dma_enable’ [-Werror=nested-externs]

The code looks like:

static inline void trace_esp_dma_enable(void)
{
    tracepoint(ust, esp_dma_enable, );
}

So I assume it's the included stuff that's falling over.

This is on Ubuntu 12.04 LTS with:

11:03 ajb@sloy/x86_64 [qemu.git] >dpkg -l | grep tng
ii  liblttng-ust-dev                                    2.0.2-0ubuntu1                                      LTTng 2.0 Userspace Tracer (development files)
ii  liblttng-ust0                                       2.0.2-0ubuntu1                                      LTTng 2.0 Userspace Tracer (libraries)

-- 
Alex Bennée