[PATCH 0/5] tracing: Add __rel_loc support

[PATCH 0/5] tracing: Add __rel_loc support

[Masami Hiramatsu]

Hello Beau,

(Sorry, if you get the same copy. I resend this because I made a typo
 on the mail address.)

Here is a series of patches to add support of '__rel_loc' relative
dynamic array attribute support. Note that this is only for the
kernel side code, user space tools might need to be updated for
decoding '__rel_loc' too.
There are 2 patches for libtraceevent and perftools, but not those
are not tested yet. I added it for review.

Here are 3 patches, [1/5] is the main and required patch, which
updates the event filter, histogram, and inject for __rel_loc.

[2/5] and [3/5] are for testing. I think those are no needed to
be merged. I just made it for the testing. [4/5] is updating
libtraceevent, which allows libtraceevent user to decode the
__rel_loc via libtraceevent API. Since perf-tools decodes the
event by itself, it needs [5/5]. (Note that I haven't tested
fully yet)

Steve, please tell me if I missed some parts or features which
need to decode the dynamic string or data.


The __rel_loc
-----

The '__data_loc' is used for encoding the dynamic data location on
the trace event record. But '__data_loc' is not useful if the writer
doesn't know the event header (e.g. user event), because it records
the dynamic data offset from the entry of the record, not the field
itself.

This new '__rel_loc' attribute encodes the data location relatively
from the next of the field. For example, when there is a record like
below (the number in the parentheses is the size of fields)

 |header(N)|common(M)|fields(K)|__data_loc(4)|fields(L)|data(G)|

In this case, '__data_loc' field will be

 __data_loc = (G << 16) | (N+M+K+4+L)

If '__rel_loc' is used, this will be

 |header(N)|common(M)|fields(K)|__rel_loc(4)|fields(L)|data(G)|

where

 __rel_loc = (G << 16) | (L)

(Because there is L bytes after the '__rel_loc' attribute field)

This is relatively easy (and no need to consider the kernel header
change) when the event data fields are composed by user who doesn't
know header and common fields.


Test Event Injection
----
/ # modprobe trace-events-sample.ko
/ # cd /sys/kernel/tracing/
/sys/kernel/tracing # echo 'foo="test" bar=0' >> events/sample-trace/foo_rel_loc
/inject 
/sys/kernel/tracing # cat trace 
# tracer: nop
#
# entries-in-buffer/entries-written: 1/1   #P:8
#
#                                _-----=> irqs-off
#                               / _----=> need-resched
#                              | / _---=> hardirq/softirq
#                              || / _--=> preempt-depth
#                              ||| / _-=> migrate-disable
#                              |||| /     delay
#           TASK-PID     CPU#  |||||  TIMESTAMP  FUNCTION
#              | |         |   |||||     |         |
           <...>-135     [002] .....   153.998779: foo_rel_loc: foo_rel_loc test, 0


Test Event Histogram
----

/sys/kernel/tracing # echo 'hist:key=foo' >> events/sample-trace/foo_rel_loc/trigger 
/sys/kernel/tracing # cat events/sample-trace/foo_rel_loc/hist 
# event histogram
#
# trigger info: hist:keys=foo:vals=hitcount:sort=hitcount:size=2048 [active]
#

{ foo: Hello __rel_loc                                    } hitcount:         11

Totals:
    Hits: 11
    Entries: 1
    Dropped: 0


Test Event Filter
----
/sys/kernel/tracing # echo 'foo == "Hello __rel_loc"' >> events/sample-trace/foo_rel_loc/filter
/sys/kernel/tracing # echo > trace
/sys/kernel/tracing # echo 1 >  events/sample-trace/foo_rel_loc/enable 
/sys/kernel/tracing # cat trace 
# tracer: nop
#
# entries-in-buffer/entries-written: 2/2   #P:8
#
#                                _-----=> irqs-off
#                               / _----=> need-resched
#                              | / _---=> hardirq/softirq
#                              || / _--=> preempt-depth
#                              ||| / _-=> migrate-disable
#                              |||| /     delay
#           TASK-PID     CPU#  |||||  TIMESTAMP  FUNCTION
#              | |         |   |||||     |         |
    event-sample-142     [005] .....   665.801471: foo_rel_loc: foo_rel_loc Hello __rel_loc, 631
    event-sample-142     [005] .....   666.825611: foo_rel_loc: foo_rel_loc Hello __rel_loc, 632


Thank you,

---

Masami Hiramatsu (5):
      tracing: Support __rel_loc relative dynamic data location attribute
      tracing: Add '__rel_loc' using trace event macros
      samples/trace_event: Add '__rel_loc' using sample event
      libtraceevent: Add __rel_loc relative location attribute support
      tools/perf: Add __rel_loc support


 include/linux/trace_events.h                       |    1 
 include/trace/bpf_probe.h                          |   13 +++
 include/trace/perf.h                               |   13 +++
 include/trace/trace_events.h                       |   83 ++++++++++++++++++++
 kernel/trace/trace.h                               |    4 +
 kernel/trace/trace_events_filter.c                 |   32 +++++++-
 kernel/trace/trace_events_hist.c                   |   21 +++++
 kernel/trace/trace_events_inject.c                 |   11 ++-
 samples/trace_events/trace-events-sample.c         |    2 
 samples/trace_events/trace-events-sample.h         |   28 +++++++
 tools/lib/traceevent/event-parse.c                 |   14 +++
 tools/lib/traceevent/event-parse.h                 |    1 
 tools/lib/traceevent/parse-filter.c                |    5 +
 tools/perf/builtin-trace.c                         |    2 
 tools/perf/util/data-convert-bt.c                  |    2 
 tools/perf/util/evsel.c                            |    2 
 tools/perf/util/python.c                           |    2 
 .../perf/util/scripting-engines/trace-event-perl.c |    2 
 .../util/scripting-engines/trace-event-python.c    |    2 
 tools/perf/util/sort.c                             |    2 
 20 files changed, 235 insertions(+), 7 deletions(-)

--
Masami Hiramatsu (Linaro) <mhiramat@kernel.org>

[PATCH 1/5] tracing: Support __rel_loc relative dynamic data location attribute

[Masami Hiramatsu]

Add '__rel_loc' new dynamic data location attribute which encodes
the data location from the next to the field itself.

The '__data_loc' is used for encoding the dynamic data location on
the trace event record. But '__data_loc' is not useful if the writer
doesn't know the event header (e.g. user event), because it records
the dynamic data offset from the entry of the record, not the field
itself.

This new '__rel_loc' attribute encodes the data location relatively
from the next of the field. For example, when there is a record like
below (the number in the parentheses is the size of fields)

 |header(N)|common(M)|fields(K)|__data_loc(4)|fields(L)|data(G)|

In this case, '__data_loc' field will be

 __data_loc = (G << 16) | (N+M+K+4+L)

If '__rel_loc' is used, this will be

 |header(N)|common(M)|fields(K)|__rel_loc(4)|fields(L)|data(G)|

where

 __rel_loc = (G << 16) | (L)

(Because there is L bytes after the '__rel_loc' attribute field)

This is relatively easy (and no need to consider the kernel header
change) when the event data fields are composed by user who doesn't
know header and common fields.

Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
---
 include/linux/trace_events.h       |    1 +
 kernel/trace/trace.h               |    1 +
 kernel/trace/trace_events_filter.c |   32 ++++++++++++++++++++++++++++++--
 kernel/trace/trace_events_hist.c   |   21 +++++++++++++++++++--
 kernel/trace/trace_events_inject.c |   11 +++++++++--
 5 files changed, 60 insertions(+), 6 deletions(-)

diff --git a/include/linux/trace_events.h b/include/linux/trace_events.h
index 50453b287615..d5b20540ff98 100644
--- a/include/linux/trace_events.h
+++ b/include/linux/trace_events.h
@@ -782,6 +782,7 @@ enum {
 	FILTER_OTHER = 0,
 	FILTER_STATIC_STRING,
 	FILTER_DYN_STRING,
+	FILTER_RDYN_STRING,
 	FILTER_PTR_STRING,
 	FILTER_TRACE_FN,
 	FILTER_COMM,
diff --git a/kernel/trace/trace.h b/kernel/trace/trace.h
index 6b60ab9475ed..4fd292c3a062 100644
--- a/kernel/trace/trace.h
+++ b/kernel/trace/trace.h
@@ -1453,6 +1453,7 @@ struct filter_pred {
 static inline bool is_string_field(struct ftrace_event_field *field)
 {
 	return field->filter_type == FILTER_DYN_STRING ||
+	       field->filter_type == FILTER_RDYN_STRING ||
 	       field->filter_type == FILTER_STATIC_STRING ||
 	       field->filter_type == FILTER_PTR_STRING ||
 	       field->filter_type == FILTER_COMM;
diff --git a/kernel/trace/trace_events_filter.c b/kernel/trace/trace_events_filter.c
index c9124038b140..996920ed1812 100644
--- a/kernel/trace/trace_events_filter.c
+++ b/kernel/trace/trace_events_filter.c
@@ -706,6 +706,29 @@ static int filter_pred_strloc(struct filter_pred *pred, void *event)
 	return match;
 }
 
+/*
+ * Filter predicate for relative dynamic sized arrays of characters.
+ * These are implemented through a list of strings at the end
+ * of the entry as same as dynamic string.
+ * The difference is that the relative one records the location offset
+ * from the field itself, not the event entry.
+ */
+static int filter_pred_strrelloc(struct filter_pred *pred, void *event)
+{
+	u32 *item = (u32 *)(event + pred->offset);
+	u32 str_item = *item;
+	int str_loc = str_item & 0xffff;
+	int str_len = str_item >> 16;
+	char *addr = (char *)(&item[1]) + str_loc;
+	int cmp, match;
+
+	cmp = pred->regex.match(addr, &pred->regex, str_len);
+
+	match = cmp ^ pred->not;
+
+	return match;
+}
+
 /* Filter predicate for CPUs. */
 static int filter_pred_cpu(struct filter_pred *pred, void *event)
 {
@@ -756,7 +779,7 @@ static int filter_pred_none(struct filter_pred *pred, void *event)
  *
  * Note:
  * - @str might not be NULL-terminated if it's of type DYN_STRING
- *   or STATIC_STRING, unless @len is zero.
+ *   RDYN_STRING, or STATIC_STRING, unless @len is zero.
  */
 
 static int regex_match_full(char *str, struct regex *r, int len)
@@ -1083,6 +1106,9 @@ int filter_assign_type(const char *type)
 	if (strstr(type, "__data_loc") && strstr(type, "char"))
 		return FILTER_DYN_STRING;
 
+	if (strstr(type, "__rel_loc") && strstr(type, "char"))
+		return FILTER_RDYN_STRING;
+
 	if (strchr(type, '[') && strstr(type, "char"))
 		return FILTER_STATIC_STRING;
 
@@ -1318,8 +1344,10 @@ static int parse_pred(const char *str, void *data,
 			pred->fn = filter_pred_string;
 			pred->regex.field_len = field->size;
 
-		} else if (field->filter_type == FILTER_DYN_STRING)
+		} else if (field->filter_type == FILTER_DYN_STRING) {
 			pred->fn = filter_pred_strloc;
+		} else if (field->filter_type == FILTER_RDYN_STRING)
+			pred->fn = filter_pred_strrelloc;
 		else
 			pred->fn = filter_pred_pchar;
 		/* go past the last quote */
diff --git a/kernel/trace/trace_events_hist.c b/kernel/trace/trace_events_hist.c
index 6a9fa34e2785..2c225f13f820 100644
--- a/kernel/trace/trace_events_hist.c
+++ b/kernel/trace/trace_events_hist.c
@@ -217,6 +217,20 @@ static u64 hist_field_dynstring(struct hist_field *hist_field,
 	return (u64)(unsigned long)addr;
 }
 
+static u64 hist_field_reldynstring(struct hist_field *hist_field,
+				   struct tracing_map_elt *elt,
+				   struct trace_buffer *buffer,
+				   struct ring_buffer_event *rbe,
+				   void *event)
+{
+	u32 *item = event + hist_field->field->offset;
+	u32 str_item = *item;
+	int str_loc = str_item & 0xffff;
+	char *addr = (char *)&item[1] + str_loc;
+
+	return (u64)(unsigned long)addr;
+}
+
 static u64 hist_field_pstring(struct hist_field *hist_field,
 			      struct tracing_map_elt *elt,
 			      struct trace_buffer *buffer,
@@ -1956,8 +1970,10 @@ static struct hist_field *create_hist_field(struct hist_trigger_data *hist_data,
 		if (field->filter_type == FILTER_STATIC_STRING) {
 			hist_field->fn = hist_field_string;
 			hist_field->size = field->size;
-		} else if (field->filter_type == FILTER_DYN_STRING)
+		} else if (field->filter_type == FILTER_DYN_STRING) {
 			hist_field->fn = hist_field_dynstring;
+		} else if (field->filter_type == FILTER_RDYN_STRING)
+			hist_field->fn = hist_field_reldynstring;
 		else
 			hist_field->fn = hist_field_pstring;
 	} else {
@@ -4952,7 +4968,8 @@ static inline void add_to_key(char *compound_key, void *key,
 		struct ftrace_event_field *field;
 
 		field = key_field->field;
-		if (field->filter_type == FILTER_DYN_STRING)
+		if (field->filter_type == FILTER_DYN_STRING ||
+		    field->filter_type == FILTER_RDYN_STRING)
 			size = *(u32 *)(rec + field->offset) >> 16;
 		else if (field->filter_type == FILTER_STATIC_STRING)
 			size = field->size;
diff --git a/kernel/trace/trace_events_inject.c b/kernel/trace/trace_events_inject.c
index c188045c5f97..d6b4935a78c0 100644
--- a/kernel/trace/trace_events_inject.c
+++ b/kernel/trace/trace_events_inject.c
@@ -168,10 +168,14 @@ static void *trace_alloc_entry(struct trace_event_call *call, int *size)
 			continue;
 		if (field->filter_type == FILTER_STATIC_STRING)
 			continue;
-		if (field->filter_type == FILTER_DYN_STRING) {
+		if (field->filter_type == FILTER_DYN_STRING ||
+		    field->filter_type == FILTER_RDYN_STRING) {
 			u32 *str_item;
 			int str_loc = entry_size & 0xffff;
 
+			if (field->filter_type == FILTER_RDYN_STRING)
+				str_loc -= field->offset + field->size;
+
 			str_item = (u32 *)(entry + field->offset);
 			*str_item = str_loc; /* string length is 0. */
 		} else {
@@ -214,7 +218,8 @@ static int parse_entry(char *str, struct trace_event_call *call, void **pentry)
 
 			if (field->filter_type == FILTER_STATIC_STRING) {
 				strlcpy(entry + field->offset, addr, field->size);
-			} else if (field->filter_type == FILTER_DYN_STRING) {
+			} else if (field->filter_type == FILTER_DYN_STRING ||
+				   field->filter_type == FILTER_RDYN_STRING) {
 				int str_len = strlen(addr) + 1;
 				int str_loc = entry_size & 0xffff;
 				u32 *str_item;
@@ -229,6 +234,8 @@ static int parse_entry(char *str, struct trace_event_call *call, void **pentry)
 
 				strlcpy(entry + (entry_size - str_len), addr, str_len);
 				str_item = (u32 *)(entry + field->offset);
+				if (field->filter_type == FILTER_RDYN_STRING)
+					str_loc -= field->offset + field->size;
 				*str_item = (str_len << 16) | str_loc;
 			} else {
 				char **paddr;

[PATCH 2/5] tracing: Add '__rel_loc' using trace event macros

[Masami Hiramatsu]

Add '__rel_loc' using trace event macros. These macros
are usually not used in the kernel, except for testing
purpose.

Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
---
 include/trace/bpf_probe.h    |   13 +++++++
 include/trace/perf.h         |   13 +++++++
 include/trace/trace_events.h |   83 ++++++++++++++++++++++++++++++++++++++++++
 kernel/trace/trace.h         |    3 ++
 4 files changed, 112 insertions(+)

diff --git a/include/trace/bpf_probe.h b/include/trace/bpf_probe.h
index a23be89119aa..ff40ec0c887d 100644
--- a/include/trace/bpf_probe.h
+++ b/include/trace/bpf_probe.h
@@ -18,6 +18,19 @@
 #undef __get_str
 #define __get_str(field) ((char *)__get_dynamic_array(field))
 
+#undef __get_rel_dynamic_array
+#define __get_rel_dynamic_array(field)	\
+		((void *)(&__entry->__rel_loc_##field) +	\
+		 sizeof(__entry->__rel_loc_##field) +		\
+		 (__entry->__rel_loc_##field & 0xffff))
+
+#undef __get_rel_dynamic_array_len
+#define __get_rel_dynamic_array_len(field)	\
+		((__entry->__rel_loc_##field >> 16) & 0xffff)
+
+#undef __get_rel_str
+#define __get_rel_str(field) ((char *)__get_rel_dynamic_array(field))
+
 #undef __get_bitmask
 #define __get_bitmask(field) (char *)__get_dynamic_array(field)
 
diff --git a/include/trace/perf.h b/include/trace/perf.h
index dbc6c74defc3..06cee508bdfa 100644
--- a/include/trace/perf.h
+++ b/include/trace/perf.h
@@ -18,6 +18,19 @@
 #undef __get_str
 #define __get_str(field) ((char *)__get_dynamic_array(field))
 
+#undef __get_rel_dynamic_array
+#define __get_rel_dynamic_array(field)	\
+		((void *)(&__entry->__rel_loc_##field) +	\
+		 sizeof(__entry->__rel_loc_##field) +		\
+		 (__entry->__rel_loc_##field & 0xffff))
+
+#undef __get_rel_dynamic_array_len
+#define __get_rel_dynamic_array_len(field)	\
+		((__entry->__rel_loc_##field >> 16) & 0xffff)
+
+#undef __get_rel_str
+#define __get_rel_str(field) ((char *)__get_rel_dynamic_array(field))
+
 #undef __get_bitmask
 #define __get_bitmask(field) (char *)__get_dynamic_array(field)
 
diff --git a/include/trace/trace_events.h b/include/trace/trace_events.h
index 08810a463880..6a730164479f 100644
--- a/include/trace/trace_events.h
+++ b/include/trace/trace_events.h
@@ -105,6 +105,15 @@ TRACE_MAKE_SYSTEM_STR();
 #undef __string_len
 #define __string_len(item, src, len) __dynamic_array(char, item, -1)
 
+#undef __rel_dynamic_array
+#define __rel_dynamic_array(type, item, len) u32 __rel_loc_##item;
+
+#undef __rel_string
+#define __rel_string(item, src) __rel_dynamic_array(char, item, -1)
+
+#undef __rel_string_len
+#define __rel_string_len(item, src, len) __rel_dynamic_array(char, item, -1)
+
 #undef __bitmask
 #define __bitmask(item, nr_bits) __dynamic_array(char, item, -1)
 
@@ -203,6 +212,15 @@ TRACE_MAKE_SYSTEM_STR();
 #undef __string_len
 #define __string_len(item, src, len) __dynamic_array(char, item, -1)
 
+#undef __rel_dynamic_array
+#define __rel_dynamic_array(type, item, len)	u32 item;
+
+#undef __rel_string
+#define __rel_string(item, src) __rel_dynamic_array(char, item, -1)
+
+#undef __rel_string_len
+#define __rel_string_len(item, src, len) __rel_dynamic_array(char, item, -1)
+
 #undef __bitmask
 #define __bitmask(item, nr_bits) __dynamic_array(unsigned long, item, -1)
 
@@ -293,6 +311,19 @@ TRACE_MAKE_SYSTEM_STR();
 #undef __get_str
 #define __get_str(field) ((char *)__get_dynamic_array(field))
 
+#undef __get_rel_dynamic_array
+#define __get_rel_dynamic_array(field)	\
+		((void *)(&__entry->__rel_loc_##field) +	\
+		 sizeof(__entry->__rel_loc_##field) +		\
+		 (__entry->__rel_loc_##field & 0xffff))
+
+#undef __get_rel_dynamic_array_len
+#define __get_rel_dynamic_array_len(field)	\
+		((__entry->__rel_loc_##field >> 16) & 0xffff)
+
+#undef __get_rel_str
+#define __get_rel_str(field) ((char *)__get_rel_dynamic_array(field))
+
 #undef __get_bitmask
 #define __get_bitmask(field)						\
 	({								\
@@ -468,6 +499,18 @@ static struct trace_event_functions trace_event_type_funcs_##call = {	\
 #undef __string_len
 #define __string_len(item, src, len) __dynamic_array(char, item, -1)
 
+#undef __rel_dynamic_array
+#define __rel_dynamic_array(_type, _item, _len) {			\
+	.type = "__rel_loc " #_type "[]", .name = #_item,		\
+	.size = 4, .align = 4,						\
+	.is_signed = is_signed_type(_type), .filter_type = FILTER_OTHER },
+
+#undef __rel_string
+#define __rel_string(item, src) __rel_dynamic_array(char, item, -1)
+
+#undef __rel_string_len
+#define __rel_string_len(item, src, len) __rel_dynamic_array(char, item, -1)
+
 #undef __bitmask
 #define __bitmask(item, nr_bits) __dynamic_array(unsigned long, item, -1)
 
@@ -519,6 +562,22 @@ static struct trace_event_fields trace_event_fields_##call[] = {	\
 #undef __string_len
 #define __string_len(item, src, len) __dynamic_array(char, item, (len) + 1)
 
+#undef __rel_dynamic_array
+#define __rel_dynamic_array(type, item, len)				\
+	__item_length = (len) * sizeof(type);				\
+	__data_offsets->item = __data_size +				\
+			       offsetof(typeof(*entry), __data) -	\
+			       offsetof(typeof(*entry), __rel_loc_##item) -	\
+			       sizeof(u32);				\
+	__data_offsets->item |= __item_length << 16;			\
+	__data_size += __item_length;
+
+#undef __rel_string
+#define __rel_string(item, src) __rel_dynamic_array(char, item,			\
+		    strlen((src) ? (const char *)(src) : "(null)") + 1)
+
+#undef __rel_string_len
+#define __rel_string_len(item, src, len) __rel_dynamic_array(char, item, (len) + 1)
 /*
  * __bitmask_size_in_bytes_raw is the number of bytes needed to hold
  * num_possible_cpus().
@@ -696,6 +755,27 @@ static inline notrace int trace_event_get_offsets_##call(		\
 		__get_str(dst)[len] = '\0';				\
 	} while(0)
 
+#undef __rel_dynamic_array
+#define __rel_dynamic_array(type, item, len)				\
+	__entry->__rel_loc_##item = __data_offsets.item;
+
+#undef __rel_string
+#define __rel_string(item, src) __rel_dynamic_array(char, item, -1)
+
+#undef __rel_string_len
+#define __rel_string_len(item, src, len) __rel_dynamic_array(char, item, -1)
+
+#undef __assign_rel_str
+#define __assign_rel_str(dst, src)					\
+	strcpy(__get_rel_str(dst), (src) ? (const char *)(src) : "(null)");
+
+#undef __assign_rel_str_len
+#define __assign_rel_str_len(dst, src, len)				\
+	do {								\
+		memcpy(__get_rel_str(dst), (src), (len));		\
+		__get_rel_str(dst)[len] = '\0';				\
+	} while (0)
+
 #undef __bitmask
 #define __bitmask(item, nr_bits) __dynamic_array(unsigned long, item, -1)
 
@@ -769,6 +849,9 @@ static inline void ftrace_test_probe_##call(void)			\
 #undef __get_dynamic_array
 #undef __get_dynamic_array_len
 #undef __get_str
+#undef __get_rel_dynamic_array
+#undef __get_rel_dynamic_array_len
+#undef __get_rel_str
 #undef __get_bitmask
 #undef __print_array
 #undef __print_hex_dump
diff --git a/kernel/trace/trace.h b/kernel/trace/trace.h
index 4fd292c3a062..f80d5612701e 100644
--- a/kernel/trace/trace.h
+++ b/kernel/trace/trace.h
@@ -83,6 +83,9 @@ enum trace_type {
 #undef __dynamic_array
 #define __dynamic_array(type, item)	type	item[];
 
+#undef __rel_dynamic_array
+#define __rel_dynamic_array(type, item)	type	item[];
+
 #undef F_STRUCT
 #define F_STRUCT(args...)		args
 

[PATCH 3/5] samples/trace_event: Add '__rel_loc' using sample event

[Masami Hiramatsu]

Add '__rel_loc' using sample event for testing.
User can use this for testing purpose. There is
no reason to use this macro from the kernel.

Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
---
 samples/trace_events/trace-events-sample.c |    2 ++
 samples/trace_events/trace-events-sample.h |   28 ++++++++++++++++++++++++++++
 2 files changed, 30 insertions(+)

diff --git a/samples/trace_events/trace-events-sample.c b/samples/trace_events/trace-events-sample.c
index 1a72b7d95cdc..b43415c31da6 100644
--- a/samples/trace_events/trace-events-sample.c
+++ b/samples/trace_events/trace-events-sample.c
@@ -43,6 +43,8 @@ static void simple_thread_func(int cnt)
 	trace_foo_with_template_cond("prints other times", cnt);
 
 	trace_foo_with_template_print("I have to be different", cnt);
+
+	trace_foo_rel_loc("Hello __rel_loc", cnt);
 }
 
 static int simple_thread(void *arg)
diff --git a/samples/trace_events/trace-events-sample.h b/samples/trace_events/trace-events-sample.h
index e61471ab7d14..be6321d4b04a 100644
--- a/samples/trace_events/trace-events-sample.h
+++ b/samples/trace_events/trace-events-sample.h
@@ -506,6 +506,34 @@ DEFINE_EVENT_PRINT(foo_template, foo_with_template_print,
 	TP_ARGS(foo, bar),
 	TP_printk("bar %s %d", __get_str(foo), __entry->bar));
 
+/*
+ * There are yet another __rel_loc dynamic data attribute. If you
+ * use __rel_dynamic_array() and __rel_string() etc. macros, you
+ * can use this attribute. There is no difference from the viewpoint
+ * of functionality with/without 'rel' but the encoding is a bit
+ * different. This is expected to be used with user-space event,
+ * there is no reason that the kernel event use this, but only for
+ * testing.
+ */
+
+TRACE_EVENT(foo_rel_loc,
+
+	TP_PROTO(const char *foo, int bar),
+
+	TP_ARGS(foo, bar),
+
+	TP_STRUCT__entry(
+		__rel_string(	foo,	foo	)
+		__field(	int,	bar	)
+	),
+
+	TP_fast_assign(
+		__assign_rel_str(foo, foo);
+		__entry->bar = bar;
+	),
+
+	TP_printk("foo_rel_loc %s, %d", __get_rel_str(foo), __entry->bar)
+);
 #endif
 
 /***** NOTICE! The #if protection ends here. *****/

[PATCH 4/5] libtraceevent: Add __rel_loc relative location attribute support

[Masami Hiramatsu]

Add '__rel_loc' new dynamic data location attribute which encodes
the data location from the next to the field itself. This is similar
to the '__data_loc' but the location offset is not from the event
entry but from the next of the field.

This patch adds '__rel_loc' decoding support in the libtraceevent.

Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
---
 tools/lib/traceevent/event-parse.c  |   14 ++++++++++++++
 tools/lib/traceevent/event-parse.h  |    1 +
 tools/lib/traceevent/parse-filter.c |    5 ++++-
 3 files changed, 19 insertions(+), 1 deletion(-)

diff --git a/tools/lib/traceevent/event-parse.c b/tools/lib/traceevent/event-parse.c
index fe58843d047c..170d2c9aa522 100644
--- a/tools/lib/traceevent/event-parse.c
+++ b/tools/lib/traceevent/event-parse.c
@@ -1367,6 +1367,14 @@ static int field_is_dynamic(struct tep_format_field *field)
 	return 0;
 }
 
+static int field_is_relative_dynamic(struct tep_format_field *field)
+{
+	if (strncmp(field->type, "__rel_loc", 9) == 0)
+		return 1;
+
+	return 0;
+}
+
 static int field_is_long(struct tep_format_field *field)
 {
 	/* includes long long */
@@ -1622,6 +1630,8 @@ static int event_read_fields(struct tep_event *event, struct tep_format_field **
 			field->flags |= TEP_FIELD_IS_STRING;
 		if (field_is_dynamic(field))
 			field->flags |= TEP_FIELD_IS_DYNAMIC;
+		if (field_is_relative_dynamic(field))
+			field->flags |= TEP_FIELD_IS_DYNAMIC|TEP_FIELD_IS_REL_DYNAMIC;
 		if (field_is_long(field))
 			field->flags |= TEP_FIELD_IS_LONG;
 
@@ -5109,6 +5119,8 @@ void tep_print_field(struct trace_seq *s, void *data,
 			offset = val;
 			len = offset >> 16;
 			offset &= 0xffff;
+			if (field->flags & TEP_FIELD_IS_REL_DYNAMIC)
+				offset += field->offset + field->size;
 		}
 		if (field->flags & TEP_FIELD_IS_STRING &&
 		    is_printable_array(data + offset, len)) {
@@ -6987,6 +6999,8 @@ void *tep_get_field_raw(struct trace_seq *s, struct tep_event *event,
 					 data + offset, field->size);
 		*len = offset >> 16;
 		offset &= 0xffff;
+		if (field->flags & TEP_FIELD_IS_REL_DYNAMIC)
+			offset += field->offset + field->size;
 	} else
 		*len = field->size;
 
diff --git a/tools/lib/traceevent/event-parse.h b/tools/lib/traceevent/event-parse.h
index a67ad9a5b835..c4138eb72605 100644
--- a/tools/lib/traceevent/event-parse.h
+++ b/tools/lib/traceevent/event-parse.h
@@ -125,6 +125,7 @@ enum tep_format_flags {
 	TEP_FIELD_IS_LONG	= 32,
 	TEP_FIELD_IS_FLAG	= 64,
 	TEP_FIELD_IS_SYMBOLIC	= 128,
+	TEP_FIELD_IS_REL_DYNAMIC = 256,
 };
 
 struct tep_format_field {
diff --git a/tools/lib/traceevent/parse-filter.c b/tools/lib/traceevent/parse-filter.c
index 368826bb5a57..9fcee7012df4 100644
--- a/tools/lib/traceevent/parse-filter.c
+++ b/tools/lib/traceevent/parse-filter.c
@@ -1712,8 +1712,11 @@ static const char *get_field_str(struct tep_filter_arg *arg, struct tep_record *
 
 		if (arg->str.field->flags & TEP_FIELD_IS_DYNAMIC) {
 			addr = *(unsigned int *)val;
-			val = record->data + (addr & 0xffff);
 			size = addr >> 16;
+			addr &= 0xffff;
+			if (arg->str.field->flags & TEP_FIELD_IS_REL_DYNAMIC)
+				addr += arg->str.field->offset + arg->str.field->size;
+			val = record->data + addr;
 		}
 
 		/*

Re: [PATCH 4/5] libtraceevent: Add __rel_loc relative location attribute support

[Steven Rostedt]

On Mon, 15 Nov 2021 19:20:36 +0900
Masami Hiramatsu <mhiramat@kernel.org> wrote:

> Add '__rel_loc' new dynamic data location attribute which encodes
> the data location from the next to the field itself. This is similar
> to the '__data_loc' but the location offset is not from the event
> entry but from the next of the field.
> 
> This patch adds '__rel_loc' decoding support in the libtraceevent.

Note, libtraceevent in the kernel is deprecated.

Care to send a patch against:

  https://git.kernel.org/pub/scm/libs/libtrace/libtraceevent.git/

And Cc linux-trace-devel@vger.kernel.org

Thanks!

-- Steve

> 
> Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
> ---
>  tools/lib/traceevent/event-parse.c  |   14 ++++++++++++++
>  tools/lib/traceevent/event-parse.h  |    1 +
>  tools/lib/traceevent/parse-filter.c |    5 ++++-
>  3 files changed, 19 insertions(+), 1 deletion(-)

Re: [PATCH 4/5] libtraceevent: Add __rel_loc relative location attribute support

[Masami Hiramatsu]

Hi Steve,

On Tue, 16 Nov 2021 17:23:32 -0500
Steven Rostedt <rostedt@goodmis.org> wrote:

> On Mon, 15 Nov 2021 19:20:36 +0900
> Masami Hiramatsu <mhiramat@kernel.org> wrote:
> 
> > Add '__rel_loc' new dynamic data location attribute which encodes
> > the data location from the next to the field itself. This is similar
> > to the '__data_loc' but the location offset is not from the event
> > entry but from the next of the field.
> > 
> > This patch adds '__rel_loc' decoding support in the libtraceevent.
> 
> Note, libtraceevent in the kernel is deprecated.

Ah, got it.

> 
> Care to send a patch against:
> 
>   https://git.kernel.org/pub/scm/libs/libtrace/libtraceevent.git/
> 
> And Cc linux-trace-devel@vger.kernel.org

Should I cc to LKML too?

Thank you,

> 
> Thanks!
> 
> -- Steve
> 
> > 
> > Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
> > ---
> >  tools/lib/traceevent/event-parse.c  |   14 ++++++++++++++
> >  tools/lib/traceevent/event-parse.h  |    1 +
> >  tools/lib/traceevent/parse-filter.c |    5 ++++-
> >  3 files changed, 19 insertions(+), 1 deletion(-)
> 


-- 
Masami Hiramatsu <mhiramat@kernel.org>

Re: [PATCH 4/5] libtraceevent: Add __rel_loc relative location attribute support

[Steven Rostedt]

On Wed, 17 Nov 2021 23:33:17 +0900
Masami Hiramatsu <mhiramat@kernel.org> wrote:

> > Care to send a patch against:
> > 
> >   https://git.kernel.org/pub/scm/libs/libtrace/libtraceevent.git/
> > 
> > And Cc linux-trace-devel@vger.kernel.org  
> 
> Should I cc to LKML too?

Only if you want to, but it is not necessary. Patches to linux-trace-devel
end up at:

   https://patchwork.kernel.org/project/linux-trace-devel/list/

-- Steve

Re: [PATCH 4/5] libtraceevent: Add __rel_loc relative location attribute support

[Masami Hiramatsu]

Hi Steve,

On Tue, 16 Nov 2021 17:23:32 -0500
Steven Rostedt <rostedt@goodmis.org> wrote:

> On Mon, 15 Nov 2021 19:20:36 +0900
> Masami Hiramatsu <mhiramat@kernel.org> wrote:
> 
> > Add '__rel_loc' new dynamic data location attribute which encodes
> > the data location from the next to the field itself. This is similar
> > to the '__data_loc' but the location offset is not from the event
> > entry but from the next of the field.
> > 
> > This patch adds '__rel_loc' decoding support in the libtraceevent.
> 
> Note, libtraceevent in the kernel is deprecated.

Without this patch, perf build is failed even if I installed

> 
> Care to send a patch against:
> 
>   https://git.kernel.org/pub/scm/libs/libtrace/libtraceevent.git/

this libtraceevent.

So it seems that the in-kernel libtraceevent source and header are
still in use.

Thank you,

-- 
Masami Hiramatsu <mhiramat@kernel.org>

Re: [PATCH 4/5] libtraceevent: Add __rel_loc relative location attribute support

[Steven Rostedt]

On Mon, 22 Nov 2021 14:05:38 +0900
Masami Hiramatsu <mhiramat@kernel.org> wrote:

> So it seems that the in-kernel libtraceevent source and header are
> still in use.

I believe it's still used if it's not found in the system.

I'll include this patch to the code as well.

Thanks!

-- Steve

Re: [PATCH 4/5] libtraceevent: Add __rel_loc relative location attribute support

[Masami Hiramatsu]

On Mon, 22 Nov 2021 11:25:20 -0500
Steven Rostedt <rostedt@goodmis.org> wrote:

> On Mon, 22 Nov 2021 14:05:38 +0900
> Masami Hiramatsu <mhiramat@kernel.org> wrote:
> 
> > So it seems that the in-kernel libtraceevent source and header are
> > still in use.
> 
> I believe it's still used if it's not found in the system.
> 
> I'll include this patch to the code as well.

Thanks! I sent v2 which fixes some bugs and covers bitmask too.
So please pick v2 up.

Thank you,

> 
> Thanks!
> 
> -- Steve


-- 
Masami Hiramatsu <mhiramat@kernel.org>

[PATCH 5/5] tools/perf: Add __rel_loc support

[Masami Hiramatsu]

Add new __rel_loc dynamic data location attribute support.
This type attribute is similar to the __data_loc but records
the offset from the field itself.
The libtraceevent adds TEP_FIELD_IS_REL_DYNAMIC to the
field->flags with TEP_FIELD_IS_DYNAMIC for __rel_loc.

Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
---
 tools/perf/builtin-trace.c                         |    2 ++
 tools/perf/util/data-convert-bt.c                  |    2 ++
 tools/perf/util/evsel.c                            |    2 ++
 tools/perf/util/python.c                           |    2 ++
 .../perf/util/scripting-engines/trace-event-perl.c |    2 ++
 .../util/scripting-engines/trace-event-python.c    |    2 ++
 tools/perf/util/sort.c                             |    2 ++
 7 files changed, 14 insertions(+)

diff --git a/tools/perf/builtin-trace.c b/tools/perf/builtin-trace.c
index 2bf21194c7b3..79f76eea0fcc 100644
--- a/tools/perf/builtin-trace.c
+++ b/tools/perf/builtin-trace.c
@@ -2722,6 +2722,8 @@ static size_t trace__fprintf_tp_fields(struct trace *trace, struct evsel *evsel,
 				offset = format_field__intval(field, sample, evsel->needs_swap);
 				syscall_arg.len = offset >> 16;
 				offset &= 0xffff;
+				if (field->flags & TEP_FIELD_IS_REL_DYNAMIC)
+					offset += field->offset + field->size;
 			}
 
 			val = (uintptr_t)(sample->raw_data + offset);
diff --git a/tools/perf/util/data-convert-bt.c b/tools/perf/util/data-convert-bt.c
index aa862a26d95c..4406f82abb64 100644
--- a/tools/perf/util/data-convert-bt.c
+++ b/tools/perf/util/data-convert-bt.c
@@ -318,6 +318,8 @@ static int add_tracepoint_field_value(struct ctf_writer *cw,
 		offset = tmp_val;
 		len = offset >> 16;
 		offset &= 0xffff;
+		if (flags & TEP_FIELD_IS_REL_DYNAMIC)
+			offset += fmtf->offset + fmtf->size;
 	}
 
 	if (flags & TEP_FIELD_IS_ARRAY) {
diff --git a/tools/perf/util/evsel.c b/tools/perf/util/evsel.c
index dbfeceb2546c..0dca46cf41a8 100644
--- a/tools/perf/util/evsel.c
+++ b/tools/perf/util/evsel.c
@@ -2597,6 +2597,8 @@ void *evsel__rawptr(struct evsel *evsel, struct perf_sample *sample, const char
 	if (field->flags & TEP_FIELD_IS_DYNAMIC) {
 		offset = *(int *)(sample->raw_data + field->offset);
 		offset &= 0xffff;
+		if (field->flags & TEP_FIELD_IS_REL_DYNAMIC)
+			offset += field->offset + field->size;
 	}
 
 	return sample->raw_data + offset;
diff --git a/tools/perf/util/python.c b/tools/perf/util/python.c
index 8feef3a05af7..f0dea973b395 100644
--- a/tools/perf/util/python.c
+++ b/tools/perf/util/python.c
@@ -416,6 +416,8 @@ tracepoint_field(struct pyrf_event *pe, struct tep_format_field *field)
 			offset  = val;
 			len     = offset >> 16;
 			offset &= 0xffff;
+			if (field->flags & TEP_FIELD_IS_REL_DYNAMIC)
+				offset += field->offset + field->size;
 		}
 		if (field->flags & TEP_FIELD_IS_STRING &&
 		    is_printable_array(data + offset, len)) {
diff --git a/tools/perf/util/scripting-engines/trace-event-perl.c b/tools/perf/util/scripting-engines/trace-event-perl.c
index 32a721b3e9a5..c535ed3b224e 100644
--- a/tools/perf/util/scripting-engines/trace-event-perl.c
+++ b/tools/perf/util/scripting-engines/trace-event-perl.c
@@ -392,6 +392,8 @@ static void perl_process_tracepoint(struct perf_sample *sample,
 			if (field->flags & TEP_FIELD_IS_DYNAMIC) {
 				offset = *(int *)(data + field->offset);
 				offset &= 0xffff;
+				if (field->flags & TEP_FIELD_IS_REL_DYNAMIC)
+					offset += field->offset + field->size;
 			} else
 				offset = field->offset;
 			XPUSHs(sv_2mortal(newSVpv((char *)data + offset, 0)));
diff --git a/tools/perf/util/scripting-engines/trace-event-python.c b/tools/perf/util/scripting-engines/trace-event-python.c
index c0c010350bc2..46344446f61c 100644
--- a/tools/perf/util/scripting-engines/trace-event-python.c
+++ b/tools/perf/util/scripting-engines/trace-event-python.c
@@ -942,6 +942,8 @@ static void python_process_tracepoint(struct perf_sample *sample,
 				offset  = val;
 				len     = offset >> 16;
 				offset &= 0xffff;
+				if (field->flags & TEP_FIELD_IS_REL_DYNAMIC)
+					offset += field->offset + field->size;
 			}
 			if (field->flags & TEP_FIELD_IS_STRING &&
 			    is_printable_array(data + offset, len)) {
diff --git a/tools/perf/util/sort.c b/tools/perf/util/sort.c
index 568a88c001c6..cffe2c484923 100644
--- a/tools/perf/util/sort.c
+++ b/tools/perf/util/sort.c
@@ -2385,6 +2385,8 @@ static int64_t __sort__hde_cmp(struct perf_hpp_fmt *fmt,
 		tep_read_number_field(field, a->raw_data, &dyn);
 		offset = dyn & 0xffff;
 		size = (dyn >> 16) & 0xffff;
+		if (field->flags & TEP_FIELD_IS_REL_DYNAMIC)
+			offset += field->offset + field->size;
 
 		/* record max width for output */
 		if (size > hde->dynamic_len)