[RFCv2 0/8] perf tools: Adding formula support

[RFCv2 0/8] perf tools: Adding formula support

[Jiri Olsa]

hi,
adding support to define counters for stat command,
by using user defined events and counters.

Initial RFC is here:
http://marc.info/?l=linux-kernel&m=135825930106535&w=2

v2 main changes:
  - counter formula grammar changed
    (details in patch 4 changelog)
  - using -e 'formula-<name>' way of using configured <name> set
    for stat command
    (details in patch 7 changelog)

thanks for comments,
jirka

Signed-off-by: Jiri Olsa <jolsa@redhat.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Corey Ashford <cjashfor@linux.vnet.ibm.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Andi Kleen <andi@firstfloor.org>
Cc: David Ahern <dsahern@gmail.com>
Cc: Ulrich Drepper <drepper@gmail.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Stephane Eranian <eranian@google.com>
---
Jiri Olsa (8):
      perf tools: Move start conditions to start of the flex file
      perf tools: Factorize event parsing to be more general
      perf tools: Add formula-* parsing support for events
      perf tools: Add formula interface to interface formula definitions
      perf tools: Add support to preload default formulas
      perf tests: Add automated tests for formula object
      perf stat: Add support to process formulas
      perf list: List formulas counters

 tools/perf/Makefile              |  20 ++++
 tools/perf/builtin-list.c        |   3 +
 tools/perf/builtin-stat.c        |  83 +++++++++++++-
 tools/perf/formulas/default.conf |  26 +++++
 tools/perf/tests/builtin-test.c  |   4 +
 tools/perf/tests/formula.c       | 183 ++++++++++++++++++++++++++++++
 tools/perf/tests/parse-events.c  |   8 --
 tools/perf/tests/tests.h         |  10 ++
 tools/perf/util/evlist.c         |  13 +++
 tools/perf/util/evlist.h         |   9 ++
 tools/perf/util/formula.c        | 600 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 tools/perf/util/formula.h        | 142 +++++++++++++++++++++++
 tools/perf/util/formula.l        | 119 ++++++++++++++++++++
 tools/perf/util/formula.y        | 249 +++++++++++++++++++++++++++++++++++++++++
 tools/perf/util/parse-events.c   |  56 ++++++++++
 tools/perf/util/parse-events.h   |  20 ++++
 tools/perf/util/parse-events.l   |  73 ++++++------
 tools/perf/util/parse-events.y   |  80 +++++++++++--
 18 files changed, 1646 insertions(+), 52 deletions(-)
 create mode 100644 tools/perf/formulas/default.conf
 create mode 100644 tools/perf/tests/formula.c
 create mode 100644 tools/perf/util/formula.c
 create mode 100644 tools/perf/util/formula.h
 create mode 100644 tools/perf/util/formula.l
 create mode 100644 tools/perf/util/formula.y

[PATCH 1/8] perf tools: Move start conditions to start of the flex file

[Jiri Olsa]

Moving start conditions to start of the flex file so it's clear
what the INITIAL condition rules are.

Plus adding default rule for INITIAL condition. This prevents
default space to be printed for events like:

$ ./perf stat -e "cycles        " kill 2>/dev/null
        $
^^^^^^^^

Signed-off-by: Jiri Olsa <jolsa@redhat.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Corey Ashford <cjashfor@linux.vnet.ibm.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Andi Kleen <andi@firstfloor.org>
Cc: David Ahern <dsahern@gmail.com>
Cc: Ulrich Drepper <drepper@gmail.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Stephane Eranian <eranian@google.com>
---
 tools/perf/util/parse-events.l | 63 +++++++++++++++++++++---------------------
 1 file changed, 32 insertions(+), 31 deletions(-)

diff --git a/tools/perf/util/parse-events.l b/tools/perf/util/parse-events.l
index e9d1134..5b4ef52 100644
--- a/tools/perf/util/parse-events.l
+++ b/tools/perf/util/parse-events.l
@@ -125,6 +125,37 @@ modifier_bp	[rwx]{1,3}
 
 }
 
+<config>{
+config			{ return term(yyscanner, PARSE_EVENTS__TERM_TYPE_CONFIG); }
+config1			{ return term(yyscanner, PARSE_EVENTS__TERM_TYPE_CONFIG1); }
+config2			{ return term(yyscanner, PARSE_EVENTS__TERM_TYPE_CONFIG2); }
+name			{ return term(yyscanner, PARSE_EVENTS__TERM_TYPE_NAME); }
+period			{ return term(yyscanner, PARSE_EVENTS__TERM_TYPE_SAMPLE_PERIOD); }
+branch_type		{ return term(yyscanner, PARSE_EVENTS__TERM_TYPE_BRANCH_SAMPLE_TYPE); }
+,			{ return ','; }
+"/"			{ BEGIN(INITIAL); return '/'; }
+{name_minus}		{ return str(yyscanner, PE_NAME); }
+}
+
+<mem>{
+{modifier_bp}		{ return str(yyscanner, PE_MODIFIER_BP); }
+:			{ return ':'; }
+{num_dec}		{ return value(yyscanner, 10); }
+{num_hex}		{ return value(yyscanner, 16); }
+	/*
+	 * We need to separate 'mem:' scanner part, in order to get specific
+	 * modifier bits parsed out. Otherwise we would need to handle PE_NAME
+	 * and we'd need to parse it manually. During the escape from <mem>
+	 * state we need to put the escaping char back, so we dont miss it.
+	 */
+.			{ unput(*yytext); BEGIN(INITIAL); }
+	/*
+	 * We destroy the scanner after reaching EOF,
+	 * but anyway just to be sure get back to INIT state.
+	 */
+<<EOF>>			{ BEGIN(INITIAL); }
+}
+
 cpu-cycles|cycles				{ return sym(yyscanner, PERF_TYPE_HARDWARE, PERF_COUNT_HW_CPU_CYCLES); }
 stalled-cycles-frontend|idle-cycles-frontend	{ return sym(yyscanner, PERF_TYPE_HARDWARE, PERF_COUNT_HW_STALLED_CYCLES_FRONTEND); }
 stalled-cycles-backend|idle-cycles-backend	{ return sym(yyscanner, PERF_TYPE_HARDWARE, PERF_COUNT_HW_STALLED_CYCLES_BACKEND); }
@@ -160,18 +191,6 @@ speculative-read|speculative-load	|
 refs|Reference|ops|access		|
 misses|miss				{ return str(yyscanner, PE_NAME_CACHE_OP_RESULT); }
 
-<config>{
-config			{ return term(yyscanner, PARSE_EVENTS__TERM_TYPE_CONFIG); }
-config1			{ return term(yyscanner, PARSE_EVENTS__TERM_TYPE_CONFIG1); }
-config2			{ return term(yyscanner, PARSE_EVENTS__TERM_TYPE_CONFIG2); }
-name			{ return term(yyscanner, PARSE_EVENTS__TERM_TYPE_NAME); }
-period			{ return term(yyscanner, PARSE_EVENTS__TERM_TYPE_SAMPLE_PERIOD); }
-branch_type		{ return term(yyscanner, PARSE_EVENTS__TERM_TYPE_BRANCH_SAMPLE_TYPE); }
-,			{ return ','; }
-"/"			{ BEGIN(INITIAL); return '/'; }
-{name_minus}		{ return str(yyscanner, PE_NAME); }
-}
-
 mem:			{ BEGIN(mem); return PE_PREFIX_MEM; }
 r{num_raw_hex}		{ return raw(yyscanner); }
 {num_dec}		{ return value(yyscanner, 10); }
@@ -187,25 +206,7 @@ r{num_raw_hex}		{ return raw(yyscanner); }
 "}"			{ return '}'; }
 =			{ return '='; }
 \n			{ }
-
-<mem>{
-{modifier_bp}		{ return str(yyscanner, PE_MODIFIER_BP); }
-:			{ return ':'; }
-{num_dec}		{ return value(yyscanner, 10); }
-{num_hex}		{ return value(yyscanner, 16); }
-	/*
-	 * We need to separate 'mem:' scanner part, in order to get specific
-	 * modifier bits parsed out. Otherwise we would need to handle PE_NAME
-	 * and we'd need to parse it manually. During the escape from <mem>
-	 * state we need to put the escaping char back, so we dont miss it.
-	 */
-.			{ unput(*yytext); BEGIN(INITIAL); }
-	/*
-	 * We destroy the scanner after reaching EOF,
-	 * but anyway just to be sure get back to INIT state.
-	 */
-<<EOF>>			{ BEGIN(INITIAL); }
-}
+.			{ }
 
 %%
 
-- 
1.7.11.7

[PATCH 2/8] perf tools: Factorize event parsing to be more general

[Jiri Olsa]

Making the event parsing processing more general so we
can plug in strings with special processing like formula
definition.

The formula changes are coming in following patches.

Signed-off-by: Jiri Olsa <jolsa@redhat.com>
Cc: Andi Kleen <andi@firstfloor.org>
Cc: Corey Ashford <cjashfor@linux.vnet.ibm.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Ulrich Drepper <drepper@gmail.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Stephane Eranian <eranian@google.com>
---
 tools/perf/util/parse-events.c | 21 ++++++++++++++++++
 tools/perf/util/parse-events.h | 17 +++++++++++++++
 tools/perf/util/parse-events.y | 48 +++++++++++++++++++++++++++++++-----------
 3 files changed, 74 insertions(+), 12 deletions(-)

diff --git a/tools/perf/util/parse-events.c b/tools/perf/util/parse-events.c
index 6c8bb0f..e8e9dc8 100644
--- a/tools/perf/util/parse-events.c
+++ b/tools/perf/util/parse-events.c
@@ -1238,3 +1238,24 @@ void parse_events__free_terms(struct list_head *terms)
 
 	free(terms);
 }
+
+int parse_events_config_process(struct parse_events_evlist *data,
+				struct list_head *head)
+{
+	struct parse_events_config *cfg, *h;
+
+	list_for_each_entry_safe(cfg, h, head, list) {
+		switch (cfg->type) {
+		case PARSE_EVENTS_CONFIG_EVENTS:
+			parse_events_update_lists(cfg->events, &data->list);
+			break;
+		default:
+			break;
+		}
+
+		list_del(&cfg->list);
+		free(cfg);
+	}
+
+	return 0;
+}
diff --git a/tools/perf/util/parse-events.h b/tools/perf/util/parse-events.h
index 8a48593..98810f1 100644
--- a/tools/perf/util/parse-events.h
+++ b/tools/perf/util/parse-events.h
@@ -71,6 +71,23 @@ struct parse_events_terms {
 	struct list_head *terms;
 };
 
+enum parse_events_config_type {
+	PARSE_EVENTS_CONFIG_EVENTS,
+};
+
+struct parse_events_config {
+	enum parse_events_config_type type;
+
+	union {
+		struct list_head *events;
+		void *val;
+	};
+
+	struct list_head list;
+};
+
+int parse_events_config_process(struct parse_events_evlist *data,
+				struct list_head *head);
 int parse_events__is_hardcoded_term(struct parse_events_term *term);
 int parse_events_term__num(struct parse_events_term **_term,
 			   int type_term, char *config, u64 num);
diff --git a/tools/perf/util/parse-events.y b/tools/perf/util/parse-events.y
index afc44c1..bae1879 100644
--- a/tools/perf/util/parse-events.y
+++ b/tools/perf/util/parse-events.y
@@ -30,6 +30,21 @@ static inc_group_count(struct list_head *list,
 		data->nr_groups++;
 }
 
+#define CONFIG(t, v) ({						\
+	struct parse_events_config *c = zalloc(sizeof(*c));	\
+	ABORT_ON(!c);						\
+	c->type = PARSE_EVENTS_CONFIG_ ## t;			\
+	c->val = v;						\
+	c;							\
+})
+
+#define HEAD() ({					\
+	struct list_head *h = zalloc(sizeof(*h));	\
+	ABORT_ON(!h);					\
+	INIT_LIST_HEAD(h);				\
+	h;						\
+})
+
 %}
 
 %token PE_START_EVENTS PE_START_TERMS
@@ -69,6 +84,7 @@ static inc_group_count(struct list_head *list,
 %type <head> group_def
 %type <head> group
 %type <head> groups
+%type <cfg> groups_config
 
 %union
 {
@@ -76,6 +92,7 @@ static inc_group_count(struct list_head *list,
 	u64 num;
 	struct list_head *head;
 	struct parse_events_term *term;
+	struct parse_events_config *cfg;
 }
 %%
 
@@ -88,31 +105,38 @@ start_events: groups
 {
 	struct parse_events_evlist *data = _data;
 
-	parse_events_update_lists($1, &data->list);
+	ABORT_ON(parse_events_config_process(data, $1));
 }
 
 groups:
-groups ',' group
+groups ',' groups_config
 {
-	struct list_head *list  = $1;
-	struct list_head *group = $3;
+	struct list_head *head = $1;
+	struct parse_events_config *cfg = $3;
 
-	parse_events_update_lists(group, list);
-	$$ = list;
+	list_add_tail(&cfg->list, head);
+	$$ = head;
 }
 |
-groups ',' event
+groups_config
 {
-	struct list_head *list  = $1;
-	struct list_head *event = $3;
+	struct list_head *head = HEAD();
+	struct parse_events_config *cfg = $1;
 
-	parse_events_update_lists(event, list);
-	$$ = list;
+	list_add_tail(&cfg->list, head);
+	$$ = head;
 }
-|
+
+groups_config:
 group
+{
+	$$ = CONFIG(EVENTS, $1);
+}
 |
 event
+{
+	$$ = CONFIG(EVENTS, $1);
+}
 
 group:
 group_def ':' PE_MODIFIER_EVENT
-- 
1.7.11.7

[PATCH 3/8] perf tools: Add formula-* parsing support for events

[Jiri Olsa]

Adding a possibility to specify formula-$NAME within
the -e option same way as an event.

This adds only the functionality to parse  out the formula
names and populate string array within struct perf_evlist.

The formula processing itself will come in next patches.

Signed-off-by: Jiri Olsa <jolsa@redhat.com>
Cc: Andi Kleen <andi@firstfloor.org>
Cc: Corey Ashford <cjashfor@linux.vnet.ibm.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Ulrich Drepper <drepper@gmail.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Stephane Eranian <eranian@google.com>
---
 tools/perf/util/evlist.h       |  5 +++++
 tools/perf/util/parse-events.c | 32 ++++++++++++++++++++++++++++++++
 tools/perf/util/parse-events.h |  3 +++
 tools/perf/util/parse-events.l | 10 +++++++++-
 tools/perf/util/parse-events.y | 34 ++++++++++++++++++++++++++++++++++
 5 files changed, 83 insertions(+), 1 deletion(-)

diff --git a/tools/perf/util/evlist.h b/tools/perf/util/evlist.h
index 0583d36..dbfb59f 100644
--- a/tools/perf/util/evlist.h
+++ b/tools/perf/util/evlist.h
@@ -42,6 +42,7 @@ struct perf_evlist {
 	struct thread_map *threads;
 	struct cpu_map	  *cpus;
 	struct perf_evsel *selected;
+	char		 **formulas;
 };
 
 struct perf_evsel_str_handler {
@@ -164,4 +165,8 @@ static inline void perf_mmap__write_tail(struct perf_mmap *md,
 	pc->data_tail = tail;
 }
 
+static inline bool perf_evlist__has_formulas(struct perf_evlist *evlist)
+{
+	return evlist->formulas != NULL;
+}
 #endif /* __PERF_EVLIST_H */
diff --git a/tools/perf/util/parse-events.c b/tools/perf/util/parse-events.c
index e8e9dc8..fe0a554 100644
--- a/tools/perf/util/parse-events.c
+++ b/tools/perf/util/parse-events.c
@@ -877,6 +877,7 @@ int parse_events(struct perf_evlist *evlist, const char *str)
 		int entries = data.idx - evlist->nr_entries;
 		perf_evlist__splice_list_tail(evlist, &data.list, entries);
 		evlist->nr_groups += data.nr_groups;
+		evlist->formulas = data.formulas;
 		return 0;
 	}
 
@@ -1239,6 +1240,31 @@ void parse_events__free_terms(struct list_head *terms)
 	free(terms);
 }
 
+static char **formula_add(char **f, char *new)
+{
+	int i;
+#define FORMULAS_CNT 20
+
+	if (!f) {
+		f = zalloc(sizeof(char *) * FORMULAS_CNT + 1);
+		if (!f)
+			return NULL;
+	}
+
+	for (i = 0; f[i] && (i < FORMULAS_CNT); i++);
+
+	if (i == FORMULAS_CNT) {
+		pr_err("Too many formula defined, max = %d\n",
+		       FORMULAS_CNT);
+		return NULL;
+	}
+
+	pr_debug("parse events: formula %s\n", new);
+
+	f[i] = new;
+	return f;
+}
+
 int parse_events_config_process(struct parse_events_evlist *data,
 				struct list_head *head)
 {
@@ -1249,6 +1275,12 @@ int parse_events_config_process(struct parse_events_evlist *data,
 		case PARSE_EVENTS_CONFIG_EVENTS:
 			parse_events_update_lists(cfg->events, &data->list);
 			break;
+		case PARSE_EVENTS_CONFIG_FORMULA:
+			data->formulas = formula_add(data->formulas,
+						     cfg->formula);
+			if (!data->formulas)
+				return -1;
+			break;
 		default:
 			break;
 		}
diff --git a/tools/perf/util/parse-events.h b/tools/perf/util/parse-events.h
index 98810f1..1b6c34d 100644
--- a/tools/perf/util/parse-events.h
+++ b/tools/perf/util/parse-events.h
@@ -63,6 +63,7 @@ struct parse_events_term {
 
 struct parse_events_evlist {
 	struct list_head list;
+	char **formulas;
 	int idx;
 	int nr_groups;
 };
@@ -73,6 +74,7 @@ struct parse_events_terms {
 
 enum parse_events_config_type {
 	PARSE_EVENTS_CONFIG_EVENTS,
+	PARSE_EVENTS_CONFIG_FORMULA,
 };
 
 struct parse_events_config {
@@ -80,6 +82,7 @@ struct parse_events_config {
 
 	union {
 		struct list_head *events;
+		char *formula;
 		void *val;
 	};
 
diff --git a/tools/perf/util/parse-events.l b/tools/perf/util/parse-events.l
index 5b4ef52..7d0d630 100644
--- a/tools/perf/util/parse-events.l
+++ b/tools/perf/util/parse-events.l
@@ -72,10 +72,12 @@ static int term(yyscan_t scanner, int type)
 %x mem
 %s config
 %x event
+%x formula
 
 group		[^,{}/]*[{][^}]*[}][^,{}/]*
 event_pmu	[^,{}/]+[/][^/]*[/][^,{}/]*
 event		[^,{}/]+
+formula		formula-[^,{}/]+
 
 num_dec		[0-9]+
 num_hex		0x[a-fA-F0-9]+
@@ -106,7 +108,7 @@ modifier_bp	[rwx]{1,3}
 %}
 
 <event>{
-
+{formula}	|
 {group}		{
 			BEGIN(INITIAL); yyless(0);
 		}
@@ -156,6 +158,11 @@ branch_type		{ return term(yyscanner, PARSE_EVENTS__TERM_TYPE_BRANCH_SAMPLE_TYPE
 <<EOF>>			{ BEGIN(INITIAL); }
 }
 
+<formula>{
+-			{ return '-'; }
+{name_minus}		{ BEGIN(INITIAL); return str(yyscanner, PE_FORMULA_NAME); }
+}
+
 cpu-cycles|cycles				{ return sym(yyscanner, PERF_TYPE_HARDWARE, PERF_COUNT_HW_CPU_CYCLES); }
 stalled-cycles-frontend|idle-cycles-frontend	{ return sym(yyscanner, PERF_TYPE_HARDWARE, PERF_COUNT_HW_STALLED_CYCLES_FRONTEND); }
 stalled-cycles-backend|idle-cycles-backend	{ return sym(yyscanner, PERF_TYPE_HARDWARE, PERF_COUNT_HW_STALLED_CYCLES_BACKEND); }
@@ -191,6 +198,7 @@ speculative-read|speculative-load	|
 refs|Reference|ops|access		|
 misses|miss				{ return str(yyscanner, PE_NAME_CACHE_OP_RESULT); }
 
+formula			{ BEGIN(formula); return PE_FORMULA; }
 mem:			{ BEGIN(mem); return PE_PREFIX_MEM; }
 r{num_raw_hex}		{ return raw(yyscanner); }
 {num_dec}		{ return value(yyscanner, 10); }
diff --git a/tools/perf/util/parse-events.y b/tools/perf/util/parse-events.y
index bae1879..ed2d174 100644
--- a/tools/perf/util/parse-events.y
+++ b/tools/perf/util/parse-events.y
@@ -55,6 +55,7 @@ static inc_group_count(struct list_head *list,
 %token PE_NAME_CACHE_TYPE PE_NAME_CACHE_OP_RESULT
 %token PE_PREFIX_MEM PE_PREFIX_RAW PE_PREFIX_GROUP
 %token PE_ERROR
+%token PE_FORMULA PE_FORMULA_NAME
 %type <num> PE_VALUE
 %type <num> PE_VALUE_SYM_HW
 %type <num> PE_VALUE_SYM_SW
@@ -66,6 +67,7 @@ static inc_group_count(struct list_head *list,
 %type <str> PE_MODIFIER_EVENT
 %type <str> PE_MODIFIER_BP
 %type <str> PE_EVENT_NAME
+%type <str> PE_FORMULA_NAME
 %type <num> value_sym
 %type <head> event_config
 %type <term> event_term
@@ -85,6 +87,8 @@ static inc_group_count(struct list_head *list,
 %type <head> group
 %type <head> groups
 %type <cfg> groups_config
+%type <cfg> groups_formula
+%type <str> formula
 
 %union
 {
@@ -118,6 +122,15 @@ groups ',' groups_config
 	$$ = head;
 }
 |
+groups ',' groups_formula
+{
+	struct list_head *head = $1;
+	struct parse_events_config *cfg = $3;
+
+	list_add_tail(&cfg->list, head);
+	$$ = head;
+}
+|
 groups_config
 {
 	struct list_head *head = HEAD();
@@ -126,6 +139,15 @@ groups_config
 	list_add_tail(&cfg->list, head);
 	$$ = head;
 }
+|
+groups_formula
+{
+	struct list_head *head = HEAD();
+	struct parse_events_config *cfg = $1;
+
+	list_add_tail(&cfg->list, head);
+	$$ = head;
+}
 
 groups_config:
 group
@@ -436,6 +458,18 @@ PE_TERM
 	$$ = term;
 }
 
+groups_formula:
+formula
+{
+	$$ = CONFIG(FORMULA, $1);
+}
+
+formula:
+PE_FORMULA '-' PE_FORMULA_NAME
+{
+	$$ = strdup($3);
+}
+
 sep_dc: ':' |
 
 sep_slash_dc: '/' | ':' |
-- 
1.7.11.7

[PATCH 4/8] perf tools: Add formula interface to interface formula definitions

[Jiri Olsa]

Adding formula object to interface formula definitions like:

  set {
       events {
               CY = cycles
               IN = instructions
               BR = branches
       }

       cpi      = CY / IN
       bdensity = IN / BR

       print cpi
       print bdensity
  }

The 'set' defines set of counter that share same events.
Each 'set' defines:
  events   - event string that would go into stat -e option
  counters - any number of counters based on above events

Each event and counter is defined as an assignment (via =)
to the name(tag), which could be later used within counter
formulas.

Each counter (cpi/branch-rate above) defines formula that
produces the counter number.

Formula grammar:

  expr: '-' expr       |
        expr '+' expr  |
        expr '-' expr  |
        expr '*' expr  |
        expr '/' expr  |
        value          |
        name

  where name could be any event or counter name(tag), like:
  (considering the architectural events name patchset
   is included)

  set {
        events {
                IN = instructions
                R0 = cpu/OFFCORE_RESPONSE_0,offcore_rsp=OFFCORE_RESPONSE.(DMND_IFETCH|LLC_MISS_LOCAL)/
        }

        kinst     = IN / 1000
        llc_imiss = R0 / kinst

        print llc_imiss
  }

It's possible to enable counter to be displayed via 'print'
label on separate line. Only counters marked like that
will be displayed to the user.

Interface:
  perf_formula__init
  - initialize perf_formula handler

  perf_formula__load
  - load file into the handler

  perf_formula__free
  - cleanup

  perf_formula__set
  - get 'set' handler

  perf_formula__evlist
  - update perf_evlist with needed events

  perf_formula__print
  - display output ratios

Signed-off-by: Jiri Olsa <jolsa@redhat.com>
Cc: Andi Kleen <andi@firstfloor.org>
Cc: Corey Ashford <cjashfor@linux.vnet.ibm.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Ulrich Drepper <drepper@gmail.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Stephane Eranian <eranian@google.com>
---
 tools/perf/Makefile       |  11 +
 tools/perf/util/evlist.c  |  13 ++
 tools/perf/util/evlist.h  |   4 +
 tools/perf/util/formula.c | 513 ++++++++++++++++++++++++++++++++++++++++++++++
 tools/perf/util/formula.h | 137 +++++++++++++
 tools/perf/util/formula.l | 119 +++++++++++
 tools/perf/util/formula.y | 249 ++++++++++++++++++++++
 7 files changed, 1046 insertions(+)
 create mode 100644 tools/perf/util/formula.c
 create mode 100644 tools/perf/util/formula.h
 create mode 100644 tools/perf/util/formula.l
 create mode 100644 tools/perf/util/formula.y

diff --git a/tools/perf/Makefile b/tools/perf/Makefile
index b0f164b..bcbc524 100644
--- a/tools/perf/Makefile
+++ b/tools/perf/Makefile
@@ -326,8 +326,15 @@ $(OUTPUT)util/pmu-flex.c: util/pmu.l $(OUTPUT)util/pmu-bison.c
 $(OUTPUT)util/pmu-bison.c: util/pmu.y
 	$(QUIET_BISON)$(BISON) -v util/pmu.y -d -o $(OUTPUT)util/pmu-bison.c -p perf_pmu_
 
+$(OUTPUT)util/formula-flex.c: util/formula.l util/formula.y $(OUTPUT)util/formula-bison.c
+	$(QUIET_FLEX)$(FLEX) --header-file=$(OUTPUT)util/formula-flex.h $(PARSER_DEBUG_FLEX) -t util/formula.l > $(OUTPUT)util/formula-flex.c
+
+$(OUTPUT)util/formula-bison.c: util/formula.y
+	$(QUIET_BISON)$(BISON) -v util/formula.y -d $(PARSER_DEBUG_BISON) -o $(OUTPUT)util/formula-bison.c
+
 $(OUTPUT)util/parse-events.o: $(OUTPUT)util/parse-events-flex.c $(OUTPUT)util/parse-events-bison.c
 $(OUTPUT)util/pmu.o: $(OUTPUT)util/pmu-flex.c $(OUTPUT)util/pmu-bison.c
+$(OUTPUT)util/formula.o: $(OUTPUT)util/formula-flex.c $(OUTPUT)util/formula-bison.c
 
 LIB_FILE=$(OUTPUT)libperf.a
 
@@ -420,6 +427,7 @@ LIB_H += util/intlist.h
 LIB_H += util/perf_regs.h
 LIB_H += util/unwind.h
 LIB_H += util/vdso.h
+LIB_H += util/formula.h
 LIB_H += ui/helpline.h
 LIB_H += ui/progress.h
 LIB_H += ui/util.h
@@ -492,6 +500,9 @@ LIB_OBJS += $(OUTPUT)util/rblist.o
 LIB_OBJS += $(OUTPUT)util/intlist.o
 LIB_OBJS += $(OUTPUT)util/vdso.o
 LIB_OBJS += $(OUTPUT)util/stat.o
+LIB_OBJS += $(OUTPUT)util/formula.o
+LIB_OBJS += $(OUTPUT)util/formula-flex.o
+LIB_OBJS += $(OUTPUT)util/formula-bison.o
 
 LIB_OBJS += $(OUTPUT)ui/setup.o
 LIB_OBJS += $(OUTPUT)ui/helpline.o
diff --git a/tools/perf/util/evlist.c b/tools/perf/util/evlist.c
index f7c7278..ec69b2b 100644
--- a/tools/perf/util/evlist.c
+++ b/tools/perf/util/evlist.c
@@ -863,3 +863,16 @@ size_t perf_evlist__fprintf(struct perf_evlist *evlist, FILE *fp)
 
 	return printed + fprintf(fp, "\n");;
 }
+
+struct perf_evsel*
+perf_evlist__find_evsel_name(struct perf_evlist *evlist, char *name)
+{
+	struct perf_evsel *evsel;
+
+	list_for_each_entry(evsel, &evlist->entries, node) {
+		if (strstr(perf_evsel__name(evsel), name))
+			return evsel;
+	}
+
+	return NULL;
+}
diff --git a/tools/perf/util/evlist.h b/tools/perf/util/evlist.h
index dbfb59f..abf76a4 100644
--- a/tools/perf/util/evlist.h
+++ b/tools/perf/util/evlist.h
@@ -169,4 +169,8 @@ static inline bool perf_evlist__has_formulas(struct perf_evlist *evlist)
 {
 	return evlist->formulas != NULL;
 }
+
+struct perf_evsel*
+perf_evlist__find_evsel_name(struct perf_evlist *evlist, char *name);
+
 #endif /* __PERF_EVLIST_H */
diff --git a/tools/perf/util/formula.c b/tools/perf/util/formula.c
new file mode 100644
index 0000000..2edc720
--- /dev/null
+++ b/tools/perf/util/formula.c
@@ -0,0 +1,513 @@
+
+#include <linux/compiler.h>
+#include <stdio.h>
+#include "stat.h"
+#include "parse-events.h"
+#include "formula.h"
+#include "formula-bison.h"
+#define YY_EXTRA_TYPE int
+#include "formula-flex.h"
+
+#define FORMULA_SET_ALL ((void *) -1)
+
+#ifdef PARSER_DEBUG
+extern int perf_formula_debug;
+#endif
+int perf_formula_parse(void *_data, void *scanner);
+
+void perf_formula__init(struct perf_formula *f)
+{
+	memset(f, 0x0, sizeof(*f));
+	INIT_LIST_HEAD(&f->head_files);
+}
+
+static int scanner_expr(const char *str, void *data)
+{
+	YY_BUFFER_STATE buffer;
+	void *scanner;
+	int ret;
+
+	ret = perf_formula_lex_init_extra(PF_START_EXPR, &scanner);
+	if (ret)
+		return ret;
+
+	buffer = perf_formula__scan_string(str, scanner);
+
+#ifdef PARSER_DEBUG
+	perf_formula_debug = 1;
+#endif
+	ret = perf_formula_parse(data, scanner);
+
+	perf_formula__flush_buffer(buffer, scanner);
+	perf_formula__delete_buffer(buffer, scanner);
+	perf_formula_lex_destroy(scanner);
+	return ret;
+}
+
+static int scanner_config(FILE *file, void *data)
+{
+	void *scanner;
+	int ret;
+
+	ret = perf_formula_lex_init_extra(PF_START_CONFIG, &scanner);
+	if (ret)
+		return ret;
+
+	perf_formula_set_in(file, scanner);
+
+#ifdef PARSER_DEBUG
+	perf_formula_debug = 1;
+#endif
+	ret = perf_formula_parse(data, scanner);
+
+	perf_formula_lex_destroy(scanner);
+	return ret;
+}
+
+static int config_parse(struct perf_formula_file *file)
+{
+	FILE *f;
+	int ret;
+
+	f = fopen(file->path, "r");
+	if (!f)
+		return -EINVAL;
+
+	ret = scanner_config(f, file);
+
+	fclose(f);
+	return ret;
+}
+
+static int counter_init(struct perf_formula_counter *counter)
+{
+	struct perf_formula_expr expr = {
+		.test_only = true,
+	};
+
+	return scanner_expr(counter->formula, &expr);
+}
+
+static int set_init(struct perf_formula_set *set)
+{
+	struct perf_formula_counter *counter;
+	int ret = 0;
+
+	list_for_each_entry(counter, &set->head_counters, list) {
+		ret = counter_init(counter);
+		if (ret)
+			break;
+
+		counter->set = set;
+	}
+
+	return ret;
+}
+
+static int file_init(struct perf_formula_file *file)
+{
+	struct perf_formula_set *set;
+	int ret;
+
+	ret = config_parse(file);
+
+	list_for_each_entry(set, &file->head_sets, list) {
+		ret = set_init(set);
+		if (ret)
+			break;
+	}
+
+	return ret;
+}
+
+static void file_free(struct perf_formula_file *file)
+{
+	struct perf_formula_set *set;
+
+	list_for_each_entry(set, &file->head_sets, list) {
+		struct perf_formula_counter *counter;
+
+		list_for_each_entry(counter, &set->head_counters, list)
+			free(counter);
+
+		free(set);
+	}
+
+	free(file->path);
+	free(file);
+}
+
+int perf_formula__load(struct perf_formula *f, char *path)
+{
+	struct perf_formula_file *file;
+	int ret;
+
+	file = zalloc(sizeof(*file));
+	if (!file)
+		return -ENOMEM;
+
+	INIT_LIST_HEAD(&file->list);
+	INIT_LIST_HEAD(&file->head_sets);
+	file->path = strdup(path);
+
+	ret = file_init(file);
+	if (ret)
+		file_free(file);
+	else
+		list_add_tail(&file->list, &f->head_files);
+
+	return ret;
+}
+
+int perf_formula__free(struct perf_formula *f)
+{
+	struct perf_formula_file *file;
+
+	list_for_each_entry(file, &f->head_files, list)
+		file_free(file);
+
+	return 0;
+}
+
+enum {
+	CB_NEXT,
+	CB_OK,
+	CB_FAIL,
+};
+
+typedef int (*set_cb)(struct perf_formula_set *set, void *data);
+
+static int for_each_set(struct perf_formula *formula,
+			set_cb cb, void *data)
+{
+	struct perf_formula_file *file;
+
+	list_for_each_entry(file, &formula->head_files, list) {
+		struct perf_formula_set *set;
+
+		list_for_each_entry(set, &file->head_sets, list) {
+			int ret = cb(set, data);
+
+			if (ret == CB_NEXT)
+				continue;
+			else if (ret == CB_OK)
+				return 0;
+			else if (ret == CB_FAIL)
+				return -1;
+		}
+	}
+
+	return 0;
+}
+
+struct find_set_data {
+	struct perf_formula_set *set;
+	char *name;
+};
+
+static int find_set_cb(struct perf_formula_set *set, void *data)
+{
+	struct find_set_data *d = data;
+
+	if (strcmp(set->name, d->name))
+		return CB_NEXT;
+
+	d->set = set;
+	return CB_OK;
+}
+
+struct perf_formula_set*
+perf_formula__set(struct perf_formula *f, char *name)
+{
+	struct find_set_data data = {
+		.name = name,
+	};
+
+	if (!strcmp(name, "all"))
+		return FORMULA_SET_ALL;
+
+	if (for_each_set(f, find_set_cb, &data))
+		return NULL;
+
+	return data.set;
+}
+
+static int eval_counter(struct perf_formula_counter *counter,
+			struct perf_formula_expr *expr)
+{
+	int ret;
+
+	ret = scanner_expr(counter->formula, expr);
+	if (!ret) {
+		counter->result = expr->result;
+		pr_debug2("formula counter eval %s = %F\n",
+			 counter->name, counter->result);
+	}
+
+	return ret;
+}
+
+static int eval_set_cb(struct perf_formula_set *set, void *data)
+{
+	struct perf_formula_expr *expr = data;
+	struct perf_formula_counter *counter;
+
+	expr->set = set;
+
+	pr_debug2("formula eval %s\n", set->name);
+
+	list_for_each_entry(counter, &set->head_counters, list) {
+		if (eval_counter(counter, expr)) {
+			pr_err("failed to eval counter %s\n",
+			        counter->name);
+			return CB_FAIL;
+		}
+
+		if (expr->print && counter->print)
+			fprintf(expr->file,
+				"%'18.8F %-25s\n",
+				counter->result, counter->name);
+	}
+
+	return CB_NEXT;
+}
+
+int perf_formula__eval(struct perf_formula *f,
+		       struct perf_formula_set *set,
+		       struct perf_formula_expr *expr)
+{
+	if (set != FORMULA_SET_ALL)
+		return eval_set_cb(set, expr);
+
+	return for_each_set(f, eval_set_cb, expr);
+}
+
+int perf_formula__print(FILE *file,
+			struct perf_formula *f,
+			struct perf_formula_set *set,
+			struct perf_evlist *evlist,
+			struct perf_formula_value **values)
+{
+	struct perf_formula_expr expr = {
+		.set = set,
+		.evlist = evlist,
+		.values = values,
+		.print = true,
+		.file = file,
+	};
+
+	return perf_formula__eval(f, set, &expr);
+}
+
+static int set_evlist(struct perf_formula_set *set,
+		      struct perf_evlist *evlist)
+{
+	struct perf_evlist *evlist_tmp = evlist;
+
+	if (set->loaded)
+		return 0;
+
+	if (parse_events(evlist_tmp, set->events))
+		return -1;
+
+	set->loaded = true;
+
+	return 0;
+}
+
+static int evlist_cb(struct perf_formula_set *set, void *data)
+{
+       struct perf_evlist *evlist = data;
+
+	if (set_evlist(set, evlist))
+		return CB_FAIL;
+
+	return CB_NEXT;
+}
+
+int perf_formula__evlist(struct perf_formula *f,
+			 struct perf_formula_set *set,
+			 struct perf_evlist *evlist)
+{
+	if (set != FORMULA_SET_ALL)
+		return set_evlist(set, evlist);
+
+	return for_each_set(f, evlist_cb, (void *) evlist);
+}
+
+static struct perf_formula_counter*
+set_counter(struct perf_formula_set *set, struct perf_formula_config *config)
+{
+	struct perf_formula_counter *counter;
+
+	counter = zalloc(sizeof(*counter));
+	if (!counter)
+		return NULL;
+
+	INIT_LIST_HEAD(&counter->list);
+	counter->name    = config->ass.name;
+	counter->formula = config->ass.value;
+	counter->set     = set;
+	counter->print   = false;
+
+	list_add_tail(&counter->list, &set->head_counters);
+	return counter;
+}
+
+static int
+set_events(struct perf_formula_set *set, struct perf_formula_config *config)
+{
+	struct perf_formula_config *event;
+#define MAX_EVENTS 4096
+	char buf[MAX_EVENTS];
+
+	/* grouped by default */
+	buf[0] = '{';
+	buf[1] = 0x0;
+
+	list_for_each_entry(event, &config->events, list) {
+		struct perf_formula_event *e;
+
+		e = zalloc(sizeof(*e));
+		if (!e)
+			goto out;
+
+		INIT_LIST_HEAD(&e->list);
+		e->name   = event->ass.name;
+		e->config = event->ass.value;
+		list_add_tail(&e->list, &set->head_events);
+
+		/* TODO length check */
+		strcat(buf, event->ass.value);
+
+		if (!list_is_last(&event->list, &config->events))
+			strcat(buf, ",");
+	}
+
+	strcat(buf, "}");
+	set->events = strdup(buf);
+	return 0;
+
+ out:
+	/* TODO free everything ;-) */
+	return -ENOMEM;
+}
+
+static int
+set_print(struct perf_formula_set *set, struct perf_formula_config *config)
+{
+	struct perf_formula_counter *counter;
+
+	list_for_each_entry(counter, &set->head_counters, list) {
+		if (!strcmp(counter->name, config->print)) {
+			counter->print = true;
+			return 0;
+		}
+	}
+
+	return -1;
+}
+
+struct perf_formula_set*
+perf_formula_set__new(char *name, struct list_head *head)
+{
+	struct perf_formula_set *set;
+	struct perf_formula_config *config;
+
+	set = zalloc(sizeof(*set));
+	if (!set)
+		return NULL;
+
+	INIT_LIST_HEAD(&set->list);
+	INIT_LIST_HEAD(&set->head_counters);
+	INIT_LIST_HEAD(&set->head_events);
+
+	list_for_each_entry(config, head, list) {
+		switch (config->type) {
+		case PERF_FORMULA_CONFIG_EVENTS:
+			if (set_events(set, config))
+				goto out;
+			break;
+
+		case PERF_FORMULA_CONFIG_PRINT:
+			if (set_print(set, config))
+				goto out;
+			break;
+
+		case PERF_FORMULA_CONFIG_ASS:
+			if (!set_counter(set, config))
+				goto out;
+			break;
+
+		default:
+			BUG_ON(1);
+		}
+	}
+
+	set->name = strdup(name);
+	return set;
+
+ out:
+	free(set);
+	return NULL;
+}
+
+static struct perf_evsel*
+resolve_evsel(struct perf_formula_expr *expr, char *name)
+{
+	struct perf_formula_set *set = expr->set;
+	struct perf_formula_event *e;
+
+	list_for_each_entry(e, &set->head_events, list) {
+		if (strcmp(e->name, name))
+			continue;
+
+		return perf_evlist__find_evsel_name(expr->evlist, e->config);
+	}
+
+	return NULL;
+}
+
+static struct perf_formula_counter*
+resolve_counter(struct perf_formula_expr *expr, char *name)
+{
+	struct perf_formula_set *set = expr->set;
+	struct perf_formula_counter *counter;
+
+	list_for_each_entry(counter, &set->head_counters, list) {
+		if (!strcmp(counter->name, name))
+			return counter;
+	}
+
+	return NULL;
+}
+
+double perf_formula_expr__resolve(struct perf_formula_expr *expr,
+				  char *name)
+{
+	struct perf_evsel *evsel;
+	double result = 0;
+
+	if (expr->test_only)
+		return 1;
+
+	evsel = resolve_evsel(expr, name);
+	if (evsel) {
+		struct perf_counts *counts = evsel->counts;
+		result = counts->aggr.val;
+	} else {
+		struct perf_formula_counter *counter;
+
+		counter = resolve_counter(expr, name);
+		if (counter)
+			result = counter->result;
+		else {
+			pr_err("formula: failed to resolve '%s'\n",
+			       name);
+			result = -1;
+		}
+	}
+
+	pr_debug2("formula resolve %s = %F\n", name, result);
+	return result;
+}
diff --git a/tools/perf/util/formula.h b/tools/perf/util/formula.h
new file mode 100644
index 0000000..90325ec
--- /dev/null
+++ b/tools/perf/util/formula.h
@@ -0,0 +1,137 @@
+#ifndef __PERF_FORMULA
+#define __PERF_FORMULA
+
+/*
+ * format:
+ * set {
+ *      events {
+ *              CY = cycles
+ *              IN = instructions
+ *              BR = branches
+ *      }
+ *
+ *      cpi      = CY / IN
+ *      bdensity = IN / BR
+ *
+ *      print cpi
+ *      print bdensity
+ * }
+ *
+ * TODO add following syntax:
+ *      print cpi = CY / IN
+ */
+
+#include <linux/list.h>
+#include "evlist.h"
+
+struct perf_formula {
+	struct list_head head_files;
+};
+
+struct perf_formula_file {
+	char *path;
+
+	struct list_head head_sets;
+	struct list_head list;
+};
+
+struct perf_formula_set {
+	char *name;
+	char *events;
+	bool  loaded;
+
+	struct list_head head_events;
+	struct list_head head_counters;
+	struct list_head list;
+};
+
+struct perf_formula_event {
+	char *name;
+	char *config;
+
+	struct perf_evsel *evsel;
+	struct list_head list;
+};
+
+struct perf_formula_counter {
+	char  *name;
+	char  *formula;
+	bool   print;
+	double result;
+
+	struct perf_formula_set *set;
+	struct list_head list;
+};
+
+struct perf_formula_value {
+	char   *name;
+	double *ptr;
+};
+
+struct perf_formula_expr {
+	bool   test_only;
+	bool   print;
+
+	double result;
+	FILE  *file;
+
+	struct perf_evlist         *evlist;
+	struct perf_formula_set    *set;
+	struct perf_formula_value **values;
+};
+
+struct perf_formula_ass {
+	char *name;
+	char *value;
+};
+
+struct perf_formula_config {
+	enum {
+		PERF_FORMULA_CONFIG_EVENTS,
+		PERF_FORMULA_CONFIG_PRINT,
+		PERF_FORMULA_CONFIG_ASS,
+	} type;
+
+	union {
+		struct perf_formula_counter *counter;
+		struct perf_formula_ass      ass;
+		struct list_head             events;
+		char                        *print;
+	};
+
+	struct list_head list;
+};
+
+
+void perf_formula__init(struct perf_formula *f);
+
+int perf_formula__load(struct perf_formula *f, char *path);
+int perf_formula__free(struct perf_formula *f);
+
+struct perf_formula_set*
+perf_formula__set(struct perf_formula *f, char *name);
+
+int perf_formula__evlist(struct perf_formula *f,
+			 struct perf_formula_set *set,
+			 struct perf_evlist *evlist);
+
+int perf_formula__print(FILE *file,
+			struct perf_formula *f,
+			struct perf_formula_set *set,
+			struct perf_evlist *evlist,
+			struct perf_formula_value **values);
+
+struct perf_formula_counter*
+perf_formula_counter__new(char *name, struct list_head *head);
+
+int perf_formula__eval(struct perf_formula *f,
+		       struct perf_formula_set *set,
+		       struct perf_formula_expr *expr);
+
+struct perf_formula_set*
+perf_formula_set__new(char *name, struct list_head *head);
+
+double perf_formula_expr__resolve(struct perf_formula_expr *expr,
+				  char *name);
+
+#endif /* __PERF_FORMULA */
diff --git a/tools/perf/util/formula.l b/tools/perf/util/formula.l
new file mode 100644
index 0000000..3e83a0e
--- /dev/null
+++ b/tools/perf/util/formula.l
@@ -0,0 +1,119 @@
+
+%option reentrant
+%option bison-bridge
+%option prefix="perf_formula_"
+%option stack
+
+%{
+#include "formula-bison.h"
+#include "formula.h"
+
+char *perf_formula_get_text(yyscan_t yyscanner);
+YYSTYPE *perf_formula_get_lval(yyscan_t yyscanner);
+
+static int __value(YYSTYPE *yylval, char *str, int base, int token)
+{
+	double num;
+
+	errno = 0;
+	num = strtoull(str, NULL, base);
+	if (errno)
+		return PF_ERROR;
+
+	yylval->num = num;
+	return token;
+}
+
+static int value(yyscan_t scanner, int base)
+{
+	YYSTYPE *yylval = perf_formula_get_lval(scanner);
+	char *text = perf_formula_get_text(scanner);
+
+	return __value(yylval, text, base, PF_VALUE);
+}
+
+static int str(yyscan_t scanner, int token)
+{
+	YYSTYPE *yylval = perf_formula_get_lval(scanner);
+	char *text = perf_formula_get_text(scanner);
+
+	yylval->str = strdup(text);
+	return token;
+}
+
+%}
+
+num_dec		[0-9]+
+num_hex		0x[a-fA-F0-9]+
+name		[a-zA-Z_*?][a-zA-Z0-9_*?\.-]*
+
+%x config
+%x expr
+%x config_eoln_str
+
+%%
+
+%{
+	{
+		int start_token;
+
+		start_token = perf_formula_get_extra(yyscanner);
+
+		if (start_token == PF_START_CONFIG)
+			BEGIN(config);
+		else if (start_token == PF_START_EXPR)
+			BEGIN(expr);
+
+		if (start_token) {
+			perf_formula_set_extra(NULL, yyscanner);
+			return start_token;
+		}
+	}
+%}
+
+<config>{
+"{"		{ return '{'; }
+"}"		{ return '}'; }
+"="		{ BEGIN(config_eoln_str); return '='; }
+
+events		{ return PF_EVENTS; }
+print		{ return PF_PRINT; }
+{name}		{ return str(yyscanner, PF_NAME); }
+
+\n		{ }
+.		{ }
+}
+
+<config_eoln_str>{
+[^\n]+		{
+			str(yyscanner, PF_EOLN_STR);
+			BEGIN(config);
+			return PF_EOLN_STR;
+                }
+
+<<EOF>>		{
+			BEGIN(config);
+		}
+}
+
+<expr>{
+"*"		{ return '*'; }
+"-"		{ return '-'; }
+"+"		{ return '+'; }
+"/"		{ return '/'; }
+"("		{ return '('; }
+")"		{ return ')'; }
+
+{name}		{ return str(yyscanner, PF_NAME); }
+{num_dec}	{ return value(yyscanner, 10); }
+{num_hex}	{ return value(yyscanner, 16); }
+
+.		{ }
+}
+
+%%
+
+int perf_formula_wrap(void *scanner __maybe_unused)
+{
+	return 1;
+}
diff --git a/tools/perf/util/formula.y b/tools/perf/util/formula.y
new file mode 100644
index 0000000..955264c
--- /dev/null
+++ b/tools/perf/util/formula.y
@@ -0,0 +1,249 @@
+%pure-parser
+%name-prefix "perf_formula_"
+%parse-param {void *_data}
+%parse-param {void *scanner}
+%lex-param {void* scanner}
+
+%left '+' '-' '*' '/'
+
+%{
+
+#define YYDEBUG 1
+
+#include "util.h"
+#include "formula.h"
+#include "formula-bison.h"
+
+extern int formula_lex(YYSTYPE* lvalp, void* scanner);
+
+
+#define ABORT() YYABORT
+
+#define ABORT_ON(val) \
+do { \
+	if (val) \
+		YYABORT; \
+} while (0)
+
+#define HEAD() ({						\
+	struct list_head *__head = zalloc(sizeof(*__head));	\
+	ABORT_ON(!__head);					\
+	INIT_LIST_HEAD(__head);					\
+	__head;							\
+})
+
+#define CONFIG() ({						\
+	struct perf_formula_config *__config;			\
+	__config = zalloc(sizeof(*__config));			\
+	ABORT_ON(!__config);					\
+	INIT_LIST_HEAD(&__config->list);			\
+	__config;						\
+})
+
+%}
+
+%token PF_START_CONFIG PF_START_EXPR
+%token PF_NAME
+%token PF_VALUE
+%token PF_FORMULA
+%token PF_DESC
+%token PF_EVENTS
+%token PF_EOLN_STR
+%token PF_ERROR
+%token PF_PRINT
+
+%type <str> PF_NAME
+%type <num> PF_VALUE
+%type <str> PF_EOLN_STR
+%type <head> set_def
+%type <config> set_token
+%type <config> events_def
+%type <config> ass
+%type <num> expr
+
+%union
+{
+	char *str;
+	double num;
+	struct list_head *head;
+	struct config *config;
+	struct perf_formula_counter *counter;
+}
+
+%%
+
+start:
+PF_START_CONFIG start_config
+|
+PF_START_EXPR start_expr
+
+start_config: sets
+
+sets:
+sets set | set
+
+set:
+PF_NAME '{' set_def '}'
+{
+	struct perf_formula_file *file = _data;
+	struct perf_formula_set *set;
+
+	set = perf_formula_set__new($1, $3);
+	ABORT_ON(!set);
+
+	list_add_tail(&set->list, &file->head_sets);
+}
+
+set_def:
+set_def set_token
+{
+	struct list_head *head = $1;
+	struct perf_formula_config *config = $2;
+
+	list_add_tail(&config->list, head);
+	$$ = head;
+}
+|
+set_token
+{
+	struct list_head *head = HEAD();
+	struct perf_formula_config *config = $1;
+
+	list_add_tail(&config->list, head);
+	$$ = head;
+}
+
+set_token:
+PF_EVENTS '{' events_def '}'
+{
+	$$ = $3;
+}
+|
+PF_PRINT PF_NAME
+{
+	struct perf_formula_config *config = CONFIG();
+
+	config->type  = PERF_FORMULA_CONFIG_PRINT;
+	config->print = strdup($2);
+
+	$$ = config;
+}
+|
+ass
+
+/*
+ * TODO add 'print ass' processing in here, like:
+ * ...
+ * |
+ * PF_PRINT ass
+ * {
+ * ...
+ * }
+ */
+
+events_def:
+events_def ass
+{
+	struct perf_formula_config *config = $1;
+	struct perf_formula_config *ass    = $2;
+
+	list_add_tail(&ass->list, &config->events);
+
+	$$ = config;
+}
+|
+ass
+{
+	struct perf_formula_config *config = CONFIG();
+	struct perf_formula_config *ass    = $1;
+
+	config->type = PERF_FORMULA_CONFIG_EVENTS;
+	INIT_LIST_HEAD(&config->events);
+	list_add_tail(&ass->list, &config->events);
+
+	$$ = config;
+}
+
+ass:
+PF_NAME '=' PF_EOLN_STR
+{
+	struct perf_formula_config *config = CONFIG();
+
+	config->type = PERF_FORMULA_CONFIG_ASS;
+	config->ass.name  = strdup($1);
+	config->ass.value = strdup($3);
+
+	$$ = config;
+}
+
+start_expr:
+expr
+{
+	struct perf_formula_expr *expr = _data;
+
+	expr->result = $1;
+}
+
+expr:
+PF_VALUE
+{
+	$$ = $1;
+	pr_debug2("value %F\n", $$);
+}
+|
+PF_NAME
+{
+	$$ = perf_formula_expr__resolve(_data, $1);
+}
+|
+'-' expr
+{
+	$$ = - $2;
+	pr_debug2("%F = - %F\n", $$, $2);
+}
+|
+expr '+' expr
+{
+	$$ = $1 + $3;
+	pr_debug2("%F = %F + %F\n", $$, $1, $3);
+}
+|
+expr '-' expr
+{
+	$$ = $1 - $3;
+	pr_debug2("%F = %F - %F\n", $$, $1, $3);
+}
+|
+expr '*' expr
+{
+	$$ = $1 * $3;
+	pr_debug2("%F = %F * %F\n", $$, $1, $3);
+}
+|
+expr '/' expr
+{
+	if (!$3) {
+		pr_debug2("formula division by zero\n");
+		struct perf_formula_expr *expr = _data;
+		expr->result = 0.0;
+		YYACCEPT;
+	}
+
+
+	$$ = $1 / $3;
+	pr_debug2("%F = %F / %F\n", $$, $1, $3);
+}
+|
+'(' expr ')'
+{
+	$$ = $2;
+	pr_debug2("( %F )\n", $$);
+}
+
+%%
+
+void perf_formula_error(void *data __maybe_unused,
+			void *scanner __maybe_unused,
+			char const *msg __maybe_unused)
+{
+}
-- 
1.7.11.7

[PATCH 5/8] perf tools: Add support to preload default formulas

[Jiri Olsa]

Adding support to read all formula definition files
under common locations:

  'formula'
    - under perf developement tree
  'PREFIX/PERF_EXEC_PATH/formulas'
    - when installed

Plus adding very basic formula definitions for CPI
and branch related counters.

Signed-off-by: Jiri Olsa <jolsa@redhat.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Corey Ashford <cjashfor@linux.vnet.ibm.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Andi Kleen <andi@firstfloor.org>
Cc: David Ahern <dsahern@gmail.com>
Cc: Ulrich Drepper <drepper@gmail.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Stephane Eranian <eranian@google.com>
---
 tools/perf/Makefile              |  8 ++++++
 tools/perf/formulas/default.conf | 26 ++++++++++++++++++++
 tools/perf/util/formula.c        | 53 ++++++++++++++++++++++++++++++++++++++++
 tools/perf/util/formula.h        |  3 +++
 4 files changed, 90 insertions(+)
 create mode 100644 tools/perf/formulas/default.conf

diff --git a/tools/perf/Makefile b/tools/perf/Makefile
index bcbc524..c35bb7f 100644
--- a/tools/perf/Makefile
+++ b/tools/perf/Makefile
@@ -1010,6 +1010,12 @@ $(OUTPUT)util/exec_cmd.o: util/exec_cmd.c $(OUTPUT)PERF-CFLAGS
 		'-DPREFIX="$(prefix_SQ)"' \
 		$<
 
+$(OUTPUT)util/formula.o: util/formula.c $(OUTPUT)PERF-CFLAGS
+	$(QUIET_CC)$(CC) -o $@ -c $(ALL_CFLAGS) \
+		'-DPERF_EXEC_PATH="$(perfexecdir_SQ)"' \
+		'-DPREFIX="$(prefix_SQ)"' \
+		$<
+
 $(OUTPUT)tests/attr.o: tests/attr.c $(OUTPUT)PERF-CFLAGS
 	$(QUIET_CC)$(CC) -o $@ -c $(ALL_CFLAGS) \
 		'-DBINDIR="$(bindir_SQ)"' -DPYTHON='"$(PYTHON_WORD)"' \
@@ -1201,6 +1207,8 @@ install-bin: all
 	$(INSTALL) tests/attr.py '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests'
 	$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/attr'
 	$(INSTALL) tests/attr/* '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/attr'
+	$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/formulas'
+	$(INSTALL) -m 644 formulas/* $(DESTDIR_SQ)$(perfexec_instdir_SQ)/formulas
 
 install: install-bin try-install-man
 
diff --git a/tools/perf/formulas/default.conf b/tools/perf/formulas/default.conf
new file mode 100644
index 0000000..56712d6
--- /dev/null
+++ b/tools/perf/formulas/default.conf
@@ -0,0 +1,26 @@
+cpi {
+	events {
+		CY = cycles:u
+		IN = instructions:u
+	}
+
+	cpi = CY / IN
+
+	print cpi
+}
+
+branch {
+	events {
+		IN = instructions:u
+		BI = branch-instructions:u
+		BM = branch-misses:u
+	}
+
+	branch-rate       = BI / IN
+	branch-miss-rate  = BM / IN
+	branch-miss-ratio = BM / BI
+
+	print branch-rate
+	print branch-miss-rate
+	print branch-miss-ratio
+}
diff --git a/tools/perf/util/formula.c b/tools/perf/util/formula.c
index 2edc720..b9d95b2 100644
--- a/tools/perf/util/formula.c
+++ b/tools/perf/util/formula.c
@@ -159,6 +159,35 @@ int perf_formula__load(struct perf_formula *f, char *path)
 	return ret;
 }
 
+int perf_formula__load_dir(struct perf_formula *f, char *path)
+{
+	struct dirent *ent;
+	DIR *dir;
+	int ret = 0;
+
+	dir = opendir(path);
+	if (!dir) {
+		pr_err("formula: can't open dir '%s' - %s\n",
+		       path, strerror(errno));
+		return -1;
+	}
+
+	while (!ret && (ent = readdir(dir))) {
+		char file[PATH_MAX];
+
+		if (!strcmp(ent->d_name, ".") ||
+		    !strcmp(ent->d_name, ".."))
+			continue;
+
+		scnprintf(file, PATH_MAX, "%s/%s", path, ent->d_name);
+
+		ret = perf_formula__load(f, file);
+	}
+
+	closedir(dir);
+	return ret;
+}
+
 int perf_formula__free(struct perf_formula *f)
 {
 	struct perf_formula_file *file;
@@ -511,3 +540,27 @@ double perf_formula_expr__resolve(struct perf_formula_expr *expr,
 	pr_debug2("formula resolve %s = %F\n", name, result);
 	return result;
 }
+
+
+__attribute__((weak))
+int perf_formula__preload_arch(struct perf_formula *f __maybe_unused)
+{
+	return 0;
+}
+
+int perf_formula__preload(struct perf_formula *f)
+{
+	struct stat st;
+	int ret = 0;
+
+	if (!lstat("./formulas", &st))
+		ret = perf_formula__load_dir(f, (char *) "./formulas");
+	else {
+		char path[PATH_MAX];
+
+		scnprintf(path, PATH_MAX, "%s/%s/formulas/", PREFIX, PERF_EXEC_PATH);
+		ret = perf_formula__load_dir(f, path);
+	}
+
+	return ret ? ret : perf_formula__preload_arch(f);
+}
diff --git a/tools/perf/util/formula.h b/tools/perf/util/formula.h
index 90325ec..b6437d8 100644
--- a/tools/perf/util/formula.h
+++ b/tools/perf/util/formula.h
@@ -106,6 +106,7 @@ struct perf_formula_config {
 void perf_formula__init(struct perf_formula *f);
 
 int perf_formula__load(struct perf_formula *f, char *path);
+int perf_formula__load_dir(struct perf_formula *f, char *path);
 int perf_formula__free(struct perf_formula *f);
 
 struct perf_formula_set*
@@ -134,4 +135,6 @@ perf_formula_set__new(char *name, struct list_head *head);
 double perf_formula_expr__resolve(struct perf_formula_expr *expr,
 				  char *name);
 
+int perf_formula__preload(struct perf_formula *f);
+int perf_formula__preload_arch(struct perf_formula *f);
 #endif /* __PERF_FORMULA */
-- 
1.7.11.7

[PATCH 6/8] perf tests: Add automated tests for formula object

[Jiri Olsa]

Adding automated tests to test basic formula
object definitions and interface.

Signed-off-by: Jiri Olsa <jolsa@redhat.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Corey Ashford <cjashfor@linux.vnet.ibm.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Andi Kleen <andi@firstfloor.org>
Cc: David Ahern <dsahern@gmail.com>
Cc: Ulrich Drepper <drepper@gmail.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Stephane Eranian <eranian@google.com>
---
 tools/perf/Makefile             |   1 +
 tools/perf/tests/builtin-test.c |   4 +
 tools/perf/tests/formula.c      | 183 ++++++++++++++++++++++++++++++++++++++++
 tools/perf/tests/parse-events.c |   8 --
 tools/perf/tests/tests.h        |  10 +++
 5 files changed, 198 insertions(+), 8 deletions(-)
 create mode 100644 tools/perf/tests/formula.c

diff --git a/tools/perf/Makefile b/tools/perf/Makefile
index c35bb7f..aac10d0 100644
--- a/tools/perf/Makefile
+++ b/tools/perf/Makefile
@@ -532,6 +532,7 @@ LIB_OBJS += $(OUTPUT)tests/bp_signal.o
 LIB_OBJS += $(OUTPUT)tests/bp_signal_overflow.o
 LIB_OBJS += $(OUTPUT)tests/task-exit.o
 LIB_OBJS += $(OUTPUT)tests/sw-clock.o
+LIB_OBJS += $(OUTPUT)tests/formula.o
 
 BUILTIN_OBJS += $(OUTPUT)builtin-annotate.o
 BUILTIN_OBJS += $(OUTPUT)builtin-bench.o
diff --git a/tools/perf/tests/builtin-test.c b/tools/perf/tests/builtin-test.c
index 0918ada..8c48adf 100644
--- a/tools/perf/tests/builtin-test.c
+++ b/tools/perf/tests/builtin-test.c
@@ -94,6 +94,10 @@ static struct test {
 		.func = test__sw_clock_freq,
 	},
 	{
+		.desc = "Test formula interface",
+		.func = test__formula,
+	},
+	{
 		.func = NULL,
 	},
 };
diff --git a/tools/perf/tests/formula.c b/tools/perf/tests/formula.c
new file mode 100644
index 0000000..db2e71c
--- /dev/null
+++ b/tools/perf/tests/formula.c
@@ -0,0 +1,183 @@
+
+#include <linux/kernel.h>
+#include <linux/compiler.h>
+#include <stdlib.h>
+#include <unistd.h>
+#include "tests.h"
+#include "formula.h"
+#include "evlist.h"
+#include "evsel.h"
+
+static void file_put(const char *file)
+{
+	unlink(file);
+}
+
+static char *file_get(const char *data, size_t size)
+{
+	char file[] = "/tmp/perf-tests-formula-XXXXXX";
+	bool failed = false;
+	int fd;
+
+	fd = mkstemp(file);
+	if (fd < 0) {
+		pr_debug("failed to create temp file\n");
+		return NULL;
+	}
+
+	if (size != (size_t) write(fd, data, size))
+		failed = true;
+
+	close(fd);
+
+	if (failed)
+		file_put(file);
+
+	return failed ? NULL : file;
+}
+
+static int __test_basics(char *file)
+{
+	struct perf_formula fml;
+	struct perf_formula_set *set;
+	struct perf_formula_counter *counter;
+	struct perf_evlist *evlist;
+	struct perf_evsel *evsel;
+	struct perf_counts *counts;
+	struct perf_formula_expr expr;
+	int ret;
+
+	pr_debug("file %s\n", file);
+
+	perf_formula__init(&fml);
+
+	ret = perf_formula__load(&fml, file);
+	TEST_ASSERT_VAL("failed to load formula", !ret);
+
+	set = perf_formula__set(&fml, (char *) "set");
+	TEST_ASSERT_VAL("failed to load set", set);
+	TEST_ASSERT_VAL("wrong set name", !strcmp(set->name, "set"));
+
+	evlist = perf_evlist__new();
+	TEST_ASSERT_VAL("failed to create evlist", evlist);
+
+	ret = perf_formula__evlist(&fml, set, evlist);
+	TEST_ASSERT_VAL("failed to load evlist with set", !ret);
+
+	evsel = perf_evlist__first(evlist);
+	TEST_ASSERT_VAL("wrong type", PERF_TYPE_HARDWARE == evsel->attr.type);
+	TEST_ASSERT_VAL("wrong config",
+		PERF_COUNT_HW_CPU_CYCLES == evsel->attr.config);
+	TEST_ASSERT_VAL("wrong exclude_user", !evsel->attr.exclude_user);
+	TEST_ASSERT_VAL("wrong exclude_kernel", evsel->attr.exclude_kernel);
+	TEST_ASSERT_VAL("wrong exclude_hv", evsel->attr.exclude_hv);
+	TEST_ASSERT_VAL("wrong exclude guest", !evsel->attr.exclude_guest);
+	TEST_ASSERT_VAL("wrong exclude host", !evsel->attr.exclude_host);
+	TEST_ASSERT_VAL("wrong precise_ip", !evsel->attr.precise_ip);
+
+	ret = perf_evsel__alloc_counts(evsel, 1);
+	TEST_ASSERT_VAL("failed to alloc coiunts for evsel", !ret);
+
+	counts = evsel->counts;
+	counts->aggr.val = 1000;
+
+	evsel = perf_evsel__next(evsel);
+	TEST_ASSERT_VAL("wrong type", PERF_TYPE_HARDWARE == evsel->attr.type);
+	TEST_ASSERT_VAL("wrong config",
+		PERF_COUNT_HW_INSTRUCTIONS == evsel->attr.config);
+	TEST_ASSERT_VAL("wrong exclude_user", !evsel->attr.exclude_user);
+	TEST_ASSERT_VAL("wrong exclude_kernel", evsel->attr.exclude_kernel);
+	TEST_ASSERT_VAL("wrong exclude_hv", evsel->attr.exclude_hv);
+	TEST_ASSERT_VAL("wrong exclude guest", !evsel->attr.exclude_guest);
+	TEST_ASSERT_VAL("wrong exclude host", !evsel->attr.exclude_host);
+	TEST_ASSERT_VAL("wrong precise_ip", !evsel->attr.precise_ip);
+
+	ret = perf_evsel__alloc_counts(evsel, 1);
+	TEST_ASSERT_VAL("failed to alloc coiunts for evsel", !ret);
+
+	counts = evsel->counts;
+	counts->aggr.val = 500;
+
+	memset(&expr, 0x0, sizeof(expr));
+	expr.set    = set;
+	expr.evlist = evlist;
+
+	perf_formula__eval(&fml, set, &expr);
+
+	counter = list_first_entry(&set->head_counters,
+				   struct perf_formula_counter,
+				   list);
+	TEST_ASSERT_VAL("failed to calculated the counter", !ret);
+	TEST_ASSERT_VAL("wrong counter final value", counter->result == 4.5);
+
+	counter = list_entry(counter->list.next,
+			     struct perf_formula_counter,
+			     list);
+	TEST_ASSERT_VAL("failed to calculated the counter", !ret);
+	TEST_ASSERT_VAL("wrong counter final value", counter->result == 2);
+
+	perf_formula__free(&fml);
+	return 0;
+}
+
+static int test_basics(void)
+{
+	int ret;
+	char *file;
+	char data[] =
+"							\n\
+set {							\n\
+	events {					\n\
+		CY = cycles:u				\n\
+		IN = instructions:u			\n\
+	}						\n\
+							\n\
+	t   = -IN / 1000 + 5				\n\
+	cpi = CY / -((t - 5) * 1000)			\n\
+							\n\
+	print cpi					\n\
+}							\n\
+";
+
+	file = file_get(data, sizeof(data));
+	TEST_ASSERT_VAL("failed to get data file", file);
+
+	ret = __test_basics(file);
+
+	file_put(file);
+	return ret;
+}
+
+typedef int (*fn_t)(void);
+
+struct formula_test {
+	const char *desc;
+	fn_t fn;
+} tests[] = {
+	{
+		.desc	= "basics",
+		.fn	= test_basics,
+	},
+	{
+		.desc	= NULL,
+		.fn	= NULL,
+	},
+};
+
+
+int test__formula(void)
+{
+	struct formula_test *t = &tests[0];
+	int ret = 0;
+
+	while (t->desc) {
+		pr_debug("test: %s\n", t->desc);
+
+		if (t->fn())
+			ret = -1;
+
+		t++;
+	}
+
+	return ret;
+}
diff --git a/tools/perf/tests/parse-events.c b/tools/perf/tests/parse-events.c
index 88e2f44..91a0004 100644
--- a/tools/perf/tests/parse-events.c
+++ b/tools/perf/tests/parse-events.c
@@ -7,14 +7,6 @@
 #include "tests.h"
 #include <linux/hw_breakpoint.h>
 
-#define TEST_ASSERT_VAL(text, cond) \
-do { \
-	if (!(cond)) { \
-		pr_debug("FAILED %s:%d %s\n", __FILE__, __LINE__, text); \
-		return -1; \
-	} \
-} while (0)
-
 #define PERF_TP_SAMPLE_TYPE (PERF_SAMPLE_RAW | PERF_SAMPLE_TIME | \
 			     PERF_SAMPLE_CPU | PERF_SAMPLE_PERIOD)
 
diff --git a/tools/perf/tests/tests.h b/tools/perf/tests/tests.h
index dd7feae..c3e3cea 100644
--- a/tools/perf/tests/tests.h
+++ b/tools/perf/tests/tests.h
@@ -7,6 +7,15 @@ enum {
 	TEST_SKIP = -2,
 };
 
+#define TEST_ASSERT_VAL(text, cond) \
+do { \
+	if (!(cond)) { \
+		pr_debug("FAILED %s:%d %s\n", __FILE__, __LINE__, text); \
+		return -1; \
+	} \
+} while (0)
+
+
 /* Tests */
 int test__vmlinux_matches_kallsyms(void);
 int test__open_syscall_event(void);
@@ -27,5 +36,6 @@ int test__bp_signal(void);
 int test__bp_signal_overflow(void);
 int test__task_exit(void);
 int test__sw_clock_freq(void);
+int test__formula(void);
 
 #endif /* TESTS_H */
-- 
1.7.11.7

[PATCH 7/8] perf stat: Add support to process formulas

[Jiri Olsa]

Adding -f option to specify the formula configuration file
to load. The formula name itself is specified within the
'-e' option.

With following formula.conf config file:

---
cpi {
        events {
                CY = cycles:u
                IN = instructions:u
        }

        cpi = CY / IN

        print cpi
}

branch {
        events {
                IN = instructions:u
                BI = branch-instructions:u
                BM = branch-misses:u
        }

        branch-rate       = BI / IN
        branch-miss-rate  = BM / IN
        branch-miss-ratio = BM / BI

        print branch-rate
        print branch-miss-rate
        print branch-miss-ratio
}
---

You'll get following result:

  $ ./perf stat -f formula.conf -e formula-branch kill
  usage: kill [ -s signal | -p ] [ -a ] pid ...
         kill -l [ signal ]

   Performance counter stats for 'kill':

           187,342  instructions:u           #    0.00  insns per cycle
            41,996  branch-instructions:u
             4,826  branch-misses:u          #   11.49% of all branches

       0.054313993 seconds time elapsed

        0.22416757 branch-rate
        0.02576037 branch-miss-rate
        0.11491571 branch-miss-ratio

Signed-off-by: Jiri Olsa <jolsa@redhat.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Corey Ashford <cjashfor@linux.vnet.ibm.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Andi Kleen <andi@firstfloor.org>
Cc: David Ahern <dsahern@gmail.com>
Cc: Ulrich Drepper <drepper@gmail.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Stephane Eranian <eranian@google.com>
---
 tools/perf/builtin-stat.c | 83 ++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 82 insertions(+), 1 deletion(-)

diff --git a/tools/perf/builtin-stat.c b/tools/perf/builtin-stat.c
index 7e910ba..b703307 100644
--- a/tools/perf/builtin-stat.c
+++ b/tools/perf/builtin-stat.c
@@ -56,6 +56,7 @@
 #include "util/cpumap.h"
 #include "util/thread.h"
 #include "util/thread_map.h"
+#include "util/formula.h"
 
 #include <stdlib.h>
 #include <sys/prctl.h>
@@ -72,6 +73,10 @@ static void print_aggr(char *prefix);
 
 static struct perf_evlist	*evsel_list;
 
+static struct perf_formula	 formula;
+static struct perf_formula_set	*formula_set;
+static bool			 formula_no_preload;
+
 static struct perf_target	target = {
 	.uid	= UINT_MAX,
 };
@@ -380,6 +385,12 @@ static void print_interval(void)
 	diff_timespec(&rs, &ts, &ref_time);
 	sprintf(prefix, "%6lu.%09lu%s", rs.tv_sec, rs.tv_nsec, csv_sep);
 
+	if (formula_set) {
+		perf_formula__print(output, &formula, formula_set,
+				    evsel_list, NULL);
+		return;
+	}
+
 	if (num_print_interval == 0 && !csv_output) {
 		switch (aggr_mode) {
 		case AGGR_SOCKET:
@@ -1340,6 +1351,51 @@ static int add_default_attributes(void)
 	return perf_evlist__add_default_attrs(evsel_list, very_very_detailed_attrs);
 }
 
+static int formula_option(const struct option *opt __maybe_unused,
+			  const char *str, int unset __maybe_unused)
+{
+	if (perf_formula__load(&formula, (char *) str)) {
+		pr_err("formula: failed to load formula file '%s'\n", str);
+		return -1;
+	}
+
+	formula_no_preload = true;
+
+	pr_debug("formula file '%s'\n", str);
+	return 0;
+}
+
+static int formula_resolve(struct perf_evlist *evlist)
+{
+	char **f = evlist->formulas;
+	char *name = *f;
+
+	if (!formula_no_preload && perf_formula__preload(&formula)) {
+		pr_err("formula: failed to preload default formulas\n");
+		return -1;
+	}
+
+	if (f[1]) {
+		pr_err("formula: only single formula supported for stat\n");
+		return -1;
+	}
+
+	formula_set = perf_formula__set(&formula, name);
+	if (!formula_set) {
+		pr_err("formula: failed to find formula '%s'\n", name);
+		return -1;
+	}
+
+	if (perf_formula__evlist(&formula, formula_set, evlist)) {
+		pr_err("formula: failed to load formula events for %s\n",
+		       name);
+		return -1;
+	}
+
+	pr_debug("loaded formula '%s'\n", name);
+	return 0;
+}
+
 int cmd_stat(int argc, const char **argv, const char *prefix __maybe_unused)
 {
 	bool append_file = false;
@@ -1351,6 +1407,9 @@ int cmd_stat(int argc, const char **argv, const char *prefix __maybe_unused)
 		     parse_events_option),
 	OPT_CALLBACK(0, "filter", &evsel_list, "filter",
 		     "event filter", parse_filter),
+	OPT_CALLBACK('f', "formula", NULL, "formula",
+		     "counters formula file",
+		     formula_option),
 	OPT_BOOLEAN('i', "no-inherit", &no_inherit,
 		    "child tasks do not inherit counters"),
 	OPT_STRING('p', "pid", &target.pid, "pid",
@@ -1412,9 +1471,22 @@ int cmd_stat(int argc, const char **argv, const char *prefix __maybe_unused)
 	if (evsel_list == NULL)
 		return -ENOMEM;
 
+	perf_formula__init(&formula);
+
 	argc = parse_options(argc, argv, options, stat_usage,
 		PARSE_OPT_STOP_AT_NON_OPTION);
 
+	if (perf_evlist__has_formulas(evsel_list)) {
+		if (aggr_mode == AGGR_NONE) {
+			pr_err("formula: Cannot run formulas with "
+			       "no aggr mode\n");
+			return -1;
+		}
+
+		if (formula_resolve(evsel_list))
+			usage_with_options(stat_usage, options);
+	}
+
 	output = stderr;
 	if (output_name && strcmp(output_name, "-"))
 		output = NULL;
@@ -1540,13 +1612,22 @@ int cmd_stat(int argc, const char **argv, const char *prefix __maybe_unused)
 		}
 	}
 
-	if (!forever && status != -1 && !interval)
+	if (!forever && status != -1 && !interval) {
 		print_stat(argc, argv);
 
+		if (formula_set)
+			perf_formula__print(output, &formula, formula_set,
+					    evsel_list, NULL);
+	}
+
 	perf_evlist__free_stats(evsel_list);
 out_free_maps:
 	perf_evlist__delete_maps(evsel_list);
 out:
 	perf_evlist__delete(evsel_list);
+
+	if (formula_set)
+		perf_formula__free(&formula);
+
 	return status;
 }
-- 
1.7.11.7

[PATCH 8/8] perf list: List formulas counters

[Jiri Olsa]

Make default formulas appear in 'perf list' command output
with [Formula counter].

Signed-off-by: Jiri Olsa <jolsa@redhat.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Corey Ashford <cjashfor@linux.vnet.ibm.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Andi Kleen <andi@firstfloor.org>
Cc: David Ahern <dsahern@gmail.com>
Cc: Ulrich Drepper <drepper@gmail.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Stephane Eranian <eranian@google.com>
---
 tools/perf/builtin-list.c      |  3 +++
 tools/perf/util/formula.c      | 34 ++++++++++++++++++++++++++++++++++
 tools/perf/util/formula.h      |  2 ++
 tools/perf/util/parse-events.c |  3 +++
 4 files changed, 42 insertions(+)

diff --git a/tools/perf/builtin-list.c b/tools/perf/builtin-list.c
index 1948ece..71e6d1f 100644
--- a/tools/perf/builtin-list.c
+++ b/tools/perf/builtin-list.c
@@ -13,6 +13,7 @@
 
 #include "util/parse-events.h"
 #include "util/cache.h"
+#include "util/formula.h"
 
 int cmd_list(int argc, const char **argv, const char *prefix __maybe_unused)
 {
@@ -37,6 +38,8 @@ int cmd_list(int argc, const char **argv, const char *prefix __maybe_unused)
 			else if (strcmp(argv[i], "cache") == 0 ||
 				 strcmp(argv[i], "hwcache") == 0)
 				print_hwcache_events(NULL, false);
+			else if (strcmp(argv[i], "formulas") == 0)
+				print_formulas();
 			else if (strcmp(argv[i], "--raw-dump") == 0)
 				print_events(NULL, true);
 			else {
diff --git a/tools/perf/util/formula.c b/tools/perf/util/formula.c
index b9d95b2..71ae117 100644
--- a/tools/perf/util/formula.c
+++ b/tools/perf/util/formula.c
@@ -564,3 +564,37 @@ int perf_formula__preload(struct perf_formula *f)
 
 	return ret ? ret : perf_formula__preload_arch(f);
 }
+
+static int print_set_cb(struct perf_formula_set *set, void *data)
+{
+	bool *printed = (bool *) data;
+	char name[100];
+
+	scnprintf(name, 100, "formula-%s", set->name);
+	printf("  %-50s [Formula counter]\n", name);
+
+	if (!*printed)
+		*printed = true;
+
+	return CB_NEXT;
+}
+
+void print_formulas(void)
+{
+	struct perf_formula f;
+	bool printed = false;
+
+	perf_formula__init(&f);
+
+	if (perf_formula__preload(&f)) {
+		pr_err("formula: failed to preload formulas\n");
+		return;
+	}
+
+	printf("\n");
+
+	for_each_set(&f, print_set_cb, &printed);
+
+	if (printed)
+		printf("\n");
+}
diff --git a/tools/perf/util/formula.h b/tools/perf/util/formula.h
index b6437d8..5288bc6 100644
--- a/tools/perf/util/formula.h
+++ b/tools/perf/util/formula.h
@@ -137,4 +137,6 @@ double perf_formula_expr__resolve(struct perf_formula_expr *expr,
 
 int perf_formula__preload(struct perf_formula *f);
 int perf_formula__preload_arch(struct perf_formula *f);
+
+void print_formulas(void);
 #endif /* __PERF_FORMULA */
diff --git a/tools/perf/util/parse-events.c b/tools/perf/util/parse-events.c
index fe0a554..405f192 100644
--- a/tools/perf/util/parse-events.c
+++ b/tools/perf/util/parse-events.c
@@ -15,6 +15,7 @@
 #define YY_EXTRA_TYPE int
 #include "parse-events-flex.h"
 #include "pmu.h"
+#include "formula.h"
 
 #define MAX_NAME_LEN 100
 
@@ -1133,6 +1134,8 @@ void print_events(const char *event_glob, bool name_only)
 
 	print_hwcache_events(event_glob, name_only);
 
+	print_formulas();
+
 	if (event_glob != NULL)
 		return;
 
-- 
1.7.11.7

Re: [PATCH 3/8] perf tools: Add formula-* parsing support for events

[David Ahern]

On 5/1/13 9:15 AM, Jiri Olsa wrote:
> +static char **formula_add(char **f, char *new)
> +{
> +	int i;
> +#define FORMULAS_CNT 20
> +
> +	if (!f) {
> +		f = zalloc(sizeof(char *) * FORMULAS_CNT + 1);
> +		if (!f)
> +			return NULL;
> +	}
> +
> +	for (i = 0; f[i] && (i < FORMULAS_CNT); i++);
> +
> +	if (i == FORMULAS_CNT) {
> +		pr_err("Too many formula defined, max = %d\n",
> +		       FORMULAS_CNT);
> +		return NULL;
> +	}

Why have an array of arbitrary limit versus a linked list?

David

Re: [PATCH 5/8] perf tools: Add support to preload default formulas

[David Ahern]

On 5/1/13 9:15 AM, Jiri Olsa wrote:
> +int perf_formula__load_dir(struct perf_formula *f, char *path)
> +{
> +	struct dirent *ent;
> +	DIR *dir;
> +	int ret = 0;
> +
> +	dir = opendir(path);
> +	if (!dir) {
> +		pr_err("formula: can't open dir '%s' - %s\n",
> +		       path, strerror(errno));
> +		return -1;
> +	}
> +
> +	while (!ret && (ent = readdir(dir))) {
> +		char file[PATH_MAX];

readdir_r? this is going into libperf, so might as well be thread safe.

David

Re: [PATCH 6/8] perf tests: Add automated tests for formula object

[David Ahern]

On 5/1/13 9:15 AM, Jiri Olsa wrote:
> +static int __test_basics(char *file)
> +{
> +	struct perf_formula fml;
> +	struct perf_formula_set *set;
> +	struct perf_formula_counter *counter;
> +	struct perf_evlist *evlist;
> +	struct perf_evsel *evsel;
> +	struct perf_counts *counts;
> +	struct perf_formula_expr expr;
> +	int ret;
> +
> +	pr_debug("file %s\n", file);
> +
> +	perf_formula__init(&fml);
> +
> +	ret = perf_formula__load(&fml, file);
> +	TEST_ASSERT_VAL("failed to load formula", !ret);

With all the asserts when something goes wrong you should delete the 
file right after perf_formula__load. so maybe

file = file_get(data, sizeof(data));
ret = perf_formula__load(&fml, file);
file_put(file);

David

Re: [RFCv2 0/8] perf tools: Adding formula support

[Jiri Olsa]

On Wed, May 01, 2013 at 05:15:38PM +0200, Jiri Olsa wrote:
> hi,
> adding support to define counters for stat command,
> by using user defined events and counters.
> 
> Initial RFC is here:
> http://marc.info/?l=linux-kernel&m=135825930106535&w=2
> 
> v2 main changes:
>   - counter formula grammar changed
>     (details in patch 4 changelog)
>   - using -e 'formula-<name>' way of using configured <name> set
>     for stat command
>     (details in patch 7 changelog)

forgot.. changes could be reached in here:
git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
perf/formula3

jirka

Re: [PATCH 3/8] perf tools: Add formula-* parsing support for events

[Jiri Olsa]

On Fri, May 03, 2013 at 09:07:56AM -0600, David Ahern wrote:
> On 5/1/13 9:15 AM, Jiri Olsa wrote:
> >+static char **formula_add(char **f, char *new)
> >+{
> >+	int i;
> >+#define FORMULAS_CNT 20
> >+
> >+	if (!f) {
> >+		f = zalloc(sizeof(char *) * FORMULAS_CNT + 1);
> >+		if (!f)
> >+			return NULL;
> >+	}
> >+
> >+	for (i = 0; f[i] && (i < FORMULAS_CNT); i++);
> >+
> >+	if (i == FORMULAS_CNT) {
> >+		pr_err("Too many formula defined, max = %d\n",
> >+		       FORMULAS_CNT);
> >+		return NULL;
> >+	}
> 
> Why have an array of arbitrary limit versus a linked list?

well, I dont need to define new struct, just an array
of string pointers..

jirka

Re: [PATCH 5/8] perf tools: Add support to preload default formulas

[Jiri Olsa]

On Fri, May 03, 2013 at 09:11:16AM -0600, David Ahern wrote:
> On 5/1/13 9:15 AM, Jiri Olsa wrote:
> >+int perf_formula__load_dir(struct perf_formula *f, char *path)
> >+{
> >+	struct dirent *ent;
> >+	DIR *dir;
> >+	int ret = 0;
> >+
> >+	dir = opendir(path);
> >+	if (!dir) {
> >+		pr_err("formula: can't open dir '%s' - %s\n",
> >+		       path, strerror(errno));
> >+		return -1;
> >+	}
> >+
> >+	while (!ret && (ent = readdir(dir))) {
> >+		char file[PATH_MAX];
> 
> readdir_r? this is going into libperf, so might as well be thread safe.

right, will do

thanks,
jirka

Re: [PATCH 6/8] perf tests: Add automated tests for formula object

[Jiri Olsa]

On Fri, May 03, 2013 at 09:18:05AM -0600, David Ahern wrote:
> On 5/1/13 9:15 AM, Jiri Olsa wrote:
> >+static int __test_basics(char *file)
> >+{
> >+	struct perf_formula fml;
> >+	struct perf_formula_set *set;
> >+	struct perf_formula_counter *counter;
> >+	struct perf_evlist *evlist;
> >+	struct perf_evsel *evsel;
> >+	struct perf_counts *counts;
> >+	struct perf_formula_expr expr;
> >+	int ret;
> >+
> >+	pr_debug("file %s\n", file);
> >+
> >+	perf_formula__init(&fml);
> >+
> >+	ret = perf_formula__load(&fml, file);
> >+	TEST_ASSERT_VAL("failed to load formula", !ret);
> 
> With all the asserts when something goes wrong you should delete the
> file right after perf_formula__load. so maybe
> 
> file = file_get(data, sizeof(data));
> ret = perf_formula__load(&fml, file);
> file_put(file);

hum, the TEST_ASSERT_VAL call 'return' from __test_basics on error and
that goes back to test_basics where file_put is called..

thanks,
jirka

Re: [PATCH 6/8] perf tests: Add automated tests for formula object

[David Ahern]

On 5/6/13 11:54 AM, Jiri Olsa wrote:
> hum, the TEST_ASSERT_VAL call 'return' from __test_basics on error and
> that goes back to test_basics where file_put is called..

ok. I filled in the meaning of TEST_ASSERT_VAL -- thinking it killed the 
program at that point as in assert().

David

Re: [PATCH 4/8] perf tools: Add formula interface to interface formula definitions

[Peter Zijlstra]

On Wed, May 01, 2013 at 05:15:42PM +0200, Jiri Olsa wrote:
> Each counter (cpi/branch-rate above) defines formula that
> produces the counter number.
> 
> Formula grammar:
> 
>   expr: '-' expr       |
>         expr '+' expr  |
>         expr '-' expr  |
>         expr '*' expr  |
>         expr '/' expr  |
>         value          |
>         name
> 

I was going to suggest adding (), but then I saw

> +<expr>{
> +"*"		{ return '*'; }
> +"-"		{ return '-'; }
> +"+"		{ return '+'; }
> +"/"		{ return '/'; }
> +"("		{ return '('; }
> +")"		{ return ')'; }
> +

> +'(' expr ')'
> +{
> +	$$ = $2;
> +	pr_debug2("( %F )\n", $$);
> +}

:-)