perf, tools: Refactor and support interval and CSV metrics v4

perf, tools: Refactor and support interval and CSV metrics v4

[Andi Kleen]

[v4: Addressed all review feedback.]
[v3: Addressed all review feedback. Update manpage for CSV. Various changes
     (see individual patches). Remove some more redundant code 
     in printout callers.]
[v2: Addressed (near) all review feedback. No manpage updates so far.
     Add support for --per-core metrics. Various cleanups.]
[v3: Everything compiles again. Some more cleanups.]

Currently perf stat does not support printing computed metrics for interval (-I xxx)
or CSV (-x,) mode. For example IPC or TSX metrics over time are quite useful to know.

This patch implements them. The main obstacle was that the
metrics printing was all open coded all over the metrics computation code.
The second patch refactors the metrics printing to work through call backs that
can be more easily changed. This also cleans up the metrics printing significantly.
The indentation is now handled through printf, no more need to manually count spaces.

Then based on that it implements metrics printing for CSV and interval mode.

Example output:

% perf stat  -I1000 -a sleep 1
#          time              counts unit events                    metric                              multiplex
     1.001301370       12020.049593      task-clock (msec)                                             (100.00%)
     1.001301370              3,952      context-switches          #    0.329 K/sec                    (100.00%)
     1.001301370                 69      cpu-migrations            #    0.006 K/sec                    (100.00%)
     1.001301370                 76      page-faults               #    0.006 K/sec                  
     1.001301370        386,582,789      cycles                    #    0.032 GHz                      (100.00%)
     1.001301370        716,441,544      stalled-cycles-frontend   #  185.33% frontend cycles idle     (100.00%)
     1.001301370    <not supported>      stalled-cycles-backend   
     1.001301370        101,751,678      instructions              #    0.26  insn per cycle         
     1.001301370                                                   #    7.04  stalled cycles per insn  (100.00%)
     1.001301370         20,914,692      branches                  #    1.740 M/sec                    (100.00%)
     1.001301370          1,943,630      branch-misses             #    9.29% of all branches        

CSV mode

% perf stat  -x, -I1000 -a sleep 1
     1.000852081,12016.143006,,task-clock
     1.000852081,4457,,context-switches,12015168277,100.00,0.371,K/sec
     1.000852081,50,,cpu-migrations,12014024424,100.00,0.004,K/sec
     1.000852081,76,,page-faults,12013076716,100.00,0.006,K/sec
     1.000852081,515854373,,cycles,12011235336,100.00,0.043,GHz
     1.000852081,1030742150,,stalled-cycles-frontend,12010984057,100.00,199.81,frontend cycles idle
     1.000852081,<not supported>,,stalled-cycles-backend,0,100.00
     1.000852081,116782495,,instructions,12011130729,100.00,0.23,insn per cycle
     1.000852081,,,,12011130729,100.00,8.83,stalled cycles per insn
     1.000852081,23748237,,branches,12010745125,100.00,1.976,M/sec
     1.000852081,1976560,,branch-misses,12010501884,100.00,8.32,of all branches

Available in
git://git.kernel.org/pub/scm/linux/kernel/git/ak/linux-misc-2.6 perf/stat-metrics-6

[PATCH 1/6] perf, tools, stat: Move sw clock metrics printout to stat-shadow

[Andi Kleen]

From: Andi Kleen <ak@linux.intel.com>

The sw clock metrics printing was missed in the earlier move to
stat-shadow of all the other metric printouts. Move it too.

Acked-by: Jiri Olsa <jolsa@kernel.org>
Signed-off-by: Andi Kleen <ak@linux.intel.com>
---
 tools/perf/builtin-stat.c     | 9 ---------
 tools/perf/util/stat-shadow.c | 3 +++
 2 files changed, 3 insertions(+), 9 deletions(-)

diff --git a/tools/perf/builtin-stat.c b/tools/perf/builtin-stat.c
index a96fb5c..77e5781 100644
--- a/tools/perf/builtin-stat.c
+++ b/tools/perf/builtin-stat.c
@@ -510,15 +510,6 @@ static void nsec_printout(int id, int nr, struct perf_evsel *evsel, double avg)
 
 	if (evsel->cgrp)
 		fprintf(output, "%s%s", csv_sep, evsel->cgrp->name);
-
-	if (csv_output || stat_config.interval)
-		return;
-
-	if (perf_evsel__match(evsel, SOFTWARE, SW_TASK_CLOCK))
-		fprintf(output, " # %8.3f CPUs utilized          ",
-			avg / avg_stats(&walltime_nsecs_stats));
-	else
-		fprintf(output, "                                   ");
 }
 
 static void abs_printout(int id, int nr, struct perf_evsel *evsel, double avg)
diff --git a/tools/perf/util/stat-shadow.c b/tools/perf/util/stat-shadow.c
index 2a5d8d7..625ab3b 100644
--- a/tools/perf/util/stat-shadow.c
+++ b/tools/perf/util/stat-shadow.c
@@ -413,6 +413,9 @@ void perf_stat__print_shadow_stats(FILE *out, struct perf_evsel *evsel,
 			ratio = total / avg;
 
 		fprintf(out, " # %8.0f cycles / elision       ", ratio);
+	} else if (perf_evsel__match(evsel, SOFTWARE, SW_TASK_CLOCK) &&
+		   (ratio = avg_stats(&walltime_nsecs_stats)) != 0) {
+		fprintf(out, " # %8.3f CPUs utilized          ", avg / ratio);
 	} else if (runtime_nsecs_stats[cpu].n != 0) {
 		char unit = 'M';
 
-- 
2.4.3

[PATCH 2/6] perf, tools, stat: Abstract stat metrics printing

[Andi Kleen]

From: Andi Kleen <ak@linux.intel.com>

Abstract the printing of shadow metrics. Instead of every
metric calling fprintf directly and taking care of indentation,
use two call backs: one to print metrics and another to
start a new line.

This will allow adding metrics to CSV mode and also
using them for other purposes.

The computation of padding is now done in the central
callback, instead of every metric doing it manually.
This makes it easier to add new metrics.

v2: Refactor functions, printout now does more. Move
shadow printing. Improve fallback callbacks. Don't
use void * callback data.
v3: Remove unnecessary hunk. Add typedef for new_line
Signed-off-by: Andi Kleen <ak@linux.intel.com>
---
 tools/perf/builtin-stat.c     | 100 ++++++++++++++------
 tools/perf/util/stat-shadow.c | 212 +++++++++++++++++++++++-------------------
 tools/perf/util/stat.h        |  15 ++-
 3 files changed, 197 insertions(+), 130 deletions(-)

diff --git a/tools/perf/builtin-stat.c b/tools/perf/builtin-stat.c
index 77e5781..fbe457c 100644
--- a/tools/perf/builtin-stat.c
+++ b/tools/perf/builtin-stat.c
@@ -484,6 +484,47 @@ static void aggr_printout(struct perf_evsel *evsel, int id, int nr)
 	}
 }
 
+struct outstate {
+	FILE *fh;
+};
+
+#define BASE_INDENT 41
+#define AGGR_INDENT  8
+#define METRIC_LEN  35
+#define NA_INDENT   16
+
+static void new_line_no_aggr_std(void *ctx)
+{
+	struct outstate *os = ctx;
+	fprintf(os->fh, "\n%*s", BASE_INDENT + NA_INDENT, "");
+}
+
+static void new_line_std(void *ctx)
+{
+	struct outstate *os = ctx;
+	fprintf(os->fh, "\n%-*s", BASE_INDENT + AGGR_INDENT, "");
+}
+
+static void print_metric_std(void *ctx, const char *color, const char *fmt,
+			     const char *unit, double val)
+{
+	struct outstate *os = ctx;
+	FILE *out = os->fh;
+	int n;
+
+	if (unit == NULL) {
+		fprintf(out, "%-*s", METRIC_LEN, "");
+		return;
+	}
+
+	n = fprintf(out, " # ");
+	if (color)
+		n += color_fprintf(out, color, fmt, val);
+	else
+		n += fprintf(out, fmt, val);
+	fprintf(out, " %-*s", METRIC_LEN - n - 1, unit);
+}
+
 static void nsec_printout(int id, int nr, struct perf_evsel *evsel, double avg)
 {
 	FILE *output = stat_config.output;
@@ -517,7 +558,6 @@ static void abs_printout(int id, int nr, struct perf_evsel *evsel, double avg)
 	FILE *output = stat_config.output;
 	double sc =  evsel->scale;
 	const char *fmt;
-	int cpu = cpu_map__id_to_cpu(id);
 
 	if (csv_output) {
 		fmt = sc != 1.0 ?  "%.2f%s" : "%.0f%s";
@@ -530,9 +570,6 @@ static void abs_printout(int id, int nr, struct perf_evsel *evsel, double avg)
 
 	aggr_printout(evsel, id, nr);
 
-	if (stat_config.aggr_mode == AGGR_GLOBAL)
-		cpu = 0;
-
 	fprintf(output, fmt, avg, csv_sep);
 
 	if (evsel->unit)
@@ -544,12 +581,32 @@ static void abs_printout(int id, int nr, struct perf_evsel *evsel, double avg)
 
 	if (evsel->cgrp)
 		fprintf(output, "%s%s", csv_sep, evsel->cgrp->name);
+}
 
-	if (csv_output || stat_config.interval)
-		return;
+static void printout(int id, int nr, struct perf_evsel *counter, double uval)
+{
+	struct outstate os = { .fh = stat_config.output };
+	struct perf_stat_output_ctx out;
+	print_metric_t pm = print_metric_std;
+	void (*nl)(void *);
+
+	if (stat_config.aggr_mode == AGGR_NONE)
+		nl = new_line_no_aggr_std;
+	else
+		nl = new_line_std;
 
-	perf_stat__print_shadow_stats(output, evsel, avg, cpu,
-				      stat_config.aggr_mode);
+	if (nsec_counter(counter))
+		nsec_printout(id, nr, counter, uval);
+	else
+		abs_printout(id, nr, counter, uval);
+
+	out.print_metric = pm;
+	out.new_line = nl;
+	out.ctx = &os;
+	perf_stat__print_shadow_stats(counter, uval,
+				stat_config.aggr_mode == AGGR_GLOBAL ? 0 :
+				cpu_map__id_to_cpu(id),
+				&out);
 }
 
 static void print_aggr(char *prefix)
@@ -605,12 +662,7 @@ static void print_aggr(char *prefix)
 				continue;
 			}
 			uval = val * counter->scale;
-
-			if (nsec_counter(counter))
-				nsec_printout(id, nr, counter, uval);
-			else
-				abs_printout(id, nr, counter, uval);
-
+			printout(id, nr, counter, uval);
 			if (!csv_output)
 				print_noise(counter, 1.0);
 
@@ -641,11 +693,7 @@ static void print_aggr_thread(struct perf_evsel *counter, char *prefix)
 			fprintf(output, "%s", prefix);
 
 		uval = val * counter->scale;
-
-		if (nsec_counter(counter))
-			nsec_printout(thread, 0, counter, uval);
-		else
-			abs_printout(thread, 0, counter, uval);
+		printout(thread, 0, counter, uval);
 
 		if (!csv_output)
 			print_noise(counter, 1.0);
@@ -695,11 +743,7 @@ static void print_counter_aggr(struct perf_evsel *counter, char *prefix)
 	}
 
 	uval = avg * counter->scale;
-
-	if (nsec_counter(counter))
-		nsec_printout(-1, 0, counter, uval);
-	else
-		abs_printout(-1, 0, counter, uval);
+	printout(-1, 0, counter, uval);
 
 	print_noise(counter, avg);
 
@@ -752,15 +796,9 @@ static void print_counter(struct perf_evsel *counter, char *prefix)
 		}
 
 		uval = val * counter->scale;
-
-		if (nsec_counter(counter))
-			nsec_printout(cpu, 0, counter, uval);
-		else
-			abs_printout(cpu, 0, counter, uval);
-
+		printout(cpu, 0, counter, uval);
 		if (!csv_output)
 			print_noise(counter, 1.0);
-		print_running(run, ena);
 
 		fputc('\n', output);
 	}
diff --git a/tools/perf/util/stat-shadow.c b/tools/perf/util/stat-shadow.c
index 625ab3b..d7d27b5 100644
--- a/tools/perf/util/stat-shadow.c
+++ b/tools/perf/util/stat-shadow.c
@@ -137,9 +137,10 @@ static const char *get_ratio_color(enum grc_type type, double ratio)
 	return color;
 }
 
-static void print_stalled_cycles_frontend(FILE *out, int cpu,
+static void print_stalled_cycles_frontend(int cpu,
 					  struct perf_evsel *evsel
-					  __maybe_unused, double avg)
+					  __maybe_unused, double avg,
+					  struct perf_stat_output_ctx *out)
 {
 	double total, ratio = 0.0;
 	const char *color;
@@ -152,14 +153,13 @@ static void print_stalled_cycles_frontend(FILE *out, int cpu,
 
 	color = get_ratio_color(GRC_STALLED_CYCLES_FE, ratio);
 
-	fprintf(out, " #  ");
-	color_fprintf(out, color, "%6.2f%%", ratio);
-	fprintf(out, " frontend cycles idle   ");
+	out->print_metric(out->ctx, color, "%7.2f%%", "frontend cycles idle", ratio);
 }
 
-static void print_stalled_cycles_backend(FILE *out, int cpu,
+static void print_stalled_cycles_backend(int cpu,
 					 struct perf_evsel *evsel
-					 __maybe_unused, double avg)
+					 __maybe_unused, double avg,
+					 struct perf_stat_output_ctx *out)
 {
 	double total, ratio = 0.0;
 	const char *color;
@@ -172,14 +172,13 @@ static void print_stalled_cycles_backend(FILE *out, int cpu,
 
 	color = get_ratio_color(GRC_STALLED_CYCLES_BE, ratio);
 
-	fprintf(out, " #  ");
-	color_fprintf(out, color, "%6.2f%%", ratio);
-	fprintf(out, " backend  cycles idle   ");
+	out->print_metric(out->ctx, color, "%6.2f%%", "backend cycles idle", ratio);
 }
 
-static void print_branch_misses(FILE *out, int cpu,
+static void print_branch_misses(int cpu,
 				struct perf_evsel *evsel __maybe_unused,
-				double avg)
+				double avg,
+				struct perf_stat_output_ctx *out)
 {
 	double total, ratio = 0.0;
 	const char *color;
@@ -192,14 +191,13 @@ static void print_branch_misses(FILE *out, int cpu,
 
 	color = get_ratio_color(GRC_CACHE_MISSES, ratio);
 
-	fprintf(out, " #  ");
-	color_fprintf(out, color, "%6.2f%%", ratio);
-	fprintf(out, " of all branches        ");
+	out->print_metric(out->ctx, color, "%7.2f%%", "of all branches", ratio);
 }
 
-static void print_l1_dcache_misses(FILE *out, int cpu,
+static void print_l1_dcache_misses(int cpu,
 				   struct perf_evsel *evsel __maybe_unused,
-				   double avg)
+				   double avg,
+				   struct perf_stat_output_ctx *out)
 {
 	double total, ratio = 0.0;
 	const char *color;
@@ -212,14 +210,13 @@ static void print_l1_dcache_misses(FILE *out, int cpu,
 
 	color = get_ratio_color(GRC_CACHE_MISSES, ratio);
 
-	fprintf(out, " #  ");
-	color_fprintf(out, color, "%6.2f%%", ratio);
-	fprintf(out, " of all L1-dcache hits  ");
+	out->print_metric(out->ctx, color, "%7.2f%%", "of all L1-dcache hits", ratio);
 }
 
-static void print_l1_icache_misses(FILE *out, int cpu,
+static void print_l1_icache_misses(int cpu,
 				   struct perf_evsel *evsel __maybe_unused,
-				   double avg)
+				   double avg,
+				   struct perf_stat_output_ctx *out)
 {
 	double total, ratio = 0.0;
 	const char *color;
@@ -231,15 +228,13 @@ static void print_l1_icache_misses(FILE *out, int cpu,
 		ratio = avg / total * 100.0;
 
 	color = get_ratio_color(GRC_CACHE_MISSES, ratio);
-
-	fprintf(out, " #  ");
-	color_fprintf(out, color, "%6.2f%%", ratio);
-	fprintf(out, " of all L1-icache hits  ");
+	out->print_metric(out->ctx, color, "%7.2f%%", "of all L1-icache hits", ratio);
 }
 
-static void print_dtlb_cache_misses(FILE *out, int cpu,
+static void print_dtlb_cache_misses(int cpu,
 				    struct perf_evsel *evsel __maybe_unused,
-				    double avg)
+				    double avg,
+				    struct perf_stat_output_ctx *out)
 {
 	double total, ratio = 0.0;
 	const char *color;
@@ -251,15 +246,13 @@ static void print_dtlb_cache_misses(FILE *out, int cpu,
 		ratio = avg / total * 100.0;
 
 	color = get_ratio_color(GRC_CACHE_MISSES, ratio);
-
-	fprintf(out, " #  ");
-	color_fprintf(out, color, "%6.2f%%", ratio);
-	fprintf(out, " of all dTLB cache hits ");
+	out->print_metric(out->ctx, color, "%7.2f%%", "of all dTLB cache hits", ratio);
 }
 
-static void print_itlb_cache_misses(FILE *out, int cpu,
+static void print_itlb_cache_misses(int cpu,
 				    struct perf_evsel *evsel __maybe_unused,
-				    double avg)
+				    double avg,
+				    struct perf_stat_output_ctx *out)
 {
 	double total, ratio = 0.0;
 	const char *color;
@@ -271,15 +264,13 @@ static void print_itlb_cache_misses(FILE *out, int cpu,
 		ratio = avg / total * 100.0;
 
 	color = get_ratio_color(GRC_CACHE_MISSES, ratio);
-
-	fprintf(out, " #  ");
-	color_fprintf(out, color, "%6.2f%%", ratio);
-	fprintf(out, " of all iTLB cache hits ");
+	out->print_metric(out->ctx, color, "%7.2f%%", "of all iTLB cache hits", ratio);
 }
 
-static void print_ll_cache_misses(FILE *out, int cpu,
+static void print_ll_cache_misses(int cpu,
 				  struct perf_evsel *evsel __maybe_unused,
-				  double avg)
+				  double avg,
+				  struct perf_stat_output_ctx *out)
 {
 	double total, ratio = 0.0;
 	const char *color;
@@ -291,15 +282,15 @@ static void print_ll_cache_misses(FILE *out, int cpu,
 		ratio = avg / total * 100.0;
 
 	color = get_ratio_color(GRC_CACHE_MISSES, ratio);
-
-	fprintf(out, " #  ");
-	color_fprintf(out, color, "%6.2f%%", ratio);
-	fprintf(out, " of all LL-cache hits   ");
+	out->print_metric(out->ctx, color, "%7.2f%%", "of all LL-cache hits", ratio);
 }
 
-void perf_stat__print_shadow_stats(FILE *out, struct perf_evsel *evsel,
-				   double avg, int cpu, enum aggr_mode aggr)
+void perf_stat__print_shadow_stats(struct perf_evsel *evsel,
+				   double avg, int cpu,
+				   struct perf_stat_output_ctx *out)
 {
+	void *ctxp = out->ctx;
+	print_metric_t print_metric = out->print_metric;
 	double total, ratio = 0.0, total2;
 	int ctx = evsel_context(evsel);
 
@@ -307,117 +298,144 @@ void perf_stat__print_shadow_stats(FILE *out, struct perf_evsel *evsel,
 		total = avg_stats(&runtime_cycles_stats[ctx][cpu]);
 		if (total) {
 			ratio = avg / total;
-			fprintf(out, " #   %5.2f  insns per cycle        ", ratio);
+			print_metric(ctxp, NULL, "%7.2f ",
+					"insn per cycle", ratio);
 		} else {
-			fprintf(out, "                                   ");
+			print_metric(ctxp, NULL, NULL, "insn per cycle", 0);
 		}
 		total = avg_stats(&runtime_stalled_cycles_front_stats[ctx][cpu]);
 		total = max(total, avg_stats(&runtime_stalled_cycles_back_stats[ctx][cpu]));
 
 		if (total && avg) {
 			ratio = total / avg;
-			fprintf(out, "\n");
-			if (aggr == AGGR_NONE)
-				fprintf(out, "        ");
-			fprintf(out, "                                                  #   %5.2f  stalled cycles per insn", ratio);
+			out->new_line(ctxp);
+			print_metric(ctxp, NULL, "%7.2f ",
+					"stalled cycles per insn",
+					ratio);
+		} else {
+			print_metric(ctxp, NULL, NULL,
+				     "stalled cycles per insn", 0);
 		}
-
-	} else if (perf_evsel__match(evsel, HARDWARE, HW_BRANCH_MISSES) &&
-			runtime_branches_stats[ctx][cpu].n != 0) {
-		print_branch_misses(out, cpu, evsel, avg);
+	} else if (perf_evsel__match(evsel, HARDWARE, HW_BRANCH_MISSES)) {
+		if (runtime_branches_stats[ctx][cpu].n != 0)
+			print_branch_misses(cpu, evsel, avg, out);
+		else
+			print_metric(ctxp, NULL, NULL, "of all branches", 0);
 	} else if (
 		evsel->attr.type == PERF_TYPE_HW_CACHE &&
 		evsel->attr.config ==  ( PERF_COUNT_HW_CACHE_L1D |
 					((PERF_COUNT_HW_CACHE_OP_READ) << 8) |
-					((PERF_COUNT_HW_CACHE_RESULT_MISS) << 16)) &&
-			runtime_l1_dcache_stats[ctx][cpu].n != 0) {
-		print_l1_dcache_misses(out, cpu, evsel, avg);
+					 ((PERF_COUNT_HW_CACHE_RESULT_MISS) << 16))) {
+		if (runtime_l1_dcache_stats[ctx][cpu].n != 0)
+			print_l1_dcache_misses(cpu, evsel, avg, out);
+		else
+			print_metric(ctxp, NULL, NULL, "of all L1-dcache hits", 0);
 	} else if (
 		evsel->attr.type == PERF_TYPE_HW_CACHE &&
 		evsel->attr.config ==  ( PERF_COUNT_HW_CACHE_L1I |
 					((PERF_COUNT_HW_CACHE_OP_READ) << 8) |
-					((PERF_COUNT_HW_CACHE_RESULT_MISS) << 16)) &&
-			runtime_l1_icache_stats[ctx][cpu].n != 0) {
-		print_l1_icache_misses(out, cpu, evsel, avg);
+					 ((PERF_COUNT_HW_CACHE_RESULT_MISS) << 16))) {
+		if (runtime_l1_icache_stats[ctx][cpu].n != 0)
+			print_l1_icache_misses(cpu, evsel, avg, out);
+		else
+			print_metric(ctxp, NULL, NULL, "of all L1-icache hits", 0);
 	} else if (
 		evsel->attr.type == PERF_TYPE_HW_CACHE &&
 		evsel->attr.config ==  ( PERF_COUNT_HW_CACHE_DTLB |
 					((PERF_COUNT_HW_CACHE_OP_READ) << 8) |
-					((PERF_COUNT_HW_CACHE_RESULT_MISS) << 16)) &&
-			runtime_dtlb_cache_stats[ctx][cpu].n != 0) {
-		print_dtlb_cache_misses(out, cpu, evsel, avg);
+					 ((PERF_COUNT_HW_CACHE_RESULT_MISS) << 16))) {
+		if (runtime_dtlb_cache_stats[ctx][cpu].n != 0)
+			print_dtlb_cache_misses(cpu, evsel, avg, out);
+		else
+			print_metric(ctxp, NULL, NULL, "of all dTLB cache hits", 0);
 	} else if (
 		evsel->attr.type == PERF_TYPE_HW_CACHE &&
 		evsel->attr.config ==  ( PERF_COUNT_HW_CACHE_ITLB |
 					((PERF_COUNT_HW_CACHE_OP_READ) << 8) |
-					((PERF_COUNT_HW_CACHE_RESULT_MISS) << 16)) &&
-			runtime_itlb_cache_stats[ctx][cpu].n != 0) {
-		print_itlb_cache_misses(out, cpu, evsel, avg);
+					 ((PERF_COUNT_HW_CACHE_RESULT_MISS) << 16))) {
+		if (runtime_itlb_cache_stats[ctx][cpu].n != 0)
+			print_itlb_cache_misses(cpu, evsel, avg, out);
+		else
+			print_metric(ctxp, NULL, NULL, "of all iTLB cache hits", 0);
 	} else if (
 		evsel->attr.type == PERF_TYPE_HW_CACHE &&
 		evsel->attr.config ==  ( PERF_COUNT_HW_CACHE_LL |
 					((PERF_COUNT_HW_CACHE_OP_READ) << 8) |
-					((PERF_COUNT_HW_CACHE_RESULT_MISS) << 16)) &&
-			runtime_ll_cache_stats[ctx][cpu].n != 0) {
-		print_ll_cache_misses(out, cpu, evsel, avg);
-	} else if (perf_evsel__match(evsel, HARDWARE, HW_CACHE_MISSES) &&
-			runtime_cacherefs_stats[ctx][cpu].n != 0) {
+					 ((PERF_COUNT_HW_CACHE_RESULT_MISS) << 16))) {
+		if (runtime_ll_cache_stats[ctx][cpu].n != 0)
+			print_ll_cache_misses(cpu, evsel, avg, out);
+		else
+			print_metric(ctxp, NULL, NULL, "of all LL-cache hits", 0);
+	} else if (perf_evsel__match(evsel, HARDWARE, HW_CACHE_MISSES)) {
 		total = avg_stats(&runtime_cacherefs_stats[ctx][cpu]);
 
 		if (total)
 			ratio = avg * 100 / total;
 
-		fprintf(out, " # %8.3f %% of all cache refs    ", ratio);
-
+		if (runtime_cacherefs_stats[ctx][cpu].n != 0)
+			print_metric(ctxp, NULL, "%8.3f %%",
+				     "of all cache refs", ratio);
+		else
+			print_metric(ctxp, NULL, NULL, "of all cache refs", 0);
 	} else if (perf_evsel__match(evsel, HARDWARE, HW_STALLED_CYCLES_FRONTEND)) {
-		print_stalled_cycles_frontend(out, cpu, evsel, avg);
+		print_stalled_cycles_frontend(cpu, evsel, avg, out);
 	} else if (perf_evsel__match(evsel, HARDWARE, HW_STALLED_CYCLES_BACKEND)) {
-		print_stalled_cycles_backend(out, cpu, evsel, avg);
+		print_stalled_cycles_backend(cpu, evsel, avg, out);
 	} else if (perf_evsel__match(evsel, HARDWARE, HW_CPU_CYCLES)) {
 		total = avg_stats(&runtime_nsecs_stats[cpu]);
 
 		if (total) {
 			ratio = avg / total;
-			fprintf(out, " # %8.3f GHz                    ", ratio);
+			print_metric(ctxp, NULL, "%8.3f", "GHz", ratio);
 		} else {
-			fprintf(out, "                                   ");
+			print_metric(ctxp, NULL, NULL, "Ghz", 0);
 		}
 	} else if (perf_stat_evsel__is(evsel, CYCLES_IN_TX)) {
 		total = avg_stats(&runtime_cycles_stats[ctx][cpu]);
 		if (total)
-			fprintf(out,
-				" #   %5.2f%% transactional cycles   ",
-				100.0 * (avg / total));
+			print_metric(ctxp, NULL,
+					"%7.2f%%", "transactional cycles",
+					100.0 * (avg / total));
+		else
+			print_metric(ctxp, NULL, NULL, "transactional cycles",
+				     0);
 	} else if (perf_stat_evsel__is(evsel, CYCLES_IN_TX_CP)) {
 		total = avg_stats(&runtime_cycles_stats[ctx][cpu]);
 		total2 = avg_stats(&runtime_cycles_in_tx_stats[ctx][cpu]);
 		if (total2 < avg)
 			total2 = avg;
 		if (total)
-			fprintf(out,
-				" #   %5.2f%% aborted cycles         ",
+			print_metric(ctxp, NULL, "%7.2f%%", "aborted cycles",
 				100.0 * ((total2-avg) / total));
-	} else if (perf_stat_evsel__is(evsel, TRANSACTION_START) &&
-		   runtime_cycles_in_tx_stats[ctx][cpu].n != 0) {
+		else
+			print_metric(ctxp, NULL, NULL, "aborted cycles", 0);
+	} else if (perf_stat_evsel__is(evsel, TRANSACTION_START)) {
 		total = avg_stats(&runtime_cycles_in_tx_stats[ctx][cpu]);
 
 		if (avg)
 			ratio = total / avg;
 
-		fprintf(out, " # %8.0f cycles / transaction   ", ratio);
-	} else if (perf_stat_evsel__is(evsel, ELISION_START) &&
-		   runtime_cycles_in_tx_stats[ctx][cpu].n != 0) {
+		if (runtime_cycles_in_tx_stats[ctx][cpu].n != 0)
+			print_metric(ctxp, NULL, "%8.0f",
+				     "cycles / transaction", ratio);
+		else
+			print_metric(ctxp, NULL, NULL, "cycles / transaction",
+				     0);
+	} else if (perf_stat_evsel__is(evsel, ELISION_START)) {
 		total = avg_stats(&runtime_cycles_in_tx_stats[ctx][cpu]);
 
 		if (avg)
 			ratio = total / avg;
-
-		fprintf(out, " # %8.0f cycles / elision       ", ratio);
-	} else if (perf_evsel__match(evsel, SOFTWARE, SW_TASK_CLOCK) &&
-		   (ratio = avg_stats(&walltime_nsecs_stats)) != 0) {
-		fprintf(out, " # %8.3f CPUs utilized          ", avg / ratio);
+		print_metric(ctxp, NULL, "%8.0f", "cycles / elision", ratio);
+	} else if (perf_evsel__match(evsel, SOFTWARE, SW_TASK_CLOCK)) {
+		if ((ratio = avg_stats(&walltime_nsecs_stats)) != 0)
+			print_metric(ctxp, NULL, "%8.3f", "CPUs utilized",
+				     avg / ratio);
+		else
+			print_metric(ctxp, NULL, NULL, "CPUs utilized", 0);
 	} else if (runtime_nsecs_stats[cpu].n != 0) {
 		char unit = 'M';
+		char unit_buf[10];
 
 		total = avg_stats(&runtime_nsecs_stats[cpu]);
 
@@ -427,9 +445,9 @@ void perf_stat__print_shadow_stats(FILE *out, struct perf_evsel *evsel,
 			ratio *= 1000;
 			unit = 'K';
 		}
-
-		fprintf(out, " # %8.3f %c/sec                  ", ratio, unit);
+		snprintf(unit_buf, sizeof(unit_buf), "%c/sec", unit);
+		print_metric(ctxp, NULL, "%8.3f", unit_buf, ratio);
 	} else {
-		fprintf(out, "                                   ");
+		print_metric(ctxp, NULL, NULL, NULL, 0);
 	}
 }
diff --git a/tools/perf/util/stat.h b/tools/perf/util/stat.h
index 62448c8..e1899f6 100644
--- a/tools/perf/util/stat.h
+++ b/tools/perf/util/stat.h
@@ -67,11 +67,22 @@ void perf_stat_evsel_id_init(struct perf_evsel *evsel);
 
 extern struct stats walltime_nsecs_stats;
 
+typedef void (*print_metric_t)(void *ctx, const char *color, const char *unit,
+			       const char *fmt, double val);
+typedef void (*new_line_t )(void *ctx);
+
 void perf_stat__reset_shadow_stats(void);
 void perf_stat__update_shadow_stats(struct perf_evsel *counter, u64 *count,
 				    int cpu);
-void perf_stat__print_shadow_stats(FILE *out, struct perf_evsel *evsel,
-				   double avg, int cpu, enum aggr_mode aggr);
+struct perf_stat_output_ctx {
+	void *ctx;
+	print_metric_t print_metric;
+	new_line_t new_line;
+};
+
+void perf_stat__print_shadow_stats(struct perf_evsel *evsel,
+				   double avg, int cpu,
+				   struct perf_stat_output_ctx *out);
 
 void perf_evsel__reset_stat_priv(struct perf_evsel *evsel);
 int perf_evsel__alloc_stat_priv(struct perf_evsel *evsel);
-- 
2.4.3

[PATCH 3/6] perf, tools, stat: Add support for metrics in interval mode

[Andi Kleen]

From: Andi Kleen <ak@linux.intel.com>

Now that we can modify the metrics printout functions easily,
it's straight forward to support metric printing for interval mode.
All that is needed is to print the time stamp on every new line.
Pass the prefix into the context and print it out.

Signed-off-by: Andi Kleen <ak@linux.intel.com>
---
 tools/perf/builtin-stat.c | 21 +++++++++++++--------
 1 file changed, 13 insertions(+), 8 deletions(-)

diff --git a/tools/perf/builtin-stat.c b/tools/perf/builtin-stat.c
index fbe457c..78b1688 100644
--- a/tools/perf/builtin-stat.c
+++ b/tools/perf/builtin-stat.c
@@ -486,6 +486,7 @@ static void aggr_printout(struct perf_evsel *evsel, int id, int nr)
 
 struct outstate {
 	FILE *fh;
+	const char *prefix;
 };
 
 #define BASE_INDENT 41
@@ -496,13 +497,13 @@ struct outstate {
 static void new_line_no_aggr_std(void *ctx)
 {
 	struct outstate *os = ctx;
-	fprintf(os->fh, "\n%*s", BASE_INDENT + NA_INDENT, "");
+	fprintf(os->fh, "\n%s%-*s", os->prefix, BASE_INDENT + NA_INDENT, "");
 }
 
 static void new_line_std(void *ctx)
 {
 	struct outstate *os = ctx;
-	fprintf(os->fh, "\n%-*s", BASE_INDENT + AGGR_INDENT, "");
+	fprintf(os->fh, "\n%s%-*s", os->prefix, BASE_INDENT + AGGR_INDENT, "");
 }
 
 static void print_metric_std(void *ctx, const char *color, const char *fmt,
@@ -583,10 +584,14 @@ static void abs_printout(int id, int nr, struct perf_evsel *evsel, double avg)
 		fprintf(output, "%s%s", csv_sep, evsel->cgrp->name);
 }
 
-static void printout(int id, int nr, struct perf_evsel *counter, double uval)
+static void printout(int id, int nr, struct perf_evsel *counter, double uval,
+		     char *prefix)
 {
-	struct outstate os = { .fh = stat_config.output };
 	struct perf_stat_output_ctx out;
+	struct outstate os = {
+		.fh = stat_config.output,
+		.prefix = prefix ? prefix : ""
+	};
 	print_metric_t pm = print_metric_std;
 	void (*nl)(void *);
 
@@ -662,7 +667,7 @@ static void print_aggr(char *prefix)
 				continue;
 			}
 			uval = val * counter->scale;
-			printout(id, nr, counter, uval);
+			printout(id, nr, counter, uval, prefix);
 			if (!csv_output)
 				print_noise(counter, 1.0);
 
@@ -693,7 +698,7 @@ static void print_aggr_thread(struct perf_evsel *counter, char *prefix)
 			fprintf(output, "%s", prefix);
 
 		uval = val * counter->scale;
-		printout(thread, 0, counter, uval);
+		printout(thread, 0, counter, uval, prefix);
 
 		if (!csv_output)
 			print_noise(counter, 1.0);
@@ -743,7 +748,7 @@ static void print_counter_aggr(struct perf_evsel *counter, char *prefix)
 	}
 
 	uval = avg * counter->scale;
-	printout(-1, 0, counter, uval);
+	printout(-1, 0, counter, uval, prefix);
 
 	print_noise(counter, avg);
 
@@ -796,7 +801,7 @@ static void print_counter(struct perf_evsel *counter, char *prefix)
 		}
 
 		uval = val * counter->scale;
-		printout(cpu, 0, counter, uval);
+		printout(cpu, 0, counter, uval, prefix);
 		if (!csv_output)
 			print_noise(counter, 1.0);
 
-- 
2.4.3

[PATCH 4/6] perf, tools, stat: Move noise/running printing into printout

[Andi Kleen]

From: Andi Kleen <ak@linux.intel.com>

Move the running/noise printing into printout to avoid
duplicated code in the callers.

v2: Merged with other patches. Remove unnecessary hunk.
Readd hunk that ended in earlier patch.
Signed-off-by: Andi Kleen <ak@linux.intel.com>
---
 tools/perf/builtin-stat.c | 52 ++++++++++-------------------------------------
 1 file changed, 11 insertions(+), 41 deletions(-)

diff --git a/tools/perf/builtin-stat.c b/tools/perf/builtin-stat.c
index 78b1688..86da2e1 100644
--- a/tools/perf/builtin-stat.c
+++ b/tools/perf/builtin-stat.c
@@ -585,7 +585,7 @@ static void abs_printout(int id, int nr, struct perf_evsel *evsel, double avg)
 }
 
 static void printout(int id, int nr, struct perf_evsel *counter, double uval,
-		     char *prefix)
+		     char *prefix, u64 run, u64 ena, double noise)
 {
 	struct perf_stat_output_ctx out;
 	struct outstate os = {
@@ -612,6 +612,12 @@ static void printout(int id, int nr, struct perf_evsel *counter, double uval,
 				stat_config.aggr_mode == AGGR_GLOBAL ? 0 :
 				cpu_map__id_to_cpu(id),
 				&out);
+
+	if (!csv_output) {
+		print_noise(counter, noise);
+		if (run != ena)
+			fprintf(stat_config.output, "  (%.2f%%)", 100.0 * run / ena);
+	}
 }
 
 static void print_aggr(char *prefix)
@@ -667,11 +673,7 @@ static void print_aggr(char *prefix)
 				continue;
 			}
 			uval = val * counter->scale;
-			printout(id, nr, counter, uval, prefix);
-			if (!csv_output)
-				print_noise(counter, 1.0);
-
-			print_running(run, ena);
+			printout(id, nr, counter, uval, prefix, run, ena, 1.0);
 			fputc('\n', output);
 		}
 	}
@@ -698,12 +700,7 @@ static void print_aggr_thread(struct perf_evsel *counter, char *prefix)
 			fprintf(output, "%s", prefix);
 
 		uval = val * counter->scale;
-		printout(thread, 0, counter, uval, prefix);
-
-		if (!csv_output)
-			print_noise(counter, 1.0);
-
-		print_running(run, ena);
+		printout(thread, 0, counter, uval, prefix, run, ena, 1.0);
 		fputc('\n', output);
 	}
 }
@@ -717,7 +714,6 @@ static void print_counter_aggr(struct perf_evsel *counter, char *prefix)
 	FILE *output = stat_config.output;
 	struct perf_stat *ps = counter->priv;
 	double avg = avg_stats(&ps->res_stats[0]);
-	int scaled = counter->counts->scaled;
 	double uval;
 	double avg_enabled, avg_running;
 
@@ -727,32 +723,8 @@ static void print_counter_aggr(struct perf_evsel *counter, char *prefix)
 	if (prefix)
 		fprintf(output, "%s", prefix);
 
-	if (scaled == -1 || !counter->supported) {
-		fprintf(output, "%*s%s",
-			csv_output ? 0 : 18,
-			counter->supported ? CNTR_NOT_COUNTED : CNTR_NOT_SUPPORTED,
-			csv_sep);
-		fprintf(output, "%-*s%s",
-			csv_output ? 0 : unit_width,
-			counter->unit, csv_sep);
-		fprintf(output, "%*s",
-			csv_output ? 0 : -25,
-			perf_evsel__name(counter));
-
-		if (counter->cgrp)
-			fprintf(output, "%s%s", csv_sep, counter->cgrp->name);
-
-		print_running(avg_running, avg_enabled);
-		fputc('\n', output);
-		return;
-	}
-
 	uval = avg * counter->scale;
-	printout(-1, 0, counter, uval, prefix);
-
-	print_noise(counter, avg);
-
-	print_running(avg_running, avg_enabled);
+	printout(-1, 0, counter, uval, prefix, avg_running, avg_enabled, avg);
 	fprintf(output, "\n");
 }
 
@@ -801,9 +773,7 @@ static void print_counter(struct perf_evsel *counter, char *prefix)
 		}
 
 		uval = val * counter->scale;
-		printout(cpu, 0, counter, uval, prefix);
-		if (!csv_output)
-			print_noise(counter, 1.0);
+		printout(cpu, 0, counter, uval, prefix, run, ena, 1.0);
 
 		fputc('\n', output);
 	}
-- 
2.4.3

[PATCH 5/6] perf, tools, stat: Implement CSV metrics output

[Andi Kleen]

From: Andi Kleen <ak@linux.intel.com>

Now support CSV output for metrics. With the new output callbacks
this is relatively straight forward by creating new callbacks.

The new line callback needs to know the number of fields to skip them
correctly

v2: Split out function argument changes
Signed-off-by: Andi Kleen <ak@linux.intel.com>
---
 tools/perf/builtin-stat.c | 64 +++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 64 insertions(+)

diff --git a/tools/perf/builtin-stat.c b/tools/perf/builtin-stat.c
index 86da2e1..f582481 100644
--- a/tools/perf/builtin-stat.c
+++ b/tools/perf/builtin-stat.c
@@ -487,6 +487,8 @@ static void aggr_printout(struct perf_evsel *evsel, int id, int nr)
 struct outstate {
 	FILE *fh;
 	const char *prefix;
+	int  nfields;
+	u64  run, ena;
 };
 
 #define BASE_INDENT 41
@@ -526,6 +528,49 @@ static void print_metric_std(void *ctx, const char *color, const char *fmt,
 	fprintf(out, " %-*s", METRIC_LEN - n - 1, unit);
 }
 
+static void new_line_csv(void *ctx)
+{
+	struct outstate *os = ctx;
+	int i;
+
+	fputc('\n', os->fh);
+	if (os->prefix)
+		fprintf(os->fh, "%s%s", os->prefix, csv_sep);
+	for (i = 0; i < os->nfields; i++)
+		fputs(csv_sep, os->fh);
+}
+
+static void print_metric_csv(void *ctx,
+			     const char *color __maybe_unused,
+			     const char *fmt, const char *unit, double val)
+{
+	struct outstate *os = ctx;
+	FILE *out = os->fh;
+	char buf[64], *vals, *ends;
+
+	if (unit == NULL) {
+		fprintf(out, "%s%s%s%s", csv_sep, csv_sep, csv_sep, csv_sep);
+		return;
+	}
+	fprintf(out, "%s%" PRIu64 "%s%.2f%s",
+		csv_sep,
+		os->run,
+		csv_sep,
+		os->ena ? 100.0 * os->run / os->ena : 100.0,
+		csv_sep);
+	snprintf(buf, sizeof(buf), fmt, val);
+	vals = buf;
+	while (isspace(*vals))
+		vals++;
+	ends = vals;
+	while (isdigit(*ends) || *ends == '.')
+		ends++;
+	*ends = 0;
+	while (isspace(*unit))
+		unit++;
+	fprintf(out, "%s%s%s", vals, csv_sep, unit);
+}
+
 static void nsec_printout(int id, int nr, struct perf_evsel *evsel, double avg)
 {
 	FILE *output = stat_config.output;
@@ -600,6 +645,25 @@ static void printout(int id, int nr, struct perf_evsel *counter, double uval,
 	else
 		nl = new_line_std;
 
+	if (csv_output) {
+		static int aggr_fields[] = {
+			[AGGR_GLOBAL] = 0,
+			[AGGR_THREAD] = 1,
+			[AGGR_NONE] = 1,
+			[AGGR_SOCKET] = 2,
+			[AGGR_CORE] = 2,
+		};
+
+		pm = print_metric_csv;
+		nl = new_line_csv;
+		os.nfields = 1;
+		os.nfields += aggr_fields[stat_config.aggr_mode];
+		if (counter->cgrp)
+			os.nfields++;
+		os.run = run;
+		os.ena = ena;
+	}
+
 	if (nsec_counter(counter))
 		nsec_printout(id, nr, counter, uval);
 	else
-- 
2.4.3

[PATCH 6/6] perf, tools, stat: Move non counting counter printing to printout

[Andi Kleen]

From: Andi Kleen <ak@linux.intel.com>

Move the special case printing for non-running counters to
printout, so it can be shared by all the output options.

Signed-off-by: Andi Kleen <ak@linux.intel.com>
---
 tools/perf/builtin-stat.c | 73 ++++++++++++++++-------------------------------
 1 file changed, 24 insertions(+), 49 deletions(-)

diff --git a/tools/perf/builtin-stat.c b/tools/perf/builtin-stat.c
index f582481..9ae8fc0 100644
--- a/tools/perf/builtin-stat.c
+++ b/tools/perf/builtin-stat.c
@@ -664,6 +664,30 @@ static void printout(int id, int nr, struct perf_evsel *counter, double uval,
 		os.ena = ena;
 	}
 
+	if (run == 0 || ena == 0) {
+		aggr_printout(counter, id, nr);
+
+		fprintf(stat_config.output, "%*s%s",
+			csv_output ? 0 : 18,
+			counter->supported ? CNTR_NOT_COUNTED : CNTR_NOT_SUPPORTED,
+			csv_sep);
+
+		fprintf(stat_config.output, "%-*s%s",
+			csv_output ? 0 : unit_width,
+			counter->unit, csv_sep);
+
+		fprintf(stat_config.output, "%*s",
+			csv_output ? 0 : -25,
+			perf_evsel__name(counter));
+
+		if (counter->cgrp)
+			fprintf(stat_config.output, "%s%s",
+				csv_sep, counter->cgrp->name);
+
+		print_running(run, ena);
+		return;
+	}
+
 	if (nsec_counter(counter))
 		nsec_printout(id, nr, counter, uval);
 	else
@@ -712,30 +736,6 @@ static void print_aggr(char *prefix)
 			if (prefix)
 				fprintf(output, "%s", prefix);
 
-			if (run == 0 || ena == 0) {
-				aggr_printout(counter, id, nr);
-
-				fprintf(output, "%*s%s",
-					csv_output ? 0 : 18,
-					counter->supported ? CNTR_NOT_COUNTED : CNTR_NOT_SUPPORTED,
-					csv_sep);
-
-				fprintf(output, "%-*s%s",
-					csv_output ? 0 : unit_width,
-					counter->unit, csv_sep);
-
-				fprintf(output, "%*s",
-					csv_output ? 0 : -25,
-					perf_evsel__name(counter));
-
-				if (counter->cgrp)
-					fprintf(output, "%s%s",
-						csv_sep, counter->cgrp->name);
-
-				print_running(run, ena);
-				fputc('\n', output);
-				continue;
-			}
 			uval = val * counter->scale;
 			printout(id, nr, counter, uval, prefix, run, ena, 1.0);
 			fputc('\n', output);
@@ -811,31 +811,6 @@ static void print_counter(struct perf_evsel *counter, char *prefix)
 		if (prefix)
 			fprintf(output, "%s", prefix);
 
-		if (run == 0 || ena == 0) {
-			fprintf(output, "CPU%*d%s%*s%s",
-				csv_output ? 0 : -4,
-				perf_evsel__cpus(counter)->map[cpu], csv_sep,
-				csv_output ? 0 : 18,
-				counter->supported ? CNTR_NOT_COUNTED : CNTR_NOT_SUPPORTED,
-				csv_sep);
-
-				fprintf(output, "%-*s%s",
-					csv_output ? 0 : unit_width,
-					counter->unit, csv_sep);
-
-				fprintf(output, "%*s",
-					csv_output ? 0 : -25,
-					perf_evsel__name(counter));
-
-			if (counter->cgrp)
-				fprintf(output, "%s%s",
-					csv_sep, counter->cgrp->name);
-
-			print_running(run, ena);
-			fputc('\n', output);
-			continue;
-		}
-
 		uval = val * counter->scale;
 		printout(cpu, 0, counter, uval, prefix, run, ena, 1.0);
 
-- 
2.4.3

Re: [PATCH 1/6] perf, tools, stat: Move sw clock metrics printout to stat-shadow

[Arnaldo Carvalho de Melo]

Em Fri, Sep 04, 2015 at 03:45:12PM -0700, Andi Kleen escreveu:
> From: Andi Kleen <ak@linux.intel.com>
> 
> The sw clock metrics printing was missed in the earlier move to
> stat-shadow of all the other metric printouts. Move it too.
> 
> Acked-by: Jiri Olsa <jolsa@kernel.org>

Merging this one, letting Jiri react to the others,

- Arnaldo

> Signed-off-by: Andi Kleen <ak@linux.intel.com>
> ---
>  tools/perf/builtin-stat.c     | 9 ---------
>  tools/perf/util/stat-shadow.c | 3 +++
>  2 files changed, 3 insertions(+), 9 deletions(-)
> 
> diff --git a/tools/perf/builtin-stat.c b/tools/perf/builtin-stat.c
> index a96fb5c..77e5781 100644
> --- a/tools/perf/builtin-stat.c
> +++ b/tools/perf/builtin-stat.c
> @@ -510,15 +510,6 @@ static void nsec_printout(int id, int nr, struct perf_evsel *evsel, double avg)
>  
>  	if (evsel->cgrp)
>  		fprintf(output, "%s%s", csv_sep, evsel->cgrp->name);
> -
> -	if (csv_output || stat_config.interval)
> -		return;
> -
> -	if (perf_evsel__match(evsel, SOFTWARE, SW_TASK_CLOCK))
> -		fprintf(output, " # %8.3f CPUs utilized          ",
> -			avg / avg_stats(&walltime_nsecs_stats));
> -	else
> -		fprintf(output, "                                   ");
>  }
>  
>  static void abs_printout(int id, int nr, struct perf_evsel *evsel, double avg)
> diff --git a/tools/perf/util/stat-shadow.c b/tools/perf/util/stat-shadow.c
> index 2a5d8d7..625ab3b 100644
> --- a/tools/perf/util/stat-shadow.c
> +++ b/tools/perf/util/stat-shadow.c
> @@ -413,6 +413,9 @@ void perf_stat__print_shadow_stats(FILE *out, struct perf_evsel *evsel,
>  			ratio = total / avg;
>  
>  		fprintf(out, " # %8.0f cycles / elision       ", ratio);
> +	} else if (perf_evsel__match(evsel, SOFTWARE, SW_TASK_CLOCK) &&
> +		   (ratio = avg_stats(&walltime_nsecs_stats)) != 0) {
> +		fprintf(out, " # %8.3f CPUs utilized          ", avg / ratio);
>  	} else if (runtime_nsecs_stats[cpu].n != 0) {
>  		char unit = 'M';
>  
> -- 
> 2.4.3

Re: [PATCH 2/6] perf, tools, stat: Abstract stat metrics printing

[Jiri Olsa]

On Fri, Sep 04, 2015 at 03:45:13PM -0700, Andi Kleen wrote:

SNIP

> @@ -695,11 +743,7 @@ static void print_counter_aggr(struct perf_evsel *counter, char *prefix)
>  	}
>  
>  	uval = avg * counter->scale;
> -
> -	if (nsec_counter(counter))
> -		nsec_printout(-1, 0, counter, uval);
> -	else
> -		abs_printout(-1, 0, counter, uval);
> +	printout(-1, 0, counter, uval);
>  
>  	print_noise(counter, avg);
>  
> @@ -752,15 +796,9 @@ static void print_counter(struct perf_evsel *counter, char *prefix)
>  		}
>  
>  		uval = val * counter->scale;
> -
> -		if (nsec_counter(counter))
> -			nsec_printout(cpu, 0, counter, uval);
> -		else
> -			abs_printout(cpu, 0, counter, uval);
> -
> +		printout(cpu, 0, counter, uval);
>  		if (!csv_output)
>  			print_noise(counter, 1.0);
> -		print_running(run, ena);

why is this one removed?

jirka

Re: [PATCH 2/6] perf, tools, stat: Abstract stat metrics printing

[Jiri Olsa]

On Fri, Sep 04, 2015 at 03:45:13PM -0700, Andi Kleen wrote:
> From: Andi Kleen <ak@linux.intel.com>
> 
> Abstract the printing of shadow metrics. Instead of every
> metric calling fprintf directly and taking care of indentation,
> use two call backs: one to print metrics and another to
> start a new line.
> 
> This will allow adding metrics to CSV mode and also
> using them for other purposes.
> 
> The computation of padding is now done in the central
> callback, instead of every metric doing it manually.
> This makes it easier to add new metrics.
> 
> v2: Refactor functions, printout now does more. Move
> shadow printing. Improve fallback callbacks. Don't
> use void * callback data.
> v3: Remove unnecessary hunk. Add typedef for new_line
> Signed-off-by: Andi Kleen <ak@linux.intel.com>


before this patch:

[jolsa@krava perf]$ ./perf stat -e cycles  -I 1000 -x, yes > /dev/null
    28.002841829,2907511555,,cycles,1000612137,100.00
    29.002944242,2893835898,,cycles,1000615661,100.00
    30.003043253,2910405767,,cycles,1000618513,100.00
    31.003140585,2909645607,,cycles,1000621442,100.00
    32.003244286,2910397988,,cycles,1000624150,100.00
    33.003347139,2908927486,,cycles,1000626890,100.00
^C    33.581210913,1677490047,,cycles,988203059,100.00
    33.581315820,157136,,cycles,959970233,100.00
yes: Interrupt

after this patch:
[jolsa@krava perf]$ ./perf stat -e cycles  -I 1000 -x, yes > /dev/null
     1.000101739,3074999967,,cycles #  Ghz                             ,1000247731,100.00
     2.000223351,3019361712,,cycles #  Ghz                             ,999945890,100.00
     3.000327235,3012966500,,cycles #  Ghz                             ,999805678,100.00
     4.000430983,3009829158,,cycles #  Ghz                             ,999871044,100.00


I think the reason is that this patch does too much and needs
to be separated to smaller changes, like:

  - introduce the print_metrics interface for perf_stat__print_shadow_stats
  - introduce printout function
  - introduce support for printing metrics in csv in interval mode
  - and enable it


or any other way you find feasible *easy to review*.. I cannot ack it at this state

jirka


> ---
>  tools/perf/builtin-stat.c     | 100 ++++++++++++++------
>  tools/perf/util/stat-shadow.c | 212 +++++++++++++++++++++++-------------------
>  tools/perf/util/stat.h        |  15 ++-
>  3 files changed, 197 insertions(+), 130 deletions(-)
> 

SNIP

> @@ -544,12 +581,32 @@ static void abs_printout(int id, int nr, struct perf_evsel *evsel, double avg)
>  
>  	if (evsel->cgrp)
>  		fprintf(output, "%s%s", csv_sep, evsel->cgrp->name);
> +}
>  
> -	if (csv_output || stat_config.interval)
> -		return;

now you printout metrics for interval/csv


> +static void printout(int id, int nr, struct perf_evsel *counter, double uval)
> +{
> +	struct outstate os = { .fh = stat_config.output };
> +	struct perf_stat_output_ctx out;
> +	print_metric_t pm = print_metric_std;
> +	void (*nl)(void *);
> +
> +	if (stat_config.aggr_mode == AGGR_NONE)
> +		nl = new_line_no_aggr_std;
> +	else
> +		nl = new_line_std;
>  
> -	perf_stat__print_shadow_stats(output, evsel, avg, cpu,
> -				      stat_config.aggr_mode);
> +	if (nsec_counter(counter))
> +		nsec_printout(id, nr, counter, uval);
> +	else
> +		abs_printout(id, nr, counter, uval);
> +
> +	out.print_metric = pm;
> +	out.new_line = nl;
> +	out.ctx = &os;
> +	perf_stat__print_shadow_stats(counter, uval,
> +				stat_config.aggr_mode == AGGR_GLOBAL ? 0 :
> +				cpu_map__id_to_cpu(id),
> +				&out);
>  }
>  
>  static void print_aggr(char *prefix)
> @@ -605,12 +662,7 @@ static void print_aggr(char *prefix)
>  				continue;
>  			}
>  			uval = val * counter->scale;
> -
> -			if (nsec_counter(counter))
> -				nsec_printout(id, nr, counter, uval);
> -			else
> -				abs_printout(id, nr, counter, uval);
> -
> +			printout(id, nr, counter, uval);
>  			if (!csv_output)
>  				print_noise(counter, 1.0);
>  
> @@ -641,11 +693,7 @@ static void print_aggr_thread(struct perf_evsel *counter, char *prefix)
>  			fprintf(output, "%s", prefix);
>  
>  		uval = val * counter->scale;
> -
> -		if (nsec_counter(counter))
> -			nsec_printout(thread, 0, counter, uval);
> -		else
> -			abs_printout(thread, 0, counter, uval);
> +		printout(thread, 0, counter, uval);
>  
>  		if (!csv_output)
>  			print_noise(counter, 1.0);
> @@ -695,11 +743,7 @@ static void print_counter_aggr(struct perf_evsel *counter, char *prefix)
>  	}
>  
>  	uval = avg * counter->scale;
> -
> -	if (nsec_counter(counter))
> -		nsec_printout(-1, 0, counter, uval);
> -	else
> -		abs_printout(-1, 0, counter, uval);
> +	printout(-1, 0, counter, uval);
>  
>  	print_noise(counter, avg);
>  
> @@ -752,15 +796,9 @@ static void print_counter(struct perf_evsel *counter, char *prefix)
>  		}
>  
>  		uval = val * counter->scale;
> -
> -		if (nsec_counter(counter))
> -			nsec_printout(cpu, 0, counter, uval);
> -		else
> -			abs_printout(cpu, 0, counter, uval);
> -
> +		printout(cpu, 0, counter, uval);
>  		if (!csv_output)
>  			print_noise(counter, 1.0);
> -		print_running(run, ena);
>  
>  		fputc('\n', output);
>  	}
> diff --git a/tools/perf/util/stat-shadow.c b/tools/perf/util/stat-shadow.c
> index 625ab3b..d7d27b5 100644
> --- a/tools/perf/util/stat-shadow.c
> +++ b/tools/perf/util/stat-shadow.c
> @@ -137,9 +137,10 @@ static const char *get_ratio_color(enum grc_type type, double ratio)
>  	return color;
>  }
>  
> -static void print_stalled_cycles_frontend(FILE *out, int cpu,
> +static void print_stalled_cycles_frontend(int cpu,
>  					  struct perf_evsel *evsel
> -					  __maybe_unused, double avg)
> +					  __maybe_unused, double avg,
> +					  struct perf_stat_output_ctx *out)
>  {
>  	double total, ratio = 0.0;
>  	const char *color;
> @@ -152,14 +153,13 @@ static void print_stalled_cycles_frontend(FILE *out, int cpu,
>  
>  	color = get_ratio_color(GRC_STALLED_CYCLES_FE, ratio);
>  
> -	fprintf(out, " #  ");
> -	color_fprintf(out, color, "%6.2f%%", ratio);
> -	fprintf(out, " frontend cycles idle   ");
> +	out->print_metric(out->ctx, color, "%7.2f%%", "frontend cycles idle", ratio);
>  }
>  
> -static void print_stalled_cycles_backend(FILE *out, int cpu,
> +static void print_stalled_cycles_backend(int cpu,
>  					 struct perf_evsel *evsel
> -					 __maybe_unused, double avg)
> +					 __maybe_unused, double avg,
> +					 struct perf_stat_output_ctx *out)
>  {
>  	double total, ratio = 0.0;
>  	const char *color;
> @@ -172,14 +172,13 @@ static void print_stalled_cycles_backend(FILE *out, int cpu,
>  
>  	color = get_ratio_color(GRC_STALLED_CYCLES_BE, ratio);
>  
> -	fprintf(out, " #  ");
> -	color_fprintf(out, color, "%6.2f%%", ratio);
> -	fprintf(out, " backend  cycles idle   ");
> +	out->print_metric(out->ctx, color, "%6.2f%%", "backend cycles idle", ratio);
>  }
>  
> -static void print_branch_misses(FILE *out, int cpu,
> +static void print_branch_misses(int cpu,
>  				struct perf_evsel *evsel __maybe_unused,
> -				double avg)
> +				double avg,
> +				struct perf_stat_output_ctx *out)
>  {
>  	double total, ratio = 0.0;
>  	const char *color;
> @@ -192,14 +191,13 @@ static void print_branch_misses(FILE *out, int cpu,
>  
>  	color = get_ratio_color(GRC_CACHE_MISSES, ratio);
>  
> -	fprintf(out, " #  ");
> -	color_fprintf(out, color, "%6.2f%%", ratio);
> -	fprintf(out, " of all branches        ");
> +	out->print_metric(out->ctx, color, "%7.2f%%", "of all branches", ratio);
>  }
>  
> -static void print_l1_dcache_misses(FILE *out, int cpu,
> +static void print_l1_dcache_misses(int cpu,
>  				   struct perf_evsel *evsel __maybe_unused,
> -				   double avg)
> +				   double avg,
> +				   struct perf_stat_output_ctx *out)
>  {
>  	double total, ratio = 0.0;
>  	const char *color;
> @@ -212,14 +210,13 @@ static void print_l1_dcache_misses(FILE *out, int cpu,
>  
>  	color = get_ratio_color(GRC_CACHE_MISSES, ratio);
>  
> -	fprintf(out, " #  ");
> -	color_fprintf(out, color, "%6.2f%%", ratio);
> -	fprintf(out, " of all L1-dcache hits  ");
> +	out->print_metric(out->ctx, color, "%7.2f%%", "of all L1-dcache hits", ratio);
>  }
>  
> -static void print_l1_icache_misses(FILE *out, int cpu,
> +static void print_l1_icache_misses(int cpu,
>  				   struct perf_evsel *evsel __maybe_unused,
> -				   double avg)
> +				   double avg,
> +				   struct perf_stat_output_ctx *out)
>  {
>  	double total, ratio = 0.0;
>  	const char *color;
> @@ -231,15 +228,13 @@ static void print_l1_icache_misses(FILE *out, int cpu,
>  		ratio = avg / total * 100.0;
>  
>  	color = get_ratio_color(GRC_CACHE_MISSES, ratio);
> -
> -	fprintf(out, " #  ");
> -	color_fprintf(out, color, "%6.2f%%", ratio);
> -	fprintf(out, " of all L1-icache hits  ");
> +	out->print_metric(out->ctx, color, "%7.2f%%", "of all L1-icache hits", ratio);
>  }
>  
> -static void print_dtlb_cache_misses(FILE *out, int cpu,
> +static void print_dtlb_cache_misses(int cpu,
>  				    struct perf_evsel *evsel __maybe_unused,
> -				    double avg)
> +				    double avg,
> +				    struct perf_stat_output_ctx *out)
>  {
>  	double total, ratio = 0.0;
>  	const char *color;
> @@ -251,15 +246,13 @@ static void print_dtlb_cache_misses(FILE *out, int cpu,
>  		ratio = avg / total * 100.0;
>  
>  	color = get_ratio_color(GRC_CACHE_MISSES, ratio);
> -
> -	fprintf(out, " #  ");
> -	color_fprintf(out, color, "%6.2f%%", ratio);
> -	fprintf(out, " of all dTLB cache hits ");
> +	out->print_metric(out->ctx, color, "%7.2f%%", "of all dTLB cache hits", ratio);
>  }
>  
> -static void print_itlb_cache_misses(FILE *out, int cpu,
> +static void print_itlb_cache_misses(int cpu,
>  				    struct perf_evsel *evsel __maybe_unused,
> -				    double avg)
> +				    double avg,
> +				    struct perf_stat_output_ctx *out)
>  {
>  	double total, ratio = 0.0;
>  	const char *color;
> @@ -271,15 +264,13 @@ static void print_itlb_cache_misses(FILE *out, int cpu,
>  		ratio = avg / total * 100.0;
>  
>  	color = get_ratio_color(GRC_CACHE_MISSES, ratio);
> -
> -	fprintf(out, " #  ");
> -	color_fprintf(out, color, "%6.2f%%", ratio);
> -	fprintf(out, " of all iTLB cache hits ");
> +	out->print_metric(out->ctx, color, "%7.2f%%", "of all iTLB cache hits", ratio);
>  }
>  
> -static void print_ll_cache_misses(FILE *out, int cpu,
> +static void print_ll_cache_misses(int cpu,
>  				  struct perf_evsel *evsel __maybe_unused,
> -				  double avg)
> +				  double avg,
> +				  struct perf_stat_output_ctx *out)
>  {
>  	double total, ratio = 0.0;
>  	const char *color;
> @@ -291,15 +282,15 @@ static void print_ll_cache_misses(FILE *out, int cpu,
>  		ratio = avg / total * 100.0;
>  
>  	color = get_ratio_color(GRC_CACHE_MISSES, ratio);
> -
> -	fprintf(out, " #  ");
> -	color_fprintf(out, color, "%6.2f%%", ratio);
> -	fprintf(out, " of all LL-cache hits   ");
> +	out->print_metric(out->ctx, color, "%7.2f%%", "of all LL-cache hits", ratio);
>  }
>  
> -void perf_stat__print_shadow_stats(FILE *out, struct perf_evsel *evsel,
> -				   double avg, int cpu, enum aggr_mode aggr)
> +void perf_stat__print_shadow_stats(struct perf_evsel *evsel,
> +				   double avg, int cpu,
> +				   struct perf_stat_output_ctx *out)
>  {
> +	void *ctxp = out->ctx;
> +	print_metric_t print_metric = out->print_metric;
>  	double total, ratio = 0.0, total2;
>  	int ctx = evsel_context(evsel);
>  
> @@ -307,117 +298,144 @@ void perf_stat__print_shadow_stats(FILE *out, struct perf_evsel *evsel,
>  		total = avg_stats(&runtime_cycles_stats[ctx][cpu]);
>  		if (total) {
>  			ratio = avg / total;
> -			fprintf(out, " #   %5.2f  insns per cycle        ", ratio);
> +			print_metric(ctxp, NULL, "%7.2f ",
> +					"insn per cycle", ratio);
>  		} else {
> -			fprintf(out, "                                   ");
> +			print_metric(ctxp, NULL, NULL, "insn per cycle", 0);
>  		}
>  		total = avg_stats(&runtime_stalled_cycles_front_stats[ctx][cpu]);
>  		total = max(total, avg_stats(&runtime_stalled_cycles_back_stats[ctx][cpu]));
>  
>  		if (total && avg) {
>  			ratio = total / avg;
> -			fprintf(out, "\n");
> -			if (aggr == AGGR_NONE)
> -				fprintf(out, "        ");
> -			fprintf(out, "                                                  #   %5.2f  stalled cycles per insn", ratio);
> +			out->new_line(ctxp);
> +			print_metric(ctxp, NULL, "%7.2f ",
> +					"stalled cycles per insn",
> +					ratio);
> +		} else {
> +			print_metric(ctxp, NULL, NULL,
> +				     "stalled cycles per insn", 0);
>  		}
> -
> -	} else if (perf_evsel__match(evsel, HARDWARE, HW_BRANCH_MISSES) &&
> -			runtime_branches_stats[ctx][cpu].n != 0) {
> -		print_branch_misses(out, cpu, evsel, avg);
> +	} else if (perf_evsel__match(evsel, HARDWARE, HW_BRANCH_MISSES)) {
> +		if (runtime_branches_stats[ctx][cpu].n != 0)
> +			print_branch_misses(cpu, evsel, avg, out);
> +		else
> +			print_metric(ctxp, NULL, NULL, "of all branches", 0);
>  	} else if (
>  		evsel->attr.type == PERF_TYPE_HW_CACHE &&
>  		evsel->attr.config ==  ( PERF_COUNT_HW_CACHE_L1D |
>  					((PERF_COUNT_HW_CACHE_OP_READ) << 8) |
> -					((PERF_COUNT_HW_CACHE_RESULT_MISS) << 16)) &&
> -			runtime_l1_dcache_stats[ctx][cpu].n != 0) {
> -		print_l1_dcache_misses(out, cpu, evsel, avg);
> +					 ((PERF_COUNT_HW_CACHE_RESULT_MISS) << 16))) {
> +		if (runtime_l1_dcache_stats[ctx][cpu].n != 0)
> +			print_l1_dcache_misses(cpu, evsel, avg, out);
> +		else
> +			print_metric(ctxp, NULL, NULL, "of all L1-dcache hits", 0);
>  	} else if (
>  		evsel->attr.type == PERF_TYPE_HW_CACHE &&
>  		evsel->attr.config ==  ( PERF_COUNT_HW_CACHE_L1I |
>  					((PERF_COUNT_HW_CACHE_OP_READ) << 8) |
> -					((PERF_COUNT_HW_CACHE_RESULT_MISS) << 16)) &&
> -			runtime_l1_icache_stats[ctx][cpu].n != 0) {
> -		print_l1_icache_misses(out, cpu, evsel, avg);
> +					 ((PERF_COUNT_HW_CACHE_RESULT_MISS) << 16))) {
> +		if (runtime_l1_icache_stats[ctx][cpu].n != 0)
> +			print_l1_icache_misses(cpu, evsel, avg, out);
> +		else
> +			print_metric(ctxp, NULL, NULL, "of all L1-icache hits", 0);
>  	} else if (
>  		evsel->attr.type == PERF_TYPE_HW_CACHE &&
>  		evsel->attr.config ==  ( PERF_COUNT_HW_CACHE_DTLB |
>  					((PERF_COUNT_HW_CACHE_OP_READ) << 8) |
> -					((PERF_COUNT_HW_CACHE_RESULT_MISS) << 16)) &&
> -			runtime_dtlb_cache_stats[ctx][cpu].n != 0) {
> -		print_dtlb_cache_misses(out, cpu, evsel, avg);
> +					 ((PERF_COUNT_HW_CACHE_RESULT_MISS) << 16))) {
> +		if (runtime_dtlb_cache_stats[ctx][cpu].n != 0)
> +			print_dtlb_cache_misses(cpu, evsel, avg, out);
> +		else
> +			print_metric(ctxp, NULL, NULL, "of all dTLB cache hits", 0);
>  	} else if (
>  		evsel->attr.type == PERF_TYPE_HW_CACHE &&
>  		evsel->attr.config ==  ( PERF_COUNT_HW_CACHE_ITLB |
>  					((PERF_COUNT_HW_CACHE_OP_READ) << 8) |
> -					((PERF_COUNT_HW_CACHE_RESULT_MISS) << 16)) &&
> -			runtime_itlb_cache_stats[ctx][cpu].n != 0) {
> -		print_itlb_cache_misses(out, cpu, evsel, avg);
> +					 ((PERF_COUNT_HW_CACHE_RESULT_MISS) << 16))) {
> +		if (runtime_itlb_cache_stats[ctx][cpu].n != 0)
> +			print_itlb_cache_misses(cpu, evsel, avg, out);
> +		else
> +			print_metric(ctxp, NULL, NULL, "of all iTLB cache hits", 0);
>  	} else if (
>  		evsel->attr.type == PERF_TYPE_HW_CACHE &&
>  		evsel->attr.config ==  ( PERF_COUNT_HW_CACHE_LL |
>  					((PERF_COUNT_HW_CACHE_OP_READ) << 8) |
> -					((PERF_COUNT_HW_CACHE_RESULT_MISS) << 16)) &&
> -			runtime_ll_cache_stats[ctx][cpu].n != 0) {
> -		print_ll_cache_misses(out, cpu, evsel, avg);
> -	} else if (perf_evsel__match(evsel, HARDWARE, HW_CACHE_MISSES) &&
> -			runtime_cacherefs_stats[ctx][cpu].n != 0) {
> +					 ((PERF_COUNT_HW_CACHE_RESULT_MISS) << 16))) {
> +		if (runtime_ll_cache_stats[ctx][cpu].n != 0)
> +			print_ll_cache_misses(cpu, evsel, avg, out);
> +		else
> +			print_metric(ctxp, NULL, NULL, "of all LL-cache hits", 0);
> +	} else if (perf_evsel__match(evsel, HARDWARE, HW_CACHE_MISSES)) {
>  		total = avg_stats(&runtime_cacherefs_stats[ctx][cpu]);
>  
>  		if (total)
>  			ratio = avg * 100 / total;
>  
> -		fprintf(out, " # %8.3f %% of all cache refs    ", ratio);
> -
> +		if (runtime_cacherefs_stats[ctx][cpu].n != 0)
> +			print_metric(ctxp, NULL, "%8.3f %%",
> +				     "of all cache refs", ratio);
> +		else
> +			print_metric(ctxp, NULL, NULL, "of all cache refs", 0);
>  	} else if (perf_evsel__match(evsel, HARDWARE, HW_STALLED_CYCLES_FRONTEND)) {
> -		print_stalled_cycles_frontend(out, cpu, evsel, avg);
> +		print_stalled_cycles_frontend(cpu, evsel, avg, out);
>  	} else if (perf_evsel__match(evsel, HARDWARE, HW_STALLED_CYCLES_BACKEND)) {
> -		print_stalled_cycles_backend(out, cpu, evsel, avg);
> +		print_stalled_cycles_backend(cpu, evsel, avg, out);
>  	} else if (perf_evsel__match(evsel, HARDWARE, HW_CPU_CYCLES)) {
>  		total = avg_stats(&runtime_nsecs_stats[cpu]);
>  
>  		if (total) {
>  			ratio = avg / total;
> -			fprintf(out, " # %8.3f GHz                    ", ratio);
> +			print_metric(ctxp, NULL, "%8.3f", "GHz", ratio);
>  		} else {
> -			fprintf(out, "                                   ");
> +			print_metric(ctxp, NULL, NULL, "Ghz", 0);
>  		}
>  	} else if (perf_stat_evsel__is(evsel, CYCLES_IN_TX)) {
>  		total = avg_stats(&runtime_cycles_stats[ctx][cpu]);
>  		if (total)
> -			fprintf(out,
> -				" #   %5.2f%% transactional cycles   ",
> -				100.0 * (avg / total));
> +			print_metric(ctxp, NULL,
> +					"%7.2f%%", "transactional cycles",
> +					100.0 * (avg / total));
> +		else
> +			print_metric(ctxp, NULL, NULL, "transactional cycles",
> +				     0);
>  	} else if (perf_stat_evsel__is(evsel, CYCLES_IN_TX_CP)) {
>  		total = avg_stats(&runtime_cycles_stats[ctx][cpu]);
>  		total2 = avg_stats(&runtime_cycles_in_tx_stats[ctx][cpu]);
>  		if (total2 < avg)
>  			total2 = avg;
>  		if (total)
> -			fprintf(out,
> -				" #   %5.2f%% aborted cycles         ",
> +			print_metric(ctxp, NULL, "%7.2f%%", "aborted cycles",
>  				100.0 * ((total2-avg) / total));
> -	} else if (perf_stat_evsel__is(evsel, TRANSACTION_START) &&
> -		   runtime_cycles_in_tx_stats[ctx][cpu].n != 0) {
> +		else
> +			print_metric(ctxp, NULL, NULL, "aborted cycles", 0);
> +	} else if (perf_stat_evsel__is(evsel, TRANSACTION_START)) {
>  		total = avg_stats(&runtime_cycles_in_tx_stats[ctx][cpu]);
>  
>  		if (avg)
>  			ratio = total / avg;
>  
> -		fprintf(out, " # %8.0f cycles / transaction   ", ratio);
> -	} else if (perf_stat_evsel__is(evsel, ELISION_START) &&
> -		   runtime_cycles_in_tx_stats[ctx][cpu].n != 0) {
> +		if (runtime_cycles_in_tx_stats[ctx][cpu].n != 0)
> +			print_metric(ctxp, NULL, "%8.0f",
> +				     "cycles / transaction", ratio);
> +		else
> +			print_metric(ctxp, NULL, NULL, "cycles / transaction",
> +				     0);
> +	} else if (perf_stat_evsel__is(evsel, ELISION_START)) {
>  		total = avg_stats(&runtime_cycles_in_tx_stats[ctx][cpu]);
>  
>  		if (avg)
>  			ratio = total / avg;
> -
> -		fprintf(out, " # %8.0f cycles / elision       ", ratio);
> -	} else if (perf_evsel__match(evsel, SOFTWARE, SW_TASK_CLOCK) &&
> -		   (ratio = avg_stats(&walltime_nsecs_stats)) != 0) {
> -		fprintf(out, " # %8.3f CPUs utilized          ", avg / ratio);
> +		print_metric(ctxp, NULL, "%8.0f", "cycles / elision", ratio);
> +	} else if (perf_evsel__match(evsel, SOFTWARE, SW_TASK_CLOCK)) {
> +		if ((ratio = avg_stats(&walltime_nsecs_stats)) != 0)
> +			print_metric(ctxp, NULL, "%8.3f", "CPUs utilized",
> +				     avg / ratio);
> +		else
> +			print_metric(ctxp, NULL, NULL, "CPUs utilized", 0);
>  	} else if (runtime_nsecs_stats[cpu].n != 0) {
>  		char unit = 'M';
> +		char unit_buf[10];
>  
>  		total = avg_stats(&runtime_nsecs_stats[cpu]);
>  
> @@ -427,9 +445,9 @@ void perf_stat__print_shadow_stats(FILE *out, struct perf_evsel *evsel,
>  			ratio *= 1000;
>  			unit = 'K';
>  		}
> -
> -		fprintf(out, " # %8.3f %c/sec                  ", ratio, unit);
> +		snprintf(unit_buf, sizeof(unit_buf), "%c/sec", unit);
> +		print_metric(ctxp, NULL, "%8.3f", unit_buf, ratio);
>  	} else {
> -		fprintf(out, "                                   ");
> +		print_metric(ctxp, NULL, NULL, NULL, 0);
>  	}
>  }
> diff --git a/tools/perf/util/stat.h b/tools/perf/util/stat.h
> index 62448c8..e1899f6 100644
> --- a/tools/perf/util/stat.h
> +++ b/tools/perf/util/stat.h
> @@ -67,11 +67,22 @@ void perf_stat_evsel_id_init(struct perf_evsel *evsel);
>  
>  extern struct stats walltime_nsecs_stats;
>  
> +typedef void (*print_metric_t)(void *ctx, const char *color, const char *unit,
> +			       const char *fmt, double val);
> +typedef void (*new_line_t )(void *ctx);
> +
>  void perf_stat__reset_shadow_stats(void);
>  void perf_stat__update_shadow_stats(struct perf_evsel *counter, u64 *count,
>  				    int cpu);
> -void perf_stat__print_shadow_stats(FILE *out, struct perf_evsel *evsel,
> -				   double avg, int cpu, enum aggr_mode aggr);
> +struct perf_stat_output_ctx {
> +	void *ctx;
> +	print_metric_t print_metric;
> +	new_line_t new_line;
> +};
> +
> +void perf_stat__print_shadow_stats(struct perf_evsel *evsel,
> +				   double avg, int cpu,
> +				   struct perf_stat_output_ctx *out);
>  
>  void perf_evsel__reset_stat_priv(struct perf_evsel *evsel);
>  int perf_evsel__alloc_stat_priv(struct perf_evsel *evsel);
> -- 
> 2.4.3
> 

Re: [PATCH 2/6] perf, tools, stat: Abstract stat metrics printing

[Andi Kleen]

>   - introduce the print_metrics interface for perf_stat__print_shadow_stats
>   - introduce printout function
>   - introduce support for printing metrics in csv in interval mode

That's already a separate patch.

>   - and enable i
> 
> 
> or any other way you find feasible *easy to review*.. I cannot ack it at this state

Well it's just printing out values. It's really not rocket science. Nobody
is asking you to review a new VM subsystems or something like that.

I expect each more "refactoring" to add more problems. It's just really hard to to do,
and usually adds more problems.

I can try one more time, but if it doesn't work out we'll have to give up on this.

-Andi

Re: [PATCH 2/6] perf, tools, stat: Abstract stat metrics printing

[Jiri Olsa]

On Tue, Sep 08, 2015 at 09:03:17AM -0700, Andi Kleen wrote:
> >   - introduce the print_metrics interface for perf_stat__print_shadow_stats
> >   - introduce printout function
> >   - introduce support for printing metrics in csv in interval mode
> 
> That's already a separate patch.
> 
> >   - and enable i
> > 
> > 
> > or any other way you find feasible *easy to review*.. I cannot ack it at this state
> 
> Well it's just printing out values. It's really not rocket science. Nobody
> is asking you to review a new VM subsystems or something like that.

right, it's easy.. and it should be done in 'easy to review' way

> 
> I expect each more "refactoring" to add more problems. It's just really hard to to do,
> and usually adds more problems.

yep, that's why we test things

> 
> I can try one more time, but if it doesn't work out we'll have to give up on this.

ok

jirka