[Qemu-devel] [PATCH] arm: Add PMU test

[Qemu-devel] [PATCH] arm: Add PMU test

[Christopher Covington]

Beginning with just a read of the control register, add plumbing
for testing the ARM Performance Monitors Unit (PMU).

Signed-off-by: Christopher Covington <cov@codeaurora.org>
---
 arm/pmu.c                    | 31 +++++++++++++++++++++++++++++++
 arm/unittests.cfg            |  5 +++++
 config/config-arm-common.mak |  4 +++-
 3 files changed, 39 insertions(+), 1 deletion(-)
 create mode 100644 arm/pmu.c

diff --git a/arm/pmu.c b/arm/pmu.c
new file mode 100644
index 0000000..b1e3c7a
--- /dev/null
+++ b/arm/pmu.c
@@ -0,0 +1,31 @@
+#include "libcflat.h"
+
+union pmcr_el0 {
+	struct {
+		unsigned int implementor:8;
+		unsigned int identification_code:8;
+		unsigned int num_counters:5;
+		unsigned int zeros:4;
+		unsigned int cycle_counter_long:1;
+		unsigned int cycle_counter_disable_when_prohibited:1;
+		unsigned int event_counter_export:1;
+		unsigned int cycle_counter_clock_divider:1;
+		unsigned int cycle_counter_reset:1;
+		unsigned int event_counter_reset:1;
+		unsigned int enable:1;
+	} split;
+	uint32_t full;
+};
+
+int main()
+{
+	union pmcr_el0 pmcr;
+
+	asm volatile("mrs %0, pmcr_el0\n" : "=r" (pmcr));
+
+	printf("PMU implementor:     0x%x\n", pmcr.split.implementor);
+	printf("Identification code: 0x%x\n", pmcr.split.identification_code);
+	printf("Event counters:      %d\n", pmcr.split.num_counters);
+
+	return report_summary();
+}
diff --git a/arm/unittests.cfg b/arm/unittests.cfg
index e068a0c..d3deb6a 100644
--- a/arm/unittests.cfg
+++ b/arm/unittests.cfg
@@ -35,3 +35,8 @@ file = selftest.flat
 smp = `getconf _NPROCESSORS_CONF`
 extra_params = -append 'smp'
 groups = selftest
+
+# Test PMU support
+[pmu]
+file = pmu.flat
+groups = pmu
diff --git a/config/config-arm-common.mak b/config/config-arm-common.mak
index 698555d..b34d04c 100644
--- a/config/config-arm-common.mak
+++ b/config/config-arm-common.mak
@@ -11,7 +11,8 @@ endif
 
 tests-common = \
 	$(TEST_DIR)/selftest.flat \
-	$(TEST_DIR)/spinlock-test.flat
+	$(TEST_DIR)/spinlock-test.flat \
+	$(TEST_DIR)/pmu.flat
 
 all: test_cases
 
@@ -70,3 +71,4 @@ test_cases: $(generated_files) $(tests-common) $(tests)
 
 $(TEST_DIR)/selftest.elf: $(cstart.o) $(TEST_DIR)/selftest.o
 $(TEST_DIR)/spinlock-test.elf: $(cstart.o) $(TEST_DIR)/spinlock-test.o
+$(TEST_DIR)/pmu.elf: $(cstart.o) $(TEST_DIR)/pmu.o
-- 
Qualcomm Innovation Center, Inc.
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project

Re: [Qemu-devel] [PATCH] arm: Add PMU test

[Andrew Jones]

On Thu, Oct 01, 2015 at 03:47:21PM -0400, Christopher Covington wrote:
> Beginning with just a read of the control register, add plumbing
> for testing the ARM Performance Monitors Unit (PMU).
> 
> Signed-off-by: Christopher Covington <cov@codeaurora.org>

Hi Christopher,

PMU tests are a great idea. I think Wei Huang has started working
on them too. I've CC'ed him so you two can work out how to proceed
without duplication of effort.

For kvm-unit-tests patches please make sure the kvm-unit-tests flag
is in the PATCH tag. You may want to run

  git config format.subjectprefix "kvm-unit-tests PATCH"

in your git repo.


> ---
>  arm/pmu.c                    | 31 +++++++++++++++++++++++++++++++
>  arm/unittests.cfg            |  5 +++++
>  config/config-arm-common.mak |  4 +++-
>  3 files changed, 39 insertions(+), 1 deletion(-)
>  create mode 100644 arm/pmu.c
> 
> diff --git a/arm/pmu.c b/arm/pmu.c
> new file mode 100644
> index 0000000..b1e3c7a
> --- /dev/null
> +++ b/arm/pmu.c
> @@ -0,0 +1,31 @@
> +#include "libcflat.h"
> +
> +union pmcr_el0 {
> +	struct {
> +		unsigned int implementor:8;
> +		unsigned int identification_code:8;
> +		unsigned int num_counters:5;
> +		unsigned int zeros:4;
> +		unsigned int cycle_counter_long:1;
> +		unsigned int cycle_counter_disable_when_prohibited:1;
> +		unsigned int event_counter_export:1;
> +		unsigned int cycle_counter_clock_divider:1;
> +		unsigned int cycle_counter_reset:1;
> +		unsigned int event_counter_reset:1;
> +		unsigned int enable:1;
> +	} split;

I'd use an anonymous struct here. And, since I prefer to avoid needing
to remember a data structure is a union, I'd also make the union anonymous
and wrap it inside a struct, i.e.

struct pmu_data {
        union {
                uint32_t pmcr_el0;
                struct {
			...
		};
	};
};

> +	uint32_t full;
> +};
> +
> +int main()
> +{
> +	union pmcr_el0 pmcr;
> +
> +	asm volatile("mrs %0, pmcr_el0\n" : "=r" (pmcr));
> +
> +	printf("PMU implementor:     0x%x\n", pmcr.split.implementor);
> +	printf("Identification code: 0x%x\n", pmcr.split.identification_code);
> +	printf("Event counters:      %d\n", pmcr.split.num_counters);
> +
> +	return report_summary();
> +}
> diff --git a/arm/unittests.cfg b/arm/unittests.cfg
> index e068a0c..d3deb6a 100644
> --- a/arm/unittests.cfg
> +++ b/arm/unittests.cfg
> @@ -35,3 +35,8 @@ file = selftest.flat
>  smp = `getconf _NPROCESSORS_CONF`
>  extra_params = -append 'smp'
>  groups = selftest
> +
> +# Test PMU support
> +[pmu]
> +file = pmu.flat
> +groups = pmu
> diff --git a/config/config-arm-common.mak b/config/config-arm-common.mak
> index 698555d..b34d04c 100644
> --- a/config/config-arm-common.mak
> +++ b/config/config-arm-common.mak
> @@ -11,7 +11,8 @@ endif
>  
>  tests-common = \
>  	$(TEST_DIR)/selftest.flat \
> -	$(TEST_DIR)/spinlock-test.flat
> +	$(TEST_DIR)/spinlock-test.flat \
> +	$(TEST_DIR)/pmu.flat
>  
>  all: test_cases
>  
> @@ -70,3 +71,4 @@ test_cases: $(generated_files) $(tests-common) $(tests)
>  
>  $(TEST_DIR)/selftest.elf: $(cstart.o) $(TEST_DIR)/selftest.o
>  $(TEST_DIR)/spinlock-test.elf: $(cstart.o) $(TEST_DIR)/spinlock-test.o
> +$(TEST_DIR)/pmu.elf: $(cstart.o) $(TEST_DIR)/pmu.o
> -- 
> Qualcomm Innovation Center, Inc.
> The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
> a Linux Foundation Collaborative Project
> 
>

Thanks,
drew 

[Qemu-devel] [kvm-unit-tests PATCHv2] arm: Add PMU test

[Christopher Covington]

Add test the ARM Performance Monitors Unit (PMU). The informational
fields from the control register are printed, but not checked, and
the number of cycles it takes to run a known-instruction-count loop
is printed, but not checked. Once QEMU is fixed, we can at least
begin to check for IPC == 1 when using -icount.

Signed-off-by: Christopher Covington <cov@codeaurora.org>
---
 arm/pmu.c               | 89 +++++++++++++++++++++++++++++++++++++++++++++++++
 arm/unittests.cfg       | 11 ++++++
 config/config-arm64.mak |  4 ++-
 3 files changed, 103 insertions(+), 1 deletion(-)
 create mode 100644 arm/pmu.c

diff --git a/arm/pmu.c b/arm/pmu.c
new file mode 100644
index 0000000..f724c2c
--- /dev/null
+++ b/arm/pmu.c
@@ -0,0 +1,89 @@
+/*
+ * Test the ARM Performance Monitors Unit (PMU).
+ *
+ * Copyright 2015 The Linux Foundation. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU Lesser General Public License version 2.1 and
+ * only version 2.1 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License
+ * for more details.
+ */
+#include "libcflat.h"
+
+struct pmu_data {
+	union {
+		uint32_t pmcr_el0;
+		struct {
+			unsigned int enable:1;
+			unsigned int event_counter_reset:1;
+			unsigned int cycle_counter_reset:1;
+			unsigned int cycle_counter_clock_divider:1;
+			unsigned int event_counter_export:1;
+			unsigned int cycle_counter_disable_when_prohibited:1;
+			unsigned int cycle_counter_long:1;
+			unsigned int zeros:4;
+			unsigned int num_counters:5;
+			unsigned int identification_code:8;
+			unsigned int implementor:8;
+		};
+	};
+};
+
+/* Execute a known number of guest instructions. Only odd instruction counts
+ * greater than or equal to 3 are supported. The control register (PMCR) is
+ * initialized with the provided value (allowing for example for the cycle
+ * counter or eventer count to be reset if needed). After the known instruction
+ * count loop, zero is written to the PMCR to disable counting, allowing the
+ * cycle counter or event counters to be read as needed at a later time.
+ */
+static void measure_instrs(int len, struct pmu_data pmcr)
+{
+	int i = (len - 1) / 2;
+
+	if (len < 3 || ((len - 1) % 2))
+		abort();
+
+	asm volatile(
+		"msr pmcr_el0, %[pmcr]\n"
+		"1: subs %[i], %[i], #1\n"
+		"b.gt 1b\n"
+		"msr pmcr_el0, xzr"
+	: [i] "+r" (i) : [pmcr] "r" (pmcr) : "cc");
+}
+
+int main()
+{
+	struct pmu_data pmcr;
+	const int samples = 10;
+
+	asm volatile("mrs %0, pmcr_el0" : "=r" (pmcr));
+
+	printf("PMU implementor:     %c\n", pmcr.implementor);
+	printf("Identification code: 0x%x\n", pmcr.identification_code);
+	printf("Event counters:      %d\n", pmcr.num_counters);
+
+	pmcr.cycle_counter_reset = 1;
+	pmcr.enable = 1;
+
+	printf("\ninstructions : cycles0 cycles1 ...\n");
+
+	for (int i = 3; i < 300; i += 32) {
+		int avg, sum = 0;
+		printf("%d :", i);
+		for (int j = 0; j < samples; j++) {
+			int val;
+			measure_instrs(i, pmcr);
+			asm volatile("mrs %0, pmccntr_el0" : "=r" (val));
+			sum += val;
+			printf(" %d", val);
+		}
+		avg = sum / samples;
+		printf(" sum=%d avg=%d avg_ipc=%d avg_cpi=%d\n", sum, avg, i / avg, avg / i);
+	}
+
+	return report_summary();
+}
diff --git a/arm/unittests.cfg b/arm/unittests.cfg
index e068a0c..b3b7ff4 100644
--- a/arm/unittests.cfg
+++ b/arm/unittests.cfg
@@ -35,3 +35,14 @@ file = selftest.flat
 smp = `getconf _NPROCESSORS_CONF`
 extra_params = -append 'smp'
 groups = selftest
+
+# Test PMU support without -icount
+[pmu]
+file = pmu.flat
+groups = pmu
+
+# Test PMU support with -icount
+[pmu-icount]
+file = pmu.flat
+groups = pmu
+extra_params = -icount 0
diff --git a/config/config-arm64.mak b/config/config-arm64.mak
index d61b703..140b611 100644
--- a/config/config-arm64.mak
+++ b/config/config-arm64.mak
@@ -12,9 +12,11 @@ cflatobjs += lib/arm64/processor.o
 cflatobjs += lib/arm64/spinlock.o
 
 # arm64 specific tests
-tests =
+tests = $(TEST_DIR)/pmu.flat
 
 include config/config-arm-common.mak
 
 arch_clean: arm_clean
 	$(RM) lib/arm64/.*.d
+
+$(TEST_DIR)/pmu.elf: $(cstart.o) $(TEST_DIR)/pmu.o
-- 
Qualcomm Innovation Center, Inc.
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project

Re: [Qemu-devel] [kvm-unit-tests PATCHv2] arm: Add PMU test

[Wei Huang]

On 10/02/2015 10:48 AM, Christopher Covington wrote:
> Add test the ARM Performance Monitors Unit (PMU). The informational
> fields from the control register are printed, but not checked, and
> the number of cycles it takes to run a known-instruction-count loop
> is printed, but not checked. Once QEMU is fixed, we can at least
> begin to check for IPC == 1 when using -icount.

Thanks for submitting it. I think this is a good starting point to add
PMU unit testing support for ARM. Some comments below.

> 
> Signed-off-by: Christopher Covington <cov@codeaurora.org>
> ---
>  arm/pmu.c               | 89 +++++++++++++++++++++++++++++++++++++++++++++++++
>  arm/unittests.cfg       | 11 ++++++
>  config/config-arm64.mak |  4 ++-
>  3 files changed, 103 insertions(+), 1 deletion(-)
>  create mode 100644 arm/pmu.c
> 
> diff --git a/arm/pmu.c b/arm/pmu.c
> new file mode 100644
> index 0000000..f724c2c
> --- /dev/null
> +++ b/arm/pmu.c
> @@ -0,0 +1,89 @@
> +/*
> + * Test the ARM Performance Monitors Unit (PMU).
> + *
> + * Copyright 2015 The Linux Foundation. All rights reserved.
> + *
> + * This program is free software; you can redistribute it and/or modify it
> + * under the terms of the GNU Lesser General Public License version 2.1 and
> + * only version 2.1 as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful, but WITHOUT
> + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
> + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License
> + * for more details.
> + */
> +#include "libcflat.h"
> +
> +struct pmu_data {
> +	union {
> +		uint32_t pmcr_el0;
> +		struct {
> +			unsigned int enable:1;
> +			unsigned int event_counter_reset:1;
> +			unsigned int cycle_counter_reset:1;
> +			unsigned int cycle_counter_clock_divider:1;
> +			unsigned int event_counter_export:1;
> +			unsigned int cycle_counter_disable_when_prohibited:1;
> +			unsigned int cycle_counter_long:1;
> +			unsigned int zeros:4;
> +			unsigned int num_counters:5;
> +			unsigned int identification_code:8;
> +			unsigned int implementor:8;
                        ^^^^^^^^
Not saying it is a problem because "unsigned int" is 32-bit on 64bit
machine. But to make it consistent with pmcr_el0, I would prefer
"unsigned int" been replaced by "uint32_t".

> +		};
> +	};
> +};
> +
> +/* Execute a known number of guest instructions. Only odd instruction counts
> + * greater than or equal to 3 are supported. The control register (PMCR) is
> + * initialized with the provided value (allowing for example for the cycle
> + * counter or eventer count to be reset if needed). After the known instruction
> + * count loop, zero is written to the PMCR to disable counting, allowing the
> + * cycle counter or event counters to be read as needed at a later time.
> + */
> +static void measure_instrs(int len, struct pmu_data pmcr)
> +{
> +	int i = (len - 1) / 2;
> +
> +	if (len < 3 || ((len - 1) % 2))
> +		abort();
> +
> +	asm volatile(
> +		"msr pmcr_el0, %[pmcr]\n"
> +		"1: subs %[i], %[i], #1\n"
> +		"b.gt 1b\n"
> +		"msr pmcr_el0, xzr"
> +	: [i] "+r" (i) : [pmcr] "r" (pmcr) : "cc");
> +}
> +
> +int main()
> +{
> +	struct pmu_data pmcr;
> +	const int samples = 10;
> +
> +	asm volatile("mrs %0, pmcr_el0" : "=r" (pmcr));
> +
> +	printf("PMU implementor:     %c\n", pmcr.implementor);
> +	printf("Identification code: 0x%x\n", pmcr.identification_code);
> +	printf("Event counters:      %d\n", pmcr.num_counters);
> +
> +	pmcr.cycle_counter_reset = 1;
> +	pmcr.enable = 1;
> +
> +	printf("\ninstructions : cycles0 cycles1 ...\n");
> +
> +	for (int i = 3; i < 300; i += 32) {
> +		int avg, sum = 0;
> +		printf("%d :", i);
> +		for (int j = 0; j < samples; j++) {
> +			int val;
> +			measure_instrs(i, pmcr);
> +			asm volatile("mrs %0, pmccntr_el0" : "=r" (val));
> +			sum += val;
> +			printf(" %d", val);
> +		}
> +		avg = sum / samples;
> +		printf(" sum=%d avg=%d avg_ipc=%d avg_cpi=%d\n", sum, avg, i / avg, avg / i);
> +	}

I understand that, as stated in commit message, it currently doesn't
check the correctness of PMU counter values. But it would be better if
testing code can be abstracted into an independent function (e.g.
instr_cycle_check) for report("Instruction Cycles",
instr_cycle_check()). You can return TRUE in the checking code for now.


> +
> +	return report_summary();
> +}
> diff --git a/arm/unittests.cfg b/arm/unittests.cfg
> index e068a0c..b3b7ff4 100644
> --- a/arm/unittests.cfg
> +++ b/arm/unittests.cfg
> @@ -35,3 +35,14 @@ file = selftest.flat
>  smp = `getconf _NPROCESSORS_CONF`
>  extra_params = -append 'smp'
>  groups = selftest
> +
> +# Test PMU support without -icount
> +[pmu]
> +file = pmu.flat
> +groups = pmu
> +
> +# Test PMU support with -icount
> +[pmu-icount]
> +file = pmu.flat
> +groups = pmu
> +extra_params = -icount 0
> diff --git a/config/config-arm64.mak b/config/config-arm64.mak
> index d61b703..140b611 100644
> --- a/config/config-arm64.mak
> +++ b/config/config-arm64.mak
> @@ -12,9 +12,11 @@ cflatobjs += lib/arm64/processor.o
>  cflatobjs += lib/arm64/spinlock.o
>  
>  # arm64 specific tests
> -tests =
> +tests = $(TEST_DIR)/pmu.flat
>  
>  include config/config-arm-common.mak
>  
>  arch_clean: arm_clean
>  	$(RM) lib/arm64/.*.d
> +
> +$(TEST_DIR)/pmu.elf: $(cstart.o) $(TEST_DIR)/pmu.o
> 

[Qemu-devel] [kvm-unit-tests PATCHv3] ARM PMU tests

[Christopher Covington]

Changes from v2:

* Explicit test for monotonically increasing cycle count
* Tests now pass or fail
* Tests broken into functions
* Tests/functions broken into separate patches in series
* Style improvements as suggested by Wei Huang and Linux checkpatch.pl
* Spelling and comment improvements

[Qemu-devel] [kvm-unit-tests PATCHv3 1/3] arm: Add PMU test

[Christopher Covington]

Beginning with a simple sanity check of the control register, add
a unit test for the ARM Performance Monitors Unit (PMU).

Signed-off-by: Christopher Covington <cov@codeaurora.org>
---
 arm/pmu.c               | 66 +++++++++++++++++++++++++++++++++++++++++++++++++
 arm/unittests.cfg       |  5 ++++
 config/config-arm64.mak |  4 ++-
 3 files changed, 74 insertions(+), 1 deletion(-)
 create mode 100644 arm/pmu.c

diff --git a/arm/pmu.c b/arm/pmu.c
new file mode 100644
index 0000000..91a3688
--- /dev/null
+++ b/arm/pmu.c
@@ -0,0 +1,66 @@
+/*
+ * Test the ARM Performance Monitors Unit (PMU).
+ *
+ * Copyright 2015 The Linux Foundation. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU Lesser General Public License version 2.1 and
+ * only version 2.1 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License
+ * for more details.
+ */
+#include "libcflat.h"
+
+struct pmu_data {
+	union {
+		uint32_t pmcr_el0;
+		struct {
+			uint32_t enable:1;
+			uint32_t event_counter_reset:1;
+			uint32_t cycle_counter_reset:1;
+			uint32_t cycle_counter_clock_divider:1;
+			uint32_t event_counter_export:1;
+			uint32_t cycle_counter_disable_when_prohibited:1;
+			uint32_t cycle_counter_long:1;
+			uint32_t zeros:4;
+			uint32_t num_counters:5;
+			uint32_t identification_code:8;
+			uint32_t implementer:8;
+		};
+	};
+};
+
+/* As a simple sanity check on the PMCR_EL0, ensure the implementer field isn't
+ * null. Also print out a couple other interesting fields for diagnostic
+ * purposes. For example, as of fall 2015, QEMU TCG mode doesn't implement
+ * event counters and therefore reports zero of them, but hopefully support for
+ * at least the instructions event will be added in the future and the reported
+ * number of event counters will become nonzero.
+ */
+static bool check_pmcr(void)
+{
+	struct pmu_data pmcr;
+
+	asm volatile("mrs %0, pmcr_el0" : "=r" (pmcr));
+
+	printf("PMU implementer:     %c\n", pmcr.implementer);
+	printf("Identification code: 0x%x\n", pmcr.identification_code);
+	printf("Event counters:      %d\n", pmcr.num_counters);
+
+	if (pmcr.implementer)
+		return true;
+
+	return false;
+}
+
+int main(void)
+{
+	report_prefix_push("pmu");
+
+	report("Control register", check_pmcr());
+
+	return report_summary();
+}
diff --git a/arm/unittests.cfg b/arm/unittests.cfg
index e068a0c..fd94adb 100644
--- a/arm/unittests.cfg
+++ b/arm/unittests.cfg
@@ -35,3 +35,8 @@ file = selftest.flat
 smp = `getconf _NPROCESSORS_CONF`
 extra_params = -append 'smp'
 groups = selftest
+
+# Test PMU support without -icount
+[pmu]
+file = pmu.flat
+groups = pmu
diff --git a/config/config-arm64.mak b/config/config-arm64.mak
index d61b703..140b611 100644
--- a/config/config-arm64.mak
+++ b/config/config-arm64.mak
@@ -12,9 +12,11 @@ cflatobjs += lib/arm64/processor.o
 cflatobjs += lib/arm64/spinlock.o
 
 # arm64 specific tests
-tests =
+tests = $(TEST_DIR)/pmu.flat
 
 include config/config-arm-common.mak
 
 arch_clean: arm_clean
 	$(RM) lib/arm64/.*.d
+
+$(TEST_DIR)/pmu.elf: $(cstart.o) $(TEST_DIR)/pmu.o
-- 
Qualcomm Innovation Center, Inc.
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project

[Qemu-devel] [kvm-unit-tests PATCHv3 2/3] arm: pmu: Check cycle count increases

[Christopher Covington]

Ensure that reads of the PMCCNTR_EL0 are monotonically increasing,
even for the smallest delta of two subsequent reads.

Signed-off-by: Christopher Covington <cov@codeaurora.org>
---
 arm/pmu.c | 29 +++++++++++++++++++++++++++++
 1 file changed, 29 insertions(+)

diff --git a/arm/pmu.c b/arm/pmu.c
index 91a3688..589e605 100644
--- a/arm/pmu.c
+++ b/arm/pmu.c
@@ -33,6 +33,8 @@ struct pmu_data {
 	};
 };
 
+static const int samples = 10;
+
 /* As a simple sanity check on the PMCR_EL0, ensure the implementer field isn't
  * null. Also print out a couple other interesting fields for diagnostic
  * purposes. For example, as of fall 2015, QEMU TCG mode doesn't implement
@@ -56,11 +58,38 @@ static bool check_pmcr(void)
 	return false;
 }
 
+/* Ensure that the cycle counter progresses between back-to-back reads.
+ */
+static bool check_cycles_increase(void)
+{
+	struct pmu_data pmcr;
+
+	pmcr.enable = 1;
+	asm volatile("msr pmcr_el0, %0" : : "r" (pmcr));
+
+	for (int i = 0; i < samples; i++) {
+		int a, b;
+
+		asm volatile(
+			"mrs %[a], pmccntr_el0\n"
+			"mrs %[b], pmccntr_el0\n"
+		: [a] "=r" (a), [b] "=r" (b));
+
+		if (a >= b) {
+			printf("Read %d then %d.\n", a, b);
+			return false;
+		}
+	}
+
+	return true;
+}
+
 int main(void)
 {
 	report_prefix_push("pmu");
 
 	report("Control register", check_pmcr());
+	report("Monotonically increasing cycle count", check_cycles_increase());
 
 	return report_summary();
 }
-- 
Qualcomm Innovation Center, Inc.
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project

[Qemu-devel] [kvm-unit-tests PATCHv3 3/3] arm: pmu: Add CPI checking

[Christopher Covington]

Check the numbers of cycles per instruction (CPI) implied by ARM PMU
cycle counter values. Check that in -icount mode these strictly
match the specified rate.

Signed-off-by: Christopher Covington <cov@codeaurora.org>
---
 arm/pmu.c         | 72 ++++++++++++++++++++++++++++++++++++++++++++++++++++++-
 arm/unittests.cfg | 13 ++++++++++
 2 files changed, 84 insertions(+), 1 deletion(-)

diff --git a/arm/pmu.c b/arm/pmu.c
index 589e605..0ad113d 100644
--- a/arm/pmu.c
+++ b/arm/pmu.c
@@ -84,12 +84,82 @@ static bool check_cycles_increase(void)
 	return true;
 }
 
-int main(void)
+/* Execute a known number of guest instructions. Only odd instruction counts
+ * greater than or equal to 3 are supported by the in-line assembly code. The
+ * control register (PMCR_EL0) is initialized with the provided value (allowing
+ * for example for the cycle counter or event counters to be reset). At the end
+ * of the exact instruction loop, zero is written to PMCR_EL0 to disable
+ * counting, allowing the cycle counter or event counters to be read at the
+ * leisure of the calling code.
+ */
+static void measure_instrs(int num, struct pmu_data pmcr)
+{
+	int i = (num - 1) / 2;
+
+	if (num < 3 || ((num - 1) % 2))
+		abort();
+
+	asm volatile(
+		"msr pmcr_el0, %[pmcr]\n"
+		"1: subs %[i], %[i], #1\n"
+		"b.gt 1b\n"
+		"msr pmcr_el0, xzr"
+	: [i] "+r" (i) : [pmcr] "r" (pmcr) : "cc");
+}
+
+/* Measure cycle counts for various known instruction counts. Ensure that the
+ * cycle counter progresses (similar to check_cycles_increase() but with more
+ * instructions and using reset and stop controls). If supplied a positive,
+ * nonzero CPI parameter, also strictly check that every measurement matches
+ * it. Strict CPI checking is used to test -icount mode.
+ */
+static bool check_cpi(int cpi)
+{
+	struct pmu_data pmcr;
+
+	pmcr.cycle_counter_reset = 1;
+	pmcr.enable = 1;
+
+	if (cpi > 0)
+		printf("Checking for CPI=%d.\n", cpi);
+	printf("instrs : cycles0 cycles1 ...\n");
+
+	for (int i = 3; i < 300; i += 32) {
+		int avg, sum = 0;
+
+		printf("%d :", i);
+		for (int j = 0; j < samples; j++) {
+			int cycles;
+
+			measure_instrs(i, pmcr);
+			asm volatile("mrs %0, pmccntr_el0" : "=r" (cycles));
+			printf(" %d", cycles);
+
+			if (!cycles || (cpi > 0 && cycles != i * cpi)) {
+				printf("\n");
+				return false;
+			}
+
+			sum += cycles;
+		}
+		avg = sum / samples;
+		printf(" sum=%d avg=%d avg_ipc=%d avg_cpi=%d\n",
+			sum, avg, i / avg, avg / i);
+	}
+
+	return true;
+}
+
+int main(int argc, char *argv[])
 {
 	report_prefix_push("pmu");
 
 	report("Control register", check_pmcr());
 	report("Monotonically increasing cycle count", check_cycles_increase());
 
+	int cpi = (argc == 1 ? atol(argv[0]) : 0);
+
+	report("Cycle/instruction ratio", check_cpi(cpi));
+
 	return report_summary();
 }
diff --git a/arm/unittests.cfg b/arm/unittests.cfg
index fd94adb..333ee0d 100644
--- a/arm/unittests.cfg
+++ b/arm/unittests.cfg
@@ -39,4 +39,17 @@ groups = selftest
 # Test PMU support without -icount
 [pmu]
 file = pmu.flat
+extra_params = -append '-1'
+groups = pmu
+
+# Test PMU support with -icount IPC=1
+[pmu-icount-1]
+file = pmu.flat
+extra_params = -icount 0 -append '1'
+groups = pmu
+
+# Test PMU support with -icount IPC=256
+[pmu-icount-256]
+file = pmu.flat
+extra_params = -icount 8 -append '256'
 groups = pmu
-- 
Qualcomm Innovation Center, Inc.
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project

Re: [Qemu-devel] [kvm-unit-tests PATCHv3 1/3] arm: Add PMU test

[Andrew Jones]

On Tue, Oct 06, 2015 at 01:49:24PM -0400, Christopher Covington wrote:
> Beginning with a simple sanity check of the control register, add
> a unit test for the ARM Performance Monitors Unit (PMU).
> 
> Signed-off-by: Christopher Covington <cov@codeaurora.org>
> ---
>  arm/pmu.c               | 66 +++++++++++++++++++++++++++++++++++++++++++++++++
>  arm/unittests.cfg       |  5 ++++
>  config/config-arm64.mak |  4 ++-
>  3 files changed, 74 insertions(+), 1 deletion(-)
>  create mode 100644 arm/pmu.c
> 
> diff --git a/arm/pmu.c b/arm/pmu.c
> new file mode 100644
> index 0000000..91a3688
> --- /dev/null
> +++ b/arm/pmu.c
> @@ -0,0 +1,66 @@
> +/*
> + * Test the ARM Performance Monitors Unit (PMU).
> + *
> + * Copyright 2015 The Linux Foundation. All rights reserved.
> + *
> + * This program is free software; you can redistribute it and/or modify it
> + * under the terms of the GNU Lesser General Public License version 2.1 and
> + * only version 2.1 as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful, but WITHOUT
> + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
> + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License
> + * for more details.
> + */
> +#include "libcflat.h"
> +
> +struct pmu_data {
> +	union {
> +		uint32_t pmcr_el0;
> +		struct {
> +			uint32_t enable:1;
> +			uint32_t event_counter_reset:1;
> +			uint32_t cycle_counter_reset:1;
> +			uint32_t cycle_counter_clock_divider:1;
> +			uint32_t event_counter_export:1;
> +			uint32_t cycle_counter_disable_when_prohibited:1;
> +			uint32_t cycle_counter_long:1;
> +			uint32_t zeros:4;

resv might be a better name than zeros. Actually, does the spec even
state these bits will always be read as zeros?

> +			uint32_t num_counters:5;
> +			uint32_t identification_code:8;
> +			uint32_t implementer:8;
> +		};
> +	};
> +};
> +
> +/* As a simple sanity check on the PMCR_EL0, ensure the implementer field isn't
> + * null. Also print out a couple other interesting fields for diagnostic
> + * purposes. For example, as of fall 2015, QEMU TCG mode doesn't implement
> + * event counters and therefore reports zero of them, but hopefully support for
                                                ^for
> + * at least the instructions event will be added in the future and the reported
> + * number of event counters will become nonzero.
> + */
> +static bool check_pmcr(void)
> +{
> +	struct pmu_data pmcr;

Thanks for making the union and bitfield anonymous. I suggested
naming the struct pmu_data because I'm expecting more stuff to
eventually get shoved in there. If not, then maybe it needs to
be renamed something more pmcr-ish. Or, if it might expand its
purpose as I expect, then this variable should be renamed, i.e.

  struct pmu_data pmu;

> +
> +	asm volatile("mrs %0, pmcr_el0" : "=r" (pmcr));

And this should be

  asm volatile("mrs %0, pmcr_el0" : "=r" (pmu.pmcr_el0));

> +
> +	printf("PMU implementer:     %c\n", pmcr.implementer);
> +	printf("Identification code: 0x%x\n", pmcr.identification_code);
> +	printf("Event counters:      %d\n", pmcr.num_counters);
> +
> +	if (pmcr.implementer)
> +		return true;
> +
> +	return false;

How about

  return pmcr.implementer != 0;

> +}
> +
> +int main(void)
> +{
> +	report_prefix_push("pmu");
> +
> +	report("Control register", check_pmcr());
> +
> +	return report_summary();
> +}
> diff --git a/arm/unittests.cfg b/arm/unittests.cfg
> index e068a0c..fd94adb 100644
> --- a/arm/unittests.cfg
> +++ b/arm/unittests.cfg
> @@ -35,3 +35,8 @@ file = selftest.flat
>  smp = `getconf _NPROCESSORS_CONF`
>  extra_params = -append 'smp'
>  groups = selftest
> +
> +# Test PMU support without -icount
> +[pmu]
> +file = pmu.flat
> +groups = pmu
> diff --git a/config/config-arm64.mak b/config/config-arm64.mak
> index d61b703..140b611 100644
> --- a/config/config-arm64.mak
> +++ b/config/config-arm64.mak
> @@ -12,9 +12,11 @@ cflatobjs += lib/arm64/processor.o
>  cflatobjs += lib/arm64/spinlock.o
>  
>  # arm64 specific tests
> -tests =
> +tests = $(TEST_DIR)/pmu.flat
>  
>  include config/config-arm-common.mak
>  
>  arch_clean: arm_clean
>  	$(RM) lib/arm64/.*.d
> +
> +$(TEST_DIR)/pmu.elf: $(cstart.o) $(TEST_DIR)/pmu.o

We can have a PMU on 32-bit ARM processors too. If this test file
needs to be specific to 64-bit, then we should probably name it
less generically, like pmu64?

Thanks,
drew

Re: [Qemu-devel] [kvm-unit-tests PATCHv3 2/3] arm: pmu: Check cycle count increases

[Andrew Jones]

On Tue, Oct 06, 2015 at 01:49:25PM -0400, Christopher Covington wrote:
> Ensure that reads of the PMCCNTR_EL0 are monotonically increasing,
> even for the smallest delta of two subsequent reads.
> 
> Signed-off-by: Christopher Covington <cov@codeaurora.org>
> ---
>  arm/pmu.c | 29 +++++++++++++++++++++++++++++
>  1 file changed, 29 insertions(+)
> 
> diff --git a/arm/pmu.c b/arm/pmu.c
> index 91a3688..589e605 100644
> --- a/arm/pmu.c
> +++ b/arm/pmu.c
> @@ -33,6 +33,8 @@ struct pmu_data {
>  	};
>  };
>  
> +static const int samples = 10;

#define NR_SAMPLES 10

> +
>  /* As a simple sanity check on the PMCR_EL0, ensure the implementer field isn't
>   * null. Also print out a couple other interesting fields for diagnostic
>   * purposes. For example, as of fall 2015, QEMU TCG mode doesn't implement
> @@ -56,11 +58,38 @@ static bool check_pmcr(void)
>  	return false;
>  }
>  
> +/* Ensure that the cycle counter progresses between back-to-back reads.
> + */

style nit: your block quotes don't have opening wing (the preferred
kernel style - and, fwiw, my preference too...)

> +static bool check_cycles_increase(void)
> +{
> +	struct pmu_data pmcr;
> +
> +	pmcr.enable = 1;
> +	asm volatile("msr pmcr_el0, %0" : : "r" (pmcr));
> +
> +	for (int i = 0; i < samples; i++) {
> +		int a, b;
> +
> +		asm volatile(
> +			"mrs %[a], pmccntr_el0\n"
> +			"mrs %[b], pmccntr_el0\n"
> +		: [a] "=r" (a), [b] "=r" (b));
> +
> +		if (a >= b) {
> +			printf("Read %d then %d.\n", a, b);
> +			return false;
> +		}
> +	}
> +
> +	return true;
> +}
> +
>  int main(void)
>  {
>  	report_prefix_push("pmu");
>  
>  	report("Control register", check_pmcr());
> +	report("Monotonically increasing cycle count", check_cycles_increase());
>  
>  	return report_summary();
>  }
> -- 
> Qualcomm Innovation Center, Inc.
> The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
> a Linux Foundation Collaborative Project
> 
> 

Re: [Qemu-devel] [kvm-unit-tests PATCHv3 3/3] arm: pmu: Add CPI checking

[Andrew Jones]

On Tue, Oct 06, 2015 at 01:49:26PM -0400, Christopher Covington wrote:
> Check the numbers of cycles per instruction (CPI) implied by ARM PMU
> cycle counter values. Check that in -icount mode these strictly
> match the specified rate.
> 
> Signed-off-by: Christopher Covington <cov@codeaurora.org>
> ---
>  arm/pmu.c         | 72 ++++++++++++++++++++++++++++++++++++++++++++++++++++++-
>  arm/unittests.cfg | 13 ++++++++++
>  2 files changed, 84 insertions(+), 1 deletion(-)
> 
> diff --git a/arm/pmu.c b/arm/pmu.c
> index 589e605..0ad113d 100644
> --- a/arm/pmu.c
> +++ b/arm/pmu.c
> @@ -84,12 +84,82 @@ static bool check_cycles_increase(void)
>  	return true;
>  }
>  
> -int main(void)
> +/* Execute a known number of guest instructions. Only odd instruction counts
> + * greater than or equal to 3 are supported by the in-line assembly code. The
> + * control register (PMCR_EL0) is initialized with the provided value (allowing
> + * for example for the cycle counter or event counters to be reset). At the end
> + * of the exact instruction loop, zero is written to PMCR_EL0 to disable
> + * counting, allowing the cycle counter or event counters to be read at the
> + * leisure of the calling code.
> + */
> +static void measure_instrs(int num, struct pmu_data pmcr)
> +{
> +	int i = (num - 1) / 2;
> +
> +	if (num < 3 || ((num - 1) % 2))
> +		abort();

assert(num >= 3 && ((num - 1) % 2) == 0);

> +
> +	asm volatile(
> +		"msr pmcr_el0, %[pmcr]\n"
                    ^\t
> +		"1: subs %[i], %[i], #1\n"
                   ^\t  ^\t
> +		"b.gt 1b\n"
                     ^\t
> +		"msr pmcr_el0, xzr"
                    ^\t
> +	: [i] "+r" (i) : [pmcr] "r" (pmcr) : "cc");
> +}
> +
> +/* Measure cycle counts for various known instruction counts. Ensure that the
> + * cycle counter progresses (similar to check_cycles_increase() but with more
> + * instructions and using reset and stop controls). If supplied a positive,
> + * nonzero CPI parameter, also strictly check that every measurement matches
> + * it. Strict CPI checking is used to test -icount mode.
> + */
> +static bool check_cpi(int cpi)
> +{
> +	struct pmu_data pmcr;
> +
> +	pmcr.cycle_counter_reset = 1;
> +	pmcr.enable = 1;
> +
> +	if (cpi > 0)
> +		printf("Checking for CPI=%d.\n", cpi);
> +	printf("instrs : cycles0 cycles1 ...\n");
> +
> +	for (int i = 3; i < 300; i += 32) {
> +		int avg, sum = 0;
> +
> +		printf("%d :", i);
> +		for (int j = 0; j < samples; j++) {
> +			int cycles;
> +
> +			measure_instrs(i, pmcr);
> +			asm volatile("mrs %0, pmccntr_el0" : "=r" (cycles));
> +			printf(" %d", cycles);
> +
> +			if (!cycles || (cpi > 0 && cycles != i * cpi)) {
> +				printf("\n");
> +				return false;
> +			}
> +
> +			sum += cycles;
> +		}
> +		avg = sum / samples;
> +		printf(" sum=%d avg=%d avg_ipc=%d avg_cpi=%d\n",
> +			sum, avg, i / avg, avg / i);
> +	}
> +
> +	return true;
> +}
> +
> +int main(int argc, char *argv[])
>  {
>  	report_prefix_push("pmu");
>  
>  	report("Control register", check_pmcr());
>  	report("Monotonically increasing cycle count", check_cycles_increase());
>  
> +	int cpi = (argc == 1 ? atol(argv[0]) : 0);

I prefer variable declarations at the top of the function, and

  int cpi = 0;

  if (argc > 1)
    cpi = atol(argv[0]);

looks a bit better to me.


> +
> +	report("Cycle/instruction ratio", check_cpi(cpi));
> +
>  	return report_summary();
>  }
> diff --git a/arm/unittests.cfg b/arm/unittests.cfg
> index fd94adb..333ee0d 100644
> --- a/arm/unittests.cfg
> +++ b/arm/unittests.cfg
> @@ -39,4 +39,17 @@ groups = selftest
>  # Test PMU support without -icount
>  [pmu]
>  file = pmu.flat
> +extra_params = -append '-1'

Why do we need this cpu == -1? Can't it just be zero?

> +groups = pmu
> +
> +# Test PMU support with -icount IPC=1
> +[pmu-icount-1]
> +file = pmu.flat
> +extra_params = -icount 0 -append '1'
> +groups = pmu
> +
> +# Test PMU support with -icount IPC=256
> +[pmu-icount-256]
> +file = pmu.flat
> +extra_params = -icount 8 -append '256'
>  groups = pmu

-icount is a tcg specific parameter. I have a patch[*] in my staging
branch which allows you to specify 'accel = tcg' in unittests.cfg for
this type of test. You'll need to use that for anything with -icount
on the extra_params list.

Thanks,
drew

[*] https://github.com/rhdrjones/kvm-unit-tests/commit/85e084cf263e76484f7d82cbc9add4e7602f80a4

[Qemu-devel] [kvm-unit-tests PATCHv4] ARM PMU tests

[Christopher Covington]

Changes from v3 in response to Drew's suggestions:

* Improved pmu_data / PMCR fields and usage
* Straightened out awkward conditionals
* Added 32-bit support
* Styling enhancements
* Deferred -icount testing to later patch

[Qemu-devel] [kvm-unit-tests PATCHv4 1/3] arm: Add PMU test

[Christopher Covington]

Beginning with a simple sanity check of the control register, add
a unit test for the ARM Performance Monitors Unit (PMU).

Signed-off-by: Christopher Covington <cov@codeaurora.org>
---
 arm/pmu.c                    | 82 ++++++++++++++++++++++++++++++++++++++++++++
 arm/unittests.cfg            |  5 +++
 config/config-arm-common.mak |  4 ++-
 3 files changed, 90 insertions(+), 1 deletion(-)
 create mode 100644 arm/pmu.c

diff --git a/arm/pmu.c b/arm/pmu.c
new file mode 100644
index 0000000..42d0ee1
--- /dev/null
+++ b/arm/pmu.c
@@ -0,0 +1,82 @@
+/*
+ * Test the ARM Performance Monitors Unit (PMU).
+ *
+ * Copyright 2015 The Linux Foundation. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU Lesser General Public License version 2.1 and
+ * only version 2.1 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License
+ * for more details.
+ */
+#include "libcflat.h"
+
+#if defined(__arm__)
+static inline uint32_t get_pmcr(void)
+{
+	uint32_t ret;
+
+	asm volatile("mrc p15, 0, %0, c9, c12, 0" : "=r" (ret));
+	return ret;
+}
+#elif defined(__aarch64__)
+static inline uint32_t get_pmcr(void)
+{
+	uint32_t ret;
+
+	asm volatile("mrs %0, pmcr_el0" : "=r" (ret));
+	return ret;
+}
+#endif
+
+struct pmu_data {
+	union {
+		uint32_t pmcr_el0;
+		struct {
+			uint32_t enable:1;
+			uint32_t event_counter_reset:1;
+			uint32_t cycle_counter_reset:1;
+			uint32_t cycle_counter_clock_divider:1;
+			uint32_t event_counter_export:1;
+			uint32_t cycle_counter_disable_when_prohibited:1;
+			uint32_t cycle_counter_long:1;
+			uint32_t reserved:4;
+			uint32_t counters:5;
+			uint32_t identification_code:8;
+			uint32_t implementer:8;
+		};
+	};
+};
+
+/*
+ * As a simple sanity check on the PMCR_EL0, ensure the implementer field isn't
+ * null. Also print out a couple other interesting fields for diagnostic
+ * purposes. For example, as of fall 2015, QEMU TCG mode doesn't implement
+ * event counters and therefore reports zero event counters, but hopefully
+ * support for at least the instructions event will be added in the future and
+ * the reported number of event counters will become nonzero.
+ */
+static bool check_pmcr(void)
+{
+	struct pmu_data pmu;
+
+	pmu.pmcr_el0 = get_pmcr();
+
+	printf("PMU implementer:     %c\n", pmu.implementer);
+	printf("Identification code: 0x%x\n", pmu.identification_code);
+	printf("Event counters:      %d\n", pmu.counters);
+
+	return pmu.implementer != 0;
+}
+
+int main(void)
+{
+	report_prefix_push("pmu");
+
+	report("Control register", check_pmcr());
+
+	return report_summary();
+}
diff --git a/arm/unittests.cfg b/arm/unittests.cfg
index e068a0c..fd94adb 100644
--- a/arm/unittests.cfg
+++ b/arm/unittests.cfg
@@ -35,3 +35,8 @@ file = selftest.flat
 smp = `getconf _NPROCESSORS_CONF`
 extra_params = -append 'smp'
 groups = selftest
+
+# Test PMU support without -icount
+[pmu]
+file = pmu.flat
+groups = pmu
diff --git a/config/config-arm-common.mak b/config/config-arm-common.mak
index 698555d..b34d04c 100644
--- a/config/config-arm-common.mak
+++ b/config/config-arm-common.mak
@@ -11,7 +11,8 @@ endif
 
 tests-common = \
 	$(TEST_DIR)/selftest.flat \
-	$(TEST_DIR)/spinlock-test.flat
+	$(TEST_DIR)/spinlock-test.flat \
+	$(TEST_DIR)/pmu.flat
 
 all: test_cases
 
@@ -70,3 +71,4 @@ test_cases: $(generated_files) $(tests-common) $(tests)
 
 $(TEST_DIR)/selftest.elf: $(cstart.o) $(TEST_DIR)/selftest.o
 $(TEST_DIR)/spinlock-test.elf: $(cstart.o) $(TEST_DIR)/spinlock-test.o
+$(TEST_DIR)/pmu.elf: $(cstart.o) $(TEST_DIR)/pmu.o
-- 
Qualcomm Innovation Center, Inc.
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project

[Qemu-devel] [kvm-unit-tests PATCHv4 2/3] arm: pmu: Check cycle count increases

[Christopher Covington]

Ensure that reads of the PMCCNTR_EL0 are monotonically increasing,
even for the smallest delta of two subsequent reads.

Signed-off-by: Christopher Covington <cov@codeaurora.org>
---
 arm/pmu.c | 54 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 54 insertions(+)

diff --git a/arm/pmu.c b/arm/pmu.c
index 42d0ee1..ae81970 100644
--- a/arm/pmu.c
+++ b/arm/pmu.c
@@ -14,6 +14,8 @@
  */
 #include "libcflat.h"
 
+#define NR_SAMPLES 10
+
 #if defined(__arm__)
 static inline uint32_t get_pmcr(void)
 {
@@ -22,6 +24,19 @@ static inline uint32_t get_pmcr(void)
 	asm volatile("mrc p15, 0, %0, c9, c12, 0" : "=r" (ret));
 	return ret;
 }
+
+static inline void set_pmcr(uint32_t pmcr)
+{
+	asm volatile("mcr p15, 0, %0, c9, c12, 0" : : "r" (pmcr));
+}
+
+static inline unsigned long get_pmccntr(void)
+{
+	unsigned long cycles;
+
+	asm volatile("mrc p15, 0, %0, c9, c13, 0" : "=r" (cycles));
+	return cycles;
+}
 #elif defined(__aarch64__)
 static inline uint32_t get_pmcr(void)
 {
@@ -30,6 +45,19 @@ static inline uint32_t get_pmcr(void)
 	asm volatile("mrs %0, pmcr_el0" : "=r" (ret));
 	return ret;
 }
+
+static inline void set_pmcr(uint32_t pmcr)
+{
+	asm volatile("msr pmcr_el0, %0" : : "r" (pmcr));
+}
+
+static inline unsigned long get_pmccntr(void)
+{
+	unsigned long cycles;
+
+	asm volatile("mrs %0, pmccntr_el0" : "=r" (cycles));
+	return cycles;
+}
 #endif
 
 struct pmu_data {
@@ -72,11 +100,37 @@ static bool check_pmcr(void)
 	return pmu.implementer != 0;
 }
 
+/*
+ * Ensure that the cycle counter progresses between back-to-back reads.
+ */
+static bool check_cycles_increase(void)
+{
+	struct pmu_data pmu;
+
+	pmu.enable = 1;
+	set_pmcr(pmu.pmcr_el0);
+
+	for (int i = 0; i < NR_SAMPLES; i++) {
+		unsigned long a, b;
+
+		a = get_pmccntr();
+		b = get_pmccntr();
+
+		if (a >= b) {
+			printf("Read %ld then %ld.\n", a, b);
+			return false;
+		}
+	}
+
+	return true;
+}
+
 int main(void)
 {
 	report_prefix_push("pmu");
 
 	report("Control register", check_pmcr());
+	report("Monotonically increasing cycle count", check_cycles_increase());
 
 	return report_summary();
 }
-- 
Qualcomm Innovation Center, Inc.
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project

[Qemu-devel] [kvm-unit-tests PATCHv4 3/3] arm: pmu: Add CPI checking

[Christopher Covington]

Calculate the numbers of cycles per instruction (CPI) implied by ARM
PMU cycle counter values. The code includes a strict checking facility
intended for the -icount option in TCG mode but it is not yet enabled
in the configuration file. Enabling it must wait on infrastructure
improvements which allow for different tests to be run on TCG versus
KVM.

Signed-off-by: Christopher Covington <cov@codeaurora.org>
---
 arm/pmu.c | 91 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 90 insertions(+), 1 deletion(-)

diff --git a/arm/pmu.c b/arm/pmu.c
index ae81970..169c36c 100644
--- a/arm/pmu.c
+++ b/arm/pmu.c
@@ -37,6 +37,18 @@ static inline unsigned long get_pmccntr(void)
 	asm volatile("mrc p15, 0, %0, c9, c13, 0" : "=r" (cycles));
 	return cycles;
 }
+
+static inline void loop(int i, uint32_t pmcr)
+{
+	uint32_t z = 0;
+
+	asm volatile(
+		"	mcr p15, 0, %[pmcr], c9, c12, 0\n"
+		"	1: subs %[i], %[i], #1\n"
+		"	bgt 1b\n"
+		"	mcr p15, 0, %[z], c9, c12, 0\n"
+	: [i] "+r" (i) : [pmcr] "r" (pmcr), [z] "r" (z) : "cc");
+}
 #elif defined(__aarch64__)
 static inline uint32_t get_pmcr(void)
 {
@@ -58,6 +70,16 @@ static inline unsigned long get_pmccntr(void)
 	asm volatile("mrs %0, pmccntr_el0" : "=r" (cycles));
 	return cycles;
 }
+
+static inline void loop(int i, uint32_t pmcr)
+{
+	asm volatile(
+		"	msr pmcr_el0, %[pmcr]\n"
+		"	1: subs %[i], %[i], #1\n"
+		"	b.gt 1b\n"
+		"	msr pmcr_el0, xzr\n"
+	: [i] "+r" (i) : [pmcr] "r" (pmcr) : "cc");
+}
 #endif
 
 struct pmu_data {
@@ -125,12 +147,79 @@ static bool check_cycles_increase(void)
 	return true;
 }
 
-int main(void)
+/*
+ * Execute a known number of guest instructions. Only odd instruction counts
+ * greater than or equal to 3 are supported by the in-line assembly code. The
+ * control register (PMCR_EL0) is initialized with the provided value (allowing
+ * for example for the cycle counter or event counters to be reset). At the end
+ * of the exact instruction loop, zero is written to PMCR_EL0 to disable
+ * counting, allowing the cycle counter or event counters to be read at the
+ * leisure of the calling code.
+ */
+static void measure_instrs(int num, uint32_t pmcr)
+{
+	int i = (num - 1) / 2;
+
+	assert(num >= 3 && ((num - 1) % 2 == 0));
+	loop(i, pmcr);
+}
+
+/*
+ * Measure cycle counts for various known instruction counts. Ensure that the
+ * cycle counter progresses (similar to check_cycles_increase() but with more
+ * instructions and using reset and stop controls). If supplied a positive,
+ * nonzero CPI parameter, also strictly check that every measurement matches
+ * it. Strict CPI checking is used to test -icount mode.
+ */
+static bool check_cpi(int cpi)
+{
+	struct pmu_data pmu;
+
+	pmu.cycle_counter_reset = 1;
+	pmu.enable = 1;
+
+	if (cpi > 0)
+		printf("Checking for CPI=%d.\n", cpi);
+	printf("instrs : cycles0 cycles1 ...\n");
+
+	for (int i = 3; i < 300; i += 32) {
+		int avg, sum = 0;
+
+		printf("%d :", i);
+		for (int j = 0; j < NR_SAMPLES; j++) {
+			int cycles;
+
+			measure_instrs(i, pmu.pmcr_el0);
+			cycles = get_pmccntr();
+			printf(" %d", cycles);
+
+			if (!cycles || (cpi > 0 && cycles != i * cpi)) {
+				printf("\n");
+				return false;
+			}
+
+			sum += cycles;
+		}
+		avg = sum / NR_SAMPLES;
+		printf(" sum=%d avg=%d avg_ipc=%d avg_cpi=%d\n",
+			sum, avg, i / avg, avg / i);
+	}
+
+	return true;
+}
+
+int main(int argc, char *argv[])
 {
+	int cpi = 0;
+
+	if (argc > 1)
+		cpi = atol(argv[0]);
+
 	report_prefix_push("pmu");
 
 	report("Control register", check_pmcr());
 	report("Monotonically increasing cycle count", check_cycles_increase());
+	report("Cycle/instruction ratio", check_cpi(cpi));
 
 	return report_summary();
 }
-- 
Qualcomm Innovation Center, Inc.
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project

Re: [Qemu-devel] [kvm-unit-tests PATCHv4 1/3] arm: Add PMU test

[Andrew Jones]

On Mon, Oct 12, 2015 at 11:07:48AM -0400, Christopher Covington wrote:
> Beginning with a simple sanity check of the control register, add
> a unit test for the ARM Performance Monitors Unit (PMU).
> 
> Signed-off-by: Christopher Covington <cov@codeaurora.org>
> ---
>  arm/pmu.c                    | 82 ++++++++++++++++++++++++++++++++++++++++++++
>  arm/unittests.cfg            |  5 +++
>  config/config-arm-common.mak |  4 ++-
>  3 files changed, 90 insertions(+), 1 deletion(-)
>  create mode 100644 arm/pmu.c
> 
> diff --git a/arm/pmu.c b/arm/pmu.c
> new file mode 100644
> index 0000000..42d0ee1
> --- /dev/null
> +++ b/arm/pmu.c
> @@ -0,0 +1,82 @@
> +/*
> + * Test the ARM Performance Monitors Unit (PMU).
> + *
> + * Copyright 2015 The Linux Foundation. All rights reserved.
> + *
> + * This program is free software; you can redistribute it and/or modify it
> + * under the terms of the GNU Lesser General Public License version 2.1 and
> + * only version 2.1 as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful, but WITHOUT
> + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
> + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License
> + * for more details.
> + */
> +#include "libcflat.h"
> +
> +#if defined(__arm__)
> +static inline uint32_t get_pmcr(void)
> +{
> +	uint32_t ret;
> +
> +	asm volatile("mrc p15, 0, %0, c9, c12, 0" : "=r" (ret));
> +	return ret;
> +}
> +#elif defined(__aarch64__)
> +static inline uint32_t get_pmcr(void)
> +{
> +	uint32_t ret;
> +
> +	asm volatile("mrs %0, pmcr_el0" : "=r" (ret));
> +	return ret;
> +}
> +#endif
> +
> +struct pmu_data {
> +	union {
> +		uint32_t pmcr_el0;
> +		struct {
> +			uint32_t enable:1;
> +			uint32_t event_counter_reset:1;
> +			uint32_t cycle_counter_reset:1;
> +			uint32_t cycle_counter_clock_divider:1;
> +			uint32_t event_counter_export:1;
> +			uint32_t cycle_counter_disable_when_prohibited:1;
> +			uint32_t cycle_counter_long:1;
> +			uint32_t reserved:4;
> +			uint32_t counters:5;
> +			uint32_t identification_code:8;
> +			uint32_t implementer:8;
> +		};
> +	};
> +};
> +
> +/*
> + * As a simple sanity check on the PMCR_EL0, ensure the implementer field isn't
> + * null. Also print out a couple other interesting fields for diagnostic
> + * purposes. For example, as of fall 2015, QEMU TCG mode doesn't implement
> + * event counters and therefore reports zero event counters, but hopefully
> + * support for at least the instructions event will be added in the future and
> + * the reported number of event counters will become nonzero.
> + */
> +static bool check_pmcr(void)
> +{
> +	struct pmu_data pmu;
> +
> +	pmu.pmcr_el0 = get_pmcr();
> +
> +	printf("PMU implementer:     %c\n", pmu.implementer);
> +	printf("Identification code: 0x%x\n", pmu.identification_code);
> +	printf("Event counters:      %d\n", pmu.counters);
> +
> +	return pmu.implementer != 0;
> +}
> +
> +int main(void)
> +{
> +	report_prefix_push("pmu");
> +
> +	report("Control register", check_pmcr());
> +
> +	return report_summary();
> +}
> diff --git a/arm/unittests.cfg b/arm/unittests.cfg
> index e068a0c..fd94adb 100644
> --- a/arm/unittests.cfg
> +++ b/arm/unittests.cfg
> @@ -35,3 +35,8 @@ file = selftest.flat
>  smp = `getconf _NPROCESSORS_CONF`
>  extra_params = -append 'smp'
>  groups = selftest
> +
> +# Test PMU support without -icount
> +[pmu]
> +file = pmu.flat
> +groups = pmu
> diff --git a/config/config-arm-common.mak b/config/config-arm-common.mak
> index 698555d..b34d04c 100644
> --- a/config/config-arm-common.mak
> +++ b/config/config-arm-common.mak
> @@ -11,7 +11,8 @@ endif
>  
>  tests-common = \
>  	$(TEST_DIR)/selftest.flat \
> -	$(TEST_DIR)/spinlock-test.flat
> +	$(TEST_DIR)/spinlock-test.flat \
> +	$(TEST_DIR)/pmu.flat
>  
>  all: test_cases
>  
> @@ -70,3 +71,4 @@ test_cases: $(generated_files) $(tests-common) $(tests)
>  
>  $(TEST_DIR)/selftest.elf: $(cstart.o) $(TEST_DIR)/selftest.o
>  $(TEST_DIR)/spinlock-test.elf: $(cstart.o) $(TEST_DIR)/spinlock-test.o
> +$(TEST_DIR)/pmu.elf: $(cstart.o) $(TEST_DIR)/pmu.o
> -- 
> Qualcomm Innovation Center, Inc.
> The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
> a Linux Foundation Collaborative Project
> 
>

Reviewed-by: Andrew Jones <drjones@redhat.com> 

Re: [Qemu-devel] [kvm-unit-tests PATCHv4 2/3] arm: pmu: Check cycle count increases

[Andrew Jones]

On Mon, Oct 12, 2015 at 11:07:49AM -0400, Christopher Covington wrote:
> Ensure that reads of the PMCCNTR_EL0 are monotonically increasing,
> even for the smallest delta of two subsequent reads.
> 
> Signed-off-by: Christopher Covington <cov@codeaurora.org>
> ---
>  arm/pmu.c | 54 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 54 insertions(+)
> 
> diff --git a/arm/pmu.c b/arm/pmu.c
> index 42d0ee1..ae81970 100644
> --- a/arm/pmu.c
> +++ b/arm/pmu.c
> @@ -14,6 +14,8 @@
>   */
>  #include "libcflat.h"
>  
> +#define NR_SAMPLES 10
> +
>  #if defined(__arm__)
>  static inline uint32_t get_pmcr(void)
>  {
> @@ -22,6 +24,19 @@ static inline uint32_t get_pmcr(void)
>  	asm volatile("mrc p15, 0, %0, c9, c12, 0" : "=r" (ret));
>  	return ret;
>  }
> +
> +static inline void set_pmcr(uint32_t pmcr)
> +{
> +	asm volatile("mcr p15, 0, %0, c9, c12, 0" : : "r" (pmcr));
> +}
> +
> +static inline unsigned long get_pmccntr(void)
> +{
> +	unsigned long cycles;
> +
> +	asm volatile("mrc p15, 0, %0, c9, c13, 0" : "=r" (cycles));
> +	return cycles;

This is a 64-bit register, even for arm, but I guess there's no
need to access more than 32-bits using mrrc?

> +}
>  #elif defined(__aarch64__)
>  static inline uint32_t get_pmcr(void)
>  {
> @@ -30,6 +45,19 @@ static inline uint32_t get_pmcr(void)
>  	asm volatile("mrs %0, pmcr_el0" : "=r" (ret));
>  	return ret;
>  }
> +
> +static inline void set_pmcr(uint32_t pmcr)
> +{
> +	asm volatile("msr pmcr_el0, %0" : : "r" (pmcr));
> +}
> +
> +static inline unsigned long get_pmccntr(void)
> +{
> +	unsigned long cycles;
> +
> +	asm volatile("mrs %0, pmccntr_el0" : "=r" (cycles));
> +	return cycles;
> +}
>  #endif
>  
>  struct pmu_data {
> @@ -72,11 +100,37 @@ static bool check_pmcr(void)
>  	return pmu.implementer != 0;
>  }
>  
> +/*
> + * Ensure that the cycle counter progresses between back-to-back reads.
> + */
> +static bool check_cycles_increase(void)
> +{
> +	struct pmu_data pmu;
> +
> +	pmu.enable = 1;
> +	set_pmcr(pmu.pmcr_el0);

You need to zero pmu out first, it's just random stack junk except
for 'enable' definitely being 1 at this point. 

> +
> +	for (int i = 0; i < NR_SAMPLES; i++) {
> +		unsigned long a, b;
> +
> +		a = get_pmccntr();
> +		b = get_pmccntr();
> +
> +		if (a >= b) {
> +			printf("Read %ld then %ld.\n", a, b);
> +			return false;
> +		}
> +	}
> +
> +	return true;
> +}
> +
>  int main(void)
>  {
>  	report_prefix_push("pmu");
>  
>  	report("Control register", check_pmcr());
> +	report("Monotonically increasing cycle count", check_cycles_increase());
>  
>  	return report_summary();
>  }
> -- 
> Qualcomm Innovation Center, Inc.
> The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
> a Linux Foundation Collaborative Project
> 
> 

Re: [Qemu-devel] [kvm-unit-tests PATCHv4 3/3] arm: pmu: Add CPI checking

[Andrew Jones]

On Mon, Oct 12, 2015 at 11:07:50AM -0400, Christopher Covington wrote:
> Calculate the numbers of cycles per instruction (CPI) implied by ARM
> PMU cycle counter values. The code includes a strict checking facility
> intended for the -icount option in TCG mode but it is not yet enabled
> in the configuration file. Enabling it must wait on infrastructure
> improvements which allow for different tests to be run on TCG versus
> KVM.
> 
> Signed-off-by: Christopher Covington <cov@codeaurora.org>
> ---
>  arm/pmu.c | 91 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
>  1 file changed, 90 insertions(+), 1 deletion(-)
> 
> diff --git a/arm/pmu.c b/arm/pmu.c
> index ae81970..169c36c 100644
> --- a/arm/pmu.c
> +++ b/arm/pmu.c
> @@ -37,6 +37,18 @@ static inline unsigned long get_pmccntr(void)
>  	asm volatile("mrc p15, 0, %0, c9, c13, 0" : "=r" (cycles));
>  	return cycles;
>  }
> +
> +static inline void loop(int i, uint32_t pmcr)
> +{
> +	uint32_t z = 0;
> +
> +	asm volatile(
> +		"	mcr p15, 0, %[pmcr], c9, c12, 0\n"
> +		"	1: subs %[i], %[i], #1\n"
> +		"	bgt 1b\n"
> +		"	mcr p15, 0, %[z], c9, c12, 0\n"
> +	: [i] "+r" (i) : [pmcr] "r" (pmcr), [z] "r" (z) : "cc");

Assembly is always ugly, but we can do a bit better formatting with tabs

	asm volatile(
	"	mcr	p15, 0, %[pmcr], c9, c12, 0\n"
	"1:	subs	%[i], %[i], #1\n"
	"	bgt	1b\n"
	"	mcr	p15, 0, %[z], c9, c12, 0\n"
	: [i] "+r" (i)
	: [pmcr] "r" (pmcr), [z] "r" (z)
	: "cc");

Actually it can be even cleaner because you already created set_pmcr()

	set_pmcr(pmcr);

	asm volatile(
	"1:	subs	%0, %0, #1\n"
	"	bgt     1b\n"
	: "+r" (i) : : "cc");

	set_pmcr(0);


> +}
>  #elif defined(__aarch64__)
>  static inline uint32_t get_pmcr(void)
>  {
> @@ -58,6 +70,16 @@ static inline unsigned long get_pmccntr(void)
>  	asm volatile("mrs %0, pmccntr_el0" : "=r" (cycles));
>  	return cycles;
>  }
> +
> +static inline void loop(int i, uint32_t pmcr)
> +{
> +	asm volatile(
> +		"	msr pmcr_el0, %[pmcr]\n"
> +		"	1: subs %[i], %[i], #1\n"
> +		"	b.gt 1b\n"
> +		"	msr pmcr_el0, xzr\n"
> +	: [i] "+r" (i) : [pmcr] "r" (pmcr) : "cc");

same comment as above

> +}
>  #endif
>  
>  struct pmu_data {
> @@ -125,12 +147,79 @@ static bool check_cycles_increase(void)
>  	return true;
>  }
>  
> -int main(void)
> +/*
> + * Execute a known number of guest instructions. Only odd instruction counts
> + * greater than or equal to 3 are supported by the in-line assembly code. The

Not all odd counts, right? But rather all multiples of 3? IIUC this is because
the loop is two instructions (sub + branch), and then the clearing of the pmcr
register counts as the 3rd?

> + * control register (PMCR_EL0) is initialized with the provided value (allowing
> + * for example for the cycle counter or event counters to be reset). At the end
> + * of the exact instruction loop, zero is written to PMCR_EL0 to disable
> + * counting, allowing the cycle counter or event counters to be read at the
> + * leisure of the calling code.
> + */
> +static void measure_instrs(int num, uint32_t pmcr)
> +{
> +	int i = (num - 1) / 2;
> +
> +	assert(num >= 3 && ((num - 1) % 2 == 0));
> +	loop(i, pmcr);
> +}
> +
> +/*
> + * Measure cycle counts for various known instruction counts. Ensure that the
> + * cycle counter progresses (similar to check_cycles_increase() but with more
> + * instructions and using reset and stop controls). If supplied a positive,
> + * nonzero CPI parameter, also strictly check that every measurement matches
> + * it. Strict CPI checking is used to test -icount mode.
> + */
> +static bool check_cpi(int cpi)
> +{
> +	struct pmu_data pmu;

memset(&pmu, 0, sizeof(pmu));

> +
> +	pmu.cycle_counter_reset = 1;
> +	pmu.enable = 1;
> +
> +	if (cpi > 0)
> +		printf("Checking for CPI=%d.\n", cpi);
> +	printf("instrs : cycles0 cycles1 ...\n");
> +
> +	for (int i = 3; i < 300; i += 32) {
> +		int avg, sum = 0;
> +
> +		printf("%d :", i);
> +		for (int j = 0; j < NR_SAMPLES; j++) {
> +			int cycles;
> +
> +			measure_instrs(i, pmu.pmcr_el0);
> +			cycles = get_pmccntr();
> +			printf(" %d", cycles);
> +
> +			if (!cycles || (cpi > 0 && cycles != i * cpi)) {
> +				printf("\n");
> +				return false;
> +			}
> +
> +			sum += cycles;
> +		}
> +		avg = sum / NR_SAMPLES;
> +		printf(" sum=%d avg=%d avg_ipc=%d avg_cpi=%d\n",
> +			sum, avg, i / avg, avg / i);
> +	}
> +
> +	return true;
> +}
> +
> +int main(int argc, char *argv[])
>  {
> +	int cpi = 0;
> +
> +	if (argc > 1)
> +		cpi = atol(argv[0]);
> +
>  	report_prefix_push("pmu");
>  
>  	report("Control register", check_pmcr());
>  	report("Monotonically increasing cycle count", check_cycles_increase());
> +	report("Cycle/instruction ratio", check_cpi(cpi));
>  
>  	return report_summary();
>  }
> -- 
> Qualcomm Innovation Center, Inc.
> The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
> a Linux Foundation Collaborative Project
> 
> 

Re: [Qemu-devel] [kvm-unit-tests PATCHv4] ARM PMU tests

[Andrew Jones]

On Mon, Oct 12, 2015 at 11:07:47AM -0400, Christopher Covington wrote:
> Changes from v3 in response to Drew's suggestions:
> 
> * Improved pmu_data / PMCR fields and usage
> * Straightened out awkward conditionals
> * Added 32-bit support
> * Styling enhancements
> * Deferred -icount testing to later patch
> 
>

Sorry I was slow to review this version. Also, just FYI, I'll be on
vacation for a week, so I'll probably be slow to review the next
version too :-) Anyway, thanks for the patches, and thanks for your
patience.

drew 

Re: [Qemu-devel] [kvm-unit-tests PATCHv4 3/3] arm: pmu: Add CPI checking

[Christopher Covington]

Hi Drew,

I appreciate your feedback on these patches.

On 10/18/2015 02:28 PM, Andrew Jones wrote:

>> --- a/arm/pmu.c
>> +++ b/arm/pmu.c
>> @@ -37,6 +37,18 @@ static inline unsigned long get_pmccntr(void)
>>  	asm volatile("mrc p15, 0, %0, c9, c13, 0" : "=r" (cycles));
>>  	return cycles;
>>  }
>> +
>> +static inline void loop(int i, uint32_t pmcr)
>> +{
>> +	uint32_t z = 0;
>> +
>> +	asm volatile(
>> +		"	mcr p15, 0, %[pmcr], c9, c12, 0\n"
>> +		"	1: subs %[i], %[i], #1\n"
>> +		"	bgt 1b\n"
>> +		"	mcr p15, 0, %[z], c9, c12, 0\n"
>> +	: [i] "+r" (i) : [pmcr] "r" (pmcr), [z] "r" (z) : "cc");
> 
> Assembly is always ugly, but we can do a bit better formatting with tabs
> 
> 	asm volatile(
> 	"	mcr	p15, 0, %[pmcr], c9, c12, 0\n"
> 	"1:	subs	%[i], %[i], #1\n"
> 	"	bgt	1b\n"
> 	"	mcr	p15, 0, %[z], c9, c12, 0\n"
> 	: [i] "+r" (i)
> 	: [pmcr] "r" (pmcr), [z] "r" (z)
> 	: "cc");
> 
> Actually it can be even cleaner because you already created set_pmcr()
> 
> 	set_pmcr(pmcr);
> 
> 	asm volatile(
> 	"1:	subs	%0, %0, #1\n"
> 	"	bgt     1b\n"
> 	: "+r" (i) : : "cc");
> 
> 	set_pmcr(0);

Is there any way to ensure that the compiler won't for example put a `mov rd,
#0` between the `bgt 1b` and the `mcr <pmcr>, rn`?

>> @@ -125,12 +147,79 @@ static bool check_cycles_increase(void)
>>  	return true;
>>  }
>>  
>> -int main(void)
>> +/*
>> + * Execute a known number of guest instructions. Only odd instruction counts
>> + * greater than or equal to 3 are supported by the in-line assembly code. The
> 
> Not all odd counts, right? But rather all multiples of 3? IIUC this is because
> the loop is two instructions (sub + branch), and then the clearing of the pmcr
> register counts as the 3rd?

Clearing the PMCR doesn't happen as part of the loop, but as part of the loop
exit or epilogue.

total_instrs = iteration_count * loop_instrs + eipilogue_instrs
total_instrs = iteration_count * 2 + 1

Thanks,
Christopher Covington

-- 
Qualcomm Innovation Center, Inc.
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project

Re: [Qemu-devel] [kvm-unit-tests PATCHv4 3/3] arm: pmu: Add CPI checking

[Andrew Jones]

On Mon, Oct 19, 2015 at 11:44:30AM -0400, Christopher Covington wrote:
> Hi Drew,
> 
> I appreciate your feedback on these patches.
> 
> On 10/18/2015 02:28 PM, Andrew Jones wrote:
> 
> >> --- a/arm/pmu.c
> >> +++ b/arm/pmu.c
> >> @@ -37,6 +37,18 @@ static inline unsigned long get_pmccntr(void)
> >>  	asm volatile("mrc p15, 0, %0, c9, c13, 0" : "=r" (cycles));
> >>  	return cycles;
> >>  }
> >> +
> >> +static inline void loop(int i, uint32_t pmcr)
> >> +{
> >> +	uint32_t z = 0;
> >> +
> >> +	asm volatile(
> >> +		"	mcr p15, 0, %[pmcr], c9, c12, 0\n"
> >> +		"	1: subs %[i], %[i], #1\n"
> >> +		"	bgt 1b\n"
> >> +		"	mcr p15, 0, %[z], c9, c12, 0\n"
> >> +	: [i] "+r" (i) : [pmcr] "r" (pmcr), [z] "r" (z) : "cc");
> > 
> > Assembly is always ugly, but we can do a bit better formatting with tabs
> > 
> > 	asm volatile(
> > 	"	mcr	p15, 0, %[pmcr], c9, c12, 0\n"
> > 	"1:	subs	%[i], %[i], #1\n"
> > 	"	bgt	1b\n"
> > 	"	mcr	p15, 0, %[z], c9, c12, 0\n"
> > 	: [i] "+r" (i)
> > 	: [pmcr] "r" (pmcr), [z] "r" (z)
> > 	: "cc");
> > 
> > Actually it can be even cleaner because you already created set_pmcr()
> > 
> > 	set_pmcr(pmcr);
> > 
> > 	asm volatile(
> > 	"1:	subs	%0, %0, #1\n"
> > 	"	bgt     1b\n"
> > 	: "+r" (i) : : "cc");
> > 
> > 	set_pmcr(0);
> 
> Is there any way to ensure that the compiler won't for example put a `mov rd,
> #0` between the `bgt 1b` and the `mcr <pmcr>, rn`?

You're right. We need to keep the clearing in the asm here in order to
make sure don't add instructions in between.

> 
> >> @@ -125,12 +147,79 @@ static bool check_cycles_increase(void)
> >>  	return true;
> >>  }
> >>  
> >> -int main(void)
> >> +/*
> >> + * Execute a known number of guest instructions. Only odd instruction counts
> >> + * greater than or equal to 3 are supported by the in-line assembly code. The
> > 
> > Not all odd counts, right? But rather all multiples of 3? IIUC this is because
> > the loop is two instructions (sub + branch), and then the clearing of the pmcr
> > register counts as the 3rd?
> 
> Clearing the PMCR doesn't happen as part of the loop, but as part of the loop
> exit or epilogue.
> 
> total_instrs = iteration_count * loop_instrs + eipilogue_instrs
> total_instrs = iteration_count * 2 + 1

Ah yeah, that makes sense.

Thanks,
drew

> 
> Thanks,
> Christopher Covington
> 
> -- 
> Qualcomm Innovation Center, Inc.
> The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
> a Linux Foundation Collaborative Project
> 

[Qemu-devel] [kvm-unit-tests PATCHv5] ARM PMU tests

[Christopher Covington]

Changes from v4:
* Add Drew's Reviewed-by to first patch.
* Explain use of 32-bit cycle count values in AArch32.
* Zero-initialize pmu_data struct before use in check_cycles_increase and check_cpi. While the insistence on not using memset is entirely my own vanity, I blame the funny syntax on https://gcc.gnu.org/bugzilla/show_bug.cgi?id=53119 (adds extra set of braces) and checkpatch (adds spaces).
* Improve formatting of inline assembly and better explain why so much code must be inline assembly.

[Qemu-devel] [kvm-unit-tests PATCHv5 1/3] arm: Add PMU test

[Christopher Covington]

Beginning with a simple sanity check of the control register, add
a unit test for the ARM Performance Monitors Unit (PMU).

Signed-off-by: Christopher Covington <cov@codeaurora.org>
Reviewed-by: Andrew Jones <drjones@redhat.com>
---
 arm/pmu.c                    | 82 ++++++++++++++++++++++++++++++++++++++++++++
 arm/unittests.cfg            |  5 +++
 config/config-arm-common.mak |  4 ++-
 3 files changed, 90 insertions(+), 1 deletion(-)
 create mode 100644 arm/pmu.c

diff --git a/arm/pmu.c b/arm/pmu.c
new file mode 100644
index 0000000..42d0ee1
--- /dev/null
+++ b/arm/pmu.c
@@ -0,0 +1,82 @@
+/*
+ * Test the ARM Performance Monitors Unit (PMU).
+ *
+ * Copyright 2015 The Linux Foundation. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU Lesser General Public License version 2.1 and
+ * only version 2.1 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License
+ * for more details.
+ */
+#include "libcflat.h"
+
+#if defined(__arm__)
+static inline uint32_t get_pmcr(void)
+{
+	uint32_t ret;
+
+	asm volatile("mrc p15, 0, %0, c9, c12, 0" : "=r" (ret));
+	return ret;
+}
+#elif defined(__aarch64__)
+static inline uint32_t get_pmcr(void)
+{
+	uint32_t ret;
+
+	asm volatile("mrs %0, pmcr_el0" : "=r" (ret));
+	return ret;
+}
+#endif
+
+struct pmu_data {
+	union {
+		uint32_t pmcr_el0;
+		struct {
+			uint32_t enable:1;
+			uint32_t event_counter_reset:1;
+			uint32_t cycle_counter_reset:1;
+			uint32_t cycle_counter_clock_divider:1;
+			uint32_t event_counter_export:1;
+			uint32_t cycle_counter_disable_when_prohibited:1;
+			uint32_t cycle_counter_long:1;
+			uint32_t reserved:4;
+			uint32_t counters:5;
+			uint32_t identification_code:8;
+			uint32_t implementer:8;
+		};
+	};
+};
+
+/*
+ * As a simple sanity check on the PMCR_EL0, ensure the implementer field isn't
+ * null. Also print out a couple other interesting fields for diagnostic
+ * purposes. For example, as of fall 2015, QEMU TCG mode doesn't implement
+ * event counters and therefore reports zero event counters, but hopefully
+ * support for at least the instructions event will be added in the future and
+ * the reported number of event counters will become nonzero.
+ */
+static bool check_pmcr(void)
+{
+	struct pmu_data pmu;
+
+	pmu.pmcr_el0 = get_pmcr();
+
+	printf("PMU implementer:     %c\n", pmu.implementer);
+	printf("Identification code: 0x%x\n", pmu.identification_code);
+	printf("Event counters:      %d\n", pmu.counters);
+
+	return pmu.implementer != 0;
+}
+
+int main(void)
+{
+	report_prefix_push("pmu");
+
+	report("Control register", check_pmcr());
+
+	return report_summary();
+}
diff --git a/arm/unittests.cfg b/arm/unittests.cfg
index e068a0c..fd94adb 100644
--- a/arm/unittests.cfg
+++ b/arm/unittests.cfg
@@ -35,3 +35,8 @@ file = selftest.flat
 smp = `getconf _NPROCESSORS_CONF`
 extra_params = -append 'smp'
 groups = selftest
+
+# Test PMU support without -icount
+[pmu]
+file = pmu.flat
+groups = pmu
diff --git a/config/config-arm-common.mak b/config/config-arm-common.mak
index 698555d..b34d04c 100644
--- a/config/config-arm-common.mak
+++ b/config/config-arm-common.mak
@@ -11,7 +11,8 @@ endif
 
 tests-common = \
 	$(TEST_DIR)/selftest.flat \
-	$(TEST_DIR)/spinlock-test.flat
+	$(TEST_DIR)/spinlock-test.flat \
+	$(TEST_DIR)/pmu.flat
 
 all: test_cases
 
@@ -70,3 +71,4 @@ test_cases: $(generated_files) $(tests-common) $(tests)
 
 $(TEST_DIR)/selftest.elf: $(cstart.o) $(TEST_DIR)/selftest.o
 $(TEST_DIR)/spinlock-test.elf: $(cstart.o) $(TEST_DIR)/spinlock-test.o
+$(TEST_DIR)/pmu.elf: $(cstart.o) $(TEST_DIR)/pmu.o
-- 
Qualcomm Innovation Center, Inc.
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project

[Qemu-devel] [kvm-unit-tests PATCHv5 2/3] arm: pmu: Check cycle count increases

[Christopher Covington]

Ensure that reads of the PMCCNTR_EL0 are monotonically increasing,
even for the smallest delta of two subsequent reads.

Signed-off-by: Christopher Covington <cov@codeaurora.org>
---
 arm/pmu.c | 60 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 60 insertions(+)

diff --git a/arm/pmu.c b/arm/pmu.c
index 42d0ee1..c44d708 100644
--- a/arm/pmu.c
+++ b/arm/pmu.c
@@ -14,6 +14,8 @@
  */
 #include "libcflat.h"
 
+#define NR_SAMPLES 10
+
 #if defined(__arm__)
 static inline uint32_t get_pmcr(void)
 {
@@ -22,6 +24,25 @@ static inline uint32_t get_pmcr(void)
 	asm volatile("mrc p15, 0, %0, c9, c12, 0" : "=r" (ret));
 	return ret;
 }
+
+static inline void set_pmcr(uint32_t pmcr)
+{
+	asm volatile("mcr p15, 0, %0, c9, c12, 0" : : "r" (pmcr));
+}
+
+/*
+ * While PMCCNTR can be accessed as a 64 bit coprocessor register, returning 64
+ * bits doesn't seem worth the trouble when differential usage of the result is
+ * expected (with differences that can easily fit in 32 bits). So just return
+ * the lower 32 bits of the cycle count in AArch32.
+ */
+static inline unsigned long get_pmccntr(void)
+{
+	unsigned long cycles;
+
+	asm volatile("mrc p15, 0, %0, c9, c13, 0" : "=r" (cycles));
+	return cycles;
+}
 #elif defined(__aarch64__)
 static inline uint32_t get_pmcr(void)
 {
@@ -30,6 +51,19 @@ static inline uint32_t get_pmcr(void)
 	asm volatile("mrs %0, pmcr_el0" : "=r" (ret));
 	return ret;
 }
+
+static inline void set_pmcr(uint32_t pmcr)
+{
+	asm volatile("msr pmcr_el0, %0" : : "r" (pmcr));
+}
+
+static inline unsigned long get_pmccntr(void)
+{
+	unsigned long cycles;
+
+	asm volatile("mrs %0, pmccntr_el0" : "=r" (cycles));
+	return cycles;
+}
 #endif
 
 struct pmu_data {
@@ -72,11 +106,37 @@ static bool check_pmcr(void)
 	return pmu.implementer != 0;
 }
 
+/*
+ * Ensure that the cycle counter progresses between back-to-back reads.
+ */
+static bool check_cycles_increase(void)
+{
+	struct pmu_data pmu = { {0} };
+
+	pmu.enable = 1;
+	set_pmcr(pmu.pmcr_el0);
+
+	for (int i = 0; i < NR_SAMPLES; i++) {
+		unsigned long a, b;
+
+		a = get_pmccntr();
+		b = get_pmccntr();
+
+		if (a >= b) {
+			printf("Read %ld then %ld.\n", a, b);
+			return false;
+		}
+	}
+
+	return true;
+}
+
 int main(void)
 {
 	report_prefix_push("pmu");
 
 	report("Control register", check_pmcr());
+	report("Monotonically increasing cycle count", check_cycles_increase());
 
 	return report_summary();
 }
-- 
Qualcomm Innovation Center, Inc.
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project

[Qemu-devel] [kvm-unit-tests PATCHv5 3/3] arm: pmu: Add CPI checking

[Christopher Covington]

Calculate the numbers of cycles per instruction (CPI) implied by ARM
PMU cycle counter values. The code includes a strict checking facility
intended for the -icount option in TCG mode but it is not yet enabled
in the configuration file. Enabling it must wait on infrastructure
improvements which allow for different tests to be run on TCG versus
KVM.

Signed-off-by: Christopher Covington <cov@codeaurora.org>
---
 arm/pmu.c | 105 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 104 insertions(+), 1 deletion(-)

diff --git a/arm/pmu.c b/arm/pmu.c
index c44d708..59f26ab 100644
--- a/arm/pmu.c
+++ b/arm/pmu.c
@@ -43,6 +43,25 @@ static inline unsigned long get_pmccntr(void)
 	asm volatile("mrc p15, 0, %0, c9, c13, 0" : "=r" (cycles));
 	return cycles;
 }
+
+/*
+ * Extra instructions such as `mov rd, #0` inserted by the compiler would be
+ * difficult to compensate for, so hand assemble everything between, and
+ * including, the PMCR accesses to start and stop counting.
+ */
+static inline void loop(int i, uint32_t pmcr)
+{
+	uint32_t z = 0;
+
+	asm volatile(
+	"	mcr p15, 0, %[pmcr], c9, c12, 0\n"
+	"1:	subs %[i], %[i], #1\n"
+	"	bgt 1b\n"
+	"	mcr p15, 0, %[z], c9, c12, 0\n"
+	: [i] "+r" (i)
+	: [pmcr] "r" (pmcr), [z] "r" (z)
+	: "cc");
+}
 #elif defined(__aarch64__)
 static inline uint32_t get_pmcr(void)
 {
@@ -64,6 +83,23 @@ static inline unsigned long get_pmccntr(void)
 	asm volatile("mrs %0, pmccntr_el0" : "=r" (cycles));
 	return cycles;
 }
+
+/*
+ * Extra instructions inserted by the compiler would be difficult to compensate
+ * for, so hand assemble everything between, and including, the PMCR accesses
+ * to start and stop counting.
+ */
+static inline void loop(int i, uint32_t pmcr)
+{
+	asm volatile(
+	"	msr pmcr_el0, %[pmcr]\n"
+	"1:	subs %[i], %[i], #1\n"
+	"	b.gt 1b\n"
+	"	msr pmcr_el0, xzr\n"
+	: [i] "+r" (i)
+	: [pmcr] "r" (pmcr)
+	: "cc");
+}
 #endif
 
 struct pmu_data {
@@ -131,12 +167,79 @@ static bool check_cycles_increase(void)
 	return true;
 }
 
-int main(void)
+/*
+ * Execute a known number of guest instructions. Only odd instruction counts
+ * greater than or equal to 3 are supported by the in-line assembly code. The
+ * control register (PMCR_EL0) is initialized with the provided value (allowing
+ * for example for the cycle counter or event counters to be reset). At the end
+ * of the exact instruction loop, zero is written to PMCR_EL0 to disable
+ * counting, allowing the cycle counter or event counters to be read at the
+ * leisure of the calling code.
+ */
+static void measure_instrs(int num, uint32_t pmcr)
+{
+	int i = (num - 1) / 2;
+
+	assert(num >= 3 && ((num - 1) % 2 == 0));
+	loop(i, pmcr);
+}
+
+/*
+ * Measure cycle counts for various known instruction counts. Ensure that the
+ * cycle counter progresses (similar to check_cycles_increase() but with more
+ * instructions and using reset and stop controls). If supplied a positive,
+ * nonzero CPI parameter, also strictly check that every measurement matches
+ * it. Strict CPI checking is used to test -icount mode.
+ */
+static bool check_cpi(int cpi)
+{
+	struct pmu_data pmu = { {0} };
+
+	pmu.cycle_counter_reset = 1;
+	pmu.enable = 1;
+
+	if (cpi > 0)
+		printf("Checking for CPI=%d.\n", cpi);
+	printf("instrs : cycles0 cycles1 ...\n");
+
+	for (int i = 3; i < 300; i += 32) {
+		int avg, sum = 0;
+
+		printf("%d :", i);
+		for (int j = 0; j < NR_SAMPLES; j++) {
+			int cycles;
+
+			measure_instrs(i, pmu.pmcr_el0);
+			cycles = get_pmccntr();
+			printf(" %d", cycles);
+
+			if (!cycles || (cpi > 0 && cycles != i * cpi)) {
+				printf("\n");
+				return false;
+			}
+
+			sum += cycles;
+		}
+		avg = sum / NR_SAMPLES;
+		printf(" sum=%d avg=%d avg_ipc=%d avg_cpi=%d\n",
+			sum, avg, i / avg, avg / i);
+	}
+
+	return true;
+}
+
+int main(int argc, char *argv[])
 {
+	int cpi = 0;
+
+	if (argc > 1)
+		cpi = atol(argv[0]);
+
 	report_prefix_push("pmu");
 
 	report("Control register", check_pmcr());
 	report("Monotonically increasing cycle count", check_cycles_increase());
+	report("Cycle/instruction ratio", check_cpi(cpi));
 
 	return report_summary();
 }
-- 
Qualcomm Innovation Center, Inc.
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project

Re: [Qemu-devel] [kvm-unit-tests PATCHv5 2/3] arm: pmu: Check cycle count increases

[Andrew Jones]

On Mon, Oct 26, 2015 at 11:38:49AM -0400, Christopher Covington wrote:
> Ensure that reads of the PMCCNTR_EL0 are monotonically increasing,
> even for the smallest delta of two subsequent reads.
> 
> Signed-off-by: Christopher Covington <cov@codeaurora.org>
> ---
>  arm/pmu.c | 60 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 60 insertions(+)
> 
> diff --git a/arm/pmu.c b/arm/pmu.c
> index 42d0ee1..c44d708 100644
> --- a/arm/pmu.c
> +++ b/arm/pmu.c
> @@ -14,6 +14,8 @@
>   */
>  #include "libcflat.h"
>  
> +#define NR_SAMPLES 10
> +
>  #if defined(__arm__)
>  static inline uint32_t get_pmcr(void)
>  {
> @@ -22,6 +24,25 @@ static inline uint32_t get_pmcr(void)
>  	asm volatile("mrc p15, 0, %0, c9, c12, 0" : "=r" (ret));
>  	return ret;
>  }
> +
> +static inline void set_pmcr(uint32_t pmcr)
> +{
> +	asm volatile("mcr p15, 0, %0, c9, c12, 0" : : "r" (pmcr));
> +}
> +
> +/*
> + * While PMCCNTR can be accessed as a 64 bit coprocessor register, returning 64
> + * bits doesn't seem worth the trouble when differential usage of the result is
> + * expected (with differences that can easily fit in 32 bits). So just return
> + * the lower 32 bits of the cycle count in AArch32.
> + */
> +static inline unsigned long get_pmccntr(void)
> +{
> +	unsigned long cycles;
> +
> +	asm volatile("mrc p15, 0, %0, c9, c13, 0" : "=r" (cycles));
> +	return cycles;
> +}
>  #elif defined(__aarch64__)
>  static inline uint32_t get_pmcr(void)
>  {
> @@ -30,6 +51,19 @@ static inline uint32_t get_pmcr(void)
>  	asm volatile("mrs %0, pmcr_el0" : "=r" (ret));
>  	return ret;
>  }
> +
> +static inline void set_pmcr(uint32_t pmcr)
> +{
> +	asm volatile("msr pmcr_el0, %0" : : "r" (pmcr));
> +}
> +
> +static inline unsigned long get_pmccntr(void)
> +{
> +	unsigned long cycles;
> +
> +	asm volatile("mrs %0, pmccntr_el0" : "=r" (cycles));
> +	return cycles;
> +}
>  #endif
>  
>  struct pmu_data {
> @@ -72,11 +106,37 @@ static bool check_pmcr(void)
>  	return pmu.implementer != 0;
>  }
>  
> +/*
> + * Ensure that the cycle counter progresses between back-to-back reads.
> + */
> +static bool check_cycles_increase(void)
> +{
> +	struct pmu_data pmu = { {0} };

One set of {} is enough, and looks better.

> +
> +	pmu.enable = 1;
> +	set_pmcr(pmu.pmcr_el0);
> +
> +	for (int i = 0; i < NR_SAMPLES; i++) {
> +		unsigned long a, b;
> +
> +		a = get_pmccntr();
> +		b = get_pmccntr();
> +
> +		if (a >= b) {
> +			printf("Read %ld then %ld.\n", a, b);
> +			return false;
> +		}
> +	}
> +
> +	return true;
> +}
> +
>  int main(void)
>  {
>  	report_prefix_push("pmu");
>  
>  	report("Control register", check_pmcr());
> +	report("Monotonically increasing cycle count", check_cycles_increase());
>  
>  	return report_summary();
>  }
> -- 
> Qualcomm Innovation Center, Inc.
> The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
> a Linux Foundation Collaborative Project
> 
>

Otherwise
Reviewed-by: Andrew Jones <drjones@redhat.com>

Re: [Qemu-devel] [kvm-unit-tests PATCHv5 2/3] arm: pmu: Check cycle count increases

[Christopher Covington]

On 10/26/2015 11:58 AM, Andrew Jones wrote:
> On Mon, Oct 26, 2015 at 11:38:49AM -0400, Christopher Covington wrote:
>> Ensure that reads of the PMCCNTR_EL0 are monotonically increasing,
>> even for the smallest delta of two subsequent reads.
>>
>> Signed-off-by: Christopher Covington <cov@codeaurora.org>
>> ---
>>  arm/pmu.c | 60 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>  1 file changed, 60 insertions(+)
>>
>> diff --git a/arm/pmu.c b/arm/pmu.c
>> index 42d0ee1..c44d708 100644
>> --- a/arm/pmu.c
>> +++ b/arm/pmu.c
>> @@ -14,6 +14,8 @@
>>   */
>>  #include "libcflat.h"
>>  
>> +#define NR_SAMPLES 10
>> +
>>  #if defined(__arm__)
>>  static inline uint32_t get_pmcr(void)
>>  {
>> @@ -22,6 +24,25 @@ static inline uint32_t get_pmcr(void)
>>  	asm volatile("mrc p15, 0, %0, c9, c12, 0" : "=r" (ret));
>>  	return ret;
>>  }
>> +
>> +static inline void set_pmcr(uint32_t pmcr)
>> +{
>> +	asm volatile("mcr p15, 0, %0, c9, c12, 0" : : "r" (pmcr));
>> +}
>> +
>> +/*
>> + * While PMCCNTR can be accessed as a 64 bit coprocessor register, returning 64
>> + * bits doesn't seem worth the trouble when differential usage of the result is
>> + * expected (with differences that can easily fit in 32 bits). So just return
>> + * the lower 32 bits of the cycle count in AArch32.
>> + */
>> +static inline unsigned long get_pmccntr(void)
>> +{
>> +	unsigned long cycles;
>> +
>> +	asm volatile("mrc p15, 0, %0, c9, c13, 0" : "=r" (cycles));
>> +	return cycles;
>> +}
>>  #elif defined(__aarch64__)
>>  static inline uint32_t get_pmcr(void)
>>  {
>> @@ -30,6 +51,19 @@ static inline uint32_t get_pmcr(void)
>>  	asm volatile("mrs %0, pmcr_el0" : "=r" (ret));
>>  	return ret;
>>  }
>> +
>> +static inline void set_pmcr(uint32_t pmcr)
>> +{
>> +	asm volatile("msr pmcr_el0, %0" : : "r" (pmcr));
>> +}
>> +
>> +static inline unsigned long get_pmccntr(void)
>> +{
>> +	unsigned long cycles;
>> +
>> +	asm volatile("mrs %0, pmccntr_el0" : "=r" (cycles));
>> +	return cycles;
>> +}
>>  #endif
>>  
>>  struct pmu_data {
>> @@ -72,11 +106,37 @@ static bool check_pmcr(void)
>>  	return pmu.implementer != 0;
>>  }
>>  
>> +/*
>> + * Ensure that the cycle counter progresses between back-to-back reads.
>> + */
>> +static bool check_cycles_increase(void)
>> +{
>> +	struct pmu_data pmu = { {0} };
> 
> One set of {} is enough, and looks better.

As I tried to mention in the cover letter, that can trigger a spurious GCC
warning (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=53119). But if you don't
mind, I don't mind.

> Otherwise
> Reviewed-by: Andrew Jones <drjones@redhat.com>

Thanks!

Christopher Covington

-- 
Qualcomm Innovation Center, Inc.
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project

Re: [Qemu-devel] [kvm-unit-tests PATCHv5 2/3] arm: pmu: Check cycle count increases

[Andrew Jones]

On Mon, Oct 26, 2015 at 04:58:43PM +0100, Andrew Jones wrote:
> On Mon, Oct 26, 2015 at 11:38:49AM -0400, Christopher Covington wrote:
> > Ensure that reads of the PMCCNTR_EL0 are monotonically increasing,
> > even for the smallest delta of two subsequent reads.
> > 
> > Signed-off-by: Christopher Covington <cov@codeaurora.org>
> > ---
> >  arm/pmu.c | 60 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >  1 file changed, 60 insertions(+)
> > 
> > diff --git a/arm/pmu.c b/arm/pmu.c
> > index 42d0ee1..c44d708 100644
> > --- a/arm/pmu.c
> > +++ b/arm/pmu.c
> > @@ -14,6 +14,8 @@
> >   */
> >  #include "libcflat.h"
> >  
> > +#define NR_SAMPLES 10
> > +
> >  #if defined(__arm__)
> >  static inline uint32_t get_pmcr(void)
> >  {
> > @@ -22,6 +24,25 @@ static inline uint32_t get_pmcr(void)
> >  	asm volatile("mrc p15, 0, %0, c9, c12, 0" : "=r" (ret));
> >  	return ret;
> >  }
> > +
> > +static inline void set_pmcr(uint32_t pmcr)
> > +{
> > +	asm volatile("mcr p15, 0, %0, c9, c12, 0" : : "r" (pmcr));
> > +}
> > +
> > +/*
> > + * While PMCCNTR can be accessed as a 64 bit coprocessor register, returning 64
> > + * bits doesn't seem worth the trouble when differential usage of the result is
> > + * expected (with differences that can easily fit in 32 bits). So just return
> > + * the lower 32 bits of the cycle count in AArch32.
> > + */
> > +static inline unsigned long get_pmccntr(void)
> > +{
> > +	unsigned long cycles;
> > +
> > +	asm volatile("mrc p15, 0, %0, c9, c13, 0" : "=r" (cycles));
> > +	return cycles;
> > +}
> >  #elif defined(__aarch64__)
> >  static inline uint32_t get_pmcr(void)
> >  {
> > @@ -30,6 +51,19 @@ static inline uint32_t get_pmcr(void)
> >  	asm volatile("mrs %0, pmcr_el0" : "=r" (ret));
> >  	return ret;
> >  }
> > +
> > +static inline void set_pmcr(uint32_t pmcr)
> > +{
> > +	asm volatile("msr pmcr_el0, %0" : : "r" (pmcr));
> > +}
> > +
> > +static inline unsigned long get_pmccntr(void)
> > +{
> > +	unsigned long cycles;
> > +
> > +	asm volatile("mrs %0, pmccntr_el0" : "=r" (cycles));
> > +	return cycles;
> > +}
> >  #endif
> >  
> >  struct pmu_data {
> > @@ -72,11 +106,37 @@ static bool check_pmcr(void)
> >  	return pmu.implementer != 0;
> >  }
> >  
> > +/*
> > + * Ensure that the cycle counter progresses between back-to-back reads.
> > + */
> > +static bool check_cycles_increase(void)
> > +{
> > +	struct pmu_data pmu = { {0} };
> 
> One set of {} is enough, and looks better.

Ah, just read your cover letter and now see that this was done on purpose.
So your compiler complains about {0}? Is there a problem besides the
warning? If not, then I'm still a bit inclined to keep the code neat. The
warnings will go away with compiler updates.

Thanks,
drew
> 
> > +
> > +	pmu.enable = 1;
> > +	set_pmcr(pmu.pmcr_el0);
> > +
> > +	for (int i = 0; i < NR_SAMPLES; i++) {
> > +		unsigned long a, b;
> > +
> > +		a = get_pmccntr();
> > +		b = get_pmccntr();
> > +
> > +		if (a >= b) {
> > +			printf("Read %ld then %ld.\n", a, b);
> > +			return false;
> > +		}
> > +	}
> > +
> > +	return true;
> > +}
> > +
> >  int main(void)
> >  {
> >  	report_prefix_push("pmu");
> >  
> >  	report("Control register", check_pmcr());
> > +	report("Monotonically increasing cycle count", check_cycles_increase());
> >  
> >  	return report_summary();
> >  }
> > -- 
> > Qualcomm Innovation Center, Inc.
> > The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
> > a Linux Foundation Collaborative Project
> > 
> >
> 
> Otherwise
> Reviewed-by: Andrew Jones <drjones@redhat.com>

Re: [Qemu-devel] [kvm-unit-tests PATCHv5 3/3] arm: pmu: Add CPI checking

[Andrew Jones]

On Mon, Oct 26, 2015 at 11:38:50AM -0400, Christopher Covington wrote:
> Calculate the numbers of cycles per instruction (CPI) implied by ARM
> PMU cycle counter values. The code includes a strict checking facility
> intended for the -icount option in TCG mode but it is not yet enabled
> in the configuration file. Enabling it must wait on infrastructure
> improvements which allow for different tests to be run on TCG versus
> KVM.
> 
> Signed-off-by: Christopher Covington <cov@codeaurora.org>
> ---
>  arm/pmu.c | 105 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
>  1 file changed, 104 insertions(+), 1 deletion(-)
> 
> diff --git a/arm/pmu.c b/arm/pmu.c
> index c44d708..59f26ab 100644
> --- a/arm/pmu.c
> +++ b/arm/pmu.c
> @@ -43,6 +43,25 @@ static inline unsigned long get_pmccntr(void)
>  	asm volatile("mrc p15, 0, %0, c9, c13, 0" : "=r" (cycles));
>  	return cycles;
>  }
> +
> +/*
> + * Extra instructions such as `mov rd, #0` inserted by the compiler would be

Probably don't need the 'such as `mov..`. Removing it also allows this
comment to directly match the 64-bit one.

> + * difficult to compensate for, so hand assemble everything between, and
> + * including, the PMCR accesses to start and stop counting.
> + */
> +static inline void loop(int i, uint32_t pmcr)
> +{
> +	uint32_t z = 0;
> +
> +	asm volatile(
> +	"	mcr p15, 0, %[pmcr], c9, c12, 0\n"
> +	"1:	subs %[i], %[i], #1\n"
> +	"	bgt 1b\n"
> +	"	mcr p15, 0, %[z], c9, c12, 0\n"

Thanks for making some formatting improvements. We're still
missing the tabs after the instructions though. The format
used by the kernel is

[label]<tab>insn<tab>operands

> +	: [i] "+r" (i)
> +	: [pmcr] "r" (pmcr), [z] "r" (z)

I don't think you should need the z variable. Just doing

[z] "r" (0)

should work.

> +	: "cc");
> +}
>  #elif defined(__aarch64__)
>  static inline uint32_t get_pmcr(void)
>  {
> @@ -64,6 +83,23 @@ static inline unsigned long get_pmccntr(void)
>  	asm volatile("mrs %0, pmccntr_el0" : "=r" (cycles));
>  	return cycles;
>  }
> +
> +/*
> + * Extra instructions inserted by the compiler would be difficult to compensate
> + * for, so hand assemble everything between, and including, the PMCR accesses
> + * to start and stop counting.
> + */
> +static inline void loop(int i, uint32_t pmcr)
> +{
> +	asm volatile(
> +	"	msr pmcr_el0, %[pmcr]\n"
> +	"1:	subs %[i], %[i], #1\n"
> +	"	b.gt 1b\n"
> +	"	msr pmcr_el0, xzr\n"

same tab after insn comment

> +	: [i] "+r" (i)
> +	: [pmcr] "r" (pmcr)
> +	: "cc");
> +}
>  #endif
>  
>  struct pmu_data {
> @@ -131,12 +167,79 @@ static bool check_cycles_increase(void)
>  	return true;
>  }
>  
> -int main(void)
> +/*
> + * Execute a known number of guest instructions. Only odd instruction counts
> + * greater than or equal to 3 are supported by the in-line assembly code. The
> + * control register (PMCR_EL0) is initialized with the provided value (allowing
> + * for example for the cycle counter or event counters to be reset). At the end
> + * of the exact instruction loop, zero is written to PMCR_EL0 to disable
> + * counting, allowing the cycle counter or event counters to be read at the
> + * leisure of the calling code.
> + */
> +static void measure_instrs(int num, uint32_t pmcr)
> +{
> +	int i = (num - 1) / 2;
> +
> +	assert(num >= 3 && ((num - 1) % 2 == 0));
> +	loop(i, pmcr);
> +}
> +
> +/*
> + * Measure cycle counts for various known instruction counts. Ensure that the
> + * cycle counter progresses (similar to check_cycles_increase() but with more
> + * instructions and using reset and stop controls). If supplied a positive,
> + * nonzero CPI parameter, also strictly check that every measurement matches
> + * it. Strict CPI checking is used to test -icount mode.
> + */
> +static bool check_cpi(int cpi)
> +{
> +	struct pmu_data pmu = { {0} };
> +
> +	pmu.cycle_counter_reset = 1;
> +	pmu.enable = 1;
> +
> +	if (cpi > 0)
> +		printf("Checking for CPI=%d.\n", cpi);
> +	printf("instrs : cycles0 cycles1 ...\n");
> +
> +	for (int i = 3; i < 300; i += 32) {
> +		int avg, sum = 0;
> +
> +		printf("%d :", i);
> +		for (int j = 0; j < NR_SAMPLES; j++) {
> +			int cycles;
> +
> +			measure_instrs(i, pmu.pmcr_el0);
> +			cycles = get_pmccntr();
> +			printf(" %d", cycles);
> +
> +			if (!cycles || (cpi > 0 && cycles != i * cpi)) {
> +				printf("\n");
> +				return false;
> +			}
> +
> +			sum += cycles;
> +		}
> +		avg = sum / NR_SAMPLES;
> +		printf(" sum=%d avg=%d avg_ipc=%d avg_cpi=%d\n",
> +			sum, avg, i / avg, avg / i);
> +	}
> +
> +	return true;
> +}
> +
> +int main(int argc, char *argv[])
>  {
> +	int cpi = 0;
> +
> +	if (argc > 1)

if (argc > 0)

Unlike a real main() we don't have the program name automatically put
in argv[0]. This inconsistency is a bit annoying, but to change it now
probably isn't worth it.

> +		cpi = atol(argv[0]);

Ah, looks like the right index is being used here though, so the above
check was probably supposed to be >=

> +
>  	report_prefix_push("pmu");
>  
>  	report("Control register", check_pmcr());
>  	report("Monotonically increasing cycle count", check_cycles_increase());
> +	report("Cycle/instruction ratio", check_cpi(cpi));
>  
>  	return report_summary();
>  }
> -- 
> Qualcomm Innovation Center, Inc.
> The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
> a Linux Foundation Collaborative Project
> 
> 

Thanks,
drew

[Qemu-devel] [kvm-unit-tests PATCHv6] ARM PMU tests

[Christopher Covington]

Changes from v5:

2/3 arm: pmu: Check cycle count increases
* Use universal initializer {0} despite spurious compiler warnings.
* Add Drew's Reviewed-by.

3/3 arm: pmu: Add CPI checking
* Use numeric constant 0 directly with no intermediate variable.
* More tabs in inline assembly.
* Make A32/A64 inline assembly justification comments uniform.
* Check argc properly.

Thanks,
Christopher Covington

[Qemu-devel] [kvm-unit-tests PATCHv5 1/3] arm: Add PMU test

[Christopher Covington]

Beginning with a simple sanity check of the control register, add
a unit test for the ARM Performance Monitors Unit (PMU).

Signed-off-by: Christopher Covington <cov@codeaurora.org>
Reviewed-by: Andrew Jones <drjones@redhat.com>
---
 arm/pmu.c                    | 82 ++++++++++++++++++++++++++++++++++++++++++++
 arm/unittests.cfg            |  5 +++
 config/config-arm-common.mak |  4 ++-
 3 files changed, 90 insertions(+), 1 deletion(-)
 create mode 100644 arm/pmu.c

diff --git a/arm/pmu.c b/arm/pmu.c
new file mode 100644
index 0000000..42d0ee1
--- /dev/null
+++ b/arm/pmu.c
@@ -0,0 +1,82 @@
+/*
+ * Test the ARM Performance Monitors Unit (PMU).
+ *
+ * Copyright 2015 The Linux Foundation. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU Lesser General Public License version 2.1 and
+ * only version 2.1 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License
+ * for more details.
+ */
+#include "libcflat.h"
+
+#if defined(__arm__)
+static inline uint32_t get_pmcr(void)
+{
+	uint32_t ret;
+
+	asm volatile("mrc p15, 0, %0, c9, c12, 0" : "=r" (ret));
+	return ret;
+}
+#elif defined(__aarch64__)
+static inline uint32_t get_pmcr(void)
+{
+	uint32_t ret;
+
+	asm volatile("mrs %0, pmcr_el0" : "=r" (ret));
+	return ret;
+}
+#endif
+
+struct pmu_data {
+	union {
+		uint32_t pmcr_el0;
+		struct {
+			uint32_t enable:1;
+			uint32_t event_counter_reset:1;
+			uint32_t cycle_counter_reset:1;
+			uint32_t cycle_counter_clock_divider:1;
+			uint32_t event_counter_export:1;
+			uint32_t cycle_counter_disable_when_prohibited:1;
+			uint32_t cycle_counter_long:1;
+			uint32_t reserved:4;
+			uint32_t counters:5;
+			uint32_t identification_code:8;
+			uint32_t implementer:8;
+		};
+	};
+};
+
+/*
+ * As a simple sanity check on the PMCR_EL0, ensure the implementer field isn't
+ * null. Also print out a couple other interesting fields for diagnostic
+ * purposes. For example, as of fall 2015, QEMU TCG mode doesn't implement
+ * event counters and therefore reports zero event counters, but hopefully
+ * support for at least the instructions event will be added in the future and
+ * the reported number of event counters will become nonzero.
+ */
+static bool check_pmcr(void)
+{
+	struct pmu_data pmu;
+
+	pmu.pmcr_el0 = get_pmcr();
+
+	printf("PMU implementer:     %c\n", pmu.implementer);
+	printf("Identification code: 0x%x\n", pmu.identification_code);
+	printf("Event counters:      %d\n", pmu.counters);
+
+	return pmu.implementer != 0;
+}
+
+int main(void)
+{
+	report_prefix_push("pmu");
+
+	report("Control register", check_pmcr());
+
+	return report_summary();
+}
diff --git a/arm/unittests.cfg b/arm/unittests.cfg
index e068a0c..fd94adb 100644
--- a/arm/unittests.cfg
+++ b/arm/unittests.cfg
@@ -35,3 +35,8 @@ file = selftest.flat
 smp = `getconf _NPROCESSORS_CONF`
 extra_params = -append 'smp'
 groups = selftest
+
+# Test PMU support without -icount
+[pmu]
+file = pmu.flat
+groups = pmu
diff --git a/config/config-arm-common.mak b/config/config-arm-common.mak
index 698555d..b34d04c 100644
--- a/config/config-arm-common.mak
+++ b/config/config-arm-common.mak
@@ -11,7 +11,8 @@ endif
 
 tests-common = \
 	$(TEST_DIR)/selftest.flat \
-	$(TEST_DIR)/spinlock-test.flat
+	$(TEST_DIR)/spinlock-test.flat \
+	$(TEST_DIR)/pmu.flat
 
 all: test_cases
 
@@ -70,3 +71,4 @@ test_cases: $(generated_files) $(tests-common) $(tests)
 
 $(TEST_DIR)/selftest.elf: $(cstart.o) $(TEST_DIR)/selftest.o
 $(TEST_DIR)/spinlock-test.elf: $(cstart.o) $(TEST_DIR)/spinlock-test.o
+$(TEST_DIR)/pmu.elf: $(cstart.o) $(TEST_DIR)/pmu.o
-- 
Qualcomm Innovation Center, Inc.
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project

[Qemu-devel] [kvm-unit-tests PATCHv5 2/3] arm: pmu: Check cycle count increases

[Christopher Covington]

Ensure that reads of the PMCCNTR_EL0 are monotonically increasing,
even for the smallest delta of two subsequent reads.

Signed-off-by: Christopher Covington <cov@codeaurora.org>
Reviewed-by: Andrew Jones <drjones@redhat.com>
---
 arm/pmu.c | 60 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 60 insertions(+)

diff --git a/arm/pmu.c b/arm/pmu.c
index 42d0ee1..4334de4 100644
--- a/arm/pmu.c
+++ b/arm/pmu.c
@@ -14,6 +14,8 @@
  */
 #include "libcflat.h"
 
+#define NR_SAMPLES 10
+
 #if defined(__arm__)
 static inline uint32_t get_pmcr(void)
 {
@@ -22,6 +24,25 @@ static inline uint32_t get_pmcr(void)
 	asm volatile("mrc p15, 0, %0, c9, c12, 0" : "=r" (ret));
 	return ret;
 }
+
+static inline void set_pmcr(uint32_t pmcr)
+{
+	asm volatile("mcr p15, 0, %0, c9, c12, 0" : : "r" (pmcr));
+}
+
+/*
+ * While PMCCNTR can be accessed as a 64 bit coprocessor register, returning 64
+ * bits doesn't seem worth the trouble when differential usage of the result is
+ * expected (with differences that can easily fit in 32 bits). So just return
+ * the lower 32 bits of the cycle count in AArch32.
+ */
+static inline unsigned long get_pmccntr(void)
+{
+	unsigned long cycles;
+
+	asm volatile("mrc p15, 0, %0, c9, c13, 0" : "=r" (cycles));
+	return cycles;
+}
 #elif defined(__aarch64__)
 static inline uint32_t get_pmcr(void)
 {
@@ -30,6 +51,19 @@ static inline uint32_t get_pmcr(void)
 	asm volatile("mrs %0, pmcr_el0" : "=r" (ret));
 	return ret;
 }
+
+static inline void set_pmcr(uint32_t pmcr)
+{
+	asm volatile("msr pmcr_el0, %0" : : "r" (pmcr));
+}
+
+static inline unsigned long get_pmccntr(void)
+{
+	unsigned long cycles;
+
+	asm volatile("mrs %0, pmccntr_el0" : "=r" (cycles));
+	return cycles;
+}
 #endif
 
 struct pmu_data {
@@ -72,11 +106,37 @@ static bool check_pmcr(void)
 	return pmu.implementer != 0;
 }
 
+/*
+ * Ensure that the cycle counter progresses between back-to-back reads.
+ */
+static bool check_cycles_increase(void)
+{
+	struct pmu_data pmu = {0};
+
+	pmu.enable = 1;
+	set_pmcr(pmu.pmcr_el0);
+
+	for (int i = 0; i < NR_SAMPLES; i++) {
+		unsigned long a, b;
+
+		a = get_pmccntr();
+		b = get_pmccntr();
+
+		if (a >= b) {
+			printf("Read %ld then %ld.\n", a, b);
+			return false;
+		}
+	}
+
+	return true;
+}
+
 int main(void)
 {
 	report_prefix_push("pmu");
 
 	report("Control register", check_pmcr());
+	report("Monotonically increasing cycle count", check_cycles_increase());
 
 	return report_summary();
 }
-- 
Qualcomm Innovation Center, Inc.
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project

[Qemu-devel] [kvm-unit-tests PATCHv5 3/3] arm: pmu: Add CPI checking

[Christopher Covington]

Calculate the numbers of cycles per instruction (CPI) implied by ARM
PMU cycle counter values. The code includes a strict checking facility
intended for the -icount option in TCG mode but it is not yet enabled
in the configuration file. Enabling it must wait on infrastructure
improvements which allow for different tests to be run on TCG versus
KVM.

Signed-off-by: Christopher Covington <cov@codeaurora.org>
---
 arm/pmu.c | 103 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 102 insertions(+), 1 deletion(-)

diff --git a/arm/pmu.c b/arm/pmu.c
index 4334de4..788886a 100644
--- a/arm/pmu.c
+++ b/arm/pmu.c
@@ -43,6 +43,23 @@ static inline unsigned long get_pmccntr(void)
 	asm volatile("mrc p15, 0, %0, c9, c13, 0" : "=r" (cycles));
 	return cycles;
 }
+
+/*
+ * Extra instructions inserted by the compiler would be difficult to compensate
+ * for, so hand assemble everything between, and including, the PMCR accesses
+ * to start and stop counting.
+ */
+static inline void loop(int i, uint32_t pmcr)
+{
+	asm volatile(
+	"	mcr	p15, 0, %[pmcr], c9, c12, 0\n"
+	"1:	subs	%[i], %[i], #1\n"
+	"	bgt	1b\n"
+	"	mcr	p15, 0, %[z], c9, c12, 0\n"
+	: [i] "+r" (i)
+	: [pmcr] "r" (pmcr), [z] "r" (0)
+	: "cc");
+}
 #elif defined(__aarch64__)
 static inline uint32_t get_pmcr(void)
 {
@@ -64,6 +81,23 @@ static inline unsigned long get_pmccntr(void)
 	asm volatile("mrs %0, pmccntr_el0" : "=r" (cycles));
 	return cycles;
 }
+
+/*
+ * Extra instructions inserted by the compiler would be difficult to compensate
+ * for, so hand assemble everything between, and including, the PMCR accesses
+ * to start and stop counting.
+ */
+static inline void loop(int i, uint32_t pmcr)
+{
+	asm volatile(
+	"	msr	pmcr_el0, %[pmcr]\n"
+	"1:	subs	%[i], %[i], #1\n"
+	"	b.gt	1b\n"
+	"	msr	pmcr_el0, xzr\n"
+	: [i] "+r" (i)
+	: [pmcr] "r" (pmcr)
+	: "cc");
+}
 #endif
 
 struct pmu_data {
@@ -131,12 +165,79 @@ static bool check_cycles_increase(void)
 	return true;
 }
 
-int main(void)
+/*
+ * Execute a known number of guest instructions. Only odd instruction counts
+ * greater than or equal to 3 are supported by the in-line assembly code. The
+ * control register (PMCR_EL0) is initialized with the provided value (allowing
+ * for example for the cycle counter or event counters to be reset). At the end
+ * of the exact instruction loop, zero is written to PMCR_EL0 to disable
+ * counting, allowing the cycle counter or event counters to be read at the
+ * leisure of the calling code.
+ */
+static void measure_instrs(int num, uint32_t pmcr)
+{
+	int i = (num - 1) / 2;
+
+	assert(num >= 3 && ((num - 1) % 2 == 0));
+	loop(i, pmcr);
+}
+
+/*
+ * Measure cycle counts for various known instruction counts. Ensure that the
+ * cycle counter progresses (similar to check_cycles_increase() but with more
+ * instructions and using reset and stop controls). If supplied a positive,
+ * nonzero CPI parameter, also strictly check that every measurement matches
+ * it. Strict CPI checking is used to test -icount mode.
+ */
+static bool check_cpi(int cpi)
+{
+	struct pmu_data pmu = {0};
+
+	pmu.cycle_counter_reset = 1;
+	pmu.enable = 1;
+
+	if (cpi > 0)
+		printf("Checking for CPI=%d.\n", cpi);
+	printf("instrs : cycles0 cycles1 ...\n");
+
+	for (int i = 3; i < 300; i += 32) {
+		int avg, sum = 0;
+
+		printf("%d :", i);
+		for (int j = 0; j < NR_SAMPLES; j++) {
+			int cycles;
+
+			measure_instrs(i, pmu.pmcr_el0);
+			cycles = get_pmccntr();
+			printf(" %d", cycles);
+
+			if (!cycles || (cpi > 0 && cycles != i * cpi)) {
+				printf("\n");
+				return false;
+			}
+
+			sum += cycles;
+		}
+		avg = sum / NR_SAMPLES;
+		printf(" sum=%d avg=%d avg_ipc=%d avg_cpi=%d\n",
+			sum, avg, i / avg, avg / i);
+	}
+
+	return true;
+}
+
+int main(int argc, char *argv[])
 {
+	int cpi = 0;
+
+	if (argc >= 1)
+		cpi = atol(argv[0]);
+
 	report_prefix_push("pmu");
 
 	report("Control register", check_pmcr());
 	report("Monotonically increasing cycle count", check_cycles_increase());
+	report("Cycle/instruction ratio", check_cpi(cpi));
 
 	return report_summary();
 }
-- 
Qualcomm Innovation Center, Inc.
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project

Re: [Qemu-devel] [kvm-unit-tests PATCHv5 3/3] arm: pmu: Add CPI checking

[Andrew Jones]

On Wed, Oct 28, 2015 at 03:12:55PM -0400, Christopher Covington wrote:
> Calculate the numbers of cycles per instruction (CPI) implied by ARM
> PMU cycle counter values. The code includes a strict checking facility
> intended for the -icount option in TCG mode but it is not yet enabled
> in the configuration file. Enabling it must wait on infrastructure
> improvements which allow for different tests to be run on TCG versus
> KVM.
> 
> Signed-off-by: Christopher Covington <cov@codeaurora.org>
> ---
>  arm/pmu.c | 103 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
>  1 file changed, 102 insertions(+), 1 deletion(-)
> 
> diff --git a/arm/pmu.c b/arm/pmu.c
> index 4334de4..788886a 100644
> --- a/arm/pmu.c
> +++ b/arm/pmu.c
> @@ -43,6 +43,23 @@ static inline unsigned long get_pmccntr(void)
>  	asm volatile("mrc p15, 0, %0, c9, c13, 0" : "=r" (cycles));
>  	return cycles;
>  }
> +
> +/*
> + * Extra instructions inserted by the compiler would be difficult to compensate
> + * for, so hand assemble everything between, and including, the PMCR accesses
> + * to start and stop counting.
> + */
> +static inline void loop(int i, uint32_t pmcr)
> +{
> +	asm volatile(
> +	"	mcr	p15, 0, %[pmcr], c9, c12, 0\n"
> +	"1:	subs	%[i], %[i], #1\n"
> +	"	bgt	1b\n"
> +	"	mcr	p15, 0, %[z], c9, c12, 0\n"
> +	: [i] "+r" (i)
> +	: [pmcr] "r" (pmcr), [z] "r" (0)
> +	: "cc");
> +}
>  #elif defined(__aarch64__)
>  static inline uint32_t get_pmcr(void)
>  {
> @@ -64,6 +81,23 @@ static inline unsigned long get_pmccntr(void)
>  	asm volatile("mrs %0, pmccntr_el0" : "=r" (cycles));
>  	return cycles;
>  }
> +
> +/*
> + * Extra instructions inserted by the compiler would be difficult to compensate
> + * for, so hand assemble everything between, and including, the PMCR accesses
> + * to start and stop counting.
> + */
> +static inline void loop(int i, uint32_t pmcr)
> +{
> +	asm volatile(
> +	"	msr	pmcr_el0, %[pmcr]\n"
> +	"1:	subs	%[i], %[i], #1\n"
> +	"	b.gt	1b\n"
> +	"	msr	pmcr_el0, xzr\n"
> +	: [i] "+r" (i)
> +	: [pmcr] "r" (pmcr)
> +	: "cc");
> +}
>  #endif
>  
>  struct pmu_data {
> @@ -131,12 +165,79 @@ static bool check_cycles_increase(void)
>  	return true;
>  }
>  
> -int main(void)
> +/*
> + * Execute a known number of guest instructions. Only odd instruction counts
> + * greater than or equal to 3 are supported by the in-line assembly code. The
> + * control register (PMCR_EL0) is initialized with the provided value (allowing
> + * for example for the cycle counter or event counters to be reset). At the end
> + * of the exact instruction loop, zero is written to PMCR_EL0 to disable
> + * counting, allowing the cycle counter or event counters to be read at the
> + * leisure of the calling code.
> + */
> +static void measure_instrs(int num, uint32_t pmcr)
> +{
> +	int i = (num - 1) / 2;
> +
> +	assert(num >= 3 && ((num - 1) % 2 == 0));
> +	loop(i, pmcr);
> +}
> +
> +/*
> + * Measure cycle counts for various known instruction counts. Ensure that the
> + * cycle counter progresses (similar to check_cycles_increase() but with more
> + * instructions and using reset and stop controls). If supplied a positive,
> + * nonzero CPI parameter, also strictly check that every measurement matches
> + * it. Strict CPI checking is used to test -icount mode.
> + */
> +static bool check_cpi(int cpi)
> +{
> +	struct pmu_data pmu = {0};
> +
> +	pmu.cycle_counter_reset = 1;
> +	pmu.enable = 1;
> +
> +	if (cpi > 0)
> +		printf("Checking for CPI=%d.\n", cpi);
> +	printf("instrs : cycles0 cycles1 ...\n");
> +
> +	for (int i = 3; i < 300; i += 32) {
> +		int avg, sum = 0;
> +
> +		printf("%d :", i);
> +		for (int j = 0; j < NR_SAMPLES; j++) {
> +			int cycles;
> +
> +			measure_instrs(i, pmu.pmcr_el0);
> +			cycles = get_pmccntr();
> +			printf(" %d", cycles);
> +
> +			if (!cycles || (cpi > 0 && cycles != i * cpi)) {
> +				printf("\n");
> +				return false;
> +			}
> +
> +			sum += cycles;
> +		}
> +		avg = sum / NR_SAMPLES;
> +		printf(" sum=%d avg=%d avg_ipc=%d avg_cpi=%d\n",
> +			sum, avg, i / avg, avg / i);
> +	}
> +
> +	return true;
> +}
> +
> +int main(int argc, char *argv[])
>  {
> +	int cpi = 0;
> +
> +	if (argc >= 1)
> +		cpi = atol(argv[0]);
> +
>  	report_prefix_push("pmu");
>  
>  	report("Control register", check_pmcr());
>  	report("Monotonically increasing cycle count", check_cycles_increase());
> +	report("Cycle/instruction ratio", check_cpi(cpi));
>  
>  	return report_summary();
>  }
> -- 
> Qualcomm Innovation Center, Inc.
> The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
> a Linux Foundation Collaborative Project
> 
>

I applied and tested this (by adding -icount 1 -append 1 to the cmdline),
but get

qemu-system-aarch64 -machine virt,accel=tcg -cpu cortex-a57 \
	-device virtio-serial-device -device virtconsole,chardev=ctd \
	-chardev testdev,id=ctd -display none -serial stdio \
	-kernel arm/pmu.flat -smp 1 -icount 1 -append 1
PMU implementer:     A
Identification code: 0x0
Event counters:      0
PASS: pmu: Control register
PASS: pmu: Monotonically increasing cycle count
Checking for CPI=1.
instrs : cycles0 cycles1 ...
3 : 6
FAIL: pmu: Cycle/instruction ratio

SUMMARY: 3 tests, 1 unexpected failures 

I'm using a latest pull of qemu master. Taking '-append 1' off the cmdline,
we can see that for all trials we're getting twice the expected cpi.

3 : 6 6 6 6 6 6 6 6 6 6 sum=60 avg=6 avg_ipc=0 avg_cpi=2
35 : 70 70 70 70 70 70 70 70 70 70 sum=700 avg=70 avg_ipc=0 avg_cpi=2
67 : 134 134 134 134 134 134 134 134 134 134 sum=1340 avg=134 avg_ipc=0 avg_cpi=2
99 : 198 198 198 198 198 198 198 198 198 198 sum=1980 avg=198 avg_ipc=0 avg_cpi=2
131 : 262 262 262 262 262 262 262 262 262 262 sum=2620 avg=262 avg_ipc=0 avg_cpi=2
163 : 326 326 326 326 326 326 326 326 326 326 sum=3260 avg=326 avg_ipc=0 avg_cpi=2
195 : 390 390 390 390 390 390 390 390 390 390 sum=3900 avg=390 avg_ipc=0 avg_cpi=2
227 : 454 454 454 454 454 454 454 454 454 454 sum=4540 avg=454 avg_ipc=0 avg_cpi=2
259 : 518 518 518 518 518 518 518 518 518 518 sum=5180 avg=518 avg_ipc=0 avg_cpi=2
291 : 582 582 582 582 582 582 582 582 582 582 sum=5820 avg=582 avg_ipc=0 avg_cpi=2

drew

Re: [Qemu-devel] [kvm-unit-tests PATCHv5 3/3] arm: pmu: Add CPI checking

[Christopher Covington]

Hi Drew,

On 10/30/2015 09:00 AM, Andrew Jones wrote:
> On Wed, Oct 28, 2015 at 03:12:55PM -0400, Christopher Covington wrote:
>> Calculate the numbers of cycles per instruction (CPI) implied by ARM
>> PMU cycle counter values. The code includes a strict checking facility
>> intended for the -icount option in TCG mode but it is not yet enabled
>> in the configuration file. Enabling it must wait on infrastructure
>> improvements which allow for different tests to be run on TCG versus
>> KVM.
>>
>> Signed-off-by: Christopher Covington <cov@codeaurora.org>
>> ---
>>  arm/pmu.c | 103 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
>>  1 file changed, 102 insertions(+), 1 deletion(-)
>>
>> diff --git a/arm/pmu.c b/arm/pmu.c
>> index 4334de4..788886a 100644
>> --- a/arm/pmu.c
>> +++ b/arm/pmu.c
>> @@ -43,6 +43,23 @@ static inline unsigned long get_pmccntr(void)
>>  	asm volatile("mrc p15, 0, %0, c9, c13, 0" : "=r" (cycles));
>>  	return cycles;
>>  }
>> +
>> +/*
>> + * Extra instructions inserted by the compiler would be difficult to compensate
>> + * for, so hand assemble everything between, and including, the PMCR accesses
>> + * to start and stop counting.
>> + */
>> +static inline void loop(int i, uint32_t pmcr)
>> +{
>> +	asm volatile(
>> +	"	mcr	p15, 0, %[pmcr], c9, c12, 0\n"
>> +	"1:	subs	%[i], %[i], #1\n"
>> +	"	bgt	1b\n"
>> +	"	mcr	p15, 0, %[z], c9, c12, 0\n"
>> +	: [i] "+r" (i)
>> +	: [pmcr] "r" (pmcr), [z] "r" (0)
>> +	: "cc");
>> +}
>>  #elif defined(__aarch64__)
>>  static inline uint32_t get_pmcr(void)
>>  {
>> @@ -64,6 +81,23 @@ static inline unsigned long get_pmccntr(void)
>>  	asm volatile("mrs %0, pmccntr_el0" : "=r" (cycles));
>>  	return cycles;
>>  }
>> +
>> +/*
>> + * Extra instructions inserted by the compiler would be difficult to compensate
>> + * for, so hand assemble everything between, and including, the PMCR accesses
>> + * to start and stop counting.
>> + */
>> +static inline void loop(int i, uint32_t pmcr)
>> +{
>> +	asm volatile(
>> +	"	msr	pmcr_el0, %[pmcr]\n"
>> +	"1:	subs	%[i], %[i], #1\n"
>> +	"	b.gt	1b\n"
>> +	"	msr	pmcr_el0, xzr\n"
>> +	: [i] "+r" (i)
>> +	: [pmcr] "r" (pmcr)
>> +	: "cc");
>> +}
>>  #endif
>>  
>>  struct pmu_data {
>> @@ -131,12 +165,79 @@ static bool check_cycles_increase(void)
>>  	return true;
>>  }
>>  
>> -int main(void)
>> +/*
>> + * Execute a known number of guest instructions. Only odd instruction counts
>> + * greater than or equal to 3 are supported by the in-line assembly code. The
>> + * control register (PMCR_EL0) is initialized with the provided value (allowing
>> + * for example for the cycle counter or event counters to be reset). At the end
>> + * of the exact instruction loop, zero is written to PMCR_EL0 to disable
>> + * counting, allowing the cycle counter or event counters to be read at the
>> + * leisure of the calling code.
>> + */
>> +static void measure_instrs(int num, uint32_t pmcr)
>> +{
>> +	int i = (num - 1) / 2;
>> +
>> +	assert(num >= 3 && ((num - 1) % 2 == 0));
>> +	loop(i, pmcr);
>> +}
>> +
>> +/*
>> + * Measure cycle counts for various known instruction counts. Ensure that the
>> + * cycle counter progresses (similar to check_cycles_increase() but with more
>> + * instructions and using reset and stop controls). If supplied a positive,
>> + * nonzero CPI parameter, also strictly check that every measurement matches
>> + * it. Strict CPI checking is used to test -icount mode.
>> + */
>> +static bool check_cpi(int cpi)
>> +{
>> +	struct pmu_data pmu = {0};
>> +
>> +	pmu.cycle_counter_reset = 1;
>> +	pmu.enable = 1;
>> +
>> +	if (cpi > 0)
>> +		printf("Checking for CPI=%d.\n", cpi);
>> +	printf("instrs : cycles0 cycles1 ...\n");
>> +
>> +	for (int i = 3; i < 300; i += 32) {
>> +		int avg, sum = 0;
>> +
>> +		printf("%d :", i);
>> +		for (int j = 0; j < NR_SAMPLES; j++) {
>> +			int cycles;
>> +
>> +			measure_instrs(i, pmu.pmcr_el0);
>> +			cycles = get_pmccntr();
>> +			printf(" %d", cycles);
>> +
>> +			if (!cycles || (cpi > 0 && cycles != i * cpi)) {
>> +				printf("\n");
>> +				return false;
>> +			}
>> +
>> +			sum += cycles;
>> +		}
>> +		avg = sum / NR_SAMPLES;
>> +		printf(" sum=%d avg=%d avg_ipc=%d avg_cpi=%d\n",
>> +			sum, avg, i / avg, avg / i);
>> +	}
>> +
>> +	return true;
>> +}
>> +
>> +int main(int argc, char *argv[])
>>  {
>> +	int cpi = 0;
>> +
>> +	if (argc >= 1)
>> +		cpi = atol(argv[0]);
>> +
>>  	report_prefix_push("pmu");
>>  
>>  	report("Control register", check_pmcr());
>>  	report("Monotonically increasing cycle count", check_cycles_increase());
>> +	report("Cycle/instruction ratio", check_cpi(cpi));
>>  
>>  	return report_summary();
>>  }
> 
> I applied and tested this (by adding -icount 1 -append 1 to the cmdline),

Thanks for giving this a spin. For whatever reason the -icount argument is the
exponent n in 2^n. I could match that logic if you prefer, but the pmu.c code
currently takes the fully calculated shift value rather than the exponent.
I've been testing with the following option pairs (dependent on 'accel = tcg'
support).

-- >8 --
Subject: [PATCH] arm: pmu: Add -icount checking configurations

Pass a couple -icount values in TCG mode and strictly check the
resulting cycle counts.

Signed-off-by: Christopher Covington <cov@codeaurora.org>
---
 arm/unittests.cfg | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/arm/unittests.cfg b/arm/unittests.cfg
index fd94adb..5ca1e6a 100644
--- a/arm/unittests.cfg
+++ b/arm/unittests.cfg
@@ -40,3 +40,17 @@ groups = selftest
 [pmu]
 file = pmu.flat
 groups = pmu
+
+# Test PMU support with -icount IPC=1
+[pmu-icount-1]
+file = pmu.flat
+extra_params = -icount 0 -append '1'
+groups = pmu
+accel = tcg
+
+# Test PMU support with -icount IPC=256
+[pmu-icount-256]
+file = pmu.flat
+extra_params = -icount 8 -append '256'
+groups = pmu
+accel = tcg
-- 
Qualcomm Innovation Center, Inc.
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project

Re: [Qemu-devel] [kvm-unit-tests PATCHv5 3/3] arm: pmu: Add CPI checking

[Andrew Jones]

On Fri, Oct 30, 2015 at 03:32:43PM -0400, Christopher Covington wrote:
> Hi Drew,
> 
> On 10/30/2015 09:00 AM, Andrew Jones wrote:
> > On Wed, Oct 28, 2015 at 03:12:55PM -0400, Christopher Covington wrote:
> >> Calculate the numbers of cycles per instruction (CPI) implied by ARM
> >> PMU cycle counter values. The code includes a strict checking facility
> >> intended for the -icount option in TCG mode but it is not yet enabled
> >> in the configuration file. Enabling it must wait on infrastructure
> >> improvements which allow for different tests to be run on TCG versus
> >> KVM.
> >>
> >> Signed-off-by: Christopher Covington <cov@codeaurora.org>
> >> ---
> >>  arm/pmu.c | 103 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
> >>  1 file changed, 102 insertions(+), 1 deletion(-)
> >>
> >> diff --git a/arm/pmu.c b/arm/pmu.c
> >> index 4334de4..788886a 100644
> >> --- a/arm/pmu.c
> >> +++ b/arm/pmu.c
> >> @@ -43,6 +43,23 @@ static inline unsigned long get_pmccntr(void)
> >>  	asm volatile("mrc p15, 0, %0, c9, c13, 0" : "=r" (cycles));
> >>  	return cycles;
> >>  }
> >> +
> >> +/*
> >> + * Extra instructions inserted by the compiler would be difficult to compensate
> >> + * for, so hand assemble everything between, and including, the PMCR accesses
> >> + * to start and stop counting.
> >> + */
> >> +static inline void loop(int i, uint32_t pmcr)
> >> +{
> >> +	asm volatile(
> >> +	"	mcr	p15, 0, %[pmcr], c9, c12, 0\n"
> >> +	"1:	subs	%[i], %[i], #1\n"
> >> +	"	bgt	1b\n"
> >> +	"	mcr	p15, 0, %[z], c9, c12, 0\n"
> >> +	: [i] "+r" (i)
> >> +	: [pmcr] "r" (pmcr), [z] "r" (0)
> >> +	: "cc");
> >> +}
> >>  #elif defined(__aarch64__)
> >>  static inline uint32_t get_pmcr(void)
> >>  {
> >> @@ -64,6 +81,23 @@ static inline unsigned long get_pmccntr(void)
> >>  	asm volatile("mrs %0, pmccntr_el0" : "=r" (cycles));
> >>  	return cycles;
> >>  }
> >> +
> >> +/*
> >> + * Extra instructions inserted by the compiler would be difficult to compensate
> >> + * for, so hand assemble everything between, and including, the PMCR accesses
> >> + * to start and stop counting.
> >> + */
> >> +static inline void loop(int i, uint32_t pmcr)
> >> +{
> >> +	asm volatile(
> >> +	"	msr	pmcr_el0, %[pmcr]\n"
> >> +	"1:	subs	%[i], %[i], #1\n"
> >> +	"	b.gt	1b\n"
> >> +	"	msr	pmcr_el0, xzr\n"
> >> +	: [i] "+r" (i)
> >> +	: [pmcr] "r" (pmcr)
> >> +	: "cc");
> >> +}
> >>  #endif
> >>  
> >>  struct pmu_data {
> >> @@ -131,12 +165,79 @@ static bool check_cycles_increase(void)
> >>  	return true;
> >>  }
> >>  
> >> -int main(void)
> >> +/*
> >> + * Execute a known number of guest instructions. Only odd instruction counts
> >> + * greater than or equal to 3 are supported by the in-line assembly code. The
> >> + * control register (PMCR_EL0) is initialized with the provided value (allowing
> >> + * for example for the cycle counter or event counters to be reset). At the end
> >> + * of the exact instruction loop, zero is written to PMCR_EL0 to disable
> >> + * counting, allowing the cycle counter or event counters to be read at the
> >> + * leisure of the calling code.
> >> + */
> >> +static void measure_instrs(int num, uint32_t pmcr)
> >> +{
> >> +	int i = (num - 1) / 2;
> >> +
> >> +	assert(num >= 3 && ((num - 1) % 2 == 0));
> >> +	loop(i, pmcr);
> >> +}
> >> +
> >> +/*
> >> + * Measure cycle counts for various known instruction counts. Ensure that the
> >> + * cycle counter progresses (similar to check_cycles_increase() but with more
> >> + * instructions and using reset and stop controls). If supplied a positive,
> >> + * nonzero CPI parameter, also strictly check that every measurement matches
> >> + * it. Strict CPI checking is used to test -icount mode.
> >> + */
> >> +static bool check_cpi(int cpi)
> >> +{
> >> +	struct pmu_data pmu = {0};
> >> +
> >> +	pmu.cycle_counter_reset = 1;
> >> +	pmu.enable = 1;
> >> +
> >> +	if (cpi > 0)
> >> +		printf("Checking for CPI=%d.\n", cpi);
> >> +	printf("instrs : cycles0 cycles1 ...\n");
> >> +
> >> +	for (int i = 3; i < 300; i += 32) {
> >> +		int avg, sum = 0;
> >> +
> >> +		printf("%d :", i);
> >> +		for (int j = 0; j < NR_SAMPLES; j++) {
> >> +			int cycles;
> >> +
> >> +			measure_instrs(i, pmu.pmcr_el0);
> >> +			cycles = get_pmccntr();
> >> +			printf(" %d", cycles);
> >> +
> >> +			if (!cycles || (cpi > 0 && cycles != i * cpi)) {
> >> +				printf("\n");
> >> +				return false;
> >> +			}
> >> +
> >> +			sum += cycles;
> >> +		}
> >> +		avg = sum / NR_SAMPLES;
> >> +		printf(" sum=%d avg=%d avg_ipc=%d avg_cpi=%d\n",
> >> +			sum, avg, i / avg, avg / i);
> >> +	}
> >> +
> >> +	return true;
> >> +}
> >> +
> >> +int main(int argc, char *argv[])
> >>  {
> >> +	int cpi = 0;
> >> +
> >> +	if (argc >= 1)
> >> +		cpi = atol(argv[0]);
> >> +
> >>  	report_prefix_push("pmu");
> >>  
> >>  	report("Control register", check_pmcr());
> >>  	report("Monotonically increasing cycle count", check_cycles_increase());
> >> +	report("Cycle/instruction ratio", check_cpi(cpi));
> >>  
> >>  	return report_summary();
> >>  }
> > 
> > I applied and tested this (by adding -icount 1 -append 1 to the cmdline),
> 
> Thanks for giving this a spin. For whatever reason the -icount argument is the
> exponent n in 2^n. I could match that logic if you prefer, but the pmu.c code

Oh yeah, I forgot how you had that in your earlier posts. So in that
case

Reviewed-by: Andrew Jones <drjones@redhat.com>

I've applied these three patches to 

https://github.com/rhdrjones/kvm-unit-tests/commits/staging

I'll send a pull request to Paolo for that branch soon.

Thanks,
drew

> currently takes the fully calculated shift value rather than the exponent.
> I've been testing with the following option pairs (dependent on 'accel = tcg'
> support).
> 
> -- >8 --
> Subject: [PATCH] arm: pmu: Add -icount checking configurations
> 
> Pass a couple -icount values in TCG mode and strictly check the
> resulting cycle counts.
> 
> Signed-off-by: Christopher Covington <cov@codeaurora.org>
> ---
>  arm/unittests.cfg | 14 ++++++++++++++
>  1 file changed, 14 insertions(+)
> 
> diff --git a/arm/unittests.cfg b/arm/unittests.cfg
> index fd94adb..5ca1e6a 100644
> --- a/arm/unittests.cfg
> +++ b/arm/unittests.cfg
> @@ -40,3 +40,17 @@ groups = selftest
>  [pmu]
>  file = pmu.flat
>  groups = pmu
> +
> +# Test PMU support with -icount IPC=1
> +[pmu-icount-1]
> +file = pmu.flat
> +extra_params = -icount 0 -append '1'
> +groups = pmu
> +accel = tcg
> +
> +# Test PMU support with -icount IPC=256
> +[pmu-icount-256]
> +file = pmu.flat
> +extra_params = -icount 8 -append '256'
> +groups = pmu
> +accel = tcg
> -- 
> Qualcomm Innovation Center, Inc.
> The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
> a Linux Foundation Collaborative Project
> 

Re: [Qemu-devel] [kvm-unit-tests PATCHv5 3/3] arm: pmu: Add CPI checking

[Andrew Jones]

On Mon, Nov 02, 2015 at 09:58:14AM -0600, Andrew Jones wrote:
> On Fri, Oct 30, 2015 at 03:32:43PM -0400, Christopher Covington wrote:
> > Hi Drew,
> > 
> > On 10/30/2015 09:00 AM, Andrew Jones wrote:
> > > On Wed, Oct 28, 2015 at 03:12:55PM -0400, Christopher Covington wrote:
> > >> Calculate the numbers of cycles per instruction (CPI) implied by ARM
> > >> PMU cycle counter values. The code includes a strict checking facility
> > >> intended for the -icount option in TCG mode but it is not yet enabled
> > >> in the configuration file. Enabling it must wait on infrastructure
> > >> improvements which allow for different tests to be run on TCG versus
> > >> KVM.
> > >>
> > >> Signed-off-by: Christopher Covington <cov@codeaurora.org>
> > >> ---
> > >>  arm/pmu.c | 103 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
> > >>  1 file changed, 102 insertions(+), 1 deletion(-)
> > >>
> > >> diff --git a/arm/pmu.c b/arm/pmu.c
> > >> index 4334de4..788886a 100644
> > >> --- a/arm/pmu.c
> > >> +++ b/arm/pmu.c
> > >> @@ -43,6 +43,23 @@ static inline unsigned long get_pmccntr(void)
> > >>  	asm volatile("mrc p15, 0, %0, c9, c13, 0" : "=r" (cycles));
> > >>  	return cycles;
> > >>  }
> > >> +
> > >> +/*
> > >> + * Extra instructions inserted by the compiler would be difficult to compensate
> > >> + * for, so hand assemble everything between, and including, the PMCR accesses
> > >> + * to start and stop counting.
> > >> + */
> > >> +static inline void loop(int i, uint32_t pmcr)
> > >> +{
> > >> +	asm volatile(
> > >> +	"	mcr	p15, 0, %[pmcr], c9, c12, 0\n"
> > >> +	"1:	subs	%[i], %[i], #1\n"
> > >> +	"	bgt	1b\n"
> > >> +	"	mcr	p15, 0, %[z], c9, c12, 0\n"
> > >> +	: [i] "+r" (i)
> > >> +	: [pmcr] "r" (pmcr), [z] "r" (0)
> > >> +	: "cc");
> > >> +}
> > >>  #elif defined(__aarch64__)
> > >>  static inline uint32_t get_pmcr(void)
> > >>  {
> > >> @@ -64,6 +81,23 @@ static inline unsigned long get_pmccntr(void)
> > >>  	asm volatile("mrs %0, pmccntr_el0" : "=r" (cycles));
> > >>  	return cycles;
> > >>  }
> > >> +
> > >> +/*
> > >> + * Extra instructions inserted by the compiler would be difficult to compensate
> > >> + * for, so hand assemble everything between, and including, the PMCR accesses
> > >> + * to start and stop counting.
> > >> + */
> > >> +static inline void loop(int i, uint32_t pmcr)
> > >> +{
> > >> +	asm volatile(
> > >> +	"	msr	pmcr_el0, %[pmcr]\n"
> > >> +	"1:	subs	%[i], %[i], #1\n"
> > >> +	"	b.gt	1b\n"
> > >> +	"	msr	pmcr_el0, xzr\n"
> > >> +	: [i] "+r" (i)
> > >> +	: [pmcr] "r" (pmcr)
> > >> +	: "cc");
> > >> +}
> > >>  #endif
> > >>  
> > >>  struct pmu_data {
> > >> @@ -131,12 +165,79 @@ static bool check_cycles_increase(void)
> > >>  	return true;
> > >>  }
> > >>  
> > >> -int main(void)
> > >> +/*
> > >> + * Execute a known number of guest instructions. Only odd instruction counts
> > >> + * greater than or equal to 3 are supported by the in-line assembly code. The
> > >> + * control register (PMCR_EL0) is initialized with the provided value (allowing
> > >> + * for example for the cycle counter or event counters to be reset). At the end
> > >> + * of the exact instruction loop, zero is written to PMCR_EL0 to disable
> > >> + * counting, allowing the cycle counter or event counters to be read at the
> > >> + * leisure of the calling code.
> > >> + */
> > >> +static void measure_instrs(int num, uint32_t pmcr)
> > >> +{
> > >> +	int i = (num - 1) / 2;
> > >> +
> > >> +	assert(num >= 3 && ((num - 1) % 2 == 0));
> > >> +	loop(i, pmcr);
> > >> +}
> > >> +
> > >> +/*
> > >> + * Measure cycle counts for various known instruction counts. Ensure that the
> > >> + * cycle counter progresses (similar to check_cycles_increase() but with more
> > >> + * instructions and using reset and stop controls). If supplied a positive,
> > >> + * nonzero CPI parameter, also strictly check that every measurement matches
> > >> + * it. Strict CPI checking is used to test -icount mode.
> > >> + */
> > >> +static bool check_cpi(int cpi)
> > >> +{
> > >> +	struct pmu_data pmu = {0};
> > >> +
> > >> +	pmu.cycle_counter_reset = 1;
> > >> +	pmu.enable = 1;
> > >> +
> > >> +	if (cpi > 0)
> > >> +		printf("Checking for CPI=%d.\n", cpi);
> > >> +	printf("instrs : cycles0 cycles1 ...\n");
> > >> +
> > >> +	for (int i = 3; i < 300; i += 32) {
> > >> +		int avg, sum = 0;
> > >> +
> > >> +		printf("%d :", i);
> > >> +		for (int j = 0; j < NR_SAMPLES; j++) {
> > >> +			int cycles;
> > >> +
> > >> +			measure_instrs(i, pmu.pmcr_el0);
> > >> +			cycles = get_pmccntr();
> > >> +			printf(" %d", cycles);
> > >> +
> > >> +			if (!cycles || (cpi > 0 && cycles != i * cpi)) {
> > >> +				printf("\n");
> > >> +				return false;
> > >> +			}
> > >> +
> > >> +			sum += cycles;
> > >> +		}
> > >> +		avg = sum / NR_SAMPLES;
> > >> +		printf(" sum=%d avg=%d avg_ipc=%d avg_cpi=%d\n",
> > >> +			sum, avg, i / avg, avg / i);
> > >> +	}
> > >> +
> > >> +	return true;
> > >> +}
> > >> +
> > >> +int main(int argc, char *argv[])
> > >>  {
> > >> +	int cpi = 0;
> > >> +
> > >> +	if (argc >= 1)
> > >> +		cpi = atol(argv[0]);
> > >> +
> > >>  	report_prefix_push("pmu");
> > >>  
> > >>  	report("Control register", check_pmcr());
> > >>  	report("Monotonically increasing cycle count", check_cycles_increase());
> > >> +	report("Cycle/instruction ratio", check_cpi(cpi));
> > >>  
> > >>  	return report_summary();
> > >>  }
> > > 
> > > I applied and tested this (by adding -icount 1 -append 1 to the cmdline),
> > 
> > Thanks for giving this a spin. For whatever reason the -icount argument is the
> > exponent n in 2^n. I could match that logic if you prefer, but the pmu.c code
> 
> Oh yeah, I forgot how you had that in your earlier posts. So in that
> case
> 
> Reviewed-by: Andrew Jones <drjones@redhat.com>
> 
> I've applied these three patches to 
> 
> https://github.com/rhdrjones/kvm-unit-tests/commits/staging
> 
> I'll send a pull request to Paolo for that branch soon.

Hi Christopher,

I still have these queued on staging, but before we ask Paolo to commit,
I think we should try them with Shannon's patches for KVM in order to
make sure they work there, as well as with TCG (I have a feeling we'll
need to initialize a couple more registers). Also, now that we've got
the 'ACCEL' patch in master, we can take the unittests.cfg -icount patch
as well. Can you please resubmit with any changes needed for KVM, and
also with a unittests.cfg patch enabling both KVM and TCG tests?

Thanks,
drew

> 
> Thanks,
> drew
> 
> > currently takes the fully calculated shift value rather than the exponent.
> > I've been testing with the following option pairs (dependent on 'accel = tcg'
> > support).
> > 
> > -- >8 --
> > Subject: [PATCH] arm: pmu: Add -icount checking configurations
> > 
> > Pass a couple -icount values in TCG mode and strictly check the
> > resulting cycle counts.
> > 
> > Signed-off-by: Christopher Covington <cov@codeaurora.org>
> > ---
> >  arm/unittests.cfg | 14 ++++++++++++++
> >  1 file changed, 14 insertions(+)
> > 
> > diff --git a/arm/unittests.cfg b/arm/unittests.cfg
> > index fd94adb..5ca1e6a 100644
> > --- a/arm/unittests.cfg
> > +++ b/arm/unittests.cfg
> > @@ -40,3 +40,17 @@ groups = selftest
> >  [pmu]
> >  file = pmu.flat
> >  groups = pmu
> > +
> > +# Test PMU support with -icount IPC=1
> > +[pmu-icount-1]
> > +file = pmu.flat
> > +extra_params = -icount 0 -append '1'
> > +groups = pmu
> > +accel = tcg
> > +
> > +# Test PMU support with -icount IPC=256
> > +[pmu-icount-256]
> > +file = pmu.flat
> > +extra_params = -icount 8 -append '256'
> > +groups = pmu
> > +accel = tcg
> > -- 
> > Qualcomm Innovation Center, Inc.
> > The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
> > a Linux Foundation Collaborative Project
> > 

Re: [Qemu-devel] [kvm-unit-tests PATCHv5 3/3] arm: pmu: Add CPI checking

[Christopher Covington]

On 11/10/2015 09:05 PM, Andrew Jones wrote:
> On Mon, Nov 02, 2015 at 09:58:14AM -0600, Andrew Jones wrote:
>> On Fri, Oct 30, 2015 at 03:32:43PM -0400, Christopher Covington wrote:
>>> Hi Drew,
>>>
>>> On 10/30/2015 09:00 AM, Andrew Jones wrote:
>>>> On Wed, Oct 28, 2015 at 03:12:55PM -0400, Christopher Covington wrote:
>>>>> Calculate the numbers of cycles per instruction (CPI) implied by ARM
>>>>> PMU cycle counter values. The code includes a strict checking facility
>>>>> intended for the -icount option in TCG mode but it is not yet enabled
>>>>> in the configuration file. Enabling it must wait on infrastructure
>>>>> improvements which allow for different tests to be run on TCG versus
>>>>> KVM.
>>>>>
>>>>> Signed-off-by: Christopher Covington <cov@codeaurora.org>
>>>>> ---
>>>>>  arm/pmu.c | 103 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
>>>>>  1 file changed, 102 insertions(+), 1 deletion(-)
>>>>>
>>>>> diff --git a/arm/pmu.c b/arm/pmu.c
>>>>> index 4334de4..788886a 100644
>>>>> --- a/arm/pmu.c
>>>>> +++ b/arm/pmu.c
>>>>> @@ -43,6 +43,23 @@ static inline unsigned long get_pmccntr(void)
>>>>>  	asm volatile("mrc p15, 0, %0, c9, c13, 0" : "=r" (cycles));
>>>>>  	return cycles;
>>>>>  }
>>>>> +
>>>>> +/*
>>>>> + * Extra instructions inserted by the compiler would be difficult to compensate
>>>>> + * for, so hand assemble everything between, and including, the PMCR accesses
>>>>> + * to start and stop counting.
>>>>> + */
>>>>> +static inline void loop(int i, uint32_t pmcr)
>>>>> +{
>>>>> +	asm volatile(
>>>>> +	"	mcr	p15, 0, %[pmcr], c9, c12, 0\n"
>>>>> +	"1:	subs	%[i], %[i], #1\n"
>>>>> +	"	bgt	1b\n"
>>>>> +	"	mcr	p15, 0, %[z], c9, c12, 0\n"
>>>>> +	: [i] "+r" (i)
>>>>> +	: [pmcr] "r" (pmcr), [z] "r" (0)
>>>>> +	: "cc");
>>>>> +}
>>>>>  #elif defined(__aarch64__)
>>>>>  static inline uint32_t get_pmcr(void)
>>>>>  {
>>>>> @@ -64,6 +81,23 @@ static inline unsigned long get_pmccntr(void)
>>>>>  	asm volatile("mrs %0, pmccntr_el0" : "=r" (cycles));
>>>>>  	return cycles;
>>>>>  }
>>>>> +
>>>>> +/*
>>>>> + * Extra instructions inserted by the compiler would be difficult to compensate
>>>>> + * for, so hand assemble everything between, and including, the PMCR accesses
>>>>> + * to start and stop counting.
>>>>> + */
>>>>> +static inline void loop(int i, uint32_t pmcr)
>>>>> +{
>>>>> +	asm volatile(
>>>>> +	"	msr	pmcr_el0, %[pmcr]\n"
>>>>> +	"1:	subs	%[i], %[i], #1\n"
>>>>> +	"	b.gt	1b\n"
>>>>> +	"	msr	pmcr_el0, xzr\n"
>>>>> +	: [i] "+r" (i)
>>>>> +	: [pmcr] "r" (pmcr)
>>>>> +	: "cc");
>>>>> +}
>>>>>  #endif
>>>>>  
>>>>>  struct pmu_data {
>>>>> @@ -131,12 +165,79 @@ static bool check_cycles_increase(void)
>>>>>  	return true;
>>>>>  }
>>>>>  
>>>>> -int main(void)
>>>>> +/*
>>>>> + * Execute a known number of guest instructions. Only odd instruction counts
>>>>> + * greater than or equal to 3 are supported by the in-line assembly code. The
>>>>> + * control register (PMCR_EL0) is initialized with the provided value (allowing
>>>>> + * for example for the cycle counter or event counters to be reset). At the end
>>>>> + * of the exact instruction loop, zero is written to PMCR_EL0 to disable
>>>>> + * counting, allowing the cycle counter or event counters to be read at the
>>>>> + * leisure of the calling code.
>>>>> + */
>>>>> +static void measure_instrs(int num, uint32_t pmcr)
>>>>> +{
>>>>> +	int i = (num - 1) / 2;
>>>>> +
>>>>> +	assert(num >= 3 && ((num - 1) % 2 == 0));
>>>>> +	loop(i, pmcr);
>>>>> +}
>>>>> +
>>>>> +/*
>>>>> + * Measure cycle counts for various known instruction counts. Ensure that the
>>>>> + * cycle counter progresses (similar to check_cycles_increase() but with more
>>>>> + * instructions and using reset and stop controls). If supplied a positive,
>>>>> + * nonzero CPI parameter, also strictly check that every measurement matches
>>>>> + * it. Strict CPI checking is used to test -icount mode.
>>>>> + */
>>>>> +static bool check_cpi(int cpi)
>>>>> +{
>>>>> +	struct pmu_data pmu = {0};
>>>>> +
>>>>> +	pmu.cycle_counter_reset = 1;
>>>>> +	pmu.enable = 1;
>>>>> +
>>>>> +	if (cpi > 0)
>>>>> +		printf("Checking for CPI=%d.\n", cpi);
>>>>> +	printf("instrs : cycles0 cycles1 ...\n");
>>>>> +
>>>>> +	for (int i = 3; i < 300; i += 32) {
>>>>> +		int avg, sum = 0;
>>>>> +
>>>>> +		printf("%d :", i);
>>>>> +		for (int j = 0; j < NR_SAMPLES; j++) {
>>>>> +			int cycles;
>>>>> +
>>>>> +			measure_instrs(i, pmu.pmcr_el0);
>>>>> +			cycles = get_pmccntr();
>>>>> +			printf(" %d", cycles);
>>>>> +
>>>>> +			if (!cycles || (cpi > 0 && cycles != i * cpi)) {
>>>>> +				printf("\n");
>>>>> +				return false;
>>>>> +			}
>>>>> +
>>>>> +			sum += cycles;
>>>>> +		}
>>>>> +		avg = sum / NR_SAMPLES;
>>>>> +		printf(" sum=%d avg=%d avg_ipc=%d avg_cpi=%d\n",
>>>>> +			sum, avg, i / avg, avg / i);
>>>>> +	}
>>>>> +
>>>>> +	return true;
>>>>> +}
>>>>> +
>>>>> +int main(int argc, char *argv[])
>>>>>  {
>>>>> +	int cpi = 0;
>>>>> +
>>>>> +	if (argc >= 1)
>>>>> +		cpi = atol(argv[0]);
>>>>> +
>>>>>  	report_prefix_push("pmu");
>>>>>  
>>>>>  	report("Control register", check_pmcr());
>>>>>  	report("Monotonically increasing cycle count", check_cycles_increase());
>>>>> +	report("Cycle/instruction ratio", check_cpi(cpi));
>>>>>  
>>>>>  	return report_summary();
>>>>>  }
>>>>
>>>> I applied and tested this (by adding -icount 1 -append 1 to the cmdline),
>>>
>>> Thanks for giving this a spin. For whatever reason the -icount argument is the
>>> exponent n in 2^n. I could match that logic if you prefer, but the pmu.c code
>>
>> Oh yeah, I forgot how you had that in your earlier posts. So in that
>> case
>>
>> Reviewed-by: Andrew Jones <drjones@redhat.com>
>>
>> I've applied these three patches to 
>>
>> https://github.com/rhdrjones/kvm-unit-tests/commits/staging
>>
>> I'll send a pull request to Paolo for that branch soon.
> 
> Hi Christopher,
> 
> I still have these queued on staging, but before we ask Paolo to commit,
> I think we should try them with Shannon's patches for KVM in order to
> make sure they work there, as well as with TCG (I have a feeling we'll
> need to initialize a couple more registers). Also, now that we've got
> the 'ACCEL' patch in master, we can take the unittests.cfg -icount patch
> as well. Can you please resubmit with any changes needed for KVM, and
> also with a unittests.cfg patch enabling both KVM and TCG tests?

Sure I'll do the additional testing and resubmit with any necessary modifications.

Christopher Covington

-- 
Qualcomm Innovation Center, Inc.
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project