* ARMv6 performance counters v2
@ 2009-12-14 14:04 Jamie Iles
2009-12-14 14:04 ` [PATCH 1/5] arm: provide a mechanism to reserve performance counters Jamie Iles
0 siblings, 1 reply; 28+ messages in thread
From: Jamie Iles @ 2009-12-14 14:04 UTC (permalink / raw)
To: linux-arm-kernel
This patch series implements performance counter support for ARMv6
architectures. The changes since the previous patch series are:
- the pmu interface (arch/arm/include/asm/pmu.h) to return the
interrupts for the PMU so that PMU interrupts are stored in a common
place (arch/arm/kernel/pmu.c) for all platforms and users.
- The addition of a pmu_init() function that sets the IRQ affinity for
each PMU to the owning CPU. This was previously done in oprofile but
also needs to be done for perf events so put it in a common place.
- hardware perf events are checked to ensure the whole group can go
onto the cpu when initialised.
- style cleanups to perf_events.c
- the use of the generic atomic64's has been split out into a separate
patch. When we have proper hardware support (as in Will Deacon's
patch) we can use that.
Jamie Iles (5):
arm: provide a mechanism to reserve performance counters
arm/oprofile: reserve the PMU when starting
arm: use the spinlocked, generic atomic64 support
arm: enable support for software perf events
arm/perfevents: implement perf event support for ARMv6
^ permalink raw reply [flat|nested] 28+ messages in thread* [PATCH 1/5] arm: provide a mechanism to reserve performance counters 2009-12-14 14:04 ARMv6 performance counters v2 Jamie Iles @ 2009-12-14 14:04 ` Jamie Iles 2009-12-14 14:04 ` [PATCH 2/5] arm/oprofile: reserve the PMU when starting Jamie Iles ` (2 more replies) 0 siblings, 3 replies; 28+ messages in thread From: Jamie Iles @ 2009-12-14 14:04 UTC (permalink / raw) To: linux-arm-kernel To add support for perf events and to allow the hardware counters to be shared with oprofile, we need a way to reserve access to the pmu (performance monitor unit). Cc: Will Deacon <will.deacon@arm.com> Signed-off-by: Jamie Iles <jamie.iles@picochip.com> --- arch/arm/include/asm/pmu.h | 76 +++++++++++++++++++++++++++++++ arch/arm/kernel/Makefile | 1 + arch/arm/kernel/pmu.c | 108 ++++++++++++++++++++++++++++++++++++++++++++ arch/arm/mm/Kconfig | 6 +++ 4 files changed, 191 insertions(+), 0 deletions(-) create mode 100644 arch/arm/include/asm/pmu.h create mode 100644 arch/arm/kernel/pmu.c diff --git a/arch/arm/include/asm/pmu.h b/arch/arm/include/asm/pmu.h new file mode 100644 index 0000000..d66a7cd --- /dev/null +++ b/arch/arm/include/asm/pmu.h @@ -0,0 +1,76 @@ +/* + * linux/arch/arm/include/asm/pmu.h + * + * Copyright (C) 2009 picoChip Designs Ltd, Jamie Iles + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + */ + +#ifndef __ARM_PMU_H__ +#define __ARM_PMU_H__ + +#ifdef CONFIG_CPU_HAS_PMU + +#define MAX_PMU_IRQS 8 + +struct pmu_irqs { + int irqs[MAX_PMU_IRQS]; + unsigned num_irqs; +}; + +/** + * reserve_pmu() - reserve the hardware performance counters + * + * Reserve the hardware performance counters in the system for exclusive use. + * The 'struct pmu_irqs' for the system is returned on success, ERR_PTR() + * encoded error on failure. + */ +extern const struct pmu_irqs * +reserve_pmu(void); + +/** + * release_pmu() - Relinquish control of the performance counters + * + * Release the performance counters and allow someone else to use them. + * Callers must have disabled the counters and released IRQs before calling + * this. The 'struct pmu_irqs' returned from reserve_pmu() must be passed as + * a cookie. + */ +extern void +release_pmu(const struct pmu_irqs *irqs); + +/** + * init_pmu() - Initialise the PMU. + * + * Initialise the system ready for PMU enabling. This should typically set the + * IRQ affinity and nothing else. The users (oprofile/perf events etc) will do + * the actual hardware initialisation. + */ +extern int +init_pmu(void); + +#else /* CONFIG_CPU_HAS_PMU */ + +static inline const struct pmu_irqs * +reserve_pmu(void) +{ + ERR_PTR(-ENODEV); +} + +static inline void +release_pmu(const struct pmu_irqs *irqs) +{ +} + +static inline int +init_pmu(void) +{ + return -ENODEV; +} + +#endif /* CONFIG_CPU_HAS_PMU */ + +#endif /* __ARM_PMU_H__ */ diff --git a/arch/arm/kernel/Makefile b/arch/arm/kernel/Makefile index e7ccf7e..286a276 100644 --- a/arch/arm/kernel/Makefile +++ b/arch/arm/kernel/Makefile @@ -46,6 +46,7 @@ obj-$(CONFIG_CPU_XSCALE) += xscale-cp0.o obj-$(CONFIG_CPU_XSC3) += xscale-cp0.o obj-$(CONFIG_CPU_MOHAWK) += xscale-cp0.o obj-$(CONFIG_IWMMXT) += iwmmxt.o +obj-$(CONFIG_CPU_HAS_PMU) += pmu.o AFLAGS_iwmmxt.o := -Wa,-mcpu=iwmmxt ifneq ($(CONFIG_ARCH_EBSA110),y) diff --git a/arch/arm/kernel/pmu.c b/arch/arm/kernel/pmu.c new file mode 100644 index 0000000..881e526 --- /dev/null +++ b/arch/arm/kernel/pmu.c @@ -0,0 +1,108 @@ +/* + * linux/arch/arm/kernel/pmu.c + * + * Copyright (C) 2009 picoChip Designs Ltd, Jamie Iles + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + */ + +#include <linux/kernel.h> +#include <linux/module.h> +#include <linux/semaphore.h> +#include <linux/err.h> +#include <linux/irq.h> + +#include <asm/pmu.h> +#include <asm/irq.h> + +/* + * Define the IRQs for the system. We could use something like a platform + * device but that seems fairly heavyweight for this. Also, the performance + * counters can't be removed or hotplugged. + * + * Ordering is important: init_pmu() will use the ordering to set the affinity + * to the corresponding core. e.g. the first interrupt will go to cpu 0, the + * second goes to cpu 1 etc. + */ +static const struct pmu_irqs pmu_irqs = { +#ifdef CONFIG_ARCH_PC3XX + .irqs = { IRQ_NPMUIRQ }, + .num_irqs = 1, +#elif defined(CONFIG_ARCH_OMAP2) + .irqs = { 3 }, + .num_irqs = 1, +#elif defined(CONFIG_ARCH_BCMRING) + .irqs = { IRQ_PMUIRQ }, + .num_irqs = 1, +#elif defined(CONFIG_MACH_REALVIEW_EB) + .irqs = { + [0] = IRQ_EB11MP_PMU_CPU0, + [1] = IRQ_EB11MP_PMU_CPU1, + [2] = IRQ_EB11MP_PMU_CPU2, + [3] = IRQ_EB11MP_PMU_CPU3 + }, + .num_irqs = 4, +#elif defined(CONFIG_ARCH_OMAP3) + .irqs = { INT_34XX_BENCH_MPU_EMUL }, + .num_irqs = 1, +#elif defined(CONFIG_ARCH_IOP32X) + .irqs = { IRQ_IOP32X_CORE_PMU }, + .num_irqs = 1, +#elif defined(CONFIG_ARCH_IOP33X) + .irqs = { IRQ_IOP33X_CORE_PMU }, + .num_irqs = 1, +#elif defined(CONFIG_ARCH_PXA) + .irqs = { IRQ_PMU }, + .num_irqs = 1, +#endif +}; + +static DECLARE_MUTEX(pmu_mutex); + +const struct pmu_irqs * +reserve_pmu(void) +{ + int ret = down_trylock(&pmu_mutex) ? -EBUSY : 0; + + return ret ? ERR_PTR(ret) : &pmu_irqs; +} +EXPORT_SYMBOL_GPL(reserve_pmu); + +void +release_pmu(const struct pmu_irqs *irqs) +{ + WARN_ON(irqs != &pmu_irqs); + up(&pmu_mutex); +} +EXPORT_SYMBOL_GPL(release_pmu); + +static void +set_irq_affinity(int irq, + unsigned int cpu) +{ +#ifdef CONFIG_SMP + struct irq_desc *desc = irq_desc + irq; + const struct cpumask *mask = cpumask_of(cpu); + unsigned long flags; + + raw_spin_lock_irqsave(&desc->lock, flags); + cpumask_copy(desc->affinity, mask); + desc->chip->set_affinity(irq, mask); + raw_spin_unlock_irqrestore(&desc->lock, flags); +#endif +} + +int +init_pmu(void) +{ + int i; + + for (i = 0; i < pmu_irqs.num_irqs; ++i) + set_irq_affinity(pmu_irqs.irqs[i], i); + + return 0; +} +EXPORT_SYMBOL_GPL(init_pmu); diff --git a/arch/arm/mm/Kconfig b/arch/arm/mm/Kconfig index dd4698c..fc5c05b 100644 --- a/arch/arm/mm/Kconfig +++ b/arch/arm/mm/Kconfig @@ -342,6 +342,7 @@ config CPU_XSCALE select CPU_PABRT_LEGACY select CPU_CACHE_VIVT select CPU_CP15_MMU + select CPU_HAS_PMU select CPU_TLB_V4WBI if MMU # XScale Core Version 3 @@ -398,6 +399,7 @@ config CPU_V6 select CPU_HAS_ASID if MMU select CPU_COPY_V6 if MMU select CPU_TLB_V6 if MMU + select CPU_HAS_PMU # ARMv6k config CPU_32v6K @@ -421,6 +423,7 @@ config CPU_V7 select CPU_CACHE_V7 select CPU_CACHE_VIPT select CPU_CP15_MMU + select CPU_HAS_PMU select CPU_HAS_ASID if MMU select CPU_COPY_V6 if MMU select CPU_TLB_V7 if MMU @@ -536,6 +539,9 @@ config CPU_COPY_FA config CPU_COPY_V6 bool +config CPU_HAS_PMU + bool + # This selects the TLB model config CPU_TLB_V3 bool -- 1.6.5.4 ^ permalink raw reply related [flat|nested] 28+ messages in thread
* [PATCH 2/5] arm/oprofile: reserve the PMU when starting 2009-12-14 14:04 ` [PATCH 1/5] arm: provide a mechanism to reserve performance counters Jamie Iles @ 2009-12-14 14:04 ` Jamie Iles 2009-12-14 14:04 ` [PATCH 3/5] arm: use the spinlocked, generic atomic64 support Jamie Iles ` (2 more replies) 2009-12-14 14:39 ` [PATCH 1/5] arm: provide a mechanism to reserve performance counters Will Deacon 2009-12-14 16:01 ` Jean Pihet 2 siblings, 3 replies; 28+ messages in thread From: Jamie Iles @ 2009-12-14 14:04 UTC (permalink / raw) To: linux-arm-kernel Make sure that we have access to the performance counters and that they aren't being used by perf events or anything else. Cc: Will Deacon <will.deacon@arm.com> Signed-off-by: Jamie Iles <jamie.iles@picochip.com> --- arch/arm/oprofile/op_model_arm11_core.c | 4 +- arch/arm/oprofile/op_model_arm11_core.h | 4 +- arch/arm/oprofile/op_model_mpcore.c | 42 ++++++++++++++++-------------- arch/arm/oprofile/op_model_v6.c | 33 ++++++++++++++---------- arch/arm/oprofile/op_model_v7.c | 30 ++++++++++++++-------- arch/arm/oprofile/op_model_v7.h | 4 +- arch/arm/oprofile/op_model_xscale.c | 35 ++++++++++++++----------- 7 files changed, 85 insertions(+), 67 deletions(-) diff --git a/arch/arm/oprofile/op_model_arm11_core.c b/arch/arm/oprofile/op_model_arm11_core.c index ad80752..ef3e265 100644 --- a/arch/arm/oprofile/op_model_arm11_core.c +++ b/arch/arm/oprofile/op_model_arm11_core.c @@ -132,7 +132,7 @@ static irqreturn_t arm11_pmu_interrupt(int irq, void *arg) return IRQ_HANDLED; } -int arm11_request_interrupts(int *irqs, int nr) +int arm11_request_interrupts(const int *irqs, int nr) { unsigned int i; int ret = 0; @@ -153,7 +153,7 @@ int arm11_request_interrupts(int *irqs, int nr) return ret; } -void arm11_release_interrupts(int *irqs, int nr) +void arm11_release_interrupts(const int *irqs, int nr) { unsigned int i; diff --git a/arch/arm/oprofile/op_model_arm11_core.h b/arch/arm/oprofile/op_model_arm11_core.h index 6f8538e..1902b99 100644 --- a/arch/arm/oprofile/op_model_arm11_core.h +++ b/arch/arm/oprofile/op_model_arm11_core.h @@ -39,7 +39,7 @@ int arm11_setup_pmu(void); int arm11_start_pmu(void); int arm11_stop_pmu(void); -int arm11_request_interrupts(int *, int); -void arm11_release_interrupts(int *, int); +int arm11_request_interrupts(const int *, int); +void arm11_release_interrupts(const int *, int); #endif diff --git a/arch/arm/oprofile/op_model_mpcore.c b/arch/arm/oprofile/op_model_mpcore.c index 4ce0f98..f73ce87 100644 --- a/arch/arm/oprofile/op_model_mpcore.c +++ b/arch/arm/oprofile/op_model_mpcore.c @@ -32,6 +32,7 @@ /* #define DEBUG */ #include <linux/types.h> #include <linux/errno.h> +#include <linux/err.h> #include <linux/sched.h> #include <linux/oprofile.h> #include <linux/interrupt.h> @@ -43,6 +44,7 @@ #include <mach/hardware.h> #include <mach/board-eb.h> #include <asm/system.h> +#include <asm/pmu.h> #include "op_counter.h" #include "op_arm_model.h" @@ -58,6 +60,7 @@ * Bitmask of used SCU counters */ static unsigned int scu_em_used; +static const struct pmu_irqs *pmu_irqs; /* * 2 helper fns take a counter number from 0-7 (not the userspace-visible counter number) @@ -225,33 +228,40 @@ static int em_setup_ctrs(void) return 0; } -static int arm11_irqs[] = { - [0] = IRQ_EB11MP_PMU_CPU0, - [1] = IRQ_EB11MP_PMU_CPU1, - [2] = IRQ_EB11MP_PMU_CPU2, - [3] = IRQ_EB11MP_PMU_CPU3 -}; - static int em_start(void) { int ret; - ret = arm11_request_interrupts(arm11_irqs, ARRAY_SIZE(arm11_irqs)); + pmu_irqs = reserve_pmu(); + if (IS_ERR(pmu_irqs)) { + ret = PTR_ERR(pmu_irqs); + goto out; + } + + ret = arm11_request_interrupts(pmu_irqs->irqs, pmu_irqs->num_irqs); if (ret == 0) { em_call_function(arm11_start_pmu); ret = scu_start(); - if (ret) - arm11_release_interrupts(arm11_irqs, ARRAY_SIZE(arm11_irqs)); + if (ret) { + arm11_release_interrupts(pmu_irqs->irqs, + pmu_irqs->num_irqs); + } else { + release_pmu(pmu_irqs); + pmu_irqs = NULL; + } } + +out: return ret; } static void em_stop(void) { em_call_function(arm11_stop_pmu); - arm11_release_interrupts(arm11_irqs, ARRAY_SIZE(arm11_irqs)); + arm11_release_interrupts(pmu_irqs->irqs, pmu_irqs->num_irqs); scu_stop(); + release_pmu(pmu_irqs); } /* @@ -283,15 +293,7 @@ static int em_setup(void) em_route_irq(IRQ_EB11MP_PMU_SCU6, 3); em_route_irq(IRQ_EB11MP_PMU_SCU7, 3); - /* - * Send CP15 PMU interrupts to the owner CPU. - */ - em_route_irq(IRQ_EB11MP_PMU_CPU0, 0); - em_route_irq(IRQ_EB11MP_PMU_CPU1, 1); - em_route_irq(IRQ_EB11MP_PMU_CPU2, 2); - em_route_irq(IRQ_EB11MP_PMU_CPU3, 3); - - return 0; + return init_pmu(); } struct op_arm_model_spec op_mpcore_spec = { diff --git a/arch/arm/oprofile/op_model_v6.c b/arch/arm/oprofile/op_model_v6.c index e468017..a22357a 100644 --- a/arch/arm/oprofile/op_model_v6.c +++ b/arch/arm/oprofile/op_model_v6.c @@ -19,42 +19,47 @@ /* #define DEBUG */ #include <linux/types.h> #include <linux/errno.h> +#include <linux/err.h> #include <linux/sched.h> #include <linux/oprofile.h> #include <linux/interrupt.h> #include <asm/irq.h> #include <asm/system.h> +#include <asm/pmu.h> #include "op_counter.h" #include "op_arm_model.h" #include "op_model_arm11_core.h" -static int irqs[] = { -#ifdef CONFIG_ARCH_OMAP2 - 3, -#endif -#ifdef CONFIG_ARCH_BCMRING - IRQ_PMUIRQ, /* for BCMRING, ARM PMU interrupt is 43 */ -#endif -#ifdef CONFIG_ARCH_PC3XX - IRQ_NPMUIRQ, -#endif -}; +static const struct pmu_irqs *pmu_irqs; static void armv6_pmu_stop(void) { arm11_stop_pmu(); - arm11_release_interrupts(irqs, ARRAY_SIZE(irqs)); + arm11_release_interrupts(pmu_irqs->irqs, pmu_irqs->num_irqs); + release_pmu(pmu_irqs); + pmu_irqs = NULL; } static int armv6_pmu_start(void) { int ret; - ret = arm11_request_interrupts(irqs, ARRAY_SIZE(irqs)); - if (ret >= 0) + pmu_irqs = reserve_pmu(); + if (IS_ERR(pmu_irqs)) { + ret = PTR_ERR(pmu_irqs); + goto out; + } + + ret = arm11_request_interrupts(pmu_irqs->irqs, pmu_irqs->num_irqs); + if (ret >= 0) { ret = arm11_start_pmu(); + } else { + release_pmu(pmu_irqs); + pmu_irqs = NULL; + } +out: return ret; } diff --git a/arch/arm/oprofile/op_model_v7.c b/arch/arm/oprofile/op_model_v7.c index f20295f..9258fca 100644 --- a/arch/arm/oprofile/op_model_v7.c +++ b/arch/arm/oprofile/op_model_v7.c @@ -11,11 +11,14 @@ */ #include <linux/types.h> #include <linux/errno.h> +#include <linux/err.h> #include <linux/oprofile.h> #include <linux/interrupt.h> #include <linux/irq.h> #include <linux/smp.h> +#include <asm/pmu.h> + #include "op_counter.h" #include "op_arm_model.h" #include "op_model_v7.h" @@ -299,7 +302,7 @@ static irqreturn_t armv7_pmnc_interrupt(int irq, void *arg) return IRQ_HANDLED; } -int armv7_request_interrupts(int *irqs, int nr) +int armv7_request_interrupts(const int *irqs, int nr) { unsigned int i; int ret = 0; @@ -322,7 +325,7 @@ int armv7_request_interrupts(int *irqs, int nr) return ret; } -void armv7_release_interrupts(int *irqs, int nr) +void armv7_release_interrupts(const int *irqs, int nr) { unsigned int i; @@ -366,12 +369,7 @@ static void armv7_pmnc_dump_regs(void) } #endif - -static int irqs[] = { -#ifdef CONFIG_ARCH_OMAP3 - INT_34XX_BENCH_MPU_EMUL, -#endif -}; +static const struct pmu_irqs *pmu_irqs; static void armv7_pmnc_stop(void) { @@ -379,19 +377,29 @@ static void armv7_pmnc_stop(void) armv7_pmnc_dump_regs(); #endif armv7_stop_pmnc(); - armv7_release_interrupts(irqs, ARRAY_SIZE(irqs)); + armv7_release_interrupts(pmu_irqs->irqs, pmu_irqs->num_irqs); + release_pmu(pmu_irqs); + pmu_irqs = NULL; } static int armv7_pmnc_start(void) { int ret; + pmu_irqs = reserve_pmu(); + if (IS_ERR(pmu_irqs)) + return PTR_ERR(pmu_irqs); + #ifdef DEBUG armv7_pmnc_dump_regs(); #endif - ret = armv7_request_interrupts(irqs, ARRAY_SIZE(irqs)); - if (ret >= 0) + ret = armv7_request_interrupts(pmu_irqs->irqs, pmu_irqs->num_irqs); + if (ret >= 0) { armv7_start_pmnc(); + } else { + release_pmu(pmu_irqs); + pmu_irqs = NULL; + } return ret; } diff --git a/arch/arm/oprofile/op_model_v7.h b/arch/arm/oprofile/op_model_v7.h index 0e19bcc..9ca334b 100644 --- a/arch/arm/oprofile/op_model_v7.h +++ b/arch/arm/oprofile/op_model_v7.h @@ -97,7 +97,7 @@ int armv7_setup_pmu(void); int armv7_start_pmu(void); int armv7_stop_pmu(void); -int armv7_request_interrupts(int *, int); -void armv7_release_interrupts(int *, int); +int armv7_request_interrupts(const int *, int); +void armv7_release_interrupts(const int *, int); #endif diff --git a/arch/arm/oprofile/op_model_xscale.c b/arch/arm/oprofile/op_model_xscale.c index 724ab9c..1d34a02 100644 --- a/arch/arm/oprofile/op_model_xscale.c +++ b/arch/arm/oprofile/op_model_xscale.c @@ -17,12 +17,14 @@ /* #define DEBUG */ #include <linux/types.h> #include <linux/errno.h> +#include <linux/err.h> #include <linux/sched.h> #include <linux/oprofile.h> #include <linux/interrupt.h> #include <linux/irq.h> #include <asm/cputype.h> +#include <asm/pmu.h> #include "op_counter.h" #include "op_arm_model.h" @@ -33,17 +35,6 @@ #define PMU_RESET (CCNT_RESET | PMN_RESET) #define PMU_CNT64 0x008 /* Make CCNT count every 64th cycle */ -/* TODO do runtime detection */ -#ifdef CONFIG_ARCH_IOP32X -#define XSCALE_PMU_IRQ IRQ_IOP32X_CORE_PMU -#endif -#ifdef CONFIG_ARCH_IOP33X -#define XSCALE_PMU_IRQ IRQ_IOP33X_CORE_PMU -#endif -#ifdef CONFIG_ARCH_PXA -#define XSCALE_PMU_IRQ IRQ_PMU -#endif - /* * Different types of events that can be counted by the XScale PMU * as used by Oprofile userspace. Here primarily for documentation @@ -367,6 +358,8 @@ static irqreturn_t xscale_pmu_interrupt(int irq, void *arg) return IRQ_HANDLED; } +static const struct pmu_irqs *pmu_irqs; + static void xscale_pmu_stop(void) { u32 pmnc = read_pmnc(); @@ -374,20 +367,30 @@ static void xscale_pmu_stop(void) pmnc &= ~PMU_ENABLE; write_pmnc(pmnc); - free_irq(XSCALE_PMU_IRQ, results); + free_irq(pmu_irqs->irqs[0], results); + release_pmu(pmu_irqs); + pmu_irqs = NULL; } static int xscale_pmu_start(void) { int ret; - u32 pmnc = read_pmnc(); + u32 pmnc; + + pmu_irqs = reserve_pmu(); + if (IS_ERR(pmu_irqs)) + return PTR_ERR(pmu_irqs); + + pmnc = read_pmnc(); - ret = request_irq(XSCALE_PMU_IRQ, xscale_pmu_interrupt, IRQF_DISABLED, - "XScale PMU", (void *)results); + ret = request_irq(pmu_irqs->irqs[0], xscale_pmu_interrupt, + IRQF_DISABLED, "XScale PMU", (void *)results); if (ret < 0) { printk(KERN_ERR "oprofile: unable to request IRQ%d for XScale PMU\n", - XSCALE_PMU_IRQ); + pmu_irqs->irqs[0]); + release_pmu(pmu_irqs); + pmu_irqs = NULL; return ret; } -- 1.6.5.4 ^ permalink raw reply related [flat|nested] 28+ messages in thread
* [PATCH 3/5] arm: use the spinlocked, generic atomic64 support 2009-12-14 14:04 ` [PATCH 2/5] arm/oprofile: reserve the PMU when starting Jamie Iles @ 2009-12-14 14:04 ` Jamie Iles 2009-12-14 14:04 ` [PATCH 4/5] arm: enable support for software perf events Jamie Iles 2009-12-14 17:38 ` [PATCH 3/5] arm: use the spinlocked, generic atomic64 support Nicolas Pitre 2009-12-14 16:01 ` [PATCH 2/5] arm/oprofile: reserve the PMU when starting Jean Pihet 2009-12-14 16:04 ` Will Deacon 2 siblings, 2 replies; 28+ messages in thread From: Jamie Iles @ 2009-12-14 14:04 UTC (permalink / raw) To: linux-arm-kernel perf events require that we can support atomic64's. There is a generic, spinlocked version that we can use until we have proper hardware support. Signed-off-by: Jamie Iles <jamie.iles@picochip.com> --- arch/arm/Kconfig | 1 + arch/arm/include/asm/atomic.h | 4 ++++ 2 files changed, 5 insertions(+), 0 deletions(-) diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig index c40efec..b13cb85 100644 --- a/arch/arm/Kconfig +++ b/arch/arm/Kconfig @@ -18,6 +18,7 @@ config ARM select HAVE_KRETPROBES if (HAVE_KPROBES) select HAVE_FUNCTION_TRACER if (!XIP_KERNEL) select HAVE_GENERIC_DMA_COHERENT + select GENERIC_ATOMIC64 help The ARM series is a line of low-power-consumption RISC chip designs licensed by ARM Ltd and targeted at embedded applications and diff --git a/arch/arm/include/asm/atomic.h b/arch/arm/include/asm/atomic.h index d0daeab..ff286a8 100644 --- a/arch/arm/include/asm/atomic.h +++ b/arch/arm/include/asm/atomic.h @@ -15,6 +15,10 @@ #include <linux/types.h> #include <asm/system.h> +#ifdef CONFIG_GENERIC_ATOMIC64 +#include <asm-generic/atomic64.h> +#endif + #define ATOMIC_INIT(i) { (i) } #ifdef __KERNEL__ -- 1.6.5.4 ^ permalink raw reply related [flat|nested] 28+ messages in thread
* [PATCH 4/5] arm: enable support for software perf events 2009-12-14 14:04 ` [PATCH 3/5] arm: use the spinlocked, generic atomic64 support Jamie Iles @ 2009-12-14 14:04 ` Jamie Iles 2009-12-14 14:04 ` [PATCH 5/5] arm/perfevents: implement perf event support for ARMv6 Jamie Iles 2009-12-14 17:38 ` [PATCH 3/5] arm: use the spinlocked, generic atomic64 support Nicolas Pitre 1 sibling, 1 reply; 28+ messages in thread From: Jamie Iles @ 2009-12-14 14:04 UTC (permalink / raw) To: linux-arm-kernel The perf events subsystem allows counting of both hardware and software events. This patch implements the bare minimum for software performance events. Signed-off-by: Jamie Iles <jamie.iles@picochip.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Ingo Molnar <mingo@elte.hu> --- arch/arm/Kconfig | 2 + arch/arm/include/asm/perf_event.h | 38 +++++++++++++++++++++++++++++++++++++ arch/arm/mm/fault.c | 7 ++++++ 3 files changed, 47 insertions(+), 0 deletions(-) create mode 100644 arch/arm/include/asm/perf_event.h diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig index b13cb85..dc4eb0f 100644 --- a/arch/arm/Kconfig +++ b/arch/arm/Kconfig @@ -19,6 +19,8 @@ config ARM select HAVE_FUNCTION_TRACER if (!XIP_KERNEL) select HAVE_GENERIC_DMA_COHERENT select GENERIC_ATOMIC64 + select HAVE_PERF_EVENTS + select PERF_USE_VMALLOC help The ARM series is a line of low-power-consumption RISC chip designs licensed by ARM Ltd and targeted at embedded applications and diff --git a/arch/arm/include/asm/perf_event.h b/arch/arm/include/asm/perf_event.h new file mode 100644 index 0000000..32a66ac --- /dev/null +++ b/arch/arm/include/asm/perf_event.h @@ -0,0 +1,38 @@ +/* + * linux/arch/arm/include/asm/perf_event.h + * + * Copyright (C) 2009 picoChip Designs Ltd, Jamie Iles + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + */ + +#ifndef __ARM_PERF_EVENT_H__ +#define __ARM_PERF_EVENT_H__ + +/* + * NOP: on *most* (read: all supported) ARM platforms, the performance + * counter interrupts are regular interrupts and not an NMI. This + * means that when we receive the interrupt we can call + * perf_event_do_pending() that handles all of the work with + * interrupts enabled. + */ +static inline void +set_perf_event_pending(void) +{ +} + +/* Get the PC. Make sure that we have a 64bit value with the upper 32 cleared. + */ +#define perf_instruction_pointer(_regs) \ + ((u64)instruction_pointer(regs) & 0xFFFFFFFFLU) +#define perf_misc_flags(regs) (user_mode(regs) ? PERF_RECORD_MISC_USER : \ + PERF_RECORD_MISC_KERNEL) + +/* ARM performance counters start from 1 (in the cp15 accesses) so use the + * same indexes here for consistency. */ +#define PERF_EVENT_INDEX_OFFSET 1 + +#endif /* __ARM_PERF_EVENT_H__ */ diff --git a/arch/arm/mm/fault.c b/arch/arm/mm/fault.c index 10e0680..9d40c34 100644 --- a/arch/arm/mm/fault.c +++ b/arch/arm/mm/fault.c @@ -18,6 +18,7 @@ #include <linux/page-flags.h> #include <linux/sched.h> #include <linux/highmem.h> +#include <linux/perf_event.h> #include <asm/system.h> #include <asm/pgtable.h> @@ -302,6 +303,12 @@ do_page_fault(unsigned long addr, unsigned int fsr, struct pt_regs *regs) fault = __do_page_fault(mm, addr, fsr, tsk); up_read(&mm->mmap_sem); + perf_sw_event(PERF_COUNT_SW_PAGE_FAULTS, 1, 0, regs, addr); + if (fault & VM_FAULT_MAJOR) + perf_sw_event(PERF_COUNT_SW_PAGE_FAULTS_MAJ, 1, 0, regs, addr); + else if (fault & VM_FAULT_MINOR) + perf_sw_event(PERF_COUNT_SW_PAGE_FAULTS_MIN, 1, 0, regs, addr); + /* * Handle the "normal" case first - VM_FAULT_MAJOR / VM_FAULT_MINOR */ -- 1.6.5.4 ^ permalink raw reply related [flat|nested] 28+ messages in thread
* [PATCH 5/5] arm/perfevents: implement perf event support for ARMv6 2009-12-14 14:04 ` [PATCH 4/5] arm: enable support for software perf events Jamie Iles @ 2009-12-14 14:04 ` Jamie Iles 2009-12-14 16:12 ` Jean Pihet 2009-12-14 16:13 ` Will Deacon 0 siblings, 2 replies; 28+ messages in thread From: Jamie Iles @ 2009-12-14 14:04 UTC (permalink / raw) To: linux-arm-kernel This patch implements support for ARMv6 performance counters in the Linux performance events subsystem. ARMv6 architectures that have the performance counters should enable HW_PERF_EVENTS and define the interrupts for the counters in arch/arm/kernel/perf_event.c Signed-off-by: Jamie Iles <jamie.iles@picochip.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Ingo Molnar <mingo@elte.hu> --- arch/arm/kernel/Makefile | 1 + arch/arm/kernel/perf_event.c | 1034 ++++++++++++++++++++++++++++++++++++++++++ arch/arm/mm/Kconfig | 8 + 3 files changed, 1043 insertions(+), 0 deletions(-) create mode 100644 arch/arm/kernel/perf_event.c diff --git a/arch/arm/kernel/Makefile b/arch/arm/kernel/Makefile index 286a276..44ebf36 100644 --- a/arch/arm/kernel/Makefile +++ b/arch/arm/kernel/Makefile @@ -47,6 +47,7 @@ obj-$(CONFIG_CPU_XSC3) += xscale-cp0.o obj-$(CONFIG_CPU_MOHAWK) += xscale-cp0.o obj-$(CONFIG_IWMMXT) += iwmmxt.o obj-$(CONFIG_CPU_HAS_PMU) += pmu.o +obj-$(CONFIG_HW_PERF_EVENTS) += perf_event.o AFLAGS_iwmmxt.o := -Wa,-mcpu=iwmmxt ifneq ($(CONFIG_ARCH_EBSA110),y) diff --git a/arch/arm/kernel/perf_event.c b/arch/arm/kernel/perf_event.c new file mode 100644 index 0000000..358fd31 --- /dev/null +++ b/arch/arm/kernel/perf_event.c @@ -0,0 +1,1034 @@ +#undef DEBUG + +/* + * ARMv6 performance counter support. + * + * Copyright (C) 2009 picoChip Designs, Ltd., Jamie Iles + * + * This code is based on the sparc64 perf event code, which is in turn based + * on the x86 code. Callchain code is based on the ARM OProfile backtrace + * code. + */ +#define pr_fmt(fmt) "armv6_perfctr: " fmt + +#include <linux/kernel.h> +#include <linux/perf_event.h> +#include <linux/interrupt.h> +#include <linux/spinlock.h> +#include <linux/uaccess.h> + +#include <asm/irq_regs.h> +#include <asm/stacktrace.h> +#include <asm/irq.h> +#include <asm/pmu.h> + +/* + * ARMv6 has 2 configurable performance counters and a single cycle counter. + * They all share a single reset bit but can be written to zero so we can use + * that for a reset. + * + * The counters can't be individually enabled or disabled so when we remove + * one event and replace it with another we could get spurious counts from the + * wrong event. However, we can take advantage of the fact that the + * performance counters can export events to the event bus, and the event bus + * itself can be monitored. This requires that we *don't* export the events to + * the event bus. The procedure for disabling a configurable counter is: + * - change the counter to count the ETMEXTOUT[0] signal (0x20). This + * effectively stops the counter from counting. + * - disable the counter's interrupt generation (each counter has it's + * own interrupt enable bit). + * Once stopped, the counter value can be written as 0 to reset. + * + * To enable a counter: + * - enable the counter's interrupt generation. + * - set the new event type. + * + * Note: the dedicated cycle counter only counts cycles and can't be + * enabled/disabled independently of the others. When we want to disable the + * cycle counter, we have to just disable the interrupt reporting and start + * ignoring that counter. When re-enabling, we have to reset the value and + * enable the interrupt. + */ + +static const struct pmu_irqs *pmu_irqs; + +/* + * Hardware lock to serialize accesses to PMU registers. Needed for the + * read/modify/write sequences. + */ +DEFINE_SPINLOCK(pmu_lock); + +enum arm_perf_types { + ARM_PERFCTR_ICACHE_MISS = 0x0, + ARM_PERFCTR_IBUF_STALL = 0x1, + ARM_PERFCTR_DDEP_STALL = 0x2, + ARM_PERFCTR_ITLB_MISS = 0x3, + ARM_PERFCTR_DTLB_MISS = 0x4, + ARM_PERFCTR_BR_EXEC = 0x5, + ARM_PERFCTR_BR_MISPREDICT = 0x6, + ARM_PERFCTR_INSTR_EXEC = 0x7, + ARM_PERFCTR_DCACHE_HIT = 0x9, + ARM_PERFCTR_DCACHE_ACCESS = 0xA, + ARM_PERFCTR_DCACHE_MISS = 0xB, + ARM_PERFCTR_DCACHE_WBACK = 0xC, + ARM_PERFCTR_SW_PC_CHANGE = 0xD, + ARM_PERFCTR_MAIN_TLB_MISS = 0xF, + ARM_PERFCTR_EXPL_D_ACCESS = 0x10, + ARM_PERFCTR_LSU_FULL_STALL = 0x11, + ARM_PERFCTR_WBUF_DRAINED = 0x12, + ARM_PERFCTR_CPU_CYCLES = 0xFF, + ARM_PERFCTR_NOP = 0x20, +}; + +/* We support using the full 32 bits of each counter. */ +#define MAX_PERIOD ((1LLU << 32) - 1) + +enum armv6_counters { + ARMV6_CYCLE_COUNTER = 1, + ARMV6_COUNTER0, + ARMV6_COUNTER1, +}; + +/* We support 3 simultaneous events, but they begin from 1. */ +#define ARMV6_MAX_HWEVENTS 4 + +/* The events for a given CPU. */ +struct cpu_hw_events { + /* + * The events that are active on the CPU for the given index. Index 0 + * is reserved. + */ + struct perf_event *events[ARMV6_MAX_HWEVENTS]; + + /* + * A 1 bit for an index indicates that the counter is being used for + * an event. A 0 means that the counter can be used. + */ + unsigned long used_mask[BITS_TO_LONGS(ARMV6_MAX_HWEVENTS)]; + + /* + * A 1 bit for an index indicates that the counter is actively being + * used. + */ + unsigned long active_mask[BITS_TO_LONGS(ARMV6_MAX_HWEVENTS)]; +}; +DEFINE_PER_CPU(struct cpu_hw_events, cpu_hw_events); + +#define HW_OP_UNSUPPORTED 0xFFFF + +/* + * The hardware events that we support. We do support cache operations but + * we have harvard caches and no way to combine instruction and data + * accesses/misses in hardware. + */ +static const unsigned v6_perf_map[PERF_COUNT_HW_MAX] = { + [PERF_COUNT_HW_CPU_CYCLES] = ARM_PERFCTR_CPU_CYCLES, + [PERF_COUNT_HW_INSTRUCTIONS] = ARM_PERFCTR_INSTR_EXEC, + [PERF_COUNT_HW_CACHE_REFERENCES] = HW_OP_UNSUPPORTED, + [PERF_COUNT_HW_CACHE_MISSES] = HW_OP_UNSUPPORTED, + [PERF_COUNT_HW_BRANCH_INSTRUCTIONS] = ARM_PERFCTR_BR_EXEC, + [PERF_COUNT_HW_BRANCH_MISSES] = ARM_PERFCTR_BR_MISPREDICT, + [PERF_COUNT_HW_BUS_CYCLES] = HW_OP_UNSUPPORTED, +}; + +static inline int +armv6_map_hw_event(u64 config) +{ + int mapping = v6_perf_map[config]; + if (HW_OP_UNSUPPORTED == mapping) + mapping = -EOPNOTSUPP; + return mapping; +} + +#define C(_x) \ + PERF_COUNT_HW_CACHE_##_x + +#define CACHE_OP_UNSUPPORTED 0xFFFF + +static const unsigned v6_perf_cache_map[PERF_COUNT_HW_CACHE_MAX] + [PERF_COUNT_HW_CACHE_OP_MAX] + [PERF_COUNT_HW_CACHE_RESULT_MAX] = { + [C(L1D)] = { + /* + * The performance counters don't differentiate between read + * and write accesses/misses so this isn't strictly correct, + * but it's the best we can do. Writes and reads get + * combined. + */ + [C(OP_READ)] = { + [C(RESULT_ACCESS)] = ARM_PERFCTR_DCACHE_ACCESS, + [C(RESULT_MISS)] = ARM_PERFCTR_DCACHE_MISS, + }, + [C(OP_WRITE)] = { + [C(RESULT_ACCESS)] = ARM_PERFCTR_DCACHE_ACCESS, + [C(RESULT_MISS)] = ARM_PERFCTR_DCACHE_MISS, + }, + [C(OP_PREFETCH)] = { + [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED, + [C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED, + }, + }, + [C(L1I)] = { + [C(OP_READ)] = { + [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED, + [C(RESULT_MISS)] = ARM_PERFCTR_ICACHE_MISS, + }, + [C(OP_WRITE)] = { + [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED, + [C(RESULT_MISS)] = ARM_PERFCTR_ICACHE_MISS, + }, + [C(OP_PREFETCH)] = { + [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED, + [C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED, + }, + }, + [C(LL)] = { + [C(OP_READ)] = { + [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED, + [C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED, + }, + [C(OP_WRITE)] = { + [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED, + [C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED, + }, + [C(OP_PREFETCH)] = { + [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED, + [C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED, + }, + }, + [C(DTLB)] = { + /* + * The ARM performance counters can count micro DTLB misses, + * micro ITLB misses and main TLB misses. There isn't an event + * for TLB misses, so use the micro misses here and if users + * want the main TLB misses they can use a raw counter. + */ + [C(OP_READ)] = { + [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED, + [C(RESULT_MISS)] = ARM_PERFCTR_DTLB_MISS, + }, + [C(OP_WRITE)] = { + [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED, + [C(RESULT_MISS)] = ARM_PERFCTR_DTLB_MISS, + }, + [C(OP_PREFETCH)] = { + [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED, + [C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED, + }, + }, + [C(ITLB)] = { + [C(OP_READ)] = { + [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED, + [C(RESULT_MISS)] = ARM_PERFCTR_ITLB_MISS, + }, + [C(OP_WRITE)] = { + [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED, + [C(RESULT_MISS)] = ARM_PERFCTR_ITLB_MISS, + }, + [C(OP_PREFETCH)] = { + [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED, + [C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED, + }, + }, + [C(BPU)] = { + [C(OP_READ)] = { + [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED, + [C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED, + }, + [C(OP_WRITE)] = { + [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED, + [C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED, + }, + [C(OP_PREFETCH)] = { + [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED, + [C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED, + }, + }, +}; + +static const int +armv6_map_cache_event(u64 config) +{ + unsigned int cache_type, cache_op, cache_result, ret; + + cache_type = (config >> 0) & 0xff; + if (cache_type >= PERF_COUNT_HW_CACHE_MAX) + return -EINVAL; + + cache_op = (config >> 8) & 0xff; + if (cache_op >= PERF_COUNT_HW_CACHE_OP_MAX) + return -EINVAL; + + cache_result = (config >> 16) & 0xff; + if (cache_result >= PERF_COUNT_HW_CACHE_RESULT_MAX) + return -EINVAL; + + ret = (int)v6_perf_cache_map[cache_type][cache_op][cache_result]; + + if (ret == CACHE_OP_UNSUPPORTED) + return -ENOENT; + + return ret; +} + +#define PMCR_ENABLE (1 << 0) +#define PMCR_CTR01_RESET (1 << 1) +#define PMCR_CCOUNT_RESET (1 << 2) +#define PMCR_CCOUNT_DIV (1 << 3) +#define PMCR_COUNT0_IEN (1 << 4) +#define PMCR_COUNT1_IEN (1 << 5) +#define PMCR_CCOUNT_IEN (1 << 6) +#define PMCR_COUNT0_OVERFLOW (1 << 8) +#define PMCR_COUNT1_OVERFLOW (1 << 9) +#define PMCR_CCOUNT_OVERFLOW (1 << 10) +#define PMCR_EVT_COUNT0_SHIFT 20 +#define PMCR_EVT_COUNT0_MASK (0xFF << PMCR_EVT_COUNT0_SHIFT) +#define PMCR_EVT_COUNT1_SHIFT 12 +#define PMCR_EVT_COUNT1_MASK (0xFF << PMCR_EVT_COUNT1_SHIFT) + +static inline unsigned long +pmcr_read(void) +{ + u32 val; + asm volatile("mrc p15, 0, %0, c15, c12, 0" : "=r"(val)); + return val; +} + +static inline void +pmcr_write(unsigned long val) +{ + asm volatile("mcr p15, 0, %0, c15, c12, 0" : : "r"(val)); +} + +#define PMCR_OVERFLOWED_MASK \ + (PMCR_COUNT0_OVERFLOW | PMCR_COUNT1_OVERFLOW | PMCR_CCOUNT_OVERFLOW) + +static inline int +pmcr_has_overflowed(unsigned long pmcr) +{ + return (pmcr & PMCR_OVERFLOWED_MASK); +} + +static inline int +pmcr_counter_has_overflowed(unsigned long pmcr, + enum armv6_counters counter) +{ + int ret; + + if (ARMV6_CYCLE_COUNTER == counter) + ret = pmcr & PMCR_CCOUNT_OVERFLOW; + else if (ARMV6_COUNTER0 == counter) + ret = pmcr & PMCR_COUNT0_OVERFLOW; + else if (ARMV6_COUNTER1 == counter) + ret = pmcr & PMCR_COUNT1_OVERFLOW; + else + BUG(); + + return ret; +} + +static inline unsigned long +armv6pmu_read_counter(enum armv6_counters counter) +{ + unsigned long value; + + if (ARMV6_CYCLE_COUNTER == counter) + asm volatile("mrc p15, 0, %0, c15, c12, 1" : "=r"(value)); + else if (ARMV6_COUNTER0 == counter) + asm volatile("mrc p15, 0, %0, c15, c12, 2" : "=r"(value)); + else if (ARMV6_COUNTER1 == counter) + asm volatile("mrc p15, 0, %0, c15, c12, 3" : "=r"(value)); + else + BUG(); + + return value; +} + +static inline void +armv6pmu_write_counter(enum armv6_counters counter, + unsigned long value) +{ + if (ARMV6_CYCLE_COUNTER == counter) + asm volatile("mcr p15, 0, %0, c15, c12, 1" : : "r"(value)); + else if (ARMV6_COUNTER0 == counter) + asm volatile("mcr p15, 0, %0, c15, c12, 2" : : "r"(value)); + else if (ARMV6_COUNTER1 == counter) + asm volatile("mcr p15, 0, %0, c15, c12, 3" : : "r"(value)); + else + BUG(); +} + +static int +armv6pmu_place_event(struct cpu_hw_events *cpuc, + struct hw_perf_event *event) +{ + /* Always place a cycle counter into the cycle counter. */ + if (ARM_PERFCTR_CPU_CYCLES == event->config_base) { + if (test_and_set_bit(ARMV6_CYCLE_COUNTER, cpuc->used_mask)) + return -EAGAIN; + + return ARMV6_CYCLE_COUNTER; + } else { + /* + * For anything other than a cycle counter, try and use + * counter0 and counter1. + */ + if (!test_and_set_bit(ARMV6_COUNTER1, cpuc->used_mask)) { + return ARMV6_COUNTER1; + } + + if (!test_and_set_bit(ARMV6_COUNTER0, cpuc->used_mask)) { + return ARMV6_COUNTER0; + } + + /* The counters are all in use. */ + return -EAGAIN; + } +} + +static void +armv6pmu_disable_event(struct cpu_hw_events *cpuc, + struct hw_perf_event *hwc, + int idx) +{ + unsigned long val, mask, evt, flags; + + if (ARMV6_CYCLE_COUNTER == idx) { + mask = PMCR_CCOUNT_IEN; + evt = 0; + } else if (ARMV6_COUNTER0 == idx) { + mask = PMCR_COUNT0_IEN | PMCR_EVT_COUNT0_MASK; + evt = ARM_PERFCTR_NOP << PMCR_EVT_COUNT0_SHIFT; + } else if (ARMV6_COUNTER1 == idx) { + mask = PMCR_COUNT1_IEN | PMCR_EVT_COUNT1_MASK; + evt = ARM_PERFCTR_NOP << PMCR_EVT_COUNT1_SHIFT; + } else { + BUG(); + } + + /* + * Mask out the current event and set the counter to count the number + * of ETM bus signal assertion cycles. The external reporting should + * be disabled and so this should never increment. + */ + spin_lock_irqsave(&pmu_lock, flags); + val = pmcr_read(); + val &= ~mask; + val |= evt; + pmcr_write(val); + spin_unlock_irqrestore(&pmu_lock, flags); +} + +static void +armv6pmu_enable_event(struct cpu_hw_events *cpuc, + struct hw_perf_event *hwc, + int idx) +{ + unsigned long val, mask, evt, flags; + + if (ARMV6_CYCLE_COUNTER == idx) { + mask = 0; + evt = PMCR_CCOUNT_IEN; + } else if (ARMV6_COUNTER0 == idx) { + mask = PMCR_EVT_COUNT0_MASK; + evt = (hwc->config_base << PMCR_EVT_COUNT0_SHIFT) | + PMCR_COUNT0_IEN; + } else if (ARMV6_COUNTER1 == idx) { + mask = PMCR_EVT_COUNT1_MASK; + evt = (hwc->config_base << PMCR_EVT_COUNT1_SHIFT) | + PMCR_COUNT1_IEN; + } else { + BUG(); + } + + /* + * Mask out the current event and set the counter to count the event + * that we're interested in. + */ + spin_lock_irqsave(&pmu_lock, flags); + val = pmcr_read(); + val &= ~mask; + val |= evt; + pmcr_write(val); + spin_unlock_irqrestore(&pmu_lock, flags); +} + +static int +armv6pmu_event_set_period(struct perf_event *event, + struct hw_perf_event *hwc, + int idx) +{ + s64 left = atomic64_read(&hwc->period_left); + s64 period = hwc->sample_period; + int ret = 0; + + if (unlikely(left <= -period)) { + left = period; + atomic64_set(&hwc->period_left, left); + hwc->last_period = period; + ret = 1; + } + + if (unlikely(left <= 0)) { + left += period; + atomic64_set(&hwc->period_left, left); + hwc->last_period = period; + ret = 1; + } + + if (left > MAX_PERIOD) + left = MAX_PERIOD; + + atomic64_set(&hwc->prev_count, (u64)-left); + + armv6pmu_write_counter(idx, (u64)(-left) & 0xffffffff); + + perf_event_update_userpage(event); + + return ret; +} + +static int +armv6pmu_enable(struct perf_event *event) +{ + struct cpu_hw_events *cpuc = &__get_cpu_var(cpu_hw_events); + struct hw_perf_event *hwc = &event->hw; + int idx; + int err = 0; + + /* If we don't have a space for the counter then finish early. */ + idx = armv6pmu_place_event(cpuc, hwc); + if (idx < 0) { + err = idx; + goto out; + } + + /* + * If there is an event in the counter we are going to use then make + * sure it is disabled. + */ + event->hw.idx = idx; + armv6pmu_disable_event(cpuc, hwc, idx); + cpuc->events[idx] = event; + set_bit(idx, cpuc->active_mask); + + /* Set the period for the event. */ + armv6pmu_event_set_period(event, hwc, idx); + + /* Enable the event. */ + armv6pmu_enable_event(cpuc, hwc, idx); + + /* Propagate our changes to the userspace mapping. */ + perf_event_update_userpage(event); + +out: + return err; +} + +static u64 +armv6pmu_event_update(struct perf_event *event, + struct hw_perf_event *hwc, + int idx) +{ + int shift = 64 - 32; + u64 prev_raw_count, new_raw_count; + s64 delta; + +again: + prev_raw_count = atomic64_read(&hwc->prev_count); + new_raw_count = armv6pmu_read_counter(idx); + + if (atomic64_cmpxchg(&hwc->prev_count, prev_raw_count, + new_raw_count) != prev_raw_count) + goto again; + + delta = (new_raw_count << shift) - (prev_raw_count << shift); + delta >>= shift; + + atomic64_add(delta, &event->count); + atomic64_sub(delta, &hwc->period_left); + + return new_raw_count; +} + +static void +armv6pmu_disable(struct perf_event *event) +{ + struct cpu_hw_events *cpuc = &__get_cpu_var(cpu_hw_events); + struct hw_perf_event *hwc = &event->hw; + int idx = hwc->idx; + + WARN_ON(idx < 0); + + clear_bit(idx, cpuc->active_mask); + armv6pmu_disable_event(cpuc, hwc, idx); + + barrier(); + + armv6pmu_event_update(event, hwc, idx); + cpuc->events[idx] = NULL; + clear_bit(idx, cpuc->used_mask); + + perf_event_update_userpage(event); +} + +static void +armv6pmu_read(struct perf_event *event) +{ + struct hw_perf_event *hwc = &event->hw; + + /* Don't read disabled counters! */ + if (hwc->idx < 0) + return; + + armv6pmu_event_update(event, hwc, hwc->idx); +} + +static void +armv6pmu_unthrottle(struct perf_event *event) +{ + struct cpu_hw_events *cpuc = &__get_cpu_var(cpu_hw_events); + struct hw_perf_event *hwc = &event->hw; + + armv6pmu_enable_event(cpuc, hwc, hwc->idx); +} + +static irqreturn_t +armv6_perfcounters_irq(int irq_num, + void *dev) +{ + unsigned long pmcr = pmcr_read(); + struct perf_sample_data data; + struct cpu_hw_events *cpuc; + struct pt_regs *regs; + int idx; + + if (!pmcr_has_overflowed(pmcr)) + return IRQ_NONE; + + regs = get_irq_regs(); + + /* + * The interrupts are cleared by writing the overflow flags back to + * the control register. All of the other bits don't have any effect + * if they are rewritten, so write the whole value back. + */ + pmcr_write(pmcr); + + data.addr = 0; + + cpuc = &__get_cpu_var(cpu_hw_events); + for (idx = 0; idx < ARMV6_MAX_HWEVENTS; ++idx) { + struct perf_event *event = cpuc->events[idx]; + struct hw_perf_event *hwc; + + if (!test_bit(idx, cpuc->active_mask)) + continue; + + /* + * We have a single interrupt for all counters. Check that + * each counter has overflowed before we process it. + */ + if (!pmcr_counter_has_overflowed(pmcr, idx)) + continue; + + hwc = &event->hw; + armv6pmu_event_update(event, hwc, idx); + data.period = event->hw.last_period; + if (!armv6pmu_event_set_period(event, hwc, idx)) + continue; + + if (perf_event_overflow(event, 0, &data, regs)) + armv6pmu_disable_event(cpuc, hwc, idx); + } + + /* + * Handle the pending perf events. + * + * Note: this call *must* be run with interrupts enabled. For + * platforms that can have the PMU interrupts raised as a PMI, this + * will not work. + */ + perf_event_do_pending(); + + return IRQ_HANDLED; +} + +void +hw_perf_enable(void) +{ + /* Enable all of the perf events on hardware. */ + int idx; + struct cpu_hw_events *cpuc = &__get_cpu_var(cpu_hw_events); + unsigned long flags, val; + + for (idx = 0; idx < ARMV6_MAX_HWEVENTS; ++idx) { + struct perf_event *event = cpuc->events[idx]; + + if (!event) + continue; + + armv6pmu_enable_event(cpuc, &event->hw, idx); + } + + spin_lock_irqsave(&pmu_lock, flags); + val = pmcr_read(); + val |= PMCR_ENABLE; + pmcr_write(val); + spin_unlock_irqrestore(&pmu_lock, flags); +} + +void +hw_perf_disable(void) +{ + unsigned long flags, val; + + spin_lock_irqsave(&pmu_lock, flags); + val = pmcr_read(); + val &= ~PMCR_ENABLE; + pmcr_write(val); + spin_unlock_irqrestore(&pmu_lock, flags); +} + +static const struct pmu armv6pmu = { + .enable = armv6pmu_enable, + .disable = armv6pmu_disable, + .read = armv6pmu_read, + .unthrottle = armv6pmu_unthrottle, +}; + +static int +validate_event(struct cpu_hw_events *cpuc, + struct perf_event *event) +{ + struct hw_perf_event fake_event = event->hw; + + if (event->pmu && event->pmu != &armv6pmu) + return 0; + + return armv6pmu_place_event(cpuc, &fake_event) >= 0; +} + +static int +validate_group(struct perf_event *event) +{ + struct perf_event *sibling, *leader = event->group_leader; + struct cpu_hw_events fake_pmu; + + memset(&fake_pmu, 0, sizeof(fake_pmu)); + + if (!validate_event(&fake_pmu, leader)) + return -ENOSPC; + + list_for_each_entry(sibling, &leader->sibling_list, group_entry) { + if (!validate_event(&fake_pmu, sibling)) + return -ENOSPC; + } + + if (!validate_event(&fake_pmu, event)) + return -ENOSPC; + + return 0; +} + +static int +armv6_hw_perf_event_init(struct perf_event *event) +{ + struct hw_perf_event *hwc = &event->hw; + int mapping, err; + + /* Decode the generic type into an ARM event identifier. */ + if (PERF_TYPE_HARDWARE == event->attr.type) { + mapping = armv6_map_hw_event(event->attr.config); + } else if (PERF_TYPE_HW_CACHE == event->attr.type) { + mapping = armv6_map_cache_event(event->attr.config); + } else if (PERF_TYPE_RAW == event->attr.type) { + /* The EvtCountN field of the PMCR is 8 bits. */ + mapping = event->attr.config & 0xFF; + } else { + pr_debug("event type %x not supported\n", event->attr.type); + return -EOPNOTSUPP; + } + + if (mapping < 0) { + pr_debug("event %x:%llx not supported\n", event->attr.type, + event->attr.config); + return mapping; + } + + /* + * Check whether we need to exclude the counter from certain modes. + * The ARM performance counters are on all of the time so if someone + * has asked us for some excludes then we have to fail. + */ + if (event->attr.exclude_kernel || event->attr.exclude_user || + event->attr.exclude_hv || event->attr.exclude_idle) { + pr_debug("ARM performance counters do not support " + "mode exclusion\n"); + return -EPERM; + } + + /* + * We don't assign an index until we actually place the event onto + * hardware. Use -1 to signify that we haven't decided where to put it + * yet. For SMP systems, each core has it's own PMU so we can't do any + * clever allocation or constraints checking at this point. + */ + hwc->idx = -1; + + /* + * Store the event encoding into the config_base field. config and + * event_base are unused as the only 2 things we need to know are + * the event mapping and the counter to use. The counter to use is + * also the indx and the config_base is the event type. + */ + hwc->config_base = (unsigned long)mapping; + hwc->config = 0; + hwc->event_base = 0; + + if (!hwc->sample_period) { + hwc->sample_period = MAX_PERIOD; + hwc->last_period = hwc->sample_period; + atomic64_set(&hwc->period_left, hwc->sample_period); + } + + err = 0; + if (event->group_leader != event) { + err = validate_group(event); + if (err) + return -EINVAL; + } + + return err; +} + +static int +armv6pmu_reserve_hardware(void) +{ + int i; + int err; + + pmu_irqs = reserve_pmu(); + if (IS_ERR(pmu_irqs)) { + pr_warning("unable to reserve pmu\n"); + return PTR_ERR(pmu_irqs); + } + + init_pmu(); + + if (pmu_irqs->num_irqs < 1) { + pr_err("no irqs for PMUs defined\n"); + } + + for (i = 0; i < pmu_irqs->num_irqs; ++i) { + err = request_irq(pmu_irqs->irqs[i], armv6_perfcounters_irq, + IRQF_DISABLED, "armv6_perfctr", NULL); + if (err) { + pr_warning("unable to request IRQ%d for ARMv6 " + "perf counters\n", pmu_irqs->irqs[i]); + break; + } + } + + if (err) { + for (i = i - 1; i >= 0; --i) + free_irq(pmu_irqs->irqs[i], NULL); + release_pmu(pmu_irqs); + pmu_irqs = NULL; + } + + return err; +} + +static void +armv6pmu_release_hardware(void) +{ + int i; + + for (i = pmu_irqs->num_irqs - 1; i >= 0; --i) { + free_irq(pmu_irqs->irqs[i], NULL); + } + pmcr_write(0); + + release_pmu(pmu_irqs); + pmu_irqs = NULL; +} + +static atomic_t active_events = ATOMIC_INIT(0); +static DEFINE_MUTEX(pmu_reserve_mutex); + +static void +hw_perf_event_destroy(struct perf_event *event) +{ + if (atomic_dec_and_mutex_lock(&active_events, &pmu_reserve_mutex)) { + armv6pmu_release_hardware(); + mutex_unlock(&pmu_reserve_mutex); + } +} + +const struct pmu * +hw_perf_event_init(struct perf_event *event) +{ + int err = 0; + + /* We support 3 events: one cycle counter and 2 programmable. */ + perf_max_events = 3; + event->destroy = hw_perf_event_destroy; + + if (!atomic_inc_not_zero(&active_events)) { + if (atomic_read(&active_events) > perf_max_events) { + atomic_dec(&active_events); + return ERR_PTR(-ENOSPC); + } + + mutex_lock(&pmu_reserve_mutex); + if (atomic_read(&active_events) == 0) { + err = armv6pmu_reserve_hardware(); + } + + if (!err) + atomic_inc(&active_events); + mutex_unlock(&pmu_reserve_mutex); + } + + if (err) + return ERR_PTR(err); + + err = armv6_hw_perf_event_init(event); + if (err) + hw_perf_event_destroy(event); + + return err ? ERR_PTR(err) : &armv6pmu; +} + +/* Callchain handling code. */ +static inline void +callchain_store(struct perf_callchain_entry *entry, + u64 ip) +{ + if (entry->nr < PERF_MAX_STACK_DEPTH) + entry->ip[entry->nr++] = ip; +} + +/* + * The registers we're interested in are at the end of the variable + * length saved register structure. The fp points at the end of this + * structure so the address of this struct is: + * (struct frame_tail *)(xxx->fp)-1 + * + * This code has been adapted from the ARM OProfile support. + */ +struct frame_tail { + struct frame_tail *fp; + unsigned long sp; + unsigned long lr; +} __attribute__((packed)); + +/* + * Get the return address for a single stackframe and return a pointer to the + * next frame tail. + */ +static struct frame_tail * +user_backtrace(struct frame_tail *tail, + struct perf_callchain_entry *entry) +{ + struct frame_tail buftail; + + /* Also check accessibility of one struct frame_tail beyond */ + if (!access_ok(VERIFY_READ, tail, sizeof(buftail))) + return NULL; + if (__copy_from_user_inatomic(&buftail, tail, sizeof(buftail))) + return NULL; + + callchain_store(entry, buftail.lr); + + /* + * Frame pointers should strictly progress back up the stack + * (towards higher addresses). + */ + if (tail >= buftail.fp) + return NULL; + + return buftail.fp - 1; +} + +static void +perf_callchain_user(struct pt_regs *regs, + struct perf_callchain_entry *entry) +{ + struct frame_tail *tail; + + callchain_store(entry, PERF_CONTEXT_USER); + + if (!user_mode(regs)) + regs = task_pt_regs(current); + + tail = (struct frame_tail *)regs->ARM_fp - 1; + + while (tail && !((unsigned long)tail & 0x3)) + tail = user_backtrace(tail, entry); +} + +/* + * Gets called by walk_stackframe() for every stackframe. This will be called + * whist unwinding the stackframe and is like a subroutine return so we use + * the PC. + */ +static int +callchain_trace(struct stackframe *fr, + void *data) +{ + struct perf_callchain_entry *entry = data; + callchain_store(entry, fr->pc); + return 0; +} + +static void +perf_callchain_kernel(struct pt_regs *regs, + struct perf_callchain_entry *entry) +{ + struct stackframe fr; + + callchain_store(entry, PERF_CONTEXT_KERNEL); + fr.fp = regs->ARM_fp; + fr.sp = regs->ARM_sp; + fr.lr = regs->ARM_lr; + fr.pc = regs->ARM_pc; + walk_stackframe(&fr, callchain_trace, entry); +} + +static void +perf_do_callchain(struct pt_regs *regs, + struct perf_callchain_entry *entry) +{ + int is_user; + + if (!regs) + return; + + is_user = user_mode(regs); + + if (!current || !current->pid) + return; + + if (is_user && current->state != TASK_RUNNING) + return; + + if (!is_user) + perf_callchain_kernel(regs, entry); + + if (current->mm) + perf_callchain_user(regs, entry); +} + +static DEFINE_PER_CPU(struct perf_callchain_entry, pmc_irq_entry); + +struct perf_callchain_entry * +perf_callchain(struct pt_regs *regs) +{ + struct perf_callchain_entry *entry = &__get_cpu_var(pmc_irq_entry); + + entry->nr = 0; + perf_do_callchain(regs, entry); + return entry; +} diff --git a/arch/arm/mm/Kconfig b/arch/arm/mm/Kconfig index fc5c05b..89a26ea 100644 --- a/arch/arm/mm/Kconfig +++ b/arch/arm/mm/Kconfig @@ -791,3 +791,11 @@ config ARM_L1_CACHE_SHIFT int default 6 if ARCH_OMAP3 || ARCH_S5PC1XX default 5 + +config HW_PERF_EVENTS + bool "Enable hardware performance counter support for perf events" + depends on PERF_EVENTS && CPU_HAS_PMU && CPU_V6 + default n + help + Enable hardware performance counter support for perf events. If + disabled, perf events will use software events only. -- 1.6.5.4 ^ permalink raw reply related [flat|nested] 28+ messages in thread
* [PATCH 5/5] arm/perfevents: implement perf event support for ARMv6 2009-12-14 14:04 ` [PATCH 5/5] arm/perfevents: implement perf event support for ARMv6 Jamie Iles @ 2009-12-14 16:12 ` Jean Pihet 2009-12-14 16:33 ` Jamie Iles 2009-12-14 17:09 ` Will Deacon 2009-12-14 16:13 ` Will Deacon 1 sibling, 2 replies; 28+ messages in thread From: Jean Pihet @ 2009-12-14 16:12 UTC (permalink / raw) To: linux-arm-kernel Hi, On Mon, 2009-12-14 at 14:04 +0000, Jamie Iles wrote: > This patch implements support for ARMv6 performance counters in the > Linux performance events subsystem. ARMv6 architectures that have the > performance counters should enable HW_PERF_EVENTS and define the > interrupts for the counters in arch/arm/kernel/perf_event.c > > Signed-off-by: Jamie Iles <jamie.iles@picochip.com> > Cc: Peter Zijlstra <peterz@infradead.org> > Cc: Ingo Molnar <mingo@elte.hu> > --- > arch/arm/kernel/Makefile | 1 + > arch/arm/kernel/perf_event.c | 1034 ++++++++++++++++++++++++++++++++++++++++++ > arch/arm/mm/Kconfig | 8 + > 3 files changed, 1043 insertions(+), 0 deletions(-) > create mode 100644 arch/arm/kernel/perf_event.c I have a question about other ARM implementations. Since all the code is in perf_event.c is it modular enough to allow the addition of other ARM implementations? The remarks are inlined below. I am interested in supporting perf events for ARMv7. What do you think? > > diff --git a/arch/arm/kernel/Makefile b/arch/arm/kernel/Makefile > index 286a276..44ebf36 100644 > --- a/arch/arm/kernel/Makefile > +++ b/arch/arm/kernel/Makefile > @@ -47,6 +47,7 @@ obj-$(CONFIG_CPU_XSC3) += xscale-cp0.o > obj-$(CONFIG_CPU_MOHAWK) += xscale-cp0.o > obj-$(CONFIG_IWMMXT) += iwmmxt.o > obj-$(CONFIG_CPU_HAS_PMU) += pmu.o > +obj-$(CONFIG_HW_PERF_EVENTS) += perf_event.o > AFLAGS_iwmmxt.o := -Wa,-mcpu=iwmmxt > > ifneq ($(CONFIG_ARCH_EBSA110),y) > diff --git a/arch/arm/kernel/perf_event.c b/arch/arm/kernel/perf_event.c > new file mode 100644 > index 0000000..358fd31 I have a generic question here: will #ifdef be needed to allow other ARM implementations (e.g. ARMv7)? Is it worth to do it that way or to duplicate this file for other ARM implementations? > --- /dev/null > +++ b/arch/arm/kernel/perf_event.c > @@ -0,0 +1,1034 @@ > +#undef DEBUG > + > +/* > + * ARMv6 performance counter support. ARMv6 only? > + * > + * Copyright (C) 2009 picoChip Designs, Ltd., Jamie Iles > + * > + * This code is based on the sparc64 perf event code, which is in turn based > + * on the x86 code. Callchain code is based on the ARM OProfile backtrace > + * code. > + */ > +#define pr_fmt(fmt) "armv6_perfctr: " fmt > + > +#include <linux/kernel.h> > +#include <linux/perf_event.h> > +#include <linux/interrupt.h> > +#include <linux/spinlock.h> > +#include <linux/uaccess.h> > + > +#include <asm/irq_regs.h> > +#include <asm/stacktrace.h> > +#include <asm/irq.h> > +#include <asm/pmu.h> > + > +/* > + * ARMv6 has 2 configurable performance counters and a single cycle counter. > + * They all share a single reset bit but can be written to zero so we can use > + * that for a reset. > + * > + * The counters can't be individually enabled or disabled so when we remove > + * one event and replace it with another we could get spurious counts from the > + * wrong event. However, we can take advantage of the fact that the > + * performance counters can export events to the event bus, and the event bus > + * itself can be monitored. This requires that we *don't* export the events to > + * the event bus. The procedure for disabling a configurable counter is: > + * - change the counter to count the ETMEXTOUT[0] signal (0x20). This > + * effectively stops the counter from counting. > + * - disable the counter's interrupt generation (each counter has it's > + * own interrupt enable bit). > + * Once stopped, the counter value can be written as 0 to reset. > + * > + * To enable a counter: > + * - enable the counter's interrupt generation. > + * - set the new event type. > + * > + * Note: the dedicated cycle counter only counts cycles and can't be > + * enabled/disabled independently of the others. When we want to disable the > + * cycle counter, we have to just disable the interrupt reporting and start > + * ignoring that counter. When re-enabling, we have to reset the value and > + * enable the interrupt. > + */ > + > +static const struct pmu_irqs *pmu_irqs; > + > +/* > + * Hardware lock to serialize accesses to PMU registers. Needed for the > + * read/modify/write sequences. > + */ > +DEFINE_SPINLOCK(pmu_lock); > + > +enum arm_perf_types { > + ARM_PERFCTR_ICACHE_MISS = 0x0, > + ARM_PERFCTR_IBUF_STALL = 0x1, > + ARM_PERFCTR_DDEP_STALL = 0x2, > + ARM_PERFCTR_ITLB_MISS = 0x3, > + ARM_PERFCTR_DTLB_MISS = 0x4, > + ARM_PERFCTR_BR_EXEC = 0x5, > + ARM_PERFCTR_BR_MISPREDICT = 0x6, > + ARM_PERFCTR_INSTR_EXEC = 0x7, > + ARM_PERFCTR_DCACHE_HIT = 0x9, > + ARM_PERFCTR_DCACHE_ACCESS = 0xA, > + ARM_PERFCTR_DCACHE_MISS = 0xB, > + ARM_PERFCTR_DCACHE_WBACK = 0xC, > + ARM_PERFCTR_SW_PC_CHANGE = 0xD, > + ARM_PERFCTR_MAIN_TLB_MISS = 0xF, > + ARM_PERFCTR_EXPL_D_ACCESS = 0x10, > + ARM_PERFCTR_LSU_FULL_STALL = 0x11, > + ARM_PERFCTR_WBUF_DRAINED = 0x12, > + ARM_PERFCTR_CPU_CYCLES = 0xFF, > + ARM_PERFCTR_NOP = 0x20, > +}; This needs an armv6 prefix in the name. > + > +/* We support using the full 32 bits of each counter. */ > +#define MAX_PERIOD ((1LLU << 32) - 1) > + > +enum armv6_counters { > + ARMV6_CYCLE_COUNTER = 1, > + ARMV6_COUNTER0, > + ARMV6_COUNTER1, > +}; > + > +/* We support 3 simultaneous events, but they begin from 1. */ > +#define ARMV6_MAX_HWEVENTS 4 > + > +/* The events for a given CPU. */ > +struct cpu_hw_events { > + /* > + * The events that are active on the CPU for the given index. Index 0 > + * is reserved. > + */ > + struct perf_event *events[ARMV6_MAX_HWEVENTS]; > + > + /* > + * A 1 bit for an index indicates that the counter is being used for > + * an event. A 0 means that the counter can be used. > + */ > + unsigned long used_mask[BITS_TO_LONGS(ARMV6_MAX_HWEVENTS)]; > + > + /* > + * A 1 bit for an index indicates that the counter is actively being > + * used. > + */ > + unsigned long active_mask[BITS_TO_LONGS(ARMV6_MAX_HWEVENTS)]; > +}; Will it need #ifdef for other ARM implementations, or a armv6 prefix? > +DEFINE_PER_CPU(struct cpu_hw_events, cpu_hw_events); > + > +#define HW_OP_UNSUPPORTED 0xFFFF > + > +/* > + * The hardware events that we support. We do support cache operations but > + * we have harvard caches and no way to combine instruction and data > + * accesses/misses in hardware. > + */ > +static const unsigned v6_perf_map[PERF_COUNT_HW_MAX] = { > + [PERF_COUNT_HW_CPU_CYCLES] = ARM_PERFCTR_CPU_CYCLES, > + [PERF_COUNT_HW_INSTRUCTIONS] = ARM_PERFCTR_INSTR_EXEC, > + [PERF_COUNT_HW_CACHE_REFERENCES] = HW_OP_UNSUPPORTED, > + [PERF_COUNT_HW_CACHE_MISSES] = HW_OP_UNSUPPORTED, > + [PERF_COUNT_HW_BRANCH_INSTRUCTIONS] = ARM_PERFCTR_BR_EXEC, > + [PERF_COUNT_HW_BRANCH_MISSES] = ARM_PERFCTR_BR_MISPREDICT, > + [PERF_COUNT_HW_BUS_CYCLES] = HW_OP_UNSUPPORTED, > +}; > + > +static inline int > +armv6_map_hw_event(u64 config) > +{ > + int mapping = v6_perf_map[config]; > + if (HW_OP_UNSUPPORTED == mapping) > + mapping = -EOPNOTSUPP; > + return mapping; > +} > + > +#define C(_x) \ > + PERF_COUNT_HW_CACHE_##_x > + > +#define CACHE_OP_UNSUPPORTED 0xFFFF > + > +static const unsigned v6_perf_cache_map[PERF_COUNT_HW_CACHE_MAX] > + [PERF_COUNT_HW_CACHE_OP_MAX] > + [PERF_COUNT_HW_CACHE_RESULT_MAX] = { > + [C(L1D)] = { > + /* > + * The performance counters don't differentiate between read > + * and write accesses/misses so this isn't strictly correct, > + * but it's the best we can do. Writes and reads get > + * combined. > + */ > + [C(OP_READ)] = { > + [C(RESULT_ACCESS)] = ARM_PERFCTR_DCACHE_ACCESS, > + [C(RESULT_MISS)] = ARM_PERFCTR_DCACHE_MISS, > + }, > + [C(OP_WRITE)] = { > + [C(RESULT_ACCESS)] = ARM_PERFCTR_DCACHE_ACCESS, > + [C(RESULT_MISS)] = ARM_PERFCTR_DCACHE_MISS, > + }, > + [C(OP_PREFETCH)] = { > + [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED, > + [C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED, > + }, > + }, > + [C(L1I)] = { > + [C(OP_READ)] = { > + [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED, > + [C(RESULT_MISS)] = ARM_PERFCTR_ICACHE_MISS, > + }, > + [C(OP_WRITE)] = { > + [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED, > + [C(RESULT_MISS)] = ARM_PERFCTR_ICACHE_MISS, > + }, > + [C(OP_PREFETCH)] = { > + [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED, > + [C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED, > + }, > + }, > + [C(LL)] = { > + [C(OP_READ)] = { > + [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED, > + [C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED, > + }, > + [C(OP_WRITE)] = { > + [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED, > + [C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED, > + }, > + [C(OP_PREFETCH)] = { > + [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED, > + [C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED, > + }, > + }, > + [C(DTLB)] = { > + /* > + * The ARM performance counters can count micro DTLB misses, > + * micro ITLB misses and main TLB misses. There isn't an event > + * for TLB misses, so use the micro misses here and if users > + * want the main TLB misses they can use a raw counter. > + */ > + [C(OP_READ)] = { > + [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED, > + [C(RESULT_MISS)] = ARM_PERFCTR_DTLB_MISS, > + }, > + [C(OP_WRITE)] = { > + [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED, > + [C(RESULT_MISS)] = ARM_PERFCTR_DTLB_MISS, > + }, > + [C(OP_PREFETCH)] = { > + [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED, > + [C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED, > + }, > + }, > + [C(ITLB)] = { > + [C(OP_READ)] = { > + [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED, > + [C(RESULT_MISS)] = ARM_PERFCTR_ITLB_MISS, > + }, > + [C(OP_WRITE)] = { > + [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED, > + [C(RESULT_MISS)] = ARM_PERFCTR_ITLB_MISS, > + }, > + [C(OP_PREFETCH)] = { > + [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED, > + [C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED, > + }, > + }, > + [C(BPU)] = { > + [C(OP_READ)] = { > + [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED, > + [C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED, > + }, > + [C(OP_WRITE)] = { > + [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED, > + [C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED, > + }, > + [C(OP_PREFETCH)] = { > + [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED, > + [C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED, > + }, > + }, > +}; > + > +static const int > +armv6_map_cache_event(u64 config) > +{ > + unsigned int cache_type, cache_op, cache_result, ret; > + > + cache_type = (config >> 0) & 0xff; > + if (cache_type >= PERF_COUNT_HW_CACHE_MAX) > + return -EINVAL; > + > + cache_op = (config >> 8) & 0xff; > + if (cache_op >= PERF_COUNT_HW_CACHE_OP_MAX) > + return -EINVAL; > + > + cache_result = (config >> 16) & 0xff; > + if (cache_result >= PERF_COUNT_HW_CACHE_RESULT_MAX) > + return -EINVAL; > + > + ret = (int)v6_perf_cache_map[cache_type][cache_op][cache_result]; > + > + if (ret == CACHE_OP_UNSUPPORTED) > + return -ENOENT; > + > + return ret; > +} > + > +#define PMCR_ENABLE (1 << 0) > +#define PMCR_CTR01_RESET (1 << 1) > +#define PMCR_CCOUNT_RESET (1 << 2) > +#define PMCR_CCOUNT_DIV (1 << 3) > +#define PMCR_COUNT0_IEN (1 << 4) > +#define PMCR_COUNT1_IEN (1 << 5) > +#define PMCR_CCOUNT_IEN (1 << 6) > +#define PMCR_COUNT0_OVERFLOW (1 << 8) > +#define PMCR_COUNT1_OVERFLOW (1 << 9) > +#define PMCR_CCOUNT_OVERFLOW (1 << 10) > +#define PMCR_EVT_COUNT0_SHIFT 20 > +#define PMCR_EVT_COUNT0_MASK (0xFF << PMCR_EVT_COUNT0_SHIFT) > +#define PMCR_EVT_COUNT1_SHIFT 12 > +#define PMCR_EVT_COUNT1_MASK (0xFF << PMCR_EVT_COUNT1_SHIFT) > + > +static inline unsigned long > +pmcr_read(void) > +{ > + u32 val; > + asm volatile("mrc p15, 0, %0, c15, c12, 0" : "=r"(val)); > + return val; > +} This needs an armv6 prefix in the name. > + > +static inline void > +pmcr_write(unsigned long val) > +{ > + asm volatile("mcr p15, 0, %0, c15, c12, 0" : : "r"(val)); > +} This needs to be armv6 specific. > + > +#define PMCR_OVERFLOWED_MASK \ > + (PMCR_COUNT0_OVERFLOW | PMCR_COUNT1_OVERFLOW | PMCR_CCOUNT_OVERFLOW) This needs to be armv6 specific. > + > +static inline int > +pmcr_has_overflowed(unsigned long pmcr) > +{ > + return (pmcr & PMCR_OVERFLOWED_MASK); > +} > + > +static inline int > +pmcr_counter_has_overflowed(unsigned long pmcr, > + enum armv6_counters counter) > +{ > + int ret; > + > + if (ARMV6_CYCLE_COUNTER == counter) > + ret = pmcr & PMCR_CCOUNT_OVERFLOW; > + else if (ARMV6_COUNTER0 == counter) > + ret = pmcr & PMCR_COUNT0_OVERFLOW; > + else if (ARMV6_COUNTER1 == counter) > + ret = pmcr & PMCR_COUNT1_OVERFLOW; > + else > + BUG(); > + > + return ret; > +} > + > +static inline unsigned long > +armv6pmu_read_counter(enum armv6_counters counter) > +{ > + unsigned long value; > + > + if (ARMV6_CYCLE_COUNTER == counter) > + asm volatile("mrc p15, 0, %0, c15, c12, 1" : "=r"(value)); > + else if (ARMV6_COUNTER0 == counter) > + asm volatile("mrc p15, 0, %0, c15, c12, 2" : "=r"(value)); > + else if (ARMV6_COUNTER1 == counter) > + asm volatile("mrc p15, 0, %0, c15, c12, 3" : "=r"(value)); > + else > + BUG(); > + > + return value; > +} > + > +static inline void > +armv6pmu_write_counter(enum armv6_counters counter, > + unsigned long value) > +{ > + if (ARMV6_CYCLE_COUNTER == counter) > + asm volatile("mcr p15, 0, %0, c15, c12, 1" : : "r"(value)); > + else if (ARMV6_COUNTER0 == counter) > + asm volatile("mcr p15, 0, %0, c15, c12, 2" : : "r"(value)); > + else if (ARMV6_COUNTER1 == counter) > + asm volatile("mcr p15, 0, %0, c15, c12, 3" : : "r"(value)); > + else > + BUG(); > +} > + > +static int > +armv6pmu_place_event(struct cpu_hw_events *cpuc, > + struct hw_perf_event *event) > +{ > + /* Always place a cycle counter into the cycle counter. */ > + if (ARM_PERFCTR_CPU_CYCLES == event->config_base) { > + if (test_and_set_bit(ARMV6_CYCLE_COUNTER, cpuc->used_mask)) > + return -EAGAIN; > + > + return ARMV6_CYCLE_COUNTER; > + } else { > + /* > + * For anything other than a cycle counter, try and use > + * counter0 and counter1. > + */ > + if (!test_and_set_bit(ARMV6_COUNTER1, cpuc->used_mask)) { > + return ARMV6_COUNTER1; > + } > + > + if (!test_and_set_bit(ARMV6_COUNTER0, cpuc->used_mask)) { > + return ARMV6_COUNTER0; > + } > + > + /* The counters are all in use. */ > + return -EAGAIN; > + } > +} > + > +static void > +armv6pmu_disable_event(struct cpu_hw_events *cpuc, > + struct hw_perf_event *hwc, > + int idx) > +{ > + unsigned long val, mask, evt, flags; > + > + if (ARMV6_CYCLE_COUNTER == idx) { > + mask = PMCR_CCOUNT_IEN; > + evt = 0; > + } else if (ARMV6_COUNTER0 == idx) { > + mask = PMCR_COUNT0_IEN | PMCR_EVT_COUNT0_MASK; > + evt = ARM_PERFCTR_NOP << PMCR_EVT_COUNT0_SHIFT; > + } else if (ARMV6_COUNTER1 == idx) { > + mask = PMCR_COUNT1_IEN | PMCR_EVT_COUNT1_MASK; > + evt = ARM_PERFCTR_NOP << PMCR_EVT_COUNT1_SHIFT; > + } else { > + BUG(); > + } > + > + /* > + * Mask out the current event and set the counter to count the number > + * of ETM bus signal assertion cycles. The external reporting should > + * be disabled and so this should never increment. > + */ > + spin_lock_irqsave(&pmu_lock, flags); > + val = pmcr_read(); > + val &= ~mask; > + val |= evt; > + pmcr_write(val); > + spin_unlock_irqrestore(&pmu_lock, flags); > +} > + > +static void > +armv6pmu_enable_event(struct cpu_hw_events *cpuc, > + struct hw_perf_event *hwc, > + int idx) > +{ > + unsigned long val, mask, evt, flags; > + > + if (ARMV6_CYCLE_COUNTER == idx) { > + mask = 0; > + evt = PMCR_CCOUNT_IEN; > + } else if (ARMV6_COUNTER0 == idx) { > + mask = PMCR_EVT_COUNT0_MASK; > + evt = (hwc->config_base << PMCR_EVT_COUNT0_SHIFT) | > + PMCR_COUNT0_IEN; > + } else if (ARMV6_COUNTER1 == idx) { > + mask = PMCR_EVT_COUNT1_MASK; > + evt = (hwc->config_base << PMCR_EVT_COUNT1_SHIFT) | > + PMCR_COUNT1_IEN; > + } else { > + BUG(); > + } > + > + /* > + * Mask out the current event and set the counter to count the event > + * that we're interested in. > + */ > + spin_lock_irqsave(&pmu_lock, flags); > + val = pmcr_read(); > + val &= ~mask; > + val |= evt; > + pmcr_write(val); > + spin_unlock_irqrestore(&pmu_lock, flags); > +} > + > +static int > +armv6pmu_event_set_period(struct perf_event *event, > + struct hw_perf_event *hwc, > + int idx) > +{ > + s64 left = atomic64_read(&hwc->period_left); > + s64 period = hwc->sample_period; > + int ret = 0; > + > + if (unlikely(left <= -period)) { > + left = period; > + atomic64_set(&hwc->period_left, left); > + hwc->last_period = period; > + ret = 1; > + } > + > + if (unlikely(left <= 0)) { > + left += period; > + atomic64_set(&hwc->period_left, left); > + hwc->last_period = period; > + ret = 1; > + } > + > + if (left > MAX_PERIOD) > + left = MAX_PERIOD; > + > + atomic64_set(&hwc->prev_count, (u64)-left); > + > + armv6pmu_write_counter(idx, (u64)(-left) & 0xffffffff); > + > + perf_event_update_userpage(event); > + > + return ret; > +} > + > +static int > +armv6pmu_enable(struct perf_event *event) > +{ > + struct cpu_hw_events *cpuc = &__get_cpu_var(cpu_hw_events); > + struct hw_perf_event *hwc = &event->hw; > + int idx; > + int err = 0; > + > + /* If we don't have a space for the counter then finish early. */ > + idx = armv6pmu_place_event(cpuc, hwc); > + if (idx < 0) { > + err = idx; > + goto out; > + } > + > + /* > + * If there is an event in the counter we are going to use then make > + * sure it is disabled. > + */ > + event->hw.idx = idx; > + armv6pmu_disable_event(cpuc, hwc, idx); > + cpuc->events[idx] = event; > + set_bit(idx, cpuc->active_mask); > + > + /* Set the period for the event. */ > + armv6pmu_event_set_period(event, hwc, idx); > + > + /* Enable the event. */ > + armv6pmu_enable_event(cpuc, hwc, idx); > + > + /* Propagate our changes to the userspace mapping. */ > + perf_event_update_userpage(event); > + > +out: > + return err; > +} > + > +static u64 > +armv6pmu_event_update(struct perf_event *event, > + struct hw_perf_event *hwc, > + int idx) > +{ > + int shift = 64 - 32; > + u64 prev_raw_count, new_raw_count; > + s64 delta; > + > +again: > + prev_raw_count = atomic64_read(&hwc->prev_count); > + new_raw_count = armv6pmu_read_counter(idx); > + > + if (atomic64_cmpxchg(&hwc->prev_count, prev_raw_count, > + new_raw_count) != prev_raw_count) > + goto again; > + > + delta = (new_raw_count << shift) - (prev_raw_count << shift); > + delta >>= shift; > + > + atomic64_add(delta, &event->count); > + atomic64_sub(delta, &hwc->period_left); > + > + return new_raw_count; > +} > + > +static void > +armv6pmu_disable(struct perf_event *event) > +{ > + struct cpu_hw_events *cpuc = &__get_cpu_var(cpu_hw_events); > + struct hw_perf_event *hwc = &event->hw; > + int idx = hwc->idx; > + > + WARN_ON(idx < 0); > + > + clear_bit(idx, cpuc->active_mask); > + armv6pmu_disable_event(cpuc, hwc, idx); > + > + barrier(); > + > + armv6pmu_event_update(event, hwc, idx); > + cpuc->events[idx] = NULL; > + clear_bit(idx, cpuc->used_mask); > + > + perf_event_update_userpage(event); > +} > + > +static void > +armv6pmu_read(struct perf_event *event) > +{ > + struct hw_perf_event *hwc = &event->hw; > + > + /* Don't read disabled counters! */ > + if (hwc->idx < 0) > + return; > + > + armv6pmu_event_update(event, hwc, hwc->idx); > +} > + > +static void > +armv6pmu_unthrottle(struct perf_event *event) > +{ > + struct cpu_hw_events *cpuc = &__get_cpu_var(cpu_hw_events); > + struct hw_perf_event *hwc = &event->hw; > + > + armv6pmu_enable_event(cpuc, hwc, hwc->idx); > +} > + > +static irqreturn_t > +armv6_perfcounters_irq(int irq_num, > + void *dev) > +{ > + unsigned long pmcr = pmcr_read(); > + struct perf_sample_data data; > + struct cpu_hw_events *cpuc; > + struct pt_regs *regs; > + int idx; > + > + if (!pmcr_has_overflowed(pmcr)) > + return IRQ_NONE; > + > + regs = get_irq_regs(); > + > + /* > + * The interrupts are cleared by writing the overflow flags back to > + * the control register. All of the other bits don't have any effect > + * if they are rewritten, so write the whole value back. > + */ > + pmcr_write(pmcr); > + > + data.addr = 0; > + > + cpuc = &__get_cpu_var(cpu_hw_events); > + for (idx = 0; idx < ARMV6_MAX_HWEVENTS; ++idx) { > + struct perf_event *event = cpuc->events[idx]; > + struct hw_perf_event *hwc; > + > + if (!test_bit(idx, cpuc->active_mask)) > + continue; > + > + /* > + * We have a single interrupt for all counters. Check that > + * each counter has overflowed before we process it. > + */ > + if (!pmcr_counter_has_overflowed(pmcr, idx)) > + continue; > + > + hwc = &event->hw; > + armv6pmu_event_update(event, hwc, idx); > + data.period = event->hw.last_period; > + if (!armv6pmu_event_set_period(event, hwc, idx)) > + continue; > + > + if (perf_event_overflow(event, 0, &data, regs)) > + armv6pmu_disable_event(cpuc, hwc, idx); > + } > + > + /* > + * Handle the pending perf events. > + * > + * Note: this call *must* be run with interrupts enabled. For > + * platforms that can have the PMU interrupts raised as a PMI, this > + * will not work. > + */ > + perf_event_do_pending(); > + > + return IRQ_HANDLED; > +} > + > +void > +hw_perf_enable(void) > +{ > + /* Enable all of the perf events on hardware. */ > + int idx; > + struct cpu_hw_events *cpuc = &__get_cpu_var(cpu_hw_events); > + unsigned long flags, val; > + > + for (idx = 0; idx < ARMV6_MAX_HWEVENTS; ++idx) { > + struct perf_event *event = cpuc->events[idx]; > + > + if (!event) > + continue; > + > + armv6pmu_enable_event(cpuc, &event->hw, idx); Can this function be made generic for all ARM implementations? > + } > + > + spin_lock_irqsave(&pmu_lock, flags); > + val = pmcr_read(); > + val |= PMCR_ENABLE; > + pmcr_write(val); > + spin_unlock_irqrestore(&pmu_lock, flags); > +} > + > +void > +hw_perf_disable(void) > +{ > + unsigned long flags, val; > + > + spin_lock_irqsave(&pmu_lock, flags); > + val = pmcr_read(); > + val &= ~PMCR_ENABLE; > + pmcr_write(val); > + spin_unlock_irqrestore(&pmu_lock, flags); > +} > + > +static const struct pmu armv6pmu = { > + .enable = armv6pmu_enable, > + .disable = armv6pmu_disable, > + .read = armv6pmu_read, > + .unthrottle = armv6pmu_unthrottle, > +}; > + > +static int > +validate_event(struct cpu_hw_events *cpuc, > + struct perf_event *event) > +{ > + struct hw_perf_event fake_event = event->hw; > + > + if (event->pmu && event->pmu != &armv6pmu) > + return 0; > + > + return armv6pmu_place_event(cpuc, &fake_event) >= 0; > +} Can this function be made generic for all ARM implementations? > + > +static int > +validate_group(struct perf_event *event) > +{ > + struct perf_event *sibling, *leader = event->group_leader; > + struct cpu_hw_events fake_pmu; > + > + memset(&fake_pmu, 0, sizeof(fake_pmu)); > + > + if (!validate_event(&fake_pmu, leader)) > + return -ENOSPC; > + > + list_for_each_entry(sibling, &leader->sibling_list, group_entry) { > + if (!validate_event(&fake_pmu, sibling)) > + return -ENOSPC; > + } > + > + if (!validate_event(&fake_pmu, event)) > + return -ENOSPC; > + > + return 0; > +} > + > +static int > +armv6_hw_perf_event_init(struct perf_event *event) > +{ > + struct hw_perf_event *hwc = &event->hw; > + int mapping, err; > + > + /* Decode the generic type into an ARM event identifier. */ > + if (PERF_TYPE_HARDWARE == event->attr.type) { > + mapping = armv6_map_hw_event(event->attr.config); > + } else if (PERF_TYPE_HW_CACHE == event->attr.type) { > + mapping = armv6_map_cache_event(event->attr.config); > + } else if (PERF_TYPE_RAW == event->attr.type) { > + /* The EvtCountN field of the PMCR is 8 bits. */ > + mapping = event->attr.config & 0xFF; > + } else { > + pr_debug("event type %x not supported\n", event->attr.type); > + return -EOPNOTSUPP; > + } > + > + if (mapping < 0) { > + pr_debug("event %x:%llx not supported\n", event->attr.type, > + event->attr.config); > + return mapping; > + } > + > + /* > + * Check whether we need to exclude the counter from certain modes. > + * The ARM performance counters are on all of the time so if someone > + * has asked us for some excludes then we have to fail. > + */ > + if (event->attr.exclude_kernel || event->attr.exclude_user || > + event->attr.exclude_hv || event->attr.exclude_idle) { > + pr_debug("ARM performance counters do not support " > + "mode exclusion\n"); > + return -EPERM; > + } > + > + /* > + * We don't assign an index until we actually place the event onto > + * hardware. Use -1 to signify that we haven't decided where to put it > + * yet. For SMP systems, each core has it's own PMU so we can't do any > + * clever allocation or constraints checking at this point. > + */ > + hwc->idx = -1; > + > + /* > + * Store the event encoding into the config_base field. config and > + * event_base are unused as the only 2 things we need to know are > + * the event mapping and the counter to use. The counter to use is > + * also the indx and the config_base is the event type. > + */ > + hwc->config_base = (unsigned long)mapping; > + hwc->config = 0; > + hwc->event_base = 0; > + > + if (!hwc->sample_period) { > + hwc->sample_period = MAX_PERIOD; > + hwc->last_period = hwc->sample_period; > + atomic64_set(&hwc->period_left, hwc->sample_period); > + } > + > + err = 0; > + if (event->group_leader != event) { > + err = validate_group(event); > + if (err) > + return -EINVAL; > + } > + > + return err; > +} > + > +static int > +armv6pmu_reserve_hardware(void) > +{ > + int i; > + int err; > + > + pmu_irqs = reserve_pmu(); > + if (IS_ERR(pmu_irqs)) { > + pr_warning("unable to reserve pmu\n"); > + return PTR_ERR(pmu_irqs); > + } > + > + init_pmu(); > + > + if (pmu_irqs->num_irqs < 1) { > + pr_err("no irqs for PMUs defined\n"); > + } > + > + for (i = 0; i < pmu_irqs->num_irqs; ++i) { > + err = request_irq(pmu_irqs->irqs[i], armv6_perfcounters_irq, > + IRQF_DISABLED, "armv6_perfctr", NULL); > + if (err) { > + pr_warning("unable to request IRQ%d for ARMv6 " > + "perf counters\n", pmu_irqs->irqs[i]); > + break; > + } > + } > + > + if (err) { > + for (i = i - 1; i >= 0; --i) > + free_irq(pmu_irqs->irqs[i], NULL); > + release_pmu(pmu_irqs); > + pmu_irqs = NULL; > + } > + > + return err; > +} > + > +static void > +armv6pmu_release_hardware(void) > +{ > + int i; > + > + for (i = pmu_irqs->num_irqs - 1; i >= 0; --i) { > + free_irq(pmu_irqs->irqs[i], NULL); > + } > + pmcr_write(0); > + > + release_pmu(pmu_irqs); > + pmu_irqs = NULL; > +} > + > +static atomic_t active_events = ATOMIC_INIT(0); > +static DEFINE_MUTEX(pmu_reserve_mutex); > + > +static void > +hw_perf_event_destroy(struct perf_event *event) > +{ > + if (atomic_dec_and_mutex_lock(&active_events, &pmu_reserve_mutex)) { > + armv6pmu_release_hardware(); Can this function be made generic for all ARM implementations? > + mutex_unlock(&pmu_reserve_mutex); > + } > +} > + > +const struct pmu * > +hw_perf_event_init(struct perf_event *event) > +{ > + int err = 0; > + > + /* We support 3 events: one cycle counter and 2 programmable. */ > + perf_max_events = 3; > + event->destroy = hw_perf_event_destroy; > + > + if (!atomic_inc_not_zero(&active_events)) { > + if (atomic_read(&active_events) > perf_max_events) { > + atomic_dec(&active_events); > + return ERR_PTR(-ENOSPC); > + } > + > + mutex_lock(&pmu_reserve_mutex); > + if (atomic_read(&active_events) == 0) { > + err = armv6pmu_reserve_hardware(); Can this function be made generic for all ARM implementations? > + } > + > + if (!err) > + atomic_inc(&active_events); > + mutex_unlock(&pmu_reserve_mutex); > + } > + > + if (err) > + return ERR_PTR(err); > + > + err = armv6_hw_perf_event_init(event); Same here. > + if (err) > + hw_perf_event_destroy(event); > + > + return err ? ERR_PTR(err) : &armv6pmu; > +} > + > +/* Callchain handling code. */ > +static inline void > +callchain_store(struct perf_callchain_entry *entry, > + u64 ip) > +{ > + if (entry->nr < PERF_MAX_STACK_DEPTH) > + entry->ip[entry->nr++] = ip; > +} > + > +/* > + * The registers we're interested in are at the end of the variable > + * length saved register structure. The fp points at the end of this > + * structure so the address of this struct is: > + * (struct frame_tail *)(xxx->fp)-1 > + * > + * This code has been adapted from the ARM OProfile support. > + */ > +struct frame_tail { > + struct frame_tail *fp; > + unsigned long sp; > + unsigned long lr; > +} __attribute__((packed)); > + > +/* > + * Get the return address for a single stackframe and return a pointer to the > + * next frame tail. > + */ > +static struct frame_tail * > +user_backtrace(struct frame_tail *tail, > + struct perf_callchain_entry *entry) > +{ > + struct frame_tail buftail; > + > + /* Also check accessibility of one struct frame_tail beyond */ > + if (!access_ok(VERIFY_READ, tail, sizeof(buftail))) > + return NULL; > + if (__copy_from_user_inatomic(&buftail, tail, sizeof(buftail))) > + return NULL; > + > + callchain_store(entry, buftail.lr); > + > + /* > + * Frame pointers should strictly progress back up the stack > + * (towards higher addresses). > + */ > + if (tail >= buftail.fp) > + return NULL; > + > + return buftail.fp - 1; > +} > + > +static void > +perf_callchain_user(struct pt_regs *regs, > + struct perf_callchain_entry *entry) > +{ > + struct frame_tail *tail; > + > + callchain_store(entry, PERF_CONTEXT_USER); > + > + if (!user_mode(regs)) > + regs = task_pt_regs(current); > + > + tail = (struct frame_tail *)regs->ARM_fp - 1; > + > + while (tail && !((unsigned long)tail & 0x3)) > + tail = user_backtrace(tail, entry); > +} > + > +/* > + * Gets called by walk_stackframe() for every stackframe. This will be called > + * whist unwinding the stackframe and is like a subroutine return so we use > + * the PC. > + */ > +static int > +callchain_trace(struct stackframe *fr, > + void *data) > +{ > + struct perf_callchain_entry *entry = data; > + callchain_store(entry, fr->pc); > + return 0; > +} > + > +static void > +perf_callchain_kernel(struct pt_regs *regs, > + struct perf_callchain_entry *entry) > +{ > + struct stackframe fr; > + > + callchain_store(entry, PERF_CONTEXT_KERNEL); > + fr.fp = regs->ARM_fp; > + fr.sp = regs->ARM_sp; > + fr.lr = regs->ARM_lr; > + fr.pc = regs->ARM_pc; > + walk_stackframe(&fr, callchain_trace, entry); > +} > + > +static void > +perf_do_callchain(struct pt_regs *regs, > + struct perf_callchain_entry *entry) > +{ > + int is_user; > + > + if (!regs) > + return; > + > + is_user = user_mode(regs); > + > + if (!current || !current->pid) > + return; > + > + if (is_user && current->state != TASK_RUNNING) > + return; > + > + if (!is_user) > + perf_callchain_kernel(regs, entry); > + > + if (current->mm) > + perf_callchain_user(regs, entry); > +} > + > +static DEFINE_PER_CPU(struct perf_callchain_entry, pmc_irq_entry); > + > +struct perf_callchain_entry * > +perf_callchain(struct pt_regs *regs) > +{ > + struct perf_callchain_entry *entry = &__get_cpu_var(pmc_irq_entry); > + > + entry->nr = 0; > + perf_do_callchain(regs, entry); > + return entry; > +} > diff --git a/arch/arm/mm/Kconfig b/arch/arm/mm/Kconfig > index fc5c05b..89a26ea 100644 > --- a/arch/arm/mm/Kconfig > +++ b/arch/arm/mm/Kconfig > @@ -791,3 +791,11 @@ config ARM_L1_CACHE_SHIFT > int > default 6 if ARCH_OMAP3 || ARCH_S5PC1XX > default 5 > + > +config HW_PERF_EVENTS > + bool "Enable hardware performance counter support for perf events" > + depends on PERF_EVENTS && CPU_HAS_PMU && CPU_V6 > + default n > + help > + Enable hardware performance counter support for perf events. If > + disabled, perf events will use software events only. > -- > 1.6.5.4 > Thanks & regards, Jean > > _______________________________________________ > linux-arm-kernel mailing list > linux-arm-kernel at lists.infradead.org > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 28+ messages in thread
* [PATCH 5/5] arm/perfevents: implement perf event support for ARMv6 2009-12-14 16:12 ` Jean Pihet @ 2009-12-14 16:33 ` Jamie Iles 2009-12-14 16:57 ` Jean Pihet 2009-12-14 17:09 ` Will Deacon 1 sibling, 1 reply; 28+ messages in thread From: Jamie Iles @ 2009-12-14 16:33 UTC (permalink / raw) To: linux-arm-kernel Hi Jean, On Mon, Dec 14, 2009 at 05:12:57PM +0100, Jean Pihet wrote: > I have a question about other ARM implementations. Since all the code is > in perf_event.c is it modular enough to allow the addition of other ARM > implementations? The remarks are inlined below. > > I am interested in supporting perf events for ARMv7. What do you think? It should be possible, and not too difficult. The x86 perf events code (arch/x86/kernel/cpu/perf_events.c) does exactly this. We would need to define an arm_pmu structure that wraps up a 'struct pmu' and provides hooks for the architecture specific call. > > > > diff --git a/arch/arm/kernel/Makefile b/arch/arm/kernel/Makefile > > index 286a276..44ebf36 100644 > > --- a/arch/arm/kernel/Makefile > > +++ b/arch/arm/kernel/Makefile > > @@ -47,6 +47,7 @@ obj-$(CONFIG_CPU_XSC3) += xscale-cp0.o > > obj-$(CONFIG_CPU_MOHAWK) += xscale-cp0.o > > obj-$(CONFIG_IWMMXT) += iwmmxt.o > > obj-$(CONFIG_CPU_HAS_PMU) += pmu.o > > +obj-$(CONFIG_HW_PERF_EVENTS) += perf_event.o > > AFLAGS_iwmmxt.o := -Wa,-mcpu=iwmmxt > > > > ifneq ($(CONFIG_ARCH_EBSA110),y) > > diff --git a/arch/arm/kernel/perf_event.c b/arch/arm/kernel/perf_event.c > > new file mode 100644 > > index 0000000..358fd31 > I have a generic question here: will #ifdef be needed to allow other ARM > implementations (e.g. ARMv7)? Is it worth to do it that way or to > duplicate this file for other ARM implementations? We can probably do it without preprocessor magic as long as the coprocessor instructions for the v6 counters still assemble under v7. We can do it all at runtime like in the x86 code and return the correct pmu from hw_perf_event_init(). > > > --- /dev/null > > +++ b/arch/arm/kernel/perf_event.c > > @@ -0,0 +1,1034 @@ > > +#undef DEBUG > > + > > +/* > > + * ARMv6 performance counter support. > ARMv6 only? Yes, at the moment ;-) [snip] > > +enum arm_perf_types { > > + ARM_PERFCTR_ICACHE_MISS = 0x0, > > + ARM_PERFCTR_IBUF_STALL = 0x1, > > + ARM_PERFCTR_DDEP_STALL = 0x2, > > + ARM_PERFCTR_ITLB_MISS = 0x3, > > + ARM_PERFCTR_DTLB_MISS = 0x4, > > + ARM_PERFCTR_BR_EXEC = 0x5, > > + ARM_PERFCTR_BR_MISPREDICT = 0x6, > > + ARM_PERFCTR_INSTR_EXEC = 0x7, > > + ARM_PERFCTR_DCACHE_HIT = 0x9, > > + ARM_PERFCTR_DCACHE_ACCESS = 0xA, > > + ARM_PERFCTR_DCACHE_MISS = 0xB, > > + ARM_PERFCTR_DCACHE_WBACK = 0xC, > > + ARM_PERFCTR_SW_PC_CHANGE = 0xD, > > + ARM_PERFCTR_MAIN_TLB_MISS = 0xF, > > + ARM_PERFCTR_EXPL_D_ACCESS = 0x10, > > + ARM_PERFCTR_LSU_FULL_STALL = 0x11, > > + ARM_PERFCTR_WBUF_DRAINED = 0x12, > > + ARM_PERFCTR_CPU_CYCLES = 0xFF, > > + ARM_PERFCTR_NOP = 0x20, > > +}; > This needs an armv6 prefix in the name. Agreed. > > +/* The events for a given CPU. */ > > +struct cpu_hw_events { > > + /* > > + * The events that are active on the CPU for the given index. Index 0 > > + * is reserved. > > + */ > > + struct perf_event *events[ARMV6_MAX_HWEVENTS]; > > + > > + /* > > + * A 1 bit for an index indicates that the counter is being used for > > + * an event. A 0 means that the counter can be used. > > + */ > > + unsigned long used_mask[BITS_TO_LONGS(ARMV6_MAX_HWEVENTS)]; > > + > > + /* > > + * A 1 bit for an index indicates that the counter is actively being > > + * used. > > + */ > > + unsigned long active_mask[BITS_TO_LONGS(ARMV6_MAX_HWEVENTS)]; > > +}; > Will it need #ifdef for other ARM implementations, or a armv6 prefix? We could make the bitmaps large enough to hold the biggest case then store the maximum number events for each PMU type in its own structure. As for making everything else implementation specific, if we adopted the same method as x86 then we could easily do this. Jamie ^ permalink raw reply [flat|nested] 28+ messages in thread
* [PATCH 5/5] arm/perfevents: implement perf event support for ARMv6 2009-12-14 16:33 ` Jamie Iles @ 2009-12-14 16:57 ` Jean Pihet 0 siblings, 0 replies; 28+ messages in thread From: Jean Pihet @ 2009-12-14 16:57 UTC (permalink / raw) To: linux-arm-kernel Hi Jamie, On Mon, 2009-12-14 at 16:33 +0000, Jamie Iles wrote: > Hi Jean, > > On Mon, Dec 14, 2009 at 05:12:57PM +0100, Jean Pihet wrote: > > I have a question about other ARM implementations. Since all the code is > > in perf_event.c is it modular enough to allow the addition of other ARM > > implementations? The remarks are inlined below. > > > > I am interested in supporting perf events for ARMv7. What do you think? > It should be possible, and not too difficult. The x86 perf events code > (arch/x86/kernel/cpu/perf_events.c) does exactly this. We would need to define > an arm_pmu structure that wraps up a 'struct pmu' and provides hooks for the > architecture specific call. Ok I see. Let's have a generic code for ARM, then add the ARMv7 support (including variations depending on the Cortex chip in use). ... > > I have a generic question here: will #ifdef be needed to allow other ARM > > implementations (e.g. ARMv7)? Is it worth to do it that way or to > > duplicate this file for other ARM implementations? > We can probably do it without preprocessor magic as long as the coprocessor > instructions for the v6 counters still assemble under v7. We can do it all at > runtime like in the x86 code and return the correct pmu from > hw_perf_event_init(). Agree. > > > > > --- /dev/null > > > +++ b/arch/arm/kernel/perf_event.c > > > @@ -0,0 +1,1034 @@ > > > +#undef DEBUG > > > + > > > +/* > > > + * ARMv6 performance counter support. > > ARMv6 only? > Yes, at the moment ;-) > > [snip] > > > +enum arm_perf_types { > > > + ARM_PERFCTR_ICACHE_MISS = 0x0, > > > + ARM_PERFCTR_IBUF_STALL = 0x1, > > > + ARM_PERFCTR_DDEP_STALL = 0x2, > > > + ARM_PERFCTR_ITLB_MISS = 0x3, > > > + ARM_PERFCTR_DTLB_MISS = 0x4, > > > + ARM_PERFCTR_BR_EXEC = 0x5, > > > + ARM_PERFCTR_BR_MISPREDICT = 0x6, > > > + ARM_PERFCTR_INSTR_EXEC = 0x7, > > > + ARM_PERFCTR_DCACHE_HIT = 0x9, > > > + ARM_PERFCTR_DCACHE_ACCESS = 0xA, > > > + ARM_PERFCTR_DCACHE_MISS = 0xB, > > > + ARM_PERFCTR_DCACHE_WBACK = 0xC, > > > + ARM_PERFCTR_SW_PC_CHANGE = 0xD, > > > + ARM_PERFCTR_MAIN_TLB_MISS = 0xF, > > > + ARM_PERFCTR_EXPL_D_ACCESS = 0x10, > > > + ARM_PERFCTR_LSU_FULL_STALL = 0x11, > > > + ARM_PERFCTR_WBUF_DRAINED = 0x12, > > > + ARM_PERFCTR_CPU_CYCLES = 0xFF, > > > + ARM_PERFCTR_NOP = 0x20, > > > +}; > > This needs an armv6 prefix in the name. > Agreed. Ok. ... > > Will it need #ifdef for other ARM implementations, or a armv6 prefix? > We could make the bitmaps large enough to hold the biggest case then store the > maximum number events for each PMU type in its own structure. > > As for making everything else implementation specific, if we adopted the same > method as x86 then we could easily do this. Agree. > > Jamie Thanks, Jean ^ permalink raw reply [flat|nested] 28+ messages in thread
* [PATCH 5/5] arm/perfevents: implement perf event support for ARMv6 2009-12-14 16:12 ` Jean Pihet 2009-12-14 16:33 ` Jamie Iles @ 2009-12-14 17:09 ` Will Deacon 1 sibling, 0 replies; 28+ messages in thread From: Will Deacon @ 2009-12-14 17:09 UTC (permalink / raw) To: linux-arm-kernel Hi Jean, * Jean Pihet wrote: > > I am interested in supporting perf events for ARMv7. What do you think? > I'm also interested in this. My plan was to add an initialisation function as an arch_initcall which would switch on the cpuid and setup a static PMU struct that could be later returned by hw_perf_event_init. This struct could also contain another struct encoding the event numbers for that particular core. Is this what you had in mind? Either way, we should get Jamie's code sorted and then build on top of that to avoid divergence. Will ^ permalink raw reply [flat|nested] 28+ messages in thread
* [PATCH 5/5] arm/perfevents: implement perf event support for ARMv6 2009-12-14 14:04 ` [PATCH 5/5] arm/perfevents: implement perf event support for ARMv6 Jamie Iles 2009-12-14 16:12 ` Jean Pihet @ 2009-12-14 16:13 ` Will Deacon 2009-12-14 16:20 ` Jamie Iles 1 sibling, 1 reply; 28+ messages in thread From: Will Deacon @ 2009-12-14 16:13 UTC (permalink / raw) To: linux-arm-kernel * Jamie Iles wrote: > This patch implements support for ARMv6 performance counters in the > Linux performance events subsystem. ARMv6 architectures that have the > performance counters should enable HW_PERF_EVENTS and define the > interrupts for the counters in arch/arm/kernel/perf_event.c > > diff --git a/arch/arm/kernel/Makefile b/arch/arm/kernel/Makefile > index 286a276..44ebf36 100644 > --- a/arch/arm/kernel/Makefile > +++ b/arch/arm/kernel/Makefile > @@ -47,6 +47,7 @@ obj-$(CONFIG_CPU_XSC3) += xscale-cp0.o > obj-$(CONFIG_CPU_MOHAWK) += xscale-cp0.o > obj-$(CONFIG_IWMMXT) += iwmmxt.o > obj-$(CONFIG_CPU_HAS_PMU) += pmu.o > +obj-$(CONFIG_HW_PERF_EVENTS) += perf_event.o > AFLAGS_iwmmxt.o := -Wa,-mcpu=iwmmxt > > ifneq ($(CONFIG_ARCH_EBSA110),y) <snip> > diff --git a/arch/arm/mm/Kconfig b/arch/arm/mm/Kconfig > index fc5c05b..89a26ea 100644 > --- a/arch/arm/mm/Kconfig > +++ b/arch/arm/mm/Kconfig > @@ -791,3 +791,11 @@ config ARM_L1_CACHE_SHIFT > int > default 6 if ARCH_OMAP3 || ARCH_S5PC1XX > default 5 > + > +config HW_PERF_EVENTS > + bool "Enable hardware performance counter support for perf events" > + depends on PERF_EVENTS && CPU_HAS_PMU && CPU_V6 > + default n > + help > + Enable hardware performance counter support for perf events. If > + disabled, perf events will use software events only. > -- > 1.6.5.4 Why are you modifying mm/Kconfig? I think it would be better to change the Makefile to use CONFIG_HAVE_PERF_EVENTS rather than define a new bool. Will ^ permalink raw reply [flat|nested] 28+ messages in thread
* [PATCH 5/5] arm/perfevents: implement perf event support for ARMv6 2009-12-14 16:13 ` Will Deacon @ 2009-12-14 16:20 ` Jamie Iles 2009-12-14 16:24 ` Will Deacon 0 siblings, 1 reply; 28+ messages in thread From: Jamie Iles @ 2009-12-14 16:20 UTC (permalink / raw) To: linux-arm-kernel On Mon, Dec 14, 2009 at 04:13:05PM -0000, Will Deacon wrote: > > diff --git a/arch/arm/mm/Kconfig b/arch/arm/mm/Kconfig > > index fc5c05b..89a26ea 100644 > > --- a/arch/arm/mm/Kconfig > > +++ b/arch/arm/mm/Kconfig > > @@ -791,3 +791,11 @@ config ARM_L1_CACHE_SHIFT > > int > > default 6 if ARCH_OMAP3 || ARCH_S5PC1XX > > default 5 > > + > > +config HW_PERF_EVENTS > > + bool "Enable hardware performance counter support for perf events" > > + depends on PERF_EVENTS && CPU_HAS_PMU && CPU_V6 > > + default n > > + help > > + Enable hardware performance counter support for perf events. If > > + disabled, perf events will use software events only. > > -- > > 1.6.5.4 > > Why are you modifying mm/Kconfig? I think it would be better to change the > Makefile to use CONFIG_HAVE_PERF_EVENTS rather than define a new bool. Because this bool says that we support hardware perf events. If it isn't set, we can still do perf events, but only the software variety. This allows us to do software events for ARMv5 and earlier, but also do hardware counter support where available. Jamie ^ permalink raw reply [flat|nested] 28+ messages in thread
* [PATCH 5/5] arm/perfevents: implement perf event support for ARMv6 2009-12-14 16:20 ` Jamie Iles @ 2009-12-14 16:24 ` Will Deacon 0 siblings, 0 replies; 28+ messages in thread From: Will Deacon @ 2009-12-14 16:24 UTC (permalink / raw) To: linux-arm-kernel * Jamie Iles wrote: > > Why are you modifying mm/Kconfig? I think it would be better to change the > > Makefile to use CONFIG_HAVE_PERF_EVENTS rather than define a new bool. > Because this bool says that we support hardware perf events. If it isn't set, > we can still do perf events, but only the software variety. This allows us to > do software events for ARMv5 and earlier, but also do hardware counter support > where available. Ok, sorry I missed that. I still feel that it should be in the top-level ARM Kconfig instead of the mm/Kconfig though. Will ^ permalink raw reply [flat|nested] 28+ messages in thread
* [PATCH 3/5] arm: use the spinlocked, generic atomic64 support 2009-12-14 14:04 ` [PATCH 3/5] arm: use the spinlocked, generic atomic64 support Jamie Iles 2009-12-14 14:04 ` [PATCH 4/5] arm: enable support for software perf events Jamie Iles @ 2009-12-14 17:38 ` Nicolas Pitre 2009-12-14 19:36 ` Will Deacon [not found] ` <001301ca7cf4$c04481a0$40cd84e0$%deacon@arm.com> 1 sibling, 2 replies; 28+ messages in thread From: Nicolas Pitre @ 2009-12-14 17:38 UTC (permalink / raw) To: linux-arm-kernel On Mon, 14 Dec 2009, Jamie Iles wrote: > perf events require that we can support atomic64's. There is a generic, > spinlocked version that we can use until we have proper hardware > support. Can't a variant of include/linux/cnt32_to_63.h be used here? typedef struct { atomic_t low; u32 high; } atomic64_t; static inline void atomic64_set(atomic64_t *ptr, u64 new_val) { u32 low = new_val; u32 high = new_val >> 32; BUG_ON(high & 0x80000000); atomic_set(&ptr->low, low); ptr->high = (high & 0x7fffffff) | (low & 0x80000000); } static inline u64 atomic64_read(atomic64_t *ptr) { u32 high, low; high = ptr->high; smp_rmb(); low = atomic_read(&ptr->low); if (unlikely((s32)(high ^ low) < 0)) ptr->high = high = (high ^ 0x80000000) + (high >> 31); return ((u64)(high & 0x7fffffff) << 32) | low; } static inline u64 atomic64_inc_return(atomic64_t *ptr) { atomic_inc(&ptr->low); return atomic64_read(ptr); } The atomic64_add_return() could be implemented the same way, however the added value would have to be smaller than 31 bits for the algorythm to work. Nicolas ^ permalink raw reply [flat|nested] 28+ messages in thread
* [PATCH 3/5] arm: use the spinlocked, generic atomic64 support 2009-12-14 17:38 ` [PATCH 3/5] arm: use the spinlocked, generic atomic64 support Nicolas Pitre @ 2009-12-14 19:36 ` Will Deacon [not found] ` <001301ca7cf4$c04481a0$40cd84e0$%deacon@arm.com> 1 sibling, 0 replies; 28+ messages in thread From: Will Deacon @ 2009-12-14 19:36 UTC (permalink / raw) To: linux-arm-kernel Hi Nicolas, *Nicolas Pitre wrote: > Can't a variant of include/linux/cnt32_to_63.h be used here?> > typedef struct { > atomic_t low; > u32 high; > } atomic64_t; > > static inline void atomic64_set(atomic64_t *ptr, u64 new_val) > { > u32 low = new_val; > u32 high = new_val >> 32; > BUG_ON(high & 0x80000000); > atomic_set(&ptr->low, low); > ptr->high = (high & 0x7fffffff) | (low & 0x80000000); > } How do you ensure that this is atomic? To me it looks like one CPU could write the lower 32-bits and another could write the upper 32, leaving the memory location in an inconsistent state. > static inline u64 atomic64_read(atomic64_t *ptr) > { > u32 high, low; > high = ptr->high; > smp_rmb(); > low = atomic_read(&ptr->low); > if (unlikely((s32)(high ^ low) < 0)) > ptr->high = high = (high ^ 0x80000000) + (high >> 31); > return ((u64)(high & 0x7fffffff) << 32) | low; > } > > static inline u64 atomic64_inc_return(atomic64_t *ptr) > { > atomic_inc(&ptr->low); > return atomic64_read(ptr); > } > > The atomic64_add_return() could be implemented the same way, however the > added value would have to be smaller than 31 bits for the algorythm to > work. I posted a patch to this list on Friday which provides 64-bit atomic operations for ARM using exclusive loads and stores: http://lists.infradead.org/pipermail/linux-arm-kernel/2009-December/005934.html Once this patch has been successfully reviewed, these routines should be used instead. For now it makes sense to use the generic spinlocks version as a placeholder. Will ^ permalink raw reply [flat|nested] 28+ messages in thread
[parent not found: <001301ca7cf4$c04481a0$40cd84e0$%deacon@arm.com>]
* [PATCH 3/5] arm: use the spinlocked, generic atomic64 support [not found] ` <001301ca7cf4$c04481a0$40cd84e0$%deacon@arm.com> @ 2009-12-14 19:52 ` Nicolas Pitre 2009-12-15 10:24 ` Catalin Marinas 0 siblings, 1 reply; 28+ messages in thread From: Nicolas Pitre @ 2009-12-14 19:52 UTC (permalink / raw) To: linux-arm-kernel On Mon, 14 Dec 2009, Will Deacon wrote: > Hi Nicolas, > > *Nicolas Pitre wrote: > > > Can't a variant of include/linux/cnt32_to_63.h be used here?> > > typedef struct { > > atomic_t low; > > u32 high; > > } atomic64_t; > > > > static inline void atomic64_set(atomic64_t *ptr, u64 new_val) > > { > > u32 low = new_val; > > u32 high = new_val >> 32; > > BUG_ON(high & 0x80000000); > > atomic_set(&ptr->low, low); > > ptr->high = (high & 0x7fffffff) | (low & 0x80000000); > > } > > How do you ensure that this is atomic? To me it looks like one CPU > could write the lower 32-bits and another could write the upper 32, > leaving the memory location in an inconsistent state. This one is indeed not exactly atomic. It isn't the interesting case either though. Probably a strd could fix that. > > static inline u64 atomic64_read(atomic64_t *ptr) > > { > > u32 high, low; > > high = ptr->high; > > smp_rmb(); > > low = atomic_read(&ptr->low); > > if (unlikely((s32)(high ^ low) < 0)) > > ptr->high = high = (high ^ 0x80000000) + (high >> 31); > > return ((u64)(high & 0x7fffffff) << 32) | low; > > } > > > > static inline u64 atomic64_inc_return(atomic64_t *ptr) > > { > > atomic_inc(&ptr->low); > > return atomic64_read(ptr); > > } > > > > The atomic64_add_return() could be implemented the same way, however the > > added value would have to be smaller than 31 bits for the algorythm to > > work. > > I posted a patch to this list on Friday which provides 64-bit atomic > operations for ARM using exclusive loads and stores: > > http://lists.infradead.org/pipermail/linux-arm-kernel/2009-December/005934.html For ARMv7 (or ARMv6K), not for ARM. > Once this patch has been successfully reviewed, these routines should be used > instead. For now it makes sense to use the generic spinlocks version as a > placeholder. On ARMv6 the above is still faster. Nicolas ^ permalink raw reply [flat|nested] 28+ messages in thread
* [PATCH 3/5] arm: use the spinlocked, generic atomic64 support 2009-12-14 19:52 ` Nicolas Pitre @ 2009-12-15 10:24 ` Catalin Marinas 0 siblings, 0 replies; 28+ messages in thread From: Catalin Marinas @ 2009-12-15 10:24 UTC (permalink / raw) To: linux-arm-kernel On Mon, 2009-12-14 at 19:52 +0000, Nicolas Pitre wrote: > On Mon, 14 Dec 2009, Will Deacon wrote: > > *Nicolas Pitre wrote: > > > > > Can't a variant of include/linux/cnt32_to_63.h be used here?> > > > typedef struct { > > > atomic_t low; > > > u32 high; > > > } atomic64_t; > > > > > > static inline void atomic64_set(atomic64_t *ptr, u64 new_val) > > > { > > > u32 low = new_val; > > > u32 high = new_val >> 32; > > > BUG_ON(high & 0x80000000); > > > atomic_set(&ptr->low, low); > > > ptr->high = (high & 0x7fffffff) | (low & 0x80000000); > > > } > > > > How do you ensure that this is atomic? To me it looks like one CPU > > could write the lower 32-bits and another could write the upper 32, > > leaving the memory location in an inconsistent state. > > This one is indeed not exactly atomic. It isn't the interesting case > either though. Probably a strd could fix that. STRD is not guaranteed to be atomic either (it may be implemented as a succession of two word stores like an STM), especially when the low latency interrupt mode is enabled. -- Catalin ^ permalink raw reply [flat|nested] 28+ messages in thread
* [PATCH 2/5] arm/oprofile: reserve the PMU when starting 2009-12-14 14:04 ` [PATCH 2/5] arm/oprofile: reserve the PMU when starting Jamie Iles 2009-12-14 14:04 ` [PATCH 3/5] arm: use the spinlocked, generic atomic64 support Jamie Iles @ 2009-12-14 16:01 ` Jean Pihet 2009-12-14 16:04 ` Will Deacon 2 siblings, 0 replies; 28+ messages in thread From: Jean Pihet @ 2009-12-14 16:01 UTC (permalink / raw) To: linux-arm-kernel Hi, On Mon, 2009-12-14 at 14:04 +0000, Jamie Iles wrote: > Make sure that we have access to the performance counters and > that they aren't being used by perf events or anything else. > > Cc: Will Deacon <will.deacon@arm.com> > Signed-off-by: Jamie Iles <jamie.iles@picochip.com> > --- > arch/arm/oprofile/op_model_arm11_core.c | 4 +- > arch/arm/oprofile/op_model_arm11_core.h | 4 +- > arch/arm/oprofile/op_model_mpcore.c | 42 ++++++++++++++++-------------- > arch/arm/oprofile/op_model_v6.c | 33 ++++++++++++++---------- > arch/arm/oprofile/op_model_v7.c | 30 ++++++++++++++-------- > arch/arm/oprofile/op_model_v7.h | 4 +- I am OK with the changes for ARMv7. Regards, Jean > arch/arm/oprofile/op_model_xscale.c | 35 ++++++++++++++----------- > 7 files changed, 85 insertions(+), 67 deletions(-) > > diff --git a/arch/arm/oprofile/op_model_arm11_core.c b/arch/arm/oprofile/op_model_arm11_core.c > index ad80752..ef3e265 100644 > --- a/arch/arm/oprofile/op_model_arm11_core.c > +++ b/arch/arm/oprofile/op_model_arm11_core.c > @@ -132,7 +132,7 @@ static irqreturn_t arm11_pmu_interrupt(int irq, void *arg) > return IRQ_HANDLED; > } > > -int arm11_request_interrupts(int *irqs, int nr) > +int arm11_request_interrupts(const int *irqs, int nr) > { > unsigned int i; > int ret = 0; > @@ -153,7 +153,7 @@ int arm11_request_interrupts(int *irqs, int nr) > return ret; > } > > -void arm11_release_interrupts(int *irqs, int nr) > +void arm11_release_interrupts(const int *irqs, int nr) > { > unsigned int i; > > diff --git a/arch/arm/oprofile/op_model_arm11_core.h b/arch/arm/oprofile/op_model_arm11_core.h > index 6f8538e..1902b99 100644 > --- a/arch/arm/oprofile/op_model_arm11_core.h > +++ b/arch/arm/oprofile/op_model_arm11_core.h > @@ -39,7 +39,7 @@ > int arm11_setup_pmu(void); > int arm11_start_pmu(void); > int arm11_stop_pmu(void); > -int arm11_request_interrupts(int *, int); > -void arm11_release_interrupts(int *, int); > +int arm11_request_interrupts(const int *, int); > +void arm11_release_interrupts(const int *, int); > > #endif > diff --git a/arch/arm/oprofile/op_model_mpcore.c b/arch/arm/oprofile/op_model_mpcore.c > index 4ce0f98..f73ce87 100644 > --- a/arch/arm/oprofile/op_model_mpcore.c > +++ b/arch/arm/oprofile/op_model_mpcore.c > @@ -32,6 +32,7 @@ > /* #define DEBUG */ > #include <linux/types.h> > #include <linux/errno.h> > +#include <linux/err.h> > #include <linux/sched.h> > #include <linux/oprofile.h> > #include <linux/interrupt.h> > @@ -43,6 +44,7 @@ > #include <mach/hardware.h> > #include <mach/board-eb.h> > #include <asm/system.h> > +#include <asm/pmu.h> > > #include "op_counter.h" > #include "op_arm_model.h" > @@ -58,6 +60,7 @@ > * Bitmask of used SCU counters > */ > static unsigned int scu_em_used; > +static const struct pmu_irqs *pmu_irqs; > > /* > * 2 helper fns take a counter number from 0-7 (not the userspace-visible counter number) > @@ -225,33 +228,40 @@ static int em_setup_ctrs(void) > return 0; > } > > -static int arm11_irqs[] = { > - [0] = IRQ_EB11MP_PMU_CPU0, > - [1] = IRQ_EB11MP_PMU_CPU1, > - [2] = IRQ_EB11MP_PMU_CPU2, > - [3] = IRQ_EB11MP_PMU_CPU3 > -}; > - > static int em_start(void) > { > int ret; > > - ret = arm11_request_interrupts(arm11_irqs, ARRAY_SIZE(arm11_irqs)); > + pmu_irqs = reserve_pmu(); > + if (IS_ERR(pmu_irqs)) { > + ret = PTR_ERR(pmu_irqs); > + goto out; > + } > + > + ret = arm11_request_interrupts(pmu_irqs->irqs, pmu_irqs->num_irqs); > if (ret == 0) { > em_call_function(arm11_start_pmu); > > ret = scu_start(); > - if (ret) > - arm11_release_interrupts(arm11_irqs, ARRAY_SIZE(arm11_irqs)); > + if (ret) { > + arm11_release_interrupts(pmu_irqs->irqs, > + pmu_irqs->num_irqs); > + } else { > + release_pmu(pmu_irqs); > + pmu_irqs = NULL; > + } > } > + > +out: > return ret; > } > > static void em_stop(void) > { > em_call_function(arm11_stop_pmu); > - arm11_release_interrupts(arm11_irqs, ARRAY_SIZE(arm11_irqs)); > + arm11_release_interrupts(pmu_irqs->irqs, pmu_irqs->num_irqs); > scu_stop(); > + release_pmu(pmu_irqs); > } > > /* > @@ -283,15 +293,7 @@ static int em_setup(void) > em_route_irq(IRQ_EB11MP_PMU_SCU6, 3); > em_route_irq(IRQ_EB11MP_PMU_SCU7, 3); > > - /* > - * Send CP15 PMU interrupts to the owner CPU. > - */ > - em_route_irq(IRQ_EB11MP_PMU_CPU0, 0); > - em_route_irq(IRQ_EB11MP_PMU_CPU1, 1); > - em_route_irq(IRQ_EB11MP_PMU_CPU2, 2); > - em_route_irq(IRQ_EB11MP_PMU_CPU3, 3); > - > - return 0; > + return init_pmu(); > } > > struct op_arm_model_spec op_mpcore_spec = { > diff --git a/arch/arm/oprofile/op_model_v6.c b/arch/arm/oprofile/op_model_v6.c > index e468017..a22357a 100644 > --- a/arch/arm/oprofile/op_model_v6.c > +++ b/arch/arm/oprofile/op_model_v6.c > @@ -19,42 +19,47 @@ > /* #define DEBUG */ > #include <linux/types.h> > #include <linux/errno.h> > +#include <linux/err.h> > #include <linux/sched.h> > #include <linux/oprofile.h> > #include <linux/interrupt.h> > #include <asm/irq.h> > #include <asm/system.h> > +#include <asm/pmu.h> > > #include "op_counter.h" > #include "op_arm_model.h" > #include "op_model_arm11_core.h" > > -static int irqs[] = { > -#ifdef CONFIG_ARCH_OMAP2 > - 3, > -#endif > -#ifdef CONFIG_ARCH_BCMRING > - IRQ_PMUIRQ, /* for BCMRING, ARM PMU interrupt is 43 */ > -#endif > -#ifdef CONFIG_ARCH_PC3XX > - IRQ_NPMUIRQ, > -#endif > -}; > +static const struct pmu_irqs *pmu_irqs; > > static void armv6_pmu_stop(void) > { > arm11_stop_pmu(); > - arm11_release_interrupts(irqs, ARRAY_SIZE(irqs)); > + arm11_release_interrupts(pmu_irqs->irqs, pmu_irqs->num_irqs); > + release_pmu(pmu_irqs); > + pmu_irqs = NULL; > } > > static int armv6_pmu_start(void) > { > int ret; > > - ret = arm11_request_interrupts(irqs, ARRAY_SIZE(irqs)); > - if (ret >= 0) > + pmu_irqs = reserve_pmu(); > + if (IS_ERR(pmu_irqs)) { > + ret = PTR_ERR(pmu_irqs); > + goto out; > + } > + > + ret = arm11_request_interrupts(pmu_irqs->irqs, pmu_irqs->num_irqs); > + if (ret >= 0) { > ret = arm11_start_pmu(); > + } else { > + release_pmu(pmu_irqs); > + pmu_irqs = NULL; > + } > > +out: > return ret; > } > > diff --git a/arch/arm/oprofile/op_model_v7.c b/arch/arm/oprofile/op_model_v7.c > index f20295f..9258fca 100644 > --- a/arch/arm/oprofile/op_model_v7.c > +++ b/arch/arm/oprofile/op_model_v7.c > @@ -11,11 +11,14 @@ > */ > #include <linux/types.h> > #include <linux/errno.h> > +#include <linux/err.h> > #include <linux/oprofile.h> > #include <linux/interrupt.h> > #include <linux/irq.h> > #include <linux/smp.h> > > +#include <asm/pmu.h> > + > #include "op_counter.h" > #include "op_arm_model.h" > #include "op_model_v7.h" > @@ -299,7 +302,7 @@ static irqreturn_t armv7_pmnc_interrupt(int irq, void *arg) > return IRQ_HANDLED; > } > > -int armv7_request_interrupts(int *irqs, int nr) > +int armv7_request_interrupts(const int *irqs, int nr) > { > unsigned int i; > int ret = 0; > @@ -322,7 +325,7 @@ int armv7_request_interrupts(int *irqs, int nr) > return ret; > } > > -void armv7_release_interrupts(int *irqs, int nr) > +void armv7_release_interrupts(const int *irqs, int nr) > { > unsigned int i; > > @@ -366,12 +369,7 @@ static void armv7_pmnc_dump_regs(void) > } > #endif > > - > -static int irqs[] = { > -#ifdef CONFIG_ARCH_OMAP3 > - INT_34XX_BENCH_MPU_EMUL, > -#endif > -}; > +static const struct pmu_irqs *pmu_irqs; > > static void armv7_pmnc_stop(void) > { > @@ -379,19 +377,29 @@ static void armv7_pmnc_stop(void) > armv7_pmnc_dump_regs(); > #endif > armv7_stop_pmnc(); > - armv7_release_interrupts(irqs, ARRAY_SIZE(irqs)); > + armv7_release_interrupts(pmu_irqs->irqs, pmu_irqs->num_irqs); > + release_pmu(pmu_irqs); > + pmu_irqs = NULL; > } > > static int armv7_pmnc_start(void) > { > int ret; > > + pmu_irqs = reserve_pmu(); > + if (IS_ERR(pmu_irqs)) > + return PTR_ERR(pmu_irqs); > + > #ifdef DEBUG > armv7_pmnc_dump_regs(); > #endif > - ret = armv7_request_interrupts(irqs, ARRAY_SIZE(irqs)); > - if (ret >= 0) > + ret = armv7_request_interrupts(pmu_irqs->irqs, pmu_irqs->num_irqs); > + if (ret >= 0) { > armv7_start_pmnc(); > + } else { > + release_pmu(pmu_irqs); > + pmu_irqs = NULL; > + } > > return ret; > } > diff --git a/arch/arm/oprofile/op_model_v7.h b/arch/arm/oprofile/op_model_v7.h > index 0e19bcc..9ca334b 100644 > --- a/arch/arm/oprofile/op_model_v7.h > +++ b/arch/arm/oprofile/op_model_v7.h > @@ -97,7 +97,7 @@ > int armv7_setup_pmu(void); > int armv7_start_pmu(void); > int armv7_stop_pmu(void); > -int armv7_request_interrupts(int *, int); > -void armv7_release_interrupts(int *, int); > +int armv7_request_interrupts(const int *, int); > +void armv7_release_interrupts(const int *, int); > > #endif > diff --git a/arch/arm/oprofile/op_model_xscale.c b/arch/arm/oprofile/op_model_xscale.c > index 724ab9c..1d34a02 100644 > --- a/arch/arm/oprofile/op_model_xscale.c > +++ b/arch/arm/oprofile/op_model_xscale.c > @@ -17,12 +17,14 @@ > /* #define DEBUG */ > #include <linux/types.h> > #include <linux/errno.h> > +#include <linux/err.h> > #include <linux/sched.h> > #include <linux/oprofile.h> > #include <linux/interrupt.h> > #include <linux/irq.h> > > #include <asm/cputype.h> > +#include <asm/pmu.h> > > #include "op_counter.h" > #include "op_arm_model.h" > @@ -33,17 +35,6 @@ > #define PMU_RESET (CCNT_RESET | PMN_RESET) > #define PMU_CNT64 0x008 /* Make CCNT count every 64th cycle */ > > -/* TODO do runtime detection */ > -#ifdef CONFIG_ARCH_IOP32X > -#define XSCALE_PMU_IRQ IRQ_IOP32X_CORE_PMU > -#endif > -#ifdef CONFIG_ARCH_IOP33X > -#define XSCALE_PMU_IRQ IRQ_IOP33X_CORE_PMU > -#endif > -#ifdef CONFIG_ARCH_PXA > -#define XSCALE_PMU_IRQ IRQ_PMU > -#endif > - > /* > * Different types of events that can be counted by the XScale PMU > * as used by Oprofile userspace. Here primarily for documentation > @@ -367,6 +358,8 @@ static irqreturn_t xscale_pmu_interrupt(int irq, void *arg) > return IRQ_HANDLED; > } > > +static const struct pmu_irqs *pmu_irqs; > + > static void xscale_pmu_stop(void) > { > u32 pmnc = read_pmnc(); > @@ -374,20 +367,30 @@ static void xscale_pmu_stop(void) > pmnc &= ~PMU_ENABLE; > write_pmnc(pmnc); > > - free_irq(XSCALE_PMU_IRQ, results); > + free_irq(pmu_irqs->irqs[0], results); > + release_pmu(pmu_irqs); > + pmu_irqs = NULL; > } > > static int xscale_pmu_start(void) > { > int ret; > - u32 pmnc = read_pmnc(); > + u32 pmnc; > + > + pmu_irqs = reserve_pmu(); > + if (IS_ERR(pmu_irqs)) > + return PTR_ERR(pmu_irqs); > + > + pmnc = read_pmnc(); > > - ret = request_irq(XSCALE_PMU_IRQ, xscale_pmu_interrupt, IRQF_DISABLED, > - "XScale PMU", (void *)results); > + ret = request_irq(pmu_irqs->irqs[0], xscale_pmu_interrupt, > + IRQF_DISABLED, "XScale PMU", (void *)results); > > if (ret < 0) { > printk(KERN_ERR "oprofile: unable to request IRQ%d for XScale PMU\n", > - XSCALE_PMU_IRQ); > + pmu_irqs->irqs[0]); > + release_pmu(pmu_irqs); > + pmu_irqs = NULL; > return ret; > } > ^ permalink raw reply [flat|nested] 28+ messages in thread
* [PATCH 2/5] arm/oprofile: reserve the PMU when starting 2009-12-14 14:04 ` [PATCH 2/5] arm/oprofile: reserve the PMU when starting Jamie Iles 2009-12-14 14:04 ` [PATCH 3/5] arm: use the spinlocked, generic atomic64 support Jamie Iles 2009-12-14 16:01 ` [PATCH 2/5] arm/oprofile: reserve the PMU when starting Jean Pihet @ 2009-12-14 16:04 ` Will Deacon 2009-12-14 16:10 ` Jamie Iles 2 siblings, 1 reply; 28+ messages in thread From: Will Deacon @ 2009-12-14 16:04 UTC (permalink / raw) To: linux-arm-kernel * Jamie Iles wrote: > Make sure that we have access to the performance counters and > that they aren't being used by perf events or anything else. > > diff --git a/arch/arm/oprofile/op_model_v6.c b/arch/arm/oprofile/op_model_v6.c > index e468017..a22357a 100644 > --- a/arch/arm/oprofile/op_model_v6.c > +++ b/arch/arm/oprofile/op_model_v6.c > @@ -19,42 +19,47 @@ > /* #define DEBUG */ > #include <linux/types.h> > #include <linux/errno.h> > +#include <linux/err.h> > #include <linux/sched.h> > #include <linux/oprofile.h> > #include <linux/interrupt.h> > #include <asm/irq.h> > #include <asm/system.h> > +#include <asm/pmu.h> > > #include "op_counter.h" > #include "op_arm_model.h" > #include "op_model_arm11_core.h" > > -static int irqs[] = { > -#ifdef CONFIG_ARCH_OMAP2 > - 3, > -#endif > -#ifdef CONFIG_ARCH_BCMRING > - IRQ_PMUIRQ, /* for BCMRING, ARM PMU interrupt is 43 */ > -#endif > -#ifdef CONFIG_ARCH_PC3XX > - IRQ_NPMUIRQ, > -#endif <snip> These last three lines don't apply cleanly. I think you've based this patch on top of your previous one. Will ^ permalink raw reply [flat|nested] 28+ messages in thread
* [PATCH 2/5] arm/oprofile: reserve the PMU when starting 2009-12-14 16:04 ` Will Deacon @ 2009-12-14 16:10 ` Jamie Iles 0 siblings, 0 replies; 28+ messages in thread From: Jamie Iles @ 2009-12-14 16:10 UTC (permalink / raw) To: linux-arm-kernel On Mon, Dec 14, 2009 at 04:04:25PM -0000, Will Deacon wrote: > > -static int irqs[] = { > > -#ifdef CONFIG_ARCH_OMAP2 > > - 3, > > -#endif > > -#ifdef CONFIG_ARCH_BCMRING > > - IRQ_PMUIRQ, /* for BCMRING, ARM PMU interrupt is 43 */ > > -#endif > > -#ifdef CONFIG_ARCH_PC3XX > > - IRQ_NPMUIRQ, > > -#endif > <snip> > > These last three lines don't apply cleanly. > I think you've based this patch on top of your previous one. Apologies, this is is from a platform (pc3xx) that isn't in mainline. I've been using this platform to test the perf events code. I'll submit the next lot directly off of tip/master. Jamie ^ permalink raw reply [flat|nested] 28+ messages in thread
* [PATCH 1/5] arm: provide a mechanism to reserve performance counters 2009-12-14 14:04 ` [PATCH 1/5] arm: provide a mechanism to reserve performance counters Jamie Iles 2009-12-14 14:04 ` [PATCH 2/5] arm/oprofile: reserve the PMU when starting Jamie Iles @ 2009-12-14 14:39 ` Will Deacon 2009-12-14 15:03 ` Jamie Iles 2009-12-14 16:01 ` Jean Pihet 2 siblings, 1 reply; 28+ messages in thread From: Will Deacon @ 2009-12-14 14:39 UTC (permalink / raw) To: linux-arm-kernel * Jamie Iles wrote: > To add support for perf events and to allow the hardware > counters to be shared with oprofile, we need a way to reserve > access to the pmu (performance monitor unit). Hi Jamie, this is looking good. It's nice to see the IRQ stuff moving out of oprofile. Comments are inline. > diff --git a/arch/arm/include/asm/pmu.h b/arch/arm/include/asm/pmu.h > new file mode 100644 > index 0000000..d66a7cd > --- /dev/null > +++ b/arch/arm/include/asm/pmu.h > @@ -0,0 +1,76 @@ > +/* > + * linux/arch/arm/include/asm/pmu.h > + * > + * Copyright (C) 2009 picoChip Designs Ltd, Jamie Iles > + * > + * This program is free software; you can redistribute it and/or modify > + * it under the terms of the GNU General Public License version 2 as > + * published by the Free Software Foundation. > + * > + */ > + > +#ifndef __ARM_PMU_H__ > +#define __ARM_PMU_H__ > + > +#ifdef CONFIG_CPU_HAS_PMU > + > +#define MAX_PMU_IRQS 8 > + > +struct pmu_irqs { > + int irqs[MAX_PMU_IRQS]; > + unsigned num_irqs; > +}; Since we're populating this struct at compile time anyway, could we make it an array and use the ARRAY_SIZE macro to get the number of irqs? This would also mean that MAX_PMU_IRQS could be removed. > diff --git a/arch/arm/kernel/pmu.c b/arch/arm/kernel/pmu.c > new file mode 100644 > index 0000000..881e526 > --- /dev/null > +++ b/arch/arm/kernel/pmu.c <snip> > +void > +release_pmu(const struct pmu_irqs *irqs) > +{ > + WARN_ON(irqs != &pmu_irqs); > + up(&pmu_mutex); > +} > +EXPORT_SYMBOL_GPL(release_pmu); I think it would be better to allow release to fail and do so if the irqs don't match, otherwise a malicious oprofile module could release on behalf of perf :). > +static void > +set_irq_affinity(int irq, > + unsigned int cpu) > +{ > +#ifdef CONFIG_SMP > + struct irq_desc *desc = irq_desc + irq; > + const struct cpumask *mask = cpumask_of(cpu); > + unsigned long flags; > + > + raw_spin_lock_irqsave(&desc->lock, flags); > + cpumask_copy(desc->affinity, mask); > + desc->chip->set_affinity(irq, mask); > + raw_spin_unlock_irqrestore(&desc->lock, flags); > +#endif > +} Why not use irq_set_affinity(irq, cpumask_of(cpu))? This function isn't exported, but I don't envisage building the pmu as a module. > diff --git a/arch/arm/mm/Kconfig b/arch/arm/mm/Kconfig > index dd4698c..fc5c05b 100644 > --- a/arch/arm/mm/Kconfig > +++ b/arch/arm/mm/Kconfig > @@ -342,6 +342,7 @@ config CPU_XSCALE > select CPU_PABRT_LEGACY > select CPU_CACHE_VIVT > select CPU_CP15_MMU > + select CPU_HAS_PMU > select CPU_TLB_V4WBI if MMU > > # XScale Core Version 3 > @@ -398,6 +399,7 @@ config CPU_V6 > select CPU_HAS_ASID if MMU > select CPU_COPY_V6 if MMU > select CPU_TLB_V6 if MMU > + select CPU_HAS_PMU > > # ARMv6k > config CPU_32v6K > @@ -421,6 +423,7 @@ config CPU_V7 > select CPU_CACHE_V7 > select CPU_CACHE_VIPT > select CPU_CP15_MMU > + select CPU_HAS_PMU > select CPU_HAS_ASID if MMU > select CPU_COPY_V6 if MMU > select CPU_TLB_V7 if MMU > @@ -536,6 +539,9 @@ config CPU_COPY_FA > config CPU_COPY_V6 > bool > > +config CPU_HAS_PMU > + bool I think all v6 cores and above have a PMU, so you could set the bool based on that (and add the exceptional cases like xscale). I've got a quad-core pb11mp box so once this is settled I'll give it a test in an SMP environment. Cheers, Will ^ permalink raw reply [flat|nested] 28+ messages in thread
* [PATCH 1/5] arm: provide a mechanism to reserve performance counters 2009-12-14 14:39 ` [PATCH 1/5] arm: provide a mechanism to reserve performance counters Will Deacon @ 2009-12-14 15:03 ` Jamie Iles 0 siblings, 0 replies; 28+ messages in thread From: Jamie Iles @ 2009-12-14 15:03 UTC (permalink / raw) To: linux-arm-kernel Hi Will, Thanks for your feedback, comments inline. Jamie On Mon, Dec 14, 2009 at 02:39:59PM -0000, Will Deacon wrote: > > +#define MAX_PMU_IRQS 8 > > + > > +struct pmu_irqs { > > + int irqs[MAX_PMU_IRQS]; > > + unsigned num_irqs; > > +}; > > Since we're populating this struct at compile time anyway, could we make it > an array and use the ARRAY_SIZE macro to get the number of irqs? This would > also mean that MAX_PMU_IRQS could be removed. Ok, good plan. > > diff --git a/arch/arm/kernel/pmu.c b/arch/arm/kernel/pmu.c > > new file mode 100644 > > index 0000000..881e526 > > --- /dev/null > > +++ b/arch/arm/kernel/pmu.c > <snip> > > +void > > +release_pmu(const struct pmu_irqs *irqs) > > +{ > > + WARN_ON(irqs != &pmu_irqs); > > + up(&pmu_mutex); > > +} > > +EXPORT_SYMBOL_GPL(release_pmu); > > I think it would be better to allow release to fail and do so if the irqs > don't match, otherwise a malicious oprofile module could release on behalf of > perf :). Ok, that sounds reasonable. I'll make release_pmu() return an int, but I doubt that it's recoverable by any of the users! > > > +static void > > +set_irq_affinity(int irq, > > + unsigned int cpu) > > +{ > > +#ifdef CONFIG_SMP > > + struct irq_desc *desc = irq_desc + irq; > > + const struct cpumask *mask = cpumask_of(cpu); > > + unsigned long flags; > > + > > + raw_spin_lock_irqsave(&desc->lock, flags); > > + cpumask_copy(desc->affinity, mask); > > + desc->chip->set_affinity(irq, mask); > > + raw_spin_unlock_irqrestore(&desc->lock, flags); > > +#endif > > +} > > Why not use irq_set_affinity(irq, cpumask_of(cpu))? > This function isn't exported, but I don't envisage building the pmu > as a module. Because I moved the code from oprofile ;-) irq_set_affinity() looks like a better option so I'll use that for the next revision. > I think all v6 cores and above have a PMU, so you could set the bool based > on that (and add the exceptional cases like xscale). Ok, I wasn't sure if that was the case but if so then that's a sensible change. ^ permalink raw reply [flat|nested] 28+ messages in thread
* [PATCH 1/5] arm: provide a mechanism to reserve performance counters 2009-12-14 14:04 ` [PATCH 1/5] arm: provide a mechanism to reserve performance counters Jamie Iles 2009-12-14 14:04 ` [PATCH 2/5] arm/oprofile: reserve the PMU when starting Jamie Iles 2009-12-14 14:39 ` [PATCH 1/5] arm: provide a mechanism to reserve performance counters Will Deacon @ 2009-12-14 16:01 ` Jean Pihet 2 siblings, 0 replies; 28+ messages in thread From: Jean Pihet @ 2009-12-14 16:01 UTC (permalink / raw) To: linux-arm-kernel Hi, I am OK with this code. It is good to have such a reservation mechanism. Regards, Jean On Mon, 2009-12-14 at 14:04 +0000, Jamie Iles wrote: > To add support for perf events and to allow the hardware > counters to be shared with oprofile, we need a way to reserve > access to the pmu (performance monitor unit). > > Cc: Will Deacon <will.deacon@arm.com> > Signed-off-by: Jamie Iles <jamie.iles@picochip.com> > --- > arch/arm/include/asm/pmu.h | 76 +++++++++++++++++++++++++++++++ > arch/arm/kernel/Makefile | 1 + > arch/arm/kernel/pmu.c | 108 ++++++++++++++++++++++++++++++++++++++++++++ > arch/arm/mm/Kconfig | 6 +++ > 4 files changed, 191 insertions(+), 0 deletions(-) > create mode 100644 arch/arm/include/asm/pmu.h > create mode 100644 arch/arm/kernel/pmu.c > > diff --git a/arch/arm/include/asm/pmu.h b/arch/arm/include/asm/pmu.h > new file mode 100644 > index 0000000..d66a7cd > --- /dev/null > +++ b/arch/arm/include/asm/pmu.h > @@ -0,0 +1,76 @@ > +/* > + * linux/arch/arm/include/asm/pmu.h > + * > + * Copyright (C) 2009 picoChip Designs Ltd, Jamie Iles > + * > + * This program is free software; you can redistribute it and/or modify > + * it under the terms of the GNU General Public License version 2 as > + * published by the Free Software Foundation. > + * > + */ > + > +#ifndef __ARM_PMU_H__ > +#define __ARM_PMU_H__ > + > +#ifdef CONFIG_CPU_HAS_PMU > + > +#define MAX_PMU_IRQS 8 > + > +struct pmu_irqs { > + int irqs[MAX_PMU_IRQS]; > + unsigned num_irqs; > +}; > + > +/** > + * reserve_pmu() - reserve the hardware performance counters > + * > + * Reserve the hardware performance counters in the system for exclusive use. > + * The 'struct pmu_irqs' for the system is returned on success, ERR_PTR() > + * encoded error on failure. > + */ > +extern const struct pmu_irqs * > +reserve_pmu(void); > + > +/** > + * release_pmu() - Relinquish control of the performance counters > + * > + * Release the performance counters and allow someone else to use them. > + * Callers must have disabled the counters and released IRQs before calling > + * this. The 'struct pmu_irqs' returned from reserve_pmu() must be passed as > + * a cookie. > + */ > +extern void > +release_pmu(const struct pmu_irqs *irqs); > + > +/** > + * init_pmu() - Initialise the PMU. > + * > + * Initialise the system ready for PMU enabling. This should typically set the > + * IRQ affinity and nothing else. The users (oprofile/perf events etc) will do > + * the actual hardware initialisation. > + */ > +extern int > +init_pmu(void); > + > +#else /* CONFIG_CPU_HAS_PMU */ > + > +static inline const struct pmu_irqs * > +reserve_pmu(void) > +{ > + ERR_PTR(-ENODEV); > +} > + > +static inline void > +release_pmu(const struct pmu_irqs *irqs) > +{ > +} > + > +static inline int > +init_pmu(void) > +{ > + return -ENODEV; > +} > + > +#endif /* CONFIG_CPU_HAS_PMU */ > + > +#endif /* __ARM_PMU_H__ */ > diff --git a/arch/arm/kernel/Makefile b/arch/arm/kernel/Makefile > index e7ccf7e..286a276 100644 > --- a/arch/arm/kernel/Makefile > +++ b/arch/arm/kernel/Makefile > @@ -46,6 +46,7 @@ obj-$(CONFIG_CPU_XSCALE) += xscale-cp0.o > obj-$(CONFIG_CPU_XSC3) += xscale-cp0.o > obj-$(CONFIG_CPU_MOHAWK) += xscale-cp0.o > obj-$(CONFIG_IWMMXT) += iwmmxt.o > +obj-$(CONFIG_CPU_HAS_PMU) += pmu.o > AFLAGS_iwmmxt.o := -Wa,-mcpu=iwmmxt > > ifneq ($(CONFIG_ARCH_EBSA110),y) > diff --git a/arch/arm/kernel/pmu.c b/arch/arm/kernel/pmu.c > new file mode 100644 > index 0000000..881e526 > --- /dev/null > +++ b/arch/arm/kernel/pmu.c > @@ -0,0 +1,108 @@ > +/* > + * linux/arch/arm/kernel/pmu.c > + * > + * Copyright (C) 2009 picoChip Designs Ltd, Jamie Iles > + * > + * This program is free software; you can redistribute it and/or modify > + * it under the terms of the GNU General Public License version 2 as > + * published by the Free Software Foundation. > + * > + */ > + > +#include <linux/kernel.h> > +#include <linux/module.h> > +#include <linux/semaphore.h> > +#include <linux/err.h> > +#include <linux/irq.h> > + > +#include <asm/pmu.h> > +#include <asm/irq.h> > + > +/* > + * Define the IRQs for the system. We could use something like a platform > + * device but that seems fairly heavyweight for this. Also, the performance > + * counters can't be removed or hotplugged. > + * > + * Ordering is important: init_pmu() will use the ordering to set the affinity > + * to the corresponding core. e.g. the first interrupt will go to cpu 0, the > + * second goes to cpu 1 etc. > + */ > +static const struct pmu_irqs pmu_irqs = { > +#ifdef CONFIG_ARCH_PC3XX > + .irqs = { IRQ_NPMUIRQ }, > + .num_irqs = 1, > +#elif defined(CONFIG_ARCH_OMAP2) > + .irqs = { 3 }, > + .num_irqs = 1, > +#elif defined(CONFIG_ARCH_BCMRING) > + .irqs = { IRQ_PMUIRQ }, > + .num_irqs = 1, > +#elif defined(CONFIG_MACH_REALVIEW_EB) > + .irqs = { > + [0] = IRQ_EB11MP_PMU_CPU0, > + [1] = IRQ_EB11MP_PMU_CPU1, > + [2] = IRQ_EB11MP_PMU_CPU2, > + [3] = IRQ_EB11MP_PMU_CPU3 > + }, > + .num_irqs = 4, > +#elif defined(CONFIG_ARCH_OMAP3) > + .irqs = { INT_34XX_BENCH_MPU_EMUL }, > + .num_irqs = 1, > +#elif defined(CONFIG_ARCH_IOP32X) > + .irqs = { IRQ_IOP32X_CORE_PMU }, > + .num_irqs = 1, > +#elif defined(CONFIG_ARCH_IOP33X) > + .irqs = { IRQ_IOP33X_CORE_PMU }, > + .num_irqs = 1, > +#elif defined(CONFIG_ARCH_PXA) > + .irqs = { IRQ_PMU }, > + .num_irqs = 1, > +#endif > +}; > + > +static DECLARE_MUTEX(pmu_mutex); > + > +const struct pmu_irqs * > +reserve_pmu(void) > +{ > + int ret = down_trylock(&pmu_mutex) ? -EBUSY : 0; > + > + return ret ? ERR_PTR(ret) : &pmu_irqs; > +} > +EXPORT_SYMBOL_GPL(reserve_pmu); > + > +void > +release_pmu(const struct pmu_irqs *irqs) > +{ > + WARN_ON(irqs != &pmu_irqs); > + up(&pmu_mutex); > +} > +EXPORT_SYMBOL_GPL(release_pmu); > + > +static void > +set_irq_affinity(int irq, > + unsigned int cpu) > +{ > +#ifdef CONFIG_SMP > + struct irq_desc *desc = irq_desc + irq; > + const struct cpumask *mask = cpumask_of(cpu); > + unsigned long flags; > + > + raw_spin_lock_irqsave(&desc->lock, flags); > + cpumask_copy(desc->affinity, mask); > + desc->chip->set_affinity(irq, mask); > + raw_spin_unlock_irqrestore(&desc->lock, flags); > +#endif > +} > + > +int > +init_pmu(void) > +{ > + int i; > + > + for (i = 0; i < pmu_irqs.num_irqs; ++i) > + set_irq_affinity(pmu_irqs.irqs[i], i); > + > + return 0; > +} > +EXPORT_SYMBOL_GPL(init_pmu); > diff --git a/arch/arm/mm/Kconfig b/arch/arm/mm/Kconfig > index dd4698c..fc5c05b 100644 > --- a/arch/arm/mm/Kconfig > +++ b/arch/arm/mm/Kconfig > @@ -342,6 +342,7 @@ config CPU_XSCALE > select CPU_PABRT_LEGACY > select CPU_CACHE_VIVT > select CPU_CP15_MMU > + select CPU_HAS_PMU > select CPU_TLB_V4WBI if MMU > > # XScale Core Version 3 > @@ -398,6 +399,7 @@ config CPU_V6 > select CPU_HAS_ASID if MMU > select CPU_COPY_V6 if MMU > select CPU_TLB_V6 if MMU > + select CPU_HAS_PMU > > # ARMv6k > config CPU_32v6K > @@ -421,6 +423,7 @@ config CPU_V7 > select CPU_CACHE_V7 > select CPU_CACHE_VIPT > select CPU_CP15_MMU > + select CPU_HAS_PMU > select CPU_HAS_ASID if MMU > select CPU_COPY_V6 if MMU > select CPU_TLB_V7 if MMU > @@ -536,6 +539,9 @@ config CPU_COPY_FA > config CPU_COPY_V6 > bool > > +config CPU_HAS_PMU > + bool > + > # This selects the TLB model > config CPU_TLB_V3 > bool ^ permalink raw reply [flat|nested] 28+ messages in thread
* ARMv6 performance counters v3 @ 2009-12-15 11:15 Jamie Iles 2009-12-15 11:15 ` [PATCH 1/5] arm: provide a mechanism to reserve performance counters Jamie Iles 0 siblings, 1 reply; 28+ messages in thread From: Jamie Iles @ 2009-12-15 11:15 UTC (permalink / raw) To: linux-arm-kernel Okay, here's the 3rd attempt at support for hardware perf events support on ARMv6. After feedback from Jean and Will, I've added an arm_pmu structure that allows other CPU versions to be supported in the future. At the moment only ARMv6 is supported in hardware. Thanks to Will and Jean for their comments. ^ permalink raw reply [flat|nested] 28+ messages in thread
* [PATCH 1/5] arm: provide a mechanism to reserve performance counters 2009-12-15 11:15 ARMv6 performance counters v3 Jamie Iles @ 2009-12-15 11:15 ` Jamie Iles 2009-12-15 11:15 ` [PATCH 2/5] arm/oprofile: reserve the PMU when starting Jamie Iles 0 siblings, 1 reply; 28+ messages in thread From: Jamie Iles @ 2009-12-15 11:15 UTC (permalink / raw) To: linux-arm-kernel To add support for perf events and to allow the hardware counters to be shared with oprofile, we need a way to reserve access to the pmu (performance monitor unit). Cc: Will Deacon <will.deacon@arm.com> Signed-off-by: Jamie Iles <jamie.iles@picochip.com> --- arch/arm/include/asm/pmu.h | 74 ++++++++++++++++++++++++++++++ arch/arm/kernel/Makefile | 1 + arch/arm/kernel/pmu.c | 108 ++++++++++++++++++++++++++++++++++++++++++++ arch/arm/mm/Kconfig | 5 ++ 4 files changed, 188 insertions(+), 0 deletions(-) create mode 100644 arch/arm/include/asm/pmu.h create mode 100644 arch/arm/kernel/pmu.c diff --git a/arch/arm/include/asm/pmu.h b/arch/arm/include/asm/pmu.h new file mode 100644 index 0000000..e7cc264 --- /dev/null +++ b/arch/arm/include/asm/pmu.h @@ -0,0 +1,74 @@ +/* + * linux/arch/arm/include/asm/pmu.h + * + * Copyright (C) 2009 picoChip Designs Ltd, Jamie Iles + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + */ + +#ifndef __ARM_PMU_H__ +#define __ARM_PMU_H__ + +#ifdef CONFIG_CPU_HAS_PMU + +struct pmu_irqs { + const int *irqs; + unsigned num_irqs; +}; + +/** + * reserve_pmu() - reserve the hardware performance counters + * + * Reserve the hardware performance counters in the system for exclusive use. + * The 'struct pmu_irqs' for the system is returned on success, ERR_PTR() + * encoded error on failure. + */ +extern const struct pmu_irqs * +reserve_pmu(void); + +/** + * release_pmu() - Relinquish control of the performance counters + * + * Release the performance counters and allow someone else to use them. + * Callers must have disabled the counters and released IRQs before calling + * this. The 'struct pmu_irqs' returned from reserve_pmu() must be passed as + * a cookie. + */ +extern int +release_pmu(const struct pmu_irqs *irqs); + +/** + * init_pmu() - Initialise the PMU. + * + * Initialise the system ready for PMU enabling. This should typically set the + * IRQ affinity and nothing else. The users (oprofile/perf events etc) will do + * the actual hardware initialisation. + */ +extern int +init_pmu(void); + +#else /* CONFIG_CPU_HAS_PMU */ + +static inline const struct pmu_irqs * +reserve_pmu(void) +{ + ERR_PTR(-ENODEV); +} + +static inline int +release_pmu(const struct pmu_irqs *irqs) +{ +} + +static inline int +init_pmu(void) +{ + return -ENODEV; +} + +#endif /* CONFIG_CPU_HAS_PMU */ + +#endif /* __ARM_PMU_H__ */ diff --git a/arch/arm/kernel/Makefile b/arch/arm/kernel/Makefile index dd00f74..216890d 100644 --- a/arch/arm/kernel/Makefile +++ b/arch/arm/kernel/Makefile @@ -46,6 +46,7 @@ obj-$(CONFIG_CPU_XSCALE) += xscale-cp0.o obj-$(CONFIG_CPU_XSC3) += xscale-cp0.o obj-$(CONFIG_CPU_MOHAWK) += xscale-cp0.o obj-$(CONFIG_IWMMXT) += iwmmxt.o +obj-$(CONFIG_CPU_HAS_PMU) += pmu.o AFLAGS_iwmmxt.o := -Wa,-mcpu=iwmmxt ifneq ($(CONFIG_ARCH_EBSA110),y) diff --git a/arch/arm/kernel/pmu.c b/arch/arm/kernel/pmu.c new file mode 100644 index 0000000..3a178bb --- /dev/null +++ b/arch/arm/kernel/pmu.c @@ -0,0 +1,108 @@ +/* + * linux/arch/arm/kernel/pmu.c + * + * Copyright (C) 2009 picoChip Designs Ltd, Jamie Iles + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + */ + +#include <linux/cpumask.h> +#include <linux/err.h> +#include <linux/interrupt.h> +#include <linux/kernel.h> +#include <linux/module.h> +#include <linux/semaphore.h> + +#include <asm/pmu.h> + +/* + * Define the IRQs for the system. We could use something like a platform + * device but that seems fairly heavyweight for this. Also, the performance + * counters can't be removed or hotplugged. + * + * Ordering is important: init_pmu() will use the ordering to set the affinity + * to the corresponding core. e.g. the first interrupt will go to cpu 0, the + * second goes to cpu 1 etc. + */ +static const int irqs[] = { +#ifdef CONFIG_ARCH_PC3XX + IRQ_NPMUIRQ, +#elif defined(CONFIG_ARCH_OMAP2) + 3, +#elif defined(CONFIG_ARCH_BCMRING) + IRQ_PMUIRQ, +#elif defined(CONFIG_MACH_REALVIEW_EB) + IRQ_EB11MP_PMU_CPU0, + IRQ_EB11MP_PMU_CPU1, + IRQ_EB11MP_PMU_CPU2, + IRQ_EB11MP_PMU_CPU3, +#elif defined(CONFIG_ARCH_OMAP3) + INT_34XX_BENCH_MPU_EMUL, +#elif defined(CONFIG_ARCH_IOP32X) + IRQ_IOP32X_CORE_PMU, +#elif defined(CONFIG_ARCH_IOP33X) + IRQ_IOP33X_CORE_PMU, +#elif defined(CONFIG_ARCH_PXA) + IRQ_PMU, +#endif +}; + +static const struct pmu_irqs pmu_irqs = { + .irqs = irqs, + .num_irqs = ARRAY_SIZE(irqs), +}; + +static DECLARE_MUTEX(pmu_mutex); + +const struct pmu_irqs * +reserve_pmu(void) +{ + int ret = down_trylock(&pmu_mutex) ? -EBUSY : 0; + + return ret ? ERR_PTR(ret) : &pmu_irqs; +} +EXPORT_SYMBOL_GPL(reserve_pmu); + +int +release_pmu(const struct pmu_irqs *irqs) +{ + if (WARN_ON(irqs != &pmu_irqs)) + return -EINVAL; + up(&pmu_mutex); + return 0; +} +EXPORT_SYMBOL_GPL(release_pmu); + +static int +set_irq_affinity(int irq, + unsigned int cpu) +{ +#ifdef CONFIG_SMP + int err = irq_set_affinity(irq, cpumask_of(cpu)); + if (err) + pr_warning("unable to set irq affinity (irq=%d, cpu=%u)\n", + irq, cpu); + return err; +#else + return 0; +#endif +} + +int +init_pmu(void) +{ + int i; + int err = 0; + + for (i = 0; i < pmu_irqs.num_irqs; ++i) { + err = set_irq_affinity(pmu_irqs.irqs[i], i); + if (err) + break; + } + + return err; +} +EXPORT_SYMBOL_GPL(init_pmu); diff --git a/arch/arm/mm/Kconfig b/arch/arm/mm/Kconfig index dd4698c..5cd0ec4 100644 --- a/arch/arm/mm/Kconfig +++ b/arch/arm/mm/Kconfig @@ -536,6 +536,11 @@ config CPU_COPY_FA config CPU_COPY_V6 bool +config CPU_HAS_PMU + depends on CPU_V6 || CPU_V7 || CPU_XSCALE + default y + bool + # This selects the TLB model config CPU_TLB_V3 bool -- 1.6.5.4 ^ permalink raw reply related [flat|nested] 28+ messages in thread
* [PATCH 2/5] arm/oprofile: reserve the PMU when starting 2009-12-15 11:15 ` [PATCH 1/5] arm: provide a mechanism to reserve performance counters Jamie Iles @ 2009-12-15 11:15 ` Jamie Iles 2009-12-15 11:15 ` [PATCH 3/5] arm: use the spinlocked, generic atomic64 support Jamie Iles 0 siblings, 1 reply; 28+ messages in thread From: Jamie Iles @ 2009-12-15 11:15 UTC (permalink / raw) To: linux-arm-kernel Make sure that we have access to the performance counters and that they aren't being used by perf events or anything else. Cc: Will Deacon <will.deacon@arm.com> Signed-off-by: Jamie Iles <jamie.iles@picochip.com> --- arch/arm/oprofile/op_model_arm11_core.c | 4 +- arch/arm/oprofile/op_model_arm11_core.h | 4 +- arch/arm/oprofile/op_model_mpcore.c | 42 ++++++++++++++++-------------- arch/arm/oprofile/op_model_v6.c | 30 ++++++++++++++-------- arch/arm/oprofile/op_model_v7.c | 30 ++++++++++++++-------- arch/arm/oprofile/op_model_v7.h | 4 +- arch/arm/oprofile/op_model_xscale.c | 35 ++++++++++++++----------- 7 files changed, 85 insertions(+), 64 deletions(-) diff --git a/arch/arm/oprofile/op_model_arm11_core.c b/arch/arm/oprofile/op_model_arm11_core.c index ad80752..ef3e265 100644 --- a/arch/arm/oprofile/op_model_arm11_core.c +++ b/arch/arm/oprofile/op_model_arm11_core.c @@ -132,7 +132,7 @@ static irqreturn_t arm11_pmu_interrupt(int irq, void *arg) return IRQ_HANDLED; } -int arm11_request_interrupts(int *irqs, int nr) +int arm11_request_interrupts(const int *irqs, int nr) { unsigned int i; int ret = 0; @@ -153,7 +153,7 @@ int arm11_request_interrupts(int *irqs, int nr) return ret; } -void arm11_release_interrupts(int *irqs, int nr) +void arm11_release_interrupts(const int *irqs, int nr) { unsigned int i; diff --git a/arch/arm/oprofile/op_model_arm11_core.h b/arch/arm/oprofile/op_model_arm11_core.h index 6f8538e..1902b99 100644 --- a/arch/arm/oprofile/op_model_arm11_core.h +++ b/arch/arm/oprofile/op_model_arm11_core.h @@ -39,7 +39,7 @@ int arm11_setup_pmu(void); int arm11_start_pmu(void); int arm11_stop_pmu(void); -int arm11_request_interrupts(int *, int); -void arm11_release_interrupts(int *, int); +int arm11_request_interrupts(const int *, int); +void arm11_release_interrupts(const int *, int); #endif diff --git a/arch/arm/oprofile/op_model_mpcore.c b/arch/arm/oprofile/op_model_mpcore.c index 4ce0f98..f73ce87 100644 --- a/arch/arm/oprofile/op_model_mpcore.c +++ b/arch/arm/oprofile/op_model_mpcore.c @@ -32,6 +32,7 @@ /* #define DEBUG */ #include <linux/types.h> #include <linux/errno.h> +#include <linux/err.h> #include <linux/sched.h> #include <linux/oprofile.h> #include <linux/interrupt.h> @@ -43,6 +44,7 @@ #include <mach/hardware.h> #include <mach/board-eb.h> #include <asm/system.h> +#include <asm/pmu.h> #include "op_counter.h" #include "op_arm_model.h" @@ -58,6 +60,7 @@ * Bitmask of used SCU counters */ static unsigned int scu_em_used; +static const struct pmu_irqs *pmu_irqs; /* * 2 helper fns take a counter number from 0-7 (not the userspace-visible counter number) @@ -225,33 +228,40 @@ static int em_setup_ctrs(void) return 0; } -static int arm11_irqs[] = { - [0] = IRQ_EB11MP_PMU_CPU0, - [1] = IRQ_EB11MP_PMU_CPU1, - [2] = IRQ_EB11MP_PMU_CPU2, - [3] = IRQ_EB11MP_PMU_CPU3 -}; - static int em_start(void) { int ret; - ret = arm11_request_interrupts(arm11_irqs, ARRAY_SIZE(arm11_irqs)); + pmu_irqs = reserve_pmu(); + if (IS_ERR(pmu_irqs)) { + ret = PTR_ERR(pmu_irqs); + goto out; + } + + ret = arm11_request_interrupts(pmu_irqs->irqs, pmu_irqs->num_irqs); if (ret == 0) { em_call_function(arm11_start_pmu); ret = scu_start(); - if (ret) - arm11_release_interrupts(arm11_irqs, ARRAY_SIZE(arm11_irqs)); + if (ret) { + arm11_release_interrupts(pmu_irqs->irqs, + pmu_irqs->num_irqs); + } else { + release_pmu(pmu_irqs); + pmu_irqs = NULL; + } } + +out: return ret; } static void em_stop(void) { em_call_function(arm11_stop_pmu); - arm11_release_interrupts(arm11_irqs, ARRAY_SIZE(arm11_irqs)); + arm11_release_interrupts(pmu_irqs->irqs, pmu_irqs->num_irqs); scu_stop(); + release_pmu(pmu_irqs); } /* @@ -283,15 +293,7 @@ static int em_setup(void) em_route_irq(IRQ_EB11MP_PMU_SCU6, 3); em_route_irq(IRQ_EB11MP_PMU_SCU7, 3); - /* - * Send CP15 PMU interrupts to the owner CPU. - */ - em_route_irq(IRQ_EB11MP_PMU_CPU0, 0); - em_route_irq(IRQ_EB11MP_PMU_CPU1, 1); - em_route_irq(IRQ_EB11MP_PMU_CPU2, 2); - em_route_irq(IRQ_EB11MP_PMU_CPU3, 3); - - return 0; + return init_pmu(); } struct op_arm_model_spec op_mpcore_spec = { diff --git a/arch/arm/oprofile/op_model_v6.c b/arch/arm/oprofile/op_model_v6.c index f7d2ec5..a22357a 100644 --- a/arch/arm/oprofile/op_model_v6.c +++ b/arch/arm/oprofile/op_model_v6.c @@ -19,39 +19,47 @@ /* #define DEBUG */ #include <linux/types.h> #include <linux/errno.h> +#include <linux/err.h> #include <linux/sched.h> #include <linux/oprofile.h> #include <linux/interrupt.h> #include <asm/irq.h> #include <asm/system.h> +#include <asm/pmu.h> #include "op_counter.h" #include "op_arm_model.h" #include "op_model_arm11_core.h" -static int irqs[] = { -#ifdef CONFIG_ARCH_OMAP2 - 3, -#endif -#ifdef CONFIG_ARCH_BCMRING - IRQ_PMUIRQ, /* for BCMRING, ARM PMU interrupt is 43 */ -#endif -}; +static const struct pmu_irqs *pmu_irqs; static void armv6_pmu_stop(void) { arm11_stop_pmu(); - arm11_release_interrupts(irqs, ARRAY_SIZE(irqs)); + arm11_release_interrupts(pmu_irqs->irqs, pmu_irqs->num_irqs); + release_pmu(pmu_irqs); + pmu_irqs = NULL; } static int armv6_pmu_start(void) { int ret; - ret = arm11_request_interrupts(irqs, ARRAY_SIZE(irqs)); - if (ret >= 0) + pmu_irqs = reserve_pmu(); + if (IS_ERR(pmu_irqs)) { + ret = PTR_ERR(pmu_irqs); + goto out; + } + + ret = arm11_request_interrupts(pmu_irqs->irqs, pmu_irqs->num_irqs); + if (ret >= 0) { ret = arm11_start_pmu(); + } else { + release_pmu(pmu_irqs); + pmu_irqs = NULL; + } +out: return ret; } diff --git a/arch/arm/oprofile/op_model_v7.c b/arch/arm/oprofile/op_model_v7.c index f20295f..9258fca 100644 --- a/arch/arm/oprofile/op_model_v7.c +++ b/arch/arm/oprofile/op_model_v7.c @@ -11,11 +11,14 @@ */ #include <linux/types.h> #include <linux/errno.h> +#include <linux/err.h> #include <linux/oprofile.h> #include <linux/interrupt.h> #include <linux/irq.h> #include <linux/smp.h> +#include <asm/pmu.h> + #include "op_counter.h" #include "op_arm_model.h" #include "op_model_v7.h" @@ -299,7 +302,7 @@ static irqreturn_t armv7_pmnc_interrupt(int irq, void *arg) return IRQ_HANDLED; } -int armv7_request_interrupts(int *irqs, int nr) +int armv7_request_interrupts(const int *irqs, int nr) { unsigned int i; int ret = 0; @@ -322,7 +325,7 @@ int armv7_request_interrupts(int *irqs, int nr) return ret; } -void armv7_release_interrupts(int *irqs, int nr) +void armv7_release_interrupts(const int *irqs, int nr) { unsigned int i; @@ -366,12 +369,7 @@ static void armv7_pmnc_dump_regs(void) } #endif - -static int irqs[] = { -#ifdef CONFIG_ARCH_OMAP3 - INT_34XX_BENCH_MPU_EMUL, -#endif -}; +static const struct pmu_irqs *pmu_irqs; static void armv7_pmnc_stop(void) { @@ -379,19 +377,29 @@ static void armv7_pmnc_stop(void) armv7_pmnc_dump_regs(); #endif armv7_stop_pmnc(); - armv7_release_interrupts(irqs, ARRAY_SIZE(irqs)); + armv7_release_interrupts(pmu_irqs->irqs, pmu_irqs->num_irqs); + release_pmu(pmu_irqs); + pmu_irqs = NULL; } static int armv7_pmnc_start(void) { int ret; + pmu_irqs = reserve_pmu(); + if (IS_ERR(pmu_irqs)) + return PTR_ERR(pmu_irqs); + #ifdef DEBUG armv7_pmnc_dump_regs(); #endif - ret = armv7_request_interrupts(irqs, ARRAY_SIZE(irqs)); - if (ret >= 0) + ret = armv7_request_interrupts(pmu_irqs->irqs, pmu_irqs->num_irqs); + if (ret >= 0) { armv7_start_pmnc(); + } else { + release_pmu(pmu_irqs); + pmu_irqs = NULL; + } return ret; } diff --git a/arch/arm/oprofile/op_model_v7.h b/arch/arm/oprofile/op_model_v7.h index 0e19bcc..9ca334b 100644 --- a/arch/arm/oprofile/op_model_v7.h +++ b/arch/arm/oprofile/op_model_v7.h @@ -97,7 +97,7 @@ int armv7_setup_pmu(void); int armv7_start_pmu(void); int armv7_stop_pmu(void); -int armv7_request_interrupts(int *, int); -void armv7_release_interrupts(int *, int); +int armv7_request_interrupts(const int *, int); +void armv7_release_interrupts(const int *, int); #endif diff --git a/arch/arm/oprofile/op_model_xscale.c b/arch/arm/oprofile/op_model_xscale.c index 724ab9c..1d34a02 100644 --- a/arch/arm/oprofile/op_model_xscale.c +++ b/arch/arm/oprofile/op_model_xscale.c @@ -17,12 +17,14 @@ /* #define DEBUG */ #include <linux/types.h> #include <linux/errno.h> +#include <linux/err.h> #include <linux/sched.h> #include <linux/oprofile.h> #include <linux/interrupt.h> #include <linux/irq.h> #include <asm/cputype.h> +#include <asm/pmu.h> #include "op_counter.h" #include "op_arm_model.h" @@ -33,17 +35,6 @@ #define PMU_RESET (CCNT_RESET | PMN_RESET) #define PMU_CNT64 0x008 /* Make CCNT count every 64th cycle */ -/* TODO do runtime detection */ -#ifdef CONFIG_ARCH_IOP32X -#define XSCALE_PMU_IRQ IRQ_IOP32X_CORE_PMU -#endif -#ifdef CONFIG_ARCH_IOP33X -#define XSCALE_PMU_IRQ IRQ_IOP33X_CORE_PMU -#endif -#ifdef CONFIG_ARCH_PXA -#define XSCALE_PMU_IRQ IRQ_PMU -#endif - /* * Different types of events that can be counted by the XScale PMU * as used by Oprofile userspace. Here primarily for documentation @@ -367,6 +358,8 @@ static irqreturn_t xscale_pmu_interrupt(int irq, void *arg) return IRQ_HANDLED; } +static const struct pmu_irqs *pmu_irqs; + static void xscale_pmu_stop(void) { u32 pmnc = read_pmnc(); @@ -374,20 +367,30 @@ static void xscale_pmu_stop(void) pmnc &= ~PMU_ENABLE; write_pmnc(pmnc); - free_irq(XSCALE_PMU_IRQ, results); + free_irq(pmu_irqs->irqs[0], results); + release_pmu(pmu_irqs); + pmu_irqs = NULL; } static int xscale_pmu_start(void) { int ret; - u32 pmnc = read_pmnc(); + u32 pmnc; + + pmu_irqs = reserve_pmu(); + if (IS_ERR(pmu_irqs)) + return PTR_ERR(pmu_irqs); + + pmnc = read_pmnc(); - ret = request_irq(XSCALE_PMU_IRQ, xscale_pmu_interrupt, IRQF_DISABLED, - "XScale PMU", (void *)results); + ret = request_irq(pmu_irqs->irqs[0], xscale_pmu_interrupt, + IRQF_DISABLED, "XScale PMU", (void *)results); if (ret < 0) { printk(KERN_ERR "oprofile: unable to request IRQ%d for XScale PMU\n", - XSCALE_PMU_IRQ); + pmu_irqs->irqs[0]); + release_pmu(pmu_irqs); + pmu_irqs = NULL; return ret; } -- 1.6.5.4 ^ permalink raw reply related [flat|nested] 28+ messages in thread
* [PATCH 3/5] arm: use the spinlocked, generic atomic64 support 2009-12-15 11:15 ` [PATCH 2/5] arm/oprofile: reserve the PMU when starting Jamie Iles @ 2009-12-15 11:15 ` Jamie Iles 0 siblings, 0 replies; 28+ messages in thread From: Jamie Iles @ 2009-12-15 11:15 UTC (permalink / raw) To: linux-arm-kernel perf events require that we can support atomic64's. There is a generic, spinlocked version that we can use until we have proper hardware support. Signed-off-by: Jamie Iles <jamie.iles@picochip.com> --- arch/arm/Kconfig | 1 + arch/arm/include/asm/atomic.h | 4 ++++ 2 files changed, 5 insertions(+), 0 deletions(-) diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig index 233a222..9580418 100644 --- a/arch/arm/Kconfig +++ b/arch/arm/Kconfig @@ -18,6 +18,7 @@ config ARM select HAVE_KRETPROBES if (HAVE_KPROBES) select HAVE_FUNCTION_TRACER if (!XIP_KERNEL) select HAVE_GENERIC_DMA_COHERENT + select GENERIC_ATOMIC64 help The ARM series is a line of low-power-consumption RISC chip designs licensed by ARM Ltd and targeted at embedded applications and diff --git a/arch/arm/include/asm/atomic.h b/arch/arm/include/asm/atomic.h index d0daeab..ff286a8 100644 --- a/arch/arm/include/asm/atomic.h +++ b/arch/arm/include/asm/atomic.h @@ -15,6 +15,10 @@ #include <linux/types.h> #include <asm/system.h> +#ifdef CONFIG_GENERIC_ATOMIC64 +#include <asm-generic/atomic64.h> +#endif + #define ATOMIC_INIT(i) { (i) } #ifdef __KERNEL__ -- 1.6.5.4 ^ permalink raw reply related [flat|nested] 28+ messages in thread
* ARM perf events support v4
@ 2010-01-04 10:48 Jamie Iles
2010-01-04 10:48 ` [PATCH 1/5] arm: provide a mechanism to reserve performance counters Jamie Iles
0 siblings, 1 reply; 28+ messages in thread
From: Jamie Iles @ 2010-01-04 10:48 UTC (permalink / raw)
To: linux-arm-kernel
This is the fourth revision of ARM performance counter support. This adds
support for software performance events on all ARM platforms and hardware
support for v6 processors. There is a generic PMU framework to allow for v7
and later to be added in the future.
Jamie Iles (5):
arm: provide a mechanism to reserve performance counters
arm/oprofile: reserve the PMU when starting
arm: use the spinlocked, generic atomic64 support
arm: enable support for software perf events
arm/perfevents: implement perf event support for ARMv6
arch/arm/Kconfig | 16 +
arch/arm/include/asm/atomic.h | 4 +
arch/arm/include/asm/perf_event.h | 38 +
arch/arm/include/asm/pmu.h | 74 ++
arch/arm/kernel/Makefile | 2 +
arch/arm/kernel/perf_event.c | 1338 +++++++++++++++++++++++++++++++
arch/arm/kernel/pmu.c | 107 +++
arch/arm/mm/fault.c | 7 +
arch/arm/oprofile/op_model_arm11_core.c | 4 +-
arch/arm/oprofile/op_model_arm11_core.h | 4 +-
arch/arm/oprofile/op_model_mpcore.c | 42 +-
arch/arm/oprofile/op_model_v6.c | 30 +-
arch/arm/oprofile/op_model_v7.c | 30 +-
arch/arm/oprofile/op_model_v7.h | 4 +-
arch/arm/oprofile/op_model_xscale.c | 35 +-
15 files changed, 1671 insertions(+), 64 deletions(-)
Jamie
^ permalink raw reply [flat|nested] 28+ messages in thread* [PATCH 1/5] arm: provide a mechanism to reserve performance counters 2010-01-04 10:48 ARM perf events support v4 Jamie Iles @ 2010-01-04 10:48 ` Jamie Iles 2010-01-04 10:48 ` [PATCH 2/5] arm/oprofile: reserve the PMU when starting Jamie Iles 0 siblings, 1 reply; 28+ messages in thread From: Jamie Iles @ 2010-01-04 10:48 UTC (permalink / raw) To: linux-arm-kernel To add support for perf events and to allow the hardware counters to be shared with oprofile, we need a way to reserve access to the pmu (performance monitor unit). Cc: Will Deacon <will.deacon@arm.com> Signed-off-by: Jamie Iles <jamie.iles@picochip.com> --- arch/arm/Kconfig | 5 ++ arch/arm/include/asm/pmu.h | 74 ++++++++++++++++++++++++++++++ arch/arm/kernel/Makefile | 1 + arch/arm/kernel/pmu.c | 107 ++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 187 insertions(+), 0 deletions(-) create mode 100644 arch/arm/include/asm/pmu.h create mode 100644 arch/arm/kernel/pmu.c diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig index 233a222..9e08891 100644 --- a/arch/arm/Kconfig +++ b/arch/arm/Kconfig @@ -863,6 +863,11 @@ config XSCALE_PMU depends on CPU_XSCALE && !XSCALE_PMU_TIMER default y +config CPU_HAS_PMU + depends on CPU_V6 || CPU_V7 || XSCALE_PMU + default y + bool + if !MMU source "arch/arm/Kconfig-nommu" endif diff --git a/arch/arm/include/asm/pmu.h b/arch/arm/include/asm/pmu.h new file mode 100644 index 0000000..5840d2d --- /dev/null +++ b/arch/arm/include/asm/pmu.h @@ -0,0 +1,74 @@ +/* + * linux/arch/arm/include/asm/pmu.h + * + * Copyright (C) 2009 picoChip Designs Ltd, Jamie Iles + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + */ + +#ifndef __ARM_PMU_H__ +#define __ARM_PMU_H__ + +#ifdef CONFIG_CPU_HAS_PMU + +struct pmu_irqs { + const int *irqs; + int num_irqs; +}; + +/** + * reserve_pmu() - reserve the hardware performance counters + * + * Reserve the hardware performance counters in the system for exclusive use. + * The 'struct pmu_irqs' for the system is returned on success, ERR_PTR() + * encoded error on failure. + */ +extern const struct pmu_irqs * +reserve_pmu(void); + +/** + * release_pmu() - Relinquish control of the performance counters + * + * Release the performance counters and allow someone else to use them. + * Callers must have disabled the counters and released IRQs before calling + * this. The 'struct pmu_irqs' returned from reserve_pmu() must be passed as + * a cookie. + */ +extern int +release_pmu(const struct pmu_irqs *irqs); + +/** + * init_pmu() - Initialise the PMU. + * + * Initialise the system ready for PMU enabling. This should typically set the + * IRQ affinity and nothing else. The users (oprofile/perf events etc) will do + * the actual hardware initialisation. + */ +extern int +init_pmu(void); + +#else /* CONFIG_CPU_HAS_PMU */ + +static inline const struct pmu_irqs * +reserve_pmu(void) +{ + ERR_PTR(-ENODEV); +} + +static inline int +release_pmu(const struct pmu_irqs *irqs) +{ +} + +static inline int +init_pmu(void) +{ + return -ENODEV; +} + +#endif /* CONFIG_CPU_HAS_PMU */ + +#endif /* __ARM_PMU_H__ */ diff --git a/arch/arm/kernel/Makefile b/arch/arm/kernel/Makefile index dd00f74..216890d 100644 --- a/arch/arm/kernel/Makefile +++ b/arch/arm/kernel/Makefile @@ -46,6 +46,7 @@ obj-$(CONFIG_CPU_XSCALE) += xscale-cp0.o obj-$(CONFIG_CPU_XSC3) += xscale-cp0.o obj-$(CONFIG_CPU_MOHAWK) += xscale-cp0.o obj-$(CONFIG_IWMMXT) += iwmmxt.o +obj-$(CONFIG_CPU_HAS_PMU) += pmu.o AFLAGS_iwmmxt.o := -Wa,-mcpu=iwmmxt ifneq ($(CONFIG_ARCH_EBSA110),y) diff --git a/arch/arm/kernel/pmu.c b/arch/arm/kernel/pmu.c new file mode 100644 index 0000000..a8c015d --- /dev/null +++ b/arch/arm/kernel/pmu.c @@ -0,0 +1,107 @@ +/* + * linux/arch/arm/kernel/pmu.c + * + * Copyright (C) 2009 picoChip Designs Ltd, Jamie Iles + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + */ + +#include <linux/cpumask.h> +#include <linux/err.h> +#include <linux/interrupt.h> +#include <linux/kernel.h> +#include <linux/module.h> +#include <linux/semaphore.h> + +#include <asm/pmu.h> + +/* + * Define the IRQs for the system. We could use something like a platform + * device but that seems fairly heavyweight for this. Also, the performance + * counters can't be removed or hotplugged. + * + * Ordering is important: init_pmu() will use the ordering to set the affinity + * to the corresponding core. e.g. the first interrupt will go to cpu 0, the + * second goes to cpu 1 etc. + */ +static const int irqs[] = { +#ifdef CONFIG_ARCH_PC3XX + IRQ_NPMUIRQ, +#elif defined(CONFIG_ARCH_OMAP2) + 3, +#elif defined(CONFIG_ARCH_BCMRING) + IRQ_PMUIRQ, +#elif defined(CONFIG_MACH_REALVIEW_EB) + IRQ_EB11MP_PMU_CPU0, + IRQ_EB11MP_PMU_CPU1, + IRQ_EB11MP_PMU_CPU2, + IRQ_EB11MP_PMU_CPU3, +#elif defined(CONFIG_ARCH_OMAP3) + INT_34XX_BENCH_MPU_EMUL, +#elif defined(CONFIG_ARCH_IOP32X) + IRQ_IOP32X_CORE_PMU, +#elif defined(CONFIG_ARCH_IOP33X) + IRQ_IOP33X_CORE_PMU, +#elif defined(CONFIG_ARCH_PXA) + IRQ_PMU, +#endif +}; + +static const struct pmu_irqs pmu_irqs = { + .irqs = irqs, + .num_irqs = ARRAY_SIZE(irqs), +}; + +static DECLARE_MUTEX(pmu_mutex); + +const struct pmu_irqs * +reserve_pmu(void) +{ + int ret = down_trylock(&pmu_mutex) ? -EBUSY : 0; + + return ret ? ERR_PTR(ret) : &pmu_irqs; +} +EXPORT_SYMBOL_GPL(reserve_pmu); + +int +release_pmu(const struct pmu_irqs *irqs) +{ + if (WARN_ON(irqs != &pmu_irqs)) + return -EINVAL; + up(&pmu_mutex); + return 0; +} +EXPORT_SYMBOL_GPL(release_pmu); + +static int +set_irq_affinity(int irq, + unsigned int cpu) +{ +#ifdef CONFIG_SMP + int err = irq_set_affinity(irq, cpumask_of(cpu)); + if (err) + pr_warning("unable to set irq affinity (irq=%d, cpu=%u)\n", + irq, cpu); + return err; +#else + return 0; +#endif +} + +int +init_pmu(void) +{ + int i, err = 0; + + for (i = 0; i < pmu_irqs.num_irqs; ++i) { + err = set_irq_affinity(pmu_irqs.irqs[i], i); + if (err) + break; + } + + return err; +} +EXPORT_SYMBOL_GPL(init_pmu); -- 1.6.5.4 ^ permalink raw reply related [flat|nested] 28+ messages in thread
* [PATCH 2/5] arm/oprofile: reserve the PMU when starting 2010-01-04 10:48 ` [PATCH 1/5] arm: provide a mechanism to reserve performance counters Jamie Iles @ 2010-01-04 10:48 ` Jamie Iles 2010-01-04 10:48 ` [PATCH 3/5] arm: use the spinlocked, generic atomic64 support Jamie Iles 0 siblings, 1 reply; 28+ messages in thread From: Jamie Iles @ 2010-01-04 10:48 UTC (permalink / raw) To: linux-arm-kernel Make sure that we have access to the performance counters and that they aren't being used by perf events or anything else. Cc: Will Deacon <will.deacon@arm.com> Cc: Jean Pihet <jpihet@mvista.com> Signed-off-by: Jamie Iles <jamie.iles@picochip.com> --- arch/arm/oprofile/op_model_arm11_core.c | 4 +- arch/arm/oprofile/op_model_arm11_core.h | 4 +- arch/arm/oprofile/op_model_mpcore.c | 42 ++++++++++++++++-------------- arch/arm/oprofile/op_model_v6.c | 30 ++++++++++++++-------- arch/arm/oprofile/op_model_v7.c | 30 ++++++++++++++-------- arch/arm/oprofile/op_model_v7.h | 4 +- arch/arm/oprofile/op_model_xscale.c | 35 ++++++++++++++----------- 7 files changed, 85 insertions(+), 64 deletions(-) diff --git a/arch/arm/oprofile/op_model_arm11_core.c b/arch/arm/oprofile/op_model_arm11_core.c index ad80752..ef3e265 100644 --- a/arch/arm/oprofile/op_model_arm11_core.c +++ b/arch/arm/oprofile/op_model_arm11_core.c @@ -132,7 +132,7 @@ static irqreturn_t arm11_pmu_interrupt(int irq, void *arg) return IRQ_HANDLED; } -int arm11_request_interrupts(int *irqs, int nr) +int arm11_request_interrupts(const int *irqs, int nr) { unsigned int i; int ret = 0; @@ -153,7 +153,7 @@ int arm11_request_interrupts(int *irqs, int nr) return ret; } -void arm11_release_interrupts(int *irqs, int nr) +void arm11_release_interrupts(const int *irqs, int nr) { unsigned int i; diff --git a/arch/arm/oprofile/op_model_arm11_core.h b/arch/arm/oprofile/op_model_arm11_core.h index 6f8538e..1902b99 100644 --- a/arch/arm/oprofile/op_model_arm11_core.h +++ b/arch/arm/oprofile/op_model_arm11_core.h @@ -39,7 +39,7 @@ int arm11_setup_pmu(void); int arm11_start_pmu(void); int arm11_stop_pmu(void); -int arm11_request_interrupts(int *, int); -void arm11_release_interrupts(int *, int); +int arm11_request_interrupts(const int *, int); +void arm11_release_interrupts(const int *, int); #endif diff --git a/arch/arm/oprofile/op_model_mpcore.c b/arch/arm/oprofile/op_model_mpcore.c index 4ce0f98..f73ce87 100644 --- a/arch/arm/oprofile/op_model_mpcore.c +++ b/arch/arm/oprofile/op_model_mpcore.c @@ -32,6 +32,7 @@ /* #define DEBUG */ #include <linux/types.h> #include <linux/errno.h> +#include <linux/err.h> #include <linux/sched.h> #include <linux/oprofile.h> #include <linux/interrupt.h> @@ -43,6 +44,7 @@ #include <mach/hardware.h> #include <mach/board-eb.h> #include <asm/system.h> +#include <asm/pmu.h> #include "op_counter.h" #include "op_arm_model.h" @@ -58,6 +60,7 @@ * Bitmask of used SCU counters */ static unsigned int scu_em_used; +static const struct pmu_irqs *pmu_irqs; /* * 2 helper fns take a counter number from 0-7 (not the userspace-visible counter number) @@ -225,33 +228,40 @@ static int em_setup_ctrs(void) return 0; } -static int arm11_irqs[] = { - [0] = IRQ_EB11MP_PMU_CPU0, - [1] = IRQ_EB11MP_PMU_CPU1, - [2] = IRQ_EB11MP_PMU_CPU2, - [3] = IRQ_EB11MP_PMU_CPU3 -}; - static int em_start(void) { int ret; - ret = arm11_request_interrupts(arm11_irqs, ARRAY_SIZE(arm11_irqs)); + pmu_irqs = reserve_pmu(); + if (IS_ERR(pmu_irqs)) { + ret = PTR_ERR(pmu_irqs); + goto out; + } + + ret = arm11_request_interrupts(pmu_irqs->irqs, pmu_irqs->num_irqs); if (ret == 0) { em_call_function(arm11_start_pmu); ret = scu_start(); - if (ret) - arm11_release_interrupts(arm11_irqs, ARRAY_SIZE(arm11_irqs)); + if (ret) { + arm11_release_interrupts(pmu_irqs->irqs, + pmu_irqs->num_irqs); + } else { + release_pmu(pmu_irqs); + pmu_irqs = NULL; + } } + +out: return ret; } static void em_stop(void) { em_call_function(arm11_stop_pmu); - arm11_release_interrupts(arm11_irqs, ARRAY_SIZE(arm11_irqs)); + arm11_release_interrupts(pmu_irqs->irqs, pmu_irqs->num_irqs); scu_stop(); + release_pmu(pmu_irqs); } /* @@ -283,15 +293,7 @@ static int em_setup(void) em_route_irq(IRQ_EB11MP_PMU_SCU6, 3); em_route_irq(IRQ_EB11MP_PMU_SCU7, 3); - /* - * Send CP15 PMU interrupts to the owner CPU. - */ - em_route_irq(IRQ_EB11MP_PMU_CPU0, 0); - em_route_irq(IRQ_EB11MP_PMU_CPU1, 1); - em_route_irq(IRQ_EB11MP_PMU_CPU2, 2); - em_route_irq(IRQ_EB11MP_PMU_CPU3, 3); - - return 0; + return init_pmu(); } struct op_arm_model_spec op_mpcore_spec = { diff --git a/arch/arm/oprofile/op_model_v6.c b/arch/arm/oprofile/op_model_v6.c index f7d2ec5..a22357a 100644 --- a/arch/arm/oprofile/op_model_v6.c +++ b/arch/arm/oprofile/op_model_v6.c @@ -19,39 +19,47 @@ /* #define DEBUG */ #include <linux/types.h> #include <linux/errno.h> +#include <linux/err.h> #include <linux/sched.h> #include <linux/oprofile.h> #include <linux/interrupt.h> #include <asm/irq.h> #include <asm/system.h> +#include <asm/pmu.h> #include "op_counter.h" #include "op_arm_model.h" #include "op_model_arm11_core.h" -static int irqs[] = { -#ifdef CONFIG_ARCH_OMAP2 - 3, -#endif -#ifdef CONFIG_ARCH_BCMRING - IRQ_PMUIRQ, /* for BCMRING, ARM PMU interrupt is 43 */ -#endif -}; +static const struct pmu_irqs *pmu_irqs; static void armv6_pmu_stop(void) { arm11_stop_pmu(); - arm11_release_interrupts(irqs, ARRAY_SIZE(irqs)); + arm11_release_interrupts(pmu_irqs->irqs, pmu_irqs->num_irqs); + release_pmu(pmu_irqs); + pmu_irqs = NULL; } static int armv6_pmu_start(void) { int ret; - ret = arm11_request_interrupts(irqs, ARRAY_SIZE(irqs)); - if (ret >= 0) + pmu_irqs = reserve_pmu(); + if (IS_ERR(pmu_irqs)) { + ret = PTR_ERR(pmu_irqs); + goto out; + } + + ret = arm11_request_interrupts(pmu_irqs->irqs, pmu_irqs->num_irqs); + if (ret >= 0) { ret = arm11_start_pmu(); + } else { + release_pmu(pmu_irqs); + pmu_irqs = NULL; + } +out: return ret; } diff --git a/arch/arm/oprofile/op_model_v7.c b/arch/arm/oprofile/op_model_v7.c index 2088a6c..8642d08 100644 --- a/arch/arm/oprofile/op_model_v7.c +++ b/arch/arm/oprofile/op_model_v7.c @@ -11,11 +11,14 @@ */ #include <linux/types.h> #include <linux/errno.h> +#include <linux/err.h> #include <linux/oprofile.h> #include <linux/interrupt.h> #include <linux/irq.h> #include <linux/smp.h> +#include <asm/pmu.h> + #include "op_counter.h" #include "op_arm_model.h" #include "op_model_v7.h" @@ -295,7 +298,7 @@ static irqreturn_t armv7_pmnc_interrupt(int irq, void *arg) return IRQ_HANDLED; } -int armv7_request_interrupts(int *irqs, int nr) +int armv7_request_interrupts(const int *irqs, int nr) { unsigned int i; int ret = 0; @@ -318,7 +321,7 @@ int armv7_request_interrupts(int *irqs, int nr) return ret; } -void armv7_release_interrupts(int *irqs, int nr) +void armv7_release_interrupts(const int *irqs, int nr) { unsigned int i; @@ -362,12 +365,7 @@ static void armv7_pmnc_dump_regs(void) } #endif - -static int irqs[] = { -#ifdef CONFIG_ARCH_OMAP3 - INT_34XX_BENCH_MPU_EMUL, -#endif -}; +static const struct pmu_irqs *pmu_irqs; static void armv7_pmnc_stop(void) { @@ -375,19 +373,29 @@ static void armv7_pmnc_stop(void) armv7_pmnc_dump_regs(); #endif armv7_stop_pmnc(); - armv7_release_interrupts(irqs, ARRAY_SIZE(irqs)); + armv7_release_interrupts(pmu_irqs->irqs, pmu_irqs->num_irqs); + release_pmu(pmu_irqs); + pmu_irqs = NULL; } static int armv7_pmnc_start(void) { int ret; + pmu_irqs = reserve_pmu(); + if (IS_ERR(pmu_irqs)) + return PTR_ERR(pmu_irqs); + #ifdef DEBUG armv7_pmnc_dump_regs(); #endif - ret = armv7_request_interrupts(irqs, ARRAY_SIZE(irqs)); - if (ret >= 0) + ret = armv7_request_interrupts(pmu_irqs->irqs, pmu_irqs->num_irqs); + if (ret >= 0) { armv7_start_pmnc(); + } else { + release_pmu(pmu_irqs); + pmu_irqs = NULL; + } return ret; } diff --git a/arch/arm/oprofile/op_model_v7.h b/arch/arm/oprofile/op_model_v7.h index 0e19bcc..9ca334b 100644 --- a/arch/arm/oprofile/op_model_v7.h +++ b/arch/arm/oprofile/op_model_v7.h @@ -97,7 +97,7 @@ int armv7_setup_pmu(void); int armv7_start_pmu(void); int armv7_stop_pmu(void); -int armv7_request_interrupts(int *, int); -void armv7_release_interrupts(int *, int); +int armv7_request_interrupts(const int *, int); +void armv7_release_interrupts(const int *, int); #endif diff --git a/arch/arm/oprofile/op_model_xscale.c b/arch/arm/oprofile/op_model_xscale.c index 724ab9c..1d34a02 100644 --- a/arch/arm/oprofile/op_model_xscale.c +++ b/arch/arm/oprofile/op_model_xscale.c @@ -17,12 +17,14 @@ /* #define DEBUG */ #include <linux/types.h> #include <linux/errno.h> +#include <linux/err.h> #include <linux/sched.h> #include <linux/oprofile.h> #include <linux/interrupt.h> #include <linux/irq.h> #include <asm/cputype.h> +#include <asm/pmu.h> #include "op_counter.h" #include "op_arm_model.h" @@ -33,17 +35,6 @@ #define PMU_RESET (CCNT_RESET | PMN_RESET) #define PMU_CNT64 0x008 /* Make CCNT count every 64th cycle */ -/* TODO do runtime detection */ -#ifdef CONFIG_ARCH_IOP32X -#define XSCALE_PMU_IRQ IRQ_IOP32X_CORE_PMU -#endif -#ifdef CONFIG_ARCH_IOP33X -#define XSCALE_PMU_IRQ IRQ_IOP33X_CORE_PMU -#endif -#ifdef CONFIG_ARCH_PXA -#define XSCALE_PMU_IRQ IRQ_PMU -#endif - /* * Different types of events that can be counted by the XScale PMU * as used by Oprofile userspace. Here primarily for documentation @@ -367,6 +358,8 @@ static irqreturn_t xscale_pmu_interrupt(int irq, void *arg) return IRQ_HANDLED; } +static const struct pmu_irqs *pmu_irqs; + static void xscale_pmu_stop(void) { u32 pmnc = read_pmnc(); @@ -374,20 +367,30 @@ static void xscale_pmu_stop(void) pmnc &= ~PMU_ENABLE; write_pmnc(pmnc); - free_irq(XSCALE_PMU_IRQ, results); + free_irq(pmu_irqs->irqs[0], results); + release_pmu(pmu_irqs); + pmu_irqs = NULL; } static int xscale_pmu_start(void) { int ret; - u32 pmnc = read_pmnc(); + u32 pmnc; + + pmu_irqs = reserve_pmu(); + if (IS_ERR(pmu_irqs)) + return PTR_ERR(pmu_irqs); + + pmnc = read_pmnc(); - ret = request_irq(XSCALE_PMU_IRQ, xscale_pmu_interrupt, IRQF_DISABLED, - "XScale PMU", (void *)results); + ret = request_irq(pmu_irqs->irqs[0], xscale_pmu_interrupt, + IRQF_DISABLED, "XScale PMU", (void *)results); if (ret < 0) { printk(KERN_ERR "oprofile: unable to request IRQ%d for XScale PMU\n", - XSCALE_PMU_IRQ); + pmu_irqs->irqs[0]); + release_pmu(pmu_irqs); + pmu_irqs = NULL; return ret; } -- 1.6.5.4 ^ permalink raw reply related [flat|nested] 28+ messages in thread
* [PATCH 3/5] arm: use the spinlocked, generic atomic64 support 2010-01-04 10:48 ` [PATCH 2/5] arm/oprofile: reserve the PMU when starting Jamie Iles @ 2010-01-04 10:48 ` Jamie Iles 2010-01-05 18:57 ` Jamie Lokier 0 siblings, 1 reply; 28+ messages in thread From: Jamie Iles @ 2010-01-04 10:48 UTC (permalink / raw) To: linux-arm-kernel perf events require that we can support atomic64's. There is a generic, spinlocked version that we can use until we have proper hardware support. Signed-off-by: Jamie Iles <jamie.iles@picochip.com> --- arch/arm/Kconfig | 1 + arch/arm/include/asm/atomic.h | 4 ++++ 2 files changed, 5 insertions(+), 0 deletions(-) diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig index 9e08891..0dbfd9b 100644 --- a/arch/arm/Kconfig +++ b/arch/arm/Kconfig @@ -18,6 +18,7 @@ config ARM select HAVE_KRETPROBES if (HAVE_KPROBES) select HAVE_FUNCTION_TRACER if (!XIP_KERNEL) select HAVE_GENERIC_DMA_COHERENT + select GENERIC_ATOMIC64 help The ARM series is a line of low-power-consumption RISC chip designs licensed by ARM Ltd and targeted at embedded applications and diff --git a/arch/arm/include/asm/atomic.h b/arch/arm/include/asm/atomic.h index d0daeab..ff286a8 100644 --- a/arch/arm/include/asm/atomic.h +++ b/arch/arm/include/asm/atomic.h @@ -15,6 +15,10 @@ #include <linux/types.h> #include <asm/system.h> +#ifdef CONFIG_GENERIC_ATOMIC64 +#include <asm-generic/atomic64.h> +#endif + #define ATOMIC_INIT(i) { (i) } #ifdef __KERNEL__ -- 1.6.5.4 ^ permalink raw reply related [flat|nested] 28+ messages in thread
* [PATCH 3/5] arm: use the spinlocked, generic atomic64 support 2010-01-04 10:48 ` [PATCH 3/5] arm: use the spinlocked, generic atomic64 support Jamie Iles @ 2010-01-05 18:57 ` Jamie Lokier 2010-01-05 19:08 ` Jamie Iles 0 siblings, 1 reply; 28+ messages in thread From: Jamie Lokier @ 2010-01-05 18:57 UTC (permalink / raw) To: linux-arm-kernel Jamie Iles wrote: > perf events require that we can support atomic64's. There is a generic, > spinlocked version that we can use until we have proper hardware > support. Is that simply waiting for LDREXD/STREXD-based atomic64's, or is it depending on something subtler? -- Jamie ^ permalink raw reply [flat|nested] 28+ messages in thread
* [PATCH 3/5] arm: use the spinlocked, generic atomic64 support 2010-01-05 18:57 ` Jamie Lokier @ 2010-01-05 19:08 ` Jamie Iles 0 siblings, 0 replies; 28+ messages in thread From: Jamie Iles @ 2010-01-05 19:08 UTC (permalink / raw) To: linux-arm-kernel On Tue, Jan 05, 2010 at 06:57:32PM +0000, Jamie Lokier wrote: > Jamie Iles wrote: > > perf events require that we can support atomic64's. There is a generic, > > spinlocked version that we can use until we have proper hardware > > support. > > Is that simply waiting for LDREXD/STREXD-based atomic64's, or is it > depending on something subtler? Yes, Will Deacon submitted a patch last month that does this but afaik it hasn't been merged in yet. Jamie ^ permalink raw reply [flat|nested] 28+ messages in thread
* ARM perf events support v5
@ 2010-01-14 12:14 Jamie Iles
2010-01-14 12:14 ` [PATCH 1/5] arm: provide a mechanism to reserve performance counters Jamie Iles
0 siblings, 1 reply; 28+ messages in thread
From: Jamie Iles @ 2010-01-14 12:14 UTC (permalink / raw)
To: linux-arm-kernel
Here is the 5th revision of the ARM performance events support. This initially
targets software events on all platforms and hardware events on v6 platforms
with hooks for later platforms to be added later.
Changes since the previous revison are:
- compile fixes for systems without a pmu.
- remove mutex from pmu.c and replace with bitset based lock
- removal of BUG()'s and replace with warnings
Also required are:
ARM: 5866/1: arm ptrace: use unsigned types for kernel pt_regs
sched/perf: Make sure irqs are disabled for perf_event_task_sched_in()
Both of these can be found in tip/master.
Jamie Iles (5):
arm: provide a mechanism to reserve performance counters
arm/oprofile: reserve the PMU when starting
arm: use the spinlocked, generic atomic64 support
arm: enable support for software perf events
arm/perfevents: implement perf event support for ARMv6
arch/arm/Kconfig | 16 +
arch/arm/include/asm/atomic.h | 4 +
arch/arm/include/asm/perf_event.h | 31 +
arch/arm/include/asm/pmu.h | 75 ++
arch/arm/kernel/Makefile | 2 +
arch/arm/kernel/perf_event.c | 1342 +++++++++++++++++++++++++++++++
arch/arm/kernel/pmu.c | 105 +++
arch/arm/mm/fault.c | 7 +
arch/arm/oprofile/op_model_arm11_core.c | 4 +-
arch/arm/oprofile/op_model_arm11_core.h | 4 +-
arch/arm/oprofile/op_model_mpcore.c | 42 +-
arch/arm/oprofile/op_model_v6.c | 30 +-
arch/arm/oprofile/op_model_v7.c | 30 +-
arch/arm/oprofile/op_model_v7.h | 4 +-
arch/arm/oprofile/op_model_xscale.c | 35 +-
15 files changed, 1667 insertions(+), 64 deletions(-)
^ permalink raw reply [flat|nested] 28+ messages in thread* [PATCH 1/5] arm: provide a mechanism to reserve performance counters 2010-01-14 12:14 ARM perf events support v5 Jamie Iles @ 2010-01-14 12:14 ` Jamie Iles 2010-01-14 12:14 ` [PATCH 2/5] arm/oprofile: reserve the PMU when starting Jamie Iles 0 siblings, 1 reply; 28+ messages in thread From: Jamie Iles @ 2010-01-14 12:14 UTC (permalink / raw) To: linux-arm-kernel To add support for perf events and to allow the hardware counters to be shared with oprofile, we need a way to reserve access to the pmu (performance monitor unit). Cc: Will Deacon <will.deacon@arm.com> Signed-off-by: Jamie Iles <jamie.iles@picochip.com> --- arch/arm/Kconfig | 5 ++ arch/arm/include/asm/pmu.h | 75 +++++++++++++++++++++++++++++++ arch/arm/kernel/Makefile | 1 + arch/arm/kernel/pmu.c | 105 ++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 186 insertions(+), 0 deletions(-) create mode 100644 arch/arm/include/asm/pmu.h create mode 100644 arch/arm/kernel/pmu.c diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig index c2238cd..31d52ed 100644 --- a/arch/arm/Kconfig +++ b/arch/arm/Kconfig @@ -866,6 +866,11 @@ config XSCALE_PMU depends on CPU_XSCALE && !XSCALE_PMU_TIMER default y +config CPU_HAS_PMU + depends on CPU_V6 || CPU_V7 || XSCALE_PMU + default y + bool + if !MMU source "arch/arm/Kconfig-nommu" endif diff --git a/arch/arm/include/asm/pmu.h b/arch/arm/include/asm/pmu.h new file mode 100644 index 0000000..2829b9f --- /dev/null +++ b/arch/arm/include/asm/pmu.h @@ -0,0 +1,75 @@ +/* + * linux/arch/arm/include/asm/pmu.h + * + * Copyright (C) 2009 picoChip Designs Ltd, Jamie Iles + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + */ + +#ifndef __ARM_PMU_H__ +#define __ARM_PMU_H__ + +#ifdef CONFIG_CPU_HAS_PMU + +struct pmu_irqs { + const int *irqs; + int num_irqs; +}; + +/** + * reserve_pmu() - reserve the hardware performance counters + * + * Reserve the hardware performance counters in the system for exclusive use. + * The 'struct pmu_irqs' for the system is returned on success, ERR_PTR() + * encoded error on failure. + */ +extern const struct pmu_irqs * +reserve_pmu(void); + +/** + * release_pmu() - Relinquish control of the performance counters + * + * Release the performance counters and allow someone else to use them. + * Callers must have disabled the counters and released IRQs before calling + * this. The 'struct pmu_irqs' returned from reserve_pmu() must be passed as + * a cookie. + */ +extern int +release_pmu(const struct pmu_irqs *irqs); + +/** + * init_pmu() - Initialise the PMU. + * + * Initialise the system ready for PMU enabling. This should typically set the + * IRQ affinity and nothing else. The users (oprofile/perf events etc) will do + * the actual hardware initialisation. + */ +extern int +init_pmu(void); + +#else /* CONFIG_CPU_HAS_PMU */ + +static inline const struct pmu_irqs * +reserve_pmu(void) +{ + return ERR_PTR(-ENODEV); +} + +static inline int +release_pmu(const struct pmu_irqs *irqs) +{ + return -ENODEV; +} + +static inline int +init_pmu(void) +{ + return -ENODEV; +} + +#endif /* CONFIG_CPU_HAS_PMU */ + +#endif /* __ARM_PMU_H__ */ diff --git a/arch/arm/kernel/Makefile b/arch/arm/kernel/Makefile index dd00f74..216890d 100644 --- a/arch/arm/kernel/Makefile +++ b/arch/arm/kernel/Makefile @@ -46,6 +46,7 @@ obj-$(CONFIG_CPU_XSCALE) += xscale-cp0.o obj-$(CONFIG_CPU_XSC3) += xscale-cp0.o obj-$(CONFIG_CPU_MOHAWK) += xscale-cp0.o obj-$(CONFIG_IWMMXT) += iwmmxt.o +obj-$(CONFIG_CPU_HAS_PMU) += pmu.o AFLAGS_iwmmxt.o := -Wa,-mcpu=iwmmxt ifneq ($(CONFIG_ARCH_EBSA110),y) diff --git a/arch/arm/kernel/pmu.c b/arch/arm/kernel/pmu.c new file mode 100644 index 0000000..688d450 --- /dev/null +++ b/arch/arm/kernel/pmu.c @@ -0,0 +1,105 @@ +/* + * linux/arch/arm/kernel/pmu.c + * + * Copyright (C) 2009 picoChip Designs Ltd, Jamie Iles + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + */ + +#include <linux/cpumask.h> +#include <linux/err.h> +#include <linux/interrupt.h> +#include <linux/kernel.h> +#include <linux/module.h> + +#include <asm/pmu.h> + +/* + * Define the IRQs for the system. We could use something like a platform + * device but that seems fairly heavyweight for this. Also, the performance + * counters can't be removed or hotplugged. + * + * Ordering is important: init_pmu() will use the ordering to set the affinity + * to the corresponding core. e.g. the first interrupt will go to cpu 0, the + * second goes to cpu 1 etc. + */ +static const int irqs[] = { +#ifdef CONFIG_ARCH_PC3XX + IRQ_NPMUIRQ, +#elif defined(CONFIG_ARCH_OMAP2) + 3, +#elif defined(CONFIG_ARCH_BCMRING) + IRQ_PMUIRQ, +#elif defined(CONFIG_MACH_REALVIEW_EB) + IRQ_EB11MP_PMU_CPU0, + IRQ_EB11MP_PMU_CPU1, + IRQ_EB11MP_PMU_CPU2, + IRQ_EB11MP_PMU_CPU3, +#elif defined(CONFIG_ARCH_OMAP3) + INT_34XX_BENCH_MPU_EMUL, +#elif defined(CONFIG_ARCH_IOP32X) + IRQ_IOP32X_CORE_PMU, +#elif defined(CONFIG_ARCH_IOP33X) + IRQ_IOP33X_CORE_PMU, +#elif defined(CONFIG_ARCH_PXA) + IRQ_PMU, +#endif +}; + +static const struct pmu_irqs pmu_irqs = { + .irqs = irqs, + .num_irqs = ARRAY_SIZE(irqs), +}; + +static volatile long pmu_lock; + +const struct pmu_irqs * +reserve_pmu(void) +{ + return test_and_set_bit_lock(0, &pmu_lock) ? ERR_PTR(-EBUSY) : + &pmu_irqs; +} +EXPORT_SYMBOL_GPL(reserve_pmu); + +int +release_pmu(const struct pmu_irqs *irqs) +{ + if (WARN_ON(irqs != &pmu_irqs)) + return -EINVAL; + clear_bit_unlock(0, &pmu_lock); + return 0; +} +EXPORT_SYMBOL_GPL(release_pmu); + +static int +set_irq_affinity(int irq, + unsigned int cpu) +{ +#ifdef CONFIG_SMP + int err = irq_set_affinity(irq, cpumask_of(cpu)); + if (err) + pr_warning("unable to set irq affinity (irq=%d, cpu=%u)\n", + irq, cpu); + return err; +#else + return 0; +#endif +} + +int +init_pmu(void) +{ + int i, err = 0; + + for (i = 0; i < pmu_irqs.num_irqs; ++i) { + err = set_irq_affinity(pmu_irqs.irqs[i], i); + if (err) + break; + } + + return err; +} +EXPORT_SYMBOL_GPL(init_pmu); -- 1.6.5.4 ^ permalink raw reply related [flat|nested] 28+ messages in thread
* [PATCH 2/5] arm/oprofile: reserve the PMU when starting 2010-01-14 12:14 ` [PATCH 1/5] arm: provide a mechanism to reserve performance counters Jamie Iles @ 2010-01-14 12:14 ` Jamie Iles 2010-01-14 12:14 ` [PATCH 3/5] arm: use the spinlocked, generic atomic64 support Jamie Iles 0 siblings, 1 reply; 28+ messages in thread From: Jamie Iles @ 2010-01-14 12:14 UTC (permalink / raw) To: linux-arm-kernel Make sure that we have access to the performance counters and that they aren't being used by perf events or anything else. Cc: Will Deacon <will.deacon@arm.com> Cc: Jean Pihet <jpihet@mvista.com> Signed-off-by: Jamie Iles <jamie.iles@picochip.com> --- arch/arm/oprofile/op_model_arm11_core.c | 4 +- arch/arm/oprofile/op_model_arm11_core.h | 4 +- arch/arm/oprofile/op_model_mpcore.c | 42 ++++++++++++++++-------------- arch/arm/oprofile/op_model_v6.c | 30 ++++++++++++++-------- arch/arm/oprofile/op_model_v7.c | 30 ++++++++++++++-------- arch/arm/oprofile/op_model_v7.h | 4 +- arch/arm/oprofile/op_model_xscale.c | 35 ++++++++++++++----------- 7 files changed, 85 insertions(+), 64 deletions(-) diff --git a/arch/arm/oprofile/op_model_arm11_core.c b/arch/arm/oprofile/op_model_arm11_core.c index ad80752..ef3e265 100644 --- a/arch/arm/oprofile/op_model_arm11_core.c +++ b/arch/arm/oprofile/op_model_arm11_core.c @@ -132,7 +132,7 @@ static irqreturn_t arm11_pmu_interrupt(int irq, void *arg) return IRQ_HANDLED; } -int arm11_request_interrupts(int *irqs, int nr) +int arm11_request_interrupts(const int *irqs, int nr) { unsigned int i; int ret = 0; @@ -153,7 +153,7 @@ int arm11_request_interrupts(int *irqs, int nr) return ret; } -void arm11_release_interrupts(int *irqs, int nr) +void arm11_release_interrupts(const int *irqs, int nr) { unsigned int i; diff --git a/arch/arm/oprofile/op_model_arm11_core.h b/arch/arm/oprofile/op_model_arm11_core.h index 6f8538e..1902b99 100644 --- a/arch/arm/oprofile/op_model_arm11_core.h +++ b/arch/arm/oprofile/op_model_arm11_core.h @@ -39,7 +39,7 @@ int arm11_setup_pmu(void); int arm11_start_pmu(void); int arm11_stop_pmu(void); -int arm11_request_interrupts(int *, int); -void arm11_release_interrupts(int *, int); +int arm11_request_interrupts(const int *, int); +void arm11_release_interrupts(const int *, int); #endif diff --git a/arch/arm/oprofile/op_model_mpcore.c b/arch/arm/oprofile/op_model_mpcore.c index 4ce0f98..f73ce87 100644 --- a/arch/arm/oprofile/op_model_mpcore.c +++ b/arch/arm/oprofile/op_model_mpcore.c @@ -32,6 +32,7 @@ /* #define DEBUG */ #include <linux/types.h> #include <linux/errno.h> +#include <linux/err.h> #include <linux/sched.h> #include <linux/oprofile.h> #include <linux/interrupt.h> @@ -43,6 +44,7 @@ #include <mach/hardware.h> #include <mach/board-eb.h> #include <asm/system.h> +#include <asm/pmu.h> #include "op_counter.h" #include "op_arm_model.h" @@ -58,6 +60,7 @@ * Bitmask of used SCU counters */ static unsigned int scu_em_used; +static const struct pmu_irqs *pmu_irqs; /* * 2 helper fns take a counter number from 0-7 (not the userspace-visible counter number) @@ -225,33 +228,40 @@ static int em_setup_ctrs(void) return 0; } -static int arm11_irqs[] = { - [0] = IRQ_EB11MP_PMU_CPU0, - [1] = IRQ_EB11MP_PMU_CPU1, - [2] = IRQ_EB11MP_PMU_CPU2, - [3] = IRQ_EB11MP_PMU_CPU3 -}; - static int em_start(void) { int ret; - ret = arm11_request_interrupts(arm11_irqs, ARRAY_SIZE(arm11_irqs)); + pmu_irqs = reserve_pmu(); + if (IS_ERR(pmu_irqs)) { + ret = PTR_ERR(pmu_irqs); + goto out; + } + + ret = arm11_request_interrupts(pmu_irqs->irqs, pmu_irqs->num_irqs); if (ret == 0) { em_call_function(arm11_start_pmu); ret = scu_start(); - if (ret) - arm11_release_interrupts(arm11_irqs, ARRAY_SIZE(arm11_irqs)); + if (ret) { + arm11_release_interrupts(pmu_irqs->irqs, + pmu_irqs->num_irqs); + } else { + release_pmu(pmu_irqs); + pmu_irqs = NULL; + } } + +out: return ret; } static void em_stop(void) { em_call_function(arm11_stop_pmu); - arm11_release_interrupts(arm11_irqs, ARRAY_SIZE(arm11_irqs)); + arm11_release_interrupts(pmu_irqs->irqs, pmu_irqs->num_irqs); scu_stop(); + release_pmu(pmu_irqs); } /* @@ -283,15 +293,7 @@ static int em_setup(void) em_route_irq(IRQ_EB11MP_PMU_SCU6, 3); em_route_irq(IRQ_EB11MP_PMU_SCU7, 3); - /* - * Send CP15 PMU interrupts to the owner CPU. - */ - em_route_irq(IRQ_EB11MP_PMU_CPU0, 0); - em_route_irq(IRQ_EB11MP_PMU_CPU1, 1); - em_route_irq(IRQ_EB11MP_PMU_CPU2, 2); - em_route_irq(IRQ_EB11MP_PMU_CPU3, 3); - - return 0; + return init_pmu(); } struct op_arm_model_spec op_mpcore_spec = { diff --git a/arch/arm/oprofile/op_model_v6.c b/arch/arm/oprofile/op_model_v6.c index f7d2ec5..a22357a 100644 --- a/arch/arm/oprofile/op_model_v6.c +++ b/arch/arm/oprofile/op_model_v6.c @@ -19,39 +19,47 @@ /* #define DEBUG */ #include <linux/types.h> #include <linux/errno.h> +#include <linux/err.h> #include <linux/sched.h> #include <linux/oprofile.h> #include <linux/interrupt.h> #include <asm/irq.h> #include <asm/system.h> +#include <asm/pmu.h> #include "op_counter.h" #include "op_arm_model.h" #include "op_model_arm11_core.h" -static int irqs[] = { -#ifdef CONFIG_ARCH_OMAP2 - 3, -#endif -#ifdef CONFIG_ARCH_BCMRING - IRQ_PMUIRQ, /* for BCMRING, ARM PMU interrupt is 43 */ -#endif -}; +static const struct pmu_irqs *pmu_irqs; static void armv6_pmu_stop(void) { arm11_stop_pmu(); - arm11_release_interrupts(irqs, ARRAY_SIZE(irqs)); + arm11_release_interrupts(pmu_irqs->irqs, pmu_irqs->num_irqs); + release_pmu(pmu_irqs); + pmu_irqs = NULL; } static int armv6_pmu_start(void) { int ret; - ret = arm11_request_interrupts(irqs, ARRAY_SIZE(irqs)); - if (ret >= 0) + pmu_irqs = reserve_pmu(); + if (IS_ERR(pmu_irqs)) { + ret = PTR_ERR(pmu_irqs); + goto out; + } + + ret = arm11_request_interrupts(pmu_irqs->irqs, pmu_irqs->num_irqs); + if (ret >= 0) { ret = arm11_start_pmu(); + } else { + release_pmu(pmu_irqs); + pmu_irqs = NULL; + } +out: return ret; } diff --git a/arch/arm/oprofile/op_model_v7.c b/arch/arm/oprofile/op_model_v7.c index 2088a6c..8642d08 100644 --- a/arch/arm/oprofile/op_model_v7.c +++ b/arch/arm/oprofile/op_model_v7.c @@ -11,11 +11,14 @@ */ #include <linux/types.h> #include <linux/errno.h> +#include <linux/err.h> #include <linux/oprofile.h> #include <linux/interrupt.h> #include <linux/irq.h> #include <linux/smp.h> +#include <asm/pmu.h> + #include "op_counter.h" #include "op_arm_model.h" #include "op_model_v7.h" @@ -295,7 +298,7 @@ static irqreturn_t armv7_pmnc_interrupt(int irq, void *arg) return IRQ_HANDLED; } -int armv7_request_interrupts(int *irqs, int nr) +int armv7_request_interrupts(const int *irqs, int nr) { unsigned int i; int ret = 0; @@ -318,7 +321,7 @@ int armv7_request_interrupts(int *irqs, int nr) return ret; } -void armv7_release_interrupts(int *irqs, int nr) +void armv7_release_interrupts(const int *irqs, int nr) { unsigned int i; @@ -362,12 +365,7 @@ static void armv7_pmnc_dump_regs(void) } #endif - -static int irqs[] = { -#ifdef CONFIG_ARCH_OMAP3 - INT_34XX_BENCH_MPU_EMUL, -#endif -}; +static const struct pmu_irqs *pmu_irqs; static void armv7_pmnc_stop(void) { @@ -375,19 +373,29 @@ static void armv7_pmnc_stop(void) armv7_pmnc_dump_regs(); #endif armv7_stop_pmnc(); - armv7_release_interrupts(irqs, ARRAY_SIZE(irqs)); + armv7_release_interrupts(pmu_irqs->irqs, pmu_irqs->num_irqs); + release_pmu(pmu_irqs); + pmu_irqs = NULL; } static int armv7_pmnc_start(void) { int ret; + pmu_irqs = reserve_pmu(); + if (IS_ERR(pmu_irqs)) + return PTR_ERR(pmu_irqs); + #ifdef DEBUG armv7_pmnc_dump_regs(); #endif - ret = armv7_request_interrupts(irqs, ARRAY_SIZE(irqs)); - if (ret >= 0) + ret = armv7_request_interrupts(pmu_irqs->irqs, pmu_irqs->num_irqs); + if (ret >= 0) { armv7_start_pmnc(); + } else { + release_pmu(pmu_irqs); + pmu_irqs = NULL; + } return ret; } diff --git a/arch/arm/oprofile/op_model_v7.h b/arch/arm/oprofile/op_model_v7.h index 0e19bcc..9ca334b 100644 --- a/arch/arm/oprofile/op_model_v7.h +++ b/arch/arm/oprofile/op_model_v7.h @@ -97,7 +97,7 @@ int armv7_setup_pmu(void); int armv7_start_pmu(void); int armv7_stop_pmu(void); -int armv7_request_interrupts(int *, int); -void armv7_release_interrupts(int *, int); +int armv7_request_interrupts(const int *, int); +void armv7_release_interrupts(const int *, int); #endif diff --git a/arch/arm/oprofile/op_model_xscale.c b/arch/arm/oprofile/op_model_xscale.c index 724ab9c..1d34a02 100644 --- a/arch/arm/oprofile/op_model_xscale.c +++ b/arch/arm/oprofile/op_model_xscale.c @@ -17,12 +17,14 @@ /* #define DEBUG */ #include <linux/types.h> #include <linux/errno.h> +#include <linux/err.h> #include <linux/sched.h> #include <linux/oprofile.h> #include <linux/interrupt.h> #include <linux/irq.h> #include <asm/cputype.h> +#include <asm/pmu.h> #include "op_counter.h" #include "op_arm_model.h" @@ -33,17 +35,6 @@ #define PMU_RESET (CCNT_RESET | PMN_RESET) #define PMU_CNT64 0x008 /* Make CCNT count every 64th cycle */ -/* TODO do runtime detection */ -#ifdef CONFIG_ARCH_IOP32X -#define XSCALE_PMU_IRQ IRQ_IOP32X_CORE_PMU -#endif -#ifdef CONFIG_ARCH_IOP33X -#define XSCALE_PMU_IRQ IRQ_IOP33X_CORE_PMU -#endif -#ifdef CONFIG_ARCH_PXA -#define XSCALE_PMU_IRQ IRQ_PMU -#endif - /* * Different types of events that can be counted by the XScale PMU * as used by Oprofile userspace. Here primarily for documentation @@ -367,6 +358,8 @@ static irqreturn_t xscale_pmu_interrupt(int irq, void *arg) return IRQ_HANDLED; } +static const struct pmu_irqs *pmu_irqs; + static void xscale_pmu_stop(void) { u32 pmnc = read_pmnc(); @@ -374,20 +367,30 @@ static void xscale_pmu_stop(void) pmnc &= ~PMU_ENABLE; write_pmnc(pmnc); - free_irq(XSCALE_PMU_IRQ, results); + free_irq(pmu_irqs->irqs[0], results); + release_pmu(pmu_irqs); + pmu_irqs = NULL; } static int xscale_pmu_start(void) { int ret; - u32 pmnc = read_pmnc(); + u32 pmnc; + + pmu_irqs = reserve_pmu(); + if (IS_ERR(pmu_irqs)) + return PTR_ERR(pmu_irqs); + + pmnc = read_pmnc(); - ret = request_irq(XSCALE_PMU_IRQ, xscale_pmu_interrupt, IRQF_DISABLED, - "XScale PMU", (void *)results); + ret = request_irq(pmu_irqs->irqs[0], xscale_pmu_interrupt, + IRQF_DISABLED, "XScale PMU", (void *)results); if (ret < 0) { printk(KERN_ERR "oprofile: unable to request IRQ%d for XScale PMU\n", - XSCALE_PMU_IRQ); + pmu_irqs->irqs[0]); + release_pmu(pmu_irqs); + pmu_irqs = NULL; return ret; } -- 1.6.5.4 ^ permalink raw reply related [flat|nested] 28+ messages in thread
* [PATCH 3/5] arm: use the spinlocked, generic atomic64 support 2010-01-14 12:14 ` [PATCH 2/5] arm/oprofile: reserve the PMU when starting Jamie Iles @ 2010-01-14 12:14 ` Jamie Iles 0 siblings, 0 replies; 28+ messages in thread From: Jamie Iles @ 2010-01-14 12:14 UTC (permalink / raw) To: linux-arm-kernel perf events require that we can support atomic64's. There is a generic, spinlocked version that we can use until we have proper hardware support. Signed-off-by: Jamie Iles <jamie.iles@picochip.com> --- arch/arm/Kconfig | 1 + arch/arm/include/asm/atomic.h | 4 ++++ 2 files changed, 5 insertions(+), 0 deletions(-) diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig index 31d52ed..293a879 100644 --- a/arch/arm/Kconfig +++ b/arch/arm/Kconfig @@ -20,6 +20,7 @@ config ARM select HAVE_GENERIC_DMA_COHERENT select HAVE_KERNEL_GZIP select HAVE_KERNEL_LZO + select GENERIC_ATOMIC64 help The ARM series is a line of low-power-consumption RISC chip designs licensed by ARM Ltd and targeted at embedded applications and diff --git a/arch/arm/include/asm/atomic.h b/arch/arm/include/asm/atomic.h index d0daeab..ff286a8 100644 --- a/arch/arm/include/asm/atomic.h +++ b/arch/arm/include/asm/atomic.h @@ -15,6 +15,10 @@ #include <linux/types.h> #include <asm/system.h> +#ifdef CONFIG_GENERIC_ATOMIC64 +#include <asm-generic/atomic64.h> +#endif + #define ATOMIC_INIT(i) { (i) } #ifdef __KERNEL__ -- 1.6.5.4 ^ permalink raw reply related [flat|nested] 28+ messages in thread
end of thread, other threads:[~2010-01-14 12:14 UTC | newest]
Thread overview: 28+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2009-12-14 14:04 ARMv6 performance counters v2 Jamie Iles
2009-12-14 14:04 ` [PATCH 1/5] arm: provide a mechanism to reserve performance counters Jamie Iles
2009-12-14 14:04 ` [PATCH 2/5] arm/oprofile: reserve the PMU when starting Jamie Iles
2009-12-14 14:04 ` [PATCH 3/5] arm: use the spinlocked, generic atomic64 support Jamie Iles
2009-12-14 14:04 ` [PATCH 4/5] arm: enable support for software perf events Jamie Iles
2009-12-14 14:04 ` [PATCH 5/5] arm/perfevents: implement perf event support for ARMv6 Jamie Iles
2009-12-14 16:12 ` Jean Pihet
2009-12-14 16:33 ` Jamie Iles
2009-12-14 16:57 ` Jean Pihet
2009-12-14 17:09 ` Will Deacon
2009-12-14 16:13 ` Will Deacon
2009-12-14 16:20 ` Jamie Iles
2009-12-14 16:24 ` Will Deacon
2009-12-14 17:38 ` [PATCH 3/5] arm: use the spinlocked, generic atomic64 support Nicolas Pitre
2009-12-14 19:36 ` Will Deacon
[not found] ` <001301ca7cf4$c04481a0$40cd84e0$%deacon@arm.com>
2009-12-14 19:52 ` Nicolas Pitre
2009-12-15 10:24 ` Catalin Marinas
2009-12-14 16:01 ` [PATCH 2/5] arm/oprofile: reserve the PMU when starting Jean Pihet
2009-12-14 16:04 ` Will Deacon
2009-12-14 16:10 ` Jamie Iles
2009-12-14 14:39 ` [PATCH 1/5] arm: provide a mechanism to reserve performance counters Will Deacon
2009-12-14 15:03 ` Jamie Iles
2009-12-14 16:01 ` Jean Pihet
-- strict thread matches above, loose matches on Subject: below --
2009-12-15 11:15 ARMv6 performance counters v3 Jamie Iles
2009-12-15 11:15 ` [PATCH 1/5] arm: provide a mechanism to reserve performance counters Jamie Iles
2009-12-15 11:15 ` [PATCH 2/5] arm/oprofile: reserve the PMU when starting Jamie Iles
2009-12-15 11:15 ` [PATCH 3/5] arm: use the spinlocked, generic atomic64 support Jamie Iles
2010-01-04 10:48 ARM perf events support v4 Jamie Iles
2010-01-04 10:48 ` [PATCH 1/5] arm: provide a mechanism to reserve performance counters Jamie Iles
2010-01-04 10:48 ` [PATCH 2/5] arm/oprofile: reserve the PMU when starting Jamie Iles
2010-01-04 10:48 ` [PATCH 3/5] arm: use the spinlocked, generic atomic64 support Jamie Iles
2010-01-05 18:57 ` Jamie Lokier
2010-01-05 19:08 ` Jamie Iles
2010-01-14 12:14 ARM perf events support v5 Jamie Iles
2010-01-14 12:14 ` [PATCH 1/5] arm: provide a mechanism to reserve performance counters Jamie Iles
2010-01-14 12:14 ` [PATCH 2/5] arm/oprofile: reserve the PMU when starting Jamie Iles
2010-01-14 12:14 ` [PATCH 3/5] arm: use the spinlocked, generic atomic64 support Jamie Iles
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).