* Re: [RFC/PATCH] mm/futex: Fix futex writes on archs with SW tracking of dirty & young
From: Mike Frysinger @ 2011-07-28 0:12 UTC (permalink / raw)
To: Benjamin Herrenschmidt
Cc: tony.luck, Shan Hai, Peter Zijlstra, linux-kernel, cmetcalf,
David Howells, paulus, uclinux-dist-devel, tglx, walken,
linuxppc-dev, akpm
In-Reply-To: <1311762043.25044.679.camel@pasglop>
On Wed, Jul 27, 2011 at 03:20, Benjamin Herrenschmidt wrote:
> Hoping the BUG() isn't trippable by userspace but then it's no mmu, it's
> not like we care what userspace can do right :-)
side note ... common misconception that "no mmu" == "no memory
protection". a few of the nommu processors have memory protection,
just no virtual<->physical translation.
thanks for the patch !
-mike
^ permalink raw reply
* [PATCH] [6/99] seqlock: Don't smp_rmb in seqlock reader spin loop
From: Andi Kleen @ 2011-07-27 21:48 UTC (permalink / raw)
To: miltonm, linuxppc-dev, torvalds, andi, npiggin, benh, anton,
paulmck, eric.dumazet, ak, tglx, gregkh, linux-kernel, stable,
tim.bird
In-Reply-To: <20110727247.325703029@firstfloor.org>
2.6.35-longterm review patch. If anyone has any objections, please let me know.
------------------
From: Milton Miller <miltonm@bga.com>
commit 5db1256a5131d3b133946fa02ac9770a784e6eb2 upstream.
Move the smp_rmb after cpu_relax loop in read_seqlock and add
ACCESS_ONCE to make sure the test and return are consistent.
A multi-threaded core in the lab didn't like the update
from 2.6.35 to 2.6.36, to the point it would hang during
boot when multiple threads were active. Bisection showed
af5ab277ded04bd9bc6b048c5a2f0e7d70ef0867 (clockevents:
Remove the per cpu tick skew) as the culprit and it is
supported with stack traces showing xtime_lock waits including
tick_do_update_jiffies64 and/or update_vsyscall.
Experimentation showed the combination of cpu_relax and smp_rmb
was significantly slowing the progress of other threads sharing
the core, and this patch is effective in avoiding the hang.
A theory is the rmb is affecting the whole core while the
cpu_relax is causing a resource rebalance flush, together they
cause an interfernce cadance that is unbroken when the seqlock
reader has interrupts disabled.
At first I was confused why the refactor in
3c22cd5709e8143444a6d08682a87f4c57902df3 (kernel: optimise
seqlock) didn't affect this patch application, but after some
study that affected seqcount not seqlock. The new seqcount was
not factored back into the seqlock. I defer that the future.
While the removal of the timer interrupt offset created
contention for the xtime lock while a cpu does the
additonal work to update the system clock, the seqlock
implementation with the tight rmb spin loop goes back much
further, and is just waiting for the right trigger.
Signed-off-by: Milton Miller <miltonm@bga.com>
Cc: <linuxppc-dev@lists.ozlabs.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Andi Kleen <andi@firstfloor.org>
Cc: Nick Piggin <npiggin@kernel.dk>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Anton Blanchard <anton@samba.org>
Cc: Paul McKenney <paulmck@linux.vnet.ibm.com>
Acked-by: Eric Dumazet <eric.dumazet@gmail.com>
Signed-off-by: Andi Kleen <ak@linux.intel.com>
Link: http://lkml.kernel.org/r/%3Cseqlock-rmb%40mdm.bga.com%3E
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
---
include/linux/seqlock.h | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
Index: linux-2.6.35.y/include/linux/seqlock.h
===================================================================
--- linux-2.6.35.y.orig/include/linux/seqlock.h
+++ linux-2.6.35.y/include/linux/seqlock.h
@@ -88,12 +88,12 @@ static __always_inline unsigned read_seq
unsigned ret;
repeat:
- ret = sl->sequence;
- smp_rmb();
+ ret = ACCESS_ONCE(sl->sequence);
if (unlikely(ret & 1)) {
cpu_relax();
goto repeat;
}
+ smp_rmb();
return ret;
}
^ permalink raw reply
* Re: HELP:PowerPc-Linux kernel
From: Scott Wood @ 2011-07-27 20:27 UTC (permalink / raw)
To: naresh.kamboju; +Cc: linuxppc-dev
In-Reply-To: <35CC4C9595855B43903A67B297EFA8E3C544FD@HYD-MKD-MBX01.wipro.com>
On Wed, 27 Jul 2011 20:33:54 +0530
<naresh.kamboju@wipro.com> wrote:
> Hi,
>
>
>
> I have take up the new assignment Board bring up activity with Linux
> kernel on PowerPC MPC8272.
>
> I have been searching Linux bring up on PowerPC processor in Google and
> IBM wiki and not found good stuff.
>
> It would be more helpful for me if you could share related documents.
Look at the support in current kernels for 82xx-based boards such as
mpc8272ads. Read the documentation on device trees
(Documentation/devicetree, devicetree.org, ePAPR).
-Scott
^ permalink raw reply
* [PATCH] ppc: Remove duplicate definition of PV_POWER7
From: Peter Zijlstra @ 2011-07-27 17:27 UTC (permalink / raw)
To: Benjamin Herrenschmidt; +Cc: linuxppc-dev, Anton Blanchard
One definition of PV_POWER7 seems enough to me.
Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
---
arch/powerpc/include/asm/reg.h | 1 -
1 files changed, 0 insertions(+), 1 deletions(-)
diff --git a/arch/powerpc/include/asm/reg.h b/arch/powerpc/include/asm/reg.=
h
index c5cae0d..fedf93b 100644
--- a/arch/powerpc/include/asm/reg.h
+++ b/arch/powerpc/include/asm/reg.h
@@ -986,7 +986,6 @@
#define PV_970 0x0039
#define PV_POWER5 0x003A
#define PV_POWER5p 0x003B
-#define PV_POWER7 0x003F
#define PV_970FX 0x003C
#define PV_POWER6 0x003E
#define PV_POWER7 0x003F
^ permalink raw reply related
* HELP:PowerPc-Linux kernel
From: naresh.kamboju @ 2011-07-27 15:03 UTC (permalink / raw)
To: linuxppc-dev; +Cc: cort, linas, hollis
[-- Attachment #1: Type: text/plain, Size: 1080 bytes --]
Hi,
I have take up the new assignment Board bring up activity with Linux
kernel on PowerPC MPC8272.
I have been searching Linux bring up on PowerPC processor in Google and
IBM wiki and not found good stuff.
It would be more helpful for me if you could share related documents.
Best regards
Naresh Kamboju
Please do not print this email unless it is absolutely necessary.
The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments.
WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email.
www.wipro.com
[-- Attachment #2: Type: text/html, Size: 5835 bytes --]
^ permalink raw reply
* [PATCH] PSeries: Cancel RTAS event scan before firmware flash
From: Ravi K. Nittala @ 2011-07-27 12:09 UTC (permalink / raw)
To: linuxppc-dev
Cc: antonb, subrata.modak, mikey, sbest, suzuki, ranittal,
divya.vikas
The firmware flash update is conducted using an RTAS call, that is serialized
by lock_rtas() which uses spin_lock. rtasd keeps scanning for the RTAS events
generated on the machine. This is performed via a delayed workqueue, invoking
an RTAS call to scan the events.
The flash update takes a while to complete and during this time, any other
RTAS call has to wait. In this case, rtas_event_scan() waits for a long time
on the spin_lock resulting in a soft lockup.
Approaches to fix the issue :
Approach 1: Stop all the other CPUs before we start flashing the firmware.
Before the rtas firmware update starts, all other CPUs should be stopped.
Which means no other CPU should be in lock_rtas(). We do not want other CPUs
execute while FW update is in progress and the system will be rebooted anyway
after the update.
--- arch/powerpc/kernel/setup-common.c.orig 2011-07-01 22:41:12.952507971 -0400
+++ arch/powerpc/kernel/setup-common.c 2011-07-01 22:48:31.182507915 -0400
@@ -109,11 +109,12 @@ void machine_shutdown(void)
void machine_restart(char *cmd)
{
machine_shutdown();
- if (ppc_md.restart)
- ppc_md.restart(cmd);
#ifdef CONFIG_SMP
- smp_send_stop();
+ smp_send_stop();
#endif
+ if (ppc_md.restart)
+ ppc_md.restart(cmd);
+
printk(KERN_EMERG "System Halted, OK to turn off power\n");
local_irq_disable();
while (1) ;
Problems with this approach:
Stopping the CPUs suddenly may cause other serious problems depending on what
was running on them. Hence, this approach cannot be considered.
Approach 2: Cancel the rtas_scan_event work before starting the firmware flash.
Just before the flash update is performed, the queued rtas_event_scan() work
item is cancelled from the work queue so that there is no other RTAS call
issued while the flash is in progress. After the flash completes, the system
reboots and the rtas_event_scan() is rescheduled.
Approach 2 looks to be a better solution than Approach 1. Kindly let us know
your thoughts. Patch attached.
Signed-off-by: Suzuki Poulose <suzuki@in.ibm.com>
Signed-off-by: Ravi Nittala <ravi.nittala@in.ibm.com>
---
arch/powerpc/include/asm/rtas.h | 2 ++
arch/powerpc/kernel/rtas_flash.c | 6 ++++++
arch/powerpc/kernel/rtasd.c | 6 ++++++
3 files changed, 14 insertions(+), 0 deletions(-)
diff --git a/arch/powerpc/include/asm/rtas.h b/arch/powerpc/include/asm/rtas.h
index 58625d1..3f26f87 100644
--- a/arch/powerpc/include/asm/rtas.h
+++ b/arch/powerpc/include/asm/rtas.h
@@ -245,6 +245,8 @@ extern int early_init_dt_scan_rtas(unsigned long node,
extern void pSeries_log_error(char *buf, unsigned int err_type, int fatal);
+extern bool rtas_cancel_event_scan(void);
+
/* Error types logged. */
#define ERR_FLAG_ALREADY_LOGGED 0x0
#define ERR_FLAG_BOOT 0x1 /* log was pulled from NVRAM on boot */
diff --git a/arch/powerpc/kernel/rtas_flash.c b/arch/powerpc/kernel/rtas_flash.c
index e037c74..4174b4b 100644
--- a/arch/powerpc/kernel/rtas_flash.c
+++ b/arch/powerpc/kernel/rtas_flash.c
@@ -568,6 +568,12 @@ static void rtas_flash_firmware(int reboot_type)
}
/*
+ * Just before starting the firmware flash, cancel the event scan work
+ * to avoid any soft lockup issues.
+ */
+ rtas_cancel_event_scan();
+
+ /*
* NOTE: the "first" block must be under 4GB, so we create
* an entry with no data blocks in the reserved buffer in
* the kernel data segment.
diff --git a/arch/powerpc/kernel/rtasd.c b/arch/powerpc/kernel/rtasd.c
index 481ef06..e8f03fa 100644
--- a/arch/powerpc/kernel/rtasd.c
+++ b/arch/powerpc/kernel/rtasd.c
@@ -472,6 +472,12 @@ static void start_event_scan(void)
&event_scan_work, event_scan_delay);
}
+/* Cancel the rtas event scan work */
+bool rtas_cancel_event_scan(void)
+{
+ return cancel_delayed_work_sync(&event_scan_work);
+}
+
static int __init rtas_init(void)
{
struct proc_dir_entry *entry;
^ permalink raw reply related
* Re: [RFC/PATCH] mm/futex: Fix futex writes on archs with SW tracking of dirty & young
From: Benjamin Herrenschmidt @ 2011-07-27 10:20 UTC (permalink / raw)
To: Peter Zijlstra
Cc: tony.luck, Mike Frysinger, Shan Hai, linux-kernel, cmetcalf,
David Howells, paulus, uclinux-dist-devel, tglx, walken,
linuxppc-dev, akpm
In-Reply-To: <1311761831.24752.413.camel@twins>
On Wed, 2011-07-27 at 12:17 +0200, Peter Zijlstra wrote:
> On Wed, 2011-07-27 at 11:09 +0100, David Howells wrote:
> > Can you inline this for the NOMMU case please?
>
> ---
> Subject: mm: Fix fixup_user_fault() for MMU=n
>
> In commit 2efaca927 ("mm/futex: fix futex writes on archs with SW
> tracking of dirty & young") we forgot about MMU=n. This patch fixes
> that.
>
> Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Hoping the BUG() isn't trippable by userspace but then it's no mmu, it's
not like we care what userspace can do right :-)
Acked-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Thanks !
Cheers,
Ben.
> ---
> Index: linux-2.6/include/linux/mm.h
> ===================================================================
> --- linux-2.6.orig/include/linux/mm.h
> +++ linux-2.6/include/linux/mm.h
> @@ -962,6 +962,8 @@ int invalidate_inode_page(struct page *p
> #ifdef CONFIG_MMU
> extern int handle_mm_fault(struct mm_struct *mm, struct vm_area_struct *vma,
> unsigned long address, unsigned int flags);
> +extern int fixup_user_fault(struct task_struct *tsk, struct mm_struct *mm,
> + unsigned long address, unsigned int fault_flags);
> #else
> static inline int handle_mm_fault(struct mm_struct *mm,
> struct vm_area_struct *vma, unsigned long address,
> @@ -971,6 +973,14 @@ static inline int handle_mm_fault(struct
> BUG();
> return VM_FAULT_SIGBUS;
> }
> +static inline int fixup_user_fault(struct task_struct *tsk,
> + struct mm_struct *mm, unsigned long address,
> + unsigned int fault_flags)
> +{
> + /* should never happen if there's no MMU */
> + BUG();
> + return -EFAULT;
> +}
> #endif
>
> extern int make_pages_present(unsigned long addr, unsigned long end);
> @@ -988,8 +998,6 @@ int get_user_pages(struct task_struct *t
> int get_user_pages_fast(unsigned long start, int nr_pages, int write,
> struct page **pages);
> struct page *get_dump_page(unsigned long addr);
> -extern int fixup_user_fault(struct task_struct *tsk, struct mm_struct *mm,
> - unsigned long address, unsigned int fault_flags);
>
> extern int try_to_release_page(struct page * page, gfp_t gfp_mask);
> extern void do_invalidatepage(struct page *page, unsigned long offset);
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply
* Re: [RFC/PATCH] mm/futex: Fix futex writes on archs with SW tracking of dirty & young
From: Peter Zijlstra @ 2011-07-27 10:17 UTC (permalink / raw)
To: David Howells
Cc: tony.luck, Mike Frysinger, Shan Hai, linux-kernel, cmetcalf,
paulus, uclinux-dist-devel, tglx, walken, linuxppc-dev, akpm
In-Reply-To: <20368.1311761379@redhat.com>
On Wed, 2011-07-27 at 11:09 +0100, David Howells wrote:
> Can you inline this for the NOMMU case please?
---
Subject: mm: Fix fixup_user_fault() for MMU=3Dn=20
In commit 2efaca927 ("mm/futex: fix futex writes on archs with SW
tracking of dirty & young") we forgot about MMU=3Dn. This patch fixes
that.
Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
---
Index: linux-2.6/include/linux/mm.h
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
--- linux-2.6.orig/include/linux/mm.h
+++ linux-2.6/include/linux/mm.h
@@ -962,6 +962,8 @@ int invalidate_inode_page(struct page *p
#ifdef CONFIG_MMU
extern int handle_mm_fault(struct mm_struct *mm, struct vm_area_struct *vm=
a,
unsigned long address, unsigned int flags);
+extern int fixup_user_fault(struct task_struct *tsk, struct mm_struct *mm,
+ unsigned long address, unsigned int fault_flags);
#else
static inline int handle_mm_fault(struct mm_struct *mm,
struct vm_area_struct *vma, unsigned long address,
@@ -971,6 +973,14 @@ static inline int handle_mm_fault(struct
BUG();
return VM_FAULT_SIGBUS;
}
+static inline int fixup_user_fault(struct task_struct *tsk,=20
+ struct mm_struct *mm, unsigned long address,
+ unsigned int fault_flags)
+{
+ /* should never happen if there's no MMU */
+ BUG();
+ return -EFAULT;
+}
#endif
=20
extern int make_pages_present(unsigned long addr, unsigned long end);
@@ -988,8 +998,6 @@ int get_user_pages(struct task_struct *t
int get_user_pages_fast(unsigned long start, int nr_pages, int write,
struct page **pages);
struct page *get_dump_page(unsigned long addr);
-extern int fixup_user_fault(struct task_struct *tsk, struct mm_struct *mm,
- unsigned long address, unsigned int fault_flags);
=20
extern int try_to_release_page(struct page * page, gfp_t gfp_mask);
extern void do_invalidatepage(struct page *page, unsigned long offset);
^ permalink raw reply
* Re: [RFC/PATCH] mm/futex: Fix futex writes on archs with SW tracking of dirty & young
From: David Howells @ 2011-07-27 10:09 UTC (permalink / raw)
To: Peter Zijlstra
Cc: tony.luck, Mike Frysinger, Shan Hai, linux-kernel, cmetcalf,
dhowells, paulus, uclinux-dist-devel, tglx, walken, linuxppc-dev,
akpm
In-Reply-To: <1311757190.24752.406.camel@twins>
Peter Zijlstra <peterz@infradead.org> wrote:
> > What should nommu do anyways ? it's not like there's much it can do
> > right ? It should never even hit the fault path to start with ...
>
> Something like the below makes a nommu arm config build.. David, is this
> indeed the correct thing to do for nommu?
>
> ---
> Index: linux-2.6/mm/nommu.c
> ===================================================================
> --- linux-2.6.orig/mm/nommu.c
> +++ linux-2.6/mm/nommu.c
> @@ -190,6 +190,12 @@ int get_user_pages(struct task_struct *t
> }
> EXPORT_SYMBOL(get_user_pages);
>
> +int fixup_user_fault(struct task_struct *tsk, struct mm_struct *mm,
> + unsigned long address, unsigned int fault_flags)
> +{
> + BUG(); /* nommu should never call this */
> +}
> +
> /**
> * follow_pfn - look up PFN at a user virtual address
> * @vma: memory mapping
Or perhaps send SEGV? Can 'address' be bad at this point?
Can you inline this for the NOMMU case please?
David
^ permalink raw reply
* [PATCHv4 01/11] atomic: add *_dec_not_zero
From: Sven Eckelmann @ 2011-07-27 9:47 UTC (permalink / raw)
To: linux-arch
Cc: linux-m32r-ja, linux-mips, linux-ia64, linux-doc, H. Peter Anvin,
Heiko Carstens, Randy Dunlap, Paul Mackerras, Helge Deller,
sparclinux, Sven Eckelmann, linux-s390, Russell King,
user-mode-linux-devel, Richard Weinberger, Hirokazu Takata, x86,
James E.J. Bottomley, Ingo Molnar, Matt Turner, Fenghua Yu,
Arnd Bergmann, Jeff Dike, Chris Metcalf, linux-m32r,
Ivan Kokshaysky, Thomas Gleixner, linux-arm-kernel,
Richard Henderson, Tony Luck, linux-parisc, linux-kernel,
Ralf Baechle, Kyle McMartin, linux-alpha, Martin Schwidefsky,
linux390, linuxppc-dev, David S. Miller
Introduce an *_dec_not_zero operation. Make this a special case of
*_add_unless because batman-adv uses atomic_dec_not_zero in different
places like re-broadcast queue or aggregation queue management. There
are other non-final patches which may also want to use this macro.
Reported-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Sven Eckelmann <sven@narfation.org>
Cc: Randy Dunlap <rdunlap@xenotime.net>
Cc: Richard Henderson <rth@twiddle.net>
Cc: Ivan Kokshaysky <ink@jurassic.park.msu.ru>
Cc: Matt Turner <mattst88@gmail.com>
Cc: Russell King <linux@arm.linux.org.uk>
Cc: Tony Luck <tony.luck@intel.com>
Cc: Fenghua Yu <fenghua.yu@intel.com>
Cc: Hirokazu Takata <takata@linux-m32r.org>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: Kyle McMartin <kyle@mcmartin.ca>
Cc: Helge Deller <deller@gmx.de>
Cc: "James E.J. Bottomley" <jejb@parisc-linux.org>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
Cc: linux390@de.ibm.com
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Chris Metcalf <cmetcalf@tilera.com>
Cc: Jeff Dike <jdike@addtoit.com>
Cc: Richard Weinberger <richard@nod.at>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: x86@kernel.org
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: linux-doc@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Cc: linux-alpha@vger.kernel.org
Cc: linux-arm-kernel@lists.infradead.org
Cc: linux-ia64@vger.kernel.org
Cc: linux-m32r@ml.linux-m32r.org
Cc: linux-m32r-ja@ml.linux-m32r.org
Cc: linux-mips@linux-mips.org
Cc: linux-parisc@vger.kernel.org
Cc: linuxppc-dev@lists.ozlabs.org
Cc: linux-s390@vger.kernel.org
Cc: sparclinux@vger.kernel.org
Cc: user-mode-linux-devel@lists.sourceforge.net
---
David S. Miller recommended this change in
https://lists.open-mesh.org/pipermail/b.a.t.m.a.n/2011-May/004560.html
Arnd Bergmann wanted to apply it in 201106172320.26476.arnd@arndb.de
... and then Arun Sharma created a big merge conflict with
https://lkml.org/lkml/2011/6/6/430
I don't think that it is a a good idea to asume that everyone still agrees
with the patch after I've rewritten it.
Documentation/atomic_ops.txt | 1 +
arch/alpha/include/asm/atomic.h | 1 +
arch/alpha/include/asm/local.h | 1 +
arch/arm/include/asm/atomic.h | 1 +
arch/ia64/include/asm/atomic.h | 1 +
arch/m32r/include/asm/local.h | 1 +
arch/mips/include/asm/atomic.h | 1 +
arch/mips/include/asm/local.h | 1 +
arch/parisc/include/asm/atomic.h | 1 +
arch/powerpc/include/asm/atomic.h | 1 +
arch/powerpc/include/asm/local.h | 1 +
arch/s390/include/asm/atomic.h | 1 +
arch/sparc/include/asm/atomic_64.h | 1 +
arch/tile/include/asm/atomic_32.h | 1 +
arch/tile/include/asm/atomic_64.h | 1 +
arch/um/sys-i386/atomic64_cx8_32.S | 28 ++++++++++++++++++++++++++++
arch/x86/include/asm/atomic64_32.h | 12 ++++++++++++
arch/x86/include/asm/atomic64_64.h | 1 +
arch/x86/include/asm/local.h | 1 +
arch/x86/lib/atomic64_32.c | 4 ++++
arch/x86/lib/atomic64_386_32.S | 21 +++++++++++++++++++++
arch/x86/lib/atomic64_cx8_32.S | 28 ++++++++++++++++++++++++++++
include/asm-generic/atomic-long.h | 2 ++
include/asm-generic/atomic64.h | 1 +
include/asm-generic/local.h | 1 +
include/asm-generic/local64.h | 2 ++
include/linux/atomic.h | 9 +++++++++
27 files changed, 125 insertions(+), 0 deletions(-)
diff --git a/Documentation/atomic_ops.txt b/Documentation/atomic_ops.txt
index 3bd585b..1eec221 100644
--- a/Documentation/atomic_ops.txt
+++ b/Documentation/atomic_ops.txt
@@ -190,6 +190,7 @@ atomic_add_unless requires explicit memory barriers around the operation
unless it fails (returns 0).
atomic_inc_not_zero, equivalent to atomic_add_unless(v, 1, 0)
+atomic_dec_not_zero, equivalent to atomic_add_unless(v, -1, 0)
If a caller requires memory barrier semantics around an atomic_t
diff --git a/arch/alpha/include/asm/atomic.h b/arch/alpha/include/asm/atomic.h
index 640f909..09d1571 100644
--- a/arch/alpha/include/asm/atomic.h
+++ b/arch/alpha/include/asm/atomic.h
@@ -225,6 +225,7 @@ static __inline__ int atomic64_add_unless(atomic64_t *v, long a, long u)
}
#define atomic64_inc_not_zero(v) atomic64_add_unless((v), 1, 0)
+#define atomic64_dec_not_zero(v) atomic64_add_unless((v), -1, 0)
#define atomic_add_negative(a, v) (atomic_add_return((a), (v)) < 0)
#define atomic64_add_negative(a, v) (atomic64_add_return((a), (v)) < 0)
diff --git a/arch/alpha/include/asm/local.h b/arch/alpha/include/asm/local.h
index 9c94b84..51eb678 100644
--- a/arch/alpha/include/asm/local.h
+++ b/arch/alpha/include/asm/local.h
@@ -79,6 +79,7 @@ static __inline__ long local_sub_return(long i, local_t * l)
c != (u); \
})
#define local_inc_not_zero(l) local_add_unless((l), 1, 0)
+#define local_dec_not_zero(l) local_add_unless((l), -1, 0)
#define local_add_negative(a, l) (local_add_return((a), (l)) < 0)
diff --git a/arch/arm/include/asm/atomic.h b/arch/arm/include/asm/atomic.h
index 86976d0..80ed975 100644
--- a/arch/arm/include/asm/atomic.h
+++ b/arch/arm/include/asm/atomic.h
@@ -458,6 +458,7 @@ static inline int atomic64_add_unless(atomic64_t *v, u64 a, u64 u)
#define atomic64_dec_return(v) atomic64_sub_return(1LL, (v))
#define atomic64_dec_and_test(v) (atomic64_dec_return((v)) == 0)
#define atomic64_inc_not_zero(v) atomic64_add_unless((v), 1LL, 0LL)
+#define atomic64_dec_not_zero(v) atomic64_add_unless((v), -1LL, 0LL)
#endif /* !CONFIG_GENERIC_ATOMIC64 */
#endif
diff --git a/arch/ia64/include/asm/atomic.h b/arch/ia64/include/asm/atomic.h
index 3fad89e..af6e9b2 100644
--- a/arch/ia64/include/asm/atomic.h
+++ b/arch/ia64/include/asm/atomic.h
@@ -122,6 +122,7 @@ static __inline__ long atomic64_add_unless(atomic64_t *v, long a, long u)
}
#define atomic64_inc_not_zero(v) atomic64_add_unless((v), 1, 0)
+#define atomic64_dec_not_zero(v) atomic64_add_unless((v), -1, 0)
#define atomic_add_return(i,v) \
({ \
diff --git a/arch/m32r/include/asm/local.h b/arch/m32r/include/asm/local.h
index 734bca8..d536082 100644
--- a/arch/m32r/include/asm/local.h
+++ b/arch/m32r/include/asm/local.h
@@ -272,6 +272,7 @@ static inline int local_add_unless(local_t *l, long a, long u)
}
#define local_inc_not_zero(l) local_add_unless((l), 1, 0)
+#define local_dec_not_zero(l) local_add_unless((l), -1, 0)
static inline void local_clear_mask(unsigned long mask, local_t *addr)
{
diff --git a/arch/mips/include/asm/atomic.h b/arch/mips/include/asm/atomic.h
index 1d93f81..babb043 100644
--- a/arch/mips/include/asm/atomic.h
+++ b/arch/mips/include/asm/atomic.h
@@ -697,6 +697,7 @@ static __inline__ int atomic64_add_unless(atomic64_t *v, long a, long u)
}
#define atomic64_inc_not_zero(v) atomic64_add_unless((v), 1, 0)
+#define atomic64_dec_not_zero(v) atomic64_add_unless((v), -1, 0)
#define atomic64_dec_return(v) atomic64_sub_return(1, (v))
#define atomic64_inc_return(v) atomic64_add_return(1, (v))
diff --git a/arch/mips/include/asm/local.h b/arch/mips/include/asm/local.h
index 94fde8d..0242256 100644
--- a/arch/mips/include/asm/local.h
+++ b/arch/mips/include/asm/local.h
@@ -137,6 +137,7 @@ static __inline__ long local_sub_return(long i, local_t * l)
c != (u); \
})
#define local_inc_not_zero(l) local_add_unless((l), 1, 0)
+#define local_dec_not_zero(l) local_add_unless((l), -1, 0)
#define local_dec_return(l) local_sub_return(1, (l))
#define local_inc_return(l) local_add_return(1, (l))
diff --git a/arch/parisc/include/asm/atomic.h b/arch/parisc/include/asm/atomic.h
index b1dc71f..8a50234 100644
--- a/arch/parisc/include/asm/atomic.h
+++ b/arch/parisc/include/asm/atomic.h
@@ -334,6 +334,7 @@ static __inline__ int atomic64_add_unless(atomic64_t *v, long a, long u)
}
#define atomic64_inc_not_zero(v) atomic64_add_unless((v), 1, 0)
+#define atomic64_dec_not_zero(v) atomic64_add_unless((v), -1, 0)
#endif /* !CONFIG_64BIT */
diff --git a/arch/powerpc/include/asm/atomic.h b/arch/powerpc/include/asm/atomic.h
index e2a4c26..c0131a6 100644
--- a/arch/powerpc/include/asm/atomic.h
+++ b/arch/powerpc/include/asm/atomic.h
@@ -468,6 +468,7 @@ static __inline__ int atomic64_add_unless(atomic64_t *v, long a, long u)
}
#define atomic64_inc_not_zero(v) atomic64_add_unless((v), 1, 0)
+#define atomic64_dec_not_zero(v) atomic64_add_unless((v), -1, 0)
#endif /* __powerpc64__ */
diff --git a/arch/powerpc/include/asm/local.h b/arch/powerpc/include/asm/local.h
index b8da913..d182e34 100644
--- a/arch/powerpc/include/asm/local.h
+++ b/arch/powerpc/include/asm/local.h
@@ -134,6 +134,7 @@ static __inline__ int local_add_unless(local_t *l, long a, long u)
}
#define local_inc_not_zero(l) local_add_unless((l), 1, 0)
+#define local_dec_not_zero(l) local_add_unless((l), -1, 0)
#define local_sub_and_test(a, l) (local_sub_return((a), (l)) == 0)
#define local_dec_and_test(l) (local_dec_return((l)) == 0)
diff --git a/arch/s390/include/asm/atomic.h b/arch/s390/include/asm/atomic.h
index 8517d2a..92e7d5d 100644
--- a/arch/s390/include/asm/atomic.h
+++ b/arch/s390/include/asm/atomic.h
@@ -325,6 +325,7 @@ static inline long long atomic64_dec_if_positive(atomic64_t *v)
#define atomic64_dec_return(_v) atomic64_sub_return(1, _v)
#define atomic64_dec_and_test(_v) (atomic64_sub_return(1, _v) == 0)
#define atomic64_inc_not_zero(v) atomic64_add_unless((v), 1, 0)
+#define atomic64_dec_not_zero(v) atomic64_add_unless((v), -1, 0)
#define smp_mb__before_atomic_dec() smp_mb()
#define smp_mb__after_atomic_dec() smp_mb()
diff --git a/arch/sparc/include/asm/atomic_64.h b/arch/sparc/include/asm/atomic_64.h
index 9f421df..94cf160 100644
--- a/arch/sparc/include/asm/atomic_64.h
+++ b/arch/sparc/include/asm/atomic_64.h
@@ -106,6 +106,7 @@ static inline long atomic64_add_unless(atomic64_t *v, long a, long u)
}
#define atomic64_inc_not_zero(v) atomic64_add_unless((v), 1, 0)
+#define atomic64_dec_not_zero(v) atomic64_add_unless((v), -1, 0)
/* Atomic operations are already serializing */
#define smp_mb__before_atomic_dec() barrier()
diff --git a/arch/tile/include/asm/atomic_32.h b/arch/tile/include/asm/atomic_32.h
index c03349e..9cfafb3 100644
--- a/arch/tile/include/asm/atomic_32.h
+++ b/arch/tile/include/asm/atomic_32.h
@@ -233,6 +233,7 @@ static inline void atomic64_set(atomic64_t *v, u64 n)
#define atomic64_dec_return(v) atomic64_sub_return(1LL, (v))
#define atomic64_dec_and_test(v) (atomic64_dec_return((v)) == 0)
#define atomic64_inc_not_zero(v) atomic64_add_unless((v), 1LL, 0LL)
+#define atomic64_dec_not_zero(v) atomic64_add_unless((v), -1LL, 0LL)
/*
* We need to barrier before modifying the word, since the _atomic_xxx()
diff --git a/arch/tile/include/asm/atomic_64.h b/arch/tile/include/asm/atomic_64.h
index 27fe667..9c22f50 100644
--- a/arch/tile/include/asm/atomic_64.h
+++ b/arch/tile/include/asm/atomic_64.h
@@ -141,6 +141,7 @@ static inline long atomic64_add_unless(atomic64_t *v, long a, long u)
#define atomic64_add_negative(i, v) (atomic64_add_return((i), (v)) < 0)
#define atomic64_inc_not_zero(v) atomic64_add_unless((v), 1, 0)
+#define atomic64_dec_not_zero(v) atomic64_add_unless((v), -1, 0)
/* Atomic dec and inc don't implement barrier, so provide them if needed. */
#define smp_mb__before_atomic_dec() smp_mb()
diff --git a/arch/um/sys-i386/atomic64_cx8_32.S b/arch/um/sys-i386/atomic64_cx8_32.S
index 1e901d3..a58a1d4 100644
--- a/arch/um/sys-i386/atomic64_cx8_32.S
+++ b/arch/um/sys-i386/atomic64_cx8_32.S
@@ -223,3 +223,31 @@ ENTRY(atomic64_inc_not_zero_cx8)
jmp 3b
CFI_ENDPROC
ENDPROC(atomic64_inc_not_zero_cx8)
+
+ENTRY(atomic64_dec_not_zero_cx8)
+ CFI_STARTPROC
+ SAVE ebx
+
+ read64 %esi
+1:
+ testl %eax, %eax
+ je 4f
+2:
+ movl %eax, %ebx
+ movl %edx, %ecx
+ subl $1, %ebx
+ sbbl $0, %ecx
+ LOCK_PREFIX
+ cmpxchg8b (%esi)
+ jne 1b
+
+ movl $1, %eax
+3:
+ RESTORE ebx
+ ret
+4:
+ testl %edx, %edx
+ jne 2b
+ jmp 3b
+ CFI_ENDPROC
+ENDPROC(atomic64_dec_not_zero_cx8)
diff --git a/arch/x86/include/asm/atomic64_32.h b/arch/x86/include/asm/atomic64_32.h
index 24098aa..3cd4431 100644
--- a/arch/x86/include/asm/atomic64_32.h
+++ b/arch/x86/include/asm/atomic64_32.h
@@ -287,6 +287,18 @@ static inline int atomic64_inc_not_zero(atomic64_t *v)
return r;
}
+
+static inline int atomic64_dec_not_zero(atomic64_t *v)
+{
+ int r;
+ asm volatile(ATOMIC64_ALTERNATIVE(dec_not_zero)
+ : "=a" (r)
+ : "S" (v)
+ : "ecx", "edx", "memory"
+ );
+ return r;
+}
+
static inline long long atomic64_dec_if_positive(atomic64_t *v)
{
long long r;
diff --git a/arch/x86/include/asm/atomic64_64.h b/arch/x86/include/asm/atomic64_64.h
index 017594d..93c9d8b 100644
--- a/arch/x86/include/asm/atomic64_64.h
+++ b/arch/x86/include/asm/atomic64_64.h
@@ -220,6 +220,7 @@ static inline int atomic64_add_unless(atomic64_t *v, long a, long u)
}
#define atomic64_inc_not_zero(v) atomic64_add_unless((v), 1, 0)
+#define atomic64_dec_not_zero(v) atomic64_add_unless((v), -1, 0)
/*
* atomic64_dec_if_positive - decrement by 1 if old value positive
diff --git a/arch/x86/include/asm/local.h b/arch/x86/include/asm/local.h
index 9cdae5d..2c8c92d 100644
--- a/arch/x86/include/asm/local.h
+++ b/arch/x86/include/asm/local.h
@@ -185,6 +185,7 @@ static inline long local_sub_return(long i, local_t *l)
c != (u); \
})
#define local_inc_not_zero(l) local_add_unless((l), 1, 0)
+#define local_dec_not_zero(l) local_add_unless((l), -1, 0)
/* On x86_32, these are no better than the atomic variants.
* On x86-64 these are better than the atomic variants on SMP kernels
diff --git a/arch/x86/lib/atomic64_32.c b/arch/x86/lib/atomic64_32.c
index 042f682..7da05c3 100644
--- a/arch/x86/lib/atomic64_32.c
+++ b/arch/x86/lib/atomic64_32.c
@@ -24,6 +24,8 @@ long long atomic64_dec_if_positive_cx8(atomic64_t *v);
EXPORT_SYMBOL(atomic64_dec_if_positive_cx8);
int atomic64_inc_not_zero_cx8(atomic64_t *v);
EXPORT_SYMBOL(atomic64_inc_not_zero_cx8);
+int atomic64_dec_not_zero_cx8(atomic64_t *v);
+EXPORT_SYMBOL(atomic64_dec_not_zero_cx8);
int atomic64_add_unless_cx8(atomic64_t *v, long long a, long long u);
EXPORT_SYMBOL(atomic64_add_unless_cx8);
@@ -54,6 +56,8 @@ long long atomic64_dec_if_positive_386(atomic64_t *v);
EXPORT_SYMBOL(atomic64_dec_if_positive_386);
int atomic64_inc_not_zero_386(atomic64_t *v);
EXPORT_SYMBOL(atomic64_inc_not_zero_386);
+int atomic64_dec_not_zero_386(atomic64_t *v);
+EXPORT_SYMBOL(atomic64_dec_not_zero_386);
int atomic64_add_unless_386(atomic64_t *v, long long a, long long u);
EXPORT_SYMBOL(atomic64_add_unless_386);
#endif
diff --git a/arch/x86/lib/atomic64_386_32.S b/arch/x86/lib/atomic64_386_32.S
index e8e7e0d..c78337b 100644
--- a/arch/x86/lib/atomic64_386_32.S
+++ b/arch/x86/lib/atomic64_386_32.S
@@ -181,6 +181,27 @@ ENDP
#undef v
#define v %esi
+BEGIN(dec_not_zero)
+ movl (v), %eax
+ movl 4(v), %edx
+ testl %eax, %eax
+ je 3f
+1:
+ subl $1, %eax
+ sbbl $0, %edx
+ movl %eax, (v)
+ movl %edx, 4(v)
+ movl $1, %eax
+2:
+ RET
+3:
+ testl %edx, %edx
+ jne 1b
+ jmp 2b
+ENDP
+#undef v
+
+#define v %esi
BEGIN(dec_if_positive)
movl (v), %eax
movl 4(v), %edx
diff --git a/arch/x86/lib/atomic64_cx8_32.S b/arch/x86/lib/atomic64_cx8_32.S
index 391a083..989638c 100644
--- a/arch/x86/lib/atomic64_cx8_32.S
+++ b/arch/x86/lib/atomic64_cx8_32.S
@@ -220,3 +220,31 @@ ENTRY(atomic64_inc_not_zero_cx8)
jmp 3b
CFI_ENDPROC
ENDPROC(atomic64_inc_not_zero_cx8)
+
+ENTRY(atomic64_dec_not_zero_cx8)
+ CFI_STARTPROC
+ SAVE ebx
+
+ read64 %esi
+1:
+ testl %eax, %eax
+ je 4f
+2:
+ movl %eax, %ebx
+ movl %edx, %ecx
+ subl $1, %ebx
+ sbbl $0, %ecx
+ LOCK_PREFIX
+ cmpxchg8b (%esi)
+ jne 1b
+
+ movl $1, %eax
+3:
+ RESTORE ebx
+ ret
+4:
+ testl %edx, %edx
+ jne 2b
+ jmp 3b
+ CFI_ENDPROC
+ENDPROC(atomic64_dec_not_zero_cx8)
diff --git a/include/asm-generic/atomic-long.h b/include/asm-generic/atomic-long.h
index b7babf0..0fe75ab 100644
--- a/include/asm-generic/atomic-long.h
+++ b/include/asm-generic/atomic-long.h
@@ -130,6 +130,7 @@ static inline long atomic_long_add_unless(atomic_long_t *l, long a, long u)
}
#define atomic_long_inc_not_zero(l) atomic64_inc_not_zero((atomic64_t *)(l))
+#define atomic_long_dec_not_zero(l) atomic64_dec_not_zero((atomic64_t *)(l))
#define atomic_long_cmpxchg(l, old, new) \
(atomic64_cmpxchg((atomic64_t *)(l), (old), (new)))
@@ -247,6 +248,7 @@ static inline long atomic_long_add_unless(atomic_long_t *l, long a, long u)
}
#define atomic_long_inc_not_zero(l) atomic_inc_not_zero((atomic_t *)(l))
+#define atomic_long_dec_not_zero(l) atomic_dec_not_zero((atomic_t *)(l))
#define atomic_long_cmpxchg(l, old, new) \
(atomic_cmpxchg((atomic_t *)(l), (old), (new)))
diff --git a/include/asm-generic/atomic64.h b/include/asm-generic/atomic64.h
index b18ce4f..90ff9b1 100644
--- a/include/asm-generic/atomic64.h
+++ b/include/asm-generic/atomic64.h
@@ -38,5 +38,6 @@ extern int atomic64_add_unless(atomic64_t *v, long long a, long long u);
#define atomic64_dec_return(v) atomic64_sub_return(1LL, (v))
#define atomic64_dec_and_test(v) (atomic64_dec_return((v)) == 0)
#define atomic64_inc_not_zero(v) atomic64_add_unless((v), 1LL, 0LL)
+#define atomic64_dec_not_zero(v) atomic64_add_unless((v), -1LL, 0LL)
#endif /* _ASM_GENERIC_ATOMIC64_H */
diff --git a/include/asm-generic/local.h b/include/asm-generic/local.h
index 9ceb03b..fabf4f3 100644
--- a/include/asm-generic/local.h
+++ b/include/asm-generic/local.h
@@ -44,6 +44,7 @@ typedef struct
#define local_xchg(l, n) atomic_long_xchg((&(l)->a), (n))
#define local_add_unless(l, _a, u) atomic_long_add_unless((&(l)->a), (_a), (u))
#define local_inc_not_zero(l) atomic_long_inc_not_zero(&(l)->a)
+#define local_dec_not_zero(l) atomic_long_dec_not_zero(&(l)->a)
/* Non-atomic variants, ie. preemption disabled and won't be touched
* in interrupt, etc. Some archs can optimize this case well. */
diff --git a/include/asm-generic/local64.h b/include/asm-generic/local64.h
index 5980002..76acbe2 100644
--- a/include/asm-generic/local64.h
+++ b/include/asm-generic/local64.h
@@ -45,6 +45,7 @@ typedef struct {
#define local64_xchg(l, n) local_xchg((&(l)->a), (n))
#define local64_add_unless(l, _a, u) local_add_unless((&(l)->a), (_a), (u))
#define local64_inc_not_zero(l) local_inc_not_zero(&(l)->a)
+#define local64_dec_not_zero(l) local_dec_not_zero(&(l)->a)
/* Non-atomic variants, ie. preemption disabled and won't be touched
* in interrupt, etc. Some archs can optimize this case well. */
@@ -83,6 +84,7 @@ typedef struct {
#define local64_xchg(l, n) atomic64_xchg((&(l)->a), (n))
#define local64_add_unless(l, _a, u) atomic64_add_unless((&(l)->a), (_a), (u))
#define local64_inc_not_zero(l) atomic64_inc_not_zero(&(l)->a)
+#define local64_dec_not_zero(l) atomic64_dec_not_zero(&(l)->a)
/* Non-atomic variants, ie. preemption disabled and won't be touched
* in interrupt, etc. Some archs can optimize this case well. */
diff --git a/include/linux/atomic.h b/include/linux/atomic.h
index 42b77b5..ad2b750 100644
--- a/include/linux/atomic.h
+++ b/include/linux/atomic.h
@@ -27,6 +27,15 @@ static inline int atomic_add_unless(atomic_t *v, int a, int u)
#define atomic_inc_not_zero(v) atomic_add_unless((v), 1, 0)
/**
+ * atomic_dec_not_zero - decrement unless the number is zero
+ * @v: pointer of type atomic_t
+ *
+ * Atomically decrements @v by 1, so long as @v is non-zero.
+ * Returns non-zero if @v was non-zero, and zero otherwise.
+ */
+#define atomic_dec_not_zero(v) atomic_add_unless((v), -1, 0)
+
+/**
* atomic_inc_not_zero_hint - increment if not null
* @v: pointer of type atomic_t
* @hint: probable value of the atomic before the increment
--
1.7.5.4
^ permalink raw reply related
* Re: [RFC/PATCH] mm/futex: Fix futex writes on archs with SW tracking of dirty & young
From: Peter Zijlstra @ 2011-07-27 8:59 UTC (permalink / raw)
To: Benjamin Herrenschmidt
Cc: tony.luck, Mike Frysinger, Shan Hai, linux-kernel, cmetcalf,
dhowells, paulus, uclinux-dist-devel, tglx, walken, linuxppc-dev,
akpm
In-Reply-To: <1311753513.25044.663.camel@pasglop>
On Wed, 2011-07-27 at 17:58 +1000, Benjamin Herrenschmidt wrote:
> What should nommu do anyways ? it's not like there's much it can do
> right ? It should never even hit the fault path to start with ...
Something like the below makes a nommu arm config build.. David, is this
indeed the correct thing to do for nommu?
---
Index: linux-2.6/mm/nommu.c
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
--- linux-2.6.orig/mm/nommu.c
+++ linux-2.6/mm/nommu.c
@@ -190,6 +190,12 @@ int get_user_pages(struct task_struct *t
}
EXPORT_SYMBOL(get_user_pages);
=20
+int fixup_user_fault(struct task_struct *tsk, struct mm_struct *mm,
+ unsigned long address, unsigned int fault_flags)
+{
+ BUG(); /* nommu should never call this */
+}
+
/**
* follow_pfn - look up PFN at a user virtual address
* @vma: memory mapping
^ permalink raw reply
* Re: [RFC/PATCH] mm/futex: Fix futex writes on archs with SW tracking of dirty & young
From: Benjamin Herrenschmidt @ 2011-07-27 7:58 UTC (permalink / raw)
To: Mike Frysinger
Cc: tony.luck, Peter Zijlstra, Shan Hai, Peter Zijlstra, linux-kernel,
cmetcalf, dhowells, paulus, uclinux-dist-devel, tglx, walken,
linuxppc-dev, akpm
In-Reply-To: <CAMjpGUdxpaYBFfKjBEPFOJohnKoXfagLUHvhrst+NembeabxcA@mail.gmail.com>
On Tue, 2011-07-26 at 23:50 -0700, Mike Frysinger wrote:
> On Mon, Jul 18, 2011 at 21:29, Benjamin Herrenschmidt wrote:
> > The futex code currently attempts to write to user memory within
> > a pagefault disabled section, and if that fails, tries to fix it
> > up using get_user_pages().
> >
> > This doesn't work on archs where the dirty and young bits are
> > maintained by software, since they will gate access permission
> > in the TLB, and will not be updated by gup().
> >
> > In addition, there's an expectation on some archs that a
> > spurious write fault triggers a local TLB flush, and that is
> > missing from the picture as well.
> >
> > I decided that adding those "features" to gup() would be too much
> > for this already too complex function, and instead added a new
> > simpler fixup_user_fault() which is essentially a wrapper around
> > handle_mm_fault() which the futex code can call.
>
> unfortunately, this breaks all nommu ports. you added
> fixup_user_fault() to mm/memory.c only which is not used by nommu
Argh. Andrew, do you want to send a fix ? I won't be able to do that
tonight, I have to go.
What should nommu do anyways ? it's not like there's much it can do
right ? It should never even hit the fault path to start with ...
Cheers,
Ben.
^ permalink raw reply
* Re: [RFC/PATCH] mm/futex: Fix futex writes on archs with SW tracking of dirty & young
From: Mike Frysinger @ 2011-07-27 6:50 UTC (permalink / raw)
To: Benjamin Herrenschmidt
Cc: tony.luck, Peter Zijlstra, Shan Hai, Peter Zijlstra, linux-kernel,
cmetcalf, dhowells, paulus, uclinux-dist-devel, tglx, walken,
linuxppc-dev, akpm
In-Reply-To: <1311049762.25044.392.camel@pasglop>
On Mon, Jul 18, 2011 at 21:29, Benjamin Herrenschmidt wrote:
> The futex code currently attempts to write to user memory within
> a pagefault disabled section, and if that fails, tries to fix it
> up using get_user_pages().
>
> This doesn't work on archs where the dirty and young bits are
> maintained by software, since they will gate access permission
> in the TLB, and will not be updated by gup().
>
> In addition, there's an expectation on some archs that a
> spurious write fault triggers a local TLB flush, and that is
> missing from the picture as well.
>
> I decided that adding those "features" to gup() would be too much
> for this already too complex function, and instead added a new
> simpler fixup_user_fault() which is essentially a wrapper around
> handle_mm_fault() which the futex code can call.
unfortunately, this breaks all nommu ports. you added
fixup_user_fault() to mm/memory.c only which is not used by nommu
logic.
-mike
^ permalink raw reply
* Re: [PATCH] powerpc: pseries: Fix kexec on machines with more than 4TB of RAM
From: Michael Neuling @ 2011-07-27 4:26 UTC (permalink / raw)
To: Anton Blanchard; +Cc: paulus, linuxppc-dev, miltonm, kexec
In-Reply-To: <20110727141503.3faa93f5@kryten>
> On a box with 8TB of RAM the MMU hashtable is 64GB in size. That
> means we have 4G PTEs. pSeries_lpar_hptab_clear was using a signed
> int to store the index which will overflow at 2G.
Wow, I hear... "Anton Blanchard's machine has more memory in it's MMU
than you have in total".
Acked-by: Michael Neuling <mikey@neuling.org>
>
> Signed-off-by: Anton Blanchard <anton@samba.org>
> Cc: <stable@kernel.org>
> ---
>
> Index: linux-powerpc/arch/powerpc/platforms/pseries/lpar.c
> ===================================================================
> --- linux-powerpc.orig/arch/powerpc/platforms/pseries/lpar.c 2011-07-27 13:5
0:18.408091576 +1000
> +++ linux-powerpc/arch/powerpc/platforms/pseries/lpar.c 2011-07-27 13:5
1:19.359196131 +1000
> @@ -203,7 +203,7 @@ static void pSeries_lpar_hptab_clear(voi
> unsigned long ptel;
> } ptes[4];
> long lpar_rc;
> - int i, j;
> + unsigned long i, j;
>
> /* Read in batches of 4,
> * invalidate only valid entries not in the VRMA
>
^ permalink raw reply
* [PATCH] powerpc: pseries: Fix kexec on machines with more than 4TB of RAM
From: Anton Blanchard @ 2011-07-27 4:15 UTC (permalink / raw)
To: benh, paulus, mikey, miltonm; +Cc: kexec, linuxppc-dev
On a box with 8TB of RAM the MMU hashtable is 64GB in size. That
means we have 4G PTEs. pSeries_lpar_hptab_clear was using a signed
int to store the index which will overflow at 2G.
Signed-off-by: Anton Blanchard <anton@samba.org>
Cc: <stable@kernel.org>
---
Index: linux-powerpc/arch/powerpc/platforms/pseries/lpar.c
===================================================================
--- linux-powerpc.orig/arch/powerpc/platforms/pseries/lpar.c 2011-07-27 13:50:18.408091576 +1000
+++ linux-powerpc/arch/powerpc/platforms/pseries/lpar.c 2011-07-27 13:51:19.359196131 +1000
@@ -203,7 +203,7 @@ static void pSeries_lpar_hptab_clear(voi
unsigned long ptel;
} ptes[4];
long lpar_rc;
- int i, j;
+ unsigned long i, j;
/* Read in batches of 4,
* invalidate only valid entries not in the VRMA
^ permalink raw reply
* [PATCH] powerpc: Jump label misalignment causes oops at boot
From: Anton Blanchard @ 2011-07-27 0:35 UTC (permalink / raw)
To: benh, paulus, michael; +Cc: linuxppc-dev
In-Reply-To: <20110727101150.449574ef@kryten>
I hit an oops at boot on the first instruction of timer_cpu_notify:
NIP [c000000000722f88] .timer_cpu_notify+0x0/0x388
The code should look like:
c000000000722f78: eb e9 00 30 ld r31,48(r9)
c000000000722f7c: 2f bf 00 00 cmpdi cr7,r31,0
c000000000722f80: 40 9e ff 44 bne+ cr7,c000000000722ec4
c000000000722f84: 4b ff ff 74 b c000000000722ef8
c000000000722f88 <.timer_cpu_notify>:
c000000000722f88: 7c 08 02 a6 mflr r0
c000000000722f8c: 2f a4 00 07 cmpdi cr7,r4,7
c000000000722f90: fb c1 ff f0 std r30,-16(r1)
c000000000722f94: fb 61 ff d8 std r27,-40(r1)
But the oops output shows:
eb61ffd8 eb81ffe0 eba1ffe8 ebc1fff0 7c0803a6 ebe1fff8 4e800020
00000000 ebe90030 c0000000 00ad0a28 00000000 2fa40007 fbc1fff0 fb61ffd8
So we scribbled over our instructions with c000000000ad0a28, which
is an address inside the jump_table ELF section.
It turns out the jump_table section is only aligned to 8 bytes but
we are aligning our entries within the section to 16 bytes. This
means our entries are offset from the table:
c000000000acd4a8 <__start___jump_table>:
...
c000000000ad0a10: c0 00 00 00 lfs f0,0(0)
c000000000ad0a14: 00 70 cd 5c .long 0x70cd5c
c000000000ad0a18: c0 00 00 00 lfs f0,0(0)
c000000000ad0a1c: 00 70 cd 90 .long 0x70cd90
c000000000ad0a20: c0 00 00 00 lfs f0,0(0)
c000000000ad0a24: 00 ac a4 20 .long 0xaca420
And the jump table sort code gets very confused and writes into the
wrong spot. Remove the alignment, and also remove the padding since
we it saves some space and we shouldn't need it.
Signed-off-by: Anton Blanchard <anton@samba.org>
---
Index: linux-powerpc/arch/powerpc/include/asm/jump_label.h
===================================================================
--- linux-powerpc.orig/arch/powerpc/include/asm/jump_label.h 2011-07-27 09:35:04.266872078 +1000
+++ linux-powerpc/arch/powerpc/include/asm/jump_label.h 2011-07-27 09:39:32.581719669 +1000
@@ -22,7 +22,6 @@ static __always_inline bool arch_static_
asm goto("1:\n\t"
"nop\n\t"
".pushsection __jump_table, \"aw\"\n\t"
- ".align 4\n\t"
JUMP_ENTRY_TYPE "1b, %l[l_yes], %c0\n\t"
".popsection \n\t"
: : "i" (key) : : l_yes);
@@ -41,7 +40,6 @@ struct jump_entry {
jump_label_t code;
jump_label_t target;
jump_label_t key;
- jump_label_t pad;
};
#endif /* _ASM_POWERPC_JUMP_LABEL_H */
^ permalink raw reply
* [PATCH] powerpc: Jump label misalignment causes oops at boot
From: Anton Blanchard @ 2011-07-27 0:11 UTC (permalink / raw)
To: benh, paulus, michael; +Cc: linuxppc-dev
I hit an oops at boot on the first instruction of timer_cpu_notify:
Oops: Exception in kernel mode, sig: 4 [#1]
...
NIP [c000000000722f88] .timer_cpu_notify+0x0/0x388
The code should look like:
c000000000722f78: eb e9 00 30 ld r31,48(r9)
c000000000722f7c: 2f bf 00 00 cmpdi cr7,r31,0
c000000000722f80: 40 9e ff 44 bne+ cr7,c000000000722ec4
c000000000722f84: 4b ff ff 74 b c000000000722ef8
c000000000722f88 <.timer_cpu_notify>:
c000000000722f88: 7c 08 02 a6 mflr r0
c000000000722f8c: 2f a4 00 07 cmpdi cr7,r4,7
c000000000722f90: fb c1 ff f0 std r30,-16(r1)
c000000000722f94: fb 61 ff d8 std r27,-40(r1)
But the oops output shows these instructions:
eb61ffd8 eb81ffe0 eba1ffe8 ebc1fff0 7c0803a6 ebe1fff8 4e800020 00000000
ebe90030 c0000000 00ad0a28 00000000 2fa40007 fbc1fff0 fb61ffd8
So we scribbled over our instructions with c000000000ad0a28 which
is an address inside the jump_table ELF section.
It turns out the jump_table section is only aligned to 8 bytes but
we are aligning our entries within the section to 16 bytes. This
means our entries can be offset from the table:
c000000000acd4a8 <__start___jump_table>:
...
c000000000ad0a10: c0 00 00 00 lfs f0,0(0)
c000000000ad0a14: 00 70 cd 5c .long 0x70cd5c
c000000000ad0a18: c0 00 00 00 lfs f0,0(0)
c000000000ad0a1c: 00 70 cd 90 .long 0x70cd90
c000000000ad0a20: c0 00 00 00 lfs f0,0(0)
c000000000ad0a24: 00 ac a4 20 .long 0xaca420
And the jump table sort code gets very confused and writes into the
wrong spot. Remove the alignment and also remove the padding since
we it saves some space and we shouldn't need it.
Signed-off-by: Anton Blanchard <anton@samba.org>
---
Index: linux-powerpc/arch/powerpc/include/asm/jump_label.h
===================================================================
--- linux-powerpc.orig/arch/powerpc/include/asm/jump_label.h
2011-07-27 09:35:04.266872078 +1000 +++
linux-powerpc/arch/powerpc/include/asm/jump_label.h 2011-07-27
09:39:32.581719669 +1000 @@ -22,7 +22,6 @@ static __always_inline bool
arch_static_ asm goto("1:\n\t" "nop\n\t"
".pushsection __jump_table, \"aw\"\n\t"
- ".align 4\n\t"
JUMP_ENTRY_TYPE "1b, %l[l_yes], %c0\n\t"
".popsection \n\t"
: : "i" (key) : : l_yes);
@@ -41,7 +40,6 @@ struct jump_entry {
jump_label_t code;
jump_label_t target;
jump_label_t key;
- jump_label_t pad;
};
#endif /* _ASM_POWERPC_JUMP_LABEL_H */
^ permalink raw reply
* Re: [PATCH RFC] perf event: Torrent, add support for the various PMUs on the Torrent chip.
From: Kumar Gala @ 2011-07-26 22:24 UTC (permalink / raw)
To: Carl E. Love; +Cc: linuxppc-dev, ltc-interlock
In-Reply-To: <1311698974.28685.4.camel@oc5652146517.ibm.com>
On Jul 26, 2011, at 11:49 AM, Carl E. Love wrote:
>=20
>=20
> We are requesting your help to review the following patch prior to =
posting it=20
> upstream. The patch is against the 2.6.39 tree (which is already out =
of date).
> Thank your for your help and input.
>=20
> Carl Love
> =
--------------------------------------------------------------------------=
> [PATCH RFC] perf event: Torrent, add support for the various PMUs on =
the Torrent chip.
>=20
> Support for the Torrent hardware performance monitor units (PMU) on =
the LL
> link, WXYZ links, MCD bus, and CAU units is added. These PMUs are =
specific
> to the Torrent system which is built using POWER7 processors. These =
PMUs
> are specifc to the IBM P7IH system's Torrent chips, which are used to
> interconnect POWER7 processors. Hence the platform specific files to
> support these PMUs are in the platform-specific directory
> arch/powerpc/platforms/torrent. The include files are added to the
> standard powerpc include directory. The Torrent PMU support uses the
> multiple PMU perf_events support. A single perf_events PMU type is
> created to cover all of the various Torrent hardware PMUs. The =
generic
> Torrent PMU type allows all of the specific Torrent hardware PMU =
events to
> be included into a group of events within the perf_events Torrent PMU =
type.
> The advantage of this model is that all of the events for the various
> links are guaranteed to be measured at the same time, providing good
> correlation between the activity on each of the different Torrent =
links.
>=20
> This patch is a forward port of the initial Torrent PMU patch for =
2.6.32
> that was done as part of the development effort on Torrent and never =
pushed
> upstream.
>=20
> Signed-off-by: Carl Love <carll@us.ibm.com>
> ---
> arch/powerpc/include/asm/cau_pmu.h | 39 +
> arch/powerpc/include/asm/hvcall.h | 14 +
> arch/powerpc/include/asm/mmu_pmu.h | 39 +
> arch/powerpc/include/asm/power_torrent_events.h | 106 ++
> arch/powerpc/include/asm/powerbus_bus_util_pmu.h | 39 +
> arch/powerpc/include/asm/powerbus_ll_pmu.h | 39 +
> arch/powerpc/include/asm/powerbus_mcd_pmu.h | 44 +
> arch/powerpc/include/asm/powerbus_wxyz_pmu.h | 38 +
> arch/powerpc/include/asm/torrent_nest_pmu.h | 192 +++
Seems like a lot of these headers could move to plarforms/p7ih/
> arch/powerpc/kernel/perf_event.c | 4 +
> arch/powerpc/platforms/Makefile | 1 +
> arch/powerpc/platforms/p7ih/Makefile | 10 +
> arch/powerpc/platforms/p7ih/cau_pmu.c | 160 ++
> arch/powerpc/platforms/p7ih/mmu_pmu.c | 156 ++
> .../powerpc/platforms/p7ih/powerbus_bus_util_pmu.c | 410 +++++
> arch/powerpc/platforms/p7ih/powerbus_ll_pmu.c | 273 ++++
> arch/powerpc/platforms/p7ih/powerbus_mcd_pmu.c | 492 ++++++
> arch/powerpc/platforms/p7ih/powerbus_wxyz_pmu.c | 244 +++
> arch/powerpc/platforms/p7ih/torrent_pmu.c | 1582 =
++++++++++++++++++++
> arch/powerpc/platforms/pseries/Kconfig | 5 +
> 20 files changed, 3887 insertions(+), 0 deletions(-)
>=20
> +#define PMU_SPACE_MASK 0xFF000000
> +#define POWERPC_CORE_SPACE 0x00000000
> +#define TORRENT_SPACE 0x01000000
> +#define IS_CORE_EVENT(x) ((x & PMU_SPACE_MASK) =3D=3D =
POWERPC_CORE_SPACE)
> +#define IS_TORRENT_EVENT(x) ((x & PMU_SPACE_MASK) =3D=3D =
TORRENT_SPACE)
>=20
> diff --git a/arch/powerpc/kernel/perf_event.c =
b/arch/powerpc/kernel/perf_event.c
> index 822f630..aa1eb43 100644
> --- a/arch/powerpc/kernel/perf_event.c
> +++ b/arch/powerpc/kernel/perf_event.c
> @@ -19,6 +19,8 @@
> #include <asm/firmware.h>
> #include <asm/ptrace.h>
>=20
> +#include <asm/torrent_nest_pmu.h>
> +
> struct cpu_hw_events {
> int n_events;
> int n_percpu;
> @@ -1092,6 +1094,8 @@ static int power_pmu_event_init(struct =
perf_event *event)
> break;
> case PERF_TYPE_RAW:
> ev =3D event->attr.config;
> + if (!IS_CORE_EVENT(ev))
> + return -ENOENT;
seems like we should put what is needed for IS_CORE_EVENT() into =
asm/pmc.h if we believe its reasonable..
pulling in <asm/torrent_nest_pmu.h> seems the torrent support is pretty =
specific.
> break;
> default:
> return -ENOENT;
- k=
^ permalink raw reply
* Re: [PATCH v3] mtd: eLBC NAND: update ecc_stats.corrected when lteccr available
From: Scott Wood @ 2011-07-26 20:10 UTC (permalink / raw)
To: MichaelHench; +Cc: linuxppc-dev, linux-mtd, mlcreech, mhench
In-Reply-To: <CAHyXW61rWcoE0Wcd9K7BsP3+VKzmQo=RPMM9+1DTw+QdF9MVew@mail.gmail.com>
On Tue, 26 Jul 2011 15:07:42 -0500
Michael Hench <michaelhench@gmail.com> wrote:
> update ecc_stats.corrected if LTECCR register is available.
>
> v2: kernel standard C formatting
>
> v3: kernel standard C formatting again, changed a comment to get under 80 chars
>
> Signed-off-by: Michael Hench <MichaelHench@gmail.com>
Acked-by: Scott Wood <scottwood@freescale.com>
-Scott
^ permalink raw reply
* [PATCH v3] mtd: eLBC NAND: update ecc_stats.corrected when lteccr available
From: Michael Hench @ 2011-07-26 20:07 UTC (permalink / raw)
To: scottwood, linux-mtd, linuxppc-dev, mhench, mlcreech
update ecc_stats.corrected if LTECCR register is available.
v2: kernel standard C formatting
v3: kernel standard C formatting again, changed a comment to get under 80 chars
Signed-off-by: Michael Hench <MichaelHench@gmail.com>
---
diff -purN orig/drivers/mtd/nand/fsl_elbc_nand.c
linux-3.0/drivers/mtd/nand/fsl_elbc_nand.c
--- orig/drivers/mtd/nand/fsl_elbc_nand.c 2011-07-25 14:50:56.838326055 -0500
+++ linux-3.0/drivers/mtd/nand/fsl_elbc_nand.c 2011-07-25
14:48:35.680940052 -0500
@@ -256,6 +256,25 @@ static int fsl_elbc_run_command(struct m
return -EIO;
}
+ if (chip->ecc.mode != NAND_ECC_HW)
+ return 0;
+
+ if (elbc_fcm_ctrl->read_bytes == mtd->writesize + mtd->oobsize) {
+ uint32_t lteccr = in_be32(&lbc->lteccr);
+ /*
+ * if command was a full page read and the ELBC
+ * has the LTECCR register, then bits 12-15 (ppc order) of
+ * LTECCR indicates which 512 byte sub-pages had fixed errors.
+ * bits 28-31 are uncorrectable errors, marked elsewhere.
+ * for small page nand only 1 bit is used.
+ * if the ELBC doesn't have the lteccr register it reads 0
+ */
+ if (lteccr & 0x000F000F)
+ out_be32(&lbc->lteccr, 0x000F000F); /* clear lteccr */
+ if (lteccr & 0x000F0000)
+ mtd->ecc_stats.corrected++;
+ }
+
return 0;
}
--
^ permalink raw reply
* Re: [PATCH v2] mtd: eLBC NAND: update ecc_stats.corrected when lteccr available
From: Scott Wood @ 2011-07-26 19:05 UTC (permalink / raw)
To: MichaelHench; +Cc: linuxppc-dev, linux-mtd, mlcreech, mhench
In-Reply-To: <CAHyXW604AxBGO=AEAyOcba2VfsjVve+RV7_A8QXaUpJRfAsq3Q@mail.gmail.com>
On Tue, 26 Jul 2011 13:59:25 -0500
Michael Hench <michaelhench@gmail.com> wrote:
> update ecc_stats.corrected if LTECCR register is available.
>
> v2: kernel standard C formatting
Missed a couple:
> + if(lteccr & 0x000F000F)
> + out_be32(&lbc->lteccr, 0x000F000F); /* clear lteccr */
> + if(lteccr & 0x000F0000)
> + mtd->ecc_stats.corrected++;
-Scott
^ permalink raw reply
* [PATCH v2] mtd: eLBC NAND: update ecc_stats.corrected when lteccr available
From: Michael Hench @ 2011-07-26 18:59 UTC (permalink / raw)
To: scottwood, linux-mtd, linuxppc-dev, mhench, mlcreech
update ecc_stats.corrected if LTECCR register is available.
v2: kernel standard C formatting
Signed-off-by: Michael Hench <MichaelHench@gmail.com>
---
diff -purN orig/drivers/mtd/nand/fsl_elbc_nand.c
linux-3.0/drivers/mtd/nand/fsl_elbc_nand.c
--- orig/drivers/mtd/nand/fsl_elbc_nand.c 2011-07-25 14:50:56.838326055 -0500
+++ linux-3.0/drivers/mtd/nand/fsl_elbc_nand.c 2011-07-25
14:48:35.680940052 -0500
@@ -256,6 +256,25 @@ static int fsl_elbc_run_command(struct m
return -EIO;
}
+ if (chip->ecc.mode != NAND_ECC_HW)
+ return 0;
+
+ if (elbc_fcm_ctrl->read_bytes == mtd->writesize + mtd->oobsize) {
+ uint32_t lteccr = in_be32(&lbc->lteccr);
+ /*
+ * if command was a full page read and the ELBC
+ * has the LTECCR register, then bits 12-15 (ppc order) of
+ * LTECCR indicate which 512 byte sub-pages had corrected errors.
+ * bits 28-31 are uncorrectable errors, marked elsewhere.
+ * for small page nand only 1 bit is used.
+ * if the ELBC doesn't have the lteccr register it reads 0
+ */
+ if(lteccr & 0x000F000F)
+ out_be32(&lbc->lteccr, 0x000F000F); /* clear lteccr */
+ if(lteccr & 0x000F0000)
+ mtd->ecc_stats.corrected++;
+ }
+
return 0;
}
--
^ permalink raw reply
* [PATCH] RapidIO: Fix use of non-compatible registers
From: Alexandre Bounine @ 2011-07-26 18:07 UTC (permalink / raw)
To: akpm, linux-kernel, linuxppc-dev
Cc: Chul Kim, Alexandre Bounine, Thomas Moll, stable
Replace/remove use of RIO v.1.2 registers/bits that are not forward-compatible
with newer versions of RapidIO specification.
RapidIO specification v. 1.3 removed Write Port CSR, Doorbell CSR,
Mailbox CSR and Mailbox and Doorbell bits of the PEF CAR.
Signed-off-by: Alexandre Bounine <alexandre.bounine@idt.com>
Cc: Kumar Gala <galak@kernel.crashing.org>
Cc: Matt Porter <mporter@kernel.crashing.org>
Cc: Li Yang <leoli@freescale.com>
Cc: Thomas Moll <thomas.moll@sysgo.com>
Cc: Chul Kim <chul.kim@idt.com>
Cc: <stable@kernel.org>
---
drivers/net/rionet.c | 23 ++++++++---------------
drivers/rapidio/rio-scan.c | 3 +--
include/linux/rio_regs.h | 18 +++++++++---------
3 files changed, 18 insertions(+), 26 deletions(-)
diff --git a/drivers/net/rionet.c b/drivers/net/rionet.c
index 86ac38c..3bb1311 100644
--- a/drivers/net/rionet.c
+++ b/drivers/net/rionet.c
@@ -80,13 +80,13 @@ static int rionet_capable = 1;
*/
static struct rio_dev **rionet_active;
-#define is_rionet_capable(pef, src_ops, dst_ops) \
- ((pef & RIO_PEF_INB_MBOX) && \
- (pef & RIO_PEF_INB_DOORBELL) && \
+#define is_rionet_capable(src_ops, dst_ops) \
+ ((src_ops & RIO_SRC_OPS_DATA_MSG) && \
+ (dst_ops & RIO_DST_OPS_DATA_MSG) && \
(src_ops & RIO_SRC_OPS_DOORBELL) && \
(dst_ops & RIO_DST_OPS_DOORBELL))
#define dev_rionet_capable(dev) \
- is_rionet_capable(dev->pef, dev->src_ops, dev->dst_ops)
+ is_rionet_capable(dev->src_ops, dev->dst_ops)
#define RIONET_MAC_MATCH(x) (*(u32 *)x == 0x00010001)
#define RIONET_GET_DESTID(x) (*(u16 *)(x + 4))
@@ -282,7 +282,6 @@ static int rionet_open(struct net_device *ndev)
{
int i, rc = 0;
struct rionet_peer *peer, *tmp;
- u32 pwdcsr;
struct rionet_private *rnet = netdev_priv(ndev);
if (netif_msg_ifup(rnet))
@@ -332,13 +331,8 @@ static int rionet_open(struct net_device *ndev)
continue;
}
- /*
- * If device has initialized inbound doorbells,
- * send a join message
- */
- rio_read_config_32(peer->rdev, RIO_WRITE_PORT_CSR, &pwdcsr);
- if (pwdcsr & RIO_DOORBELL_AVAIL)
- rio_send_doorbell(peer->rdev, RIONET_DOORBELL_JOIN);
+ /* Send a join message */
+ rio_send_doorbell(peer->rdev, RIONET_DOORBELL_JOIN);
}
out:
@@ -492,7 +486,7 @@ static int rionet_setup_netdev(struct rio_mport *mport, struct net_device *ndev)
static int rionet_probe(struct rio_dev *rdev, const struct rio_device_id *id)
{
int rc = -ENODEV;
- u32 lpef, lsrc_ops, ldst_ops;
+ u32 lsrc_ops, ldst_ops;
struct rionet_peer *peer;
struct net_device *ndev = NULL;
@@ -515,12 +509,11 @@ static int rionet_probe(struct rio_dev *rdev, const struct rio_device_id *id)
* on later probes
*/
if (!rionet_check) {
- rio_local_read_config_32(rdev->net->hport, RIO_PEF_CAR, &lpef);
rio_local_read_config_32(rdev->net->hport, RIO_SRC_OPS_CAR,
&lsrc_ops);
rio_local_read_config_32(rdev->net->hport, RIO_DST_OPS_CAR,
&ldst_ops);
- if (!is_rionet_capable(lpef, lsrc_ops, ldst_ops)) {
+ if (!is_rionet_capable(lsrc_ops, ldst_ops)) {
printk(KERN_ERR
"%s: local device is not network capable\n",
DRV_NAME);
diff --git a/drivers/rapidio/rio-scan.c b/drivers/rapidio/rio-scan.c
index ee89358..ebe77dd 100644
--- a/drivers/rapidio/rio-scan.c
+++ b/drivers/rapidio/rio-scan.c
@@ -505,8 +505,7 @@ static struct rio_dev __devinit *rio_setup_device(struct rio_net *net,
rdev->dev.dma_mask = &rdev->dma_mask;
rdev->dev.coherent_dma_mask = DMA_BIT_MASK(32);
- if ((rdev->pef & RIO_PEF_INB_DOORBELL) &&
- (rdev->dst_ops & RIO_DST_OPS_DOORBELL))
+ if (rdev->dst_ops & RIO_DST_OPS_DOORBELL)
rio_init_dbell_res(&rdev->riores[RIO_DOORBELL_RESOURCE],
0, 0xffff);
diff --git a/include/linux/rio_regs.h b/include/linux/rio_regs.h
index 9026b30..218168a 100644
--- a/include/linux/rio_regs.h
+++ b/include/linux/rio_regs.h
@@ -36,12 +36,12 @@
#define RIO_PEF_PROCESSOR 0x20000000 /* [I] Processor */
#define RIO_PEF_SWITCH 0x10000000 /* [I] Switch */
#define RIO_PEF_MULTIPORT 0x08000000 /* [VI, 2.1] Multiport */
-#define RIO_PEF_INB_MBOX 0x00f00000 /* [II] Mailboxes */
-#define RIO_PEF_INB_MBOX0 0x00800000 /* [II] Mailbox 0 */
-#define RIO_PEF_INB_MBOX1 0x00400000 /* [II] Mailbox 1 */
-#define RIO_PEF_INB_MBOX2 0x00200000 /* [II] Mailbox 2 */
-#define RIO_PEF_INB_MBOX3 0x00100000 /* [II] Mailbox 3 */
-#define RIO_PEF_INB_DOORBELL 0x00080000 /* [II] Doorbells */
+#define RIO_PEF_INB_MBOX 0x00f00000 /* [II, <= 1.2] Mailboxes */
+#define RIO_PEF_INB_MBOX0 0x00800000 /* [II, <= 1.2] Mailbox 0 */
+#define RIO_PEF_INB_MBOX1 0x00400000 /* [II, <= 1.2] Mailbox 1 */
+#define RIO_PEF_INB_MBOX2 0x00200000 /* [II, <= 1.2] Mailbox 2 */
+#define RIO_PEF_INB_MBOX3 0x00100000 /* [II, <= 1.2] Mailbox 3 */
+#define RIO_PEF_INB_DOORBELL 0x00080000 /* [II, <= 1.2] Doorbells */
#define RIO_PEF_EXT_RT 0x00000200 /* [III, 1.3] Extended route table support */
#define RIO_PEF_STD_RT 0x00000100 /* [III, 1.3] Standard route table support */
#define RIO_PEF_CTLS 0x00000010 /* [III] CTLS */
@@ -102,7 +102,7 @@
#define RIO_SWITCH_RT_LIMIT 0x34 /* [III, 1.3] Switch Route Table Destination ID Limit CAR */
#define RIO_RT_MAX_DESTID 0x0000ffff
-#define RIO_MBOX_CSR 0x40 /* [II] Mailbox CSR */
+#define RIO_MBOX_CSR 0x40 /* [II, <= 1.2] Mailbox CSR */
#define RIO_MBOX0_AVAIL 0x80000000 /* [II] Mbox 0 avail */
#define RIO_MBOX0_FULL 0x40000000 /* [II] Mbox 0 full */
#define RIO_MBOX0_EMPTY 0x20000000 /* [II] Mbox 0 empty */
@@ -128,8 +128,8 @@
#define RIO_MBOX3_FAIL 0x00000008 /* [II] Mbox 3 fail */
#define RIO_MBOX3_ERROR 0x00000004 /* [II] Mbox 3 error */
-#define RIO_WRITE_PORT_CSR 0x44 /* [I] Write Port CSR */
-#define RIO_DOORBELL_CSR 0x44 /* [II] Doorbell CSR */
+#define RIO_WRITE_PORT_CSR 0x44 /* [I, <= 1.2] Write Port CSR */
+#define RIO_DOORBELL_CSR 0x44 /* [II, <= 1.2] Doorbell CSR */
#define RIO_DOORBELL_AVAIL 0x80000000 /* [II] Doorbell avail */
#define RIO_DOORBELL_FULL 0x40000000 /* [II] Doorbell full */
#define RIO_DOORBELL_EMPTY 0x20000000 /* [II] Doorbell empty */
--
1.7.6
^ permalink raw reply related
* Re: [PATCH] mtd: eLBC NAND: update ecc_stats.corrected when lteccr available
From: Scott Wood @ 2011-07-26 18:20 UTC (permalink / raw)
To: MichaelHench; +Cc: linuxppc-dev, linux-mtd, mlcreech, mhench
In-Reply-To: <CAHyXW61fOhuDcxCVQQ6inO_Uoz6m+hN_eB0pQ-rW0c=miJC=3w@mail.gmail.com>
On Tue, 26 Jul 2011 08:45:42 -0500
Michael Hench <michaelhench@gmail.com> wrote:
> update ecc_stats.corrected if LTECCR register is available.
>
> Signed-off-by: Michael Hench <MichaelHench@gmail.com>
> ---
> diff -purN orig/drivers/mtd/nand/fsl_elbc_nand.c
> linux-3.0/drivers/mtd/nand/fsl_elbc_nand.c
> --- orig/drivers/mtd/nand/fsl_elbc_nand.c 2011-07-25 14:50:56.838326055 -0500
> +++ linux-3.0/drivers/mtd/nand/fsl_elbc_nand.c 2011-07-25
> 14:48:35.680940052 -0500
> @@ -256,6 +256,25 @@ static int fsl_elbc_run_command(struct m
> return -EIO;
> }
>
> + if(chip->ecc.mode != NAND_ECC_HW)
> + return(0);
> +
> + if(elbc_fcm_ctrl->read_bytes == mtd->writesize + mtd->oobsize) {
> + uint32_t lteccr = in_be32(&lbc->lteccr);
> + /*
> + * if command was a full page read and the ELBC
> + * has the LTECCR register, then bits 12-15 (ppc order) of
> + * LTECCR indicate which 512 byte sub-pages had corrected errors.
> + * bits 28-31 are uncorrectable errors, marked elsewhere.
> + * for small page nand only 1 bit is used.
> + * if the ELBC doesn't have the lteccr register it reads 0
> + */
> + if(lteccr & 0x000F000F)
> + out_be32(&lbc->lteccr, 0x000F000F); /* clear lteccr */
> + if(lteccr & 0x000F0000)
> + mtd->ecc_stats.corrected++;
> + }
Add a space after "if" and remove the parentheses in "return(0)", but
otherwise ACK.
-Scott
^ permalink raw reply
* [PATCH RFC] perf event: Torrent, add support for the various PMUs on the Torrent chip.
From: Carl E. Love @ 2011-07-26 16:49 UTC (permalink / raw)
To: ltc-interlock, linuxppc-dev
We are requesting your help to review the following patch prior to posting it
upstream. The patch is against the 2.6.39 tree (which is already out of date).
Thank your for your help and input.
Carl Love
--------------------------------------------------------------------------
[PATCH RFC] perf event: Torrent, add support for the various PMUs on the Torrent chip.
Support for the Torrent hardware performance monitor units (PMU) on the LL
link, WXYZ links, MCD bus, and CAU units is added. These PMUs are specific
to the Torrent system which is built using POWER7 processors. These PMUs
are specifc to the IBM P7IH system's Torrent chips, which are used to
interconnect POWER7 processors. Hence the platform specific files to
support these PMUs are in the platform-specific directory
arch/powerpc/platforms/torrent. The include files are added to the
standard powerpc include directory. The Torrent PMU support uses the
multiple PMU perf_events support. A single perf_events PMU type is
created to cover all of the various Torrent hardware PMUs. The generic
Torrent PMU type allows all of the specific Torrent hardware PMU events to
be included into a group of events within the perf_events Torrent PMU type.
The advantage of this model is that all of the events for the various
links are guaranteed to be measured at the same time, providing good
correlation between the activity on each of the different Torrent links.
This patch is a forward port of the initial Torrent PMU patch for 2.6.32
that was done as part of the development effort on Torrent and never pushed
upstream.
Signed-off-by: Carl Love <carll@us.ibm.com>
---
arch/powerpc/include/asm/cau_pmu.h | 39 +
arch/powerpc/include/asm/hvcall.h | 14 +
arch/powerpc/include/asm/mmu_pmu.h | 39 +
arch/powerpc/include/asm/power_torrent_events.h | 106 ++
arch/powerpc/include/asm/powerbus_bus_util_pmu.h | 39 +
arch/powerpc/include/asm/powerbus_ll_pmu.h | 39 +
arch/powerpc/include/asm/powerbus_mcd_pmu.h | 44 +
arch/powerpc/include/asm/powerbus_wxyz_pmu.h | 38 +
arch/powerpc/include/asm/torrent_nest_pmu.h | 192 +++
arch/powerpc/kernel/perf_event.c | 4 +
arch/powerpc/platforms/Makefile | 1 +
arch/powerpc/platforms/p7ih/Makefile | 10 +
arch/powerpc/platforms/p7ih/cau_pmu.c | 160 ++
arch/powerpc/platforms/p7ih/mmu_pmu.c | 156 ++
.../powerpc/platforms/p7ih/powerbus_bus_util_pmu.c | 410 +++++
arch/powerpc/platforms/p7ih/powerbus_ll_pmu.c | 273 ++++
arch/powerpc/platforms/p7ih/powerbus_mcd_pmu.c | 492 ++++++
arch/powerpc/platforms/p7ih/powerbus_wxyz_pmu.c | 244 +++
arch/powerpc/platforms/p7ih/torrent_pmu.c | 1582 ++++++++++++++++++++
arch/powerpc/platforms/pseries/Kconfig | 5 +
20 files changed, 3887 insertions(+), 0 deletions(-)
create mode 100644 arch/powerpc/include/asm/cau_pmu.h
create mode 100644 arch/powerpc/include/asm/mmu_pmu.h
create mode 100644 arch/powerpc/include/asm/power_torrent_events.h
create mode 100644 arch/powerpc/include/asm/powerbus_bus_util_pmu.h
create mode 100644 arch/powerpc/include/asm/powerbus_ll_pmu.h
create mode 100644 arch/powerpc/include/asm/powerbus_mcd_pmu.h
create mode 100644 arch/powerpc/include/asm/powerbus_wxyz_pmu.h
create mode 100644 arch/powerpc/include/asm/torrent_nest_pmu.h
create mode 100644 arch/powerpc/platforms/p7ih/Makefile
create mode 100644 arch/powerpc/platforms/p7ih/cau_pmu.c
create mode 100644 arch/powerpc/platforms/p7ih/mmu_pmu.c
create mode 100644 arch/powerpc/platforms/p7ih/powerbus_bus_util_pmu.c
create mode 100644 arch/powerpc/platforms/p7ih/powerbus_ll_pmu.c
create mode 100644 arch/powerpc/platforms/p7ih/powerbus_mcd_pmu.c
create mode 100644 arch/powerpc/platforms/p7ih/powerbus_wxyz_pmu.c
create mode 100644 arch/powerpc/platforms/p7ih/torrent_pmu.c
diff --git a/arch/powerpc/include/asm/cau_pmu.h b/arch/powerpc/include/asm/cau_pmu.h
new file mode 100644
index 0000000..1fc6f48
--- /dev/null
+++ b/arch/powerpc/include/asm/cau_pmu.h
@@ -0,0 +1,39 @@
+/*
+ * Torrent Performance Monitor
+ *
+ * Copyright Carl Love, Corey Ashford IBM Corporation 2010, 2011
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2, or (at your option)
+ * any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+ */
+
+#ifndef __ASM_CAU_PMU_H_
+#define __ASM_CAU_PMU_H_
+
+extern int cau_compute_pmc_reg(struct torrent_pmu_events *torrent_pmu_events,
+ struct hcall_data *hcall_write);
+extern int cau_pmu_check_constraints(struct torrent_pmu_events
+ *torrent_pmu_events,
+ struct unit_config *unit_cfg);
+extern void cau_enable_disable_hw_cntr(int op,
+ struct torrent_pmu_events
+ *torrent_pmu_events,
+ struct hcall_data *hcall_write);
+extern void cau_pmd_write(struct torrent_pmu_events *torrent_pmu_events,
+ struct hcall_data *hcall_read);
+extern int cau_pmd_read(struct torrent_pmu_events *torrent_pmu_events,
+ struct hcall_data *hcall_read);
+extern int cau_get_phys_pmd_reg(u64 event_code);
+
+#endif
diff --git a/arch/powerpc/include/asm/hvcall.h b/arch/powerpc/include/asm/hvcall.h
index fd8201d..2343636 100644
--- a/arch/powerpc/include/asm/hvcall.h
+++ b/arch/powerpc/include/asm/hvcall.h
@@ -238,6 +238,20 @@
#define H_GET_MPP_X 0x314
#define MAX_HCALL_OPCODE H_GET_MPP_X
+/* PHyp calls for P7IH */
+#define H_TOR_ACCESS_PMU_SCOM_REGS 0xF048ULL
+#define H_HFI_DUMP_INFO 0xF01CULL
+
+/* Dump request codes for the H_HFI_DUMP_INFO PHyp call */
+/* request code is in bits 32..47 */
+#define DUMP_REQUEST_SHIFT (63 - 47)
+#define DUMP_WINDOW_REGS (0x1ULL << DUMP_REQUEST_SHIFT)
+#define DUMP_NON_WINDOW_REGS (0x2ULL << DUMP_REQUEST_SHIFT)
+#define DUMP_BROADCAST_COUNTERS (0x3ULL << DUMP_REQUEST_SHIFT)
+#define DUMP_PERFORMANCE_COUNTERS (0x4ULL << DUMP_REQUEST_SHIFT)
+#define DUMP_PHYP_INTERNALS (0x5ULL << DUMP_REQUEST_SHIFT)
+#define DUMP_MMIO_PERFORMANCE_COUNTERS (0x6ULL << DUMP_REQUEST_SHIFT)
+
#ifndef __ASSEMBLY__
/**
diff --git a/arch/powerpc/include/asm/mmu_pmu.h b/arch/powerpc/include/asm/mmu_pmu.h
new file mode 100644
index 0000000..ad433ea
--- /dev/null
+++ b/arch/powerpc/include/asm/mmu_pmu.h
@@ -0,0 +1,39 @@
+/*
+ * Torrent Performance Monitor
+ *
+ * Copyright Carl Love, Corey Ashford IBM Corporation 2010, 2011
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2, or (at your option)
+ * any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+ */
+
+#ifndef __ASM_MMU_PMU_H_
+#define __ASM_MMU_PMU_H_
+
+extern int mmu_compute_pmc_reg(struct torrent_pmu_events *torrent_pmu_events,
+ struct hcall_data *hcall_write);
+extern int mmu_pmu_check_constraints(struct torrent_pmu_events
+ *torrent_pmu_events,
+ struct unit_config *unit_cfg);
+extern void mmu_enable_disable_hw_cntr(int op,
+ struct torrent_pmu_events
+ *torrent_pmu_events,
+ struct hcall_data *hcall_write);
+extern void mmu_pmd_write(struct torrent_pmu_events *torrent_pmu_events,
+ struct hcall_data *hcall_read);
+extern int mmu_pmd_read(struct torrent_pmu_events *torrent_pmu_events,
+ struct hcall_data *hcall_read);
+extern int mmu_get_phys_pmd_reg(u64 event_code);
+
+#endif
diff --git a/arch/powerpc/include/asm/power_torrent_events.h b/arch/powerpc/include/asm/power_torrent_events.h
new file mode 100644
index 0000000..7d71cab
--- /dev/null
+++ b/arch/powerpc/include/asm/power_torrent_events.h
@@ -0,0 +1,106 @@
+/*
+ * Torrent Performance Monitor
+ *
+ * Copyright Carl Love, Corey Ashford IBM Corporation 2010, 2011
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2, or (at your option)
+ * any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+ */
+
+/* Power Torrent PMU event codes */
+
+#ifndef __POWER_TORRENT_EVENTS_H__
+#define __POWER_TORRENT_EVENTS_H__
+
+/* PRELIMINARY EVENT ENCODING
+ * 0x0000_0000 - 0x00FF_FFFF = POWER core events
+ * 0x0100_0000 - 0x01FF_FFFF = Torrent events
+ * 0x0200_0000 - 0xFFFF_FFFF = reserved
+ * For Torrent events:
+ * Reserve encodings 0x0..0x00FF_FFFF for core POWER events.
+ * For Torrent events
+ * 0x00F0_0000 = Torrent PMU id
+ * 0x000F_0000 = PMU unit number (e.g. 0 for MCD0, 1 for MCD1)
+ * 0x0000_FF00 = virtual counter number (unused on MCD)
+ * 0x0000_00FF = PMC event selector mux value (unused on Util, MMU, CAU)
+ * (Note that some of these fields are wider than necessary)
+ *
+ * The upper bits 0xFFFF_FFFF_0000_0000 are reserved for attribute
+ * fields.
+ */
+
+#define PMU_SPACE_MASK 0xFF000000
+#define POWERPC_CORE_SPACE 0x00000000
+#define TORRENT_SPACE 0x01000000
+#define IS_CORE_EVENT(x) ((x & PMU_SPACE_MASK) == POWERPC_CORE_SPACE)
+#define IS_TORRENT_EVENT(x) ((x & PMU_SPACE_MASK) == TORRENT_SPACE)
+#define TORRENT_PMU_SHIFT 20
+#define TORRENT_PMU_MASK (0xF << TORRENT_PMU_SHIFT)
+#define TORRENT_PMU_GET(x) ((x & TORRENT_PMU_MASK) >> TORRENT_PMU_SHIFT)
+#define TORRENT_UNIT_SHIFT 16
+#define TORRENT_UNIT_MASK (0xF << TORRENT_UNIT_SHIFT)
+#define TORRENT_UNIT_GET(x) ((x & TORRENT_UNIT_MASK) >> TORRENT_UNIT_SHIFT)
+#define TORRENT_VIRT_CTR_SHIFT 8
+#define TORRENT_VIRT_CTR_MASK (0xFF << TORRENT_VIRT_CTR_SHIFT)
+#define TORRENT_VIRT_CTR_GET(x) ((x & TORRENT_VIRT_CTR_MASK) >> \
+ TORRENT_VIRT_CTR_SHIFT)
+#define TORRENT_MUX_SHIFT 0
+#define TORRENT_MUX_MASK 0xFF
+#define TORRENT_MUX_GET(x) ((x & TORRENT_MUX_MASK) >> TORRENT_MUX_SHIFT)
+
+#define TORRENT_ATTR_UTIL_SEL_SHIFT 32
+#define TORRENT_ATTR_UTIL_SEL_MASK (0x3ULL << TORRENT_ATTR_UTIL_SEL_SHIFT)
+#define TORRENT_ATTR_UTIL_CMP_SHIFT 34
+#define TORRENT_ATTR_UTIL_CMP_MASK (0x1FULL << TORRENT_ATTR_UTIL_CMP_SHIFT)
+
+#define TORRENT_PBUS_WXYZ_ID 0x0
+#define TORRENT_PBUS_LL_ID 0x1
+#define TORRENT_PBUS_MCD_ID 0x2
+#define TORRENT_PBUS_UTIL_ID 0x3
+#define TORRENT_MMU_ID 0x4
+#define TORRENT_CAU_ID 0x5
+
+#define TORRENT_LAST_PMU_ID (TORRENT_CAU_ID)
+#define TORRENT_NUM_PMU_TYPES (TORRENT_LAST_PMU_ID + 1)
+#define TORRENT_LAST_PBUS_PMU_ID (TORRENT_PBUS_UTIL_ID)
+#define TORRENT_NUM_PBUS_PMU_TYPES (TORRENT_LAST_PBUS_PMU_ID + 1)
+
+#define TORRENT_PMU(pmu) (TORRENT_SPACE | \
+ TORRENT_##pmu##_ID << TORRENT_PMU_SHIFT)
+
+#define TORRENT_PBUS_WXYZ TORRENT_PMU(PBUS_WXYZ)
+#define TORRENT_PBUS_LL TORRENT_PMU(PBUS_LL)
+#define TORRENT_PBUS_MCD TORRENT_PMU(PBUS_MCD)
+#define TORRENT_PBUS_UTIL TORRENT_PMU(PBUS_UTIL)
+#define TORRENT_MMU TORRENT_PMU(MMU)
+#define TORRENT_CAU TORRENT_PMU(CAU)
+
+#define COUNTER_W (0 << TORRENT_VIRT_CTR_SHIFT)
+#define COUNTER_X (1 << TORRENT_VIRT_CTR_SHIFT)
+#define COUNTER_Y (2 << TORRENT_VIRT_CTR_SHIFT)
+#define COUNTER_Z (3 << TORRENT_VIRT_CTR_SHIFT)
+#define COUNTER_LL0 (0 << TORRENT_VIRT_CTR_SHIFT)
+#define COUNTER_LL1 (1 << TORRENT_VIRT_CTR_SHIFT)
+#define COUNTER_LL2 (2 << TORRENT_VIRT_CTR_SHIFT)
+#define COUNTER_LL3 (3 << TORRENT_VIRT_CTR_SHIFT)
+#define COUNTER_LL4 (4 << TORRENT_VIRT_CTR_SHIFT)
+#define COUNTER_LL5 (5 << TORRENT_VIRT_CTR_SHIFT)
+#define COUNTER_LL6 (6 << TORRENT_VIRT_CTR_SHIFT)
+
+/* Attributes */
+
+#define TORRENT_ATTR_MCD_TYPE_SHIFT 32
+#define TORRENT_ATTR_MCD_TYPE_MASK (0x3ULL << TORRENT_ATTR_MCD_TYPE_SHIFT)
+
+#endif
diff --git a/arch/powerpc/include/asm/powerbus_bus_util_pmu.h b/arch/powerpc/include/asm/powerbus_bus_util_pmu.h
new file mode 100644
index 0000000..17a30d3
--- /dev/null
+++ b/arch/powerpc/include/asm/powerbus_bus_util_pmu.h
@@ -0,0 +1,39 @@
+/*
+ * Torrent Performance Monitor
+ *
+ * Copyright Carl Love, Corey Ashford IBM Corporation 2010, 2011
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2, or (at your option)
+ * any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+ */
+
+#ifndef __ASM_POWERBUS_BUS_UTIL_PMU_H_
+#define __ASM_POWERBUS_BUS_UTIL_PMU_H_
+
+extern int bus_util_compute_pmc_reg(
+ struct torrent_pmu_events *torrent_pmu_events,
+ struct hcall_data *hcall_write);
+extern int bus_util_pmu_check_constraints(
+ struct torrent_pmu_events *torrent_pmu_events,
+ struct unit_config *unit_cfg);
+extern void bus_util_enable_disable_hw_cntr(int op,
+ struct torrent_pmu_events *torrent_pmu_events,
+ struct hcall_data *hcall_write);
+extern void bus_util_pmd_write(struct torrent_pmu_events *torrent_pmu_events,
+ struct hcall_data *hcall_read);
+extern int bus_util_pmd_read(struct torrent_pmu_events *torrent_pmu_events,
+ struct hcall_data *hcall_read);
+extern int bus_util_get_phys_pmd_reg(u64 event_code);
+
+#endif
diff --git a/arch/powerpc/include/asm/powerbus_ll_pmu.h b/arch/powerpc/include/asm/powerbus_ll_pmu.h
new file mode 100644
index 0000000..a3d289d
--- /dev/null
+++ b/arch/powerpc/include/asm/powerbus_ll_pmu.h
@@ -0,0 +1,39 @@
+/*
+ * Torrent Performance Monitor
+ *
+ * Copyright Carl Love, Corey Ashford IBM Corporation 2010, 2011
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2, or (at your option)
+ * any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+ */
+
+#ifndef __ASM_POWERBUS_LL_PMU_H_
+#define __ASM_POWERBUS_LL_PMU_H_
+
+extern int ll_link_compute_pmc_reg(
+ struct torrent_pmu_events *torrent_pmu_events,
+ struct hcall_data *hcall_write);
+extern int ll_link_pmu_check_constraints(
+ struct torrent_pmu_events *torrent_pmu_events,
+ struct unit_config *unit_cfg);
+extern void ll_link_enable_disable_hw_cntr(int op,
+ struct torrent_pmu_events *torrent_pmu_events,
+ struct hcall_data *hcall_write);
+extern void ll_link_pmd_write(struct torrent_pmu_events *torrent_pmu_events,
+ struct hcall_data *hcall_read);
+extern int ll_link_pmd_read(struct torrent_pmu_events *torrent_pmu_events,
+ struct hcall_data *hcall_read);
+extern int ll_link_get_phys_pmd_reg(u64 event_code);
+
+#endif
diff --git a/arch/powerpc/include/asm/powerbus_mcd_pmu.h b/arch/powerpc/include/asm/powerbus_mcd_pmu.h
new file mode 100644
index 0000000..acd2fcd
--- /dev/null
+++ b/arch/powerpc/include/asm/powerbus_mcd_pmu.h
@@ -0,0 +1,44 @@
+/*
+ * Torrent Performance Monitor
+ *
+ * Copyright Carl Love, Corey Ashford IBM Corporation 2010, 2011
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2, or (at your option)
+ * any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+ */
+
+#ifndef __ASM_POWERBUS_MCD_PMU_H_
+#define __ASM_POWERBUS_MCD_PMU_H_
+
+extern int mcd_compute_pmc_reg(struct torrent_pmu_events *torrent_pmu_events,
+ struct hcall_data *hcall_write);
+extern int mcd_pmu_check_constraints(
+ struct torrent_pmu_events *torrent_pmu_events,
+ struct unit_config *unit_cfg);
+extern void mcd_enable_disable_hw_cntr(int op,
+ struct torrent_pmu_events *torrent_pmu_events,
+ struct hcall_data *hcall_write);
+extern void mcd_pmd_write(struct torrent_pmu_events *torrent_pmu_events,
+ struct hcall_data *hcall_read);
+extern int mcd_pmd_read(struct torrent_pmu_events *torrent_pmu_events,
+ struct hcall_data *hcall_read);
+extern int mcd_get_phys_pmd_reg(u64 event_code);
+
+/* The following is the shift/mask for the physical register location */
+#define TORRENT_REG_ATTR_MCD_TYPE_SHIFT 61
+
+#define TORRENT_REG_ATTR_MCD_TYPE_MASK (0x3ULL << \
+ TORRENT_REG_ATTR_MCD_TYPE_SHIFT)
+
+#endif
diff --git a/arch/powerpc/include/asm/powerbus_wxyz_pmu.h b/arch/powerpc/include/asm/powerbus_wxyz_pmu.h
new file mode 100644
index 0000000..198ca71
--- /dev/null
+++ b/arch/powerpc/include/asm/powerbus_wxyz_pmu.h
@@ -0,0 +1,38 @@
+/*
+ * Torrent Performance Monitor
+ *
+ * Copyright Carl Love, Corey Ashford IBM Corporation 2010, 2011
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2, or (at your option)
+ * any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+ */
+
+#ifndef __ASM_POWERBUS_WXYZ_PMU_H_
+#define __ASM_POWERBUS_WXYZ_PMU_H_
+
+extern int wxyz_link_compute_pmc_reg(struct torrent_pmu_events *pmu_events,
+ struct hcall_data *hcall_write);
+extern int wxyz_link_pmu_check_constraints(
+ struct torrent_pmu_events *pmu_events,
+ struct unit_config *unit_cfg);
+extern void wxyz_link_enable_disable_hw_cntr(int op,
+ struct torrent_pmu_events *pmu_events,
+ struct hcall_data *hcall_write);
+extern void wxyz_link_pmd_write(struct torrent_pmu_events *pmu_events,
+ struct hcall_data *hcall_read);
+extern int wxyz_link_pmd_read(struct torrent_pmu_events *pmu_events,
+ struct hcall_data *hcall_read);
+extern int wxyz_link_get_phys_pmd_reg(u64 event_code);
+
+#endif
diff --git a/arch/powerpc/include/asm/torrent_nest_pmu.h b/arch/powerpc/include/asm/torrent_nest_pmu.h
new file mode 100644
index 0000000..88339cf
--- /dev/null
+++ b/arch/powerpc/include/asm/torrent_nest_pmu.h
@@ -0,0 +1,192 @@
+/*
+ * Torrent Performance Monitor
+ *
+ * Copyright Carl Love, Corey Ashford IBM Corporation 2010, 2011
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2, or (at your option)
+ * any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+ */
+
+#ifndef __ASM_POWERPC_TORRENT_PMU_H_
+#define __ASM_POWERPC_TORRENT_PMU_H_
+
+#include <asm/power_torrent_events.h>
+
+/* Currently, partitions never span octants, which means that exactly one
+ * Torrent chip is visible from each partition.
+ */
+#define MAX_TORRENT_CHIPS 1
+
+/* Max number of counters in each PMU type */
+#define MAX_CNTRS_PER_WXYZ_LINK_PMU 4
+#define MAX_CNTRS_PER_LL_LINK_PMU 7
+#define MAX_CNTRS_PER_BUS_UTIL_PMU 12
+#define MAX_CNTRS_PER_MCD_PMU 4
+#define MAX_CNTRS_PER_MMU_PMU 4
+#define MAX_CNTRS_PER_CAU_PMU 1
+
+/* Overall max of counters across all PMU types */
+#define MAX_CNTRS_TORRENT_PMUS 12
+
+/* Torrent HCall defines */
+#define H_WRITE_TORRENT_PMU_SCOM_REGS 0
+#define H_READ_TORRENT_PMU_SCOM_REGS 1
+
+/* Torrent PowerBus physical register */
+#define PBE_LINKS_TRACE 0
+#define PBE_LINKS_CTRS 1
+#define PBL_LINKS_TRACE_PERF_CTR_CFG 2
+#define PBL_LINKS_CTRS_1 3
+#define PBL_LINKS_CTRS_2 4
+#define PBUS_REG21_PERF_CTR 5
+#define PBUS_REG22_PERF_CTR 6
+#define PBUS_REG23_PERF_CTR 7
+#define PBUS_REG24_PERF_CTR 8
+#define PBUS_REG25_PERF_CTR 9
+#define PBUS_REG26_PERF_CTR 10
+#define PBUS_MCD_PERF_CTRS_CFG 11
+#define PBUS_MCD_PERF_CTRS_SEL 12
+#define PBUS_MCD_PERF_CTRS 13
+#define MAX_HCALL_REGS 14
+
+/* Hcall token IDs and argument array */
+#define PBUS_HCALL_READ 0
+#define PBUS_HCALL_WRITE 1
+#define PBUS_HCALL_LOCK 2
+#define PBUS_HCALL_UNLOCK 3
+
+/* counter use flags */
+#define CNTR_FREE 0
+#define CNTR_ALLOCATED 1 /* event assigned to physical cntr
+ * but not being counted yet.
+ */
+#define CNTR_IN_USE 2 /* event is being counted on its
+ * assigned physical counter.
+ */
+
+/* These two structures are defined by PHyp */
+struct tor_pmu_reg {
+ u64 regId;
+ u64 regValue;
+};
+
+struct dump_mmio_perf_counters {
+ /* CAU */
+ u64 cycles_waiting_on_a_credit;
+ /* MMU */
+ u64 G_MMCHIT;
+ u64 G_MMCMIS;
+ u64 G_MMATHIT;
+ u64 G_MMATMIS;
+};
+
+/* Torrent Hcall data structs */
+#define MAX_HCALL_WRITE_ARGS 4
+#define MAX_HCALL_READ_ARGS 8
+struct hcall_data {
+ int num_regs; /* number of physical registers to read */
+
+ /* Map each Torrent PMU register number to a request array index
+ * in the Hcall memory buffer, if there is currently is one for
+ * that register, and if not the mapped value is -1.
+ */
+ int request_idx[MAX_HCALL_REGS];
+
+ /* Which request index has the virtual PMD value for the
+ * counter.
+ */
+ int virt_cntr_to_request_idx
+ [TORRENT_NUM_PMU_TYPES][MAX_CNTRS_TORRENT_PMUS];
+
+ struct tor_pmu_reg tor_pmu_reg[MAX_HCALL_REGS];
+
+ int do_dump_mmio_perf_counters;
+ struct dump_mmio_perf_counters mmio_perf_counters;
+};
+
+#define MAX_TORRENT_PMC_REGS 2
+#define MAX_TORRENT_PMD_REGS 8 /* Maximum number for all PMUs */
+
+#define BUS_UTIL_CNTR_SEL_AVAIL 0xFFFFFFFF /* code for sel field not in use */
+struct unit_config {
+ u8 cntr_state[MAX_CNTRS_TORRENT_PMUS];
+ /* Place any needed PMU-specific config values here
+ * as a union of structs.
+ */
+ u64 mcd_cntr_attr;
+ u64 bus_util_enable_mask;
+ u64 bus_util_cntr_sel;
+};
+
+/* This structure tracks the events being counted and being added for an
+ * individual HW PMU.
+ */
+struct torrent_pmu_events {
+ int max_events; /* max physical counters in this PMU */
+ struct unit_config unit; /* PMU configuration array */
+
+ struct perf_event *event[MAX_CNTRS_TORRENT_PMUS];
+ int n_events; /* number of Torrent events being counted */
+ int n_add; /* number of Torrent events being added.
+ */
+ int update; /* boolean - The number of events has changed
+ * so the PMU hardware needs to be updated.
+ */
+ int disabled;
+
+ /* The rest of the entries are only used by PMUs whose registers are
+ * accessed via the Hcall.
+ */
+ u64 shadow_pmc_reg[MAX_TORRENT_PMC_REGS];
+};
+
+/* The Torrent PMU structure tracks the PMC value written to the HW register so
+ * it is not necessary to read the physical register when updating the value.
+ * This is done to minimize the number of hypervisor and SCOM register reads
+ * that are needed.
+ */
+struct torrent_pmu_counters {
+ int (*check_constraints)(struct torrent_pmu_events *torrent_pmu_events,
+ struct unit_config *unit_cfg);
+ int (*compute_pmc_regs)(struct torrent_pmu_events *torrent_pmu_events,
+ struct hcall_data *hcall_write);
+ void (*enable_disable_cntr)(int op,
+ struct torrent_pmu_events *torrent_pmu_events,
+ struct hcall_data *hcall_write);
+ int (*read_pmd)(struct torrent_pmu_events *torrent_pmu_events,
+ struct hcall_data *hcall_read);
+ void (*write_pmd)(struct torrent_pmu_events *torrent_pmu_events,
+ struct hcall_data *hcall_read);
+ int (*get_phys_pmd_reg)(u64 event_code);
+};
+
+/* This is the max number of instances of each type of Torrent PMU. Currently,
+ * all supported PMUs have a single instance. Supporting HFI non-windowed
+ * counters would require changing this to 2.
+ */
+#define MAX_TORRENT_PMUS_PER_TYPE 1
+
+struct torrent_events {
+ int count[TORRENT_NUM_PMU_TYPES][MAX_TORRENT_PMUS_PER_TYPE];
+ int max_count[TORRENT_NUM_PMU_TYPES];
+ struct torrent_pmu_counters
+ cntrs[MAX_TORRENT_CHIPS][TORRENT_NUM_PMU_TYPES];
+};
+
+extern void torrent_pmu_enable(struct pmu *pmu);
+extern void torrent_pmu_disable(struct pmu *pmu);
+extern void record_hcall_request_idx(int phys_register_index,
+ struct hcall_data *hcall_request,
+ u64 shadow_reg_index);
+#endif
diff --git a/arch/powerpc/kernel/perf_event.c b/arch/powerpc/kernel/perf_event.c
index 822f630..aa1eb43 100644
--- a/arch/powerpc/kernel/perf_event.c
+++ b/arch/powerpc/kernel/perf_event.c
@@ -19,6 +19,8 @@
#include <asm/firmware.h>
#include <asm/ptrace.h>
+#include <asm/torrent_nest_pmu.h>
+
struct cpu_hw_events {
int n_events;
int n_percpu;
@@ -1092,6 +1094,8 @@ static int power_pmu_event_init(struct perf_event *event)
break;
case PERF_TYPE_RAW:
ev = event->attr.config;
+ if (!IS_CORE_EVENT(ev))
+ return -ENOENT;
break;
default:
return -ENOENT;
diff --git a/arch/powerpc/platforms/Makefile b/arch/powerpc/platforms/Makefile
index 73e2116..d5db464 100644
--- a/arch/powerpc/platforms/Makefile
+++ b/arch/powerpc/platforms/Makefile
@@ -23,3 +23,4 @@ obj-$(CONFIG_PPC_PS3) += ps3/
obj-$(CONFIG_EMBEDDED6xx) += embedded6xx/
obj-$(CONFIG_AMIGAONE) += amigaone/
obj-$(CONFIG_PPC_WSP) += wsp/
+obj-$(CONFIG_PPC_P7IH) += p7ih/
diff --git a/arch/powerpc/platforms/p7ih/Makefile b/arch/powerpc/platforms/p7ih/Makefile
new file mode 100644
index 0000000..5b7b8db
--- /dev/null
+++ b/arch/powerpc/platforms/p7ih/Makefile
@@ -0,0 +1,10 @@
+
+subdir-ccflags-$(CONFIG_PPC_WERROR) := -Werror
+
+obj-$(CONFIG_PPC_P7IH) += mmu_pmu.o powerbus_bus_util_pmu.o \
+ powerbus_ll_pmu.o \
+ powerbus_mcd_pmu.o \
+ powerbus_wxyz_pmu.o \
+ mmu_pmu.o \
+ cau_pmu.o \
+ torrent_pmu.o
diff --git a/arch/powerpc/platforms/p7ih/cau_pmu.c b/arch/powerpc/platforms/p7ih/cau_pmu.c
new file mode 100644
index 0000000..3f70298
--- /dev/null
+++ b/arch/powerpc/platforms/p7ih/cau_pmu.c
@@ -0,0 +1,160 @@
+/*
+ * Performance counter support for IBM Torrent interconnect chip
+ *
+ * Copyright 2010, 2011 Carl Love, Corey Ashford, IBM Corporation.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ */
+
+#include <linux/kernel.h>
+#include <linux/perf_event.h>
+#include <asm/torrent_nest_pmu.h>
+
+
+int cau_pmu_check_constraints(struct torrent_pmu_events *pmu_events,
+ struct unit_config *unit_cfg)
+{
+ int total_events, i;
+ u8 virt_cntr;
+ u64 event_code;
+
+ total_events = pmu_events->n_events + pmu_events->n_add;
+ for (i = pmu_events->n_events; i < total_events; i++) {
+ /*
+ * Get the event code from attr.config value as the hw.config
+ * field is not set yet since the event has not been added.
+ */
+ event_code = pmu_events->event[i]->attr.config;
+ virt_cntr = (u8)TORRENT_VIRT_CTR_GET(event_code);
+ if (virt_cntr != 0) {
+ pr_err("%s:%d, CAU virt_cntr is %d instead of 0\n",
+ __func__, __LINE__, virt_cntr);
+ return -1;
+ }
+ if (unit_cfg->cntr_state[virt_cntr] != CNTR_FREE)
+ return -1;
+ unit_cfg->cntr_state[virt_cntr] = CNTR_IN_USE;
+ }
+ return 0;
+}
+
+int cau_compute_pmc_reg(struct torrent_pmu_events *pmu_events,
+ struct hcall_data *hcall_write)
+{
+ int i, total_events;
+ u8 virt_cntr;
+ u64 event_code;
+
+ /*
+ *There are no control registers in the MMU PMU. Just
+ * assign the hw.idx field.
+ */
+ total_events = pmu_events->n_events + pmu_events->n_add;
+ for (i = 0; i < total_events; i++) {
+ event_code = pmu_events->event[i]->hw.config;
+ virt_cntr = (u8)TORRENT_VIRT_CTR_GET(event_code);
+ if (virt_cntr != 0) {
+ pr_err("%s:%d, CAU virt_cntr is %u instead of 0\n",
+ __func__, __LINE__, virt_cntr);
+ return -1;
+ }
+ pmu_events->event[i]->hw.idx = virt_cntr;
+ }
+ return 0;
+}
+
+void cau_enable_disable_hw_cntr(int op, struct torrent_pmu_events *pmu_events)
+{
+ /*
+ * The MMU performance counters are free running, and can't be
+ * disabled or enabled.
+ *
+ * Function is needed as it is an argument to the function
+ * torrent_pmu_initialize() which is used to initialize each of the
+ * physical PMUs in the Torrent chip.
+ */
+}
+
+void cau_pmd_write(struct torrent_pmu_events *pmu_events,
+ struct hcall_data *hcall_read)
+{
+ int i;
+ u8 virt_cntr;
+ u64 value, event_code;
+
+ for (i = 0; i < pmu_events->n_events; i++) {
+ event_code = pmu_events->event[i]->hw.config;
+ virt_cntr = (u8)TORRENT_VIRT_CTR_GET(event_code);
+ /* CAU PMU has just a single counter */
+ if (virt_cntr != 0) {
+ pr_err("%s:%d, CAU virt_cntr is %u instead of 0\n",
+ __func__, __LINE__, virt_cntr);
+ return;
+ }
+ if (pmu_events->unit.cntr_state[virt_cntr] != CNTR_FREE) {
+ /*
+ * The counters are stopped. The HCall to
+ * read the counters has been done. The
+ * current count must be set to the previous
+ * count. The counters will be enabled.
+ * When the counters are stopped and read
+ * again, the actual count will be the
+ * new count minus the previous count.
+ */
+ value = hcall_read->
+ mmio_perf_counters.cycles_waiting_on_a_credit;
+ local64_set(&pmu_events->event[i]->hw.prev_count,
+ value);
+ }
+ }
+}
+
+int cau_pmd_read(struct torrent_pmu_events *pmu_events,
+ struct hcall_data *hcall_read)
+{
+ /*
+ * This PMU is accessed via hypervisor calls. The HCall to read the
+ * counters has been done. The counter data is in hcall_read.
+ */
+ int i;
+ u8 virt_cntr;
+ u64 value, prev, delta;
+ struct perf_event *event;
+
+ for (i = 0; i < pmu_events->n_events; i++) {
+ event = pmu_events->event[i];
+ virt_cntr = (u8)TORRENT_VIRT_CTR_GET(event->hw.config);
+ /* CAU PMU has just a single counter */
+ if (virt_cntr != 0) {
+ pr_err("%s:%d, CAU virt_cntr is %d instead of 0\n",
+ __func__, __LINE__, virt_cntr);
+ return -1;
+ }
+ if (pmu_events->unit.cntr_state[virt_cntr] == CNTR_IN_USE) {
+ /*
+ *Get the initial value, which was saved by the pmd
+ * write call into prev_count.
+ */
+ prev = local64_read(
+ &event->hw.prev_count);
+
+ value = hcall_read->
+ mmio_perf_counters.cycles_waiting_on_a_credit;
+
+ /* calculate/save the count */
+ delta = value - prev;
+ local64_add(delta, &event->count);
+ local64_set(&event->hw.prev_count, value);
+ }
+ }
+ return 0;
+}
+
+int cau_get_phys_pmd_reg(u64 event_code)
+{
+ /* This function never should be called */
+ return -1;
+}
diff --git a/arch/powerpc/platforms/p7ih/mmu_pmu.c b/arch/powerpc/platforms/p7ih/mmu_pmu.c
new file mode 100644
index 0000000..4d6d5c2
--- /dev/null
+++ b/arch/powerpc/platforms/p7ih/mmu_pmu.c
@@ -0,0 +1,156 @@
+/*
+ * Performance counter support for IBM Torrent interconnect chip
+ *
+ * Copyright 2010, 2011 Carl Love, Corey Ashford, IBM Corporation.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ */
+
+#include <linux/kernel.h>
+#include <linux/perf_event.h>
+#include <asm/torrent_nest_pmu.h>
+
+u64 get_mmu_counter_value(struct hcall_data *hcall_read, u8 virt_cntr)
+{
+ switch (virt_cntr) {
+ case 0:
+ return hcall_read->mmio_perf_counters.G_MMCHIT;
+ case 1:
+ return hcall_read->mmio_perf_counters.G_MMCMIS;
+ case 2:
+ return hcall_read->mmio_perf_counters.G_MMATHIT;
+ case 3:
+ return hcall_read->mmio_perf_counters.G_MMATMIS;
+ default:
+ pr_err("MMU event code has illegal virt counter field: %u",
+ virt_cntr);
+ return 0;
+ }
+}
+
+int mmu_pmu_check_constraints(struct torrent_pmu_events *pmu_events,
+ struct unit_config *unit_cfg)
+{
+ int total_events, i;
+ u8 virt_cntr;
+ u64 event_code;
+
+ total_events = pmu_events->n_events + pmu_events->n_add;
+ for (i = pmu_events->n_events; i < total_events; i++) {
+ /*
+ * Get the event code from attr.config value as the hw.config
+ * field is not set yet since the event has not been added.
+ */
+ event_code = pmu_events->event[i]->attr.config;
+ virt_cntr = (u8)TORRENT_VIRT_CTR_GET(event_code);
+
+ if (unit_cfg->cntr_state[virt_cntr] == CNTR_IN_USE)
+ return -1;
+
+ unit_cfg->cntr_state[virt_cntr] = CNTR_IN_USE;
+ }
+ return 0;
+}
+
+int mmu_compute_pmc_reg(struct torrent_pmu_events *pmu_events,
+ struct hcall_data *hcall_write)
+{
+ int i, total_events;
+ u8 virt_cntr;
+ u64 event_code;
+
+ /*
+ * There are no control registers in the MMU PMU. Just
+ * assign the hw.idx field.
+ */
+ total_events = pmu_events->n_events + pmu_events->n_add;
+ for (i = 0; i < total_events; i++) {
+ event_code = pmu_events->event[i]->hw.config;
+ virt_cntr = (u8)TORRENT_VIRT_CTR_GET(event_code);
+ pmu_events->event[i]->hw.idx = virt_cntr;
+ }
+ return 0;
+}
+
+void mmu_enable_disable_hw_cntr(int op, struct torrent_pmu_events *pmu_events)
+{
+ /*
+ * The MMU performance counters are free running, and can't be
+ * disabled or enabled.
+ *
+ * Function is needed as it is an argument to the function
+ * torrent_pmu_initialize() which is used to initialize each of the
+ * physical PMUs in the Torrent chip.
+ */
+}
+
+void mmu_pmd_write(struct torrent_pmu_events *pmu_events,
+ struct hcall_data *hcall_read)
+{
+ int i;
+ u8 virt_cntr;
+ u64 value, event_code;
+
+ for (i = 0; i < pmu_events->n_events; i++) {
+ event_code = pmu_events->event[i]->hw.config;
+ virt_cntr = (u8)TORRENT_VIRT_CTR_GET(event_code);
+
+ if (pmu_events->unit.cntr_state[virt_cntr] == CNTR_IN_USE) {
+ /*
+ * The counters are stopped. The HCall to
+ * read the counters has been done. The
+ * current count must be set to the previous
+ * count. The counters will be enabled.
+ * When the counters are stopped and read
+ * again, the actual count will be the
+ * new count minus the previous count.
+ */
+ value = get_mmu_counter_value(hcall_read, virt_cntr);
+ local64_set(&pmu_events->event[i]->hw.prev_count,
+ value);
+ }
+ }
+}
+
+int mmu_pmd_read(struct torrent_pmu_events *pmu_events,
+ struct hcall_data *hcall_read)
+{
+ /*
+ * This PMU is accessed via hypervisor calls. The HCall to read the
+ * counters has been done. The counter data is in hcall_read.
+ */
+ int i;
+ u8 virt_cntr;
+ u64 value, prev, delta;
+ struct perf_event *event;
+
+ for (i = 0; i < pmu_events->n_events; i++) {
+ event = pmu_events->event[i];
+ virt_cntr = (u8)TORRENT_VIRT_CTR_GET(event->hw.config);
+
+ if (pmu_events->unit.cntr_state[virt_cntr] == CNTR_IN_USE) {
+ /*
+ * Get the initial value, which was saved by the pmd
+ * write call into prev_count.
+ */
+ prev = local64_read(&event->hw.prev_count);
+
+ value = get_mmu_counter_value(hcall_read, virt_cntr);
+
+ /* calculate/save the count */
+ delta = value - prev;
+ local64_add(delta, &event->count);
+ local64_set(&event->hw.prev_count, value);
+ }
+ }
+ return 0;
+}
+
+int mmu_get_phys_pmd_reg(u64 event_code)
+{
+ /* This function never should be called */
+ return -1;
+}
diff --git a/arch/powerpc/platforms/p7ih/powerbus_bus_util_pmu.c b/arch/powerpc/platforms/p7ih/powerbus_bus_util_pmu.c
new file mode 100644
index 0000000..213c500
--- /dev/null
+++ b/arch/powerpc/platforms/p7ih/powerbus_bus_util_pmu.c
@@ -0,0 +1,410 @@
+/*
+ * Performance counter support for IBM Torrent interconnect chip
+ *
+ * Copyright 2010, 2011 Carl Love, Corey Ashford, IBM Corporation.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ */
+
+#include <linux/kernel.h>
+#include <linux/perf_event.h>
+#include <asm/torrent_nest_pmu.h>
+#include <asm/powerbus_mcd_pmu.h>
+#include <asm/power_torrent_events.h>
+
+#define BUS_UTIL_PMC_FIELD_WIDTH 8
+#define NUM_BUS_UTIL_PMC_REGS 4
+#define BUS_UTIL_CNTR_SIZE 16
+#define BUS_UTIL_OPCODE_MASK 0xFF
+#define NUM_BUS_UTIL_COUNTERS 4
+
+#define BUS_UTIL_APM_ENABLE_MASK 0x8000000000000000UL
+#define BUS_UTIL_PMU_ENABLE_MASK 0x1000000000000000UL
+#define BUS_UTIL_16_BIT_CNTR 16
+#define BUS_UTIL_32_BIT_CNTR 32
+#define BUS_UTIL_16_BIT_CNTR_MASK 0xFFFF
+#define BUS_UTIL_32_BIT_CNTR_MASK 0xFFFFFFFF
+#define BUS_UTIL_SHADOW_REG21 0
+#define BUS_UTIL_SHADOW_REG22 1
+#define BUS_UTIL_SHADOW_REG23 2
+#define BUS_UTIL_SHADOW_REG24 3
+#define BUS_UTIL_SHADOW_REG25 4
+#define BUS_UTIL_SHADOW_REG26 5
+#define BUS_UTIL_REG21_SEL_BASE (63 - 2)
+#define BUS_UTIL_HI_COMP_BASE (63 - 10)
+#define BUS_UTIL_LO_COMP_BASE (63 - 15)
+
+/*
+ * Bus utilization counters consist of a couple of configurable counters and
+ * a set of 10 dedicated counters. Register 21 has two enable bits:
+ * pb_cfg_apm_en and pb_cfg_pmucnt_en. Only one of these bits can be set at
+ * a time. The pb_cfg_apm_en enables the two 16-bit counters to count event.
+ * The pb_cfg_pmucnt_en enable is used to enable the counters to count the
+ * events specified in the pb_cfg_pmucnt_sel field. The select field options
+ * are:
+ * 00 - rcmd0 (reflected command 0)
+ * 01 - rcmd1 (reflected command 1)
+ * 1x - rcmd0 or rcmd 1
+ *
+ * The counters in registers 22 to 26 are enabled if pb_cfg_apm_en OR
+ * pb_cfg_pmucnt_en are enabled.
+ */
+
+int virtual_cntr_to_phys_reg(u8 virt_cntr)
+{
+ int phys_reg_num;
+
+ /*
+ * Map event to the physical hardware register that has the counter to
+ * count the specified event.
+ */
+
+ switch (virt_cntr) {
+ case 0:
+ case 1:
+ phys_reg_num = PBUS_REG21_PERF_CTR;
+ break;
+
+ case 2:
+ case 3:
+ phys_reg_num = PBUS_REG22_PERF_CTR;
+ break;
+
+ case 4:
+ case 5:
+ phys_reg_num = PBUS_REG23_PERF_CTR;
+ break;
+
+ case 6:
+ case 7:
+ phys_reg_num = PBUS_REG24_PERF_CTR;
+ break;
+
+ case 8:
+ case 9:
+ phys_reg_num = PBUS_REG25_PERF_CTR;
+ break;
+
+ case 10:
+ case 11:
+ phys_reg_num = PBUS_REG26_PERF_CTR;
+ break;
+
+ default:
+ pr_err("%s, %d ERROR not able to map virt counter to physical register\n",
+ __func__, __LINE__);
+ phys_reg_num = -1;
+ break;
+ }
+ return phys_reg_num;
+}
+
+int virt_cntr_to_shadow_reg(u8 virt_cntr)
+{
+ int phys_reg;
+
+ /*
+ * Map the virtual counter that counts the specified event to the
+ * physical register that contains the counter for that event.
+ *
+ * virtual counters 0, 1 are in BUS_UTIL_SHADOW_REG21 (index 0)
+ * virtual counters 2, 3 are in BUS_UTIL_SHADOW_REG22
+ * virtual counters 4, 5 are in BUS_UTIL_SHADOW_REG23
+ * virtual counters 6, 7 are in BUS_UTIL_SHADOW_REG24
+ * virtual counters 8, 9 are in BUS_UTIL_SHADOW_REG25
+ * virtual counters 10, 11 are in BUS_UTIL_SHADOW_REG26 (index 5)
+ */
+
+ if (virt_cntr < 12) {
+ phys_reg = virt_cntr >> 1; /* just divide virt counter by 2 */
+ } else {
+ pr_err("%s, %d ERROR not able to map virtual counter to physical register number..\n",
+ __func__, __LINE__);
+ phys_reg = -1;
+ }
+ return phys_reg;
+}
+
+int bus_util_pmu_check_constraints(struct torrent_pmu_events *pmu_events,
+ struct unit_config *unit_cfg)
+{
+ int i, total_events;
+ u8 virt_cntr;
+ u64 event_code, sel, enable_mask;
+
+ total_events = pmu_events->n_events + pmu_events->n_add;
+
+ for (i = pmu_events->n_events; i < total_events; i++) {
+ if (pmu_events->unit.cntr_state[i] != CNTR_FREE)
+ /*
+ * Each event has an assigned counter that is used to
+ * count the event. If the counter is already in use,
+ * the constraint check fails.
+ */
+ return -1;
+
+ /*
+ * Get the event code from attr.config value as the hw.config
+ * field is not set yet since the event has not been added.
+ */
+ event_code = pmu_events->event[i]->attr.config;
+ virt_cntr = TORRENT_VIRT_CTR_GET(event_code);
+
+ if ((virt_cntr >= 0) && (virt_cntr <= 3)) {
+ /* Check constraints for the reg21 and reg22
+ * registers.
+ *
+ * The sel field is shared by counters 0 through 3.
+ * The enable bits define which set of events the
+ * counters can count.
+ */
+ if (TORRENT_MUX_GET(event_code) == 0)
+ /* APM event */
+ enable_mask = BUS_UTIL_APM_ENABLE_MASK;
+ else
+ /* PMU event */
+ enable_mask = BUS_UTIL_PMU_ENABLE_MASK;
+
+ if (unit_cfg->bus_util_enable_mask == 0)
+ /* PMU configuration is not set yet */
+ unit_cfg->bus_util_enable_mask = enable_mask;
+
+ else
+ if (unit_cfg->bus_util_enable_mask
+ != enable_mask)
+ /* event and PMU config conflict */
+ return -1;
+
+ sel = TORRENT_ATTR_UTIL_SEL_MASK & event_code;
+
+ if (unit_cfg->bus_util_cntr_sel
+ == BUS_UTIL_CNTR_SEL_AVAIL)
+ unit_cfg->bus_util_cntr_sel = sel;
+ else if (unit_cfg->bus_util_cntr_sel != sel)
+ /*
+ * PMU sel configuration and event sel
+ * value are not compatible.
+ */
+ return -1;
+ }
+
+ /*
+ * Assign the event to the counter but it has not been
+ * enabled yet.
+ */
+ unit_cfg->cntr_state[virt_cntr] = CNTR_ALLOCATED;
+ }
+ return 0;
+}
+
+int bus_util_compute_pmc_reg(struct torrent_pmu_events *pmu_events,
+ struct hcall_data *hcall_write)
+{
+
+ int i, phys_reg, total_events, shift;
+ u8 virt_cntr;
+ u64 event_code, sel, comp;
+
+ /*
+ * The assumption is that we have passed the constraint test before
+ * this routine is called so we know that for the counter configuration
+ * (two or four counters) there are enough counters available for the
+ * new events.
+ */
+
+ total_events = pmu_events->n_events + pmu_events->n_add;
+
+ for (i = 0; i < total_events; i++) {
+ event_code = pmu_events->event[i]->hw.config;
+ virt_cntr = TORRENT_VIRT_CTR_GET(event_code);
+
+ pmu_events->event[i]->hw.idx = virt_cntr;
+
+ phys_reg = virtual_cntr_to_phys_reg(virt_cntr);
+ if (phys_reg == -1)
+ return -1; /* could not map event to phys_reg */
+
+ /*
+ * Assign the event to the counter but it has not been
+ * enabled yet.
+ */
+ pmu_events->unit.cntr_state[virt_cntr] = CNTR_ALLOCATED;
+
+ if (phys_reg >= PBUS_REG23_PERF_CTR)
+ /* no additional configuration information to write */
+ return 0;
+
+ /* set the sel and comp fields */
+ shift = 63 + TORRENT_ATTR_UTIL_SEL_SHIFT
+ - BUS_UTIL_REG21_SEL_BASE;
+ pmu_events->shadow_pmc_reg[virt_cntr] &=
+ ~(TORRENT_ATTR_UTIL_SEL_MASK << shift);
+
+ sel = (pmu_events->unit.bus_util_cntr_sel) << shift;
+
+ comp = event_code && TORRENT_ATTR_UTIL_CMP_MASK;
+ switch (virt_cntr) {
+ case 0:
+ case 2:
+ shift = 63 + TORRENT_ATTR_UTIL_CMP_SHIFT
+ - BUS_UTIL_HI_COMP_BASE;
+ break;
+
+ case 1:
+ case 3:
+ shift = 63 + TORRENT_ATTR_UTIL_CMP_SHIFT
+ - BUS_UTIL_LO_COMP_BASE;
+ break;
+
+ default:
+ pr_err("%s, ERROR, virtual counter value not in expected range. virt_cntr = %u\n",
+ __func__, virt_cntr);
+ return -1;
+ break;
+
+ }
+ comp = comp << shift;
+ pmu_events->shadow_pmc_reg[virt_cntr]
+ &= ~(TORRENT_ATTR_UTIL_CMP_MASK << shift);
+
+ pmu_events->shadow_pmc_reg[virt_cntr] |= comp
+ || pmu_events->unit.bus_util_cntr_sel;
+
+ /* Configuration register needs to be written */
+ record_hcall_request_idx(phys_reg, hcall_write,
+ pmu_events->shadow_pmc_reg[virt_cntr]);
+ }
+ return 0;
+}
+
+void bus_util_enable_disable_hw_cntr(int op,
+ struct torrent_pmu_events *pmu_events,
+ struct hcall_data *hcall_write)
+{
+ int index, cntr_num, phys_reg;
+ u8 virt_cntr;
+ u64 enable_bit;
+
+ for (cntr_num = 0; cntr_num < pmu_events->max_events;
+ cntr_num++) {
+ if (pmu_events->unit.cntr_state[cntr_num] != CNTR_FREE) {
+ /*
+ * Which enable bit to use is based on what events
+ * the first two counters are counting. The compute
+ * function will set the needed enable bit based on
+ * the events being counted. The physical register
+ * for virtual counter 0 contains the enable bits used
+ * by all of the counters.
+ */
+ virt_cntr = 0;
+ if (pmu_events->unit.bus_util_enable_mask == 0)
+ /*
+ * No REG21 events are configured, set the
+ * enable to APM enable to enable the counters
+ * for the other registers.
+ */
+ enable_bit = BUS_UTIL_APM_ENABLE_MASK;
+ else
+ enable_bit = pmu_events->
+ unit.bus_util_enable_mask;
+
+ phys_reg = virtual_cntr_to_phys_reg(virt_cntr);
+ index = virt_cntr_to_shadow_reg(virt_cntr);
+
+ /*
+ * The enable bits for the first two counters are used
+ * to control all of the counters.
+ */
+ if (op) {
+ pmu_events->shadow_pmc_reg[index] |=
+ enable_bit;
+ pmu_events->unit.cntr_state[cntr_num] =
+ CNTR_IN_USE;
+ } else if (pmu_events->unit.cntr_state[cntr_num]
+ == CNTR_IN_USE)
+ /*
+ * Disable only if the counter is actually
+ * in use.
+ */
+ pmu_events->shadow_pmc_reg[index] &=
+ ~enable_bit;
+
+ /* get entry, record which PMC needs to be written */
+ record_hcall_request_idx(phys_reg, hcall_write,
+ pmu_events->
+ shadow_pmc_reg[index]);
+ }
+ }
+}
+
+void bus_util_pmd_write(struct torrent_pmu_events *pmu_events,
+ struct hcall_data *hcall_read)
+{
+ int i, index;
+ u8 virt_cntr;
+ u64 value, event_code;
+
+ for (i = 0; i < pmu_events->n_events; i++) {
+ event_code = pmu_events->event[i]->hw.config;
+ virt_cntr = (u8)pmu_events->event[i]->hw.idx;
+
+ if (pmu_events->unit.cntr_state[virt_cntr] == CNTR_IN_USE) {
+ index = hcall_read->virt_cntr_to_request_idx
+ [TORRENT_PBUS_UTIL_ID][virt_cntr];
+ value = hcall_read->tor_pmu_reg[index].regValue;
+
+ local64_set(&pmu_events->event[i]->hw.prev_count,
+ value);
+ }
+ }
+}
+
+int bus_util_pmd_read(struct torrent_pmu_events *pmu_events,
+ struct hcall_data *hcall_read)
+{
+ /*
+ * This PMU is accessed via hypervisor calls. The HCall to read the
+ * counters has been done. The counter data is in hcall_read.
+ */
+ int i, index;
+ u8 virt_cntr;
+ u64 value, prev, delta;
+ struct perf_event *event;
+
+ for (i = 0; i < pmu_events->n_events; i++) {
+ event = pmu_events->event[i];
+ virt_cntr = (u8)pmu_events->event[i]->hw.idx;
+
+ if (pmu_events->unit.cntr_state[virt_cntr] == CNTR_IN_USE) {
+ /*
+ * Read the PMD register. Its count value is the
+ * current value minus the previous value from when
+ * the counter was started.
+ */
+
+ /* get current count, from the HCall */
+ index = hcall_read->virt_cntr_to_request_idx
+ [TORRENT_PBUS_UTIL_ID][virt_cntr];
+
+ /*
+ * Get initial value, which was saved by the pmd write
+ * call into prev_count.
+ */
+ prev = local64_read(&event->hw.prev_count);
+ value = hcall_read->tor_pmu_reg[index].regValue;
+ delta = ((value - prev) & BUS_UTIL_16_BIT_CNTR_MASK);
+
+ local64_add(delta, &event->count);
+ local64_set(&event->hw.prev_count, value);
+ }
+ }
+ return 0;
+}
+
+int bus_util_get_phys_pmd_reg(u64 event_code)
+{
+ return virtual_cntr_to_phys_reg(TORRENT_VIRT_CTR_GET(event_code));
+}
diff --git a/arch/powerpc/platforms/p7ih/powerbus_ll_pmu.c b/arch/powerpc/platforms/p7ih/powerbus_ll_pmu.c
new file mode 100644
index 0000000..bf8d453
--- /dev/null
+++ b/arch/powerpc/platforms/p7ih/powerbus_ll_pmu.c
@@ -0,0 +1,273 @@
+/*
+ * Performance counter support for IBM Torrent interconnect chip
+ *
+ * Copyright 2010, 2011 Carl Love, Corey Ashford, IBM Corporation.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ */
+
+#include <linux/kernel.h>
+#include <linux/perf_event.h>
+#include <asm/torrent_nest_pmu.h>
+#include <asm/powerbus_ll_pmu.h>
+#include <asm/power_torrent_events.h>
+
+#define LL_PMC_FIELD_WIDTH 4
+#define LL_NUM_VIRT_CTRS 7
+#define LL_CTR_SIZE 16
+#define LL_OPCODE_MASK 0x7ULL
+#define LL_PMD_MASK 0xFFFFULL
+#define LL_ENABLE_MASK (1ULL << (63 - 6))
+#define LL_SHADOW_IDX 0
+
+#define PB_CFG_PERF_CNT_MODE (1ULL << (63 - 7))
+
+int ll_link_pmu_check_constraints(struct torrent_pmu_events *pmu_events,
+ struct unit_config *unit_cfg)
+{
+ /*
+ * Check that there is only one event for each link (LL0..LL7)
+ * being counted at a time.
+ */
+ int i, total_events;
+ u8 virt_cntr;
+ u64 event_code;
+
+ total_events = pmu_events->n_events + pmu_events->n_add;
+
+ for (i = pmu_events->n_events; i < total_events; i++) {
+ /*
+ * Get the event code from attr.config value as the hw.config
+ * field is not set yet since the event has not been added.
+ */
+ event_code = pmu_events->event[i]->attr.config;
+ virt_cntr = (u8)TORRENT_VIRT_CTR_GET(event_code);
+ if (unit_cfg->cntr_state[virt_cntr] != CNTR_FREE)
+ return -1;
+
+ /*
+ * Assign the event to the counter but it has not been
+ * enabled yet.
+ */
+ unit_cfg->cntr_state[virt_cntr] = CNTR_ALLOCATED;
+ }
+ return 0;
+}
+
+int ll_link_compute_pmc_reg(struct torrent_pmu_events *pmu_events,
+ struct hcall_data *hcall_write)
+{
+ int i, total_events, shift_by;
+ u64 event_code;
+ u8 mux, virt_cntr;
+
+ total_events = pmu_events->n_events + pmu_events->n_add;
+
+ for (i = 0; i < total_events; i++) {
+ event_code = pmu_events->event[i]->hw.config;
+ virt_cntr = (u8)TORRENT_VIRT_CTR_GET(event_code);
+ pmu_events->event[i]->hw.idx = virt_cntr;
+ mux = (u8) TORRENT_MUX_GET(event_code);
+
+ /*
+ * The PBL_LINKS_TRACE_PERF_CTR_CFG event fields are:
+ * LL0 counter [52:49]
+ * LL1 counter [48:45]
+ * LL2 counter [44:41]
+ * LL3 counter [40:37]
+ * LL4 counter [36:33]
+ * LL5 counter [32:29]
+ * LL6 counter [28:25]
+ */
+ shift_by = 25 + (LL_NUM_VIRT_CTRS - virt_cntr - 1)
+ * LL_PMC_FIELD_WIDTH;
+
+ pmu_events->shadow_pmc_reg[LL_SHADOW_IDX]
+ &= ~(LL_OPCODE_MASK << shift_by);
+ pmu_events->shadow_pmc_reg[LL_SHADOW_IDX]
+ |= (u64)mux << shift_by | PB_CFG_PERF_CNT_MODE;
+
+ /* record which PMC needs to be written */
+ record_hcall_request_idx(PBL_LINKS_TRACE_PERF_CTR_CFG,
+ hcall_write,
+ pmu_events->
+ shadow_pmc_reg[LL_SHADOW_IDX]);
+ }
+ return 0;
+}
+
+void ll_link_enable_disable_hw_cntr(int op,
+ struct torrent_pmu_events *pmu_events,
+ struct hcall_data *hcall_write)
+{
+ int cntr_num;
+
+ for (cntr_num = 0; cntr_num < pmu_events->max_events;
+ cntr_num++) {
+ if (op) {
+ /* enable if cntr is in use or has been allocated */
+ if (pmu_events->unit.cntr_state[cntr_num] !=
+ CNTR_FREE) {
+ pmu_events->shadow_pmc_reg[LL_SHADOW_IDX] |=
+ LL_ENABLE_MASK;
+ pmu_events->unit.cntr_state[cntr_num] =
+ CNTR_IN_USE;
+ }
+ } else {
+ /* disable, only if the counter is actually in use */
+ if (pmu_events->unit.cntr_state[cntr_num] ==
+ CNTR_IN_USE)
+ pmu_events->shadow_pmc_reg[LL_SHADOW_IDX] &=
+ ~LL_ENABLE_MASK;
+ }
+
+ /* record which PMC needs to be written */
+ record_hcall_request_idx(PBL_LINKS_TRACE_PERF_CTR_CFG,
+ hcall_write,
+ pmu_events->shadow_pmc_reg
+ [LL_SHADOW_IDX]);
+ }
+}
+
+void ll_link_pmd_write(struct torrent_pmu_events *pmu_events,
+ struct hcall_data *hcall_read)
+{
+ /*
+ * The physical counters are 16 bits wide each.
+ * The eight counters are packed into two registers:
+ * LL0 counter -> PBL Links Counters 1 [47:32]
+ * LL1 counter -> PBL Links Counters 1 [31:16]
+ * LL2 counter -> PBL Links Counters 1 [15:0]
+ * LL3 counter -> PBL Links Counters 2 [63:48]
+ * LL4 counter -> PBL Links Counters 2 [47:32]
+ * LL5 counter -> PBL Links Counters 2 [31:16]
+ * LL6 counter -> PBL Links Counters 2 [15:0]
+ */
+
+ /*
+ * The PMD can only be accessed via an HCall. The four counters are
+ * packed into a single register. It is not practical to read the
+ * register, reset one counter field, then write the counter back.
+ * Secondly, being able to write a non-zero value is only needed when
+ * collecting a profile. Profiling on non-CPU counters doesn't make
+ * sense since there is no reliable way to associate the number of
+ * events that have occurred back to a specific instruction.
+ * Therefore, we will not support writing a value. The enable/disable
+ * HCall will need to sample the counts before the counter is enabled
+ * and then generate a delta count when the counters are disabled
+ */
+ int i, index;
+ u8 virt_cntr;
+ u64 event_code, value;
+
+ for (i = 0; i < pmu_events->n_events; i++) {
+ event_code = pmu_events->event[i]->hw.config;
+ virt_cntr = TORRENT_VIRT_CTR_GET(event_code);
+
+ if (pmu_events->unit.cntr_state[virt_cntr] == CNTR_IN_USE) {
+ /*
+ * The counters are stopped. The HCall to
+ * read the counters has been done. The
+ * current count must be set to the previous
+ * count. The counters will be enabled.
+ * When the counters are stopped and read
+ * again, the actual count will be the
+ * new count minus the previous count.
+ */
+
+ /*
+ * Get the request entry that contains the PMD for
+ * this virtual counter that is in use.
+ */
+ index = hcall_read->virt_cntr_to_request_idx
+ [TORRENT_PBUS_LL_ID][virt_cntr];
+
+ value = hcall_read->tor_pmu_reg[index].regValue;
+ if (virt_cntr < 3)
+ value = value >> ((2 - virt_cntr)
+ * LL_CTR_SIZE) & LL_PMD_MASK;
+ else
+ value = value >> ((6 - virt_cntr)
+ * LL_CTR_SIZE) & LL_PMD_MASK;
+ local64_set(&pmu_events->event[i]->hw.prev_count,
+ value);
+ }
+ }
+}
+
+int ll_link_pmd_read(struct torrent_pmu_events *pmu_events,
+ struct hcall_data *hcall_read)
+{
+ /*
+ * This PMU is accessed via hypervisor calls. The HCall to read the
+ * counters has been done. The counter data is in hcall_read.
+ */
+ int i, index;
+ u8 virt_cntr;
+ u64 value, prev, delta;
+ struct perf_event *event;
+
+ /*
+ * The physical counters are 16 bits wide each.
+ * The eight counters are packed into two registers:
+ * LL0 counter -> PBL Links Counters 1 [47:32]
+ * LL1 counter -> PBL Links Counters 1 [31:16]
+ * LL2 counter -> PBL Links Counters 1 [15:0]
+ * LL3 counter -> PBL Links Counters 2 [63:48]
+ * LL4 counter -> PBL Links Counters 2 [47:32]
+ * LL5 counter -> PBL Links Counters 2 [31:16]
+ * LL6 counter -> PBL Links Counters 2 [15:0]
+ */
+
+ for (i = 0; i < pmu_events->n_events; i++) {
+ event = pmu_events->event[i];
+ virt_cntr = (u8)TORRENT_VIRT_CTR_GET(event->hw.config);
+
+ if (pmu_events->unit.cntr_state[virt_cntr] == CNTR_IN_USE) {
+ /*
+ * Read the PMD register. Its count value is the
+ * current value minus the previous value from when
+ * the counter was started.
+ */
+
+ /* get current count, from the HCall */
+ index = hcall_read->virt_cntr_to_request_idx
+ [TORRENT_PBUS_LL_ID][virt_cntr];
+
+ /*
+ * Get initial value, which was saved by the pmd write
+ * call into prev_count.
+ */
+ prev = local64_read(&event->hw.prev_count);
+ value = hcall_read->tor_pmu_reg[index].regValue;
+
+ if (virt_cntr < 3)
+ value = value >> ((2 - virt_cntr)
+ * LL_CTR_SIZE) & LL_PMD_MASK;
+ else
+ value = value >> ((6 - virt_cntr)
+ * LL_CTR_SIZE) & LL_PMD_MASK;
+
+ /* calculate/save the count */
+ delta = ((value - prev) & 0xffffUL) << 16;
+ local64_add(delta, &event->count);
+ local64_set(&event->hw.prev_count, value);
+ }
+ }
+ return 0;
+}
+
+int ll_link_get_phys_pmd_reg(u64 event_code)
+{
+ int virt_cntr = TORRENT_VIRT_CTR_GET(event_code);
+
+ if (virt_cntr < 3)
+ /* LL0..LL2 */
+ return PBL_LINKS_CTRS_1;
+ else
+ /* LL3.. LL6 */
+ return PBL_LINKS_CTRS_2;
+}
diff --git a/arch/powerpc/platforms/p7ih/powerbus_mcd_pmu.c b/arch/powerpc/platforms/p7ih/powerbus_mcd_pmu.c
new file mode 100644
index 0000000..57070e5
--- /dev/null
+++ b/arch/powerpc/platforms/p7ih/powerbus_mcd_pmu.c
@@ -0,0 +1,492 @@
+/*
+ * Performance counter support for IBM Torrent interconnect chip
+ *
+ * Copyright 2010, 2011 Carl Love, Corey Ashford, IBM Corporation.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ */
+
+#include <linux/kernel.h>
+#include <linux/perf_event.h>
+#include <asm/torrent_nest_pmu.h>
+#include <asm/powerbus_mcd_pmu.h>
+#include <asm/power_torrent_events.h>
+
+#define MCD_PMC_FIELD_WIDTH 8
+#define NUM_MCD_PMC_REGS 4
+#define MCD_CNTR_SIZE 16
+#define MCD_OPCODE_MASK 0xFF
+#define NUM_MCD_COUNTERS 4
+#define MCD_ENABLE_MASK 0x8000000000000000ULL
+#define MCD_UNIT_ROUTE_MASK 0x1L
+#define MCD_UNIT_ROUTE_CLEAR_MASK 0x11L
+#define MCD_16_BIT_CNTR 16
+#define MCD_32_BIT_CNTR 32
+#define MCD_16_BIT_CNTR_MASK 0xFFFF
+#define MCD_32_BIT_CNTR_MASK 0xFFFFFFFF
+#define MCD_CFG_SHADOW_REG_IDX 0
+#define MCD_SEL_SHADOW_REG_IDX 1
+
+u64 get_counter_attr(u64 event_code)
+{
+ u64 event_counter_type;
+
+ /* extract the counter type from the attribute field */
+ event_counter_type = TORRENT_ATTR_MCD_TYPE_MASK & event_code;
+
+ /*
+ * Move the type field to the location for the physical
+ * register.
+ */
+ event_counter_type >>= TORRENT_ATTR_MCD_TYPE_SHIFT;
+
+ return event_counter_type;
+}
+
+int mcd_get_prescale(u64 event_counter_type)
+{
+ switch (event_counter_type) {
+ case 0x0L: /* two 64-bit counters with 32-bit prescale */
+ return 32;
+ break;
+
+ /* four 32-bit counters with 16-bit prescale */
+ case 0x1L:
+ return 16;
+ break;
+
+ /* two 32-bit counters no prescale. */
+ case 0x2L:
+ /* four 16-bit counters no prescale. */
+ case 0x3L:
+ return 0;
+ break;
+
+ default:
+ pr_err("%s ERROR, UNKNOWN counter configuration setting 0x%Lx.\n",
+ __func__, event_counter_type);
+ return -1;
+ }
+}
+
+int mcd_get_num_ctrs(u64 event_counter_type)
+{
+ switch (event_counter_type) {
+ /* two 64-bit counters with 32-bit prescale */
+ case 0x0L:
+ /* two 32-bit counters no prescale. */
+ case 0x2L:
+ return 2;
+ break;
+
+ /* four 32-bit counters with 16-bit prescale */
+ case 0x1L:
+ /* four 16-bit counters no prescale. */
+ case 0x3L:
+ return 4;
+ break;
+
+ default:
+ pr_err("%s ERROR, UNKNOWN counter configuration setting, 0x%Lx.\n",
+ __func__, event_counter_type);
+ return -1;
+ }
+}
+
+int mcd_assign_counter(u8 *physical_counter_num,
+ struct torrent_pmu_events *pmu_events,
+ int num_config_cntrs)
+{
+ int i;
+ /*
+ * If the physical_counter_num is not equal to -1, then the event
+ * has already been assigned to a counter. Otherwise, assign
+ * physical_counter_num to the next available counter.
+ */
+
+ /* Look for first available counter, assign it */
+ for (i = 0; i < num_config_cntrs; i++) {
+ if (pmu_events->unit.cntr_state[i] == CNTR_FREE) {
+ pmu_events->unit.cntr_state[i] = CNTR_IN_USE;
+ *physical_counter_num = i;
+ return 0;
+ }
+ }
+
+ pr_err("%s, ERROR could not find an available counter to use.\n",
+ __func__);
+ return -1;
+}
+
+int mcd_pmu_check_constraints(struct torrent_pmu_events *pmu_events,
+ struct unit_config *unit_cfg)
+{
+ /*
+ * The MCD 0 and 1 units share the same physical counters. Therefore,
+ * copy_unit_config sends unit 0's unit config structure to check
+ * constraints so both units share the same counter reservation.
+ */
+
+ int i, total_events;
+ int config_max_counters;
+ u64 event_code;
+
+ total_events = pmu_events->n_events + pmu_events->n_add;
+
+ for (i = pmu_events->n_events; i < total_events; i++) {
+ /*
+ * Get the event code from attr.config value as the hw.config
+ * field is not set yet since the event has not been added.
+ */
+ event_code = pmu_events->event[i]->attr.config;
+
+ /*
+ * If the PMU is currently not counting any events, set the
+ * counter attribute. Make sure counter attribute for all
+ * events are consistent. Also, if the events are
+ * accepted, i.e. n_counters for this unit is updated, the
+ * counter attribute will also be set. If the events are not
+ * accepted, the counter attribute is not changed
+ */
+ if ((pmu_events->n_events == 0) && (i == 0))
+ /*
+ * If no counters are in use and this is the first
+ * event in the group, set the counter attribute and
+ * check that all the other events in the group are
+ * consistent.
+ */
+ unit_cfg->mcd_cntr_attr = get_counter_attr(event_code);
+
+ /*
+ * The event counter configuration must match the
+ * current counter configuration.
+ */
+ if (get_counter_attr(event_code) != unit_cfg->mcd_cntr_attr) {
+ pr_warning("%s Warning, rejecting MCD PMU events due to inconsistent counter attribute.\n",
+ __func__);
+ return -1;
+ }
+ }
+
+ /*
+ * Check that there are enough counters for the events given the
+ * number of counters configured as specified by the counter attribute.
+ */
+ config_max_counters = mcd_get_num_ctrs(unit_cfg->mcd_cntr_attr);
+
+ if (config_max_counters == -1)
+ return -1;
+
+ if (total_events > config_max_counters) {
+ pr_warning("%s Warning, rejecting PMU events due to insufficient number of counters\n",
+ __func__);
+ return -1;
+ }
+ return 0;
+}
+
+int mcd_compute_pmc_reg(struct torrent_pmu_events *pmu_events,
+ struct hcall_data *hcall_write)
+{
+ /* There are two MCD units however, they share the same physical
+ * PMU counters. Hence we only have one MCD PMU unit. The unit
+ * number is used to calculate the appropriate shift to write the
+ * unit specific counter event field.
+ */
+
+ int i, total_events, shift_by, input_unit, num_config_cntrs;
+ u8 virt_cntr;
+ u64 event_code, event_counter_type, err, mux;
+
+ /* The assumption is that we have passed the constraint test before
+ * this routine is called so we know that for the counter configuration
+ * (two or four counters) there are enough counters available for the
+ * new events.
+ */
+
+ total_events = pmu_events->n_events + pmu_events->n_add;
+
+ for (i = 0; i < total_events; i++) {
+ event_code = pmu_events->event[i]->hw.config;
+
+ /* extract the counter type from the attribute field */
+ event_counter_type = get_counter_attr(event_code);
+ num_config_cntrs = mcd_get_num_ctrs(event_counter_type);
+ err = mcd_assign_counter(&virt_cntr, pmu_events,
+ num_config_cntrs);
+ if (err)
+ return -1;
+
+ pmu_events->event[i]->hw.idx = virt_cntr;
+ pmu_events->unit.cntr_state[i] = CNTR_ALLOCATED;
+
+ /* determine which of the input MCD unit signals to count */
+ input_unit = TORRENT_UNIT_GET(event_code);
+
+ /*
+ * Setup the MCD config register fields:
+ * counter configuration field
+ * Unit signal routing field for the counter
+ */
+ shift_by = 52 + (NUM_MCD_PMC_REGS - virt_cntr - 1);
+
+ /* Clear signal routing bits for both units.e */
+ pmu_events->shadow_pmc_reg[MCD_CFG_SHADOW_REG_IDX] &=
+ ~(MCD_UNIT_ROUTE_CLEAR_MASK << shift_by);
+
+ if (input_unit == 0)
+ shift_by += 4;
+
+ pmu_events->shadow_pmc_reg[MCD_CFG_SHADOW_REG_IDX] |=
+ MCD_UNIT_ROUTE_MASK << shift_by;
+
+ /* set the counter number/prescale bits in config register */
+ pmu_events->shadow_pmc_reg[MCD_CFG_SHADOW_REG_IDX]
+ &= ~TORRENT_REG_ATTR_MCD_TYPE_MASK;
+
+ pmu_events->shadow_pmc_reg[MCD_CFG_SHADOW_REG_IDX]
+ |= event_counter_type
+ << TORRENT_REG_ATTR_MCD_TYPE_SHIFT;
+
+ /* Configuration register needs to be written */
+ record_hcall_request_idx(PBUS_MCD_PERF_CTRS_CFG, hcall_write,
+ pmu_events->shadow_pmc_reg
+ [MCD_CFG_SHADOW_REG_IDX]);
+
+ /* Setup the MCD selection register */
+ mux = (u64) TORRENT_MUX_GET(event_code);
+
+ /* The MCD event fields are:
+ * MCD 0 counter 0 [63:56]
+ * MCD 0 counter 1 [55:48]
+ * MCD 0 counter 2 [47:40]
+ * MCD 0 counter 3 [39:32]
+ * MCD 1 counter 0 [31:24]
+ * MCD 1 counter 1 [23:16]
+ * MCD 1 counter 2 [15:8]
+ * MCD 1 counter 3 [7:0]
+ */
+ if (input_unit == 0) /* adjust position based on input unit */
+ shift_by = 32;
+ else
+ shift_by = 0;
+
+ shift_by += (NUM_MCD_PMC_REGS - virt_cntr - 1)
+ * MCD_PMC_FIELD_WIDTH;
+
+ /*
+ * Set the event specifier field for the counter in the
+ * selection register
+ */
+ pmu_events->shadow_pmc_reg[MCD_SEL_SHADOW_REG_IDX] &=
+ MCD_OPCODE_MASK << shift_by;
+ pmu_events->shadow_pmc_reg[MCD_SEL_SHADOW_REG_IDX]
+ |= mux << shift_by;
+
+ /* Get request_idx and record which PMC needs to be written */
+ record_hcall_request_idx(PBUS_MCD_PERF_CTRS_SEL,
+ hcall_write,
+ pmu_events->shadow_pmc_reg
+ [MCD_SEL_SHADOW_REG_IDX]);
+ }
+ return 0;
+}
+
+void mcd_enable_disable_hw_cntr(int op, struct torrent_pmu_events *pmu_events,
+ struct hcall_data *hcall_write)
+{
+ int cntr_num;
+
+ /* There is a single PMU enable bit for all counters
+ *
+ * Note, the PMU can be configured with 2 or 4 physical counters.
+ */
+
+ for (cntr_num = 0; cntr_num < pmu_events->max_events;
+ cntr_num++) {
+ if (op) {
+ /* enable if cntr is in use or has been allocated */
+ if (pmu_events->unit.cntr_state[cntr_num] !=
+ CNTR_FREE) {
+ pmu_events->shadow_pmc_reg
+ [MCD_CFG_SHADOW_REG_IDX] |=
+ MCD_ENABLE_MASK;
+ pmu_events->unit.cntr_state[cntr_num] =
+ CNTR_IN_USE;
+ }
+ } else {
+ /* disable, only if the counter is actually in use */
+ if (pmu_events->unit.cntr_state[cntr_num] ==
+ CNTR_IN_USE)
+ pmu_events->shadow_pmc_reg
+ [MCD_CFG_SHADOW_REG_IDX] &=
+ ~MCD_ENABLE_MASK;
+ }
+ /* record which PMC needs to be written */
+ record_hcall_request_idx(PBUS_MCD_PERF_CTRS_CFG, hcall_write,
+ pmu_events->shadow_pmc_reg
+ [MCD_CFG_SHADOW_REG_IDX]);
+ }
+}
+
+void mcd_pmd_write(struct torrent_pmu_events *pmu_events,
+ struct hcall_data *hcall_read)
+{
+ /*
+ * All counters are packed into a single physical register:
+ * Four counter layout: (16-bit counters)
+ * counter 0 [63:48]
+ * counter 1 [47:32]
+ * counter 2 [31:16]
+ * counter 3 [15:0]
+ *
+ * Two counter layout: (32-bit counters)
+ * counter 0 [63:32]
+ * counter 1 [31:0]
+ */
+
+ /*
+ * The PMD can only be accessed via an HCall. The counters are
+ * packed into a single register. It is not practical to read the
+ * register, reset one counter field, then write the counter back.
+ * Secondly, being able to write a non-zero value is only needed when
+ * collecting a profile. Profiling on non-CPU counters doesn't make
+ * sense since there is no reliable way to associate the number of
+ * events that have occurred back to a specific instruction.
+ * Therefore, we will not support writing a value. The enable/disable
+ * HCall will need to sample the counts before the counter is enabled
+ * and then generate a delta count when the counters are disabled.
+ */
+ int i, index, num_config_cntrs;
+ u8 virt_cntr;
+ u64 value, event_code;
+
+ for (i = 0; i < pmu_events->n_events; i++) {
+ event_code = pmu_events->event[i]->hw.config;
+ virt_cntr = pmu_events->event[i]->hw.idx;
+ num_config_cntrs = mcd_get_num_ctrs(
+ pmu_events->unit.mcd_cntr_attr);
+
+ if (pmu_events->unit.cntr_state[virt_cntr] == CNTR_IN_USE) {
+ /*
+ * The counters are stopped. The HCall to
+ * read the counters has been done. The
+ * current count must be set to the previous
+ * count. The counters will be enabled.
+ * When the counters are stopped and read
+ * again, the actual count will be the
+ * new count minus the previous count.
+ */
+
+ /*
+ * Get the request entry that contains the PMD for
+ * this virtual counter.
+ */
+ index = hcall_read->virt_cntr_to_request_idx
+ [TORRENT_PBUS_MCD_ID][virt_cntr];
+ value = hcall_read->tor_pmu_reg[index].regValue;
+
+ if (num_config_cntrs == 2)
+ value >>= ((num_config_cntrs - 1 - virt_cntr)
+ * MCD_32_BIT_CNTR) &
+ MCD_32_BIT_CNTR_MASK;
+ else
+ value >>= ((num_config_cntrs - 1 - virt_cntr)
+ * MCD_16_BIT_CNTR) &
+ MCD_16_BIT_CNTR_MASK;
+ local64_set(&pmu_events->event[i]->hw.prev_count,
+ value);
+ }
+ }
+}
+
+int mcd_pmd_read(struct torrent_pmu_events *pmu_events,
+ struct hcall_data *hcall_read)
+{
+ /*
+ * This PMU is accessed via hypervisor calls. The HCall to read the
+ * counters has been done. The counter data is in hcall_read.
+ */
+ int i, index, num_config_cntrs, prescaler;
+ u8 virt_cntr;
+ u64 value, prev, delta;
+ struct perf_event *event;
+
+ /*
+ * All counters are packed into a single physical register:
+ * Four-counter layout: (16-bit counters)
+ * counter 0 [63:48]
+ * counter 1 [47:32]
+ * counter 2 [31:16]
+ * counter 3 [15:0]
+ *
+ * Two-counter layout: (32-bit counters)
+ * counter 0 [63:32]
+ * counter 1 [31:0]
+ */
+
+ for (i = 0; i < pmu_events->n_events; i++) {
+ event = pmu_events->event[i];
+ virt_cntr = pmu_events->event[i]->hw.idx;
+ prescaler = mcd_get_prescale(pmu_events->unit.mcd_cntr_attr);
+ num_config_cntrs = mcd_get_num_ctrs(
+ pmu_events->unit.mcd_cntr_attr);
+
+ if (prescaler == -1) {
+ pr_err("%s ERROR, Unknown counter configuration.\n",
+ __func__);
+ return -1;
+ }
+
+ if (pmu_events->unit.cntr_state[virt_cntr] == CNTR_IN_USE) {
+ /*
+ * Read the PMD register. Its count value is the
+ * current value minus the previous value from when
+ * the counter was started.
+ */
+
+ /* get current count, from the HCall */
+ index = hcall_read->virt_cntr_to_request_idx
+ [TORRENT_PBUS_MCD_ID][virt_cntr];
+
+ /*
+ * Get initial value, which was saved by the PMD write
+ * call into prev_count.
+ */
+ prev = local64_read(&event->hw.prev_count);
+ value = hcall_read->tor_pmu_reg[index].regValue;
+
+ if (num_config_cntrs == 2) {
+ value >>= ((num_config_cntrs - 1
+ - virt_cntr) * MCD_32_BIT_CNTR)
+ & MCD_32_BIT_CNTR_MASK;
+
+ delta = (value - prev) & MCD_32_BIT_CNTR_MASK;
+
+ } else {
+ value = value >> ((num_config_cntrs - 1
+ - virt_cntr)
+ * MCD_16_BIT_CNTR)
+ & MCD_16_BIT_CNTR_MASK;
+
+ delta = ((value - prev)
+ & MCD_16_BIT_CNTR_MASK);
+ }
+ delta = delta << prescaler;
+ local64_add(delta, &event->count);
+ local64_set(&event->hw.prev_count, value);
+ }
+ }
+ return 0;
+}
+
+
+int mcd_get_phys_pmd_reg(u64 event_code)
+{
+ /* All of the MCD counters use this single physical PMD
+ * register. */
+ return PBUS_MCD_PERF_CTRS;
+}
diff --git a/arch/powerpc/platforms/p7ih/powerbus_wxyz_pmu.c b/arch/powerpc/platforms/p7ih/powerbus_wxyz_pmu.c
new file mode 100644
index 0000000..e541469
--- /dev/null
+++ b/arch/powerpc/platforms/p7ih/powerbus_wxyz_pmu.c
@@ -0,0 +1,244 @@
+/*
+ * Performance counter support for IBM Torrent interconnect chip
+ *
+ * Copyright 2010, 2011 Carl Love, Corey Ashford, IBM Corporation.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ */
+
+#include <linux/kernel.h>
+#include <linux/perf_event.h>
+#include <asm/torrent_nest_pmu.h>
+#include <asm/powerbus_wxyz_pmu.h>
+#include <asm/power_torrent_events.h>
+
+#define WXYZ_PMC_FIELD_WIDTH 3
+#define WXYZ_NUM_VIRT_CTRS 4
+#define WXYZ_CNTR_SIZE 16
+#define WXYZ_OPCODE_MASK 0x7ULL
+#define WXYZ_PMD_MASK 0xFFFFULL
+#define WXYZ_ENABLE_MASK (1ULL << (63 - 6))
+#define WXYZ_SHADOW_IDX 0
+
+int wxyz_link_pmu_check_constraints(struct torrent_pmu_events *pmu_events,
+ struct unit_config *unit_cfg)
+{
+ /*
+ * Check that there is only one event for each link (W, X,
+ * Y and Z) being counted at a time.
+ */
+ int i, total_events;
+ u8 virt_cntr;
+ u64 event_code;
+
+ total_events = pmu_events->n_events + pmu_events->n_add;
+
+ for (i = pmu_events->n_events; i < total_events; i++) {
+ /*
+ * Get the event code from attr.config value as the hw.config
+ * field is not set yet since the event has not been added.
+ */
+ event_code = pmu_events->event[i]->attr.config;
+ virt_cntr = (u8)TORRENT_VIRT_CTR_GET(event_code);
+ if (unit_cfg->cntr_state[virt_cntr] != CNTR_FREE)
+ return -1;
+
+ /*
+ * Assign the event to the counter but it has not been
+ * enabled yet.
+ */
+ unit_cfg->cntr_state[virt_cntr] = CNTR_ALLOCATED;
+ }
+ return 0;
+}
+
+int wxyz_link_compute_pmc_reg(struct torrent_pmu_events *pmu_events,
+ struct hcall_data *hcall_write)
+{
+ int i, total_events, shift_by;
+ u8 mux, virt_cntr;
+ u64 event_code;
+
+ total_events = pmu_events->n_events + pmu_events->n_add;
+
+ for (i = 0; i < total_events; i++) {
+ event_code = pmu_events->event[i]->hw.config;
+ virt_cntr = TORRENT_VIRT_CTR_GET(event_code);
+ pmu_events->event[i]->hw.idx = virt_cntr;
+ mux = (u8)TORRENT_MUX_GET(event_code);
+
+ /* The PBE_LINKS_TRACE event fields are:
+ * W counter [51:49]
+ * X counter [48:46]
+ * Y counter [45:43]
+ * Z counter [42:40]
+ */
+ shift_by = 40 +
+ (WXYZ_NUM_VIRT_CTRS - virt_cntr - 1) *
+ WXYZ_PMC_FIELD_WIDTH;
+
+ pmu_events->shadow_pmc_reg[WXYZ_SHADOW_IDX]
+ &= ~(WXYZ_OPCODE_MASK << shift_by);
+ pmu_events->shadow_pmc_reg[WXYZ_SHADOW_IDX]
+ |= (u64)mux << shift_by;
+
+ /* record which PMC needs to be written */
+ record_hcall_request_idx(PBE_LINKS_TRACE,
+ hcall_write,
+ pmu_events->
+ shadow_pmc_reg[WXYZ_SHADOW_IDX]);
+ }
+ return 0;
+}
+
+void wxyz_link_enable_disable_hw_cntr(int op,
+ struct torrent_pmu_events *pmu_events,
+ struct hcall_data *hcall_write)
+{
+ int cntr_num;
+
+ for (cntr_num = 0; cntr_num < pmu_events->max_events; cntr_num++) {
+ if (op) {
+ /* enable if cntr is in use or has been allocated */
+ if (pmu_events->unit.cntr_state[cntr_num] !=
+ CNTR_FREE) {
+ pmu_events->shadow_pmc_reg[WXYZ_SHADOW_IDX] |=
+ WXYZ_ENABLE_MASK;
+ pmu_events->unit.cntr_state[cntr_num] =
+ CNTR_IN_USE;
+ }
+ } else {
+ /* disable, only if the counter is actually in use */
+ if (pmu_events->unit.cntr_state[cntr_num]
+ == CNTR_IN_USE)
+ pmu_events->shadow_pmc_reg[WXYZ_SHADOW_IDX] &=
+ ~WXYZ_ENABLE_MASK;
+ }
+ /* record which PMC needs to be written */
+ record_hcall_request_idx(PBE_LINKS_TRACE,
+ hcall_write,
+ pmu_events->shadow_pmc_reg
+ [WXYZ_SHADOW_IDX]);
+ }
+}
+
+void wxyz_link_pmd_write(struct torrent_pmu_events *pmu_events,
+ struct hcall_data *hcall_read)
+{
+ /*
+ * The physical counters are 16 bits wide each.
+ * All four counters are packed into a single physical register:
+ * W counter [63:48]
+ * X counter [47:32]
+ * Y counter [31:16]
+ * Z counter [15:0]
+ */
+
+ /*
+ * The PMD can only be accessed via an HCall. The four counters are
+ * packed into a single register. It is not practical to read the
+ * register, reset one counter field, then write the counter back.
+ * Secondly, being able to write a non-zero value is only needed when
+ * collecting a profile. Profiling on non-CPU counters doesn't make
+ * sense since there is no reliable way to associate the number of
+ * events that have occurred back to a specific instruction.
+ * Therefore, we will not support writing a value. The enable/disable
+ * HCall will need to sample the counts before the counter is enabled
+ * and then generate a delta count when the counters are disabled
+ */
+
+ int i, index;
+ u8 virt_cntr;
+ u64 value, event_code;
+
+ for (i = 0; i < pmu_events->n_events; i++) {
+ event_code = pmu_events->event[i]->hw.config;
+ virt_cntr = TORRENT_VIRT_CTR_GET(event_code);
+
+ if (pmu_events->unit.cntr_state[virt_cntr] == CNTR_IN_USE) {
+ /*
+ * The counters are stopped. The HCall to
+ * read the counters has been done. The
+ * current count must be set to the previous
+ * count. The counters will be enabled.
+ * When the counters are stopped and read
+ * again, the actual count will be the
+ * new count minus the previous count.
+ */
+
+ /* Get the request entry that contains the PMD for
+ * this virtual counter.
+ */
+ index = hcall_read->virt_cntr_to_request_idx
+ [TORRENT_PBUS_WXYZ_ID][virt_cntr];
+ value = hcall_read->tor_pmu_reg[index].regValue;
+ value = value >> ((WXYZ_NUM_VIRT_CTRS - 1 - virt_cntr)
+ * WXYZ_CNTR_SIZE) & WXYZ_PMD_MASK;
+ local64_set(&pmu_events->event[i]->hw.prev_count,
+ value);
+ }
+ }
+}
+
+int wxyz_link_pmd_read(struct torrent_pmu_events *pmu_events,
+ struct hcall_data *hcall_read)
+{
+ /*
+ * This PMU is accessed via hypervisor calls. The HCall to read the
+ * counters has been done. The counter data is in hcall_read.
+ */
+ int i, index;
+ u8 virt_cntr;
+ u64 value, prev, delta;
+ struct perf_event *event;
+
+ /*
+ * The physical counters are 16 bits wide each.
+ * All four counters are packed into a single physical register:
+ * W counter [63:48]
+ * X counter [47:32]
+ * Y counter [31:16]
+ * Z counter [15:0]
+ */
+ for (i = 0; i < pmu_events->n_events; i++) {
+ event = pmu_events->event[i];
+ virt_cntr = TORRENT_VIRT_CTR_GET(event->hw.config);
+
+ if (pmu_events->unit.cntr_state[virt_cntr] == CNTR_IN_USE) {
+ /*
+ * Read the PMD register. Its count value is the
+ * current value minus the previous value from when
+ * the counter was started.
+ */
+ /* get current count, from the HCall */
+ index = hcall_read->virt_cntr_to_request_idx
+ [TORRENT_PBUS_WXYZ_ID][virt_cntr];
+
+ /*
+ * Get initial value, which was saved by the pmd write
+ * call into prev_count.
+ */
+ prev = local64_read(&event->hw.prev_count);
+ value = hcall_read->tor_pmu_reg[index].regValue;
+ value = value >> ((WXYZ_NUM_VIRT_CTRS - 1
+ - virt_cntr) * WXYZ_CNTR_SIZE)
+ & WXYZ_PMD_MASK;
+
+ /* calculate/save the count */
+ delta = ((value - prev) & 0xffffUL) << 16;
+ local64_add(delta, &event->count);
+ local64_set(&event->hw.prev_count, value);
+ }
+ }
+ return 0;
+}
+
+int wxyz_link_get_phys_pmd_reg(u64 event_code)
+{
+ /* All of the WXYZ link counters use this single physical PMD
+ * register. */
+ return PBE_LINKS_CTRS;
+}
diff --git a/arch/powerpc/platforms/p7ih/torrent_pmu.c b/arch/powerpc/platforms/p7ih/torrent_pmu.c
new file mode 100644
index 0000000..5005fd1
--- /dev/null
+++ b/arch/powerpc/platforms/p7ih/torrent_pmu.c
@@ -0,0 +1,1582 @@
+/*
+ * Performance event support for IBM Torrent interconnect chip
+ *
+ * Copyright 2010, 2011 Carl Love, Corey Ashford, IBM Corporation.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ */
+
+#include <linux/hrtimer.h>
+#include <linux/io.h>
+#include <linux/kernel.h>
+#include <linux/slab.h>
+#include <linux/list.h>
+#include <linux/vmalloc.h>
+#include <linux/of.h>
+#include <linux/perf_event.h>
+#include <linux/string.h>
+#include <asm/power_torrent_events.h>
+#include <asm/torrent_nest_pmu.h>
+#include <asm/powerbus_bus_util_pmu.h>
+#include <asm/powerbus_wxyz_pmu.h>
+#include <asm/powerbus_ll_pmu.h>
+#include <asm/powerbus_mcd_pmu.h>
+#include <asm/mmu_pmu.h>
+#include <asm/cau_pmu.h>
+#include <asm/cputable.h>
+#include <asm/reg.h>
+#include <asm/hvcall.h>
+
+/*
+ * There are a number of Torrent PMUs that count non-core-specific events.
+ * These are referred to as Torrent events.
+ */
+
+static u64 torrent_chip_id;
+static u64 hfi_unit_id;
+
+struct torrent_txns {
+ unsigned int group_flag;
+ int n_txn_start[TORRENT_NUM_PMU_TYPES];
+};
+
+static struct torrent_txns torrent_txn;
+struct torrent_pmu_events txn_all_events[TORRENT_NUM_PMU_TYPES];
+
+/*
+ * The variable torrent_pmu_cntrs is doubly-indexed array. The first index of
+ * the array is the PMU type. Then each instance of a given type is indexed
+ * by its unit number.
+ */
+static struct torrent_pmu_counters ***torrent_pmu_cntrs;
+
+static int num_torrent_chips = 1;
+static int pbus_pmu_reservation_cnt[TORRENT_NUM_PBUS_PMU_TYPES];
+
+/* Store the list of all events, being counted and those to be added */
+struct torrent_pmu_events all_torrent_events[TORRENT_NUM_PMU_TYPES];
+
+#define HFI_SHIFT_OCTANT 3
+#define HFI_MAX_OCTANT 7
+
+#define PLPAR_HCALL plpar_hcall_norets
+
+static DEFINE_SPINLOCK(torrent_lock);
+#define PAGE_4K (1ULL << 12)
+#define ALIGN_4K(x) ALIGN(x, PAGE_4K)
+
+static u64 unaligned_buffer[2 * PAGE_4K / sizeof(u64)];
+static u64 *aligned_buffer;
+
+/*
+ * The Torrent counter polling interval needs to be set so that there's a good
+ * margin between counting to 2^32 at the speed of the fastest possible event
+ * rate. Right now, the fastest rate is the WXYZ link idle which counts at
+ * 3 GHz. At 3 GHz, it will take about 1.4 seconds to count to 2^32. Note,
+ * the MCD counters can be setup as 16 bit, 32 or 64 bit counters. It is not
+ * clear what the maximum MCD event increment rate might be. Since the user
+ * can control the size of these counters, for now we will leave it to the user
+ * to select the correct size based on the event. To be conservative, the
+ * polling period is set to about 1/3 of the maximum counter-wrap period.
+ */
+#define POLLING_INTERVAL_SEC 0
+#define POLLING_INTERVAL_NS 500000000
+
+static ktime_t torrent_counter_poll_interval;
+struct hrtimer torrent_poll_timer;
+u64 poll_start_time;
+
+static void initialize_event_struct(struct torrent_pmu_events *event_struct)
+{
+ int pmu_type, i;
+
+ event_struct[TORRENT_PBUS_WXYZ_ID].max_events =
+ MAX_CNTRS_PER_WXYZ_LINK_PMU;
+ event_struct[TORRENT_PBUS_LL_ID].max_events =
+ MAX_CNTRS_PER_LL_LINK_PMU;
+ event_struct[TORRENT_PBUS_MCD_ID].max_events =
+ MAX_CNTRS_PER_MCD_PMU;
+ event_struct[TORRENT_PBUS_UTIL_ID].max_events =
+ MAX_CNTRS_PER_BUS_UTIL_PMU;
+ event_struct[TORRENT_MMU_ID].max_events = MAX_CNTRS_PER_MMU_PMU;
+ event_struct[TORRENT_CAU_ID].max_events = MAX_CNTRS_PER_CAU_PMU;
+
+ for (pmu_type = 0; pmu_type < TORRENT_NUM_PMU_TYPES; pmu_type++) {
+ event_struct[pmu_type].n_events = 0;
+ event_struct[pmu_type].n_add = 0;
+ event_struct[pmu_type].update = false;
+ event_struct[pmu_type].disabled = 0;
+ event_struct[pmu_type].unit.mcd_cntr_attr = 0;
+ event_struct[pmu_type].unit.bus_util_enable_mask = 0;
+ event_struct[pmu_type].unit.bus_util_cntr_sel = 0;
+
+ for (i = 0; i < MAX_CNTRS_TORRENT_PMUS; i++)
+ event_struct[pmu_type].unit.cntr_state[i] = CNTR_FREE;
+
+ /* Initialize the PMC shadow regs to all zero.
+ * *NOTE* this assumes that the correct initial value for
+ * every PMC reg is zero. If this turns out not to be the
+ * case for some PMU(s), we may want to pass in a pointer to a
+ * function which initializes the shadow regs correctly.
+ */
+ for (i = 0; i < MAX_TORRENT_PMC_REGS; i++)
+ event_struct[pmu_type].shadow_pmc_reg[i] = 0x0;
+
+ if (pmu_type == TORRENT_PBUS_MCD_ID)
+ /* MCD specific set up */
+ event_struct[pmu_type].unit.mcd_cntr_attr = 0;
+
+ if (pmu_type == TORRENT_PBUS_UTIL_ID) {
+ /* Bus util specific set up */
+ event_struct[pmu_type].unit.bus_util_enable_mask = 0;
+ event_struct[pmu_type].unit.bus_util_cntr_sel
+ = BUS_UTIL_CNTR_SEL_AVAIL;
+ }
+ }
+}
+
+static int wrap_hcall_hfi_dump_info(u64 torrent_chip_id,
+ struct dump_mmio_perf_counters *mmio_perf_counters)
+{
+ struct dump_mmio_perf_counters *aligned_mmio_perf_counters =
+ (struct dump_mmio_perf_counters *)aligned_buffer;
+ int ret = 0;
+
+ ret = PLPAR_HCALL(H_HFI_DUMP_INFO, hfi_unit_id,
+ DUMP_MMIO_PERFORMANCE_COUNTERS,
+ sizeof(struct dump_mmio_perf_counters),
+ __pa(aligned_buffer));
+
+ if (ret == H_SUCCESS)
+ /* Copy the data out of the aligned buffer into the caller's
+ * memory.
+ */
+ *mmio_perf_counters = *aligned_mmio_perf_counters;
+
+ return ret;
+}
+
+static int wrap_hcall_tor_access_pmu_scom_regs(u64 torrent_chip_id,
+ int req_type,
+ int num_regs,
+ struct tor_pmu_reg *tor_pmu_regs)
+{
+ struct tor_pmu_reg *aligned_tor_pmu_reg =
+ (struct tor_pmu_reg *)aligned_buffer;
+ int i, ret;
+
+ for (i = 0; i < num_regs; i++) {
+ aligned_tor_pmu_reg[i].regId = tor_pmu_regs[i].regId;
+ if (req_type == PBUS_HCALL_WRITE)
+ aligned_tor_pmu_reg[i].regValue =
+ tor_pmu_regs[i].regValue;
+ }
+ ret = PLPAR_HCALL(H_TOR_ACCESS_PMU_SCOM_REGS, torrent_chip_id,
+ (u64)req_type, (u64)num_regs,
+ __pa(aligned_buffer));
+
+ if (req_type == PBUS_HCALL_READ)
+ for (i = 0; i < num_regs; i++)
+ tor_pmu_regs[i].regValue =
+ aligned_tor_pmu_reg[i].regValue;
+ return ret;
+}
+
+/* Generic Torrent PMU functions
+ *
+ * These routines make calls via function pointers to the PMU-specific
+ * functions to do the PMU-specific PMU register accesses. The number and
+ * layout of the control registers, the PMD registers and the event constraints
+ * are very PMU-specific. Hence the use of function pointers to deal with the
+ * specifics of a given PMU.
+ */
+int get_chip(void)
+{
+ return 0; /* Currently only supporting a single Torrent chip system */
+}
+
+int get_max_nest_events(int pmu_type)
+{
+ /* The maximum number of PMU events is independent of the chip number.
+ * So, just get it for chip 0.
+ */
+ return all_torrent_events[pmu_type].max_events;
+}
+
+static void hcall_data_reset(struct hcall_data *hcall_data)
+{
+ int i, pmu_type;
+
+ hcall_data->num_regs = 0;
+ for (i = 0; i < MAX_HCALL_REGS; i++)
+ hcall_data->request_idx[i] = -1;
+
+ for (pmu_type = 0; pmu_type < TORRENT_NUM_PMU_TYPES; pmu_type++) {
+ for (i = 0; i < MAX_CNTRS_TORRENT_PMUS; i++)
+ hcall_data->virt_cntr_to_request_idx[pmu_type][i] =
+ 0xbadbeef;
+ }
+ hcall_data->do_dump_mmio_perf_counters = 0;
+}
+
+static int hcall_read_setup(struct torrent_pmu_events *torrent_pmu_events,
+ int pmu_type,
+ struct hcall_data *hcall_read)
+{
+ struct torrent_pmu_counters *torrent_pmu;
+ int num_events, i, phys_reg = 0, *num_regs_p, request_idx;
+ u8 virt_cntr;
+ u64 event_code;
+
+ torrent_pmu = torrent_pmu_cntrs[get_chip()][pmu_type];
+ num_events = torrent_pmu_events->n_events;
+
+ if (num_events == 0)
+ return 0; /* no counters to read on this PMU. */
+
+ switch (pmu_type) {
+ case TORRENT_MMU_ID:
+ case TORRENT_CAU_ID:
+ hcall_read->do_dump_mmio_perf_counters = 1;
+ return 0;
+ default:
+ /* it must be one of the Powerbus PMU's */
+ break;
+ }
+ num_regs_p = &hcall_read->num_regs;
+
+ for (i = 0; i < num_events; i++) {
+ event_code = torrent_pmu_events->event[i]->hw.config;
+ phys_reg = torrent_pmu->get_phys_pmd_reg(event_code);
+ virt_cntr = (u8)torrent_pmu_events->event[i]->hw.idx;
+
+ /* Check if the physical reg is already in the list
+ * to be read.
+ */
+ if (hcall_read->request_idx[phys_reg] == -1) {
+ request_idx = *num_regs_p;
+ hcall_read->request_idx[phys_reg] = *num_regs_p;
+ hcall_read->tor_pmu_reg[*num_regs_p].regId
+ = phys_reg;
+ (*num_regs_p)++;
+
+ if (*num_regs_p == MAX_HCALL_REGS) {
+ pr_err("%s, ERROR, "
+ "MAX_HCALL_REGS is "
+ "too small.\n", __func__);
+ return 1;
+ }
+ } else {
+ request_idx = hcall_read->request_idx[phys_reg];
+ }
+ hcall_read->virt_cntr_to_request_idx[pmu_type][virt_cntr]
+ = request_idx;
+ }
+ return 0;
+}
+
+void do_hcall_pmd_read(struct hcall_data *hcall_read,
+ struct torrent_pmu_events *torrent_pmu_events)
+{
+ int pmu_type, ret;
+
+ /*
+ * Set up variable hcall_read with all the physical registers to read.
+ * Then bundle up the registers to be read into as few HCalls as
+ * possible to minimize the HCall overhead.
+ */
+
+ hcall_data_reset(hcall_read);
+ for (pmu_type = 0; pmu_type < TORRENT_NUM_PMU_TYPES; pmu_type++)
+ hcall_read_setup(&torrent_pmu_events[pmu_type], pmu_type,
+ hcall_read);
+
+ if (hcall_read->num_regs != 0) {
+ ret = wrap_hcall_tor_access_pmu_scom_regs(torrent_chip_id,
+ PBUS_HCALL_READ, hcall_read->num_regs,
+ hcall_read->tor_pmu_reg);
+ if (ret != H_SUCCESS)
+ pr_err("%s, ERROR, HCall H_TOR_ACCESS_PMU_SCOM_REGS:PBUS_HCALL_READ returned an error %d\n", __func__,
+ ret);
+ }
+ if (hcall_read->do_dump_mmio_perf_counters) {
+ ret = wrap_hcall_hfi_dump_info(torrent_chip_id,
+ &hcall_read->mmio_perf_counters);
+ if (ret != H_SUCCESS)
+ pr_err("%s, ERROR, HCall H_HFI_DUMP_INFO:DUMP_MMIO_PERFORMANCE_COUNTERS HCall returned an error %d\n", __func__,
+ ret);
+ }
+}
+
+void do_hcall_pmc_write(struct hcall_data *hcall_write)
+{
+ int ret;
+
+ if (hcall_write->num_regs != 0) {
+ ret = wrap_hcall_tor_access_pmu_scom_regs(torrent_chip_id,
+ PBUS_HCALL_WRITE, hcall_write->num_regs,
+ hcall_write->tor_pmu_reg);
+
+ if (ret != H_SUCCESS)
+ pr_err("%s, ERROR, HCall H_TOR_ACCESS_PMU_SCOM_REGS:PBUS_HCALL_WRITE returned an error %d\n", __func__,
+ ret);
+ }
+}
+
+void record_hcall_request_idx(int phys_register_index,
+ struct hcall_data *hcall_request,
+ u64 shadow_reg_index)
+{
+ int request_idx;
+
+ /* Configuration register needs to be written */
+ request_idx = hcall_request->num_regs;
+
+ if (hcall_request->request_idx[phys_register_index] == -1) {
+ hcall_request->request_idx[phys_register_index]
+ = request_idx;
+ hcall_request->tor_pmu_reg[request_idx].regId
+ = phys_register_index;
+ hcall_request->num_regs++;
+ } else {
+ /* store the updated value to write */
+ request_idx = hcall_request->
+ request_idx[phys_register_index];
+ }
+
+ hcall_request->tor_pmu_reg[request_idx].regValue = shadow_reg_index;
+}
+
+static void copy_unit_config(struct unit_config *unit_cfg_src,
+ struct unit_config *unit_cfg_dest, int pmu_type)
+{
+ int i;
+
+ for (i = 0; i < MAX_CNTRS_TORRENT_PMUS; i++)
+ unit_cfg_dest->cntr_state[i] = unit_cfg_src->cntr_state[i];
+
+ if (pmu_type == TORRENT_PBUS_MCD_ID)
+ unit_cfg_dest->mcd_cntr_attr = unit_cfg_src->mcd_cntr_attr;
+
+ if (pmu_type == TORRENT_PBUS_UTIL_ID) {
+ unit_cfg_dest->bus_util_enable_mask =
+ unit_cfg_src->bus_util_enable_mask;
+ unit_cfg_dest->bus_util_cntr_sel =
+ unit_cfg_src->bus_util_cntr_sel;
+ }
+}
+
+static void copy_all_events(struct torrent_pmu_events *src_events,
+ struct torrent_pmu_events *dst_events)
+{
+ int pmu_type, i;
+
+ for (pmu_type = 0; pmu_type < TORRENT_NUM_PMU_TYPES; pmu_type++) {
+ dst_events[pmu_type].max_events =
+ src_events[pmu_type].max_events;
+ dst_events[pmu_type].n_events = src_events[pmu_type].n_events;
+ dst_events[pmu_type].n_add = src_events[pmu_type].n_add;
+
+ copy_unit_config(&src_events[pmu_type].unit,
+ &dst_events[pmu_type].unit, pmu_type);
+ for (i = 0; i < MAX_CNTRS_TORRENT_PMUS; i++)
+ dst_events[pmu_type].event[i] =
+ src_events[pmu_type].event[i];
+ }
+}
+
+int get_max_torrent_counters(int pmu_type)
+{
+ /* The maximum number of PMU events is independent of the chip number.
+ * So, just get it for chip 0.
+ */
+ return all_torrent_events[pmu_type].max_events;
+}
+
+static void collect_events(struct perf_event *group,
+ struct torrent_pmu_events *tmp_pmus)
+{
+ int pmu_type;
+ struct torrent_pmu_events *torrent_pmu_events;
+ struct perf_event *event;
+
+ /*
+ * Get the event code from attr.config value as the hw.config
+ * field is not set yet since the event has not been added.
+ */
+ pmu_type = TORRENT_PMU_GET(group->attr.config);
+
+ if (pmu_type < 0)
+ pr_err("%s: ERROR, collect_events: could not determine pmu_type or unit number.\n",
+ __func__);
+
+ torrent_pmu_events = &tmp_pmus[pmu_type];
+
+ BUG_ON(torrent_pmu_events == NULL);
+
+ list_for_each_entry(event, &group->sibling_list, group_entry) {
+ if (event->state != PERF_EVENT_STATE_OFF) {
+ int pmu_type, index;
+ struct torrent_pmu_events *torrent_pmu_events;
+
+ pmu_type = TORRENT_PMU_GET(event->attr.config);
+
+ torrent_pmu_events = &tmp_pmus[pmu_type];
+ /*
+ * Index is what is on the PMU plus pending
+ * events to add.
+ */
+ index = torrent_pmu_events->n_events +
+ torrent_pmu_events->n_add;
+
+ if (index >= get_max_nest_events(pmu_type))
+ return;
+
+ torrent_pmu_events->event[index] = event;
+ /*
+ * Initialize perf event as not yet
+ * being assigned to a physical counter.
+ * Physical counters are numbered from 0
+ * to n-1.
+ */
+ torrent_pmu_events->event[index]->hw.idx = -1;
+ torrent_pmu_events->n_add++;
+ }
+ }
+}
+
+void accept_torrent_events(struct torrent_pmu_events *new_all_events,
+ struct torrent_pmu_events *existing_all_events)
+{
+ int pmu_type, i, index, debug_total_events = 0;
+ struct torrent_pmu_events *new_events, *existing_events;
+
+ for (pmu_type = 0; pmu_type < TORRENT_NUM_PMU_TYPES;
+ pmu_type++) {
+
+ new_events = &new_all_events[pmu_type];
+ existing_events = &existing_all_events[pmu_type];
+
+ if (new_events->n_add) {
+ /*
+ * Move the events from the list of new events to the
+ * list of events being measured by the PMU.
+ */
+ for (i = 0; i < new_events->n_add; i++) {
+ index = i + existing_events->n_events;
+ existing_events->event[index] =
+ new_events->event[index];
+ }
+ existing_events->n_events += new_events->n_add;
+ debug_total_events += existing_events->n_events;
+ copy_unit_config(&new_events->unit,
+ &existing_events->unit, pmu_type);
+ existing_events->update = 1;
+ }
+ }
+}
+
+int hw_perf_group_sched_in_torrent_check(struct torrent_pmu_events
+ *new_pmu_events)
+{
+ struct torrent_pmu_counters *torrent_pmu;
+ struct torrent_pmu_events *torrent_pmu_events;
+ int pmu_type, n, n0, i, max_events;
+
+ for (pmu_type = 0; pmu_type < TORRENT_NUM_PMU_TYPES; pmu_type++) {
+ /*
+ * torrent_pmu is the structure that tracks what the physical
+ * PMU usage
+ */
+ torrent_pmu_events = &new_pmu_events[pmu_type];
+
+ /*
+ * We are just testing to see if the group is
+ * self-consistent, so use a PMU config which has no
+ * registers already reserved, nor other configuration
+ * bits preset.
+ */
+
+ n = torrent_pmu_events->n_add;
+ n0 = torrent_pmu_events->n_events; /* being counted */
+
+ if (n == 0)
+ continue;
+
+ /* Get the number of physical counters for the PMU */
+ max_events = torrent_pmu_events->max_events;
+
+ if (n + n0 > max_events) {
+ /*
+ * Current plus new events exceeds physical
+ * number of counters
+ */
+ pr_err("%s %d, CPU %d, !!! ERROR n %d exceeded max %d\n",
+ __func__, __LINE__, smp_processor_id(), n+n0,
+ max_events);
+ return -EAGAIN;
+ }
+
+ /*
+ * See if we can put the new events on with the
+ * existing events. We need to use the state of the real
+ * unit config structure so we can see what is being
+ * used. If we fail, we need to make sure we throw
+ * away any changes from the check constraints call.
+ */
+
+ torrent_pmu = torrent_pmu_cntrs[get_chip()][pmu_type];
+ i = torrent_pmu->check_constraints(torrent_pmu_events,
+ &torrent_pmu_events->unit);
+ if (i < 0)
+ return -EAGAIN;
+ }
+ return 0;
+}
+
+void torrent_pmu_enable(struct pmu *pmu)
+{
+ /*
+ * This function adds the new events to all of the PMUs. The n_add
+ * value for each PMU will specify how many new events are being
+ * added. This routine will add n_add to n__events then set n_add
+ * to zero to indicate the event has been added.
+ */
+ int pmu_type, ret, dump_ret;
+ unsigned long lock_flags;
+ struct torrent_pmu_counters *torrent_pmu;
+ struct torrent_pmu_events *torrent_pmu_events;
+ struct hcall_data hcall_read;
+ struct hcall_data hcall_write;
+ bool start_timer = false;
+
+ /*
+ * Need to do a single HCall to stop all the PMUs that need to have
+ * events programmed on them, i.e. n_add for the PMU is not zero.
+ */
+ spin_lock_irqsave(&torrent_lock, lock_flags); /* ensure only one CPU
+ * at a time access the
+ * HCall data structures
+ */
+ hcall_data_reset(&hcall_write);
+
+ /* Disable the PMUs with new events to add */
+ for (pmu_type = 0; pmu_type < TORRENT_NUM_PMU_TYPES; pmu_type++) {
+ torrent_pmu = torrent_pmu_cntrs[get_chip()][pmu_type];
+ torrent_pmu_events = &all_torrent_events[pmu_type];
+
+ /*
+ * We only need to stop PMUs where there are new
+ * events added to the PMU.
+ */
+ if (torrent_pmu_events->update)
+ torrent_pmu->enable_disable_cntr(0,
+ torrent_pmu_events,
+ &hcall_write);
+ }
+
+ if (hcall_write.num_regs != 0) {
+ do_hcall_pmc_write(&hcall_write);
+ hcall_data_reset(&hcall_write);
+ }
+
+ if (hcall_write.do_dump_mmio_perf_counters)
+ dump_ret = wrap_hcall_hfi_dump_info(torrent_chip_id,
+ &hcall_write.mmio_perf_counters);
+
+ for (pmu_type = 0; pmu_type < TORRENT_NUM_PMU_TYPES;
+ pmu_type++) {
+ torrent_pmu = torrent_pmu_cntrs[get_chip()][pmu_type];
+ torrent_pmu_events = &all_torrent_events[pmu_type];
+
+ if (torrent_pmu_events->update) {
+ torrent_pmu_events->update = 0;
+
+ if (pmu_type <= TORRENT_LAST_PBUS_PMU_ID) {
+ /* Reserve the Pbus PMU */
+
+ pbus_pmu_reservation_cnt[pmu_type] =
+ torrent_pmu_events->n_events;
+ }
+
+ /*
+ * Set up the physical PMCs with the new
+ * events and set the PMU enable bit,
+ * assign events to counters
+ */
+ ret = torrent_pmu->compute_pmc_regs(
+ torrent_pmu_events, &hcall_write);
+
+ if (ret) {
+ pr_debug("%s:%d, compute_pmc_regs returned error = %d\n",
+ __func__, __LINE__, ret);
+ return;
+ }
+
+ torrent_pmu->enable_disable_cntr(1, torrent_pmu_events,
+ &hcall_write);
+ torrent_pmu_events->disabled = 0;
+ start_timer = true;
+ }
+ }
+
+ /* Read the initial value of all of the PMDs that will be enabled */
+ do_hcall_pmd_read(&hcall_read, all_torrent_events);
+
+ /*
+ * Now read the virtual counter PMD values from the physical registers
+ * returned in variable hcall_read. Set up the new PMC values.
+ */
+ for (pmu_type = 0; pmu_type < TORRENT_NUM_PMU_TYPES;
+ pmu_type++) {
+ torrent_pmu = torrent_pmu_cntrs[get_chip()][pmu_type];
+ torrent_pmu_events = &all_torrent_events[pmu_type];
+ /*
+ * Call pmd_write to update the prev_count values
+ * with the current values.
+ */
+ torrent_pmu->write_pmd(torrent_pmu_events, &hcall_read);
+ }
+
+ /* Enable the PMUs with new events to add */
+ do_hcall_pmc_write(&hcall_write);
+
+ if (start_timer && !hrtimer_active(&torrent_poll_timer)) {
+ int ret;
+
+ poll_start_time = get_tb();
+ ret = hrtimer_start(&torrent_poll_timer,
+ torrent_counter_poll_interval,
+ HRTIMER_MODE_REL);
+ WARN_ON(ret != 0);
+ }
+
+ spin_unlock_irqrestore(&torrent_lock, lock_flags);
+}
+
+
+static void release_torrent_pmu_counters(struct perf_event *event,
+ struct torrent_pmu_events
+ *torrent_pmu_events,
+ int pmu_type)
+{
+ int hw_cntr;
+
+ /*
+ * The HCall has been done to stop the counters. The counters have
+ * been read. This function removes the counters from the PMU.
+ */
+
+ hw_cntr = event->hw.idx;
+ torrent_pmu_events->unit.cntr_state[hw_cntr] = CNTR_FREE;
+
+ if (pmu_type <= TORRENT_LAST_PBUS_PMU_ID)
+ /* Release the PBUS PMU */
+ pbus_pmu_reservation_cnt[pmu_type]--;
+
+ event->hw.idx = -1;
+}
+
+
+void torrent_pmu_disable(struct pmu *pmu)
+{
+ struct torrent_pmu_events *torrent_pmu_events;
+ struct torrent_pmu_counters *torrent_pmu;
+ struct hcall_data hcall_write;
+ unsigned long lock_flags;
+ int pmu_type;
+
+ /* Disable and read all of the counters on all of the PMUs */
+
+ /* Set up the HCall to write the PMC's to stop the counters. */
+ spin_lock_irqsave(&torrent_lock, lock_flags); /* Ensure only one CPU
+ * at a time access the
+ * HCall data structures
+ */
+ /*
+ * The first time that torrent_pmu_disable is called is when a
+ * measurement is started, the poll_timer won't be running yet, but
+ * all other times, it will be. So hrtimer_cancel will return 0 on the
+ * first call, and 1 subsequently.
+ */
+ hrtimer_cancel(&torrent_poll_timer);
+
+ hcall_data_reset(&hcall_write);
+
+ for (pmu_type = 0; pmu_type < TORRENT_NUM_PMU_TYPES;
+ pmu_type++) {
+ torrent_pmu = torrent_pmu_cntrs[get_chip()][pmu_type];
+
+ torrent_pmu_events = &all_torrent_events[pmu_type];
+ torrent_pmu->enable_disable_cntr(0, torrent_pmu_events,
+ &hcall_write);
+ torrent_pmu_events->disabled = 1;
+ }
+
+ /* Do the HCall to write the physical registers. */
+ do_hcall_pmc_write(&hcall_write);
+ spin_unlock_irqrestore(&torrent_lock, lock_flags);
+}
+
+struct torrent_pmu_counters *map_to_pmu(u64 event_code)
+{
+ /* Extract the PMU type from the event and return the pointer to the
+ * torrent_pmu_counters struct for that PMU type.
+ */
+ int pmu_type;
+ int chip = get_chip();
+
+ BUG_ON(!IS_TORRENT_EVENT(event_code));
+ pmu_type = TORRENT_PMU_GET(event_code);
+
+ BUG_ON(pmu_type > TORRENT_LAST_PMU_ID);
+ return torrent_pmu_cntrs[chip][pmu_type];
+}
+
+int torrent_pmu_add(struct perf_event *event, int ef_flags)
+{
+ /*
+ * This function adds the event to the list of events for
+ * the PMU that counts the specified event.
+ *
+ * This function is called directly from the arch-independent code.
+ */
+ int pmu_type, index, err, ret = -EAGAIN;
+ unsigned long lock_flags;
+ struct torrent_pmu_events tmp_all_events[TORRENT_NUM_PMU_TYPES];
+ struct torrent_pmu_events *torrent_pmu_events;
+
+ initialize_event_struct(tmp_all_events);
+
+ spin_lock_irqsave(&torrent_lock, lock_flags); /* Ensure only one CPU
+ * at a time access the
+ * data structures.
+ */
+
+ copy_all_events(all_torrent_events, tmp_all_events);
+
+ if (event->group_leader != event) {
+ collect_events(event->group_leader, tmp_all_events);
+ } else {
+
+ /*
+ * Add the new group leader event to the list of existing
+ * nest events.
+ */
+ pmu_type = TORRENT_PMU_GET(event->attr.config);
+
+ if (pmu_type < 0) {
+ pr_err("%s: ERROR, pmu_type < 0\n",
+ __func__);
+ ret = -EINVAL;
+ goto out;
+ }
+
+ torrent_pmu_events = &tmp_all_events[pmu_type];
+
+ index = torrent_pmu_events->n_events
+ + torrent_pmu_events->n_add;
+
+ if (index > get_max_nest_events(pmu_type)) {
+ pr_err("%s: ERROR, index out of range\n",
+ __func__);
+ ret = -EINVAL;
+ goto out;
+ }
+
+ torrent_pmu_events->event[index] = event;
+ torrent_pmu_events->n_add++;
+ }
+
+ if (torrent_txn.group_flag & PERF_EVENT_TXN) {
+ /*
+ * If group events scheduling transaction was started,
+ * skip the schedulability test here, it will be performed
+ * at commit time(->commit_txn) as a whole. Save the event
+ * added to the list of transaction events being added.
+ */
+ copy_all_events(tmp_all_events, txn_all_events);
+
+ } else {
+ /* check constraints for all pmus */
+ err = hw_perf_group_sched_in_torrent_check(tmp_all_events);
+
+ if (err) {
+ pr_err("%s, ERROR, Torrent sched in check failed\n",
+ __func__);
+ ret = -EINVAL;
+ goto out;
+ }
+
+ /*
+ * Can now accept the events from the tmp list into the list
+ * of current events for the Torrent PMU an add to list for
+ * each physical PMU.
+ */
+ accept_torrent_events(tmp_all_events, all_torrent_events);
+ }
+
+ /* Setup the group leader period info */
+ event->hw.config = event->attr.config;
+ event->hw.config_base = event->attr.config;
+ event->hw.event_base = 0; /* nest doesn't use the flags */
+ event->hw.last_period = event->hw.sample_period;
+ local64_set(&event->hw.period_left, event->hw.last_period);
+ ret = 0;
+
+out:
+ spin_unlock_irqrestore(&torrent_lock, lock_flags);
+ return ret;
+}
+
+void torrent_pmu_del(struct perf_event *event, int ef_flags)
+{
+ /*
+ * This function removes a single counter from a specific PMU.
+ * The function is called from the arch-independent perf code.
+ */
+
+ struct torrent_pmu_events *torrent_pmu_events;
+ struct torrent_pmu_counters *torrent_pmu;
+ struct hcall_data hcall_read;
+ struct hcall_data hcall_write;
+ struct unit_config *unit_config;
+ unsigned long lock_flags;
+ int i, pmu_type, events_released = 0, ret;
+ u64 event_code = event->hw.config;
+
+ /* Remove a single counter */
+ pmu_type = TORRENT_PMU_GET(event_code);
+
+ torrent_pmu = map_to_pmu(event_code);
+ if (!torrent_pmu)
+ return;
+
+ spin_lock_irqsave(&torrent_lock, lock_flags); /* Ensure only one CPU
+ * at a time access the
+ * HCall data structures
+ */
+ torrent_pmu_events = &all_torrent_events[pmu_type];
+
+ /* Set up variable hcall_write for the enable/disable call */
+ hcall_data_reset(&hcall_write);
+ torrent_pmu->enable_disable_cntr(0, torrent_pmu_events, &hcall_write);
+ do_hcall_pmc_write(&hcall_write);
+
+ /* Set up to do the PMD read call */
+ do_hcall_pmd_read(&hcall_read, all_torrent_events);
+
+ /* read the virtual counters */
+ torrent_pmu->read_pmd(torrent_pmu_events, &hcall_read);
+
+ /* Update the number of events being counted by the PMU */
+ unit_config = &torrent_pmu_events->unit;
+
+ /* Remove the event from the PMU list of events to count */
+ if (event->hw.idx >= 0)
+ /*
+ * Only need to release counter if event was assigned to a
+ * counter. If the event failed the constraint test, it was
+ * not assigned to a counter.
+ */
+ unit_config->cntr_state[event->hw.idx] = CNTR_FREE;
+
+ for (i = 0; i < torrent_pmu_events->n_events; i++) {
+ if (event == torrent_pmu_events->event[i]) {
+ release_torrent_pmu_counters(event, torrent_pmu_events,
+ pmu_type);
+
+ while (++i < torrent_pmu_events->n_events)
+ torrent_pmu_events->event[i - 1] =
+ torrent_pmu_events->event[i];
+
+ events_released++;
+ perf_event_update_userpage(event);
+ }
+ }
+
+ torrent_pmu_events->n_events -= events_released;
+ BUG_ON(torrent_pmu_events->n_events < 0);
+
+ if (pmu_type <= TORRENT_LAST_PBUS_PMU_ID)
+ /* Release the PBUS PMU */
+ pbus_pmu_reservation_cnt[pmu_type] =
+ torrent_pmu_events->n_events;
+
+ /* release PMU specific configuration values */
+ if (pmu_type == TORRENT_PBUS_MCD_ID)
+ unit_config->mcd_cntr_attr = 0x3;
+
+ if (pmu_type == TORRENT_PBUS_UTIL_ID) {
+ unit_config->bus_util_enable_mask = 0;
+ unit_config->bus_util_cntr_sel = BUS_UTIL_CNTR_SEL_AVAIL;
+ }
+
+ /* Enable the hardware counters on the specific PMU for the new list
+ * of events
+ */
+ ret = torrent_pmu->compute_pmc_regs(torrent_pmu_events, &hcall_write);
+ if (ret) {
+ pr_debug("%s:%d, compute_pmc_regs returned error = %d\n",
+ __func__, __LINE__, ret);
+ spin_unlock_irqrestore(&torrent_lock, lock_flags);
+ return;
+ }
+
+ torrent_pmu->enable_disable_cntr(1, torrent_pmu_events, &hcall_write);
+
+ do_hcall_pmc_write(&hcall_write);
+ spin_unlock_irqrestore(&torrent_lock, lock_flags);
+}
+
+static void poll_all_torrent_pmus(void)
+{
+ /*
+ * The counters need to be read and reset periodically to ensure the
+ * counters do not overflow. Most of the counters do not support
+ * interrupts so in general it is not possible to only update when the
+ * counters overflow. This routine will stop all the counters, read
+ * the physical PMD registers, update the virtual count for each event,
+ * reset the counter and then restart the counters.
+ */
+
+ int pmu_type, dump_ret;
+ unsigned long lock_flags;
+ struct torrent_pmu_counters *torrent_pmu;
+ struct torrent_pmu_events *torrent_pmu_events;
+ struct hcall_data hcall_read;
+ struct hcall_data hcall_write;
+
+ spin_lock_irqsave(&torrent_lock, lock_flags); /* Ensure only one CPU
+ * at a time access the
+ * HCall data structures
+ */
+ hcall_data_reset(&hcall_write);
+
+ /* Disable all of the active PMUs */
+ for (pmu_type = 0; pmu_type < TORRENT_NUM_PMU_TYPES; pmu_type++) {
+ torrent_pmu = torrent_pmu_cntrs[get_chip()][pmu_type];
+ torrent_pmu_events = &all_torrent_events[pmu_type];
+
+ if (torrent_pmu_events->n_events)
+ /* Only disable the PMUs with events being counted */
+ torrent_pmu->enable_disable_cntr(0, torrent_pmu_events,
+ &hcall_write);
+ }
+
+ if (hcall_write.num_regs != 0) {
+ do_hcall_pmc_write(&hcall_write);
+ hcall_data_reset(&hcall_write);
+ }
+
+ if (hcall_write.do_dump_mmio_perf_counters)
+ dump_ret = wrap_hcall_hfi_dump_info(torrent_chip_id,
+ &hcall_write.mmio_perf_counters);
+
+ /* Setup the HCall to enable all of the PMUs */
+ for (pmu_type = 0; pmu_type < TORRENT_NUM_PMU_TYPES; pmu_type++) {
+ torrent_pmu = torrent_pmu_cntrs[get_chip()][pmu_type];
+ torrent_pmu_events = &all_torrent_events[pmu_type];
+
+ if (torrent_pmu_events->n_events)
+ /* Only enablee the PMUs with events being counted */
+ torrent_pmu->enable_disable_cntr(1, torrent_pmu_events,
+ &hcall_write);
+ }
+
+ if (hcall_write.num_regs != 0) {
+ do_hcall_pmc_write(&hcall_write);
+ hcall_data_reset(&hcall_write);
+ }
+
+ if (hcall_write.do_dump_mmio_perf_counters)
+ dump_ret = wrap_hcall_hfi_dump_info(torrent_chip_id,
+ &hcall_write.mmio_perf_counters);
+
+ /* Read the initial value of all of the PMDs that will be enabled */
+ do_hcall_pmd_read(&hcall_read, all_torrent_events);
+
+ /* Now read the virtual counter PMD values from the physical registers
+ * returned in hcall_read. Set up the new PMC values.
+ */
+ for (pmu_type = 0; pmu_type < TORRENT_NUM_PMU_TYPES;
+ pmu_type++) {
+ torrent_pmu = torrent_pmu_cntrs[get_chip()][pmu_type];
+ torrent_pmu_events = &all_torrent_events[pmu_type];
+ /*
+ * Call pmd_write to update the prev_count values
+ * with the current values.
+ */
+ torrent_pmu->write_pmd(torrent_pmu_events, &hcall_read);
+ }
+
+ /* Enable the PMUs */
+ do_hcall_pmc_write(&hcall_write);
+ spin_unlock_irqrestore(&torrent_lock, lock_flags);
+}
+
+static enum hrtimer_restart poll_torrent_pmus(struct hrtimer *timer)
+{
+ poll_all_torrent_pmus();
+
+ poll_start_time = get_tb();
+ hrtimer_forward_now(timer, torrent_counter_poll_interval);
+ return HRTIMER_RESTART;
+}
+
+/*
+ * These typedefs are present mostly to clean up the declaration of
+ * torrent_pmu_initialize, keeping the lines from going beyond column 80.
+ */
+
+typedef int (*compute_pmc_regs_fptr)(struct torrent_pmu_events *pmu_events,
+ struct hcall_data *hcall_read);
+
+typedef int (*check_constraints_fptr)(struct torrent_pmu_events *pmu_events,
+ struct unit_config *unit_cfg);
+
+typedef void (*enable_disable_cntr_fptr)(int op,
+ struct torrent_pmu_events *pmu_events,
+ struct hcall_data *hcall_write);
+
+typedef int (*pmd_read_fptr)(struct torrent_pmu_events *torrent_pmu_events,
+ struct hcall_data *hcall_read);
+
+typedef void (*pmd_write_fptr)(struct torrent_pmu_events *torrent_pmu_events,
+ struct hcall_data *hcall_read);
+
+typedef int (*get_phys_pmd_reg_fptr)(u64 event_code);
+
+static int torrent_pmu_initialize(int chip, int pmu_type,
+ compute_pmc_regs_fptr compute_pmc_regs,
+ check_constraints_fptr check_constraints,
+ enable_disable_cntr_fptr enable_disable_cntr,
+ pmd_read_fptr pmd_read,
+ pmd_write_fptr pmd_write,
+ get_phys_pmd_reg_fptr get_phys_pmd_reg)
+{
+ struct torrent_pmu_counters *torrent_pmu;
+
+ /*
+ * This function allocates and initializes the data structure for the
+ * Torrent PMU. The PMU structure tracks the events, how many are on
+ * the PMU, what function to call to read/write the physical PMU data
+ * and control registers, etc.
+ *
+ * This routine is assumed to return 0 on no error or -ENOMEM.
+ */
+ torrent_pmu = torrent_pmu_cntrs[chip][pmu_type];
+
+ torrent_pmu->compute_pmc_regs = compute_pmc_regs;
+ torrent_pmu->check_constraints = check_constraints;
+ torrent_pmu->enable_disable_cntr = enable_disable_cntr;
+ torrent_pmu->read_pmd = pmd_read;
+ torrent_pmu->write_pmd = pmd_write;
+ torrent_pmu->get_phys_pmd_reg = get_phys_pmd_reg;
+
+ hrtimer_init(&torrent_poll_timer, CLOCK_MONOTONIC,
+ HRTIMER_MODE_REL);
+ torrent_poll_timer.function = &poll_torrent_pmus;
+
+ spin_lock_init(&torrent_lock);
+
+ return 0;
+}
+
+static const struct {
+ int counters;
+} per_pmu_info[TORRENT_NUM_PMU_TYPES] = {
+ [TORRENT_PBUS_WXYZ_ID] = {
+ .counters = MAX_CNTRS_PER_WXYZ_LINK_PMU
+ },
+ [TORRENT_PBUS_LL_ID] = {
+ .counters = MAX_CNTRS_PER_LL_LINK_PMU
+ },
+ [TORRENT_PBUS_MCD_ID] = {
+ .counters = MAX_CNTRS_PER_MCD_PMU
+ },
+ [TORRENT_PBUS_UTIL_ID] = {
+ .counters = MAX_CNTRS_PER_BUS_UTIL_PMU
+ },
+ [TORRENT_MMU_ID] = {
+ .counters = MAX_CNTRS_PER_MMU_PMU
+ },
+ [TORRENT_CAU_ID] = {
+ .counters = MAX_CNTRS_PER_CAU_PMU
+ }
+};
+
+static int alloc_torrent_pmu_cntrs(int num_torrent_chips)
+{
+ int chip, pmu_type;
+
+ /* Allocate space for the torrent_pmu_cntrs struct. */
+
+ /* Allocate space for the top level index. */
+ torrent_pmu_cntrs = kmalloc(sizeof(void *) * num_torrent_chips,
+ GFP_KERNEL);
+
+ if (!torrent_pmu_cntrs)
+ return -ENOMEM;
+
+ for (chip = 0; chip < num_torrent_chips; chip++) {
+ /* Allocate space for the node-level pointer array. */
+ torrent_pmu_cntrs[chip] = kmalloc(
+ sizeof(void *) * TORRENT_NUM_PMU_TYPES, GFP_KERNEL);
+
+ if (!torrent_pmu_cntrs[chip])
+ return -ENOMEM;
+
+ for (pmu_type = 0; pmu_type < TORRENT_NUM_PMU_TYPES;
+ pmu_type++) {
+ torrent_pmu_cntrs[chip][pmu_type] =
+ kmalloc(
+ sizeof(struct torrent_pmu_counters),
+ GFP_KERNEL);
+
+ if (!torrent_pmu_cntrs[chip][pmu_type]) {
+ pr_err("%s: ran out of memory for torrent_pmu_cntrs",
+ __func__);
+ return -ENOMEM;
+ }
+ }
+ }
+ return 0;
+}
+
+static void dealloc_torrent_pmu_cntrs(int num_torrent_chips)
+{
+ int chip, pmu_type;
+
+ /* Release the torrent_pmu_cntrs structs. */
+
+ for (chip = 0; chip < num_torrent_chips; chip++) {
+
+ for (pmu_type = 0; pmu_type < TORRENT_NUM_PMU_TYPES;
+ pmu_type++)
+ kfree(torrent_pmu_cntrs[chip][pmu_type]);
+
+ kfree(torrent_pmu_cntrs[chip]);
+ }
+ torrent_pmu_cntrs = kmalloc(sizeof(void *) * num_torrent_chips,
+ GFP_KERNEL);
+}
+
+static u64 get_torrent_chip_id(void)
+{
+ struct device_node *node;
+ u64 *lp;
+
+ node = of_find_node_by_name(NULL, "hfi-iohub");
+ if (!node) {
+ pr_err("%s: of_find_node_by_name"
+ " 'hfi-iohub' failed\n", __func__);
+ return -EINVAL;
+ }
+
+ lp = (u64 *)of_get_property(node, "reg", NULL);
+ if (!lp) {
+ pr_err("%s: of_get_property 'hfi-iohub/reg' failed\n",
+ __func__);
+ return -EINVAL;
+ }
+ return *lp;
+}
+
+static u64 get_hfi_unit_id(void)
+{
+ u8 octant, id;
+ u32 *p;
+ struct device_node *node;
+ struct device_node *child_node = NULL;
+
+ node = of_find_node_by_name(NULL, "hfi-iohub");
+ if (!node) {
+ pr_err("%s: of_find_node_by_name 'hfi-iohub' failed\n",
+ __func__);
+ return -EINVAL;
+ }
+ octant = (node->full_name[strlen(node->full_name) - 1] - '0');
+ if (octant > HFI_MAX_OCTANT) {
+ pr_err("%s: invalid hfi-iohub octant '%s'\n",
+ __func__, node->full_name);
+ return -EINVAL;
+ }
+
+ id = octant << HFI_SHIFT_OCTANT;
+
+ while ((child_node = of_get_next_child(node, child_node))) {
+ p = (u32 *)of_get_property(child_node, "reg", NULL);
+ if (!p) {
+ pr_err("%s: of_get_property 'reg' failed\n", __func__);
+ return -EINVAL;
+ }
+ if (id == (u8)*p)
+ return id;
+ }
+ pr_err("%s: can not find child\n", __func__);
+ return -EINVAL;
+}
+
+void torrent_pmu_start(struct perf_event *event, int ef_flags)
+{
+ /* Don't support enable/disable on individual hardware counters */
+
+ if (!event->hw.idx || !event->hw.sample_period)
+ return;
+
+ if (!(event->hw.state & PERF_HES_STOPPED))
+ return;
+
+ event->hw.state = 0;
+
+}
+
+void torrent_pmu_stop(struct perf_event *event, int ef_flags)
+{
+
+ if (!event->hw.idx || !event->hw.sample_period)
+ return;
+
+ if (event->hw.state & PERF_HES_STOPPED)
+ return;
+
+ event->hw.state |= PERF_HES_STOPPED | PERF_HES_UPTODATE;
+}
+
+void torrent_pmu_read(struct perf_event *event)
+{
+
+ s64 delta, prev, value;
+ unsigned long lock_flags;
+ u8 virt_cntr;
+ int pmu_type;
+ struct hcall_data hcall_read;
+ struct torrent_pmu_events *torrent_pmu_events;
+ struct torrent_pmu_counters *torrent_pmu;
+ u64 event_code = event->hw.config;
+
+ if (!event->hw.idx)
+ return;
+
+ spin_lock_irqsave(&torrent_lock, lock_flags); /* Ensure only one CPU
+ * at a time access the
+ * HCall data structures
+ */
+ pmu_type = TORRENT_PMU_GET(event_code);
+ torrent_pmu = map_to_pmu(event_code);
+ torrent_pmu_events = &all_torrent_events[pmu_type];
+
+ /* Set up variable hcall_write for the enable/disable call. */
+ do_hcall_pmd_read(&hcall_read, all_torrent_events);
+ virt_cntr = TORRENT_VIRT_CTR_GET(event->hw.config);
+
+ prev = local64_read(&event->hw.prev_count);
+ value = torrent_pmu->read_pmd(torrent_pmu_events, &hcall_read);
+
+ /* calculate/save the count */
+ delta = value - prev;
+ local64_add(delta, &event->count);
+ local64_set(&event->hw.prev_count, value);
+
+ local64_add(delta, &event->count);
+ local64_sub(delta, &event->hw.period_left);
+ spin_unlock_irqrestore(&torrent_lock, lock_flags);
+}
+
+/*
+ * The transaction scheduling is done across all of the physical PMUs within
+ * the Torrent chip. When using transaction scheduling, the compatibility
+ * check is skipped for the events as each event is being added.. Once the
+ * entire set of events has been added, the constraint check is done for all
+ * of the events that have been added.
+ *
+ * Set the flag to make pmu::enable() not perform the constraint check.
+ */
+void torrent_pmu_start_txn(struct pmu *pmu)
+{
+ perf_pmu_disable(pmu);
+
+ torrent_txn.group_flag |= PERF_EVENT_TXN;
+ copy_all_events(all_torrent_events, txn_all_events);
+}
+
+void torrent_pmu_cancel_txn(struct pmu *pmu)
+{
+ perf_pmu_disable(pmu);
+
+ torrent_txn.group_flag &= PERF_EVENT_TXN;
+}
+
+int torrent_pmu_commit_txn(struct pmu *pmu)
+{
+ /*
+ * Do constraint check for new events across all physical PMUs within
+ * the Torrent chip.
+ */
+ int err;
+ unsigned long lock_flags;
+
+ spin_lock_irqsave(&torrent_lock, lock_flags);
+
+ /* check constraints for all pmus */
+ err = hw_perf_group_sched_in_torrent_check(txn_all_events);
+
+ if (err) {
+ pr_err("%s, ERROR, Torrent sched in check failed\n", __func__);
+ spin_unlock_irqrestore(&torrent_lock, lock_flags);
+ return -EAGAIN;
+ }
+
+ /*
+ * Can now accept the events from the txn list into the list of
+ * current events for the Torrent PMU an add to list for each physical
+ * PMU.
+ */
+
+ accept_torrent_events(txn_all_events, all_torrent_events);
+ spin_unlock_irqrestore(&torrent_lock, lock_flags);
+
+ perf_pmu_enable(pmu);
+ return 0;
+}
+
+int torrent_pmu_event_init(struct perf_event *event)
+{
+ int pmu_type, index, err, rtn = 0;
+ unsigned long flags;
+ struct torrent_pmu_events *torrent_pmu_events;
+ struct torrent_pmu_events tmp_all_events[TORRENT_NUM_PMU_TYPES];
+
+ /*
+ * If this is in a group, check if it can go on with all the
+ * other hardware events in the group. We assume the event
+ * hasn't been linked into its leader's sibling list at this point.
+ */
+
+ local_irq_save(flags);
+ switch (event->attr.type) {
+ case PERF_TYPE_RAW:
+ if (!IS_TORRENT_EVENT(event->attr.config)) {
+ rtn = -ENOENT;
+ goto out;
+ }
+ break;
+
+ default:
+ rtn = -ENOENT;
+ goto out;
+ }
+
+ /*
+ * Use an empty data structure to see if the events are self
+ * consistent.
+ */
+ initialize_event_struct(tmp_all_events);
+
+ if (event->group_leader != event) {
+ /*
+ * Use a temporary array of pmu events in case the entire
+ * group needs to be rejected.
+ */
+ collect_events(event->group_leader, tmp_all_events);
+ }
+
+ /* Include the group leader event to list of existing nest events */
+ pmu_type = TORRENT_PMU_GET(event->attr.config);
+
+ if (pmu_type < 0) {
+ pr_err("%s: ERROR, pmu_type < 0\n", __func__);
+ rtn = -EINVAL;
+ goto out;
+ }
+
+ torrent_pmu_events = &tmp_all_events[pmu_type];
+ index = torrent_pmu_events->n_events + torrent_pmu_events->n_add;
+ if (index >= get_max_nest_events(pmu_type)) {
+ pr_err("%s: ERROR, index out of range\n",
+ __func__);
+ rtn = -EINVAL;
+ goto out;
+ }
+ torrent_pmu_events->event[index] = event;
+ torrent_pmu_events->n_add++;
+
+ /* check constraints of new events */
+ err = hw_perf_group_sched_in_torrent_check(tmp_all_events);
+ if (err) {
+ pr_err("%s, ERROR, Torrent sched in check failed\n", __func__);
+ rtn = -EINVAL;
+ goto out;
+ }
+
+out:
+ local_irq_restore(flags);
+ return rtn;
+}
+
+struct pmu torrent_pmu = {
+ .pmu_enable = torrent_pmu_enable,
+ .pmu_disable = torrent_pmu_disable,
+ .event_init = torrent_pmu_event_init,
+ .add = torrent_pmu_add,
+ .del = torrent_pmu_del,
+ .start = torrent_pmu_start,
+ .stop = torrent_pmu_stop,
+ .read = torrent_pmu_read,
+ .start_txn = torrent_pmu_start_txn,
+ .cancel_txn = torrent_pmu_cancel_txn,
+ .commit_txn = torrent_pmu_commit_txn,
+};
+
+int __init torrent_pmu_init(void)
+{
+ int chip, ret;
+
+ /*
+ * Set up the structure for each of the PMU types to track what
+ * events are being counted by that PMU.
+ */
+
+ /* Attempt to determine the correct Torrent chip id, the HFI unit id
+ * and lock the counter facility. If we can't do these things, then
+ * we are not on a P7IH system, exit and do not setup the Torrent PMU.
+ */
+ torrent_chip_id = get_torrent_chip_id();
+ if (torrent_chip_id < 0)
+ /* torrent_chip_id will be set to the errno if not valid */
+ return torrent_chip_id;
+
+ hfi_unit_id = get_hfi_unit_id();
+ if (hfi_unit_id < 0)
+ /* torrent_chip_id will be set to the errno if not valid */
+ return torrent_chip_id;
+
+ /*
+ * Lock the counter facility for this partition's access. Note that
+ * we don't ever unlock it.
+ */
+
+ ret = PLPAR_HCALL(H_TOR_ACCESS_PMU_SCOM_REGS, torrent_chip_id,
+ PBUS_HCALL_LOCK, 0, (unsigned long)NULL);
+ if (ret != H_SUCCESS) {
+ pr_err("%s, ERROR, lock counter facility HCall returned an error %d\n",
+ __func__, ret);
+ return -EINVAL;
+ }
+
+ /*
+ * Looks like we are on a Torrent system, allocate space for the
+ * torrent_pmu_cntrs struct.
+ */
+ pr_debug("Number of Torrent chips %d\n", num_torrent_chips);
+ ret = alloc_torrent_pmu_cntrs(num_torrent_chips);
+ if (ret) {
+ pr_err("%s,%d: Failed to allocate torrent_pmu_cntrs, retval = %d\n",
+ __func__, __LINE__, ret);
+ return ret;
+ }
+
+ /* Align the 4K buffer used for HCalls */
+ aligned_buffer = (u64 *)((u64)unaligned_buffer + 0xFFFULL);
+ aligned_buffer = (u64 *)((u64)aligned_buffer & ~0xFFFULL);
+
+ /* We currently only support one Torrent chip */
+ chip = get_chip();
+
+ pr_debug("%s: set up Torrent chip %d WXYZ link PMU\n", __func__, chip);
+
+ if (torrent_pmu_initialize(chip, TORRENT_PBUS_WXYZ_ID,
+ wxyz_link_compute_pmc_reg,
+ wxyz_link_pmu_check_constraints,
+ wxyz_link_enable_disable_hw_cntr,
+ wxyz_link_pmd_read,
+ wxyz_link_pmd_write,
+ wxyz_link_get_phys_pmd_reg)) {
+ pr_err("%s: ERROR WXYZ Link PMU initialize failed\n",
+ __func__);
+ goto out;
+ }
+
+ if (torrent_pmu_initialize(chip, TORRENT_PBUS_LL_ID,
+ ll_link_compute_pmc_reg,
+ ll_link_pmu_check_constraints,
+ ll_link_enable_disable_hw_cntr,
+ ll_link_pmd_read,
+ ll_link_pmd_write,
+ ll_link_get_phys_pmd_reg)) {
+ pr_err("%s: ERROR LL Link PMU initialize failed\n",
+ __func__);
+ goto out;
+ }
+
+ if (torrent_pmu_initialize(chip, TORRENT_PBUS_MCD_ID,
+ mcd_compute_pmc_reg,
+ mcd_pmu_check_constraints,
+ mcd_enable_disable_hw_cntr,
+ mcd_pmd_read,
+ mcd_pmd_write,
+ mcd_get_phys_pmd_reg)) {
+ pr_err("%s: ERROR MCD PMU initialize failed\n",
+ __func__);
+ goto out;
+ }
+
+ if (torrent_pmu_initialize(chip, TORRENT_PBUS_UTIL_ID,
+ bus_util_compute_pmc_reg,
+ bus_util_pmu_check_constraints,
+ bus_util_enable_disable_hw_cntr,
+ bus_util_pmd_read,
+ bus_util_pmd_write,
+ bus_util_get_phys_pmd_reg)) {
+ pr_err("%s: ERROR BUS UTIL PMU initialize failed\n",
+ __func__);
+ goto out;
+ }
+
+ if (torrent_pmu_initialize(chip, TORRENT_MMU_ID,
+ mmu_compute_pmc_reg,
+ mmu_pmu_check_constraints,
+ mmu_enable_disable_hw_cntr,
+ mmu_pmd_read,
+ mmu_pmd_write,
+ mmu_get_phys_pmd_reg)) {
+ pr_err("%s: ERROR MMU PMU initialize failed\n",
+ __func__);
+ goto out;
+ }
+
+ if (torrent_pmu_initialize(chip, TORRENT_CAU_ID,
+ cau_compute_pmc_reg,
+ cau_pmu_check_constraints,
+ cau_enable_disable_hw_cntr,
+ cau_pmd_read,
+ cau_pmd_write,
+ cau_get_phys_pmd_reg)) {
+ pr_err("%s: ERROR CAU PMU initialize failed\n",
+ __func__);
+ goto out;
+ }
+
+ /* Initialize the structure to track the events being counted by
+ * the physical Torrent PMUs.
+ */
+ initialize_event_struct(all_torrent_events);
+
+ /* Initialize the poll interval value. It's used as a constant
+ * elsewhere.
+ */
+ torrent_counter_poll_interval = ktime_set(POLLING_INTERVAL_SEC,
+ POLLING_INTERVAL_NS);
+ ret = perf_pmu_register(&torrent_pmu, "torrent", -1);
+ return 0;
+out:
+ dealloc_torrent_pmu_cntrs(num_torrent_chips);
+ return -ENOMEM;
+
+}
+
+arch_initcall(torrent_pmu_init);
diff --git a/arch/powerpc/platforms/pseries/Kconfig b/arch/powerpc/platforms/pseries/Kconfig
index 71af4c5..402a029 100644
--- a/arch/powerpc/platforms/pseries/Kconfig
+++ b/arch/powerpc/platforms/pseries/Kconfig
@@ -26,6 +26,11 @@ config PPC_SPLPAR
processors, that is, which share physical processors between
two or more partitions.
+config PPC_P7IH
+ bool "P7IH (w/Torrent interconnect)"
+ depends on PPC_PSERIES
+ default n
+
config EEH
bool "PCI Extended Error Handling (EEH)" if EXPERT
depends on PPC_PSERIES && PCI
--
1.7.0.4
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox