* spufs: User space thread library
2005-09-16 12:16 [patch 00/11] spufs: latest spufs snapshot for 2.6.14-rc1 Arnd Bergmann
@ 2005-09-16 6:40 ` Arnd Bergmann
2005-09-16 12:16 ` [patch 01/11] spufs: The SPU file system, base Arnd Bergmann
` (10 subsequent siblings)
11 siblings, 0 replies; 14+ messages in thread
From: Arnd Bergmann @ 2005-09-16 6:40 UTC (permalink / raw)
To: ppc64-dev, HERREND
Cc: linux-kernel, Paul Mackerras, Benjamin Herrenschmidt,
jordi_caubet, Hiroyuki Machida, Geoff Levand
[-- Attachment #1: Type: text/plain, Size: 217 bytes --]
This is the user space library for using Cell SPE, for the
2.6.14-rc1 release.
It is maintained by Dirk Herrendoerfer <HERREND@de.ibm.com>, I'm
sending this out as a companion to the spufs implementation.
Arnd <><
[-- Attachment #2: libspe.tar.gz --]
[-- Type: application/x-tgz, Size: 35615 bytes --]
^ permalink raw reply [flat|nested] 14+ messages in thread* [patch 01/11] spufs: The SPU file system, base
2005-09-16 12:16 [patch 00/11] spufs: latest spufs snapshot for 2.6.14-rc1 Arnd Bergmann
2005-09-16 6:40 ` spufs: User space thread library Arnd Bergmann
@ 2005-09-16 12:16 ` Arnd Bergmann
2005-09-16 12:16 ` [patch 02/11] spufs: switchable spu contexts Arnd Bergmann
` (9 subsequent siblings)
11 siblings, 0 replies; 14+ messages in thread
From: Arnd Bergmann @ 2005-09-16 12:16 UTC (permalink / raw)
To: ppc64-dev
Cc: linux-kernel, Paul Mackerras, Benjamin Herrenschmidt,
jordi_caubet, Hiroyuki Machida, Geoff Levand
[-- Attachment #1: spufs-9.diff --]
[-- Type: text/plain, Size: 73190 bytes --]
This is a work-in-progress version of the SPU file system.
The base patch now is a bit smaller, since we removed some
of the spufs file that were only there for debugging.
See Documentation/filesystems/spufs.txt on how to use
spufs.
Signed-off-by: Arnd Bergmann <arndb@de.ibm.com>
--
Documentation/filesystems/spufs.txt | 107 +++++
arch/ppc/kernel/ppc_ksyms.c | 1
arch/ppc64/kernel/Makefile | 1
arch/ppc64/kernel/spu_base.c | 733 ++++++++++++++++++++++++++++++++++++
arch/ppc64/mm/hash_utils.c | 1
fs/Kconfig | 10
fs/Makefile | 1
fs/spufs/Makefile | 3
fs/spufs/context.c | 67 +++
fs/spufs/file.c | 653 ++++++++++++++++++++++++++++++++
fs/spufs/inode.c | 435 +++++++++++++++++++++
fs/spufs/spufs.h | 65 +++
include/asm-ppc64/spu.h | 473 +++++++++++++++++++++++
mm/memory.c | 2
14 files changed, 2551 insertions(+), 1 deletion(-)
Index: linux-cg/Documentation/filesystems/spufs.txt
===================================================================
--- /dev/null
+++ linux-cg/Documentation/filesystems/spufs.txt
@@ -0,0 +1,107 @@
+ *** The SPU file system ***
+
+Spufs is used to run code on the Synergistic Processing Units
+of the Cell Broadband Engine.
+
+The file system provides a name space similar to posix shared
+memory or message queues. Users that have write permissions
+on the file system can create directories in the spufs root.
+
+Every directory represents an SPU context, which is currently
+mapped to a physical SPU, but that is going to change to a
+virtualization scheme in future updates.
+
+An SPU context directory contains a predefined set of files
+used for manipulating the state of the logical SPU. Users
+can change permissions on those files, but not actually
+add or remove files without removing the complete directory.
+
+The current set of files is:
+
+/mem the contents of the local store memory of the SPU.
+ This can be accessed like a regular shared memory
+ file and contains both code and data in the address
+ space of the SPU.
+ The implemented file operations currently are read(),
+ write() and mmap(). We will need our own address
+ space operations as soon as we allow the SPU context
+ to be scheduled away from the physical SPU into
+ page cache.
+
+/run A stub file that lets us do the spu_run() call.
+ spu_run suspends the current thread from the host
+ CPU and transfers the flow of execution to the SPU.
+ The spu_run call returns to the calling thread when a
+ state is entered that can not be handled by the kernel,
+ e.g. an error in the SPU code or an exit() from it.
+ When a signal is pending for the host CPU thread, the
+ ioctl is interrupted and the SPU stopped in order to
+ call the signal handler.
+ Currently, there are three different methods how
+ spu_run can be entered with the /run file:
+ - The spu_run() syscall (see other patch)
+ - ioctl(), as used in previous versions
+ - read(), to return only the status, while the npc
+ is updated in kernel or through the /npc file.
+ Very likely, the concept of the /run file will change
+ in the near future and it will be replaced with
+ something completely different.
+
+/mbox The first SPU to CPU communication mailbox. This file
+ is read-only and can be read in units of 32 bits.
+ The file can only be used in non-blocking mode and
+ it even poll() will not block on it.
+ When no data is available in the mailbox, read() returns
+ EAGAIN.
+
+/ibox The second SPU to CPU communication mailbox. This file
+ is similar to the first mailbox file, but can be read
+ in blocking I/O mode, and the poll familiy of system
+ calls can be used to wait for it.
+
+/wbox The CPU to SPU communation mailbox. It is write-only
+ can can be written in units of 32 bits. If the mailbox
+ is full, write() will block and poll can be used to
+ wait for it becoming empty again.
+
+/mbox_stat
+/ibox_stat
+/wbox_stat
+ Read-only files that contain the length of the current
+ queue, i.e. how many words can be read from mbox or
+ ibox or how many words can be written to wbox without
+ blocking.
+ The files can be read only in 4-byte units and return
+ a big-endian binary integer number.
+
+/npc The next program counter of the SPU. The representation
+ is an ASCII string with the numeric value of the
+ next instruction to be executed. It can be used in
+ read/write mode.
+
+/signal1
+/signal2
+ The two signal notification channels of an SPU. These
+ are read-write files that operate on a 32 bit word,
+ similar to the npc file.
+ Writing to one of these files triggers an interrupt on
+ the SPU. The value writting to the signal files can
+ be read from the SPU through a channel read or from
+ host user space through the file.
+ After the value has been read by the SPU, it is reset
+ to zero.
+
+/signal1_type
+/signal2_type
+ These two files change the behavior of the signal1 and
+ signal2 notification files. The contain a numerical
+ ASCII string which is read as either "1" or "0".
+ In mode 0 (overwrite), the hardware replaces the contents
+ of the signal channel with the data that is written to it.
+ in mode 1 (logical OR), the hardware accumulates the bits
+ that are subsequently written to it.
+
+There are also a number of files that are only there for
+debugging purposes and give low-level access to hardware
+registers of an SPE. These will certainly go away and
+should not be considered official interfaces.
Index: linux-cg/arch/ppc64/kernel/Makefile
===================================================================
--- linux-cg.orig/arch/ppc64/kernel/Makefile
+++ linux-cg/arch/ppc64/kernel/Makefile
@@ -56,6 +56,7 @@ vio-obj-$(CONFIG_PPC_ISERIES) += iSeries
obj-$(CONFIG_IBMVIO) += vio.o $(vio-obj-y)
obj-$(CONFIG_XICS) += xics.o
obj-$(CONFIG_MPIC) += mpic.o
+obj-$(CONFIG_SPU_FS) += spu_base.o
obj-$(CONFIG_PPC_PMAC) += pmac_setup.o pmac_feature.o pmac_pci.o \
pmac_time.o pmac_nvram.o pmac_low_i2c.o \
Index: linux-cg/arch/ppc64/kernel/spu_base.c
===================================================================
--- /dev/null
+++ linux-cg/arch/ppc64/kernel/spu_base.c
@@ -0,0 +1,733 @@
+/*
+ * Low-level SPU handling
+ *
+ * (C) Copyright IBM Deutschland Entwicklung GmbH 2005
+ *
+ * Author: Arnd Bergmann <arndb@de.ibm.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2, or (at your option)
+ * any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+ */
+
+#define DEBUG 1
+
+#include <linux/interrupt.h>
+#include <linux/list.h>
+#include <linux/module.h>
+#include <linux/poll.h>
+#include <linux/ptrace.h>
+#include <linux/slab.h>
+#include <linux/wait.h>
+
+#include <asm/io.h>
+#include <asm/prom.h>
+#include <asm/semaphore.h>
+#include <asm/spu.h>
+#include <asm/mmu_context.h>
+
+#include "bpa_iic.h"
+
+static int __spu_trap_invalid_dma(struct spu *spu)
+{
+ pr_debug("%s\n", __FUNCTION__);
+ force_sig(SIGBUS, /* info, */ current);
+ return 0;
+}
+
+static int __spu_trap_dma_align(struct spu *spu)
+{
+ pr_debug("%s\n", __FUNCTION__);
+ force_sig(SIGBUS, /* info, */ current);
+ return 0;
+}
+
+static int __spu_trap_error(struct spu *spu)
+{
+ pr_debug("%s\n", __FUNCTION__);
+ force_sig(SIGILL, /* info, */ current);
+ return 0;
+}
+
+static void spu_restart_dma(struct spu *spu)
+{
+ struct spu_priv2 __iomem *priv2 = spu->priv2;
+ out_be64(&priv2->mfc_control_RW, MFC_CNTL_RESTART_DMA_COMMAND);
+}
+
+static int __spu_trap_data_seg(struct spu *spu, unsigned long ea)
+{
+ struct spu_priv2 __iomem *priv2;
+ struct mm_struct *mm;
+
+ pr_debug("%s\n", __FUNCTION__);
+
+ if (REGION_ID(ea) != USER_REGION_ID) {
+ pr_debug("invalid region access at %016lx\n", ea);
+ return 1;
+ }
+
+ priv2 = spu->priv2;
+ mm = spu->mm;
+
+ if (spu->slb_replace >= 8)
+ spu->slb_replace = 0;
+
+ out_be64(&priv2->slb_index_W, spu->slb_replace);
+ out_be64(&priv2->slb_vsid_RW,
+ (get_vsid(mm->context.id, ea) << SLB_VSID_SHIFT)
+ | SLB_VSID_USER);
+ out_be64(&priv2->slb_esid_RW, (ea & ESID_MASK) | SLB_ESID_V);
+
+ spu_restart_dma(spu);
+
+ pr_debug("set slb %d context %lx, ea %016lx, vsid %016lx, esid %016lx\n",
+ spu->slb_replace, mm->context.id, ea,
+ (get_vsid(mm->context.id, ea) << SLB_VSID_SHIFT)| SLB_VSID_USER,
+ (ea & ESID_MASK) | SLB_ESID_V);
+ return 0;
+}
+
+static int __spu_trap_data_map(struct spu *spu, unsigned long ea)
+{
+ unsigned long dsisr;
+ struct spu_priv1 __iomem *priv1;
+
+ pr_debug("%s\n", __FUNCTION__);
+ priv1 = spu->priv1;
+ dsisr = in_be64(&priv1->mfc_dsisr_RW);
+
+ wake_up(&spu->stop_wq);
+
+ return 0;
+}
+
+static int __spu_trap_mailbox(struct spu *spu)
+{
+ wake_up_all(&spu->ibox_wq);
+
+ /* atomically disable SPU mailbox interrupts */
+ spin_lock(&spu->register_lock);
+ out_be64(&spu->priv1->int_mask_class2_RW,
+ in_be64(&spu->priv1->int_mask_class2_RW) & ~0x1);
+ spin_unlock(&spu->register_lock);
+ return 0;
+}
+
+static int __spu_trap_stop(struct spu *spu)
+{
+ pr_debug("%s\n", __FUNCTION__);
+ spu->stop_code = in_be32(&spu->problem->spu_status_R);
+ wake_up(&spu->stop_wq);
+ return 0;
+}
+
+static int __spu_trap_halt(struct spu *spu)
+{
+ pr_debug("%s\n", __FUNCTION__);
+ spu->stop_code = in_be32(&spu->problem->spu_status_R);
+ wake_up(&spu->stop_wq);
+ return 0;
+}
+
+static int __spu_trap_tag_group(struct spu *spu)
+{
+ pr_debug("%s\n", __FUNCTION__);
+ /* wake_up(&spu->dma_wq); */
+ return 0;
+}
+
+static int __spu_trap_spubox(struct spu *spu)
+{
+ wake_up_all(&spu->wbox_wq);
+
+ /* atomically disable SPU mailbox interrupts */
+ spin_lock(&spu->register_lock);
+ out_be64(&spu->priv1->int_mask_class2_RW,
+ in_be64(&spu->priv1->int_mask_class2_RW) & ~0x10);
+ spin_unlock(&spu->register_lock);
+ return 0;
+}
+
+static irqreturn_t
+spu_irq_class_0(int irq, void *data, struct pt_regs *regs)
+{
+ struct spu *spu;
+
+ spu = data;
+ spu->class_0_pending = 1;
+ wake_up(&spu->stop_wq);
+
+ return IRQ_HANDLED;
+}
+
+static int
+spu_irq_class_0_bottom(struct spu *spu)
+{
+ unsigned long stat;
+
+ spu->class_0_pending = 0;
+
+ stat = in_be64(&spu->priv1->int_stat_class0_RW);
+
+ if (stat & 1) /* invalid MFC DMA */
+ __spu_trap_invalid_dma(spu);
+
+ if (stat & 2) /* invalid DMA alignment */
+ __spu_trap_dma_align(spu);
+
+ if (stat & 4) /* error on SPU */
+ __spu_trap_error(spu);
+
+ out_be64(&spu->priv1->int_stat_class0_RW, stat);
+ return 0;
+}
+
+static irqreturn_t
+spu_irq_class_1(int irq, void *data, struct pt_regs *regs)
+{
+ struct spu *spu;
+ unsigned long stat, dar;
+
+ spu = data;
+ stat = in_be64(&spu->priv1->int_stat_class1_RW);
+ dar = in_be64(&spu->priv1->mfc_dar_RW);
+
+ if (stat & 1) /* segment fault */
+ __spu_trap_data_seg(spu, dar);
+
+ if (stat & 2) { /* mapping fault */
+ __spu_trap_data_map(spu, dar);
+ }
+
+ if (stat & 4) /* ls compare & suspend on get */
+ ;
+
+ if (stat & 8) /* ls compare & suspend on put */
+ ;
+
+ out_be64(&spu->priv1->int_stat_class1_RW, stat);
+ return stat ? IRQ_HANDLED : IRQ_NONE;
+}
+
+static irqreturn_t
+spu_irq_class_2(int irq, void *data, struct pt_regs *regs)
+{
+ struct spu *spu;
+ unsigned long stat;
+
+ spu = data;
+ stat = in_be64(&spu->priv1->int_stat_class2_RW);
+
+ pr_debug("class 2 interrupt %d, %lx, %lx\n", irq, stat,
+ in_be64(&spu->priv1->int_mask_class2_RW));
+
+
+ if (stat & 1) /* PPC core mailbox */
+ __spu_trap_mailbox(spu);
+
+ if (stat & 2) /* SPU stop-and-signal */
+ __spu_trap_stop(spu);
+
+ if (stat & 4) /* SPU halted */
+ __spu_trap_halt(spu);
+
+ if (stat & 8) /* DMA tag group complete */
+ __spu_trap_tag_group(spu);
+
+ if (stat & 0x10) /* SPU mailbox threshold */
+ __spu_trap_spubox(spu);
+
+ out_be64(&spu->priv1->int_stat_class2_RW, stat);
+ return stat ? IRQ_HANDLED : IRQ_NONE;
+}
+
+static int
+spu_request_irqs(struct spu *spu)
+{
+ int ret;
+ int irq_base;
+
+ irq_base = IIC_NODE_STRIDE * spu->node + IIC_SPE_OFFSET;
+
+ snprintf(spu->irq_c0, sizeof (spu->irq_c0), "spe%02d.0", spu->number);
+ ret = request_irq(irq_base + spu->isrc,
+ spu_irq_class_0, 0, spu->irq_c0, spu);
+ if (ret)
+ goto out;
+ out_be64(&spu->priv1->int_mask_class0_RW, 0x7);
+
+ snprintf(spu->irq_c1, sizeof (spu->irq_c1), "spe%02d.1", spu->number);
+ ret = request_irq(irq_base + IIC_CLASS_STRIDE + spu->isrc,
+ spu_irq_class_1, 0, spu->irq_c1, spu);
+ if (ret)
+ goto out1;
+ out_be64(&spu->priv1->int_mask_class1_RW, 0x3);
+
+ snprintf(spu->irq_c2, sizeof (spu->irq_c2), "spe%02d.2", spu->number);
+ ret = request_irq(irq_base + 2*IIC_CLASS_STRIDE + spu->isrc,
+ spu_irq_class_2, 0, spu->irq_c2, spu);
+ if (ret)
+ goto out2;
+ out_be64(&spu->priv1->int_mask_class2_RW, 0xe);
+ goto out;
+
+out2:
+ free_irq(irq_base + IIC_CLASS_STRIDE + spu->isrc, spu);
+out1:
+ free_irq(irq_base + spu->isrc, spu);
+out:
+ return ret;
+}
+
+static void
+spu_free_irqs(struct spu *spu)
+{
+ int irq_base;
+
+ irq_base = IIC_NODE_STRIDE * spu->node + IIC_SPE_OFFSET;
+
+ free_irq(irq_base + spu->isrc, spu);
+ free_irq(irq_base + IIC_CLASS_STRIDE + spu->isrc, spu);
+ free_irq(irq_base + 2*IIC_CLASS_STRIDE + spu->isrc, spu);
+}
+
+static LIST_HEAD(spu_list);
+static DECLARE_MUTEX(spu_mutex);
+
+static void spu_init_channels(struct spu *spu)
+{
+ static const struct {
+ unsigned channel;
+ unsigned count;
+ } zero_list[] = {
+ { 0x00, 1, }, { 0x01, 1, }, { 0x03, 1, }, { 0x04, 1, },
+ { 0x18, 1, }, { 0x19, 1, }, { 0x1b, 1, }, { 0x1d, 1, },
+ }, count_list[] = {
+ { 0x00, 0, }, { 0x03, 0, }, { 0x04, 0, }, { 0x15, 16, },
+ { 0x17, 1, }, { 0x18, 0, }, { 0x19, 0, }, { 0x1b, 0, },
+ { 0x1c, 1, }, { 0x1d, 0, }, { 0x1e, 1, },
+ };
+ struct spu_priv2 *priv2;
+ int i;
+
+ priv2 = spu->priv2;
+
+ /* initialize all channel data to zero */
+ for (i = 0; i < ARRAY_SIZE(zero_list); i++) {
+ int count;
+
+ out_be64(&priv2->spu_chnlcntptr_RW, zero_list[i].channel);
+ for (count = 0; count < zero_list[i].count; count++)
+ out_be64(&priv2->spu_chnldata_RW, 0);
+ }
+
+ /* initialize channel counts to meaningful values */
+ for (i = 0; i < ARRAY_SIZE(count_list); i++) {
+ out_be64(&priv2->spu_chnlcntptr_RW, count_list[i].channel);
+ out_be64(&priv2->spu_chnlcnt_RW, count_list[i].count);
+ }
+}
+
+static void spu_init_regs(struct spu *spu)
+{
+ out_be64(&spu->priv1->int_mask_class0_RW, 0x7);
+ out_be64(&spu->priv1->int_mask_class1_RW, 0x3);
+ out_be64(&spu->priv1->int_mask_class2_RW, 0xe);
+}
+
+struct spu *spu_alloc(void)
+{
+ struct spu *spu;
+
+ down(&spu_mutex);
+ if (!list_empty(&spu_list)) {
+ spu = list_entry(spu_list.next, struct spu, list);
+ list_del_init(&spu->list);
+ pr_debug("Got SPU %x %d\n", spu->isrc, spu->number);
+ } else {
+ pr_debug("No SPU left\n");
+ spu = NULL;
+ }
+ up(&spu_mutex);
+
+ if (spu) {
+ spu_init_channels(spu);
+ spu_init_regs(spu);
+ }
+
+ return spu;
+}
+EXPORT_SYMBOL(spu_alloc);
+
+void spu_free(struct spu *spu)
+{
+ down(&spu_mutex);
+ list_add_tail(&spu->list, &spu_list);
+ up(&spu_mutex);
+}
+EXPORT_SYMBOL(spu_free);
+
+extern int hash_page(unsigned long ea, unsigned long access, unsigned long trap); //XXX
+static int spu_handle_mm_fault(struct spu *spu)
+{
+ struct spu_priv1 __iomem *priv1;
+ struct mm_struct *mm = spu->mm;
+ struct vm_area_struct *vma;
+ u64 ea, dsisr, is_write;
+ int ret;
+
+ priv1 = spu->priv1;
+ ea = in_be64(&priv1->mfc_dar_RW);
+ dsisr = in_be64(&priv1->mfc_dsisr_RW);
+#if 0
+ if (!IS_VALID_EA(ea)) {
+ return -EFAULT;
+ }
+#endif /* XXX */
+ if (mm == NULL) {
+ return -EFAULT;
+ }
+ if (mm->pgd == NULL) {
+ return -EFAULT;
+ }
+
+ down_read(&mm->mmap_sem);
+ vma = find_vma(mm, ea);
+ if (!vma)
+ goto bad_area;
+ if (vma->vm_start <= ea)
+ goto good_area;
+ if (!(vma->vm_flags & VM_GROWSDOWN))
+ goto bad_area;
+#if 0
+ if (expand_stack(vma, ea))
+ goto bad_area;
+#endif /* XXX */
+good_area:
+ is_write = dsisr & MFC_DSISR_ACCESS_PUT;
+ if (is_write) {
+ if (!(vma->vm_flags & VM_WRITE))
+ goto bad_area;
+ } else {
+ if (dsisr & MFC_DSISR_ACCESS_DENIED)
+ goto bad_area;
+ if (!(vma->vm_flags & (VM_READ | VM_EXEC)))
+ goto bad_area;
+ }
+ ret = 0;
+ switch (handle_mm_fault(mm, vma, ea, is_write)) {
+ case VM_FAULT_MINOR:
+ current->min_flt++;
+ break;
+ case VM_FAULT_MAJOR:
+ current->maj_flt++;
+ break;
+ case VM_FAULT_SIGBUS:
+ ret = -EFAULT;
+ goto bad_area;
+ case VM_FAULT_OOM:
+ ret = -ENOMEM;
+ goto bad_area;
+ default:
+ BUG();
+ }
+ up_read(&mm->mmap_sem);
+ return ret;
+
+bad_area:
+ up_read(&mm->mmap_sem);
+ return -EFAULT;
+}
+
+static int spu_handle_pte_fault(struct spu *spu)
+{
+ struct spu_priv1 __iomem *priv1;
+ u64 ea, dsisr, access, error = 0UL;
+ int ret = 0;
+
+ priv1 = spu->priv1;
+ ea = in_be64(&priv1->mfc_dar_RW);
+ dsisr = in_be64(&priv1->mfc_dsisr_RW);
+ access = (_PAGE_PRESENT | _PAGE_USER);
+ if (dsisr & MFC_DSISR_PTE_NOT_FOUND) {
+ if (hash_page(ea, access, 0x300) != 0)
+ error |= CLASS1_ENABLE_STORAGE_FAULT_INTR;
+ }
+ if ((error & CLASS1_ENABLE_STORAGE_FAULT_INTR) ||
+ (dsisr & MFC_DSISR_ACCESS_DENIED)) {
+ if ((ret = spu_handle_mm_fault(spu)) != 0)
+ error |= CLASS1_ENABLE_STORAGE_FAULT_INTR;
+ else
+ error &= ~CLASS1_ENABLE_STORAGE_FAULT_INTR;
+ }
+ if (!error)
+ spu_restart_dma(spu);
+
+ return ret;
+}
+
+int spu_run(struct spu *spu)
+{
+ struct spu_problem __iomem *prob;
+ struct spu_priv1 __iomem *priv1;
+ struct spu_priv2 __iomem *priv2;
+ unsigned long status;
+ int ret;
+
+ prob = spu->problem;
+ priv1 = spu->priv1;
+ priv2 = spu->priv2;
+
+ /* Let SPU run. */
+ spu->mm = current->mm;
+ eieio();
+ out_be32(&prob->spu_runcntl_RW, SPU_RUNCNTL_RUNNABLE);
+
+ do {
+ ret = wait_event_interruptible(spu->stop_wq,
+ (!((status = in_be32(&prob->spu_status_R)) & 0x1))
+ || (in_be64(&priv1->mfc_dsisr_RW) & MFC_DSISR_PTE_NOT_FOUND)
+ || spu->class_0_pending);
+
+ if (status & SPU_STATUS_STOPPED_BY_STOP)
+ ret = -EAGAIN;
+ else if (status & SPU_STATUS_STOPPED_BY_HALT)
+ ret = -EIO;
+ else if (in_be64(&priv1->mfc_dsisr_RW) & MFC_DSISR_PTE_NOT_FOUND)
+ ret = spu_handle_pte_fault(spu);
+
+ if (spu->class_0_pending)
+ spu_irq_class_0_bottom(spu);
+
+ if (!ret && signal_pending(current))
+ ret = -ERESTARTSYS;
+
+ } while (!ret);
+
+ /* Ensure SPU is stopped. */
+ out_be32(&prob->spu_runcntl_RW, SPU_RUNCNTL_STOP);
+ eieio();
+ while (in_be32(&prob->spu_status_R) & SPU_STATUS_RUNNING)
+ cpu_relax();
+
+ out_be64(&priv2->slb_invalidate_all_W, 0);
+ out_be64(&priv1->tlb_invalidate_entry_W, 0UL);
+ eieio();
+
+ spu->mm = NULL;
+
+ /* Check for SPU breakpoint. */
+ if (unlikely(current->ptrace & PT_PTRACED)) {
+ status = in_be32(&prob->spu_status_R);
+
+ if ((status & SPU_STATUS_STOPPED_BY_STOP)
+ && status >> SPU_STOP_STATUS_SHIFT == 0x3fff) {
+ force_sig(SIGTRAP, current);
+ ret = -ERESTARTSYS;
+ }
+ }
+
+ return ret;
+}
+EXPORT_SYMBOL(spu_run);
+
+static void __iomem * __init map_spe_prop(struct device_node *n,
+ const char *name)
+{
+ struct address_prop {
+ unsigned long address;
+ unsigned int len;
+ } __attribute__((packed)) *prop;
+
+ void *p;
+ int proplen;
+
+ p = get_property(n, name, &proplen);
+ if (proplen != sizeof (struct address_prop))
+ return NULL;
+
+ prop = p;
+
+ return ioremap(prop->address, prop->len);
+}
+
+static void spu_unmap(struct spu *spu)
+{
+ iounmap(spu->priv2);
+ iounmap(spu->priv1);
+ iounmap(spu->problem);
+ iounmap((u8 __iomem *)spu->local_store);
+}
+
+static int __init spu_map_device(struct spu *spu, struct device_node *spe)
+{
+ char *prop;
+ int ret;
+
+ ret = -ENODEV;
+ prop = get_property(spe, "isrc", NULL);
+ if (!prop)
+ goto out;
+ spu->isrc = *(unsigned int *)prop;
+
+ spu->name = get_property(spe, "name", NULL);
+ if (!spu->name)
+ goto out;
+
+ prop = get_property(spe, "local-store", NULL);
+ if (!prop)
+ goto out;
+ spu->local_store_phys = *(unsigned long *)prop;
+
+ /* we use local store as ram, not io memory */
+ spu->local_store = (void __force *)map_spe_prop(spe, "local-store");
+ if (!spu->local_store)
+ goto out;
+
+ spu->problem= map_spe_prop(spe, "problem");
+ if (!spu->problem)
+ goto out_unmap;
+
+ spu->priv1= map_spe_prop(spe, "priv1");
+ if (!spu->priv1)
+ goto out_unmap;
+
+ spu->priv2= map_spe_prop(spe, "priv2");
+ if (!spu->priv2)
+ goto out_unmap;
+ ret = 0;
+ goto out;
+
+out_unmap:
+ spu_unmap(spu);
+out:
+ return ret;
+}
+
+static int __init find_spu_node_id(struct device_node *spe)
+{
+ unsigned int *id;
+ struct device_node *cpu;
+
+ cpu = spe->parent->parent;
+ id = (unsigned int *)get_property(cpu, "node-id", NULL);
+
+ return id ? *id : 0;
+}
+
+static int __init create_spu(struct device_node *spe)
+{
+ struct spu *spu;
+ int ret;
+ static int number;
+
+ ret = -ENOMEM;
+ spu = kmalloc(sizeof (*spu), GFP_KERNEL);
+ if (!spu)
+ goto out;
+
+ ret = spu_map_device(spu, spe);
+ if (ret)
+ goto out_free;
+
+ spu->node = find_spu_node_id(spe);
+ spu->stop_code = 0;
+ spu->slb_replace = 0;
+ spu->mm = NULL;
+ spu->class_0_pending = 0;
+ spin_lock_init(&spu->register_lock);
+
+ out_be64(&spu->priv1->mfc_sdr_RW, mfspr(SPRN_SDR1));
+ out_be64(&spu->priv1->mfc_sr1_RW, 0x33);
+
+ init_waitqueue_head(&spu->stop_wq);
+ init_waitqueue_head(&spu->wbox_wq);
+ init_waitqueue_head(&spu->ibox_wq);
+
+ down(&spu_mutex);
+ spu->number = number++;
+ ret = spu_request_irqs(spu);
+ if (ret)
+ goto out_unmap;
+
+ list_add(&spu->list, &spu_list);
+ up(&spu_mutex);
+
+ pr_debug(KERN_DEBUG "Using SPE %s %02x %p %p %p %p %d\n",
+ spu->name, spu->isrc, spu->local_store,
+ spu->problem, spu->priv1, spu->priv2, spu->number);
+ goto out;
+
+out_unmap:
+ up(&spu_mutex);
+ spu_unmap(spu);
+out_free:
+ kfree(spu);
+out:
+ return ret;
+}
+
+static void destroy_spu(struct spu *spu)
+{
+ list_del_init(&spu->list);
+
+ spu_free_irqs(spu);
+ spu_unmap(spu);
+ kfree(spu);
+}
+
+static void cleanup_spu_base(void)
+{
+ struct spu *spu, *tmp;
+ down(&spu_mutex);
+ list_for_each_entry_safe(spu, tmp, &spu_list, list)
+ destroy_spu(spu);
+ up(&spu_mutex);
+}
+module_exit(cleanup_spu_base);
+
+static int __init init_spu_base(void)
+{
+ struct device_node *node;
+ int ret;
+
+ ret = -ENODEV;
+ for (node = of_find_node_by_type(NULL, "spe");
+ node; node = of_find_node_by_type(node, "spe")) {
+ ret = create_spu(node);
+ if (ret) {
+ printk(KERN_WARNING "%s: Error initializing %s\n",
+ __FUNCTION__, node->name);
+ cleanup_spu_base();
+ break;
+ }
+ }
+ /* in some old firmware versions, the spe is called 'spc', so we
+ look for that as well */
+ for (node = of_find_node_by_type(NULL, "spc");
+ node; node = of_find_node_by_type(node, "spc")) {
+ ret = create_spu(node);
+ if (ret) {
+ printk(KERN_WARNING "%s: Error initializing %s\n",
+ __FUNCTION__, node->name);
+ cleanup_spu_base();
+ break;
+ }
+ }
+ return ret;
+}
+module_init(init_spu_base);
+
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Arnd Bergmann <arndb@de.ibm.com>");
Index: linux-cg/arch/ppc64/mm/hash_utils.c
===================================================================
--- linux-cg.orig/arch/ppc64/mm/hash_utils.c
+++ linux-cg/arch/ppc64/mm/hash_utils.c
@@ -354,6 +354,7 @@ int hash_page(unsigned long ea, unsigned
return ret;
}
+EXPORT_SYMBOL_GPL(hash_page);
void flush_hash_page(unsigned long context, unsigned long ea, pte_t pte,
int local)
Index: linux-cg/fs/Kconfig
===================================================================
--- linux-cg.orig/fs/Kconfig
+++ linux-cg/fs/Kconfig
@@ -812,6 +812,16 @@ config HUGETLBFS
config HUGETLB_PAGE
def_bool HUGETLBFS
+config SPU_FS
+ tristate "SPU file system"
+ default m
+ depends on PPC_BPA
+ help
+ The SPU file system is used to access Synergistic Processing
+ Units on machines implementing the Broadband Processor
+ Architecture.
+
+
config RAMFS
bool
default y
Index: linux-cg/fs/Makefile
===================================================================
--- linux-cg.orig/fs/Makefile
+++ linux-cg/fs/Makefile
@@ -101,3 +101,4 @@ obj-$(CONFIG_BEFS_FS) += befs/
obj-$(CONFIG_HOSTFS) += hostfs/
obj-$(CONFIG_HPPFS) += hppfs/
obj-$(CONFIG_DEBUG_FS) += debugfs/
+obj-$(CONFIG_SPU_FS) += spufs/
Index: linux-cg/fs/spufs/Makefile
===================================================================
--- /dev/null
+++ linux-cg/fs/spufs/Makefile
@@ -0,0 +1,3 @@
+obj-$(CONFIG_SPU_FS) += spufs.o
+
+spufs-y += inode.o file.o context.o
Index: linux-cg/fs/spufs/context.c
===================================================================
--- /dev/null
+++ linux-cg/fs/spufs/context.c
@@ -0,0 +1,67 @@
+/*
+ * SPU file system -- SPU context management
+ *
+ * (C) Copyright IBM Deutschland Entwicklung GmbH 2005
+ *
+ * Author: Arnd Bergmann <arndb@de.ibm.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2, or (at your option)
+ * any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+ */
+
+#include <linux/slab.h>
+#include <asm/spu.h>
+#include "spufs.h"
+
+struct spu_context *alloc_spu_context(void)
+{
+ struct spu_context *ctx;
+ ctx = kmalloc(sizeof *ctx, GFP_KERNEL);
+ if (!ctx)
+ goto out;
+ ctx->spu = spu_alloc();
+ if (!ctx->spu)
+ goto out_free;
+ init_rwsem(&ctx->backing_sema);
+ spin_lock_init(&ctx->mmio_lock);
+ kref_init(&ctx->kref);
+ goto out;
+out_free:
+ kfree(ctx);
+ ctx = NULL;
+out:
+ return ctx;
+}
+
+void destroy_spu_context(struct kref *kref)
+{
+ struct spu_context *ctx;
+ ctx = container_of(kref, struct spu_context, kref);
+ if (ctx->spu)
+ spu_free(ctx->spu);
+ kfree(ctx);
+}
+
+struct spu_context * get_spu_context(struct spu_context *ctx)
+{
+ kref_get(&ctx->kref);
+ return ctx;
+}
+
+void put_spu_context(struct spu_context *ctx)
+{
+ kref_put(&ctx->kref, &destroy_spu_context);
+}
+
+
Index: linux-cg/fs/spufs/file.c
===================================================================
--- /dev/null
+++ linux-cg/fs/spufs/file.c
@@ -0,0 +1,653 @@
+/*
+ * SPU file system -- file contents
+ *
+ * (C) Copyright IBM Deutschland Entwicklung GmbH 2005
+ *
+ * Author: Arnd Bergmann <arndb@de.ibm.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2, or (at your option)
+ * any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+ */
+
+#include <linux/fs.h>
+#include <linux/ioctl.h>
+#include <linux/module.h>
+#include <linux/poll.h>
+
+#include <asm/io.h>
+#include <asm/semaphore.h>
+#include <asm/spu.h>
+#include <asm/uaccess.h>
+
+#include "spufs.h"
+
+static int
+spufs_mem_open(struct inode *inode, struct file *file)
+{
+ struct spufs_inode_info *i = SPUFS_I(inode);
+ file->private_data = i->i_ctx;
+ return 0;
+}
+
+static ssize_t
+spufs_mem_read(struct file *file, char __user *buffer,
+ size_t size, loff_t *pos)
+{
+ struct spu *spu;
+ struct spu_context *ctx;
+ int ret;
+
+ ctx = file->private_data;
+ spu = ctx->spu;
+
+ down_read(&ctx->backing_sema);
+ if (spu->number & 0/*1*/) {
+ ret = generic_file_read(file, buffer, size, pos);
+ goto out;
+ }
+
+ ret = simple_read_from_buffer(buffer, size, pos,
+ spu->local_store, LS_SIZE);
+out:
+ up_read(&ctx->backing_sema);
+ return ret;
+}
+
+static ssize_t
+spufs_mem_write(struct file *file, const char __user *buffer,
+ size_t size, loff_t *pos)
+{
+ struct spu_context *ctx = file->private_data;
+ struct spu *spu = ctx->spu;
+
+ if (spu->number & 0) //1)
+ return generic_file_write(file, buffer, size, pos);
+
+ size = min_t(ssize_t, LS_SIZE - *pos, size);
+ if (size <= 0)
+ return -EFBIG;
+ *pos += size;
+ return copy_from_user(spu->local_store + *pos - size,
+ buffer, size) ? -EFAULT : size;
+}
+
+static int
+spufs_mem_mmap(struct file *file, struct vm_area_struct *vma)
+{
+ struct spu_context *ctx = file->private_data;
+ struct spu *spu = ctx->spu;
+ unsigned long pfn;
+
+ if (spu->number & 0) //1)
+ return generic_file_mmap(file, vma);
+
+ vma->vm_flags |= VM_RESERVED;
+ vma->vm_page_prot = __pgprot(pgprot_val (vma->vm_page_prot)
+ | _PAGE_NO_CACHE);
+ pfn = spu->local_store_phys >> PAGE_SHIFT;
+ /*
+ * This will work for actual SPUs, but not for vmalloc memory:
+ */
+ if (remap_pfn_range(vma, vma->vm_start, pfn,
+ vma->vm_end-vma->vm_start, vma->vm_page_prot))
+ return -EAGAIN;
+ return 0;
+}
+
+static struct file_operations spufs_mem_fops = {
+ .open = spufs_mem_open,
+ .read = spufs_mem_read,
+ .write = spufs_mem_write,
+ .mmap = spufs_mem_mmap,
+ .llseek = generic_file_llseek,
+};
+
+/* generic open function for all pipe-like files */
+static int spufs_pipe_open(struct inode *inode, struct file *file)
+{
+ struct spufs_inode_info *i = SPUFS_I(inode);
+ file->private_data = i->i_ctx;
+
+ return nonseekable_open(inode, file);
+}
+
+static ssize_t spufs_mbox_read(struct file *file, char __user *buf,
+ size_t len, loff_t *pos)
+{
+ struct spu_context *ctx;
+ struct spu_problem __iomem *prob;
+ u32 mbox_stat;
+ u32 mbox_data;
+
+ if (len < 4)
+ return -EINVAL;
+
+ ctx = file->private_data;
+ prob = ctx->spu->problem;
+ mbox_stat = in_be32(&prob->mb_stat_R);
+ if (!(mbox_stat & 0x0000ff))
+ return -EAGAIN;
+
+ mbox_data = in_be32(&prob->pu_mb_R);
+
+ if (copy_to_user(buf, &mbox_data, sizeof mbox_data))
+ return -EFAULT;
+
+ return 4;
+}
+
+static struct file_operations spufs_mbox_fops = {
+ .open = spufs_pipe_open,
+ .read = spufs_mbox_read,
+};
+
+static ssize_t spufs_mbox_stat_read(struct file *file, char __user *buf,
+ size_t len, loff_t *pos)
+{
+ struct spu_context *ctx;
+ u32 mbox_stat;
+
+ if (len < 4)
+ return -EINVAL;
+
+ ctx = file->private_data;
+ mbox_stat = in_be32(&ctx->spu->problem->mb_stat_R) & 0xff;
+
+ if (copy_to_user(buf, &mbox_stat, sizeof mbox_stat))
+ return -EFAULT;
+
+ return 4;
+}
+
+static struct file_operations spufs_mbox_stat_fops = {
+ .open = spufs_pipe_open,
+ .read = spufs_mbox_stat_read,
+};
+
+/* low-level ibox access function */
+size_t spu_ibox_read(struct spu *spu, u32 *data)
+{
+ int ret;
+
+ spin_lock_irq(&spu->register_lock);
+
+ if (in_be32(&spu->problem->mb_stat_R) & 0xff0000) {
+ /* read the first available word */
+ *data = in_be64(&spu->priv2->puint_mb_R);
+ ret = 4;
+ } else {
+ /* make sure we get woken up by the interrupt */
+ out_be64(&spu->priv1->int_mask_class2_RW,
+ in_be64(&spu->priv1->int_mask_class2_RW) | 0x1);
+ ret = 0;
+ }
+
+ spin_unlock_irq(&spu->register_lock);
+ return ret;
+}
+EXPORT_SYMBOL(spu_ibox_read);
+
+static ssize_t spufs_ibox_read(struct file *file, char __user *buf,
+ size_t len, loff_t *pos)
+{
+ struct spu_context *ctx;
+ u32 ibox_data;
+ ssize_t ret;
+
+ if (len < 4)
+ return -EINVAL;
+
+ ctx = file->private_data;
+
+ ret = 0;
+ if (file->f_flags & O_NONBLOCK) {
+ if (!spu_ibox_read(ctx->spu, &ibox_data))
+ ret = -EAGAIN;
+ } else {
+ ret = wait_event_interruptible(ctx->spu->ibox_wq,
+ spu_ibox_read(ctx->spu, &ibox_data));
+ }
+
+ if (ret)
+ return ret;
+
+ ret = 4;
+ if (copy_to_user(buf, &ibox_data, sizeof ibox_data))
+ ret = -EFAULT;
+
+ return ret;
+}
+
+static unsigned int spufs_ibox_poll(struct file *file, poll_table *wait)
+{
+ struct spu_context *ctx;
+ struct spu_problem __iomem *prob;
+ u32 mbox_stat;
+ unsigned int mask;
+
+ ctx = file->private_data;
+ prob = ctx->spu->problem;
+ mbox_stat = in_be32(&prob->mb_stat_R);
+
+ poll_wait(file, &ctx->spu->ibox_wq, wait);
+
+ mask = 0;
+ if (mbox_stat & 0xff0000)
+ mask |= POLLIN | POLLRDNORM;
+
+ return mask;
+}
+
+static struct file_operations spufs_ibox_fops = {
+ .open = spufs_pipe_open,
+ .read = spufs_ibox_read,
+ .poll = spufs_ibox_poll,
+};
+
+static ssize_t spufs_ibox_stat_read(struct file *file, char __user *buf,
+ size_t len, loff_t *pos)
+{
+ struct spu_context *ctx;
+ u32 ibox_stat;
+
+ if (len < 4)
+ return -EINVAL;
+
+ ctx = file->private_data;
+ ibox_stat = (in_be32(&ctx->spu->problem->mb_stat_R) >> 16) & 0xff;
+
+ if (copy_to_user(buf, &ibox_stat, sizeof ibox_stat))
+ return -EFAULT;
+
+ return 4;
+}
+
+static struct file_operations spufs_ibox_stat_fops = {
+ .open = spufs_pipe_open,
+ .read = spufs_ibox_stat_read,
+};
+
+/* low-level mailbox write */
+size_t spu_wbox_write(struct spu *spu, u32 data)
+{
+ int ret;
+
+ spin_lock_irq(&spu->register_lock);
+
+ if (in_be32(&spu->problem->mb_stat_R) & 0x00ff00) {
+ /* we have space to write wbox_data to */
+ out_be32(&spu->problem->spu_mb_W, data);
+ ret = 4;
+ } else {
+ /* make sure we get woken up by the interrupt when space
+ becomes available */
+ out_be64(&spu->priv1->int_mask_class2_RW,
+ in_be64(&spu->priv1->int_mask_class2_RW) | 0x10);
+ ret = 0;
+ }
+
+ spin_unlock_irq(&spu->register_lock);
+ return ret;
+}
+EXPORT_SYMBOL(spu_wbox_write);
+
+static ssize_t spufs_wbox_write(struct file *file, const char __user *buf,
+ size_t len, loff_t *pos)
+{
+ struct spu_context *ctx;
+ u32 wbox_data;
+ int ret;
+
+ if (len < 4)
+ return -EINVAL;
+
+ ctx = file->private_data;
+
+ if (copy_from_user(&wbox_data, buf, sizeof wbox_data))
+ return -EFAULT;
+
+ ret = 0;
+ if (file->f_flags & O_NONBLOCK) {
+ if (!spu_wbox_write(ctx->spu, wbox_data))
+ ret = -EAGAIN;
+ } else {
+ ret = wait_event_interruptible(ctx->spu->wbox_wq,
+ spu_wbox_write(ctx->spu, wbox_data));
+ }
+
+ return ret ? ret : sizeof wbox_data;
+}
+
+static unsigned int spufs_wbox_poll(struct file *file, poll_table *wait)
+{
+ struct spu_context *ctx;
+ struct spu_problem __iomem *prob;
+ u32 mbox_stat;
+ unsigned int mask;
+
+ ctx = file->private_data;
+ prob = ctx->spu->problem;
+ mbox_stat = in_be32(&prob->mb_stat_R);
+
+ poll_wait(file, &ctx->spu->wbox_wq, wait);
+
+ mask = 0;
+ if (mbox_stat & 0x00ff00)
+ mask = POLLOUT | POLLWRNORM;
+
+ return mask;
+}
+
+static struct file_operations spufs_wbox_fops = {
+ .open = spufs_pipe_open,
+ .write = spufs_wbox_write,
+ .poll = spufs_wbox_poll,
+};
+
+static ssize_t spufs_wbox_stat_read(struct file *file, char __user *buf,
+ size_t len, loff_t *pos)
+{
+ struct spu_context *ctx;
+ u32 wbox_stat;
+
+ if (len < 4)
+ return -EINVAL;
+
+ ctx = file->private_data;
+ wbox_stat = (in_be32(&ctx->spu->problem->mb_stat_R) >> 8) & 0xff;
+
+ if (copy_to_user(buf, &wbox_stat, sizeof wbox_stat))
+ return -EFAULT;
+
+ return 4;
+}
+
+static struct file_operations spufs_wbox_stat_fops = {
+ .open = spufs_pipe_open,
+ .read = spufs_wbox_stat_read,
+};
+
+static long spufs_run_spu(struct file *file, struct spu_context *ctx,
+ u32 *npc, u32 *status)
+{
+ struct spu_problem __iomem *prob;
+ int ret;
+
+ if (file->f_flags & O_NONBLOCK) {
+ ret = -EAGAIN;
+ if (!down_write_trylock(&ctx->backing_sema))
+ goto out;
+ } else {
+ down_write(&ctx->backing_sema);
+ }
+
+ prob = ctx->spu->problem;
+ out_be32(&prob->spu_npc_RW, *npc);
+
+ ret = spu_run(ctx->spu);
+
+ *status = in_be32(&prob->spu_status_R);
+ *npc = in_be32(&prob->spu_npc_RW);
+
+ up_write(&ctx->backing_sema);
+
+out:
+ return ret;
+}
+
+struct spufs_run_arg {
+ u32 npc; /* inout: Next Program Counter */
+ u32 status; /* out: SPU status */
+};
+
+static ssize_t spufs_run_read(struct file *file, char __user *buf,
+ size_t len, loff_t *pos)
+{
+ struct spu_context *ctx;
+ struct spufs_run_arg arg;
+ int ret;
+
+ ctx = file->private_data;
+
+ ret = -EINVAL;
+ if (len < 8)
+ goto out;
+
+ arg.npc = in_be32(&ctx->spu->problem->spu_npc_RW);
+
+ ret = spufs_run_spu(file, file->private_data, &arg.npc, &arg.status);
+ if (ret == -EAGAIN)
+ ret = 0;
+
+ if ((arg.status & 0xffff0002) == 0x21000002) {
+ /* library callout */
+ u32 npc = arg.npc;
+ arg.npc = *(u32*) (ctx->spu->local_store + npc);
+ npc += 4;
+ out_be32(&ctx->spu->problem->spu_npc_RW, npc);
+ }
+
+ if (ret)
+ goto out;
+
+ ret = 8;
+ if (copy_to_user(buf, &arg, 8))
+ ret = -EFAULT;
+
+out:
+ return ret;
+}
+
+/* either this ioctl function or the system call needs to die! */
+static long spufs_run_ioctl(struct file *file, unsigned int num,
+ unsigned long arg)
+{
+ struct spufs_run_arg data;
+ int ret;
+
+ if (num != _IOWR('s', 0, struct spufs_run_arg))
+ return -EINVAL;
+
+ if (copy_from_user(&data, (void __user *)arg, sizeof data))
+ return -EFAULT;
+
+ ret = spufs_run_spu(file, file->private_data,
+ &data.npc, &data.status);
+
+ if (copy_to_user((void __user *)arg, &data, sizeof data))
+ ret = -EFAULT;
+
+ return ret;
+}
+
+static struct file_operations spufs_run_fops = {
+ .open = spufs_pipe_open,
+ .unlocked_ioctl = spufs_run_ioctl,
+ .compat_ioctl = spufs_run_ioctl,
+ .read = spufs_run_read,
+};
+
+static ssize_t spufs_signal1_read(struct file *file, char __user *buf,
+ size_t len, loff_t *pos)
+{
+ struct spu_context *ctx;
+ struct spu_problem *prob;
+ u32 data;
+
+ ctx = file->private_data;
+ prob = ctx->spu->problem;
+
+ if (len < 4)
+ return -EINVAL;
+
+ data = in_be32(&prob->signal_notify1);
+ if (copy_to_user(buf, &data, 4))
+ return -EFAULT;
+
+ return 4;
+}
+
+static ssize_t spufs_signal1_write(struct file *file, const char __user *buf,
+ size_t len, loff_t *pos)
+{
+ struct spu_context *ctx;
+ struct spu_problem *prob;
+ u32 data;
+
+ ctx = file->private_data;
+ prob = ctx->spu->problem;
+
+ if (len < 4)
+ return -EINVAL;
+
+ if (copy_from_user(&data, buf, 4))
+ return -EFAULT;
+
+ out_be32(&prob->signal_notify1, data);
+
+ return 4;
+}
+
+static struct file_operations spufs_signal1_fops = {
+ .open = spufs_pipe_open,
+ .read = spufs_signal1_read,
+ .write = spufs_signal1_write,
+};
+
+static ssize_t spufs_signal2_read(struct file *file, char __user *buf,
+ size_t len, loff_t *pos)
+{
+ struct spu_context *ctx;
+ struct spu_problem *prob;
+ u32 data;
+
+ ctx = file->private_data;
+ prob = ctx->spu->problem;
+
+ if (len < 4)
+ return -EINVAL;
+
+ data = in_be32(&prob->signal_notify2);
+ if (copy_to_user(buf, &data, 4))
+ return -EFAULT;
+
+ return 4;
+}
+
+static ssize_t spufs_signal2_write(struct file *file, const char __user *buf,
+ size_t len, loff_t *pos)
+{
+ struct spu_context *ctx;
+ struct spu_problem *prob;
+ u32 data;
+
+ ctx = file->private_data;
+ prob = ctx->spu->problem;
+
+ if (len < 4)
+ return -EINVAL;
+
+ if (copy_from_user(&data, buf, 4))
+ return -EFAULT;
+
+ out_be32(&prob->signal_notify2, data);
+
+ return 4;
+}
+
+static struct file_operations spufs_signal2_fops = {
+ .open = spufs_pipe_open,
+ .read = spufs_signal2_read,
+ .write = spufs_signal2_write,
+};
+
+static void spufs_signal1_type_set(void *data, u64 val)
+{
+ struct spu_context *ctx = data;
+ struct spu_priv2 *priv2 = ctx->spu->priv2;
+ u64 tmp;
+
+ spin_lock_irq(&ctx->spu->register_lock);
+ tmp = in_be64(&priv2->spu_cfg_RW);
+ if (val)
+ tmp |= 1;
+ else
+ tmp &= ~1;
+ out_be64(&priv2->spu_cfg_RW, tmp);
+ spin_unlock_irq(&ctx->spu->register_lock);
+}
+
+static u64 spufs_signal1_type_get(void *data)
+{
+ struct spu_context *ctx = data;
+ return (in_be64(&ctx->spu->priv2->spu_cfg_RW) & 1) != 0;
+}
+DEFINE_SIMPLE_ATTRIBUTE(spufs_signal1_type, spufs_signal1_type_get,
+ spufs_signal1_type_set, "%llu");
+
+static void spufs_signal2_type_set(void *data, u64 val)
+{
+ struct spu_context *ctx = data;
+ struct spu_priv2 *priv2 = ctx->spu->priv2;
+ u64 tmp;
+
+ spin_lock_irq(&ctx->spu->register_lock);
+ tmp = in_be64(&priv2->spu_cfg_RW);
+ if (val)
+ tmp |= 2;
+ else
+ tmp &= ~2;
+ out_be64(&priv2->spu_cfg_RW, tmp);
+ spin_unlock_irq(&ctx->spu->register_lock);
+}
+
+static u64 spufs_signal2_type_get(void *data)
+{
+ struct spu_context *ctx = data;
+ return (in_be64(&ctx->spu->priv2->spu_cfg_RW) & 2) != 0;
+}
+DEFINE_SIMPLE_ATTRIBUTE(spufs_signal2_type, spufs_signal2_type_get,
+ spufs_signal2_type_set, "%llu");
+
+static void spufs_npc_set(void *data, u64 val)
+{
+ struct spu_context *ctx = data;
+ out_be32(&ctx->spu->problem->spu_npc_RW, val);
+}
+
+static u64 spufs_npc_get(void *data)
+{
+ struct spu_context *ctx = data;
+ u64 ret;
+ ret = in_be32(&ctx->spu->problem->spu_npc_RW);
+ return ret;
+}
+DEFINE_SIMPLE_ATTRIBUTE(spufs_npc_ops, spufs_npc_get, spufs_npc_set, "%llx\n")
+
+struct tree_descr spufs_dir_contents[] = {
+ { "mem", &spufs_mem_fops, 0666, },
+ { "run", &spufs_run_fops, 0444, },
+ { "mbox", &spufs_mbox_fops, 0444, },
+ { "ibox", &spufs_ibox_fops, 0444, },
+ { "wbox", &spufs_wbox_fops, 0222, },
+ { "mbox_stat", &spufs_mbox_stat_fops, 0444, },
+ { "ibox_stat", &spufs_ibox_stat_fops, 0444, },
+ { "wbox_stat", &spufs_wbox_stat_fops, 0444, },
+ { "signal1", &spufs_signal1_fops, 0666, },
+ { "signal2", &spufs_signal2_fops, 0666, },
+ { "signal1_type", &spufs_signal1_type, 0666, },
+ { "signal2_type", &spufs_signal2_type, 0666, },
+ { "npc", &spufs_npc_ops, 0666, },
+ {},
+};
Index: linux-cg/fs/spufs/inode.c
===================================================================
--- /dev/null
+++ linux-cg/fs/spufs/inode.c
@@ -0,0 +1,435 @@
+/*
+ * SPU file system
+ *
+ * (C) Copyright IBM Deutschland Entwicklung GmbH 2005
+ *
+ * Author: Arnd Bergmann <arndb@de.ibm.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2, or (at your option)
+ * any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+ */
+
+#include <linux/fs.h>
+#include <linux/backing-dev.h>
+#include <linux/init.h>
+#include <linux/ioctl.h>
+#include <linux/module.h>
+#include <linux/pagemap.h>
+#include <linux/poll.h>
+#include <linux/slab.h>
+#include <linux/parser.h>
+
+#include <asm/io.h>
+#include <asm/semaphore.h>
+#include <asm/spu.h>
+#include <asm/uaccess.h>
+
+#include "spufs.h"
+
+static kmem_cache_t *spufs_inode_cache;
+
+/* Information about the backing dev, same as ramfs */
+#if 0
+static struct backing_dev_info spufs_backing_dev_info = {
+ .ra_pages = 0, /* No readahead */
+ .capabilities = BDI_CAP_NO_ACCT_DIRTY | BDI_CAP_NO_WRITEBACK |
+ BDI_CAP_MAP_DIRECT | BDI_CAP_MAP_COPY | BDI_CAP_READ_MAP |
+ BDI_CAP_WRITE_MAP,
+};
+
+static struct address_space_operations spufs_aops = {
+ .readpage = simple_readpage,
+ .prepare_write = simple_prepare_write,
+ .commit_write = simple_commit_write,
+};
+#endif
+
+/* Inode operations */
+
+static struct inode *
+spufs_alloc_inode(struct super_block *sb)
+{
+ struct spufs_inode_info *ei;
+
+ ei = kmem_cache_alloc(spufs_inode_cache, SLAB_KERNEL);
+ if (!ei)
+ return NULL;
+ return &ei->vfs_inode;
+}
+
+static void
+spufs_destroy_inode(struct inode *inode)
+{
+ kmem_cache_free(spufs_inode_cache, SPUFS_I(inode));
+}
+
+static void
+spufs_init_once(void *p, kmem_cache_t * cachep, unsigned long flags)
+{
+ struct spufs_inode_info *ei = p;
+
+ if ((flags & (SLAB_CTOR_VERIFY|SLAB_CTOR_CONSTRUCTOR)) ==
+ SLAB_CTOR_CONSTRUCTOR) {
+ inode_init_once(&ei->vfs_inode);
+ }
+}
+
+static struct inode *
+spufs_new_inode(struct super_block *sb, int mode)
+{
+ struct inode *inode;
+
+ inode = new_inode(sb);
+ if (!inode)
+ goto out;
+
+ inode->i_mode = mode;
+ inode->i_uid = current->fsuid;
+ inode->i_gid = current->fsgid;
+ inode->i_blksize = PAGE_CACHE_SIZE;
+ inode->i_blocks = 0;
+ inode->i_atime = inode->i_mtime = inode->i_ctime = CURRENT_TIME;
+out:
+ return inode;
+}
+
+static int
+spufs_setattr(struct dentry *dentry, struct iattr *attr)
+{
+ struct inode *inode = dentry->d_inode;
+
+/* dump_stack();
+ pr_debug("ia_size %lld, i_size:%lld\n", attr->ia_size, inode->i_size);
+*/
+ if ((attr->ia_valid & ATTR_SIZE) &&
+ (attr->ia_size != inode->i_size))
+ return -EINVAL;
+ return inode_setattr(inode, attr);
+}
+
+
+static int
+spufs_new_file(struct super_block *sb, struct dentry *dentry,
+ struct file_operations *fops, int mode,
+ struct spu_context *ctx)
+{
+ static struct inode_operations spufs_file_iops = {
+ .getattr = simple_getattr,
+ .setattr = spufs_setattr,
+ .unlink = simple_unlink,
+ };
+ struct inode *inode;
+ int ret;
+
+ ret = -ENOSPC;
+ inode = spufs_new_inode(sb, S_IFREG | mode);
+ if (!inode)
+ goto out;
+
+ ret = 0;
+ inode->i_op = &spufs_file_iops;
+ inode->i_fop = fops;
+// inode->i_mapping->a_ops = &spufs_aops;
+// inode->i_mapping->backing_dev_info = &spufs_backing_dev_info;
+ inode->u.generic_ip = SPUFS_I(inode)->i_ctx = get_spu_context(ctx);
+ d_add(dentry, inode);
+out:
+ return ret;
+}
+
+static int
+spufs_create(struct inode *dir, struct dentry *dentry,
+ int mode, struct nameidata *nd)
+{
+ struct tree_descr *descr;
+ struct spu_context *ctx;
+ int ret;
+
+ descr = spufs_dir_contents;
+
+ /* search spufs_dir_contents for a file with the name we are
+ trying to create */
+ ret = -EINVAL;
+ while (strcmp(descr->name, dentry->d_name.name) != 0) {
+ descr++;
+ if (!descr->name || !descr->name[0])
+ goto out;
+ }
+
+ ctx = SPUFS_I(dir)->i_ctx;
+ mode &= descr->mode;
+
+ ret = spufs_new_file(dir->i_sb, dentry, descr->ops, mode, ctx);
+ /* get an extra reference to pin the dentry */
+ dget(dentry);
+out:
+ return ret;
+}
+
+static void
+spufs_delete_inode(struct inode *inode)
+{
+ if (SPUFS_I(inode)->i_ctx)
+ put_spu_context(SPUFS_I(inode)->i_ctx);
+ clear_inode(inode);
+}
+
+static int
+spufs_fill_dir(struct dentry *dir, struct tree_descr *files,
+ int mode, struct spu_context *ctx)
+{
+ struct dentry *dentry;
+ int ret;
+
+ while (files->name && files->name[0]) {
+ ret = -ENOMEM;
+ dentry = d_alloc_name(dir, files->name);
+ if (!dentry)
+ goto out;
+ ret = spufs_new_file(dir->d_sb, dentry, files->ops,
+ files->mode & mode, ctx);
+ if (ret)
+ goto out;
+ files++;
+ }
+ return 0;
+out:
+ // FIXME: remove all files that are left
+
+ return ret;
+}
+
+struct inode_operations spufs_dir_inode_operations = {
+ .lookup = simple_lookup,
+ .unlink = simple_unlink,
+ .create = spufs_create,
+};
+
+static int
+spufs_mkdir(struct inode *dir, struct dentry *dentry, int mode)
+{
+ int ret;
+ struct inode *inode;
+ struct spu_context *ctx;
+
+ ret = -ENOSPC;
+ inode = spufs_new_inode(dir->i_sb, mode | S_IFDIR);
+ if (!inode)
+ goto out;
+
+ if (dir->i_mode & S_ISGID) {
+ inode->i_gid = dir->i_gid;
+ inode->i_mode &= S_ISGID;
+ }
+ ctx = alloc_spu_context();
+ SPUFS_I(inode)->i_ctx = ctx;
+ if (!ctx)
+ goto out_iput;
+
+ inode->i_op = &spufs_dir_inode_operations;
+ inode->i_fop = &simple_dir_operations;
+ ret = spufs_fill_dir(dentry, spufs_dir_contents, mode, ctx);
+ if (ret)
+ goto out_free_ctx;
+
+ d_instantiate(dentry, inode);
+ dget(dentry);
+ dir->i_nlink++;
+ goto out;
+
+out_free_ctx:
+ put_spu_context(ctx);
+out_iput:
+ iput(inode);
+out:
+ return ret;
+}
+
+/* This looks really wrong! */
+static int spufs_rmdir(struct inode *root, struct dentry *dir_dentry)
+{
+ struct dentry *dentry;
+ int err;
+
+ spin_lock(&dcache_lock);
+#if 0
+ /* check if any entry is used */
+ err = -EBUSY;
+ list_for_each_entry(dentry, &dir_dentry->d_subdirs, d_child) {
+ if (d_unhashed(dentry) || !dentry->d_inode)
+ continue;
+ if (atomic_read(&dentry->d_count) != 1)
+ goto out;
+ }
+#endif
+ /* remove all entries */
+ err = 0;
+ list_for_each_entry(dentry, &dir_dentry->d_subdirs, d_child) {
+ if (d_unhashed(dentry) || !dentry->d_inode)
+ continue;
+ atomic_dec(&dentry->d_count);
+ __d_drop(dentry);
+ }
+#if 0
+out:
+#endif
+ spin_unlock(&dcache_lock);
+ if (!err) {
+ shrink_dcache_parent(dir_dentry);
+ err = simple_rmdir(root, dir_dentry);
+ }
+ return err;
+}
+
+/* File system initialization */
+enum {
+ Opt_uid, Opt_gid, Opt_err,
+};
+
+static match_table_t spufs_tokens = {
+ { Opt_uid, "uid=%d" },
+ { Opt_gid, "gid=%d" },
+ { Opt_err, NULL },
+};
+
+static int
+spufs_parse_options(char *options, struct inode *root)
+{
+ char *p;
+ substring_t args[MAX_OPT_ARGS];
+
+ while ((p = strsep(&options, ",")) != NULL) {
+ int token, option;
+
+ if (!*p)
+ continue;
+
+ token = match_token(p, spufs_tokens, args);
+ switch (token) {
+ case Opt_uid:
+ if (match_int(&args[0], &option))
+ return 0;
+ root->i_uid = option;
+ break;
+ case Opt_gid:
+ if (match_int(&args[0], &option))
+ return 0;
+ root->i_gid = option;
+ break;
+ default:
+ return 0;
+ }
+ }
+ return 1;
+}
+
+static int
+spufs_create_root(struct super_block *sb, void *data) {
+ static struct inode_operations spufs_root_inode_operations = {
+ .lookup = simple_lookup,
+ .mkdir = spufs_mkdir,
+ .rmdir = spufs_rmdir,
+// .rename = simple_rename, // XXX maybe
+ };
+
+ struct inode *inode;
+ int ret;
+
+ ret = -ENOMEM;
+ inode = spufs_new_inode(sb, S_IFDIR | 0775);
+ if (!inode)
+ goto out;
+
+ inode->i_op = &spufs_root_inode_operations;
+ inode->i_fop = &simple_dir_operations;
+ SPUFS_I(inode)->i_ctx = NULL;
+
+ ret = -EINVAL;
+ if (!spufs_parse_options(data, inode))
+ goto out_iput;
+
+ ret = -ENOMEM;
+ sb->s_root = d_alloc_root(inode);
+ if (!sb->s_root)
+ goto out_iput;
+
+ return 0;
+out_iput:
+ iput(inode);
+out:
+ return ret;
+}
+
+static int
+spufs_fill_super(struct super_block *sb, void *data, int silent)
+{
+ static struct super_operations s_ops = {
+ .alloc_inode = spufs_alloc_inode,
+ .destroy_inode = spufs_destroy_inode,
+ .statfs = simple_statfs,
+ .delete_inode = spufs_delete_inode,
+ .drop_inode = generic_delete_inode,
+ };
+
+ sb->s_maxbytes = MAX_LFS_FILESIZE;
+ sb->s_blocksize = PAGE_CACHE_SIZE;
+ sb->s_blocksize_bits = PAGE_CACHE_SHIFT;
+ sb->s_magic = SPUFS_MAGIC;
+ sb->s_op = &s_ops;
+
+ return spufs_create_root(sb, data);
+}
+
+static struct super_block *
+spufs_get_sb(struct file_system_type *fstype, int flags,
+ const char *name, void *data)
+{
+ return get_sb_single(fstype, flags, data, spufs_fill_super);
+}
+
+static struct file_system_type spufs_type = {
+ .owner = THIS_MODULE,
+ .name = "spufs",
+ .get_sb = spufs_get_sb,
+ .kill_sb = kill_litter_super,
+};
+
+static int spufs_init(void)
+{
+ int ret;
+ ret = -ENOMEM;
+ spufs_inode_cache = kmem_cache_create("spufs_inode_cache",
+ sizeof(struct spufs_inode_info), 0,
+ SLAB_HWCACHE_ALIGN, spufs_init_once, NULL);
+
+ if (!spufs_inode_cache)
+ goto out;
+ ret = register_filesystem(&spufs_type);
+ if (ret)
+ kmem_cache_destroy(spufs_inode_cache);
+out:
+ return ret;
+}
+module_init(spufs_init);
+
+static void spufs_exit(void)
+{
+ unregister_filesystem(&spufs_type);
+ kmem_cache_destroy(spufs_inode_cache);
+}
+module_exit(spufs_exit);
+
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Arnd Bergmann <arndb@de.ibm.com>");
+
Index: linux-cg/fs/spufs/spufs.h
===================================================================
--- /dev/null
+++ linux-cg/fs/spufs/spufs.h
@@ -0,0 +1,65 @@
+/*
+ * SPU file system
+ *
+ * (C) Copyright IBM Deutschland Entwicklung GmbH 2005
+ *
+ * Author: Arnd Bergmann <arndb@de.ibm.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2, or (at your option)
+ * any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+ */
+#ifndef SPUFS_H
+#define SPUFS_H
+
+#include <linux/kref.h>
+#include <linux/rwsem.h>
+#include <linux/spinlock.h>
+#include <linux/fs.h>
+
+#include <asm/spu.h>
+
+/* The magic number for our file system */
+enum {
+ SPUFS_MAGIC = 0x23c9b64e,
+};
+
+struct spu_context {
+ struct spu *spu; /* pointer to a physical SPU */
+ struct rw_semaphore backing_sema; /* protects the above */
+ spinlock_t mmio_lock; /* protects mmio access */
+
+ struct kref kref;
+};
+
+struct spufs_inode_info {
+ struct spu_context *i_ctx;
+ struct inode vfs_inode;
+};
+#define SPUFS_I(inode) \
+ container_of(inode, struct spufs_inode_info, vfs_inode)
+
+extern struct tree_descr spufs_dir_contents[];
+
+/* context management */
+struct spu_context * alloc_spu_context(void);
+void destroy_spu_context(struct kref *kref);
+struct spu_context * get_spu_context(struct spu_context *ctx);
+void put_spu_context(struct spu_context *ctx);
+
+void spu_acquire(struct spu_context *ctx);
+void spu_release(struct spu_context *ctx);
+void spu_acquire_runnable(struct spu_context *ctx);
+void spu_acquire_saved(struct spu_context *ctx);
+
+#endif
Index: linux-cg/include/asm-ppc64/spu.h
===================================================================
--- /dev/null
+++ linux-cg/include/asm-ppc64/spu.h
@@ -0,0 +1,473 @@
+/*
+ * SPU core / file system interface and HW structures
+ *
+ * (C) Copyright IBM Deutschland Entwicklung GmbH 2005
+ *
+ * Author: Arnd Bergmann <arndb@de.ibm.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2, or (at your option)
+ * any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+ */
+
+#ifndef _SPU_H
+#define _SPU_H
+#include <linux/kref.h>
+#include <linux/workqueue.h>
+
+#define LS_ORDER (6) /* 256 kb */
+
+#define LS_SIZE (PAGE_SIZE << LS_ORDER)
+
+struct spu {
+ char *name;
+ unsigned long local_store_phys;
+ u8 *local_store;
+ struct spu_problem __iomem *problem;
+ struct spu_priv1 __iomem *priv1;
+ struct spu_priv2 __iomem *priv2;
+ struct list_head list;
+ int number;
+ u32 isrc;
+ u32 node;
+ struct kref kref;
+ size_t ls_size;
+ unsigned int slb_replace;
+ struct mm_struct *mm;
+ int class_0_pending;
+ spinlock_t register_lock;
+
+ u32 stop_code;
+ wait_queue_head_t stop_wq;
+ wait_queue_head_t ibox_wq;
+ wait_queue_head_t wbox_wq;
+
+ char irq_c0[8];
+ char irq_c1[8];
+ char irq_c2[8];
+};
+
+struct spu *spu_alloc(void);
+void spu_free(struct spu *spu);
+int spu_run(struct spu *spu);
+
+size_t spu_wbox_write(struct spu *spu, u32 data);
+size_t spu_ibox_read(struct spu *spu, u32 *data);
+
+/*
+ * This defines the Local Store, Problem Area and Privlege Area of an SPU.
+ */
+
+union mfc_tag_size_class_cmd {
+ struct {
+ u16 mfc_size;
+ u16 mfc_tag;
+ u8 pad;
+ u8 mfc_rclassid;
+ u16 mfc_cmd;
+ } u;
+ struct {
+ u32 mfc_size_tag32;
+ u32 mfc_class_cmd32;
+ } by32;
+ u64 all64;
+};
+
+struct mfc_cq_sr {
+ u64 mfc_cq_data0_RW;
+ u64 mfc_cq_data1_RW;
+ u64 mfc_cq_data2_RW;
+ u64 mfc_cq_data3_RW;
+};
+
+struct spu_problem {
+#define MS_SYNC_PENDING 1L
+ u64 spc_mssync_RW; /* 0x0000 */
+ u8 pad_0x0008_0x3000[0x3000 - 0x0008];
+
+ /* DMA Area */
+ u8 pad_0x3000_0x3004[0x4]; /* 0x3000 */
+ u32 mfc_lsa_W; /* 0x3004 */
+ u64 mfc_ea_W; /* 0x3008 */
+ union mfc_tag_size_class_cmd mfc_union_W; /* 0x3010 */
+ u8 pad_0x3018_0x3104[0xec]; /* 0x3018 */
+ u32 dma_qstatus_R; /* 0x3104 */
+ u8 pad_0x3108_0x3204[0xfc]; /* 0x3108 */
+ u32 dma_querytype_RW; /* 0x3204 */
+ u8 pad_0x3208_0x321c[0x14]; /* 0x3208 */
+ u32 dma_querymask_RW; /* 0x321c */
+ u8 pad_0x3220_0x322c[0xc]; /* 0x3220 */
+ u32 dma_tagstatus_R; /* 0x322c */
+#define DMA_TAGSTATUS_INTR_ANY 1u
+#define DMA_TAGSTATUS_INTR_ALL 2u
+ u8 pad_0x3230_0x4000[0x4000 - 0x3230]; /* 0x3230 */
+
+ /* SPU Control Area */
+ u8 pad_0x4000_0x4004[0x4]; /* 0x4000 */
+ u32 pu_mb_R; /* 0x4004 */
+ u8 pad_0x4008_0x400c[0x4]; /* 0x4008 */
+ u32 spu_mb_W; /* 0x400c */
+ u8 pad_0x4010_0x4014[0x4]; /* 0x4010 */
+ u32 mb_stat_R; /* 0x4014 */
+ u8 pad_0x4018_0x401c[0x4]; /* 0x4018 */
+ u32 spu_runcntl_RW; /* 0x401c */
+#define SPU_RUNCNTL_STOP 0L
+#define SPU_RUNCNTL_RUNNABLE 1L
+ u8 pad_0x4020_0x4024[0x4]; /* 0x4020 */
+ u32 spu_status_R; /* 0x4024 */
+#define SPU_STOP_STATUS_SHIFT 16
+#define SPU_STATUS_STOPPED 0x0
+#define SPU_STATUS_RUNNING 0x1
+#define SPU_STATUS_STOPPED_BY_STOP 0x2
+#define SPU_STATUS_STOPPED_BY_HALT 0x4
+#define SPU_STATUS_WAITING_FOR_CHANNEL 0x8
+#define SPU_STATUS_SINGLE_STEP 0x10
+#define SPU_STATUS_INVALID_INSTR 0x20
+#define SPU_STATUS_INVALID_CH 0x40
+#define SPU_STATUS_ISOLATED_STATE 0x80
+#define SPU_STATUS_ISOLATED_LOAD_STAUTUS 0x200
+#define SPU_STATUS_ISOLATED_EXIT_STAUTUS 0x400
+ u8 pad_0x4028_0x402c[0x4]; /* 0x4028 */
+ u32 spu_spe_R; /* 0x402c */
+ u8 pad_0x4030_0x4034[0x4]; /* 0x4030 */
+ u32 spu_npc_RW; /* 0x4034 */
+ u8 pad_0x4038_0x14000[0x14000 - 0x4038]; /* 0x4038 */
+
+ /* Signal Notification Area */
+ u8 pad_0x14000_0x1400c[0xc]; /* 0x14000 */
+ u32 signal_notify1; /* 0x1400c */
+ u8 pad_0x14010_0x1c00c[0x7ffc]; /* 0x14010 */
+ u32 signal_notify2; /* 0x1c00c */
+} __attribute__ ((aligned(0x20000)));
+
+/* SPU Privilege 2 State Area */
+struct spu_priv2 {
+ /* MFC Registers */
+ u8 pad_0x0000_0x1100[0x1100 - 0x0000]; /* 0x0000 */
+
+ /* SLB Management Registers */
+ u8 pad_0x1100_0x1108[0x8]; /* 0x1100 */
+ u64 slb_index_W; /* 0x1108 */
+#define SLB_INDEX_MASK 0x7L
+ u64 slb_esid_RW; /* 0x1110 */
+ u64 slb_vsid_RW; /* 0x1118 */
+#define SLB_VSID_SUPERVISOR_STATE (0x1ull << 11)
+#define SLB_VSID_SUPERVISOR_STATE_MASK (0x1ull << 11)
+#define SLB_VSID_PROBLEM_STATE (0x1ull << 10)
+#define SLB_VSID_PROBLEM_STATE_MASK (0x1ull << 10)
+#define SLB_VSID_EXECUTE_SEGMENT (0x1ull << 9)
+#define SLB_VSID_NO_EXECUTE_SEGMENT (0x1ull << 9)
+#define SLB_VSID_EXECUTE_SEGMENT_MASK (0x1ull << 9)
+#define SLB_VSID_4K_PAGE (0x0 << 8)
+#define SLB_VSID_LARGE_PAGE (0x1ull << 8)
+#define SLB_VSID_PAGE_SIZE_MASK (0x1ull << 8)
+#define SLB_VSID_CLASS_MASK (0x1ull << 7)
+#define SLB_VSID_VIRTUAL_PAGE_SIZE_MASK (0x1ull << 6)
+ u64 slb_invalidate_entry_W; /* 0x1120 */
+ u64 slb_invalidate_all_W; /* 0x1128 */
+ u8 pad_0x1130_0x2000[0x2000 - 0x1130]; /* 0x1130 */
+
+ /* Context Save / Restore Area */
+ struct mfc_cq_sr spuq[16]; /* 0x2000 */
+ struct mfc_cq_sr puq[8]; /* 0x2200 */
+ u8 pad_0x2300_0x3000[0x3000 - 0x2300]; /* 0x2300 */
+
+ /* MFC Control */
+ u64 mfc_control_RW; /* 0x3000 */
+#define MFC_CNTL_RESUME_DMA_QUEUE (0ull << 0)
+#define MFC_CNTL_SUSPEND_DMA_QUEUE (1ull << 0)
+#define MFC_CNTL_SUSPEND_DMA_QUEUE_MASK (1ull << 0)
+#define MFC_CNTL_NORMAL_DMA_QUEUE_OPERATION (0ull << 8)
+#define MFC_CNTL_SUSPEND_IN_PROGRESS (1ull << 8)
+#define MFC_CNTL_SUSPEND_COMPLETE (3ull << 8)
+#define MFC_CNTL_SUSPEND_DMA_STATUS_MASK (3ull << 8)
+#define MFC_CNTL_DMA_QUEUES_EMPTY (1ull << 14)
+#define MFC_CNTL_DMA_QUEUES_EMPTY_MASK (1ull << 14)
+#define MFC_CNTL_PURGE_DMA_REQUEST (1ull << 15)
+#define MFC_CNTL_PURGE_DMA_IN_PROGRESS (1ull << 24)
+#define MFC_CNTL_PURGE_DMA_COMPLETE (3ull << 24)
+#define MFC_CNTL_PURGE_DMA_STATUS_MASK (3ull << 24)
+#define MFC_CNTL_RESTART_DMA_COMMAND (1ull << 32)
+#define MFC_CNTL_DMA_COMMAND_REISSUE_PENDING (1ull << 32)
+#define MFC_CNTL_DMA_COMMAND_REISSUE_STATUS_MASK (1ull << 32)
+#define MFC_CNTL_MFC_PRIVILEGE_STATE (2ull << 33)
+#define MFC_CNTL_MFC_PROBLEM_STATE (3ull << 33)
+#define MFC_CNTL_MFC_KEY_PROTECTION_STATE_MASK (3ull << 33)
+#define MFC_CNTL_DECREMENTER_HALTED (1ull << 35)
+#define MFC_CNTL_DECREMENTER_RUNNING (1ull << 40)
+#define MFC_CNTL_DECREMENTER_STATUS_MASK (1ull << 40)
+ u8 pad_0x3008_0x4000[0x4000 - 0x3008]; /* 0x3008 */
+
+ /* Interrupt Mailbox */
+ u64 puint_mb_R; /* 0x4000 */
+ u8 pad_0x4008_0x4040[0x4040 - 0x4008]; /* 0x4008 */
+
+ /* SPU Control */
+ u64 spu_privcntl_RW; /* 0x4040 */
+#define SPU_PRIVCNTL_MODE_NORMAL (0x0ull << 0)
+#define SPU_PRIVCNTL_MODE_SINGLE_STEP (0x1ull << 0)
+#define SPU_PRIVCNTL_MODE_MASK (0x1ull << 0)
+#define SPU_PRIVCNTL_NO_ATTENTION_EVENT (0x0ull << 1)
+#define SPU_PRIVCNTL_ATTENTION_EVENT (0x1ull << 1)
+#define SPU_PRIVCNTL_ATTENTION_EVENT_MASK (0x1ull << 1)
+#define SPU_PRIVCNT_LOAD_REQUEST_NORMAL (0x0ull << 2)
+#define SPU_PRIVCNT_LOAD_REQUEST_ENABLE_MASK (0x1ull << 2)
+ u8 pad_0x4048_0x4058[0x10]; /* 0x4048 */
+ u64 spu_lslr_RW; /* 0x4058 */
+ u64 spu_chnlcntptr_RW; /* 0x4060 */
+ u64 spu_chnlcnt_RW; /* 0x4068 */
+ u64 spu_chnldata_RW; /* 0x4070 */
+ u64 spu_cfg_RW; /* 0x4078 */
+ u8 pad_0x4080_0x5000[0x5000 - 0x4080]; /* 0x4080 */
+
+ /* PV2_ImplRegs: Implementation-specific privileged-state 2 regs */
+ u64 spu_pm_trace_tag_status_RW; /* 0x5000 */
+ u64 spu_tag_status_query_RW; /* 0x5008 */
+#define TAG_STATUS_QUERY_CONDITION_BITS (0x3ull << 32)
+#define TAG_STATUS_QUERY_MASK_BITS (0xffffffffull)
+ u64 spu_cmd_buf1_RW; /* 0x5010 */
+#define SPU_COMMAND_BUFFER_1_LSA_BITS (0x7ffffull << 32)
+#define SPU_COMMAND_BUFFER_1_EAH_BITS (0xffffffffull)
+ u64 spu_cmd_buf2_RW; /* 0x5018 */
+#define SPU_COMMAND_BUFFER_2_EAL_BITS ((0xffffffffull) << 32)
+#define SPU_COMMAND_BUFFER_2_TS_BITS (0xffffull << 16)
+#define SPU_COMMAND_BUFFER_2_TAG_BITS (0x3full)
+ u64 spu_atomic_status_RW; /* 0x5020 */
+} __attribute__ ((aligned(0x20000)));
+
+/* SPU Privilege 1 State Area */
+struct spu_priv1 {
+ /* Control and Configuration Area */
+ u64 mfc_sr1_RW; /* 0x000 */
+#define MFC_STATE1_LOCAL_STORAGE_DECODE_MASK 0x01ull
+#define MFC_STATE1_BUS_TLBIE_MASK 0x02ull
+#define MFC_STATE1_REAL_MODE_OFFSET_ENABLE_MASK 0x04ull
+#define MFC_STATE1_PROBLEM_STATE_MASK 0x08ull
+#define MFC_STATE1_RELOCATE_MASK 0x10ull
+#define MFC_STATE1_MASTER_RUN_CONTROL_MASK 0x20ull
+ u64 mfc_lpid_RW; /* 0x008 */
+ u64 spu_idr_RW; /* 0x010 */
+ u64 mfc_vr_RO; /* 0x018 */
+#define MFC_VERSION_BITS (0xffff << 16)
+#define MFC_REVISION_BITS (0xffff)
+#define MFC_GET_VERSION_BITS(vr) (((vr) & MFC_VERSION_BITS) >> 16)
+#define MFC_GET_REVISION_BITS(vr) ((vr) & MFC_REVISION_BITS)
+ u64 spu_vr_RO; /* 0x020 */
+#define SPU_VERSION_BITS (0xffff << 16)
+#define SPU_REVISION_BITS (0xffff)
+#define SPU_GET_VERSION_BITS(vr) (vr & SPU_VERSION_BITS) >> 16
+#define SPU_GET_REVISION_BITS(vr) (vr & SPU_REVISION_BITS)
+ u8 pad_0x28_0x100[0x100 - 0x28]; /* 0x28 */
+
+
+ /* Interrupt Area */
+ u64 int_mask_class0_RW; /* 0x100 */
+#define CLASS0_ENABLE_DMA_ALIGNMENT_INTR 0x1L
+#define CLASS0_ENABLE_INVALID_DMA_COMMAND_INTR 0x2L
+#define CLASS0_ENABLE_SPU_ERROR_INTR 0x4L
+#define CLASS0_ENABLE_MFC_FIR_INTR 0x8L
+ u64 int_mask_class1_RW; /* 0x108 */
+#define CLASS1_ENABLE_SEGMENT_FAULT_INTR 0x1L
+#define CLASS1_ENABLE_STORAGE_FAULT_INTR 0x2L
+#define CLASS1_ENABLE_LS_COMPARE_SUSPEND_ON_GET_INTR 0x4L
+#define CLASS1_ENABLE_LS_COMPARE_SUSPEND_ON_PUT_INTR 0x8L
+ u64 int_mask_class2_RW; /* 0x110 */
+#define CLASS2_ENABLE_MAILBOX_INTR 0x1L
+#define CLASS2_ENABLE_SPU_STOP_INTR 0x2L
+#define CLASS2_ENABLE_SPU_HALT_INTR 0x4L
+#define CLASS2_ENABLE_SPU_DMA_TAG_GROUP_COMPLETE_INTR 0x8L
+ u8 pad_0x118_0x140[0x28]; /* 0x118 */
+ u64 int_stat_class0_RW; /* 0x140 */
+ u64 int_stat_class1_RW; /* 0x148 */
+ u64 int_stat_class2_RW; /* 0x150 */
+ u8 pad_0x158_0x180[0x28]; /* 0x158 */
+ u64 int_route_RW; /* 0x180 */
+
+ /* Interrupt Routing */
+ u8 pad_0x188_0x200[0x200 - 0x188]; /* 0x188 */
+
+ /* Atomic Unit Control Area */
+ u64 mfc_atomic_flush_RW; /* 0x200 */
+#define mfc_atomic_flush_enable 0x1L
+ u8 pad_0x208_0x280[0x78]; /* 0x208 */
+ u64 resource_allocation_groupID_RW; /* 0x280 */
+ u64 resource_allocation_enable_RW; /* 0x288 */
+ u8 pad_0x290_0x3c8[0x3c8 - 0x290]; /* 0x290 */
+
+ /* SPU_Cache_ImplRegs: Implementation-dependent cache registers */
+
+ u64 smf_sbi_signal_sel; /* 0x3c8 */
+#define smf_sbi_mask_lsb 56
+#define smf_sbi_shift (63 - smf_sbi_mask_lsb)
+#define smf_sbi_mask (0x301LL << smf_sbi_shift)
+#define smf_sbi_bus0_bits (0x001LL << smf_sbi_shift)
+#define smf_sbi_bus2_bits (0x100LL << smf_sbi_shift)
+#define smf_sbi2_bus0_bits (0x201LL << smf_sbi_shift)
+#define smf_sbi2_bus2_bits (0x300LL << smf_sbi_shift)
+ u64 smf_ato_signal_sel; /* 0x3d0 */
+#define smf_ato_mask_lsb 35
+#define smf_ato_shift (63 - smf_ato_mask_lsb)
+#define smf_ato_mask (0x3LL << smf_ato_shift)
+#define smf_ato_bus0_bits (0x2LL << smf_ato_shift)
+#define smf_ato_bus2_bits (0x1LL << smf_ato_shift)
+ u8 pad_0x3d8_0x400[0x400 - 0x3d8]; /* 0x3d8 */
+
+ /* TLB Management Registers */
+ u64 mfc_sdr_RW; /* 0x400 */
+ u8 pad_0x408_0x500[0xf8]; /* 0x408 */
+ u64 tlb_index_hint_RO; /* 0x500 */
+ u64 tlb_index_W; /* 0x508 */
+ u64 tlb_vpn_RW; /* 0x510 */
+ u64 tlb_rpn_RW; /* 0x518 */
+ u8 pad_0x520_0x540[0x20]; /* 0x520 */
+ u64 tlb_invalidate_entry_W; /* 0x540 */
+ u64 tlb_invalidate_all_W; /* 0x548 */
+ u8 pad_0x550_0x580[0x580 - 0x550]; /* 0x550 */
+
+ /* SPU_MMU_ImplRegs: Implementation-dependent MMU registers */
+ u64 smm_hid; /* 0x580 */
+#define PAGE_SIZE_MASK 0xf000000000000000ull
+#define PAGE_SIZE_16MB_64KB 0x2000000000000000ull
+ u8 pad_0x588_0x600[0x600 - 0x588]; /* 0x588 */
+
+ /* MFC Status/Control Area */
+ u64 mfc_accr_RW; /* 0x600 */
+#define MFC_ACCR_EA_ACCESS_GET (1 << 0)
+#define MFC_ACCR_EA_ACCESS_PUT (1 << 1)
+#define MFC_ACCR_LS_ACCESS_GET (1 << 3)
+#define MFC_ACCR_LS_ACCESS_PUT (1 << 4)
+ u8 pad_0x608_0x610[0x8]; /* 0x608 */
+ u64 mfc_dsisr_RW; /* 0x610 */
+#define MFC_DSISR_PTE_NOT_FOUND (1 << 30)
+#define MFC_DSISR_ACCESS_DENIED (1 << 27)
+#define MFC_DSISR_ATOMIC (1 << 26)
+#define MFC_DSISR_ACCESS_PUT (1 << 25)
+#define MFC_DSISR_ADDR_MATCH (1 << 22)
+#define MFC_DSISR_LS (1 << 17)
+#define MFC_DSISR_L (1 << 16)
+#define MFC_DSISR_ADDRESS_OVERFLOW (1 << 0)
+ u8 pad_0x618_0x620[0x8]; /* 0x618 */
+ u64 mfc_dar_RW; /* 0x620 */
+ u8 pad_0x628_0x700[0x700 - 0x628]; /* 0x628 */
+
+ /* Replacement Management Table (RMT) Area */
+ u64 rmt_index_RW; /* 0x700 */
+ u8 pad_0x708_0x710[0x8]; /* 0x708 */
+ u64 rmt_data1_RW; /* 0x710 */
+ u8 pad_0x718_0x800[0x800 - 0x718]; /* 0x718 */
+
+ /* Control/Configuration Registers */
+ u64 mfc_dsir_R; /* 0x800 */
+#define MFC_DSIR_Q (1 << 31)
+#define MFC_DSIR_SPU_QUEUE MFC_DSIR_Q
+ u64 mfc_lsacr_RW; /* 0x808 */
+#define MFC_LSACR_COMPARE_MASK ((~0ull) << 32)
+#define MFC_LSACR_COMPARE_ADDR ((~0ull) >> 32)
+ u64 mfc_lscrr_R; /* 0x810 */
+#define MFC_LSCRR_Q (1 << 31)
+#define MFC_LSCRR_SPU_QUEUE MFC_LSCRR_Q
+#define MFC_LSCRR_QI_SHIFT 32
+#define MFC_LSCRR_QI_MASK ((~0ull) << MFC_LSCRR_QI_SHIFT)
+ u8 pad_0x818_0x820[0x8]; /* 0x818 */
+ u64 mfc_tclass_id_RW; /* 0x820 */
+#define MFC_TCLASS_ID_ENABLE (1L << 0L)
+#define MFC_TCLASS_SLOT2_ENABLE (1L << 5L)
+#define MFC_TCLASS_SLOT1_ENABLE (1L << 6L)
+#define MFC_TCLASS_SLOT0_ENABLE (1L << 7L)
+#define MFC_TCLASS_QUOTA_2_SHIFT 8L
+#define MFC_TCLASS_QUOTA_1_SHIFT 16L
+#define MFC_TCLASS_QUOTA_0_SHIFT 24L
+#define MFC_TCLASS_QUOTA_2_MASK (0x1FL << MFC_TCLASS_QUOTA_2_SHIFT)
+#define MFC_TCLASS_QUOTA_1_MASK (0x1FL << MFC_TCLASS_QUOTA_1_SHIFT)
+#define MFC_TCLASS_QUOTA_0_MASK (0x1FL << MFC_TCLASS_QUOTA_0_SHIFT)
+ u8 pad_0x828_0x900[0x900 - 0x828]; /* 0x828 */
+
+ /* Real Mode Support Registers */
+ u64 mfc_rm_boundary; /* 0x900 */
+ u8 pad_0x908_0x938[0x30]; /* 0x908 */
+ u64 smf_dma_signal_sel; /* 0x938 */
+#define mfc_dma1_mask_lsb 41
+#define mfc_dma1_shift (63 - mfc_dma1_mask_lsb)
+#define mfc_dma1_mask (0x3LL << mfc_dma1_shift)
+#define mfc_dma1_bits (0x1LL << mfc_dma1_shift)
+#define mfc_dma2_mask_lsb 43
+#define mfc_dma2_shift (63 - mfc_dma2_mask_lsb)
+#define mfc_dma2_mask (0x3LL << mfc_dma2_shift)
+#define mfc_dma2_bits (0x1LL << mfc_dma2_shift)
+ u8 pad_0x940_0xa38[0xf8]; /* 0x940 */
+ u64 smm_signal_sel; /* 0xa38 */
+#define smm_sig_mask_lsb 12
+#define smm_sig_shift (63 - smm_sig_mask_lsb)
+#define smm_sig_mask (0x3LL << smm_sig_shift)
+#define smm_sig_bus0_bits (0x2LL << smm_sig_shift)
+#define smm_sig_bus2_bits (0x1LL << smm_sig_shift)
+ u8 pad_0xa40_0xc00[0xc00 - 0xa40]; /* 0xa40 */
+
+ /* DMA Command Error Area */
+ u64 mfc_cer_R; /* 0xc00 */
+#define MFC_CER_Q (1 << 31)
+#define MFC_CER_SPU_QUEUE MFC_CER_Q
+ u8 pad_0xc08_0x1000[0x1000 - 0xc08]; /* 0xc08 */
+
+ /* PV1_ImplRegs: Implementation-dependent privileged-state 1 regs */
+ /* DMA Command Error Area */
+ u64 spu_ecc_cntl_RW; /* 0x1000 */
+#define SPU_ECC_CNTL_E (1ull << 0ull)
+#define SPU_ECC_CNTL_ENABLE SPU_ECC_CNTL_E
+#define SPU_ECC_CNTL_DISABLE (~SPU_ECC_CNTL_E & 1L)
+#define SPU_ECC_CNTL_S (1ull << 1ull)
+#define SPU_ECC_STOP_AFTER_ERROR SPU_ECC_CNTL_S
+#define SPU_ECC_CONTINUE_AFTER_ERROR (~SPU_ECC_CNTL_S & 2L)
+#define SPU_ECC_CNTL_B (1ull << 2ull)
+#define SPU_ECC_BACKGROUND_ENABLE SPU_ECC_CNTL_B
+#define SPU_ECC_BACKGROUND_DISABLE (~SPU_ECC_CNTL_B & 4L)
+#define SPU_ECC_CNTL_I_SHIFT 3ull
+#define SPU_ECC_CNTL_I_MASK (3ull << SPU_ECC_CNTL_I_SHIFT)
+#define SPU_ECC_WRITE_ALWAYS (~SPU_ECC_CNTL_I & 12L)
+#define SPU_ECC_WRITE_CORRECTABLE (1ull << SPU_ECC_CNTL_I_SHIFT)
+#define SPU_ECC_WRITE_UNCORRECTABLE (3ull << SPU_ECC_CNTL_I_SHIFT)
+#define SPU_ECC_CNTL_D (1ull << 5ull)
+#define SPU_ECC_DETECTION_ENABLE SPU_ECC_CNTL_D
+#define SPU_ECC_DETECTION_DISABLE (~SPU_ECC_CNTL_D & 32L)
+ u64 spu_ecc_stat_RW; /* 0x1008 */
+#define SPU_ECC_CORRECTED_ERROR (1ull << 0ul)
+#define SPU_ECC_UNCORRECTED_ERROR (1ull << 1ul)
+#define SPU_ECC_SCRUB_COMPLETE (1ull << 2ul)
+#define SPU_ECC_SCRUB_IN_PROGRESS (1ull << 3ul)
+#define SPU_ECC_INSTRUCTION_ERROR (1ull << 4ul)
+#define SPU_ECC_DATA_ERROR (1ull << 5ul)
+#define SPU_ECC_DMA_ERROR (1ull << 6ul)
+#define SPU_ECC_STATUS_CNT_MASK (256ull << 8)
+ u64 spu_ecc_addr_RW; /* 0x1010 */
+ u64 spu_err_mask_RW; /* 0x1018 */
+#define SPU_ERR_ILLEGAL_INSTR (1ull << 0ul)
+#define SPU_ERR_ILLEGAL_CHANNEL (1ull << 1ul)
+ u8 pad_0x1020_0x1028[0x1028 - 0x1020]; /* 0x1020 */
+
+ /* SPU Debug-Trace Bus (DTB) Selection Registers */
+ u64 spu_trig0_sel; /* 0x1028 */
+ u64 spu_trig1_sel; /* 0x1030 */
+ u64 spu_trig2_sel; /* 0x1038 */
+ u64 spu_trig3_sel; /* 0x1040 */
+ u64 spu_trace_sel; /* 0x1048 */
+#define spu_trace_sel_mask 0x1f1fLL
+#define spu_trace_sel_bus0_bits 0x1000LL
+#define spu_trace_sel_bus2_bits 0x0010LL
+ u64 spu_event0_sel; /* 0x1050 */
+ u64 spu_event1_sel; /* 0x1058 */
+ u64 spu_event2_sel; /* 0x1060 */
+ u64 spu_event3_sel; /* 0x1068 */
+ u64 spu_trace_cntl; /* 0x1070 */
+} __attribute__ ((aligned(0x2000)));
+
+#endif
Index: linux-cg/mm/memory.c
===================================================================
--- linux-cg.orig/mm/memory.c
+++ linux-cg/mm/memory.c
@@ -2074,6 +2074,8 @@ int __handle_mm_fault(struct mm_struct *
return VM_FAULT_OOM;
}
+EXPORT_SYMBOL_GPL(__handle_mm_fault);
+
#ifndef __PAGETABLE_PUD_FOLDED
/*
* Allocate page upper directory.
Index: linux-cg/arch/ppc/kernel/ppc_ksyms.c
===================================================================
--- linux-cg.orig/arch/ppc/kernel/ppc_ksyms.c
+++ linux-cg/arch/ppc/kernel/ppc_ksyms.c
@@ -322,7 +322,6 @@ EXPORT_SYMBOL(__res);
EXPORT_SYMBOL(next_mmu_context);
EXPORT_SYMBOL(set_context);
-EXPORT_SYMBOL_GPL(__handle_mm_fault); /* For MOL */
EXPORT_SYMBOL(disarm_decr);
#ifdef CONFIG_PPC_STD_MMU
extern long mol_trampoline;
--
^ permalink raw reply [flat|nested] 14+ messages in thread* [patch 02/11] spufs: switchable spu contexts
2005-09-16 12:16 [patch 00/11] spufs: latest spufs snapshot for 2.6.14-rc1 Arnd Bergmann
2005-09-16 6:40 ` spufs: User space thread library Arnd Bergmann
2005-09-16 12:16 ` [patch 01/11] spufs: The SPU file system, base Arnd Bergmann
@ 2005-09-16 12:16 ` Arnd Bergmann
2005-09-16 12:16 ` [patch 03/11] spufs: kernel-side context switch code Arnd Bergmann
` (8 subsequent siblings)
11 siblings, 0 replies; 14+ messages in thread
From: Arnd Bergmann @ 2005-09-16 12:16 UTC (permalink / raw)
To: ppc64-dev
Cc: linux-kernel, Paul Mackerras, Benjamin Herrenschmidt,
jordi_caubet, Hiroyuki Machida, Geoff Levand
[-- Attachment #1: spufs-context-4.diff --]
[-- Type: text/plain, Size: 42615 bytes --]
Add some infrastructure for saving and restoring the context of an
SPE. This patch creates a new structure that can hold the whole
state of a physical SPE in memory. It also contains code that
avoids races during the context switch and the binary code that
is loaded to the SPU in order to access its registers.
The actual PPE- and SPE-side context switch code are two separate
patches.
From: Mark Nutter <mnutter@us.ibm.com>
Signed-off-by: Arnd Bergmann <arndb@de.ibm.com>
--
arch/ppc64/kernel/spu_base.c | 27 ++
fs/spufs/Makefile | 4
fs/spufs/context.c | 18 +
fs/spufs/spu_restore_dump.h_shipped | 342 ++++++++++++++++++++++++++++++++++++
fs/spufs/spu_save_dump.h_shipped | 290 ++++++++++++++++++++++++++++++
fs/spufs/spufs.h | 2
fs/spufs/switch.c | 174 ++++++++++++++++++
include/asm-ppc64/spu.h | 76 ++++++++
include/asm-ppc64/spu_csa.h | 256 ++++++++++++++++++++++++++
9 files changed, 1185 insertions(+), 4 deletions(-)
Index: linux-cg/arch/ppc64/kernel/spu_base.c
===================================================================
--- linux-cg.orig/arch/ppc64/kernel/spu_base.c
+++ linux-cg/arch/ppc64/kernel/spu_base.c
@@ -62,7 +62,9 @@ static int __spu_trap_error(struct spu *
static void spu_restart_dma(struct spu *spu)
{
struct spu_priv2 __iomem *priv2 = spu->priv2;
- out_be64(&priv2->mfc_control_RW, MFC_CNTL_RESTART_DMA_COMMAND);
+
+ if (!test_bit(SPU_CONTEXT_SWITCH_PENDING_nr, &spu->flags))
+ out_be64(&priv2->mfc_control_RW, MFC_CNTL_RESTART_DMA_COMMAND);
}
static int __spu_trap_data_seg(struct spu *spu, unsigned long ea)
@@ -72,6 +74,11 @@ static int __spu_trap_data_seg(struct sp
pr_debug("%s\n", __FUNCTION__);
+ if (test_bit(SPU_CONTEXT_SWITCH_ACTIVE_nr, &spu->flags)) {
+ printk("%s: invalid access during switch!\n", __func__);
+ return 1;
+ }
+
if (REGION_ID(ea) != USER_REGION_ID) {
pr_debug("invalid region access at %016lx\n", ea);
return 1;
@@ -98,6 +105,7 @@ static int __spu_trap_data_seg(struct sp
return 0;
}
+extern int hash_page(unsigned long ea, unsigned long access, unsigned long trap); //XXX
static int __spu_trap_data_map(struct spu *spu, unsigned long ea)
{
unsigned long dsisr;
@@ -107,8 +115,21 @@ static int __spu_trap_data_map(struct sp
priv1 = spu->priv1;
dsisr = in_be64(&priv1->mfc_dsisr_RW);
- wake_up(&spu->stop_wq);
+ /* Handle kernel space hash faults immediately.
+ User hash faults need to be deferred to process context. */
+ if ((dsisr & MFC_DSISR_PTE_NOT_FOUND)
+ && REGION_ID(ea) != USER_REGION_ID
+ && hash_page(ea, _PAGE_PRESENT, 0x300) == 0) {
+ spu_restart_dma(spu);
+ return 0;
+ }
+
+ if (test_bit(SPU_CONTEXT_SWITCH_ACTIVE_nr, &spu->flags)) {
+ printk("%s: invalid access during switch!\n", __func__);
+ return 1;
+ }
+ wake_up(&spu->stop_wq);
return 0;
}
@@ -378,7 +399,6 @@ void spu_free(struct spu *spu)
}
EXPORT_SYMBOL(spu_free);
-extern int hash_page(unsigned long ea, unsigned long access, unsigned long trap); //XXX
static int spu_handle_mm_fault(struct spu *spu)
{
struct spu_priv1 __iomem *priv1;
@@ -646,6 +666,7 @@ static int __init create_spu(struct devi
spu->slb_replace = 0;
spu->mm = NULL;
spu->class_0_pending = 0;
+ spu->flags = 0UL;
spin_lock_init(&spu->register_lock);
out_be64(&spu->priv1->mfc_sdr_RW, mfspr(SPRN_SDR1));
Index: linux-cg/fs/spufs/Makefile
===================================================================
--- linux-cg.orig/fs/spufs/Makefile
+++ linux-cg/fs/spufs/Makefile
@@ -1,3 +1,5 @@
obj-$(CONFIG_SPU_FS) += spufs.o
-spufs-y += inode.o file.o context.o
+spufs-y += inode.o file.o context.o switch.o
+
+$(obj)/switch.o: $(obj)/spu_save_dump.h $(obj)/spu_restore_dump.h
Index: linux-cg/fs/spufs/context.c
===================================================================
--- linux-cg.orig/fs/spufs/context.c
+++ linux-cg/fs/spufs/context.c
@@ -22,6 +22,7 @@
#include <linux/slab.h>
#include <asm/spu.h>
+#include <asm/spu_csa.h>
#include "spufs.h"
struct spu_context *alloc_spu_context(void)
@@ -30,9 +31,25 @@ struct spu_context *alloc_spu_context(vo
ctx = kmalloc(sizeof *ctx, GFP_KERNEL);
if (!ctx)
goto out;
+ /* Future enhancement: do not call spu_alloc()
+ * here. This step should be deferred until
+ * spu_run()!!
+ *
+ * More work needs to be done to read(),
+ * write(), mmap(), etc., so that operations
+ * are performed on CSA when the context is
+ * not currently being run. In this way we
+ * can support arbitrarily large number of
+ * entries in /spu, allow state queries, etc.
+ */
ctx->spu = spu_alloc();
if (!ctx->spu)
goto out_free;
+ spu_init_csa(&ctx->csa);
+ if (!ctx->csa.lscsa) {
+ spu_free(ctx->spu);
+ goto out_free;
+ }
init_rwsem(&ctx->backing_sema);
spin_lock_init(&ctx->mmio_lock);
kref_init(&ctx->kref);
@@ -50,6 +67,7 @@ void destroy_spu_context(struct kref *kr
ctx = container_of(kref, struct spu_context, kref);
if (ctx->spu)
spu_free(ctx->spu);
+ spu_fini_csa(&ctx->csa);
kfree(ctx);
}
Index: linux-cg/fs/spufs/spu_restore_dump.h_shipped
===================================================================
--- /dev/null
+++ linux-cg/fs/spufs/spu_restore_dump.h_shipped
@@ -0,0 +1,231 @@
+/*
+ * spu_restore_dump.h: Copyright (C) 2005 IBM.
+ * Hex-dump auto generated from spu_restore.c.
+ * Do not edit!
+ */
+static unsigned int spu_restore_code[] __page_aligned = {
+0x40800000, 0x409ff801, 0x24000080, 0x24fd8081,
+0x1cd80081, 0x33001180, 0x42030003, 0x33800284,
+0x1c010204, 0x40200000, 0x40200000, 0x40200000,
+0x34000190, 0x34004191, 0x34008192, 0x3400c193,
+0x141fc205, 0x23fffd84, 0x1c100183, 0x217ffa85,
+0x3080a000, 0x3080a201, 0x3080a402, 0x3080a603,
+0x3080a804, 0x3080aa05, 0x3080ac06, 0x3080ae07,
+0x3080b008, 0x3080b209, 0x3080b40a, 0x3080b60b,
+0x3080b80c, 0x3080ba0d, 0x3080bc0e, 0x3080be0f,
+0x00003ffc, 0x00000000, 0x00000000, 0x00000000,
+0x01a00182, 0x3ec00083, 0xb0a14103, 0x01a00204,
+0x3ec10082, 0x4202800e, 0x04000703, 0xb0a14202,
+0x21a00803, 0x3fbf028d, 0x3f20068d, 0x3fbe0682,
+0x3fe30102, 0x21a00882, 0x3f82028f, 0x3fe3078f,
+0x3fbf0784, 0x3f200204, 0x3fbe0204, 0x3fe30204,
+0x04000203, 0x21a00903, 0x40848002, 0x21a00982,
+0x40800003, 0x21a00a03, 0x40802002, 0x21a00a82,
+0x21a00083, 0x40800082, 0x21a00b02, 0x10002818,
+0x40a80002, 0x32800007, 0x4207000c, 0x18008208,
+0x40a0000b, 0x4080020a, 0x40800709, 0x00200000,
+0x42070002, 0x3ac30384, 0x1cffc489, 0x00200000,
+0x18008383, 0x38830382, 0x4cffc486, 0x3ac28185,
+0xb0408584, 0x28830382, 0x1c020387, 0x38828182,
+0xb0408405, 0x1802c408, 0x28828182, 0x217ff886,
+0x04000583, 0x21a00803, 0x3fbe0682, 0x3fe30102,
+0x04000106, 0x21a00886, 0x04000603, 0x21a00903,
+0x40803c02, 0x21a00982, 0x40800003, 0x04000184,
+0x21a00a04, 0x40802202, 0x21a00a82, 0x42028005,
+0x34208702, 0x21002282, 0x21a00804, 0x21a00886,
+0x3fbf0782, 0x3f200102, 0x3fbe0102, 0x3fe30102,
+0x21a00902, 0x40804003, 0x21a00983, 0x21a00a04,
+0x40805a02, 0x21a00a82, 0x40800083, 0x21a00b83,
+0x01a00c02, 0x01a00d83, 0x3420c282, 0x21a00e02,
+0x34210283, 0x21a00f03, 0x34200284, 0x77400200,
+0x3421c282, 0x21a00702, 0x34218283, 0x21a00083,
+0x34214282, 0x21a00b02, 0x4200480c, 0x00200000,
+0x1c010286, 0x34220284, 0x34220302, 0x0f608203,
+0x5c024204, 0x3b81810b, 0x42013c02, 0x00200000,
+0x18008185, 0x38808183, 0x3b814182, 0x21004e84,
+0x4020007f, 0x35000100, 0x000004e0, 0x000002a0,
+0x000002e8, 0x00000428, 0x00000360, 0x000002e8,
+0x000004a0, 0x00000468, 0x000003c8, 0x00000360,
+0x409ffe02, 0x30801203, 0x40800204, 0x3ec40085,
+0x10009c09, 0x3ac10606, 0xb060c105, 0x4020007f,
+0x4020007f, 0x20801203, 0x38810602, 0xb0408586,
+0x28810602, 0x32004180, 0x34204702, 0x21a00382,
+0x4020007f, 0x327fdc80, 0x409ffe02, 0x30801203,
+0x40800204, 0x3ec40087, 0x40800405, 0x00200000,
+0x40800606, 0x3ac10608, 0x3ac14609, 0x3ac1860a,
+0xb060c107, 0x20801203, 0x41004003, 0x38810602,
+0x4020007f, 0xb0408188, 0x4020007f, 0x28810602,
+0x41201002, 0x38814603, 0x10009c09, 0xb060c109,
+0x4020007f, 0x28814603, 0x41193f83, 0x38818602,
+0x60ffc003, 0xb040818a, 0x28818602, 0x32003080,
+0x409ffe02, 0x30801203, 0x40800204, 0x3ec40087,
+0x41201008, 0x10009c14, 0x40800405, 0x3ac10609,
+0x40800606, 0x3ac1460a, 0xb060c107, 0x3ac1860b,
+0x20801203, 0x38810602, 0xb0408409, 0x28810602,
+0x38814603, 0xb060c40a, 0x4020007f, 0x28814603,
+0x41193f83, 0x38818602, 0x60ffc003, 0xb040818b,
+0x28818602, 0x32002380, 0x409ffe02, 0x30801204,
+0x40800205, 0x3ec40083, 0x40800406, 0x3ac14607,
+0x3ac18608, 0xb0810103, 0x41004002, 0x20801204,
+0x4020007f, 0x38814603, 0x10009c0b, 0xb060c107,
+0x4020007f, 0x4020007f, 0x28814603, 0x38818602,
+0x4020007f, 0x4020007f, 0xb0408588, 0x28818602,
+0x4020007f, 0x32001780, 0x409ffe02, 0x1000640e,
+0x40800204, 0x30801203, 0x40800405, 0x3ec40087,
+0x40800606, 0x3ac10608, 0x3ac14609, 0x3ac1860a,
+0xb060c107, 0x20801203, 0x413d8003, 0x38810602,
+0x4020007f, 0x327fd780, 0x409ffe02, 0x10007f0c,
+0x40800205, 0x30801204, 0x40800406, 0x3ec40083,
+0x3ac14607, 0x3ac18608, 0xb0810103, 0x413d8002,
+0x20801204, 0x38814603, 0x4020007f, 0x327feb80,
+0x409ffe02, 0x30801203, 0x40800204, 0x3ec40087,
+0x40800405, 0x1000650a, 0x40800606, 0x3ac10608,
+0x3ac14609, 0x3ac1860a, 0xb060c107, 0x20801203,
+0x38810602, 0xb0408588, 0x4020007f, 0x327fc980,
+0x00400000, 0x40800003, 0x4020007f, 0x35000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+};
Index: linux-cg/fs/spufs/spu_save_dump.h_shipped
===================================================================
--- /dev/null
+++ linux-cg/fs/spufs/spu_save_dump.h_shipped
@@ -0,0 +1,191 @@
+/*
+ * spu_save_dump.h: Copyright (C) 2005 IBM.
+ * Hex-dump auto generated from spu_save.c.
+ * Do not edit!
+ */
+static unsigned int spu_save_code[] __page_aligned = {
+0x20805000, 0x20805201, 0x20805402, 0x20805603,
+0x20805804, 0x20805a05, 0x20805c06, 0x20805e07,
+0x20806008, 0x20806209, 0x2080640a, 0x2080660b,
+0x2080680c, 0x20806a0d, 0x20806c0e, 0x20806e0f,
+0x4201c003, 0x33800184, 0x1c010204, 0x40200000,
+0x24000190, 0x24004191, 0x24008192, 0x2400c193,
+0x141fc205, 0x23fffd84, 0x1c100183, 0x217ffb85,
+0x40800000, 0x409ff801, 0x24000080, 0x24fd8081,
+0x1cd80081, 0x33000180, 0x00000000, 0x00000000,
+0x01a00182, 0x3ec00083, 0xb1c38103, 0x01a00204,
+0x3ec10082, 0x4201400d, 0xb1c38202, 0x01a00583,
+0x34218682, 0x3ed80684, 0xb0408184, 0x24218682,
+0x01a00603, 0x00200000, 0x34214682, 0x3ed40684,
+0xb0408184, 0x40800003, 0x24214682, 0x21a00083,
+0x40800082, 0x21a00b02, 0x4020007f, 0x1000251e,
+0x40a80002, 0x32800008, 0x4205c00c, 0x00200000,
+0x40a0000b, 0x3f82070f, 0x4080020a, 0x40800709,
+0x3fe3078f, 0x3fbf0783, 0x3f200183, 0x3fbe0183,
+0x3fe30187, 0x18008387, 0x4205c002, 0x3ac30404,
+0x1cffc489, 0x00200000, 0x18008403, 0x38830402,
+0x4cffc486, 0x3ac28185, 0xb0408584, 0x28830402,
+0x1c020408, 0x38828182, 0xb0408385, 0x1802c387,
+0x28828182, 0x217ff886, 0x04000582, 0x32800007,
+0x21a00802, 0x3fbf0705, 0x3f200285, 0x3fbe0285,
+0x3fe30285, 0x21a00885, 0x04000603, 0x21a00903,
+0x40803c02, 0x21a00982, 0x04000386, 0x21a00a06,
+0x40801202, 0x21a00a82, 0x73000003, 0x24200683,
+0x01a00404, 0x00200000, 0x34204682, 0x3ec40683,
+0xb0408203, 0x24204682, 0x01a00783, 0x00200000,
+0x3421c682, 0x3edc0684, 0xb0408184, 0x2421c682,
+0x21a00806, 0x21a00885, 0x3fbf0784, 0x3f200204,
+0x3fbe0204, 0x3fe30204, 0x21a00904, 0x40804002,
+0x21a00982, 0x21a00a06, 0x40805a02, 0x21a00a82,
+0x04000683, 0x21a00803, 0x21a00885, 0x21a00904,
+0x40848002, 0x21a00982, 0x21a00a06, 0x40801002,
+0x21a00a82, 0x21a00a06, 0x40806602, 0x00200000,
+0x35800009, 0x21a00a82, 0x40800083, 0x21a00b83,
+0x01a00c02, 0x01a00d83, 0x00003ffb, 0x40800003,
+0x4020007f, 0x35000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+0x00000000, 0x00000000, 0x00000000, 0x00000000,
+};
Index: linux-cg/fs/spufs/spufs.h
===================================================================
--- linux-cg.orig/fs/spufs/spufs.h
+++ linux-cg/fs/spufs/spufs.h
@@ -28,6 +28,7 @@
#include <linux/fs.h>
#include <asm/spu.h>
+#include <asm/spu_csa.h>
/* The magic number for our file system */
enum {
@@ -36,6 +37,7 @@ enum {
struct spu_context {
struct spu *spu; /* pointer to a physical SPU */
+ struct spu_state csa; /* SPU context save area. */
struct rw_semaphore backing_sema; /* protects the above */
spinlock_t mmio_lock; /* protects mmio access */
Index: linux-cg/fs/spufs/switch.c
===================================================================
--- /dev/null
+++ linux-cg/fs/spufs/switch.c
@@ -0,0 +1,174 @@
+/*
+ * spu_switch.c
+ *
+ * (C) Copyright IBM Corp. 2005
+ *
+ * Author: Mark Nutter <mnutter@us.ibm.com>
+ *
+ * Host-side part of SPU context switch sequence outlined in
+ * Synergistic Processor Element, Book IV.
+ *
+ * A fully premptive switch of an SPE is very expensive in terms
+ * of time and system resources. SPE Book IV indicates that SPE
+ * allocation should follow a "serially reusable device" model,
+ * in which the SPE is assigned a task until it completes. When
+ * this is not possible, this sequence may be used to premptively
+ * save, and then later (optionally) restore the context of a
+ * program executing on an SPE.
+ *
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2, or (at your option)
+ * any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+ */
+
+#include <linux/config.h>
+#include <linux/module.h>
+#include <linux/errno.h>
+#include <linux/sched.h>
+#include <linux/kernel.h>
+#include <linux/mm.h>
+#include <linux/vmalloc.h>
+#include <linux/smp.h>
+#include <linux/smp_lock.h>
+#include <linux/stddef.h>
+#include <linux/unistd.h>
+
+#include <asm/io.h>
+#include <asm/spu.h>
+#include <asm/spu_csa.h>
+#include <asm/mmu_context.h>
+
+#include "spu_save_dump.h"
+#include "spu_restore_dump.h"
+
+/**
+ * spu_save - SPU context save, with locking.
+ * @prev: pointer to SPU context save area, to be saved.
+ * @spu: pointer to SPU iomem structure.
+ *
+ * Acquire locks, perform the save operation then return.
+ */
+int spu_save(struct spu_state *prev, struct spu *spu)
+{
+ /* XXX missing */
+
+ return 0;
+}
+
+/**
+ * spu_restore - SPU context restore, with harvest and locking.
+ * @new: pointer to SPU context save area, to be restored.
+ * @spu: pointer to SPU iomem structure.
+ *
+ * Perform harvest + restore, as we may not be coming
+ * from a previous succesful save operation, and the
+ * hardware state is unknown.
+ */
+int spu_restore(struct spu_state *new, struct spu *spu)
+{
+ /* XXX missing */
+
+ return 0;
+}
+
+/**
+ * spu_switch - SPU context switch (save + restore).
+ * @prev: pointer to SPU context save area, to be saved.
+ * @new: pointer to SPU context save area, to be restored.
+ * @spu: pointer to SPU iomem structure.
+ *
+ * Perform save, then restore. Only harvest if the
+ * save fails, as cleanup is otherwise not needed.
+ */
+int spu_switch(struct spu_state *prev, struct spu_state *new, struct spu *spu)
+{
+ /* XXX missing */
+
+ return 0;
+}
+
+static void init_prob(struct spu_state *csa)
+{
+ csa->spu_chnlcnt_RW[9] = 1;
+ csa->spu_chnlcnt_RW[21] = 16;
+ csa->spu_chnlcnt_RW[23] = 1;
+ csa->spu_chnlcnt_RW[28] = 1;
+ csa->spu_chnlcnt_RW[30] = 1;
+ csa->prob.spu_runcntl_RW = SPU_RUNCNTL_STOP;
+}
+
+static void init_priv1(struct spu_state *csa)
+{
+ /* Enable decode, relocate, tlbie response, master runcntl. */
+ csa->priv1.mfc_sr1_RW = MFC_STATE1_LOCAL_STORAGE_DECODE_MASK |
+ MFC_STATE1_MASTER_RUN_CONTROL_MASK |
+ MFC_STATE1_PROBLEM_STATE_MASK |
+ MFC_STATE1_RELOCATE_MASK | MFC_STATE1_BUS_TLBIE_MASK;
+
+ /* Set storage description. */
+ csa->priv1.mfc_sdr_RW = mfspr(SPRN_SDR1);
+
+ /* Enable OS-specific set of interrupts. */
+ csa->priv1.int_mask_class0_RW = CLASS0_ENABLE_DMA_ALIGNMENT_INTR |
+ CLASS0_ENABLE_INVALID_DMA_COMMAND_INTR |
+ CLASS0_ENABLE_SPU_ERROR_INTR;
+ csa->priv1.int_mask_class1_RW = CLASS1_ENABLE_SEGMENT_FAULT_INTR |
+ CLASS1_ENABLE_STORAGE_FAULT_INTR;
+ csa->priv1.int_mask_class2_RW = CLASS2_ENABLE_MAILBOX_INTR |
+ CLASS2_ENABLE_SPU_STOP_INTR | CLASS2_ENABLE_SPU_HALT_INTR;
+}
+
+static void init_priv2(struct spu_state *csa)
+{
+ csa->priv2.spu_lslr_RW = LS_ADDR_MASK;
+ csa->priv2.mfc_control_RW = MFC_CNTL_RESUME_DMA_QUEUE |
+ MFC_CNTL_NORMAL_DMA_QUEUE_OPERATION |
+ MFC_CNTL_DMA_QUEUES_EMPTY_MASK;
+}
+
+/**
+ * spu_alloc_csa - allocate and initialize an SPU context save area.
+ *
+ * Allocate and initialize the contents of an SPU context save area.
+ * This includes enabling address translation, interrupt masks, etc.,
+ * as appropriate for the given OS environment.
+ *
+ * Note that storage for the 'lscsa' is allocated separately,
+ * as it is by far the largest of the context save regions,
+ * and may need to be pinned or otherwise specially aligned.
+ */
+void spu_init_csa(struct spu_state *csa)
+{
+ struct spu_lscsa *lscsa;
+
+ if (!csa)
+ return;
+ memset(csa, 0, sizeof(struct spu_state));
+
+ lscsa = vmalloc(sizeof(struct spu_lscsa));
+ if (!lscsa)
+ return;
+
+ memset(lscsa, 0, sizeof(struct spu_lscsa));
+ csa->lscsa = lscsa;
+
+ init_prob(csa);
+ init_priv1(csa);
+ init_priv2(csa);
+}
+
+void spu_fini_csa(struct spu_state *csa)
+{
+ vfree(csa->lscsa);
+}
Index: linux-cg/include/asm-ppc64/spu.h
===================================================================
--- linux-cg.orig/include/asm-ppc64/spu.h
+++ linux-cg/include/asm-ppc64/spu.h
@@ -28,6 +28,81 @@
#define LS_ORDER (6) /* 256 kb */
#define LS_SIZE (PAGE_SIZE << LS_ORDER)
+#define LS_ADDR_MASK (LS_SIZE - 1)
+
+#define MFC_PUT_CMD 0x20
+#define MFC_PUTS_CMD 0x28
+#define MFC_PUTR_CMD 0x30
+#define MFC_PUTF_CMD 0x22
+#define MFC_PUTB_CMD 0x21
+#define MFC_PUTFS_CMD 0x2A
+#define MFC_PUTBS_CMD 0x29
+#define MFC_PUTRF_CMD 0x32
+#define MFC_PUTRB_CMD 0x31
+#define MFC_PUTL_CMD 0x24
+#define MFC_PUTRL_CMD 0x34
+#define MFC_PUTLF_CMD 0x26
+#define MFC_PUTLB_CMD 0x25
+#define MFC_PUTRLF_CMD 0x36
+#define MFC_PUTRLB_CMD 0x35
+
+#define MFC_GET_CMD 0x40
+#define MFC_GETS_CMD 0x48
+#define MFC_GETF_CMD 0x42
+#define MFC_GETB_CMD 0x41
+#define MFC_GETFS_CMD 0x4A
+#define MFC_GETBS_CMD 0x49
+#define MFC_GETL_CMD 0x44
+#define MFC_GETLF_CMD 0x46
+#define MFC_GETLB_CMD 0x45
+
+#define MFC_SDCRT_CMD 0x80
+#define MFC_SDCRTST_CMD 0x81
+#define MFC_SDCRZ_CMD 0x89
+#define MFC_SDCRS_CMD 0x8D
+#define MFC_SDCRF_CMD 0x8F
+
+#define MFC_GETLLAR_CMD 0xD0
+#define MFC_PUTLLC_CMD 0xB4
+#define MFC_PUTLLUC_CMD 0xB0
+#define MFC_PUTQLLUC_CMD 0xB8
+#define MFC_SNDSIG_CMD 0xA0
+#define MFC_SNDSIGB_CMD 0xA1
+#define MFC_SNDSIGF_CMD 0xA2
+#define MFC_BARRIER_CMD 0xC0
+#define MFC_EIEIO_CMD 0xC8
+#define MFC_SYNC_CMD 0xCC
+
+#define MFC_MIN_DMA_SIZE_SHIFT 4 /* 16 bytes */
+#define MFC_MAX_DMA_SIZE_SHIFT 14 /* 16384 bytes */
+#define MFC_MIN_DMA_SIZE (1 << MFC_MIN_DMA_SIZE_SHIFT)
+#define MFC_MAX_DMA_SIZE (1 << MFC_MAX_DMA_SIZE_SHIFT)
+#define MFC_MIN_DMA_SIZE_MASK (MFC_MIN_DMA_SIZE - 1)
+#define MFC_MAX_DMA_SIZE_MASK (MFC_MAX_DMA_SIZE - 1)
+#define MFC_MIN_DMA_LIST_SIZE 0x0008 /* 8 bytes */
+#define MFC_MAX_DMA_LIST_SIZE 0x4000 /* 16K bytes */
+
+#define MFC_TAGID_TO_TAGMASK(tag_id) (1 << (tag_id & 0x1F))
+
+/* Events for Channels 0-2 */
+#define MFC_DMA_TAG_STATUS_UPDATE_EVENT 0x00000001
+#define MFC_DMA_TAG_CMD_STALL_NOTIFY_EVENT 0x00000002
+#define MFC_DMA_QUEUE_AVAILABLE_EVENT 0x00000008
+#define MFC_SPU_MAILBOX_WRITTEN_EVENT 0x00000010
+#define MFC_DECREMENTER_EVENT 0x00000020
+#define MFC_PU_INT_MAILBOX_AVAILABLE_EVENT 0x00000040
+#define MFC_PU_MAILBOX_AVAILABLE_EVENT 0x00000080
+#define MFC_SIGNAL_2_EVENT 0x00000100
+#define MFC_SIGNAL_1_EVENT 0x00000200
+#define MFC_LLR_LOST_EVENT 0x00000400
+#define MFC_PRIV_ATTN_EVENT 0x00000800
+#define MFC_MULTI_SRC_EVENT 0x00001000
+
+/* Flags indicating progress during context switch. */
+#define SPU_CONTEXT_SWITCH_PENDING_nr 0UL
+#define SPU_CONTEXT_SWITCH_ACTIVE_nr 1UL
+#define SPU_CONTEXT_SWITCH_PENDING (1UL << SPU_CONTEXT_SWITCH_PENDING_nr)
+#define SPU_CONTEXT_SWITCH_ACTIVE (1UL << SPU_CONTEXT_SWITCH_ACTIVE_nr)
struct spu {
char *name;
@@ -40,6 +115,7 @@ struct spu {
int number;
u32 isrc;
u32 node;
+ u64 flags;
struct kref kref;
size_t ls_size;
unsigned int slb_replace;
Index: linux-cg/include/asm-ppc64/spu_csa.h
===================================================================
--- /dev/null
+++ linux-cg/include/asm-ppc64/spu_csa.h
@@ -0,0 +1,256 @@
+/*
+ * spu_csa.h: Definitions for SPU context save area (CSA).
+ *
+ * (C) Copyright IBM 2005
+ *
+ * Author: Mark Nutter <mnutter@us.ibm.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2, or (at your option)
+ * any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+ */
+
+#ifndef _SPU_CSA_H_
+#define _SPU_CSA_H_
+
+/*
+ * Total number of 128-bit registers.
+ */
+#define NR_SPU_GPRS 128
+#define NR_SPU_SPRS 9
+#define NR_SPU_REGS_PAD 7
+#define NR_SPU_SPILL_REGS 144 /* GPRS + SPRS + PAD */
+#define SIZEOF_SPU_SPILL_REGS NR_SPU_SPILL_REGS * 16
+
+#define SPU_SAVE_COMPLETE 0x3FFB
+#define SPU_RESTORE_COMPLETE 0x3FFC
+
+/*
+ * Definitions for various 'stopped' status conditions,
+ * to be recreated during context restore.
+ */
+#define SPU_STOPPED_STATUS_P 1
+#define SPU_STOPPED_STATUS_I 2
+#define SPU_STOPPED_STATUS_H 3
+#define SPU_STOPPED_STATUS_S 4
+#define SPU_STOPPED_STATUS_S_I 5
+#define SPU_STOPPED_STATUS_S_P 6
+#define SPU_STOPPED_STATUS_P_H 7
+#define SPU_STOPPED_STATUS_P_I 8
+#define SPU_STOPPED_STATUS_R 9
+
+#ifndef __ASSEMBLY__
+/**
+ * spu_reg128 - generic 128-bit register definition.
+ */
+struct spu_reg128 {
+ u32 slot[4];
+};
+
+/**
+ * struct spu_lscsa - Local Store Context Save Area.
+ * @gprs: Array of saved registers.
+ * @fpcr: Saved floating point status control register.
+ * @decr: Saved decrementer value.
+ * @decr_status: Indicates decrementer run status.
+ * @ppu_mb: Saved PPU mailbox data.
+ * @ppuint_mb: Saved PPU interrupting mailbox data.
+ * @tag_mask: Saved tag group mask.
+ * @event_mask: Saved event mask.
+ * @srr0: Saved SRR0.
+ * @stopped_status: Conditions to be recreated by restore.
+ * @ls: Saved contents of Local Storage Area.
+ *
+ * The LSCSA represents state that is primarily saved and
+ * restored by SPU-side code.
+ */
+struct spu_lscsa {
+ struct spu_reg128 gprs[128];
+ struct spu_reg128 fpcr;
+ struct spu_reg128 decr;
+ struct spu_reg128 decr_status;
+ struct spu_reg128 ppu_mb;
+ struct spu_reg128 ppuint_mb;
+ struct spu_reg128 tag_mask;
+ struct spu_reg128 event_mask;
+ struct spu_reg128 srr0;
+ struct spu_reg128 stopped_status;
+ struct spu_reg128 pad[119]; /* 'ls' must be page-aligned. */
+ unsigned char ls[LS_SIZE];
+};
+
+#ifdef __KERNEL__
+
+/*
+ * struct spu_problem_collapsed - condensed problem state area, w/o pads.
+ */
+struct spu_problem_collapsed {
+ u64 spc_mssync_RW;
+ u32 mfc_lsa_W;
+ u32 unused_pad0;
+ u64 mfc_ea_W;
+ union mfc_tag_size_class_cmd mfc_union_W;
+ u32 dma_qstatus_R;
+ u32 dma_querytype_RW;
+ u32 dma_querymask_RW;
+ u32 dma_tagstatus_R;
+ u32 pu_mb_R;
+ u32 spu_mb_W;
+ u32 mb_stat_R;
+ u32 spu_runcntl_RW;
+ u32 spu_status_R;
+ u32 spu_spc_R;
+ u32 spu_npc_RW;
+ u32 signal_notify1;
+ u32 signal_notify2;
+ u32 unused_pad1;
+};
+
+/*
+ * struct spu_priv1_collapsed - condensed privileged 1 area, w/o pads.
+ */
+struct spu_priv1_collapsed {
+ u64 mfc_sr1_RW;
+ u64 mfc_lpid_RW;
+ u64 spu_idr_RW;
+ u64 mfc_vr_RO;
+ u64 spu_vr_RO;
+ u64 int_mask_class0_RW;
+ u64 int_mask_class1_RW;
+ u64 int_mask_class2_RW;
+ u64 int_stat_class0_RW;
+ u64 int_stat_class1_RW;
+ u64 int_stat_class2_RW;
+ u64 int_route_RW;
+ u64 mfc_atomic_flush_RW;
+ u64 resource_allocation_groupID_RW;
+ u64 resource_allocation_enable_RW;
+ u64 mfc_fir_R;
+ u64 mfc_fir_status_or_W;
+ u64 mfc_fir_status_and_W;
+ u64 mfc_fir_mask_R;
+ u64 mfc_fir_mask_or_W;
+ u64 mfc_fir_mask_and_W;
+ u64 mfc_fir_chkstp_enable_RW;
+ u64 smf_sbi_signal_sel;
+ u64 smf_ato_signal_sel;
+ u64 mfc_sdr_RW;
+ u64 tlb_index_hint_RO;
+ u64 tlb_index_W;
+ u64 tlb_vpn_RW;
+ u64 tlb_rpn_RW;
+ u64 tlb_invalidate_entry_W;
+ u64 tlb_invalidate_all_W;
+ u64 smm_hid;
+ u64 mfc_accr_RW;
+ u64 mfc_dsisr_RW;
+ u64 mfc_dar_RW;
+ u64 rmt_index_RW;
+ u64 rmt_data1_RW;
+ u64 mfc_dsir_R;
+ u64 mfc_lsacr_RW;
+ u64 mfc_lscrr_R;
+ u64 mfc_tclass_id_RW;
+ u64 mfc_rm_boundary;
+ u64 smf_dma_signal_sel;
+ u64 smm_signal_sel;
+ u64 mfc_cer_R;
+ u64 pu_ecc_cntl_RW;
+ u64 pu_ecc_stat_RW;
+ u64 spu_ecc_addr_RW;
+ u64 spu_err_mask_RW;
+ u64 spu_trig0_sel;
+ u64 spu_trig1_sel;
+ u64 spu_trig2_sel;
+ u64 spu_trig3_sel;
+ u64 spu_trace_sel;
+ u64 spu_event0_sel;
+ u64 spu_event1_sel;
+ u64 spu_event2_sel;
+ u64 spu_event3_sel;
+ u64 spu_trace_cntl;
+};
+
+/*
+ * struct spu_priv2_collapsed - condensed priviliged 2 area, w/o pads.
+ */
+struct spu_priv2_collapsed {
+ u64 slb_index_W;
+ u64 slb_esid_RW;
+ u64 slb_vsid_RW;
+ u64 slb_invalidate_entry_W;
+ u64 slb_invalidate_all_W;
+ struct mfc_cq_sr spuq[16];
+ struct mfc_cq_sr puq[8];
+ u64 mfc_control_RW;
+ u64 puint_mb_R;
+ u64 spu_privcntl_RW;
+ u64 spu_lslr_RW;
+ u64 spu_chnlcntptr_RW;
+ u64 spu_chnlcnt_RW;
+ u64 spu_chnldata_RW;
+ u64 spu_cfg_RW;
+ u64 spu_pm_trace_tag_status_RW;
+ u64 spu_tag_status_query_RW;
+ u64 spu_cmd_buf1_RW;
+ u64 spu_cmd_buf2_RW;
+ u64 spu_atomic_status_RW;
+};
+
+/**
+ * struct spu_state
+ * @lscsa: Local Store Context Save Area.
+ * @prob: Collapsed Problem State Area, w/o pads.
+ * @priv1: Collapsed Privileged 1 Area, w/o pads.
+ * @priv2: Collapsed Privileged 2 Area, w/o pads.
+ * @spu_chnlcnt_RW: Array of saved channel counts.
+ * @spu_chnldata_RW: Array of saved channel data.
+ * @suspend_time: Time stamp when decrementer disabled.
+ * @slb_esid_RW: Array of saved SLB esid entries.
+ * @slb_vsid_RW: Array of saved SLB vsid entries.
+ *
+ * Structure representing the whole of the SPU
+ * context save area (CSA). This struct contains
+ * all of the state necessary to suspend and then
+ * later optionally resume execution of an SPU
+ * context.
+ *
+ * The @lscsa region is by far the largest, and is
+ * allocated separately so that it may either be
+ * pinned or mapped to/from application memory, as
+ * appropriate for the OS environment.
+ */
+struct spu_state {
+ struct spu_lscsa *lscsa;
+ struct spu_problem_collapsed prob;
+ struct spu_priv1_collapsed priv1;
+ struct spu_priv2_collapsed priv2;
+ u64 spu_chnlcnt_RW[32];
+ u64 spu_chnldata_RW[32];
+ u32 spu_mailbox_data[4];
+ u32 pu_mailbox_data[1];
+ unsigned long suspend_time;
+ u64 slb_esid_RW[8];
+ u64 slb_vsid_RW[8];
+};
+
+extern void spu_init_csa(struct spu_state *csa);
+extern void spu_fini_csa(struct spu_state *csa);
+extern int spu_save(struct spu_state *prev, struct spu *spu);
+extern int spu_restore(struct spu_state *new, struct spu *spu);
+extern int spu_switch(struct spu_state *prev, struct spu_state *new,
+ struct spu *spu);
+
+#endif /* __KERNEL__ */
+#endif /* !__ASSEMBLY__ */
+#endif /* _SPU_CSA_H_ */
--
^ permalink raw reply [flat|nested] 14+ messages in thread* [patch 03/11] spufs: kernel-side context switch code
2005-09-16 12:16 [patch 00/11] spufs: latest spufs snapshot for 2.6.14-rc1 Arnd Bergmann
` (2 preceding siblings ...)
2005-09-16 12:16 ` [patch 02/11] spufs: switchable spu contexts Arnd Bergmann
@ 2005-09-16 12:16 ` Arnd Bergmann
2005-09-16 12:16 ` [patch 04/11] spufs: add spu-side " Arnd Bergmann
` (7 subsequent siblings)
11 siblings, 0 replies; 14+ messages in thread
From: Arnd Bergmann @ 2005-09-16 12:16 UTC (permalink / raw)
To: ppc64-dev
Cc: linux-kernel, Paul Mackerras, Benjamin Herrenschmidt,
jordi_caubet, Hiroyuki Machida, Geoff Levand
[-- Attachment #1: spufs-context-3-part2.diff --]
[-- Type: text/plain, Size: 62158 bytes --]
This adds the code needed to perform a context switch from
spufs, following the recommended 76-step sequence.
From: Mark Nutter <mnutter@us.ibm.com>
Signed-off-by: Arnd Bergmann <arndb@de.ibm.com>
--
switch.c | 2052 ++++++++++++++++++++++++++++++++++++++++++++++++++++++-
1 files changed, 2046 insertions(+), 6 deletions(-)
--- linux-cg.orig/fs/spufs/switch.c 2005-08-17 19:15:27.324893240 -0400
+++ linux-cg/fs/spufs/switch.c 2005-08-17 19:22:16.112922576 -0400
@@ -52,6 +52,2029 @@
#include "spu_save_dump.h"
#include "spu_restore_dump.h"
+#if 0
+#define POLL_WHILE_TRUE(_c) { \
+ do { \
+ } while (_c); \
+ }
+#else
+#define RELAX_SPIN_COUNT 1000
+#define POLL_WHILE_TRUE(_c) { \
+ do { \
+ int _i; \
+ for (_i=0; _i<RELAX_SPIN_COUNT && (_c); _i++) { \
+ cpu_relax(); \
+ } \
+ if (unlikely(_c)) yield(); \
+ else break; \
+ } while (_c); \
+ }
+#endif /* debug */
+
+#define POLL_WHILE_FALSE(_c) POLL_WHILE_TRUE(!(_c))
+
+static inline void acquire_spu_lock(struct spu *spu)
+{
+ /* Save, Step 1:
+ * Restore, Step 1:
+ * Acquire SPU-specific mutual exclusion lock.
+ * TBD.
+ */
+}
+
+static inline void release_spu_lock(struct spu *spu)
+{
+ /* Restore, Step 76:
+ * Release SPU-specific mutual exclusion lock.
+ * TBD.
+ */
+}
+
+static inline int check_spu_isolate(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_problem __iomem *prob = spu->problem;
+ u32 isolate_state;
+
+ /* Save, Step 2:
+ * Save, Step 6:
+ * If SPU_Status[E,L,IS] any field is '1', this
+ * SPU is in isolate state and cannot be context
+ * saved at this time.
+ */
+ isolate_state = SPU_STATUS_ISOLATED_STATE |
+ SPU_STATUS_ISOLATED_LOAD_STAUTUS | SPU_STATUS_ISOLATED_EXIT_STAUTUS;
+ return (in_be32(&prob->spu_status_R) & isolate_state) ? 1 : 0;
+}
+
+static inline void disable_interrupts(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_priv1 __iomem *priv1 = spu->priv1;
+ unsigned long flags;
+
+ /* Save, Step 3:
+ * Restore, Step 2:
+ * Save INT_Mask_class0 in CSA.
+ * Write INT_MASK_class0 with value of 0.
+ * Save INT_Mask_class1 in CSA.
+ * Write INT_MASK_class1 with value of 0.
+ * Save INT_Mask_class2 in CSA.
+ * Write INT_MASK_class2 with value of 0.
+ */
+ local_irq_save(flags);
+ if (csa) {
+ csa->priv1.int_mask_class0_RW =
+ in_be64(&priv1->int_mask_class0_RW);
+ csa->priv1.int_mask_class1_RW =
+ in_be64(&priv1->int_mask_class1_RW);
+ csa->priv1.int_mask_class2_RW =
+ in_be64(&priv1->int_mask_class2_RW);
+ }
+ out_be64(&priv1->int_mask_class0_RW, 0UL);
+ out_be64(&priv1->int_mask_class1_RW, 0UL);
+ out_be64(&priv1->int_mask_class2_RW, 0UL);
+ eieio();
+ local_irq_restore(flags);
+}
+
+static inline void set_watchdog_timer(struct spu_state *csa, struct spu *spu)
+{
+ /* Save, Step 4:
+ * Restore, Step 25.
+ * Set a software watchdog timer, which specifies the
+ * maximum allowable time for a context save sequence.
+ *
+ * For present, this implementation will not set a global
+ * watchdog timer, as virtualization & variable system load
+ * may cause unpredictable execution times.
+ */
+}
+
+static inline void inhibit_user_access(struct spu_state *csa, struct spu *spu)
+{
+ /* Save, Step 5:
+ * Restore, Step 3:
+ * Inhibit user-space access (if provided) to this
+ * SPU by unmapping the virtual pages assigned to
+ * the SPU memory-mapped I/O (MMIO) for problem
+ * state. TBD.
+ */
+}
+
+static inline void set_switch_pending(struct spu_state *csa, struct spu *spu)
+{
+ /* Save, Step 7:
+ * Restore, Step 5:
+ * Set a software context switch pending flag.
+ */
+ set_bit(SPU_CONTEXT_SWITCH_PENDING_nr, &spu->flags);
+ mb();
+}
+
+static inline void save_mfc_cntl(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_priv2 __iomem *priv2 = spu->priv2;
+
+ /* Save, Step 8:
+ * Read and save MFC_CNTL[Ss].
+ */
+ if (csa) {
+ csa->priv2.mfc_control_RW = in_be64(&priv2->mfc_control_RW) &
+ MFC_CNTL_SUSPEND_DMA_STATUS_MASK;
+ }
+}
+
+static inline void save_spu_runcntl(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_problem __iomem *prob = spu->problem;
+
+ /* Save, Step 9:
+ * Save SPU_Runcntl in the CSA. This value contains
+ * the "Application Desired State".
+ */
+ csa->prob.spu_runcntl_RW = in_be32(&prob->spu_runcntl_RW);
+}
+
+static inline void save_mfc_sr1(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_priv1 __iomem *priv1 = spu->priv1;
+
+ /* Save, Step 10:
+ * Save MFC_SR1 in the CSA.
+ */
+ csa->priv1.mfc_sr1_RW = in_be64(&priv1->mfc_sr1_RW);
+}
+
+static inline void save_spu_status(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_problem __iomem *prob = spu->problem;
+
+ /* Save, Step 11:
+ * Read SPU_Status[R], and save to CSA.
+ */
+ if ((in_be32(&prob->spu_status_R) & SPU_STATUS_RUNNING) == 0) {
+ csa->prob.spu_status_R = in_be32(&prob->spu_status_R);
+ } else {
+ u32 stopped;
+
+ out_be32(&prob->spu_runcntl_RW, SPU_RUNCNTL_STOP);
+ eieio();
+ POLL_WHILE_TRUE(in_be32(&prob->spu_status_R) &
+ SPU_STATUS_RUNNING);
+ stopped =
+ SPU_STATUS_INVALID_INSTR | SPU_STATUS_SINGLE_STEP |
+ SPU_STATUS_STOPPED_BY_HALT | SPU_STATUS_STOPPED_BY_STOP;
+ if ((in_be32(&prob->spu_status_R) & stopped) == 0)
+ csa->prob.spu_status_R = SPU_STATUS_RUNNING;
+ else
+ csa->prob.spu_status_R = in_be32(&prob->spu_status_R);
+ }
+}
+
+static inline void save_mfc_decr(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_priv2 __iomem *priv2 = spu->priv2;
+
+ /* Save, Step 12:
+ * Read MFC_CNTL[Ds]. Update saved copy of
+ * CSA.MFC_CNTL[Ds].
+ */
+ if (in_be64(&priv2->mfc_control_RW) & MFC_CNTL_DECREMENTER_RUNNING) {
+ csa->priv2.mfc_control_RW |= MFC_CNTL_DECREMENTER_RUNNING;
+ csa->suspend_time = get_cycles();
+ out_be64(&priv2->spu_chnlcntptr_RW, 7ULL);
+ eieio();
+ csa->spu_chnldata_RW[7] = in_be64(&priv2->spu_chnldata_RW);
+ eieio();
+ }
+}
+
+static inline void halt_mfc_decr(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_priv2 __iomem *priv2 = spu->priv2;
+
+ /* Save, Step 13:
+ * Write MFC_CNTL[Dh] set to a '1' to halt
+ * the decrementer.
+ */
+ out_be64(&priv2->mfc_control_RW, MFC_CNTL_DECREMENTER_HALTED);
+ eieio();
+}
+
+static inline void save_timebase(struct spu_state *csa, struct spu *spu)
+{
+ /* Save, Step 14:
+ * Read PPE Timebase High and Timebase low registers
+ * and save in CSA. TBD.
+ */
+ csa->suspend_time = get_cycles();
+}
+
+static inline void remove_other_spu_access(struct spu_state *csa,
+ struct spu *spu)
+{
+ /* Save, Step 15:
+ * Remove other SPU access to this SPU by unmapping
+ * this SPU's pages from their address space. TBD.
+ */
+}
+
+static inline void do_mfc_mssync(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_problem __iomem *prob = spu->problem;
+
+ /* Save, Step 16:
+ * Restore, Step 11.
+ * Write SPU_MSSync register. Poll SPU_MSSync[P]
+ * for a value of 0.
+ */
+ out_be64(&prob->spc_mssync_RW, 1UL);
+ POLL_WHILE_TRUE(in_be64(&prob->spc_mssync_RW) & MS_SYNC_PENDING);
+}
+
+static inline void issue_mfc_tlbie(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_priv1 __iomem *priv1 = spu->priv1;
+
+ /* Save, Step 17:
+ * Restore, Step 12.
+ * Restore, Step 48.
+ * Write TLB_Invalidate_Entry[IS,VPN,L,Lp]=0 register.
+ * Then issue a PPE sync instruction.
+ */
+ out_be64(&priv1->tlb_invalidate_entry_W, 0UL);
+ mb();
+}
+
+static inline void handle_pending_interrupts(struct spu_state *csa,
+ struct spu *spu)
+{
+ /* Save, Step 18:
+ * Handle any pending interrupts from this SPU
+ * here. This is OS or hypervisor specific. One
+ * option is to re-enable interrupts to handle any
+ * pending interrupts, with the interrupt handlers
+ * recognizing the software Context Switch Pending
+ * flag, to ensure the SPU execution or MFC command
+ * queue is not restarted. TBD.
+ */
+}
+
+static inline void save_mfc_queues(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_priv2 __iomem *priv2 = spu->priv2;
+ int i;
+
+ /* Save, Step 19:
+ * If MFC_Cntl[Se]=0 then save
+ * MFC command queues.
+ */
+ if ((in_be64(&priv2->mfc_control_RW) & MFC_CNTL_DMA_QUEUES_EMPTY) == 0) {
+ for (i = 0; i < 8; i++) {
+ csa->priv2.puq[i].mfc_cq_data0_RW =
+ in_be64(&priv2->puq[i].mfc_cq_data0_RW);
+ csa->priv2.puq[i].mfc_cq_data1_RW =
+ in_be64(&priv2->puq[i].mfc_cq_data1_RW);
+ csa->priv2.puq[i].mfc_cq_data2_RW =
+ in_be64(&priv2->puq[i].mfc_cq_data2_RW);
+ csa->priv2.puq[i].mfc_cq_data3_RW =
+ in_be64(&priv2->puq[i].mfc_cq_data3_RW);
+ }
+ for (i = 0; i < 16; i++) {
+ csa->priv2.spuq[i].mfc_cq_data0_RW =
+ in_be64(&priv2->spuq[i].mfc_cq_data0_RW);
+ csa->priv2.spuq[i].mfc_cq_data1_RW =
+ in_be64(&priv2->spuq[i].mfc_cq_data1_RW);
+ csa->priv2.spuq[i].mfc_cq_data2_RW =
+ in_be64(&priv2->spuq[i].mfc_cq_data2_RW);
+ csa->priv2.spuq[i].mfc_cq_data3_RW =
+ in_be64(&priv2->spuq[i].mfc_cq_data3_RW);
+ }
+ }
+}
+
+static inline void save_ppu_querymask(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_problem __iomem *prob = spu->problem;
+
+ /* Save, Step 20:
+ * Save the PPU_QueryMask register
+ * in the CSA.
+ */
+ csa->prob.dma_querymask_RW = in_be32(&prob->dma_querymask_RW);
+}
+
+static inline void save_ppu_querytype(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_problem __iomem *prob = spu->problem;
+
+ /* Save, Step 21:
+ * Save the PPU_QueryType register
+ * in the CSA.
+ */
+ csa->prob.dma_querytype_RW = in_be32(&prob->dma_querytype_RW);
+}
+
+static inline void save_mfc_csr_tsq(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_priv2 __iomem *priv2 = spu->priv2;
+
+ /* Save, Step 22:
+ * Save the MFC_CSR_TSQ register
+ * in the LSCSA.
+ */
+ csa->priv2.spu_tag_status_query_RW =
+ in_be64(&priv2->spu_tag_status_query_RW);
+}
+
+static inline void save_mfc_csr_cmd(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_priv2 __iomem *priv2 = spu->priv2;
+
+ /* Save, Step 23:
+ * Save the MFC_CSR_CMD1 and MFC_CSR_CMD2
+ * registers in the CSA.
+ */
+ csa->priv2.spu_cmd_buf1_RW = in_be64(&priv2->spu_cmd_buf1_RW);
+ csa->priv2.spu_cmd_buf2_RW = in_be64(&priv2->spu_cmd_buf2_RW);
+}
+
+static inline void save_mfc_csr_ato(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_priv2 __iomem *priv2 = spu->priv2;
+
+ /* Save, Step 24:
+ * Save the MFC_CSR_ATO register in
+ * the CSA.
+ */
+ csa->priv2.spu_atomic_status_RW = in_be64(&priv2->spu_atomic_status_RW);
+}
+
+static inline void save_mfc_tclass_id(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_priv1 __iomem *priv1 = spu->priv1;
+
+ /* Save, Step 25:
+ * Save the MFC_TCLASS_ID register in
+ * the CSA.
+ */
+ csa->priv1.mfc_tclass_id_RW = in_be64(&priv1->mfc_tclass_id_RW);
+}
+
+static inline void set_mfc_tclass_id(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_priv1 __iomem *priv1 = spu->priv1;
+
+ /* Save, Step 26:
+ * Restore, Step 23.
+ * Write the MFC_TCLASS_ID register with
+ * the value 0x10000000.
+ */
+ out_be64(&priv1->mfc_tclass_id_RW, 0x10000000);
+ eieio();
+}
+
+static inline void purge_mfc_queue(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_priv2 __iomem *priv2 = spu->priv2;
+
+ /* Save, Step 27:
+ * Restore, Step 14.
+ * Write MFC_CNTL[Pc]=1 (purge queue).
+ */
+ out_be64(&priv2->mfc_control_RW, MFC_CNTL_PURGE_DMA_REQUEST);
+ eieio();
+}
+
+static inline void wait_purge_complete(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_priv2 __iomem *priv2 = spu->priv2;
+
+ /* Save, Step 28:
+ * Poll MFC_CNTL[Ps] until value '11' is read
+ * (purge complete).
+ */
+ POLL_WHILE_FALSE(in_be64(&priv2->mfc_control_RW) &
+ MFC_CNTL_PURGE_DMA_COMPLETE);
+}
+
+static inline void save_mfc_slbs(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_priv1 __iomem *priv1 = spu->priv1;
+ struct spu_priv2 __iomem *priv2 = spu->priv2;
+ int i;
+
+ /* Save, Step 29:
+ * If MFC_SR1[R]='1', save SLBs in CSA.
+ */
+ if (in_be64(&priv1->mfc_sr1_RW) & MFC_STATE1_RELOCATE_MASK) {
+ csa->priv2.slb_index_W = in_be64(&priv2->slb_index_W);
+ for (i = 0; i < 8; i++) {
+ out_be64(&priv2->slb_index_W, i);
+ eieio();
+ csa->slb_esid_RW[i] = in_be64(&priv2->slb_esid_RW);
+ csa->slb_vsid_RW[i] = in_be64(&priv2->slb_vsid_RW);
+ eieio();
+ }
+ }
+}
+
+static inline void setup_mfc_sr1(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_priv1 __iomem *priv1 = spu->priv1;
+
+ /* Save, Step 30:
+ * Restore, Step 18:
+ * Write MFC_SR1 with MFC_SR1[D=0,S=1] and
+ * MFC_SR1[TL,R,Pr,T] set correctly for the
+ * OS specific environment.
+ *
+ * Implementation note: The SPU-side code
+ * for save/restore is privileged, so the
+ * MFC_SR1[Pr] bit is not set.
+ *
+ */
+ out_be64(&priv1->mfc_sr1_RW, (MFC_STATE1_MASTER_RUN_CONTROL_MASK |
+ MFC_STATE1_RELOCATE_MASK |
+ MFC_STATE1_BUS_TLBIE_MASK));
+}
+
+static inline void save_spu_npc(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_problem __iomem *prob = spu->problem;
+
+ /* Save, Step 31:
+ * Save SPU_NPC in the CSA.
+ */
+ csa->prob.spu_npc_RW = in_be32(&prob->spu_npc_RW);
+}
+
+static inline void save_spu_privcntl(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_priv2 __iomem *priv2 = spu->priv2;
+
+ /* Save, Step 32:
+ * Save SPU_PrivCntl in the CSA.
+ */
+ csa->priv2.spu_privcntl_RW = in_be64(&priv2->spu_privcntl_RW);
+}
+
+static inline void reset_spu_privcntl(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_priv2 __iomem *priv2 = spu->priv2;
+
+ /* Save, Step 33:
+ * Restore, Step 16:
+ * Write SPU_PrivCntl[S,Le,A] fields reset to 0.
+ */
+ out_be64(&priv2->spu_privcntl_RW, 0UL);
+ eieio();
+}
+
+static inline void save_spu_lslr(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_priv2 __iomem *priv2 = spu->priv2;
+
+ /* Save, Step 34:
+ * Save SPU_LSLR in the CSA.
+ */
+ csa->priv2.spu_lslr_RW = in_be64(&priv2->spu_lslr_RW);
+}
+
+static inline void reset_spu_lslr(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_priv2 __iomem *priv2 = spu->priv2;
+
+ /* Save, Step 35:
+ * Restore, Step 17.
+ * Reset SPU_LSLR.
+ */
+ out_be64(&priv2->spu_lslr_RW, LS_ADDR_MASK);
+ eieio();
+}
+
+static inline void save_spu_cfg(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_priv2 __iomem *priv2 = spu->priv2;
+
+ /* Save, Step 36:
+ * Save SPU_Cfg in the CSA.
+ */
+ csa->priv2.spu_cfg_RW = in_be64(&priv2->spu_cfg_RW);
+}
+
+static inline void save_pm_trace(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_priv2 __iomem *priv2 = spu->priv2;
+
+ /* Save, Step 37:
+ * Save PM_Trace_Tag_Wait_Mask in the CSA.
+ */
+ csa->priv2.spu_pm_trace_tag_status_RW =
+ in_be64(&priv2->spu_pm_trace_tag_status_RW);
+}
+
+static inline void save_mfc_rag(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_priv1 __iomem *priv1 = spu->priv1;
+
+ /* Save, Step 38:
+ * Save RA_GROUP_ID register and the
+ * RA_ENABLE reigster in the CSA.
+ */
+ csa->priv1.resource_allocation_groupID_RW =
+ in_be64(&priv1->resource_allocation_groupID_RW);
+ csa->priv1.resource_allocation_enable_RW =
+ in_be64(&priv1->resource_allocation_enable_RW);
+}
+
+static inline void save_ppu_mb_stat(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_problem __iomem *prob = spu->problem;
+
+ /* Save, Step 39:
+ * Save MB_Stat register in the CSA.
+ */
+ csa->prob.mb_stat_R = in_be32(&prob->mb_stat_R);
+}
+
+static inline void save_ppu_mb(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_problem __iomem *prob = spu->problem;
+
+ /* Save, Step 40:
+ * Save the PPU_MB register in the CSA.
+ */
+ csa->prob.pu_mb_R = in_be32(&prob->pu_mb_R);
+}
+
+static inline void save_ppuint_mb(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_priv2 __iomem *priv2 = spu->priv2;
+
+ /* Save, Step 41:
+ * Save the PPUINT_MB register in the CSA.
+ */
+ csa->priv2.puint_mb_R = in_be64(&priv2->puint_mb_R);
+}
+
+static inline void save_ch_part1(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_priv2 __iomem *priv2 = spu->priv2;
+ u64 idx, ch_indices[7] = { 0UL, 1UL, 3UL, 4UL, 24UL, 25UL, 27UL };
+ int i;
+
+ /* Save, Step 42:
+ * Save the following CH: [0,1,3,4,24,25,27]
+ */
+ for (i = 0; i < 7; i++) {
+ idx = ch_indices[i];
+ out_be64(&priv2->spu_chnlcntptr_RW, idx);
+ eieio();
+ csa->spu_chnldata_RW[idx] = in_be64(&priv2->spu_chnldata_RW);
+ csa->spu_chnlcnt_RW[idx] = in_be64(&priv2->spu_chnlcnt_RW);
+ out_be64(&priv2->spu_chnldata_RW, 0UL);
+ out_be64(&priv2->spu_chnlcnt_RW, 0UL);
+ eieio();
+ }
+}
+
+static inline void save_spu_mb(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_priv2 __iomem *priv2 = spu->priv2;
+ int i;
+
+ /* Save, Step 43:
+ * Save SPU Read Mailbox Channel.
+ */
+ out_be64(&priv2->spu_chnlcntptr_RW, 29UL);
+ eieio();
+ csa->spu_chnlcnt_RW[29] = in_be64(&priv2->spu_chnlcnt_RW);
+ for (i = 0; i < 4; i++) {
+ csa->pu_mailbox_data[i] = in_be64(&priv2->spu_chnldata_RW);
+ }
+ out_be64(&priv2->spu_chnlcnt_RW, 0UL);
+ eieio();
+}
+
+static inline void save_mfc_cmd(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_priv2 __iomem *priv2 = spu->priv2;
+
+ /* Save, Step 44:
+ * Save MFC_CMD Channel.
+ */
+ out_be64(&priv2->spu_chnlcntptr_RW, 21UL);
+ eieio();
+ csa->spu_chnlcnt_RW[21] = in_be64(&priv2->spu_chnlcnt_RW);
+ eieio();
+}
+
+static inline void reset_ch(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_priv2 __iomem *priv2 = spu->priv2;
+ u64 ch_indices[4] = { 21UL, 23UL, 28UL, 30UL };
+ u64 ch_counts[4] = { 16UL, 1UL, 1UL, 1UL };
+ u64 idx;
+ int i;
+
+ /* Save, Step 45:
+ * Reset the following CH: [21, 23, 28, 30]
+ */
+ for (i = 0; i < 4; i++) {
+ idx = ch_indices[i];
+ out_be64(&priv2->spu_chnlcntptr_RW, idx);
+ eieio();
+ out_be64(&priv2->spu_chnlcnt_RW, ch_counts[i]);
+ eieio();
+ }
+}
+
+static inline void resume_mfc_queue(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_priv2 __iomem *priv2 = spu->priv2;
+
+ /* Save, Step 46:
+ * Restore, Step 25.
+ * Write MFC_CNTL[Sc]=0 (resume queue processing).
+ */
+ out_be64(&priv2->mfc_control_RW, MFC_CNTL_RESUME_DMA_QUEUE);
+}
+
+static inline void invalidate_slbs(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_priv1 __iomem *priv1 = spu->priv1;
+ struct spu_priv2 __iomem *priv2 = spu->priv2;
+
+ /* Save, Step 45:
+ * Restore, Step 19:
+ * If MFC_SR1[R]=1, write 0 to SLB_Invalidate_All.
+ */
+ if (in_be64(&priv1->mfc_sr1_RW) & MFC_STATE1_RELOCATE_MASK) {
+ out_be64(&priv2->slb_invalidate_all_W, 0UL);
+ eieio();
+ }
+}
+
+static inline void get_kernel_slb(u64 ea, u64 slb[2])
+{
+ slb[0] = (get_kernel_vsid(ea) << SLB_VSID_SHIFT) | SLB_VSID_KERNEL;
+ slb[1] = (ea & ESID_MASK) | SLB_ESID_V;
+
+ /* Large pages are used for kernel text/data, but not vmalloc. */
+ if (cpu_has_feature(CPU_FTR_16M_PAGE)
+ && REGION_ID(ea) == KERNEL_REGION_ID)
+ slb[0] |= SLB_VSID_L;
+}
+
+static inline void load_mfc_slb(struct spu *spu, u64 slb[2], int slbe)
+{
+ struct spu_priv2 __iomem *priv2 = spu->priv2;
+
+ out_be64(&priv2->slb_index_W, slbe);
+ eieio();
+ out_be64(&priv2->slb_vsid_RW, slb[0]);
+ out_be64(&priv2->slb_esid_RW, slb[1]);
+ eieio();
+}
+
+static inline void setup_mfc_slbs(struct spu_state *csa, struct spu *spu)
+{
+ u64 code_slb[2];
+ u64 lscsa_slb[2];
+
+ /* Save, Step 47:
+ * Restore, Step 30.
+ * If MFC_SR1[R]=1, write 0 to SLB_Invalidate_All
+ * register, then initialize SLB_VSID and SLB_ESID
+ * to provide access to SPU context save code and
+ * LSCSA.
+ *
+ * This implementation places both the context
+ * switch code and LSCSA in kernel address space.
+ *
+ * Further this implementation assumes that the
+ * MFC_SR1[R]=1 (in other words, assume that
+ * translation is desired by OS environment).
+ */
+ invalidate_slbs(csa, spu);
+ get_kernel_slb((unsigned long)&spu_save_code[0], code_slb);
+ get_kernel_slb((unsigned long)csa->lscsa, lscsa_slb);
+ load_mfc_slb(spu, code_slb, 0);
+ if ((lscsa_slb[0] != code_slb[0]) || (lscsa_slb[1] != code_slb[1]))
+ load_mfc_slb(spu, lscsa_slb, 1);
+}
+
+static inline void set_switch_active(struct spu_state *csa, struct spu *spu)
+{
+ /* Save, Step 48:
+ * Restore, Step 23.
+ * Change the software context switch pending flag
+ * to context switch active.
+ */
+ set_bit(SPU_CONTEXT_SWITCH_ACTIVE_nr, &spu->flags);
+ clear_bit(SPU_CONTEXT_SWITCH_PENDING_nr, &spu->flags);
+ mb();
+}
+
+static inline void enable_interrupts(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_priv1 __iomem *priv1 = spu->priv1;
+ unsigned long flags, class1_mask = CLASS1_ENABLE_SEGMENT_FAULT_INTR |
+ CLASS1_ENABLE_STORAGE_FAULT_INTR;
+
+ /* Save, Step 49:
+ * Restore, Step 22:
+ * Reset and then enable interrupts, as
+ * needed by OS.
+ *
+ * This implementation enables only class1
+ * (translation) interrupts.
+ */
+ local_irq_save(flags);
+ out_be64(&priv1->int_stat_class0_RW, ~(0UL));
+ out_be64(&priv1->int_stat_class1_RW, ~(0UL));
+ out_be64(&priv1->int_stat_class2_RW, ~(0UL));
+ out_be64(&priv1->int_mask_class0_RW, 0UL);
+ out_be64(&priv1->int_mask_class1_RW, class1_mask);
+ out_be64(&priv1->int_mask_class2_RW, 0UL);
+ local_irq_restore(flags);
+}
+
+static inline int send_mfc_dma(struct spu *spu, unsigned long ea,
+ unsigned int ls_offset, unsigned int size,
+ unsigned int tag, unsigned int rclass,
+ unsigned int cmd)
+{
+ struct spu_problem __iomem *prob = spu->problem;
+ union mfc_tag_size_class_cmd command;
+ unsigned int transfer_size;
+ volatile unsigned int status = 0x0;
+
+ while (size > 0) {
+ transfer_size =
+ (size > MFC_MAX_DMA_SIZE) ? MFC_MAX_DMA_SIZE : size;
+ command.u.mfc_size = transfer_size;
+ command.u.mfc_tag = tag;
+ command.u.mfc_rclassid = rclass;
+ command.u.mfc_cmd = cmd;
+ do {
+ out_be32(&prob->mfc_lsa_W, ls_offset);
+ out_be64(&prob->mfc_ea_W, ea);
+ out_be64(&prob->mfc_union_W.all64, command.all64);
+ status =
+ in_be32(&prob->mfc_union_W.by32.mfc_class_cmd32);
+ if (unlikely(status & 0x2)) {
+ cpu_relax();
+ }
+ } while (status & 0x3);
+ size -= transfer_size;
+ ea += transfer_size;
+ ls_offset += transfer_size;
+ }
+ return 0;
+}
+
+static inline void save_ls_16kb(struct spu_state *csa, struct spu *spu)
+{
+ unsigned long addr = (unsigned long)&csa->lscsa->ls[0];
+ unsigned int ls_offset = 0x0;
+ unsigned int size = 16384;
+ unsigned int tag = 0;
+ unsigned int rclass = 0;
+ unsigned int cmd = MFC_PUT_CMD;
+
+ /* Save, Step 50:
+ * Issue a DMA command to copy the first 16K bytes
+ * of local storage to the CSA.
+ */
+ send_mfc_dma(spu, addr, ls_offset, size, tag, rclass, cmd);
+}
+
+static inline void set_spu_npc(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_problem __iomem *prob = spu->problem;
+
+ /* Save, Step 51:
+ * Restore, Step 31.
+ * Write SPU_NPC[IE]=0 and SPU_NPC[LSA] to entry
+ * point address of context save code in local
+ * storage.
+ *
+ * This implementation uses SPU-side save/restore
+ * programs with entry points at LSA of 0.
+ */
+ out_be32(&prob->spu_npc_RW, 0);
+ eieio();
+}
+
+static inline void set_signot1(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_problem __iomem *prob = spu->problem;
+ union {
+ u64 ull;
+ u32 ui[2];
+ } addr64;
+
+ /* Save, Step 52:
+ * Restore, Step 32:
+ * Write SPU_Sig_Notify_1 register with upper 32-bits
+ * of the CSA.LSCSA effective address.
+ */
+ addr64.ull = (u64) csa->lscsa;
+ out_be32(&prob->signal_notify1, addr64.ui[0]);
+ eieio();
+}
+
+static inline void set_signot2(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_problem __iomem *prob = spu->problem;
+ union {
+ u64 ull;
+ u32 ui[2];
+ } addr64;
+
+ /* Save, Step 53:
+ * Restore, Step 33:
+ * Write SPU_Sig_Notify_2 register with lower 32-bits
+ * of the CSA.LSCSA effective address.
+ */
+ addr64.ull = (u64) csa->lscsa;
+ out_be32(&prob->signal_notify2, addr64.ui[1]);
+ eieio();
+}
+
+static inline void send_save_code(struct spu_state *csa, struct spu *spu)
+{
+ unsigned long addr = (unsigned long)&spu_save_code[0];
+ unsigned int ls_offset = 0x0;
+ unsigned int size = sizeof(spu_save_code);
+ unsigned int tag = 0;
+ unsigned int rclass = 0;
+ unsigned int cmd = MFC_GETFS_CMD;
+
+ /* Save, Step 54:
+ * Issue a DMA command to copy context save code
+ * to local storage and start SPU.
+ */
+ send_mfc_dma(spu, addr, ls_offset, size, tag, rclass, cmd);
+}
+
+static inline void set_ppu_querymask(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_problem __iomem *prob = spu->problem;
+
+ /* Save, Step 55:
+ * Restore, Step 38.
+ * Write PPU_QueryMask=1 (enable Tag Group 0)
+ * and issue eieio instruction.
+ */
+ out_be32(&prob->dma_querymask_RW, MFC_TAGID_TO_TAGMASK(0));
+ eieio();
+}
+
+static inline void wait_tag_complete(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_priv1 __iomem *priv1 = spu->priv1;
+ struct spu_problem __iomem *prob = spu->problem;
+ u32 mask = MFC_TAGID_TO_TAGMASK(0);
+ unsigned long flags;
+
+ /* Save, Step 56:
+ * Restore, Step 39.
+ * Restore, Step 39.
+ * Restore, Step 46.
+ * Poll PPU_TagStatus[gn] until 01 (Tag group 0 complete)
+ * or write PPU_QueryType[TS]=01 and wait for Tag Group
+ * Complete Interrupt. Write INT_Stat_Class0 or
+ * INT_Stat_Class2 with value of 'handled'.
+ */
+ POLL_WHILE_FALSE(in_be32(&prob->dma_tagstatus_R) & mask);
+
+ local_irq_save(flags);
+ out_be64(&priv1->int_stat_class0_RW, ~(0UL));
+ out_be64(&priv1->int_stat_class2_RW, ~(0UL));
+ local_irq_restore(flags);
+}
+
+static inline void wait_spu_stopped(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_priv1 __iomem *priv1 = spu->priv1;
+ struct spu_problem __iomem *prob = spu->problem;
+ unsigned long flags;
+
+ /* Save, Step 57:
+ * Restore, Step 40.
+ * Poll until SPU_Status[R]=0 or wait for SPU Class 0
+ * or SPU Class 2 interrupt. Write INT_Stat_class0
+ * or INT_Stat_class2 with value of handled.
+ */
+ POLL_WHILE_TRUE(in_be32(&prob->spu_status_R) & SPU_STATUS_RUNNING);
+
+ local_irq_save(flags);
+ out_be64(&priv1->int_stat_class0_RW, ~(0UL));
+ out_be64(&priv1->int_stat_class2_RW, ~(0UL));
+ local_irq_restore(flags);
+}
+
+static inline int check_save_status(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_problem __iomem *prob = spu->problem;
+ u32 complete;
+
+ /* Save, Step 54:
+ * If SPU_Status[P]=1 and SPU_Status[SC] = "success",
+ * context save succeeded, otherwise context save
+ * failed.
+ */
+ complete = ((SPU_SAVE_COMPLETE << SPU_STOP_STATUS_SHIFT) |
+ SPU_STATUS_STOPPED_BY_STOP);
+ return (in_be32(&prob->spu_status_R) != complete) ? 1 : 0;
+}
+
+static inline void terminate_spu_app(struct spu_state *csa, struct spu *spu)
+{
+ /* Restore, Step 4:
+ * If required, notify the "using application" that
+ * the SPU task has been terminated. TBD.
+ */
+}
+
+static inline void suspend_mfc(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_priv2 __iomem *priv2 = spu->priv2;
+
+ /* Restore, Step 7:
+ * Restore, Step 47.
+ * Write MFC_Cntl[Dh,Sc]='1','1' to suspend
+ * the queue and halt the decrementer.
+ */
+ out_be64(&priv2->mfc_control_RW, MFC_CNTL_SUSPEND_DMA_QUEUE |
+ MFC_CNTL_DECREMENTER_HALTED);
+ eieio();
+}
+
+static inline void wait_suspend_mfc_complete(struct spu_state *csa,
+ struct spu *spu)
+{
+ struct spu_priv2 __iomem *priv2 = spu->priv2;
+
+ /* Restore, Step 8:
+ * Restore, Step 47.
+ * Poll MFC_CNTL[Ss] until 11 is returned.
+ */
+ POLL_WHILE_FALSE(in_be64(&priv2->mfc_control_RW) &
+ MFC_CNTL_SUSPEND_COMPLETE);
+}
+
+static inline int suspend_spe(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_problem __iomem *prob = spu->problem;
+
+ /* Restore, Step 9:
+ * If SPU_Status[R]=1, stop SPU execution
+ * and wait for stop to complete.
+ *
+ * Returns 1 if SPU_Status[R]=1 on entry.
+ * 0 otherwise
+ */
+ if (in_be32(&prob->spu_status_R) & SPU_STATUS_RUNNING) {
+ if (in_be32(&prob->spu_status_R) &
+ SPU_STATUS_ISOLATED_EXIT_STAUTUS) {
+ POLL_WHILE_TRUE(in_be32(&prob->spu_status_R) &
+ SPU_STATUS_RUNNING);
+ }
+ if ((in_be32(&prob->spu_status_R) &
+ SPU_STATUS_ISOLATED_LOAD_STAUTUS)
+ || (in_be32(&prob->spu_status_R) &
+ SPU_STATUS_ISOLATED_STATE)) {
+ out_be32(&prob->spu_runcntl_RW, SPU_RUNCNTL_STOP);
+ eieio();
+ POLL_WHILE_TRUE(in_be32(&prob->spu_status_R) &
+ SPU_STATUS_RUNNING);
+ out_be32(&prob->spu_runcntl_RW, 0x2);
+ eieio();
+ POLL_WHILE_TRUE(in_be32(&prob->spu_status_R) &
+ SPU_STATUS_RUNNING);
+ }
+ if (in_be32(&prob->spu_status_R) &
+ SPU_STATUS_WAITING_FOR_CHANNEL) {
+ out_be32(&prob->spu_runcntl_RW, SPU_RUNCNTL_STOP);
+ eieio();
+ POLL_WHILE_TRUE(in_be32(&prob->spu_status_R) &
+ SPU_STATUS_RUNNING);
+ }
+ return 1;
+ }
+ return 0;
+}
+
+static inline void clear_spu_status(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_problem __iomem *prob = spu->problem;
+ struct spu_priv1 __iomem *priv1 = spu->priv1;
+
+ /* Restore, Step 10:
+ * If SPU_Status[R]=0 and SPU_Status[E,L,IS]=1,
+ * release SPU from isolate state.
+ */
+ if (!(in_be32(&prob->spu_status_R) & SPU_STATUS_RUNNING)) {
+ if (in_be32(&prob->spu_status_R) &
+ SPU_STATUS_ISOLATED_EXIT_STAUTUS) {
+ out_be64(&priv1->mfc_sr1_RW,
+ MFC_STATE1_MASTER_RUN_CONTROL_MASK);
+ eieio();
+ out_be32(&prob->spu_runcntl_RW, SPU_RUNCNTL_RUNNABLE);
+ eieio();
+ POLL_WHILE_TRUE(in_be32(&prob->spu_status_R) &
+ SPU_STATUS_RUNNING);
+ }
+ if ((in_be32(&prob->spu_status_R) &
+ SPU_STATUS_ISOLATED_LOAD_STAUTUS)
+ || (in_be32(&prob->spu_status_R) &
+ SPU_STATUS_ISOLATED_STATE)) {
+ out_be64(&priv1->mfc_sr1_RW,
+ MFC_STATE1_MASTER_RUN_CONTROL_MASK);
+ eieio();
+ out_be32(&prob->spu_runcntl_RW, 0x2);
+ eieio();
+ POLL_WHILE_TRUE(in_be32(&prob->spu_status_R) &
+ SPU_STATUS_RUNNING);
+ }
+ }
+}
+
+static inline void reset_ch_part1(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_priv2 __iomem *priv2 = spu->priv2;
+ u64 ch_indices[7] = { 0UL, 1UL, 3UL, 4UL, 24UL, 25UL, 27UL };
+ u64 idx;
+ int i;
+
+ /* Restore, Step 20:
+ * Reset the following CH: [0,1,3,4,24,25,27]
+ */
+ for (i = 0; i < 7; i++) {
+ idx = ch_indices[i];
+ out_be64(&priv2->spu_chnlcntptr_RW, idx);
+ eieio();
+ out_be64(&priv2->spu_chnldata_RW, 0UL);
+ out_be64(&priv2->spu_chnlcnt_RW, 0UL);
+ eieio();
+ }
+}
+
+static inline void reset_ch_part2(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_priv2 __iomem *priv2 = spu->priv2;
+ u64 ch_indices[5] = { 21UL, 23UL, 28UL, 29UL, 30UL };
+ u64 ch_counts[5] = { 16UL, 1UL, 1UL, 0UL, 1UL };
+ u64 idx;
+ int i;
+
+ /* Restore, Step 21:
+ * Reset the following CH: [21, 23, 28, 29, 30]
+ */
+ for (i = 0; i < 5; i++) {
+ idx = ch_indices[i];
+ out_be64(&priv2->spu_chnlcntptr_RW, idx);
+ eieio();
+ out_be64(&priv2->spu_chnlcnt_RW, ch_counts[i]);
+ eieio();
+ }
+}
+
+static inline void setup_spu_status_part1(struct spu_state *csa,
+ struct spu *spu)
+{
+ u32 status_P = SPU_STATUS_STOPPED_BY_STOP;
+ u32 status_I = SPU_STATUS_INVALID_INSTR;
+ u32 status_H = SPU_STATUS_STOPPED_BY_HALT;
+ u32 status_S = SPU_STATUS_SINGLE_STEP;
+ u32 status_S_I = SPU_STATUS_SINGLE_STEP | SPU_STATUS_INVALID_INSTR;
+ u32 status_S_P = SPU_STATUS_SINGLE_STEP | SPU_STATUS_STOPPED_BY_STOP;
+ u32 status_P_H = SPU_STATUS_STOPPED_BY_HALT |SPU_STATUS_STOPPED_BY_STOP;
+ u32 status_P_I = SPU_STATUS_STOPPED_BY_STOP |SPU_STATUS_INVALID_INSTR;
+ u32 status_code;
+
+ /* Restore, Step 27:
+ * If the CSA.SPU_Status[I,S,H,P]=1 then add the correct
+ * instruction sequence to the end of the SPU based restore
+ * code (after the "context restored" stop and signal) to
+ * restore the correct SPU status.
+ *
+ * NOTE: Rather than modifying the SPU executable, we
+ * instead add a new 'stopped_status' field to the
+ * LSCSA. The SPU-side restore reads this field and
+ * takes the appropriate action when exiting.
+ */
+
+ status_code =
+ (csa->prob.spu_status_R >> SPU_STOP_STATUS_SHIFT) & 0xFFFF;
+ if (csa->prob.spu_status_R & status_P_I) {
+
+ /* SPU_Status[P,I]=1 - Illegal Instruction followed
+ * by Stop and Signal instruction, followed by 'br -4'.
+ *
+ */
+ csa->lscsa->stopped_status.slot[0] = SPU_STOPPED_STATUS_P_I;
+ csa->lscsa->stopped_status.slot[1] = status_code;
+
+ } else if (csa->prob.spu_status_R & status_P_H) {
+
+ /* SPU_Status[P,H]=1 - Halt Conditional, followed
+ * by Stop and Signal instruction, followed by
+ * 'br -4'.
+ */
+ csa->lscsa->stopped_status.slot[0] = SPU_STOPPED_STATUS_P_H;
+ csa->lscsa->stopped_status.slot[1] = status_code;
+
+ } else if (csa->prob.spu_status_R & status_S_P) {
+
+ /* SPU_Status[S,P]=1 - Stop and Signal instruction
+ * followed by 'br -4'.
+ */
+ csa->lscsa->stopped_status.slot[0] = SPU_STOPPED_STATUS_S_P;
+ csa->lscsa->stopped_status.slot[1] = status_code;
+
+ } else if (csa->prob.spu_status_R & status_S_I) {
+
+ /* SPU_Status[S,I]=1 - Illegal instruction followed
+ * by 'br -4'.
+ */
+ csa->lscsa->stopped_status.slot[0] = SPU_STOPPED_STATUS_S_I;
+ csa->lscsa->stopped_status.slot[1] = status_code;
+
+ } else if (csa->prob.spu_status_R & status_P) {
+
+ /* SPU_Status[P]=1 - Stop and Signal instruction
+ * followed by 'br -4'.
+ */
+ csa->lscsa->stopped_status.slot[0] = SPU_STOPPED_STATUS_P;
+ csa->lscsa->stopped_status.slot[1] = status_code;
+
+ } else if (csa->prob.spu_status_R & status_H) {
+
+ /* SPU_Status[H]=1 - Halt Conditional, followed
+ * by 'br -4'.
+ */
+ csa->lscsa->stopped_status.slot[0] = SPU_STOPPED_STATUS_H;
+
+ } else if (csa->prob.spu_status_R & status_S) {
+
+ /* SPU_Status[S]=1 - Two nop instructions.
+ */
+ csa->lscsa->stopped_status.slot[0] = SPU_STOPPED_STATUS_S;
+
+ } else if (csa->prob.spu_status_R & status_I) {
+
+ /* SPU_Status[I]=1 - Illegal instruction followed
+ * by 'br -4'.
+ */
+ csa->lscsa->stopped_status.slot[0] = SPU_STOPPED_STATUS_I;
+
+ }
+}
+
+static inline void setup_spu_status_part2(struct spu_state *csa,
+ struct spu *spu)
+{
+ u32 mask;
+
+ /* Restore, Step 28:
+ * If the CSA.SPU_Status[I,S,H,P,R]=0 then
+ * add a 'br *' instruction to the end of
+ * the SPU based restore code.
+ *
+ * NOTE: Rather than modifying the SPU executable, we
+ * instead add a new 'stopped_status' field to the
+ * LSCSA. The SPU-side restore reads this field and
+ * takes the appropriate action when exiting.
+ */
+ mask = SPU_STATUS_INVALID_INSTR |
+ SPU_STATUS_SINGLE_STEP |
+ SPU_STATUS_STOPPED_BY_HALT |
+ SPU_STATUS_STOPPED_BY_STOP | SPU_STATUS_RUNNING;
+ if (!(csa->prob.spu_status_R & mask)) {
+ csa->lscsa->stopped_status.slot[0] = SPU_STOPPED_STATUS_R;
+ }
+}
+
+static inline void restore_mfc_rag(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_priv1 __iomem *priv1 = spu->priv1;
+
+ /* Restore, Step 29:
+ * Restore RA_GROUP_ID register and the
+ * RA_ENABLE reigster from the CSA.
+ */
+ out_be64(&priv1->resource_allocation_groupID_RW,
+ csa->priv1.resource_allocation_groupID_RW);
+ out_be64(&priv1->resource_allocation_enable_RW,
+ csa->priv1.resource_allocation_enable_RW);
+}
+
+static inline void send_restore_code(struct spu_state *csa, struct spu *spu)
+{
+ unsigned long addr = (unsigned long)&spu_restore_code[0];
+ unsigned int ls_offset = 0x0;
+ unsigned int size = sizeof(spu_restore_code);
+ unsigned int tag = 0;
+ unsigned int rclass = 0;
+ unsigned int cmd = MFC_GETFS_CMD;
+
+ /* Restore, Step 37:
+ * Issue MFC DMA command to copy context
+ * restore code to local storage.
+ */
+ send_mfc_dma(spu, addr, ls_offset, size, tag, rclass, cmd);
+}
+
+static inline void setup_decr(struct spu_state *csa, struct spu *spu)
+{
+ /* Restore, Step 34:
+ * If CSA.MFC_CNTL[Ds]=1 (decrementer was
+ * running) then adjust decrementer, set
+ * decrementer running status in LSCSA,
+ * and set decrementer "wrapped" status
+ * in LSCSA.
+ */
+ if (csa->priv2.mfc_control_RW & MFC_CNTL_DECREMENTER_RUNNING) {
+ cycles_t resume_time = get_cycles();
+ cycles_t delta_time = resume_time - csa->suspend_time;
+
+ csa->lscsa->decr.slot[0] = delta_time;
+ }
+}
+
+static inline void setup_ppu_mb(struct spu_state *csa, struct spu *spu)
+{
+ /* Restore, Step 35:
+ * Copy the CSA.PU_MB data into the LSCSA.
+ */
+ csa->lscsa->ppu_mb.slot[0] = csa->prob.pu_mb_R;
+}
+
+static inline void setup_ppuint_mb(struct spu_state *csa, struct spu *spu)
+{
+ /* Restore, Step 36:
+ * Copy the CSA.PUINT_MB data into the LSCSA.
+ */
+ csa->lscsa->ppuint_mb.slot[0] = csa->priv2.puint_mb_R;
+}
+
+static inline int check_restore_status(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_problem __iomem *prob = spu->problem;
+ u32 complete;
+
+ /* Restore, Step 40:
+ * If SPU_Status[P]=1 and SPU_Status[SC] = "success",
+ * context restore succeeded, otherwise context restore
+ * failed.
+ */
+ complete = ((SPU_RESTORE_COMPLETE << SPU_STOP_STATUS_SHIFT) |
+ SPU_STATUS_STOPPED_BY_STOP);
+ return (in_be32(&prob->spu_status_R) != complete) ? 1 : 0;
+}
+
+static inline void restore_spu_privcntl(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_priv2 __iomem *priv2 = spu->priv2;
+
+ /* Restore, Step 41:
+ * Restore SPU_PrivCntl from the CSA.
+ */
+ out_be64(&priv2->spu_privcntl_RW, csa->priv2.spu_privcntl_RW);
+ eieio();
+}
+
+static inline void restore_status_part1(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_problem __iomem *prob = spu->problem;
+ u32 mask;
+
+ /* Restore, Step 42:
+ * If any CSA.SPU_Status[I,S,H,P]=1, then
+ * restore the error or single step state.
+ */
+ mask = SPU_STATUS_INVALID_INSTR |
+ SPU_STATUS_SINGLE_STEP |
+ SPU_STATUS_STOPPED_BY_HALT | SPU_STATUS_STOPPED_BY_STOP;
+ if (csa->prob.spu_status_R & mask) {
+ out_be32(&prob->spu_runcntl_RW, SPU_RUNCNTL_RUNNABLE);
+ eieio();
+ POLL_WHILE_TRUE(in_be32(&prob->spu_status_R) &
+ SPU_STATUS_RUNNING);
+ }
+}
+
+static inline void restore_status_part2(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_problem __iomem *prob = spu->problem;
+ u32 mask;
+
+ /* Restore, Step 43:
+ * If all CSA.SPU_Status[I,S,H,P,R]=0 then write
+ * SPU_RunCntl[R0R1]='01', wait for SPU_Status[R]=1,
+ * then write '00' to SPU_RunCntl[R0R1] and wait
+ * for SPU_Status[R]=0.
+ */
+ mask = SPU_STATUS_INVALID_INSTR |
+ SPU_STATUS_SINGLE_STEP |
+ SPU_STATUS_STOPPED_BY_HALT |
+ SPU_STATUS_STOPPED_BY_STOP | SPU_STATUS_RUNNING;
+ if (!(csa->prob.spu_status_R & mask)) {
+ out_be32(&prob->spu_runcntl_RW, SPU_RUNCNTL_RUNNABLE);
+ eieio();
+ POLL_WHILE_FALSE(in_be32(&prob->spu_status_R) &
+ SPU_STATUS_RUNNING);
+ out_be32(&prob->spu_runcntl_RW, SPU_RUNCNTL_STOP);
+ eieio();
+ POLL_WHILE_TRUE(in_be32(&prob->spu_status_R) &
+ SPU_STATUS_RUNNING);
+ }
+}
+
+static inline void restore_ls_16kb(struct spu_state *csa, struct spu *spu)
+{
+ unsigned long addr = (unsigned long)&csa->lscsa->ls[0];
+ unsigned int ls_offset = 0x0;
+ unsigned int size = 16384;
+ unsigned int tag = 0;
+ unsigned int rclass = 0;
+ unsigned int cmd = MFC_GET_CMD;
+
+ /* Restore, Step 44:
+ * Issue a DMA command to restore the first
+ * 16kb of local storage from CSA.
+ */
+ send_mfc_dma(spu, addr, ls_offset, size, tag, rclass, cmd);
+}
+
+static inline void clear_interrupts(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_priv1 __iomem *priv1 = spu->priv1;
+ unsigned long flags;
+
+ /* Restore, Step 49:
+ * Write INT_MASK_class0 with value of 0.
+ * Write INT_MASK_class1 with value of 0.
+ * Write INT_MASK_class2 with value of 0.
+ * Write INT_STAT_class0 with value of -1.
+ * Write INT_STAT_class1 with value of -1.
+ * Write INT_STAT_class2 with value of -1.
+ */
+ local_irq_save(flags);
+ out_be64(&priv1->int_mask_class0_RW, 0UL);
+ out_be64(&priv1->int_mask_class1_RW, 0UL);
+ out_be64(&priv1->int_mask_class2_RW, 0UL);
+ out_be64(&priv1->int_stat_class0_RW, ~(0UL));
+ out_be64(&priv1->int_stat_class1_RW, ~(0UL));
+ out_be64(&priv1->int_stat_class2_RW, ~(0UL));
+ local_irq_restore(flags);
+}
+
+static inline void restore_mfc_queues(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_priv2 __iomem *priv2 = spu->priv2;
+ int i;
+
+ /* Restore, Step 50:
+ * If MFC_Cntl[Se]!=0 then restore
+ * MFC command queues.
+ */
+ if ((csa->priv2.mfc_control_RW & MFC_CNTL_DMA_QUEUES_EMPTY_MASK) == 0) {
+ for (i = 0; i < 8; i++) {
+ out_be64(&priv2->puq[i].mfc_cq_data0_RW,
+ csa->priv2.puq[i].mfc_cq_data0_RW);
+ out_be64(&priv2->puq[i].mfc_cq_data1_RW,
+ csa->priv2.puq[i].mfc_cq_data1_RW);
+ out_be64(&priv2->puq[i].mfc_cq_data2_RW,
+ csa->priv2.puq[i].mfc_cq_data2_RW);
+ out_be64(&priv2->puq[i].mfc_cq_data3_RW,
+ csa->priv2.puq[i].mfc_cq_data3_RW);
+ }
+ for (i = 0; i < 16; i++) {
+ out_be64(&priv2->spuq[i].mfc_cq_data0_RW,
+ csa->priv2.spuq[i].mfc_cq_data0_RW);
+ out_be64(&priv2->spuq[i].mfc_cq_data1_RW,
+ csa->priv2.spuq[i].mfc_cq_data1_RW);
+ out_be64(&priv2->spuq[i].mfc_cq_data2_RW,
+ csa->priv2.spuq[i].mfc_cq_data2_RW);
+ out_be64(&priv2->spuq[i].mfc_cq_data3_RW,
+ csa->priv2.spuq[i].mfc_cq_data3_RW);
+ }
+ }
+ eieio();
+}
+
+static inline void restore_ppu_querymask(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_problem __iomem *prob = spu->problem;
+
+ /* Restore, Step 51:
+ * Restore the PPU_QueryMask register from CSA.
+ */
+ out_be32(&prob->dma_querymask_RW, csa->prob.dma_querymask_RW);
+ eieio();
+}
+
+static inline void restore_ppu_querytype(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_problem __iomem *prob = spu->problem;
+
+ /* Restore, Step 52:
+ * Restore the PPU_QueryType register from CSA.
+ */
+ out_be32(&prob->dma_querytype_RW, csa->prob.dma_querytype_RW);
+ eieio();
+}
+
+static inline void restore_mfc_csr_tsq(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_priv2 __iomem *priv2 = spu->priv2;
+
+ /* Restore, Step 53:
+ * Restore the MFC_CSR_TSQ register from CSA.
+ */
+ out_be64(&priv2->spu_tag_status_query_RW,
+ csa->priv2.spu_tag_status_query_RW);
+ eieio();
+}
+
+static inline void restore_mfc_csr_cmd(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_priv2 __iomem *priv2 = spu->priv2;
+
+ /* Restore, Step 54:
+ * Restore the MFC_CSR_CMD1 and MFC_CSR_CMD2
+ * registers from CSA.
+ */
+ out_be64(&priv2->spu_cmd_buf1_RW, csa->priv2.spu_cmd_buf1_RW);
+ out_be64(&priv2->spu_cmd_buf2_RW, csa->priv2.spu_cmd_buf2_RW);
+ eieio();
+}
+
+static inline void restore_mfc_csr_ato(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_priv2 __iomem *priv2 = spu->priv2;
+
+ /* Restore, Step 55:
+ * Restore the MFC_CSR_ATO register from CSA.
+ */
+ out_be64(&priv2->spu_atomic_status_RW, csa->priv2.spu_atomic_status_RW);
+}
+
+static inline void restore_mfc_tclass_id(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_priv1 __iomem *priv1 = spu->priv1;
+
+ /* Restore, Step 56:
+ * Restore the MFC_TCLASS_ID register from CSA.
+ */
+ out_be64(&priv1->mfc_tclass_id_RW, csa->priv1.mfc_tclass_id_RW);
+ eieio();
+}
+
+static inline void set_llr_event(struct spu_state *csa, struct spu *spu)
+{
+ u64 ch0_cnt, ch0_data;
+ u64 ch1_data;
+
+ /* Restore, Step 57:
+ * Set the Lock Line Reservation Lost Event by:
+ * 1. OR CSA.SPU_Event_Status with bit 21 (Lr) set to 1.
+ * 2. If CSA.SPU_Channel_0_Count=0 and
+ * CSA.SPU_Wr_Event_Mask[Lr]=1 and
+ * CSA.SPU_Event_Status[Lr]=0 then set
+ * CSA.SPU_Event_Status_Count=1.
+ */
+ ch0_cnt = csa->spu_chnlcnt_RW[0];
+ ch0_data = csa->spu_chnldata_RW[0];
+ ch1_data = csa->spu_chnldata_RW[1];
+ csa->spu_chnldata_RW[0] |= MFC_LLR_LOST_EVENT;
+ if ((ch0_cnt == 0) && !(ch0_data & MFC_LLR_LOST_EVENT) &&
+ (ch1_data & MFC_LLR_LOST_EVENT)) {
+ csa->spu_chnlcnt_RW[0] = 1;
+ }
+}
+
+static inline void restore_decr_wrapped(struct spu_state *csa, struct spu *spu)
+{
+ /* Restore, Step 58:
+ * If the status of the CSA software decrementer
+ * "wrapped" flag is set, OR in a '1' to
+ * CSA.SPU_Event_Status[Tm].
+ */
+ if (csa->lscsa->decr_status.slot[0] == 1) {
+ csa->spu_chnldata_RW[0] |= 0x20;
+ }
+ if ((csa->lscsa->decr_status.slot[0] == 1) &&
+ (csa->spu_chnlcnt_RW[0] == 0 &&
+ ((csa->spu_chnldata_RW[2] & 0x20) == 0x0) &&
+ ((csa->spu_chnldata_RW[0] & 0x20) != 0x1))) {
+ csa->spu_chnlcnt_RW[0] = 1;
+ }
+}
+
+static inline void restore_ch_part1(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_priv2 __iomem *priv2 = spu->priv2;
+ u64 idx, ch_indices[7] = { 0UL, 1UL, 3UL, 4UL, 24UL, 25UL, 27UL };
+ int i;
+
+ /* Restore, Step 59:
+ * Restore the following CH: [0,1,3,4,24,25,27]
+ */
+ for (i = 0; i < 7; i++) {
+ idx = ch_indices[i];
+ out_be64(&priv2->spu_chnlcntptr_RW, idx);
+ eieio();
+ out_be64(&priv2->spu_chnldata_RW, csa->spu_chnldata_RW[idx]);
+ out_be64(&priv2->spu_chnlcnt_RW, csa->spu_chnlcnt_RW[idx]);
+ eieio();
+ }
+}
+
+static inline void restore_ch_part2(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_priv2 __iomem *priv2 = spu->priv2;
+ u64 ch_indices[3] = { 9UL, 21UL, 23UL };
+ u64 ch_counts[3] = { 1UL, 16UL, 1UL };
+ u64 idx;
+ int i;
+
+ /* Restore, Step 60:
+ * Restore the following CH: [9,21,23].
+ */
+ ch_counts[0] = 1UL;
+ ch_counts[1] = csa->spu_chnlcnt_RW[21];
+ ch_counts[2] = 1UL;
+ for (i = 0; i < 3; i++) {
+ idx = ch_indices[i];
+ out_be64(&priv2->spu_chnlcntptr_RW, idx);
+ eieio();
+ out_be64(&priv2->spu_chnlcnt_RW, ch_counts[i]);
+ eieio();
+ }
+}
+
+static inline void restore_spu_lslr(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_priv2 __iomem *priv2 = spu->priv2;
+
+ /* Restore, Step 61:
+ * Restore the SPU_LSLR register from CSA.
+ */
+ out_be64(&priv2->spu_lslr_RW, csa->priv2.spu_lslr_RW);
+ eieio();
+}
+
+static inline void restore_spu_cfg(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_priv2 __iomem *priv2 = spu->priv2;
+
+ /* Restore, Step 62:
+ * Restore the SPU_Cfg register from CSA.
+ */
+ out_be64(&priv2->spu_cfg_RW, csa->priv2.spu_cfg_RW);
+ eieio();
+}
+
+static inline void restore_pm_trace(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_priv2 __iomem *priv2 = spu->priv2;
+
+ /* Restore, Step 63:
+ * Restore PM_Trace_Tag_Wait_Mask from CSA.
+ */
+ out_be64(&priv2->spu_pm_trace_tag_status_RW,
+ csa->priv2.spu_pm_trace_tag_status_RW);
+ eieio();
+}
+
+static inline void restore_spu_npc(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_problem __iomem *prob = spu->problem;
+
+ /* Restore, Step 64:
+ * Restore SPU_NPC from CSA.
+ */
+ out_be32(&prob->spu_npc_RW, csa->prob.spu_npc_RW);
+ eieio();
+}
+
+static inline void restore_spu_mb(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_priv2 __iomem *priv2 = spu->priv2;
+ int i;
+
+ /* Restore, Step 65:
+ * Restore MFC_RdSPU_MB from CSA.
+ */
+ out_be64(&priv2->spu_chnlcntptr_RW, 29UL);
+ eieio();
+ out_be64(&priv2->spu_chnlcnt_RW, csa->spu_chnlcnt_RW[29]);
+ for (i = 0; i < 4; i++) {
+ out_be64(&priv2->spu_chnldata_RW, csa->pu_mailbox_data[i]);
+ }
+ eieio();
+}
+
+static inline void check_ppu_mb_stat(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_problem __iomem *prob = spu->problem;
+ u32 dummy = 0;
+
+ /* Restore, Step 66:
+ * If CSA.MB_Stat[P]=0 (mailbox empty) then
+ * read from the PPU_MB register.
+ */
+ if ((csa->prob.mb_stat_R & 0xFF) == 0) {
+ dummy = in_be32(&prob->pu_mb_R);
+ eieio();
+ }
+}
+
+static inline void check_ppuint_mb_stat(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_priv1 __iomem *priv1 = spu->priv1;
+ struct spu_priv2 __iomem *priv2 = spu->priv2;
+ u64 dummy = 0UL;
+
+ /* Restore, Step 66:
+ * If CSA.MB_Stat[I]=0 (mailbox empty) then
+ * read from the PPUINT_MB register.
+ */
+ if ((csa->prob.mb_stat_R & 0xFF0000) == 0) {
+ dummy = in_be64(&priv2->puint_mb_R);
+ eieio();
+ out_be64(&priv1->int_stat_class2_RW,
+ CLASS2_ENABLE_MAILBOX_INTR);
+ eieio();
+ }
+}
+
+static inline void restore_mfc_slbs(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_priv2 __iomem *priv2 = spu->priv2;
+ int i;
+
+ /* Restore, Step 68:
+ * If MFC_SR1[R]='1', restore SLBs from CSA.
+ */
+ if (csa->priv1.mfc_sr1_RW & MFC_STATE1_RELOCATE_MASK) {
+ for (i = 0; i < 8; i++) {
+ out_be64(&priv2->slb_index_W, i);
+ eieio();
+ out_be64(&priv2->slb_esid_RW, csa->slb_esid_RW[i]);
+ out_be64(&priv2->slb_vsid_RW, csa->slb_vsid_RW[i]);
+ eieio();
+ }
+ out_be64(&priv2->slb_index_W, csa->priv2.slb_index_W);
+ eieio();
+ }
+}
+
+static inline void restore_mfc_sr1(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_priv1 __iomem *priv1 = spu->priv1;
+
+ /* Restore, Step 69:
+ * Restore the MFC_SR1 register from CSA.
+ */
+ out_be64(&priv1->mfc_sr1_RW, csa->priv1.mfc_sr1_RW);
+ eieio();
+}
+
+static inline void restore_other_spu_access(struct spu_state *csa,
+ struct spu *spu)
+{
+ /* Restore, Step 70:
+ * Restore other SPU mappings to this SPU. TBD.
+ */
+}
+
+static inline void restore_spu_runcntl(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_problem __iomem *prob = spu->problem;
+
+ /* Restore, Step 71:
+ * If CSA.SPU_Status[R]=1 then write
+ * SPU_RunCntl[R0R1]='01'.
+ */
+ if (csa->prob.spu_status_R & SPU_STATUS_RUNNING) {
+ out_be32(&prob->spu_runcntl_RW, SPU_RUNCNTL_RUNNABLE);
+ eieio();
+ }
+}
+
+static inline void restore_mfc_cntl(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_priv2 __iomem *priv2 = spu->priv2;
+
+ /* Restore, Step 72:
+ * Restore the MFC_CNTL register for the CSA.
+ */
+ out_be64(&priv2->mfc_control_RW, csa->priv2.mfc_control_RW);
+ eieio();
+}
+
+static inline void enable_user_access(struct spu_state *csa, struct spu *spu)
+{
+ /* Restore, Step 73:
+ * Enable user-space access (if provided) to this
+ * SPU by mapping the virtual pages assigned to
+ * the SPU memory-mapped I/O (MMIO) for problem
+ * state. TBD.
+ */
+}
+
+static inline void reset_switch_active(struct spu_state *csa, struct spu *spu)
+{
+ /* Restore, Step 74:
+ * Reset the "context switch active" flag.
+ */
+ clear_bit(SPU_CONTEXT_SWITCH_ACTIVE_nr, &spu->flags);
+ mb();
+}
+
+static inline void reenable_interrupts(struct spu_state *csa, struct spu *spu)
+{
+ struct spu_priv1 __iomem *priv1 = spu->priv1;
+ unsigned long flags;
+
+ /* Restore, Step 75:
+ * Re-enable SPU interrupts.
+ */
+ local_irq_save(flags);
+ out_be64(&priv1->int_mask_class0_RW, csa->priv1.int_mask_class0_RW);
+ out_be64(&priv1->int_mask_class1_RW, csa->priv1.int_mask_class1_RW);
+ out_be64(&priv1->int_mask_class2_RW, csa->priv1.int_mask_class2_RW);
+ local_irq_restore(flags);
+}
+
+static int quiece_spu(struct spu_state *prev, struct spu *spu)
+{
+ /*
+ * Combined steps 2-18 of SPU context save sequence, which
+ * quiesce the SPU state (disable SPU execution, MFC command
+ * queues, decrementer, SPU interrupts, etc.).
+ *
+ * Returns 0 on success.
+ * 2 if failed step 2.
+ * 6 if failed step 6.
+ */
+
+ if (check_spu_isolate(prev, spu)) { /* Step 2. */
+ return 2;
+ }
+ disable_interrupts(prev, spu); /* Step 3. */
+ set_watchdog_timer(prev, spu); /* Step 4. */
+ inhibit_user_access(prev, spu); /* Step 5. */
+ if (check_spu_isolate(prev, spu)) { /* Step 6. */
+ return 6;
+ }
+ set_switch_pending(prev, spu); /* Step 7. */
+ save_mfc_cntl(prev, spu); /* Step 8. */
+ save_spu_runcntl(prev, spu); /* Step 9. */
+ save_mfc_sr1(prev, spu); /* Step 10. */
+ save_spu_status(prev, spu); /* Step 11. */
+ save_mfc_decr(prev, spu); /* Step 12. */
+ halt_mfc_decr(prev, spu); /* Step 13. */
+ save_timebase(prev, spu); /* Step 14. */
+ remove_other_spu_access(prev, spu); /* Step 15. */
+ do_mfc_mssync(prev, spu); /* Step 16. */
+ issue_mfc_tlbie(prev, spu); /* Step 17. */
+ handle_pending_interrupts(prev, spu); /* Step 18. */
+
+ return 0;
+}
+
+static void save_csa(struct spu_state *prev, struct spu *spu)
+{
+ /*
+ * Combine steps 19-44 of SPU context save sequence, which
+ * save regions of the privileged & problem state areas.
+ */
+
+ save_mfc_queues(prev, spu); /* Step 19. */
+ save_ppu_querymask(prev, spu); /* Step 20. */
+ save_ppu_querytype(prev, spu); /* Step 21. */
+ save_mfc_csr_tsq(prev, spu); /* Step 22. */
+ save_mfc_csr_cmd(prev, spu); /* Step 23. */
+ save_mfc_csr_ato(prev, spu); /* Step 24. */
+ save_mfc_tclass_id(prev, spu); /* Step 25. */
+ set_mfc_tclass_id(prev, spu); /* Step 26. */
+ purge_mfc_queue(prev, spu); /* Step 27. */
+ wait_purge_complete(prev, spu); /* Step 28. */
+ save_mfc_slbs(prev, spu); /* Step 29. */
+ setup_mfc_sr1(prev, spu); /* Step 30. */
+ save_spu_npc(prev, spu); /* Step 31. */
+ save_spu_privcntl(prev, spu); /* Step 32. */
+ reset_spu_privcntl(prev, spu); /* Step 33. */
+ save_spu_lslr(prev, spu); /* Step 34. */
+ reset_spu_lslr(prev, spu); /* Step 35. */
+ save_spu_cfg(prev, spu); /* Step 36. */
+ save_pm_trace(prev, spu); /* Step 37. */
+ save_mfc_rag(prev, spu); /* Step 38. */
+ save_ppu_mb_stat(prev, spu); /* Step 39. */
+ save_ppu_mb(prev, spu); /* Step 40. */
+ save_ppuint_mb(prev, spu); /* Step 41. */
+ save_ch_part1(prev, spu); /* Step 42. */
+ save_spu_mb(prev, spu); /* Step 43. */
+ save_mfc_cmd(prev, spu); /* Step 44. */
+ reset_ch(prev, spu); /* Step 45. */
+}
+
+static void save_lscsa(struct spu_state *prev, struct spu *spu)
+{
+ /*
+ * Perform steps 46-57 of SPU context save sequence,
+ * which save regions of the local store and register
+ * file.
+ */
+
+ resume_mfc_queue(prev, spu); /* Step 46. */
+ setup_mfc_slbs(prev, spu); /* Step 47. */
+ set_switch_active(prev, spu); /* Step 48. */
+ enable_interrupts(prev, spu); /* Step 49. */
+ save_ls_16kb(prev, spu); /* Step 50. */
+ set_spu_npc(prev, spu); /* Step 51. */
+ set_signot1(prev, spu); /* Step 52. */
+ set_signot2(prev, spu); /* Step 53. */
+ send_save_code(prev, spu); /* Step 54. */
+ set_ppu_querymask(prev, spu); /* Step 55. */
+ wait_tag_complete(prev, spu); /* Step 56. */
+ wait_spu_stopped(prev, spu); /* Step 57. */
+}
+
+static void harvest(struct spu_state *prev, struct spu *spu)
+{
+ /*
+ * Perform steps 2-25 of SPU context restore sequence,
+ * which resets an SPU either after a failed save, or
+ * when using SPU for first time.
+ */
+
+ disable_interrupts(prev, spu); /* Step 2. */
+ inhibit_user_access(prev, spu); /* Step 3. */
+ terminate_spu_app(prev, spu); /* Step 4. */
+ set_switch_pending(prev, spu); /* Step 5. */
+ remove_other_spu_access(prev, spu); /* Step 6. */
+ suspend_mfc(prev, spu); /* Step 7. */
+ wait_suspend_mfc_complete(prev, spu); /* Step 8. */
+ if (!suspend_spe(prev, spu)) /* Step 9. */
+ clear_spu_status(prev, spu); /* Step 10. */
+ do_mfc_mssync(prev, spu); /* Step 11. */
+ issue_mfc_tlbie(prev, spu); /* Step 12. */
+ handle_pending_interrupts(prev, spu); /* Step 13. */
+ purge_mfc_queue(prev, spu); /* Step 14. */
+ wait_purge_complete(prev, spu); /* Step 15. */
+ reset_spu_privcntl(prev, spu); /* Step 16. */
+ reset_spu_lslr(prev, spu); /* Step 17. */
+ setup_mfc_sr1(prev, spu); /* Step 18. */
+ invalidate_slbs(prev, spu); /* Step 19. */
+ reset_ch_part1(prev, spu); /* Step 20. */
+ reset_ch_part2(prev, spu); /* Step 21. */
+ enable_interrupts(prev, spu); /* Step 22. */
+ set_switch_active(prev, spu); /* Step 23. */
+ set_mfc_tclass_id(prev, spu); /* Step 24. */
+ resume_mfc_queue(prev, spu); /* Step 25. */
+}
+
+static void restore_lscsa(struct spu_state *next, struct spu *spu)
+{
+ /*
+ * Perform steps 26-40 of SPU context restore sequence,
+ * which restores regions of the local store and register
+ * file.
+ */
+
+ set_watchdog_timer(next, spu); /* Step 26. */
+ setup_spu_status_part1(next, spu); /* Step 27. */
+ setup_spu_status_part2(next, spu); /* Step 28. */
+ restore_mfc_rag(next, spu); /* Step 29. */
+ setup_mfc_slbs(next, spu); /* Step 30. */
+ set_spu_npc(next, spu); /* Step 31. */
+ set_signot1(next, spu); /* Step 32. */
+ set_signot2(next, spu); /* Step 33. */
+ setup_decr(next, spu); /* Step 34. */
+ setup_ppu_mb(next, spu); /* Step 35. */
+ setup_ppuint_mb(next, spu); /* Step 36. */
+ send_restore_code(next, spu); /* Step 37. */
+ set_ppu_querymask(next, spu); /* Step 38. */
+ wait_tag_complete(next, spu); /* Step 39. */
+ wait_spu_stopped(next, spu); /* Step 40. */
+}
+
+static void restore_csa(struct spu_state *next, struct spu *spu)
+{
+ /*
+ * Combine steps 41-76 of SPU context restore sequence, which
+ * restore regions of the privileged & problem state areas.
+ */
+
+ restore_spu_privcntl(next, spu); /* Step 41. */
+ restore_status_part1(next, spu); /* Step 42. */
+ restore_status_part2(next, spu); /* Step 43. */
+ restore_ls_16kb(next, spu); /* Step 44. */
+ wait_tag_complete(next, spu); /* Step 45. */
+ suspend_mfc(next, spu); /* Step 46. */
+ wait_suspend_mfc_complete(next, spu); /* Step 47. */
+ issue_mfc_tlbie(next, spu); /* Step 48. */
+ clear_interrupts(next, spu); /* Step 49. */
+ restore_mfc_queues(next, spu); /* Step 50. */
+ restore_ppu_querymask(next, spu); /* Step 51. */
+ restore_ppu_querytype(next, spu); /* Step 52. */
+ restore_mfc_csr_tsq(next, spu); /* Step 53. */
+ restore_mfc_csr_cmd(next, spu); /* Step 54. */
+ restore_mfc_csr_ato(next, spu); /* Step 55. */
+ restore_mfc_tclass_id(next, spu); /* Step 56. */
+ set_llr_event(next, spu); /* Step 57. */
+ restore_decr_wrapped(next, spu); /* Step 58. */
+ restore_ch_part1(next, spu); /* Step 59. */
+ restore_ch_part2(next, spu); /* Step 60. */
+ restore_spu_lslr(next, spu); /* Step 61. */
+ restore_spu_cfg(next, spu); /* Step 62. */
+ restore_pm_trace(next, spu); /* Step 63. */
+ restore_spu_npc(next, spu); /* Step 64. */
+ restore_spu_mb(next, spu); /* Step 65. */
+ check_ppu_mb_stat(next, spu); /* Step 66. */
+ check_ppuint_mb_stat(next, spu); /* Step 67. */
+ restore_mfc_slbs(next, spu); /* Step 68. */
+ restore_mfc_sr1(next, spu); /* Step 69. */
+ restore_other_spu_access(next, spu); /* Step 70. */
+ restore_spu_runcntl(next, spu); /* Step 71. */
+ restore_mfc_cntl(next, spu); /* Step 72. */
+ enable_user_access(next, spu); /* Step 73. */
+ reset_switch_active(next, spu); /* Step 74. */
+ reenable_interrupts(next, spu); /* Step 75. */
+}
+
+static int __do_spu_save(struct spu_state *prev, struct spu *spu)
+{
+ int rc;
+
+ /*
+ * SPU context save can be broken into three phases:
+ *
+ * (a) quiesce [steps 2-16].
+ * (b) save of CSA, performed by PPE [steps 17-42]
+ * (c) save of LSCSA, mostly performed by SPU [steps 43-52].
+ *
+ * Returns 0 on success.
+ * 2,6 if failed to quiece SPU
+ * 53 if SPU-side of save failed.
+ */
+
+ rc = quiece_spu(prev, spu); /* Steps 2-16. */
+ switch (rc) {
+ default:
+ case 2:
+ case 6:
+ harvest(prev, spu);
+ return rc;
+ break;
+ case 0:
+ break;
+ }
+ save_csa(prev, spu); /* Steps 17-43. */
+ save_lscsa(prev, spu); /* Steps 44-53. */
+ return check_save_status(prev, spu); /* Step 54. */
+}
+
+static int __do_spu_restore(struct spu_state *next, struct spu *spu)
+{
+ int rc;
+
+ /*
+ * SPU context restore can be broken into three phases:
+ *
+ * (a) harvest (or reset) SPU [steps 2-24].
+ * (b) restore LSCSA [steps 25-40], mostly performed by SPU.
+ * (c) restore CSA [steps 41-76], performed by PPE.
+ *
+ * The 'harvest' step is not performed here, but rather
+ * as needed below.
+ */
+
+ restore_lscsa(next, spu); /* Steps 24-39. */
+ rc = check_restore_status(next, spu); /* Step 40. */
+ switch (rc) {
+ default:
+ /* Failed. Return now. */
+ return rc;
+ break;
+ case 0:
+ /* Fall through to next step. */
+ break;
+ }
+ restore_csa(next, spu);
+
+ return 0;
+}
+
/**
* spu_save - SPU context save, with locking.
* @prev: pointer to SPU context save area, to be saved.
@@ -61,9 +2084,13 @@
*/
int spu_save(struct spu_state *prev, struct spu *spu)
{
- /* XXX missing */
+ int rc;
- return 0;
+ acquire_spu_lock(spu); /* Step 1. */
+ rc = __do_spu_save(prev, spu); /* Steps 2-53. */
+ release_spu_lock(spu);
+
+ return rc;
}
/**
@@ -77,9 +2104,14 @@ int spu_save(struct spu_state *prev, str
*/
int spu_restore(struct spu_state *new, struct spu *spu)
{
- /* XXX missing */
+ int rc;
- return 0;
+ acquire_spu_lock(spu);
+ harvest(NULL, spu);
+ rc = __do_spu_restore(new, spu);
+ release_spu_lock(spu);
+
+ return rc;
}
/**
@@ -93,9 +2125,17 @@ int spu_restore(struct spu_state *new, s
*/
int spu_switch(struct spu_state *prev, struct spu_state *new, struct spu *spu)
{
- /* XXX missing */
+ int rc;
- return 0;
+ acquire_spu_lock(spu); /* Save, Step 1. */
+ rc = __do_spu_save(prev, spu); /* Save, Steps 2-53. */
+ if (rc != 0) {
+ harvest(prev, spu);
+ }
+ rc = __do_spu_restore(new, spu);
+ release_spu_lock(spu);
+
+ return rc;
}
static void init_prob(struct spu_state *csa)
--
^ permalink raw reply [flat|nested] 14+ messages in thread* [patch 04/11] spufs: add spu-side context switch code
2005-09-16 12:16 [patch 00/11] spufs: latest spufs snapshot for 2.6.14-rc1 Arnd Bergmann
` (3 preceding siblings ...)
2005-09-16 12:16 ` [patch 03/11] spufs: kernel-side context switch code Arnd Bergmann
@ 2005-09-16 12:16 ` Arnd Bergmann
2005-09-16 12:16 ` [patch 05/11] spufs: Use a system call instead of ioctl Arnd Bergmann
` (6 subsequent siblings)
11 siblings, 0 replies; 14+ messages in thread
From: Arnd Bergmann @ 2005-09-16 12:16 UTC (permalink / raw)
To: ppc64-dev
Cc: linux-kernel, Paul Mackerras, Benjamin Herrenschmidt,
jordi_caubet, Hiroyuki Machida, Geoff Levand
[-- Attachment #1: spufs-generated-files.diff --]
[-- Type: text/plain, Size: 28036 bytes --]
Add the source code that is used to generate spu_save_dump.h and
spu_restore_dump.h. Since a full spu tool chain is needed to
generate these files, the default remains to use the shipped
versions in order to keep the number of tools for building the
kernel down.
From: Mark Nutter: <mnutter@us.ibm.com>
Signed-off-by: Arnd Bergmann <arndb@de.ibm.com>
--
Makefile | 49 ++++
spu_restore.c | 336 +++++++++++++++++++++++++++++++++
spu_restore_crt0.S | 116 +++++++++++
spu_save.c | 195 +++++++++++++++++++
spu_save_crt0.S | 102 ++++++++++
spu_utils.h | 160 +++++++++++++++
6 files changed, 958 insertions(+)
Index: linux-cg/fs/spufs/Makefile
===================================================================
--- linux-cg.orig/fs/spufs/Makefile
+++ linux-cg/fs/spufs/Makefile
@@ -2,4 +2,53 @@ obj-$(CONFIG_SPU_FS) += spufs.o
spufs-y += inode.o file.o context.o switch.o
+# Rules to build switch.o with the help of SPU tool chain
+SPU_CROSS := spu-
+SPU_CC := $(SPU_CROSS)gcc
+SPU_AS := $(SPU_CROSS)gcc
+SPU_LD := $(SPU_CROSS)ld
+SPU_OBJCOPY := $(SPU_CROSS)objcopy
+SPU_CFLAGS := -O2 -Wall -I$(srctree)/include -I$(objtree)/include2
+SPU_AFLAGS := -c -D__ASSEMBLY__ -I$(srctree)/include -I$(objtree)/include2
+SPU_LDFLAGS := -N -Ttext=0x0
+
$(obj)/switch.o: $(obj)/spu_save_dump.h $(obj)/spu_restore_dump.h
+
+# Compile SPU files
+ cmd_spu_cc = $(SPU_CC) $(SPU_CFLAGS) -c -o $@ $<
+quiet_cmd_spu_cc = SPU_CC $@
+$(obj)/spu_%.o: $(src)/spu_%.c
+ $(call if_changed,spu_cc)
+
+# Assemble SPU files
+ cmd_spu_as = $(SPU_AS) $(SPU_AFLAGS) -o $@ $<
+quiet_cmd_spu_as = SPU_AS $@
+$(obj)/spu_%.o: $(src)/spu_%.S
+ $(call if_changed,spu_as)
+
+# Link SPU Executables
+ cmd_spu_ld = $(SPU_LD) $(SPU_LDFLAGS) -o $@ $^
+quiet_cmd_spu_ld = SPU_LD $@
+$(obj)/spu_%: $(obj)/spu_%_crt0.o $(obj)/spu_%.o
+ $(call if_changed,spu_ld)
+
+# Copy into binary format
+ cmd_spu_objcopy = $(SPU_OBJCOPY) -O binary $< $@
+quiet_cmd_spu_objcopy = OBJCOPY $@
+$(obj)/spu_%.bin: $(src)/spu_%
+ $(call if_changed,spu_objcopy)
+
+# create C code from ELF executable
+cmd_hexdump = ( \
+ echo "/*" ; \
+ echo " * $*_dump.h: Copyright (C) 2005 IBM." ; \
+ echo " * Hex-dump auto generated from $*.c." ; \
+ echo " * Do not edit!" ; \
+ echo " */" ; \
+ echo "static unsigned int $*_code[] __page_aligned = {" ; \
+ hexdump -v -e '4/4 "0x%08x, " "\n"' $< ; \
+ echo "};" ; \
+ ) > $@
+quiet_cmd_hexdump = HEXDUMP $@
+$(obj)/%_dump.h: $(obj)/%.bin
+ $(call if_changed,hexdump)
Index: linux-cg/fs/spufs/spu_restore.c
===================================================================
--- /dev/null
+++ linux-cg/fs/spufs/spu_restore.c
@@ -0,0 +1,336 @@
+/*
+ * spu_restore.c
+ *
+ * (C) Copyright IBM Corp. 2005
+ *
+ * SPU-side context restore sequence outlined in
+ * Synergistic Processor Element Book IV
+ *
+ * Author: Mark Nutter <mnutter@us.ibm.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2, or (at your option)
+ * any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+ *
+ */
+
+
+#ifndef LS_SIZE
+#define LS_SIZE 0x40000 /* 256K (in bytes) */
+#endif
+
+typedef unsigned int u32;
+typedef unsigned long long u64;
+
+#include <spu_intrinsics.h>
+#include <asm/spu_csa.h>
+#include "spu_utils.h"
+
+#define BR_INSTR 0x327fff80 /* br -4 */
+#define NOP_INSTR 0x40200000 /* nop */
+#define HEQ_INSTR 0x7b000000 /* heq $0, $0 */
+#define STOP_INSTR 0x00000000 /* stop 0x0 */
+#define ILLEGAL_INSTR 0x00800000 /* illegal instr */
+#define RESTORE_COMPLETE 0x00003ffc /* stop 0x3ffc */
+
+static inline void fetch_regs_from_mem(addr64 lscsa_ea)
+{
+ unsigned int ls = (unsigned int)®s_spill[0];
+ unsigned int size = sizeof(regs_spill);
+ unsigned int tag_id = 0;
+ unsigned int cmd = 0x40; /* GET */
+
+ spu_writech(MFC_LSA, ls);
+ spu_writech(MFC_EAH, lscsa_ea.ui[0]);
+ spu_writech(MFC_EAL, lscsa_ea.ui[1]);
+ spu_writech(MFC_Size, size);
+ spu_writech(MFC_TagID, tag_id);
+ spu_writech(MFC_Cmd, cmd);
+}
+
+static inline void restore_upper_240kb(addr64 lscsa_ea)
+{
+ unsigned int ls = 16384;
+ unsigned int list = (unsigned int)&dma_list[0];
+ unsigned int size = sizeof(dma_list);
+ unsigned int tag_id = 0;
+ unsigned int cmd = 0x44; /* GETL */
+
+ /* Restore, Step 4:
+ * Enqueue the GETL command (tag 0) to the MFC SPU command
+ * queue to transfer the upper 240 kb of LS from CSA.
+ */
+ spu_writech(MFC_LSA, ls);
+ spu_writech(MFC_EAH, lscsa_ea.ui[0]);
+ spu_writech(MFC_EAL, list);
+ spu_writech(MFC_Size, size);
+ spu_writech(MFC_TagID, tag_id);
+ spu_writech(MFC_Cmd, cmd);
+}
+
+static inline void restore_decr(void)
+{
+ unsigned int offset;
+ unsigned int decr_running;
+ unsigned int decr;
+
+ /* Restore, Step 6:
+ * If the LSCSA "decrementer running" flag is set
+ * then write the SPU_WrDec channel with the
+ * decrementer value from LSCSA.
+ */
+ offset = LSCSA_QW_OFFSET(decr_status);
+ decr_running = regs_spill[offset].slot[0];
+ if (decr_running) {
+ offset = LSCSA_QW_OFFSET(decr);
+ decr = regs_spill[offset].slot[0];
+ spu_writech(SPU_WrDec, decr);
+ }
+}
+
+static inline void write_ppu_mb(void)
+{
+ unsigned int offset;
+ unsigned int data;
+
+ /* Restore, Step 11:
+ * Write the MFC_WrOut_MB channel with the PPU_MB
+ * data from LSCSA.
+ */
+ offset = LSCSA_QW_OFFSET(ppu_mb);
+ data = regs_spill[offset].slot[0];
+ spu_writech(SPU_WrOutMbox, data);
+}
+
+static inline void write_ppuint_mb(void)
+{
+ unsigned int offset;
+ unsigned int data;
+
+ /* Restore, Step 12:
+ * Write the MFC_WrInt_MB channel with the PPUINT_MB
+ * data from LSCSA.
+ */
+ offset = LSCSA_QW_OFFSET(ppuint_mb);
+ data = regs_spill[offset].slot[0];
+ spu_writech(SPU_WrOutIntrMbox, data);
+}
+
+static inline void restore_fpcr(void)
+{
+ unsigned int offset;
+ vector unsigned int fpcr;
+
+ /* Restore, Step 13:
+ * Restore the floating-point status and control
+ * register from the LSCSA.
+ */
+ offset = LSCSA_QW_OFFSET(fpcr);
+ fpcr = regs_spill[offset].v;
+ spu_mtfpscr(fpcr);
+}
+
+static inline void restore_srr0(void)
+{
+ unsigned int offset;
+ unsigned int srr0;
+
+ /* Restore, Step 14:
+ * Restore the SPU SRR0 data from the LSCSA.
+ */
+ offset = LSCSA_QW_OFFSET(srr0);
+ srr0 = regs_spill[offset].slot[0];
+ spu_writech(SPU_WrSRR0, srr0);
+}
+
+static inline void restore_event_mask(void)
+{
+ unsigned int offset;
+ unsigned int event_mask;
+
+ /* Restore, Step 15:
+ * Restore the SPU_RdEventMsk data from the LSCSA.
+ */
+ offset = LSCSA_QW_OFFSET(event_mask);
+ event_mask = regs_spill[offset].slot[0];
+ spu_writech(SPU_WrEventMask, event_mask);
+}
+
+static inline void restore_tag_mask(void)
+{
+ unsigned int offset;
+ unsigned int tag_mask;
+
+ /* Restore, Step 16:
+ * Restore the SPU_RdTagMsk data from the LSCSA.
+ */
+ offset = LSCSA_QW_OFFSET(tag_mask);
+ tag_mask = regs_spill[offset].slot[0];
+ spu_writech(MFC_WrTagMask, tag_mask);
+}
+
+static inline void restore_complete(void)
+{
+ extern void exit_fini(void);
+ unsigned int *exit_instrs = (unsigned int *)exit_fini;
+ unsigned int offset;
+ unsigned int stopped_status;
+ unsigned int stopped_code;
+
+ /* Restore, Step 18:
+ * Issue a stop-and-signal instruction with
+ * "good context restore" signal value.
+ *
+ * Restore, Step 19:
+ * There may be additional instructions placed
+ * here by the PPE Sequence for SPU Context
+ * Restore in order to restore the correct
+ * "stopped state".
+ *
+ * This step is handled here by analyzing the
+ * LSCSA.stopped_status and then modifying the
+ * exit() function to behave appropriately.
+ */
+
+ offset = LSCSA_QW_OFFSET(stopped_status);
+ stopped_status = regs_spill[offset].slot[0];
+ stopped_code = regs_spill[offset].slot[1];
+
+ switch (stopped_status) {
+ case SPU_STOPPED_STATUS_P_I:
+ /* SPU_Status[P,I]=1. Add illegal instruction
+ * followed by stop-and-signal instruction after
+ * end of restore code.
+ */
+ exit_instrs[0] = RESTORE_COMPLETE;
+ exit_instrs[1] = ILLEGAL_INSTR;
+ exit_instrs[2] = STOP_INSTR | stopped_code;
+ break;
+ case SPU_STOPPED_STATUS_P_H:
+ /* SPU_Status[P,H]=1. Add 'heq $0, $0' followed
+ * by stop-and-signal instruction after end of
+ * restore code.
+ */
+ exit_instrs[0] = RESTORE_COMPLETE;
+ exit_instrs[1] = HEQ_INSTR;
+ exit_instrs[2] = STOP_INSTR | stopped_code;
+ break;
+ case SPU_STOPPED_STATUS_S_P:
+ /* SPU_Status[S,P]=1. Add nop instruction
+ * followed by 'br -4' after end of restore
+ * code.
+ */
+ exit_instrs[0] = RESTORE_COMPLETE;
+ exit_instrs[1] = STOP_INSTR | stopped_code;
+ exit_instrs[2] = NOP_INSTR;
+ exit_instrs[3] = BR_INSTR;
+ break;
+ case SPU_STOPPED_STATUS_S_I:
+ /* SPU_Status[S,I]=1. Add illegal instruction
+ * followed by 'br -4' after end of restore code.
+ */
+ exit_instrs[0] = RESTORE_COMPLETE;
+ exit_instrs[1] = ILLEGAL_INSTR;
+ exit_instrs[2] = NOP_INSTR;
+ exit_instrs[3] = BR_INSTR;
+ break;
+ case SPU_STOPPED_STATUS_I:
+ /* SPU_Status[I]=1. Add illegal instruction followed
+ * by infinite loop after end of restore sequence.
+ */
+ exit_instrs[0] = RESTORE_COMPLETE;
+ exit_instrs[1] = ILLEGAL_INSTR;
+ exit_instrs[2] = NOP_INSTR;
+ exit_instrs[3] = BR_INSTR;
+ break;
+ case SPU_STOPPED_STATUS_S:
+ /* SPU_Status[S]=1. Add two 'nop' instructions. */
+ exit_instrs[0] = RESTORE_COMPLETE;
+ exit_instrs[1] = NOP_INSTR;
+ exit_instrs[2] = NOP_INSTR;
+ exit_instrs[3] = BR_INSTR;
+ break;
+ case SPU_STOPPED_STATUS_H:
+ /* SPU_Status[H]=1. Add 'heq $0, $0' instruction
+ * after end of restore code.
+ */
+ exit_instrs[0] = RESTORE_COMPLETE;
+ exit_instrs[1] = HEQ_INSTR;
+ exit_instrs[2] = NOP_INSTR;
+ exit_instrs[3] = BR_INSTR;
+ break;
+ case SPU_STOPPED_STATUS_P:
+ /* SPU_Status[P]=1. Add stop-and-signal instruction
+ * after end of restore code.
+ */
+ exit_instrs[0] = RESTORE_COMPLETE;
+ exit_instrs[1] = STOP_INSTR | stopped_code;
+ break;
+ case SPU_STOPPED_STATUS_R:
+ /* SPU_Status[I,S,H,P,R]=0. Add infinite loop. */
+ exit_instrs[0] = RESTORE_COMPLETE;
+ exit_instrs[1] = NOP_INSTR;
+ exit_instrs[2] = NOP_INSTR;
+ exit_instrs[3] = BR_INSTR;
+ break;
+ default:
+ /* SPU_Status[R]=1. No additonal instructions. */
+ break;
+ }
+ spu_sync();
+}
+
+/**
+ * main - entry point for SPU-side context restore.
+ *
+ * This code deviates from the documented sequence in the
+ * following aspects:
+ *
+ * 1. The EA for LSCSA is passed from PPE in the
+ * signal notification channels.
+ * 2. The register spill area is pulled by SPU
+ * into LS, rather than pushed by PPE.
+ * 3. All 128 registers are restored by exit().
+ * 4. The exit() function is modified at run
+ * time in order to properly restore the
+ * SPU_Status register.
+ */
+int main()
+{
+ addr64 lscsa_ea;
+
+ lscsa_ea.ui[0] = spu_readch(SPU_RdSigNotify1);
+ lscsa_ea.ui[1] = spu_readch(SPU_RdSigNotify2);
+ fetch_regs_from_mem(lscsa_ea);
+
+ set_event_mask(); /* Step 1. */
+ set_tag_mask(); /* Step 2. */
+ build_dma_list(lscsa_ea); /* Step 3. */
+ restore_upper_240kb(lscsa_ea); /* Step 4. */
+ /* Step 5: done by 'exit'. */
+ restore_decr(); /* Step 6. */
+ enqueue_putllc(lscsa_ea); /* Step 7. */
+ set_tag_update(); /* Step 8. */
+ read_tag_status(); /* Step 9. */
+ read_llar_status(); /* Step 10. */
+ write_ppu_mb(); /* Step 11. */
+ write_ppuint_mb(); /* Step 12. */
+ restore_fpcr(); /* Step 13. */
+ restore_srr0(); /* Step 14. */
+ restore_event_mask(); /* Step 15. */
+ restore_tag_mask(); /* Step 16. */
+ /* Step 17. done by 'exit'. */
+ restore_complete(); /* Step 18. */
+
+ return 0;
+}
Index: linux-cg/fs/spufs/spu_restore_crt0.S
===================================================================
--- /dev/null
+++ linux-cg/fs/spufs/spu_restore_crt0.S
@@ -0,0 +1,116 @@
+/*
+ * crt0_r.S: Entry function for SPU-side context restore.
+ *
+ * Copyright (C) 2005 IBM
+ *
+ * Entry and exit function for SPU-side of the context restore
+ * sequence. Sets up an initial stack frame, then branches to
+ * 'main'. On return, restores all 128 registers from the LSCSA
+ * and exits.
+ *
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2, or (at your option)
+ * any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+ */
+
+#include <asm/spu_csa.h>
+
+.data
+.align 7
+.globl regs_spill
+regs_spill:
+.space SIZEOF_SPU_SPILL_REGS, 0x0
+
+.text
+.global _start
+_start:
+ /* Initialize the stack pointer to point to 16368
+ * (16kb-16). The back chain pointer is initialized
+ * to NULL.
+ */
+ il $0, 0
+ il $SP, 16368
+ stqd $0, 0($SP)
+
+ /* Allocate a minimum stack frame for the called main.
+ * This is needed so that main has a place to save the
+ * link register when it calls another function.
+ */
+ stqd $SP, -160($SP)
+ ai $SP, $SP, -160
+
+ /* Call the program's main function. */
+ brsl $0, main
+
+.global exit
+.global _exit
+exit:
+_exit:
+ /* SPU Context Restore, Step 5: Restore the remaining 112 GPRs. */
+ ila $3, regs_spill + 256
+restore_regs:
+ lqr $4, restore_reg_insts
+restore_reg_loop:
+ ai $4, $4, 4
+ .balignl 16, 0x40200000
+restore_reg_insts: /* must be quad-word aligned. */
+ lqd $16, 0($3)
+ lqd $17, 16($3)
+ lqd $18, 32($3)
+ lqd $19, 48($3)
+ andi $5, $4, 0x7F
+ stqr $4, restore_reg_insts
+ ai $3, $3, 64
+ brnz $5, restore_reg_loop
+
+ /* SPU Context Restore Step 17: Restore the first 16 GPRs. */
+ lqa $0, regs_spill + 0
+ lqa $1, regs_spill + 16
+ lqa $2, regs_spill + 32
+ lqa $3, regs_spill + 48
+ lqa $4, regs_spill + 64
+ lqa $5, regs_spill + 80
+ lqa $6, regs_spill + 96
+ lqa $7, regs_spill + 112
+ lqa $8, regs_spill + 128
+ lqa $9, regs_spill + 144
+ lqa $10, regs_spill + 160
+ lqa $11, regs_spill + 176
+ lqa $12, regs_spill + 192
+ lqa $13, regs_spill + 208
+ lqa $14, regs_spill + 224
+ lqa $15, regs_spill + 240
+
+ /* Under normal circumstances, the 'exit' function
+ * terminates with 'stop SPU_RESTORE_COMPLETE',
+ * indicating that the SPU-side restore code has
+ * completed.
+ *
+ * However it is possible that instructions immediately
+ * following the 'stop 0x3ffc' have been modified at run
+ * time so as to recreate the exact SPU_Status settings
+ * from the application, e.g. illegal instruciton, halt,
+ * etc.
+ */
+.global exit_fini
+.global _exit_fini
+exit_fini:
+_exit_fini:
+ stop SPU_RESTORE_COMPLETE
+ stop 0
+ stop 0
+ stop 0
+
+ /* Pad the size of this crt0.o to be multiple of 16 bytes. */
+.balignl 16, 0x0
Index: linux-cg/fs/spufs/spu_save.c
===================================================================
--- /dev/null
+++ linux-cg/fs/spufs/spu_save.c
@@ -0,0 +1,195 @@
+/*
+ * spu_save.c
+ *
+ * (C) Copyright IBM Corp. 2005
+ *
+ * SPU-side context save sequence outlined in
+ * Synergistic Processor Element Book IV
+ *
+ * Author: Mark Nutter <mnutter@us.ibm.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2, or (at your option)
+ * any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+ *
+ */
+
+
+#ifndef LS_SIZE
+#define LS_SIZE 0x40000 /* 256K (in bytes) */
+#endif
+
+typedef unsigned int u32;
+typedef unsigned long long u64;
+
+#include <spu_intrinsics.h>
+#include <asm/spu_csa.h>
+#include "spu_utils.h"
+
+static inline void save_event_mask(void)
+{
+ unsigned int offset;
+
+ /* Save, Step 2:
+ * Read the SPU_RdEventMsk channel and save to the LSCSA.
+ */
+ offset = LSCSA_QW_OFFSET(event_mask);
+ regs_spill[offset].slot[0] = spu_readch(SPU_RdEventStatMask);
+}
+
+static inline void save_tag_mask(void)
+{
+ unsigned int offset;
+
+ /* Save, Step 3:
+ * Read the SPU_RdTagMsk channel and save to the LSCSA.
+ */
+ offset = LSCSA_QW_OFFSET(tag_mask);
+ regs_spill[offset].slot[0] = spu_readch(MFC_RdTagMask);
+}
+
+static inline void save_upper_240kb(addr64 lscsa_ea)
+{
+ unsigned int ls = 16384;
+ unsigned int list = (unsigned int)&dma_list[0];
+ unsigned int size = sizeof(dma_list);
+ unsigned int tag_id = 0;
+ unsigned int cmd = 0x24; /* PUTL */
+
+ /* Save, Step 7:
+ * Enqueue the PUTL command (tag 0) to the MFC SPU command
+ * queue to transfer the remaining 240 kb of LS to CSA.
+ */
+ spu_writech(MFC_LSA, ls);
+ spu_writech(MFC_EAH, lscsa_ea.ui[0]);
+ spu_writech(MFC_EAL, list);
+ spu_writech(MFC_Size, size);
+ spu_writech(MFC_TagID, tag_id);
+ spu_writech(MFC_Cmd, cmd);
+}
+
+static inline void save_fpcr(void)
+{
+ // vector unsigned int fpcr;
+ unsigned int offset;
+
+ /* Save, Step 9:
+ * Issue the floating-point status and control register
+ * read instruction, and save to the LSCSA.
+ */
+ offset = LSCSA_QW_OFFSET(fpcr);
+ regs_spill[offset].v = spu_mffpscr();
+}
+
+static inline void save_decr(void)
+{
+ unsigned int offset;
+
+ /* Save, Step 10:
+ * Read and save the SPU_RdDec channel data to
+ * the LSCSA.
+ */
+ offset = LSCSA_QW_OFFSET(decr);
+ regs_spill[offset].slot[0] = spu_readch(SPU_RdDec);
+}
+
+static inline void save_srr0(void)
+{
+ unsigned int offset;
+
+ /* Save, Step 11:
+ * Read and save the SPU_WSRR0 channel data to
+ * the LSCSA.
+ */
+ offset = LSCSA_QW_OFFSET(srr0);
+ regs_spill[offset].slot[0] = spu_readch(SPU_RdSRR0);
+}
+
+static inline void spill_regs_to_mem(addr64 lscsa_ea)
+{
+ unsigned int ls = (unsigned int)®s_spill[0];
+ unsigned int size = sizeof(regs_spill);
+ unsigned int tag_id = 0;
+ unsigned int cmd = 0x20; /* PUT */
+
+ /* Save, Step 13:
+ * Enqueue a PUT command (tag 0) to send the LSCSA
+ * to the CSA.
+ */
+ spu_writech(MFC_LSA, ls);
+ spu_writech(MFC_EAH, lscsa_ea.ui[0]);
+ spu_writech(MFC_EAL, lscsa_ea.ui[1]);
+ spu_writech(MFC_Size, size);
+ spu_writech(MFC_TagID, tag_id);
+ spu_writech(MFC_Cmd, cmd);
+}
+
+static inline void enqueue_sync(addr64 lscsa_ea)
+{
+ unsigned int tag_id = 0;
+ unsigned int cmd = 0xCC;
+
+ /* Save, Step 14:
+ * Enqueue an MFC_SYNC command (tag 0).
+ */
+ spu_writech(MFC_TagID, tag_id);
+ spu_writech(MFC_Cmd, cmd);
+}
+
+static inline void save_complete(void)
+{
+ /* Save, Step 18:
+ * Issue a stop-and-signal instruction indicating
+ * "save complete". Note: This function will not
+ * return!!
+ */
+ spu_stop(SPU_SAVE_COMPLETE);
+}
+
+/**
+ * main - entry point for SPU-side context save.
+ *
+ * This code deviates from the documented sequence as follows:
+ *
+ * 1. The EA for LSCSA is passed from PPE in the
+ * signal notification channels.
+ * 2. All 128 registers are saved by crt0.o.
+ */
+int main()
+{
+ addr64 lscsa_ea;
+
+ lscsa_ea.ui[0] = spu_readch(SPU_RdSigNotify1);
+ lscsa_ea.ui[1] = spu_readch(SPU_RdSigNotify2);
+
+ /* Step 1: done by exit(). */
+ save_event_mask(); /* Step 2. */
+ save_tag_mask(); /* Step 3. */
+ set_event_mask(); /* Step 4. */
+ set_tag_mask(); /* Step 5. */
+ build_dma_list(lscsa_ea); /* Step 6. */
+ save_upper_240kb(lscsa_ea); /* Step 7. */
+ /* Step 8: done by exit(). */
+ save_fpcr(); /* Step 9. */
+ save_decr(); /* Step 10. */
+ save_srr0(); /* Step 11. */
+ enqueue_putllc(lscsa_ea); /* Step 12. */
+ spill_regs_to_mem(lscsa_ea); /* Step 13. */
+ enqueue_sync(lscsa_ea); /* Step 14. */
+ set_tag_update(); /* Step 15. */
+ read_tag_status(); /* Step 16. */
+ read_llar_status(); /* Step 17. */
+ save_complete(); /* Step 18. */
+
+ return 0;
+}
Index: linux-cg/fs/spufs/spu_save_crt0.S
===================================================================
--- /dev/null
+++ linux-cg/fs/spufs/spu_save_crt0.S
@@ -0,0 +1,102 @@
+/*
+ * crt0_s.S: Entry function for SPU-side context save.
+ *
+ * Copyright (C) 2005 IBM
+ *
+ * Entry function for SPU-side of the context save sequence.
+ * Saves all 128 GPRs, sets up an initial stack frame, then
+ * branches to 'main'.
+ *
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2, or (at your option)
+ * any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+ */
+
+#include <asm/spu_csa.h>
+
+.data
+.align 7
+.globl regs_spill
+regs_spill:
+.space SIZEOF_SPU_SPILL_REGS, 0x0
+
+.text
+.global _start
+_start:
+ /* SPU Context Save Step 1: Save the first 16 GPRs. */
+ stqa $0, regs_spill + 0
+ stqa $1, regs_spill + 16
+ stqa $2, regs_spill + 32
+ stqa $3, regs_spill + 48
+ stqa $4, regs_spill + 64
+ stqa $5, regs_spill + 80
+ stqa $6, regs_spill + 96
+ stqa $7, regs_spill + 112
+ stqa $8, regs_spill + 128
+ stqa $9, regs_spill + 144
+ stqa $10, regs_spill + 160
+ stqa $11, regs_spill + 176
+ stqa $12, regs_spill + 192
+ stqa $13, regs_spill + 208
+ stqa $14, regs_spill + 224
+ stqa $15, regs_spill + 240
+
+ /* SPU Context Save, Step 8: Save the remaining 112 GPRs. */
+ ila $3, regs_spill + 256
+save_regs:
+ lqr $4, save_reg_insts
+save_reg_loop:
+ ai $4, $4, 4
+ .balignl 16, 0x40200000
+save_reg_insts: /* must be quad-word aligned. */
+ stqd $16, 0($3)
+ stqd $17, 16($3)
+ stqd $18, 32($3)
+ stqd $19, 48($3)
+ andi $5, $4, 0x7F
+ stqr $4, save_reg_insts
+ ai $3, $3, 64
+ brnz $5, save_reg_loop
+
+ /* Initialize the stack pointer to point to 16368
+ * (16kb-16). The back chain pointer is initialized
+ * to NULL.
+ */
+ il $0, 0
+ il $SP, 16368
+ stqd $0, 0($SP)
+
+ /* Allocate a minimum stack frame for the called main.
+ * This is needed so that main has a place to save the
+ * link register when it calls another function.
+ */
+ stqd $SP, -160($SP)
+ ai $SP, $SP, -160
+
+ /* Call the program's main function. */
+ brsl $0, main
+
+ /* In this case main should not return; if it does
+ * there has been an error in the sequence. Execute
+ * stop-and-signal with code=0.
+ */
+.global exit
+.global _exit
+exit:
+_exit:
+ stop 0x0
+
+ /* Pad the size of this crt0.o to be multiple of 16 bytes. */
+.balignl 16, 0x0
+
Index: linux-cg/fs/spufs/spu_utils.h
===================================================================
--- /dev/null
+++ linux-cg/fs/spufs/spu_utils.h
@@ -0,0 +1,160 @@
+/*
+ * utils.h: Utilities for SPU-side of the context switch operation.
+ *
+ * (C) Copyright IBM 2005
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2, or (at your option)
+ * any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+ */
+
+#ifndef _SPU_CONTEXT_UTILS_H_
+#define _SPU_CONTEXT_UTILS_H_
+
+/*
+ * 64-bit safe EA.
+ */
+typedef union {
+ unsigned long long ull;
+ unsigned int ui[2];
+} addr64;
+
+/*
+ * 128-bit register template.
+ */
+typedef union {
+ unsigned int slot[4];
+ vector unsigned int v;
+} spu_reg128v;
+
+/*
+ * DMA list structure.
+ */
+struct dma_list_elem {
+ unsigned int size;
+ unsigned int ea_low;
+};
+
+/*
+ * Declare storage for 8-byte aligned DMA list.
+ */
+struct dma_list_elem dma_list[15] __attribute__ ((aligned(8)));
+
+/*
+ * External definition for storage
+ * declared in crt0.
+ */
+extern spu_reg128v regs_spill[NR_SPU_SPILL_REGS];
+
+/*
+ * Compute LSCSA byte offset for a given field.
+ */
+static struct spu_lscsa *dummy = (struct spu_lscsa *)0;
+#define LSCSA_BYTE_OFFSET(_field) \
+ ((char *)(&(dummy->_field)) - (char *)(&(dummy->gprs[0].slot[0])))
+#define LSCSA_QW_OFFSET(_field) (LSCSA_BYTE_OFFSET(_field) >> 4)
+
+static inline void set_event_mask(void)
+{
+ unsigned int event_mask = 0;
+
+ /* Save, Step 4:
+ * Restore, Step 1:
+ * Set the SPU_RdEventMsk channel to zero to mask
+ * all events.
+ */
+ spu_writech(SPU_WrEventMask, event_mask);
+}
+
+static inline void set_tag_mask(void)
+{
+ unsigned int tag_mask = 1;
+
+ /* Save, Step 5:
+ * Restore, Step 2:
+ * Set the SPU_WrTagMsk channel to '01' to unmask
+ * only tag group 0.
+ */
+ spu_writech(MFC_WrTagMask, tag_mask);
+}
+
+static inline void build_dma_list(addr64 lscsa_ea)
+{
+ unsigned int ea_low;
+ int i;
+
+ /* Save, Step 6:
+ * Restore, Step 3:
+ * Update the effective address for the CSA in the
+ * pre-canned DMA-list in local storage.
+ */
+ ea_low = lscsa_ea.ui[1];
+ ea_low += LSCSA_BYTE_OFFSET(ls[16384]);
+
+ for (i = 0; i < 15; i++, ea_low += 16384) {
+ dma_list[i].size = 16384;
+ dma_list[i].ea_low = ea_low;
+ }
+}
+
+static inline void enqueue_putllc(addr64 lscsa_ea)
+{
+ unsigned int ls = 0;
+ unsigned int size = 128;
+ unsigned int tag_id = 0;
+ unsigned int cmd = 0xB4; /* PUTLLC */
+
+ /* Save, Step 12:
+ * Restore, Step 7:
+ * Send a PUTLLC (tag 0) command to the MFC using
+ * an effective address in the CSA in order to
+ * remove any possible lock-line reservation.
+ */
+ spu_writech(MFC_LSA, ls);
+ spu_writech(MFC_EAH, lscsa_ea.ui[0]);
+ spu_writech(MFC_EAL, lscsa_ea.ui[1]);
+ spu_writech(MFC_Size, size);
+ spu_writech(MFC_TagID, tag_id);
+ spu_writech(MFC_Cmd, cmd);
+}
+
+static inline void set_tag_update(void)
+{
+ unsigned int update_any = 1;
+
+ /* Save, Step 15:
+ * Restore, Step 8:
+ * Write the MFC_TagUpdate channel with '01'.
+ */
+ spu_writech(MFC_WrTagUpdate, update_any);
+}
+
+static inline void read_tag_status(void)
+{
+ /* Save, Step 16:
+ * Restore, Step 9:
+ * Read the MFC_TagStat channel data.
+ */
+ spu_readch(MFC_RdTagStat);
+}
+
+static inline void read_llar_status(void)
+{
+ /* Save, Step 17:
+ * Restore, Step 10:
+ * Read the MFC_AtomicStat channel data.
+ */
+ spu_readch(MFC_RdAtomicStat);
+}
+
+#endif /* _SPU_CONTEXT_UTILS_H_ */
--
^ permalink raw reply [flat|nested] 14+ messages in thread* [patch 05/11] spufs: Use a system call instead of ioctl
2005-09-16 12:16 [patch 00/11] spufs: latest spufs snapshot for 2.6.14-rc1 Arnd Bergmann
` (4 preceding siblings ...)
2005-09-16 12:16 ` [patch 04/11] spufs: add spu-side " Arnd Bergmann
@ 2005-09-16 12:16 ` Arnd Bergmann
2005-09-16 12:16 ` [patch 06/11] spufs: allow O_ASYNC on mailbox files Arnd Bergmann
` (5 subsequent siblings)
11 siblings, 0 replies; 14+ messages in thread
From: Arnd Bergmann @ 2005-09-16 12:16 UTC (permalink / raw)
To: ppc64-dev
Cc: linux-kernel, Paul Mackerras, Benjamin Herrenschmidt,
jordi_caubet, Hiroyuki Machida, Geoff Levand
[-- Attachment #1: spufs-syscall-4.diff --]
[-- Type: text/plain, Size: 17338 bytes --]
We are moving to a system call based approach for some
low-level operations now. Two system calls are introduced,
spu_run and spu_create. The spu_create call can be
used in place of the mkdir syscall to create an spu
context. It returns an open file descriptor to the
new directory. When the fd is closed (e.g. on process
exit), the context is automatically destroyed.
The spu_run call takes over the role of the ioctl
with the same name. It operates on the file descriptor
returned from spu_create, so we don't need the "run"
file any more.
The old interfaces are retained in this patch, so
that libraries continue to work.
Signed-off-by: Arnd Bergmann <arndb@de.ibm.com.de>
--
arch/ppc64/kernel/Makefile | 2
arch/ppc64/kernel/misc.S | 4
arch/ppc64/kernel/spu_syscalls.c | 86 ++++++++++++
fs/spufs/Makefile | 3
fs/spufs/context.c | 4
fs/spufs/file.c | 2
fs/spufs/inode.c | 135 ++++++++++++++-----
fs/spufs/spufs.h | 8 -
fs/spufs/syscalls.c | 106 ++++++++++++++
include/asm-ppc/unistd.h | 4
include/asm-ppc64/spu.h | 23 +++
include/asm-ppc64/unistd.h | 5
include/linux/syscalls.h | 3
kernel/sys_ni.c | 2
14 files changed, 349 insertions(+), 38 deletions(-)
Index: linux-cg/arch/ppc64/kernel/misc.S
===================================================================
--- linux-cg.orig/arch/ppc64/kernel/misc.S
+++ linux-cg/arch/ppc64/kernel/misc.S
@@ -1230,6 +1230,8 @@ _GLOBAL(sys_call_table32)
.llong .sys_inotify_init /* 275 */
.llong .sys_inotify_add_watch
.llong .sys_inotify_rm_watch
+ .llong .sys_spu_run
+ .llong .sys_spu_create_thread
.balign 8
_GLOBAL(sys_call_table)
@@ -1511,3 +1513,5 @@ _GLOBAL(sys_call_table)
.llong .sys_inotify_init /* 275 */
.llong .sys_inotify_add_watch
.llong .sys_inotify_rm_watch
+ .llong .sys_spu_run
+ .llong .sys_spu_create_thread
Index: linux-cg/fs/spufs/Makefile
===================================================================
--- linux-cg.orig/fs/spufs/Makefile
+++ linux-cg/fs/spufs/Makefile
@@ -1,6 +1,5 @@
obj-$(CONFIG_SPU_FS) += spufs.o
-
-spufs-y += inode.o file.o context.o switch.o
+spufs-y += inode.o file.o context.o switch.o syscalls.o
# Rules to build switch.o with the help of SPU tool chain
SPU_CROSS := spu-
Index: linux-cg/fs/spufs/file.c
===================================================================
--- linux-cg.orig/fs/spufs/file.c
+++ linux-cg/fs/spufs/file.c
@@ -378,7 +378,7 @@ static struct file_operations spufs_wbox
.read = spufs_wbox_stat_read,
};
-static long spufs_run_spu(struct file *file, struct spu_context *ctx,
+long spufs_run_spu(struct file *file, struct spu_context *ctx,
u32 *npc, u32 *status)
{
struct spu_problem __iomem *prob;
Index: linux-cg/fs/spufs/spufs.h
===================================================================
--- linux-cg.orig/fs/spufs/spufs.h
+++ linux-cg/fs/spufs/spufs.h
@@ -53,11 +53,17 @@ struct spufs_inode_info {
extern struct tree_descr spufs_dir_contents[];
+/* system call implementation */
+long spufs_run_spu(struct file *file,
+ struct spu_context *ctx, u32 *npc, u32 *status);
+long spufs_create_thread(struct nameidata *nd, const char *name,
+ unsigned int flags, mode_t mode);
+
/* context management */
struct spu_context * alloc_spu_context(void);
void destroy_spu_context(struct kref *kref);
struct spu_context * get_spu_context(struct spu_context *ctx);
-void put_spu_context(struct spu_context *ctx);
+int put_spu_context(struct spu_context *ctx);
void spu_acquire(struct spu_context *ctx);
void spu_release(struct spu_context *ctx);
Index: linux-cg/include/asm-ppc/unistd.h
===================================================================
--- linux-cg.orig/include/asm-ppc/unistd.h
+++ linux-cg/include/asm-ppc/unistd.h
@@ -282,8 +282,10 @@
#define __NR_inotify_init 275
#define __NR_inotify_add_watch 276
#define __NR_inotify_rm_watch 277
+#define __NR_spu_run 278
+#define __NR_spu_create_thread 279
-#define __NR_syscalls 278
+#define __NR_syscalls 280
#define __NR(n) #n
Index: linux-cg/include/asm-ppc64/unistd.h
===================================================================
--- linux-cg.orig/include/asm-ppc64/unistd.h
+++ linux-cg/include/asm-ppc64/unistd.h
@@ -288,8 +288,11 @@
#define __NR_inotify_init 275
#define __NR_inotify_add_watch 276
#define __NR_inotify_rm_watch 277
+#define __NR_spu_run 278
+#define __NR_spu_create_thread 279
+#define __NR_syscalls 280
+
-#define __NR_syscalls 278
#ifdef __KERNEL__
#define NR_syscalls __NR_syscalls
#endif
Index: linux-cg/include/linux/syscalls.h
===================================================================
--- linux-cg.orig/include/linux/syscalls.h
+++ linux-cg/include/linux/syscalls.h
@@ -509,4 +509,7 @@ asmlinkage long sys_keyctl(int cmd, unsi
asmlinkage long sys_ioprio_set(int which, int who, int ioprio);
asmlinkage long sys_ioprio_get(int which, int who);
+asmlinkage long sys_spu_run(int fd, __u32 __user *unpc,
+ __u32 __user *ustatus);
+
#endif
Index: linux-cg/kernel/sys_ni.c
===================================================================
--- linux-cg.orig/kernel/sys_ni.c
+++ linux-cg/kernel/sys_ni.c
@@ -90,3 +90,5 @@ cond_syscall(sys_pciconfig_iobase);
cond_syscall(sys32_ipc);
cond_syscall(sys32_sysctl);
cond_syscall(ppc_rtas);
+cond_syscall(sys_spu_run);
+cond_syscall(sys_spu_create_thread);
Index: linux-cg/arch/ppc64/kernel/Makefile
===================================================================
--- linux-cg.orig/arch/ppc64/kernel/Makefile
+++ linux-cg/arch/ppc64/kernel/Makefile
@@ -57,6 +57,8 @@ obj-$(CONFIG_IBMVIO) += vio.o $(vio-obj
obj-$(CONFIG_XICS) += xics.o
obj-$(CONFIG_MPIC) += mpic.o
obj-$(CONFIG_SPU_FS) += spu_base.o
+spu_syscalls-$(CONFIG_SPU_FS) += spu_syscalls.o
+obj-y += $(spu_syscalls-m)
obj-$(CONFIG_PPC_PMAC) += pmac_setup.o pmac_feature.o pmac_pci.o \
pmac_time.o pmac_nvram.o pmac_low_i2c.o \
Index: linux-cg/arch/ppc64/kernel/spu_syscalls.c
===================================================================
--- /dev/null
+++ linux-cg/arch/ppc64/kernel/spu_syscalls.c
@@ -0,0 +1,86 @@
+/*
+ * SPU file system -- system call stubs
+ *
+ * (C) Copyright IBM Deutschland Entwicklung GmbH 2005
+ *
+ * Author: Arnd Bergmann <arndb@de.ibm.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2, or (at your option)
+ * any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+ */
+#include <linux/file.h>
+#include <linux/module.h>
+#include <linux/syscalls.h>
+
+#include <asm/spu.h>
+
+struct spufs_calls spufs_calls = {
+ .owner = NULL,
+};
+
+/* These stub syscalls are needed to have the actual implementation
+ * within a loadable module. When spufs is built into the kernel,
+ * this file is not used and the syscalls directly enter the fs code */
+
+asmlinkage long sys_spu_create_thread(const char __user *name,
+ unsigned int flags, mode_t mode)
+{
+ long ret;
+
+ ret = -ENOSYS;
+ if (try_module_get(spufs_calls.owner)) {
+ ret = spufs_calls.create_thread(name, flags, mode);
+ module_put(spufs_calls.owner);
+ }
+ return ret;
+}
+
+asmlinkage long sys_spu_run(int fd, __u32 __user *unpc, __u32 __user *ustatus)
+{
+ long ret;
+ struct file *filp;
+ int fput_needed;
+
+ ret = -ENOSYS;
+ if (try_module_get(spufs_calls.owner)) {
+ ret = -EBADF;
+ filp = fget_light(fd, &fput_needed);
+ if (filp) {
+ ret = spufs_calls.spu_run(filp, unpc, ustatus);
+ fput_light(filp, fput_needed);
+ }
+ module_put(spufs_calls.owner);
+ }
+ return ret;
+}
+
+int register_spu_syscalls(struct spufs_calls *calls)
+{
+ if (spufs_calls.owner)
+ return -EBUSY;
+
+ spufs_calls.create_thread = calls->create_thread;
+ spufs_calls.spu_run = calls->spu_run;
+ smp_mb();
+ spufs_calls.owner = calls->owner;
+ return 0;
+}
+EXPORT_SYMBOL_GPL(register_spu_syscalls);
+
+void unregister_spu_syscalls(struct spufs_calls *calls)
+{
+ BUG_ON(spufs_calls.owner != calls->owner);
+ spufs_calls.owner = NULL;
+}
+EXPORT_SYMBOL_GPL(unregister_spu_syscalls);
Index: linux-cg/fs/spufs/context.c
===================================================================
--- linux-cg.orig/fs/spufs/context.c
+++ linux-cg/fs/spufs/context.c
@@ -77,9 +77,9 @@ struct spu_context * get_spu_context(str
return ctx;
}
-void put_spu_context(struct spu_context *ctx)
+int put_spu_context(struct spu_context *ctx)
{
- kref_put(&ctx->kref, &destroy_spu_context);
+ return kref_put(&ctx->kref, &destroy_spu_context);
}
Index: linux-cg/fs/spufs/inode.c
===================================================================
--- linux-cg.orig/fs/spufs/inode.c
+++ linux-cg/fs/spufs/inode.c
@@ -20,11 +20,13 @@
* Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
*/
+#include <linux/file.h>
#include <linux/fs.h>
#include <linux/backing-dev.h>
#include <linux/init.h>
#include <linux/ioctl.h>
#include <linux/module.h>
+#include <linux/namei.h>
#include <linux/pagemap.h>
#include <linux/poll.h>
#include <linux/slab.h>
@@ -210,12 +212,60 @@ out:
return ret;
}
+static int spufs_rmdir(struct inode *root, struct dentry *dir_dentry)
+{
+ struct dentry *dentry;
+ int err;
+
+ spin_lock(&dcache_lock);
+ /* remove all entries */
+ err = 0;
+ list_for_each_entry(dentry, &dir_dentry->d_subdirs, d_child) {
+ if (d_unhashed(dentry) || !dentry->d_inode)
+ continue;
+ atomic_dec(&dentry->d_count);
+ spin_lock(&dentry->d_lock);
+ __d_drop(dentry);
+ spin_unlock(&dentry->d_lock);
+ }
+ spin_unlock(&dcache_lock);
+ if (!err) {
+ shrink_dcache_parent(dir_dentry);
+ err = simple_rmdir(root, dir_dentry);
+ }
+ return err;
+}
+
+static int spufs_dir_close(struct inode *inode, struct file *file)
+{
+ struct inode *dir;
+ struct dentry *dentry;
+ int ret;
+
+ dentry = file->f_dentry;
+ dir = dentry->d_parent->d_inode;
+ down(&dir->i_sem);
+ ret = spufs_rmdir(dir, file->f_dentry);
+ WARN_ON(ret);
+ up(&dir->i_sem);
+ return dcache_dir_close(inode, file);
+}
+
struct inode_operations spufs_dir_inode_operations = {
.lookup = simple_lookup,
.unlink = simple_unlink,
.create = spufs_create,
};
+struct file_operations spufs_autodelete_dir_operations = {
+ .open = dcache_dir_open,
+ .release = spufs_dir_close,
+ .llseek = dcache_dir_lseek,
+ .read = generic_read_dir,
+ .readdir = dcache_readdir,
+ .fsync = simple_sync_file,
+};
+
static int
spufs_mkdir(struct inode *dir, struct dentry *dentry, int mode)
{
@@ -256,40 +306,56 @@ out:
return ret;
}
-/* This looks really wrong! */
-static int spufs_rmdir(struct inode *root, struct dentry *dir_dentry)
+long
+spufs_create_thread(struct nameidata *nd, const char *name,
+ unsigned int flags, mode_t mode)
{
struct dentry *dentry;
- int err;
+ struct file *filp;
+ int ret;
- spin_lock(&dcache_lock);
-#if 0
- /* check if any entry is used */
- err = -EBUSY;
- list_for_each_entry(dentry, &dir_dentry->d_subdirs, d_child) {
- if (d_unhashed(dentry) || !dentry->d_inode)
- continue;
- if (atomic_read(&dentry->d_count) != 1)
- goto out;
- }
-#endif
- /* remove all entries */
- err = 0;
- list_for_each_entry(dentry, &dir_dentry->d_subdirs, d_child) {
- if (d_unhashed(dentry) || !dentry->d_inode)
- continue;
- atomic_dec(&dentry->d_count);
- __d_drop(dentry);
+ /* need to be at the root of spufs */
+ ret = -EINVAL;
+ if (nd->dentry->d_sb->s_magic != SPUFS_MAGIC ||
+ nd->dentry != nd->dentry->d_sb->s_root)
+ goto out;
+
+ dentry = lookup_create(nd, 1);
+ ret = PTR_ERR(dentry);
+ if (IS_ERR(dentry))
+ goto out_dir;
+
+ ret = -EEXIST;
+ if (dentry->d_inode)
+ goto out_dput;
+
+ mode &= ~current->fs->umask;
+ ret = spufs_mkdir(nd->dentry->d_inode, dentry, mode & S_IRWXUGO);
+ if (ret)
+ goto out_dput;
+
+ ret = get_unused_fd();
+ if (ret < 0)
+ goto out_dput;
+
+ dentry->d_inode->i_nlink++;
+
+ filp = filp_open(name, O_RDONLY, mode);
+ if (IS_ERR(filp)) {
+ // FIXME: remove directory again
+ put_unused_fd(ret);
+ ret = PTR_ERR(filp);
+ } else {
+ filp->f_op = &spufs_autodelete_dir_operations;
+ fd_install(ret, filp);
}
-#if 0
+
+out_dput:
+ dput(dentry);
+out_dir:
+ up(&nd->dentry->d_inode->i_sem);
out:
-#endif
- spin_unlock(&dcache_lock);
- if (!err) {
- shrink_dcache_parent(dir_dentry);
- err = simple_rmdir(root, dir_dentry);
- }
- return err;
+ return ret;
}
/* File system initialization */
@@ -417,7 +483,15 @@ static int spufs_init(void)
goto out;
ret = register_filesystem(&spufs_type);
if (ret)
- kmem_cache_destroy(spufs_inode_cache);
+ goto out_cache;
+ ret = register_spu_syscalls(&spufs_calls);
+ if (ret)
+ goto out_fs;
+ return 0;
+out_fs:
+ unregister_filesystem(&spufs_type);
+out_cache:
+ kmem_cache_destroy(spufs_inode_cache);
out:
return ret;
}
@@ -425,6 +499,7 @@ module_init(spufs_init);
static void spufs_exit(void)
{
+ unregister_spu_syscalls(&spufs_calls);
unregister_filesystem(&spufs_type);
kmem_cache_destroy(spufs_inode_cache);
}
Index: linux-cg/fs/spufs/syscalls.c
===================================================================
--- /dev/null
+++ linux-cg/fs/spufs/syscalls.c
@@ -0,0 +1,106 @@
+#include <linux/file.h>
+#include <linux/fs.h>
+#include <linux/module.h>
+#include <linux/mount.h>
+#include <linux/namei.h>
+
+#include <asm/uaccess.h>
+
+#include "spufs.h"
+
+/**
+ * sys_spu_run - run code loaded into an SPU
+ *
+ * @unpc: next program counter for the SPU
+ * @ustatus: status of the SPU
+ *
+ * This system call transfers the control of execution of a
+ * user space thread to an SPU. It will return when the
+ * SPU has finished executing or when it hits an error
+ * condition and it will be interrupted if a signal needs
+ * to be delivered to a handler in user space.
+ *
+ * The next program counter is set to the passed value
+ * before the SPU starts fetching code and the user space
+ * pointer gets updated with the new value when returning
+ * from kernel space.
+ *
+ * The status value returned from spu_run reflects the
+ * value of the spu_status register after the SPU has stopped.
+ *
+ */
+long do_spu_run(struct file *filp, __u32 __user *unpc, __u32 __user *ustatus)
+{
+ long ret;
+ struct spufs_inode_info *i;
+ u32 npc, status;
+
+ ret = -EFAULT;
+ if (get_user(npc, unpc))
+ goto out;
+
+ ret = -EINVAL;
+ if (filp->f_vfsmnt->mnt_sb->s_magic != SPUFS_MAGIC)
+ goto out;
+
+ i = SPUFS_I(filp->f_dentry->d_inode);
+ ret = spufs_run_spu(filp, i->i_ctx, &npc, &status);
+
+ if (ret ==-EAGAIN || ret == -EIO)
+ ret = status;
+
+ if (put_user(npc, unpc))
+ ret = -EFAULT;
+
+ if (ustatus && put_user(status, ustatus))
+ ret = -EFAULT;
+out:
+ return ret;
+}
+
+#ifndef MODULE
+asmlinkage long sys_spu_run(int fd, __u32 __user *unpc, __u32 __user *ustatus)
+{
+ int fput_needed;
+ struct file *filp;
+ long ret;
+
+ ret = -EBADF;
+ filp = fget_light(fd, &fput_needed);
+ if (filp) {
+ ret = do_spu_run(filp, unpc, ustatus);
+ fput_light(filp, fput_needed);
+ }
+
+ return ret;
+}
+#endif
+
+asmlinkage long sys_spu_create_thread(const char __user *pathname,
+ unsigned int flags, mode_t mode)
+{
+ char *tmp;
+ int ret;
+
+ tmp = getname(pathname);
+ ret = PTR_ERR(tmp);
+ if (!IS_ERR(tmp)) {
+ struct nameidata nd;
+
+ ret = path_lookup(tmp, LOOKUP_PARENT|
+ LOOKUP_OPEN|LOOKUP_CREATE, &nd);
+ if (!ret) {
+ ret = spufs_create_thread(&nd, pathname, flags, mode);
+ path_release(&nd);
+ }
+ putname(tmp);
+ }
+
+ return ret;
+}
+
+struct spufs_calls spufs_calls = {
+ .create_thread = sys_spu_create_thread,
+ .spu_run = do_spu_run,
+ .owner = THIS_MODULE,
+};
Index: linux-cg/include/asm-ppc64/spu.h
===================================================================
--- linux-cg.orig/include/asm-ppc64/spu.h
+++ linux-cg/include/asm-ppc64/spu.h
@@ -22,6 +22,7 @@
#ifndef _SPU_H
#define _SPU_H
+#include <linux/config.h>
#include <linux/kref.h>
#include <linux/workqueue.h>
@@ -140,6 +141,28 @@ int spu_run(struct spu *spu);
size_t spu_wbox_write(struct spu *spu, u32 data);
size_t spu_ibox_read(struct spu *spu, u32 *data);
+extern struct spufs_calls {
+ asmlinkage long (*create_thread)(const char __user *name,
+ unsigned int flags, mode_t mode);
+ asmlinkage long (*spu_run)(struct file *filp, __u32 __user *unpc,
+ __u32 __user *ustatus);
+ struct module *owner;
+} spufs_calls;
+
+#ifdef CONFIG_SPU_FS_MODULE
+int register_spu_syscalls(struct spufs_calls *calls);
+void unregister_spu_syscalls(struct spufs_calls *calls);
+#else
+static inline int register_spu_syscalls(struct spufs_calls *calls)
+{
+ return 0;
+}
+static inline void unregister_spu_syscalls(struct spufs_calls *calls)
+{
+}
+#endif /* MODULE */
+
+
/*
* This defines the Local Store, Problem Area and Privlege Area of an SPU.
*/
--
^ permalink raw reply [flat|nested] 14+ messages in thread* [patch 06/11] spufs: allow O_ASYNC on mailbox files
2005-09-16 12:16 [patch 00/11] spufs: latest spufs snapshot for 2.6.14-rc1 Arnd Bergmann
` (5 preceding siblings ...)
2005-09-16 12:16 ` [patch 05/11] spufs: Use a system call instead of ioctl Arnd Bergmann
@ 2005-09-16 12:16 ` Arnd Bergmann
2005-09-16 12:16 ` [patch 07/11] spufs: Add a register file for the debugger Arnd Bergmann
` (4 subsequent siblings)
11 siblings, 0 replies; 14+ messages in thread
From: Arnd Bergmann @ 2005-09-16 12:16 UTC (permalink / raw)
To: ppc64-dev
Cc: linux-kernel, Paul Mackerras, Benjamin Herrenschmidt,
jordi_caubet, Hiroyuki Machida, Geoff Levand
[-- Attachment #1: spufs-fasync-2.diff --]
[-- Type: text/plain, Size: 3701 bytes --]
This patch makes it possible to receive user-defined
signals when the spufs ibox and wbox files are accessed
from an SPE, so data can be read/written from/to
them again.
Unlike what I wrote previously, we now intend to keep
this as an official interface, in order to implement
one of the OS-independent library interfaces (namely
spe_get_event).
Signed-off-by: Arnd Bergmann <arndb@de.ibm.com>
--
arch/ppc64/kernel/spu_base.c | 7 +++++++
fs/spufs/file.c | 16 ++++++++++++++++
include/asm-ppc64/spu.h | 2 ++
3 files changed, 25 insertions(+)
Index: linux-cg/arch/ppc64/kernel/spu_base.c
===================================================================
--- linux-cg.orig/arch/ppc64/kernel/spu_base.c
+++ linux-cg/arch/ppc64/kernel/spu_base.c
@@ -136,6 +136,7 @@ static int __spu_trap_data_map(struct sp
static int __spu_trap_mailbox(struct spu *spu)
{
wake_up_all(&spu->ibox_wq);
+ kill_fasync(&spu->ibox_fasync, SIGIO, POLLIN);
/* atomically disable SPU mailbox interrupts */
spin_lock(&spu->register_lock);
@@ -171,6 +172,7 @@ static int __spu_trap_tag_group(struct s
static int __spu_trap_spubox(struct spu *spu)
{
wake_up_all(&spu->wbox_wq);
+ kill_fasync(&spu->wbox_fasync, SIGIO, POLLOUT);
/* atomically disable SPU mailbox interrupts */
spin_lock(&spu->register_lock);
@@ -394,6 +396,8 @@ EXPORT_SYMBOL(spu_alloc);
void spu_free(struct spu *spu)
{
down(&spu_mutex);
+ spu->ibox_fasync = NULL;
+ spu->wbox_fasync = NULL;
list_add_tail(&spu->list, &spu_list);
up(&spu_mutex);
}
@@ -676,6 +680,9 @@ static int __init create_spu(struct devi
init_waitqueue_head(&spu->wbox_wq);
init_waitqueue_head(&spu->ibox_wq);
+ spu->ibox_fasync = NULL;
+ spu->wbox_fasync = NULL;
+
down(&spu_mutex);
spu->number = number++;
ret = spu_request_irqs(spu);
Index: linux-cg/fs/spufs/file.c
===================================================================
--- linux-cg.orig/fs/spufs/file.c
+++ linux-cg/fs/spufs/file.c
@@ -198,6 +198,13 @@ size_t spu_ibox_read(struct spu *spu, u3
}
EXPORT_SYMBOL(spu_ibox_read);
+static int spufs_ibox_fasync(int fd, struct file *file, int on)
+{
+ struct spu_context *ctx;
+ ctx = file->private_data;
+ return fasync_helper(fd, file, on, &ctx->spu->ibox_fasync);
+}
+
static ssize_t spufs_ibox_read(struct file *file, char __user *buf,
size_t len, loff_t *pos)
{
@@ -253,6 +260,7 @@ static struct file_operations spufs_ibox
.open = spufs_pipe_open,
.read = spufs_ibox_read,
.poll = spufs_ibox_poll,
+ .fasync = spufs_ibox_fasync,
};
static ssize_t spufs_ibox_stat_read(struct file *file, char __user *buf,
@@ -302,6 +310,13 @@ size_t spu_wbox_write(struct spu *spu, u
}
EXPORT_SYMBOL(spu_wbox_write);
+static int spufs_wbox_fasync(int fd, struct file *file, int on)
+{
+ struct spu_context *ctx;
+ ctx = file->private_data;
+ return fasync_helper(fd, file, on, &ctx->spu->wbox_fasync);
+}
+
static ssize_t spufs_wbox_write(struct file *file, const char __user *buf,
size_t len, loff_t *pos)
{
@@ -353,6 +368,7 @@ static struct file_operations spufs_wbox
.open = spufs_pipe_open,
.write = spufs_wbox_write,
.poll = spufs_wbox_poll,
+ .fasync = spufs_wbox_fasync,
};
static ssize_t spufs_wbox_stat_read(struct file *file, char __user *buf,
Index: linux-cg/include/asm-ppc64/spu.h
===================================================================
--- linux-cg.orig/include/asm-ppc64/spu.h
+++ linux-cg/include/asm-ppc64/spu.h
@@ -128,6 +128,8 @@ struct spu {
wait_queue_head_t stop_wq;
wait_queue_head_t ibox_wq;
wait_queue_head_t wbox_wq;
+ struct fasync_struct *ibox_fasync;
+ struct fasync_struct *wbox_fasync;
char irq_c0[8];
char irq_c1[8];
--
^ permalink raw reply [flat|nested] 14+ messages in thread* [patch 07/11] spufs: Add a register file for the debugger
2005-09-16 12:16 [patch 00/11] spufs: latest spufs snapshot for 2.6.14-rc1 Arnd Bergmann
` (6 preceding siblings ...)
2005-09-16 12:16 ` [patch 06/11] spufs: allow O_ASYNC on mailbox files Arnd Bergmann
@ 2005-09-16 12:16 ` Arnd Bergmann
2005-09-16 12:16 ` [patch 08/11] spufs: make mem files mmappable Arnd Bergmann
` (3 subsequent siblings)
11 siblings, 0 replies; 14+ messages in thread
From: Arnd Bergmann @ 2005-09-16 12:16 UTC (permalink / raw)
To: ppc64-dev
Cc: linux-kernel, Paul Mackerras, Benjamin Herrenschmidt,
jordi_caubet, Hiroyuki Machida, Geoff Levand
[-- Attachment #1: spufs-debug-3.diff --]
[-- Type: text/plain, Size: 18287 bytes --]
In order to debug spu threads, we need access to the registers
of the running SPU. Unfortunately, this is only possible when
the SPU context is saved to memory.
This patch adds operations that enable accessing an SPU
in either runnable or saved state. We use an RW semaphore
to protect the state of the SPU from changing underneath
us, while we are holding it readable. In order to change
the state, it is acquired writeable and a context save
or restore is executed before downgrading the semaphore
to read-only.
Aside from the debugger, the same functionality is also
used by the SPU scheduler, which follows in the next patch.
From: Uli Weigand <Ulrich.Weigand@de.ibm.com>
Signed-off-by: Arnd Bergmann <arndb@de.ibm.com>
--
context.c | 48 +++++++
file.c | 272 ++++++++++++++++++++++++++++++++----------
spufs.h | 3
3 files changed, 264 insertions(+), 59 deletions(-)
Index: linux-cg/fs/spufs/context.c
===================================================================
--- linux-cg.orig/fs/spufs/context.c
+++ linux-cg/fs/spufs/context.c
@@ -53,6 +53,8 @@ struct spu_context *alloc_spu_context(vo
init_rwsem(&ctx->backing_sema);
spin_lock_init(&ctx->mmio_lock);
kref_init(&ctx->kref);
+ init_rwsem(&ctx->state_sema);
+ ctx->state = SPU_STATE_SAVED;
goto out;
out_free:
kfree(ctx);
@@ -82,4 +84,50 @@ int put_spu_context(struct spu_context *
return kref_put(&ctx->kref, &destroy_spu_context);
}
+void spu_acquire(struct spu_context *ctx)
+{
+ down_read(&ctx->state_sema);
+}
+
+void spu_release(struct spu_context *ctx)
+{
+ up_read(&ctx->state_sema);
+}
+
+void spu_acquire_runnable(struct spu_context *ctx)
+{
+ down_read(&ctx->state_sema);
+
+ if (ctx->state == SPU_STATE_RUNNABLE
+ || ctx->state == SPU_STATE_LOCKED)
+ return;
+
+ up_read(&ctx->state_sema);
+ down_write(&ctx->state_sema);
+ if (ctx->state == SPU_STATE_SAVED) {
+ spu_restore(&ctx->csa, ctx->spu);
+ ctx->state = SPU_STATE_RUNNABLE;
+ }
+
+ downgrade_write(&ctx->state_sema);
+}
+
+void spu_acquire_saved(struct spu_context *ctx)
+{
+ down_read(&ctx->state_sema);
+
+ if (ctx->state == SPU_STATE_SAVED
+ || ctx->state == SPU_STATE_LOCKED)
+ return;
+
+ up_read(&ctx->state_sema);
+ down_write(&ctx->state_sema);
+
+ if (ctx->state == SPU_STATE_RUNNABLE) {
+ spu_save(&ctx->csa, ctx->spu);
+ ctx->state = SPU_STATE_SAVED;
+ }
+
+ downgrade_write(&ctx->state_sema);
+}
Index: linux-cg/fs/spufs/file.c
===================================================================
--- linux-cg.orig/fs/spufs/file.c
+++ linux-cg/fs/spufs/file.c
@@ -32,6 +32,7 @@
#include "spufs.h"
+
static int
spufs_mem_open(struct inode *inode, struct file *file)
{
@@ -44,23 +45,22 @@ static ssize_t
spufs_mem_read(struct file *file, char __user *buffer,
size_t size, loff_t *pos)
{
- struct spu *spu;
- struct spu_context *ctx;
+ struct spu_context *ctx = file->private_data;
+ char *local_store;
int ret;
- ctx = file->private_data;
- spu = ctx->spu;
-
+ spu_acquire(ctx);
down_read(&ctx->backing_sema);
- if (spu->number & 0/*1*/) {
- ret = generic_file_read(file, buffer, size, pos);
- goto out;
- }
- ret = simple_read_from_buffer(buffer, size, pos,
- spu->local_store, LS_SIZE);
-out:
+ if (ctx->state == SPU_STATE_SAVED)
+ local_store = ctx->csa.lscsa->ls;
+ else
+ local_store = ctx->spu->local_store;
+
+ ret = simple_read_from_buffer(buffer, size, pos, local_store, LS_SIZE);
+
up_read(&ctx->backing_sema);
+ spu_release(ctx);
return ret;
}
@@ -69,17 +69,28 @@ spufs_mem_write(struct file *file, const
size_t size, loff_t *pos)
{
struct spu_context *ctx = file->private_data;
- struct spu *spu = ctx->spu;
-
- if (spu->number & 0) //1)
- return generic_file_write(file, buffer, size, pos);
+ char *local_store;
+ int ret;
size = min_t(ssize_t, LS_SIZE - *pos, size);
if (size <= 0)
return -EFBIG;
*pos += size;
- return copy_from_user(spu->local_store + *pos - size,
- buffer, size) ? -EFAULT : size;
+
+ spu_acquire(ctx);
+ down_read(&ctx->backing_sema);
+
+ if (ctx->state == SPU_STATE_SAVED)
+ local_store = ctx->csa.lscsa->ls;
+ else
+ local_store = ctx->spu->local_store;
+
+ ret = copy_from_user(local_store + *pos - size,
+ buffer, size) ? -EFAULT : size;
+
+ up_read(&ctx->backing_sema);
+ spu_release(ctx);
+ return ret;
}
static int
@@ -88,9 +99,9 @@ spufs_mem_mmap(struct file *file, struct
struct spu_context *ctx = file->private_data;
struct spu *spu = ctx->spu;
unsigned long pfn;
+ int ret = 0;
- if (spu->number & 0) //1)
- return generic_file_mmap(file, vma);
+ spu_acquire_runnable(ctx);
vma->vm_flags |= VM_RESERVED;
vma->vm_page_prot = __pgprot(pgprot_val (vma->vm_page_prot)
@@ -101,8 +112,13 @@ spufs_mem_mmap(struct file *file, struct
*/
if (remap_pfn_range(vma, vma->vm_start, pfn,
vma->vm_end-vma->vm_start, vma->vm_page_prot))
- return -EAGAIN;
- return 0;
+ ret = -EAGAIN;
+
+ if (!ret)
+ ctx->state = SPU_STATE_LOCKED;
+
+ spu_release(ctx);
+ return ret;
}
static struct file_operations spufs_mem_fops = {
@@ -113,6 +129,68 @@ static struct file_operations spufs_mem_
.llseek = generic_file_llseek,
};
+static int
+spufs_regs_open(struct inode *inode, struct file *file)
+{
+ struct spufs_inode_info *i = SPUFS_I(inode);
+ file->private_data = i->i_ctx;
+ return 0;
+}
+
+static ssize_t
+spufs_regs_read(struct file *file, char __user *buffer,
+ size_t size, loff_t *pos)
+{
+ struct spu_context *ctx = file->private_data;
+ struct spu_lscsa *lscsa = ctx->csa.lscsa;
+ int ret;
+
+ spu_acquire_saved(ctx);
+ if (ctx->state == SPU_STATE_LOCKED) {
+ spu_release(ctx);
+ return -EAGAIN;
+ }
+
+ ret = simple_read_from_buffer(buffer, size, pos,
+ lscsa->gprs, sizeof lscsa->gprs);
+
+ spu_release(ctx);
+ return ret;
+}
+
+static ssize_t
+spufs_regs_write(struct file *file, const char __user *buffer,
+ size_t size, loff_t *pos)
+{
+ struct spu_context *ctx = file->private_data;
+ struct spu_lscsa *lscsa = ctx->csa.lscsa;
+ int ret;
+
+ size = min_t(ssize_t, sizeof lscsa->gprs - *pos, size);
+ if (size <= 0)
+ return -EFBIG;
+ *pos += size;
+
+ spu_acquire_saved(ctx);
+ if (ctx->state == SPU_STATE_LOCKED) {
+ spu_release(ctx);
+ return -EAGAIN;
+ }
+
+ ret = copy_from_user(lscsa->gprs + *pos - size,
+ buffer, size) ? -EFAULT : size;
+
+ spu_release(ctx);
+ return ret;
+}
+
+static struct file_operations spufs_regs_fops = {
+ .open = spufs_regs_open,
+ .read = spufs_regs_read,
+ .write = spufs_regs_write,
+ .llseek = generic_file_llseek,
+};
+
/* generic open function for all pipe-like files */
static int spufs_pipe_open(struct inode *inode, struct file *file)
{
@@ -125,7 +203,7 @@ static int spufs_pipe_open(struct inode
static ssize_t spufs_mbox_read(struct file *file, char __user *buf,
size_t len, loff_t *pos)
{
- struct spu_context *ctx;
+ struct spu_context *ctx = file->private_data;
struct spu_problem __iomem *prob;
u32 mbox_stat;
u32 mbox_data;
@@ -133,14 +211,19 @@ static ssize_t spufs_mbox_read(struct fi
if (len < 4)
return -EINVAL;
- ctx = file->private_data;
+ spu_acquire_runnable(ctx);
+
prob = ctx->spu->problem;
mbox_stat = in_be32(&prob->mb_stat_R);
- if (!(mbox_stat & 0x0000ff))
+ if (!(mbox_stat & 0x0000ff)) {
+ spu_release(ctx);
return -EAGAIN;
+ }
mbox_data = in_be32(&prob->pu_mb_R);
+ spu_release(ctx);
+
if (copy_to_user(buf, &mbox_data, sizeof mbox_data))
return -EFAULT;
@@ -155,14 +238,15 @@ static struct file_operations spufs_mbox
static ssize_t spufs_mbox_stat_read(struct file *file, char __user *buf,
size_t len, loff_t *pos)
{
- struct spu_context *ctx;
+ struct spu_context *ctx = file->private_data;
u32 mbox_stat;
if (len < 4)
return -EINVAL;
- ctx = file->private_data;
+ spu_acquire_runnable(ctx);
mbox_stat = in_be32(&ctx->spu->problem->mb_stat_R) & 0xff;
+ spu_release(ctx);
if (copy_to_user(buf, &mbox_stat, sizeof mbox_stat))
return -EFAULT;
@@ -200,22 +284,27 @@ EXPORT_SYMBOL(spu_ibox_read);
static int spufs_ibox_fasync(int fd, struct file *file, int on)
{
- struct spu_context *ctx;
- ctx = file->private_data;
- return fasync_helper(fd, file, on, &ctx->spu->ibox_fasync);
+ struct spu_context *ctx = file->private_data;
+ int ret;
+
+ spu_acquire_runnable(ctx);
+ ret = fasync_helper(fd, file, on, &ctx->spu->ibox_fasync);
+ spu_release(ctx);
+
+ return ret;
}
static ssize_t spufs_ibox_read(struct file *file, char __user *buf,
size_t len, loff_t *pos)
{
- struct spu_context *ctx;
+ struct spu_context *ctx = file->private_data;
u32 ibox_data;
ssize_t ret;
if (len < 4)
return -EINVAL;
- ctx = file->private_data;
+ spu_acquire_runnable(ctx);
ret = 0;
if (file->f_flags & O_NONBLOCK) {
@@ -226,6 +315,8 @@ static ssize_t spufs_ibox_read(struct fi
spu_ibox_read(ctx->spu, &ibox_data));
}
+ spu_release(ctx);
+
if (ret)
return ret;
@@ -238,17 +329,20 @@ static ssize_t spufs_ibox_read(struct fi
static unsigned int spufs_ibox_poll(struct file *file, poll_table *wait)
{
- struct spu_context *ctx;
+ struct spu_context *ctx = file->private_data;
struct spu_problem __iomem *prob;
u32 mbox_stat;
unsigned int mask;
- ctx = file->private_data;
+ spu_acquire_runnable(ctx);
+
prob = ctx->spu->problem;
mbox_stat = in_be32(&prob->mb_stat_R);
poll_wait(file, &ctx->spu->ibox_wq, wait);
+ spu_release(ctx);
+
mask = 0;
if (mbox_stat & 0xff0000)
mask |= POLLIN | POLLRDNORM;
@@ -266,14 +360,15 @@ static struct file_operations spufs_ibox
static ssize_t spufs_ibox_stat_read(struct file *file, char __user *buf,
size_t len, loff_t *pos)
{
- struct spu_context *ctx;
+ struct spu_context *ctx = file->private_data;
u32 ibox_stat;
if (len < 4)
return -EINVAL;
- ctx = file->private_data;
+ spu_acquire_runnable(ctx);
ibox_stat = (in_be32(&ctx->spu->problem->mb_stat_R) >> 16) & 0xff;
+ spu_release(ctx);
if (copy_to_user(buf, &ibox_stat, sizeof ibox_stat))
return -EFAULT;
@@ -312,26 +407,31 @@ EXPORT_SYMBOL(spu_wbox_write);
static int spufs_wbox_fasync(int fd, struct file *file, int on)
{
- struct spu_context *ctx;
- ctx = file->private_data;
- return fasync_helper(fd, file, on, &ctx->spu->wbox_fasync);
+ struct spu_context *ctx = file->private_data;
+ int ret;
+
+ spu_acquire_runnable(ctx);
+ ret = fasync_helper(fd, file, on, &ctx->spu->wbox_fasync);
+ spu_release(ctx);
+
+ return ret;
}
static ssize_t spufs_wbox_write(struct file *file, const char __user *buf,
size_t len, loff_t *pos)
{
- struct spu_context *ctx;
+ struct spu_context *ctx = file->private_data;
u32 wbox_data;
int ret;
if (len < 4)
return -EINVAL;
- ctx = file->private_data;
-
if (copy_from_user(&wbox_data, buf, sizeof wbox_data))
return -EFAULT;
+ spu_acquire_runnable(ctx);
+
ret = 0;
if (file->f_flags & O_NONBLOCK) {
if (!spu_wbox_write(ctx->spu, wbox_data))
@@ -341,22 +441,27 @@ static ssize_t spufs_wbox_write(struct f
spu_wbox_write(ctx->spu, wbox_data));
}
+ spu_release(ctx);
+
return ret ? ret : sizeof wbox_data;
}
static unsigned int spufs_wbox_poll(struct file *file, poll_table *wait)
{
- struct spu_context *ctx;
+ struct spu_context *ctx = file->private_data;
struct spu_problem __iomem *prob;
u32 mbox_stat;
unsigned int mask;
- ctx = file->private_data;
+ spu_acquire_runnable(ctx);
+
prob = ctx->spu->problem;
mbox_stat = in_be32(&prob->mb_stat_R);
poll_wait(file, &ctx->spu->wbox_wq, wait);
+ spu_release(ctx);
+
mask = 0;
if (mbox_stat & 0x00ff00)
mask = POLLOUT | POLLWRNORM;
@@ -374,14 +479,15 @@ static struct file_operations spufs_wbox
static ssize_t spufs_wbox_stat_read(struct file *file, char __user *buf,
size_t len, loff_t *pos)
{
- struct spu_context *ctx;
+ struct spu_context *ctx = file->private_data;
u32 wbox_stat;
if (len < 4)
return -EINVAL;
- ctx = file->private_data;
+ spu_acquire_runnable(ctx);
wbox_stat = (in_be32(&ctx->spu->problem->mb_stat_R) >> 8) & 0xff;
+ spu_release(ctx);
if (copy_to_user(buf, &wbox_stat, sizeof wbox_stat))
return -EFAULT;
@@ -408,6 +514,8 @@ long spufs_run_spu(struct file *file, st
down_write(&ctx->backing_sema);
}
+ spu_acquire_runnable(ctx);
+
prob = ctx->spu->problem;
out_be32(&prob->spu_npc_RW, *npc);
@@ -416,6 +524,7 @@ long spufs_run_spu(struct file *file, st
*status = in_be32(&prob->spu_status_R);
*npc = in_be32(&prob->spu_npc_RW);
+ spu_release(ctx);
up_write(&ctx->backing_sema);
out:
@@ -440,12 +549,23 @@ static ssize_t spufs_run_read(struct fil
if (len < 8)
goto out;
- arg.npc = in_be32(&ctx->spu->problem->spu_npc_RW);
+ if (file->f_flags & O_NONBLOCK) {
+ ret = -EAGAIN;
+ if (!down_write_trylock(&ctx->backing_sema))
+ goto out;
+ } else {
+ down_write(&ctx->backing_sema);
+ }
- ret = spufs_run_spu(file, file->private_data, &arg.npc, &arg.status);
+ spu_acquire_runnable(ctx);
+
+ ret = spu_run(ctx->spu);
if (ret == -EAGAIN)
ret = 0;
+ arg.status = in_be32(&ctx->spu->problem->spu_status_R);
+ arg.npc = in_be32(&ctx->spu->problem->spu_npc_RW);
+
if ((arg.status & 0xffff0002) == 0x21000002) {
/* library callout */
u32 npc = arg.npc;
@@ -454,6 +574,9 @@ static ssize_t spufs_run_read(struct fil
out_be32(&ctx->spu->problem->spu_npc_RW, npc);
}
+ spu_release(ctx);
+ up_write(&ctx->backing_sema);
+
if (ret)
goto out;
@@ -497,17 +620,18 @@ static struct file_operations spufs_run_
static ssize_t spufs_signal1_read(struct file *file, char __user *buf,
size_t len, loff_t *pos)
{
- struct spu_context *ctx;
+ struct spu_context *ctx = file->private_data;
struct spu_problem *prob;
u32 data;
- ctx = file->private_data;
- prob = ctx->spu->problem;
-
if (len < 4)
return -EINVAL;
+ spu_acquire_runnable(ctx);
+ prob = ctx->spu->problem;
data = in_be32(&prob->signal_notify1);
+ spu_release(ctx);
+
if (copy_to_user(buf, &data, 4))
return -EFAULT;
@@ -522,7 +646,6 @@ static ssize_t spufs_signal1_write(struc
u32 data;
ctx = file->private_data;
- prob = ctx->spu->problem;
if (len < 4)
return -EINVAL;
@@ -530,7 +653,10 @@ static ssize_t spufs_signal1_write(struc
if (copy_from_user(&data, buf, 4))
return -EFAULT;
+ spu_acquire_runnable(ctx);
+ prob = ctx->spu->problem;
out_be32(&prob->signal_notify1, data);
+ spu_release(ctx);
return 4;
}
@@ -549,12 +675,15 @@ static ssize_t spufs_signal2_read(struct
u32 data;
ctx = file->private_data;
- prob = ctx->spu->problem;
if (len < 4)
return -EINVAL;
+ spu_acquire_runnable(ctx);
+ prob = ctx->spu->problem;
data = in_be32(&prob->signal_notify2);
+ spu_release(ctx);
+
if (copy_to_user(buf, &data, 4))
return -EFAULT;
@@ -569,7 +698,6 @@ static ssize_t spufs_signal2_write(struc
u32 data;
ctx = file->private_data;
- prob = ctx->spu->problem;
if (len < 4)
return -EINVAL;
@@ -577,7 +705,10 @@ static ssize_t spufs_signal2_write(struc
if (copy_from_user(&data, buf, 4))
return -EFAULT;
+ spu_acquire_runnable(ctx);
+ prob = ctx->spu->problem;
out_be32(&prob->signal_notify2, data);
+ spu_release(ctx);
return 4;
}
@@ -591,9 +722,11 @@ static struct file_operations spufs_sign
static void spufs_signal1_type_set(void *data, u64 val)
{
struct spu_context *ctx = data;
- struct spu_priv2 *priv2 = ctx->spu->priv2;
+ struct spu_priv2 *priv2;
u64 tmp;
+ spu_acquire_runnable(ctx);
+ priv2 = ctx->spu->priv2;
spin_lock_irq(&ctx->spu->register_lock);
tmp = in_be64(&priv2->spu_cfg_RW);
if (val)
@@ -602,12 +735,19 @@ static void spufs_signal1_type_set(void
tmp &= ~1;
out_be64(&priv2->spu_cfg_RW, tmp);
spin_unlock_irq(&ctx->spu->register_lock);
+ spu_release(ctx);
}
static u64 spufs_signal1_type_get(void *data)
{
struct spu_context *ctx = data;
- return (in_be64(&ctx->spu->priv2->spu_cfg_RW) & 1) != 0;
+ u64 ret;
+
+ spu_acquire_runnable(ctx);
+ ret = ((in_be64(&ctx->spu->priv2->spu_cfg_RW) & 1) != 0);
+ spu_release(ctx);
+
+ return ret;
}
DEFINE_SIMPLE_ATTRIBUTE(spufs_signal1_type, spufs_signal1_type_get,
spufs_signal1_type_set, "%llu");
@@ -615,9 +755,11 @@ DEFINE_SIMPLE_ATTRIBUTE(spufs_signal1_ty
static void spufs_signal2_type_set(void *data, u64 val)
{
struct spu_context *ctx = data;
- struct spu_priv2 *priv2 = ctx->spu->priv2;
+ struct spu_priv2 *priv2;
u64 tmp;
+ spu_acquire_runnable(ctx);
+ priv2 = ctx->spu->priv2;
spin_lock_irq(&ctx->spu->register_lock);
tmp = in_be64(&priv2->spu_cfg_RW);
if (val)
@@ -626,12 +768,19 @@ static void spufs_signal2_type_set(void
tmp &= ~2;
out_be64(&priv2->spu_cfg_RW, tmp);
spin_unlock_irq(&ctx->spu->register_lock);
+ spu_release(ctx);
}
static u64 spufs_signal2_type_get(void *data)
{
struct spu_context *ctx = data;
- return (in_be64(&ctx->spu->priv2->spu_cfg_RW) & 2) != 0;
+ u64 ret;
+
+ spu_acquire_runnable(ctx);
+ ret = ((in_be64(&ctx->spu->priv2->spu_cfg_RW) & 2) != 0);
+ spu_release(ctx);
+
+ return ret;
}
DEFINE_SIMPLE_ATTRIBUTE(spufs_signal2_type, spufs_signal2_type_get,
spufs_signal2_type_set, "%llu");
@@ -639,20 +788,25 @@ DEFINE_SIMPLE_ATTRIBUTE(spufs_signal2_ty
static void spufs_npc_set(void *data, u64 val)
{
struct spu_context *ctx = data;
+ spu_acquire_runnable(ctx);
out_be32(&ctx->spu->problem->spu_npc_RW, val);
+ spu_release(ctx);
}
static u64 spufs_npc_get(void *data)
{
struct spu_context *ctx = data;
u64 ret;
+ spu_acquire_runnable(ctx);
ret = in_be32(&ctx->spu->problem->spu_npc_RW);
+ spu_release(ctx);
return ret;
}
DEFINE_SIMPLE_ATTRIBUTE(spufs_npc_ops, spufs_npc_get, spufs_npc_set, "%llx\n")
struct tree_descr spufs_dir_contents[] = {
{ "mem", &spufs_mem_fops, 0666, },
+ { "regs", &spufs_regs_fops, 0666, },
{ "run", &spufs_run_fops, 0444, },
{ "mbox", &spufs_mbox_fops, 0444, },
{ "ibox", &spufs_ibox_fops, 0444, },
Index: linux-cg/fs/spufs/spufs.h
===================================================================
--- linux-cg.orig/fs/spufs/spufs.h
+++ linux-cg/fs/spufs/spufs.h
@@ -41,6 +41,9 @@ struct spu_context {
struct rw_semaphore backing_sema; /* protects the above */
spinlock_t mmio_lock; /* protects mmio access */
+ enum { SPU_STATE_RUNNABLE, SPU_STATE_SAVED, SPU_STATE_LOCKED } state;
+ struct rw_semaphore state_sema;
+
struct kref kref;
};
--
^ permalink raw reply [flat|nested] 14+ messages in thread* [patch 08/11] spufs: make mem files mmappable
2005-09-16 12:16 [patch 00/11] spufs: latest spufs snapshot for 2.6.14-rc1 Arnd Bergmann
` (7 preceding siblings ...)
2005-09-16 12:16 ` [patch 07/11] spufs: Add a register file for the debugger Arnd Bergmann
@ 2005-09-16 12:16 ` Arnd Bergmann
2005-09-17 6:58 ` Arnd Bergmann
2005-09-16 12:16 ` [patch 09/11] spufs: SPU scheduler Arnd Bergmann
` (2 subsequent siblings)
11 siblings, 1 reply; 14+ messages in thread
From: Arnd Bergmann @ 2005-09-16 12:16 UTC (permalink / raw)
To: ppc64-dev
Cc: linux-kernel, Paul Mackerras, Benjamin Herrenschmidt,
jordi_caubet, Hiroyuki Machida, Geoff Levand
[-- Attachment #1: spu-backingstore-2.diff --]
[-- Type: text/plain, Size: 11087 bytes --]
This patch makes it possible to mmap the "mem" files in spufs
even when they are alternating between direct mapping to
on-chip memory and the context save area.
We discard all page table entries during the SPU context switch
and get them back using the nopage operation. Unfortunately,
that requires struct page pointers for the local storage, which
is typically at the top of the 42 bit address space. In order
to get this, we use the current sparsemem support, but that still
wastes 2MB of RAM for its section pointers.
This should get better as soon as extreme sparsemem gets merged.
From: Ulrich Weigand <Ulrich.Weigand@de.ibm.com>
Signed-off-by: Arnd Bergmann <arndb@de.ibm.com>
--
arch/ppc64/Kconfig | 2
arch/ppc64/kernel/bpa_setup.c | 75 ++++++++++++++++++++++
fs/spufs/context.c | 12 +++
fs/spufs/file.c | 69 ++++++++++++++++----
fs/spufs/inode.c | 2
fs/spufs/spufs.h | 3
fs/spufs/switch.c | 10 ++
include/asm-ppc64/sparsemem.h | 4 -
8 files changed, 157 insertions(+), 20 deletions(-)
Index: linux-cg/arch/ppc64/Kconfig
===================================================================
--- linux-cg.orig/arch/ppc64/Kconfig
+++ linux-cg/arch/ppc64/Kconfig
@@ -258,7 +258,7 @@ config ARCH_FLATMEM_ENABLE
config ARCH_SPARSEMEM_ENABLE
def_bool y
- depends on ARCH_DISCONTIGMEM_ENABLE
+ depends on ARCH_DISCONTIGMEM_ENABLE || PPC_BPA
source "mm/Kconfig"
Index: linux-cg/arch/ppc64/kernel/bpa_setup.c
===================================================================
--- linux-cg.orig/arch/ppc64/kernel/bpa_setup.c
+++ linux-cg/arch/ppc64/kernel/bpa_setup.c
@@ -66,6 +66,77 @@ void bpa_get_cpuinfo(struct seq_file *m)
of_node_put(root);
}
+#ifdef CONFIG_SPARSEMEM
+static int __init find_spu_node_id(struct device_node *spe)
+{
+ unsigned int *id;
+#ifdef CONFIG_NUMA
+ struct device_node *cpu;
+ cpu = spe->parent->parent;
+ id = (unsigned int *)get_property(cpu, "node-id", NULL);
+#else
+ id = NULL;
+#endif
+ return id ? *id : 0;
+}
+
+static void __init bpa_spuprop_present(struct device_node *spe,
+ const char *prop, int early)
+{
+ struct address_prop {
+ unsigned long address;
+ unsigned int len;
+ } __attribute__((packed)) *p;
+ int proplen;
+
+ unsigned long start_pfn, end_pfn, pfn;
+ int node_id;
+
+ p = (void*)get_property(spe, prop, &proplen);
+ WARN_ON(proplen != sizeof (*p));
+
+ node_id = find_spu_node_id(spe);
+
+ start_pfn = p->address >> PAGE_SHIFT;
+ end_pfn = (p->address + p->len + PAGE_SIZE - 1) >> PAGE_SHIFT;
+
+ /* We need to call memory_present *before* the call to sparse_init,
+ but we can initialize the page structs only *after* that call.
+ Thus, we're being called twice. */
+ if (early)
+ memory_present(node_id, start_pfn, end_pfn);
+ else {
+ /* As the pages backing SPU LS and I/O are outside the range
+ of regular memory, their page structs were not initialized
+ by free_area_init. Do it here instead. */
+ for (pfn = start_pfn; pfn < end_pfn; pfn++) {
+ struct page *page = pfn_to_page(pfn);
+ set_page_links(page, ZONE_DMA, node_id, pfn);
+ set_page_count(page, 0);
+ reset_page_mapcount(page);
+ SetPageReserved(page);
+ INIT_LIST_HEAD(&page->lru);
+ }
+ }
+}
+
+static void __init bpa_spumem_init(int early)
+{
+ struct device_node *node;
+ for (node = of_find_node_by_type(NULL, "spe");
+ node; node = of_find_node_by_type(node, "spe")) {
+ bpa_spuprop_present(node, "local-store", early);
+ bpa_spuprop_present(node, "problem", early);
+ bpa_spuprop_present(node, "priv1", early);
+ bpa_spuprop_present(node, "priv2", early);
+ }
+}
+#else
+static void __init bpa_spumem_init(int early)
+{
+}
+#endif
+
static void bpa_progress(char *s, unsigned short hex)
{
printk("*** %04x : %s\n", hex, s ? s : "");
@@ -97,6 +168,8 @@ static void __init bpa_setup_arch(void)
#endif
bpa_nvram_init();
+
+ bpa_spumem_init(0);
}
/*
@@ -112,6 +185,8 @@ static void __init bpa_init_early(void)
ppc64_interrupt_controller = IC_BPA_IIC;
+ bpa_spumem_init(1);
+
DBG(" <- bpa_init_early()\n");
}
Index: linux-cg/include/asm-ppc64/sparsemem.h
===================================================================
--- linux-cg.orig/include/asm-ppc64/sparsemem.h
+++ linux-cg/include/asm-ppc64/sparsemem.h
@@ -8,8 +8,8 @@
* MAX_PHYSMEM_BITS 2^N: how much memory we can have in that space
*/
#define SECTION_SIZE_BITS 24
-#define MAX_PHYSADDR_BITS 38
-#define MAX_PHYSMEM_BITS 36
+#define MAX_PHYSADDR_BITS 42
+#define MAX_PHYSMEM_BITS 42
#endif /* CONFIG_SPARSEMEM */
Index: linux-cg/fs/spufs/context.c
===================================================================
--- linux-cg.orig/fs/spufs/context.c
+++ linux-cg/fs/spufs/context.c
@@ -20,12 +20,14 @@
* Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
*/
+#include <linux/fs.h>
+#include <linux/mm.h>
#include <linux/slab.h>
#include <asm/spu.h>
#include <asm/spu_csa.h>
#include "spufs.h"
-struct spu_context *alloc_spu_context(void)
+struct spu_context *alloc_spu_context(struct address_space *local_store)
{
struct spu_context *ctx;
ctx = kmalloc(sizeof *ctx, GFP_KERNEL);
@@ -55,6 +57,7 @@ struct spu_context *alloc_spu_context(vo
kref_init(&ctx->kref);
init_rwsem(&ctx->state_sema);
ctx->state = SPU_STATE_SAVED;
+ ctx->local_store = local_store;
goto out;
out_free:
kfree(ctx);
@@ -94,6 +97,11 @@ void spu_release(struct spu_context *ctx
up_read(&ctx->state_sema);
}
+static void spu_unmap_mappings(struct spu_context *ctx)
+{
+ unmap_mapping_range(ctx->local_store, 0, LS_SIZE, 1);
+}
+
void spu_acquire_runnable(struct spu_context *ctx)
{
down_read(&ctx->state_sema);
@@ -106,6 +114,7 @@ void spu_acquire_runnable(struct spu_con
down_write(&ctx->state_sema);
if (ctx->state == SPU_STATE_SAVED) {
+ spu_unmap_mappings(ctx);
spu_restore(&ctx->csa, ctx->spu);
ctx->state = SPU_STATE_RUNNABLE;
}
@@ -125,6 +134,7 @@ void spu_acquire_saved(struct spu_contex
down_write(&ctx->state_sema);
if (ctx->state == SPU_STATE_RUNNABLE) {
+ spu_unmap_mappings(ctx);
spu_save(&ctx->csa, ctx->spu);
ctx->state = SPU_STATE_SAVED;
}
Index: linux-cg/fs/spufs/file.c
===================================================================
--- linux-cg.orig/fs/spufs/file.c
+++ linux-cg/fs/spufs/file.c
@@ -38,6 +38,7 @@ spufs_mem_open(struct inode *inode, stru
{
struct spufs_inode_info *i = SPUFS_I(inode);
file->private_data = i->i_ctx;
+ file->f_mapping = i->i_ctx->local_store;
return 0;
}
@@ -93,32 +94,72 @@ spufs_mem_write(struct file *file, const
return ret;
}
+#ifdef CONFIG_SPARSEMEM
+static struct page *
+spufs_mem_mmap_nopage(struct vm_area_struct *vma,
+ unsigned long address, int *type)
+{
+ struct page *page = NOPAGE_SIGBUS;
+
+ struct spu_context *ctx = vma->vm_file->private_data;
+ unsigned long offset = address - vma->vm_start;
+ offset += vma->vm_pgoff << PAGE_SHIFT;
+
+ spu_acquire(ctx);
+
+ if (ctx->state == SPU_STATE_SAVED)
+ page = vmalloc_to_page(ctx->csa.lscsa->ls + offset);
+ else
+ page = pfn_to_page((ctx->spu->local_store_phys + offset)
+ >> PAGE_SHIFT);
+
+ spu_release(ctx);
+
+ if (type)
+ *type = VM_FAULT_MINOR;
+
+ return page;
+}
+
+static struct vm_operations_struct spufs_mem_mmap_vmops = {
+ .nopage = spufs_mem_mmap_nopage,
+};
+#endif
+
static int
spufs_mem_mmap(struct file *file, struct vm_area_struct *vma)
{
+#ifndef CONFIG_SPARSEMEM
struct spu_context *ctx = file->private_data;
- struct spu *spu = ctx->spu;
- unsigned long pfn;
- int ret = 0;
+#endif
- spu_acquire_runnable(ctx);
+ if (!(vma->vm_flags & VM_SHARED))
+ return -EINVAL;
+ /* FIXME: */
vma->vm_flags |= VM_RESERVED;
- vma->vm_page_prot = __pgprot(pgprot_val (vma->vm_page_prot)
- | _PAGE_NO_CACHE);
- pfn = spu->local_store_phys >> PAGE_SHIFT;
+ vma->vm_page_prot = __pgprot(pgprot_val(vma->vm_page_prot)
+ | _PAGE_NO_CACHE);
+
+#ifndef CONFIG_SPARSEMEM
+ spu_acquire_runnable(ctx);
+
/*
* This will work for actual SPUs, but not for vmalloc memory:
*/
- if (remap_pfn_range(vma, vma->vm_start, pfn,
- vma->vm_end-vma->vm_start, vma->vm_page_prot))
- ret = -EAGAIN;
-
- if (!ret)
- ctx->state = SPU_STATE_LOCKED;
+ if (remap_pfn_range(vma, vma->vm_start,
+ ctx->spu->local_store_phys >> PAGE_SHIFT,
+ vma->vm_end - vma->vm_start, vma->vm_page_prot)) {
+ spu_release(ctx);
+ return -EAGAIN;
+ }
+ ctx->state = SPU_STATE_LOCKED;
spu_release(ctx);
- return ret;
+#else
+ vma->vm_ops = &spufs_mem_mmap_vmops;
+#endif
+ return 0;
}
static struct file_operations spufs_mem_fops = {
Index: linux-cg/fs/spufs/inode.c
===================================================================
--- linux-cg.orig/fs/spufs/inode.c
+++ linux-cg/fs/spufs/inode.c
@@ -282,7 +282,7 @@ spufs_mkdir(struct inode *dir, struct de
inode->i_gid = dir->i_gid;
inode->i_mode &= S_ISGID;
}
- ctx = alloc_spu_context();
+ ctx = alloc_spu_context(inode->i_mapping);
SPUFS_I(inode)->i_ctx = ctx;
if (!ctx)
goto out_iput;
Index: linux-cg/fs/spufs/spufs.h
===================================================================
--- linux-cg.orig/fs/spufs/spufs.h
+++ linux-cg/fs/spufs/spufs.h
@@ -40,6 +40,7 @@ struct spu_context {
struct spu_state csa; /* SPU context save area. */
struct rw_semaphore backing_sema; /* protects the above */
spinlock_t mmio_lock; /* protects mmio access */
+ struct address_space *local_store;/* local store backing store */
enum { SPU_STATE_RUNNABLE, SPU_STATE_SAVED, SPU_STATE_LOCKED } state;
struct rw_semaphore state_sema;
@@ -63,7 +64,7 @@ long spufs_create_thread(struct nameidat
unsigned int flags, mode_t mode);
/* context management */
-struct spu_context * alloc_spu_context(void);
+struct spu_context * alloc_spu_context(struct address_space *local_store);
void destroy_spu_context(struct kref *kref);
struct spu_context * get_spu_context(struct spu_context *ctx);
int put_spu_context(struct spu_context *ctx);
Index: linux-cg/fs/spufs/switch.c
===================================================================
--- linux-cg.orig/fs/spufs/switch.c
+++ linux-cg/fs/spufs/switch.c
@@ -2191,6 +2191,7 @@ static void init_priv2(struct spu_state
void spu_init_csa(struct spu_state *csa)
{
struct spu_lscsa *lscsa;
+ unsigned char *p;
if (!csa)
return;
@@ -2203,6 +2204,10 @@ void spu_init_csa(struct spu_state *csa)
memset(lscsa, 0, sizeof(struct spu_lscsa));
csa->lscsa = lscsa;
+ /* Set LS pages reserved to allow for user-space mapping. */
+ for (p = lscsa->ls; p < lscsa->ls + LS_SIZE; p += PAGE_SIZE)
+ SetPageReserved(vmalloc_to_page(p));
+
init_prob(csa);
init_priv1(csa);
init_priv2(csa);
@@ -2210,5 +2215,10 @@ void spu_init_csa(struct spu_state *csa)
void spu_fini_csa(struct spu_state *csa)
{
+ /* Clear reserved bit before vfree. */
+ unsigned char *p;
+ for (p = csa->lscsa->ls; p < csa->lscsa->ls + LS_SIZE; p += PAGE_SIZE)
+ ClearPageReserved(vmalloc_to_page(p));
+
vfree(csa->lscsa);
}
--
^ permalink raw reply [flat|nested] 14+ messages in thread* Re: [patch 08/11] spufs: make mem files mmappable
2005-09-16 12:16 ` [patch 08/11] spufs: make mem files mmappable Arnd Bergmann
@ 2005-09-17 6:58 ` Arnd Bergmann
0 siblings, 0 replies; 14+ messages in thread
From: Arnd Bergmann @ 2005-09-17 6:58 UTC (permalink / raw)
To: linuxppc64-dev; +Cc: linux-kernel, Paul Mackerras, Hiroyuki Machida
On Freedag 16 September 2005 14:16, Arnd Bergmann wrote:
> This should get better as soon as extreme sparsemem gets merged.
>
Actually, it first got worse.
The initialization for the SPU page structures broke
with the inclusion of extreme sparsemem in current
kernels. This patch works around that problem by further
moving code around.
I still need to find a way to do this in a cleaner way,
but for now, it restores the basic functionality.
Signed-off-by: Arnd Bergmann <arndb@de.ibm.com>
Index: linux-cg/include/asm-ppc64/spu.h
===================================================================
--- linux-cg.orig/include/asm-ppc64/spu.h
+++ linux-cg/include/asm-ppc64/spu.h
@@ -167,6 +167,13 @@ static inline void unregister_spu_syscal
}
#endif /* MODULE */
+#if defined(CONFIG_SPARSEMEM) && defined(CONFIG_PPC_BPA)
+void __init bpa_spumem_init(int early);
+#else
+static inline void bpa_spumem_init(int early)
+{
+}
+#endif
/*
* This defines the Local Store, Problem Area and Privlege Area of an SPU.
Index: linux-cg/arch/ppc64/kernel/bpa_setup.c
===================================================================
--- linux-cg.orig/arch/ppc64/kernel/bpa_setup.c
+++ linux-cg/arch/ppc64/kernel/bpa_setup.c
@@ -122,7 +122,7 @@ static void __init bpa_spuprop_present(s
}
}
-static void __init bpa_spumem_init(int early)
+void __init bpa_spumem_init(int early)
{
struct device_node *node;
for (node = of_find_node_by_type(NULL, "spe");
@@ -133,10 +133,6 @@ static void __init bpa_spumem_init(int e
bpa_spuprop_present(node, "priv2", early);
}
}
-#else
-static void __init bpa_spumem_init(int early)
-{
-}
#endif
static void bpa_progress(char *s, unsigned short hex)
@@ -187,8 +183,6 @@ static void __init bpa_init_early(void)
ppc64_interrupt_controller = IC_BPA_IIC;
- bpa_spumem_init(1);
-
DBG(" <- bpa_init_early()\n");
}
Index: linux-cg/arch/ppc64/kernel/setup.c
===================================================================
--- linux-cg.orig/arch/ppc64/kernel/setup.c
+++ linux-cg/arch/ppc64/kernel/setup.c
@@ -58,6 +58,7 @@
#include <asm/mmu.h>
#include <asm/lmb.h>
#include <asm/iSeries/ItLpNaca.h>
+#include <asm/spu.h>
#ifdef DEBUG
#define DBG(fmt...) udbg_printf(fmt)
@@ -1042,6 +1043,8 @@ void __init setup_arch(char **cmdline_p)
/* set up the bootmem stuff with available memory */
do_init_bootmem();
+ bpa_spumem_init(1);
+
sparse_init();
/* initialize the syscall map in systemcfg */
^ permalink raw reply [flat|nested] 14+ messages in thread
* [patch 09/11] spufs: SPU scheduler
2005-09-16 12:16 [patch 00/11] spufs: latest spufs snapshot for 2.6.14-rc1 Arnd Bergmann
` (8 preceding siblings ...)
2005-09-16 12:16 ` [patch 08/11] spufs: make mem files mmappable Arnd Bergmann
@ 2005-09-16 12:16 ` Arnd Bergmann
2005-09-16 12:16 ` [patch 10/11] spufs: new entries for SPU special purpose registers Arnd Bergmann
2005-09-16 12:16 ` [patch 11/11] spufs: remove old user interfaces Arnd Bergmann
11 siblings, 0 replies; 14+ messages in thread
From: Arnd Bergmann @ 2005-09-16 12:16 UTC (permalink / raw)
To: ppc64-dev
Cc: linux-kernel, Paul Mackerras, Benjamin Herrenschmidt,
jordi_caubet, Hiroyuki Machida, Geoff Levand
[-- Attachment #1: spu-sched-5.diff --]
[-- Type: text/plain, Size: 53169 bytes --]
This adds a scheduler for SPUs to make it possible to use
more logical SPUs than physical ones are present in the
system.
Currently, there is no support for preempting a running
SPU thread, they have to leave the SPU by either triggering
an event on the SPU that causes it to return to the
owning thread or by sending a signal to it.
This patch also gives access to most parts of the saved
state without scheduling back to the physical SPU.
From: Mark Nutter <mnutter@us.ibm.com>
Signed-off-by: Arnd Bergmann <arndb@de.ibm.com>
--
arch/ppc64/kernel/spu_base.c | 23 -
fs/spufs/Makefile | 1
fs/spufs/backing_ops.c | 252 ++++++++++++++
fs/spufs/context.c | 72 ++--
fs/spufs/file.c | 292 ++++++----------
fs/spufs/hw_ops.c | 206 +++++++++++
fs/spufs/inode.c | 5
fs/spufs/sched.c | 409 +++++++++++++++++++++++
fs/spufs/spufs.h | 47 ++
fs/spufs/switch.c | 36 --
include/asm-ppc64/spu.h | 17
include/asm-ppc64/spu_csa.h | 1
12 files changed, 1120 insertions(+), 241 deletions(-)
Index: linux-cg/arch/ppc64/kernel/spu_base.c
===================================================================
--- linux-cg.orig/arch/ppc64/kernel/spu_base.c
+++ linux-cg/arch/ppc64/kernel/spu_base.c
@@ -135,8 +135,8 @@ static int __spu_trap_data_map(struct sp
static int __spu_trap_mailbox(struct spu *spu)
{
- wake_up_all(&spu->ibox_wq);
- kill_fasync(&spu->ibox_fasync, SIGIO, POLLIN);
+ if (spu->ibox_callback)
+ spu->ibox_callback(spu);
/* atomically disable SPU mailbox interrupts */
spin_lock(&spu->register_lock);
@@ -171,8 +171,8 @@ static int __spu_trap_tag_group(struct s
static int __spu_trap_spubox(struct spu *spu)
{
- wake_up_all(&spu->wbox_wq);
- kill_fasync(&spu->wbox_fasync, SIGIO, POLLOUT);
+ if (spu->wbox_callback)
+ spu->wbox_callback(spu);
/* atomically disable SPU mailbox interrupts */
spin_lock(&spu->register_lock);
@@ -396,8 +396,6 @@ EXPORT_SYMBOL(spu_alloc);
void spu_free(struct spu *spu)
{
down(&spu_mutex);
- spu->ibox_fasync = NULL;
- spu->wbox_fasync = NULL;
list_add_tail(&spu->list, &spu_list);
up(&spu_mutex);
}
@@ -514,7 +512,6 @@ int spu_run(struct spu *spu)
priv2 = spu->priv2;
/* Let SPU run. */
- spu->mm = current->mm;
eieio();
out_be32(&prob->spu_runcntl_RW, SPU_RUNCNTL_RUNNABLE);
@@ -549,8 +546,6 @@ int spu_run(struct spu *spu)
out_be64(&priv1->tlb_invalidate_entry_W, 0UL);
eieio();
- spu->mm = NULL;
-
/* Check for SPU breakpoint. */
if (unlikely(current->ptrace & PT_PTRACED)) {
status = in_be32(&prob->spu_status_R);
@@ -669,6 +664,9 @@ static int __init create_spu(struct devi
spu->stop_code = 0;
spu->slb_replace = 0;
spu->mm = NULL;
+ spu->ctx = NULL;
+ spu->rq = NULL;
+ spu->pid = 0;
spu->class_0_pending = 0;
spu->flags = 0UL;
spin_lock_init(&spu->register_lock);
@@ -677,11 +675,8 @@ static int __init create_spu(struct devi
out_be64(&spu->priv1->mfc_sr1_RW, 0x33);
init_waitqueue_head(&spu->stop_wq);
- init_waitqueue_head(&spu->wbox_wq);
- init_waitqueue_head(&spu->ibox_wq);
-
- spu->ibox_fasync = NULL;
- spu->wbox_fasync = NULL;
+ spu->ibox_callback = NULL;
+ spu->wbox_callback = NULL;
down(&spu_mutex);
spu->number = number++;
Index: linux-cg/fs/spufs/Makefile
===================================================================
--- linux-cg.orig/fs/spufs/Makefile
+++ linux-cg/fs/spufs/Makefile
@@ -1,5 +1,6 @@
obj-$(CONFIG_SPU_FS) += spufs.o
spufs-y += inode.o file.o context.o switch.o syscalls.o
+spufs-y += sched.o backing_ops.o hw_ops.o
# Rules to build switch.o with the help of SPU tool chain
SPU_CROSS := spu-
Index: linux-cg/fs/spufs/context.c
===================================================================
--- linux-cg.orig/fs/spufs/context.c
+++ linux-cg/fs/spufs/context.c
@@ -33,31 +33,24 @@ struct spu_context *alloc_spu_context(st
ctx = kmalloc(sizeof *ctx, GFP_KERNEL);
if (!ctx)
goto out;
- /* Future enhancement: do not call spu_alloc()
- * here. This step should be deferred until
- * spu_run()!!
- *
- * More work needs to be done to read(),
- * write(), mmap(), etc., so that operations
- * are performed on CSA when the context is
- * not currently being run. In this way we
- * can support arbitrarily large number of
- * entries in /spu, allow state queries, etc.
+ /* Binding to physical processor deferred
+ * until spu_activate().
*/
- ctx->spu = spu_alloc();
- if (!ctx->spu)
- goto out_free;
spu_init_csa(&ctx->csa);
if (!ctx->csa.lscsa) {
- spu_free(ctx->spu);
goto out_free;
}
- init_rwsem(&ctx->backing_sema);
spin_lock_init(&ctx->mmio_lock);
kref_init(&ctx->kref);
init_rwsem(&ctx->state_sema);
+ init_waitqueue_head(&ctx->ibox_wq);
+ init_waitqueue_head(&ctx->wbox_wq);
+ ctx->ibox_fasync = NULL;
+ ctx->wbox_fasync = NULL;
ctx->state = SPU_STATE_SAVED;
ctx->local_store = local_store;
+ ctx->spu = NULL;
+ ctx->ops = &spu_backing_ops;
goto out;
out_free:
kfree(ctx);
@@ -70,8 +63,11 @@ void destroy_spu_context(struct kref *kr
{
struct spu_context *ctx;
ctx = container_of(kref, struct spu_context, kref);
- if (ctx->spu)
- spu_free(ctx->spu);
+ down_write(&ctx->state_sema);
+ spu_deactivate(ctx);
+ ctx->ibox_fasync = NULL;
+ ctx->wbox_fasync = NULL;
+ up_write(&ctx->state_sema);
spu_fini_csa(&ctx->csa);
kfree(ctx);
}
@@ -102,24 +98,52 @@ static void spu_unmap_mappings(struct sp
unmap_mapping_range(ctx->local_store, 0, LS_SIZE, 1);
}
-void spu_acquire_runnable(struct spu_context *ctx)
+int spu_acquire_runnable_nonblock(struct spu_context *ctx)
{
- down_read(&ctx->state_sema);
+ int ret = 0;
+ if (!down_read_trylock(&ctx->state_sema))
+ return -EAGAIN;
if (ctx->state == SPU_STATE_RUNNABLE
|| ctx->state == SPU_STATE_LOCKED)
- return;
-
+ return 0;
up_read(&ctx->state_sema);
- down_write(&ctx->state_sema);
+ if (!down_write_trylock(&ctx->state_sema))
+ return -EAGAIN;
if (ctx->state == SPU_STATE_SAVED) {
spu_unmap_mappings(ctx);
- spu_restore(&ctx->csa, ctx->spu);
+ ret = spu_activate(ctx, 0);
ctx->state = SPU_STATE_RUNNABLE;
}
+ downgrade_write(&ctx->state_sema);
+ if (ret)
+ /* Release here, to simplify calling code. */
+ spu_release(ctx);
+ return ret;
+}
+
+int spu_acquire_runnable(struct spu_context *ctx)
+{
+ int ret = 0;
+ down_read(&ctx->state_sema);
+ if (ctx->state == SPU_STATE_RUNNABLE
+ || ctx->state == SPU_STATE_LOCKED)
+ return 0;
+ up_read(&ctx->state_sema);
+
+ down_write(&ctx->state_sema);
+ if (ctx->state == SPU_STATE_SAVED) {
+ spu_unmap_mappings(ctx);
+ ret = spu_activate(ctx, 0);
+ ctx->state = SPU_STATE_RUNNABLE;
+ }
downgrade_write(&ctx->state_sema);
+ if (ret)
+ /* Release here, to simplify calling code. */
+ spu_release(ctx);
+ return ret;
}
void spu_acquire_saved(struct spu_context *ctx)
@@ -135,7 +159,7 @@ void spu_acquire_saved(struct spu_contex
if (ctx->state == SPU_STATE_RUNNABLE) {
spu_unmap_mappings(ctx);
- spu_save(&ctx->csa, ctx->spu);
+ spu_deactivate(ctx);
ctx->state = SPU_STATE_SAVED;
}
Index: linux-cg/fs/spufs/file.c
===================================================================
--- linux-cg.orig/fs/spufs/file.c
+++ linux-cg/fs/spufs/file.c
@@ -51,16 +51,10 @@ spufs_mem_read(struct file *file, char _
int ret;
spu_acquire(ctx);
- down_read(&ctx->backing_sema);
-
- if (ctx->state == SPU_STATE_SAVED)
- local_store = ctx->csa.lscsa->ls;
- else
- local_store = ctx->spu->local_store;
+ local_store = ctx->ops->get_ls(ctx);
ret = simple_read_from_buffer(buffer, size, pos, local_store, LS_SIZE);
- up_read(&ctx->backing_sema);
spu_release(ctx);
return ret;
}
@@ -79,17 +73,11 @@ spufs_mem_write(struct file *file, const
*pos += size;
spu_acquire(ctx);
- down_read(&ctx->backing_sema);
-
- if (ctx->state == SPU_STATE_SAVED)
- local_store = ctx->csa.lscsa->ls;
- else
- local_store = ctx->spu->local_store;
+ local_store = ctx->ops->get_ls(ctx);
ret = copy_from_user(local_store + *pos - size,
buffer, size) ? -EFAULT : size;
- up_read(&ctx->backing_sema);
spu_release(ctx);
return ret;
}
@@ -131,6 +119,7 @@ spufs_mem_mmap(struct file *file, struct
{
#ifndef CONFIG_SPARSEMEM
struct spu_context *ctx = file->private_data;
+ int ret;
#endif
if (!(vma->vm_flags & VM_SHARED))
@@ -142,7 +131,9 @@ spufs_mem_mmap(struct file *file, struct
| _PAGE_NO_CACHE);
#ifndef CONFIG_SPARSEMEM
- spu_acquire_runnable(ctx);
+ ret = spu_acquire_runnable(ctx);
+ if (ret != 0)
+ return ret;
/*
* This will work for actual SPUs, but not for vmalloc memory:
@@ -245,25 +236,18 @@ static ssize_t spufs_mbox_read(struct fi
size_t len, loff_t *pos)
{
struct spu_context *ctx = file->private_data;
- struct spu_problem __iomem *prob;
- u32 mbox_stat;
u32 mbox_data;
+ int ret;
if (len < 4)
return -EINVAL;
- spu_acquire_runnable(ctx);
+ spu_acquire(ctx);
+ ret = ctx->ops->mbox_read(ctx, &mbox_data);
+ spu_release(ctx);
- prob = ctx->spu->problem;
- mbox_stat = in_be32(&prob->mb_stat_R);
- if (!(mbox_stat & 0x0000ff)) {
- spu_release(ctx);
+ if (!ret)
return -EAGAIN;
- }
-
- mbox_data = in_be32(&prob->pu_mb_R);
-
- spu_release(ctx);
if (copy_to_user(buf, &mbox_data, sizeof mbox_data))
return -EFAULT;
@@ -285,8 +269,10 @@ static ssize_t spufs_mbox_stat_read(stru
if (len < 4)
return -EINVAL;
- spu_acquire_runnable(ctx);
- mbox_stat = in_be32(&ctx->spu->problem->mb_stat_R) & 0xff;
+ spu_acquire(ctx);
+
+ mbox_stat = ctx->ops->mbox_stat_read(ctx) & 0xff;
+
spu_release(ctx);
if (copy_to_user(buf, &mbox_stat, sizeof mbox_stat))
@@ -300,39 +286,54 @@ static struct file_operations spufs_mbox
.read = spufs_mbox_stat_read,
};
+/*
+ * spufs_wait
+ * Same as wait_event_interruptible(), except that here
+ * we need to call spu_release(ctx) before sleeping, and
+ * then spu_acquire(ctx) when awoken.
+ */
+
+#define spufs_wait(wq, condition) \
+({ \
+ int __ret = 0; \
+ DEFINE_WAIT(__wait); \
+ for (;;) { \
+ prepare_to_wait(&(wq), &__wait, TASK_INTERRUPTIBLE); \
+ if (condition) \
+ break; \
+ if (!signal_pending(current)) { \
+ spu_release(ctx); \
+ schedule(); \
+ spu_acquire(ctx); \
+ continue; \
+ } \
+ __ret = -ERESTARTSYS; \
+ break; \
+ } \
+ finish_wait(&(wq), &__wait); \
+ __ret; \
+})
+
/* low-level ibox access function */
-size_t spu_ibox_read(struct spu *spu, u32 *data)
+size_t spu_ibox_read(struct spu_context *ctx, u32 *data)
{
- int ret;
-
- spin_lock_irq(&spu->register_lock);
-
- if (in_be32(&spu->problem->mb_stat_R) & 0xff0000) {
- /* read the first available word */
- *data = in_be64(&spu->priv2->puint_mb_R);
- ret = 4;
- } else {
- /* make sure we get woken up by the interrupt */
- out_be64(&spu->priv1->int_mask_class2_RW,
- in_be64(&spu->priv1->int_mask_class2_RW) | 0x1);
- ret = 0;
- }
-
- spin_unlock_irq(&spu->register_lock);
- return ret;
+ return ctx->ops->ibox_read(ctx, data);
}
-EXPORT_SYMBOL(spu_ibox_read);
static int spufs_ibox_fasync(int fd, struct file *file, int on)
{
struct spu_context *ctx = file->private_data;
- int ret;
- spu_acquire_runnable(ctx);
- ret = fasync_helper(fd, file, on, &ctx->spu->ibox_fasync);
- spu_release(ctx);
+ return fasync_helper(fd, file, on, &ctx->ibox_fasync);
+}
- return ret;
+/* interrupt-level ibox callback function. */
+void spufs_ibox_callback(struct spu *spu)
+{
+ struct spu_context *ctx = spu->ctx;
+
+ wake_up_all(&ctx->ibox_wq);
+ kill_fasync(&ctx->ibox_fasync, SIGIO, POLLIN);
}
static ssize_t spufs_ibox_read(struct file *file, char __user *buf,
@@ -345,15 +346,14 @@ static ssize_t spufs_ibox_read(struct fi
if (len < 4)
return -EINVAL;
- spu_acquire_runnable(ctx);
+ spu_acquire(ctx);
ret = 0;
if (file->f_flags & O_NONBLOCK) {
- if (!spu_ibox_read(ctx->spu, &ibox_data))
+ if (!spu_ibox_read(ctx, &ibox_data))
ret = -EAGAIN;
} else {
- ret = wait_event_interruptible(ctx->spu->ibox_wq,
- spu_ibox_read(ctx->spu, &ibox_data));
+ ret = spufs_wait(ctx->ibox_wq, spu_ibox_read(ctx, &ibox_data));
}
spu_release(ctx);
@@ -371,19 +371,17 @@ static ssize_t spufs_ibox_read(struct fi
static unsigned int spufs_ibox_poll(struct file *file, poll_table *wait)
{
struct spu_context *ctx = file->private_data;
- struct spu_problem __iomem *prob;
u32 mbox_stat;
unsigned int mask;
- spu_acquire_runnable(ctx);
-
- prob = ctx->spu->problem;
- mbox_stat = in_be32(&prob->mb_stat_R);
+ spu_acquire(ctx);
- poll_wait(file, &ctx->spu->ibox_wq, wait);
+ mbox_stat = ctx->ops->mbox_stat_read(ctx);
spu_release(ctx);
+ poll_wait(file, &ctx->ibox_wq, wait);
+
mask = 0;
if (mbox_stat & 0xff0000)
mask |= POLLIN | POLLRDNORM;
@@ -407,8 +405,8 @@ static ssize_t spufs_ibox_stat_read(stru
if (len < 4)
return -EINVAL;
- spu_acquire_runnable(ctx);
- ibox_stat = (in_be32(&ctx->spu->problem->mb_stat_R) >> 16) & 0xff;
+ spu_acquire(ctx);
+ ibox_stat = (ctx->ops->mbox_stat_read(ctx) >> 16) & 0xff;
spu_release(ctx);
if (copy_to_user(buf, &ibox_stat, sizeof ibox_stat))
@@ -423,41 +421,30 @@ static struct file_operations spufs_ibox
};
/* low-level mailbox write */
-size_t spu_wbox_write(struct spu *spu, u32 data)
+size_t spu_wbox_write(struct spu_context *ctx, u32 data)
{
- int ret;
-
- spin_lock_irq(&spu->register_lock);
-
- if (in_be32(&spu->problem->mb_stat_R) & 0x00ff00) {
- /* we have space to write wbox_data to */
- out_be32(&spu->problem->spu_mb_W, data);
- ret = 4;
- } else {
- /* make sure we get woken up by the interrupt when space
- becomes available */
- out_be64(&spu->priv1->int_mask_class2_RW,
- in_be64(&spu->priv1->int_mask_class2_RW) | 0x10);
- ret = 0;
- }
-
- spin_unlock_irq(&spu->register_lock);
- return ret;
+ return ctx->ops->wbox_write(ctx, data);
}
-EXPORT_SYMBOL(spu_wbox_write);
static int spufs_wbox_fasync(int fd, struct file *file, int on)
{
struct spu_context *ctx = file->private_data;
int ret;
- spu_acquire_runnable(ctx);
- ret = fasync_helper(fd, file, on, &ctx->spu->wbox_fasync);
- spu_release(ctx);
+ ret = fasync_helper(fd, file, on, &ctx->wbox_fasync);
return ret;
}
+/* interrupt-level wbox callback function. */
+void spufs_wbox_callback(struct spu *spu)
+{
+ struct spu_context *ctx = spu->ctx;
+
+ wake_up_all(&ctx->wbox_wq);
+ kill_fasync(&ctx->wbox_fasync, SIGIO, POLLOUT);
+}
+
static ssize_t spufs_wbox_write(struct file *file, const char __user *buf,
size_t len, loff_t *pos)
{
@@ -471,15 +458,14 @@ static ssize_t spufs_wbox_write(struct f
if (copy_from_user(&wbox_data, buf, sizeof wbox_data))
return -EFAULT;
- spu_acquire_runnable(ctx);
+ spu_acquire(ctx);
ret = 0;
if (file->f_flags & O_NONBLOCK) {
- if (!spu_wbox_write(ctx->spu, wbox_data))
+ if (!spu_wbox_write(ctx, wbox_data))
ret = -EAGAIN;
} else {
- ret = wait_event_interruptible(ctx->spu->wbox_wq,
- spu_wbox_write(ctx->spu, wbox_data));
+ ret = spufs_wait(ctx->wbox_wq, spu_wbox_write(ctx, wbox_data));
}
spu_release(ctx);
@@ -490,19 +476,15 @@ static ssize_t spufs_wbox_write(struct f
static unsigned int spufs_wbox_poll(struct file *file, poll_table *wait)
{
struct spu_context *ctx = file->private_data;
- struct spu_problem __iomem *prob;
u32 mbox_stat;
unsigned int mask;
- spu_acquire_runnable(ctx);
-
- prob = ctx->spu->problem;
- mbox_stat = in_be32(&prob->mb_stat_R);
-
- poll_wait(file, &ctx->spu->wbox_wq, wait);
-
+ spu_acquire(ctx);
+ mbox_stat = ctx->ops->mbox_stat_read(ctx);
spu_release(ctx);
+ poll_wait(file, &ctx->wbox_wq, wait);
+
mask = 0;
if (mbox_stat & 0x00ff00)
mask = POLLOUT | POLLWRNORM;
@@ -526,8 +508,8 @@ static ssize_t spufs_wbox_stat_read(stru
if (len < 4)
return -EINVAL;
- spu_acquire_runnable(ctx);
- wbox_stat = (in_be32(&ctx->spu->problem->mb_stat_R) >> 8) & 0xff;
+ spu_acquire(ctx);
+ wbox_stat = (ctx->ops->mbox_stat_read(ctx) >> 8) & 0xff;
spu_release(ctx);
if (copy_to_user(buf, &wbox_stat, sizeof wbox_stat))
@@ -544,31 +526,25 @@ static struct file_operations spufs_wbox
long spufs_run_spu(struct file *file, struct spu_context *ctx,
u32 *npc, u32 *status)
{
- struct spu_problem __iomem *prob;
int ret;
if (file->f_flags & O_NONBLOCK) {
- ret = -EAGAIN;
- if (!down_write_trylock(&ctx->backing_sema))
- goto out;
+ ret = spu_acquire_runnable_nonblock(ctx);
} else {
- down_write(&ctx->backing_sema);
+ ret = spu_acquire_runnable(ctx);
}
+ if (ret != 0)
+ return ret;
- spu_acquire_runnable(ctx);
-
- prob = ctx->spu->problem;
- out_be32(&prob->spu_npc_RW, *npc);
+ ctx->ops->npc_write(ctx, *npc);
ret = spu_run(ctx->spu);
- *status = in_be32(&prob->spu_status_R);
- *npc = in_be32(&prob->spu_npc_RW);
+ *status = ctx->ops->status_read(ctx);
+ *npc = ctx->ops->npc_read(ctx);
spu_release(ctx);
- up_write(&ctx->backing_sema);
-
-out:
+ spu_yield(ctx);
return ret;
}
@@ -591,21 +567,19 @@ static ssize_t spufs_run_read(struct fil
goto out;
if (file->f_flags & O_NONBLOCK) {
- ret = -EAGAIN;
- if (!down_write_trylock(&ctx->backing_sema))
- goto out;
+ ret = spu_acquire_runnable_nonblock(ctx);
} else {
- down_write(&ctx->backing_sema);
+ ret = spu_acquire_runnable(ctx);
}
-
- spu_acquire_runnable(ctx);
+ if (ret != 0)
+ return ret;
ret = spu_run(ctx->spu);
if (ret == -EAGAIN)
ret = 0;
- arg.status = in_be32(&ctx->spu->problem->spu_status_R);
- arg.npc = in_be32(&ctx->spu->problem->spu_npc_RW);
+ arg.status = ctx->ops->status_read(ctx);
+ arg.npc = ctx->ops->npc_read(ctx);
if ((arg.status & 0xffff0002) == 0x21000002) {
/* library callout */
@@ -616,8 +590,6 @@ static ssize_t spufs_run_read(struct fil
}
spu_release(ctx);
- up_write(&ctx->backing_sema);
-
if (ret)
goto out;
@@ -662,15 +634,13 @@ static ssize_t spufs_signal1_read(struct
size_t len, loff_t *pos)
{
struct spu_context *ctx = file->private_data;
- struct spu_problem *prob;
u32 data;
if (len < 4)
return -EINVAL;
- spu_acquire_runnable(ctx);
- prob = ctx->spu->problem;
- data = in_be32(&prob->signal_notify1);
+ spu_acquire(ctx);
+ data = ctx->ops->signal1_read(ctx);
spu_release(ctx);
if (copy_to_user(buf, &data, 4))
@@ -683,7 +653,6 @@ static ssize_t spufs_signal1_write(struc
size_t len, loff_t *pos)
{
struct spu_context *ctx;
- struct spu_problem *prob;
u32 data;
ctx = file->private_data;
@@ -694,9 +663,8 @@ static ssize_t spufs_signal1_write(struc
if (copy_from_user(&data, buf, 4))
return -EFAULT;
- spu_acquire_runnable(ctx);
- prob = ctx->spu->problem;
- out_be32(&prob->signal_notify1, data);
+ spu_acquire(ctx);
+ ctx->ops->signal1_write(ctx, data);
spu_release(ctx);
return 4;
@@ -712,7 +680,6 @@ static ssize_t spufs_signal2_read(struct
size_t len, loff_t *pos)
{
struct spu_context *ctx;
- struct spu_problem *prob;
u32 data;
ctx = file->private_data;
@@ -720,9 +687,8 @@ static ssize_t spufs_signal2_read(struct
if (len < 4)
return -EINVAL;
- spu_acquire_runnable(ctx);
- prob = ctx->spu->problem;
- data = in_be32(&prob->signal_notify2);
+ spu_acquire(ctx);
+ data = ctx->ops->signal2_read(ctx);
spu_release(ctx);
if (copy_to_user(buf, &data, 4))
@@ -735,7 +701,6 @@ static ssize_t spufs_signal2_write(struc
size_t len, loff_t *pos)
{
struct spu_context *ctx;
- struct spu_problem *prob;
u32 data;
ctx = file->private_data;
@@ -746,9 +711,8 @@ static ssize_t spufs_signal2_write(struc
if (copy_from_user(&data, buf, 4))
return -EFAULT;
- spu_acquire_runnable(ctx);
- prob = ctx->spu->problem;
- out_be32(&prob->signal_notify2, data);
+ spu_acquire(ctx);
+ ctx->ops->signal2_write(ctx, data);
spu_release(ctx);
return 4;
@@ -763,19 +727,9 @@ static struct file_operations spufs_sign
static void spufs_signal1_type_set(void *data, u64 val)
{
struct spu_context *ctx = data;
- struct spu_priv2 *priv2;
- u64 tmp;
- spu_acquire_runnable(ctx);
- priv2 = ctx->spu->priv2;
- spin_lock_irq(&ctx->spu->register_lock);
- tmp = in_be64(&priv2->spu_cfg_RW);
- if (val)
- tmp |= 1;
- else
- tmp &= ~1;
- out_be64(&priv2->spu_cfg_RW, tmp);
- spin_unlock_irq(&ctx->spu->register_lock);
+ spu_acquire(ctx);
+ ctx->ops->signal1_type_set(ctx, val);
spu_release(ctx);
}
@@ -784,8 +738,8 @@ static u64 spufs_signal1_type_get(void *
struct spu_context *ctx = data;
u64 ret;
- spu_acquire_runnable(ctx);
- ret = ((in_be64(&ctx->spu->priv2->spu_cfg_RW) & 1) != 0);
+ spu_acquire(ctx);
+ ret = ctx->ops->signal1_type_get(ctx);
spu_release(ctx);
return ret;
@@ -796,19 +750,9 @@ DEFINE_SIMPLE_ATTRIBUTE(spufs_signal1_ty
static void spufs_signal2_type_set(void *data, u64 val)
{
struct spu_context *ctx = data;
- struct spu_priv2 *priv2;
- u64 tmp;
- spu_acquire_runnable(ctx);
- priv2 = ctx->spu->priv2;
- spin_lock_irq(&ctx->spu->register_lock);
- tmp = in_be64(&priv2->spu_cfg_RW);
- if (val)
- tmp |= 2;
- else
- tmp &= ~2;
- out_be64(&priv2->spu_cfg_RW, tmp);
- spin_unlock_irq(&ctx->spu->register_lock);
+ spu_acquire(ctx);
+ ctx->ops->signal2_type_set(ctx, val);
spu_release(ctx);
}
@@ -817,8 +761,8 @@ static u64 spufs_signal2_type_get(void *
struct spu_context *ctx = data;
u64 ret;
- spu_acquire_runnable(ctx);
- ret = ((in_be64(&ctx->spu->priv2->spu_cfg_RW) & 2) != 0);
+ spu_acquire(ctx);
+ ret = ctx->ops->signal2_type_get(ctx);
spu_release(ctx);
return ret;
@@ -829,8 +773,8 @@ DEFINE_SIMPLE_ATTRIBUTE(spufs_signal2_ty
static void spufs_npc_set(void *data, u64 val)
{
struct spu_context *ctx = data;
- spu_acquire_runnable(ctx);
- out_be32(&ctx->spu->problem->spu_npc_RW, val);
+ spu_acquire(ctx);
+ ctx->ops->npc_write(ctx, val);
spu_release(ctx);
}
@@ -838,8 +782,8 @@ static u64 spufs_npc_get(void *data)
{
struct spu_context *ctx = data;
u64 ret;
- spu_acquire_runnable(ctx);
- ret = in_be32(&ctx->spu->problem->spu_npc_RW);
+ spu_acquire(ctx);
+ ret = ctx->ops->npc_read(ctx);
spu_release(ctx);
return ret;
}
Index: linux-cg/fs/spufs/inode.c
===================================================================
--- linux-cg.orig/fs/spufs/inode.c
+++ linux-cg/fs/spufs/inode.c
@@ -481,6 +481,10 @@ static int spufs_init(void)
if (!spufs_inode_cache)
goto out;
+ if (spu_sched_init() != 0) {
+ kmem_cache_destroy(spufs_inode_cache);
+ goto out;
+ }
ret = register_filesystem(&spufs_type);
if (ret)
goto out_cache;
@@ -499,6 +503,7 @@ module_init(spufs_init);
static void spufs_exit(void)
{
+ spu_sched_exit();
unregister_spu_syscalls(&spufs_calls);
unregister_filesystem(&spufs_type);
kmem_cache_destroy(spufs_inode_cache);
Index: linux-cg/fs/spufs/sched.c
===================================================================
--- /dev/null
+++ linux-cg/fs/spufs/sched.c
@@ -0,0 +1,409 @@
+/* sched.c - SPU scheduler.
+ *
+ * Copyright (C) IBM 2005
+ * Author: Mark Nutter <mnutter@us.ibm.com>
+ *
+ * SPU scheduler, based on Linux thread priority. For now use
+ * a simple "cooperative" yield model with no preemption. SPU
+ * scheduling will eventually be preemptive: When a thread with
+ * a higher static priority gets ready to run, then an active SPU
+ * context will be preempted and returned to the waitq.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2, or (at your option)
+ * any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+ */
+
+#define DEBUG 1
+#include <linux/config.h>
+#include <linux/module.h>
+#include <linux/errno.h>
+#include <linux/sched.h>
+#include <linux/kernel.h>
+#include <linux/mm.h>
+#include <linux/completion.h>
+#include <linux/vmalloc.h>
+#include <linux/smp.h>
+#include <linux/smp_lock.h>
+#include <linux/stddef.h>
+#include <linux/unistd.h>
+
+#include <asm/io.h>
+#include <asm/mmu_context.h>
+#include <asm/spu.h>
+#include <asm/spu_csa.h>
+#include "spufs.h"
+
+#define SPU_BITMAP_SIZE (((MAX_PRIO+BITS_PER_LONG)/BITS_PER_LONG)+1)
+struct spu_prio_array {
+ atomic_t nr_blocked;
+ unsigned long bitmap[SPU_BITMAP_SIZE];
+ wait_queue_head_t waitq[MAX_PRIO];
+};
+
+/* spu_runqueue - This is the main runqueue data structure for SPUs. */
+struct spu_runqueue {
+ struct semaphore sem;
+ unsigned long nr_active;
+ unsigned long nr_idle;
+ unsigned long nr_switches;
+ struct list_head active_list;
+ struct list_head idle_list;
+ struct spu_prio_array prio;
+};
+
+static struct spu_runqueue *spu_runqueues = NULL;
+
+static inline struct spu_runqueue *spu_rq(void)
+{
+ /* Future: make this a per-NODE array,
+ * and use cpu_to_node(smp_processor_id())
+ */
+ return spu_runqueues;
+}
+
+static inline struct spu *del_idle(struct spu_runqueue *rq)
+{
+ struct spu *spu;
+
+ BUG_ON(rq->nr_idle <= 0);
+ BUG_ON(list_empty(&rq->idle_list));
+ /* Future: Move SPU out of low-power SRI state. */
+ spu = list_entry(rq->idle_list.next, struct spu, sched_list);
+ list_del_init(&spu->sched_list);
+ rq->nr_idle--;
+ return spu;
+}
+
+static inline void del_active(struct spu_runqueue *rq, struct spu *spu)
+{
+ BUG_ON(rq->nr_active <= 0);
+ BUG_ON(list_empty(&rq->active_list));
+ list_del_init(&spu->sched_list);
+ rq->nr_active--;
+}
+
+static inline void add_idle(struct spu_runqueue *rq, struct spu *spu)
+{
+ /* Future: Put SPU into low-power SRI state. */
+ list_add_tail(&spu->sched_list, &rq->idle_list);
+ rq->nr_idle++;
+}
+
+static inline void add_active(struct spu_runqueue *rq, struct spu *spu)
+{
+ rq->nr_active++;
+ rq->nr_switches++;
+ list_add_tail(&spu->sched_list, &rq->active_list);
+}
+
+static void prio_wakeup(struct spu_runqueue *rq)
+{
+ if (atomic_read(&rq->prio.nr_blocked) && rq->nr_idle) {
+ int best = sched_find_first_bit(rq->prio.bitmap);
+ if (best < MAX_PRIO) {
+ wait_queue_head_t *wq = &rq->prio.waitq[best];
+ wake_up_interruptible_nr(wq, 1);
+ }
+ }
+}
+
+static void prio_wait(struct spu_runqueue *rq, u64 flags)
+{
+ int prio = current->prio;
+ wait_queue_head_t *wq = &rq->prio.waitq[prio];
+ DEFINE_WAIT(wait);
+
+ __set_bit(prio, rq->prio.bitmap);
+ atomic_inc(&rq->prio.nr_blocked);
+ prepare_to_wait_exclusive(wq, &wait, TASK_INTERRUPTIBLE);
+ if (!signal_pending(current)) {
+ up(&rq->sem);
+ pr_debug("%s: pid=%d prio=%d\n", __FUNCTION__,
+ current->pid, current->prio);
+ schedule();
+ down(&rq->sem);
+ }
+ finish_wait(wq, &wait);
+ atomic_dec(&rq->prio.nr_blocked);
+ if (!waitqueue_active(wq))
+ __clear_bit(prio, rq->prio.bitmap);
+}
+
+static inline int is_best_prio(struct spu_runqueue *rq)
+{
+ int best_prio;
+
+ best_prio = sched_find_first_bit(rq->prio.bitmap);
+ return (current->prio < best_prio) ? 1 : 0;
+}
+
+static inline void bind_context(struct spu *spu, struct spu_context *ctx)
+{
+ pr_debug("%s: pid=%d SPU=%d\n", __FUNCTION__, current->pid,
+ spu->number);
+ spu->ctx = ctx;
+ spu->flags = 0;
+ ctx->spu = spu;
+ ctx->ops = &spu_hw_ops;
+ spu->pid = current->pid;
+ spu->prio = current->prio;
+ spu->mm = get_task_mm(current);
+ spu->ibox_callback = spufs_ibox_callback;
+ spu->wbox_callback = spufs_wbox_callback;
+ mb();
+ spu_restore(&ctx->csa, spu);
+}
+
+static inline void unbind_context(struct spu *spu, struct spu_context *ctx)
+{
+ pr_debug("%s: unbind pid=%d SPU=%d\n", __FUNCTION__,
+ spu->pid, spu->number);
+ spu_save(&ctx->csa, spu);
+ ctx->state = SPU_STATE_SAVED;
+ spu->ibox_callback = NULL;
+ spu->wbox_callback = NULL;
+ mmput(spu->mm);
+ spu->mm = NULL;
+ spu->pid = 0;
+ spu->prio = MAX_PRIO;
+ ctx->ops = &spu_backing_ops;
+ ctx->spu = NULL;
+ spu->ctx = NULL;
+}
+
+static struct spu *preempt_active(struct spu_runqueue *rq)
+{
+ struct list_head *p;
+ struct spu_context *ctx;
+ struct spu *spu;
+
+ /* Future: implement real preemption. For now just
+ * boot a lower priority ctx that is in "detached"
+ * state, i.e. on a processor but not currently in
+ * spu_run().
+ */
+ list_for_each(p, &rq->active_list) {
+ spu = list_entry(p, struct spu, sched_list);
+ if (current->prio < spu->prio) {
+ ctx = spu->ctx;
+ if (down_write_trylock(&ctx->state_sema)) {
+ if (ctx->state != SPU_STATE_RUNNABLE) {
+ up_write(&ctx->state_sema);
+ continue;
+ }
+ pr_debug("%s: booting pid=%d from SPU %d\n",
+ __FUNCTION__, spu->pid, spu->number);
+ del_active(rq, spu);
+ up(&rq->sem);
+ unbind_context(spu, ctx);
+ up_write(&ctx->state_sema);
+ return spu;
+ }
+ }
+ }
+ return NULL;
+}
+
+static struct spu *get_idle_spu(u64 flags)
+{
+ struct spu_runqueue *rq;
+ struct spu *spu = NULL;
+
+ rq = spu_rq();
+ down(&rq->sem);
+ for (;;) {
+ if (rq->nr_idle > 0) {
+ if (is_best_prio(rq)) {
+ /* Fall through. */
+ spu = del_idle(rq);
+ break;
+ } else {
+ prio_wakeup(rq);
+ up(&rq->sem);
+ yield();
+ if (signal_pending(current)) {
+ return NULL;
+ }
+ rq = spu_rq();
+ down(&rq->sem);
+ continue;
+ }
+ } else {
+ if (is_best_prio(rq)) {
+ if ((spu = preempt_active(rq)) != NULL)
+ return spu;
+ }
+ prio_wait(rq, flags);
+ if (signal_pending(current)) {
+ prio_wakeup(rq);
+ spu = NULL;
+ break;
+ }
+ continue;
+ }
+ }
+ up(&rq->sem);
+ return spu;
+}
+
+static void put_idle_spu(struct spu *spu)
+{
+ struct spu_runqueue *rq = spu->rq;
+
+ down(&rq->sem);
+ add_idle(rq, spu);
+ prio_wakeup(rq);
+ up(&rq->sem);
+}
+
+static int get_active_spu(struct spu *spu)
+{
+ struct spu_runqueue *rq = spu->rq;
+ struct list_head *p;
+ struct spu *tmp;
+ int rc = 0;
+
+ down(&rq->sem);
+ list_for_each(p, &rq->active_list) {
+ tmp = list_entry(p, struct spu, sched_list);
+ if (tmp == spu) {
+ del_active(rq, spu);
+ rc = 1;
+ break;
+ }
+ }
+ up(&rq->sem);
+ return rc;
+}
+
+static void put_active_spu(struct spu *spu)
+{
+ struct spu_runqueue *rq = spu->rq;
+
+ down(&rq->sem);
+ add_active(rq, spu);
+ up(&rq->sem);
+}
+
+/* Lock order:
+ * spu_activate() & spu_deactivate() require the
+ * caller to have down_write(&ctx->state_sema).
+ *
+ * The rq->sem is breifly held (inside or outside a
+ * given ctx lock) for list management, but is never
+ * held during save/restore.
+ */
+
+int spu_activate(struct spu_context *ctx, u64 flags)
+{
+ struct spu *spu;
+
+ if (ctx->spu)
+ return 0;
+ spu = get_idle_spu(flags);
+ if (!spu)
+ return (signal_pending(current)) ? -ERESTARTSYS : -EAGAIN;
+ bind_context(spu, ctx);
+ put_active_spu(spu);
+ return 0;
+}
+
+void spu_deactivate(struct spu_context *ctx)
+{
+ struct spu *spu;
+ int needs_idle;
+
+ spu = ctx->spu;
+ if (!spu)
+ return;
+ needs_idle = get_active_spu(spu);
+ unbind_context(spu, ctx);
+ if (needs_idle)
+ put_idle_spu(spu);
+}
+
+void spu_yield(struct spu_context *ctx)
+{
+ struct spu *spu;
+
+ if (!down_write_trylock(&ctx->state_sema))
+ return;
+ spu = ctx->spu;
+ if ((ctx->state == SPU_STATE_RUNNABLE) &&
+ (sched_find_first_bit(spu->rq->prio.bitmap) <= current->prio)) {
+ pr_debug("%s: yielding SPU %d\n", __FUNCTION__, spu->number);
+ spu_deactivate(ctx);
+ ctx->state = SPU_STATE_SAVED;
+ }
+ up_write(&ctx->state_sema);
+}
+
+int __init spu_sched_init(void)
+{
+ struct spu_runqueue *rq;
+ struct spu *spu;
+ int i;
+
+ rq = spu_runqueues = kmalloc(sizeof(struct spu_runqueue), GFP_KERNEL);
+ if (!rq) {
+ printk(KERN_WARNING "%s: Unable to allocate runqueues.\n",
+ __FUNCTION__);
+ return 1;
+ }
+ memset(rq, 0, sizeof(struct spu_runqueue));
+ init_MUTEX(&rq->sem);
+ INIT_LIST_HEAD(&rq->active_list);
+ INIT_LIST_HEAD(&rq->idle_list);
+ rq->nr_active = 0;
+ rq->nr_idle = 0;
+ rq->nr_switches = 0;
+ atomic_set(&rq->prio.nr_blocked, 0);
+ for (i = 0; i < MAX_PRIO; i++) {
+ init_waitqueue_head(&rq->prio.waitq[i]);
+ __clear_bit(i, rq->prio.bitmap);
+ }
+ __set_bit(MAX_PRIO, rq->prio.bitmap);
+ for (;;) {
+ spu = spu_alloc();
+ if (!spu)
+ break;
+ pr_debug("%s: adding SPU[%d]\n", __FUNCTION__, spu->number);
+ add_idle(rq, spu);
+ spu->rq = rq;
+ }
+ if (!rq->nr_idle) {
+ printk(KERN_WARNING "%s: No available SPUs.\n", __FUNCTION__);
+ kfree(rq);
+ return 1;
+ }
+ return 0;
+}
+
+void __exit spu_sched_exit(void)
+{
+ struct spu_runqueue *rq = spu_rq();
+ struct spu *spu;
+
+ if (!rq) {
+ printk(KERN_WARNING "%s: no runqueues!\n", __FUNCTION__);
+ return;
+ }
+ while (rq->nr_idle > 0) {
+ spu = del_idle(rq);
+ if (!spu)
+ break;
+ spu_free(spu);
+ }
+ kfree(rq);
+}
Index: linux-cg/fs/spufs/spufs.h
===================================================================
--- linux-cg.orig/fs/spufs/spufs.h
+++ linux-cg/fs/spufs/spufs.h
@@ -35,10 +35,11 @@ enum {
SPUFS_MAGIC = 0x23c9b64e,
};
+struct spu_context_ops;
+
struct spu_context {
struct spu *spu; /* pointer to a physical SPU */
struct spu_state csa; /* SPU context save area. */
- struct rw_semaphore backing_sema; /* protects the above */
spinlock_t mmio_lock; /* protects mmio access */
struct address_space *local_store;/* local store backing store */
@@ -46,8 +47,36 @@ struct spu_context {
struct rw_semaphore state_sema;
struct kref kref;
+ wait_queue_head_t ibox_wq;
+ wait_queue_head_t wbox_wq;
+ struct fasync_struct *ibox_fasync;
+ struct fasync_struct *wbox_fasync;
+ struct spu_context_ops *ops;
+};
+
+/* SPU context query/set operations. */
+struct spu_context_ops {
+ int (*mbox_read) (struct spu_context * ctx, u32 * data);
+ u32(*mbox_stat_read) (struct spu_context * ctx);
+ int (*ibox_read) (struct spu_context * ctx, u32 * data);
+ int (*wbox_write) (struct spu_context * ctx, u32 data);
+ u32(*signal1_read) (struct spu_context * ctx);
+ void (*signal1_write) (struct spu_context * ctx, u32 data);
+ u32(*signal2_read) (struct spu_context * ctx);
+ void (*signal2_write) (struct spu_context * ctx, u32 data);
+ void (*signal1_type_set) (struct spu_context * ctx, u64 val);
+ u64(*signal1_type_get) (struct spu_context * ctx);
+ void (*signal2_type_set) (struct spu_context * ctx, u64 val);
+ u64(*signal2_type_get) (struct spu_context * ctx);
+ u32(*npc_read) (struct spu_context * ctx);
+ void (*npc_write) (struct spu_context * ctx, u32 data);
+ u32(*status_read) (struct spu_context * ctx);
+ char*(*get_ls) (struct spu_context * ctx);
};
+extern struct spu_context_ops spu_hw_ops;
+extern struct spu_context_ops spu_backing_ops;
+
struct spufs_inode_info {
struct spu_context *i_ctx;
struct inode vfs_inode;
@@ -71,7 +100,21 @@ int put_spu_context(struct spu_context *
void spu_acquire(struct spu_context *ctx);
void spu_release(struct spu_context *ctx);
-void spu_acquire_runnable(struct spu_context *ctx);
+int spu_acquire_runnable(struct spu_context *ctx);
+int spu_acquire_runnable_nonblock(struct spu_context *ctx);
void spu_acquire_saved(struct spu_context *ctx);
+int spu_activate(struct spu_context *ctx, u64 flags);
+void spu_deactivate(struct spu_context *ctx);
+void spu_yield(struct spu_context *ctx);
+int __init spu_sched_init(void);
+void __exit spu_sched_exit(void);
+
+size_t spu_wbox_write(struct spu_context *ctx, u32 data);
+size_t spu_ibox_read(struct spu_context *ctx, u32 *data);
+
+/* irq callback funcs. */
+void spufs_ibox_callback(struct spu *spu);
+void spufs_wbox_callback(struct spu *spu);
+
#endif
Index: linux-cg/fs/spufs/switch.c
===================================================================
--- linux-cg.orig/fs/spufs/switch.c
+++ linux-cg/fs/spufs/switch.c
@@ -650,7 +650,7 @@ static inline void save_spu_mb(struct sp
eieio();
csa->spu_chnlcnt_RW[29] = in_be64(&priv2->spu_chnlcnt_RW);
for (i = 0; i < 4; i++) {
- csa->pu_mailbox_data[i] = in_be64(&priv2->spu_chnldata_RW);
+ csa->spu_mailbox_data[i] = in_be64(&priv2->spu_chnldata_RW);
}
out_be64(&priv2->spu_chnlcnt_RW, 0UL);
eieio();
@@ -1676,7 +1676,7 @@ static inline void restore_spu_mb(struct
eieio();
out_be64(&priv2->spu_chnlcnt_RW, csa->spu_chnlcnt_RW[29]);
for (i = 0; i < 4; i++) {
- out_be64(&priv2->spu_chnldata_RW, csa->pu_mailbox_data[i]);
+ out_be64(&priv2->spu_chnldata_RW, csa->spu_mailbox_data[i]);
}
eieio();
}
@@ -2089,7 +2089,10 @@ int spu_save(struct spu_state *prev, str
acquire_spu_lock(spu); /* Step 1. */
rc = __do_spu_save(prev, spu); /* Steps 2-53. */
release_spu_lock(spu);
-
+ if (rc) {
+ panic("%s failed on SPU[%d], rc=%d.\n",
+ __func__, spu->number, rc);
+ }
return rc;
}
@@ -2110,32 +2113,24 @@ int spu_restore(struct spu_state *new, s
harvest(NULL, spu);
rc = __do_spu_restore(new, spu);
release_spu_lock(spu);
-
+ if (rc) {
+ panic("%s failed on SPU[%d] rc=%d.\n",
+ __func__, spu->number, rc);
+ }
return rc;
}
/**
- * spu_switch - SPU context switch (save + restore).
- * @prev: pointer to SPU context save area, to be saved.
- * @new: pointer to SPU context save area, to be restored.
+ * spu_harvest - SPU harvest (reset) operation
* @spu: pointer to SPU iomem structure.
*
- * Perform save, then restore. Only harvest if the
- * save fails, as cleanup is otherwise not needed.
+ * Perform SPU harvest (reset) operation.
*/
-int spu_switch(struct spu_state *prev, struct spu_state *new, struct spu *spu)
+void spu_harvest(struct spu *spu)
{
- int rc;
-
- acquire_spu_lock(spu); /* Save, Step 1. */
- rc = __do_spu_save(prev, spu); /* Save, Steps 2-53. */
- if (rc != 0) {
- harvest(prev, spu);
- }
- rc = __do_spu_restore(new, spu);
+ acquire_spu_lock(spu);
+ harvest(NULL, spu);
release_spu_lock(spu);
-
- return rc;
}
static void init_prob(struct spu_state *csa)
@@ -2203,6 +2198,7 @@ void spu_init_csa(struct spu_state *csa)
memset(lscsa, 0, sizeof(struct spu_lscsa));
csa->lscsa = lscsa;
+ csa->register_lock = SPIN_LOCK_UNLOCKED;
/* Set LS pages reserved to allow for user-space mapping. */
for (p = lscsa->ls; p < lscsa->ls + LS_SIZE; p += PAGE_SIZE)
Index: linux-cg/include/asm-ppc64/spu.h
===================================================================
--- linux-cg.orig/include/asm-ppc64/spu.h
+++ linux-cg/include/asm-ppc64/spu.h
@@ -105,6 +105,9 @@
#define SPU_CONTEXT_SWITCH_PENDING (1UL << SPU_CONTEXT_SWITCH_PENDING_nr)
#define SPU_CONTEXT_SWITCH_ACTIVE (1UL << SPU_CONTEXT_SWITCH_ACTIVE_nr)
+struct spu_context;
+struct spu_runqueue;
+
struct spu {
char *name;
unsigned long local_store_phys;
@@ -113,6 +116,7 @@ struct spu {
struct spu_priv1 __iomem *priv1;
struct spu_priv2 __iomem *priv2;
struct list_head list;
+ struct list_head sched_list;
int number;
u32 isrc;
u32 node;
@@ -121,15 +125,17 @@ struct spu {
size_t ls_size;
unsigned int slb_replace;
struct mm_struct *mm;
+ struct spu_context *ctx;
+ struct spu_runqueue *rq;
+ pid_t pid;
+ int prio;
int class_0_pending;
spinlock_t register_lock;
u32 stop_code;
wait_queue_head_t stop_wq;
- wait_queue_head_t ibox_wq;
- wait_queue_head_t wbox_wq;
- struct fasync_struct *ibox_fasync;
- struct fasync_struct *wbox_fasync;
+ void (* wbox_callback)(struct spu *spu);
+ void (* ibox_callback)(struct spu *spu);
char irq_c0[8];
char irq_c1[8];
@@ -140,9 +146,6 @@ struct spu *spu_alloc(void);
void spu_free(struct spu *spu);
int spu_run(struct spu *spu);
-size_t spu_wbox_write(struct spu *spu, u32 data);
-size_t spu_ibox_read(struct spu *spu, u32 *data);
-
extern struct spufs_calls {
asmlinkage long (*create_thread)(const char __user *name,
unsigned int flags, mode_t mode);
Index: linux-cg/fs/spufs/backing_ops.c
===================================================================
--- /dev/null
+++ linux-cg/fs/spufs/backing_ops.c
@@ -0,0 +1,252 @@
+/* backing_ops.c - query/set operations on saved SPU context.
+ *
+ * Copyright (C) IBM 2005
+ * Author: Mark Nutter <mnutter@us.ibm.com>
+ *
+ * These register operations allow SPUFS to operate on saved
+ * SPU contexts rather than hardware.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2, or (at your option)
+ * any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+ */
+
+#include <linux/config.h>
+#include <linux/module.h>
+#include <linux/errno.h>
+#include <linux/sched.h>
+#include <linux/kernel.h>
+#include <linux/mm.h>
+#include <linux/vmalloc.h>
+#include <linux/smp.h>
+#include <linux/smp_lock.h>
+#include <linux/stddef.h>
+#include <linux/unistd.h>
+
+#include <asm/io.h>
+#include <asm/spu.h>
+#include <asm/spu_csa.h>
+#include <asm/mmu_context.h>
+#include "spufs.h"
+
+/*
+ * Reads/writes to various problem and priv2 registers require
+ * state changes, i.e. generate SPU events, modify channel
+ * counts, etc.
+ */
+
+static void gen_spu_event(struct spu_context *ctx, u32 event)
+{
+ u64 ch0_cnt;
+ u64 ch0_data;
+ u64 ch1_data;
+
+ ch0_cnt = ctx->csa.spu_chnlcnt_RW[0];
+ ch0_data = ctx->csa.spu_chnldata_RW[0];
+ ch1_data = ctx->csa.spu_chnldata_RW[1];
+ ctx->csa.spu_chnldata_RW[0] |= event;
+ if ((ch0_cnt == 0) && !(ch0_data & event) && (ch1_data & event)) {
+ ctx->csa.spu_chnlcnt_RW[0] = 1;
+ }
+}
+
+static int spu_backing_mbox_read(struct spu_context *ctx, u32 * data)
+{
+ u32 mbox_stat;
+ int ret = 0;
+
+ spin_lock(&ctx->csa.register_lock);
+ mbox_stat = ctx->csa.prob.mb_stat_R;
+ if (mbox_stat & 0x0000ff) {
+ /* Read the first available word.
+ * Implementation note: the depth
+ * of pu_mb_R is currently 1.
+ */
+ *data = ctx->csa.prob.pu_mb_R;
+ ctx->csa.prob.mb_stat_R &= ~(0x0000ff);
+ ctx->csa.spu_chnlcnt_RW[28] = 1;
+ gen_spu_event(ctx, MFC_PU_MAILBOX_AVAILABLE_EVENT);
+ ret = 4;
+ }
+ spin_unlock(&ctx->csa.register_lock);
+ return ret;
+}
+
+static u32 spu_backing_mbox_stat_read(struct spu_context *ctx)
+{
+ return ctx->csa.prob.mb_stat_R;
+}
+
+static int spu_backing_ibox_read(struct spu_context *ctx, u32 * data)
+{
+ int ret;
+
+ spin_lock(&ctx->csa.register_lock);
+ if (ctx->csa.prob.mb_stat_R & 0xff0000) {
+ /* Read the first available word.
+ * Implementation note: the depth
+ * of puint_mb_R is currently 1.
+ */
+ *data = ctx->csa.priv2.puint_mb_R;
+ ctx->csa.prob.mb_stat_R &= ~(0xff0000);
+ ctx->csa.spu_chnlcnt_RW[30] = 1;
+ gen_spu_event(ctx, MFC_PU_INT_MAILBOX_AVAILABLE_EVENT);
+ ret = 4;
+ } else {
+ /* make sure we get woken up by the interrupt */
+ ctx->csa.priv1.int_mask_class2_RW |= 0x1UL;
+ ret = 0;
+ }
+ spin_unlock(&ctx->csa.register_lock);
+ return ret;
+}
+
+static int spu_backing_wbox_write(struct spu_context *ctx, u32 data)
+{
+ int ret;
+
+ spin_lock(&ctx->csa.register_lock);
+ if ((ctx->csa.prob.mb_stat_R) & 0x00ff00) {
+ int slot = ctx->csa.spu_chnlcnt_RW[29];
+ int avail = (ctx->csa.prob.mb_stat_R & 0x00ff00) >> 8;
+
+ /* We have space to write wbox_data.
+ * Implementation note: the depth
+ * of spu_mb_W is currently 4.
+ */
+ BUG_ON(avail != (4 - slot));
+ ctx->csa.spu_mailbox_data[slot] = data;
+ ctx->csa.spu_chnlcnt_RW[29] = ++slot;
+ ctx->csa.prob.mb_stat_R = (((4 - slot) & 0xff) << 8);
+ gen_spu_event(ctx, MFC_SPU_MAILBOX_WRITTEN_EVENT);
+ ret = 4;
+ } else {
+ /* make sure we get woken up by the interrupt when space
+ becomes available */
+ ctx->csa.priv1.int_mask_class2_RW |= 0x10;
+ ret = 0;
+ }
+ spin_unlock(&ctx->csa.register_lock);
+ return ret;
+}
+
+static u32 spu_backing_signal1_read(struct spu_context *ctx)
+{
+ return ctx->csa.spu_chnldata_RW[3];
+}
+
+static void spu_backing_signal1_write(struct spu_context *ctx, u32 data)
+{
+ spin_lock(&ctx->csa.register_lock);
+ if (ctx->csa.priv2.spu_cfg_RW & 0x1)
+ ctx->csa.spu_chnldata_RW[3] |= data;
+ else
+ ctx->csa.spu_chnldata_RW[3] = data;
+ ctx->csa.spu_chnlcnt_RW[3] = 1;
+ gen_spu_event(ctx, MFC_SIGNAL_1_EVENT);
+ spin_unlock(&ctx->csa.register_lock);
+}
+
+static u32 spu_backing_signal2_read(struct spu_context *ctx)
+{
+ return ctx->csa.spu_chnldata_RW[4];
+}
+
+static void spu_backing_signal2_write(struct spu_context *ctx, u32 data)
+{
+ spin_lock(&ctx->csa.register_lock);
+ if (ctx->csa.priv2.spu_cfg_RW & 0x2)
+ ctx->csa.spu_chnldata_RW[4] |= data;
+ else
+ ctx->csa.spu_chnldata_RW[4] = data;
+ ctx->csa.spu_chnlcnt_RW[4] = 1;
+ gen_spu_event(ctx, MFC_SIGNAL_2_EVENT);
+ spin_unlock(&ctx->csa.register_lock);
+}
+
+static void spu_backing_signal1_type_set(struct spu_context *ctx, u64 val)
+{
+ u64 tmp;
+
+ spin_lock(&ctx->csa.register_lock);
+ tmp = ctx->csa.priv2.spu_cfg_RW;
+ if (val)
+ tmp |= 1;
+ else
+ tmp &= ~1;
+ ctx->csa.priv2.spu_cfg_RW = tmp;
+ spin_unlock(&ctx->csa.register_lock);
+}
+
+static u64 spu_backing_signal1_type_get(struct spu_context *ctx)
+{
+ return ((ctx->csa.priv2.spu_cfg_RW & 1) != 0);
+}
+
+static void spu_backing_signal2_type_set(struct spu_context *ctx, u64 val)
+{
+ u64 tmp;
+
+ spin_lock(&ctx->csa.register_lock);
+ tmp = ctx->csa.priv2.spu_cfg_RW;
+ if (val)
+ tmp |= 2;
+ else
+ tmp &= ~2;
+ ctx->csa.priv2.spu_cfg_RW = tmp;
+ spin_unlock(&ctx->csa.register_lock);
+}
+
+static u64 spu_backing_signal2_type_get(struct spu_context *ctx)
+{
+ return ((ctx->csa.priv2.spu_cfg_RW & 2) != 0);
+}
+
+static u32 spu_backing_npc_read(struct spu_context *ctx)
+{
+ return ctx->csa.prob.spu_npc_RW;
+}
+
+static void spu_backing_npc_write(struct spu_context *ctx, u32 val)
+{
+ ctx->csa.prob.spu_npc_RW = val;
+}
+
+static u32 spu_backing_status_read(struct spu_context *ctx)
+{
+ return ctx->csa.prob.spu_status_R;
+}
+
+static char *spu_backing_get_ls(struct spu_context *ctx)
+{
+ return ctx->csa.lscsa->ls;
+}
+
+struct spu_context_ops spu_backing_ops = {
+ .mbox_read = spu_backing_mbox_read,
+ .mbox_stat_read = spu_backing_mbox_stat_read,
+ .ibox_read = spu_backing_ibox_read,
+ .wbox_write = spu_backing_wbox_write,
+ .signal1_read = spu_backing_signal1_read,
+ .signal1_write = spu_backing_signal1_write,
+ .signal2_read = spu_backing_signal2_read,
+ .signal2_write = spu_backing_signal2_write,
+ .signal1_type_set = spu_backing_signal1_type_set,
+ .signal1_type_get = spu_backing_signal1_type_get,
+ .signal2_type_set = spu_backing_signal2_type_set,
+ .signal2_type_get = spu_backing_signal2_type_get,
+ .npc_read = spu_backing_npc_read,
+ .npc_write = spu_backing_npc_write,
+ .status_read = spu_backing_status_read,
+ .get_ls = spu_backing_get_ls,
+};
Index: linux-cg/fs/spufs/hw_ops.c
===================================================================
--- /dev/null
+++ linux-cg/fs/spufs/hw_ops.c
@@ -0,0 +1,206 @@
+/* hw_ops.c - query/set operations on active SPU context.
+ *
+ * Copyright (C) IBM 2005
+ * Author: Mark Nutter <mnutter@us.ibm.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2, or (at your option)
+ * any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+ */
+
+#include <linux/config.h>
+#include <linux/module.h>
+#include <linux/errno.h>
+#include <linux/sched.h>
+#include <linux/kernel.h>
+#include <linux/mm.h>
+#include <linux/vmalloc.h>
+#include <linux/smp.h>
+#include <linux/smp_lock.h>
+#include <linux/stddef.h>
+#include <linux/unistd.h>
+
+#include <asm/io.h>
+#include <asm/spu.h>
+#include <asm/spu_csa.h>
+#include <asm/mmu_context.h>
+#include "spufs.h"
+
+static int spu_hw_mbox_read(struct spu_context *ctx, u32 * data)
+{
+ struct spu *spu = ctx->spu;
+ struct spu_problem __iomem *prob = spu->problem;
+ u32 mbox_stat;
+ int ret = 0;
+
+ spin_lock_irq(&spu->register_lock);
+ mbox_stat = in_be32(&prob->mb_stat_R);
+ if (mbox_stat & 0x0000ff) {
+ *data = in_be32(&prob->pu_mb_R);
+ ret = 4;
+ }
+ spin_unlock_irq(&spu->register_lock);
+ return ret;
+}
+
+static u32 spu_hw_mbox_stat_read(struct spu_context *ctx)
+{
+ return in_be32(&ctx->spu->problem->mb_stat_R);
+}
+
+static int spu_hw_ibox_read(struct spu_context *ctx, u32 * data)
+{
+ struct spu *spu = ctx->spu;
+ struct spu_problem __iomem *prob = spu->problem;
+ struct spu_priv1 __iomem *priv1 = spu->priv1;
+ struct spu_priv2 __iomem *priv2 = spu->priv2;
+ int ret;
+
+ spin_lock_irq(&spu->register_lock);
+ if (in_be32(&prob->mb_stat_R) & 0xff0000) {
+ /* read the first available word */
+ *data = in_be64(&priv2->puint_mb_R);
+ ret = 4;
+ } else {
+ /* make sure we get woken up by the interrupt */
+ out_be64(&priv1->int_mask_class2_RW,
+ in_be64(&priv1->int_mask_class2_RW) | 0x1);
+ ret = 0;
+ }
+ spin_unlock_irq(&spu->register_lock);
+ return ret;
+}
+
+static int spu_hw_wbox_write(struct spu_context *ctx, u32 data)
+{
+ struct spu *spu = ctx->spu;
+ struct spu_problem __iomem *prob = spu->problem;
+ struct spu_priv1 __iomem *priv1 = spu->priv1;
+ int ret;
+
+ spin_lock_irq(&spu->register_lock);
+ if (in_be32(&prob->mb_stat_R) & 0x00ff00) {
+ /* we have space to write wbox_data to */
+ out_be32(&prob->spu_mb_W, data);
+ ret = 4;
+ } else {
+ /* make sure we get woken up by the interrupt when space
+ becomes available */
+ out_be64(&priv1->int_mask_class2_RW,
+ in_be64(&priv1->int_mask_class2_RW) | 0x10);
+ ret = 0;
+ }
+ spin_unlock_irq(&spu->register_lock);
+ return ret;
+}
+
+static u32 spu_hw_signal1_read(struct spu_context *ctx)
+{
+ return in_be32(&ctx->spu->problem->signal_notify1);
+}
+
+static void spu_hw_signal1_write(struct spu_context *ctx, u32 data)
+{
+ out_be32(&ctx->spu->problem->signal_notify1, data);
+}
+
+static u32 spu_hw_signal2_read(struct spu_context *ctx)
+{
+ return in_be32(&ctx->spu->problem->signal_notify1);
+}
+
+static void spu_hw_signal2_write(struct spu_context *ctx, u32 data)
+{
+ out_be32(&ctx->spu->problem->signal_notify2, data);
+}
+
+static void spu_hw_signal1_type_set(struct spu_context *ctx, u64 val)
+{
+ struct spu *spu = ctx->spu;
+ struct spu_priv2 __iomem *priv2 = spu->priv2;
+ u64 tmp;
+
+ spin_lock_irq(&spu->register_lock);
+ tmp = in_be64(&priv2->spu_cfg_RW);
+ if (val)
+ tmp |= 1;
+ else
+ tmp &= ~1;
+ out_be64(&priv2->spu_cfg_RW, tmp);
+ spin_unlock_irq(&spu->register_lock);
+}
+
+static u64 spu_hw_signal1_type_get(struct spu_context *ctx)
+{
+ return ((in_be64(&ctx->spu->priv2->spu_cfg_RW) & 1) != 0);
+}
+
+static void spu_hw_signal2_type_set(struct spu_context *ctx, u64 val)
+{
+ struct spu *spu = ctx->spu;
+ struct spu_priv2 __iomem *priv2 = spu->priv2;
+ u64 tmp;
+
+ spin_lock_irq(&spu->register_lock);
+ tmp = in_be64(&priv2->spu_cfg_RW);
+ if (val)
+ tmp |= 2;
+ else
+ tmp &= ~2;
+ out_be64(&priv2->spu_cfg_RW, tmp);
+ spin_unlock_irq(&spu->register_lock);
+}
+
+static u64 spu_hw_signal2_type_get(struct spu_context *ctx)
+{
+ return ((in_be64(&ctx->spu->priv2->spu_cfg_RW) & 2) != 0);
+}
+
+static u32 spu_hw_npc_read(struct spu_context *ctx)
+{
+ return in_be32(&ctx->spu->problem->spu_npc_RW);
+}
+
+static void spu_hw_npc_write(struct spu_context *ctx, u32 val)
+{
+ out_be32(&ctx->spu->problem->spu_npc_RW, val);
+}
+
+static u32 spu_hw_status_read(struct spu_context *ctx)
+{
+ return in_be32(&ctx->spu->problem->spu_status_R);
+}
+
+static char *spu_hw_get_ls(struct spu_context *ctx)
+{
+ return ctx->spu->local_store;
+}
+
+struct spu_context_ops spu_hw_ops = {
+ .mbox_read = spu_hw_mbox_read,
+ .mbox_stat_read = spu_hw_mbox_stat_read,
+ .ibox_read = spu_hw_ibox_read,
+ .wbox_write = spu_hw_wbox_write,
+ .signal1_read = spu_hw_signal1_read,
+ .signal1_write = spu_hw_signal1_write,
+ .signal2_read = spu_hw_signal2_read,
+ .signal2_write = spu_hw_signal2_write,
+ .signal1_type_set = spu_hw_signal1_type_set,
+ .signal1_type_get = spu_hw_signal1_type_get,
+ .signal2_type_set = spu_hw_signal2_type_set,
+ .signal2_type_get = spu_hw_signal2_type_get,
+ .npc_read = spu_hw_npc_read,
+ .npc_write = spu_hw_npc_write,
+ .status_read = spu_hw_status_read,
+ .get_ls = spu_hw_get_ls,
+};
Index: linux-cg/include/asm-ppc64/spu_csa.h
===================================================================
--- linux-cg.orig/include/asm-ppc64/spu_csa.h
+++ linux-cg/include/asm-ppc64/spu_csa.h
@@ -242,6 +242,7 @@ struct spu_state {
unsigned long suspend_time;
u64 slb_esid_RW[8];
u64 slb_vsid_RW[8];
+ spinlock_t register_lock;
};
extern void spu_init_csa(struct spu_state *csa);
--
^ permalink raw reply [flat|nested] 14+ messages in thread* [patch 10/11] spufs: new entries for SPU special purpose registers.
2005-09-16 12:16 [patch 00/11] spufs: latest spufs snapshot for 2.6.14-rc1 Arnd Bergmann
` (9 preceding siblings ...)
2005-09-16 12:16 ` [patch 09/11] spufs: SPU scheduler Arnd Bergmann
@ 2005-09-16 12:16 ` Arnd Bergmann
2005-09-16 12:16 ` [patch 11/11] spufs: remove old user interfaces Arnd Bergmann
11 siblings, 0 replies; 14+ messages in thread
From: Arnd Bergmann @ 2005-09-16 12:16 UTC (permalink / raw)
To: ppc64-dev
Cc: linux-kernel, Paul Mackerras, Benjamin Herrenschmidt,
jordi_caubet, Hiroyuki Machida, Geoff Levand
[-- Attachment #1: spufs-add-sprs.diff --]
[-- Type: text/plain, Size: 5669 bytes --]
This patch implements SPUFS directory entries for various special
purpose registers. These include the SPU floating point control
register (fpcr), decrementer (decr), and machine status register
(srr0). It should now be possible to query/set all user state
from SPUFS, implement get/set_context interfaces, etc.
From: Mark Nutter <mnutter@us.ibm.com>
Signed-off-by: Arnd Bergmann <arndb@de.ibm.com>
--
file.c | 170 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 files changed, 170 insertions(+)
Index: linux-cg/fs/spufs/file.c
===================================================================
--- linux-cg.orig/fs/spufs/file.c
+++ linux-cg/fs/spufs/file.c
@@ -223,6 +223,60 @@ static struct file_operations spufs_regs
.llseek = generic_file_llseek,
};
+static ssize_t
+spufs_fpcr_read(struct file *file, char __user * buffer,
+ size_t size, loff_t * pos)
+{
+ struct spu_context *ctx = file->private_data;
+ struct spu_lscsa *lscsa = ctx->csa.lscsa;
+ int ret;
+
+ spu_acquire_saved(ctx);
+ if (ctx->state == SPU_STATE_LOCKED) {
+ spu_release(ctx);
+ return -EAGAIN;
+ }
+
+ ret = simple_read_from_buffer(buffer, size, pos,
+ &lscsa->fpcr, sizeof(lscsa->fpcr));
+
+ spu_release(ctx);
+ return ret;
+}
+
+static ssize_t
+spufs_fpcr_write(struct file *file, const char __user * buffer,
+ size_t size, loff_t * pos)
+{
+ struct spu_context *ctx = file->private_data;
+ struct spu_lscsa *lscsa = ctx->csa.lscsa;
+ int ret;
+
+ size = min_t(ssize_t, sizeof(lscsa->fpcr) - *pos, size);
+ if (size <= 0)
+ return -EFBIG;
+ *pos += size;
+
+ spu_acquire_saved(ctx);
+ if (ctx->state == SPU_STATE_LOCKED) {
+ spu_release(ctx);
+ return -EAGAIN;
+ }
+
+ ret = copy_from_user((char *)&lscsa->fpcr + *pos - size,
+ buffer, size) ? -EFAULT : size;
+
+ spu_release(ctx);
+ return ret;
+}
+
+static struct file_operations spufs_fpcr_fops = {
+ .open = spufs_regs_open,
+ .read = spufs_fpcr_read,
+ .write = spufs_fpcr_write,
+ .llseek = generic_file_llseek,
+};
+
/* generic open function for all pipe-like files */
static int spufs_pipe_open(struct inode *inode, struct file *file)
{
@@ -789,6 +843,116 @@ static u64 spufs_npc_get(void *data)
}
DEFINE_SIMPLE_ATTRIBUTE(spufs_npc_ops, spufs_npc_get, spufs_npc_set, "%llx\n")
+static void spufs_decr_set(void *data, u64 val)
+{
+ struct spu_context *ctx = data;
+ struct spu_lscsa *lscsa = ctx->csa.lscsa;
+ spu_acquire_saved(ctx);
+ lscsa->decr.slot[0] = (u32) val;
+ spu_release(ctx);
+}
+
+static u64 spufs_decr_get(void *data)
+{
+ struct spu_context *ctx = data;
+ struct spu_lscsa *lscsa = ctx->csa.lscsa;
+ u64 ret;
+ spu_acquire_saved(ctx);
+ ret = lscsa->decr.slot[0];
+ spu_release(ctx);
+ return ret;
+}
+DEFINE_SIMPLE_ATTRIBUTE(spufs_decr_ops, spufs_decr_get, spufs_decr_set,
+ "%llx\n")
+
+static void spufs_decr_status_set(void *data, u64 val)
+{
+ struct spu_context *ctx = data;
+ struct spu_lscsa *lscsa = ctx->csa.lscsa;
+ spu_acquire_saved(ctx);
+ lscsa->decr_status.slot[0] = (u32) val;
+ spu_release(ctx);
+}
+
+static u64 spufs_decr_status_get(void *data)
+{
+ struct spu_context *ctx = data;
+ struct spu_lscsa *lscsa = ctx->csa.lscsa;
+ u64 ret;
+ spu_acquire_saved(ctx);
+ ret = lscsa->decr_status.slot[0];
+ spu_release(ctx);
+ return ret;
+}
+DEFINE_SIMPLE_ATTRIBUTE(spufs_decr_status_ops, spufs_decr_status_get,
+ spufs_decr_status_set, "%llx\n")
+
+static void spufs_spu_tag_mask_set(void *data, u64 val)
+{
+ struct spu_context *ctx = data;
+ struct spu_lscsa *lscsa = ctx->csa.lscsa;
+ spu_acquire_saved(ctx);
+ lscsa->tag_mask.slot[0] = (u32) val;
+ spu_release(ctx);
+}
+
+static u64 spufs_spu_tag_mask_get(void *data)
+{
+ struct spu_context *ctx = data;
+ struct spu_lscsa *lscsa = ctx->csa.lscsa;
+ u64 ret;
+ spu_acquire_saved(ctx);
+ ret = lscsa->tag_mask.slot[0];
+ spu_release(ctx);
+ return ret;
+}
+DEFINE_SIMPLE_ATTRIBUTE(spufs_spu_tag_mask_ops, spufs_spu_tag_mask_get,
+ spufs_spu_tag_mask_set, "%llx\n")
+
+static void spufs_event_mask_set(void *data, u64 val)
+{
+ struct spu_context *ctx = data;
+ struct spu_lscsa *lscsa = ctx->csa.lscsa;
+ spu_acquire_saved(ctx);
+ lscsa->event_mask.slot[0] = (u32) val;
+ spu_release(ctx);
+}
+
+static u64 spufs_event_mask_get(void *data)
+{
+ struct spu_context *ctx = data;
+ struct spu_lscsa *lscsa = ctx->csa.lscsa;
+ u64 ret;
+ spu_acquire_saved(ctx);
+ ret = lscsa->event_mask.slot[0];
+ spu_release(ctx);
+ return ret;
+}
+DEFINE_SIMPLE_ATTRIBUTE(spufs_event_mask_ops, spufs_event_mask_get,
+ spufs_event_mask_set, "%llx\n")
+
+static void spufs_srr0_set(void *data, u64 val)
+{
+ struct spu_context *ctx = data;
+ struct spu_lscsa *lscsa = ctx->csa.lscsa;
+ spu_acquire_saved(ctx);
+ lscsa->srr0.slot[0] = (u32) val;
+ spu_release(ctx);
+}
+
+static u64 spufs_srr0_get(void *data)
+{
+ struct spu_context *ctx = data;
+ struct spu_lscsa *lscsa = ctx->csa.lscsa;
+ u64 ret;
+ spu_acquire_saved(ctx);
+ ret = lscsa->srr0.slot[0];
+ spu_release(ctx);
+ return ret;
+}
+DEFINE_SIMPLE_ATTRIBUTE(spufs_srr0_ops, spufs_srr0_get, spufs_srr0_set,
+ "%llx\n")
+
struct tree_descr spufs_dir_contents[] = {
{ "mem", &spufs_mem_fops, 0666, },
{ "regs", &spufs_regs_fops, 0666, },
@@ -804,5 +968,11 @@ struct tree_descr spufs_dir_contents[] =
{ "signal1_type", &spufs_signal1_type, 0666, },
{ "signal2_type", &spufs_signal2_type, 0666, },
{ "npc", &spufs_npc_ops, 0666, },
+ { "fpcr", &spufs_fpcr_fops, 0666, },
+ { "decr", &spufs_decr_ops, 0666, },
+ { "decr_status", &spufs_decr_status_ops, 0666, },
+ { "spu_tag_mask", &spufs_spu_tag_mask_ops, 0666, },
+ { "event_mask", &spufs_event_mask_ops, 0666, },
+ { "srr0", &spufs_srr0_ops, 0666, },
{},
};
--
^ permalink raw reply [flat|nested] 14+ messages in thread* [patch 11/11] spufs: remove old user interfaces
2005-09-16 12:16 [patch 00/11] spufs: latest spufs snapshot for 2.6.14-rc1 Arnd Bergmann
` (10 preceding siblings ...)
2005-09-16 12:16 ` [patch 10/11] spufs: new entries for SPU special purpose registers Arnd Bergmann
@ 2005-09-16 12:16 ` Arnd Bergmann
11 siblings, 0 replies; 14+ messages in thread
From: Arnd Bergmann @ 2005-09-16 12:16 UTC (permalink / raw)
To: ppc64-dev
Cc: linux-kernel, Paul Mackerras, Benjamin Herrenschmidt,
jordi_caubet, Hiroyuki Machida, Geoff Levand
[-- Attachment #1: spufs-remove-legacy-2.diff --]
[-- Type: text/plain, Size: 10392 bytes --]
This removes support for the old interfaces that we have
been using up to this point. With this patch, the run
file will not be provided any more, and it is no longer
possible to create or remove files or directories in
the spufs.
Signed-off-by: Arnd Bergmann <arndb@de.ibm.com>
--
arch/ppc64/kernel/spu_base.c | 10 --
fs/spufs/context.c | 25 -----
fs/spufs/file.c | 97 +----------------------
fs/spufs/inode.c | 68 ----------------
fs/spufs/spufs.h | 1
fs/spufs/syscalls.c | 10 --
6 files changed, 15 insertions(+), 196 deletions(-)
Index: linux-cg/fs/spufs/syscalls.c
===================================================================
--- linux-cg.orig/fs/spufs/syscalls.c
+++ linux-cg/fs/spufs/syscalls.c
@@ -36,7 +36,7 @@ long do_spu_run(struct file *filp, __u32
u32 npc, status;
ret = -EFAULT;
- if (get_user(npc, unpc))
+ if (get_user(npc, unpc) || get_user(status, ustatus))
goto out;
ret = -EINVAL;
@@ -46,13 +46,7 @@ long do_spu_run(struct file *filp, __u32
i = SPUFS_I(filp->f_dentry->d_inode);
ret = spufs_run_spu(filp, i->i_ctx, &npc, &status);
- if (ret ==-EAGAIN || ret == -EIO)
- ret = status;
-
- if (put_user(npc, unpc))
- ret = -EFAULT;
-
- if (ustatus && put_user(status, ustatus))
+ if (put_user(npc, unpc) || put_user(status, ustatus))
ret = -EFAULT;
out:
return ret;
Index: linux-cg/arch/ppc64/kernel/spu_base.c
===================================================================
--- linux-cg.orig/arch/ppc64/kernel/spu_base.c
+++ linux-cg/arch/ppc64/kernel/spu_base.c
@@ -521,11 +521,7 @@ int spu_run(struct spu *spu)
|| (in_be64(&priv1->mfc_dsisr_RW) & MFC_DSISR_PTE_NOT_FOUND)
|| spu->class_0_pending);
- if (status & SPU_STATUS_STOPPED_BY_STOP)
- ret = -EAGAIN;
- else if (status & SPU_STATUS_STOPPED_BY_HALT)
- ret = -EIO;
- else if (in_be64(&priv1->mfc_dsisr_RW) & MFC_DSISR_PTE_NOT_FOUND)
+ if (in_be64(&priv1->mfc_dsisr_RW) & MFC_DSISR_PTE_NOT_FOUND)
ret = spu_handle_pte_fault(spu);
if (spu->class_0_pending)
@@ -534,7 +530,9 @@ int spu_run(struct spu *spu)
if (!ret && signal_pending(current))
ret = -ERESTARTSYS;
- } while (!ret);
+ } while (!ret && !(status &
+ (SPU_STATUS_STOPPED_BY_STOP |
+ SPU_STATUS_STOPPED_BY_HALT)));
/* Ensure SPU is stopped. */
out_be32(&prob->spu_runcntl_RW, SPU_RUNCNTL_STOP);
Index: linux-cg/fs/spufs/context.c
===================================================================
--- linux-cg.orig/fs/spufs/context.c
+++ linux-cg/fs/spufs/context.c
@@ -98,31 +98,6 @@ static void spu_unmap_mappings(struct sp
unmap_mapping_range(ctx->local_store, 0, LS_SIZE, 1);
}
-int spu_acquire_runnable_nonblock(struct spu_context *ctx)
-{
- int ret = 0;
-
- if (!down_read_trylock(&ctx->state_sema))
- return -EAGAIN;
- if (ctx->state == SPU_STATE_RUNNABLE
- || ctx->state == SPU_STATE_LOCKED)
- return 0;
- up_read(&ctx->state_sema);
-
- if (!down_write_trylock(&ctx->state_sema))
- return -EAGAIN;
- if (ctx->state == SPU_STATE_SAVED) {
- spu_unmap_mappings(ctx);
- ret = spu_activate(ctx, 0);
- ctx->state = SPU_STATE_RUNNABLE;
- }
- downgrade_write(&ctx->state_sema);
- if (ret)
- /* Release here, to simplify calling code. */
- spu_release(ctx);
- return ret;
-}
-
int spu_acquire_runnable(struct spu_context *ctx)
{
int ret = 0;
Index: linux-cg/fs/spufs/file.c
===================================================================
--- linux-cg.orig/fs/spufs/file.c
+++ linux-cg/fs/spufs/file.c
@@ -578,23 +578,21 @@ static struct file_operations spufs_wbox
};
long spufs_run_spu(struct file *file, struct spu_context *ctx,
- u32 *npc, u32 *status)
+ u32 *npc, u32 *status)
{
int ret;
- if (file->f_flags & O_NONBLOCK) {
- ret = spu_acquire_runnable_nonblock(ctx);
- } else {
- ret = spu_acquire_runnable(ctx);
- }
- if (ret != 0)
+ ret = spu_acquire_runnable(ctx);
+ if (ret)
return ret;
ctx->ops->npc_write(ctx, *npc);
ret = spu_run(ctx->spu);
- *status = ctx->ops->status_read(ctx);
+ if (!ret)
+ ret = ctx->ops->status_read(ctx);
+
*npc = ctx->ops->npc_read(ctx);
spu_release(ctx);
@@ -602,88 +600,6 @@ long spufs_run_spu(struct file *file, st
return ret;
}
-struct spufs_run_arg {
- u32 npc; /* inout: Next Program Counter */
- u32 status; /* out: SPU status */
-};
-
-static ssize_t spufs_run_read(struct file *file, char __user *buf,
- size_t len, loff_t *pos)
-{
- struct spu_context *ctx;
- struct spufs_run_arg arg;
- int ret;
-
- ctx = file->private_data;
-
- ret = -EINVAL;
- if (len < 8)
- goto out;
-
- if (file->f_flags & O_NONBLOCK) {
- ret = spu_acquire_runnable_nonblock(ctx);
- } else {
- ret = spu_acquire_runnable(ctx);
- }
- if (ret != 0)
- return ret;
-
- ret = spu_run(ctx->spu);
- if (ret == -EAGAIN)
- ret = 0;
-
- arg.status = ctx->ops->status_read(ctx);
- arg.npc = ctx->ops->npc_read(ctx);
-
- if ((arg.status & 0xffff0002) == 0x21000002) {
- /* library callout */
- u32 npc = arg.npc;
- arg.npc = *(u32*) (ctx->spu->local_store + npc);
- npc += 4;
- out_be32(&ctx->spu->problem->spu_npc_RW, npc);
- }
-
- spu_release(ctx);
- if (ret)
- goto out;
-
- ret = 8;
- if (copy_to_user(buf, &arg, 8))
- ret = -EFAULT;
-
-out:
- return ret;
-}
-
-/* either this ioctl function or the system call needs to die! */
-static long spufs_run_ioctl(struct file *file, unsigned int num,
- unsigned long arg)
-{
- struct spufs_run_arg data;
- int ret;
-
- if (num != _IOWR('s', 0, struct spufs_run_arg))
- return -EINVAL;
-
- if (copy_from_user(&data, (void __user *)arg, sizeof data))
- return -EFAULT;
-
- ret = spufs_run_spu(file, file->private_data,
- &data.npc, &data.status);
-
- if (copy_to_user((void __user *)arg, &data, sizeof data))
- ret = -EFAULT;
-
- return ret;
-}
-
-static struct file_operations spufs_run_fops = {
- .open = spufs_pipe_open,
- .unlocked_ioctl = spufs_run_ioctl,
- .compat_ioctl = spufs_run_ioctl,
- .read = spufs_run_read,
-};
-
static ssize_t spufs_signal1_read(struct file *file, char __user *buf,
size_t len, loff_t *pos)
{
@@ -956,7 +872,6 @@ DEFINE_SIMPLE_ATTRIBUTE(spufs_srr0_ops,
struct tree_descr spufs_dir_contents[] = {
{ "mem", &spufs_mem_fops, 0666, },
{ "regs", &spufs_regs_fops, 0666, },
- { "run", &spufs_run_fops, 0444, },
{ "mbox", &spufs_mbox_fops, 0444, },
{ "ibox", &spufs_ibox_fops, 0444, },
{ "wbox", &spufs_wbox_fops, 0222, },
Index: linux-cg/fs/spufs/inode.c
===================================================================
--- linux-cg.orig/fs/spufs/inode.c
+++ linux-cg/fs/spufs/inode.c
@@ -41,24 +41,6 @@
static kmem_cache_t *spufs_inode_cache;
-/* Information about the backing dev, same as ramfs */
-#if 0
-static struct backing_dev_info spufs_backing_dev_info = {
- .ra_pages = 0, /* No readahead */
- .capabilities = BDI_CAP_NO_ACCT_DIRTY | BDI_CAP_NO_WRITEBACK |
- BDI_CAP_MAP_DIRECT | BDI_CAP_MAP_COPY | BDI_CAP_READ_MAP |
- BDI_CAP_WRITE_MAP,
-};
-
-static struct address_space_operations spufs_aops = {
- .readpage = simple_readpage,
- .prepare_write = simple_prepare_write,
- .commit_write = simple_commit_write,
-};
-#endif
-
-/* Inode operations */
-
static struct inode *
spufs_alloc_inode(struct super_block *sb)
{
@@ -111,9 +93,6 @@ spufs_setattr(struct dentry *dentry, str
{
struct inode *inode = dentry->d_inode;
-/* dump_stack();
- pr_debug("ia_size %lld, i_size:%lld\n", attr->ia_size, inode->i_size);
-*/
if ((attr->ia_valid & ATTR_SIZE) &&
(attr->ia_size != inode->i_size))
return -EINVAL;
@@ -127,9 +106,7 @@ spufs_new_file(struct super_block *sb, s
struct spu_context *ctx)
{
static struct inode_operations spufs_file_iops = {
- .getattr = simple_getattr,
.setattr = spufs_setattr,
- .unlink = simple_unlink,
};
struct inode *inode;
int ret;
@@ -142,43 +119,12 @@ spufs_new_file(struct super_block *sb, s
ret = 0;
inode->i_op = &spufs_file_iops;
inode->i_fop = fops;
-// inode->i_mapping->a_ops = &spufs_aops;
-// inode->i_mapping->backing_dev_info = &spufs_backing_dev_info;
inode->u.generic_ip = SPUFS_I(inode)->i_ctx = get_spu_context(ctx);
d_add(dentry, inode);
out:
return ret;
}
-static int
-spufs_create(struct inode *dir, struct dentry *dentry,
- int mode, struct nameidata *nd)
-{
- struct tree_descr *descr;
- struct spu_context *ctx;
- int ret;
-
- descr = spufs_dir_contents;
-
- /* search spufs_dir_contents for a file with the name we are
- trying to create */
- ret = -EINVAL;
- while (strcmp(descr->name, dentry->d_name.name) != 0) {
- descr++;
- if (!descr->name || !descr->name[0])
- goto out;
- }
-
- ctx = SPUFS_I(dir)->i_ctx;
- mode &= descr->mode;
-
- ret = spufs_new_file(dir->i_sb, dentry, descr->ops, mode, ctx);
- /* get an extra reference to pin the dentry */
- dget(dentry);
-out:
- return ret;
-}
-
static void
spufs_delete_inode(struct inode *inode)
{
@@ -253,8 +199,6 @@ static int spufs_dir_close(struct inode
struct inode_operations spufs_dir_inode_operations = {
.lookup = simple_lookup,
- .unlink = simple_unlink,
- .create = spufs_create,
};
struct file_operations spufs_autodelete_dir_operations = {
@@ -401,14 +345,8 @@ spufs_parse_options(char *options, struc
}
static int
-spufs_create_root(struct super_block *sb, void *data) {
- static struct inode_operations spufs_root_inode_operations = {
- .lookup = simple_lookup,
- .mkdir = spufs_mkdir,
- .rmdir = spufs_rmdir,
-// .rename = simple_rename, // XXX maybe
- };
-
+spufs_create_root(struct super_block *sb, void *data)
+{
struct inode *inode;
int ret;
@@ -417,7 +355,7 @@ spufs_create_root(struct super_block *sb
if (!inode)
goto out;
- inode->i_op = &spufs_root_inode_operations;
+ inode->i_op = &spufs_dir_inode_operations;
inode->i_fop = &simple_dir_operations;
SPUFS_I(inode)->i_ctx = NULL;
Index: linux-cg/fs/spufs/spufs.h
===================================================================
--- linux-cg.orig/fs/spufs/spufs.h
+++ linux-cg/fs/spufs/spufs.h
@@ -101,7 +101,6 @@ int put_spu_context(struct spu_context *
void spu_acquire(struct spu_context *ctx);
void spu_release(struct spu_context *ctx);
int spu_acquire_runnable(struct spu_context *ctx);
-int spu_acquire_runnable_nonblock(struct spu_context *ctx);
void spu_acquire_saved(struct spu_context *ctx);
int spu_activate(struct spu_context *ctx, u64 flags);
--
^ permalink raw reply [flat|nested] 14+ messages in thread