* Re: [patch 01/35] pasemi: rename platform
From: Olof Johansson @ 2007-07-09 0:18 UTC (permalink / raw)
To: Stephen Rothwell; +Cc: linuxppc-dev
In-Reply-To: <20070709095259.0720669b.sfr@canb.auug.org.au>
On Mon, Jul 09, 2007 at 09:52:59AM +1000, Stephen Rothwell wrote:
> On Thu, 05 Jul 2007 12:02:34 -0500 Olof Johansson <olof@lixom.net> wrote:
> >
> > Rename the pasemi platform to "pasemi" to be in line with the platforms
> > directory name.
>
> This should also fix a problem further up in that file (in
> pasemi_publish_devices) where we check for "machine_is(pasemi)".
Yup, that's how I discovered the machine name being less than intuitive.
-Olof
^ permalink raw reply
* Re: ARCH=ppc vs ARCH=powerpc
From: Arnd Bergmann @ 2007-07-09 0:34 UTC (permalink / raw)
To: linuxppc-embedded; +Cc: Mark Fortescue
In-Reply-To: <Pine.LNX.4.61.0707061706140.4792@mtfhpc.demon.co.uk>
On Friday 06 July 2007, Mark Fortescue wrote:
> For me, the requirement is to keep support for the cards indicated in
> arch/ppc/config/xxx_defconfig (as I have one that I might what to use with
> linux) in the same way as they are supported in ARCH=ppc.
The one thing that is holding up the transition to arch/powerpc is lack
of people with access to the respective boards. If you have one board
that you care about, it would be good if you look into getting it moved
over. Tell us if you have problems doing so. It shouldn't be all that
hard any more, and if it is, maybe some documentation is missing.
> I have just made some tweeks to get ARCH=ppc building for me but I still
> need to sort out what is required to get around *** Error: Headers not
> exportable for this architecture (ppc).
You should simply install the headers with ARCH=powerc, even when the
kernel is built with ARCH=ppc. From the user space point of view, there
is only one set of headers.
Arnd <><
^ permalink raw reply
* RE: mpc83xx USB support status
From: Li Yang-r58472 @ 2007-07-09 3:01 UTC (permalink / raw)
To: Grant Likely, Tabi Timur-B04825, Behan Webster,
Linux PPC Linux PPC
In-Reply-To: <fa686aa40707061302o6ed9ec7cq2b9cdba2553cee7a@mail.gmail.com>
> -----Original Message-----
> From: glikely@secretlab.ca [mailto:glikely@secretlab.ca] On Behalf Of
Grant
> Likely
> Sent: Saturday, July 07, 2007 4:02 AM
> To: Li Yang-r58472; Tabi Timur-B04825; Behan Webster; Linux PPC Linux
PPC
> Subject: mpc83xx USB support status
>=20
> Li,
>=20
> What is the state of mpc83xx support in arch/powerpc? I saw some of
> the patches you posted to the ppc and usb-devel lists, but I want to
> make sure I've got the whole set. Also, can you tell me what the
> state of those patches are (ie. which ones will be picked up for
> 2.6.23?)
The USB host/gadget should be working in arch/powerpc with all the
patches I submitted to ppc and usb-devel applied. AFAIK, all the
patches posted to usb-devel will be picked up for 2.6.23. I hope the
platform rework code will also be picked up; I see nothing preventing it
from being included.
>=20
> I'm working on the mpc8349-mitx-gp board, and I can do the work to get
> your changes working on that board.
Good, the USB drivers are intended to be platform independent. The soc
specific configuration will be done in the rework patch I submitted.
The only thing you need to take care of is board specific setup. Feel
free to contact me if you have any problem.
- Leo
^ permalink raw reply
* Re: ARCH=ppc vs ARCH=powerpc
From: Josh Boyer @ 2007-07-09 3:37 UTC (permalink / raw)
To: Arnd Bergmann; +Cc: Mark Fortescue, linuxppc-embedded
In-Reply-To: <200707090234.14851.arnd@arndb.de>
On Mon, 2007-07-09 at 02:34 +0200, Arnd Bergmann wrote:
> On Friday 06 July 2007, Mark Fortescue wrote:
> > For me, the requirement is to keep support for the cards indicated in
> > arch/ppc/config/xxx_defconfig (as I have one that I might what to use with
> > linux) in the same way as they are supported in ARCH=ppc.
>
> The one thing that is holding up the transition to arch/powerpc is lack
> of people with access to the respective boards. If you have one board
> that you care about, it would be good if you look into getting it moved
> over. Tell us if you have problems doing so. It shouldn't be all that
> hard any more, and if it is, maybe some documentation is missing.
It's not just lack of boards. It's lack of understanding of the
requirements to get a board port into arch/powerpc. Namely, the device
tree stuff.
josh
^ permalink raw reply
* Re: ARCH=ppc vs ARCH=powerpc
From: Josh Boyer @ 2007-07-09 3:38 UTC (permalink / raw)
To: Mark Fortescue; +Cc: linuxppc-embedded
In-Reply-To: <Pine.LNX.4.61.0707061706140.4792@mtfhpc.demon.co.uk>
On Fri, 2007-07-06 at 17:21 +0100, Mark Fortescue wrote:
> Hi,
>
> I have noticed and read a few comments regarding the demise of ARCH=ppc.
>
> Having two PowerPC arcitectures and only supprting one of them makes
> little sense. Is it not time to migrate (where required) the remaining
> code from arch/ppc to arch/powerpc and asm-ppc to asm-powerpc. There are
> not many files left that are unique to ARCH=ppc so it should not take too
> long.
It was started in 2005. It's taken too long already.
Bug fixes will still be taken until arch/ppc dies. New platform/board
support, no.
josh
^ permalink raw reply
* Re: [PATCH] Add USB support to mpc8349-mitx board port
From: John Rigby @ 2007-07-09 3:55 UTC (permalink / raw)
To: Grant Likely; +Cc: Linuxppc-embedded
In-Reply-To: <fa686aa40707071748g7f00f4c7jb761cceb7d72456d@mail.gmail.com>
[-- Attachment #1: Type: text/plain, Size: 1113 bytes --]
I see your point, I agree the two layers of glue is bad. But this means
the double code is spread across many drivers instead of being in one
place (fsl_soc.c).
On 7/7/07, Grant Likely <grant.likely@secretlab.ca> wrote:
>
> On 7/7/07, John Rigby <jcrigby@gmail.com> wrote:
> > > This same comment probably goes for the other arch_initcall functions
> > > in fsl_soc.c which do exactly the same thing for other devices.
> >
> > This depends, some devices in fsl_soc.c may exist on non-powerpc SoCs
> > that do not have OF. The USB core in 8349 for example also is in the
> > arm based mx27 and mx31. These devices should remain platform devices
> > and the glue in fls_soc.c will continure to be needed.
>
> I disagree; the current method is "glue for the glue". There is no
> reason why the driver cannot have two bindings; one for
> platform_device and one for of_platform_device. It's about the same
> amount of code, but uses less indirection for the device tree case.
>
> Cheers,
> g.
>
>
> --
> Grant Likely, B.Sc., P.Eng.
> Secret Lab Technologies Ltd.
> grant.likely@secretlab.ca
> (403) 399-0195
>
[-- Attachment #2: Type: text/html, Size: 1617 bytes --]
^ permalink raw reply
* [PATCH] i2c-mpc: work around missing-9th-clock-pulse bug
From: Domen Puncer @ 2007-07-09 7:19 UTC (permalink / raw)
To: khali; +Cc: i2c, linuxppc-embedded
Work around a problem reported on:
http://ozlabs.org/pipermail/linuxppc-embedded/2005-July/019038.html
Without this patch I2C on mpc5200 becomes unusable after a while.
Tested on mpc5200 based boards by Matthias and me.
Signed-off-by: Domen Puncer <domen.puncer@telargo.com>
---
drivers/i2c/busses/i2c-mpc.c | 19 +++++++++++++++++++
1 file changed, 19 insertions(+)
Index: work-powerpc.git/drivers/i2c/busses/i2c-mpc.c
===================================================================
--- work-powerpc.git.orig/drivers/i2c/busses/i2c-mpc.c
+++ work-powerpc.git/drivers/i2c/busses/i2c-mpc.c
@@ -74,6 +74,20 @@ static irqreturn_t mpc_i2c_isr(int irq,
return IRQ_HANDLED;
}
+static void mpc_i2c_fixup(struct mpc_i2c *i2c)
+{
+ writeccr(i2c, 0);
+ udelay(30);
+ writeccr(i2c, CCR_MEN);
+ udelay(30);
+ writeccr(i2c, CCR_MSTA | CCR_MTX);
+ udelay(30);
+ writeccr(i2c, CCR_MSTA | CCR_MTX | CCR_MEN);
+ udelay(30);
+ writeccr(i2c, CCR_MEN);
+ udelay(30);
+}
+
static int i2c_wait(struct mpc_i2c *i2c, unsigned timeout, int writing)
{
unsigned long orig_jiffies = jiffies;
@@ -153,6 +167,9 @@ static void mpc_i2c_start(struct mpc_i2c
static void mpc_i2c_stop(struct mpc_i2c *i2c)
{
writeccr(i2c, CCR_MEN);
+ mb();
+ writeccr(i2c, 0);
+ mb();
}
static int mpc_write(struct mpc_i2c *i2c, int target,
@@ -245,6 +262,8 @@ static int mpc_xfer(struct i2c_adapter *
}
if (time_after(jiffies, orig_jiffies + HZ)) {
pr_debug("I2C: timeout\n");
+ if (readb(i2c->base + MPC_I2C_SR) == (CSR_MCF | CSR_MBB | CSR_RXAK))
+ mpc_i2c_fixup(i2c);
return -EIO;
}
schedule();
^ permalink raw reply
* Re: [patch 01/35] pasemi: rename platform
From: Stephen Rothwell @ 2007-07-09 7:23 UTC (permalink / raw)
To: Olof Johansson; +Cc: linuxppc-dev
In-Reply-To: <20070709001835.GA30821@lixom.net>
[-- Attachment #1: Type: text/plain, Size: 518 bytes --]
On Sun, 8 Jul 2007 19:18:35 -0500 Olof Johansson <olof@lixom.net> wrote:
>
> On Mon, Jul 09, 2007 at 09:52:59AM +1000, Stephen Rothwell wrote:
> >
> > This should also fix a problem further up in that file (in
> > pasemi_publish_devices) where we check for "machine_is(pasemi)".
>
> Yup, that's how I discovered the machine name being less than intuitive.
So it is worth noting in the chnge log.
--
Cheers,
Stephen Rothwell sfr@canb.auug.org.au
http://www.canb.auug.org.au/~sfr/
[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply
* [PATCH] mpc5200: low power mode
From: Domen Puncer @ 2007-07-09 7:49 UTC (permalink / raw)
To: linuxppc-embedded; +Cc: Sylvain Munaut
Low-power mode implementation for Lite5200b.
Some I/O registers are also saved here.
A recent U-Boot that supports this (lite5200b_PM_config) is needed.
Signed-off-by: Domen Puncer <domen.puncer@telargo.com>
---
It would be nice to get this merged.
arch/powerpc/platforms/52xx/Makefile | 3
arch/powerpc/platforms/52xx/lite5200.c | 14
arch/powerpc/platforms/52xx/lite5200_pm.c | 213 +++++++++++++
arch/powerpc/platforms/52xx/lite5200_sleep.S | 413 +++++++++++++++++++++++++++
include/asm-powerpc/mpc52xx.h | 10
5 files changed, 648 insertions(+), 5 deletions(-)
Index: work-powerpc.git/arch/powerpc/platforms/52xx/lite5200_pm.c
===================================================================
--- /dev/null
+++ work-powerpc.git/arch/powerpc/platforms/52xx/lite5200_pm.c
@@ -0,0 +1,213 @@
+#include <linux/init.h>
+#include <linux/pm.h>
+#include <asm/io.h>
+#include <asm/time.h>
+#include <asm/mpc52xx.h>
+#include "mpc52xx_pic.h"
+
+/* defined in lite5200_sleep.S and only used here */
+extern void lite5200_low_power(void __iomem *sram, void __iomem *mbar);
+
+static struct mpc52xx_cdm __iomem *cdm;
+static struct mpc52xx_intr __iomem *pic;
+static struct mpc52xx_sdma __iomem *bes;
+static struct mpc52xx_xlb __iomem *xlb;
+static struct mpc52xx_gpio __iomem *gps;
+static struct mpc52xx_gpio_wkup __iomem *gpw;
+static void __iomem *sram;
+static const int sram_size = 0x4000; /* 16 kBytes */
+static void __iomem *mbar;
+
+static int lite5200_pm_valid(suspend_state_t state)
+{
+ switch (state) {
+ case PM_SUSPEND_STANDBY:
+ case PM_SUSPEND_MEM:
+ return 1;
+ default:
+ return 0;
+ }
+}
+
+static int lite5200_pm_prepare(suspend_state_t state)
+{
+ /* deep sleep? let mpc52xx code handle that */
+ if (state == PM_SUSPEND_STANDBY)
+ return mpc52xx_pm_prepare(state);
+
+ if (state != PM_SUSPEND_MEM)
+ return -EINVAL;
+
+ /* map registers */
+ mbar = mpc52xx_find_and_map("mpc5200");
+ if (!mbar) {
+ printk(KERN_ERR "%s:%i Error mapping registers\n", __func__, __LINE__);
+ return -ENOSYS;
+ }
+
+ cdm = mbar + 0x200;
+ pic = mbar + 0x500;
+ gps = mbar + 0xb00;
+ gpw = mbar + 0xc00;
+ bes = mbar + 0x1200;
+ xlb = mbar + 0x1f00;
+ sram = mbar + 0x8000;
+
+ return 0;
+}
+
+/* save and restore registers not bound to any real devices */
+static struct mpc52xx_cdm scdm;
+static struct mpc52xx_intr spic;
+static struct mpc52xx_sdma sbes;
+static struct mpc52xx_xlb sxlb;
+static struct mpc52xx_gpio sgps;
+static struct mpc52xx_gpio_wkup sgpw;
+
+static void lite5200_save_regs(void)
+{
+ _memcpy_fromio(&spic, pic, sizeof(*pic));
+ _memcpy_fromio(&sbes, bes, sizeof(*bes));
+ _memcpy_fromio(&scdm, cdm, sizeof(*cdm));
+ _memcpy_fromio(&sxlb, xlb, sizeof(*xlb));
+ _memcpy_fromio(&sgps, gps, sizeof(*gps));
+ _memcpy_fromio(&sgpw, gpw, sizeof(*gpw));
+
+ _memcpy_fromio(saved_sram, sram, sram_size);
+}
+
+static void lite5200_restore_regs(void)
+{
+ int i;
+ _memcpy_toio(sram, saved_sram, sram_size);
+
+
+ /*
+ * GPIOs. Interrupt Master Enable has higher address then other
+ * registers, so just memcpy is ok.
+ */
+ _memcpy_toio(gpw, &sgpw, sizeof(*gpw));
+ _memcpy_toio(gps, &sgps, sizeof(*gps));
+
+
+ /* XLB Arbitrer */
+ out_be32(&xlb->snoop_window, sxlb.snoop_window);
+ out_be32(&xlb->master_priority, sxlb.master_priority);
+ out_be32(&xlb->master_pri_enable, sxlb.master_pri_enable);
+
+ /* enable */
+ out_be32(&xlb->int_enable, sxlb.int_enable);
+ out_be32(&xlb->config, sxlb.config);
+
+
+ /* CDM - Clock Distribution Module */
+ out_8(&cdm->ipb_clk_sel, scdm.ipb_clk_sel);
+ out_8(&cdm->pci_clk_sel, scdm.pci_clk_sel);
+
+ out_8(&cdm->ext_48mhz_en, scdm.ext_48mhz_en);
+ out_8(&cdm->fd_enable, scdm.fd_enable);
+ out_be16(&cdm->fd_counters, scdm.fd_counters);
+
+ out_be32(&cdm->clk_enables, scdm.clk_enables);
+
+ out_8(&cdm->osc_disable, scdm.osc_disable);
+
+ out_be16(&cdm->mclken_div_psc1, scdm.mclken_div_psc1);
+ out_be16(&cdm->mclken_div_psc2, scdm.mclken_div_psc2);
+ out_be16(&cdm->mclken_div_psc3, scdm.mclken_div_psc3);
+ out_be16(&cdm->mclken_div_psc6, scdm.mclken_div_psc6);
+
+
+ /* BESTCOMM */
+ out_be32(&bes->taskBar, sbes.taskBar);
+ out_be32(&bes->currentPointer, sbes.currentPointer);
+ out_be32(&bes->endPointer, sbes.endPointer);
+ out_be32(&bes->variablePointer, sbes.variablePointer);
+
+ out_8(&bes->IntVect1, sbes.IntVect1);
+ out_8(&bes->IntVect2, sbes.IntVect2);
+ out_be16(&bes->PtdCntrl, sbes.PtdCntrl);
+
+ for (i=0; i<32; i++)
+ out_8(&bes->ipr[i], sbes.ipr[i]);
+
+ out_be32(&bes->cReqSelect, sbes.cReqSelect);
+ out_be32(&bes->task_size0, sbes.task_size0);
+ out_be32(&bes->task_size1, sbes.task_size1);
+ out_be32(&bes->MDEDebug, sbes.MDEDebug);
+ out_be32(&bes->ADSDebug, sbes.ADSDebug);
+ out_be32(&bes->Value1, sbes.Value1);
+ out_be32(&bes->Value2, sbes.Value2);
+ out_be32(&bes->Control, sbes.Control);
+ out_be32(&bes->Status, sbes.Status);
+ out_be32(&bes->PTDDebug, sbes.PTDDebug);
+
+ /* restore tasks */
+ for (i=0; i<16; i++)
+ out_be16(&bes->tcr[i], sbes.tcr[i]);
+
+ /* enable interrupts */
+ out_be32(&bes->IntPend, sbes.IntPend);
+ out_be32(&bes->IntMask, sbes.IntMask);
+
+
+ /* PIC */
+ out_be32(&pic->per_pri1, spic.per_pri1);
+ out_be32(&pic->per_pri2, spic.per_pri2);
+ out_be32(&pic->per_pri3, spic.per_pri3);
+
+ out_be32(&pic->main_pri1, spic.main_pri1);
+ out_be32(&pic->main_pri2, spic.main_pri2);
+
+ out_be32(&pic->enc_status, spic.enc_status);
+
+ /* unmask and enable interrupts */
+ out_be32(&pic->per_mask, spic.per_mask);
+ out_be32(&pic->main_mask, spic.main_mask);
+ out_be32(&pic->ctrl, spic.ctrl);
+}
+
+static int lite5200_pm_enter(suspend_state_t state)
+{
+ /* deep sleep? let mpc52xx code handle that */
+ if (state == PM_SUSPEND_STANDBY) {
+ return mpc52xx_pm_enter(state);
+ }
+
+ lite5200_save_regs();
+
+ /* effectively save FP regs */
+ enable_kernel_fp();
+
+ lite5200_low_power(sram, mbar);
+
+ lite5200_restore_regs();
+
+ /* restart jiffies */
+ wakeup_decrementer();
+
+ iounmap(mbar);
+ return 0;
+}
+
+static int lite5200_pm_finish(suspend_state_t state)
+{
+ /* deep sleep? let mpc52xx code handle that */
+ if (state == PM_SUSPEND_STANDBY) {
+ return mpc52xx_pm_finish(state);
+ }
+ return 0;
+}
+
+static struct pm_ops lite5200_pm_ops = {
+ .valid = lite5200_pm_valid,
+ .prepare = lite5200_pm_prepare,
+ .enter = lite5200_pm_enter,
+ .finish = lite5200_pm_finish,
+};
+
+int __init lite5200_pm_init(void)
+{
+ pm_set_ops(&lite5200_pm_ops);
+ return 0;
+}
Index: work-powerpc.git/arch/powerpc/platforms/52xx/lite5200_sleep.S
===================================================================
--- /dev/null
+++ work-powerpc.git/arch/powerpc/platforms/52xx/lite5200_sleep.S
@@ -0,0 +1,413 @@
+#include <asm/reg.h>
+#include <asm/ppc_asm.h>
+#include <asm/processor.h>
+#include <asm/cache.h>
+
+
+#define SDRAM_CTRL 0x104
+#define SC_MODE_EN (1<<31)
+#define SC_CKE (1<<30)
+#define SC_REF_EN (1<<28)
+#define SC_SOFT_PRE (1<<1)
+
+#define GPIOW_GPIOE 0xc00
+#define GPIOW_DDR 0xc08
+#define GPIOW_DVO 0xc0c
+
+#define CDM_CE 0x214
+#define CDM_SDRAM (1<<3)
+
+
+/* helpers... beware: r10 and r4 are overwritten */
+#define SAVE_SPRN(reg, addr) \
+ mfspr r10, SPRN_##reg; \
+ stw r10, ((addr)*4)(r4);
+
+#define LOAD_SPRN(reg, addr) \
+ lwz r10, ((addr)*4)(r4); \
+ mtspr SPRN_##reg, r10; \
+ sync; \
+ isync;
+
+
+ .data
+registers:
+ .space 0x5c*4
+ .text
+
+/* ---------------------------------------------------------------------- */
+/* low-power mode with help of M68HLC908QT1 */
+
+ .globl lite5200_low_power
+lite5200_low_power:
+
+ mr r7, r3 /* save SRAM va */
+ mr r8, r4 /* save MBAR va */
+
+ /* setup wakeup address for u-boot at physical location 0x0 */
+ lis r3, CONFIG_KERNEL_START@h
+ lis r4, lite5200_wakeup@h
+ ori r4, r4, lite5200_wakeup@l
+ sub r4, r4, r3
+ stw r4, 0(r3)
+
+
+ /*
+ * save stuff BDI overwrites
+ * 0xf0 (0xe0->0x100 gets overwritten when BDI connected;
+ * even when CONFIG_BDI* is disabled and MMU XLAT commented; heisenbug?))
+ * WARNING: self-refresh doesn't seem to work when BDI2000 is connected,
+ * possibly because BDI sets SDRAM registers before wakeup code does
+ */
+ lis r4, registers@h
+ ori r4, r4, registers@l
+ lwz r10, 0xf0(r3)
+ stw r10, (0x1d*4)(r4)
+
+ /* save registers to r4 [destroys r10] */
+ SAVE_SPRN(LR, 0x1c)
+ bl save_regs
+
+ /* flush caches [destroys r3, r4] */
+ bl flush_data_cache
+
+
+ /* copy code to sram */
+ mr r4, r7
+ li r3, (sram_code_end - sram_code)/4
+ mtctr r3
+ lis r3, sram_code@h
+ ori r3, r3, sram_code@l
+1:
+ lwz r5, 0(r3)
+ stw r5, 0(r4)
+ addi r3, r3, 4
+ addi r4, r4, 4
+ bdnz 1b
+
+ /* get tb_ticks_per_usec */
+ lis r3, tb_ticks_per_usec@h
+ lwz r11, tb_ticks_per_usec@l(r3)
+
+ /* disable I and D caches */
+ mfspr r3, SPRN_HID0
+ ori r3, r3, HID0_ICE | HID0_DCE
+ xori r3, r3, HID0_ICE | HID0_DCE
+ sync; isync;
+ mtspr SPRN_HID0, r3
+ sync; isync;
+
+ /* jump to sram */
+ mtlr r7
+ blrl
+ /* doesn't return */
+
+
+sram_code:
+ /* self refresh */
+ lwz r4, SDRAM_CTRL(r8)
+
+ /* send NOP (precharge) */
+ oris r4, r4, SC_MODE_EN@h /* mode_en */
+ stw r4, SDRAM_CTRL(r8)
+ sync
+
+ ori r4, r4, SC_SOFT_PRE /* soft_pre */
+ stw r4, SDRAM_CTRL(r8)
+ sync
+ xori r4, r4, SC_SOFT_PRE
+
+ xoris r4, r4, SC_MODE_EN@h /* !mode_en */
+ stw r4, SDRAM_CTRL(r8)
+ sync
+
+ /* delay (for NOP to finish) */
+ li r12, 1
+ bl udelay
+
+ /*
+ * mode_en must not be set when enabling self-refresh
+ * send AR with CKE low (self-refresh)
+ */
+ oris r4, r4, (SC_REF_EN | SC_CKE)@h
+ xoris r4, r4, (SC_CKE)@h /* ref_en !cke */
+ stw r4, SDRAM_CTRL(r8)
+ sync
+
+ /* delay (after !CKE there should be two cycles) */
+ li r12, 1
+ bl udelay
+
+ /* disable clock */
+ lwz r4, CDM_CE(r8)
+ ori r4, r4, CDM_SDRAM
+ xori r4, r4, CDM_SDRAM
+ stw r4, CDM_CE(r8)
+ sync
+
+ /* delay a bit */
+ li r12, 1
+ bl udelay
+
+
+ /* turn off with QT chip */
+ li r4, 0x02
+ stb r4, GPIOW_GPIOE(r8) /* enable gpio_wkup1 */
+ sync
+
+ stb r4, GPIOW_DVO(r8) /* "output" high */
+ sync
+ stb r4, GPIOW_DDR(r8) /* output */
+ sync
+ stb r4, GPIOW_DVO(r8) /* output high */
+ sync
+
+ /* 10uS delay */
+ li r12, 10
+ bl udelay
+
+ /* turn off */
+ li r4, 0
+ stb r4, GPIOW_DVO(r8) /* output low */
+ sync
+
+ /* wait until we're offline */
+ 1:
+ b 1b
+
+
+ /* local udelay in sram is needed */
+ udelay: /* r11 - tb_ticks_per_usec, r12 - usecs, overwrites r13 */
+ mullw r12, r12, r11
+ mftb r13 /* start */
+ addi r12, r13, r12 /* end */
+ 1:
+ mftb r13 /* current */
+ cmp cr0, r13, r12
+ blt 1b
+ blr
+
+sram_code_end:
+
+
+
+/* uboot jumps here on resume */
+lite5200_wakeup:
+ bl restore_regs
+
+
+ /* HIDs, MSR */
+ LOAD_SPRN(HID1, 0x19)
+ LOAD_SPRN(HID2, 0x1a)
+
+
+ /* address translation is tricky (see turn_on_mmu) */
+ mfmsr r10
+ ori r10, r10, MSR_DR | MSR_IR
+
+
+ mtspr SPRN_SRR1, r10
+ lis r10, mmu_on@h
+ ori r10, r10, mmu_on@l
+ mtspr SPRN_SRR0, r10
+ sync
+ rfi
+mmu_on:
+ /* kernel offset (r4 is still set from restore_registers) */
+ addis r4, r4, CONFIG_KERNEL_START@h
+
+
+ /* restore MSR */
+ lwz r10, (4*0x1b)(r4)
+ mtmsr r10
+ sync; isync;
+
+ /* invalidate caches */
+ mfspr r10, SPRN_HID0
+ ori r5, r10, HID0_ICFI | HID0_DCI
+ mtspr SPRN_HID0, r5 /* invalidate caches */
+ sync; isync;
+ mtspr SPRN_HID0, r10
+ sync; isync;
+
+ /* enable caches */
+ lwz r10, (4*0x18)(r4)
+ mtspr SPRN_HID0, r10 /* restore (enable caches, DPM) */
+ /* ^ this has to be after address translation set in MSR */
+ sync
+ isync
+
+
+ /* restore 0xf0 (BDI2000) */
+ lis r3, CONFIG_KERNEL_START@h
+ lwz r10, (0x1d*4)(r4)
+ stw r10, 0xf0(r3)
+
+ LOAD_SPRN(LR, 0x1c)
+
+
+ blr
+
+
+/* ---------------------------------------------------------------------- */
+/* boring code: helpers */
+
+/* save registers */
+#define SAVE_BAT(n, addr) \
+ SAVE_SPRN(DBAT##n##L, addr); \
+ SAVE_SPRN(DBAT##n##U, addr+1); \
+ SAVE_SPRN(IBAT##n##L, addr+2); \
+ SAVE_SPRN(IBAT##n##U, addr+3);
+
+#define SAVE_SR(n, addr) \
+ mfsr r10, n; \
+ stw r10, ((addr)*4)(r4);
+
+#define SAVE_4SR(n, addr) \
+ SAVE_SR(n, addr); \
+ SAVE_SR(n+1, addr+1); \
+ SAVE_SR(n+2, addr+2); \
+ SAVE_SR(n+3, addr+3);
+
+save_regs:
+ stw r0, 0(r4)
+ stw r1, 0x4(r4)
+ stw r2, 0x8(r4)
+ stmw r11, 0xc(r4) /* 0xc -> 0x5f, (0x18*4-1) */
+
+ SAVE_SPRN(HID0, 0x18)
+ SAVE_SPRN(HID1, 0x19)
+ SAVE_SPRN(HID2, 0x1a)
+ mfmsr r10
+ stw r10, (4*0x1b)(r4)
+ /*SAVE_SPRN(LR, 0x1c) have to save it before the call */
+ /* 0x1d reserved by 0xf0 */
+ SAVE_SPRN(RPA, 0x1e)
+ SAVE_SPRN(SDR1, 0x1f)
+
+ /* save MMU regs */
+ SAVE_BAT(0, 0x20)
+ SAVE_BAT(1, 0x24)
+ SAVE_BAT(2, 0x28)
+ SAVE_BAT(3, 0x2c)
+ SAVE_BAT(4, 0x30)
+ SAVE_BAT(5, 0x34)
+ SAVE_BAT(6, 0x38)
+ SAVE_BAT(7, 0x3c)
+
+ SAVE_4SR(0, 0x40)
+ SAVE_4SR(4, 0x44)
+ SAVE_4SR(8, 0x48)
+ SAVE_4SR(12, 0x4c)
+
+ SAVE_SPRN(SPRG0, 0x50)
+ SAVE_SPRN(SPRG1, 0x51)
+ SAVE_SPRN(SPRG2, 0x52)
+ SAVE_SPRN(SPRG3, 0x53)
+ SAVE_SPRN(SPRG4, 0x54)
+ SAVE_SPRN(SPRG5, 0x55)
+ SAVE_SPRN(SPRG6, 0x56)
+ SAVE_SPRN(SPRG7, 0x57)
+
+ SAVE_SPRN(IABR, 0x58)
+ SAVE_SPRN(DABR, 0x59)
+ SAVE_SPRN(TBRL, 0x5a)
+ SAVE_SPRN(TBRU, 0x5b)
+
+ blr
+
+
+/* restore registers */
+#define LOAD_BAT(n, addr) \
+ LOAD_SPRN(DBAT##n##L, addr); \
+ LOAD_SPRN(DBAT##n##U, addr+1); \
+ LOAD_SPRN(IBAT##n##L, addr+2); \
+ LOAD_SPRN(IBAT##n##U, addr+3);
+
+#define LOAD_SR(n, addr) \
+ lwz r10, ((addr)*4)(r4); \
+ mtsr n, r10;
+
+#define LOAD_4SR(n, addr) \
+ LOAD_SR(n, addr); \
+ LOAD_SR(n+1, addr+1); \
+ LOAD_SR(n+2, addr+2); \
+ LOAD_SR(n+3, addr+3);
+
+restore_regs:
+ lis r4, registers@h
+ ori r4, r4, registers@l
+
+ /* MMU is not up yet */
+ subis r4, r4, CONFIG_KERNEL_START@h
+
+ lwz r0, 0(r4)
+ lwz r1, 0x4(r4)
+ lwz r2, 0x8(r4)
+ lmw r11, 0xc(r4)
+
+ /*
+ * these are a bit tricky
+ *
+ * 0x18 - HID0
+ * 0x19 - HID1
+ * 0x1a - HID2
+ * 0x1b - MSR
+ * 0x1c - LR
+ * 0x1d - reserved by 0xf0 (BDI2000)
+ */
+ LOAD_SPRN(RPA, 0x1e);
+ LOAD_SPRN(SDR1, 0x1f);
+
+ /* restore MMU regs */
+ LOAD_BAT(0, 0x20)
+ LOAD_BAT(1, 0x24)
+ LOAD_BAT(2, 0x28)
+ LOAD_BAT(3, 0x2c)
+ LOAD_BAT(4, 0x30)
+ LOAD_BAT(5, 0x34)
+ LOAD_BAT(6, 0x38)
+ LOAD_BAT(7, 0x3c)
+
+ LOAD_4SR(0, 0x40)
+ LOAD_4SR(4, 0x44)
+ LOAD_4SR(8, 0x48)
+ LOAD_4SR(12, 0x4c)
+
+ /* rest of regs */
+ LOAD_SPRN(SPRG0, 0x50);
+ LOAD_SPRN(SPRG1, 0x51);
+ LOAD_SPRN(SPRG2, 0x52);
+ LOAD_SPRN(SPRG3, 0x53);
+ LOAD_SPRN(SPRG4, 0x54);
+ LOAD_SPRN(SPRG5, 0x55);
+ LOAD_SPRN(SPRG6, 0x56);
+ LOAD_SPRN(SPRG7, 0x57);
+
+ LOAD_SPRN(IABR, 0x58);
+ LOAD_SPRN(DABR, 0x59);
+ LOAD_SPRN(TBWL, 0x5a); /* these two have separate R/W regs */
+ LOAD_SPRN(TBWU, 0x5b);
+
+ blr
+
+
+
+/* cache flushing code. copied from arch/ppc/boot/util.S */
+#define NUM_CACHE_LINES (128*8)
+
+/*
+ * Flush data cache
+ * Do this by just reading lots of stuff into the cache.
+ */
+ .globl flush_data_cache
+flush_data_cache:
+ lis r3,CONFIG_KERNEL_START@h
+ ori r3,r3,CONFIG_KERNEL_START@l
+ li r4,NUM_CACHE_LINES
+ mtctr r4
+1:
+ lwz r4,0(r3)
+ addi r3,r3,L1_CACHE_BYTES /* Next line, please */
+ bdnz 1b
+ blr
Index: work-powerpc.git/arch/powerpc/platforms/52xx/Makefile
===================================================================
--- work-powerpc.git.orig/arch/powerpc/platforms/52xx/Makefile
+++ work-powerpc.git/arch/powerpc/platforms/52xx/Makefile
@@ -10,3 +10,6 @@ obj-$(CONFIG_PPC_EFIKA) += efika.o
obj-$(CONFIG_PPC_LITE5200) += lite5200.o
obj-$(CONFIG_PM) += mpc52xx_sleep.o mpc52xx_pm.o
+ifeq ($(CONFIG_PPC_LITE5200),y)
+ obj-$(CONFIG_PM) += lite5200_sleep.o lite5200_pm.o
+endif
Index: work-powerpc.git/arch/powerpc/platforms/52xx/lite5200.c
===================================================================
--- work-powerpc.git.orig/arch/powerpc/platforms/52xx/lite5200.c
+++ work-powerpc.git/arch/powerpc/platforms/52xx/lite5200.c
@@ -86,7 +86,6 @@ error:
}
#ifdef CONFIG_PM
-static u32 descr_a;
static void lite5200_suspend_prepare(void __iomem *mbar)
{
u8 pin = 1; /* GPIO_WKUP_1 (GPIO_PSC2_4) */
@@ -97,13 +96,18 @@ static void lite5200_suspend_prepare(voi
* power down usb port
* this needs to be called before of-ohci suspend code
*/
- descr_a = in_be32(mbar + 0x1048);
- out_be32(mbar + 0x1048, (descr_a & ~0x200) | 0x100);
+
+ /* set ports to "power switched" and "powered at the same time"
+ * USB Rh descriptor A: NPS = 0, PSM = 0 */
+ out_be32(mbar + 0x1048, in_be32(mbar + 0x1048) & ~0x300);
+ /* USB Rh status: LPS = 1 - turn off power */
+ out_be32(mbar + 0x1050, 0x00000001);
}
static void lite5200_resume_finish(void __iomem *mbar)
{
- out_be32(mbar + 0x1048, descr_a);
+ /* USB Rh status: LPSC = 1 - turn on power */
+ out_be32(mbar + 0x1050, 0x00010000);
}
#endif
@@ -132,7 +136,7 @@ static void __init lite5200_setup_arch(v
#ifdef CONFIG_PM
mpc52xx_suspend.board_suspend_prepare = lite5200_suspend_prepare;
mpc52xx_suspend.board_resume_finish = lite5200_resume_finish;
- mpc52xx_pm_init();
+ lite5200_pm_init();
#endif
#ifdef CONFIG_PCI
Index: work-powerpc.git/include/asm-powerpc/mpc52xx.h
===================================================================
--- work-powerpc.git.orig/include/asm-powerpc/mpc52xx.h
+++ work-powerpc.git/include/asm-powerpc/mpc52xx.h
@@ -262,6 +262,16 @@ struct mpc52xx_suspend {
extern struct mpc52xx_suspend mpc52xx_suspend;
extern int __init mpc52xx_pm_init(void);
extern int mpc52xx_set_wakeup_gpio(u8 pin, u8 level);
+
+#ifdef CONFIG_PPC_LITE5200
+extern int __init lite5200_pm_init(void);
+
+/* lite5200 calls mpc5200 suspend functions, so here they are */
+extern int mpc52xx_pm_prepare(suspend_state_t);
+extern int mpc52xx_pm_enter(suspend_state_t);
+extern int mpc52xx_pm_finish(suspend_state_t);
+extern char saved_sram[0x4000]; /* reuse buffer from mpc52xx suspend */
+#endif
#endif /* CONFIG_PM */
#endif /* __ASM_POWERPC_MPC52xx_H__ */
^ permalink raw reply
* [PATCH] mpc52xx: sparse fixes
From: Domen Puncer @ 2007-07-09 7:52 UTC (permalink / raw)
To: linuxppc-embedded; +Cc: Sylvain Munaut
In-Reply-To: <20070621074507.GL23294@moe.telargo.com>
sparse caught these static functions / __iomem annotations
under arch/powerpc/platform/52xx/
Signed-off-by: Domen Puncer <domen.puncer@telargo.com>
---
Reposting, adding tnt to Cc, and requesting inclusion.
The patch is pretty much harmless :-)
arch/powerpc/platforms/52xx/efika.c | 4 ++--
arch/powerpc/platforms/52xx/lite5200.c | 2 +-
arch/powerpc/platforms/52xx/mpc52xx_pm.c | 8 ++++----
3 files changed, 7 insertions(+), 7 deletions(-)
Index: work-powerpc.git/arch/powerpc/platforms/52xx/efika.c
===================================================================
--- work-powerpc.git.orig/arch/powerpc/platforms/52xx/efika.c
+++ work-powerpc.git/arch/powerpc/platforms/52xx/efika.c
@@ -83,7 +83,7 @@ static struct pci_ops rtas_pci_ops = {
};
-void __init efika_pcisetup(void)
+static void __init efika_pcisetup(void)
{
const int *bus_range;
int len;
@@ -145,7 +145,7 @@ void __init efika_pcisetup(void)
}
#else
-void __init efika_pcisetup(void)
+static void __init efika_pcisetup(void)
{}
#endif
Index: work-powerpc.git/arch/powerpc/platforms/52xx/lite5200.c
===================================================================
--- work-powerpc.git.orig/arch/powerpc/platforms/52xx/lite5200.c
+++ work-powerpc.git/arch/powerpc/platforms/52xx/lite5200.c
@@ -156,7 +156,7 @@ static void __init lite5200_setup_arch(v
}
-void lite5200_show_cpuinfo(struct seq_file *m)
+static void lite5200_show_cpuinfo(struct seq_file *m)
{
struct device_node* np = of_find_all_nodes(NULL);
const char *model = NULL;
Index: work-powerpc.git/arch/powerpc/platforms/52xx/mpc52xx_pm.c
===================================================================
--- work-powerpc.git.orig/arch/powerpc/platforms/52xx/mpc52xx_pm.c
+++ work-powerpc.git/arch/powerpc/platforms/52xx/mpc52xx_pm.c
@@ -9,8 +9,8 @@
/* these are defined in mpc52xx_sleep.S, and only used here */
-extern void mpc52xx_deep_sleep(void *sram, void *sdram_regs,
- struct mpc52xx_cdm *, struct mpc52xx_intr *);
+extern void mpc52xx_deep_sleep(void __iomem *sram, void __iomem *sdram_regs,
+ struct mpc52xx_cdm __iomem *, struct mpc52xx_intr __iomem*);
extern void mpc52xx_ds_sram(void);
extern const long mpc52xx_ds_sram_size;
extern void mpc52xx_ds_cached(void);
@@ -21,7 +21,7 @@ static void __iomem *sdram;
static struct mpc52xx_cdm __iomem *cdm;
static struct mpc52xx_intr __iomem *intr;
static struct mpc52xx_gpio_wkup __iomem *gpiow;
-static void *sram;
+static void __iomem *sram;
static int sram_size;
struct mpc52xx_suspend mpc52xx_suspend;
@@ -100,7 +100,7 @@ int mpc52xx_pm_enter(suspend_state_t sta
u32 clk_enables;
u32 msr, hid0;
u32 intr_main_mask;
- void __iomem * irq_0x500 = (void *)CONFIG_KERNEL_START + 0x500;
+ void __iomem * irq_0x500 = (void __iomem *)CONFIG_KERNEL_START + 0x500;
unsigned long irq_0x500_stop = (unsigned long)irq_0x500 + mpc52xx_ds_cached_size;
char saved_0x500[mpc52xx_ds_cached_size];
^ permalink raw reply
* Re: [PATCH 2/5] Add legacy devices to mpc8641_hpcn.dts
From: Sergei Shtylyov @ 2007-07-09 12:43 UTC (permalink / raw)
To: Segher Boessenkool; +Cc: linuxppc-dev
In-Reply-To: <3D4AB802-A3D2-46CC-9BD1-1064EE0AC84C@kernel.crashing.org>
Hello.
Segher Boessenkool wrote:
>>>>> + 8042@60 {
>>>>> + device_type = "8042";
>>> Drop the device_type. A number as a name isn't
>>> all that great, either.
>> Yet it's kinda accepted years ago, see:
>> http://playground.sun.com/1275/proposals/Closed/Accepted/381-it.txt
> That's not a published recommendation or anything like that.
There's also this document with a status of "unapproved draft":
http://playground.sun.com/1275/bindings/devices/html/8042.html
So, all this never got final approval? Somewhat confusing (well, what one
could expect from OF? :-)
> Segher
WBR, Sergei
^ permalink raw reply
* [PATCH] [PPC]: Add linux/pagemap.h to arch/ppc/mm/tlb.c
From: Laurent Pinchart @ 2007-07-09 12:58 UTC (permalink / raw)
To: linuxppc-embedded
When compiled without swap support, arch/mm/tlb.c complains about missing
function declarations. This patch fixes the warnings.
Signed-off-by: Laurent Pinchart <laurent.pinchart@technotrade.biz>
---
arch/ppc/mm/tlb.c | 1 +
1 files changed, 1 insertions(+), 0 deletions(-)
diff --git a/arch/ppc/mm/tlb.c b/arch/ppc/mm/tlb.c
index fa29740..4ff260b 100644
--- a/arch/ppc/mm/tlb.c
+++ b/arch/ppc/mm/tlb.c
@@ -27,6 +27,7 @@
#include <linux/mm.h>
#include <linux/init.h>
#include <linux/highmem.h>
+#include <linux/pagemap.h>
#include <asm/tlbflush.h>
#include <asm/tlb.h>
--
1.5.0
^ permalink raw reply related
* [PATCH 00/13] IB/ehca: eHCA2 enablement & some fixes
From: Joachim Fenkes @ 2007-07-09 13:02 UTC (permalink / raw)
To: LinuxPPC-Dev, LKML, OF-General, Roland Dreier
Cc: Stefan Roscher, Christoph Raisch
This patch series enables the eHCA device driver to support new functions of
the eHCA2 chip. In addition, there are some bug fixes, code optimizations
and general new features included. Another set of patches will follow.
The patches, in detail, are:
[01/13] fixes a wrong parameter description
[02/13] adds HW capabilities autodetection
[03/13] restructures the QP code, preparing for Share Receive Queues (SRQ)
[04/13] adds SRQ support
[05/13] adds support for UD low latency QPs
[06/13] sets a flag that needs to be set on eHCA2
[07/13] adds RDMA atomic attributes to the data returned by query_qp()
[08/13] straightens out lock flag naming and adds static initializers
[09/13] refactors synchronization between completions and destroy_cq()
[10/13] changes the global idr spinlocks into rwlocks
[11/13] returns the QP pointer in poll_cq() instead of NULL
[12/13] adds notifications in case the SM LID etc. changes
[13/13] adds a slight latency improvement
The patches should apply cleanly, in order, against Roland's git. Please
review the changes and apply the patches for 2.6.23 if they are okay.
Regards,
Joachim
=2D-=20
Joachim Fenkes =A0-- =A0eHCA Linux Driver Developer and Hardware Tamer
IBM Deutschland Entwicklung GmbH =A0-- =A0Dept. 3627 (I/O Firmware Dev. 2)
Schoenaicher Strasse 220 =A0-- =A071032 Boeblingen =A0-- =A0Germany
eMail: fenkes@de.ibm.com
^ permalink raw reply
* [PATCH 01/13] IB/ehca: change scaling_code parameter description to match default value
From: Joachim Fenkes @ 2007-07-09 13:20 UTC (permalink / raw)
To: LinuxPPC-Dev, LKML, OF-General, Roland Dreier
Cc: Stefan Roscher, Christoph Raisch
In-Reply-To: <200707091502.22407.fenkes@de.ibm.com>
From: Hoang-Nam Nguyen <hnguyen@de.ibm.com>
Signed-off-by: Joachim Fenkes <fenkes@de.ibm.com>
---
drivers/infiniband/hw/ehca/ehca_main.c | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/drivers/infiniband/hw/ehca/ehca_main.c b/drivers/infiniband/hw/ehca/ehca_main.c
index c3f99f3..fea199f 100644
--- a/drivers/infiniband/hw/ehca/ehca_main.c
+++ b/drivers/infiniband/hw/ehca/ehca_main.c
@@ -94,7 +94,7 @@ MODULE_PARM_DESC(poll_all_eqs,
MODULE_PARM_DESC(static_rate,
"set permanent static rate (default: disabled)");
MODULE_PARM_DESC(scaling_code,
- "set scaling code (0: disabled, 1: enabled/default)");
+ "set scaling code (0: disabled/default, 1: enabled)");
spinlock_t ehca_qp_idr_lock;
spinlock_t ehca_cq_idr_lock;
--
1.5.2
^ permalink raw reply related
* [PATCH 02/13] IB/ehca: HW level, HW caps and MTU autodetection
From: Joachim Fenkes @ 2007-07-09 13:21 UTC (permalink / raw)
To: LinuxPPC-Dev, LKML, OF-General, Roland Dreier
Cc: Stefan Roscher, Christoph Raisch
In-Reply-To: <200707091502.22407.fenkes@de.ibm.com>
In preparation for support of new eHCA2 features, change adapter probing:
- Hardware level is changed to encode major and minor chip version
- Hardware capabilities are queried from the firmware
- The maximum MTU is queried from the firmware instead of assuming a
fixed value
Signed-off-by: Joachim Fenkes <fenkes@de.ibm.com>
---
drivers/infiniband/hw/ehca/ehca_av.c | 6 ++-
drivers/infiniband/hw/ehca/ehca_classes.h | 2 +
drivers/infiniband/hw/ehca/ehca_hca.c | 27 +++++++++++-
drivers/infiniband/hw/ehca/ehca_main.c | 62 ++++++++++++++++++++++++++---
drivers/infiniband/hw/ehca/hipz_hw.h | 18 ++++++++
5 files changed, 104 insertions(+), 11 deletions(-)
diff --git a/drivers/infiniband/hw/ehca/ehca_av.c b/drivers/infiniband/hw/ehca/ehca_av.c
index 0d6e2c4..3cd6bf3 100644
--- a/drivers/infiniband/hw/ehca/ehca_av.c
+++ b/drivers/infiniband/hw/ehca/ehca_av.c
@@ -118,7 +118,7 @@ struct ib_ah *ehca_create_ah(struct ib_pd *pd, struct ib_ah_attr *ah_attr)
}
memcpy(&av->av.grh.word_1, &gid, sizeof(gid));
}
- av->av.pmtu = EHCA_MAX_MTU;
+ av->av.pmtu = shca->max_mtu;
/* dgid comes in grh.word_3 */
memcpy(&av->av.grh.word_3, &ah_attr->grh.dgid,
@@ -137,6 +137,8 @@ int ehca_modify_ah(struct ib_ah *ah, struct ib_ah_attr *ah_attr)
struct ehca_av *av;
struct ehca_ud_av new_ehca_av;
struct ehca_pd *my_pd = container_of(ah->pd, struct ehca_pd, ib_pd);
+ struct ehca_shca *shca = container_of(ah->pd->device, struct ehca_shca,
+ ib_device);
u32 cur_pid = current->tgid;
if (my_pd->ib_pd.uobject && my_pd->ib_pd.uobject->context &&
@@ -192,7 +194,7 @@ int ehca_modify_ah(struct ib_ah *ah, struct ib_ah_attr *ah_attr)
memcpy(&new_ehca_av.grh.word_1, &gid, sizeof(gid));
}
- new_ehca_av.pmtu = EHCA_MAX_MTU;
+ new_ehca_av.pmtu = shca->max_mtu;
memcpy(&new_ehca_av.grh.word_3, &ah_attr->grh.dgid,
sizeof(ah_attr->grh.dgid));
diff --git a/drivers/infiniband/hw/ehca/ehca_classes.h b/drivers/infiniband/hw/ehca/ehca_classes.h
index 1d286d3..35d948f 100644
--- a/drivers/infiniband/hw/ehca/ehca_classes.h
+++ b/drivers/infiniband/hw/ehca/ehca_classes.h
@@ -107,6 +107,8 @@ struct ehca_shca {
struct ehca_pd *pd;
struct h_galpas galpas;
struct mutex modify_mutex;
+ u64 hca_cap;
+ int max_mtu;
};
struct ehca_pd {
diff --git a/drivers/infiniband/hw/ehca/ehca_hca.c b/drivers/infiniband/hw/ehca/ehca_hca.c
index 32b55a4..b310de5 100644
--- a/drivers/infiniband/hw/ehca/ehca_hca.c
+++ b/drivers/infiniband/hw/ehca/ehca_hca.c
@@ -45,11 +45,25 @@
int ehca_query_device(struct ib_device *ibdev, struct ib_device_attr *props)
{
- int ret = 0;
+ int i, ret = 0;
struct ehca_shca *shca = container_of(ibdev, struct ehca_shca,
ib_device);
struct hipz_query_hca *rblock;
+ static const u32 cap_mapping[] = {
+ IB_DEVICE_RESIZE_MAX_WR, HCA_CAP_WQE_RESIZE,
+ IB_DEVICE_BAD_PKEY_CNTR, HCA_CAP_BAD_P_KEY_CTR,
+ IB_DEVICE_BAD_QKEY_CNTR, HCA_CAP_Q_KEY_VIOL_CTR,
+ IB_DEVICE_RAW_MULTI, HCA_CAP_RAW_PACKET_MCAST,
+ IB_DEVICE_AUTO_PATH_MIG, HCA_CAP_AUTO_PATH_MIG,
+ IB_DEVICE_CHANGE_PHY_PORT, HCA_CAP_SQD_RTS_PORT_CHANGE,
+ IB_DEVICE_UD_AV_PORT_ENFORCE, HCA_CAP_AH_PORT_NR_CHECK,
+ IB_DEVICE_CURR_QP_STATE_MOD, HCA_CAP_CUR_QP_STATE_MOD,
+ IB_DEVICE_SHUTDOWN_PORT, HCA_CAP_SHUTDOWN_PORT,
+ IB_DEVICE_INIT_TYPE, HCA_CAP_INIT_TYPE,
+ IB_DEVICE_PORT_ACTIVE_EVENT, HCA_CAP_PORT_ACTIVE_EVENT,
+ };
+
rblock = ehca_alloc_fw_ctrlblock(GFP_KERNEL);
if (!rblock) {
ehca_err(&shca->ib_device, "Can't allocate rblock memory.");
@@ -96,6 +110,13 @@ int ehca_query_device(struct ib_device *ibdev, struct ib_device_attr *props)
props->max_total_mcast_qp_attach
= min_t(int, rblock->max_total_mcast_qp_attach, INT_MAX);
+ /* translate device capabilities */
+ props->device_cap_flags = IB_DEVICE_SYS_IMAGE_GUID |
+ IB_DEVICE_RC_RNR_NAK_GEN | IB_DEVICE_N_NOTIFY_CQ;
+ for (i = 0; i < ARRAY_SIZE(cap_mapping); i += 2)
+ if (rblock->hca_cap_indicators & cap_mapping[i + 1])
+ props->device_cap_flags |= cap_mapping[i];
+
query_device1:
ehca_free_fw_ctrlblock(rblock);
@@ -261,7 +282,7 @@ int ehca_modify_port(struct ib_device *ibdev,
}
if (mutex_lock_interruptible(&shca->modify_mutex))
- return -ERESTARTSYS;
+ return -ERESTARTSYS;
rblock = ehca_alloc_fw_ctrlblock(GFP_KERNEL);
if (!rblock) {
@@ -290,7 +311,7 @@ modify_port2:
ehca_free_fw_ctrlblock(rblock);
modify_port1:
- mutex_unlock(&shca->modify_mutex);
+ mutex_unlock(&shca->modify_mutex);
return ret;
}
diff --git a/drivers/infiniband/hw/ehca/ehca_main.c b/drivers/infiniband/hw/ehca/ehca_main.c
index fea199f..befbb9c 100644
--- a/drivers/infiniband/hw/ehca/ehca_main.c
+++ b/drivers/infiniband/hw/ehca/ehca_main.c
@@ -205,11 +205,35 @@ static void ehca_destroy_slab_caches(void)
#define EHCA_HCAAVER EHCA_BMASK_IBM(32,39)
#define EHCA_REVID EHCA_BMASK_IBM(40,63)
+static struct cap_descr {
+ u64 mask;
+ char *descr;
+} hca_cap_descr[] = {
+ { HCA_CAP_AH_PORT_NR_CHECK, "HCA_CAP_AH_PORT_NR_CHECK" },
+ { HCA_CAP_ATOMIC, "HCA_CAP_ATOMIC" },
+ { HCA_CAP_AUTO_PATH_MIG, "HCA_CAP_AUTO_PATH_MIG" },
+ { HCA_CAP_BAD_P_KEY_CTR, "HCA_CAP_BAD_P_KEY_CTR" },
+ { HCA_CAP_SQD_RTS_PORT_CHANGE, "HCA_CAP_SQD_RTS_PORT_CHANGE" },
+ { HCA_CAP_CUR_QP_STATE_MOD, "HCA_CAP_CUR_QP_STATE_MOD" },
+ { HCA_CAP_INIT_TYPE, "HCA_CAP_INIT_TYPE" },
+ { HCA_CAP_PORT_ACTIVE_EVENT, "HCA_CAP_PORT_ACTIVE_EVENT" },
+ { HCA_CAP_Q_KEY_VIOL_CTR, "HCA_CAP_Q_KEY_VIOL_CTR" },
+ { HCA_CAP_WQE_RESIZE, "HCA_CAP_WQE_RESIZE" },
+ { HCA_CAP_RAW_PACKET_MCAST, "HCA_CAP_RAW_PACKET_MCAST" },
+ { HCA_CAP_SHUTDOWN_PORT, "HCA_CAP_SHUTDOWN_PORT" },
+ { HCA_CAP_RC_LL_QP, "HCA_CAP_RC_LL_QP" },
+ { HCA_CAP_SRQ, "HCA_CAP_SRQ" },
+ { HCA_CAP_UD_LL_QP, "HCA_CAP_UD_LL_QP" },
+ { HCA_CAP_RESIZE_MR, "HCA_CAP_RESIZE_MR" },
+ { HCA_CAP_MINI_QP, "HCA_CAP_MINI_QP" },
+};
+
int ehca_sense_attributes(struct ehca_shca *shca)
{
- int ret = 0;
+ int i, ret = 0;
u64 h_ret;
struct hipz_query_hca *rblock;
+ struct hipz_query_port *port;
rblock = ehca_alloc_fw_ctrlblock(GFP_KERNEL);
if (!rblock) {
@@ -222,7 +246,7 @@ int ehca_sense_attributes(struct ehca_shca *shca)
ehca_gen_err("Cannot query device properties. h_ret=%lx",
h_ret);
ret = -EPERM;
- goto num_ports1;
+ goto sense_attributes1;
}
if (ehca_nr_ports == 1)
@@ -242,18 +266,44 @@ int ehca_sense_attributes(struct ehca_shca *shca)
ehca_gen_dbg(" ... hardware version=%x:%x", hcaaver, revid);
if ((hcaaver == 1) && (revid == 0))
- shca->hw_level = 0;
+ shca->hw_level = 0x11;
else if ((hcaaver == 1) && (revid == 1))
- shca->hw_level = 1;
+ shca->hw_level = 0x12;
else if ((hcaaver == 1) && (revid == 2))
- shca->hw_level = 2;
+ shca->hw_level = 0x13;
+ else if ((hcaaver == 2) && (revid == 0))
+ shca->hw_level = 0x21;
+ else if ((hcaaver == 2) && (revid == 0x10))
+ shca->hw_level = 0x22;
+ else {
+ ehca_gen_warn("unknown hardware version"
+ " - assuming default level");
+ shca->hw_level = 0x22;
+ }
}
ehca_gen_dbg(" ... hardware level=%x", shca->hw_level);
shca->sport[0].rate = IB_RATE_30_GBPS;
shca->sport[1].rate = IB_RATE_30_GBPS;
-num_ports1:
+ shca->hca_cap = rblock->hca_cap_indicators;
+ ehca_gen_dbg(" ... HCA capabilities:");
+ for (i = 0; i < ARRAY_SIZE(hca_cap_descr); i++)
+ if (EHCA_BMASK_GET(hca_cap_descr[i].mask, shca->hca_cap))
+ ehca_gen_dbg(" %s", hca_cap_descr[i].descr);
+
+ port = (struct hipz_query_port*)rblock;
+ h_ret = hipz_h_query_port(shca->ipz_hca_handle, 1, port);
+ if (h_ret != H_SUCCESS) {
+ ehca_gen_err("Cannot query port properties. h_ret=%lx",
+ h_ret);
+ ret = -EPERM;
+ goto sense_attributes1;
+ }
+
+ shca->max_mtu = port->max_mtu;
+
+sense_attributes1:
ehca_free_fw_ctrlblock(rblock);
return ret;
}
diff --git a/drivers/infiniband/hw/ehca/hipz_hw.h b/drivers/infiniband/hw/ehca/hipz_hw.h
index fad9136..9fe8367 100644
--- a/drivers/infiniband/hw/ehca/hipz_hw.h
+++ b/drivers/infiniband/hw/ehca/hipz_hw.h
@@ -360,6 +360,24 @@ struct hipz_query_hca {
u32 max_neq;
} __attribute__ ((packed));
+#define HCA_CAP_AH_PORT_NR_CHECK EHCA_BMASK_IBM(0,0)
+#define HCA_CAP_ATOMIC EHCA_BMASK_IBM(1,1)
+#define HCA_CAP_AUTO_PATH_MIG EHCA_BMASK_IBM(2,2)
+#define HCA_CAP_BAD_P_KEY_CTR EHCA_BMASK_IBM(3,3)
+#define HCA_CAP_SQD_RTS_PORT_CHANGE EHCA_BMASK_IBM(4,4)
+#define HCA_CAP_CUR_QP_STATE_MOD EHCA_BMASK_IBM(5,5)
+#define HCA_CAP_INIT_TYPE EHCA_BMASK_IBM(6,6)
+#define HCA_CAP_PORT_ACTIVE_EVENT EHCA_BMASK_IBM(7,7)
+#define HCA_CAP_Q_KEY_VIOL_CTR EHCA_BMASK_IBM(8,8)
+#define HCA_CAP_WQE_RESIZE EHCA_BMASK_IBM(9,9)
+#define HCA_CAP_RAW_PACKET_MCAST EHCA_BMASK_IBM(10,10)
+#define HCA_CAP_SHUTDOWN_PORT EHCA_BMASK_IBM(11,11)
+#define HCA_CAP_RC_LL_QP EHCA_BMASK_IBM(12,12)
+#define HCA_CAP_SRQ EHCA_BMASK_IBM(13,13)
+#define HCA_CAP_UD_LL_QP EHCA_BMASK_IBM(16,16)
+#define HCA_CAP_RESIZE_MR EHCA_BMASK_IBM(17,17)
+#define HCA_CAP_MINI_QP EHCA_BMASK_IBM(18,18)
+
/* query port response block */
struct hipz_query_port {
u32 state;
--
1.5.2
^ permalink raw reply related
* [PATCH 03/13] IB/ehca: QP code restructuring in preparation for SRQ
From: Joachim Fenkes @ 2007-07-09 13:23 UTC (permalink / raw)
To: LinuxPPC-Dev, LKML, OF-General, Roland Dreier
Cc: Stefan Roscher, Christoph Raisch
In-Reply-To: <200707091502.22407.fenkes@de.ibm.com>
- Replace init_qp_queues() by a shorter init_qp_queue(), eliminating
duplicate code.
- hipz_h_alloc_resource_qp() doesn't need a pointer to struct ehca_qp any
longer. All input and output data is transferred through the parms
parameter.
- Change the interface to also support SRQ.
Signed-off-by: Joachim Fenkes <fenkes@de.ibm.com>
---
drivers/infiniband/hw/ehca/ehca_classes.h | 46 +++++-
drivers/infiniband/hw/ehca/ehca_qp.c | 254 +++++++++++++----------------
drivers/infiniband/hw/ehca/hcp_if.c | 35 ++---
drivers/infiniband/hw/ehca/hcp_if.h | 1 -
4 files changed, 166 insertions(+), 170 deletions(-)
diff --git a/drivers/infiniband/hw/ehca/ehca_classes.h b/drivers/infiniband/hw/ehca/ehca_classes.h
index 35d948f..6e75db6 100644
--- a/drivers/infiniband/hw/ehca/ehca_classes.h
+++ b/drivers/infiniband/hw/ehca/ehca_classes.h
@@ -322,14 +322,49 @@ struct ehca_alloc_cq_parms {
struct ipz_eq_handle eq_handle;
};
+enum ehca_service_type {
+ ST_RC = 0,
+ ST_UC = 1,
+ ST_RD = 2,
+ ST_UD = 3,
+};
+
+enum ehca_ext_qp_type {
+ EQPT_NORMAL = 0,
+ EQPT_LLQP = 1,
+ EQPT_SRQBASE = 2,
+ EQPT_SRQ = 3,
+};
+
+enum ehca_ll_comp_flags {
+ LLQP_SEND_COMP = 0x20,
+ LLQP_RECV_COMP = 0x40,
+ LLQP_COMP_MASK = 0x60,
+};
+
struct ehca_alloc_qp_parms {
- int servicetype;
+/* input parameters */
+ enum ehca_service_type servicetype;
int sigtype;
- int daqp_ctrl;
- int max_send_sge;
- int max_recv_sge;
+ enum ehca_ext_qp_type ext_type;
+ enum ehca_ll_comp_flags ll_comp_flags;
+
+ int max_send_wr, max_recv_wr;
+ int max_send_sge, max_recv_sge;
int ud_av_l_key_ctl;
+ u32 token;
+ struct ipz_eq_handle eq_handle;
+ struct ipz_pd pd;
+ struct ipz_cq_handle send_cq_handle, recv_cq_handle;
+
+ u32 srq_qpn, srq_token, srq_limit;
+
+/* output parameters */
+ u32 real_qp_num;
+ struct ipz_qp_handle qp_handle;
+ struct h_galpas galpas;
+
u16 act_nr_send_wqes;
u16 act_nr_recv_wqes;
u8 act_nr_recv_sges;
@@ -337,9 +372,6 @@ struct ehca_alloc_qp_parms {
u32 nr_rq_pages;
u32 nr_sq_pages;
-
- struct ipz_eq_handle ipz_eq_handle;
- struct ipz_pd pd;
};
int ehca_cq_assign_qp(struct ehca_cq *cq, struct ehca_qp *qp);
diff --git a/drivers/infiniband/hw/ehca/ehca_qp.c b/drivers/infiniband/hw/ehca/ehca_qp.c
index b5bc787..ec1d555 100644
--- a/drivers/infiniband/hw/ehca/ehca_qp.c
+++ b/drivers/infiniband/hw/ehca/ehca_qp.c
@@ -234,13 +234,6 @@ static inline enum ib_qp_statetrans get_modqp_statetrans(int ib_fromstate,
return index;
}
-enum ehca_service_type {
- ST_RC = 0,
- ST_UC = 1,
- ST_RD = 2,
- ST_UD = 3
-};
-
/*
* ibqptype2servicetype returns hcp service type corresponding to given
* ib qp type used by create_qp()
@@ -268,15 +261,16 @@ static inline int ibqptype2servicetype(enum ib_qp_type ibqptype)
}
/*
- * init_qp_queues initializes/constructs r/squeue and registers queue pages.
+ * init_qp_queue initializes/constructs r/squeue and registers queue pages.
*/
-static inline int init_qp_queues(struct ehca_shca *shca,
- struct ehca_qp *my_qp,
- int nr_sq_pages,
- int nr_rq_pages,
- int swqe_size,
- int rwqe_size,
- int nr_send_sges, int nr_receive_sges)
+static inline int init_qp_queue(struct ehca_shca *shca,
+ struct ehca_qp *my_qp,
+ struct ipz_queue *queue,
+ int q_type,
+ u64 expected_hret,
+ int nr_q_pages,
+ int wqe_size,
+ int nr_sges)
{
int ret, cnt, ipz_rc;
void *vpage;
@@ -284,104 +278,63 @@ static inline int init_qp_queues(struct ehca_shca *shca,
struct ib_device *ib_dev = &shca->ib_device;
struct ipz_adapter_handle ipz_hca_handle = shca->ipz_hca_handle;
- ipz_rc = ipz_queue_ctor(&my_qp->ipz_squeue,
- nr_sq_pages,
- EHCA_PAGESIZE, swqe_size, nr_send_sges);
+ if (!nr_q_pages)
+ return 0;
+
+ ipz_rc = ipz_queue_ctor(queue, nr_q_pages, EHCA_PAGESIZE,
+ wqe_size, nr_sges);
if (!ipz_rc) {
- ehca_err(ib_dev,"Cannot allocate page for squeue. ipz_rc=%x",
+ ehca_err(ib_dev,"Cannot allocate page for queue. ipz_rc=%x",
ipz_rc);
return -EBUSY;
}
- ipz_rc = ipz_queue_ctor(&my_qp->ipz_rqueue,
- nr_rq_pages,
- EHCA_PAGESIZE, rwqe_size, nr_receive_sges);
- if (!ipz_rc) {
- ehca_err(ib_dev, "Cannot allocate page for rqueue. ipz_rc=%x",
- ipz_rc);
- ret = -EBUSY;
- goto init_qp_queues0;
- }
- /* register SQ pages */
- for (cnt = 0; cnt < nr_sq_pages; cnt++) {
- vpage = ipz_qpageit_get_inc(&my_qp->ipz_squeue);
+ /* register queue pages */
+ for (cnt = 0; cnt < nr_q_pages; cnt++) {
+ vpage = ipz_qpageit_get_inc(queue);
if (!vpage) {
- ehca_err(ib_dev, "SQ ipz_qpageit_get_inc() "
+ ehca_err(ib_dev, "ipz_qpageit_get_inc() "
"failed p_vpage= %p", vpage);
ret = -EINVAL;
- goto init_qp_queues1;
+ goto init_qp_queue1;
}
rpage = virt_to_abs(vpage);
h_ret = hipz_h_register_rpage_qp(ipz_hca_handle,
my_qp->ipz_qp_handle,
- &my_qp->pf, 0, 0,
+ NULL, 0, q_type,
rpage, 1,
my_qp->galpas.kernel);
- if (h_ret < H_SUCCESS) {
- ehca_err(ib_dev, "SQ hipz_qp_register_rpage()"
- " failed rc=%lx", h_ret);
- ret = ehca2ib_return_code(h_ret);
- goto init_qp_queues1;
- }
- }
-
- ipz_qeit_reset(&my_qp->ipz_squeue);
-
- /* register RQ pages */
- for (cnt = 0; cnt < nr_rq_pages; cnt++) {
- vpage = ipz_qpageit_get_inc(&my_qp->ipz_rqueue);
- if (!vpage) {
- ehca_err(ib_dev, "RQ ipz_qpageit_get_inc() "
- "failed p_vpage = %p", vpage);
- ret = -EINVAL;
- goto init_qp_queues1;
- }
-
- rpage = virt_to_abs(vpage);
-
- h_ret = hipz_h_register_rpage_qp(ipz_hca_handle,
- my_qp->ipz_qp_handle,
- &my_qp->pf, 0, 1,
- rpage, 1,my_qp->galpas.kernel);
- if (h_ret < H_SUCCESS) {
- ehca_err(ib_dev, "RQ hipz_qp_register_rpage() failed "
- "rc=%lx", h_ret);
- ret = ehca2ib_return_code(h_ret);
- goto init_qp_queues1;
- }
- if (cnt == (nr_rq_pages - 1)) { /* last page! */
- if (h_ret != H_SUCCESS) {
- ehca_err(ib_dev, "RQ hipz_qp_register_rpage() "
+ if (cnt == (nr_q_pages - 1)) { /* last page! */
+ if (h_ret != expected_hret) {
+ ehca_err(ib_dev, "hipz_qp_register_rpage() "
"h_ret= %lx ", h_ret);
ret = ehca2ib_return_code(h_ret);
- goto init_qp_queues1;
+ goto init_qp_queue1;
}
vpage = ipz_qpageit_get_inc(&my_qp->ipz_rqueue);
if (vpage) {
ehca_err(ib_dev, "ipz_qpageit_get_inc() "
"should not succeed vpage=%p", vpage);
ret = -EINVAL;
- goto init_qp_queues1;
+ goto init_qp_queue1;
}
} else {
if (h_ret != H_PAGE_REGISTERED) {
- ehca_err(ib_dev, "RQ hipz_qp_register_rpage() "
+ ehca_err(ib_dev, "hipz_qp_register_rpage() "
"h_ret= %lx ", h_ret);
ret = ehca2ib_return_code(h_ret);
- goto init_qp_queues1;
+ goto init_qp_queue1;
}
}
}
- ipz_qeit_reset(&my_qp->ipz_rqueue);
+ ipz_qeit_reset(queue);
return 0;
-init_qp_queues1:
- ipz_queue_dtor(&my_qp->ipz_rqueue);
-init_qp_queues0:
- ipz_queue_dtor(&my_qp->ipz_squeue);
+init_qp_queue1:
+ ipz_queue_dtor(queue);
return ret;
}
@@ -397,14 +350,17 @@ struct ib_qp *ehca_create_qp(struct ib_pd *pd,
ib_device);
struct ib_ucontext *context = NULL;
u64 h_ret;
- int max_send_sge, max_recv_sge, ret;
+ int is_llqp = 0, has_srq = 0;
+ int qp_type, max_send_sge, max_recv_sge, ret;
/* h_call's out parameters */
struct ehca_alloc_qp_parms parms;
u32 swqe_size = 0, rwqe_size = 0;
- u8 daqp_completion, isdaqp;
unsigned long flags;
+ memset(&parms, 0, sizeof(parms));
+ qp_type = init_attr->qp_type;
+
if (init_attr->sq_sig_type != IB_SIGNAL_REQ_WR &&
init_attr->sq_sig_type != IB_SIGNAL_ALL_WR) {
ehca_err(pd->device, "init_attr->sg_sig_type=%x not allowed",
@@ -412,38 +368,47 @@ struct ib_qp *ehca_create_qp(struct ib_pd *pd,
return ERR_PTR(-EINVAL);
}
- /* save daqp completion bits */
- daqp_completion = init_attr->qp_type & 0x60;
- /* save daqp bit */
- isdaqp = (init_attr->qp_type & 0x80) ? 1 : 0;
- init_attr->qp_type = init_attr->qp_type & 0x1F;
+ /* save LLQP info */
+ if (qp_type & 0x80) {
+ is_llqp = 1;
+ parms.ext_type = EQPT_LLQP;
+ parms.ll_comp_flags = qp_type & LLQP_COMP_MASK;
+ }
+ qp_type &= 0x1F;
+
+ /* check for SRQ */
+ has_srq = !!(init_attr->srq);
+ if (is_llqp && has_srq) {
+ ehca_err(pd->device, "LLQPs can't have an SRQ");
+ return ERR_PTR(-EINVAL);
+ }
- if (init_attr->qp_type != IB_QPT_UD &&
- init_attr->qp_type != IB_QPT_SMI &&
- init_attr->qp_type != IB_QPT_GSI &&
- init_attr->qp_type != IB_QPT_UC &&
- init_attr->qp_type != IB_QPT_RC) {
- ehca_err(pd->device, "wrong QP Type=%x", init_attr->qp_type);
+ /* check QP type */
+ if (qp_type != IB_QPT_UD &&
+ qp_type != IB_QPT_UC &&
+ qp_type != IB_QPT_RC &&
+ qp_type != IB_QPT_SMI &&
+ qp_type != IB_QPT_GSI) {
+ ehca_err(pd->device, "wrong QP Type=%x", qp_type);
return ERR_PTR(-EINVAL);
}
- if ((init_attr->qp_type != IB_QPT_RC && init_attr->qp_type != IB_QPT_UD)
- && isdaqp) {
- ehca_err(pd->device, "unsupported LL QP Type=%x",
- init_attr->qp_type);
+
+ if (is_llqp && (qp_type != IB_QPT_RC && qp_type != IB_QPT_UD)) {
+ ehca_err(pd->device, "unsupported LL QP Type=%x", qp_type);
return ERR_PTR(-EINVAL);
- } else if (init_attr->qp_type == IB_QPT_RC && isdaqp &&
+ } else if (is_llqp && qp_type == IB_QPT_RC &&
(init_attr->cap.max_send_wr > 255 ||
init_attr->cap.max_recv_wr > 255 )) {
- ehca_err(pd->device, "Invalid Number of max_sq_wr =%x "
- "or max_rq_wr=%x for QP Type=%x",
- init_attr->cap.max_send_wr,
- init_attr->cap.max_recv_wr,init_attr->qp_type);
- return ERR_PTR(-EINVAL);
- } else if (init_attr->qp_type == IB_QPT_UD && isdaqp &&
- init_attr->cap.max_send_wr > 255) {
+ ehca_err(pd->device, "Invalid Number of max_sq_wr=%x "
+ "or max_rq_wr=%x for RC LLQP",
+ init_attr->cap.max_send_wr,
+ init_attr->cap.max_recv_wr);
+ return ERR_PTR(-EINVAL);
+ } else if (is_llqp && qp_type == IB_QPT_UD &&
+ init_attr->cap.max_send_wr > 255) {
ehca_err(pd->device,
"Invalid Number of max_send_wr=%x for UD QP_TYPE=%x",
- init_attr->cap.max_send_wr, init_attr->qp_type);
+ init_attr->cap.max_send_wr, qp_type);
return ERR_PTR(-EINVAL);
}
@@ -456,7 +421,6 @@ struct ib_qp *ehca_create_qp(struct ib_pd *pd,
return ERR_PTR(-ENOMEM);
}
- memset (&parms, 0, sizeof(struct ehca_alloc_qp_parms));
spin_lock_init(&my_qp->spinlock_s);
spin_lock_init(&my_qp->spinlock_r);
@@ -465,8 +429,6 @@ struct ib_qp *ehca_create_qp(struct ib_pd *pd,
my_qp->send_cq =
container_of(init_attr->send_cq, struct ehca_cq, ib_cq);
- my_qp->init_attr = *init_attr;
-
do {
if (!idr_pre_get(&ehca_qp_idr, GFP_KERNEL)) {
ret = -ENOMEM;
@@ -486,10 +448,10 @@ struct ib_qp *ehca_create_qp(struct ib_pd *pd,
goto create_qp_exit0;
}
- parms.servicetype = ibqptype2servicetype(init_attr->qp_type);
+ parms.servicetype = ibqptype2servicetype(qp_type);
if (parms.servicetype < 0) {
ret = -EINVAL;
- ehca_err(pd->device, "Invalid qp_type=%x", init_attr->qp_type);
+ ehca_err(pd->device, "Invalid qp_type=%x", qp_type);
goto create_qp_exit0;
}
@@ -501,21 +463,23 @@ struct ib_qp *ehca_create_qp(struct ib_pd *pd,
/* UD_AV CIRCUMVENTION */
max_send_sge = init_attr->cap.max_send_sge;
max_recv_sge = init_attr->cap.max_recv_sge;
- if (IB_QPT_UD == init_attr->qp_type ||
- IB_QPT_GSI == init_attr->qp_type ||
- IB_QPT_SMI == init_attr->qp_type) {
+ if (parms.servicetype == ST_UD) {
max_send_sge += 2;
max_recv_sge += 2;
}
- parms.ipz_eq_handle = shca->eq.ipz_eq_handle;
- parms.daqp_ctrl = isdaqp | daqp_completion;
+ parms.token = my_qp->token;
+ parms.eq_handle = shca->eq.ipz_eq_handle;
parms.pd = my_pd->fw_pd;
- parms.max_recv_sge = max_recv_sge;
- parms.max_send_sge = max_send_sge;
+ parms.send_cq_handle = my_qp->send_cq->ipz_cq_handle;
+ parms.recv_cq_handle = my_qp->recv_cq->ipz_cq_handle;
- h_ret = hipz_h_alloc_resource_qp(shca->ipz_hca_handle, my_qp, &parms);
+ parms.max_send_wr = init_attr->cap.max_send_wr;
+ parms.max_recv_wr = init_attr->cap.max_recv_wr;
+ parms.max_send_sge = max_send_sge;
+ parms.max_recv_sge = max_recv_sge;
+ h_ret = hipz_h_alloc_resource_qp(shca->ipz_hca_handle, &parms);
if (h_ret != H_SUCCESS) {
ehca_err(pd->device, "h_alloc_resource_qp() failed h_ret=%lx",
h_ret);
@@ -523,16 +487,18 @@ struct ib_qp *ehca_create_qp(struct ib_pd *pd,
goto create_qp_exit1;
}
- my_qp->ib_qp.qp_num = my_qp->real_qp_num;
+ my_qp->ib_qp.qp_num = my_qp->real_qp_num = parms.real_qp_num;
+ my_qp->ipz_qp_handle = parms.qp_handle;
+ my_qp->galpas = parms.galpas;
- switch (init_attr->qp_type) {
+ switch (qp_type) {
case IB_QPT_RC:
- if (isdaqp == 0) {
+ if (!is_llqp) {
swqe_size = offsetof(struct ehca_wqe, u.nud.sg_list[
(parms.act_nr_send_sges)]);
rwqe_size = offsetof(struct ehca_wqe, u.nud.sg_list[
(parms.act_nr_recv_sges)]);
- } else { /* for daqp we need to use msg size, not wqe size */
+ } else { /* for LLQP we need to use msg size, not wqe size */
swqe_size = da_rc_msg_size[max_send_sge];
rwqe_size = da_rc_msg_size[max_recv_sge];
parms.act_nr_send_sges = 1;
@@ -552,7 +518,7 @@ struct ib_qp *ehca_create_qp(struct ib_pd *pd,
/* UD circumvention */
parms.act_nr_recv_sges -= 2;
parms.act_nr_send_sges -= 2;
- if (isdaqp) {
+ if (is_llqp) {
swqe_size = da_ud_sq_msg_size[max_send_sge];
rwqe_size = da_rc_msg_size[max_recv_sge];
parms.act_nr_send_sges = 1;
@@ -564,14 +530,12 @@ struct ib_qp *ehca_create_qp(struct ib_pd *pd,
u.ud_av.sg_list[parms.act_nr_recv_sges]);
}
- if (IB_QPT_GSI == init_attr->qp_type ||
- IB_QPT_SMI == init_attr->qp_type) {
+ if (IB_QPT_GSI == qp_type || IB_QPT_SMI == qp_type) {
parms.act_nr_send_wqes = init_attr->cap.max_send_wr;
parms.act_nr_recv_wqes = init_attr->cap.max_recv_wr;
parms.act_nr_send_sges = init_attr->cap.max_send_sge;
parms.act_nr_recv_sges = init_attr->cap.max_recv_sge;
- my_qp->ib_qp.qp_num =
- (init_attr->qp_type == IB_QPT_SMI) ? 0 : 1;
+ my_qp->ib_qp.qp_num = (qp_type == IB_QPT_SMI) ? 0 : 1;
}
break;
@@ -580,26 +544,33 @@ struct ib_qp *ehca_create_qp(struct ib_pd *pd,
break;
}
- /* initializes r/squeue and registers queue pages */
- ret = init_qp_queues(shca, my_qp,
- parms.nr_sq_pages, parms.nr_rq_pages,
- swqe_size, rwqe_size,
- parms.act_nr_send_sges, parms.act_nr_recv_sges);
+ /* initialize r/squeue and register queue pages */
+ ret = init_qp_queue(shca, my_qp, &my_qp->ipz_squeue, 0,
+ has_srq ? H_SUCCESS : H_PAGE_REGISTERED,
+ parms.nr_sq_pages, swqe_size,
+ parms.act_nr_send_sges);
if (ret) {
ehca_err(pd->device,
- "Couldn't initialize r/squeue and pages ret=%x", ret);
+ "Couldn't initialize squeue and pages ret=%x", ret);
goto create_qp_exit2;
}
+ ret = init_qp_queue(shca, my_qp, &my_qp->ipz_rqueue, 1, H_SUCCESS,
+ parms.nr_rq_pages, rwqe_size,
+ parms.act_nr_recv_sges);
+ if (ret) {
+ ehca_err(pd->device,
+ "Couldn't initialize rqueue and pages ret=%x", ret);
+ goto create_qp_exit3;
+ }
+
my_qp->ib_qp.pd = &my_pd->ib_pd;
my_qp->ib_qp.device = my_pd->ib_pd.device;
my_qp->ib_qp.recv_cq = init_attr->recv_cq;
my_qp->ib_qp.send_cq = init_attr->send_cq;
- my_qp->ib_qp.qp_type = init_attr->qp_type;
-
- my_qp->qp_type = init_attr->qp_type;
+ my_qp->ib_qp.qp_type = my_qp->qp_type = qp_type;
my_qp->ib_qp.srq = init_attr->srq;
my_qp->ib_qp.qp_context = init_attr->qp_context;
@@ -610,15 +581,16 @@ struct ib_qp *ehca_create_qp(struct ib_pd *pd,
init_attr->cap.max_recv_wr = parms.act_nr_recv_wqes;
init_attr->cap.max_send_sge = parms.act_nr_send_sges;
init_attr->cap.max_send_wr = parms.act_nr_send_wqes;
+ my_qp->init_attr = *init_attr;
/* NOTE: define_apq0() not supported yet */
- if (init_attr->qp_type == IB_QPT_GSI) {
+ if (qp_type == IB_QPT_GSI) {
h_ret = ehca_define_sqp(shca, my_qp, init_attr);
if (h_ret != H_SUCCESS) {
ehca_err(pd->device, "ehca_define_sqp() failed rc=%lx",
h_ret);
ret = ehca2ib_return_code(h_ret);
- goto create_qp_exit3;
+ goto create_qp_exit4;
}
}
if (init_attr->send_cq) {
@@ -628,7 +600,7 @@ struct ib_qp *ehca_create_qp(struct ib_pd *pd,
if (ret) {
ehca_err(pd->device, "Couldn't assign qp to send_cq ret=%x",
ret);
- goto create_qp_exit3;
+ goto create_qp_exit4;
}
my_qp->send_cq = cq;
}
@@ -659,14 +631,16 @@ struct ib_qp *ehca_create_qp(struct ib_pd *pd,
if (ib_copy_to_udata(udata, &resp, sizeof resp)) {
ehca_err(pd->device, "Copy to udata failed");
ret = -EINVAL;
- goto create_qp_exit3;
+ goto create_qp_exit4;
}
}
return &my_qp->ib_qp;
-create_qp_exit3:
+create_qp_exit4:
ipz_queue_dtor(&my_qp->ipz_rqueue);
+
+create_qp_exit3:
ipz_queue_dtor(&my_qp->ipz_squeue);
create_qp_exit2:
diff --git a/drivers/infiniband/hw/ehca/hcp_if.c b/drivers/infiniband/hw/ehca/hcp_if.c
index 5766ae3..7efc4a2 100644
--- a/drivers/infiniband/hw/ehca/hcp_if.c
+++ b/drivers/infiniband/hw/ehca/hcp_if.c
@@ -74,11 +74,6 @@
#define H_MP_SHUTDOWN EHCA_BMASK_IBM(48, 48)
#define H_MP_RESET_QKEY_CTR EHCA_BMASK_IBM(49, 49)
-/* direct access qp controls */
-#define DAQP_CTRL_ENABLE 0x01
-#define DAQP_CTRL_SEND_COMP 0x20
-#define DAQP_CTRL_RECV_COMP 0x40
-
static u32 get_longbusy_msecs(int longbusy_rc)
{
switch (longbusy_rc) {
@@ -284,36 +279,31 @@ u64 hipz_h_alloc_resource_cq(const struct ipz_adapter_handle adapter_handle,
}
u64 hipz_h_alloc_resource_qp(const struct ipz_adapter_handle adapter_handle,
- struct ehca_qp *qp,
struct ehca_alloc_qp_parms *parms)
{
u64 ret;
u64 allocate_controls;
u64 max_r10_reg;
u64 outs[PLPAR_HCALL9_BUFSIZE];
- u16 max_nr_receive_wqes = qp->init_attr.cap.max_recv_wr + 1;
- u16 max_nr_send_wqes = qp->init_attr.cap.max_send_wr + 1;
- int daqp_ctrl = parms->daqp_ctrl;
allocate_controls =
- EHCA_BMASK_SET(H_ALL_RES_QP_ENHANCED_OPS,
- (daqp_ctrl & DAQP_CTRL_ENABLE) ? 1 : 0)
+ EHCA_BMASK_SET(H_ALL_RES_QP_ENHANCED_OPS, parms->ext_type)
| EHCA_BMASK_SET(H_ALL_RES_QP_PTE_PIN, 0)
| EHCA_BMASK_SET(H_ALL_RES_QP_SERVICE_TYPE, parms->servicetype)
| EHCA_BMASK_SET(H_ALL_RES_QP_SIGNALING_TYPE, parms->sigtype)
| EHCA_BMASK_SET(H_ALL_RES_QP_LL_RQ_CQE_POSTING,
- (daqp_ctrl & DAQP_CTRL_RECV_COMP) ? 1 : 0)
+ !!(parms->ll_comp_flags & LLQP_RECV_COMP))
| EHCA_BMASK_SET(H_ALL_RES_QP_LL_SQ_CQE_POSTING,
- (daqp_ctrl & DAQP_CTRL_SEND_COMP) ? 1 : 0)
+ !!(parms->ll_comp_flags & LLQP_SEND_COMP))
| EHCA_BMASK_SET(H_ALL_RES_QP_UD_AV_LKEY_CTRL,
parms->ud_av_l_key_ctl)
| EHCA_BMASK_SET(H_ALL_RES_QP_RESOURCE_TYPE, 1);
max_r10_reg =
EHCA_BMASK_SET(H_ALL_RES_QP_MAX_OUTST_SEND_WR,
- max_nr_send_wqes)
+ parms->max_send_wr + 1)
| EHCA_BMASK_SET(H_ALL_RES_QP_MAX_OUTST_RECV_WR,
- max_nr_receive_wqes)
+ parms->max_recv_wr + 1)
| EHCA_BMASK_SET(H_ALL_RES_QP_MAX_SEND_SGE,
parms->max_send_sge)
| EHCA_BMASK_SET(H_ALL_RES_QP_MAX_RECV_SGE,
@@ -322,15 +312,16 @@ u64 hipz_h_alloc_resource_qp(const struct ipz_adapter_handle adapter_handle,
ret = ehca_plpar_hcall9(H_ALLOC_RESOURCE, outs,
adapter_handle.handle, /* r4 */
allocate_controls, /* r5 */
- qp->send_cq->ipz_cq_handle.handle,
- qp->recv_cq->ipz_cq_handle.handle,
- parms->ipz_eq_handle.handle,
- ((u64)qp->token << 32) | parms->pd.value,
+ parms->send_cq_handle.handle,
+ parms->recv_cq_handle.handle,
+ parms->eq_handle.handle,
+ ((u64)parms->token << 32) | parms->pd.value,
max_r10_reg, /* r10 */
parms->ud_av_l_key_ctl, /* r11 */
0);
- qp->ipz_qp_handle.handle = outs[0];
- qp->real_qp_num = (u32)outs[1];
+
+ parms->qp_handle.handle = outs[0];
+ parms->real_qp_num = (u32)outs[1];
parms->act_nr_send_wqes =
(u16)EHCA_BMASK_GET(H_ALL_RES_QP_ACT_OUTST_SEND_WR, outs[2]);
parms->act_nr_recv_wqes =
@@ -345,7 +336,7 @@ u64 hipz_h_alloc_resource_qp(const struct ipz_adapter_handle adapter_handle,
(u32)EHCA_BMASK_GET(H_ALL_RES_QP_RQUEUE_SIZE_PAGES, outs[4]);
if (ret == H_SUCCESS)
- hcp_galpas_ctor(&qp->galpas, outs[6], outs[6]);
+ hcp_galpas_ctor(&parms->galpas, outs[6], outs[6]);
if (ret == H_NOT_ENOUGH_RESOURCES)
ehca_gen_err("Not enough resources. ret=%lx", ret);
diff --git a/drivers/infiniband/hw/ehca/hcp_if.h b/drivers/infiniband/hw/ehca/hcp_if.h
index 2869f7d..60ce02b 100644
--- a/drivers/infiniband/hw/ehca/hcp_if.h
+++ b/drivers/infiniband/hw/ehca/hcp_if.h
@@ -78,7 +78,6 @@ u64 hipz_h_alloc_resource_cq(const struct ipz_adapter_handle adapter_handle,
* initialize resources, create empty QPPTs (2 rings).
*/
u64 hipz_h_alloc_resource_qp(const struct ipz_adapter_handle adapter_handle,
- struct ehca_qp *qp,
struct ehca_alloc_qp_parms *parms);
u64 hipz_h_query_port(const struct ipz_adapter_handle adapter_handle,
--
1.5.2
^ permalink raw reply related
* [PATCH 04/13] IB/ehca: add Shared Receive Queue support
From: Joachim Fenkes @ 2007-07-09 13:25 UTC (permalink / raw)
To: LinuxPPC-Dev, LKML, OF-General, Roland Dreier
Cc: Stefan Roscher, Christoph Raisch
In-Reply-To: <200707091502.22407.fenkes@de.ibm.com>
Support SRQs on eHCA2. Since an SRQ is a QP for eHCA2, a lot of code
(structures, create, destroy, post_recv) can be shared between QP and SRQ.
Signed-off-by: Joachim Fenkes <fenkes@de.ibm.com>
---
drivers/infiniband/hw/ehca/ehca_classes.h | 26 +-
drivers/infiniband/hw/ehca/ehca_classes_pSeries.h | 4 +-
drivers/infiniband/hw/ehca/ehca_iverbs.h | 15 +
drivers/infiniband/hw/ehca/ehca_main.c | 16 +-
drivers/infiniband/hw/ehca/ehca_qp.c | 451 +++++++++++++++++----
drivers/infiniband/hw/ehca/ehca_reqs.c | 47 ++-
drivers/infiniband/hw/ehca/ehca_uverbs.c | 4 +-
drivers/infiniband/hw/ehca/hcp_if.c | 23 +-
drivers/infiniband/hw/ehca/hipz_hw.h | 1 +
9 files changed, 480 insertions(+), 107 deletions(-)
diff --git a/drivers/infiniband/hw/ehca/ehca_classes.h b/drivers/infiniband/hw/ehca/ehca_classes.h
index 6e75db6..9d689ae 100644
--- a/drivers/infiniband/hw/ehca/ehca_classes.h
+++ b/drivers/infiniband/hw/ehca/ehca_classes.h
@@ -5,6 +5,7 @@
*
* Authors: Heiko J Schick <schickhj@de.ibm.com>
* Christoph Raisch <raisch@de.ibm.com>
+ * Joachim Fenkes <fenkes@de.ibm.com>
*
* Copyright (c) 2005 IBM Corporation
*
@@ -117,9 +118,20 @@ struct ehca_pd {
u32 ownpid;
};
+enum ehca_ext_qp_type {
+ EQPT_NORMAL = 0,
+ EQPT_LLQP = 1,
+ EQPT_SRQBASE = 2,
+ EQPT_SRQ = 3,
+};
+
struct ehca_qp {
- struct ib_qp ib_qp;
+ union {
+ struct ib_qp ib_qp;
+ struct ib_srq ib_srq;
+ };
u32 qp_type;
+ enum ehca_ext_qp_type ext_type;
struct ipz_queue ipz_squeue;
struct ipz_queue ipz_rqueue;
struct h_galpas galpas;
@@ -142,6 +154,10 @@ struct ehca_qp {
u32 mm_count_galpa;
};
+#define IS_SRQ(qp) (qp->ext_type == EQPT_SRQ)
+#define HAS_SQ(qp) (qp->ext_type != EQPT_SRQ)
+#define HAS_RQ(qp) (qp->ext_type != EQPT_SRQBASE)
+
/* must be power of 2 */
#define QP_HASHTAB_LEN 8
@@ -307,6 +323,7 @@ struct ehca_create_qp_resp {
u32 qp_num;
u32 token;
u32 qp_type;
+ u32 ext_type;
u32 qkey;
/* qp_num assigned by ehca: sqp0/1 may have got different numbers */
u32 real_qp_num;
@@ -329,13 +346,6 @@ enum ehca_service_type {
ST_UD = 3,
};
-enum ehca_ext_qp_type {
- EQPT_NORMAL = 0,
- EQPT_LLQP = 1,
- EQPT_SRQBASE = 2,
- EQPT_SRQ = 3,
-};
-
enum ehca_ll_comp_flags {
LLQP_SEND_COMP = 0x20,
LLQP_RECV_COMP = 0x40,
diff --git a/drivers/infiniband/hw/ehca/ehca_classes_pSeries.h b/drivers/infiniband/hw/ehca/ehca_classes_pSeries.h
index 5665f21..fb3df5c 100644
--- a/drivers/infiniband/hw/ehca/ehca_classes_pSeries.h
+++ b/drivers/infiniband/hw/ehca/ehca_classes_pSeries.h
@@ -228,8 +228,8 @@ struct hcp_modify_qp_control_block {
#define MQPCB_QP_NUMBER EHCA_BMASK_IBM(8,31)
#define MQPCB_MASK_QP_ENABLE EHCA_BMASK_IBM(48,48)
#define MQPCB_QP_ENABLE EHCA_BMASK_IBM(31,31)
-#define MQPCB_MASK_CURR_SQR_LIMIT EHCA_BMASK_IBM(49,49)
-#define MQPCB_CURR_SQR_LIMIT EHCA_BMASK_IBM(15,31)
+#define MQPCB_MASK_CURR_SRQ_LIMIT EHCA_BMASK_IBM(49,49)
+#define MQPCB_CURR_SRQ_LIMIT EHCA_BMASK_IBM(16,31)
#define MQPCB_MASK_QP_AFF_ASYN_EV_LOG_REG EHCA_BMASK_IBM(50,50)
#define MQPCB_MASK_SHARED_RQ_HNDL EHCA_BMASK_IBM(51,51)
diff --git a/drivers/infiniband/hw/ehca/ehca_iverbs.h b/drivers/infiniband/hw/ehca/ehca_iverbs.h
index 37e7fe0..fd84a80 100644
--- a/drivers/infiniband/hw/ehca/ehca_iverbs.h
+++ b/drivers/infiniband/hw/ehca/ehca_iverbs.h
@@ -154,6 +154,21 @@ int ehca_post_send(struct ib_qp *qp, struct ib_send_wr *send_wr,
int ehca_post_recv(struct ib_qp *qp, struct ib_recv_wr *recv_wr,
struct ib_recv_wr **bad_recv_wr);
+int ehca_post_srq_recv(struct ib_srq *srq,
+ struct ib_recv_wr *recv_wr,
+ struct ib_recv_wr **bad_recv_wr);
+
+struct ib_srq *ehca_create_srq(struct ib_pd *pd,
+ struct ib_srq_init_attr *init_attr,
+ struct ib_udata *udata);
+
+int ehca_modify_srq(struct ib_srq *srq, struct ib_srq_attr *attr,
+ enum ib_srq_attr_mask attr_mask, struct ib_udata *udata);
+
+int ehca_query_srq(struct ib_srq *srq, struct ib_srq_attr *srq_attr);
+
+int ehca_destroy_srq(struct ib_srq *srq);
+
u64 ehca_define_sqp(struct ehca_shca *shca, struct ehca_qp *ibqp,
struct ib_qp_init_attr *qp_init_attr);
diff --git a/drivers/infiniband/hw/ehca/ehca_main.c b/drivers/infiniband/hw/ehca/ehca_main.c
index befbb9c..9bd749c 100644
--- a/drivers/infiniband/hw/ehca/ehca_main.c
+++ b/drivers/infiniband/hw/ehca/ehca_main.c
@@ -343,7 +343,7 @@ int ehca_init_device(struct ehca_shca *shca)
strlcpy(shca->ib_device.name, "ehca%d", IB_DEVICE_NAME_MAX);
shca->ib_device.owner = THIS_MODULE;
- shca->ib_device.uverbs_abi_ver = 6;
+ shca->ib_device.uverbs_abi_ver = 7;
shca->ib_device.uverbs_cmd_mask =
(1ull << IB_USER_VERBS_CMD_GET_CONTEXT) |
(1ull << IB_USER_VERBS_CMD_QUERY_DEVICE) |
@@ -411,6 +411,20 @@ int ehca_init_device(struct ehca_shca *shca)
/* shca->ib_device.process_mad = ehca_process_mad; */
shca->ib_device.mmap = ehca_mmap;
+ if (EHCA_BMASK_GET(HCA_CAP_SRQ, shca->hca_cap)) {
+ shca->ib_device.uverbs_cmd_mask |=
+ (1ull << IB_USER_VERBS_CMD_CREATE_SRQ) |
+ (1ull << IB_USER_VERBS_CMD_MODIFY_SRQ) |
+ (1ull << IB_USER_VERBS_CMD_QUERY_SRQ) |
+ (1ull << IB_USER_VERBS_CMD_DESTROY_SRQ);
+
+ shca->ib_device.create_srq = ehca_create_srq;
+ shca->ib_device.modify_srq = ehca_modify_srq;
+ shca->ib_device.query_srq = ehca_query_srq;
+ shca->ib_device.destroy_srq = ehca_destroy_srq;
+ shca->ib_device.post_srq_recv = ehca_post_srq_recv;
+ }
+
return ret;
}
diff --git a/drivers/infiniband/hw/ehca/ehca_qp.c b/drivers/infiniband/hw/ehca/ehca_qp.c
index ec1d555..9486a44 100644
--- a/drivers/infiniband/hw/ehca/ehca_qp.c
+++ b/drivers/infiniband/hw/ehca/ehca_qp.c
@@ -3,7 +3,9 @@
*
* QP functions
*
- * Authors: Waleri Fomin <fomin@de.ibm.com>
+ * Authors: Joachim Fenkes <fenkes@de.ibm.com>
+ * Stefan Roscher <stefan.roscher@de.ibm.com>
+ * Waleri Fomin <fomin@de.ibm.com>
* Hoang-Nam Nguyen <hnguyen@de.ibm.com>
* Reinhard Ernst <rernst@de.ibm.com>
* Heiko J Schick <schickhj@de.ibm.com>
@@ -261,6 +263,19 @@ static inline int ibqptype2servicetype(enum ib_qp_type ibqptype)
}
/*
+ * init userspace queue info from ipz_queue data
+ */
+static inline void queue2resp(struct ipzu_queue_resp *resp,
+ struct ipz_queue *queue)
+{
+ resp->qe_size = queue->qe_size;
+ resp->act_nr_of_sg = queue->act_nr_of_sg;
+ resp->queue_length = queue->queue_length;
+ resp->pagesize = queue->pagesize;
+ resp->toggle_state = queue->toggle_state;
+}
+
+/*
* init_qp_queue initializes/constructs r/squeue and registers queue pages.
*/
static inline int init_qp_queue(struct ehca_shca *shca,
@@ -338,11 +353,17 @@ init_qp_queue1:
return ret;
}
-struct ib_qp *ehca_create_qp(struct ib_pd *pd,
- struct ib_qp_init_attr *init_attr,
- struct ib_udata *udata)
+/*
+ * Create an ib_qp struct that is either a QP or an SRQ, depending on
+ * the value of the is_srq parameter. If init_attr and srq_init_attr share
+ * fields, the field out of init_attr is used.
+ */
+struct ehca_qp *internal_create_qp(struct ib_pd *pd,
+ struct ib_qp_init_attr *init_attr,
+ struct ib_srq_init_attr *srq_init_attr,
+ struct ib_udata *udata, int is_srq)
{
- static int da_rc_msg_size[]={ 128, 256, 512, 1024, 2048, 4096 };
+ static int da_rc_msg_size[] = { 128, 256, 512, 1024, 2048, 4096 };
static int da_ud_sq_msg_size[]={ 128, 384, 896, 1920, 3968 };
struct ehca_qp *my_qp;
struct ehca_pd *my_pd = container_of(pd, struct ehca_pd, ib_pd);
@@ -355,7 +376,7 @@ struct ib_qp *ehca_create_qp(struct ib_pd *pd,
/* h_call's out parameters */
struct ehca_alloc_qp_parms parms;
- u32 swqe_size = 0, rwqe_size = 0;
+ u32 swqe_size = 0, rwqe_size = 0, ib_qp_num;
unsigned long flags;
memset(&parms, 0, sizeof(parms));
@@ -376,13 +397,34 @@ struct ib_qp *ehca_create_qp(struct ib_pd *pd,
}
qp_type &= 0x1F;
- /* check for SRQ */
- has_srq = !!(init_attr->srq);
+ /* handle SRQ base QPs */
+ if (init_attr->srq) {
+ struct ehca_qp *my_srq =
+ container_of(init_attr->srq, struct ehca_qp, ib_srq);
+
+ has_srq = 1;
+ parms.ext_type = EQPT_SRQBASE;
+ parms.srq_qpn = my_srq->real_qp_num;
+ parms.srq_token = my_srq->token;
+ }
+
if (is_llqp && has_srq) {
ehca_err(pd->device, "LLQPs can't have an SRQ");
return ERR_PTR(-EINVAL);
}
+ /* handle SRQs */
+ if (is_srq) {
+ parms.ext_type = EQPT_SRQ;
+ parms.srq_limit = srq_init_attr->attr.srq_limit;
+ if (init_attr->cap.max_recv_sge > 3) {
+ ehca_err(pd->device, "no more than three SGEs "
+ "supported for SRQ pd=%p max_sge=%x",
+ pd, init_attr->cap.max_recv_sge);
+ return ERR_PTR(-EINVAL);
+ }
+ }
+
/* check QP type */
if (qp_type != IB_QPT_UD &&
qp_type != IB_QPT_UC &&
@@ -423,11 +465,15 @@ struct ib_qp *ehca_create_qp(struct ib_pd *pd,
spin_lock_init(&my_qp->spinlock_s);
spin_lock_init(&my_qp->spinlock_r);
+ my_qp->qp_type = qp_type;
+ my_qp->ext_type = parms.ext_type;
- my_qp->recv_cq =
- container_of(init_attr->recv_cq, struct ehca_cq, ib_cq);
- my_qp->send_cq =
- container_of(init_attr->send_cq, struct ehca_cq, ib_cq);
+ if (init_attr->recv_cq)
+ my_qp->recv_cq =
+ container_of(init_attr->recv_cq, struct ehca_cq, ib_cq);
+ if (init_attr->send_cq)
+ my_qp->send_cq =
+ container_of(init_attr->send_cq, struct ehca_cq, ib_cq);
do {
if (!idr_pre_get(&ehca_qp_idr, GFP_KERNEL)) {
@@ -471,8 +517,10 @@ struct ib_qp *ehca_create_qp(struct ib_pd *pd,
parms.token = my_qp->token;
parms.eq_handle = shca->eq.ipz_eq_handle;
parms.pd = my_pd->fw_pd;
- parms.send_cq_handle = my_qp->send_cq->ipz_cq_handle;
- parms.recv_cq_handle = my_qp->recv_cq->ipz_cq_handle;
+ if (my_qp->send_cq)
+ parms.send_cq_handle = my_qp->send_cq->ipz_cq_handle;
+ if (my_qp->recv_cq)
+ parms.recv_cq_handle = my_qp->recv_cq->ipz_cq_handle;
parms.max_send_wr = init_attr->cap.max_send_wr;
parms.max_recv_wr = init_attr->cap.max_recv_wr;
@@ -487,7 +535,7 @@ struct ib_qp *ehca_create_qp(struct ib_pd *pd,
goto create_qp_exit1;
}
- my_qp->ib_qp.qp_num = my_qp->real_qp_num = parms.real_qp_num;
+ ib_qp_num = my_qp->real_qp_num = parms.real_qp_num;
my_qp->ipz_qp_handle = parms.qp_handle;
my_qp->galpas = parms.galpas;
@@ -535,7 +583,7 @@ struct ib_qp *ehca_create_qp(struct ib_pd *pd,
parms.act_nr_recv_wqes = init_attr->cap.max_recv_wr;
parms.act_nr_send_sges = init_attr->cap.max_send_sge;
parms.act_nr_recv_sges = init_attr->cap.max_recv_sge;
- my_qp->ib_qp.qp_num = (qp_type == IB_QPT_SMI) ? 0 : 1;
+ ib_qp_num = (qp_type == IB_QPT_SMI) ? 0 : 1;
}
break;
@@ -545,36 +593,51 @@ struct ib_qp *ehca_create_qp(struct ib_pd *pd,
}
/* initialize r/squeue and register queue pages */
- ret = init_qp_queue(shca, my_qp, &my_qp->ipz_squeue, 0,
- has_srq ? H_SUCCESS : H_PAGE_REGISTERED,
- parms.nr_sq_pages, swqe_size,
- parms.act_nr_send_sges);
- if (ret) {
- ehca_err(pd->device,
- "Couldn't initialize squeue and pages ret=%x", ret);
- goto create_qp_exit2;
+ if (HAS_SQ(my_qp)) {
+ ret = init_qp_queue(
+ shca, my_qp, &my_qp->ipz_squeue, 0,
+ HAS_RQ(my_qp) ? H_PAGE_REGISTERED : H_SUCCESS,
+ parms.nr_sq_pages, swqe_size,
+ parms.act_nr_send_sges);
+ if (ret) {
+ ehca_err(pd->device, "Couldn't initialize squeue "
+ "and pages ret=%x", ret);
+ goto create_qp_exit2;
+ }
}
- ret = init_qp_queue(shca, my_qp, &my_qp->ipz_rqueue, 1, H_SUCCESS,
- parms.nr_rq_pages, rwqe_size,
- parms.act_nr_recv_sges);
- if (ret) {
- ehca_err(pd->device,
- "Couldn't initialize rqueue and pages ret=%x", ret);
- goto create_qp_exit3;
+ if (HAS_RQ(my_qp)) {
+ ret = init_qp_queue(
+ shca, my_qp, &my_qp->ipz_rqueue, 1,
+ H_SUCCESS, parms.nr_rq_pages, rwqe_size,
+ parms.act_nr_recv_sges);
+ if (ret) {
+ ehca_err(pd->device, "Couldn't initialize rqueue "
+ "and pages ret=%x", ret);
+ goto create_qp_exit3;
+ }
}
- my_qp->ib_qp.pd = &my_pd->ib_pd;
- my_qp->ib_qp.device = my_pd->ib_pd.device;
+ if (is_srq) {
+ my_qp->ib_srq.pd = &my_pd->ib_pd;
+ my_qp->ib_srq.device = my_pd->ib_pd.device;
- my_qp->ib_qp.recv_cq = init_attr->recv_cq;
- my_qp->ib_qp.send_cq = init_attr->send_cq;
+ my_qp->ib_srq.srq_context = init_attr->qp_context;
+ my_qp->ib_srq.event_handler = init_attr->event_handler;
+ } else {
+ my_qp->ib_qp.qp_num = ib_qp_num;
+ my_qp->ib_qp.pd = &my_pd->ib_pd;
+ my_qp->ib_qp.device = my_pd->ib_pd.device;
+
+ my_qp->ib_qp.recv_cq = init_attr->recv_cq;
+ my_qp->ib_qp.send_cq = init_attr->send_cq;
- my_qp->ib_qp.qp_type = my_qp->qp_type = qp_type;
- my_qp->ib_qp.srq = init_attr->srq;
+ my_qp->ib_qp.qp_type = qp_type;
+ my_qp->ib_qp.srq = init_attr->srq;
- my_qp->ib_qp.qp_context = init_attr->qp_context;
- my_qp->ib_qp.event_handler = init_attr->event_handler;
+ my_qp->ib_qp.qp_context = init_attr->qp_context;
+ my_qp->ib_qp.event_handler = init_attr->event_handler;
+ }
init_attr->cap.max_inline_data = 0; /* not supported yet */
init_attr->cap.max_recv_sge = parms.act_nr_recv_sges;
@@ -593,41 +656,32 @@ struct ib_qp *ehca_create_qp(struct ib_pd *pd,
goto create_qp_exit4;
}
}
- if (init_attr->send_cq) {
- struct ehca_cq *cq = container_of(init_attr->send_cq,
- struct ehca_cq, ib_cq);
- ret = ehca_cq_assign_qp(cq, my_qp);
+
+ if (my_qp->send_cq) {
+ ret = ehca_cq_assign_qp(my_qp->send_cq, my_qp);
if (ret) {
ehca_err(pd->device, "Couldn't assign qp to send_cq ret=%x",
ret);
goto create_qp_exit4;
}
- my_qp->send_cq = cq;
}
+
/* copy queues, galpa data to user space */
if (context && udata) {
- struct ipz_queue *ipz_rqueue = &my_qp->ipz_rqueue;
- struct ipz_queue *ipz_squeue = &my_qp->ipz_squeue;
struct ehca_create_qp_resp resp;
memset(&resp, 0, sizeof(resp));
resp.qp_num = my_qp->real_qp_num;
resp.token = my_qp->token;
resp.qp_type = my_qp->qp_type;
+ resp.ext_type = my_qp->ext_type;
resp.qkey = my_qp->qkey;
resp.real_qp_num = my_qp->real_qp_num;
- /* rqueue properties */
- resp.ipz_rqueue.qe_size = ipz_rqueue->qe_size;
- resp.ipz_rqueue.act_nr_of_sg = ipz_rqueue->act_nr_of_sg;
- resp.ipz_rqueue.queue_length = ipz_rqueue->queue_length;
- resp.ipz_rqueue.pagesize = ipz_rqueue->pagesize;
- resp.ipz_rqueue.toggle_state = ipz_rqueue->toggle_state;
- /* squeue properties */
- resp.ipz_squeue.qe_size = ipz_squeue->qe_size;
- resp.ipz_squeue.act_nr_of_sg = ipz_squeue->act_nr_of_sg;
- resp.ipz_squeue.queue_length = ipz_squeue->queue_length;
- resp.ipz_squeue.pagesize = ipz_squeue->pagesize;
- resp.ipz_squeue.toggle_state = ipz_squeue->toggle_state;
+ if (HAS_SQ(my_qp))
+ queue2resp(&resp.ipz_squeue, &my_qp->ipz_squeue);
+ if (HAS_RQ(my_qp))
+ queue2resp(&resp.ipz_rqueue, &my_qp->ipz_rqueue);
+
if (ib_copy_to_udata(udata, &resp, sizeof resp)) {
ehca_err(pd->device, "Copy to udata failed");
ret = -EINVAL;
@@ -635,13 +689,15 @@ struct ib_qp *ehca_create_qp(struct ib_pd *pd,
}
}
- return &my_qp->ib_qp;
+ return my_qp;
create_qp_exit4:
- ipz_queue_dtor(&my_qp->ipz_rqueue);
+ if (HAS_RQ(my_qp))
+ ipz_queue_dtor(&my_qp->ipz_rqueue);
create_qp_exit3:
- ipz_queue_dtor(&my_qp->ipz_squeue);
+ if (HAS_SQ(my_qp))
+ ipz_queue_dtor(&my_qp->ipz_squeue);
create_qp_exit2:
hipz_h_destroy_qp(shca->ipz_hca_handle, my_qp);
@@ -656,6 +712,114 @@ create_qp_exit0:
return ERR_PTR(ret);
}
+struct ib_qp *ehca_create_qp(struct ib_pd *pd,
+ struct ib_qp_init_attr *qp_init_attr,
+ struct ib_udata *udata)
+{
+ struct ehca_qp *ret;
+
+ ret = internal_create_qp(pd, qp_init_attr, NULL, udata, 0);
+ return IS_ERR(ret) ? (struct ib_qp*)ret : &ret->ib_qp;
+}
+
+int internal_destroy_qp(struct ib_device *dev, struct ehca_qp *my_qp,
+ struct ib_uobject *uobject);
+
+struct ib_srq *ehca_create_srq(struct ib_pd *pd,
+ struct ib_srq_init_attr *srq_init_attr,
+ struct ib_udata *udata)
+{
+ struct ib_qp_init_attr qp_init_attr;
+ struct ehca_qp *my_qp;
+ struct ib_srq *ret;
+ struct ehca_shca *shca = container_of(pd->device, struct ehca_shca,
+ ib_device);
+ struct hcp_modify_qp_control_block *mqpcb;
+ u64 hret, update_mask;
+
+ /* For common attributes, internal_create_qp() takes its info
+ * out of qp_init_attr, so copy all common attrs there.
+ */
+ memset(&qp_init_attr, 0, sizeof(qp_init_attr));
+ qp_init_attr.event_handler = srq_init_attr->event_handler;
+ qp_init_attr.qp_context = srq_init_attr->srq_context;
+ qp_init_attr.sq_sig_type = IB_SIGNAL_ALL_WR;
+ qp_init_attr.qp_type = IB_QPT_RC;
+ qp_init_attr.cap.max_recv_wr = srq_init_attr->attr.max_wr;
+ qp_init_attr.cap.max_recv_sge = srq_init_attr->attr.max_sge;
+
+ my_qp = internal_create_qp(pd, &qp_init_attr, srq_init_attr, udata, 1);
+ if (IS_ERR(my_qp))
+ return (struct ib_srq*)my_qp;
+
+ /* copy back return values */
+ srq_init_attr->attr.max_wr = qp_init_attr.cap.max_recv_wr;
+ srq_init_attr->attr.max_sge = qp_init_attr.cap.max_recv_sge;
+
+ /* drive SRQ into RTR state */
+ mqpcb = ehca_alloc_fw_ctrlblock(GFP_KERNEL);
+ if (!mqpcb) {
+ ehca_err(pd->device, "Could not get zeroed page for mqpcb "
+ "ehca_qp=%p qp_num=%x ", my_qp, my_qp->real_qp_num);
+ ret = ERR_PTR(-ENOMEM);
+ goto create_srq1;
+ }
+
+ mqpcb->qp_state = EHCA_QPS_INIT;
+ mqpcb->prim_phys_port = 1;
+ update_mask = EHCA_BMASK_SET(MQPCB_MASK_QP_STATE, 1);
+ hret = hipz_h_modify_qp(shca->ipz_hca_handle,
+ my_qp->ipz_qp_handle,
+ &my_qp->pf,
+ update_mask,
+ mqpcb, my_qp->galpas.kernel);
+ if (hret != H_SUCCESS) {
+ ehca_err(pd->device, "Could not modify SRQ to INIT"
+ "ehca_qp=%p qp_num=%x hret=%lx",
+ my_qp, my_qp->real_qp_num, hret);
+ goto create_srq2;
+ }
+
+ mqpcb->qp_enable = 1;
+ update_mask = EHCA_BMASK_SET(MQPCB_MASK_QP_ENABLE, 1);
+ hret = hipz_h_modify_qp(shca->ipz_hca_handle,
+ my_qp->ipz_qp_handle,
+ &my_qp->pf,
+ update_mask,
+ mqpcb, my_qp->galpas.kernel);
+ if (hret != H_SUCCESS) {
+ ehca_err(pd->device, "Could not enable SRQ"
+ "ehca_qp=%p qp_num=%x hret=%lx",
+ my_qp, my_qp->real_qp_num, hret);
+ goto create_srq2;
+ }
+
+ mqpcb->qp_state = EHCA_QPS_RTR;
+ update_mask = EHCA_BMASK_SET(MQPCB_MASK_QP_STATE, 1);
+ hret = hipz_h_modify_qp(shca->ipz_hca_handle,
+ my_qp->ipz_qp_handle,
+ &my_qp->pf,
+ update_mask,
+ mqpcb, my_qp->galpas.kernel);
+ if (hret != H_SUCCESS) {
+ ehca_err(pd->device, "Could not modify SRQ to RTR"
+ "ehca_qp=%p qp_num=%x hret=%lx",
+ my_qp, my_qp->real_qp_num, hret);
+ goto create_srq2;
+ }
+
+ return &my_qp->ib_srq;
+
+create_srq2:
+ ret = ERR_PTR(ehca2ib_return_code(hret));
+ ehca_free_fw_ctrlblock(mqpcb);
+
+create_srq1:
+ internal_destroy_qp(pd->device, my_qp, my_qp->ib_srq.uobject);
+
+ return ret;
+}
+
/*
* prepare_sqe_rts called by internal_modify_qp() at trans sqe -> rts
* set purge bit of bad wqe and subsequent wqes to avoid reentering sqe
@@ -1341,42 +1505,159 @@ query_qp_exit1:
return ret;
}
-int ehca_destroy_qp(struct ib_qp *ibqp)
+int ehca_modify_srq(struct ib_srq *ibsrq, struct ib_srq_attr *attr,
+ enum ib_srq_attr_mask attr_mask, struct ib_udata *udata)
{
- struct ehca_qp *my_qp = container_of(ibqp, struct ehca_qp, ib_qp);
- struct ehca_shca *shca = container_of(ibqp->device, struct ehca_shca,
+ struct ehca_qp *my_qp =
+ container_of(ibsrq, struct ehca_qp, ib_srq);
+ struct ehca_pd *my_pd =
+ container_of(ibsrq->pd, struct ehca_pd, ib_pd);
+ struct ehca_shca *shca =
+ container_of(ibsrq->pd->device, struct ehca_shca, ib_device);
+ struct hcp_modify_qp_control_block *mqpcb;
+ u64 update_mask;
+ u64 h_ret;
+ int ret = 0;
+
+ u32 cur_pid = current->tgid;
+ if (my_pd->ib_pd.uobject && my_pd->ib_pd.uobject->context &&
+ my_pd->ownpid != cur_pid) {
+ ehca_err(ibsrq->pd->device, "Invalid caller pid=%x ownpid=%x",
+ cur_pid, my_pd->ownpid);
+ return -EINVAL;
+ }
+
+ mqpcb = ehca_alloc_fw_ctrlblock(GFP_KERNEL);
+ if (!mqpcb) {
+ ehca_err(ibsrq->device, "Could not get zeroed page for mqpcb "
+ "ehca_qp=%p qp_num=%x ", my_qp, my_qp->real_qp_num);
+ return -ENOMEM;
+ }
+
+ update_mask = 0;
+ if (attr_mask & IB_SRQ_LIMIT) {
+ attr_mask &= ~IB_SRQ_LIMIT;
+ update_mask |=
+ EHCA_BMASK_SET(MQPCB_MASK_CURR_SRQ_LIMIT, 1)
+ | EHCA_BMASK_SET(MQPCB_MASK_QP_AFF_ASYN_EV_LOG_REG, 1);
+ mqpcb->curr_srq_limit =
+ EHCA_BMASK_SET(MQPCB_CURR_SRQ_LIMIT, attr->srq_limit);
+ mqpcb->qp_aff_asyn_ev_log_reg =
+ EHCA_BMASK_SET(QPX_AAELOG_RESET_SRQ_LIMIT, 1);
+ }
+
+ /* by now, all bits in attr_mask should have been cleared */
+ if (attr_mask) {
+ ehca_err(ibsrq->device, "invalid attribute mask bits set "
+ "attr_mask=%x", attr_mask);
+ ret = -EINVAL;
+ goto modify_srq_exit0;
+ }
+
+ if (ehca_debug_level)
+ ehca_dmp(mqpcb, 4*70, "qp_num=%x", my_qp->real_qp_num);
+
+ h_ret = hipz_h_modify_qp(shca->ipz_hca_handle, my_qp->ipz_qp_handle,
+ NULL, update_mask, mqpcb,
+ my_qp->galpas.kernel);
+
+ if (h_ret != H_SUCCESS) {
+ ret = ehca2ib_return_code(h_ret);
+ ehca_err(ibsrq->device, "hipz_h_modify_qp() failed rc=%lx "
+ "ehca_qp=%p qp_num=%x",
+ h_ret, my_qp, my_qp->real_qp_num);
+ }
+
+modify_srq_exit0:
+ ehca_free_fw_ctrlblock(mqpcb);
+
+ return ret;
+}
+
+int ehca_query_srq(struct ib_srq *srq, struct ib_srq_attr *srq_attr)
+{
+ struct ehca_qp *my_qp = container_of(srq, struct ehca_qp, ib_srq);
+ struct ehca_pd *my_pd = container_of(srq->pd, struct ehca_pd, ib_pd);
+ struct ehca_shca *shca = container_of(srq->device, struct ehca_shca,
ib_device);
+ struct ipz_adapter_handle adapter_handle = shca->ipz_hca_handle;
+ struct hcp_modify_qp_control_block *qpcb;
+ u32 cur_pid = current->tgid;
+ int ret = 0;
+ u64 h_ret;
+
+ if (my_pd->ib_pd.uobject && my_pd->ib_pd.uobject->context &&
+ my_pd->ownpid != cur_pid) {
+ ehca_err(srq->device, "Invalid caller pid=%x ownpid=%x",
+ cur_pid, my_pd->ownpid);
+ return -EINVAL;
+ }
+
+ qpcb = ehca_alloc_fw_ctrlblock(GFP_KERNEL);
+ if (!qpcb) {
+ ehca_err(srq->device,"Out of memory for qpcb "
+ "ehca_qp=%p qp_num=%x", my_qp, my_qp->real_qp_num);
+ return -ENOMEM;
+ }
+
+ h_ret = hipz_h_query_qp(adapter_handle, my_qp->ipz_qp_handle,
+ NULL, qpcb, my_qp->galpas.kernel);
+
+ if (h_ret != H_SUCCESS) {
+ ret = ehca2ib_return_code(h_ret);
+ ehca_err(srq->device,"hipz_h_query_qp() failed "
+ "ehca_qp=%p qp_num=%x h_ret=%lx",
+ my_qp, my_qp->real_qp_num, h_ret);
+ goto query_srq_exit1;
+ }
+
+ srq_attr->max_wr = qpcb->max_nr_outst_recv_wr - 1;
+ srq_attr->srq_limit = EHCA_BMASK_GET(
+ MQPCB_CURR_SRQ_LIMIT, qpcb->curr_srq_limit);
+
+ if (ehca_debug_level)
+ ehca_dmp(qpcb, 4*70, "qp_num=%x", my_qp->real_qp_num);
+
+query_srq_exit1:
+ ehca_free_fw_ctrlblock(qpcb);
+
+ return ret;
+}
+
+int internal_destroy_qp(struct ib_device *dev, struct ehca_qp *my_qp,
+ struct ib_uobject *uobject)
+{
+ struct ehca_shca *shca = container_of(dev, struct ehca_shca, ib_device);
struct ehca_pd *my_pd = container_of(my_qp->ib_qp.pd, struct ehca_pd,
ib_pd);
u32 cur_pid = current->tgid;
- u32 qp_num = ibqp->qp_num;
+ u32 qp_num = my_qp->real_qp_num;
int ret;
u64 h_ret;
u8 port_num;
enum ib_qp_type qp_type;
unsigned long flags;
- if (ibqp->uobject) {
+ if (uobject) {
if (my_qp->mm_count_galpa ||
my_qp->mm_count_rqueue || my_qp->mm_count_squeue) {
- ehca_err(ibqp->device, "Resources still referenced in "
- "user space qp_num=%x", ibqp->qp_num);
+ ehca_err(dev, "Resources still referenced in "
+ "user space qp_num=%x", qp_num);
return -EINVAL;
}
if (my_pd->ownpid != cur_pid) {
- ehca_err(ibqp->device, "Invalid caller pid=%x ownpid=%x",
+ ehca_err(dev, "Invalid caller pid=%x ownpid=%x",
cur_pid, my_pd->ownpid);
return -EINVAL;
}
}
if (my_qp->send_cq) {
- ret = ehca_cq_unassign_qp(my_qp->send_cq,
- my_qp->real_qp_num);
+ ret = ehca_cq_unassign_qp(my_qp->send_cq, qp_num);
if (ret) {
- ehca_err(ibqp->device, "Couldn't unassign qp from "
+ ehca_err(dev, "Couldn't unassign qp from "
"send_cq ret=%x qp_num=%x cq_num=%x", ret,
- my_qp->ib_qp.qp_num, my_qp->send_cq->cq_number);
+ qp_num, my_qp->send_cq->cq_number);
return ret;
}
}
@@ -1387,7 +1668,7 @@ int ehca_destroy_qp(struct ib_qp *ibqp)
h_ret = hipz_h_destroy_qp(shca->ipz_hca_handle, my_qp);
if (h_ret != H_SUCCESS) {
- ehca_err(ibqp->device, "hipz_h_destroy_qp() failed rc=%lx "
+ ehca_err(dev, "hipz_h_destroy_qp() failed rc=%lx "
"ehca_qp=%p qp_num=%x", h_ret, my_qp, qp_num);
return ehca2ib_return_code(h_ret);
}
@@ -1398,7 +1679,7 @@ int ehca_destroy_qp(struct ib_qp *ibqp)
/* no support for IB_QPT_SMI yet */
if (qp_type == IB_QPT_GSI) {
struct ib_event event;
- ehca_info(ibqp->device, "device %s: port %x is inactive.",
+ ehca_info(dev, "device %s: port %x is inactive.",
shca->ib_device.name, port_num);
event.device = &shca->ib_device;
event.event = IB_EVENT_PORT_ERR;
@@ -1407,12 +1688,28 @@ int ehca_destroy_qp(struct ib_qp *ibqp)
ib_dispatch_event(&event);
}
- ipz_queue_dtor(&my_qp->ipz_rqueue);
- ipz_queue_dtor(&my_qp->ipz_squeue);
+ if (HAS_RQ(my_qp))
+ ipz_queue_dtor(&my_qp->ipz_rqueue);
+ if (HAS_SQ(my_qp))
+ ipz_queue_dtor(&my_qp->ipz_squeue);
kmem_cache_free(qp_cache, my_qp);
return 0;
}
+int ehca_destroy_qp(struct ib_qp *qp)
+{
+ return internal_destroy_qp(qp->device,
+ container_of(qp, struct ehca_qp, ib_qp),
+ qp->uobject);
+}
+
+int ehca_destroy_srq(struct ib_srq *srq)
+{
+ return internal_destroy_qp(srq->device,
+ container_of(srq, struct ehca_qp, ib_srq),
+ srq->uobject);
+}
+
int ehca_init_qp_cache(void)
{
qp_cache = kmem_cache_create("ehca_cache_qp",
diff --git a/drivers/infiniband/hw/ehca/ehca_reqs.c b/drivers/infiniband/hw/ehca/ehca_reqs.c
index 56c4527..b5664fa 100644
--- a/drivers/infiniband/hw/ehca/ehca_reqs.c
+++ b/drivers/infiniband/hw/ehca/ehca_reqs.c
@@ -3,8 +3,9 @@
*
* post_send/recv, poll_cq, req_notify
*
- * Authors: Waleri Fomin <fomin@de.ibm.com>
- * Hoang-Nam Nguyen <hnguyen@de.ibm.com>
+ * Authors: Hoang-Nam Nguyen <hnguyen@de.ibm.com>
+ * Waleri Fomin <fomin@de.ibm.com>
+ * Joachim Fenkes <fenkes@de.ibm.com>
* Reinhard Ernst <rernst@de.ibm.com>
*
* Copyright (c) 2005 IBM Corporation
@@ -413,17 +414,23 @@ post_send_exit0:
return ret;
}
-int ehca_post_recv(struct ib_qp *qp,
- struct ib_recv_wr *recv_wr,
- struct ib_recv_wr **bad_recv_wr)
+static int internal_post_recv(struct ehca_qp *my_qp,
+ struct ib_device *dev,
+ struct ib_recv_wr *recv_wr,
+ struct ib_recv_wr **bad_recv_wr)
{
- struct ehca_qp *my_qp = container_of(qp, struct ehca_qp, ib_qp);
struct ib_recv_wr *cur_recv_wr;
struct ehca_wqe *wqe_p;
int wqe_cnt = 0;
int ret = 0;
unsigned long spl_flags;
+ if (unlikely(!HAS_RQ(my_qp))) {
+ ehca_err(dev, "QP has no RQ ehca_qp=%p qp_num=%x ext_type=%d",
+ my_qp, my_qp->real_qp_num, my_qp->ext_type);
+ return -ENODEV;
+ }
+
/* LOCK the QUEUE */
spin_lock_irqsave(&my_qp->spinlock_r, spl_flags);
@@ -439,8 +446,8 @@ int ehca_post_recv(struct ib_qp *qp,
*bad_recv_wr = cur_recv_wr;
if (wqe_cnt == 0) {
ret = -ENOMEM;
- ehca_err(qp->device, "Too many posted WQEs "
- "qp_num=%x", qp->qp_num);
+ ehca_err(dev, "Too many posted WQEs "
+ "qp_num=%x", my_qp->real_qp_num);
}
goto post_recv_exit0;
}
@@ -455,14 +462,14 @@ int ehca_post_recv(struct ib_qp *qp,
*bad_recv_wr = cur_recv_wr;
if (wqe_cnt == 0) {
ret = -EINVAL;
- ehca_err(qp->device, "Could not write WQE "
- "qp_num=%x", qp->qp_num);
+ ehca_err(dev, "Could not write WQE "
+ "qp_num=%x", my_qp->real_qp_num);
}
goto post_recv_exit0;
}
wqe_cnt++;
- ehca_gen_dbg("ehca_qp=%p qp_num=%x wqe_cnt=%d",
- my_qp, qp->qp_num, wqe_cnt);
+ ehca_dbg(dev, "ehca_qp=%p qp_num=%x wqe_cnt=%d",
+ my_qp, my_qp->real_qp_num, wqe_cnt);
} /* eof for cur_recv_wr */
post_recv_exit0:
@@ -472,6 +479,22 @@ post_recv_exit0:
return ret;
}
+int ehca_post_recv(struct ib_qp *qp,
+ struct ib_recv_wr *recv_wr,
+ struct ib_recv_wr **bad_recv_wr)
+{
+ return internal_post_recv(container_of(qp, struct ehca_qp, ib_qp),
+ qp->device, recv_wr, bad_recv_wr);
+}
+
+int ehca_post_srq_recv(struct ib_srq *srq,
+ struct ib_recv_wr *recv_wr,
+ struct ib_recv_wr **bad_recv_wr)
+{
+ return internal_post_recv(container_of(srq, struct ehca_qp, ib_srq),
+ srq->device, recv_wr, bad_recv_wr);
+}
+
/*
* ib_wc_opcode table converts ehca wc opcode to ib
* Since we use zero to indicate invalid opcode, the actual ib opcode must
diff --git a/drivers/infiniband/hw/ehca/ehca_uverbs.c b/drivers/infiniband/hw/ehca/ehca_uverbs.c
index 73db920..d8fe37d 100644
--- a/drivers/infiniband/hw/ehca/ehca_uverbs.c
+++ b/drivers/infiniband/hw/ehca/ehca_uverbs.c
@@ -257,6 +257,7 @@ int ehca_mmap(struct ib_ucontext *context, struct vm_area_struct *vma)
struct ehca_cq *cq;
struct ehca_qp *qp;
struct ehca_pd *pd;
+ struct ib_uobject *uobject;
switch (q_type) {
case 1: /* CQ */
@@ -304,7 +305,8 @@ int ehca_mmap(struct ib_ucontext *context, struct vm_area_struct *vma)
return -ENOMEM;
}
- if (!qp->ib_qp.uobject || qp->ib_qp.uobject->context != context)
+ uobject = IS_SRQ(qp) ? qp->ib_srq.uobject : qp->ib_qp.uobject;
+ if (!uobject || uobject->context != context)
return -EINVAL;
ret = ehca_mmap_qp(vma, qp, rsrc_type);
diff --git a/drivers/infiniband/hw/ehca/hcp_if.c b/drivers/infiniband/hw/ehca/hcp_if.c
index 7efc4a2..b078377 100644
--- a/drivers/infiniband/hw/ehca/hcp_if.c
+++ b/drivers/infiniband/hw/ehca/hcp_if.c
@@ -5,6 +5,7 @@
*
* Authors: Christoph Raisch <raisch@de.ibm.com>
* Hoang-Nam Nguyen <hnguyen@de.ibm.com>
+ * Joachim Fenkes <fenkes@de.ibm.com>
* Gerd Bayer <gerd.bayer@de.ibm.com>
* Waleri Fomin <fomin@de.ibm.com>
*
@@ -62,6 +63,12 @@
#define H_ALL_RES_QP_MAX_SEND_SGE EHCA_BMASK_IBM(32, 39)
#define H_ALL_RES_QP_MAX_RECV_SGE EHCA_BMASK_IBM(40, 47)
+#define H_ALL_RES_QP_UD_AV_LKEY EHCA_BMASK_IBM(32, 63)
+#define H_ALL_RES_QP_SRQ_QP_TOKEN EHCA_BMASK_IBM(0, 31)
+#define H_ALL_RES_QP_SRQ_QP_HANDLE EHCA_BMASK_IBM(0, 64)
+#define H_ALL_RES_QP_SRQ_LIMIT EHCA_BMASK_IBM(48, 63)
+#define H_ALL_RES_QP_SRQ_QPN EHCA_BMASK_IBM(40, 63)
+
#define H_ALL_RES_QP_ACT_OUTST_SEND_WR EHCA_BMASK_IBM(16, 31)
#define H_ALL_RES_QP_ACT_OUTST_RECV_WR EHCA_BMASK_IBM(48, 63)
#define H_ALL_RES_QP_ACT_SEND_SGE EHCA_BMASK_IBM(8, 15)
@@ -150,7 +157,7 @@ static long ehca_plpar_hcall9(unsigned long opcode,
{
long ret;
int i, sleep_msecs, lock_is_set = 0;
- unsigned long flags;
+ unsigned long flags = 0;
ehca_gen_dbg("opcode=%lx arg1=%lx arg2=%lx arg3=%lx arg4=%lx "
"arg5=%lx arg6=%lx arg7=%lx arg8=%lx arg9=%lx",
@@ -282,8 +289,7 @@ u64 hipz_h_alloc_resource_qp(const struct ipz_adapter_handle adapter_handle,
struct ehca_alloc_qp_parms *parms)
{
u64 ret;
- u64 allocate_controls;
- u64 max_r10_reg;
+ u64 allocate_controls, max_r10_reg, r11, r12;
u64 outs[PLPAR_HCALL9_BUFSIZE];
allocate_controls =
@@ -309,6 +315,13 @@ u64 hipz_h_alloc_resource_qp(const struct ipz_adapter_handle adapter_handle,
| EHCA_BMASK_SET(H_ALL_RES_QP_MAX_RECV_SGE,
parms->max_recv_sge);
+ r11 = EHCA_BMASK_SET(H_ALL_RES_QP_SRQ_QP_TOKEN, parms->srq_token);
+
+ if (parms->ext_type == EQPT_SRQ)
+ r12 = EHCA_BMASK_SET(H_ALL_RES_QP_SRQ_LIMIT, parms->srq_limit);
+ else
+ r12 = EHCA_BMASK_SET(H_ALL_RES_QP_SRQ_QPN, parms->srq_qpn);
+
ret = ehca_plpar_hcall9(H_ALLOC_RESOURCE, outs,
adapter_handle.handle, /* r4 */
allocate_controls, /* r5 */
@@ -316,9 +329,7 @@ u64 hipz_h_alloc_resource_qp(const struct ipz_adapter_handle adapter_handle,
parms->recv_cq_handle.handle,
parms->eq_handle.handle,
((u64)parms->token << 32) | parms->pd.value,
- max_r10_reg, /* r10 */
- parms->ud_av_l_key_ctl, /* r11 */
- 0);
+ max_r10_reg, r11, r12);
parms->qp_handle.handle = outs[0];
parms->real_qp_num = (u32)outs[1];
diff --git a/drivers/infiniband/hw/ehca/hipz_hw.h b/drivers/infiniband/hw/ehca/hipz_hw.h
index 9fe8367..d46b18c 100644
--- a/drivers/infiniband/hw/ehca/hipz_hw.h
+++ b/drivers/infiniband/hw/ehca/hipz_hw.h
@@ -163,6 +163,7 @@ struct hipz_qptemm {
#define QPX_SQADDER EHCA_BMASK_IBM(48,63)
#define QPX_RQADDER EHCA_BMASK_IBM(48,63)
+#define QPX_AAELOG_RESET_SRQ_LIMIT EHCA_BMASK_IBM(3,3)
#define QPTEMM_OFFSET(x) offsetof(struct hipz_qptemm,x)
--
1.5.2
^ permalink raw reply related
* [PATCH 05/13] IB/ehca: Support UD low latency QPs
From: Joachim Fenkes @ 2007-07-09 13:26 UTC (permalink / raw)
To: LinuxPPC-Dev, LKML, OF-General, Roland Dreier
Cc: Stefan Roscher, Christoph Raisch
In-Reply-To: <200707091502.22407.fenkes@de.ibm.com>
From: Stefan Roscher <stefan.roscher@de.ibm.com>
Signed-off-by: Joachim Fenkes <fenkes@de.ibm.com>
---
drivers/infiniband/hw/ehca/ehca_qp.c | 84 +++++++++++++++++++++++-----------
1 files changed, 57 insertions(+), 27 deletions(-)
diff --git a/drivers/infiniband/hw/ehca/ehca_qp.c b/drivers/infiniband/hw/ehca/ehca_qp.c
index 9486a44..ffd1ce9 100644
--- a/drivers/infiniband/hw/ehca/ehca_qp.c
+++ b/drivers/infiniband/hw/ehca/ehca_qp.c
@@ -275,6 +275,11 @@ static inline void queue2resp(struct ipzu_queue_resp *resp,
resp->toggle_state = queue->toggle_state;
}
+static inline int ll_qp_msg_size(int nr_sge)
+{
+ return 128 << nr_sge;
+}
+
/*
* init_qp_queue initializes/constructs r/squeue and registers queue pages.
*/
@@ -363,8 +368,6 @@ struct ehca_qp *internal_create_qp(struct ib_pd *pd,
struct ib_srq_init_attr *srq_init_attr,
struct ib_udata *udata, int is_srq)
{
- static int da_rc_msg_size[] = { 128, 256, 512, 1024, 2048, 4096 };
- static int da_ud_sq_msg_size[]={ 128, 384, 896, 1920, 3968 };
struct ehca_qp *my_qp;
struct ehca_pd *my_pd = container_of(pd, struct ehca_pd, ib_pd);
struct ehca_shca *shca = container_of(pd->device, struct ehca_shca,
@@ -396,6 +399,7 @@ struct ehca_qp *internal_create_qp(struct ib_pd *pd,
parms.ll_comp_flags = qp_type & LLQP_COMP_MASK;
}
qp_type &= 0x1F;
+ init_attr->qp_type &= 0x1F;
/* handle SRQ base QPs */
if (init_attr->srq) {
@@ -435,23 +439,49 @@ struct ehca_qp *internal_create_qp(struct ib_pd *pd,
return ERR_PTR(-EINVAL);
}
- if (is_llqp && (qp_type != IB_QPT_RC && qp_type != IB_QPT_UD)) {
- ehca_err(pd->device, "unsupported LL QP Type=%x", qp_type);
- return ERR_PTR(-EINVAL);
- } else if (is_llqp && qp_type == IB_QPT_RC &&
- (init_attr->cap.max_send_wr > 255 ||
- init_attr->cap.max_recv_wr > 255 )) {
- ehca_err(pd->device, "Invalid Number of max_sq_wr=%x "
- "or max_rq_wr=%x for RC LLQP",
- init_attr->cap.max_send_wr,
- init_attr->cap.max_recv_wr);
- return ERR_PTR(-EINVAL);
- } else if (is_llqp && qp_type == IB_QPT_UD &&
- init_attr->cap.max_send_wr > 255) {
- ehca_err(pd->device,
- "Invalid Number of max_send_wr=%x for UD QP_TYPE=%x",
- init_attr->cap.max_send_wr, qp_type);
- return ERR_PTR(-EINVAL);
+ if (is_llqp) {
+ switch (qp_type) {
+ case IB_QPT_RC:
+ if ((init_attr->cap.max_send_wr > 255) ||
+ (init_attr->cap.max_recv_wr > 255)) {
+ ehca_err(pd->device,
+ "Invalid Number of max_sq_wr=%x "
+ "or max_rq_wr=%x for RC LLQP",
+ init_attr->cap.max_send_wr,
+ init_attr->cap.max_recv_wr);
+ return ERR_PTR(-EINVAL);
+ }
+ break;
+ case IB_QPT_UD:
+ if (!EHCA_BMASK_GET(HCA_CAP_UD_LL_QP, shca->hca_cap)) {
+ ehca_err(pd->device, "UD LLQP not supported "
+ "by this adapter");
+ return ERR_PTR(-ENOSYS);
+ }
+ if (!(init_attr->cap.max_send_sge <= 5
+ && init_attr->cap.max_send_sge >= 1
+ && init_attr->cap.max_recv_sge <= 5
+ && init_attr->cap.max_recv_sge >= 1)) {
+ ehca_err(pd->device,
+ "Invalid Number of max_send_sge=%x "
+ "or max_recv_sge=%x for UD LLQP",
+ init_attr->cap.max_send_sge,
+ init_attr->cap.max_recv_sge);
+ return ERR_PTR(-EINVAL);
+ } else if (init_attr->cap.max_send_wr > 255) {
+ ehca_err(pd->device,
+ "Invalid Number of "
+ "ax_send_wr=%x for UD QP_TYPE=%x",
+ init_attr->cap.max_send_wr, qp_type);
+ return ERR_PTR(-EINVAL);
+ }
+ break;
+ default:
+ ehca_err(pd->device, "unsupported LL QP Type=%x",
+ qp_type);
+ return ERR_PTR(-EINVAL);
+ break;
+ }
}
if (pd->uobject && udata)
@@ -509,7 +539,7 @@ struct ehca_qp *internal_create_qp(struct ib_pd *pd,
/* UD_AV CIRCUMVENTION */
max_send_sge = init_attr->cap.max_send_sge;
max_recv_sge = init_attr->cap.max_recv_sge;
- if (parms.servicetype == ST_UD) {
+ if (parms.servicetype == ST_UD && !is_llqp) {
max_send_sge += 2;
max_recv_sge += 2;
}
@@ -547,8 +577,8 @@ struct ehca_qp *internal_create_qp(struct ib_pd *pd,
rwqe_size = offsetof(struct ehca_wqe, u.nud.sg_list[
(parms.act_nr_recv_sges)]);
} else { /* for LLQP we need to use msg size, not wqe size */
- swqe_size = da_rc_msg_size[max_send_sge];
- rwqe_size = da_rc_msg_size[max_recv_sge];
+ swqe_size = ll_qp_msg_size(max_send_sge);
+ rwqe_size = ll_qp_msg_size(max_recv_sge);
parms.act_nr_send_sges = 1;
parms.act_nr_recv_sges = 1;
}
@@ -563,15 +593,15 @@ struct ehca_qp *internal_create_qp(struct ib_pd *pd,
case IB_QPT_UD:
case IB_QPT_GSI:
case IB_QPT_SMI:
- /* UD circumvention */
- parms.act_nr_recv_sges -= 2;
- parms.act_nr_send_sges -= 2;
if (is_llqp) {
- swqe_size = da_ud_sq_msg_size[max_send_sge];
- rwqe_size = da_rc_msg_size[max_recv_sge];
+ swqe_size = ll_qp_msg_size(parms.act_nr_send_sges);
+ rwqe_size = ll_qp_msg_size(parms.act_nr_recv_sges);
parms.act_nr_send_sges = 1;
parms.act_nr_recv_sges = 1;
} else {
+ /* UD circumvention */
+ parms.act_nr_send_sges -= 2;
+ parms.act_nr_recv_sges -= 2;
swqe_size = offsetof(struct ehca_wqe,
u.ud_av.sg_list[parms.act_nr_send_sges]);
rwqe_size = offsetof(struct ehca_wqe,
--
1.5.2
^ permalink raw reply related
* [PATCH 06/13] IB/ehca: Set SEND_GRH flag for all non-LL UD QPs on eHCA2
From: Joachim Fenkes @ 2007-07-09 13:27 UTC (permalink / raw)
To: LinuxPPC-Dev, LKML, OF-General, Roland Dreier
Cc: Stefan Roscher, Christoph Raisch
In-Reply-To: <200707091502.22407.fenkes@de.ibm.com>
From: Stefan Roscher <stefan.roscher@de.ibm.com>
Signed-off-by: Joachim Fenkes <fenkes@de.ibm.com>
---
drivers/infiniband/hw/ehca/ehca_qp.c | 11 +++++++++++
1 files changed, 11 insertions(+), 0 deletions(-)
diff --git a/drivers/infiniband/hw/ehca/ehca_qp.c b/drivers/infiniband/hw/ehca/ehca_qp.c
index ffd1ce9..cbb8b5b 100644
--- a/drivers/infiniband/hw/ehca/ehca_qp.c
+++ b/drivers/infiniband/hw/ehca/ehca_qp.c
@@ -1054,6 +1054,17 @@ static int internal_modify_qp(struct ib_qp *ibqp,
"ehca_qp=%p qp_num=%x <VALID STATE CHANGE> qp_state_xsit=%x",
my_qp, ibqp->qp_num, statetrans);
+ /* eHCA2 rev2 and higher require the SEND_GRH_FLAG to be set
+ * in non-LL UD QPs.
+ */
+ if ((my_qp->qp_type == IB_QPT_UD) &&
+ (my_qp->ext_type != EQPT_LLQP) &&
+ (statetrans == IB_QPST_INIT2RTR) &&
+ (shca->hw_level >= 0x22)){
+ update_mask |= EHCA_BMASK_SET(MQPCB_MASK_SEND_GRH_FLAG, 1);
+ mqpcb->send_grh_flag = 1;
+ }
+
/* sqe -> rts: set purge bit of bad wqe before actual trans */
if ((my_qp->qp_type == IB_QPT_UD ||
my_qp->qp_type == IB_QPT_GSI ||
--
1.5.2
^ permalink raw reply related
* [PATCH 07/13] IB/ehca: Report RDMA atomic attributes in query_qp()
From: Joachim Fenkes @ 2007-07-09 13:28 UTC (permalink / raw)
To: LinuxPPC-Dev, LKML, OF-General, Roland Dreier
Cc: Stefan Roscher, Christoph Raisch
In-Reply-To: <200707091502.22407.fenkes@de.ibm.com>
From: Hoang-Nam Nguyen <hnguyen@de.ibm.com>
Signed-off-by: Joachim Fenkes <fenkes@de.ibm.com>
---
drivers/infiniband/hw/ehca/ehca_qp.c | 3 +++
1 files changed, 3 insertions(+), 0 deletions(-)
diff --git a/drivers/infiniband/hw/ehca/ehca_qp.c b/drivers/infiniband/hw/ehca/ehca_qp.c
index cbb8b5b..989f75e 100644
--- a/drivers/infiniband/hw/ehca/ehca_qp.c
+++ b/drivers/infiniband/hw/ehca/ehca_qp.c
@@ -1491,6 +1491,9 @@ int ehca_query_qp(struct ib_qp *qp,
qp_attr->alt_port_num = qpcb->alt_phys_port;
qp_attr->alt_timeout = qpcb->timeout_al;
+ qp_attr->max_dest_rd_atomic = qpcb->rdma_nr_atomic_resp_res;
+ qp_attr->max_rd_atomic = qpcb->rdma_atomic_outst_dest_qp;
+
/* primary av */
qp_attr->ah_attr.sl = qpcb->service_level;
--
1.5.2
^ permalink raw reply related
* [PATCH 08/13] IB/ehca: Lock renaming, static initializers
From: Joachim Fenkes @ 2007-07-09 13:29 UTC (permalink / raw)
To: LinuxPPC-Dev, LKML, OF-General, Roland Dreier
Cc: Stefan Roscher, Christoph Raisch
In-Reply-To: <200707091502.22407.fenkes@de.ibm.com>
- Renamed all spinlock flags to "flags", matching the vast majority of kernel
code.
- Moved hcall_lock into the only module it's used in.
- Replaced spin_lock_init() and friends with static initializers for
global variables.
Signed-off-by: Joachim Fenkes <fenkes@de.ibm.com>
---
drivers/infiniband/hw/ehca/ehca_classes.h | 1 -
drivers/infiniband/hw/ehca/ehca_cq.c | 12 ++++++------
drivers/infiniband/hw/ehca/ehca_main.c | 18 ++++--------------
drivers/infiniband/hw/ehca/ehca_qp.c | 6 +++---
drivers/infiniband/hw/ehca/ehca_reqs.c | 24 ++++++++++++------------
drivers/infiniband/hw/ehca/hcp_if.c | 2 ++
6 files changed, 27 insertions(+), 36 deletions(-)
diff --git a/drivers/infiniband/hw/ehca/ehca_classes.h b/drivers/infiniband/hw/ehca/ehca_classes.h
index 9d689ae..3550047 100644
--- a/drivers/infiniband/hw/ehca/ehca_classes.h
+++ b/drivers/infiniband/hw/ehca/ehca_classes.h
@@ -295,7 +295,6 @@ void ehca_cleanup_mrmw_cache(void);
extern spinlock_t ehca_qp_idr_lock;
extern spinlock_t ehca_cq_idr_lock;
-extern spinlock_t hcall_lock;
extern struct idr ehca_qp_idr;
extern struct idr ehca_cq_idr;
diff --git a/drivers/infiniband/hw/ehca/ehca_cq.c b/drivers/infiniband/hw/ehca/ehca_cq.c
index 67f0670..94bad27 100644
--- a/drivers/infiniband/hw/ehca/ehca_cq.c
+++ b/drivers/infiniband/hw/ehca/ehca_cq.c
@@ -56,11 +56,11 @@ int ehca_cq_assign_qp(struct ehca_cq *cq, struct ehca_qp *qp)
{
unsigned int qp_num = qp->real_qp_num;
unsigned int key = qp_num & (QP_HASHTAB_LEN-1);
- unsigned long spl_flags;
+ unsigned long flags;
- spin_lock_irqsave(&cq->spinlock, spl_flags);
+ spin_lock_irqsave(&cq->spinlock, flags);
hlist_add_head(&qp->list_entries, &cq->qp_hashtab[key]);
- spin_unlock_irqrestore(&cq->spinlock, spl_flags);
+ spin_unlock_irqrestore(&cq->spinlock, flags);
ehca_dbg(cq->ib_cq.device, "cq_num=%x real_qp_num=%x",
cq->cq_number, qp_num);
@@ -74,9 +74,9 @@ int ehca_cq_unassign_qp(struct ehca_cq *cq, unsigned int real_qp_num)
unsigned int key = real_qp_num & (QP_HASHTAB_LEN-1);
struct hlist_node *iter;
struct ehca_qp *qp;
- unsigned long spl_flags;
+ unsigned long flags;
- spin_lock_irqsave(&cq->spinlock, spl_flags);
+ spin_lock_irqsave(&cq->spinlock, flags);
hlist_for_each(iter, &cq->qp_hashtab[key]) {
qp = hlist_entry(iter, struct ehca_qp, list_entries);
if (qp->real_qp_num == real_qp_num) {
@@ -88,7 +88,7 @@ int ehca_cq_unassign_qp(struct ehca_cq *cq, unsigned int real_qp_num)
break;
}
}
- spin_unlock_irqrestore(&cq->spinlock, spl_flags);
+ spin_unlock_irqrestore(&cq->spinlock, flags);
if (ret)
ehca_err(cq->ib_cq.device,
"qp not found cq_num=%x real_qp_num=%x",
diff --git a/drivers/infiniband/hw/ehca/ehca_main.c b/drivers/infiniband/hw/ehca/ehca_main.c
index 9bd749c..77db890 100644
--- a/drivers/infiniband/hw/ehca/ehca_main.c
+++ b/drivers/infiniband/hw/ehca/ehca_main.c
@@ -96,15 +96,13 @@ MODULE_PARM_DESC(static_rate,
MODULE_PARM_DESC(scaling_code,
"set scaling code (0: disabled/default, 1: enabled)");
-spinlock_t ehca_qp_idr_lock;
-spinlock_t ehca_cq_idr_lock;
-spinlock_t hcall_lock;
+DEFINE_SPINLOCK(ehca_qp_idr_lock);
+DEFINE_SPINLOCK(ehca_cq_idr_lock);
DEFINE_IDR(ehca_qp_idr);
DEFINE_IDR(ehca_cq_idr);
-
-static struct list_head shca_list; /* list of all registered ehcas */
-static spinlock_t shca_list_lock;
+static LIST_HEAD(shca_list); /* list of all registered ehcas */
+static DEFINE_SPINLOCK(shca_list_lock);
static struct timer_list poll_eqs_timer;
@@ -864,14 +862,6 @@ int __init ehca_module_init(void)
printk(KERN_INFO "eHCA Infiniband Device Driver "
"(Rel.: SVNEHCA_0023)\n");
- idr_init(&ehca_qp_idr);
- idr_init(&ehca_cq_idr);
- spin_lock_init(&ehca_qp_idr_lock);
- spin_lock_init(&ehca_cq_idr_lock);
- spin_lock_init(&hcall_lock);
-
- INIT_LIST_HEAD(&shca_list);
- spin_lock_init(&shca_list_lock);
if ((ret = ehca_create_comp_pool())) {
ehca_gen_err("Cannot create comp pool.");
diff --git a/drivers/infiniband/hw/ehca/ehca_qp.c b/drivers/infiniband/hw/ehca/ehca_qp.c
index 989f75e..ac4ff26 100644
--- a/drivers/infiniband/hw/ehca/ehca_qp.c
+++ b/drivers/infiniband/hw/ehca/ehca_qp.c
@@ -933,7 +933,7 @@ static int internal_modify_qp(struct ib_qp *ibqp,
u64 h_ret;
int bad_wqe_cnt = 0;
int squeue_locked = 0;
- unsigned long spl_flags = 0;
+ unsigned long flags = 0;
/* do query_qp to obtain current attr values */
mqpcb = ehca_alloc_fw_ctrlblock(GFP_KERNEL);
@@ -1074,7 +1074,7 @@ static int internal_modify_qp(struct ib_qp *ibqp,
if (!ibqp->uobject) {
struct ehca_wqe *wqe;
/* lock send queue */
- spin_lock_irqsave(&my_qp->spinlock_s, spl_flags);
+ spin_lock_irqsave(&my_qp->spinlock_s, flags);
squeue_locked = 1;
/* mark next free wqe */
wqe = (struct ehca_wqe*)
@@ -1360,7 +1360,7 @@ static int internal_modify_qp(struct ib_qp *ibqp,
modify_qp_exit2:
if (squeue_locked) { /* this means: sqe -> rts */
- spin_unlock_irqrestore(&my_qp->spinlock_s, spl_flags);
+ spin_unlock_irqrestore(&my_qp->spinlock_s, flags);
my_qp->sqerr_purgeflag = 1;
}
diff --git a/drivers/infiniband/hw/ehca/ehca_reqs.c b/drivers/infiniband/hw/ehca/ehca_reqs.c
index b5664fa..73f0c06 100644
--- a/drivers/infiniband/hw/ehca/ehca_reqs.c
+++ b/drivers/infiniband/hw/ehca/ehca_reqs.c
@@ -363,10 +363,10 @@ int ehca_post_send(struct ib_qp *qp,
struct ehca_wqe *wqe_p;
int wqe_cnt = 0;
int ret = 0;
- unsigned long spl_flags;
+ unsigned long flags;
/* LOCK the QUEUE */
- spin_lock_irqsave(&my_qp->spinlock_s, spl_flags);
+ spin_lock_irqsave(&my_qp->spinlock_s, flags);
/* loop processes list of send reqs */
for (cur_send_wr = send_wr; cur_send_wr != NULL;
@@ -408,7 +408,7 @@ int ehca_post_send(struct ib_qp *qp,
post_send_exit0:
/* UNLOCK the QUEUE */
- spin_unlock_irqrestore(&my_qp->spinlock_s, spl_flags);
+ spin_unlock_irqrestore(&my_qp->spinlock_s, flags);
iosync(); /* serialize GAL register access */
hipz_update_sqa(my_qp, wqe_cnt);
return ret;
@@ -423,7 +423,7 @@ static int internal_post_recv(struct ehca_qp *my_qp,
struct ehca_wqe *wqe_p;
int wqe_cnt = 0;
int ret = 0;
- unsigned long spl_flags;
+ unsigned long flags;
if (unlikely(!HAS_RQ(my_qp))) {
ehca_err(dev, "QP has no RQ ehca_qp=%p qp_num=%x ext_type=%d",
@@ -432,7 +432,7 @@ static int internal_post_recv(struct ehca_qp *my_qp,
}
/* LOCK the QUEUE */
- spin_lock_irqsave(&my_qp->spinlock_r, spl_flags);
+ spin_lock_irqsave(&my_qp->spinlock_r, flags);
/* loop processes list of send reqs */
for (cur_recv_wr = recv_wr; cur_recv_wr != NULL;
@@ -473,7 +473,7 @@ static int internal_post_recv(struct ehca_qp *my_qp,
} /* eof for cur_recv_wr */
post_recv_exit0:
- spin_unlock_irqrestore(&my_qp->spinlock_r, spl_flags);
+ spin_unlock_irqrestore(&my_qp->spinlock_r, flags);
iosync(); /* serialize GAL register access */
hipz_update_rqa(my_qp, wqe_cnt);
return ret;
@@ -536,7 +536,7 @@ poll_cq_one_read_cqe:
if (unlikely(cqe->status & WC_STATUS_PURGE_BIT)) {
struct ehca_qp *qp=ehca_cq_get_qp(my_cq, cqe->local_qp_number);
int purgeflag;
- unsigned long spl_flags;
+ unsigned long flags;
if (!qp) {
ehca_err(cq->device, "cq_num=%x qp_num=%x "
"could not find qp -> ignore cqe",
@@ -546,9 +546,9 @@ poll_cq_one_read_cqe:
/* ignore this purged cqe */
goto poll_cq_one_read_cqe;
}
- spin_lock_irqsave(&qp->spinlock_s, spl_flags);
+ spin_lock_irqsave(&qp->spinlock_s, flags);
purgeflag = qp->sqerr_purgeflag;
- spin_unlock_irqrestore(&qp->spinlock_s, spl_flags);
+ spin_unlock_irqrestore(&qp->spinlock_s, flags);
if (purgeflag) {
ehca_dbg(cq->device, "Got CQE with purged bit qp_num=%x "
@@ -633,7 +633,7 @@ int ehca_poll_cq(struct ib_cq *cq, int num_entries, struct ib_wc *wc)
int nr;
struct ib_wc *current_wc = wc;
int ret = 0;
- unsigned long spl_flags;
+ unsigned long flags;
if (num_entries < 1) {
ehca_err(cq->device, "Invalid num_entries=%d ehca_cq=%p "
@@ -642,14 +642,14 @@ int ehca_poll_cq(struct ib_cq *cq, int num_entries, struct ib_wc *wc)
goto poll_cq_exit0;
}
- spin_lock_irqsave(&my_cq->spinlock, spl_flags);
+ spin_lock_irqsave(&my_cq->spinlock, flags);
for (nr = 0; nr < num_entries; nr++) {
ret = ehca_poll_cq_one(cq, current_wc);
if (ret)
break;
current_wc++;
} /* eof for nr */
- spin_unlock_irqrestore(&my_cq->spinlock, spl_flags);
+ spin_unlock_irqrestore(&my_cq->spinlock, flags);
if (ret == -EAGAIN || !ret)
ret = nr;
diff --git a/drivers/infiniband/hw/ehca/hcp_if.c b/drivers/infiniband/hw/ehca/hcp_if.c
index b078377..5b927a6 100644
--- a/drivers/infiniband/hw/ehca/hcp_if.c
+++ b/drivers/infiniband/hw/ehca/hcp_if.c
@@ -81,6 +81,8 @@
#define H_MP_SHUTDOWN EHCA_BMASK_IBM(48, 48)
#define H_MP_RESET_QKEY_CTR EHCA_BMASK_IBM(49, 49)
+DEFINE_SPINLOCK(hcall_lock);
+
static u32 get_longbusy_msecs(int longbusy_rc)
{
switch (longbusy_rc) {
--
1.5.2
^ permalink raw reply related
* [PATCH 09/13] IB/ehca: Refactor synchronization between completions and destroy_cq using atomic_t
From: Joachim Fenkes @ 2007-07-09 13:30 UTC (permalink / raw)
To: LinuxPPC-Dev, LKML, OF-General, Roland Dreier
Cc: Stefan Roscher, Christoph Raisch
In-Reply-To: <200707091502.22407.fenkes@de.ibm.com>
- ehca_cq.nr_events is made an atomic_t, eliminating a lot of locking.
- The CQ is removed from the CQ idr first now to make sure no more
completions are scheduled on that CQ. The "wait for all completions to
end" code becomes much simpler this way.
Signed-off-by: Joachim Fenkes <fenkes@de.ibm.com>
---
drivers/infiniband/hw/ehca/ehca_classes.h | 4 +-
drivers/infiniband/hw/ehca/ehca_cq.c | 26 +++++++-------------
drivers/infiniband/hw/ehca/ehca_irq.c | 36 +++++++++++++---------------
drivers/infiniband/hw/ehca/ehca_irq.h | 1 -
drivers/infiniband/hw/ehca/ehca_tools.h | 1 +
5 files changed, 29 insertions(+), 39 deletions(-)
diff --git a/drivers/infiniband/hw/ehca/ehca_classes.h b/drivers/infiniband/hw/ehca/ehca_classes.h
index 3550047..8580f2a 100644
--- a/drivers/infiniband/hw/ehca/ehca_classes.h
+++ b/drivers/infiniband/hw/ehca/ehca_classes.h
@@ -174,8 +174,8 @@ struct ehca_cq {
spinlock_t cb_lock;
struct hlist_head qp_hashtab[QP_HASHTAB_LEN];
struct list_head entry;
- u32 nr_callbacks; /* #events assigned to cpu by scaling code */
- u32 nr_events; /* #events seen */
+ u32 nr_callbacks; /* #events assigned to cpu by scaling code */
+ atomic_t nr_events; /* #events seen */
wait_queue_head_t wait_completion;
spinlock_t task_lock;
u32 ownpid;
diff --git a/drivers/infiniband/hw/ehca/ehca_cq.c b/drivers/infiniband/hw/ehca/ehca_cq.c
index 94bad27..3729997 100644
--- a/drivers/infiniband/hw/ehca/ehca_cq.c
+++ b/drivers/infiniband/hw/ehca/ehca_cq.c
@@ -146,6 +146,7 @@ struct ib_cq *ehca_create_cq(struct ib_device *device, int cqe, int comp_vector,
spin_lock_init(&my_cq->spinlock);
spin_lock_init(&my_cq->cb_lock);
spin_lock_init(&my_cq->task_lock);
+ atomic_set(&my_cq->nr_events, 0);
init_waitqueue_head(&my_cq->wait_completion);
my_cq->ownpid = current->tgid;
@@ -303,16 +304,6 @@ create_cq_exit1:
return cq;
}
-static int get_cq_nr_events(struct ehca_cq *my_cq)
-{
- int ret;
- unsigned long flags;
- spin_lock_irqsave(&ehca_cq_idr_lock, flags);
- ret = my_cq->nr_events;
- spin_unlock_irqrestore(&ehca_cq_idr_lock, flags);
- return ret;
-}
-
int ehca_destroy_cq(struct ib_cq *cq)
{
u64 h_ret;
@@ -339,17 +330,18 @@ int ehca_destroy_cq(struct ib_cq *cq)
}
}
+ /*
+ * remove the CQ from the idr first to make sure
+ * no more interrupt tasklets will touch this CQ
+ */
spin_lock_irqsave(&ehca_cq_idr_lock, flags);
- while (my_cq->nr_events) {
- spin_unlock_irqrestore(&ehca_cq_idr_lock, flags);
- wait_event(my_cq->wait_completion, !get_cq_nr_events(my_cq));
- spin_lock_irqsave(&ehca_cq_idr_lock, flags);
- /* recheck nr_events to assure no cqe has just arrived */
- }
-
idr_remove(&ehca_cq_idr, my_cq->token);
spin_unlock_irqrestore(&ehca_cq_idr_lock, flags);
+ /* now wait until all pending events have completed */
+ wait_event(my_cq->wait_completion, !atomic_read(&my_cq->nr_events));
+
+ /* nobody's using our CQ any longer -- we can destroy it */
h_ret = hipz_h_destroy_cq(adapter_handle, my_cq, 0);
if (h_ret == H_R_STATE) {
/* cq in err: read err data and destroy it forcibly */
diff --git a/drivers/infiniband/hw/ehca/ehca_irq.c b/drivers/infiniband/hw/ehca/ehca_irq.c
index 100329b..3e790a3 100644
--- a/drivers/infiniband/hw/ehca/ehca_irq.c
+++ b/drivers/infiniband/hw/ehca/ehca_irq.c
@@ -5,6 +5,8 @@
*
* Authors: Heiko J Schick <schickhj@de.ibm.com>
* Khadija Souissi <souissi@de.ibm.com>
+ * Hoang-Nam Nguyen <hnguyen@de.ibm.com>
+ * Joachim Fenkes <fenkes@de.ibm.com>
*
* Copyright (c) 2005 IBM Corporation
*
@@ -212,6 +214,8 @@ static void cq_event_callback(struct ehca_shca *shca,
spin_lock_irqsave(&ehca_cq_idr_lock, flags);
cq = idr_find(&ehca_cq_idr, token);
+ if (cq)
+ atomic_inc(&cq->nr_events);
spin_unlock_irqrestore(&ehca_cq_idr_lock, flags);
if (!cq)
@@ -219,6 +223,9 @@ static void cq_event_callback(struct ehca_shca *shca,
ehca_error_data(shca, cq, cq->ipz_cq_handle.handle);
+ if (atomic_dec_and_test(&cq->nr_events))
+ wake_up(&cq->wait_completion);
+
return;
}
@@ -414,25 +421,22 @@ static inline void process_eqe(struct ehca_shca *shca, struct ehca_eqe *eqe)
token = EHCA_BMASK_GET(EQE_CQ_TOKEN, eqe_value);
spin_lock_irqsave(&ehca_cq_idr_lock, flags);
cq = idr_find(&ehca_cq_idr, token);
+ if (cq)
+ atomic_inc(&cq->nr_events);
+ spin_unlock_irqrestore(&ehca_cq_idr_lock, flags);
if (cq == NULL) {
- spin_unlock_irqrestore(&ehca_cq_idr_lock, flags);
ehca_err(&shca->ib_device,
"Invalid eqe for non-existing cq token=%x",
token);
return;
}
reset_eq_pending(cq);
- cq->nr_events++;
- spin_unlock_irqrestore(&ehca_cq_idr_lock, flags);
if (ehca_scaling_code)
queue_comp_task(cq);
else {
comp_event_callback(cq);
- spin_lock_irqsave(&ehca_cq_idr_lock, flags);
- cq->nr_events--;
- if (!cq->nr_events)
+ if (atomic_dec_and_test(&cq->nr_events))
wake_up(&cq->wait_completion);
- spin_unlock_irqrestore(&ehca_cq_idr_lock, flags);
}
} else {
ehca_dbg(&shca->ib_device, "Got non completion event");
@@ -478,15 +482,15 @@ void ehca_process_eq(struct ehca_shca *shca, int is_irq)
token = EHCA_BMASK_GET(EQE_CQ_TOKEN, eqe_value);
spin_lock(&ehca_cq_idr_lock);
eqe_cache[eqe_cnt].cq = idr_find(&ehca_cq_idr, token);
+ if (eqe_cache[eqe_cnt].cq)
+ atomic_inc(&eqe_cache[eqe_cnt].cq->nr_events);
+ spin_unlock(&ehca_cq_idr_lock);
if (!eqe_cache[eqe_cnt].cq) {
- spin_unlock(&ehca_cq_idr_lock);
ehca_err(&shca->ib_device,
"Invalid eqe for non-existing cq "
"token=%x", token);
continue;
}
- eqe_cache[eqe_cnt].cq->nr_events++;
- spin_unlock(&ehca_cq_idr_lock);
} else
eqe_cache[eqe_cnt].cq = NULL;
eqe_cnt++;
@@ -517,11 +521,8 @@ void ehca_process_eq(struct ehca_shca *shca, int is_irq)
else {
struct ehca_cq *cq = eq->eqe_cache[i].cq;
comp_event_callback(cq);
- spin_lock(&ehca_cq_idr_lock);
- cq->nr_events--;
- if (!cq->nr_events)
+ if (atomic_dec_and_test(&cq->nr_events))
wake_up(&cq->wait_completion);
- spin_unlock(&ehca_cq_idr_lock);
}
} else {
ehca_dbg(&shca->ib_device, "Got non completion event");
@@ -621,13 +622,10 @@ static void run_comp_task(struct ehca_cpu_comp_task* cct)
while (!list_empty(&cct->cq_list)) {
cq = list_entry(cct->cq_list.next, struct ehca_cq, entry);
spin_unlock_irqrestore(&cct->task_lock, flags);
- comp_event_callback(cq);
- spin_lock_irqsave(&ehca_cq_idr_lock, flags);
- cq->nr_events--;
- if (!cq->nr_events)
+ comp_event_callback(cq);
+ if (atomic_dec_and_test(&cq->nr_events))
wake_up(&cq->wait_completion);
- spin_unlock_irqrestore(&ehca_cq_idr_lock, flags);
spin_lock_irqsave(&cct->task_lock, flags);
spin_lock(&cq->task_lock);
diff --git a/drivers/infiniband/hw/ehca/ehca_irq.h b/drivers/infiniband/hw/ehca/ehca_irq.h
index 6ed06ee..3346cb0 100644
--- a/drivers/infiniband/hw/ehca/ehca_irq.h
+++ b/drivers/infiniband/hw/ehca/ehca_irq.h
@@ -47,7 +47,6 @@ struct ehca_shca;
#include <linux/interrupt.h>
#include <linux/types.h>
-#include <asm/atomic.h>
int ehca_error_data(struct ehca_shca *shca, void *data, u64 resource);
diff --git a/drivers/infiniband/hw/ehca/ehca_tools.h b/drivers/infiniband/hw/ehca/ehca_tools.h
index 973c4b5..03b185f 100644
--- a/drivers/infiniband/hw/ehca/ehca_tools.h
+++ b/drivers/infiniband/hw/ehca/ehca_tools.h
@@ -59,6 +59,7 @@
#include <linux/cpu.h>
#include <linux/device.h>
+#include <asm/atomic.h>
#include <asm/abs_addr.h>
#include <asm/ibmebus.h>
#include <asm/io.h>
--
1.5.2
^ permalink raw reply related
* [PATCH 10/13] IB/ehca: Change idr spinlocks into rwlocks
From: Joachim Fenkes @ 2007-07-09 13:31 UTC (permalink / raw)
To: LinuxPPC-Dev, LKML, OF-General, Roland Dreier
Cc: Stefan Roscher, Christoph Raisch
In-Reply-To: <200707091502.22407.fenkes@de.ibm.com>
This eliminates lock contention among IRQs as well as the need to disable
IRQs around idr_find, because there are no IRQ writers.
Signed-off-by: Joachim Fenkes <fenkes@de.ibm.com>
---
drivers/infiniband/hw/ehca/ehca_classes.h | 4 ++--
drivers/infiniband/hw/ehca/ehca_cq.c | 12 ++++++------
drivers/infiniband/hw/ehca/ehca_irq.c | 19 ++++++++-----------
drivers/infiniband/hw/ehca/ehca_main.c | 4 ++--
drivers/infiniband/hw/ehca/ehca_qp.c | 12 ++++++------
drivers/infiniband/hw/ehca/ehca_uverbs.c | 9 ++++-----
6 files changed, 28 insertions(+), 32 deletions(-)
diff --git a/drivers/infiniband/hw/ehca/ehca_classes.h b/drivers/infiniband/hw/ehca/ehca_classes.h
index 8580f2a..f1e0db2 100644
--- a/drivers/infiniband/hw/ehca/ehca_classes.h
+++ b/drivers/infiniband/hw/ehca/ehca_classes.h
@@ -293,8 +293,8 @@ void ehca_cleanup_av_cache(void);
int ehca_init_mrmw_cache(void);
void ehca_cleanup_mrmw_cache(void);
-extern spinlock_t ehca_qp_idr_lock;
-extern spinlock_t ehca_cq_idr_lock;
+extern rwlock_t ehca_qp_idr_lock;
+extern rwlock_t ehca_cq_idr_lock;
extern struct idr ehca_qp_idr;
extern struct idr ehca_cq_idr;
diff --git a/drivers/infiniband/hw/ehca/ehca_cq.c b/drivers/infiniband/hw/ehca/ehca_cq.c
index 3729997..01d4a14 100644
--- a/drivers/infiniband/hw/ehca/ehca_cq.c
+++ b/drivers/infiniband/hw/ehca/ehca_cq.c
@@ -163,9 +163,9 @@ struct ib_cq *ehca_create_cq(struct ib_device *device, int cqe, int comp_vector,
goto create_cq_exit1;
}
- spin_lock_irqsave(&ehca_cq_idr_lock, flags);
+ write_lock_irqsave(&ehca_cq_idr_lock, flags);
ret = idr_get_new(&ehca_cq_idr, my_cq, &my_cq->token);
- spin_unlock_irqrestore(&ehca_cq_idr_lock, flags);
+ write_unlock_irqrestore(&ehca_cq_idr_lock, flags);
} while (ret == -EAGAIN);
@@ -294,9 +294,9 @@ create_cq_exit3:
"cq_num=%x h_ret=%lx", my_cq, my_cq->cq_number, h_ret);
create_cq_exit2:
- spin_lock_irqsave(&ehca_cq_idr_lock, flags);
+ write_lock_irqsave(&ehca_cq_idr_lock, flags);
idr_remove(&ehca_cq_idr, my_cq->token);
- spin_unlock_irqrestore(&ehca_cq_idr_lock, flags);
+ write_unlock_irqrestore(&ehca_cq_idr_lock, flags);
create_cq_exit1:
kmem_cache_free(cq_cache, my_cq);
@@ -334,9 +334,9 @@ int ehca_destroy_cq(struct ib_cq *cq)
* remove the CQ from the idr first to make sure
* no more interrupt tasklets will touch this CQ
*/
- spin_lock_irqsave(&ehca_cq_idr_lock, flags);
+ write_lock_irqsave(&ehca_cq_idr_lock, flags);
idr_remove(&ehca_cq_idr, my_cq->token);
- spin_unlock_irqrestore(&ehca_cq_idr_lock, flags);
+ write_unlock_irqrestore(&ehca_cq_idr_lock, flags);
/* now wait until all pending events have completed */
wait_event(my_cq->wait_completion, !atomic_read(&my_cq->nr_events));
diff --git a/drivers/infiniband/hw/ehca/ehca_irq.c b/drivers/infiniband/hw/ehca/ehca_irq.c
index 3e790a3..02b73c8 100644
--- a/drivers/infiniband/hw/ehca/ehca_irq.c
+++ b/drivers/infiniband/hw/ehca/ehca_irq.c
@@ -180,12 +180,11 @@ static void qp_event_callback(struct ehca_shca *shca,
{
struct ib_event event;
struct ehca_qp *qp;
- unsigned long flags;
u32 token = EHCA_BMASK_GET(EQE_QP_TOKEN, eqe);
- spin_lock_irqsave(&ehca_qp_idr_lock, flags);
+ read_lock(&ehca_qp_idr_lock);
qp = idr_find(&ehca_qp_idr, token);
- spin_unlock_irqrestore(&ehca_qp_idr_lock, flags);
+ read_unlock(&ehca_qp_idr_lock);
if (!qp)
@@ -209,14 +208,13 @@ static void cq_event_callback(struct ehca_shca *shca,
u64 eqe)
{
struct ehca_cq *cq;
- unsigned long flags;
u32 token = EHCA_BMASK_GET(EQE_CQ_TOKEN, eqe);
- spin_lock_irqsave(&ehca_cq_idr_lock, flags);
+ read_lock(&ehca_cq_idr_lock);
cq = idr_find(&ehca_cq_idr, token);
if (cq)
atomic_inc(&cq->nr_events);
- spin_unlock_irqrestore(&ehca_cq_idr_lock, flags);
+ read_unlock(&ehca_cq_idr_lock);
if (!cq)
return;
@@ -411,7 +409,6 @@ static inline void process_eqe(struct ehca_shca *shca, struct ehca_eqe *eqe)
{
u64 eqe_value;
u32 token;
- unsigned long flags;
struct ehca_cq *cq;
eqe_value = eqe->entry;
@@ -419,11 +416,11 @@ static inline void process_eqe(struct ehca_shca *shca, struct ehca_eqe *eqe)
if (EHCA_BMASK_GET(EQE_COMPLETION_EVENT, eqe_value)) {
ehca_dbg(&shca->ib_device, "Got completion event");
token = EHCA_BMASK_GET(EQE_CQ_TOKEN, eqe_value);
- spin_lock_irqsave(&ehca_cq_idr_lock, flags);
+ read_lock(&ehca_cq_idr_lock);
cq = idr_find(&ehca_cq_idr, token);
if (cq)
atomic_inc(&cq->nr_events);
- spin_unlock_irqrestore(&ehca_cq_idr_lock, flags);
+ read_unlock(&ehca_cq_idr_lock);
if (cq == NULL) {
ehca_err(&shca->ib_device,
"Invalid eqe for non-existing cq token=%x",
@@ -480,11 +477,11 @@ void ehca_process_eq(struct ehca_shca *shca, int is_irq)
eqe_value = eqe_cache[eqe_cnt].eqe->entry;
if (EHCA_BMASK_GET(EQE_COMPLETION_EVENT, eqe_value)) {
token = EHCA_BMASK_GET(EQE_CQ_TOKEN, eqe_value);
- spin_lock(&ehca_cq_idr_lock);
+ read_lock(&ehca_cq_idr_lock);
eqe_cache[eqe_cnt].cq = idr_find(&ehca_cq_idr, token);
if (eqe_cache[eqe_cnt].cq)
atomic_inc(&eqe_cache[eqe_cnt].cq->nr_events);
- spin_unlock(&ehca_cq_idr_lock);
+ read_unlock(&ehca_cq_idr_lock);
if (!eqe_cache[eqe_cnt].cq) {
ehca_err(&shca->ib_device,
"Invalid eqe for non-existing cq "
diff --git a/drivers/infiniband/hw/ehca/ehca_main.c b/drivers/infiniband/hw/ehca/ehca_main.c
index 77db890..e58e821 100644
--- a/drivers/infiniband/hw/ehca/ehca_main.c
+++ b/drivers/infiniband/hw/ehca/ehca_main.c
@@ -96,8 +96,8 @@ MODULE_PARM_DESC(static_rate,
MODULE_PARM_DESC(scaling_code,
"set scaling code (0: disabled/default, 1: enabled)");
-DEFINE_SPINLOCK(ehca_qp_idr_lock);
-DEFINE_SPINLOCK(ehca_cq_idr_lock);
+DEFINE_RWLOCK(ehca_qp_idr_lock);
+DEFINE_RWLOCK(ehca_cq_idr_lock);
DEFINE_IDR(ehca_qp_idr);
DEFINE_IDR(ehca_cq_idr);
diff --git a/drivers/infiniband/hw/ehca/ehca_qp.c b/drivers/infiniband/hw/ehca/ehca_qp.c
index ac4ff26..7452ef4 100644
--- a/drivers/infiniband/hw/ehca/ehca_qp.c
+++ b/drivers/infiniband/hw/ehca/ehca_qp.c
@@ -512,9 +512,9 @@ struct ehca_qp *internal_create_qp(struct ib_pd *pd,
goto create_qp_exit0;
}
- spin_lock_irqsave(&ehca_qp_idr_lock, flags);
+ write_lock_irqsave(&ehca_qp_idr_lock, flags);
ret = idr_get_new(&ehca_qp_idr, my_qp, &my_qp->token);
- spin_unlock_irqrestore(&ehca_qp_idr_lock, flags);
+ write_unlock_irqrestore(&ehca_qp_idr_lock, flags);
} while (ret == -EAGAIN);
@@ -733,9 +733,9 @@ create_qp_exit2:
hipz_h_destroy_qp(shca->ipz_hca_handle, my_qp);
create_qp_exit1:
- spin_lock_irqsave(&ehca_qp_idr_lock, flags);
+ write_lock_irqsave(&ehca_qp_idr_lock, flags);
idr_remove(&ehca_qp_idr, my_qp->token);
- spin_unlock_irqrestore(&ehca_qp_idr_lock, flags);
+ write_unlock_irqrestore(&ehca_qp_idr_lock, flags);
create_qp_exit0:
kmem_cache_free(qp_cache, my_qp);
@@ -1706,9 +1706,9 @@ int internal_destroy_qp(struct ib_device *dev, struct ehca_qp *my_qp,
}
}
- spin_lock_irqsave(&ehca_qp_idr_lock, flags);
+ write_lock_irqsave(&ehca_qp_idr_lock, flags);
idr_remove(&ehca_qp_idr, my_qp->token);
- spin_unlock_irqrestore(&ehca_qp_idr_lock, flags);
+ write_unlock_irqrestore(&ehca_qp_idr_lock, flags);
h_ret = hipz_h_destroy_qp(shca->ipz_hca_handle, my_qp);
if (h_ret != H_SUCCESS) {
diff --git a/drivers/infiniband/hw/ehca/ehca_uverbs.c b/drivers/infiniband/hw/ehca/ehca_uverbs.c
index d8fe37d..3031b3b 100644
--- a/drivers/infiniband/hw/ehca/ehca_uverbs.c
+++ b/drivers/infiniband/hw/ehca/ehca_uverbs.c
@@ -253,7 +253,6 @@ int ehca_mmap(struct ib_ucontext *context, struct vm_area_struct *vma)
u32 rsrc_type = (fileoffset >> 24) & 0xF; /* sq,rq,cmnd_window */
u32 cur_pid = current->tgid;
u32 ret;
- unsigned long flags;
struct ehca_cq *cq;
struct ehca_qp *qp;
struct ehca_pd *pd;
@@ -261,9 +260,9 @@ int ehca_mmap(struct ib_ucontext *context, struct vm_area_struct *vma)
switch (q_type) {
case 1: /* CQ */
- spin_lock_irqsave(&ehca_cq_idr_lock, flags);
+ read_lock(&ehca_cq_idr_lock);
cq = idr_find(&ehca_cq_idr, idr_handle);
- spin_unlock_irqrestore(&ehca_cq_idr_lock, flags);
+ read_unlock(&ehca_cq_idr_lock);
/* make sure this mmap really belongs to the authorized user */
if (!cq)
@@ -289,9 +288,9 @@ int ehca_mmap(struct ib_ucontext *context, struct vm_area_struct *vma)
break;
case 2: /* QP */
- spin_lock_irqsave(&ehca_qp_idr_lock, flags);
+ read_lock(&ehca_qp_idr_lock);
qp = idr_find(&ehca_qp_idr, idr_handle);
- spin_unlock_irqrestore(&ehca_qp_idr_lock, flags);
+ read_unlock(&ehca_qp_idr_lock);
/* make sure this mmap really belongs to the authorized user */
if (!qp)
--
1.5.2
^ permalink raw reply related
* [PATCH 11/13] IB/ehca: return QP pointer in poll_cq(), add two unlikely() statements
From: Joachim Fenkes @ 2007-07-09 13:31 UTC (permalink / raw)
To: LinuxPPC-Dev, LKML, OF-General, Roland Dreier
Cc: Stefan Roscher, Christoph Raisch
In-Reply-To: <200707091502.22407.fenkes@de.ibm.com>
Signed-off-by: Joachim Fenkes <fenkes@de.ibm.com>
---
drivers/infiniband/hw/ehca/ehca_reqs.c | 11 ++++++++---
1 files changed, 8 insertions(+), 3 deletions(-)
diff --git a/drivers/infiniband/hw/ehca/ehca_reqs.c b/drivers/infiniband/hw/ehca/ehca_reqs.c
index 73f0c06..fd3ba22 100644
--- a/drivers/infiniband/hw/ehca/ehca_reqs.c
+++ b/drivers/infiniband/hw/ehca/ehca_reqs.c
@@ -517,6 +517,7 @@ static inline int ehca_poll_cq_one(struct ib_cq *cq, struct ib_wc *wc)
int ret = 0;
struct ehca_cq *my_cq = container_of(cq, struct ehca_cq, ib_cq);
struct ehca_cqe *cqe;
+ struct ehca_qp *my_qp;
int cqe_count = 0;
poll_cq_one_read_cqe:
@@ -568,7 +569,7 @@ poll_cq_one_read_cqe:
}
/* tracing cqe */
- if (ehca_debug_level) {
+ if (unlikely(ehca_debug_level)) {
ehca_dbg(cq->device,
"Received COMPLETION ehca_cq=%p cq_num=%x -----",
my_cq, my_cq->cq_number);
@@ -602,7 +603,11 @@ poll_cq_one_read_cqe:
} else
wc->status = IB_WC_SUCCESS;
- wc->qp = NULL;
+ read_lock(&ehca_qp_idr_lock);
+ my_qp = idr_find(&ehca_qp_idr, cqe->qp_token);
+ wc->qp = &my_qp->ib_qp;
+ read_unlock(&ehca_qp_idr_lock);
+
wc->byte_len = cqe->nr_bytes_transferred;
wc->pkey_index = cqe->pkey_index;
wc->slid = cqe->rlid;
@@ -612,7 +617,7 @@ poll_cq_one_read_cqe:
wc->imm_data = cpu_to_be32(cqe->immediate_data);
wc->sl = cqe->service_level;
- if (wc->status != IB_WC_SUCCESS)
+ if (unlikely(wc->status != IB_WC_SUCCESS))
ehca_dbg(cq->device,
"ehca_cq=%p cq_num=%x WARNING unsuccessful cqe "
"OPType=%x status=%x qp_num=%x src_qp=%x wr_id=%lx "
--
1.5.2
^ permalink raw reply related
* [PATCH 12/13] IB/ehca: notify consumers of LID/PKEY/SM changes after nondisruptive events
From: Joachim Fenkes @ 2007-07-09 13:32 UTC (permalink / raw)
To: LinuxPPC-Dev, LKML, OF-General, Roland Dreier
Cc: Stefan Roscher, Christoph Raisch
In-Reply-To: <200707091502.22407.fenkes@de.ibm.com>
When firmware reports a nondisruptive port configuration change event,
previous versions of the eHCA driver didn't forward the event to consumers
like IPoIB. Add code that determines the type of configuration change by
comparing old and new port attributes and reports it.
Signed-off-by: Joachim Fenkes <fenkes@de.ibm.com>
---
drivers/infiniband/hw/ehca/ehca_classes.h | 6 ++
drivers/infiniband/hw/ehca/ehca_hca.c | 34 +++++++++++
drivers/infiniband/hw/ehca/ehca_irq.c | 89 +++++++++++++++++++----------
drivers/infiniband/hw/ehca/ehca_iverbs.h | 3 +
4 files changed, 101 insertions(+), 31 deletions(-)
diff --git a/drivers/infiniband/hw/ehca/ehca_classes.h b/drivers/infiniband/hw/ehca/ehca_classes.h
index f1e0db2..daf823e 100644
--- a/drivers/infiniband/hw/ehca/ehca_classes.h
+++ b/drivers/infiniband/hw/ehca/ehca_classes.h
@@ -87,11 +87,17 @@ struct ehca_eq {
struct ehca_eqe_cache_entry eqe_cache[EHCA_EQE_CACHE_SIZE];
};
+struct ehca_sma_attr {
+ u16 lid, lmc, sm_sl, sm_lid;
+ u16 pkey_tbl_len, pkeys[16];
+};
+
struct ehca_sport {
struct ib_cq *ibcq_aqp1;
struct ib_qp *ibqp_aqp1;
enum ib_rate rate;
enum ib_port_state port_state;
+ struct ehca_sma_attr saved_attr;
};
struct ehca_shca {
diff --git a/drivers/infiniband/hw/ehca/ehca_hca.c b/drivers/infiniband/hw/ehca/ehca_hca.c
index b310de5..bbd3c6a 100644
--- a/drivers/infiniband/hw/ehca/ehca_hca.c
+++ b/drivers/infiniband/hw/ehca/ehca_hca.c
@@ -193,6 +193,40 @@ query_port1:
return ret;
}
+int ehca_query_sma_attr(struct ehca_shca *shca,
+ u8 port, struct ehca_sma_attr *attr)
+{
+ int ret = 0;
+ struct hipz_query_port *rblock;
+
+ rblock = ehca_alloc_fw_ctrlblock(GFP_ATOMIC);
+ if (!rblock) {
+ ehca_err(&shca->ib_device, "Can't allocate rblock memory.");
+ return -ENOMEM;
+ }
+
+ if (hipz_h_query_port(shca->ipz_hca_handle, port, rblock) != H_SUCCESS) {
+ ehca_err(&shca->ib_device, "Can't query port properties");
+ ret = -EINVAL;
+ goto query_sma_attr1;
+ }
+
+ memset(attr, 0, sizeof(struct ehca_sma_attr));
+
+ attr->lid = rblock->lid;
+ attr->lmc = rblock->lmc;
+ attr->sm_sl = rblock->sm_sl;
+ attr->sm_lid = rblock->sm_lid;
+
+ attr->pkey_tbl_len = rblock->pkey_tbl_len;
+ memcpy(attr->pkeys, rblock->pkey_entries, sizeof(attr->pkeys));
+
+query_sma_attr1:
+ ehca_free_fw_ctrlblock(rblock);
+
+ return ret;
+}
+
int ehca_query_pkey(struct ib_device *ibdev, u8 port, u16 index, u16 *pkey)
{
int ret = 0;
diff --git a/drivers/infiniband/hw/ehca/ehca_irq.c b/drivers/infiniband/hw/ehca/ehca_irq.c
index 02b73c8..96eba38 100644
--- a/drivers/infiniband/hw/ehca/ehca_irq.c
+++ b/drivers/infiniband/hw/ehca/ehca_irq.c
@@ -61,6 +61,7 @@
#define NEQE_EVENT_CODE EHCA_BMASK_IBM(2,7)
#define NEQE_PORT_NUMBER EHCA_BMASK_IBM(8,15)
#define NEQE_PORT_AVAILABILITY EHCA_BMASK_IBM(16,16)
+#define NEQE_DISRUPTIVE EHCA_BMASK_IBM(16,16)
#define ERROR_DATA_LENGTH EHCA_BMASK_IBM(52,63)
#define ERROR_DATA_TYPE EHCA_BMASK_IBM(0,7)
@@ -286,30 +287,61 @@ static void parse_identifier(struct ehca_shca *shca, u64 eqe)
return;
}
-static void parse_ec(struct ehca_shca *shca, u64 eqe)
+static void dispatch_port_event(struct ehca_shca *shca, int port_num,
+ enum ib_event_type type, const char *msg)
{
struct ib_event event;
+
+ ehca_info(&shca->ib_device, "port %d %s.", port_num, msg);
+ event.device = &shca->ib_device;
+ event.event = type;
+ event.element.port_num = port_num;
+ ib_dispatch_event(&event);
+}
+
+static void notify_port_conf_change(struct ehca_shca *shca, int port_num)
+{
+ struct ehca_sma_attr new_attr;
+ struct ehca_sma_attr *old_attr = &shca->sport[port_num - 1].saved_attr;
+
+ ehca_query_sma_attr(shca, port_num, &new_attr);
+
+ if (new_attr.sm_sl != old_attr->sm_sl ||
+ new_attr.sm_lid != old_attr->sm_lid)
+ dispatch_port_event(shca, port_num, IB_EVENT_SM_CHANGE,
+ "SM changed");
+
+ if (new_attr.lid != old_attr->lid ||
+ new_attr.lmc != old_attr->lmc)
+ dispatch_port_event(shca, port_num, IB_EVENT_LID_CHANGE,
+ "LID changed");
+
+ if (new_attr.pkey_tbl_len != old_attr->pkey_tbl_len ||
+ memcmp(new_attr.pkeys, old_attr->pkeys,
+ sizeof(u16) * new_attr.pkey_tbl_len))
+ dispatch_port_event(shca, port_num, IB_EVENT_PKEY_CHANGE,
+ "P_Key changed");
+
+ *old_attr = new_attr;
+}
+
+static void parse_ec(struct ehca_shca *shca, u64 eqe)
+{
u8 ec = EHCA_BMASK_GET(NEQE_EVENT_CODE, eqe);
u8 port = EHCA_BMASK_GET(NEQE_PORT_NUMBER, eqe);
switch (ec) {
case 0x30: /* port availability change */
if (EHCA_BMASK_GET(NEQE_PORT_AVAILABILITY, eqe)) {
- ehca_info(&shca->ib_device,
- "port %x is active.", port);
- event.device = &shca->ib_device;
- event.event = IB_EVENT_PORT_ACTIVE;
- event.element.port_num = port;
shca->sport[port - 1].port_state = IB_PORT_ACTIVE;
- ib_dispatch_event(&event);
+ dispatch_port_event(shca, port, IB_EVENT_PORT_ACTIVE,
+ "is active");
+ ehca_query_sma_attr(shca, port,
+ &shca->sport[port - 1].saved_attr);
} else {
- ehca_info(&shca->ib_device,
- "port %x is inactive.", port);
- event.device = &shca->ib_device;
- event.event = IB_EVENT_PORT_ERR;
- event.element.port_num = port;
shca->sport[port - 1].port_state = IB_PORT_DOWN;
- ib_dispatch_event(&event);
+ dispatch_port_event(shca, port, IB_EVENT_PORT_ERR,
+ "is inactive");
}
break;
case 0x31:
@@ -317,24 +349,19 @@ static void parse_ec(struct ehca_shca *shca, u64 eqe)
* disruptive change is caused by
* LID, PKEY or SM change
*/
- ehca_warn(&shca->ib_device,
- "disruptive port %x configuration change", port);
-
- ehca_info(&shca->ib_device,
- "port %x is inactive.", port);
- event.device = &shca->ib_device;
- event.event = IB_EVENT_PORT_ERR;
- event.element.port_num = port;
- shca->sport[port - 1].port_state = IB_PORT_DOWN;
- ib_dispatch_event(&event);
-
- ehca_info(&shca->ib_device,
- "port %x is active.", port);
- event.device = &shca->ib_device;
- event.event = IB_EVENT_PORT_ACTIVE;
- event.element.port_num = port;
- shca->sport[port - 1].port_state = IB_PORT_ACTIVE;
- ib_dispatch_event(&event);
+ if (EHCA_BMASK_GET(NEQE_DISRUPTIVE, eqe)) {
+ ehca_warn(&shca->ib_device, "disruptive port "
+ "%d configuration change", port);
+
+ shca->sport[port - 1].port_state = IB_PORT_DOWN;
+ dispatch_port_event(shca, port, IB_EVENT_PORT_ERR,
+ "is inactive");
+
+ shca->sport[port - 1].port_state = IB_PORT_ACTIVE;
+ dispatch_port_event(shca, port, IB_EVENT_PORT_ACTIVE,
+ "is active");
+ } else
+ notify_port_conf_change(shca, port);
break;
case 0x32: /* adapter malfunction */
ehca_err(&shca->ib_device, "Adapter malfunction.");
diff --git a/drivers/infiniband/hw/ehca/ehca_iverbs.h b/drivers/infiniband/hw/ehca/ehca_iverbs.h
index fd84a80..77aeca6 100644
--- a/drivers/infiniband/hw/ehca/ehca_iverbs.h
+++ b/drivers/infiniband/hw/ehca/ehca_iverbs.h
@@ -49,6 +49,9 @@ int ehca_query_device(struct ib_device *ibdev, struct ib_device_attr *props);
int ehca_query_port(struct ib_device *ibdev, u8 port,
struct ib_port_attr *props);
+int ehca_query_sma_attr(struct ehca_shca *shca, u8 port,
+ struct ehca_sma_attr *attr);
+
int ehca_query_pkey(struct ib_device *ibdev, u8 port, u16 index, u16 * pkey);
int ehca_query_gid(struct ib_device *ibdev, u8 port, int index,
--
1.5.2
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox