* [PATCH 04/13] IB/hfi1: Increase default settings of max_cqes and max_qps
From: Dennis Dalessandro @ 2016-09-25 14:41 UTC (permalink / raw)
To: dledford-H+wXaHxf7aLQT0dZR+AlfA
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA, Mike Marciniszyn,
Jianxin Xiong
In-Reply-To: <20160925143656.10261.85231.stgit-9QXIwq+3FY+1XWohqUldA0EOCMrvLtNR@public.gmane.org>
From: Jianxin Xiong <jianxin.xiong-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
The ib_write_bw test allows using up to 16384 QPs. When a relatively
large number of QPs (within that range) is used, the test can fail
because the number of CQ entries needed exceeds the limit set by the
driver.
This patch increases the default setting of max_cqes from 0x2FFFF
(196607) to 0x2FFFFF(3145727), which is sufficient to cover the
maximum number needed by the ib_write_bw test (2097152). The default
setting of max_qps is also increased from 16384 to 32768 to allow
the test to run successfully with 16383 or 16384 QPs.
Reviewed-by: Mike Marciniszyn <mike.marciniszyn-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Jianxin Xiong <jianxin.xiong-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Dennis Dalessandro <dennis.dalessandro-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
drivers/infiniband/hw/hfi1/verbs.c | 4 ++--
1 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/infiniband/hw/hfi1/verbs.c b/drivers/infiniband/hw/hfi1/verbs.c
index 0a0e93e..1f19334 100644
--- a/drivers/infiniband/hw/hfi1/verbs.c
+++ b/drivers/infiniband/hw/hfi1/verbs.c
@@ -76,7 +76,7 @@ static unsigned int hfi1_max_ahs = 0xFFFF;
module_param_named(max_ahs, hfi1_max_ahs, uint, S_IRUGO);
MODULE_PARM_DESC(max_ahs, "Maximum number of address handles to support");
-unsigned int hfi1_max_cqes = 0x2FFFF;
+unsigned int hfi1_max_cqes = 0x2FFFFF;
module_param_named(max_cqes, hfi1_max_cqes, uint, S_IRUGO);
MODULE_PARM_DESC(max_cqes,
"Maximum number of completion queue entries to support");
@@ -89,7 +89,7 @@ unsigned int hfi1_max_qp_wrs = 0x3FFF;
module_param_named(max_qp_wrs, hfi1_max_qp_wrs, uint, S_IRUGO);
MODULE_PARM_DESC(max_qp_wrs, "Maximum number of QP WRs to support");
-unsigned int hfi1_max_qps = 16384;
+unsigned int hfi1_max_qps = 32768;
module_param_named(max_qps, hfi1_max_qps, uint, S_IRUGO);
MODULE_PARM_DESC(max_qps, "Maximum number of QPs to support");
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related
* [PATCH 03/13] IB/hfi1: Remove filtering of Set(PkeyTable) in HFI SMA
From: Dennis Dalessandro @ 2016-09-25 14:41 UTC (permalink / raw)
To: dledford-H+wXaHxf7aLQT0dZR+AlfA
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA, Ira Weiny, Sebastian Sanchez
In-Reply-To: <20160925143656.10261.85231.stgit-9QXIwq+3FY+1XWohqUldA0EOCMrvLtNR@public.gmane.org>
From: Sebastian Sanchez <sebastian.sanchez-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
The FM should have full control to set the pkeys in the
driver pkey table. Remove filtering done by the driver.
Reviewed-by: Ira Weiny <ira.weiny-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Sebastian Sanchez <sebastian.sanchez-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Dennis Dalessandro <dennis.dalessandro-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
drivers/infiniband/hw/hfi1/mad.c | 6 ------
1 files changed, 0 insertions(+), 6 deletions(-)
diff --git a/drivers/infiniband/hw/hfi1/mad.c b/drivers/infiniband/hw/hfi1/mad.c
index aecfc5b..9487c9b 100644
--- a/drivers/infiniband/hw/hfi1/mad.c
+++ b/drivers/infiniband/hw/hfi1/mad.c
@@ -1406,12 +1406,6 @@ static int set_pkeys(struct hfi1_devdata *dd, u8 port, u16 *pkeys)
if (key == okey)
continue;
/*
- * Don't update pkeys[2], if an HFI port without MgmtAllowed
- * by neighbor is a switch.
- */
- if (i == 2 && !ppd->mgmt_allowed && ppd->neighbor_type == 1)
- continue;
- /*
* The SM gives us the complete PKey table. We have
* to ensure that we put the PKeys in the matching
* slots.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related
* [PATCH 02/13] IB/qib: Remove qpt_mask global
From: Dennis Dalessandro @ 2016-09-25 14:41 UTC (permalink / raw)
To: dledford-H+wXaHxf7aLQT0dZR+AlfA
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA, Mike Marciniszyn
In-Reply-To: <20160925143656.10261.85231.stgit-9QXIwq+3FY+1XWohqUldA0EOCMrvLtNR@public.gmane.org>
There is no need to have a global qpt_mask as that does not support the
multiple chip model which qib has. Instead rely on the value which
exists already in the device data (dd).
Fixes: 898fa52b4ac3 "IB/qib: Remove qpn, qp tables and related variables from qib"
Reviewed-by: Mike Marciniszyn <mike.marciniszyn-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Dennis Dalessandro <dennis.dalessandro-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
drivers/infiniband/hw/qib/qib.h | 1 -
drivers/infiniband/hw/qib/qib_qp.c | 13 +++----------
drivers/infiniband/hw/qib/qib_verbs.c | 2 --
3 files changed, 3 insertions(+), 13 deletions(-)
diff --git a/drivers/infiniband/hw/qib/qib.h b/drivers/infiniband/hw/qib/qib.h
index bebc9a5..a3e21a2 100644
--- a/drivers/infiniband/hw/qib/qib.h
+++ b/drivers/infiniband/hw/qib/qib.h
@@ -1132,7 +1132,6 @@ extern spinlock_t qib_devs_lock;
extern struct qib_devdata *qib_lookup(int unit);
extern u32 qib_cpulist_count;
extern unsigned long *qib_cpulist;
-extern u16 qpt_mask;
extern unsigned qib_cc_table_size;
int qib_init(struct qib_devdata *, int);
diff --git a/drivers/infiniband/hw/qib/qib_qp.c b/drivers/infiniband/hw/qib/qib_qp.c
index f9b8cd2..99d31ef 100644
--- a/drivers/infiniband/hw/qib/qib_qp.c
+++ b/drivers/infiniband/hw/qib/qib_qp.c
@@ -41,14 +41,6 @@
#include "qib.h"
-/*
- * mask field which was present in now deleted qib_qpn_table
- * is not present in rvt_qpn_table. Defining the same field
- * as qpt_mask here instead of adding the mask field to
- * rvt_qpn_table.
- */
-u16 qpt_mask;
-
static inline unsigned mk_qpn(struct rvt_qpn_table *qpt,
struct rvt_qpn_map *map, unsigned off)
{
@@ -57,7 +49,7 @@ static inline unsigned mk_qpn(struct rvt_qpn_table *qpt,
static inline unsigned find_next_offset(struct rvt_qpn_table *qpt,
struct rvt_qpn_map *map, unsigned off,
- unsigned n)
+ unsigned n, u16 qpt_mask)
{
if (qpt_mask) {
off++;
@@ -179,6 +171,7 @@ int qib_alloc_qpn(struct rvt_dev_info *rdi, struct rvt_qpn_table *qpt,
struct qib_ibdev *verbs_dev = container_of(rdi, struct qib_ibdev, rdi);
struct qib_devdata *dd = container_of(verbs_dev, struct qib_devdata,
verbs_dev);
+ u16 qpt_mask = dd->qpn_mask;
if (type == IB_QPT_SMI || type == IB_QPT_GSI) {
unsigned n;
@@ -215,7 +208,7 @@ int qib_alloc_qpn(struct rvt_dev_info *rdi, struct rvt_qpn_table *qpt,
goto bail;
}
offset = find_next_offset(qpt, map, offset,
- dd->n_krcv_queues);
+ dd->n_krcv_queues, qpt_mask);
qpn = mk_qpn(qpt, map, offset);
/*
* This test differs from alloc_pidmap().
diff --git a/drivers/infiniband/hw/qib/qib_verbs.c b/drivers/infiniband/hw/qib/qib_verbs.c
index c12ec8f..876ebb4 100644
--- a/drivers/infiniband/hw/qib/qib_verbs.c
+++ b/drivers/infiniband/hw/qib/qib_verbs.c
@@ -1606,8 +1606,6 @@ int qib_register_ib_device(struct qib_devdata *dd)
/* Only need to initialize non-zero fields. */
setup_timer(&dev->mem_timer, mem_timer, (unsigned long)dev);
- qpt_mask = dd->qpn_mask;
-
INIT_LIST_HEAD(&dev->piowait);
INIT_LIST_HEAD(&dev->dmawait);
INIT_LIST_HEAD(&dev->txwait);
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related
* [PATCH 01/13] IB/hfi1: Consolidate pio control masks into single definition
From: Dennis Dalessandro @ 2016-09-25 14:40 UTC (permalink / raw)
To: dledford-H+wXaHxf7aLQT0dZR+AlfA
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA, Mike Marciniszyn
In-Reply-To: <20160925143656.10261.85231.stgit-9QXIwq+3FY+1XWohqUldA0EOCMrvLtNR@public.gmane.org>
From: Mike Marciniszyn <mike.marciniszyn-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
This allows for adding additional pages of adaptive pio
opcode control including manufacturer specific ones.
Reviewed-by: Dennis Dalessandro <dennis.dalessandro-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Mike Marciniszyn <mike.marciniszyn-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Dennis Dalessandro <dennis.dalessandro-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
drivers/infiniband/hw/hfi1/rc.c | 14 +-------------
drivers/infiniband/hw/hfi1/uc.c | 9 +--------
drivers/infiniband/hw/hfi1/verbs.c | 36 +++++++++++++++++++++++++++---------
drivers/infiniband/hw/hfi1/verbs.h | 3 +++
4 files changed, 32 insertions(+), 30 deletions(-)
diff --git a/drivers/infiniband/hw/hfi1/rc.c b/drivers/infiniband/hw/hfi1/rc.c
index db63967..d32f0c8 100644
--- a/drivers/infiniband/hw/hfi1/rc.c
+++ b/drivers/infiniband/hw/hfi1/rc.c
@@ -55,7 +55,7 @@
#include "trace.h"
/* cut down ridiculously long IB macro names */
-#define OP(x) IB_OPCODE_RC_##x
+#define OP(x) RC_OP(x)
/**
* hfi1_add_retry_timer - add/start a retry timer
@@ -184,18 +184,6 @@ void hfi1_del_timers_sync(struct rvt_qp *qp)
del_timer_sync(&priv->s_rnr_timer);
}
-/* only opcode mask for adaptive pio */
-const u32 rc_only_opcode =
- BIT(OP(SEND_ONLY) & 0x1f) |
- BIT(OP(SEND_ONLY_WITH_IMMEDIATE & 0x1f)) |
- BIT(OP(RDMA_WRITE_ONLY & 0x1f)) |
- BIT(OP(RDMA_WRITE_ONLY_WITH_IMMEDIATE & 0x1f)) |
- BIT(OP(RDMA_READ_REQUEST & 0x1f)) |
- BIT(OP(ACKNOWLEDGE & 0x1f)) |
- BIT(OP(ATOMIC_ACKNOWLEDGE & 0x1f)) |
- BIT(OP(COMPARE_SWAP & 0x1f)) |
- BIT(OP(FETCH_ADD & 0x1f));
-
static u32 restart_sge(struct rvt_sge_state *ss, struct rvt_swqe *wqe,
u32 psn, u32 pmtu)
{
diff --git a/drivers/infiniband/hw/hfi1/uc.c b/drivers/infiniband/hw/hfi1/uc.c
index e7fa176..5e6d1ba 100644
--- a/drivers/infiniband/hw/hfi1/uc.c
+++ b/drivers/infiniband/hw/hfi1/uc.c
@@ -50,14 +50,7 @@
#include "qp.h"
/* cut down ridiculously long IB macro names */
-#define OP(x) IB_OPCODE_UC_##x
-
-/* only opcode mask for adaptive pio */
-const u32 uc_only_opcode =
- BIT(OP(SEND_ONLY) & 0x1f) |
- BIT(OP(SEND_ONLY_WITH_IMMEDIATE & 0x1f)) |
- BIT(OP(RDMA_WRITE_ONLY & 0x1f)) |
- BIT(OP(RDMA_WRITE_ONLY_WITH_IMMEDIATE & 0x1f));
+#define OP(x) UC_OP(x)
/**
* hfi1_make_uc_req - construct a request packet (SEND, RDMA write)
diff --git a/drivers/infiniband/hw/hfi1/verbs.c b/drivers/infiniband/hw/hfi1/verbs.c
index d502ba0..0a0e93e 100644
--- a/drivers/infiniband/hw/hfi1/verbs.c
+++ b/drivers/infiniband/hw/hfi1/verbs.c
@@ -403,6 +403,28 @@ static const opcode_handler opcode_handler_tbl[256] = {
[IB_OPCODE_CNP] = &hfi1_cnp_rcv
};
+#define OPMASK 0x1f
+
+static const u32 pio_opmask[BIT(3)] = {
+ /* RC */
+ [IB_OPCODE_RC >> 5] =
+ BIT(RC_OP(SEND_ONLY) & OPMASK) |
+ BIT(RC_OP(SEND_ONLY_WITH_IMMEDIATE) & OPMASK) |
+ BIT(RC_OP(RDMA_WRITE_ONLY) & OPMASK) |
+ BIT(RC_OP(RDMA_WRITE_ONLY_WITH_IMMEDIATE) & OPMASK) |
+ BIT(RC_OP(RDMA_READ_REQUEST) & OPMASK) |
+ BIT(RC_OP(ACKNOWLEDGE) & OPMASK) |
+ BIT(RC_OP(ATOMIC_ACKNOWLEDGE) & OPMASK) |
+ BIT(RC_OP(COMPARE_SWAP) & OPMASK) |
+ BIT(RC_OP(FETCH_ADD) & OPMASK),
+ /* UC */
+ [IB_OPCODE_UC >> 5] =
+ BIT(UC_OP(SEND_ONLY) & OPMASK) |
+ BIT(UC_OP(SEND_ONLY_WITH_IMMEDIATE) & OPMASK) |
+ BIT(UC_OP(RDMA_WRITE_ONLY) & OPMASK) |
+ BIT(UC_OP(RDMA_WRITE_ONLY_WITH_IMMEDIATE) & OPMASK),
+};
+
/*
* System image GUID.
*/
@@ -1210,22 +1232,18 @@ static inline send_routine get_send_routine(struct rvt_qp *qp,
case IB_QPT_GSI:
case IB_QPT_UD:
break;
- case IB_QPT_RC:
- if (piothreshold &&
- qp->s_cur_size <= min(piothreshold, qp->pmtu) &&
- (BIT(get_opcode(h) & 0x1f) & rc_only_opcode) &&
- iowait_sdma_pending(&priv->s_iowait) == 0 &&
- !sdma_txreq_built(&tx->txreq))
- return dd->process_pio_send;
- break;
case IB_QPT_UC:
+ case IB_QPT_RC: {
+ u8 op = get_opcode(h);
+
if (piothreshold &&
qp->s_cur_size <= min(piothreshold, qp->pmtu) &&
- (BIT(get_opcode(h) & 0x1f) & uc_only_opcode) &&
+ (BIT(op & OPMASK) & pio_opmask[op >> 5]) &&
iowait_sdma_pending(&priv->s_iowait) == 0 &&
!sdma_txreq_built(&tx->txreq))
return dd->process_pio_send;
break;
+ }
default:
break;
}
diff --git a/drivers/infiniband/hw/hfi1/verbs.h b/drivers/infiniband/hw/hfi1/verbs.h
index 0dd65eb..1c3815d 100644
--- a/drivers/infiniband/hw/hfi1/verbs.h
+++ b/drivers/infiniband/hw/hfi1/verbs.h
@@ -97,6 +97,9 @@ struct hfi1_packet;
#define IB_DEFAULT_GID_PREFIX cpu_to_be64(0xfe80000000000000ULL)
+#define RC_OP(x) IB_OPCODE_RC_##x
+#define UC_OP(x) IB_OPCODE_UC_##x
+
/* flags passed by hfi1_ib_rcv() */
enum {
HFI1_HAS_GRH = (1 << 0),
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related
* [PATCH 00/13] IB/hfi1,qib: Round 2 of hfi1/qib fixes for 4.9
From: Dennis Dalessandro @ 2016-09-25 14:40 UTC (permalink / raw)
To: dledford-H+wXaHxf7aLQT0dZR+AlfA
Cc: Mike Marciniszyn, Dean Luick, Jakub Pawlak,
linux-rdma-u79uwXL29TY76Z2rM5mHXA, Ira Weiny,
stable-u79uwXL29TY76Z2rM5mHXA, Harish Chegondi, Sebastian Sanchez,
Jianxin Xiong
Here is the next round of fixes and clean ups for hfi1/qib for 4.9. Mostly minor
fixes but there is a fix for stable in here. It is marked to Cc the stable list
in the commit message so no special handling should be needed on your part.
Patches should apply on your GitHub hfi1 branch and are availabe in my GitRepo as
well:
https://github.com/ddalessa/kernel/tree/for-4.9
---
Dean Luick (2):
IB/hfi1: Extend i2c timeout
IB/hfi1: Act on external device timeout
Dennis Dalessandro (3):
IB/qib: Remove qpt_mask global
IB/hfi1: Cleanup tasklet refs in comments
IB/hfi1: Remove unused variable from devdata
Harish Chegondi (1):
IB/hfi1: Adjust hardware buffering parameter
Jakub Pawlak (1):
IB/hfi1: Fix resource release in context allocation
Jianxin Xiong (1):
IB/hfi1: Increase default settings of max_cqes and max_qps
Mike Marciniszyn (2):
IB/hfi1: Consolidate pio control masks into single definition
IB/hfi1: Fix defered ack race with qp destroy
Sebastian Sanchez (3):
IB/hfi1: Remove filtering of Set(PkeyTable) in HFI SMA
IB/hfi1: Do not read more than a SGE length
IB/hfi1: Combine shift copy and byte copy for SGE reads
drivers/infiniband/hw/hfi1/chip.c | 8 +
drivers/infiniband/hw/hfi1/chip.h | 6 +
drivers/infiniband/hw/hfi1/file_ops.c | 17 ++
drivers/infiniband/hw/hfi1/hfi.h | 2
drivers/infiniband/hw/hfi1/init.c | 1
drivers/infiniband/hw/hfi1/mad.c | 6 -
drivers/infiniband/hw/hfi1/pio_copy.c | 246 ++++++++-------------------------
drivers/infiniband/hw/hfi1/qsfp.c | 2
drivers/infiniband/hw/hfi1/rc.c | 33 ++--
drivers/infiniband/hw/hfi1/ruc.c | 6 -
drivers/infiniband/hw/hfi1/uc.c | 9 -
drivers/infiniband/hw/hfi1/verbs.c | 40 ++++-
drivers/infiniband/hw/hfi1/verbs.h | 3
drivers/infiniband/hw/qib/qib.h | 1
drivers/infiniband/hw/qib/qib_qp.c | 13 --
drivers/infiniband/hw/qib/qib_verbs.c | 2
16 files changed, 136 insertions(+), 259 deletions(-)
--
-Denny
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH v5 02/16] IB/pvrdma: Add user-level shared functions
From: Leon Romanovsky @ 2016-09-25 12:29 UTC (permalink / raw)
To: Adit Ranadive
Cc: dledford, linux-rdma, pv-drivers, netdev, linux-pci, jhansen,
asarwade, georgezhang, bryantan
In-Reply-To: <20160925072624.GV4088@leon.nu>
[-- Attachment #1: Type: text/plain, Size: 1524 bytes --]
On Sun, Sep 25, 2016 at 10:26:24AM +0300, Leon Romanovsky wrote:
<...>
> > +
> > +/* PVRDMA send queue work request */
> > +struct pvrdma_sq_wqe_hdr {
> > + __u64 wr_id; /* wr id */
> > + __u32 num_sge; /* size of s/g array */
> > + __u32 total_len; /* reserved */
> > + __u32 opcode; /* operation type */
> > + __u32 send_flags; /* wr flags */
> > + union {
> > + __u32 imm_data;
> > + __u32 invalidate_rkey;
> > + } ex;
> > + __u32 reserved;
> > + union {
> > + struct {
> > + __u64 remote_addr;
> > + __u32 rkey;
> > + __u8 reserved[4];
> > + } rdma;
> > + struct {
> > + __u64 remote_addr;
> > + __u64 compare_add;
> > + __u64 swap;
> > + __u32 rkey;
> > + __u32 reserved;
> > + } atomic;
> > + struct {
> > + __u64 remote_addr;
> > + __u32 log_arg_sz;
> > + __u32 rkey;
> > + union {
> > + struct pvrdma_exp_cmp_swap cmp_swap;
> > + struct pvrdma_exp_fetch_add fetch_add;
> > + } wr_data;
> > + } masked_atomics;
> > + struct {
> > + __u64 iova_start;
> > + __u64 pl_pdir_dma;
> > + __u32 page_shift;
> > + __u32 page_list_len;
> > + __u32 length;
> > + __u32 access_flags;
> > + __u32 rkey;
> > + } fast_reg;
> > + struct {
> > + __u32 remote_qpn;
> > + __u32 remote_qkey;
> > + struct pvrdma_av av;
> > + } ud;
> > + } wr;
> > +};
>
> No, I have half-baked patch series which refactors this structure in kernel.
Sorry, this patch series is not needed in kernel.
> There is no need to put this structure in UAPI.
This is still relevant.
Thanks
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply
* Re: [PATCH TRIVIAL rdma-core] README.md: Fix umad library name
From: Leon Romanovsky @ 2016-09-25 9:20 UTC (permalink / raw)
To: Hal Rosenstock
Cc: Jason Gunthorpe, Doug Ledford,
linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
In-Reply-To: <544ef813-8900-35e1-f7c9-c7135cf92dc5-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
[-- Attachment #1: Type: text/plain, Size: 162 bytes --]
On Fri, Sep 23, 2016 at 12:56:15PM -0400, Hal Rosenstock wrote:
>
> Signed-off-by: Hal Rosenstock <hal-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Thanks, applied.
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply
* Re: [PATCH] libcxgb4: fix firmware version string
From: Leon Romanovsky @ 2016-09-25 9:19 UTC (permalink / raw)
To: Steve Wise
Cc: dledford-H+wXaHxf7aLQT0dZR+AlfA,
linux-rdma-u79uwXL29TY76Z2rM5mHXA
In-Reply-To: <20160922201503.CFB88E08CA-/5N3P9jjx0xzbRFIqnYvSA@public.gmane.org>
[-- Attachment #1: Type: text/plain, Size: 367 bytes --]
On Thu, Sep 22, 2016 at 12:29:55PM -0700, Steve Wise wrote:
> c4iw_query_device() was incorrectly unpacking
> the 64b version number from the driver. This resulted
> in ibv_devinfo showing garbage fw verion strings.
>
> Signed-off-by: Steve Wise <swise-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW@public.gmane.org>
>
> ---
>
> Testing out the new process. :)
Thanks, applied.
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply
* Re: [PATCH v5 13/16] IB/pvrdma: Add the main driver module for PVRDMA
From: Leon Romanovsky @ 2016-09-25 7:57 UTC (permalink / raw)
To: Adit Ranadive
Cc: dledford-H+wXaHxf7aLQT0dZR+AlfA,
linux-rdma-u79uwXL29TY76Z2rM5mHXA,
pv-drivers-pghWNbHTmq7QT0dZR+AlfA, netdev-u79uwXL29TY76Z2rM5mHXA,
linux-pci-u79uwXL29TY76Z2rM5mHXA, jhansen-pghWNbHTmq7QT0dZR+AlfA,
asarwade-pghWNbHTmq7QT0dZR+AlfA,
georgezhang-pghWNbHTmq7QT0dZR+AlfA,
bryantan-pghWNbHTmq7QT0dZR+AlfA
In-Reply-To: <8db982c37a0a96b37df02826564474a216922f25.1474759181.git.aditr-pghWNbHTmq7QT0dZR+AlfA@public.gmane.org>
[-- Attachment #1: Type: text/plain, Size: 6775 bytes --]
On Sat, Sep 24, 2016 at 04:21:37PM -0700, Adit Ranadive wrote:
> This patch adds the support to register a RDMA device with the kernel RDMA
> stack as well as a kernel module. This also initializes the underlying
> virtual PCI device.
>
> Reviewed-by: Yuval Shaia <yuval.shaia-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
> Reviewed-by: Jorgen Hansen <jhansen-pghWNbHTmq7QT0dZR+AlfA@public.gmane.org>
> Reviewed-by: George Zhang <georgezhang-pghWNbHTmq7QT0dZR+AlfA@public.gmane.org>
> Reviewed-by: Aditya Sarwade <asarwade-pghWNbHTmq7QT0dZR+AlfA@public.gmane.org>
> Reviewed-by: Bryan Tan <bryantan-pghWNbHTmq7QT0dZR+AlfA@public.gmane.org>
> Signed-off-by: Adit Ranadive <aditr-pghWNbHTmq7QT0dZR+AlfA@public.gmane.org>
> ---
> Changes v4->v5:
> - Removed two unnecessary lines.
> - Updated include for headers in UAPI folder.
> - Update to pvrdma_cmd_post for add/delete GIDs.
> - Add error code in dev_warn if pvrdma_cmd_post failed.
>
> Changes v3->v4:
> - Fixed some checkpatch warnings.
> - Added support for new get_dev_fw_str API.
> - Added event workqueue for netdevice events.
> - Restructured the pvrdma_pci_remove function a little bit.
>
> Changes v2->v3:
> - Removed boolean in pvrdma_cmd_post.
>
> Changes v1->v2:
> - Addressed 32-bit build errors
> - Cosmetic change to avoid if else in intr0_handler
> - Removed unnecessary return assignment.
> ---
> drivers/infiniband/hw/pvrdma/pvrdma_main.c | 1220 ++++++++++++++++++++++++++++
> 1 file changed, 1220 insertions(+)
> create mode 100644 drivers/infiniband/hw/pvrdma/pvrdma_main.c
>
> diff --git a/drivers/infiniband/hw/pvrdma/pvrdma_main.c b/drivers/infiniband/hw/pvrdma/pvrdma_main.c
> new file mode 100644
> index 0000000..94cbbb9
> --- /dev/null
> +++ b/drivers/infiniband/hw/pvrdma/pvrdma_main.c
> @@ -0,0 +1,1220 @@
> +/*
> + * Copyright (c) 2012-2016 VMware, Inc. All rights reserved.
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify it under the terms of EITHER the GNU General Public License
> + * version 2 as published by the Free Software Foundation or the BSD
> + * 2-Clause License. This program is distributed in the hope that it
> + * will be useful, but WITHOUT ANY WARRANTY; WITHOUT EVEN THE IMPLIED
> + * WARRANTY OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
> + * See the GNU General Public License version 2 for more details at
> + * http://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program available in the file COPYING in the main
> + * directory of this source tree.
> + *
> + * The BSD 2-Clause License
> + *
> + * Redistribution and use in source and binary forms, with or
> + * without modification, are permitted provided that the following
> + * conditions are met:
> + *
> + * - Redistributions of source code must retain the above
> + * copyright notice, this list of conditions and the following
> + * disclaimer.
> + *
> + * - Redistributions in binary form must reproduce the above
> + * copyright notice, this list of conditions and the following
> + * disclaimer in the documentation and/or other materials
> + * provided with the distribution.
> + *
> + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
> + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
> + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
> + * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
> + * COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
> + * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
> + * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
> + * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
> + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
> + * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
> + * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED
> + * OF THE POSSIBILITY OF SUCH DAMAGE.
> + */
> +
> +#include <linux/errno.h>
> +#include <linux/inetdevice.h>
> +#include <linux/init.h>
> +#include <linux/module.h>
> +#include <linux/slab.h>
> +#include <rdma/ib_addr.h>
> +#include <rdma/ib_smi.h>
> +#include <rdma/ib_user_verbs.h>
> +#include <net/addrconf.h>
> +#include <rdma/pvrdma-abi.h>
> +
> +#include "pvrdma.h"
> +
> +#define DRV_NAME "pvrdma"
> +#define DRV_VERSION "1.0"
> +#define DRV_RELDATE "January 1, 2013"
> +
> +static const char pvrdma_version[] =
> + DRV_NAME ": PVRDMA InfiniBand driver v"
> + DRV_VERSION " (" DRV_RELDATE ")\n";
This is a good example why driver version and reldate are useless in
kernel. We are in 2016 and not in 2013. All these data are forgotten to
update right after the developer copied it from other pre-historic driver
(very common practice in netdev). It is worth to stop to use this totally
useless defines.
> +
> +static DEFINE_MUTEX(pvrdma_device_list_lock);
> +static LIST_HEAD(pvrdma_device_list);
> +static struct workqueue_struct *event_wq;
> +
> +static int pvrdma_add_gid(struct ib_device *ibdev,
> + u8 port_num,
> + unsigned int index,
> + const union ib_gid *gid,
> + const struct ib_gid_attr *attr,
> + void **context);
> +static int pvrdma_del_gid(struct ib_device *ibdev,
> + u8 port_num,
> + unsigned int index,
> + void **context);
> +
> +
> +static ssize_t show_hca(struct device *device, struct device_attribute *attr,
> + char *buf)
> +{
> + return sprintf(buf, "PVRDMA%s\n", DRV_VERSION);
> +}
> +
> +static ssize_t show_rev(struct device *device, struct device_attribute *attr,
> + char *buf)
> +{
> + return sprintf(buf, "%d\n", PVRDMA_REV_ID);
> +}
> +
> +static ssize_t show_board(struct device *device, struct device_attribute *attr,
> + char *buf)
> +{
> + return sprintf(buf, "%d\n", PVRDMA_BOARD_ID);
> +}
> +
> +static DEVICE_ATTR(hw_rev, S_IRUGO, show_rev, NULL);
> +static DEVICE_ATTR(hca_type, S_IRUGO, show_hca, NULL);
> +static DEVICE_ATTR(board_id, S_IRUGO, show_board, NULL);
> +
> +static struct device_attribute *pvrdma_class_attributes[] = {
> + &dev_attr_hw_rev,
> + &dev_attr_hca_type,
> + &dev_attr_board_id
> +};
> +
> +static void pvrdma_get_fw_ver_str(struct ib_device *device, char *str,
> + size_t str_len)
> +{
> + struct pvrdma_dev *dev =
> + container_of(device, struct pvrdma_dev, ib_dev);
> + snprintf(str, str_len, "%d.%d.%d\n",
> + (int) (dev->dsr->caps.fw_ver >> 32),
> + (int) (dev->dsr->caps.fw_ver >> 16) & 0xffff,
> + (int) dev->dsr->caps.fw_ver & 0xffff);
> +}
Yuval already pointed it to you.
Thanks to Ira, we have general function in core to print FW version.
Please use them.
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply
* Re: [PATCH v5 16/16] MAINTAINERS: Update for PVRDMA driver
From: Leon Romanovsky @ 2016-09-25 7:30 UTC (permalink / raw)
To: Adit Ranadive
Cc: dledford, linux-rdma, pv-drivers, netdev, linux-pci, jhansen,
asarwade, georgezhang, bryantan
In-Reply-To: <fe5e7704757254073963cd40bff5240421b1fce0.1474759181.git.aditr@vmware.com>
[-- Attachment #1: Type: text/plain, Size: 1223 bytes --]
On Sat, Sep 24, 2016 at 04:21:40PM -0700, Adit Ranadive wrote:
> Add maintainer info for the PVRDMA driver.
>
> Reviewed-by: Jorgen Hansen <jhansen@vmware.com>
> Reviewed-by: George Zhang <georgezhang@vmware.com>
> Reviewed-by: Aditya Sarwade <asarwade@vmware.com>
> Reviewed-by: Bryan Tan <bryantan@vmware.com>
> Signed-off-by: Adit Ranadive <aditr@vmware.com>
> ---
> Changes v4->v5:
> - Added pvrdma files to common UAPI folder.
> ---
> MAINTAINERS | 9 +++++++++
> 1 file changed, 9 insertions(+)
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 87e23cd..5023dc0 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -12615,6 +12615,15 @@ S: Maintained
> F: drivers/scsi/vmw_pvscsi.c
> F: drivers/scsi/vmw_pvscsi.h
>
> +VMWARE PVRDMA DRIVER
> +M: Adit Ranadive <aditr@vmware.com>
> +M: VMware PV-Drivers <pv-drivers@vmware.com>
> +L: linux-rdma@vger.kernel.org
> +S: Maintained
> +F: drivers/infiniband/hw/pvrdma/
> +F: include/uapi/rdma/pvrdma-abi.h
> +F: include/uapi/rdma/pvrdma-uapi.h
Please remove the last two lines, these files will be maintained by
Doug.
Thanks
> +
> VOLTAGE AND CURRENT REGULATOR FRAMEWORK
> M: Liam Girdwood <lgirdwood@gmail.com>
> M: Mark Brown <broonie@kernel.org>
> --
> 2.7.4
>
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply
* Re: [PATCH v5 02/16] IB/pvrdma: Add user-level shared functions
From: Leon Romanovsky @ 2016-09-25 7:26 UTC (permalink / raw)
To: Adit Ranadive
Cc: dledford-H+wXaHxf7aLQT0dZR+AlfA,
linux-rdma-u79uwXL29TY76Z2rM5mHXA,
pv-drivers-pghWNbHTmq7QT0dZR+AlfA, netdev-u79uwXL29TY76Z2rM5mHXA,
linux-pci-u79uwXL29TY76Z2rM5mHXA, jhansen-pghWNbHTmq7QT0dZR+AlfA,
asarwade-pghWNbHTmq7QT0dZR+AlfA,
georgezhang-pghWNbHTmq7QT0dZR+AlfA,
bryantan-pghWNbHTmq7QT0dZR+AlfA
In-Reply-To: <b52db7f59d69089f7b1d53311574143c4da8252a.1474759181.git.aditr-pghWNbHTmq7QT0dZR+AlfA@public.gmane.org>
[-- Attachment #1: Type: text/plain, Size: 14143 bytes --]
On Sat, Sep 24, 2016 at 04:21:26PM -0700, Adit Ranadive wrote:
> We share some common structures with the user-level driver. This patch adds
> those structures and shared functions to traverse the QP/CQ rings.
>
> Reviewed-by: Yuval Shaia <yuval.shaia-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
> Reviewed-by: Jorgen Hansen <jhansen-pghWNbHTmq7QT0dZR+AlfA@public.gmane.org>
> Reviewed-by: George Zhang <georgezhang-pghWNbHTmq7QT0dZR+AlfA@public.gmane.org>
> Reviewed-by: Aditya Sarwade <asarwade-pghWNbHTmq7QT0dZR+AlfA@public.gmane.org>
> Reviewed-by: Bryan Tan <bryantan-pghWNbHTmq7QT0dZR+AlfA@public.gmane.org>
> Signed-off-by: Adit Ranadive <aditr-pghWNbHTmq7QT0dZR+AlfA@public.gmane.org>
> ---
> Changes v4->v5:
> - Moved pvrdma_uapi.h and pvrdma_user.h into common UAPI folder.
> - Renamed to pvrdma-uapi.h and pvrdma-abi.h respectively.
> - Prefixed unsigned vars with __.
>
> Changes v3->v4:
> - Moved pvrdma_sge into pvrdma_uapi.h
> ---
> include/uapi/rdma/Kbuild | 2 +
> include/uapi/rdma/pvrdma-abi.h | 99 ++++++++++++++++
> include/uapi/rdma/pvrdma-uapi.h | 255 ++++++++++++++++++++++++++++++++++++++++
> 3 files changed, 356 insertions(+)
> create mode 100644 include/uapi/rdma/pvrdma-abi.h
> create mode 100644 include/uapi/rdma/pvrdma-uapi.h
>
> diff --git a/include/uapi/rdma/Kbuild b/include/uapi/rdma/Kbuild
> index 4edb0f2..fc2b285 100644
> --- a/include/uapi/rdma/Kbuild
> +++ b/include/uapi/rdma/Kbuild
> @@ -7,3 +7,5 @@ header-y += rdma_netlink.h
> header-y += rdma_user_cm.h
> header-y += hfi/
> header-y += rdma_user_rxe.h
> +header-y += pvrdma-abi.h
> +header-y += pvrdma-uapi.h
> diff --git a/include/uapi/rdma/pvrdma-abi.h b/include/uapi/rdma/pvrdma-abi.h
> new file mode 100644
> index 0000000..6fa0ab6
> --- /dev/null
> +++ b/include/uapi/rdma/pvrdma-abi.h
> @@ -0,0 +1,99 @@
> +/*
> + * Copyright (c) 2012-2016 VMware, Inc. All rights reserved.
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify it under the terms of EITHER the GNU General Public License
> + * version 2 as published by the Free Software Foundation or the BSD
> + * 2-Clause License. This program is distributed in the hope that it
> + * will be useful, but WITHOUT ANY WARRANTY; WITHOUT EVEN THE IMPLIED
> + * WARRANTY OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
> + * See the GNU General Public License version 2 for more details at
> + * http://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program available in the file COPYING in the main
> + * directory of this source tree.
> + *
> + * The BSD 2-Clause License
> + *
> + * Redistribution and use in source and binary forms, with or
> + * without modification, are permitted provided that the following
> + * conditions are met:
> + *
> + * - Redistributions of source code must retain the above
> + * copyright notice, this list of conditions and the following
> + * disclaimer.
> + *
> + * - Redistributions in binary form must reproduce the above
> + * copyright notice, this list of conditions and the following
> + * disclaimer in the documentation and/or other materials
> + * provided with the distribution.
> + *
> + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
> + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
> + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
> + * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
> + * COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
> + * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
> + * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
> + * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
> + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
> + * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
> + * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED
> + * OF THE POSSIBILITY OF SUCH DAMAGE.
> + */
> +
> +#ifndef __PVRDMA_USER_H__
> +#define __PVRDMA_USER_H__
> +
> +#include <linux/types.h>
> +
> +#define PVRDMA_UVERBS_ABI_VERSION 3
> +#define PVRDMA_BOARD_ID 1
> +#define PVRDMA_REV_ID 1
Please don't add defines which you are not using in the library and the
two above are not in use.
> +
> +struct pvrdma_alloc_ucontext_resp {
> + __u32 qp_tab_size;
> + __u32 reserved;
> +};
> +
> +struct pvrdma_alloc_pd_resp {
> + __u32 pdn;
> + __u32 reserved;
> +};
> +
> +struct pvrdma_create_cq {
> + __u64 buf_addr;
> + __u32 buf_size;
> + __u32 reserved;
> +};
> +
> +struct pvrdma_create_cq_resp {
> + __u32 cqn;
> + __u32 reserved;
> +};
> +
> +struct pvrdma_resize_cq {
> + __u64 buf_addr;
> + __u32 buf_size;
> + __u32 reserved;
> +};
> +
> +struct pvrdma_create_srq {
> + __u64 buf_addr;
> +};
> +
> +struct pvrdma_create_srq_resp {
> + __u32 srqn;
> + __u32 reserved;
> +};
> +
> +struct pvrdma_create_qp {
> + __u64 rbuf_addr;
> + __u64 sbuf_addr;
> + __u32 rbuf_size;
> + __u32 sbuf_size;
> + __u64 qp_addr;
> +};
> +
> +#endif /* __PVRDMA_USER_H__ */
> diff --git a/include/uapi/rdma/pvrdma-uapi.h b/include/uapi/rdma/pvrdma-uapi.h
> new file mode 100644
> index 0000000..430d8a5
> --- /dev/null
> +++ b/include/uapi/rdma/pvrdma-uapi.h
> @@ -0,0 +1,255 @@
> +/*
> + * Copyright (c) 2012-2016 VMware, Inc. All rights reserved.
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify it under the terms of EITHER the GNU General Public License
> + * version 2 as published by the Free Software Foundation or the BSD
> + * 2-Clause License. This program is distributed in the hope that it
> + * will be useful, but WITHOUT ANY WARRANTY; WITHOUT EVEN THE IMPLIED
> + * WARRANTY OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
> + * See the GNU General Public License version 2 for more details at
> + * http://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program available in the file COPYING in the main
> + * directory of this source tree.
> + *
> + * The BSD 2-Clause License
> + *
> + * Redistribution and use in source and binary forms, with or
> + * without modification, are permitted provided that the following
> + * conditions are met:
> + *
> + * - Redistributions of source code must retain the above
> + * copyright notice, this list of conditions and the following
> + * disclaimer.
> + *
> + * - Redistributions in binary form must reproduce the above
> + * copyright notice, this list of conditions and the following
> + * disclaimer in the documentation and/or other materials
> + * provided with the distribution.
> + *
> + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
> + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
> + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
> + * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
> + * COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
> + * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
> + * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
> + * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
> + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
> + * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
> + * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED
> + * OF THE POSSIBILITY OF SUCH DAMAGE.
> + */
> +
> +#ifndef __PVRDMA_UAPI_H__
> +#define __PVRDMA_UAPI_H__
> +
> +#include <linux/types.h>
> +
> +#define PVRDMA_VERSION 17
What do you plan to do with this VERSION?
How is it related to ABI?
> +
> +#define PVRDMA_UAR_HANDLE_MASK 0x00FFFFFF /* Bottom 24 bits. */
> +#define PVRDMA_UAR_QP_OFFSET 0 /* Offset of QP doorbell. */
> +#define PVRDMA_UAR_QP_SEND BIT(30) /* Send bit. */
> +#define PVRDMA_UAR_QP_RECV BIT(31) /* Recv bit. */
> +#define PVRDMA_UAR_CQ_OFFSET 4 /* Offset of CQ doorbell. */
> +#define PVRDMA_UAR_CQ_ARM_SOL BIT(29) /* Arm solicited bit. */
> +#define PVRDMA_UAR_CQ_ARM BIT(30) /* Arm bit. */
> +#define PVRDMA_UAR_CQ_POLL BIT(31) /* Poll bit. */
> +#define PVRDMA_INVALID_IDX -1 /* Invalid index. */
> +
> +/* PVRDMA atomic compare and swap */
> +struct pvrdma_exp_cmp_swap {
_EXP_ looks very similar to MLNX_OFED naming convention.
> + __u64 swap_val;
> + __u64 compare_val;
> + __u64 swap_mask;
> + __u64 compare_mask;
> +};
> +
> +/* PVRDMA atomic fetch and add */
> +struct pvrdma_exp_fetch_add {
The same as above.
> + __u64 add_val;
> + __u64 field_boundary;
> +};
> +
> +/* PVRDMA address vector. */
> +struct pvrdma_av {
> + __u32 port_pd;
> + __u32 sl_tclass_flowlabel;
> + __u8 dgid[16];
> + __u8 src_path_bits;
> + __u8 gid_index;
> + __u8 stat_rate;
> + __u8 hop_limit;
> + __u8 dmac[6];
> + __u8 reserved[6];
> +};
> +
> +/* PVRDMA scatter/gather entry */
> +struct pvrdma_sge {
> + __u64 addr;
> + __u32 length;
> + __u32 lkey;
> +};
> +
> +/* PVRDMA receive queue work request */
> +struct pvrdma_rq_wqe_hdr {
> + __u64 wr_id; /* wr id */
> + __u32 num_sge; /* size of s/g array */
> + __u32 total_len; /* reserved */
> +};
> +/* Use pvrdma_sge (ib_sge) for receive queue s/g array elements. */
> +
> +/* PVRDMA send queue work request */
> +struct pvrdma_sq_wqe_hdr {
> + __u64 wr_id; /* wr id */
> + __u32 num_sge; /* size of s/g array */
> + __u32 total_len; /* reserved */
> + __u32 opcode; /* operation type */
> + __u32 send_flags; /* wr flags */
> + union {
> + __u32 imm_data;
> + __u32 invalidate_rkey;
> + } ex;
> + __u32 reserved;
> + union {
> + struct {
> + __u64 remote_addr;
> + __u32 rkey;
> + __u8 reserved[4];
> + } rdma;
> + struct {
> + __u64 remote_addr;
> + __u64 compare_add;
> + __u64 swap;
> + __u32 rkey;
> + __u32 reserved;
> + } atomic;
> + struct {
> + __u64 remote_addr;
> + __u32 log_arg_sz;
> + __u32 rkey;
> + union {
> + struct pvrdma_exp_cmp_swap cmp_swap;
> + struct pvrdma_exp_fetch_add fetch_add;
> + } wr_data;
> + } masked_atomics;
> + struct {
> + __u64 iova_start;
> + __u64 pl_pdir_dma;
> + __u32 page_shift;
> + __u32 page_list_len;
> + __u32 length;
> + __u32 access_flags;
> + __u32 rkey;
> + } fast_reg;
> + struct {
> + __u32 remote_qpn;
> + __u32 remote_qkey;
> + struct pvrdma_av av;
> + } ud;
> + } wr;
> +};
No, I have half-baked patch series which refactors this structure in kernel.
There is no need to put this structure in UAPI.
> +/* Use pvrdma_sge (ib_sge) for send queue s/g array elements. */
> +
> +/* Completion queue element. */
> +struct pvrdma_cqe {
> + __u64 wr_id;
> + __u64 qp;
> + __u32 opcode;
> + __u32 status;
> + __u32 byte_len;
> + __u32 imm_data;
> + __u32 src_qp;
> + __u32 wc_flags;
> + __u32 vendor_err;
> + __u16 pkey_index;
> + __u16 slid;
> + __u8 sl;
> + __u8 dlid_path_bits;
> + __u8 port_num;
> + __u8 smac[6];
> + __u8 reserved2[7]; /* Pad to next power of 2 (64). */
> +};
> +
> +struct pvrdma_ring {
> + atomic_t prod_tail; /* Producer tail. */
> + atomic_t cons_head; /* Consumer head. */
> +};
> +
> +struct pvrdma_ring_state {
> + struct pvrdma_ring tx; /* Tx ring. */
> + struct pvrdma_ring rx; /* Rx ring. */
> +};
> +
> +static inline int pvrdma_idx_valid(__u32 idx, __u32 max_elems)
> +{
> + /* Generates fewer instructions than a less-than. */
> + return (idx & ~((max_elems << 1) - 1)) == 0;
> +}
> +
> +static inline __s32 pvrdma_idx(atomic_t *var, __u32 max_elems)
> +{
> + const unsigned int idx = atomic_read(var);
> +
> + if (pvrdma_idx_valid(idx, max_elems))
> + return idx & (max_elems - 1);
> + return PVRDMA_INVALID_IDX;
> +}
> +
> +static inline void pvrdma_idx_ring_inc(atomic_t *var, __u32 max_elems)
> +{
> + __u32 idx = atomic_read(var) + 1; /* Increment. */
It is definitely different atomic_read than you expect. From my grep
searches on my machine, linux kernel doesn't export in standard headers
the atomic_* functions and C has their implementation of that functions.
> +
> + idx &= (max_elems << 1) - 1; /* Modulo size, flip gen. */
> + atomic_set(var, idx);
> +}
> +
> +static inline __s32 pvrdma_idx_ring_has_space(const struct pvrdma_ring *r,
> + __u32 max_elems, __u32 *out_tail)
> +{
> + const __u32 tail = atomic_read(&r->prod_tail);
> + const __u32 head = atomic_read(&r->cons_head);
> +
> + if (pvrdma_idx_valid(tail, max_elems) &&
> + pvrdma_idx_valid(head, max_elems)) {
> + *out_tail = tail & (max_elems - 1);
> + return tail != (head ^ max_elems);
> + }
> + return PVRDMA_INVALID_IDX;
> +}
> +
> +static inline __s32 pvrdma_idx_ring_has_data(const struct pvrdma_ring *r,
> + __u32 max_elems, __u32 *out_head)
> +{
> + const __u32 tail = atomic_read(&r->prod_tail);
> + const __u32 head = atomic_read(&r->cons_head);
> +
> + if (pvrdma_idx_valid(tail, max_elems) &&
> + pvrdma_idx_valid(head, max_elems)) {
> + *out_head = head & (max_elems - 1);
> + return tail != head;
> + }
> + return PVRDMA_INVALID_IDX;
> +}
> +
> +static inline bool pvrdma_idx_ring_is_valid_idx(const struct pvrdma_ring *r,
> + __u32 max_elems, __u32 *idx)
> +{
> + const __u32 tail = atomic_read(&r->prod_tail);
> + const __u32 head = atomic_read(&r->cons_head);
> +
> + if (pvrdma_idx_valid(tail, max_elems) &&
> + pvrdma_idx_valid(head, max_elems) &&
> + pvrdma_idx_valid(*idx, max_elems)) {
> + if (tail > head && (*idx < tail && *idx >= head))
> + return true;
> + else if (head > tail && (*idx >= head || *idx < tail))
> + return true;
> + }
> + return false;
> +}
> +
> +#endif /* __PVRDMA_UAPI_H__ */
I suggest completely remove this file from UAPI headers folder.
Thanks.
> --
> 2.7.4
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply
* Re: [PATCH v5 00/16] Add Paravirtual RDMA Driver
From: Leon Romanovsky @ 2016-09-25 7:03 UTC (permalink / raw)
To: Adit Ranadive
Cc: dledford, linux-rdma, pv-drivers, netdev, linux-pci, jhansen,
asarwade, georgezhang, bryantan
In-Reply-To: <cover.1474759181.git.aditr@vmware.com>
[-- Attachment #1: Type: text/plain, Size: 354 bytes --]
On Sat, Sep 24, 2016 at 04:21:24PM -0700, Adit Ranadive wrote:
<...>
> include/uapi/rdma/pvrdma-abi.h | 99 ++
> include/uapi/rdma/pvrdma-uapi.h | 255 +++++
As Jason said, you need a very good reason to split and create number of
files per-driver in UAPI folder.
Please see my comments in respective patch.
Thanks
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply
* [PATCH rdma-core 5/5] libmlx5: Convert libmlx5 to use common list implementation
From: Leon Romanovsky @ 2016-09-25 6:50 UTC (permalink / raw)
To: dledford-H+wXaHxf7aLQT0dZR+AlfA,
linux-rdma-u79uwXL29TY76Z2rM5mHXA
Cc: jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/,
yishaih-VPRAkNaXOzVWk0Htik3J/w
In-Reply-To: <1474786207-2149-1-git-send-email-leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
Signed-off-by: Leon Romanovsky <leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
---
libmlx5/src/buf.c | 13 +--
libmlx5/src/list.h | 312 -----------------------------------------------------
libmlx5/src/mlx5.h | 4 +-
utils/list.h | 9 +-
4 files changed, 17 insertions(+), 321 deletions(-)
delete mode 100644 libmlx5/src/list.h
diff --git a/libmlx5/src/buf.c b/libmlx5/src/buf.c
index c827930..ec8611e 100644
--- a/libmlx5/src/buf.c
+++ b/libmlx5/src/buf.c
@@ -259,16 +259,17 @@ static int alloc_huge_buf(struct mlx5_context *mctx, struct mlx5_buf *buf,
size_t size, int page_size)
{
int found = 0;
- LIST_HEAD(slist);
int nchunk;
struct mlx5_hugetlb_mem *hmem;
+ struct list_node *tmp, *cur;
int ret;
buf->length = align(size, MLX5_Q_CHUNK_SIZE);
nchunk = buf->length / MLX5_Q_CHUNK_SIZE;
mlx5_spin_lock(&mctx->hugetlb_lock);
- list_for_each_entry(hmem, &mctx->hugetlb_list, list) {
+ list_for_each_node_safe(cur, tmp, &mctx->hugetlb_list) {
+ hmem = list_node(cur, struct mlx5_hugetlb_mem, entry);
if (bitmap_avail(&hmem->bitmap)) {
buf->base = bitmap_alloc_range(&hmem->bitmap, nchunk, 1);
if (buf->base != -1) {
@@ -297,9 +298,9 @@ static int alloc_huge_buf(struct mlx5_context *mctx, struct mlx5_buf *buf,
mlx5_spin_lock(&mctx->hugetlb_lock);
if (bitmap_avail(&hmem->bitmap))
- list_add(&hmem->list, &mctx->hugetlb_list);
+ list_add_node(&hmem->entry, &mctx->hugetlb_list);
else
- list_add_tail(&hmem->list, &mctx->hugetlb_list);
+ list_add_node_tail(&hmem->entry, &mctx->hugetlb_list);
mlx5_spin_unlock(&mctx->hugetlb_lock);
}
@@ -318,7 +319,7 @@ out_fork:
mlx5_spin_lock(&mctx->hugetlb_lock);
bitmap_free_range(&hmem->bitmap, buf->base, nchunk);
if (bitmap_empty(&hmem->bitmap)) {
- list_del(&hmem->list);
+ list_del_node(&hmem->entry);
mlx5_spin_unlock(&mctx->hugetlb_lock);
free_huge_mem(hmem);
} else
@@ -335,7 +336,7 @@ static void free_huge_buf(struct mlx5_context *ctx, struct mlx5_buf *buf)
mlx5_spin_lock(&ctx->hugetlb_lock);
bitmap_free_range(&buf->hmem->bitmap, buf->base, nchunk);
if (bitmap_empty(&buf->hmem->bitmap)) {
- list_del(&buf->hmem->list);
+ list_del_node(&buf->hmem->entry);
mlx5_spin_unlock(&ctx->hugetlb_lock);
free_huge_mem(buf->hmem);
} else
diff --git a/libmlx5/src/list.h b/libmlx5/src/list.h
deleted file mode 100644
index 4f96482..0000000
--- a/libmlx5/src/list.h
+++ /dev/null
@@ -1,312 +0,0 @@
-#ifndef _LINUX_LIST_H
-#define _LINUX_LIST_H
-
-/*
- * These are non-NULL pointers that will result in page faults
- * under normal circumstances, used to verify that nobody uses
- * non-initialized list entries.
- */
-#define LIST_POISON1 ((void *) 0x00100100)
-#define LIST_POISON2 ((void *) 0x00200200)
-
-/*
- * Simple doubly linked list implementation.
- *
- * Some of the internal functions ("__xxx") are useful when
- * manipulating whole lists rather than single entries, as
- * sometimes we already know the next/prev entries and we can
- * generate better code by using them directly rather than
- * using the generic single-entry routines.
- */
-
-struct list_head {
- struct list_head *next, *prev;
-};
-
-#define LIST_HEAD_INIT(name) { &(name), &(name) }
-
-#define LIST_HEAD(name) \
- struct list_head name = LIST_HEAD_INIT(name)
-
-#define INIT_LIST_HEAD(ptr) do { \
- (ptr)->next = (ptr); (ptr)->prev = (ptr); \
-} while (0)
-
-/*
- * Insert a new entry between two known consecutive entries.
- *
- * This is only for internal list manipulation where we know
- * the prev/next entries already!
- */
-static inline void __list_add(struct list_head *new,
- struct list_head *prev,
- struct list_head *next)
-{
- next->prev = new;
- new->next = next;
- new->prev = prev;
- prev->next = new;
-}
-
-/**
- * list_add - add a new entry
- * @new: new entry to be added
- * @head: list head to add it after
- *
- * Insert a new entry after the specified head.
- * This is good for implementing stacks.
- */
-static inline void list_add(struct list_head *new, struct list_head *head)
-{
- __list_add(new, head, head->next);
-}
-
-/**
- * list_add_tail - add a new entry
- * @new: new entry to be added
- * @head: list head to add it before
- *
- * Insert a new entry before the specified head.
- * This is useful for implementing queues.
- */
-static inline void list_add_tail(struct list_head *new, struct list_head *head)
-{
- __list_add(new, head->prev, head);
-}
-
-/*
- * Delete a list entry by making the prev/next entries
- * point to each other.
- *
- * This is only for internal list manipulation where we know
- * the prev/next entries already!
- */
-static inline void __list_del(struct list_head *prev, struct list_head *next)
-{
- next->prev = prev;
- prev->next = next;
-}
-
-/**
- * list_del - deletes entry from list.
- * @entry: the element to delete from the list.
- * Note: list_empty on entry does not return true after this, the entry is
- * in an undefined state.
- */
-static inline void list_del(struct list_head *entry)
-{
- __list_del(entry->prev, entry->next);
- entry->next = LIST_POISON1;
- entry->prev = LIST_POISON2;
-}
-
-/**
- * list_del_init - deletes entry from list and reinitialize it.
- * @entry: the element to delete from the list.
- */
-static inline void list_del_init(struct list_head *entry)
-{
- __list_del(entry->prev, entry->next);
- INIT_LIST_HEAD(entry);
-}
-
-/**
- * list_move - delete from one list and add as another's head
- * @list: the entry to move
- * @head: the head that will precede our entry
- */
-static inline void list_move(struct list_head *list, struct list_head *head)
-{
- __list_del(list->prev, list->next);
- list_add(list, head);
-}
-
-/**
- * list_move_tail - delete from one list and add as another's tail
- * @list: the entry to move
- * @head: the head that will follow our entry
- */
-static inline void list_move_tail(struct list_head *list,
- struct list_head *head)
-{
- __list_del(list->prev, list->next);
- list_add_tail(list, head);
-}
-
-/**
- * list_empty - tests whether a list is empty
- * @head: the list to test.
- */
-static inline int list_empty(const struct list_head *head)
-{
- return head->next == head;
-}
-
-/**
- * list_empty_careful - tests whether a list is
- * empty _and_ checks that no other CPU might be
- * in the process of still modifying either member
- *
- * NOTE: using list_empty_careful() without synchronization
- * can only be safe if the only activity that can happen
- * to the list entry is list_del_init(). Eg. it cannot be used
- * if another CPU could re-list_add() it.
- *
- * @head: the list to test.
- */
-static inline int list_empty_careful(const struct list_head *head)
-{
- struct list_head *next = head->next;
- return (next == head) && (next == head->prev);
-}
-
-static inline void __list_splice(struct list_head *list,
- struct list_head *head)
-{
- struct list_head *first = list->next;
- struct list_head *last = list->prev;
- struct list_head *at = head->next;
-
- first->prev = head;
- head->next = first;
-
- last->next = at;
- at->prev = last;
-}
-
-/**
- * list_splice - join two lists
- * @list: the new list to add.
- * @head: the place to add it in the first list.
- */
-static inline void list_splice(struct list_head *list, struct list_head *head)
-{
- if (!list_empty(list))
- __list_splice(list, head);
-}
-
-/**
- * list_splice_init - join two lists and reinitialise the emptied list.
- * @list: the new list to add.
- * @head: the place to add it in the first list.
- *
- * The list at @list is reinitialised
- */
-static inline void list_splice_init(struct list_head *list,
- struct list_head *head)
-{
- if (!list_empty(list)) {
- __list_splice(list, head);
- INIT_LIST_HEAD(list);
- }
-}
-
-/**
- * list_entry - get the struct for this entry
- * @ptr: the &struct list_head pointer.
- * @type: the type of the struct this is embedded in.
- * @member: the name of the list_struct within the struct.
- */
-#define list_entry(ptr, type, member) \
- container_of(ptr, type, member)
-
-/**
- * list_for_each - iterate over a list
- * @pos: the &struct list_head to use as a loop counter.
- * @head: the head for your list.
- */
-#define list_for_each(pos, head) \
- for (pos = (head)->next; prefetch(pos->next), pos != (head); \
- pos->next)
-
-/**
- * __list_for_each - iterate over a list
- * @pos: the &struct list_head to use as a loop counter.
- * @head: the head for your list.
- *
- * This variant differs from list_for_each() in that it's the
- * simplest possible list iteration code, no prefetching is done.
- * Use this for code that knows the list to be very short (empty
- * or 1 entry) most of the time.
- */
-#define __list_for_each(pos, head) \
- for (pos = (head)->next; pos != (head); pos = pos->next)
-
-/**
- * list_for_each_prev - iterate over a list backwards
- * @pos: the &struct list_head to use as a loop counter.
- * @head: the head for your list.
- */
-#define list_for_each_prev(pos, head) \
- for (pos = (head)->prev; prefetch(pos->prev), pos != (head); \
- pos = pos->prev)
-
-/**
- * list_for_each_safe - iterate over a list safe against removal of list entry
- * @pos: the &struct list_head to use as a loop counter.
- * @n: another &struct list_head to use as temporary storage
- * @head: the head for your list.
- */
-#define list_for_each_safe(pos, n, head) \
- for (pos = (head)->next, n = pos->next; pos != (head); \
- pos = n, n = pos->next)
-
-/**
- * list_for_each_entry - iterate over list of given type
- * @pos: the type * to use as a loop counter.
- * @head: the head for your list.
- * @member: the name of the list_struct within the struct.
- */
-#define list_for_each_entry(pos, head, member) \
- for (pos = list_entry((head)->next, typeof(*pos), member); \
- &pos->member != (head); \
- pos = list_entry(pos->member.next, typeof(*pos), member))
-
-/**
- * list_for_each_entry_reverse - iterate backwards over list of given type.
- * @pos: the type * to use as a loop counter.
- * @head: the head for your list.
- * @member: the name of the list_struct within the struct.
- */
-#define list_for_each_entry_reverse(pos, head, member) \
- for (pos = list_entry((head)->prev, typeof(*pos), member); \
- prefetch(pos->member.prev), &pos->member != (head); \
- pos = list_entry(pos->member.prev, typeof(*pos), member))
-
-/**
- * list_prepare_entry - prepare a pos entry for use as a start point in
- * list_for_each_entry_continue
- * @pos: the type * to use as a start point
- * @head: the head of the list
- * @member: the name of the list_struct within the struct.
- */
-#define list_prepare_entry(pos, head, member) \
- ((pos) ? : list_entry(head, typeof(*pos), member))
-
-/**
- * list_for_each_entry_continue - iterate over list of given type
- * continuing after existing point
- * @pos: the type * to use as a loop counter.
- * @head: the head for your list.
- * @member: the name of the list_struct within the struct.
- */
-#define list_for_each_entry_continue(pos, head, member) \
- for (pos = list_entry(pos->member.next, typeof(*pos), member); \
- prefetch(pos->member.next), &pos->member != (head); \
- pos = list_entry(pos->member.next, typeof(*pos), member))
-
-/**
- * list_for_each_entry_safe - iterate over list of given type safe against removal of list entry
- * @pos: the type * to use as a loop counter.
- * @n: another type * to use as temporary storage
- * @head: the head for your list.
- * @member: the name of the list_struct within the struct.
- */
-#define list_for_each_entry_safe(pos, n, head, member) \
- for (pos = list_entry((head)->next, typeof(*pos), member), \
- n = list_entry(pos->member.next, typeof(*pos), member); \
- &pos->member != (head); \
- pos = n, n = list_entry(n->member.next, typeof(*n), member))
-
-#endif
-
diff --git a/libmlx5/src/mlx5.h b/libmlx5/src/mlx5.h
index 453aeb8..7392499 100644
--- a/libmlx5/src/mlx5.h
+++ b/libmlx5/src/mlx5.h
@@ -39,10 +39,10 @@
#include <infiniband/driver.h>
#include <infiniband/arch.h>
#include "mlx5-abi.h"
-#include "list.h"
#include "bitmap.h"
#include "../../utils/math.h"
+#include "../../utils/list.h"
#ifdef __GNUC__
#define likely(x) __builtin_expect((x), 1)
@@ -355,7 +355,7 @@ struct mlx5_hugetlb_mem {
int shmid;
void *shmaddr;
struct mlx5_bitmap bitmap;
- struct list_head list;
+ struct list_node entry;
};
struct mlx5_buf {
diff --git a/utils/list.h b/utils/list.h
index e229348..bc03e2b 100644
--- a/utils/list.h
+++ b/utils/list.h
@@ -48,7 +48,7 @@ struct list_head {
PTHREAD_MUTEX_INITIALIZER }
#define LIST_HEAD(name) \
- struct list_head name = LIST_HEAD_INIT(name); \
+ struct list_head name = LIST_HEAD_INIT(name);
#define INIT_LIST_NODE(ptr) do { \
(ptr)->next = (ptr); (ptr)->prev = (ptr); \
@@ -72,6 +72,13 @@ static inline void list_add_node_tail(struct list_node *new,
__list_add_node(new, head->node.prev, &head->node);
}
+static inline void list_add_node(struct list_node *new,
+ struct list_head *head)
+{
+ __list_add_node(new, &head->node, head->node.next);
+}
+
+
static inline void __list_del_node(struct list_node *prev,
struct list_node *next)
{
--
2.7.4
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related
* [PATCH rdma-core 4/5] libocrdma: Move ocrdma's list implementation into common directory
From: Leon Romanovsky @ 2016-09-25 6:50 UTC (permalink / raw)
To: dledford-H+wXaHxf7aLQT0dZR+AlfA,
linux-rdma-u79uwXL29TY76Z2rM5mHXA
Cc: jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/,
yishaih-VPRAkNaXOzVWk0Htik3J/w
In-Reply-To: <1474786207-2149-1-git-send-email-leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
Signed-off-by: Leon Romanovsky <leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
---
libocrdma/src/ocrdma_list.h | 104 -------------------------------------------
libocrdma/src/ocrdma_main.c | 8 ++--
libocrdma/src/ocrdma_main.h | 12 ++---
libocrdma/src/ocrdma_verbs.c | 16 +++----
utils/list.h | 104 +++++++++++++++++++++++++++++++++++++++++++
5 files changed, 122 insertions(+), 122 deletions(-)
delete mode 100644 libocrdma/src/ocrdma_list.h
create mode 100644 utils/list.h
diff --git a/libocrdma/src/ocrdma_list.h b/libocrdma/src/ocrdma_list.h
deleted file mode 100644
index 1e0f1ff..0000000
--- a/libocrdma/src/ocrdma_list.h
+++ /dev/null
@@ -1,104 +0,0 @@
-/*
- * Copyright (C) 2008-2013 Emulex. All rights reserved.
- * This software is available to you under a choice of one of two
- * licenses. You may choose to be licensed under the terms of the GNU
- * General Public License (GPL) Version 2, available from the file
- * COPYING in the main directory of this source tree, or the
- * BSD license below:
- *
- * Redistribution and use in source and binary forms, with or
- * without modification, are permitted provided that the following
- * conditions are met:
- *
- * - Redistributions of source code must retain the above
- * copyright notice, this list of conditions and the following
- * disclaimer.
- *
- * - Redistributions in binary form must reproduce the above
- * copyright notice, this list of conditions and the following
- * disclaimer in the documentation and/or other materials
- * provided with the distribution.
- *
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
- * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
- * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
- * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
- * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
- * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
- * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
- * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
- * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
- * THE POSSIBILITY OF SUCH DAMAGE.
- */
-
-#ifndef __OCRDMA_LIST_H__
-#define __OCRDMA_LIST_H__
-
-struct ocrdma_list_node {
- struct ocrdma_list_node *next, *prev;
-};
-
-struct ocrdma_list_head {
- struct ocrdma_list_node node;
- pthread_mutex_t lock;
-};
-
-#define DBLY_LIST_HEAD_INIT(name) { { &(name.node), &(name.node) } , \
- PTHREAD_MUTEX_INITIALIZER }
-
-#define DBLY_LIST_HEAD(name) \
- struct ocrdma_list_head name = DBLY_LIST_HEAD_INIT(name); \
-
-#define INIT_DBLY_LIST_NODE(ptr) do { \
- (ptr)->next = (ptr); (ptr)->prev = (ptr); \
-} while (0)
-
-#define INIT_DBLY_LIST_HEAD(ptr) INIT_DBLY_LIST_NODE(ptr.node)
-
-static inline void __list_add_node(struct ocrdma_list_node *new,
- struct ocrdma_list_node *prev,
- struct ocrdma_list_node *next)
-{
- next->prev = new;
- new->next = next;
- new->prev = prev;
- prev->next = new;
-}
-
-static inline void list_add_node_tail(struct ocrdma_list_node *new,
- struct ocrdma_list_head *head)
-{
- __list_add_node(new, head->node.prev, &head->node);
-}
-
-static inline void __list_del_node(struct ocrdma_list_node *prev,
- struct ocrdma_list_node *next)
-{
- next->prev = prev;
- prev->next = next;
-}
-
-static inline void list_del_node(struct ocrdma_list_node *entry)
-{
- __list_del_node(entry->prev, entry->next);
- entry->next = entry->prev = 0;
-}
-
-#define list_lock(head) pthread_mutex_lock(&((head)->lock))
-#define list_unlock(head) pthread_mutex_unlock(&((head)->lock))
-
-#define list_node(ptr, type, member) \
- ((type *)((char *)(ptr)-(unsigned long)(&((type *)0)->member)))
-
-/**
- * list_for_each_node_safe - iterate over a list safe against removal of list entry
- * @pos: the &struct ocrdma_list_head to use as a loop counter.
- * @n: another &struct ocrdma_list_head to use as temporary storage
- * @head: the head for your list.
- */
-#define list_for_each_node_safe(pos, n, head) \
- for (pos = (head)->node.next, n = pos->next; pos != &((head)->node); \
- pos = n, n = pos->next)
-
-#endif /* __OCRDMA_LIST_H__ */
diff --git a/libocrdma/src/ocrdma_main.c b/libocrdma/src/ocrdma_main.c
index 5c494d8..6bd892c 100644
--- a/libocrdma/src/ocrdma_main.c
+++ b/libocrdma/src/ocrdma_main.c
@@ -46,7 +46,7 @@
#include "ocrdma_main.h"
#include "ocrdma_abi.h"
-#include "ocrdma_list.h"
+#include "../../utils/list.h"
#include <sys/types.h>
#include <sys/stat.h>
@@ -68,7 +68,7 @@ struct {
UCNA(EMULEX, GEN1), UCNA(EMULEX, GEN2), UCNA(EMULEX, GEN2_VF)
};
-static DBLY_LIST_HEAD(ocrdma_dev_list);
+static LIST_HEAD(ocrdma_dev_list);
static struct ibv_context *ocrdma_alloc_context(struct ibv_device *, int);
static void ocrdma_free_context(struct ibv_context *);
@@ -222,7 +222,7 @@ found:
pthread_mutex_init(&dev->dev_lock, NULL);
pthread_spin_init(&dev->flush_q_lock, PTHREAD_PROCESS_PRIVATE);
dev->ibv_dev.ops = ocrdma_dev_ops;
- INIT_DBLY_LIST_NODE(&dev->entry);
+ INIT_LIST_NODE(&dev->entry);
list_lock(&ocrdma_dev_list);
list_add_node_tail(&dev->entry, &ocrdma_dev_list);
list_unlock(&ocrdma_dev_list);
@@ -244,7 +244,7 @@ void ocrdma_register_driver(void)
static __attribute__ ((destructor))
void ocrdma_unregister_driver(void)
{
- struct ocrdma_list_node *cur, *tmp;
+ struct list_node *cur, *tmp;
struct ocrdma_device *dev;
list_lock(&ocrdma_dev_list);
list_for_each_node_safe(cur, tmp, &ocrdma_dev_list) {
diff --git a/libocrdma/src/ocrdma_main.h b/libocrdma/src/ocrdma_main.h
index c81188b..c7359c8 100644
--- a/libocrdma/src/ocrdma_main.h
+++ b/libocrdma/src/ocrdma_main.h
@@ -42,7 +42,7 @@
#include <infiniband/driver.h>
#include <infiniband/arch.h>
-#include "ocrdma_list.h"
+#include "../../utils/list.h"
#define ocrdma_err(format, arg...) printf(format, ##arg)
@@ -58,7 +58,7 @@ struct ocrdma_device {
struct ocrdma_qp **qp_tbl;
pthread_mutex_t dev_lock;
pthread_spinlock_t flush_q_lock;
- struct ocrdma_list_node entry;
+ struct list_node entry;
int id;
int gen;
uint32_t wqe_size;
@@ -106,8 +106,8 @@ struct ocrdma_cq {
uint8_t deferred_arm;
uint8_t deferred_sol;
uint8_t first_arm;
- struct ocrdma_list_head sq_head;
- struct ocrdma_list_head rq_head;
+ struct list_head sq_head;
+ struct list_head rq_head;
};
enum {
@@ -203,8 +203,8 @@ struct ocrdma_qp {
enum ibv_qp_type qp_type;
enum ocrdma_qp_state state;
- struct ocrdma_list_node sq_entry;
- struct ocrdma_list_node rq_entry;
+ struct list_node sq_entry;
+ struct list_node rq_entry;
uint16_t id;
uint16_t rsvd;
uint32_t db_shift;
diff --git a/libocrdma/src/ocrdma_verbs.c b/libocrdma/src/ocrdma_verbs.c
index 6062626..3efc8bb 100644
--- a/libocrdma/src/ocrdma_verbs.c
+++ b/libocrdma/src/ocrdma_verbs.c
@@ -51,7 +51,7 @@
#include "ocrdma_main.h"
#include "ocrdma_abi.h"
-#include "ocrdma_list.h"
+#include "../../utils/list.h"
static void ocrdma_ring_cq_db(struct ocrdma_cq *cq, uint32_t armed,
int solicited, uint32_t num_cqe);
@@ -307,8 +307,8 @@ static struct ibv_cq *ocrdma_create_cq_common(struct ibv_context *context,
ocrdma_ring_cq_db(cq, 0, 0, 0);
}
cq->ibv_cq.cqe = cqe;
- INIT_DBLY_LIST_HEAD(&cq->sq_head);
- INIT_DBLY_LIST_HEAD(&cq->rq_head);
+ INIT_LIST_HEAD(&cq->sq_head);
+ INIT_LIST_HEAD(&cq->rq_head);
return &cq->ibv_cq;
cq_err2:
(void)ibv_cmd_destroy_cq(&cq->ibv_cq);
@@ -621,8 +621,8 @@ struct ibv_qp *ocrdma_create_qp(struct ibv_pd *pd,
}
}
qp->state = OCRDMA_QPS_RST;
- INIT_DBLY_LIST_NODE(&qp->sq_entry);
- INIT_DBLY_LIST_NODE(&qp->rq_entry);
+ INIT_LIST_NODE(&qp->sq_entry);
+ INIT_LIST_NODE(&qp->rq_entry);
return &qp->ibv_qp;
map_err:
@@ -663,7 +663,7 @@ static int ocrdma_is_qp_in_sq_flushlist(struct ocrdma_cq *cq,
struct ocrdma_qp *qp)
{
struct ocrdma_qp *list_qp;
- struct ocrdma_list_node *cur, *tmp;
+ struct list_node *cur, *tmp;
int found = 0;
list_for_each_node_safe(cur, tmp, &cq->sq_head) {
list_qp = list_node(cur, struct ocrdma_qp, sq_entry);
@@ -679,7 +679,7 @@ static int ocrdma_is_qp_in_rq_flushlist(struct ocrdma_cq *cq,
struct ocrdma_qp *qp)
{
struct ocrdma_qp *list_qp;
- struct ocrdma_list_node *cur, *tmp;
+ struct list_node *cur, *tmp;
int found = 0;
list_for_each_node_safe(cur, tmp, &cq->rq_head) {
list_qp = list_node(cur, struct ocrdma_qp, rq_entry);
@@ -2034,7 +2034,7 @@ int ocrdma_poll_cq(struct ibv_cq *ibcq, int num_entries, struct ibv_wc *wc)
int cqes_to_poll = num_entries;
int num_os_cqe = 0, err_cqes = 0;
struct ocrdma_qp *qp;
- struct ocrdma_list_node *cur, *tmp;
+ struct list_node *cur, *tmp;
cq = get_ocrdma_cq(ibcq);
pthread_spin_lock(&cq->cq_lock);
diff --git a/utils/list.h b/utils/list.h
new file mode 100644
index 0000000..e229348
--- /dev/null
+++ b/utils/list.h
@@ -0,0 +1,104 @@
+/*
+ * Copyright (C) 2008-2013 Emulex. All rights reserved.
+ * This software is available to you under a choice of one of two
+ * licenses. You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * BSD license below:
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
+ * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _RDMA_LIST_H__
+#define _RDMA_LIST_H__
+
+struct list_node {
+ struct list_node *next, *prev;
+};
+
+struct list_head {
+ struct list_node node;
+ pthread_mutex_t lock;
+};
+
+#define LIST_HEAD_INIT(name) { { &(name.node), &(name.node) } , \
+ PTHREAD_MUTEX_INITIALIZER }
+
+#define LIST_HEAD(name) \
+ struct list_head name = LIST_HEAD_INIT(name); \
+
+#define INIT_LIST_NODE(ptr) do { \
+ (ptr)->next = (ptr); (ptr)->prev = (ptr); \
+} while (0)
+
+#define INIT_LIST_HEAD(ptr) INIT_LIST_NODE(ptr.node)
+
+static inline void __list_add_node(struct list_node *new,
+ struct list_node *prev,
+ struct list_node *next)
+{
+ next->prev = new;
+ new->next = next;
+ new->prev = prev;
+ prev->next = new;
+}
+
+static inline void list_add_node_tail(struct list_node *new,
+ struct list_head *head)
+{
+ __list_add_node(new, head->node.prev, &head->node);
+}
+
+static inline void __list_del_node(struct list_node *prev,
+ struct list_node *next)
+{
+ next->prev = prev;
+ prev->next = next;
+}
+
+static inline void list_del_node(struct list_node *entry)
+{
+ __list_del_node(entry->prev, entry->next);
+ entry->next = entry->prev = 0;
+}
+
+#define list_lock(head) pthread_mutex_lock(&((head)->lock))
+#define list_unlock(head) pthread_mutex_unlock(&((head)->lock))
+
+#define list_node(ptr, type, member) \
+ ((type *)((char *)(ptr)-(unsigned long)(&((type *)0)->member)))
+
+/**
+ * list_for_each_node_safe - iterate over a list safe against removal of list entry
+ * @pos: the &struct list_head to use as a loop counter.
+ * @n: another &struct list_head to use as temporary storage
+ * @head: the head for your list.
+ */
+#define list_for_each_node_safe(pos, n, head) \
+ for (pos = (head)->node.next, n = pos->next; pos != &((head)->node); \
+ pos = n, n = pos->next)
+
+#endif /* _RDMA_LIST_H_ */
--
2.7.4
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related
* [PATCH rdma-core 3/5] utils: Remove container_of and offset local declarations
From: Leon Romanovsky @ 2016-09-25 6:50 UTC (permalink / raw)
To: dledford-H+wXaHxf7aLQT0dZR+AlfA,
linux-rdma-u79uwXL29TY76Z2rM5mHXA
Cc: jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/,
yishaih-VPRAkNaXOzVWk0Htik3J/w
In-Reply-To: <1474786207-2149-1-git-send-email-leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
Signed-off-by: Leon Romanovsky <leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
---
ibacm/linux/osd.h | 3 ---
libmlx5/src/list.h | 19 -------------------
librdmacm/src/cma.h | 6 ------
3 files changed, 28 deletions(-)
diff --git a/ibacm/linux/osd.h b/ibacm/linux/osd.h
index ec18f7b..1a738e0 100644
--- a/ibacm/linux/osd.h
+++ b/ibacm/linux/osd.h
@@ -55,9 +55,6 @@
#define LIB_DESTRUCTOR __attribute__((destructor))
#define CDECL_FUNC
-#define container_of(ptr, type, field) \
- ((type *) ((void *) ptr - offsetof(type, field)))
-
#if __BYTE_ORDER == __LITTLE_ENDIAN
#define htonll(x) bswap_64(x)
#else
diff --git a/libmlx5/src/list.h b/libmlx5/src/list.h
index cd7d25b..4f96482 100644
--- a/libmlx5/src/list.h
+++ b/libmlx5/src/list.h
@@ -201,25 +201,6 @@ static inline void list_splice_init(struct list_head *list,
}
}
-#ifndef offsetof
-#define offsetof(TYPE, MEMBER) ((size_t) &((TYPE *)0)->MEMBER)
-#endif
-
-/**
- * container_of - cast a member of a structure out to the containing structure
- *
- * @ptr: the pointer to the member.
- * @type: the type of the container struct this is embedded in.
- * @member: the name of the member within the struct.
- *
- */
-#ifndef container_of
-#define container_of(ptr, type, member) ({ \
- const typeof(((type *)0)->member)*__mptr = (ptr); \
- (type *)((char *)__mptr - offsetof(type, member)); })
-#endif
-
-
/**
* list_entry - get the struct for this entry
* @ptr: the &struct list_head pointer.
diff --git a/librdmacm/src/cma.h b/librdmacm/src/cma.h
index d640688..83d98b0 100644
--- a/librdmacm/src/cma.h
+++ b/librdmacm/src/cma.h
@@ -70,12 +70,6 @@ static inline uint64_t htonll(uint64_t x) { return x; }
static inline uint64_t ntohll(uint64_t x) { return x; }
#endif
-#ifndef container_of
-#define container_of(ptr, type, field) \
- ((type *) ((void *)ptr - offsetof(type, field)))
-#endif
-
-
/*
* Fast synchronization for low contention locking.
*/
--
2.7.4
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related
* [PATCH rdma-core 2/5] utils: Create utils directory to put all common code and move min/max into it
From: Leon Romanovsky @ 2016-09-25 6:50 UTC (permalink / raw)
To: dledford-H+wXaHxf7aLQT0dZR+AlfA,
linux-rdma-u79uwXL29TY76Z2rM5mHXA
Cc: jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/,
yishaih-VPRAkNaXOzVWk0Htik3J/w
In-Reply-To: <1474786207-2149-1-git-send-email-leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
Signed-off-by: Leon Romanovsky <leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
---
CMakeLists.txt | 6 ++++++
ibacm/linux/osd.h | 5 ++---
libibverbs/examples/rc_pingpong.c | 8 +------
libmlx5/src/mlx5.h | 16 ++------------
librdmacm/src/cma.h | 5 ++---
utils/math.h | 44 +++++++++++++++++++++++++++++++++++++++
6 files changed, 57 insertions(+), 27 deletions(-)
create mode 100644 utils/math.h
diff --git a/CMakeLists.txt b/CMakeLists.txt
index 557d3a8..c997c6d 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -38,6 +38,8 @@ set(BUILD_INCLUDE ${CMAKE_BINARY_DIR}/include)
set(BUILD_BIN ${CMAKE_BINARY_DIR}/bin)
# Libraries
set(BUILD_LIB ${CMAKE_BINARY_DIR}/lib)
+# Common code
+set(COMMON_CODE utils)
# Location to place provider .driver files
set(CONFIG_DIR "${CMAKE_INSTALL_FULL_SYSCONFDIR}/libibverbs.d")
@@ -139,6 +141,10 @@ RDMA_AddOptCFlag(CMAKE_SHARED_LINKER_FLAGS SUPPORTS_NO_UNDEFINED "-Wl,--no-undef
find_package(LDSymVer REQUIRED)
#-------------------------
+# Common code
+include_directories(${COMMON_CODE})
+
+#-------------------------
# Find libraries
# pthread
FIND_PACKAGE (Threads REQUIRED)
diff --git a/ibacm/linux/osd.h b/ibacm/linux/osd.h
index 5ca4c6f..ec18f7b 100644
--- a/ibacm/linux/osd.h
+++ b/ibacm/linux/osd.h
@@ -46,6 +46,8 @@
#include <sys/time.h>
#include <netinet/in.h>
+#include "../../utils/math.h"
+
#define ACM_CONF_DIR IBACM_CONFIG_PATH
#define ACM_ADDR_FILE "ibacm_addr.cfg"
#define ACM_OPTS_FILE "ibacm_opts.cfg"
@@ -56,9 +58,6 @@
#define container_of(ptr, type, field) \
((type *) ((void *) ptr - offsetof(type, field)))
-#define min(a, b) (a < b ? a : b)
-#define max(a, b) (a > b ? a : b)
-
#if __BYTE_ORDER == __LITTLE_ENDIAN
#define htonll(x) bswap_64(x)
#else
diff --git a/libibverbs/examples/rc_pingpong.c b/libibverbs/examples/rc_pingpong.c
index 9676783..cc45741 100644
--- a/libibverbs/examples/rc_pingpong.c
+++ b/libibverbs/examples/rc_pingpong.c
@@ -49,13 +49,7 @@
#include "pingpong.h"
-#ifndef max
-#define max(x, y) (((x) > (y)) ? (x) : (y))
-#endif
-
-#ifndef min
-#define min(x, y) (((x) < (y)) ? (x) : (y))
-#endif
+#include "../../utils/math.h"
enum {
PINGPONG_RECV_WRID = 1,
diff --git a/libmlx5/src/mlx5.h b/libmlx5/src/mlx5.h
index dc90892..453aeb8 100644
--- a/libmlx5/src/mlx5.h
+++ b/libmlx5/src/mlx5.h
@@ -42,6 +42,8 @@
#include "list.h"
#include "bitmap.h"
+#include "../../utils/math.h"
+
#ifdef __GNUC__
#define likely(x) __builtin_expect((x), 1)
#define unlikely(x) __builtin_expect((x), 0)
@@ -91,20 +93,6 @@
#endif
-#ifndef min
-#define min(a, b) \
- ({ typeof(a) _a = (a); \
- typeof(b) _b = (b); \
- _a < _b ? _a : _b; })
-#endif
-
-#ifndef max
-#define max(a, b) \
- ({ typeof(a) _a = (a); \
- typeof(b) _b = (b); \
- _a > _b ? _a : _b; })
-#endif
-
#define HIDDEN __attribute__((visibility("hidden")))
#ifdef HAVE_FUNC_ATTRIBUTE_ALWAYS_INLINE
diff --git a/librdmacm/src/cma.h b/librdmacm/src/cma.h
index 98eba8d..d640688 100644
--- a/librdmacm/src/cma.h
+++ b/librdmacm/src/cma.h
@@ -47,6 +47,8 @@
#include <rdma/rdma_cma.h>
#include <infiniband/ib.h>
+#include "../../utils/math.h"
+
#ifdef INCLUDE_VALGRIND
# include <valgrind/memcheck.h>
# ifndef VALGRIND_MAKE_MEM_DEFINED
@@ -68,9 +70,6 @@ static inline uint64_t htonll(uint64_t x) { return x; }
static inline uint64_t ntohll(uint64_t x) { return x; }
#endif
-#define max(a, b) ((a) > (b) ? a : b)
-#define min(a, b) ((a) < (b) ? a : b)
-
#ifndef container_of
#define container_of(ptr, type, field) \
((type *) ((void *)ptr - offsetof(type, field)))
diff --git a/utils/math.h b/utils/math.h
new file mode 100644
index 0000000..ffef543
--- /dev/null
+++ b/utils/math.h
@@ -0,0 +1,44 @@
+/*
+ * This software is available to you under a choice of one of two
+ * licenses. You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ *
+ */
+
+#ifndef _RDMA_MATH_H_
+#define _RDMA_MATH_H_
+/*
+ * This include contains all common mathematical functions
+ */
+#ifndef min
+#define min(a, b) (((a) < (b)) ? (a) : (b))
+#endif
+
+#ifndef max
+#define max(a, b) (((a) > (b)) ? (a) : (b))
+#endif
+#endif /* _RDMA_MATH_H_ */
--
2.7.4
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related
* [PATCH rdma-core 1/5] cmake: Update build instructions in CmakeList
From: Leon Romanovsky @ 2016-09-25 6:50 UTC (permalink / raw)
To: dledford-H+wXaHxf7aLQT0dZR+AlfA,
linux-rdma-u79uwXL29TY76Z2rM5mHXA
Cc: jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/,
yishaih-VPRAkNaXOzVWk0Htik3J/w
In-Reply-To: <1474786207-2149-1-git-send-email-leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
Signed-off-by: Leon Romanovsky <leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
---
CMakeLists.txt | 1 +
1 file changed, 1 insertion(+)
diff --git a/CMakeLists.txt b/CMakeLists.txt
index a829cd4..557d3a8 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -1,6 +1,7 @@
# COPYRIGHT (c) 2016 Obsidian Research Corporation. See COPYING file
# Run cmake as:
# mkdir build
+# cd build
# cmake -GNinja ..
# ninja
#
--
2.7.4
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related
* [PATCH rdma-core 0/5] Add directory to place common code and move trivial functions into it
From: Leon Romanovsky @ 2016-09-25 6:50 UTC (permalink / raw)
To: dledford-H+wXaHxf7aLQT0dZR+AlfA,
linux-rdma-u79uwXL29TY76Z2rM5mHXA
Cc: jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/,
yishaih-VPRAkNaXOzVWk0Htik3J/w
This is patch series which adds directory to place common code.
Highlights of this this series:
1. The min/max functions were moved into it.
2. OCRDMA list implementation was chosen as common code.
3. Libmlx5 was converted to use that list.h.
4. Remove container_of and offset local declarations.
View the commits on github:
https://github.com/linux-rdma/rdma-core/pull/4
Leon Romanovsky (5):
cmake: Update build instructions in CmakeList
utils: Create utils directory to put all common code and move min/max
into it
utils: Remove container_of and offset local declarations
libocrdma: Move ocrdma's list implementation into common directory
libmlx5: Convert libmlx5 to use common list implementation
CMakeLists.txt | 7 +
ibacm/linux/osd.h | 8 +-
libibverbs/examples/rc_pingpong.c | 8 +-
libmlx5/src/buf.c | 13 +-
libmlx5/src/list.h | 331 --------------------------------------
libmlx5/src/mlx5.h | 20 +--
libocrdma/src/ocrdma_list.h | 104 ------------
libocrdma/src/ocrdma_main.c | 8 +-
libocrdma/src/ocrdma_main.h | 12 +-
libocrdma/src/ocrdma_verbs.c | 16 +-
librdmacm/src/cma.h | 11 +-
utils/list.h | 111 +++++++++++++
utils/math.h | 44 +++++
13 files changed, 196 insertions(+), 497 deletions(-)
delete mode 100644 libmlx5/src/list.h
delete mode 100644 libocrdma/src/ocrdma_list.h
create mode 100644 utils/list.h
create mode 100644 utils/math.h
--
2.7.4
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Towards qedr patch series
From: Amrani, Ram @ 2016-09-25 5:39 UTC (permalink / raw)
To: dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org
Cc: Elior, Ariel, Kalderon, Michal, Mintz, Yuval, Borundia, Rajesh,
linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
Hi Doug,
We will release qedr RFC v3 this week that will address all of the comments received on v1 and v2. Following this RFC we would like to generate a patch series.
Note that the RFC contains patches to qed and qede that are maintained in netdev so the submission may be affected by netdev activity.
Currently the RFC is based on netdev's net-next but we can rebase against your git (git://github.com/dledford/linux.git).
What guidelines should we follow with this submission?
What is the time frame to make it into kernel 4.9?
Thanks,
Ram
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH] rdma_rxe: Fix kernel crash when IPV6 disabled.
From: Parav Pandit @ 2016-09-25 5:16 UTC (permalink / raw)
To: Leon Romanovsky
Cc: Moni Shoua, Doug Ledford, linux-rdma-u79uwXL29TY76Z2rM5mHXA,
Hal Rosenstock
In-Reply-To: <20160925043648.GT4088-2ukJVAZIZ/Y@public.gmane.org>
On Sun, Sep 25, 2016 at 10:06 AM, Leon Romanovsky <leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org> wrote:
> On Sat, Sep 24, 2016 at 03:24:34PM +0000, Parav Pandit wrote:
>> This patch fixes below kernel crash when CONFIG_IPV6 is disabled by
>> avoiding ip6 socket initialization.
>>
>> Additionally it has below minor related cleanup around recv_sockets elements.
>>
>> 1. Avoid extern for structure which is locally used in file
>> 2. Define recv_sockets as static as its used by single file
>> 3. Follow reverse deallocation order than allocation order for
>> recv_sockets elements
>
> Thanks Parav,
> This crash was already fixed upstream.
> https://git.kernel.org/cgit/linux/kernel/git/dledford/rdma.git/commit/drivers/infiniband/sw/rxe?id=dfdd6158ca2c8838313470c5ab79243d04da70b2
ok. Thanks. I was in 4.8-rc branch and I searched for rdma_rxe tag for
commits based on Stephen's rdma_rxe tag. This one was done using
IB/rxe.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH] rdma_rxe: Fix kernel crash when IPV6 disabled.
From: Leon Romanovsky @ 2016-09-25 4:36 UTC (permalink / raw)
To: Parav Pandit
Cc: monis-VPRAkNaXOzVWk0Htik3J/w, dledford-H+wXaHxf7aLQT0dZR+AlfA,
linux-rdma-u79uwXL29TY76Z2rM5mHXA,
hal.rosenstock-Re5JQEeQqe8AvxtiuMwx3w
In-Reply-To: <1474730674-4017-1-git-send-email-pandit.parav-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
[-- Attachment #1: Type: text/plain, Size: 661 bytes --]
On Sat, Sep 24, 2016 at 03:24:34PM +0000, Parav Pandit wrote:
> This patch fixes below kernel crash when CONFIG_IPV6 is disabled by
> avoiding ip6 socket initialization.
>
> Additionally it has below minor related cleanup around recv_sockets elements.
>
> 1. Avoid extern for structure which is locally used in file
> 2. Define recv_sockets as static as its used by single file
> 3. Follow reverse deallocation order than allocation order for
> recv_sockets elements
Thanks Parav,
This crash was already fixed upstream.
https://git.kernel.org/cgit/linux/kernel/git/dledford/rdma.git/commit/drivers/infiniband/sw/rxe?id=dfdd6158ca2c8838313470c5ab79243d04da70b2
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply
* [PATCH v5 16/16] MAINTAINERS: Update for PVRDMA driver
From: Adit Ranadive @ 2016-09-24 23:21 UTC (permalink / raw)
To: dledford, linux-rdma, pv-drivers
Cc: Adit Ranadive, netdev, linux-pci, jhansen, asarwade, georgezhang,
bryantan
In-Reply-To: <cover.1474759181.git.aditr@vmware.com>
Add maintainer info for the PVRDMA driver.
Reviewed-by: Jorgen Hansen <jhansen@vmware.com>
Reviewed-by: George Zhang <georgezhang@vmware.com>
Reviewed-by: Aditya Sarwade <asarwade@vmware.com>
Reviewed-by: Bryan Tan <bryantan@vmware.com>
Signed-off-by: Adit Ranadive <aditr@vmware.com>
---
Changes v4->v5:
- Added pvrdma files to common UAPI folder.
---
MAINTAINERS | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/MAINTAINERS b/MAINTAINERS
index 87e23cd..5023dc0 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -12615,6 +12615,15 @@ S: Maintained
F: drivers/scsi/vmw_pvscsi.c
F: drivers/scsi/vmw_pvscsi.h
+VMWARE PVRDMA DRIVER
+M: Adit Ranadive <aditr@vmware.com>
+M: VMware PV-Drivers <pv-drivers@vmware.com>
+L: linux-rdma@vger.kernel.org
+S: Maintained
+F: drivers/infiniband/hw/pvrdma/
+F: include/uapi/rdma/pvrdma-abi.h
+F: include/uapi/rdma/pvrdma-uapi.h
+
VOLTAGE AND CURRENT REGULATOR FRAMEWORK
M: Liam Girdwood <lgirdwood@gmail.com>
M: Mark Brown <broonie@kernel.org>
--
2.7.4
^ permalink raw reply related
* [PATCH v5 15/16] IB: Add PVRDMA driver
From: Adit Ranadive @ 2016-09-24 23:21 UTC (permalink / raw)
To: dledford, linux-rdma, pv-drivers
Cc: Adit Ranadive, netdev, linux-pci, jhansen, asarwade, georgezhang,
bryantan
In-Reply-To: <cover.1474759181.git.aditr@vmware.com>
This patch updates the InfiniBand subsystem to build the PVRDMA driver.
Reviewed-by: Jorgen Hansen <jhansen@vmware.com>
Reviewed-by: George Zhang <georgezhang@vmware.com>
Reviewed-by: Aditya Sarwade <asarwade@vmware.com>
Reviewed-by: Bryan Tan <bryantan@vmware.com>
Signed-off-by: Adit Ranadive <aditr@vmware.com>
---
drivers/infiniband/Kconfig | 1 +
drivers/infiniband/hw/Makefile | 1 +
2 files changed, 2 insertions(+)
diff --git a/drivers/infiniband/Kconfig b/drivers/infiniband/Kconfig
index 19a418a..dff4bcf 100644
--- a/drivers/infiniband/Kconfig
+++ b/drivers/infiniband/Kconfig
@@ -88,5 +88,6 @@ source "drivers/infiniband/sw/rdmavt/Kconfig"
source "drivers/infiniband/sw/rxe/Kconfig"
source "drivers/infiniband/hw/hfi1/Kconfig"
+source "drivers/infiniband/hw/pvrdma/Kconfig"
endif # INFINIBAND
diff --git a/drivers/infiniband/hw/Makefile b/drivers/infiniband/hw/Makefile
index 21fe401..c8a7a36 100644
--- a/drivers/infiniband/hw/Makefile
+++ b/drivers/infiniband/hw/Makefile
@@ -10,3 +10,4 @@ obj-$(CONFIG_INFINIBAND_OCRDMA) += ocrdma/
obj-$(CONFIG_INFINIBAND_USNIC) += usnic/
obj-$(CONFIG_INFINIBAND_HFI1) += hfi1/
obj-$(CONFIG_INFINIBAND_HNS) += hns/
+obj-$(CONFIG_INFINIBAND_PVRDMA) += pvrdma/
--
2.7.4
^ permalink raw reply related
* [PATCH v5 14/16] IB/pvrdma: Add Kconfig and Makefile
From: Adit Ranadive @ 2016-09-24 23:21 UTC (permalink / raw)
To: dledford, linux-rdma, pv-drivers
Cc: Adit Ranadive, netdev, linux-pci, jhansen, asarwade, georgezhang,
bryantan
In-Reply-To: <cover.1474759181.git.aditr@vmware.com>
This patch adds a Kconfig and Makefile for the PVRDMA driver.
Reviewed-by: Jorgen Hansen <jhansen@vmware.com>
Reviewed-by: George Zhang <georgezhang@vmware.com>
Reviewed-by: Aditya Sarwade <asarwade@vmware.com>
Reviewed-by: Bryan Tan <bryantan@vmware.com>
Signed-off-by: Adit Ranadive <aditr@vmware.com>
---
Changes v3->v4:
- Enforced dependency on VMXNet3
---
drivers/infiniband/hw/pvrdma/Kconfig | 7 +++++++
drivers/infiniband/hw/pvrdma/Makefile | 3 +++
2 files changed, 10 insertions(+)
create mode 100644 drivers/infiniband/hw/pvrdma/Kconfig
create mode 100644 drivers/infiniband/hw/pvrdma/Makefile
diff --git a/drivers/infiniband/hw/pvrdma/Kconfig b/drivers/infiniband/hw/pvrdma/Kconfig
new file mode 100644
index 0000000..b345679
--- /dev/null
+++ b/drivers/infiniband/hw/pvrdma/Kconfig
@@ -0,0 +1,7 @@
+config INFINIBAND_PVRDMA
+ tristate "VMware Paravirtualized RDMA Driver"
+ depends on NETDEVICES && ETHERNET && PCI && INET && VMXNET3
+ ---help---
+ This driver provides low-level support for VMware Paravirtual
+ RDMA adapter. It interacts with the VMXNet3 driver to provide
+ Ethernet capabilities.
diff --git a/drivers/infiniband/hw/pvrdma/Makefile b/drivers/infiniband/hw/pvrdma/Makefile
new file mode 100644
index 0000000..e6f078b
--- /dev/null
+++ b/drivers/infiniband/hw/pvrdma/Makefile
@@ -0,0 +1,3 @@
+obj-$(CONFIG_INFINIBAND_PVRDMA) += pvrdma.o
+
+pvrdma-y := pvrdma_cmd.o pvrdma_cq.o pvrdma_doorbell.o pvrdma_main.o pvrdma_misc.o pvrdma_mr.o pvrdma_qp.o pvrdma_verbs.o
--
2.7.4
^ permalink raw reply related
* [PATCH v5 13/16] IB/pvrdma: Add the main driver module for PVRDMA
From: Adit Ranadive @ 2016-09-24 23:21 UTC (permalink / raw)
To: dledford, linux-rdma, pv-drivers
Cc: Adit Ranadive, netdev, linux-pci, jhansen, asarwade, georgezhang,
bryantan
In-Reply-To: <cover.1474759181.git.aditr@vmware.com>
This patch adds the support to register a RDMA device with the kernel RDMA
stack as well as a kernel module. This also initializes the underlying
virtual PCI device.
Reviewed-by: Yuval Shaia <yuval.shaia@oracle.com>
Reviewed-by: Jorgen Hansen <jhansen@vmware.com>
Reviewed-by: George Zhang <georgezhang@vmware.com>
Reviewed-by: Aditya Sarwade <asarwade@vmware.com>
Reviewed-by: Bryan Tan <bryantan@vmware.com>
Signed-off-by: Adit Ranadive <aditr@vmware.com>
---
Changes v4->v5:
- Removed two unnecessary lines.
- Updated include for headers in UAPI folder.
- Update to pvrdma_cmd_post for add/delete GIDs.
- Add error code in dev_warn if pvrdma_cmd_post failed.
Changes v3->v4:
- Fixed some checkpatch warnings.
- Added support for new get_dev_fw_str API.
- Added event workqueue for netdevice events.
- Restructured the pvrdma_pci_remove function a little bit.
Changes v2->v3:
- Removed boolean in pvrdma_cmd_post.
Changes v1->v2:
- Addressed 32-bit build errors
- Cosmetic change to avoid if else in intr0_handler
- Removed unnecessary return assignment.
---
drivers/infiniband/hw/pvrdma/pvrdma_main.c | 1220 ++++++++++++++++++++++++++++
1 file changed, 1220 insertions(+)
create mode 100644 drivers/infiniband/hw/pvrdma/pvrdma_main.c
diff --git a/drivers/infiniband/hw/pvrdma/pvrdma_main.c b/drivers/infiniband/hw/pvrdma/pvrdma_main.c
new file mode 100644
index 0000000..94cbbb9
--- /dev/null
+++ b/drivers/infiniband/hw/pvrdma/pvrdma_main.c
@@ -0,0 +1,1220 @@
+/*
+ * Copyright (c) 2012-2016 VMware, Inc. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of EITHER the GNU General Public License
+ * version 2 as published by the Free Software Foundation or the BSD
+ * 2-Clause License. This program is distributed in the hope that it
+ * will be useful, but WITHOUT ANY WARRANTY; WITHOUT EVEN THE IMPLIED
+ * WARRANTY OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
+ * See the GNU General Public License version 2 for more details at
+ * http://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program available in the file COPYING in the main
+ * directory of this source tree.
+ *
+ * The BSD 2-Clause License
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
+ * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
+ * COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
+ * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+ * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+ * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+ * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED
+ * OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <linux/errno.h>
+#include <linux/inetdevice.h>
+#include <linux/init.h>
+#include <linux/module.h>
+#include <linux/slab.h>
+#include <rdma/ib_addr.h>
+#include <rdma/ib_smi.h>
+#include <rdma/ib_user_verbs.h>
+#include <net/addrconf.h>
+#include <rdma/pvrdma-abi.h>
+
+#include "pvrdma.h"
+
+#define DRV_NAME "pvrdma"
+#define DRV_VERSION "1.0"
+#define DRV_RELDATE "January 1, 2013"
+
+static const char pvrdma_version[] =
+ DRV_NAME ": PVRDMA InfiniBand driver v"
+ DRV_VERSION " (" DRV_RELDATE ")\n";
+
+static DEFINE_MUTEX(pvrdma_device_list_lock);
+static LIST_HEAD(pvrdma_device_list);
+static struct workqueue_struct *event_wq;
+
+static int pvrdma_add_gid(struct ib_device *ibdev,
+ u8 port_num,
+ unsigned int index,
+ const union ib_gid *gid,
+ const struct ib_gid_attr *attr,
+ void **context);
+static int pvrdma_del_gid(struct ib_device *ibdev,
+ u8 port_num,
+ unsigned int index,
+ void **context);
+
+
+static ssize_t show_hca(struct device *device, struct device_attribute *attr,
+ char *buf)
+{
+ return sprintf(buf, "PVRDMA%s\n", DRV_VERSION);
+}
+
+static ssize_t show_rev(struct device *device, struct device_attribute *attr,
+ char *buf)
+{
+ return sprintf(buf, "%d\n", PVRDMA_REV_ID);
+}
+
+static ssize_t show_board(struct device *device, struct device_attribute *attr,
+ char *buf)
+{
+ return sprintf(buf, "%d\n", PVRDMA_BOARD_ID);
+}
+
+static DEVICE_ATTR(hw_rev, S_IRUGO, show_rev, NULL);
+static DEVICE_ATTR(hca_type, S_IRUGO, show_hca, NULL);
+static DEVICE_ATTR(board_id, S_IRUGO, show_board, NULL);
+
+static struct device_attribute *pvrdma_class_attributes[] = {
+ &dev_attr_hw_rev,
+ &dev_attr_hca_type,
+ &dev_attr_board_id
+};
+
+static void pvrdma_get_fw_ver_str(struct ib_device *device, char *str,
+ size_t str_len)
+{
+ struct pvrdma_dev *dev =
+ container_of(device, struct pvrdma_dev, ib_dev);
+ snprintf(str, str_len, "%d.%d.%d\n",
+ (int) (dev->dsr->caps.fw_ver >> 32),
+ (int) (dev->dsr->caps.fw_ver >> 16) & 0xffff,
+ (int) dev->dsr->caps.fw_ver & 0xffff);
+}
+
+static int pvrdma_init_device(struct pvrdma_dev *dev)
+{
+ /* Initialize some device related stuff */
+ spin_lock_init(&dev->cmd_lock);
+ sema_init(&dev->cmd_sema, 1);
+ atomic_set(&dev->num_qps, 0);
+ atomic_set(&dev->num_cqs, 0);
+ atomic_set(&dev->num_pds, 0);
+ atomic_set(&dev->num_ahs, 0);
+
+ return 0;
+}
+
+static int pvrdma_port_immutable(struct ib_device *ibdev, u8 port_num,
+ struct ib_port_immutable *immutable)
+{
+ struct ib_port_attr attr;
+ int err;
+
+ err = pvrdma_query_port(ibdev, port_num, &attr);
+ if (err)
+ return err;
+
+ immutable->pkey_tbl_len = attr.pkey_tbl_len;
+ immutable->gid_tbl_len = attr.gid_tbl_len;
+ immutable->core_cap_flags = RDMA_CORE_PORT_IBA_ROCE;
+ immutable->max_mad_size = IB_MGMT_MAD_SIZE;
+ return 0;
+}
+
+static struct net_device *pvrdma_get_netdev(struct ib_device *ibdev,
+ u8 port_num)
+{
+ struct net_device *netdev;
+ struct pvrdma_dev *dev = to_vdev(ibdev);
+
+ if (port_num != 1)
+ return NULL;
+
+ rcu_read_lock();
+ netdev = dev->netdev;
+ if (netdev)
+ dev_hold(netdev);
+ rcu_read_unlock();
+
+ return netdev;
+}
+
+static int pvrdma_register_device(struct pvrdma_dev *dev)
+{
+ int ret = -1;
+ int i = 0;
+
+ strlcpy(dev->ib_dev.name, "pvrdma%d", IB_DEVICE_NAME_MAX);
+ dev->ib_dev.node_guid = dev->dsr->caps.node_guid;
+ dev->sys_image_guid = dev->dsr->caps.sys_image_guid;
+ dev->flags = 0;
+ dev->ib_dev.owner = THIS_MODULE;
+ dev->ib_dev.num_comp_vectors = 1;
+ dev->ib_dev.dma_device = &dev->pdev->dev;
+ dev->ib_dev.uverbs_abi_ver = PVRDMA_UVERBS_ABI_VERSION;
+ dev->ib_dev.uverbs_cmd_mask =
+ (1ull << IB_USER_VERBS_CMD_GET_CONTEXT) |
+ (1ull << IB_USER_VERBS_CMD_QUERY_DEVICE) |
+ (1ull << IB_USER_VERBS_CMD_QUERY_PORT) |
+ (1ull << IB_USER_VERBS_CMD_ALLOC_PD) |
+ (1ull << IB_USER_VERBS_CMD_DEALLOC_PD) |
+ (1ull << IB_USER_VERBS_CMD_REG_MR) |
+ (1ull << IB_USER_VERBS_CMD_DEREG_MR) |
+ (1ull << IB_USER_VERBS_CMD_CREATE_COMP_CHANNEL) |
+ (1ull << IB_USER_VERBS_CMD_CREATE_CQ) |
+ (1ull << IB_USER_VERBS_CMD_POLL_CQ) |
+ (1ull << IB_USER_VERBS_CMD_REQ_NOTIFY_CQ) |
+ (1ull << IB_USER_VERBS_CMD_DESTROY_CQ) |
+ (1ull << IB_USER_VERBS_CMD_CREATE_QP) |
+ (1ull << IB_USER_VERBS_CMD_MODIFY_QP) |
+ (1ull << IB_USER_VERBS_CMD_QUERY_QP) |
+ (1ull << IB_USER_VERBS_CMD_DESTROY_QP) |
+ (1ull << IB_USER_VERBS_CMD_POST_SEND) |
+ (1ull << IB_USER_VERBS_CMD_POST_RECV) |
+ (1ull << IB_USER_VERBS_CMD_CREATE_AH) |
+ (1ull << IB_USER_VERBS_CMD_DESTROY_AH);
+
+ dev->ib_dev.node_type = RDMA_NODE_IB_CA;
+ dev->ib_dev.phys_port_cnt = dev->dsr->caps.phys_port_cnt;
+
+ dev->ib_dev.query_device = pvrdma_query_device;
+ dev->ib_dev.query_port = pvrdma_query_port;
+ dev->ib_dev.query_gid = pvrdma_query_gid;
+ dev->ib_dev.query_pkey = pvrdma_query_pkey;
+ dev->ib_dev.modify_port = pvrdma_modify_port;
+ dev->ib_dev.alloc_ucontext = pvrdma_alloc_ucontext;
+ dev->ib_dev.dealloc_ucontext = pvrdma_dealloc_ucontext;
+ dev->ib_dev.mmap = pvrdma_mmap;
+ dev->ib_dev.alloc_pd = pvrdma_alloc_pd;
+ dev->ib_dev.dealloc_pd = pvrdma_dealloc_pd;
+ dev->ib_dev.create_ah = pvrdma_create_ah;
+ dev->ib_dev.destroy_ah = pvrdma_destroy_ah;
+ dev->ib_dev.create_qp = pvrdma_create_qp;
+ dev->ib_dev.modify_qp = pvrdma_modify_qp;
+ dev->ib_dev.query_qp = pvrdma_query_qp;
+ dev->ib_dev.destroy_qp = pvrdma_destroy_qp;
+ dev->ib_dev.post_send = pvrdma_post_send;
+ dev->ib_dev.post_recv = pvrdma_post_recv;
+ dev->ib_dev.create_cq = pvrdma_create_cq;
+ dev->ib_dev.modify_cq = pvrdma_modify_cq;
+ dev->ib_dev.resize_cq = pvrdma_resize_cq;
+ dev->ib_dev.destroy_cq = pvrdma_destroy_cq;
+ dev->ib_dev.poll_cq = pvrdma_poll_cq;
+ dev->ib_dev.req_notify_cq = pvrdma_req_notify_cq;
+ dev->ib_dev.get_dma_mr = pvrdma_get_dma_mr;
+ dev->ib_dev.reg_user_mr = pvrdma_reg_user_mr;
+ dev->ib_dev.dereg_mr = pvrdma_dereg_mr;
+ dev->ib_dev.alloc_mr = pvrdma_alloc_mr;
+ dev->ib_dev.map_mr_sg = pvrdma_map_mr_sg;
+ dev->ib_dev.add_gid = pvrdma_add_gid;
+ dev->ib_dev.del_gid = pvrdma_del_gid;
+ dev->ib_dev.get_netdev = pvrdma_get_netdev;
+ dev->ib_dev.get_port_immutable = pvrdma_port_immutable;
+ dev->ib_dev.get_link_layer = pvrdma_port_link_layer;
+ dev->ib_dev.get_dev_fw_str = pvrdma_get_fw_ver_str;
+
+ mutex_init(&dev->port_mutex);
+ spin_lock_init(&dev->desc_lock);
+
+ dev->cq_tbl = kcalloc(dev->dsr->caps.max_cq, sizeof(void *),
+ GFP_KERNEL);
+ if (!dev->cq_tbl)
+ return ret;
+ spin_lock_init(&dev->cq_tbl_lock);
+
+ dev->qp_tbl = kcalloc(dev->dsr->caps.max_qp, sizeof(void *),
+ GFP_KERNEL);
+ if (!dev->qp_tbl)
+ goto err_cq_free;
+ spin_lock_init(&dev->qp_tbl_lock);
+
+ ret = ib_register_device(&dev->ib_dev, NULL);
+ if (ret)
+ goto err_qp_free;
+
+ for (i = 0; i < ARRAY_SIZE(pvrdma_class_attributes); ++i) {
+ ret = device_create_file(&dev->ib_dev.dev,
+ pvrdma_class_attributes[i]);
+ if (ret)
+ goto err_class;
+ }
+
+ dev->ib_active = true;
+
+ return 0;
+
+err_class:
+ ib_unregister_device(&dev->ib_dev);
+err_qp_free:
+ kfree(dev->qp_tbl);
+err_cq_free:
+ kfree(dev->cq_tbl);
+
+ return ret;
+}
+
+static irqreturn_t pvrdma_intr0_handler(int irq, void *dev_id)
+{
+ u32 icr = PVRDMA_INTR_CAUSE_RESPONSE;
+ struct pvrdma_dev *dev = dev_id;
+
+ dev_dbg(&dev->pdev->dev, "interrupt 0 (response) handler\n");
+
+ if (dev->intr.type != PVRDMA_INTR_TYPE_MSIX) {
+ /* Legacy intr */
+ icr = pvrdma_read_reg(dev, PVRDMA_REG_ICR);
+ if (icr == 0)
+ return IRQ_NONE;
+ }
+
+ if (icr == PVRDMA_INTR_CAUSE_RESPONSE)
+ complete(&dev->cmd_done);
+
+ return IRQ_HANDLED;
+}
+
+static void pvrdma_qp_event(struct pvrdma_dev *dev, u32 qpn, int type)
+{
+ struct pvrdma_qp *qp;
+ unsigned long flags;
+
+ spin_lock_irqsave(&dev->qp_tbl_lock, flags);
+ qp = dev->qp_tbl[qpn % dev->dsr->caps.max_qp];
+ if (qp)
+ atomic_inc(&qp->refcnt);
+ spin_unlock_irqrestore(&dev->qp_tbl_lock, flags);
+
+ if (qp && qp->ibqp.event_handler) {
+ struct ib_qp *ibqp = &qp->ibqp;
+ struct ib_event e;
+
+ e.device = ibqp->device;
+ e.element.qp = ibqp;
+ e.event = type; /* 1:1 mapping for now. */
+ ibqp->event_handler(&e, ibqp->qp_context);
+ }
+ if (qp) {
+ atomic_dec(&qp->refcnt);
+ if (atomic_read(&qp->refcnt) == 0)
+ wake_up(&qp->wait);
+ }
+}
+
+static void pvrdma_cq_event(struct pvrdma_dev *dev, u32 cqn, int type)
+{
+ struct pvrdma_cq *cq;
+ unsigned long flags;
+
+ spin_lock_irqsave(&dev->cq_tbl_lock, flags);
+ cq = dev->cq_tbl[cqn % dev->dsr->caps.max_cq];
+ if (cq)
+ atomic_inc(&cq->refcnt);
+ spin_unlock_irqrestore(&dev->cq_tbl_lock, flags);
+
+ if (cq && cq->ibcq.event_handler) {
+ struct ib_cq *ibcq = &cq->ibcq;
+ struct ib_event e;
+
+ e.device = ibcq->device;
+ e.element.cq = ibcq;
+ e.event = type; /* 1:1 mapping for now. */
+ ibcq->event_handler(&e, ibcq->cq_context);
+ }
+ if (cq) {
+ atomic_dec(&cq->refcnt);
+ if (atomic_read(&cq->refcnt) == 0)
+ wake_up(&cq->wait);
+ }
+}
+
+static void pvrdma_dispatch_event(struct pvrdma_dev *dev, int port,
+ enum ib_event_type event)
+{
+ struct ib_event ib_event;
+
+ memset(&ib_event, 0, sizeof(ib_event));
+ ib_event.device = &dev->ib_dev;
+ ib_event.element.port_num = port;
+ ib_event.event = event;
+ ib_dispatch_event(&ib_event);
+}
+
+static void pvrdma_dev_event(struct pvrdma_dev *dev, u8 port, int type)
+{
+ if (port < 1 || port > dev->dsr->caps.phys_port_cnt) {
+ dev_warn(&dev->pdev->dev, "event on port %d\n", port);
+ return;
+ }
+
+ pvrdma_dispatch_event(dev, port, type);
+}
+
+static inline struct pvrdma_eqe *get_eqe(struct pvrdma_dev *dev, unsigned int i)
+{
+ return (struct pvrdma_eqe *)pvrdma_page_dir_get_ptr(
+ &dev->async_pdir,
+ PAGE_SIZE +
+ sizeof(struct pvrdma_eqe) * i);
+}
+
+static irqreturn_t pvrdma_intr1_handler(int irq, void *dev_id)
+{
+ struct pvrdma_dev *dev = dev_id;
+ struct pvrdma_ring *ring = &dev->async_ring_state->rx;
+ int ring_slots = (dev->dsr->async_ring_pages.num_pages - 1) *
+ PAGE_SIZE / sizeof(struct pvrdma_eqe);
+ unsigned int head;
+
+ dev_dbg(&dev->pdev->dev, "interrupt 1 (async event) handler\n");
+
+ /*
+ * Don't process events until the IB device is registered. Otherwise
+ * we'll try to ib_dispatch_event() on an invalid device.
+ */
+ if (!dev->ib_active)
+ return IRQ_HANDLED;
+
+ while (pvrdma_idx_ring_has_data(ring, ring_slots, &head) > 0) {
+ struct pvrdma_eqe *eqe;
+
+ eqe = get_eqe(dev, head);
+
+ switch (eqe->type) {
+ case PVRDMA_EVENT_QP_FATAL:
+ case PVRDMA_EVENT_QP_REQ_ERR:
+ case PVRDMA_EVENT_QP_ACCESS_ERR:
+ case PVRDMA_EVENT_COMM_EST:
+ case PVRDMA_EVENT_SQ_DRAINED:
+ case PVRDMA_EVENT_PATH_MIG:
+ case PVRDMA_EVENT_PATH_MIG_ERR:
+ case PVRDMA_EVENT_QP_LAST_WQE_REACHED:
+ pvrdma_qp_event(dev, eqe->info, eqe->type);
+ break;
+
+ case PVRDMA_EVENT_CQ_ERR:
+ pvrdma_cq_event(dev, eqe->info, eqe->type);
+ break;
+
+ case PVRDMA_EVENT_SRQ_ERR:
+ case PVRDMA_EVENT_SRQ_LIMIT_REACHED:
+ break;
+
+ case PVRDMA_EVENT_PORT_ACTIVE:
+ case PVRDMA_EVENT_PORT_ERR:
+ case PVRDMA_EVENT_LID_CHANGE:
+ case PVRDMA_EVENT_PKEY_CHANGE:
+ case PVRDMA_EVENT_SM_CHANGE:
+ case PVRDMA_EVENT_CLIENT_REREGISTER:
+ case PVRDMA_EVENT_GID_CHANGE:
+ pvrdma_dev_event(dev, eqe->info, eqe->type);
+ break;
+
+ case PVRDMA_EVENT_DEVICE_FATAL:
+ pvrdma_dev_event(dev, 1, eqe->type);
+ break;
+
+ default:
+ break;
+ }
+
+ pvrdma_idx_ring_inc(&ring->cons_head, ring_slots);
+ }
+
+ return IRQ_HANDLED;
+}
+
+static inline struct pvrdma_cqne *get_cqne(struct pvrdma_dev *dev,
+ unsigned int i)
+{
+ return (struct pvrdma_cqne *)pvrdma_page_dir_get_ptr(
+ &dev->cq_pdir,
+ PAGE_SIZE +
+ sizeof(struct pvrdma_cqne) * i);
+}
+
+static irqreturn_t pvrdma_intrx_handler(int irq, void *dev_id)
+{
+ struct pvrdma_dev *dev = dev_id;
+ struct pvrdma_ring *ring = &dev->cq_ring_state->rx;
+ int ring_slots = (dev->dsr->cq_ring_pages.num_pages - 1) * PAGE_SIZE /
+ sizeof(struct pvrdma_cqne);
+ unsigned int head;
+ unsigned long flags;
+
+ dev_dbg(&dev->pdev->dev, "interrupt x (completion) handler\n");
+
+ while (pvrdma_idx_ring_has_data(ring, ring_slots, &head) > 0) {
+ struct pvrdma_cqne *cqne;
+ struct pvrdma_cq *cq;
+
+ cqne = get_cqne(dev, head);
+ spin_lock_irqsave(&dev->cq_tbl_lock, flags);
+ cq = dev->cq_tbl[cqne->info % dev->dsr->caps.max_cq];
+ if (cq)
+ atomic_inc(&cq->refcnt);
+ spin_unlock_irqrestore(&dev->cq_tbl_lock, flags);
+
+ if (cq && cq->ibcq.comp_handler)
+ cq->ibcq.comp_handler(&cq->ibcq, cq->ibcq.cq_context);
+ if (cq) {
+ atomic_dec(&cq->refcnt);
+ if (atomic_read(&cq->refcnt))
+ wake_up(&cq->wait);
+ }
+ pvrdma_idx_ring_inc(&ring->cons_head, ring_slots);
+ }
+
+ return IRQ_HANDLED;
+}
+
+static void pvrdma_disable_msi_all(struct pvrdma_dev *dev)
+{
+ if (dev->intr.type == PVRDMA_INTR_TYPE_MSIX)
+ pci_disable_msix(dev->pdev);
+ else if (dev->intr.type == PVRDMA_INTR_TYPE_MSI)
+ pci_disable_msi(dev->pdev);
+}
+
+static void pvrdma_free_irq(struct pvrdma_dev *dev)
+{
+ int i;
+
+ dev_dbg(&dev->pdev->dev, "freeing interrupts\n");
+
+ if (dev->intr.type == PVRDMA_INTR_TYPE_MSIX) {
+ for (i = 0; i < dev->intr.size; i++) {
+ if (dev->intr.enabled[i]) {
+ free_irq(dev->intr.msix_entry[i].vector, dev);
+ dev->intr.enabled[i] = 0;
+ }
+ }
+ } else if (dev->intr.type == PVRDMA_INTR_TYPE_INTX ||
+ dev->intr.type == PVRDMA_INTR_TYPE_MSI) {
+ free_irq(dev->pdev->irq, dev);
+ }
+}
+
+static void pvrdma_enable_intrs(struct pvrdma_dev *dev)
+{
+ dev_dbg(&dev->pdev->dev, "enable interrupts\n");
+ pvrdma_write_reg(dev, PVRDMA_REG_IMR, 0);
+}
+
+static void pvrdma_disable_intrs(struct pvrdma_dev *dev)
+{
+ dev_dbg(&dev->pdev->dev, "disable interrupts\n");
+ pvrdma_write_reg(dev, PVRDMA_REG_IMR, ~0);
+}
+
+static int pvrdma_enable_msix(struct pci_dev *pdev, struct pvrdma_dev *dev)
+{
+ int i;
+ int ret;
+
+ for (i = 0; i < PVRDMA_MAX_INTERRUPTS; i++) {
+ dev->intr.msix_entry[i].entry = i;
+ dev->intr.msix_entry[i].vector = i;
+
+ switch (i) {
+ case 0:
+ /* CMD ring handler */
+ dev->intr.handler[i] = pvrdma_intr0_handler;
+ break;
+ case 1:
+ /* Async event ring handler */
+ dev->intr.handler[i] = pvrdma_intr1_handler;
+ break;
+ default:
+ /* Completion queue handler */
+ dev->intr.handler[i] = pvrdma_intrx_handler;
+ break;
+ }
+ }
+
+ ret = pci_enable_msix(pdev, dev->intr.msix_entry,
+ PVRDMA_MAX_INTERRUPTS);
+ if (!ret) {
+ dev->intr.type = PVRDMA_INTR_TYPE_MSIX;
+ dev->intr.size = PVRDMA_MAX_INTERRUPTS;
+ } else if (ret > 0) {
+ ret = pci_enable_msix(pdev, dev->intr.msix_entry, ret);
+ if (!ret) {
+ dev->intr.type = PVRDMA_INTR_TYPE_MSIX;
+ dev->intr.size = ret;
+ } else {
+ dev->intr.size = 0;
+ }
+ }
+
+ dev_dbg(&pdev->dev, "using interrupt type %d, size %d\n",
+ dev->intr.type, dev->intr.size);
+
+ return ret;
+}
+
+static int pvrdma_alloc_intrs(struct pvrdma_dev *dev)
+{
+ int ret = 0;
+ int i;
+
+ if (pci_find_capability(dev->pdev, PCI_CAP_ID_MSIX) &&
+ pvrdma_enable_msix(dev->pdev, dev)) {
+ /* Try MSI */
+ ret = pci_enable_msi(dev->pdev);
+ if (!ret) {
+ dev->intr.type = PVRDMA_INTR_TYPE_MSI;
+ } else {
+ /* Legacy INTR */
+ dev->intr.type = PVRDMA_INTR_TYPE_INTX;
+ }
+ }
+
+ /* Request First IRQ */
+ switch (dev->intr.type) {
+ case PVRDMA_INTR_TYPE_INTX:
+ case PVRDMA_INTR_TYPE_MSI:
+ ret = request_irq(dev->pdev->irq, pvrdma_intr0_handler,
+ IRQF_SHARED, DRV_NAME, dev);
+ if (ret) {
+ dev_err(&dev->pdev->dev,
+ "failed to request interrupt\n");
+ goto disable_msi;
+ }
+ break;
+ case PVRDMA_INTR_TYPE_MSIX:
+ ret = request_irq(dev->intr.msix_entry[0].vector,
+ pvrdma_intr0_handler, 0, DRV_NAME, dev);
+ if (ret) {
+ dev_err(&dev->pdev->dev,
+ "failed to request interrupt 0\n");
+ goto disable_msi;
+ }
+ dev->intr.enabled[0] = 1;
+ break;
+ default:
+ /* Not reached */
+ break;
+ }
+
+ /* For MSIX: request intr for each vector */
+ if (dev->intr.size > 1) {
+ ret = request_irq(dev->intr.msix_entry[1].vector,
+ pvrdma_intr1_handler, 0, DRV_NAME, dev);
+ if (ret) {
+ dev_err(&dev->pdev->dev,
+ "failed to request interrupt 1\n");
+ goto free_irq;
+ }
+ dev->intr.enabled[1] = 1;
+
+ for (i = 2; i < dev->intr.size; i++) {
+ ret = request_irq(dev->intr.msix_entry[i].vector,
+ pvrdma_intrx_handler, 0,
+ DRV_NAME, dev);
+ if (ret) {
+ dev_err(&dev->pdev->dev,
+ "failed to request interrupt %d\n", i);
+ goto free_irq;
+ }
+ dev->intr.enabled[i] = 1;
+ }
+ }
+
+ return 0;
+
+free_irq:
+ pvrdma_free_irq(dev);
+disable_msi:
+ pvrdma_disable_msi_all(dev);
+ return ret;
+}
+
+static void pvrdma_free_slots(struct pvrdma_dev *dev)
+{
+ struct pci_dev *pdev = dev->pdev;
+
+ if (dev->resp_slot)
+ dma_free_coherent(&pdev->dev, PAGE_SIZE, dev->resp_slot,
+ dev->dsr->resp_slot_dma);
+ if (dev->cmd_slot)
+ dma_free_coherent(&pdev->dev, PAGE_SIZE, dev->cmd_slot,
+ dev->dsr->cmd_slot_dma);
+}
+
+static int pvrdma_add_gid_at_index(struct pvrdma_dev *dev,
+ const union ib_gid *gid,
+ int index)
+{
+ int ret;
+ union pvrdma_cmd_req req;
+ struct pvrdma_cmd_create_bind *cmd_bind = &req.create_bind;
+
+ if (!dev->sgid_tbl) {
+ dev_warn(&dev->pdev->dev, "sgid table not initialized\n");
+ return -EINVAL;
+ }
+
+ memset(cmd_bind, 0, sizeof(*cmd_bind));
+ cmd_bind->hdr.cmd = PVRDMA_CMD_CREATE_BIND;
+ memcpy(cmd_bind->new_gid, gid->raw, 16);
+ cmd_bind->mtu = ib_mtu_enum_to_int(IB_MTU_1024);
+ cmd_bind->vlan = 0xfff;
+ cmd_bind->index = index;
+ cmd_bind->gid_type = PVRDMA_GID_TYPE_FLAG_ROCE_V1;
+
+ ret = pvrdma_cmd_post(dev, &req, NULL, 0);
+ if (ret < 0) {
+ dev_warn(&dev->pdev->dev,
+ "could not create binding, error: %d\n", ret);
+ return -EINVAL;
+ }
+ memcpy(&dev->sgid_tbl[index], gid, sizeof(*gid));
+ return 0;
+}
+
+static int pvrdma_add_gid(struct ib_device *ibdev,
+ u8 port_num,
+ unsigned int index,
+ const union ib_gid *gid,
+ const struct ib_gid_attr *attr,
+ void **context)
+{
+ struct pvrdma_dev *dev = to_vdev(ibdev);
+
+ return pvrdma_add_gid_at_index(dev, gid, index);
+}
+
+static int pvrdma_del_gid_at_index(struct pvrdma_dev *dev, int index)
+{
+ int ret;
+ union pvrdma_cmd_req req;
+ struct pvrdma_cmd_destroy_bind *cmd_dest = &req.destroy_bind;
+
+ /* Update sgid table. */
+ if (!dev->sgid_tbl) {
+ dev_warn(&dev->pdev->dev, "sgid table not initialized\n");
+ return -EINVAL;
+ }
+
+ memset(cmd_dest, 0, sizeof(*cmd_dest));
+ cmd_dest->hdr.cmd = PVRDMA_CMD_DESTROY_BIND;
+ memcpy(cmd_dest->dest_gid, &dev->sgid_tbl[index], 16);
+ cmd_dest->index = index;
+
+ ret = pvrdma_cmd_post(dev, &req, NULL, 0);
+ if (ret < 0) {
+ dev_warn(&dev->pdev->dev,
+ "could not destroy binding, error: %d\n", ret);
+ return ret;
+ }
+ memset(&dev->sgid_tbl[index], 0, 16);
+ return 0;
+}
+
+static int pvrdma_del_gid(struct ib_device *ibdev,
+ u8 port_num,
+ unsigned int index,
+ void **context)
+{
+ struct pvrdma_dev *dev = to_vdev(ibdev);
+
+ dev_dbg(&dev->pdev->dev, "removing gid at index %u from %s",
+ index, dev->netdev->name);
+
+ return pvrdma_del_gid_at_index(dev, index);
+}
+
+static void pvrdma_netdevice_event_handle(struct pvrdma_dev *dev,
+ unsigned long event)
+{
+ switch (event) {
+ case NETDEV_REBOOT:
+ case NETDEV_DOWN:
+ pvrdma_dispatch_event(dev, 1, IB_EVENT_PORT_ERR);
+ break;
+ case NETDEV_UP:
+ pvrdma_dispatch_event(dev, 1, IB_EVENT_PORT_ACTIVE);
+ break;
+ default:
+ dev_dbg(&dev->pdev->dev, "ignore netdevice event %ld on %s\n",
+ event, dev->ib_dev.name);
+ break;
+ }
+}
+
+static void pvrdma_netdevice_event_work(struct work_struct *work)
+{
+ struct pvrdma_netdevice_work *netdev_work;
+ struct pvrdma_dev *dev;
+
+ netdev_work = container_of(work, struct pvrdma_netdevice_work, work);
+
+ mutex_lock(&pvrdma_device_list_lock);
+ list_for_each_entry(dev, &pvrdma_device_list, device_link) {
+ if (dev->netdev == netdev_work->event_netdev) {
+ pvrdma_netdevice_event_handle(dev, netdev_work->event);
+ break;
+ }
+ }
+ mutex_unlock(&pvrdma_device_list_lock);
+
+ kfree(netdev_work);
+}
+
+static int pvrdma_netdevice_event(struct notifier_block *this,
+ unsigned long event, void *ptr)
+{
+ struct net_device *event_netdev = netdev_notifier_info_to_dev(ptr);
+ struct pvrdma_netdevice_work *netdev_work;
+
+ netdev_work = kmalloc(sizeof(*netdev_work), GFP_ATOMIC);
+ if (!netdev_work)
+ return NOTIFY_BAD;
+
+ INIT_WORK(&netdev_work->work, pvrdma_netdevice_event_work);
+ netdev_work->event_netdev = event_netdev;
+ netdev_work->event = event;
+ queue_work(event_wq, &netdev_work->work);
+
+ return NOTIFY_DONE;
+}
+
+static int pvrdma_pci_probe(struct pci_dev *pdev,
+ const struct pci_device_id *id)
+{
+ struct pci_dev *pdev_net;
+ struct pvrdma_dev *dev;
+ int ret;
+ unsigned long start;
+ unsigned long len;
+ unsigned int version;
+ dma_addr_t slot_dma = 0;
+
+ dev_dbg(&pdev->dev, "initializing driver %s\n", pci_name(pdev));
+
+ /* Allocate zero-out device */
+ dev = (struct pvrdma_dev *)ib_alloc_device(sizeof(*dev));
+ if (!dev) {
+ dev_err(&pdev->dev, "failed to allocate IB device\n");
+ return -ENOMEM;
+ }
+
+ mutex_lock(&pvrdma_device_list_lock);
+ list_add(&dev->device_link, &pvrdma_device_list);
+ mutex_unlock(&pvrdma_device_list_lock);
+
+ ret = pvrdma_init_device(dev);
+ if (ret)
+ goto err_free_device;
+
+ dev->pdev = pdev;
+ pci_set_drvdata(pdev, dev);
+
+ ret = pci_enable_device(pdev);
+ if (ret) {
+ dev_err(&pdev->dev, "cannot enable PCI device\n");
+ goto err_free_device;
+ }
+
+ /*
+ * DEBUG: Checking PCI: 0x20200; _MEM; len=0x2000
+ */
+ dev_dbg(&pdev->dev, "PCI resource flags BAR0 %#lx\n",
+ pci_resource_flags(pdev, 0));
+ dev_dbg(&pdev->dev, "PCI resource len %#llx\n",
+ (unsigned long long)pci_resource_len(pdev, 0));
+ dev_dbg(&pdev->dev, "PCI resource start %#llx\n",
+ (unsigned long long)pci_resource_start(pdev, 0));
+ dev_dbg(&pdev->dev, "PCI resource flags BAR1 %#lx\n",
+ pci_resource_flags(pdev, 1));
+ dev_dbg(&pdev->dev, "PCI resource len %#llx\n",
+ (unsigned long long)pci_resource_len(pdev, 1));
+ dev_dbg(&pdev->dev, "PCI resource start %#llx\n",
+ (unsigned long long)pci_resource_start(pdev, 1));
+
+ if (!(pci_resource_flags(pdev, 0) & IORESOURCE_MEM) ||
+ !(pci_resource_flags(pdev, 1) & IORESOURCE_MEM)) {
+ dev_err(&pdev->dev, "PCI BAR region not MMIO\n");
+ ret = -ENOMEM;
+ goto err_free_device;
+ }
+
+ ret = pci_request_regions(pdev, DRV_NAME);
+ if (ret) {
+ dev_err(&pdev->dev, "cannot request PCI resources\n");
+ goto err_disable_pdev;
+ }
+
+ /* Enable 64-Bit DMA */
+ if (pci_set_dma_mask(pdev, DMA_BIT_MASK(64)) == 0) {
+ ret = pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(64));
+ if (ret != 0) {
+ dev_err(&pdev->dev,
+ "pci_set_consistent_dma_mask failed\n");
+ goto err_free_resource;
+ }
+ } else {
+ ret = pci_set_dma_mask(pdev, DMA_BIT_MASK(32));
+ if (ret != 0) {
+ dev_err(&pdev->dev,
+ "pci_set_dma_mask failed\n");
+ goto err_free_resource;
+ }
+ }
+
+ pci_set_master(pdev);
+
+ /* Map register space */
+ start = pci_resource_start(dev->pdev, PVRDMA_PCI_RESOURCE_REG);
+ len = pci_resource_len(dev->pdev, PVRDMA_PCI_RESOURCE_REG);
+ dev->regs = ioremap(start, len);
+ if (!dev->regs) {
+ dev_err(&pdev->dev, "register mapping failed\n");
+ ret = -ENOMEM;
+ goto err_free_resource;
+ }
+
+ /* Setup per-device UAR. */
+ dev->driver_uar.index = 0;
+ dev->driver_uar.pfn =
+ pci_resource_start(dev->pdev, PVRDMA_PCI_RESOURCE_UAR) >>
+ PAGE_SHIFT;
+ dev->driver_uar.map =
+ ioremap(dev->driver_uar.pfn << PAGE_SHIFT, PAGE_SIZE);
+ if (!dev->driver_uar.map) {
+ dev_err(&pdev->dev, "failed to remap UAR pages\n");
+ ret = -ENOMEM;
+ goto err_unmap_regs;
+ }
+
+ version = pvrdma_read_reg(dev, PVRDMA_REG_VERSION);
+ dev_info(&pdev->dev, "device version %d, driver version %d\n",
+ version, PVRDMA_VERSION);
+ if (version < PVRDMA_VERSION) {
+ dev_err(&pdev->dev, "incompatible device version\n");
+ goto err_uar_unmap;
+ }
+
+ dev->dsr = dma_alloc_coherent(&pdev->dev, sizeof(*dev->dsr),
+ &dev->dsrbase, GFP_KERNEL);
+ if (!dev->dsr) {
+ dev_err(&pdev->dev, "failed to allocate shared region\n");
+ ret = -ENOMEM;
+ goto err_uar_unmap;
+ }
+
+ /* Setup the shared region */
+ memset(dev->dsr, 0, sizeof(*dev->dsr));
+ dev->dsr->driver_version = PVRDMA_VERSION;
+ dev->dsr->gos_info.gos_bits = sizeof(void *) == 4 ?
+ PVRDMA_GOS_BITS_32 :
+ PVRDMA_GOS_BITS_64;
+ dev->dsr->gos_info.gos_type = PVRDMA_GOS_TYPE_LINUX;
+ dev->dsr->gos_info.gos_ver = 1;
+ dev->dsr->uar_pfn = dev->driver_uar.pfn;
+
+ /* Command slot. */
+ dev->cmd_slot = dma_alloc_coherent(&pdev->dev, PAGE_SIZE,
+ &slot_dma, GFP_KERNEL);
+ if (!dev->cmd_slot) {
+ ret = -ENOMEM;
+ goto err_free_dsr;
+ }
+
+ dev->dsr->cmd_slot_dma = (u64)slot_dma;
+
+ /* Response slot. */
+ dev->resp_slot = dma_alloc_coherent(&pdev->dev, PAGE_SIZE,
+ &slot_dma, GFP_KERNEL);
+ if (!dev->resp_slot) {
+ ret = -ENOMEM;
+ goto err_free_slots;
+ }
+
+ dev->dsr->resp_slot_dma = (u64)slot_dma;
+
+ /* Async event ring */
+ dev->dsr->async_ring_pages.num_pages = 4;
+ ret = pvrdma_page_dir_init(dev, &dev->async_pdir,
+ dev->dsr->async_ring_pages.num_pages, true);
+ if (ret)
+ goto err_free_slots;
+ dev->async_ring_state = dev->async_pdir.pages[0];
+ dev->dsr->async_ring_pages.pdir_dma = dev->async_pdir.dir_dma;
+
+ /* CQ notification ring */
+ dev->dsr->cq_ring_pages.num_pages = 4;
+ ret = pvrdma_page_dir_init(dev, &dev->cq_pdir,
+ dev->dsr->cq_ring_pages.num_pages, true);
+ if (ret)
+ goto err_free_async_ring;
+ dev->cq_ring_state = dev->cq_pdir.pages[0];
+ dev->dsr->cq_ring_pages.pdir_dma = dev->cq_pdir.dir_dma;
+
+ /*
+ * Write the PA of the shared region to the device. The writes must be
+ * ordered such that the high bits are written last. When the writes
+ * complete, the device will have filled out the capabilities.
+ */
+
+ pvrdma_write_reg(dev, PVRDMA_REG_DSRLOW, (u32)dev->dsrbase);
+ pvrdma_write_reg(dev, PVRDMA_REG_DSRHIGH,
+ (u32)((u64)(dev->dsrbase) >> 32));
+
+ /* Make sure the write is complete before reading status. */
+ mb();
+
+ /* Currently, the driver only supports RoCE mode. */
+ if (dev->dsr->caps.mode != PVRDMA_DEVICE_MODE_ROCE) {
+ dev_err(&pdev->dev, "unsupported transport %d\n",
+ dev->dsr->caps.mode);
+ ret = -EINVAL;
+ goto err_free_cq_ring;
+ }
+
+ /* Currently, the driver only supports RoCE V1. */
+ if (!(dev->dsr->caps.gid_types & PVRDMA_GID_TYPE_FLAG_ROCE_V1)) {
+ dev_err(&pdev->dev, "driver needs RoCE v1 support\n");
+ ret = -EINVAL;
+ goto err_free_cq_ring;
+ }
+
+ /* Paired vmxnet3 will have same bus, slot. But func will be 0 */
+ pdev_net = pci_get_slot(pdev->bus, PCI_DEVFN(PCI_SLOT(pdev->devfn), 0));
+ if (!pdev_net) {
+ dev_err(&pdev->dev, "failed to find paired net device\n");
+ ret = -ENODEV;
+ goto err_free_cq_ring;
+ }
+
+ if (pdev_net->vendor != PCI_VENDOR_ID_VMWARE ||
+ pdev_net->device != PCI_DEVICE_ID_VMWARE_VMXNET3) {
+ dev_err(&pdev->dev, "failed to find paired vmxnet3 device\n");
+ pci_dev_put(pdev_net);
+ ret = -ENODEV;
+ goto err_free_cq_ring;
+ }
+
+ dev->netdev = pci_get_drvdata(pdev_net);
+ pci_dev_put(pdev_net);
+ if (!dev->netdev) {
+ dev_err(&pdev->dev, "failed to get vmxnet3 device\n");
+ ret = -ENODEV;
+ goto err_free_cq_ring;
+ }
+
+ dev_info(&pdev->dev, "paired device to %s\n", dev->netdev->name);
+
+ /* Interrupt setup */
+ ret = pvrdma_alloc_intrs(dev);
+ if (ret) {
+ dev_err(&pdev->dev, "failed to allocate interrupts\n");
+ ret = -ENOMEM;
+ goto err_netdevice;
+ }
+
+ /* Allocate UAR table. */
+ ret = pvrdma_uar_table_init(dev);
+ if (ret) {
+ dev_err(&pdev->dev, "failed to allocate UAR table\n");
+ ret = -ENOMEM;
+ goto err_free_intrs;
+ }
+
+ /* Allocate GID table */
+ dev->sgid_tbl = kcalloc(dev->dsr->caps.gid_tbl_len,
+ sizeof(union ib_gid), GFP_KERNEL);
+ if (!dev->sgid_tbl) {
+ ret = -ENOMEM;
+ goto err_free_uar_table;
+ }
+ dev_dbg(&pdev->dev, "gid table len %d\n", dev->dsr->caps.gid_tbl_len);
+
+ pvrdma_enable_intrs(dev);
+
+ /* Activate pvrdma device */
+ pvrdma_write_reg(dev, PVRDMA_REG_CTL, PVRDMA_DEVICE_CTL_ACTIVATE);
+
+ /* Make sure the write is complete before reading status. */
+ mb();
+
+ /* Check if device was successfully activated */
+ ret = pvrdma_read_reg(dev, PVRDMA_REG_ERR);
+ if (ret != 0) {
+ dev_err(&pdev->dev, "failed to activate device\n");
+ ret = -EINVAL;
+ goto err_disable_intr;
+ }
+
+ /* Register IB device */
+ ret = pvrdma_register_device(dev);
+ if (ret) {
+ dev_err(&pdev->dev, "failed to register IB device\n");
+ goto err_disable_intr;
+ }
+
+ dev->nb_netdev.notifier_call = pvrdma_netdevice_event;
+ ret = register_netdevice_notifier(&dev->nb_netdev);
+ if (ret) {
+ dev_err(&pdev->dev, "failed to register netdevice events\n");
+ goto err_unreg_ibdev;
+ }
+
+ dev_info(&pdev->dev, "attached to device\n");
+ return 0;
+
+err_unreg_ibdev:
+ ib_unregister_device(&dev->ib_dev);
+err_disable_intr:
+ pvrdma_disable_intrs(dev);
+ kfree(dev->sgid_tbl);
+err_free_uar_table:
+ pvrdma_uar_table_cleanup(dev);
+err_free_intrs:
+ pvrdma_free_irq(dev);
+ pvrdma_disable_msi_all(dev);
+err_netdevice:
+ unregister_netdevice_notifier(&dev->nb_netdev);
+err_free_cq_ring:
+ pvrdma_page_dir_cleanup(dev, &dev->cq_pdir);
+err_free_async_ring:
+ pvrdma_page_dir_cleanup(dev, &dev->async_pdir);
+err_free_slots:
+ pvrdma_free_slots(dev);
+err_free_dsr:
+ dma_free_coherent(&pdev->dev, sizeof(*dev->dsr), dev->dsr,
+ dev->dsrbase);
+err_uar_unmap:
+ iounmap(dev->driver_uar.map);
+err_unmap_regs:
+ iounmap(dev->regs);
+err_free_resource:
+ pci_release_regions(pdev);
+err_disable_pdev:
+ pci_disable_device(pdev);
+ pci_set_drvdata(pdev, NULL);
+err_free_device:
+ mutex_lock(&pvrdma_device_list_lock);
+ list_del(&dev->device_link);
+ mutex_unlock(&pvrdma_device_list_lock);
+ ib_dealloc_device(&dev->ib_dev);
+ return ret;
+}
+
+static void pvrdma_pci_remove(struct pci_dev *pdev)
+{
+ struct pvrdma_dev *dev = pci_get_drvdata(pdev);
+
+ if (!dev)
+ return;
+
+ dev_info(&pdev->dev, "detaching from device\n");
+
+ unregister_netdevice_notifier(&dev->nb_netdev);
+ dev->nb_netdev.notifier_call = NULL;
+
+ flush_workqueue(event_wq);
+
+ /* Unregister ib device */
+ ib_unregister_device(&dev->ib_dev);
+
+ mutex_lock(&pvrdma_device_list_lock);
+ list_del(&dev->device_link);
+ mutex_unlock(&pvrdma_device_list_lock);
+
+ pvrdma_disable_intrs(dev);
+ pvrdma_free_irq(dev);
+ pvrdma_disable_msi_all(dev);
+
+ /* Deactivate pvrdma device */
+ pvrdma_write_reg(dev, PVRDMA_REG_CTL, PVRDMA_DEVICE_CTL_RESET);
+ pvrdma_page_dir_cleanup(dev, &dev->cq_pdir);
+ pvrdma_page_dir_cleanup(dev, &dev->async_pdir);
+ pvrdma_free_slots(dev);
+
+ iounmap(dev->regs);
+ kfree(dev->sgid_tbl);
+ kfree(dev->cq_tbl);
+ kfree(dev->qp_tbl);
+ pvrdma_uar_table_cleanup(dev);
+ iounmap(dev->driver_uar.map);
+
+ ib_dealloc_device(&dev->ib_dev);
+
+ /* Free pci resources */
+ pci_release_regions(pdev);
+ pci_disable_device(pdev);
+ pci_set_drvdata(pdev, NULL);
+}
+
+static struct pci_device_id pvrdma_pci_table[] = {
+ { PCI_DEVICE(PCI_VENDOR_ID_VMWARE, PCI_DEVICE_ID_VMWARE_PVRDMA), },
+ { 0 },
+};
+
+MODULE_DEVICE_TABLE(pci, pvrdma_pci_table);
+
+static struct pci_driver pvrdma_driver = {
+ .name = DRV_NAME,
+ .id_table = pvrdma_pci_table,
+ .probe = pvrdma_pci_probe,
+ .remove = pvrdma_pci_remove,
+};
+
+static int __init pvrdma_init(void)
+{
+ int err;
+
+ event_wq = alloc_ordered_workqueue("pvrdma_event_wq", WQ_MEM_RECLAIM);
+ if (!event_wq)
+ return -ENOMEM;
+
+ err = pci_register_driver(&pvrdma_driver);
+ if (err)
+ destroy_workqueue(event_wq);
+
+ return err;
+}
+
+static void __exit pvrdma_cleanup(void)
+{
+ pci_unregister_driver(&pvrdma_driver);
+
+ destroy_workqueue(event_wq);
+}
+
+module_init(pvrdma_init);
+module_exit(pvrdma_cleanup);
+
+MODULE_AUTHOR("VMware, Inc");
+MODULE_DESCRIPTION("VMware Virtual RDMA/InfiniBand driver");
+MODULE_VERSION(DRV_VERSION);
+MODULE_LICENSE("Dual BSD/GPL");
--
2.7.4
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox