* SDP: test bug 2027 and new error
From: Andrea Gozzelino @ 2010-04-28 14:00 UTC (permalink / raw)
To: Amir Vadai
Cc: swise-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW@public.gmane.org,
linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
In-Reply-To: <4BD6C9C3.6070905-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
On Apr 27, 2010 01:25 PM, Amir Vadai <amirv-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org> wrote:
> Andrea Hi,
>
> I am sorry - the commit missed the 24.4.2010 build which I asked you
> to
> test.
> Please test this build (the latest): OFED-1.5.2-20100426-0600
>
> - Amir
>
>
> On 04/27/2010 01:34 PM, Andrea Gozzelino wrote:
> > Hi Amir,
> > I have checked the bug solution with OFED-1.5.2-20100425-0600
> > version
> > without success.
> >
> > TEST LOG:
> > client machine
> > export LD_PRELOAD=/usr/lib64/libsdp.so
> > netperf -H address -c -C -l 60 -- -m 65536
> > tail -f /var/log/libsdp.log
> >
> > OUTPUT
> > Tue Apr 27 10:08:38 2010 netperf[19051] libsdp Error connect: failed
> > for
> > SDP
> > fd:6 with error:Cannot allocate memory
> > Tue Apr 27 11:47:55 2010 netperf[19081] libsdp Error connect: failed
> > for
> > SDP
> > fd:6 with error:Cannot allocate memory
> >
> > dmesg
> >
> > OUTPUT
> > client_script[13876]: segfault at 00007fffef8de000 rip
> > 000000387d07ad8a
> > rsp
> > 00007fffef8d5648 error 6
> > C_copiato[20437]: segfault at 00007ffff0257b3c rip 00000000004009e3
> > rsp
> > 00007ffff0257b30 error 6
> > client_20[1539]: segfault at 00007fffa2acd85c rip 0000000000400a80
> > rsp
> > 00007fffa2acd850 error 6
> > sdp_init_qp:95 sdp_sock( 2100:2 55190:0): recv sge's. capability: 4
> > needed: 9
> > sdp_init_qp:95 sdp_sock( 2100:2 41003:0): recv sge's. capability: 4
> > needed: 9
> > sdp_init_qp:95 sdp_sock( 2100:2 55192:0): recv sge's. capability: 4
> > needed: 9
> > sdp_init_qp:95 sdp_sock( 2100:2 38542:0): recv sge's. capability: 4
> > needed: 9
> >
> > ------------------
> >
> > server machine
> > export LD_PRELOAD=/usr/lib64/libsdp.so
> > netserver
> > Starting netserver at port 12865
> > Starting netserver at hostname 0.0.0.0 port 12865 and family
> > AF_UNSPEC
> >
> > ------------------
> > Amir, I think that bug 2028 is also involved in NE020 SDP working.
> > Steve, is it possible?
> > If yes, I will check the bug 2027 and 2028 solutions in July 2010.
> > Is it correct?
> >
> > I reopen the bug 2027 because I don't see any changes.
> > Please consider that the bug 2027 could be connected with the 2028
> > one
> > in Bugzilla.
> >
> > Keep in touch.
> > Thank you very much.
> > Regards,
> > Andrea
> >
> >
> >
> >
> > Andrea Gozzelino
> >
> > INFN - Laboratori Nazionali di Legnaro (LNL)
> > Viale dell'Universita' 2
> > I-35020 - Legnaro (PD)- ITALIA
> > Tel: +39 049 8068346
> > Fax: +39 049 641925
> > Mail: andrea.gozzelino-PK20h7lG/Rc1GQ1Ptb7lUw@public.gmane.org
> >
>
Hi Amir,
I have tested your solution of bug 2027 with OFED-1.5.2-20100426-0600.
Netserver/netperf does not work with Socket Direct Protocol SDP.
Now the error in the log file is:
libsdp error connect: failed for SDP fd with error No such device
sdp_init_qp:105 SDP not available on device nes0
I have a look in sdp_cma.c file: I see the message text near the
previous one about # SGE capability.
Amir, could you please comment these error?
Probably I can close the bug 2027 because the solution is OK and I will
open another one about "SDP not available on device nes0".
Steve, is there a connession with the Fast Memory Region (bug 2028)?
At the end, #SGE capability is now OK but SDP on NE020 cards - driver
iw_nes does not work yet properly.
I will wait your suggestions.
Regards,
Andrea
Andrea Gozzelino
INFN - Laboratori Nazionali di Legnaro (LNL)
Viale dell'Universita' 2
I-35020 - Legnaro (PD)- ITALIA
Tel: +39 049 8068346
Fax: +39 049 641925
Mail: andrea.gozzelino-PK20h7lG/Rc1GQ1Ptb7lUw@public.gmane.org
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: SDP: test bug 2027 and new error
From: Amir Vadai @ 2010-04-28 14:15 UTC (permalink / raw)
To: Andrea Gozzelino
Cc: swise-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW@public.gmane.org,
linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
In-Reply-To: <2307255.1272463222662.SLOX.WebMail.wwwrun-XDIR3SKYeFbgKi2NxijLtw@public.gmane.org>
Hi,
I think this is because sdp_add_device() require FMR to be supported.
I need to put a fix to disable ZCopy and not create FMR pool if doesn't
have support - I thought it would be part of BZ2027 fix.
As a workaround - you could comment out all the calls to
sdp_dev->fmr_pool. and make sure ZCopy is disabled by setting 0 in
sdp_zcopy_thresh.
- Amir
On 04/28/2010 05:00 PM, Andrea Gozzelino wrote:
> On Apr 27, 2010 01:25 PM, Amir Vadai <amirv-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org> wrote:
>
>
>> Andrea Hi,
>>
>> I am sorry - the commit missed the 24.4.2010 build which I asked you
>> to
>> test.
>> Please test this build (the latest): OFED-1.5.2-20100426-0600
>>
>> - Amir
>>
>>
>> On 04/27/2010 01:34 PM, Andrea Gozzelino wrote:
>>
>>> Hi Amir,
>>> I have checked the bug solution with OFED-1.5.2-20100425-0600
>>> version
>>> without success.
>>>
>>> TEST LOG:
>>> client machine
>>> export LD_PRELOAD=/usr/lib64/libsdp.so
>>> netperf -H address -c -C -l 60 -- -m 65536
>>> tail -f /var/log/libsdp.log
>>>
>>> OUTPUT
>>> Tue Apr 27 10:08:38 2010 netperf[19051] libsdp Error connect: failed
>>> for
>>> SDP
>>> fd:6 with error:Cannot allocate memory
>>> Tue Apr 27 11:47:55 2010 netperf[19081] libsdp Error connect: failed
>>> for
>>> SDP
>>> fd:6 with error:Cannot allocate memory
>>>
>>> dmesg
>>>
>>> OUTPUT
>>> client_script[13876]: segfault at 00007fffef8de000 rip
>>> 000000387d07ad8a
>>> rsp
>>> 00007fffef8d5648 error 6
>>> C_copiato[20437]: segfault at 00007ffff0257b3c rip 00000000004009e3
>>> rsp
>>> 00007ffff0257b30 error 6
>>> client_20[1539]: segfault at 00007fffa2acd85c rip 0000000000400a80
>>> rsp
>>> 00007fffa2acd850 error 6
>>> sdp_init_qp:95 sdp_sock( 2100:2 55190:0): recv sge's. capability: 4
>>> needed: 9
>>> sdp_init_qp:95 sdp_sock( 2100:2 41003:0): recv sge's. capability: 4
>>> needed: 9
>>> sdp_init_qp:95 sdp_sock( 2100:2 55192:0): recv sge's. capability: 4
>>> needed: 9
>>> sdp_init_qp:95 sdp_sock( 2100:2 38542:0): recv sge's. capability: 4
>>> needed: 9
>>>
>>> ------------------
>>>
>>> server machine
>>> export LD_PRELOAD=/usr/lib64/libsdp.so
>>> netserver
>>> Starting netserver at port 12865
>>> Starting netserver at hostname 0.0.0.0 port 12865 and family
>>> AF_UNSPEC
>>>
>>> ------------------
>>> Amir, I think that bug 2028 is also involved in NE020 SDP working.
>>> Steve, is it possible?
>>> If yes, I will check the bug 2027 and 2028 solutions in July 2010.
>>> Is it correct?
>>>
>>> I reopen the bug 2027 because I don't see any changes.
>>> Please consider that the bug 2027 could be connected with the 2028
>>> one
>>> in Bugzilla.
>>>
>>> Keep in touch.
>>> Thank you very much.
>>> Regards,
>>> Andrea
>>>
>>>
>>>
>>>
>>> Andrea Gozzelino
>>>
>>> INFN - Laboratori Nazionali di Legnaro (LNL)
>>> Viale dell'Universita' 2
>>> I-35020 - Legnaro (PD)- ITALIA
>>> Tel: +39 049 8068346
>>> Fax: +39 049 641925
>>> Mail: andrea.gozzelino-PK20h7lG/Rc1GQ1Ptb7lUw@public.gmane.org
>>>
>>>
>>
> Hi Amir,
>
> I have tested your solution of bug 2027 with OFED-1.5.2-20100426-0600.
> Netserver/netperf does not work with Socket Direct Protocol SDP.
>
> Now the error in the log file is:
> libsdp error connect: failed for SDP fd with error No such device
> sdp_init_qp:105 SDP not available on device nes0
>
> I have a look in sdp_cma.c file: I see the message text near the
> previous one about # SGE capability.
>
> Amir, could you please comment these error?
>
> Probably I can close the bug 2027 because the solution is OK and I will
> open another one about "SDP not available on device nes0".
>
> Steve, is there a connession with the Fast Memory Region (bug 2028)?
>
> At the end, #SGE capability is now OK but SDP on NE020 cards - driver
> iw_nes does not work yet properly.
> I will wait your suggestions.
>
> Regards,
> Andrea
>
>
> Andrea Gozzelino
>
> INFN - Laboratori Nazionali di Legnaro (LNL)
> Viale dell'Universita' 2
> I-35020 - Legnaro (PD)- ITALIA
> Tel: +39 049 8068346
> Fax: +39 049 641925
> Mail: andrea.gozzelino-PK20h7lG/Rc1GQ1Ptb7lUw@public.gmane.org
>
>
>
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* [PATCH] RDMA/cxgb3: Shrink .text with compile-time init of handlers arrays
From: Roland Dreier @ 2010-04-28 19:10 UTC (permalink / raw)
To: linux-rdma-u79uwXL29TY76Z2rM5mHXA, Steve Wise
Hey Steve,
What do you think about this for the next merge window? I think it
arguably makes the source cleaner, and it definitely makes the compiled
code better:
add/remove: 0/1 grow/shrink: 4/3 up/down: 4/-1682 (-1678)
function old new delta
tx_ack 167 168 +1
state_set 55 56 +1
start_ep_timer 99 100 +1
pass_establish 177 178 +1
act_open_req_arp_failure 39 38 -1
sched 84 82 -2
iwch_cm_init 442 91 -351
work_handlers 1328 - -1328
---
drivers/infiniband/hw/cxgb3/iwch.c | 2 -
drivers/infiniband/hw/cxgb3/iwch_cm.c | 131 +++++++++++++++++----------------
2 files changed, 67 insertions(+), 66 deletions(-)
diff --git a/drivers/infiniband/hw/cxgb3/iwch.c b/drivers/infiniband/hw/cxgb3/iwch.c
index 63f975f..8e77dc5 100644
--- a/drivers/infiniband/hw/cxgb3/iwch.c
+++ b/drivers/infiniband/hw/cxgb3/iwch.c
@@ -47,8 +47,6 @@ MODULE_DESCRIPTION("Chelsio T3 RDMA Driver");
MODULE_LICENSE("Dual BSD/GPL");
MODULE_VERSION(DRV_VERSION);
-cxgb3_cpl_handler_func t3c_handlers[NUM_CPL_CMDS];
-
static void open_rnic_dev(struct t3cdev *);
static void close_rnic_dev(struct t3cdev *);
static void iwch_event_handler(struct t3cdev *, u32, u32);
diff --git a/drivers/infiniband/hw/cxgb3/iwch_cm.c b/drivers/infiniband/hw/cxgb3/iwch_cm.c
index cfd6db0..f2d2c29 100644
--- a/drivers/infiniband/hw/cxgb3/iwch_cm.c
+++ b/drivers/infiniband/hw/cxgb3/iwch_cm.c
@@ -102,12 +102,9 @@ static unsigned int cong_flavor = 1;
module_param(cong_flavor, uint, 0644);
MODULE_PARM_DESC(cong_flavor, "TCP Congestion control flavor (default=1)");
-static void process_work(struct work_struct *work);
static struct workqueue_struct *workq;
-static DECLARE_WORK(skb_work, process_work);
static struct sk_buff_head rxq;
-static cxgb3_cpl_handler_func work_handlers[NUM_CPL_CMDS];
static struct sk_buff *get_skb(struct sk_buff *skb, int len, gfp_t gfp);
static void ep_timeout(unsigned long arg);
@@ -302,27 +299,6 @@ static void release_ep_resources(struct iwch_ep *ep)
put_ep(&ep->com);
}
-static void process_work(struct work_struct *work)
-{
- struct sk_buff *skb = NULL;
- void *ep;
- struct t3cdev *tdev;
- int ret;
-
- while ((skb = skb_dequeue(&rxq))) {
- ep = *((void **) (skb->cb));
- tdev = *((struct t3cdev **) (skb->cb + sizeof(void *)));
- ret = work_handlers[G_OPCODE(ntohl((__force __be32)skb->csum))](tdev, skb, ep);
- if (ret & CPL_RET_BUF_DONE)
- kfree_skb(skb);
-
- /*
- * ep was referenced in sched(), and is freed here.
- */
- put_ep((struct iwch_ep_common *)ep);
- }
-}
-
static int status2errno(int status)
{
switch (status) {
@@ -2158,6 +2134,50 @@ int iwch_ep_redirect(void *ctx, struct dst_entry *old, struct dst_entry *new,
/*
* All the CM events are handled on a work queue to have a safe context.
*/
+/*
+ * These are the real handlers that are called from a work queue.
+ */
+static const cxgb3_cpl_handler_func work_handlers[NUM_CPL_CMDS] = {
+ [CPL_ACT_ESTABLISH] = act_establish,
+ [CPL_ACT_OPEN_RPL] = act_open_rpl,
+ [CPL_RX_DATA] = rx_data,
+ [CPL_TX_DMA_ACK] = tx_ack,
+ [CPL_ABORT_RPL_RSS] = abort_rpl,
+ [CPL_ABORT_RPL] = abort_rpl,
+ [CPL_PASS_OPEN_RPL] = pass_open_rpl,
+ [CPL_CLOSE_LISTSRV_RPL] = close_listsrv_rpl,
+ [CPL_PASS_ACCEPT_REQ] = pass_accept_req,
+ [CPL_PASS_ESTABLISH] = pass_establish,
+ [CPL_PEER_CLOSE] = peer_close,
+ [CPL_ABORT_REQ_RSS] = peer_abort,
+ [CPL_CLOSE_CON_RPL] = close_con_rpl,
+ [CPL_RDMA_TERMINATE] = terminate,
+ [CPL_RDMA_EC_STATUS] = ec_status,
+};
+
+static void process_work(struct work_struct *work)
+{
+ struct sk_buff *skb = NULL;
+ void *ep;
+ struct t3cdev *tdev;
+ int ret;
+
+ while ((skb = skb_dequeue(&rxq))) {
+ ep = *((void **) (skb->cb));
+ tdev = *((struct t3cdev **) (skb->cb + sizeof(void *)));
+ ret = work_handlers[G_OPCODE(ntohl((__force __be32)skb->csum))](tdev, skb, ep);
+ if (ret & CPL_RET_BUF_DONE)
+ kfree_skb(skb);
+
+ /*
+ * ep was referenced in sched(), and is freed here.
+ */
+ put_ep((struct iwch_ep_common *)ep);
+ }
+}
+
+static DECLARE_WORK(skb_work, process_work);
+
static int sched(struct t3cdev *tdev, struct sk_buff *skb, void *ctx)
{
struct iwch_ep_common *epc = ctx;
@@ -2189,6 +2209,29 @@ static int set_tcb_rpl(struct t3cdev *tdev, struct sk_buff *skb, void *ctx)
return CPL_RET_BUF_DONE;
}
+/*
+ * All upcalls from the T3 Core go to sched() to schedule the
+ * processing on a work queue.
+ */
+cxgb3_cpl_handler_func t3c_handlers[NUM_CPL_CMDS] = {
+ [CPL_ACT_ESTABLISH] = sched,
+ [CPL_ACT_OPEN_RPL] = sched,
+ [CPL_RX_DATA] = sched,
+ [CPL_TX_DMA_ACK] = sched,
+ [CPL_ABORT_RPL_RSS] = sched,
+ [CPL_ABORT_RPL] = sched,
+ [CPL_PASS_OPEN_RPL] = sched,
+ [CPL_CLOSE_LISTSRV_RPL] = sched,
+ [CPL_PASS_ACCEPT_REQ] = sched,
+ [CPL_PASS_ESTABLISH] = sched,
+ [CPL_PEER_CLOSE] = sched,
+ [CPL_CLOSE_CON_RPL] = sched,
+ [CPL_ABORT_REQ_RSS] = sched,
+ [CPL_RDMA_TERMINATE] = sched,
+ [CPL_RDMA_EC_STATUS] = sched,
+ [CPL_SET_TCB_RPL] = set_tcb_rpl,
+};
+
int __init iwch_cm_init(void)
{
skb_queue_head_init(&rxq);
@@ -2197,46 +2240,6 @@ int __init iwch_cm_init(void)
if (!workq)
return -ENOMEM;
- /*
- * All upcalls from the T3 Core go to sched() to
- * schedule the processing on a work queue.
- */
- t3c_handlers[CPL_ACT_ESTABLISH] = sched;
- t3c_handlers[CPL_ACT_OPEN_RPL] = sched;
- t3c_handlers[CPL_RX_DATA] = sched;
- t3c_handlers[CPL_TX_DMA_ACK] = sched;
- t3c_handlers[CPL_ABORT_RPL_RSS] = sched;
- t3c_handlers[CPL_ABORT_RPL] = sched;
- t3c_handlers[CPL_PASS_OPEN_RPL] = sched;
- t3c_handlers[CPL_CLOSE_LISTSRV_RPL] = sched;
- t3c_handlers[CPL_PASS_ACCEPT_REQ] = sched;
- t3c_handlers[CPL_PASS_ESTABLISH] = sched;
- t3c_handlers[CPL_PEER_CLOSE] = sched;
- t3c_handlers[CPL_CLOSE_CON_RPL] = sched;
- t3c_handlers[CPL_ABORT_REQ_RSS] = sched;
- t3c_handlers[CPL_RDMA_TERMINATE] = sched;
- t3c_handlers[CPL_RDMA_EC_STATUS] = sched;
- t3c_handlers[CPL_SET_TCB_RPL] = set_tcb_rpl;
-
- /*
- * These are the real handlers that are called from a
- * work queue.
- */
- work_handlers[CPL_ACT_ESTABLISH] = act_establish;
- work_handlers[CPL_ACT_OPEN_RPL] = act_open_rpl;
- work_handlers[CPL_RX_DATA] = rx_data;
- work_handlers[CPL_TX_DMA_ACK] = tx_ack;
- work_handlers[CPL_ABORT_RPL_RSS] = abort_rpl;
- work_handlers[CPL_ABORT_RPL] = abort_rpl;
- work_handlers[CPL_PASS_OPEN_RPL] = pass_open_rpl;
- work_handlers[CPL_CLOSE_LISTSRV_RPL] = close_listsrv_rpl;
- work_handlers[CPL_PASS_ACCEPT_REQ] = pass_accept_req;
- work_handlers[CPL_PASS_ESTABLISH] = pass_establish;
- work_handlers[CPL_PEER_CLOSE] = peer_close;
- work_handlers[CPL_ABORT_REQ_RSS] = peer_abort;
- work_handlers[CPL_CLOSE_CON_RPL] = close_con_rpl;
- work_handlers[CPL_RDMA_TERMINATE] = terminate;
- work_handlers[CPL_RDMA_EC_STATUS] = ec_status;
return 0;
}
--
Roland Dreier <rolandd-FYB4Gu1CFyUAvxtiuMwx3w@public.gmane.org> || For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/index.html
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related
* Re: [PATCH] RDMA/cxgb3: Shrink .text with compile-time init of handlers arrays
From: Steve Wise @ 2010-04-28 19:33 UTC (permalink / raw)
To: Roland Dreier; +Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA
In-Reply-To: <adaiq7b1jhe.fsf-BjVyx320WGW9gfZ95n9DRSW4+XlvGpQz@public.gmane.org>
Looks ok to me. Functionally it doesn't change anything I guess.
Roland Dreier wrote:
> Hey Steve,
>
> What do you think about this for the next merge window? I think it
> arguably makes the source cleaner, and it definitely makes the compiled
> code better:
>
> add/remove: 0/1 grow/shrink: 4/3 up/down: 4/-1682 (-1678)
> function old new delta
> tx_ack 167 168 +1
> state_set 55 56 +1
> start_ep_timer 99 100 +1
> pass_establish 177 178 +1
> act_open_req_arp_failure 39 38 -1
> sched 84 82 -2
> iwch_cm_init 442 91 -351
> work_handlers 1328 - -1328
> ---
> drivers/infiniband/hw/cxgb3/iwch.c | 2 -
> drivers/infiniband/hw/cxgb3/iwch_cm.c | 131 +++++++++++++++++----------------
> 2 files changed, 67 insertions(+), 66 deletions(-)
>
> diff --git a/drivers/infiniband/hw/cxgb3/iwch.c b/drivers/infiniband/hw/cxgb3/iwch.c
> index 63f975f..8e77dc5 100644
> --- a/drivers/infiniband/hw/cxgb3/iwch.c
> +++ b/drivers/infiniband/hw/cxgb3/iwch.c
> @@ -47,8 +47,6 @@ MODULE_DESCRIPTION("Chelsio T3 RDMA Driver");
> MODULE_LICENSE("Dual BSD/GPL");
> MODULE_VERSION(DRV_VERSION);
>
> -cxgb3_cpl_handler_func t3c_handlers[NUM_CPL_CMDS];
> -
> static void open_rnic_dev(struct t3cdev *);
> static void close_rnic_dev(struct t3cdev *);
> static void iwch_event_handler(struct t3cdev *, u32, u32);
> diff --git a/drivers/infiniband/hw/cxgb3/iwch_cm.c b/drivers/infiniband/hw/cxgb3/iwch_cm.c
> index cfd6db0..f2d2c29 100644
> --- a/drivers/infiniband/hw/cxgb3/iwch_cm.c
> +++ b/drivers/infiniband/hw/cxgb3/iwch_cm.c
> @@ -102,12 +102,9 @@ static unsigned int cong_flavor = 1;
> module_param(cong_flavor, uint, 0644);
> MODULE_PARM_DESC(cong_flavor, "TCP Congestion control flavor (default=1)");
>
> -static void process_work(struct work_struct *work);
> static struct workqueue_struct *workq;
> -static DECLARE_WORK(skb_work, process_work);
>
> static struct sk_buff_head rxq;
> -static cxgb3_cpl_handler_func work_handlers[NUM_CPL_CMDS];
>
> static struct sk_buff *get_skb(struct sk_buff *skb, int len, gfp_t gfp);
> static void ep_timeout(unsigned long arg);
> @@ -302,27 +299,6 @@ static void release_ep_resources(struct iwch_ep *ep)
> put_ep(&ep->com);
> }
>
> -static void process_work(struct work_struct *work)
> -{
> - struct sk_buff *skb = NULL;
> - void *ep;
> - struct t3cdev *tdev;
> - int ret;
> -
> - while ((skb = skb_dequeue(&rxq))) {
> - ep = *((void **) (skb->cb));
> - tdev = *((struct t3cdev **) (skb->cb + sizeof(void *)));
> - ret = work_handlers[G_OPCODE(ntohl((__force __be32)skb->csum))](tdev, skb, ep);
> - if (ret & CPL_RET_BUF_DONE)
> - kfree_skb(skb);
> -
> - /*
> - * ep was referenced in sched(), and is freed here.
> - */
> - put_ep((struct iwch_ep_common *)ep);
> - }
> -}
> -
> static int status2errno(int status)
> {
> switch (status) {
> @@ -2158,6 +2134,50 @@ int iwch_ep_redirect(void *ctx, struct dst_entry *old, struct dst_entry *new,
> /*
> * All the CM events are handled on a work queue to have a safe context.
> */
> +/*
> + * These are the real handlers that are called from a work queue.
> + */
> +static const cxgb3_cpl_handler_func work_handlers[NUM_CPL_CMDS] = {
> + [CPL_ACT_ESTABLISH] = act_establish,
> + [CPL_ACT_OPEN_RPL] = act_open_rpl,
> + [CPL_RX_DATA] = rx_data,
> + [CPL_TX_DMA_ACK] = tx_ack,
> + [CPL_ABORT_RPL_RSS] = abort_rpl,
> + [CPL_ABORT_RPL] = abort_rpl,
> + [CPL_PASS_OPEN_RPL] = pass_open_rpl,
> + [CPL_CLOSE_LISTSRV_RPL] = close_listsrv_rpl,
> + [CPL_PASS_ACCEPT_REQ] = pass_accept_req,
> + [CPL_PASS_ESTABLISH] = pass_establish,
> + [CPL_PEER_CLOSE] = peer_close,
> + [CPL_ABORT_REQ_RSS] = peer_abort,
> + [CPL_CLOSE_CON_RPL] = close_con_rpl,
> + [CPL_RDMA_TERMINATE] = terminate,
> + [CPL_RDMA_EC_STATUS] = ec_status,
> +};
> +
> +static void process_work(struct work_struct *work)
> +{
> + struct sk_buff *skb = NULL;
> + void *ep;
> + struct t3cdev *tdev;
> + int ret;
> +
> + while ((skb = skb_dequeue(&rxq))) {
> + ep = *((void **) (skb->cb));
> + tdev = *((struct t3cdev **) (skb->cb + sizeof(void *)));
> + ret = work_handlers[G_OPCODE(ntohl((__force __be32)skb->csum))](tdev, skb, ep);
> + if (ret & CPL_RET_BUF_DONE)
> + kfree_skb(skb);
> +
> + /*
> + * ep was referenced in sched(), and is freed here.
> + */
> + put_ep((struct iwch_ep_common *)ep);
> + }
> +}
> +
> +static DECLARE_WORK(skb_work, process_work);
> +
> static int sched(struct t3cdev *tdev, struct sk_buff *skb, void *ctx)
> {
> struct iwch_ep_common *epc = ctx;
> @@ -2189,6 +2209,29 @@ static int set_tcb_rpl(struct t3cdev *tdev, struct sk_buff *skb, void *ctx)
> return CPL_RET_BUF_DONE;
> }
>
> +/*
> + * All upcalls from the T3 Core go to sched() to schedule the
> + * processing on a work queue.
> + */
> +cxgb3_cpl_handler_func t3c_handlers[NUM_CPL_CMDS] = {
> + [CPL_ACT_ESTABLISH] = sched,
> + [CPL_ACT_OPEN_RPL] = sched,
> + [CPL_RX_DATA] = sched,
> + [CPL_TX_DMA_ACK] = sched,
> + [CPL_ABORT_RPL_RSS] = sched,
> + [CPL_ABORT_RPL] = sched,
> + [CPL_PASS_OPEN_RPL] = sched,
> + [CPL_CLOSE_LISTSRV_RPL] = sched,
> + [CPL_PASS_ACCEPT_REQ] = sched,
> + [CPL_PASS_ESTABLISH] = sched,
> + [CPL_PEER_CLOSE] = sched,
> + [CPL_CLOSE_CON_RPL] = sched,
> + [CPL_ABORT_REQ_RSS] = sched,
> + [CPL_RDMA_TERMINATE] = sched,
> + [CPL_RDMA_EC_STATUS] = sched,
> + [CPL_SET_TCB_RPL] = set_tcb_rpl,
> +};
> +
> int __init iwch_cm_init(void)
> {
> skb_queue_head_init(&rxq);
> @@ -2197,46 +2240,6 @@ int __init iwch_cm_init(void)
> if (!workq)
> return -ENOMEM;
>
> - /*
> - * All upcalls from the T3 Core go to sched() to
> - * schedule the processing on a work queue.
> - */
> - t3c_handlers[CPL_ACT_ESTABLISH] = sched;
> - t3c_handlers[CPL_ACT_OPEN_RPL] = sched;
> - t3c_handlers[CPL_RX_DATA] = sched;
> - t3c_handlers[CPL_TX_DMA_ACK] = sched;
> - t3c_handlers[CPL_ABORT_RPL_RSS] = sched;
> - t3c_handlers[CPL_ABORT_RPL] = sched;
> - t3c_handlers[CPL_PASS_OPEN_RPL] = sched;
> - t3c_handlers[CPL_CLOSE_LISTSRV_RPL] = sched;
> - t3c_handlers[CPL_PASS_ACCEPT_REQ] = sched;
> - t3c_handlers[CPL_PASS_ESTABLISH] = sched;
> - t3c_handlers[CPL_PEER_CLOSE] = sched;
> - t3c_handlers[CPL_CLOSE_CON_RPL] = sched;
> - t3c_handlers[CPL_ABORT_REQ_RSS] = sched;
> - t3c_handlers[CPL_RDMA_TERMINATE] = sched;
> - t3c_handlers[CPL_RDMA_EC_STATUS] = sched;
> - t3c_handlers[CPL_SET_TCB_RPL] = set_tcb_rpl;
> -
> - /*
> - * These are the real handlers that are called from a
> - * work queue.
> - */
> - work_handlers[CPL_ACT_ESTABLISH] = act_establish;
> - work_handlers[CPL_ACT_OPEN_RPL] = act_open_rpl;
> - work_handlers[CPL_RX_DATA] = rx_data;
> - work_handlers[CPL_TX_DMA_ACK] = tx_ack;
> - work_handlers[CPL_ABORT_RPL_RSS] = abort_rpl;
> - work_handlers[CPL_ABORT_RPL] = abort_rpl;
> - work_handlers[CPL_PASS_OPEN_RPL] = pass_open_rpl;
> - work_handlers[CPL_CLOSE_LISTSRV_RPL] = close_listsrv_rpl;
> - work_handlers[CPL_PASS_ACCEPT_REQ] = pass_accept_req;
> - work_handlers[CPL_PASS_ESTABLISH] = pass_establish;
> - work_handlers[CPL_PEER_CLOSE] = peer_close;
> - work_handlers[CPL_ABORT_REQ_RSS] = peer_abort;
> - work_handlers[CPL_CLOSE_CON_RPL] = close_con_rpl;
> - work_handlers[CPL_RDMA_TERMINATE] = terminate;
> - work_handlers[CPL_RDMA_EC_STATUS] = ec_status;
> return 0;
> }
>
>
>
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH] RDMA/cxgb3: Shrink .text with compile-time init of handlers arrays
From: Roland Dreier @ 2010-04-28 21:54 UTC (permalink / raw)
To: Steve Wise; +Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA
In-Reply-To: <4BD88D8B.8050000-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW@public.gmane.org>
> Looks ok to me. Functionally it doesn't change anything I guess.
That's the hope at least...
--
Roland Dreier <rolandd-FYB4Gu1CFyUAvxtiuMwx3w@public.gmane.org> || For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/index.html
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* [RESEND] [PATCH/RFC] opensm: toggle sweeping
From: Arthur Kepner @ 2010-04-28 22:33 UTC (permalink / raw)
To: linux-rdma-u79uwXL29TY76Z2rM5mHXA; +Cc: sashak-smomgflXvOZWk0Htik3J/w
One of our customers recently merged some new systems into a
large, existing cluster. They requested a mechanism to prevent
opensm from sweeping while the new equipment was being added to
the IB fabric, and then resume sweeping once they felt confident
that the newly added (sub)fabric was correctly cabled, and fully
functional. They used the following patch.
Would it be worth adding this (or something with similar functionality)
to opensm?
Signed-off-by: Dale Talcott <Dale.R.Talcott-NSQ8wuThN14@public.gmane.org>
Signed-off-by: Arthur Kepner <akepner-sJ/iWh9BUns@public.gmane.org>
---
main.c | 16 ++++++++++++++++
osm_state_mgr.c | 9 ++++++++-
osm_trap_rcv.c | 40 ++++++++++++++++++++++++----------------
3 files changed, 48 insertions(+), 17 deletions(-)
diff --git a/opensm/opensm/main.c b/opensm/opensm/main.c
index 0093aa7..c3d71bc 100644
--- a/opensm/opensm/main.c
+++ b/opensm/opensm/main.c
@@ -86,6 +86,12 @@ static void mark_usr1_flag(int signum)
osm_usr1_flag = 1;
}
+int sweeping = 1;
+static void toggle_sweeping(int signum)
+{
+ sweeping = !sweeping;
+}
+
static sigset_t saved_sigset;
static void block_signals()
@@ -99,6 +105,7 @@ static void block_signals()
#ifndef HAVE_OLD_LINUX_THREADS
sigaddset(&set, SIGUSR1);
#endif
+ sigaddset(&set, SIGUSR2);
pthread_sigmask(SIG_SETMASK, &set, &saved_sigset);
}
@@ -118,6 +125,8 @@ static void setup_signals()
act.sa_handler = mark_usr1_flag;
sigaction(SIGUSR1, &act, NULL);
#endif
+ act.sa_handler = toggle_sweeping;
+ sigaction(SIGUSR2, &act, NULL);
pthread_sigmask(SIG_SETMASK, &saved_sigset, NULL);
}
@@ -498,6 +507,7 @@ static int daemonize(osm_opensm_t * osm)
int osm_manager_loop(osm_subn_opt_t * p_opt, osm_opensm_t * p_osm)
{
int console_init_flag = 0;
+ int prev_sweeping = sweeping;
if (is_console_enabled(p_opt)) {
if (!osm_console_init(p_opt, &p_osm->console, &p_osm->log))
@@ -524,6 +534,12 @@ int osm_manager_loop(osm_subn_opt_t * p_opt, osm_opensm_t * p_osm)
p_osm->subn.force_heavy_sweep = TRUE;
osm_opensm_sweep(p_osm);
}
+ if (prev_sweeping != sweeping) {
+ prev_sweeping = sweeping;
+ OSM_LOG(&p_osm->log, OSM_LOG_INFO,
+ "Sweeping is now %s\n",
+ (sweeping ? "enabled" : "disabled") );
+ }
}
if (is_console_enabled(p_opt))
osm_console_exit(&p_osm->console, &p_osm->log);
diff --git a/opensm/opensm/osm_state_mgr.c b/opensm/opensm/osm_state_mgr.c
index e43463f..e8eb47b 100644
--- a/opensm/opensm/osm_state_mgr.c
+++ b/opensm/opensm/osm_state_mgr.c
@@ -1405,6 +1405,7 @@ static void do_process_mgrp_queue(osm_sm_t * sm)
void osm_state_mgr_process(IN osm_sm_t * sm, IN osm_signal_t signal)
{
+ extern int sweeping;
CL_ASSERT(sm);
OSM_LOG_ENTER(sm->p_log);
@@ -1415,7 +1416,13 @@ void osm_state_mgr_process(IN osm_sm_t * sm, IN osm_signal_t signal)
switch (signal) {
case OSM_SIGNAL_SWEEP:
- do_sweep(sm);
+ if (!sweeping)
+ OSM_LOG(sm->p_log, OSM_LOG_DEBUG, "sweeping disabled - "
+ "ignoring signal %s in state %s\n",
+ osm_get_sm_signal_str(signal),
+ osm_get_sm_mgr_state_str(sm->p_subn->sm_state));
+ else
+ do_sweep(sm);
break;
case OSM_SIGNAL_IDLE_TIME_PROCESS_REQUEST:
do_process_mgrp_queue(sm);
diff --git a/opensm/opensm/osm_trap_rcv.c b/opensm/opensm/osm_trap_rcv.c
index bf13239..42e9b32 100644
--- a/opensm/opensm/osm_trap_rcv.c
+++ b/opensm/opensm/osm_trap_rcv.c
@@ -332,6 +332,7 @@ static void trap_rcv_process_request(IN osm_sm_t * sm,
boolean_t physp_change_trap = FALSE;
uint64_t event_wheel_timeout = OSM_DEFAULT_TRAP_SUPRESSION_TIMEOUT;
boolean_t run_heavy_sweep = FALSE;
+ extern int sweeping;
OSM_LOG_ENTER(sm->p_log);
@@ -515,23 +516,30 @@ static void trap_rcv_process_request(IN osm_sm_t * sm,
check_sweep:
/* do a sweep if we received a trap */
if (sm->p_subn->opt.sweep_on_trap) {
- /* if this is trap number 128 or run_heavy_sweep is TRUE -
- update the force_heavy_sweep flag of the subnet.
- Sweep also on traps 144 - these traps signal a change of
- certain port capabilities.
- TODO: In the future this can be changed to just getting
- PortInfo on this port instead of sweeping the entire subnet. */
- if (ib_notice_is_generic(p_ntci) &&
- (cl_ntoh16(p_ntci->g_or_v.generic.trap_num) == 128 ||
- cl_ntoh16(p_ntci->g_or_v.generic.trap_num) == 144 ||
- run_heavy_sweep)) {
- OSM_LOG(sm->p_log, OSM_LOG_VERBOSE,
- "Forcing heavy sweep. Received trap:%u\n",
- cl_ntoh16(p_ntci->g_or_v.generic.trap_num));
-
- sm->p_subn->force_heavy_sweep = TRUE;
+ if (!sweeping) {
+ OSM_LOG(sm->p_log, OSM_LOG_DEBUG,
+ "sweeping disabled - ignoring trap %u\n",
+ cl_ntoh16(p_ntci->g_or_v.generic.trap_num));
+ } else {
+ /* if this is trap number 128 or run_heavy_sweep is
+ TRUE - update the force_heavy_sweep flag of the
+ subnet. Sweep also on traps 144 - these traps signal
+ a change of certain port capabilities.
+ TODO: In the future this can be changed to just
+ getting PortInfo on this port instead of sweeping
+ the entire subnet. */
+ if (ib_notice_is_generic(p_ntci) &&
+ (cl_ntoh16(p_ntci->g_or_v.generic.trap_num) == 128 ||
+ cl_ntoh16(p_ntci->g_or_v.generic.trap_num) == 144 ||
+ run_heavy_sweep)) {
+ OSM_LOG(sm->p_log, OSM_LOG_VERBOSE,
+ "Forcing heavy sweep. Received trap:%u\n",
+ cl_ntoh16(p_ntci->g_or_v.generic.trap_num));
+
+ sm->p_subn->force_heavy_sweep = TRUE;
+ }
+ osm_sm_signal(sm, OSM_SIGNAL_SWEEP);
}
- osm_sm_signal(sm, OSM_SIGNAL_SWEEP);
}
/* If we reached here due to trap 129/130/131 - do not need to do
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related
* about SDP on NE020 card
From: Andrea Gozzelino @ 2010-04-29 8:55 UTC (permalink / raw)
To: Amir Vadai
Cc: swise-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW,
linux-rdma-u79uwXL29TY76Z2rM5mHXA
In-Reply-To: <4BD84C68.5000407-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
On Apr 28, 2010 04:55 PM, Amir Vadai <amirv-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org> wrote:
> comment it out in sdp_cma.c at:
> sdp_remove_device() and sdp_add_device()
>
> If I am right, the problem that you see, is that ib_create_fmr_pool()
> is
> failing (you should see an error in dmesg), and because of that
> ib_set_client_data() is not called. And this is the reason for the
> error
> that you saw in the logs.
>
> Because you won't be using ZCopy, no need to call ib_create_fmr_pool()
> -
> ofcourse, make sure you comment out the calls to ib_destroy_fmr_pool
> and
> ib_flush_fmr_pool()
>
> - Amir
Hi Amir,
I put in off status the ZCopy with command:
# echo 0 > /sys/module/ib_sdp/parameters/sdp_zcopy_thresh
I comment out some lines in file sdp_main.c, but now the programs don't
compile with a lot of errors.
I list out the lines that I have commented out:
file sdp_main.c
line 2625: struct ib_fmr_pool_param fmr_param
lines from 2645 to 2661
lines 2665 2666
lines from 2739 to 2745
All these lines contain a word "fmr".
I don't find word "fmr" in file sdp_cma.c; sdp_main.c calls objects of
sdp_cma.c
SDP doesn't work. The output message of dmesg command is:
sdp_init_qp:105 sdp_sock SDP not available on device nes0.
I would like identify the problem: if we are almost sure that Fast
Memory Region FMR is responsible for that, the bug 2028 solution in July
will remove the error and I hope that SDP will work fine. On the other
side, if FMR is not involved, there is an unknown situation and I am not
happy.
Could you evaluate the written error?
Could you give me your opinion?
Amir and Steve, thank you very much for help.
Regards,
Andrea
Andrea Gozzelino
INFN - Laboratori Nazionali di Legnaro (LNL)
Viale dell'Universita' 2
I-35020 - Legnaro (PD)- ITALIA
Tel: +39 049 8068346
Fax: +39 049 641925
Mail: andrea.gozzelino-PK20h7lG/Rc1GQ1Ptb7lUw@public.gmane.org
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: RoCE supplement to IB Spec
From: Eli Cohen @ 2010-04-29 12:18 UTC (permalink / raw)
To: Pradeep Satyanarayana; +Cc: linux-rdma, EWG
In-Reply-To: <4BD781CF.2080403-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>
Yes, we're compiant to the spec. If any bugs are found please report
them.
On Tue, Apr 27, 2010 at 05:31:11PM -0700, Pradeep Satyanarayana wrote:
> Is it the intent that the RoCE implementation in OFED-1.5.1 corresponds to the RoCE supplement to the IB spec dated April 6th 2010,
> excepting bugs of course? Or are there known deviations from the spec?
>
> Pradeep
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH 2/4] ib_core: implement XRC RCV qp's
From: Roland Dreier @ 2010-04-29 19:46 UTC (permalink / raw)
To: Jack Morgenstein
Cc: rolandd-FYB4Gu1CFyUAvxtiuMwx3w, linux-rdma-u79uwXL29TY76Z2rM5mHXA
In-Reply-To: <201002281102.21207.jackm-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
> Note that for users who do not wish to utilize the reg/unreg verbs,
> a destroy_xrc_rcv_qp verb is also provided. Thus, usage is:
> Either: create/destroy_xrc_rcv_qp
> Or: create/reg/unreg_xrc_rcv_qp (the unreg is used in place of destroy)
I don't really understand the semantics here. What is supposed to
happen if I do create/reg/destroy? What happens if one process does
destroy while another process is still registered? To make everything
even more confusing, mlx4 defines unreg_rxc_rcv_qp to be equivalent to
destroy_xrc_rcv_qp. I'm not even clear why the low-level driver has two
entry points for these two methods -- shouldn't the uverbs core be
handling the counting/listing of xrc rcv qps and just ask the low-level
driver to destroy the QP when it's really done with it?
(By the way, should we use the name "target QP" instead of "rcv QP" to
match the actual IB spec?)
- R.
--
Roland Dreier <rolandd-FYB4Gu1CFyUAvxtiuMwx3w@public.gmane.org> || For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/index.html
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH 4/4] mlx4: implement XRC RCV qp's
From: Roland Dreier @ 2010-04-29 20:03 UTC (permalink / raw)
To: Jack Morgenstein
Cc: rolandd-FYB4Gu1CFyUAvxtiuMwx3w, linux-rdma-u79uwXL29TY76Z2rM5mHXA
In-Reply-To: <201002281102.55944.jackm-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
> @@ -175,7 +175,7 @@ struct ib_cq *mlx4_ib_create_cq(struct ib_device *ibdev, int entries, int vector
> if (entries < 1 || entries > dev->dev->caps.max_cqes)
> return ERR_PTR(-EINVAL);
>
> - cq = kmalloc(sizeof *cq, GFP_KERNEL);
> + cq = kzalloc(sizeof *cq, GFP_KERNEL);
What's the reason for this change?
> @@ -477,23 +483,51 @@ static struct ib_xrcd *mlx4_ib_alloc_xrcd(struct ib_device *ibdev,
> + pd = mlx4_ib_alloc_pd(ibdev, NULL, NULL);
> + cq = mlx4_ib_create_cq(ibdev, 1, 0, NULL, NULL);
Why does every xrcd get a PD and a CQ now? Just in case someone wants
to create a rcv QP? (The spec is unclear on this -- for "create XRC
target QP" it says "A set of initial QP attributes must be specified by
the Consumer," but then doesn't mention anything in the input modifiers,
so it's not clear what PD/CQ is supposed to be used)
> + (1ull << IB_USER_VERBS_CMD_DESTROY_XRC_RCV_QP);
> }
>
> -
This seems to be repairing whitespace damage from the previous patch.
> +int mlx4_ib_reg_xrc_rcv_qp(struct ib_xrcd *xrcd, void *context, u32 qp_num)
> + mutex_lock(&mibqp->mutex);
> + list_for_each_entry(tmp, &mibqp->xrc_reg_list, list)
> + if (tmp->context == context) {
> + mutex_unlock(&mibqp->mutex);
> + kfree(ctx_entry);
> + mutex_unlock(&to_mdev(xrcd->device)->xrc_reg_mutex);
> + return 0;
> + }
> +
> + ctx_entry->context = context;
> + list_add_tail(&ctx_entry->list, &mibqp->xrc_reg_list);
> + mutex_unlock(&mibqp->mutex);
This list handling looks completely generic and is what I was saying
should probably be in the core uverbs module.
> +int mlx4_ib_query_xrc_rcv_qp(struct ib_xrcd *ibxrcd, u32 qp_num,
> + struct ib_qp_attr *qp_attr, int qp_attr_mask,
> + struct ib_qp_init_attr *qp_init_attr)
Virtually all of this function seems identical to the existing query QP
operation. We should avoid the mass duplication of code.
Also I'm not clear why this function takes a qp_num instead of a QP
handle. Why does the consumer have to pass in the XRCD? The IB spec
XRC annex just shows the QP handle as input to this verb. Is it because
the reg_xrc_rcv_qp doesn't give a QP handle back?
Finally, in this implementation, what happens if the consumer passes in
a QP that isn't an XRC rcv QP?
> + mqp = __mlx4_qp_lookup(dev->dev, qp_num);
> + if (unlikely(!mqp)) {
> + printk(KERN_WARNING "mlx4_ib_reg_xrc_rcv_qp: "
> + "unknown QPN %06x\n", qp_num);
> + goto err_out;
> + }
> +
> + qp = to_mibqp(mqp);
> + if (xrcd->xrcdn != to_mxrcd(qp->ibqp.xrcd)->xrcdn)
> + goto err_out;
In other words is that dereference of ->xrcdn safe if ibqp.xrcd is not set?
(And the error message talks about reg_xrc_rcv_qp instead of query)
--
Roland Dreier <rolandd-FYB4Gu1CFyUAvxtiuMwx3w@public.gmane.org> || For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/index.html
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* [PATCH] mlx4: Fix chunk sg list overflow in mlx4_alloc_icm()
From: sebastien dugue @ 2010-04-30 11:21 UTC (permalink / raw)
To: linux-rdma; +Cc: Roland Dreier
If the number of sg entries in the ICM chunk reaches MLX4_ICM_CHUNK_LEN,
we must set chunk = NULL __even__ for coherent mappings (as is done for
mthca). Otherwise we may overflow the sg list.
Signed-off-by: Sebastien Dugue <sebastien.dugue-6ktuUTfB/bM@public.gmane.org>
---
drivers/net/mlx4/icm.c | 3 ++-
1 files changed, 2 insertions(+), 1 deletions(-)
diff --git a/drivers/net/mlx4/icm.c b/drivers/net/mlx4/icm.c
index 57288ca..ef62f17 100644
--- a/drivers/net/mlx4/icm.c
+++ b/drivers/net/mlx4/icm.c
@@ -175,9 +175,10 @@ struct mlx4_icm *mlx4_alloc_icm(struct mlx4_dev *dev, int npages,
if (chunk->nsg <= 0)
goto fail;
+ }
+ if (chunk->npages == MLX4_ICM_CHUNK_LEN)
chunk = NULL;
- }
npages -= 1 << cur_order;
} else {
--
1.6.0.4
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related
* [PATCH 0/2] RDMA/core: add support for iWarp multicast acceleration over IB_QPT_RAW_ETY QP type
From: Mirek Walukiewicz @ 2010-04-30 16:53 UTC (permalink / raw)
To: rdreier-FYB4Gu1CFyUAvxtiuMwx3w; +Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA
This patch series implements iWarp multicast acceleration over raw eth QP type.
In ibv_attach_mcast and ibv_detach_mcast, new case is added for RDMA_TRANSPORT_IWARP to allow multicast
on IB_QPT_RAW_ETY QP type. This change is for iWarp only and does not affect IB.
Changes to nes demenstrate how IB_QPT_RAW_ETY QP is used to accelerate multicast traffic.
Miroslaw Walukiewicz
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* [PATCH 1/2] RDMA/core: add support for iWarp multicast acceleration over IB_QPT_RAW_ETY QP type
From: miroslaw.walukiewicz-ral2JQCrhuEAvxtiuMwx3w @ 2010-04-30 16:54 UTC (permalink / raw)
To: rdreier-FYB4Gu1CFyUAvxtiuMwx3w; +Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA
In-Reply-To: <20100430165249.1386.73960.stgit-dAdtdUp2yJRU7keBU/FxOFDQ4js95KgL@public.gmane.org>
The patch enables usage of attach_mcast()/detach_mcast driver
specific verbs for iWARP devices.
IB path is unchanged.
iWARP path adds checking for raw ethernet QP type
Signed-off-by: Mirek Walukiewicz <miroslaw.walukiewicz-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
drivers/infiniband/core/verbs.c | 24 ++++++++++++++++++++----
1 files changed, 20 insertions(+), 4 deletions(-)
diff --git a/drivers/infiniband/core/verbs.c b/drivers/infiniband/core/verbs.c
index a7da9be..b5023f2 100644
--- a/drivers/infiniband/core/verbs.c
+++ b/drivers/infiniband/core/verbs.c
@@ -887,9 +887,17 @@ int ib_attach_mcast(struct ib_qp *qp, union ib_gid *gid, u16 lid)
{
if (!qp->device->attach_mcast)
return -ENOSYS;
- if (gid->raw[0] != 0xff || qp->qp_type != IB_QPT_UD)
- return -EINVAL;
+ switch (qp->device->node_type) {
+ case RDMA_TRANSPORT_IB:
+ if (gid->raw[0] != 0xff || qp->qp_type != IB_QPT_UD)
+ return -EINVAL;
+ break;
+ case RDMA_TRANSPORT_IWARP:
+ if (qp->qp_type != IB_QPT_RAW_ETY)
+ return -EINVAL;
+ break;
+ }
return qp->device->attach_mcast(qp, gid, lid);
}
EXPORT_SYMBOL(ib_attach_mcast);
@@ -898,9 +906,17 @@ int ib_detach_mcast(struct ib_qp *qp, union ib_gid *gid, u16 lid)
{
if (!qp->device->detach_mcast)
return -ENOSYS;
- if (gid->raw[0] != 0xff || qp->qp_type != IB_QPT_UD)
- return -EINVAL;
+ switch (qp->device->node_type) {
+ case RDMA_TRANSPORT_IB:
+ if (gid->raw[0] != 0xff || qp->qp_type != IB_QPT_UD)
+ return -EINVAL;
+ break;
+ case RDMA_TRANSPORT_IWARP:
+ if (qp->qp_type != IB_QPT_RAW_ETY)
+ return -EINVAL;
+ break;
+ }
return qp->device->detach_mcast(qp, gid, lid);
}
EXPORT_SYMBOL(ib_detach_mcast);
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related
* [PATCH 2/2] RDMA/nes: add support of iWARP multicast acceleration over IB_QPT_RAW_ETY QP type
From: miroslaw.walukiewicz-ral2JQCrhuEAvxtiuMwx3w @ 2010-04-30 16:55 UTC (permalink / raw)
To: rdreier-FYB4Gu1CFyUAvxtiuMwx3w; +Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA
In-Reply-To: <20100430165249.1386.73960.stgit-dAdtdUp2yJRU7keBU/FxOFDQ4js95KgL@public.gmane.org>
This patch implements iWarp multicast acceleration (IMA)
over IB_QPT_RAW_ETY QP type in nes driver.
Application creates a raw eth QP (IBV_QPT_RAW_ETH in user-space) and
manages the multicast via ibv_attach_mcast and ibv_detach_mcast calls.
Calling ibv_attach_mcast/ibv_datach_mcast has an effect of
enabling/disabling L2 MAC address filters in HW.
Signed-off-by: Mirek Walukiewicz <miroslaw.walukiewicz-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
drivers/infiniband/hw/nes/Makefile | 2
drivers/infiniband/hw/nes/nes.c | 4
drivers/infiniband/hw/nes/nes.h | 2
drivers/infiniband/hw/nes/nes_nic.c | 11
drivers/infiniband/hw/nes/nes_ud.c | 2070 +++++++++++++++++++++++++++++++++
drivers/infiniband/hw/nes/nes_ud.h | 86 +
drivers/infiniband/hw/nes/nes_verbs.c | 221 +++-
drivers/infiniband/hw/nes/nes_verbs.h | 7
8 files changed, 2388 insertions(+), 15 deletions(-)
create mode 100644 drivers/infiniband/hw/nes/nes_ud.c
create mode 100644 drivers/infiniband/hw/nes/nes_ud.h
diff --git a/drivers/infiniband/hw/nes/Makefile b/drivers/infiniband/hw/nes/Makefile
index 3514851..850df8e 100644
--- a/drivers/infiniband/hw/nes/Makefile
+++ b/drivers/infiniband/hw/nes/Makefile
@@ -1,3 +1,3 @@
obj-$(CONFIG_INFINIBAND_NES) += iw_nes.o
-iw_nes-objs := nes.o nes_hw.o nes_nic.o nes_utils.o nes_verbs.o nes_cm.o
+iw_nes-objs := nes.o nes_hw.o nes_nic.o nes_utils.o nes_verbs.o nes_cm.o nes_ud.o
diff --git a/drivers/infiniband/hw/nes/nes.c b/drivers/infiniband/hw/nes/nes.c
index de7b9d7..e430804 100644
--- a/drivers/infiniband/hw/nes/nes.c
+++ b/drivers/infiniband/hw/nes/nes.c
@@ -60,6 +60,8 @@
#include <linux/route.h>
#include <net/ip_fib.h>
+#include "nes_ud.h"
+
MODULE_AUTHOR("NetEffect");
MODULE_DESCRIPTION("NetEffect RNIC Low-level iWARP Driver");
MODULE_LICENSE("Dual BSD/GPL");
@@ -1205,6 +1207,7 @@ static int __init nes_init_module(void)
if (retval1 < 0)
printk(KERN_ERR PFX "Unable to create NetEffect sys files.\n");
}
+ nes_ud_init();
return retval;
}
@@ -1214,6 +1217,7 @@ static int __init nes_init_module(void)
*/
static void __exit nes_exit_module(void)
{
+ nes_ud_exit();
nes_cm_stop();
nes_remove_driver_sysfs(&nes_pci_driver);
diff --git a/drivers/infiniband/hw/nes/nes.h b/drivers/infiniband/hw/nes/nes.h
index cc78fee..faf420f 100644
--- a/drivers/infiniband/hw/nes/nes.h
+++ b/drivers/infiniband/hw/nes/nes.h
@@ -102,6 +102,7 @@
#define NES_DRV_OPT_NO_INLINE_DATA 0x00000080
#define NES_DRV_OPT_DISABLE_INT_MOD 0x00000100
#define NES_DRV_OPT_DISABLE_VIRT_WQ 0x00000200
+#define NES_DRV_OPT_MCAST_LOGPORT_MAP 0x00000800
#define NES_AEQ_EVENT_TIMEOUT 2500
#define NES_DISCONNECT_EVENT_TIMEOUT 2000
@@ -128,6 +129,7 @@
#define NES_DBG_IW_RX 0x00020000
#define NES_DBG_IW_TX 0x00040000
#define NES_DBG_SHUTDOWN 0x00080000
+#define NES_DBG_UD 0x00100000
#define NES_DBG_RSVD1 0x10000000
#define NES_DBG_RSVD2 0x20000000
#define NES_DBG_RSVD3 0x40000000
diff --git a/drivers/infiniband/hw/nes/nes_nic.c b/drivers/infiniband/hw/nes/nes_nic.c
index b7c813f..c7bbb83 100644
--- a/drivers/infiniband/hw/nes/nes_nic.c
+++ b/drivers/infiniband/hw/nes/nes_nic.c
@@ -897,7 +897,7 @@ static void nes_netdev_set_multicast_list(struct net_device *netdev)
((mc_nic_index = nesvnic->mcrq_mcast_filter(nesvnic,
get_addr(addrs, i++))) == 0));
if (mc_nic_index < 0)
- mc_nic_index = nesvnic->nic_index;
+ mc_nic_index = (1 << nesvnic->nic_index);
while (nesadapter->pft_mcast_map[mc_index] < 16 &&
nesadapter->pft_mcast_map[mc_index] !=
nesvnic->nic_index &&
@@ -930,7 +930,7 @@ static void nes_netdev_set_multicast_list(struct net_device *netdev)
nes_write_indexed(nesdev,
perfect_filter_register_address+4+(mc_index * 8),
(u32)macaddr_high | NES_MAC_ADDR_VALID |
- ((((u32)(1<<mc_nic_index)) << 16)));
+ ((((u32)(mc_nic_index)) << 16)));
nesadapter->pft_mcast_map[mc_index] =
nesvnic->nic_index;
} else {
@@ -1676,8 +1676,11 @@ struct net_device *nes_netdev_init(struct nes_device *nesdev,
(nesvnic->nesdev->nesadapter->port_count == 1 &&
nesvnic->nesdev->nesadapter->adapter_fcn_count == 2)) {
nesvnic->qp_nic_index[0] = nesvnic->nic_index;
- nesvnic->qp_nic_index[1] = nesvnic->nic_index
- + 2;
+
+ if (nes_drv_opt & NES_DRV_OPT_MCAST_LOGPORT_MAP)
+ nesvnic->qp_nic_index[1] = 0xf;
+ else
+ nesvnic->qp_nic_index[1] = nesvnic->nic_index+2;
nesvnic->qp_nic_index[2] = 0xf;
nesvnic->qp_nic_index[3] = 0xf;
} else {
diff --git a/drivers/infiniband/hw/nes/nes_ud.c b/drivers/infiniband/hw/nes/nes_ud.c
new file mode 100644
index 0000000..f004855
--- /dev/null
+++ b/drivers/infiniband/hw/nes/nes_ud.c
@@ -0,0 +1,2070 @@
+/*
+ * Copyright (c) 2008 - 2010 Intel Corporation. All rights reserved.
+ * Copyright (c) 2006 - 2008 Neteffect, All rights reserved.
+ * Copyright (c) 2005 Open Grid Computing, Inc. All rights reserved.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses. You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#include <linux/version.h>
+#include <linux/completion.h>
+#include <linux/mutex.h>
+#include <linux/poll.h>
+#include <linux/idr.h>
+#include <linux/in.h>
+#include <linux/in6.h>
+#include <linux/device.h>
+#include <linux/netdevice.h>
+#include <linux/list.h>
+#include <linux/miscdevice.h>
+#include <linux/device.h>
+
+#include <rdma/ib_umem.h>
+#include <rdma/ib_user_verbs.h>
+
+#include "nes.h"
+#include "nes_ud.h"
+
+#define NES_UD_BASE_XMIT_NIC_QPID 28
+#define NES_UD_BASE_RECV_NIC_IDX 12
+#define NES_UD_BASE_XMIT_NIC_IDX 8
+#define NES_UD_MAX_NIC_CNT 8
+#define NES_UD_CLEANUP_TIMEOUT (HZ)
+#define NES_UD_MCAST_TBL_SZ 128
+#define NES_UD_SINGLE_HDR_SZ 64
+#define NES_UD_CQE_NUM NES_NIC_WQ_SIZE
+#define NES_UD_SKSQ_WAIT_TIMEOUT 100000
+#define NES_UD_MAX_REG_CNT 128
+
+#define NES_UD_MAX_ADAPTERS 4 /* number of supported interfaces for RAW ETH */
+
+#define NES_UD_MAX_REG_HASH_CNT 256 /* last byte of the STAG is hash key */
+
+/*
+ * the same multicast could be allocated up to 2 owners so there could be
+ * two differentmcast entries allocated for the same mcas address
+ */
+struct nes_ud_file;
+struct nes_ud_mcast {
+ u8 addr[3];
+ u8 in_use;
+ struct nes_ud_file *owner;
+ u8 nic_mask;
+};
+
+struct nes_ud_mem_region {
+ struct list_head list;
+ dma_addr_t *addrs;
+ u64 va;
+ u64 length;
+ u32 pg_cnt;
+ u32 in_use;
+ u32 stag; /* stag related this structure */
+};
+
+struct nic_queue_info {
+ u32 qpn;
+ u32 nic_index;
+ u32 logical_port;
+ enum nes_ud_dev_priority prio;
+ enum nes_ud_queue_type queue_type;
+ struct nes_ud_file *file;
+ struct nes_ud_file file_body;
+};
+
+struct nes_ud_resources {
+ int num_logport_confed;
+ int num_allocated_nics;
+ u8 logport_2_map;
+ u8 logport_3_map;
+ u32 original_6000;
+ u32 original_60b8;
+ struct nic_queue_info nics[NES_UD_MAX_NIC_CNT];
+ struct mutex mutex;
+ struct nes_ud_mcast mcast[NES_UD_MCAST_TBL_SZ];
+ u32 adapter_no; /* the allocated adapter no */
+
+ /* the unique ID of the NE020 adapter */
+ /*- it is allocated once per HW */
+ struct nes_adapter *pAdap;
+};
+
+/* memory hash list entry */
+struct nes_ud_hash_mem {
+ struct list_head list;
+ int read_stats;
+};
+
+
+
+struct nes_ud_mem {
+ /* hash list of registered STAGs */
+ struct nes_ud_hash_mem mrs[NES_UD_MAX_REG_HASH_CNT];
+ struct mutex mutex;
+};
+
+/* the QP in format x.y.z where x is adapter no, */
+/* y is ud file idx in adapter, z is a qp no */
+static struct nes_ud_mem ud_mem;
+
+struct nes_ud_send_wr {
+ u32 wr_cnt;
+ u32 qpn;
+ u32 flags;
+ u32 resv[1];
+ struct ib_sge sg_list[64];
+};
+
+struct nes_ud_recv_wr {
+ u32 wr_cnt;
+ u32 qpn;
+ u32 resv[2];
+ struct ib_sge sg_list[64];
+};
+
+static struct nes_ud_resources nes_ud_rsc[NES_UD_MAX_ADAPTERS];
+static struct workqueue_struct *nes_ud_workqueue;
+
+/*
+ * locate_ud_adapter
+ *
+ * the function locates the UD adapter
+* on base of the adapter unique ID (structure nes_adapter)
+ */
+static inline
+struct nes_ud_resources *locate_ud_adapter(struct nes_adapter *pAdapt)
+{
+ int i;
+ struct nes_ud_resources *pRsc;
+
+ for (i = 0; i < NES_UD_MAX_ADAPTERS; i++) {
+ pRsc = &nes_ud_rsc[i];
+
+ if (pRsc->pAdap == pAdapt)
+ return pRsc;
+
+ }
+ return NULL;
+}
+
+/*
+ * allocate_ud_adapter()
+ *
+ * function allocates a new adapter
+ */
+static inline
+struct nes_ud_resources *allocate_ud_adapter(struct nes_adapter *pAdapt)
+{
+ int i;
+ struct nes_ud_resources *pRsc;
+
+ for (i = 0; i < NES_UD_MAX_ADAPTERS; i++) {
+ pRsc = &nes_ud_rsc[i];
+ if (pRsc->pAdap == NULL) {
+ pRsc->pAdap = pAdapt;
+ nes_debug(NES_DBG_UD, "new UD Adapter allocated %d"
+ " for adapter %p no =%d\n", i, pAdapt, pRsc->adapter_no);
+ return pRsc;
+ }
+ }
+ nes_debug(NES_DBG_UD, "Unable to allocate adapter\n");
+ return NULL;
+}
+
+static inline
+struct nes_ud_file *allocate_nic_queue(struct nes_vnic *nesvnic,
+ enum nes_ud_queue_type queue_type)
+{
+ struct nes_ud_file *file = NULL;
+ int i = 0;
+ u8 select_log_port = 0xf;
+ struct nes_device *nesdev = nesvnic->nesdev;
+ int log_port_2_alloced = 0;
+ int log_port_3_alloced = 0;
+ int ret = 0;
+ struct nes_ud_resources *pRsc;
+
+ /* the first thing that must be done is determine the adapter */
+ /* number max the adapter could have up to 2 interfaces */
+ if (nesvnic->nic_index != 0 && nesvnic->nic_index != 1) {
+ nes_debug(NES_DBG_UD, "nic queue allocation failed"
+ " nesvnic->nic_index = %d\n", nesvnic->nic_index);
+ return NULL;
+ }
+
+ /* locate device on base of nesvnic */
+ /* - when it is an unknown card a new one is allocated */
+ pRsc = locate_ud_adapter(nesdev->nesadapter);
+ if (pRsc == NULL)
+ return NULL;
+
+ for (i = 0; i < NES_UD_MAX_NIC_CNT; i++) {
+ if (pRsc->nics[i].file->active == 0)
+ continue;
+ if (pRsc->nics[i].logical_port == 2 &&
+ queue_type == pRsc->nics[i].queue_type)
+ log_port_2_alloced++;
+ if (pRsc->nics[i].logical_port == 3 &&
+ queue_type == pRsc->nics[i].queue_type)
+ log_port_3_alloced++;
+ }
+
+ /* check dual/single card */
+ if (pRsc->logport_2_map != pRsc->logport_3_map) {
+ /* a dual port card */
+ /* allocation is NIC2, NIC2, NIC3, NIC3 */
+ /*- no RX packat replication supported */
+ if (log_port_2_alloced < 2 &&
+ pRsc->logport_2_map == nesvnic->nic_index)
+ select_log_port = 2;
+ else if (log_port_3_alloced < 2 &&
+ pRsc->logport_3_map == nesvnic->nic_index)
+ select_log_port = 3;
+ } else {
+ /* single port card */
+ /* change allocation scheme to NIC2,NIC3,NIC2,NIC3 */
+ switch (log_port_2_alloced + log_port_3_alloced) {
+ case 0: /* no QPs allocated - use NIC2 */
+ if (pRsc->logport_2_map == nesvnic->nic_index)
+ select_log_port = 2;
+
+ break;
+ case 1: /* NIC2 or NIC3 allocated */
+ if (log_port_2_alloced > 0) {
+ /* if NIC2 allocated use NIC3 */
+ if (pRsc->logport_3_map == nesvnic->nic_index)
+ select_log_port = 3;
+
+ } else {
+ /* when NIC3 allocated use NIC2 */
+ if (pRsc->logport_2_map == nesvnic->nic_index)
+ select_log_port = 2;
+
+ }
+ break;
+
+ case 2:
+ /* NIC2 and NIC3 allocated or both ports on NIC3 - use NIC2 */
+ if ((log_port_2_alloced == 1) ||
+ (log_port_3_alloced == 2)) {
+ if (pRsc->logport_2_map == nesvnic->nic_index)
+ select_log_port = 2;
+
+ } else {
+ /* both ports allocated on NIC2 - use NIC3 */
+ if (pRsc->logport_3_map == nesvnic->nic_index)
+ select_log_port = 3;
+
+ }
+ break;
+ case 3:
+ /* when both NIC2 allocated use NIC3 */
+ if (log_port_2_alloced == 2) {
+ if (pRsc->logport_3_map == nesvnic->nic_index)
+ select_log_port = 3;
+
+ } else {
+ /* when both NIC3 alloced use NIC2 */
+ if (pRsc->logport_2_map == nesvnic->nic_index)
+ select_log_port = 2;
+ }
+ break;
+
+ default:
+ break;
+ }
+ }
+ if (select_log_port == 0xf) {
+ ret = -1;
+ nes_debug(NES_DBG_UD, "%s(%d) logport allocation failed "
+ "log_port_2_alloced=%d log_port_3_alloced=%d\n",
+ __func__, __LINE__, log_port_2_alloced,
+ log_port_3_alloced);
+ goto out;
+ }
+
+ nes_debug(NES_DBG_UD, "%s(%d) log_port_2_alloced=%d "
+ "log_port_3_alloced=%d select_log_port=%d\n",
+ __func__, __LINE__, log_port_2_alloced,
+ log_port_3_alloced, select_log_port);
+
+ for (i = 0; i < NES_UD_MAX_NIC_CNT; i++) {
+ if (pRsc->nics[i].file->active == 1)
+ continue;
+ if (pRsc->nics[i].logical_port == select_log_port &&
+ queue_type == pRsc->nics[i].queue_type) {
+
+ /* file is preallocated during initialization */
+ file = pRsc->nics[i].file;
+ memset(file, 0, sizeof(*file));
+
+ file->nesvnic = nesvnic;
+ file->queue_type = queue_type;
+
+ file->prio = pRsc->nics[i].prio;
+ file->qpn = pRsc->nics[i].qpn;
+ file->nes_ud_nic_index = pRsc->nics[i].nic_index;
+ file->rsc_idx = i;
+ file->adapter_no = pRsc->adapter_no;
+ goto out;
+ }
+ }
+
+out:
+ return file;
+}
+
+static inline int del_rsc_list(struct nes_ud_file *file)
+{
+ int logport_2_cnt = 0;
+ int logport_3_cnt = 0;
+ struct nes_device *nesdev = file->nesvnic->nesdev;
+ int i = 0;
+ struct nes_ud_resources *pRsc;
+
+ if (file == NULL) {
+ nes_debug(NES_DBG_UD, "%s(%d) file is NULL\n",
+ __func__, __LINE__);
+ return -EFAULT;
+ }
+ if (file->nesvnic == NULL) {
+ nes_debug(NES_DBG_UD, "%s(%d) file->nesvnic is NULL\n",
+ __func__, __LINE__);
+ return -EFAULT;
+ }
+ if (nesdev == NULL) {
+ nes_debug(NES_DBG_UD, "%s(%d) nesdev is NULL\n",
+ __func__, __LINE__);
+ return -EFAULT;
+ }
+
+ /* locate device on base of nesvnic */
+ /*- when it is an unknown card a new one is allocated */
+ pRsc = locate_ud_adapter(nesdev->nesadapter);
+ if (pRsc == NULL) {
+ nes_debug(NES_DBG_UD, "%s(%d) cannot locate an allocated "
+ "adapter is NULL\n", __func__, __LINE__);
+ return -EFAULT;
+ }
+ if (--pRsc->num_allocated_nics == 0) {
+ nes_write_indexed(nesdev, 0x60b8, pRsc->original_60b8);
+ nes_write_indexed(nesdev, 0x6000, pRsc->original_6000);
+ pRsc->num_logport_confed = 0;
+ }
+ BUG_ON(pRsc->num_allocated_nics < 0);
+ BUG_ON(file->rsc_idx >= NES_UD_MAX_NIC_CNT);
+
+ for (i = 0; i < NES_UD_MAX_NIC_CNT; i++) {
+ if (pRsc->nics[i].file->active &&
+ pRsc->nics[i].logical_port == 2)
+ logport_2_cnt++;
+ if (pRsc->nics[i].file->active &&
+ pRsc->nics[i].logical_port == 3)
+ logport_3_cnt++;
+ }
+
+ if (pRsc->num_logport_confed != 0x3 && logport_2_cnt == 0)
+ pRsc->logport_2_map = 0xf;
+
+ if (pRsc->num_logport_confed != 0x3 && logport_3_cnt == 0)
+ pRsc->logport_3_map = 0xf;
+ return 0;
+}
+
+/*
+* the QPN contains now the number of the RAW ETH
+* adapter and QPN number on the adapter
+* the adapter number is located in the highier
+* 8 bits so QPN is stored as [adapter:qpn]
+*/
+static inline
+struct nes_ud_file *get_file_by_qpn(struct nes_ud_resources *pRsc, int qpn)
+{
+ int i = 0;
+
+ for (i = 0; i < NES_UD_MAX_NIC_CNT; i++) {
+ if (pRsc->nics[i].file->active &&
+ pRsc->nics[i].qpn == (qpn & 0xff))
+ return pRsc->nics[i].file;
+
+ }
+ return NULL;
+}
+
+/* function counts all ETH RAW entities that have */
+/* a specific type and relation to specific vnic */
+static inline
+int count_files_by_nic(struct nes_vnic *nesvnic,
+ enum nes_ud_queue_type queue_type)
+{
+ int count = 0;
+ int i = 0;
+ struct nes_ud_resources *pRsc;
+
+ pRsc = locate_ud_adapter(nesvnic->nesdev->nesadapter);
+ if (pRsc == NULL)
+ return 0;
+
+ for (i = 0; i < NES_UD_MAX_NIC_CNT; i++) {
+ if (pRsc->nics[i].file->active &&
+ pRsc->nics[i].file->nesvnic == nesvnic &&
+ pRsc->nics[i].queue_type == queue_type)
+ count++;
+ }
+ return count;
+}
+
+/* function counts all RAW ETH entities the have a specific type */
+static inline
+int count_files(struct nes_vnic *nesvnic, enum nes_ud_queue_type queue_type)
+{
+ int count = 0;
+ int i = 0;
+ struct nes_ud_resources *pRsc;
+
+ pRsc = locate_ud_adapter(nesvnic->nesdev->nesadapter);
+ if (pRsc == NULL)
+ return 0;
+
+ for (i = 0; i < NES_UD_MAX_NIC_CNT; i++) {
+ if (pRsc->nics[i].file->active &&
+ pRsc->nics[i].queue_type == queue_type)
+ count++;
+ }
+ return count;
+}
+
+/*
+ * the function locates the entry allocated by IGMP and modifies the
+ * PFT entry with the list of the NICs allowed to receive that multicast
+ * the NIC0/NIC1 are removed due to performance issue so tcpdum
+ * like tools cannot receive the accelerated multicasts
+ */
+static void mcast_fix_filter_table_single(struct nes_ud_file *file, u8 *addr)
+{
+ struct nes_device *nesdev = file->nesvnic->nesdev;
+ int i = 0;
+ u32 macaddr_low;
+ u32 orig_low;
+ u32 macaddr_high;
+ u32 prev_high;
+
+ for (i = 0; i < 48; i++) {
+ macaddr_low = nes_read_indexed(nesdev,
+ NES_IDX_PERFECT_FILTER_LOW + i*8);
+ orig_low = macaddr_low;
+ macaddr_high = nes_read_indexed(nesdev,
+ NES_IDX_PERFECT_FILTER_LOW + 4 + i*8);
+ if (!(macaddr_high & NES_MAC_ADDR_VALID))
+ continue;
+ if ((macaddr_high & 0xffff) != 0x0100)
+ continue;
+ if ((macaddr_low & 0xff) != addr[2])
+ continue;
+ macaddr_low >>= 8;
+ if ((macaddr_low & 0xff) != addr[1])
+ continue;
+ macaddr_low >>= 8;
+ if ((macaddr_low & 0xff) != addr[0])
+ continue;
+ macaddr_low >>= 8;
+ if ((macaddr_low & 0xff) != 0x5e)
+ continue;
+ /* hit - that means Linux or other UD set this bit earlier */
+ prev_high = macaddr_high;
+ nes_write_indexed(nesdev, NES_IDX_PERFECT_FILTER_LOW + 4 + i*8, 0);
+ macaddr_high = (macaddr_high & 0xfffcffff) |
+ ((1<<file->nes_ud_nic_index) << 16);
+
+ nes_debug(NES_DBG_UD, "%s(%d) found addr to fix, "
+ "i=%d, macaddr_high=0x%x macaddr_low=0x%x "
+ "nic_idx=%d prev_high=0x%x\n",
+ __func__, __LINE__, i, macaddr_high, orig_low,
+ file->nes_ud_nic_index, prev_high);
+ nes_write_indexed(nesdev,
+ NES_IDX_PERFECT_FILTER_LOW + 4 + i*8, macaddr_high);
+ break;
+ }
+}
+
+/* this function is implemented that way because the Linux multicast API
+ use the multicast list approach. When a new multicast address is added
+ all PFT table is reinitialized by linux and all entries must be fixed
+ by this procedure
+*/
+static void mcast_fix_filter_table(struct nes_ud_file *file)
+{
+ int i;
+ struct nes_ud_resources *pRsc;
+
+ pRsc = locate_ud_adapter(file->nesvnic->nesdev->nesadapter);
+ if (pRsc == NULL)
+ return;
+
+ for (i = 0; i < NES_UD_MCAST_TBL_SZ; i++) {
+ if (pRsc->mcast[i].in_use != 0)
+ mcast_fix_filter_table_single(pRsc->mcast[i].owner,
+ pRsc->mcast[i].addr);
+ }
+}
+
+/* function invalidates the PFT entry */
+static void remove_mcast_from_pft(struct nes_ud_file *file, u8 *addr)
+{
+ struct nes_device *nesdev = file->nesvnic->nesdev;
+ int i = 0;
+ u32 macaddr_low;
+ u32 orig_low;
+ u32 macaddr_high;
+ u32 prev_high;
+
+ for (i = 0; i < 48; i++) {
+ macaddr_low = nes_read_indexed(nesdev,
+ NES_IDX_PERFECT_FILTER_LOW + i*8);
+ orig_low = macaddr_low;
+ macaddr_high = nes_read_indexed(nesdev,
+ NES_IDX_PERFECT_FILTER_LOW + 4 + i*8);
+ if (!(macaddr_high & NES_MAC_ADDR_VALID))
+ continue;
+
+ if ((macaddr_high & 0xffff) != 0x0100)
+ continue;
+ if ((macaddr_low & 0xff) != addr[2])
+ continue;
+ macaddr_low >>= 8;
+ if ((macaddr_low & 0xff) != addr[1])
+ continue;
+ macaddr_low >>= 8;
+ if ((macaddr_low & 0xff) != addr[0])
+ continue;
+ macaddr_low >>= 8;
+ if ((macaddr_low & 0xff) != 0x5e)
+ continue;
+ /* hit - that means Linux or other UD set this bit earlier */
+ /* so remove the NIC from MAC address reception */
+ prev_high = macaddr_high;
+ macaddr_high = (macaddr_high & 0xfffcffff) &
+ ~((1<<file->nes_ud_nic_index) << 16);
+ nes_debug(NES_DBG_UD, "%s(%d) found addr to mcast remove,"
+ "i=%d, macaddr_high=0x%x macaddr_low=0x%x "
+ "nic_idx=%d prev_high=0x%x\n", __func__, __LINE__, i,
+ macaddr_high, orig_low, file->nes_ud_nic_index, prev_high);
+ nes_write_indexed(nesdev, NES_IDX_PERFECT_FILTER_LOW + 4 + i*8,
+ macaddr_high);
+ break;
+ }
+
+}
+
+/*
+* the function returns a mask of the NICs
+* assotiated with given multicast address
+*/
+static int nes_ud_mcast_filter(struct nes_vnic *nesvnic, __u8 *dmi_addr)
+{
+ int i = 0;
+ int ret = 0;
+ int mask = 0;
+ struct nes_ud_resources *pRsc;
+
+ pRsc = locate_ud_adapter(nesvnic->nesdev->nesadapter);
+ if (pRsc == NULL)
+ return 0;
+
+ for (i = 0; i < NES_UD_MCAST_TBL_SZ; i++) {
+ if (pRsc->mcast[i].in_use &&
+ pRsc->mcast[i].addr[0] == dmi_addr[3] &&
+ pRsc->mcast[i].addr[1] == dmi_addr[4] &&
+ pRsc->mcast[i].addr[2] == dmi_addr[5]) {
+ mask = (pRsc->mcast[i].owner->mcast_mode ==
+ NES_UD_MCAST_PFT_MODE) ?
+ pRsc->mcast[i].owner->nes_ud_nic_index : 0;
+
+ ret = ret | (1 << mask);
+ nes_debug(NES_DBG_UD, "mcast filter, "
+ "fpr=%02X%02X%02X ret=%d\n",
+ dmi_addr[3], dmi_addr[4], dmi_addr[5], ret);
+ }
+ }
+ if (ret == 0)
+ return -1;
+ else
+ return ret;
+
+}
+
+static __u32 mqueue_key[4] = { 0x0, 0x80, 0x0, 0x0 };
+
+static inline __u8 nes_ud_calculate_hash(__u8 dest_addr_lsb)
+{
+ __u8 in[8];
+ __u32 key_arr[4];
+ int i;
+ __u32 result = 0;
+ int j, k;
+ __u8 shift_in, next_shift_in;
+
+ in[0] = 0;
+ in[1] = 0;
+ in[2] = 0;
+ in[3] = 0;
+
+ in[4] = 0;
+
+ in[5] = 0;
+ in[6] = 0;
+ in[7] = dest_addr_lsb;
+
+
+
+ for (i = 0; i < 4; i++)
+ key_arr[3-i] = mqueue_key[i];
+
+
+
+ for (i = 0; i < 8; i++) {
+ for (j = 7; j >= 0; j--) {
+ if (in[i] & (1 << j))
+ result = result ^ key_arr[0];
+
+ shift_in = 0;
+ for (k = 3; k >= 0; k--) {
+ next_shift_in = key_arr[k] >> 31;
+ key_arr[k] = (key_arr[k] << 1) + shift_in;
+ shift_in = next_shift_in;
+ }
+ }
+ }
+ return result & 0x7f;
+}
+
+static inline void nes_ud_enable_mqueue(struct nes_ud_file *file)
+{
+ struct nes_device *nesdev = file->nesvnic->nesdev;
+ int mqueue_config0;
+ int mqueue_config2;
+ int instance = file->nes_ud_nic_index & 0x1;
+
+ mqueue_config0 = nes_read_indexed(nesdev, 0x6400);
+ mqueue_config0 |= (4 | (instance & 0x3)) << (file->nes_ud_nic_index*3);
+ nes_write_indexed(nesdev, 0x6400, mqueue_config0);
+ mqueue_config0 = nes_read_indexed(nesdev, 0x6400);
+
+ mqueue_config2 = nes_read_indexed(nesdev, 0x6408);
+ mqueue_config2 |= (2 << (instance*2)) | (6 << (instance*3+8));
+ nes_write_indexed(nesdev, 0x6408, mqueue_config2);
+ mqueue_config2 = nes_read_indexed(nesdev, 0x6408);
+
+ nes_write_indexed(nesdev, 0x64a0+instance*0x100, mqueue_key[0]);
+ nes_write_indexed(nesdev, 0x64a4+instance*0x100, mqueue_key[1]);
+ nes_write_indexed(nesdev, 0x64a8+instance*0x100, mqueue_key[2]);
+ nes_write_indexed(nesdev, 0x64ac+instance*0x100, mqueue_key[3]);
+
+ nes_debug(NES_DBG_UD, "mq_config0=0x%x mq_config2=0x%x nic_idx= %d\n",
+ mqueue_config0, mqueue_config2, file->nes_ud_nic_index);
+
+}
+
+
+
+static inline
+void nes_ud_redirect_from_mqueue(struct nes_ud_file *file, int num_queues)
+{
+ struct nes_device *nesdev = file->nesvnic->nesdev;
+ int instance = file->nes_ud_nic_index & 0x1;
+ unsigned addr = 0x6420+instance*0x100;
+ unsigned value;
+ int i;
+
+ value = (file->prio == NES_UD_DEV_PRIO_LOW || num_queues == 1) ?
+ 0x0 : 0x11111111;
+ for (i = 0; i < 16; i++)
+ nes_write_indexed(nesdev, addr+i*4, value);
+}
+
+
+static int nes_ud_create_nic(struct nes_ud_file *file)
+{
+ struct nes_vnic *nesvnic = file->nesvnic;
+ struct nes_device *nesdev = nesvnic->nesdev;
+ struct nes_hw_nic_qp_context *nic_context;
+ struct nes_hw_cqp_wqe *cqp_wqe;
+ struct nes_cqp_request *cqp_request;
+ unsigned long flags;
+ void *vmem;
+ dma_addr_t pmem;
+ u64 u64temp;
+ int ret = 0;
+
+ BUG_ON(file->nic_vbase != NULL);
+
+ file->nic_mem_size = 256 +
+ (NES_NIC_WQ_SIZE * sizeof(struct nes_hw_nic_sq_wqe)) +
+ sizeof(struct nes_hw_nic_qp_context);
+
+ file->nic_vbase = pci_alloc_consistent(nesdev->pcidev,
+ file->nic_mem_size,
+ &file->nic_pbase);
+ if (!file->nic_vbase) {
+ nes_debug(NES_DBG_UD, "Unable to allocate memory for NIC host "
+ "descriptor rings\n");
+ return -ENOMEM;
+ }
+
+ memset(file->nic_vbase, 0, file->nic_mem_size);
+
+ vmem = (void *)(((unsigned long long)file->nic_vbase + (256 - 1)) &
+ ~(unsigned long long)(256 - 1));
+ pmem = (dma_addr_t)(((unsigned long long)file->nic_pbase + (256 - 1)) &
+ ~(unsigned long long)(256 - 1));
+
+ file->wq_vbase = vmem;
+ file->wq_pbase = pmem;
+ file->head = 0;
+ file->tail = 0;
+
+ vmem += (NES_NIC_WQ_SIZE * sizeof(struct nes_hw_nic_sq_wqe));
+ pmem += (NES_NIC_WQ_SIZE * sizeof(struct nes_hw_nic_sq_wqe));
+
+ cqp_request = nesvnic->get_cqp_request(nesdev);
+ if (cqp_request == NULL) {
+ nes_debug(NES_DBG_QP, "Failed to get a cqp_request.\n");
+ goto fail_cqp_req_alloc;
+ }
+ cqp_request->waiting = 1;
+ cqp_wqe = &cqp_request->cqp_wqe;
+
+ cqp_wqe->wqe_words[NES_CQP_WQE_OPCODE_IDX] =
+ cpu_to_le32(NES_CQP_CREATE_QP | NES_CQP_QP_TYPE_NIC);
+ cqp_wqe->wqe_words[NES_CQP_WQE_ID_IDX] = cpu_to_le32(file->qpn);
+ cqp_wqe->wqe_words[NES_CQP_WQE_COMP_CTX_LOW_IDX] =
+ cpu_to_le32((u32)((u64)(&nesdev->cqp)));
+ cqp_wqe->wqe_words[NES_CQP_WQE_COMP_CTX_HIGH_IDX] =
+ cpu_to_le32((u32)(((u64)(&nesdev->cqp))>>32));
+ cqp_wqe->wqe_words[NES_CQP_WQE_COMP_SCRATCH_LOW_IDX] = 0;
+ cqp_wqe->wqe_words[NES_CQP_WQE_COMP_SCRATCH_HIGH_IDX] = 0;
+
+
+ nic_context = vmem;
+
+ nic_context->context_words[NES_NIC_CTX_MISC_IDX] =
+ cpu_to_le32((u32)NES_NIC_CTX_SIZE |
+ ((u32)PCI_FUNC(nesdev->pcidev->devfn) << 12) |
+ (1 << 18));
+
+ nic_context->context_words[NES_NIC_CTX_SQ_LOW_IDX] = 0;
+ nic_context->context_words[NES_NIC_CTX_SQ_HIGH_IDX] = 0;
+ nic_context->context_words[NES_NIC_CTX_RQ_LOW_IDX] = 0;
+ nic_context->context_words[NES_NIC_CTX_RQ_HIGH_IDX] = 0;
+
+ u64temp = (u64)file->wq_pbase;
+ if (file->queue_type == NES_UD_SEND_QUEUE) {
+ nic_context->context_words[NES_NIC_CTX_SQ_LOW_IDX] =
+ cpu_to_le32((u32)u64temp);
+ nic_context->context_words[NES_NIC_CTX_SQ_HIGH_IDX] =
+ cpu_to_le32((u32)(u64temp >> 32));
+ } else {
+ nic_context->context_words[NES_NIC_CTX_RQ_LOW_IDX] =
+ cpu_to_le32((u32)u64temp);
+ nic_context->context_words[NES_NIC_CTX_RQ_HIGH_IDX] =
+ cpu_to_le32((u32)(u64temp >> 32));
+ }
+
+ u64temp = (u64)pmem;
+
+ cqp_wqe->wqe_words[NES_CQP_QP_WQE_CONTEXT_LOW_IDX] =
+ cpu_to_le32((u32)u64temp);
+ cqp_wqe->wqe_words[NES_CQP_QP_WQE_CONTEXT_HIGH_IDX] =
+ cpu_to_le32((u32)(u64temp >> 32));
+
+ atomic_set(&cqp_request->refcount, 2);
+ nesvnic->post_cqp_request(nesdev, cqp_request);
+
+ /* Wait for CQP */
+ ret = wait_event_timeout(cqp_request->waitq,
+ (cqp_request->request_done != 0),
+ NES_EVENT_TIMEOUT);
+ if (!ret)
+ nes_debug(NES_DBG_UD, "NES_UD NIC QP%u "
+ "create timeout expired\n", file->qpn);
+
+
+ if (atomic_dec_and_test(&cqp_request->refcount)) {
+ if (cqp_request->dynamic) {
+ kfree(cqp_request);
+ } else {
+ spin_lock_irqsave(&nesdev->cqp.lock, flags);
+ list_add_tail(&cqp_request->list,
+ &nesdev->cqp_avail_reqs);
+ spin_unlock_irqrestore(&nesdev->cqp.lock, flags);
+ }
+ }
+ nes_debug(NES_DBG_UD, "Created NIC, qpn=%d, SQ/RQ pa=0x%p va=%p "
+ "virt_to_phys=%p\n", file->qpn,
+ (void *)file->wq_pbase, (void *)file->nic_vbase,
+ (void *)virt_to_phys(file->nic_vbase));
+ return ret;
+
+ fail_cqp_req_alloc:
+ pci_free_consistent(nesdev->pcidev, file->nic_mem_size, file->nic_vbase,
+ file->nic_pbase);
+ file->nic_vbase = NULL;
+ return -EFAULT;
+}
+
+
+static void nes_ud_destroy_nic(struct nes_ud_file *file)
+{
+ struct nes_vnic *nesvnic = file->nesvnic;
+ struct nes_device *nesdev = nesvnic->nesdev;
+ struct nes_hw_cqp_wqe *cqp_wqe;
+ struct nes_cqp_request *cqp_request;
+ unsigned long flags;
+ int ret = 0;
+
+ cqp_request = nesvnic->get_cqp_request(nesdev);
+ if (cqp_request == NULL) {
+ nes_debug(NES_DBG_QP, "Failed to get a cqp_request.\n");
+ return;
+ }
+ cqp_request->waiting = 1;
+ cqp_wqe = &cqp_request->cqp_wqe;
+
+ cqp_wqe->wqe_words[NES_CQP_WQE_OPCODE_IDX] =
+ cpu_to_le32(NES_CQP_DESTROY_QP | NES_CQP_QP_TYPE_NIC);
+ cqp_wqe->wqe_words[NES_CQP_WQE_ID_IDX] = cpu_to_le32(file->qpn);
+ cqp_wqe->wqe_words[NES_CQP_WQE_COMP_CTX_LOW_IDX] =
+ cpu_to_le32((u32)((u64)(&nesdev->cqp)));
+ cqp_wqe->wqe_words[NES_CQP_WQE_COMP_CTX_HIGH_IDX] =
+ cpu_to_le32((u32)(((u64)(&nesdev->cqp)) >> 32));
+ cqp_wqe->wqe_words[NES_CQP_WQE_COMP_SCRATCH_LOW_IDX] = 0;
+ cqp_wqe->wqe_words[NES_CQP_WQE_COMP_SCRATCH_HIGH_IDX] = 0;
+
+ atomic_set(&cqp_request->refcount, 2);
+ nesvnic->post_cqp_request(nesdev, cqp_request);
+
+ /* Wait for CQP */
+ ret = wait_event_timeout(cqp_request->waitq,
+ (cqp_request->request_done != 0),
+ NES_EVENT_TIMEOUT);
+ if (!ret)
+ nes_debug(NES_DBG_UD, "NES_UD NIC QP%u "
+ "destroy timeout expired\n", file->qpn);
+
+ if (atomic_dec_and_test(&cqp_request->refcount)) {
+ if (cqp_request->dynamic) {
+ kfree(cqp_request);
+ } else {
+ spin_lock_irqsave(&nesdev->cqp.lock, flags);
+ list_add_tail(&cqp_request->list,
+ &nesdev->cqp_avail_reqs);
+ spin_unlock_irqrestore(&nesdev->cqp.lock, flags);
+ }
+ }
+
+ pci_free_consistent(nesdev->pcidev, file->nic_mem_size, file->nic_vbase,
+ file->nic_pbase);
+ file->nic_vbase = NULL;
+ file->qp_ptr = NULL;
+
+ return;
+}
+
+static void nes_ud_free_resources(struct nes_ud_file *file)
+{
+ struct nes_device *nesdev = file->nesvnic->nesdev;
+ int nic_active = 0;
+ int mcast_all = 0;
+ int mcast_en = 0;
+ int wqm_config0 = 0;
+ wait_queue_head_t waitq;
+ int num_queues = 0;
+ nes_debug(NES_DBG_UD, " %s(%d) NAME=%s nes_ud_qpid=%d\n",
+ __func__, __LINE__, file->ifrn_name, file->qpn);
+
+ if (!file->nesvnic || !file->active)
+ return;
+
+ if (file->queue_type == NES_UD_SEND_QUEUE) {
+ nic_active = nes_read_indexed(nesdev, NES_IDX_NIC_ACTIVE);
+ nic_active &= ~(1 << file->nes_ud_nic_index);
+ nes_write_indexed(nesdev, NES_IDX_NIC_ACTIVE, nic_active);
+ nic_active = nes_read_indexed(nesdev, NES_IDX_NIC_ACTIVE);
+ } else {
+ num_queues = count_files_by_nic(file->nesvnic,
+ file->queue_type);
+
+ if (num_queues == 1) {
+
+ nic_active = nes_read_indexed(nesdev, NES_IDX_NIC_ACTIVE);
+ nic_active &= ~(1 << file->nes_ud_nic_index);
+ nes_write_indexed(nesdev, NES_IDX_NIC_ACTIVE, nic_active);
+ nic_active = nes_read_indexed(nesdev, NES_IDX_NIC_ACTIVE);
+
+ mcast_all = nes_read_indexed(nesdev, NES_IDX_NIC_MULTICAST_ALL);
+ mcast_all &= ~(1 << file->nes_ud_nic_index);
+ nes_write_indexed(nesdev, NES_IDX_NIC_MULTICAST_ALL, mcast_all);
+ mcast_all = nes_read_indexed(nesdev, NES_IDX_NIC_MULTICAST_ALL);
+
+ mcast_en = nes_read_indexed(nesdev,
+ NES_IDX_NIC_MULTICAST_ENABLE);
+ mcast_en &= ~(1 << file->nes_ud_nic_index);
+ nes_write_indexed(nesdev, NES_IDX_NIC_MULTICAST_ENABLE,
+ mcast_en);
+ mcast_en = nes_read_indexed(nesdev,
+ NES_IDX_NIC_MULTICAST_ENABLE);
+
+ nes_debug(NES_DBG_UD, "nic_active=0x%x, mcast_en=0x%x, "
+ "mcast_all=0x%x nic_index=%d num_queues=%d\n",
+ nic_active, mcast_en, mcast_all,
+ file->nes_ud_nic_index, num_queues);
+ }
+
+ nes_ud_redirect_from_mqueue(file, num_queues);
+ num_queues = count_files(file->nesvnic, file->queue_type);
+ if (num_queues == 1) {
+ nes_debug(NES_DBG_UD, "Last receive queue, "
+ "restoring MPP debug register\n");
+ nes_write_indexed(nesdev, 0xA00, 0x200);
+ nes_write_indexed(nesdev, 0xA40, 0x200);
+ }
+ }
+
+
+
+ nes_ud_destroy_nic(file);
+
+ if (file->queue_type == NES_UD_RECV_QUEUE) {
+ wqm_config0 = nes_read_indexed(nesdev, 0x5000);
+ wqm_config0 &= ~0x8000;
+ nes_write_indexed(nesdev, 0x5000, wqm_config0);
+
+ init_waitqueue_head(&waitq);
+
+ wait_event_timeout(waitq, 0, NES_UD_CLEANUP_TIMEOUT);
+
+ nes_debug(NES_DBG_UD, "%s(%d) enabling stall_no_wqes\n",
+ __func__, __LINE__);
+ wqm_config0 = nes_read_indexed(nesdev, 0x5000);
+ wqm_config0 |= 0x8000;
+ nes_write_indexed(nesdev, 0x5000, wqm_config0);
+ }
+
+ dev_put(file->nesvnic->netdev);
+
+ file->active = 0;
+
+ nes_debug(NES_DBG_UD, "%s(%d) done\n", __func__, __LINE__);
+}
+
+
+static int nes_ud_init_channel(struct nes_ud_file *file)
+{
+ struct nes_device *nesdev = NULL;
+ int ret = 0;
+ int nic_active = 0;
+ int mcast_all = 0;
+ int mcast_en = 0;
+ int link_ag = 0;
+ int mpp4_dbg = 0;
+
+ nesdev = file->nesvnic->nesdev;
+
+ ret = nes_ud_create_nic(file);
+ if (ret != 0)
+ return ret;
+
+ if (file->queue_type == NES_UD_RECV_QUEUE) {
+
+ file->nesvnic->mcrq_mcast_filter = nes_ud_mcast_filter;
+
+ mcast_en = nes_read_indexed(nesdev,
+ NES_IDX_NIC_MULTICAST_ENABLE);
+ mcast_en |= 1 << file->nes_ud_nic_index;
+ nes_write_indexed(nesdev, NES_IDX_NIC_MULTICAST_ENABLE,
+ mcast_en);
+ mcast_en = nes_read_indexed(nesdev,
+ NES_IDX_NIC_MULTICAST_ENABLE);
+
+ /* the only case when we use PFT is for single port
+ two functions, which probably would be the
+ most common usage model :), but anyway */
+ if (file->mcast_mode == NES_UD_MCAST_ALL_MODE) {
+ mcast_all = nes_read_indexed(nesdev, NES_IDX_NIC_MULTICAST_ALL);
+ mcast_all |= 1 << file->nes_ud_nic_index;
+ nes_write_indexed(nesdev, NES_IDX_NIC_MULTICAST_ALL, mcast_all);
+ mcast_all = nes_read_indexed(nesdev, NES_IDX_NIC_MULTICAST_ALL);
+ }
+ if (nesdev->nesadapter->port_count <= 2) {
+ link_ag = 0x00;
+ nes_write_indexed(nesdev, 0x6038, link_ag);
+ link_ag = nes_read_indexed(nesdev, 0x6038);
+ }
+ if (nesdev->nesadapter->netdev_count <= 2)
+ nes_ud_enable_mqueue(file);
+
+ nes_write_indexed(nesdev, 0xA00, 0x245);
+ nes_write_indexed(nesdev, 0xA40, 0x245);
+
+ }
+ /* NES_UD_SEND_QUEUE */
+ else {
+ mpp4_dbg = nes_read_indexed(nesdev, 0xb00);
+ mpp4_dbg |= 1 << 12;
+ nes_write_indexed(nesdev, 0xb00, mpp4_dbg);
+ mpp4_dbg = nes_read_indexed(nesdev, 0xb00);
+ }
+
+ nic_active = nes_read_indexed(nesdev, NES_IDX_NIC_ACTIVE);
+ nic_active |= 1 << file->nes_ud_nic_index;
+ nes_write_indexed(nesdev, NES_IDX_NIC_ACTIVE, nic_active);
+ nic_active = nes_read_indexed(nesdev, NES_IDX_NIC_ACTIVE);
+
+ nes_debug(NES_DBG_UD, "nic_active=0x%x, mcast_en=0x%x, "
+ "mcast_all=0x%x nic_index=%d link_ag=0x%x mpp4_dbg=0x%x\n",
+ nic_active, mcast_en, mcast_all, file->nes_ud_nic_index,
+ link_ag, mpp4_dbg);
+
+ return ret;
+}
+
+static struct nes_ud_file *nes_ud_get_nxt_channel(struct nes_vnic *nesvnic,
+ enum nes_ud_queue_type queue_type)
+{
+ struct nes_ud_file *file = NULL;
+ struct net_device *netdev = NULL;
+ struct nes_device *nesdev = NULL;
+ struct nes_ud_resources *pRsc;
+
+ netdev = nesvnic->netdev;
+ nesdev = nesvnic->nesdev;
+
+ pRsc = locate_ud_adapter(nesdev->nesadapter);
+ if (pRsc == NULL) {
+ pRsc = allocate_ud_adapter(nesdev->nesadapter);
+ if (pRsc == NULL)
+ return NULL;
+
+ }
+ if (pRsc->num_logport_confed == 0) {
+ pRsc->original_60b8 = nes_read_indexed(nesdev, 0x60b8);
+ pRsc->original_6000 = nes_read_indexed(nesdev, 0x6000);
+ /* everything goes to port 0x0 */
+ if ((nesvnic->nesdev->nesadapter->port_count == 1) ||
+ (nes_drv_opt & NES_DRV_OPT_MCAST_LOGPORT_MAP)) {
+ /* single port card or dual port using single if */
+ pRsc->num_logport_confed = 0x3;
+ pRsc->logport_2_map = 0x0;
+ pRsc->logport_3_map = 0x0;
+ nes_write_indexed(nesdev, 0x60b8, 0x3);
+ nes_write_indexed(nesdev, 0x6000, 0x0);
+ } else {
+ pRsc->num_logport_confed = 0x3;
+ pRsc->logport_2_map = 0x0;
+ pRsc->logport_3_map = 0x1;
+ }
+ nes_debug(NES_DBG_UD, "%s(%d) num_logport_confed=%d "
+ "original_6000=%d logport_3_map = %d nes_drv_opt=%x\n",
+ __func__, __LINE__, pRsc->num_logport_confed,
+ pRsc->original_6000, pRsc->logport_3_map, nes_drv_opt);
+ }
+
+ nes_debug(NES_DBG_UD, "%s(%d) logport_2_map=%d logport_3_map=%d\n",
+ __func__, __LINE__, pRsc->logport_2_map, pRsc->logport_3_map);
+
+ file = allocate_nic_queue(nesvnic, queue_type);
+ if (file == NULL) {
+ nes_debug(NES_DBG_UD, "%s(%d) failed to allocate NIC\n",
+ __func__, __LINE__);
+ return NULL;
+ }
+
+ file->active = 1;
+ memcpy(file->ifrn_name, netdev->name, IFNAMSIZ);
+
+ /* for now use pft always */
+ file->mcast_mode = NES_UD_MCAST_PFT_MODE;
+
+ nes_debug(NES_DBG_UD, " %s(%d) NAME=%s qpn=%d nes_ud_nic_index=%d "
+ "nes_ud_nic.qp_id=%d mcast_mode=%d port_count=%d "
+ "netdev_count=%d\n", __func__, __LINE__, file->ifrn_name,
+ file->qpn, file->nes_ud_nic_index, file->nesvnic->mcrq_qp_id,
+ file->mcast_mode, nesdev->nesadapter->port_count,
+ nesdev->nesadapter->netdev_count);
+
+ file->mss = netdev->mtu-28;
+ pRsc->num_allocated_nics++;
+ BUG_ON(pRsc->num_allocated_nics > 8);
+
+ return file;
+
+}
+
+static struct nes_ud_mem_region *nes_ud_allocate_mr(u32 npages)
+{
+ struct nes_ud_mem_region *mr = NULL;
+
+ mr = vmalloc(sizeof(*mr));
+ if (mr == NULL)
+ return NULL;
+
+
+ mr->addrs = vmalloc(npages * sizeof(dma_addr_t));
+ if (!mr->addrs) {
+ nes_debug(NES_DBG_UD, "%s(%d) Cannot allocate mr struct "
+ "for %d pages\n", __func__, __LINE__, npages);
+ vfree(mr);
+ return NULL;
+ }
+ mr->pg_cnt = npages;
+ mr->in_use = 1;
+
+ INIT_LIST_HEAD(&mr->list);
+
+ return mr;
+}
+
+static void nes_ud_free_mr(struct nes_ud_mem_region *mr)
+{
+ if (mr->addrs != NULL)
+ vfree(mr->addrs);
+
+ vfree(mr);
+}
+
+/* nes_ud_get_hash_entry()
+ *
+ * function returns a key for hash table
+ */
+static inline
+int nes_ud_get_hash_entry(u32 stag)
+{
+ return stag & 0xff;
+}
+
+
+/* nes_ud_lookup_mr()
+ *
+ * function returns a pointer to mr realized by specific STAG
+ */
+static inline
+struct nes_ud_mem_region *nes_ud_lookup_mr(u32 stag)
+{
+ int key;
+ struct nes_ud_mem_region *mr;
+
+ key = nes_ud_get_hash_entry(stag);
+
+ list_for_each_entry(mr, &ud_mem.mrs[key].list, list) {
+ ud_mem.mrs[key].read_stats++;
+ if (mr->stag == stag)
+ return mr;
+
+ }
+ return NULL;
+}
+
+/* nes_ud_add_mr_hash()
+ *
+ * the function inserts the mr entry into the hash list
+ * the stag is a key
+ */
+static inline
+int nes_ud_add_mr_hash(struct nes_ud_mem_region *mr)
+{
+ int key;
+
+ /* first check if the stag is unique */
+ if (nes_ud_lookup_mr(mr->stag) != NULL) {
+ nes_debug(NES_DBG_UD, "%s(%d) double STAG error stag=%x\n",
+ __func__, __LINE__, mr->stag);
+ return -1;
+ }
+ key = nes_ud_get_hash_entry(mr->stag);
+
+ /* structure is global so mutexes are necessary */
+ mutex_lock(&ud_mem.mutex);
+
+ /* add mr to the list at start */
+ list_add(&mr->list, &ud_mem.mrs[key].list);
+
+ mutex_unlock(&ud_mem.mutex);
+
+ return 0;
+
+}
+
+/* nes_ud_del_mr()
+ *
+ * the function removes the entry from the hash list
+ * the stag is the key
+ */
+static inline
+void nes_ud_del_mr(struct nes_ud_mem_region *mr)
+{
+ /* structure is global so mutexes are necessary */
+ mutex_lock(&ud_mem.mutex);
+
+ list_del(&mr->list);
+
+ /* init entry */
+ INIT_LIST_HEAD(&mr->list);
+
+ mutex_unlock(&ud_mem.mutex);
+}
+
+/* nes_ud_cleanup_mr()
+ *
+ * function deletes and and frees all hash entries
+ */
+static inline
+void nes_ud_cleanup_mr(void)
+{
+ struct nes_ud_mem_region *mr;
+ struct nes_ud_mem_region *next;
+ int i;
+
+ /* structure is global so mutexes are necessary */
+ mutex_lock(&ud_mem.mutex);
+
+ for (i = 0; i < NES_UD_MAX_REG_HASH_CNT; i++) {
+ if (list_empty(&ud_mem.mrs[i].list))
+ continue;
+
+ list_for_each_entry_safe(mr, next, &ud_mem.mrs[i].list, list) {
+ nes_debug(NES_DBG_UD, "%s(%d) non free stag=%x\n",
+ __func__, __LINE__, mr->stag);
+ list_del_init(&mr->list);
+
+ nes_ud_free_mr(mr);
+ }
+ }
+
+ mutex_unlock(&ud_mem.mutex);
+}
+
+u32 nes_ud_reg_mr(struct ib_umem *region, u64 length, u64 virt, u32 stag)
+{
+ unsigned long npages =
+ PAGE_ALIGN(region->length + region->offset) >> PAGE_SHIFT;
+ struct nes_ud_mem_region *mr = nes_ud_allocate_mr(npages);
+ struct ib_umem_chunk *chunk;
+ dma_addr_t page;
+ u32 chunk_pages = 0;
+ int nmap_index;
+ int i = 0;
+ int mr_id = 0;
+ nes_debug(NES_DBG_UD, "%s(%d) mr=%p length=%d virt=%p\n",
+ __func__, __LINE__, mr, (int)length, (void *)virt);
+ if (!mr)
+ return 0;
+
+
+ mr->stag = stag;
+
+ mr->va = virt;
+ mr->length = length;
+ list_for_each_entry(chunk, ®ion->chunk_list, list) {
+ for (nmap_index = 0; nmap_index < chunk->nmap; ++nmap_index) {
+ page = sg_dma_address(&chunk->page_list[nmap_index]);
+ chunk_pages = sg_dma_len(&chunk->page_list[nmap_index]) >> 12;
+ if (page & ~PAGE_MASK)
+ goto reg_user_mr_err;
+ if (!chunk_pages)
+ goto reg_user_mr_err;
+
+ for (i = 0; i < chunk_pages; i++) {
+ mr->addrs[mr_id] = page;
+ page += PAGE_SIZE;
+ if (++mr_id > npages)
+ goto reg_user_mr_err;
+ }
+ }
+ }
+ nes_debug(NES_DBG_UD, "%s(%d) stag=0x%x mr_id=%d npages=%d\n",
+ __func__, __LINE__, stag, mr_id, (int)npages);
+ nes_ud_add_mr_hash(mr);
+ return stag;
+
+reg_user_mr_err:
+ if (mr)
+ nes_ud_free_mr(mr);
+
+ return 0;
+}
+
+
+int nes_ud_dereg_mr(u32 stag)
+{
+ struct nes_ud_mem_region *mr = NULL;
+
+ nes_debug(NES_DBG_UD, "%s(%d) stag=0x%x\n", __func__, __LINE__, stag);
+
+ mr = nes_ud_lookup_mr(stag);
+ if (mr != NULL) {
+ nes_ud_del_mr(mr);
+ nes_ud_free_mr(mr);
+ } else {
+ nes_debug(NES_DBG_UD, "%s(%d) unknown stag=0x%x\n",
+ __func__, __LINE__, stag);
+ }
+
+ nes_debug(NES_DBG_UD, "%s(%d) done\n", __func__, __LINE__);
+ return 0;
+}
+
+
+int nes_ud_unsubscribe_mcast(struct nes_ud_file *file, union ib_gid *gid)
+{
+ int ret = 0;
+ int i;
+ struct nes_ud_resources *pRsc;
+
+ if (file->queue_type == NES_UD_SEND_QUEUE)
+ return -EFAULT;
+
+ pRsc = locate_ud_adapter(file->nesvnic->nesdev->nesadapter);
+ if (pRsc == NULL)
+ return -EFAULT;
+
+ for (i = 0; i < NES_UD_MCAST_TBL_SZ; i++) {
+ if (pRsc->mcast[i].in_use &&
+ pRsc->mcast[i].owner == file &&
+ pRsc->mcast[i].addr[0] == gid->raw[13] &&
+ pRsc->mcast[i].addr[1] == gid->raw[14] &&
+ pRsc->mcast[i].addr[2] == gid->raw[15]) {
+ pRsc->mcast[i].in_use = 0;
+ goto out;
+ }
+ }
+
+ ret = -EFAULT;
+out:
+ nes_debug(NES_DBG_UD, "%s(%d) %2.2X:%2.2X:%2.2X:%2.2X:%2.2X:%2.2X \
+ ret=%d mcast=%d\n", __func__, __LINE__, gid->raw[10],
+ gid->raw[11], gid->raw[12], gid->raw[13], gid->raw[14],
+ gid->raw[15], ret , i);
+ return ret;
+
+}
+
+/* function returns a number of allocated multicast entries in given adapter */
+static int get_mcast_number_alloced(struct nes_ud_resources *pRsc)
+{
+ int i;
+ int no = 0;
+
+ for (i = 0; i < NES_UD_MCAST_TBL_SZ; i++) {
+ if (pRsc->mcast[i].in_use != 0)
+ no++;
+
+ }
+ return no;
+}
+
+/* function subscribe a multicast group in the system - PFT modification */
+int nes_ud_subscribe_mcast(struct nes_ud_file *file, union ib_gid *gid)
+{
+ struct nes_device *nesdev = file->nesvnic->nesdev;
+ int ret = 0;
+ int i;
+ __u8 hash_idx = 0;
+ __u8 instance = file->nes_ud_nic_index & 0x1;
+ unsigned addr = 0;
+ unsigned mqueue_ind_tbl;
+ struct nes_ud_resources *pRsc;
+
+ struct net_device *netdev = file->nesvnic->netdev;
+ struct dev_mc_list *mc_list;
+ int multicast_address_exist = 0;
+
+
+ if (file->queue_type == NES_UD_SEND_QUEUE)
+ return -EFAULT;
+
+ pRsc = locate_ud_adapter(nesdev->nesadapter);
+ if (pRsc == NULL)
+ return -EFAULT;
+
+ for (mc_list = netdev->mc_list;
+ mc_list != NULL;
+ mc_list = mc_list->next) {
+ if (mc_list != NULL) {
+ if ((mc_list->dmi_addr[3] == gid->raw[13]) &&
+ (mc_list->dmi_addr[4] == gid->raw[14]) &&
+ (mc_list->dmi_addr[5] == gid->raw[15]) &&
+ (mc_list->dmi_addr[0] == 0x01) &&
+ (mc_list->dmi_addr[1] == 0) &&
+ (mc_list->dmi_addr[2] == 0x5e)) {
+ multicast_address_exist = 1;
+ break;
+ }
+ } else {
+ break;
+ }
+ }
+
+ if (multicast_address_exist == 0) {
+ nes_debug(NES_DBG_UD, "WARNING: multicast address not exist "
+ "on multicast list\n");
+ return -EFAULT;
+ }
+
+ /* first check that we have not subecribed to this mcast address, yet */
+ for (i = 0; i < NES_UD_MCAST_TBL_SZ; i++) {
+ if ((pRsc->mcast[i].in_use > 0) &&
+ (pRsc->mcast[i].addr[0] == gid->raw[13]) &&
+ (pRsc->mcast[i].addr[1] == gid->raw[14]) &&
+ (pRsc->mcast[i].addr[2] == gid->raw[15])) {
+ if (pRsc->mcast[i].owner == file) {
+ nes_debug(NES_DBG_UD, "WARNING - subscribing "
+ "mcast to the same nes_ud more than once\n");
+ break;
+ } else {
+ /* receiving the same multicast on different NICs is allowed:
+ 1. when two different NICS are used
+ 2. exactly one QP exists on this adapter
+ 3. The existing QP was allocated as first
+ or the second in the system
+ */
+ if (pRsc->mcast[i].owner->nes_ud_nic_index !=
+ file->nes_ud_nic_index) {
+ if (get_mcast_number_alloced(pRsc) == 1) {
+ if ((i == 0) || (i == 1)) {
+ /* add the mask of other nics
+ that subscribe this address */
+ break;
+ }
+ }
+ }
+ nes_debug(NES_DBG_UD, "ERROR - subscribing same mcast "
+ "to the diff nes_ud's and NIC owner_idx = %d "
+ "file_idx = %d\n",
+ pRsc->mcast[i].owner->nes_ud_nic_index,
+ file->nes_ud_nic_index);
+ ret = -EFAULT;
+ }
+ goto out;
+ }
+ }
+
+ for (i = 0; i < NES_UD_MCAST_TBL_SZ; i++) {
+ if (!pRsc->mcast[i].in_use) {
+ pRsc->mcast[i].addr[0] = gid->raw[13];
+ pRsc->mcast[i].addr[1] = gid->raw[14];
+ pRsc->mcast[i].addr[2] = gid->raw[15];
+ pRsc->mcast[i].owner = file;
+ pRsc->mcast[i].in_use = 1;
+
+ hash_idx =
+ nes_ud_calculate_hash(pRsc->mcast[i].addr[2]);
+
+ addr = 0x6420 + ((hash_idx >> 3) << 2) + instance*0x100;
+ mqueue_ind_tbl = nes_read_indexed(nesdev, addr);
+ if (file->prio == NES_UD_DEV_PRIO_HIGH)
+ mqueue_ind_tbl &= ~(1 << ((hash_idx & 0x7)*4));
+ else
+ mqueue_ind_tbl |= 1 << ((hash_idx & 0x7)*4);
+
+ nes_write_indexed(nesdev, addr, mqueue_ind_tbl);
+ mqueue_ind_tbl = nes_read_indexed(nesdev, addr);
+
+ nes_debug(NES_DBG_UD, "%s(%d) addr=0x%x "
+ "mqueue_ind_tbl=0x%x hash=0x%x, mac=0x%x\n",
+ __func__, __LINE__, addr, mqueue_ind_tbl,
+ hash_idx, pRsc->mcast[i].addr[2]);
+ /* take care of the case when linux join_mcast
+ is called before mcast_attach in that case our pft
+ will already be programmed with that mcast address,
+ just with wrong NIC we need just to find an address,
+ and fix the NIC additionally the mask with other NICs
+ that subscribed the address are added*/
+
+ mcast_fix_filter_table(file);
+ goto out;
+ }
+ }
+ ret = -EFAULT;
+
+out:
+
+ nes_debug(NES_DBG_UD, "%s(%d) %2.2X:%2.2X:%2.2X:%2.2X:%2.2X:%2.2X \
+ ret=%d\n", __func__, __LINE__, gid->raw[10], gid->raw[11],
+ gid->raw[12], gid->raw[13], gid->raw[14], gid->raw[15], ret);
+
+ return ret;
+}
+
+
+static inline
+int nes_ud_post_recv(struct nes_ud_file *file,
+ u32 adap_no,
+ struct nes_ud_recv_wr *nes_ud_wr)
+{
+ struct nes_hw_nic_rq_wqe *nic_rqe;
+ struct nes_hw_nic_rq_wqe *rq_vbase =
+ (struct nes_hw_nic_rq_wqe *)file->wq_vbase;
+ struct nes_device *nesdev = file->nesvnic->nesdev;
+ u16 *wqe_fragment_length = NULL;
+ u32 mr_offset;
+ u32 page_offset;
+ u32 page_id;
+ struct nes_ud_mem_region *mr = NULL;
+ int remaining_length = 0;
+ int wqe_fragment_index = 0;
+ int err = 0;
+ int i = 0;
+ struct nes_ud_resources *pRsc;
+
+ /* check if qp is activated */
+ if (file->active == 0)
+ return -EFAULT;
+
+ pRsc = &nes_ud_rsc[adap_no];
+
+ /* let's assume for now that max sge count is 1 */
+ for (i = 0; i < nes_ud_wr->wr_cnt; i++) {
+ nic_rqe = &rq_vbase[file->head];
+
+ mr = nes_ud_lookup_mr(nes_ud_wr->sg_list[i].lkey);
+ if (mr == NULL)
+ return -EFAULT;
+
+
+ if (mr->va > nes_ud_wr->sg_list[i].addr ||
+ (nes_ud_wr->sg_list[i].addr + nes_ud_wr->sg_list[i].length >
+ mr->va + mr->length)) {
+ err = -EFAULT;
+ goto out;
+ }
+
+ mr_offset = nes_ud_wr->sg_list[i].addr - mr->va;
+ page_offset = nes_ud_wr->sg_list[i].addr & ~PAGE_MASK;
+ page_id = ((mr->va & ~PAGE_MASK) + mr_offset) >> PAGE_SHIFT;
+
+ wqe_fragment_length =
+ (u16 *)&nic_rqe->wqe_words[NES_NIC_RQ_WQE_LENGTH_1_0_IDX];
+
+ remaining_length = nes_ud_wr->sg_list[i].length;
+ wqe_fragment_index = 0;
+
+ while (remaining_length > 0) {
+ if (wqe_fragment_index >= 4) {
+ err = -EFAULT;
+ goto out;
+ }
+
+ set_wqe_64bit_value(nic_rqe->wqe_words,
+ NES_NIC_RQ_WQE_FRAG0_LOW_IDX + 2*wqe_fragment_index,
+ mr->addrs[page_id]+page_offset);
+
+ wqe_fragment_length[wqe_fragment_index] =
+ cpu_to_le16(PAGE_SIZE - page_offset);
+
+ remaining_length -= PAGE_SIZE - page_offset;
+ page_offset = 0;
+ page_id++;
+ wqe_fragment_index++;
+ }
+
+ nes_write32(nesdev->regs+NES_WQE_ALLOC, (1 << 24) | file->qpn);
+
+ file->head = (file->head+1) & ~NES_NIC_WQ_SIZE;
+ }
+out:
+ return err;
+}
+
+static inline
+int nes_ud_post_send(struct nes_ud_file *file,
+ u32 adap_no,
+ struct nes_ud_send_wr *nes_ud_wr)
+{
+ struct nes_hw_nic_sq_wqe *nic_sqe;
+ struct nes_hw_nic_sq_wqe *sq_vbase =
+ (struct nes_hw_nic_sq_wqe *)file->wq_vbase;
+ struct nes_device *nesdev = file->nesvnic->nesdev;
+ u16 *wqe_fragment_length = NULL;
+ u32 mr_offset;
+ u32 page_offset;
+ u32 page_id;
+ struct nes_ud_mem_region *mr = NULL;
+ int remaining_length = 0;
+ int wqe_fragment_index = 0;
+ int err = 0;
+ int misc_flags = NES_NIC_SQ_WQE_COMPLETION;
+ int i = 0;
+ struct nes_ud_resources *pRsc;
+
+ /* check if qp is activated */
+ if (file->active == 0)
+ return -EFAULT;
+
+ pRsc = &nes_ud_rsc[adap_no];
+
+ /* check if is not set checksum */
+ if (!(nes_ud_wr->flags & IB_SEND_IP_CSUM))
+ misc_flags |= NES_NIC_SQ_WQE_DISABLE_CHKSUM;
+
+ /* let's assume for now that max sge count is 1 */
+ for (i = 0; i < nes_ud_wr->wr_cnt; i++) {
+ nic_sqe = &sq_vbase[file->head];
+
+ mr = nes_ud_lookup_mr(nes_ud_wr->sg_list[i].lkey);
+ if (mr == NULL)
+ return -EFAULT;
+
+
+ if ((mr->va > nes_ud_wr->sg_list[i].addr) ||
+ (nes_ud_wr->sg_list[i].addr+nes_ud_wr->sg_list[i].length >
+ mr->va + mr->length)) {
+
+ err = -EFAULT;
+ goto out;
+ }
+
+ mr_offset = nes_ud_wr->sg_list[i].addr - mr->va;
+ page_offset = nes_ud_wr->sg_list[i].addr & ~PAGE_MASK;
+ page_id = ((mr->va & ~PAGE_MASK) + mr_offset) >> PAGE_SHIFT;
+
+ wqe_fragment_length =
+ (u16 *)&nic_sqe->wqe_words[NES_NIC_SQ_WQE_LENGTH_0_TAG_IDX];
+
+ wqe_fragment_length++; /* skip vlan tag */
+ remaining_length = nes_ud_wr->sg_list[i].length;
+ wqe_fragment_index = 0;
+
+ while (remaining_length > 0) {
+ if (wqe_fragment_index >= 4) {
+ err = -EFAULT;
+ goto out;
+ }
+ set_wqe_64bit_value(nic_sqe->wqe_words,
+ NES_NIC_SQ_WQE_FRAG0_LOW_IDX +
+ 2*wqe_fragment_index,
+ mr->addrs[page_id]+page_offset);
+ wqe_fragment_length[wqe_fragment_index] =
+ cpu_to_le16(PAGE_SIZE - page_offset);
+ remaining_length -= PAGE_SIZE - page_offset;
+ page_offset = 0;
+ page_id++;
+ wqe_fragment_index++;
+ }
+ nic_sqe->wqe_words[NES_IWARP_SQ_WQE_TOTAL_PAYLOAD_IDX] =
+ cpu_to_le32(nes_ud_wr->sg_list[i].length);
+ nic_sqe->wqe_words[NES_NIC_SQ_WQE_MISC_IDX] =
+ cpu_to_le32(misc_flags);
+
+ nes_write32(nesdev->regs+NES_WQE_ALLOC,
+ (1 << 24) | (1 << 23) | file->qpn);
+
+ file->head = (file->head+1) & ~NES_NIC_WQ_SIZE;
+ }
+out:
+ return err;
+}
+
+
+
+static void nes_ud_mcast_cleanup_work(struct nes_ud_file *file)
+{
+ int i = 0;
+ int num_queues = count_files_by_nic(file->nesvnic, file->queue_type);
+ struct nes_ud_resources *pRsc;
+
+ pRsc = locate_ud_adapter(file->nesvnic->nesdev->nesadapter);
+ if (pRsc == NULL)
+ return;
+
+
+ nes_debug(NES_DBG_UD, "%s(%d) file->rsc_idx=%d\n",
+ __func__, __LINE__, file->rsc_idx);
+
+ mutex_lock(&pRsc->mutex);
+ for (i = 0; i < NES_UD_MCAST_TBL_SZ; i++) {
+ if (pRsc->mcast[i].owner == file) {
+ nes_debug(NES_DBG_UD, "%s(%d) mcast cleared idx=%d "
+ "%2.2X:%2.2X:%2.2X\n", __func__, __LINE__,
+ i, pRsc->mcast[i].addr[0],
+ pRsc->mcast[i].addr[1],
+ pRsc->mcast[i].addr[2]);
+
+ pRsc->mcast[i].in_use = 0;
+ remove_mcast_from_pft(file, pRsc->mcast[i].addr);
+ }
+ }
+
+ if (del_rsc_list(file) == 0) {
+ if (num_queues == 1)
+ file->nesvnic->mcrq_mcast_filter = NULL;
+
+ }
+ mutex_unlock(&pRsc->mutex);
+}
+
+struct nes_ud_file *nes_ud_create_wq(struct nes_vnic *nesvnic, int isrecv)
+{
+ struct nes_ud_file *file;
+ int ret = 0;
+ file = nes_ud_get_nxt_channel(nesvnic, (isrecv) ?
+ NES_UD_RECV_QUEUE : NES_UD_SEND_QUEUE);
+ if (!file)
+ return NULL;
+
+
+ ret = nes_ud_init_channel(file);
+ if (ret != 0) {
+ del_rsc_list(file);
+ return NULL;
+ }
+
+ dev_hold(file->nesvnic->netdev);
+
+ nes_debug(NES_DBG_UD, "%s(%d) file=%p\n", __func__, __LINE__, file);
+ return file;
+}
+
+
+
+int nes_ud_destroy_wq(struct nes_ud_file *file)
+{
+ struct nes_ud_resources *pRsc;
+ int count = 0;
+ int i;
+ pRsc = locate_ud_adapter(file->nesvnic->nesdev->nesadapter);
+ if (pRsc == NULL)
+ return -EFAULT;
+
+ if (file->active) {
+ nes_ud_mcast_cleanup_work(file);
+ nes_ud_free_resources(file);
+ }
+
+ /* check if the the adapter has any queues */
+ for (i = 0; i < NES_UD_MAX_NIC_CNT; i++) {
+ if (pRsc->nics[i].file->active != 0)
+ count++;
+
+ }
+ if (count == 0) {
+ nes_debug(NES_DBG_UD, "%s(%d) adapter %d "
+ "is ready to next use\n",
+ __func__, __LINE__, pRsc->adapter_no);
+ pRsc->pAdap = NULL;
+ }
+ nes_debug(NES_DBG_UD, "%s(%d) done\n", __func__, __LINE__);
+ return 0;
+}
+
+
+struct nes_ud_sksq_file {
+ unsigned long shared_page;
+ struct nes_ud_file *nes_ud_send_file;
+ struct nes_ud_file *nes_ud_recv_file;
+};
+
+static ssize_t nes_ud_sksq_write(struct file *filp, const char __user *buf,
+ size_t len, loff_t *pos)
+{
+ struct nes_ud_sksq_file *file = filp->private_data;
+ struct nes_ud_send_wr *nes_ud_wr =
+ (struct nes_ud_send_wr *)file->shared_page;
+ u32 adap_no;
+ u32 nic_no;
+
+ nic_no = ((nes_ud_wr->qpn >> 16) & 0x0f00) >> 8;
+ adap_no = ((nes_ud_wr->qpn >> 16) & 0xf000) >> 12;
+ if (unlikely(!file->nes_ud_send_file)) {
+ struct nes_ud_file *nes_ud_file = NULL;
+
+ nes_ud_file = nes_ud_rsc[adap_no].nics[nic_no].file;
+ /* the nic must be active and previously activated */
+ if ((nes_ud_file->active == 0) ||
+ (nes_ud_file->qpn != ((nes_ud_wr->qpn >> 16) & 0xff)))
+ return -EAGAIN;
+
+ file->nes_ud_send_file = nes_ud_file;
+ nes_debug(NES_DBG_UD, "send shared page addr = %p "
+ "adap_no = %d nic_no=%d qpn=%x\n",
+ nes_ud_wr, adap_no, nic_no, nes_ud_wr->qpn);
+ }
+ return nes_ud_post_send(file->nes_ud_send_file, adap_no, nes_ud_wr);
+
+}
+
+static ssize_t nes_ud_sksq_read(struct file *filp, char __user *buf,
+ size_t len, loff_t *pos)
+{
+ struct nes_ud_sksq_file *file = filp->private_data;
+ struct nes_ud_recv_wr *nes_ud_recv_wr;
+ u32 adap_no;
+ u32 nic_no;
+
+ nes_ud_recv_wr = (struct nes_ud_recv_wr *)(file->shared_page+2048);
+ adap_no = (nes_ud_recv_wr->qpn & 0xf000) >> 12;
+ nic_no = (nes_ud_recv_wr->qpn & 0x0f00) >> 8;
+
+ if (unlikely(!file->nes_ud_recv_file)) {
+ struct nes_ud_file *nes_ud_file = NULL;
+
+ nes_ud_file = nes_ud_rsc[adap_no].nics[nic_no].file;
+ /* the nic must be active and previously activated */
+ if ((nes_ud_file->active == 0) ||
+ (nes_ud_file->qpn != (nes_ud_recv_wr->qpn & 0xff)))
+ return -EAGAIN;
+
+ file->nes_ud_recv_file = nes_ud_file;
+ nes_debug(NES_DBG_UD, "recv shared page addr = %p "
+ "adap_no = %d nic_no=%d qpn=%x\n",
+ nes_ud_recv_wr, adap_no, nic_no, nes_ud_recv_wr->qpn);
+ }
+ return nes_ud_post_recv(file->nes_ud_recv_file,
+ adap_no, nes_ud_recv_wr);
+}
+
+static int nes_ud_sksq_mmap(struct file *filp, struct vm_area_struct *vma)
+{
+ struct nes_ud_sksq_file *file = filp->private_data;
+
+ nes_debug(NES_DBG_UD, "shared mem pgprot_val(prot)=0x%x pa=%p\n",
+ (unsigned int)pgprot_val(vma->vm_page_prot),
+ (void *)virt_to_phys((void *)file->shared_page));
+ if (remap_pfn_range(vma, vma->vm_start,
+ virt_to_phys((void *)file->shared_page) >> PAGE_SHIFT,
+ PAGE_SIZE, vma->vm_page_prot)) {
+ printk(KERN_ERR "remap_pfn_range failed.\n");
+ return -EAGAIN;
+ }
+ return 0;
+}
+
+
+static int nes_ud_sksq_open(struct inode *inode, struct file *filp)
+{
+ struct nes_ud_sksq_file *file;
+
+ file = kmalloc(sizeof *file, GFP_KERNEL);
+ if (!file)
+ return -ENOMEM;
+
+ memset(file, 0, sizeof *file);
+ nes_debug(NES_DBG_UD, "%s(%d) file=%p\n",
+ __func__, __LINE__, file);
+
+ filp->private_data = file;
+ file->nes_ud_send_file = NULL;
+ file->nes_ud_recv_file = NULL;
+
+ file->shared_page = __get_free_page(GFP_USER);
+ return 0;
+}
+
+static int nes_ud_sksq_close(struct inode *inode, struct file *filp)
+{
+
+ struct nes_ud_sksq_file *file = filp->private_data;
+
+ if (file->shared_page) {
+ free_page(file->shared_page);
+ file->shared_page = 0;
+ }
+ kfree(file);
+ return 0;
+}
+
+static const struct file_operations nes_ud_sksq_fops = {
+ .owner = THIS_MODULE,
+ .open = nes_ud_sksq_open,
+ .release = nes_ud_sksq_close,
+ .write = nes_ud_sksq_write,
+ .read = nes_ud_sksq_read,
+ .mmap = nes_ud_sksq_mmap,
+};
+
+
+static struct miscdevice nes_ud_sksq_misc = {
+ .minor = MISC_DYNAMIC_MINOR,
+ .name = "nes_ud_sksq",
+ .fops = &nes_ud_sksq_fops,
+};
+
+/*
+ * function replaces the CQ pointer in QP stored in the file
+ * the QP must have a valid CQ pointers assotiated with it
+ */
+int nes_ud_cq_replace(struct nes_vnic *nesvnic, struct nes_cq *cq)
+{
+ u32 cq_num;
+ struct nes_ud_file *file;
+ struct nes_ud_resources *pRsc;
+
+ BUG_ON(!cq);
+
+ pRsc = locate_ud_adapter(nesvnic->nesdev->nesadapter);
+ if (pRsc == NULL)
+ return -EFAULT;
+
+
+ /* now create a QP number on base cq and adapter no */
+ cq_num = cq->hw_cq.cq_number;
+
+ nes_debug(NES_DBG_UD, "%s(%d) cq_number=%d\n",
+ __func__, __LINE__, cq_num);
+
+ /* the QP number should have the same number like CQ number */
+ file = get_file_by_qpn(pRsc, cq_num);
+ if (!file) {
+ nes_debug(NES_DBG_UD, "%s(%d) file not found\n",
+ __func__, __LINE__);
+ return -EFAULT;
+ }
+ if (file->qp_ptr) {
+ if (file->queue_type == NES_UD_RECV_QUEUE) {
+ nes_debug(NES_DBG_UD, "%s(%d) RECV file found "
+ "old=%p new=%p\n", __func__, __LINE__,
+ file->qp_ptr->ibqp.recv_cq, cq);
+ file->qp_ptr->ibqp.recv_cq = &cq->ibcq;
+ }
+ if (file->queue_type == NES_UD_SEND_QUEUE) {
+ nes_debug(NES_DBG_UD, "%s(%d) SEND file found "
+ "old=%p new=%p\n", __func__, __LINE__,
+ file->qp_ptr->ibqp.send_cq, cq);
+
+ file->qp_ptr->ibqp.send_cq = &cq->ibcq;
+ }
+ }
+ return 0;
+}
+int nes_ud_init(void)
+{
+ int i = 0;
+ int adap_no;
+ struct nes_ud_resources *pRsc;
+
+ nes_debug(NES_DBG_UD, "%s(%d)\n", __func__, __LINE__);
+
+ /* the memory registration is global for all NICS */
+ memset(&ud_mem, 0, sizeof(ud_mem));
+
+ /* init hash list of memory entries */
+ for (i = 0; i < NES_UD_MAX_REG_HASH_CNT; i++) {
+ INIT_LIST_HEAD(&ud_mem.mrs[i].list);
+ ud_mem.mrs[i].read_stats = 0;
+ }
+ mutex_init(&ud_mem.mutex);
+
+ /*allocate resources fro each adapter */
+ for (adap_no = 0; adap_no < NES_UD_MAX_ADAPTERS; adap_no++) {
+ pRsc = &nes_ud_rsc[adap_no];
+
+ memset(pRsc, 0, sizeof(*pRsc));
+
+ mutex_init(&pRsc->mutex);
+
+ pRsc->adapter_no = adap_no;
+ pRsc->pAdap = NULL;
+
+ pRsc->num_logport_confed = 0;
+ pRsc->num_allocated_nics = 0;
+ pRsc->logport_2_map = 0xf;
+ pRsc->logport_3_map = 0xf;
+ for (i = 0; i < NES_UD_MCAST_TBL_SZ; i++)
+ pRsc->mcast[i].in_use = 0;
+
+ pRsc->nics[0].qpn = 20;
+ pRsc->nics[0].nic_index = 2;
+ pRsc->nics[0].logical_port = 2;
+ pRsc->nics[0].prio = NES_UD_DEV_PRIO_HIGH;
+ pRsc->nics[0].queue_type = NES_UD_RECV_QUEUE;
+ pRsc->nics[0].file = &pRsc->nics[0].file_body;
+
+ pRsc->nics[1].qpn = 22;
+ pRsc->nics[1].nic_index = 3;
+ pRsc->nics[1].logical_port = 3;
+ pRsc->nics[1].prio = NES_UD_DEV_PRIO_HIGH;
+ pRsc->nics[1].queue_type = NES_UD_RECV_QUEUE;
+ pRsc->nics[1].file = &pRsc->nics[1].file_body;
+
+ pRsc->nics[2].qpn = 21;
+ pRsc->nics[2].nic_index = 2;
+ pRsc->nics[2].logical_port = 2;
+ pRsc->nics[2].prio = NES_UD_DEV_PRIO_LOW;
+ pRsc->nics[2].queue_type = NES_UD_RECV_QUEUE;
+ pRsc->nics[2].file = &pRsc->nics[2].file_body;
+
+ pRsc->nics[3].qpn = 23;
+ pRsc->nics[3].nic_index = 3;
+ pRsc->nics[3].logical_port = 3;
+ pRsc->nics[3].prio = NES_UD_DEV_PRIO_LOW;
+ pRsc->nics[3].queue_type = NES_UD_RECV_QUEUE;
+ pRsc->nics[3].file = &pRsc->nics[3].file_body;
+
+ pRsc->nics[4].qpn = 26;
+ pRsc->nics[4].nic_index = 6;
+ pRsc->nics[4].logical_port = 2;
+ pRsc->nics[4].prio = NES_UD_DEV_PRIO_HIGH;
+ pRsc->nics[4].queue_type = NES_UD_SEND_QUEUE;
+ pRsc->nics[4].file = &pRsc->nics[4].file_body;
+
+ pRsc->nics[5].qpn = 27;
+ pRsc->nics[5].nic_index = 7;
+ pRsc->nics[5].logical_port = 3;
+ pRsc->nics[5].prio = NES_UD_DEV_PRIO_HIGH;
+ pRsc->nics[5].queue_type = NES_UD_SEND_QUEUE;
+ pRsc->nics[5].file = &pRsc->nics[5].file_body;
+
+ pRsc->nics[6].qpn = 30;
+ pRsc->nics[6].nic_index = 10;
+ pRsc->nics[6].logical_port = 2;
+ pRsc->nics[6].prio = NES_UD_DEV_PRIO_LOW;
+ pRsc->nics[6].queue_type = NES_UD_SEND_QUEUE;
+ pRsc->nics[6].file = &pRsc->nics[6].file_body;
+
+ pRsc->nics[7].qpn = 31;
+ pRsc->nics[7].nic_index = 11;
+ pRsc->nics[7].logical_port = 3;
+ pRsc->nics[7].prio = NES_UD_DEV_PRIO_LOW;
+ pRsc->nics[7].queue_type = NES_UD_SEND_QUEUE;
+ pRsc->nics[7].file = &pRsc->nics[7].file_body;
+
+ }
+ nes_ud_workqueue = create_singlethread_workqueue("nes_ud");
+
+ return misc_register(&nes_ud_sksq_misc);
+}
+
+
+int nes_ud_exit(void)
+{
+ /* clean memory hash list */
+ nes_ud_cleanup_mr();
+ misc_deregister(&nes_ud_sksq_misc);
+ return 0;
+}
+
diff --git a/drivers/infiniband/hw/nes/nes_ud.h b/drivers/infiniband/hw/nes/nes_ud.h
new file mode 100644
index 0000000..5a03b33
--- /dev/null
+++ b/drivers/infiniband/hw/nes/nes_ud.h
@@ -0,0 +1,86 @@
+/*
+ * Copyright (c) 2008 - 2010 Intel Corporation. All rights reserved.
+ * Copyright (c) 2006 - 2008 Neteffect, All rights reserved.
+ * Copyright (c) 2005 Open Grid Computing, Inc. All rights reserved.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses. You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+#ifndef __NES_UD_H
+#define __NES_UD_H
+
+enum nes_ud_dev_priority {
+ NES_UD_DEV_PRIO_HIGH,
+ NES_UD_DEV_PRIO_LOW,
+};
+
+enum nes_ud_queue_type {
+ NES_UD_RECV_QUEUE,
+ NES_UD_SEND_QUEUE,
+};
+
+enum nes_ud_mcast_mode {
+ NES_UD_MCAST_ALL_MODE,
+ NES_UD_MCAST_PFT_MODE,
+};
+
+
+struct nes_ud_file {
+ struct nes_vnic *nesvnic;
+ u8 active;
+ char ifrn_name[IFNAMSIZ];
+ int nes_ud_nic_index;
+ int qpn;
+ enum nes_ud_dev_priority prio;
+ enum nes_ud_mcast_mode mcast_mode;
+ enum nes_ud_queue_type queue_type;
+ void *nic_vbase;
+ dma_addr_t nic_pbase;
+ int nic_mem_size;
+ void *wq_vbase;
+ dma_addr_t wq_pbase;
+ int mss;
+ struct delayed_work mcast_cleanup_work;
+ int head;
+ int tail;
+ u32 rsc_idx;
+ struct nes_qp *qp_ptr; /* it is association used for CQ replacement */
+ u32 adapter_no; /* assotiation to allocated adapter */
+};
+
+int nes_ud_init(void);
+int nes_ud_exit(void);
+struct nes_ud_file *nes_ud_create_wq(struct nes_vnic *nesvnic, int isrecv);
+int nes_ud_destroy_wq(struct nes_ud_file *file);
+u32 nes_ud_reg_mr(struct ib_umem *region, u64 length, u64 virt, u32 stag);
+int nes_ud_dereg_mr(u32 stag);
+int nes_ud_subscribe_mcast(struct nes_ud_file *file, union ib_gid *gid);
+int nes_ud_unsubscribe_mcast(struct nes_ud_file *file, union ib_gid *gid);
+int nes_ud_cq_replace(struct nes_vnic *nesvnic, struct nes_cq *cq);
+
+#endif
diff --git a/drivers/infiniband/hw/nes/nes_verbs.c b/drivers/infiniband/hw/nes/nes_verbs.c
index e54f312..ff39235 100644
--- a/drivers/infiniband/hw/nes/nes_verbs.c
+++ b/drivers/infiniband/hw/nes/nes_verbs.c
@@ -46,6 +46,8 @@
#include <rdma/ib_umem.h>
+#include "nes_ud.h"
+
atomic_t mod_qp_timouts;
atomic_t qps_created;
atomic_t sw_qps_destroyed;
@@ -1139,7 +1141,6 @@ static struct ib_qp *nes_create_qp(struct ib_pd *ibpd,
if (init_attr->create_flags)
return ERR_PTR(-EINVAL);
- atomic_inc(&qps_created);
switch (init_attr->qp_type) {
case IB_QPT_RC:
if (nes_drv_opt & NES_DRV_OPT_NO_INLINE_DATA) {
@@ -1405,10 +1406,122 @@ static struct ib_qp *nes_create_qp(struct ib_pd *ibpd,
nesqp->hwqp.qp_id, nesqp, (u32)sizeof(*nesqp));
spin_lock_init(&nesqp->lock);
nes_add_ref(&nesqp->ibqp);
+ /* moved here to be sure that QP is really created */
+ /*(now it counted a number of QP creation trials */
+ atomic_inc(&qps_created);
break;
- default:
- nes_debug(NES_DBG_QP, "Invalid QP type: %d\n", init_attr->qp_type);
- return ERR_PTR(-EINVAL);
+
+ case IB_QPT_RAW_ETY:
+ if (!ibpd->uobject)
+ return ERR_PTR(-EINVAL);
+
+ /* we are about to destroy those cqs w/o destroying qp
+ now free memory for nespbl that is not used
+ first map nespbl with the qp created */
+ if (ibpd->uobject->context) {
+ nes_ucontext = to_nesucontext(ibpd->uobject->context);
+ if (udata) {
+ if (ib_copy_from_udata(&req,
+ udata,
+ sizeof(struct nes_create_qp_req))) {
+ return ERR_PTR(-EFAULT);
+ }
+ if (req.user_wqe_buffers) {
+ err = 1;
+ list_for_each_entry(nespbl,
+ &nes_ucontext->qp_reg_mem_list,
+ list) {
+ if (nespbl->user_base ==
+ req.user_wqe_buffers) {
+ list_del(&nespbl->list);
+ err = 0;
+ /* done with memory allocated
+ during nes_reg_user_mr() */
+ pci_free_consistent(
+ nesdev->pcidev,
+ nespbl->pbl_size,
+ nespbl->pbl_vbase,
+ nespbl->pbl_pbase);
+ kfree(nespbl);
+ break;
+ }
+ }
+ }
+ }
+ }
+ /* Need 512 (actually now 1024) byte alignment on this structure */
+ mem = kzalloc(sizeof(*nesqp)+NES_SW_CONTEXT_ALIGN-1, GFP_KERNEL);
+ if (!mem) {
+ nes_debug(NES_DBG_UD, "Unable to allocate QP\n");
+ return ERR_PTR(-ENOMEM);
+ }
+ u64nesqp = (unsigned long)mem;
+ u64nesqp += ((u64)NES_SW_CONTEXT_ALIGN) - 1;
+ u64temp = ((u64)NES_SW_CONTEXT_ALIGN) - 1;
+ u64nesqp &= ~u64temp;
+ nesqp = (struct nes_qp *)(unsigned long)u64nesqp;
+ nesqp->allocated_buffer = mem;
+
+ nesqp->rx_ud_wq = nes_ud_create_wq(nesvnic, 1);
+ nesqp->tx_ud_wq = nes_ud_create_wq(nesvnic, 0);
+ if ((!nesqp->rx_ud_wq) || (!nesqp->tx_ud_wq)) {
+ kfree(nesqp->allocated_buffer);
+ return ERR_PTR(-EFAULT);
+ }
+
+ /* create association between qp and tx/rx files
+ it is used when CQ is replaced from user space */
+ nesqp->rx_ud_wq->qp_ptr = nesqp;
+ nesqp->tx_ud_wq->qp_ptr = nesqp;
+
+ sq_size = init_attr->cap.max_send_wr;
+ rq_size = init_attr->cap.max_recv_wr;
+ nes_debug(NES_DBG_UD, "%s(%d) sq_size=%d rq_size=%d\n",
+ __func__,
+ __LINE__, sq_size, rq_size);
+ uresp.actual_sq_size = sq_size;
+ uresp.actual_rq_size = rq_size;
+
+ /* Init qp size due to ibv_query_qp requirements */
+ nesqp->hwqp.sq_size = sq_size;
+ nesqp->hwqp.rq_size = rq_size;
+
+ /* enhance the response qp number with adapter number and QP number
+ on this adapter
+ user space will use this identifier when packets will be posted */
+ uresp.qp_id = nesqp->rx_ud_wq->qpn |
+ (nesqp->rx_ud_wq->adapter_no << 12) |
+ (nesqp->rx_ud_wq->rsc_idx << 8);
+ uresp.qp_id = uresp.qp_id |
+ ((nesqp->tx_ud_wq->qpn |
+ (nesqp->tx_ud_wq->adapter_no << 12) |
+ (nesqp->tx_ud_wq->rsc_idx << 8)) << 16);
+
+ nesqp->hwqp.qp_id = uresp.qp_id;
+ nesqp->ibqp.qp_num = uresp.qp_id;
+
+ nes_debug(NES_DBG_UD, "%s(%d) qpid=0x%x\n",
+ __func__, __LINE__, uresp.qp_id);
+ if (ib_copy_to_udata(udata, &uresp, sizeof uresp)) {
+ kfree(nesqp->allocated_buffer);
+ return ERR_PTR(-EFAULT);
+ }
+ /* the usecount is decreased because without it
+ the cq re-creation in user-spce will fail */
+ atomic_dec(&init_attr->send_cq->usecnt);
+ atomic_dec(&init_attr->recv_cq->usecnt);
+ nes_add_ref(&nesqp->ibqp);
+ spin_lock_init(&nesqp->lock);
+
+ /* moved here to be sure that QP is really created
+ (now it counted a number of QP creation trials */
+ atomic_inc(&qps_created);
+ return &nesqp->ibqp;
+
+ default:
+ nes_debug(NES_DBG_QP, "Invalid QP type: %d\n",
+ init_attr->qp_type);
+ return ERR_PTR(-EINVAL);
}
nesqp->sig_all = (init_attr->sq_sig_type == IB_SIGNAL_ALL_WR);
@@ -1462,6 +1575,8 @@ static void nes_clean_cq(struct nes_qp *nesqp, struct nes_cq *nescq)
static int nes_destroy_qp(struct ib_qp *ibqp)
{
struct nes_qp *nesqp = to_nesqp(ibqp);
+ struct nes_cq *scq;
+ struct nes_cq *rcq;
struct nes_ucontext *nes_ucontext;
struct ib_qp_attr attr;
struct iw_cm_id *cm_id;
@@ -1471,6 +1586,39 @@ static int nes_destroy_qp(struct ib_qp *ibqp)
atomic_inc(&sw_qps_destroyed);
nesqp->destroyed = 1;
+ if (nesqp->ibqp.qp_type == IB_QPT_RAW_ETY) {
+ /* check the QP refernece count */
+ if (atomic_read(&nesqp->refcount) == 0)
+ BUG();
+ if (atomic_dec_and_test(&nesqp->refcount)) {
+ /* destroy send and rcv QPs */
+ if (nesqp->rx_ud_wq)
+ nes_ud_destroy_wq(nesqp->rx_ud_wq);
+ nesqp->rx_ud_wq = 0;
+
+ if (nesqp->tx_ud_wq)
+ nes_ud_destroy_wq(nesqp->tx_ud_wq);
+ nesqp->tx_ud_wq = 0;
+ atomic_inc(&qps_destroyed);
+
+ /* to prevent the destroy of cq before QP
+ destroy the usecount is used */
+ if (ibqp->send_cq) {
+ scq = to_nescq(ibqp->send_cq);
+ atomic_inc(&ibqp->send_cq->usecnt);
+ atomic_dec(&scq->usecnt);
+ }
+ if (ibqp->recv_cq) {
+ rcq = to_nescq(ibqp->recv_cq);
+ atomic_inc(&ibqp->recv_cq->usecnt);
+ atomic_dec(&rcq->usecnt);
+ }
+ /* free memory for the qp */
+ kfree(nesqp->allocated_buffer);
+ }
+ return 0;
+ }
+
/* Blow away the connection if it exists. */
if (nesqp->ibqp_state >= IB_QPS_INIT && nesqp->ibqp_state <= IB_QPS_RTS) {
/* if (nesqp->ibqp_state == IB_QPS_RTS) { */
@@ -1567,9 +1715,18 @@ static struct ib_cq *nes_create_cq(struct ib_device *ibdev, int entries,
return ERR_PTR(-ENOMEM);
}
+ /* to make sure that RAW ETH cq will be not destoyed
+ without qp destroy the internal usecount is used
+ the ibcq usecount cannot be used because the RAW ETH makes
+ recreation of the CQs after QP creation
+ when this situation occured (mcrqf != 0) the usecount is increase
+ the ibcq usecount is cleared after successfull CQ creation */
+ atomic_set(&nescq->usecnt, 0);
+
nescq->hw_cq.cq_size = max(entries + 1, 5);
nescq->hw_cq.cq_number = cq_num;
nescq->ibcq.cqe = nescq->hw_cq.cq_size - 1;
+ nescq->mcrqf = 0;
if (context) {
@@ -1586,8 +1743,23 @@ static struct ib_cq *nes_create_cq(struct ib_device *ibdev, int entries,
nescq->hw_cq.cq_number = nesvnic->nic.qp_id + 28 + 2 * ((nes_ucontext->mcrqf & 0xf) - 1);
else if (nes_ucontext->mcrqf & 0x40000000)
nescq->hw_cq.cq_number = nes_ucontext->mcrqf & 0xffff;
+ else if (nes_ucontext->mcrqf & 0x20000000) {
+ /* the cq number is coded
+ adapter:4/nic:4/cq_num:8 */
+ nescq->hw_cq.cq_number =
+ nes_ucontext->mcrqf & 0x00ff;
+
+ /* to prevent the cq destroy before qp destroy
+ the internal usecount is increased
+ in this place it is the RAW ETH specific CQ
+ (after re-creation)
+ only RAW ETH type QP destroy can decrease
+ this usecounter */
+ atomic_inc(&nescq->usecnt);
+ }
else
nescq->hw_cq.cq_number = nesvnic->mcrq_qp_id + nes_ucontext->mcrqf-1;
+
nescq->mcrqf = nes_ucontext->mcrqf;
nes_free_resource(nesadapter, nesadapter->allocated_cqs, cq_num);
}
@@ -1776,6 +1948,10 @@ static struct ib_cq *nes_create_cq(struct ib_device *ibdev, int entries,
kfree(nescq);
return ERR_PTR(-EFAULT);
}
+ if (nes_ucontext->mcrqf & 0x20000000) {
+ /* change the cq address only for RAW in QP pointer */
+ nes_ud_cq_replace(nesvnic, nescq);
+ }
}
return &nescq->ibcq;
@@ -1805,6 +1981,11 @@ static int nes_destroy_cq(struct ib_cq *ib_cq)
nesdev = nesvnic->nesdev;
nesadapter = nesdev->nesadapter;
+ if (atomic_read(&nescq->usecnt) != 0) {
+ nes_debug(NES_DBG_CQ, "CQ is in use now. %d\n",
+ (int) atomic_read(&nescq->usecnt));
+ return -EBUSY;
+ }
nes_debug(NES_DBG_CQ, "Destroy CQ%u\n", nescq->hw_cq.cq_number);
/* Send DestroyCQ request to CQP */
@@ -2540,6 +2721,13 @@ static struct ib_mr *nes_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,
nesmr->ibmr.lkey = stag;
nesmr->mode = IWNES_MEMREG_TYPE_MEM;
ibmr = &nesmr->ibmr;
+ /* register memory parallelly for RAW ETH */
+ if (nes_ud_reg_mr(region, length,
+ virt, stag) == 0) {
+ ib_umem_release(region);
+ kfree(nesmr);
+ ibmr = ERR_PTR(-ENOMEM);
+ }
} else {
ib_umem_release(region);
kfree(nesmr);
@@ -2733,6 +2921,9 @@ static int nes_dereg_mr(struct ib_mr *ib_mr)
}
nes_free_resource(nesadapter, nesadapter->allocated_mrs,
(ib_mr->rkey & 0x0fffff00) >> 8);
+ ret = nes_ud_dereg_mr(ib_mr->rkey);
+ if (ret != 0)
+ return ret;
kfree(nesmr);
@@ -2939,6 +3130,9 @@ int nes_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr,
nesqp->hwqp.qp_id, attr->qp_state, nesqp->ibqp_state,
nesqp->iwarp_state, atomic_read(&nesqp->refcount));
+ if (ibqp->qp_type == IB_QPT_RAW_ETY)
+ return 0;
+
spin_lock_irqsave(&nesqp->lock, qplockflags);
nes_debug(NES_DBG_MOD_QP, "QP%u: hw_iwarp_state=0x%X, hw_tcp_state=0x%X,"
@@ -3208,8 +3402,10 @@ int nes_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr,
*/
static int nes_multicast_attach(struct ib_qp *ibqp, union ib_gid *gid, u16 lid)
{
- nes_debug(NES_DBG_INIT, "\n");
- return -ENOSYS;
+ int ret = -ENOSYS;
+ struct nes_qp *nesqp = to_nesqp(ibqp);
+ ret = nes_ud_subscribe_mcast(nesqp->rx_ud_wq, gid);
+ return ret;
}
@@ -3218,8 +3414,10 @@ static int nes_multicast_attach(struct ib_qp *ibqp, union ib_gid *gid, u16 lid)
*/
static int nes_multicast_detach(struct ib_qp *ibqp, union ib_gid *gid, u16 lid)
{
- nes_debug(NES_DBG_INIT, "\n");
- return -ENOSYS;
+ int ret = -ENOSYS;
+ struct nes_qp *nesqp = to_nesqp(ibqp);
+ ret = nes_ud_unsubscribe_mcast(nesqp->rx_ud_wq, gid);
+ return ret;
}
@@ -3846,6 +4044,7 @@ struct nes_ib_device *nes_init_ofa_device(struct net_device *netdev)
return NULL;
}
strlcpy(nesibdev->ibdev.name, "nes%d", IB_DEVICE_NAME_MAX);
+ strcpy(nesibdev->ibdev.name, netdev->name);
nesibdev->ibdev.owner = THIS_MODULE;
nesibdev->ibdev.node_type = RDMA_NODE_RNIC;
@@ -3868,6 +4067,9 @@ struct nes_ib_device *nes_init_ofa_device(struct net_device *netdev)
(1ull << IB_USER_VERBS_CMD_REQ_NOTIFY_CQ) |
(1ull << IB_USER_VERBS_CMD_CREATE_QP) |
(1ull << IB_USER_VERBS_CMD_MODIFY_QP) |
+ (1ull << IB_USER_VERBS_CMD_QUERY_QP) |
+ (1ull << IB_USER_VERBS_CMD_ATTACH_MCAST) |
+ (1ull << IB_USER_VERBS_CMD_DETACH_MCAST) |
(1ull << IB_USER_VERBS_CMD_POLL_CQ) |
(1ull << IB_USER_VERBS_CMD_DESTROY_QP) |
(1ull << IB_USER_VERBS_CMD_ALLOC_MW) |
@@ -3911,8 +4113,9 @@ struct nes_ib_device *nes_init_ofa_device(struct net_device *netdev)
nesibdev->ibdev.alloc_fast_reg_page_list = nes_alloc_fast_reg_page_list;
nesibdev->ibdev.free_fast_reg_page_list = nes_free_fast_reg_page_list;
- nesibdev->ibdev.attach_mcast = nes_multicast_attach;
nesibdev->ibdev.detach_mcast = nes_multicast_detach;
+ nesibdev->ibdev.attach_mcast = nes_multicast_attach;
+
nesibdev->ibdev.process_mad = nes_process_mad;
nesibdev->ibdev.req_notify_cq = nes_req_notify_cq;
diff --git a/drivers/infiniband/hw/nes/nes_verbs.h b/drivers/infiniband/hw/nes/nes_verbs.h
index 2df9993..cbb6585 100644
--- a/drivers/infiniband/hw/nes/nes_verbs.h
+++ b/drivers/infiniband/hw/nes/nes_verbs.h
@@ -79,6 +79,7 @@ struct nes_mr {
u16 pbls_used;
u8 mode;
u8 pbl_4k;
+ u32 mcrqf;
};
struct nes_hw_pb {
@@ -116,7 +117,8 @@ struct nes_cq {
spinlock_t lock;
u8 virtual_cq;
u8 pad[3];
- u32 mcrqf;
+ atomic_t usecnt;
+ u32 mcrqf;
};
struct nes_wq {
@@ -130,6 +132,7 @@ struct disconn_work {
struct iw_cm_id;
struct ietf_mpa_frame;
+struct nes_ud_file;
struct nes_qp {
struct ib_qp ibqp;
@@ -176,5 +179,7 @@ struct nes_qp {
u8 hw_tcp_state;
u8 term_flags;
u8 sq_kmapped;
+ struct nes_ud_file *rx_ud_wq;
+ struct nes_ud_file *tx_ud_wq;
};
#endif /* NES_VERBS_H */
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related
* Re: [PATCH] mlx4: Fix chunk sg list overflow in mlx4_alloc_icm()
From: Eli Cohen @ 2010-05-03 6:27 UTC (permalink / raw)
To: sebastien dugue; +Cc: linux-rdma, Roland Dreier
In-Reply-To: <20100430132131.0d829b3c@frecb007965>
Looks like a valid fix to me.
On Fri, Apr 30, 2010 at 01:21:31PM +0200, sebastien dugue wrote:
>
> If the number of sg entries in the ICM chunk reaches MLX4_ICM_CHUNK_LEN,
> we must set chunk = NULL __even__ for coherent mappings (as is done for
> mthca). Otherwise we may overflow the sg list.
>
> Signed-off-by: Sebastien Dugue <sebastien.dugue-6ktuUTfB/bM@public.gmane.org>
> ---
> drivers/net/mlx4/icm.c | 3 ++-
> 1 files changed, 2 insertions(+), 1 deletions(-)
>
> diff --git a/drivers/net/mlx4/icm.c b/drivers/net/mlx4/icm.c
> index 57288ca..ef62f17 100644
> --- a/drivers/net/mlx4/icm.c
> +++ b/drivers/net/mlx4/icm.c
> @@ -175,9 +175,10 @@ struct mlx4_icm *mlx4_alloc_icm(struct mlx4_dev *dev, int npages,
>
> if (chunk->nsg <= 0)
> goto fail;
> + }
>
> + if (chunk->npages == MLX4_ICM_CHUNK_LEN)
> chunk = NULL;
> - }
>
> npages -= 1 << cur_order;
> } else {
> --
> 1.6.0.4
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* [ANNOUNCE] OFED 1.5.2 beta1 is available
From: Vladimir Sokolovsky @ 2010-05-03 7:35 UTC (permalink / raw)
To: OpenFabrics EWG; +Cc: linux-rdma
[-- Attachment #1: Type: text/plain, Size: 1991 bytes --]
OFED 1.5.2-beta1 is available
Notes:
The tarball is available on:
http://www.openfabrics.org/downloads/OFED/ofed-1.5.2/OFED-1.5.2-beta1.tgz
To get BUILD_ID run ofed_info
Please report any issues in bugzilla https://bugs.openfabrics.org/ for
OFED 1.5.2
Vladimir & Tziporet
========================================================================
Supported Platforms and Operating Systems
---------------------------------------------
o CPU architectures:
- x86_64
- x86
- ppc64
- ia64
o Linux Operating Systems:
- RedHat EL4 up7 2.6.9-78.ELsmp
- RedHat EL4 up8 2.6.9-89.ELsmp
- RedHat EL5 up3 2.6.18-128.el5
- RedHat EL5 up4 2.6.18-164.el5
- RedHat EL5 up5 2.6.18-194.el5
- SLES10 SP2 2.6.16.60-0.21-smp
- SLES10 SP3 2.6.16.60-0.54-smp
- SLES11 2.6.27.19-5-default
- OEL 4 up7 2.6.9-78.ELsmp
- OEL 4 up8 2.6.9-89.ELsmp
- CentOS5.3 2.6.18-128.el5
- CentOS5.4 2.6.18-164.el5
- Fedora Core12 2.6.31.5-127.fc12 *
- OpenSuSE 11.2 2.6.31.5-0.1-default *
- kernel.org 2.6.29, 2.6.30,
2.6.31 and 2.6.32 *
* Minimal QA for these versions
Main changes from 1.5.1:
===========================
1. Updated packages:
- Management
Using latest daily builds from http://www.openfabrics.org/downloads/management/daily
- Updated libnes
libnes-1.0.1-0.1.g89ea0ee.tar.gz
- Updated libsdp
libsdp-1.1.101-0.3.gc767eee.tar.gz
- Updated perftest
perftest-1.2.4-0.15.g82b7e29.tar.gz
- Updated mpitests
mpitests-3.2-923.src.rpm
2. Added RHEL5.5 support
3. Use files under /etc/modprobe.d/ instead of /etc/modprobe.conf
4. Bug fixes
[-- Attachment #2: ofed_kernel-1.5.1_1.5.2-beta1.log --]
[-- Type: text/plain, Size: 16300 bytes --]
commit 8986850c7198bf32fc3d87884c72875a272fdb5e
Author: Vladimir Sokolovsky <vlad-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
Date: Sun May 2 15:08:03 2010 +0300
mlx4_en.conf: load mlx4_en when mlx4_core is loaded
Signed-off-by: Vladimir Sokolovsky <vlad-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
commit b03fe7f8623c2c4d9a716f90a848ab679d6b10a2
Author: Vladimir Sokolovsky <vlad-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
Date: Sun May 2 10:33:58 2010 +0300
ofa_kernel.spec: Stop using depecated /etc/modprobe.conf
Signed-off-by: Vladimir Sokolovsky <vlad-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
commit ac488f9096cd4d146f5a99fba48c17f9d4235239
Author: Eli Cohen <eli-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
Date: Thu Apr 29 15:14:49 2010 +0300
mlx4: Fix layout of QPC.AH
Signed-off-by: Eli Cohen <eli-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
commit 3aab0ebcfd1bb676b754cf9ca69a9b2609dd847e
Author: Eli Cohen <eli-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
Date: Thu Apr 29 14:36:39 2010 +0300
rdma_cm: Add default translation from ToS to SL
Add a default translation from for ToS to SL for rdma_cm over RoCE. The default
translation is dictated by the value of the module paramter, def_prec2sl. Any
value of ToS will be translated to this value. In the future we will add a more
generic translation table.
Signed-off-by: Eli Cohen <eli-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
commit 489dfc902a7b1e8f60ad996fec00113642301225
Author: Ralph Campbell (QLogic) <ralphc-ZwoEplunGu1OwGhvXhtEPSCwEArCW2h5@public.gmane.org>
Date: Wed Apr 28 16:12:30 2010 -0700
IB/qib: more fixes for OFED 1.5.2
More serdes tuning changes for interop.
Fix a MR reference count bug with UC RDMA writes with immediate.
Fix a deadlock race when registering more than one device with DCA.
Signed-off-by: Ralph Campbell <ralph.campbell-h88ZbnxC6KDQT0dZR+AlfA@public.gmane.org>
commit 1270cdb8ed5f16079a2cdf83a73bc6e5df9c9333
Author: Vladimir Sokolovsky <vlad-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
Date: Wed Apr 28 16:38:07 2010 +0300
RDMA/ucma: Add option to manually set IB path
Export rdma_set_ib_paths to user space to allow applications to
manually set the IB path used for connections. This allows
alternative ways for a user space application or library to obtain
path record information, including retrieving path information
from cached data, avoiding direct interaction with the IB SA.
The IB SA is a single, centralized entity that can limit scaling
on large clusters running MPI applications.
Future changes to the rdma cm can expand on this framework to
support the full range of features allowed by the IB CM, such as
separate forward and reverse paths and APM.
Signed-off-by: Sean Hefty <sean.hefty-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Reviewed-By: Jason Gunthorpe <jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
Signed-off-by: Roland Dreier <rolandd-FYB4Gu1CFyUAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Vladimir Sokolovsky <vlad-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
commit e0ed697e60968670d4badfb1c8aa9125065d1823
Merge: e59e4bc 7b1e558
Author: Vladimir Sokolovsky <vlad-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
Date: Tue Apr 27 10:03:13 2010 +0300
Merge branch 'ofed_kernel_1_5' of git://sofa.openfabrics.org/~ctung/ofed-1.5 into ofed_kernel_1_5
commit e59e4bc5fe76a5a8d9bdfa9407746d5227873bd9
Author: Ralph Campbell (QLogic) <ralphc-ZwoEplunGu1OwGhvXhtEPSCwEArCW2h5@public.gmane.org>
Date: Mon Apr 26 10:58:37 2010 -0700
IB/qib: fix qib backport to RHEL 5.5
Signed-off-by: Ralph Campbell <ralph.campbell-h88ZbnxC6KDQT0dZR+AlfA@public.gmane.org>
commit 30f0b496eefe781cb098ddfc774ff35b99217042
Author: Amir Vadai <amirv-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
Date: Mon Apr 26 14:53:23 2010 +0300
sdp: fix a leak when ib_post_xxx fail + small fixes
Signed-off-by: Amir Vadai <amirv-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
commit e3bf1a8d2da1759611f985493e79c61a0311e5d2
Author: Eldad Zinger <eldadz-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
Date: Mon Apr 26 10:48:48 2010 +0300
sdp: on device removal, ref count taken so that socket won't be destructed
On device removal, socket resources that are based on device are destroyed,
but the socket itself will be destroyed later, upon user request.
Signed-off-by: Eldad Zinger <eldadz-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
commit 93bc358d9895e27474c35527188165a4d1188ca5
Merge: 8c94349 6c80f9c
Author: Vladimir Sokolovsky <vlad-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
Date: Sun Apr 25 16:13:44 2010 +0300
Merge remote branch 'amirv/ofed_kernel_1_5' into ofed_kernel_1_5
commit 6c80f9ce0605f57c06629b2e4bec81752a78d0e1
Author: Amir Vadai <amirv-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
Date: Thu Apr 22 11:26:57 2010 +0300
sdp: use max number of SGE from HW capabilities
Instead of using hard coded max number of SGE's take
it from device capabilities.
Signed-off-by: Amir Vadai <amirv-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
commit 4bc6920c83fb63a510fa86dec5de95ba6fc6dfa8
Author: Amir Vadai <amirv-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
Date: Sun Apr 25 11:55:29 2010 +0300
sdp: Fix a hang when ib_post_recv is failed
Signed-off-by: Amir Vadai <amirv-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
commit 0d56a06939d7c03091c26bd1deeb689a6cbba2f5
Author: Amir Vadai <amirv-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
Date: Sun Apr 25 11:50:44 2010 +0300
sdp: fix compilation warning on debug prints
Signed-off-by: Amir Vadai <amirv-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
commit 8c94349521f73054bfcb08748f33ad815defd854
Author: Eli Cohen <eli-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
Date: Sun Apr 25 10:41:37 2010 +0300
Bug fix in counters
Take the port number from qp->port rather than attr->port_num. qp->port is
update when modifying from reset to init. For special QPs it is updated at
creation.
Signed-off-by: Eli Cohen <eli-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
commit b322c88f10bb2212101346e39523e32ce9f71739
Merge: 6d6ac62 6a54ece
Author: Vladimir Sokolovsky <vlad-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
Date: Thu Apr 22 17:29:08 2010 +0300
Merge remote branch 'amirv/ofed_kernel_1_5' into ofed_kernel_1_5
commit 6d6ac621f1ddec038cd73685fc11c60dff76f706
Author: Vladimir Sokolovsky <vlad-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
Date: Thu Apr 22 16:27:38 2010 +0300
Update support for enhanced atomic operations
Patches updated with the version committed to the upstream kernel.
Signed-off-by: Vladimir Sokolovsky <vlad-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
commit 6a54ece400371447a383c59597e961cb6d51977a
Author: Eldad Zinger <eldadz-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
Date: Thu Apr 22 10:34:59 2010 +0300
sdp: sdp_bzcopy_thresh module parameter removal
Field [bzcopy_thresh] inside 'struct sdp_sock' changed to [zcopy_thresh] and
the ability to disable zcopy per-socket was added.
Signed-off-by: Eldad Zinger <eldadz-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
commit aaf5a11f6b1d466b5aaf2c9dd40b43dbe8f389bc
Author: Ralph Campbell (QLogic) <ralphc-ZwoEplunGu1OwGhvXhtEPSCwEArCW2h5@public.gmane.org>
Date: Tue Apr 20 17:20:16 2010 -0700
IB/qib: backport fixes for OFED 1.5.2
Update the backport patches for OFED 1.5.2.
Signed-off-by: Ralph Campbell <ralph.campbell-h88ZbnxC6KDQT0dZR+AlfA@public.gmane.org>
commit eceda2cf579365ee90fcb9c5c4ca647a16484777
Merge: b35c420 b48db55
Author: Vladimir Sokolovsky <vlad-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
Date: Tue Apr 20 14:57:14 2010 +0300
Merge branch 'ofed_kernel_1_5' of git://git.openfabrics.org/~ralphc/linux-2.6/ into ofed_kernel_1_5
commit b48db55de53486afcdf0422d79684627045731de
Author: Ralph Campbell (QLogic) <ralphc-ZwoEplunGu1OwGhvXhtEPSCwEArCW2h5@public.gmane.org>
Date: Mon Apr 19 11:35:04 2010 -0700
IB/qib: fixes for OFED 1.5.2
This updates the QDR HCA serdes settings to improve reliability
and is needed for the next IB interoperability event.
Signed-off-by: Ralph Campbell <ralph.campbell-h88ZbnxC6KDQT0dZR+AlfA@public.gmane.org>
commit b35c4207dce433c1e3b623d9bdd4259f474e38e2
Author: Eldad Zinger <eldadz-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
Date: Sun Apr 18 16:56:03 2010 +0300
sdp: BUG2017 - better initialization implementation for ssk->nagle_timer
Signed-off-by: Eldad Zinger <eldadz-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
commit e79bd3e79785a6974feace5acea1e43fb1b7cd03
Author: Eldad Zinger <eldadz-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
Date: Sun Apr 18 14:34:33 2010 +0300
sdp: fix for brutal device removing
Kernel always crashed in the following test case:
user program: socket+bind+listen+accept. socket accpted.
shell: rmmod mlx4_ib
user program: close <<< CRASH
The fix closes any socket that its ib_device is the device being removed.
Signed-off-by: Eldad Zinger <eldadz-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
commit 5a9dbaf793268ecf4d44f53fdca42edda0f54e16
Author: Amir Vadai <amirv-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
Date: Thu Apr 15 11:57:11 2010 +0300
sdp: Don't try to allocate FMR larger than RLIMIT_MEMLOCK
During ZCopy. If don't have CAP_IPC_LOCK capability and current
max number of locked pages is smaller than the buffer size, split
the send into small fragments.
Signed-off-by: Amir Vadai <amirv-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
commit f988c01bf259206e9575bcc65d3d98e700f10911
Author: Amir Vadai <amirv-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
Date: Thu Apr 15 11:52:19 2010 +0300
sdp: Don't count sdp header twice when calculating size_goal
sizeof(struct sdp_bsdh) is included inside skb->len. Ignore it
when calculating maximum payload of the skb.
This mistake caused every BCopy send of 32K (and its multiples) to
be split and thus got bad performance.
Signed-off-by: Amir Vadai <amirv-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
commit e233ae1662b884deb9bdf77193e7a2b2a0f0b7d3
Author: Eldad Zinger <eldadz-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
Date: Tue Apr 13 13:39:41 2010 +0300
sdp: timeout for abortive close updated
SDP_FIN_WAIT_TIMEOUT updated from 10[sec] to 60[sec] to comply with TCP
TCP_FIN_TIMEOUT.
Signed-off-by: Eldad Zinger <eldadz-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
commit 6cac8adf70072d141d403203693af9d94adb0e24
Author: Eli Cohen <eli-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
Date: Thu Apr 15 15:08:13 2010 +0300
Fix counters for RoCE
counter index is assigned in modify from init to rtr.
Signed-off-by: Eli Cohen <eli-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
commit e5414cccaa13e6dd80d8d6fc3dafe95355facdef
Author: Amir Vadai <amirv-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
Date: Mon Apr 12 15:42:32 2010 +0300
sdp: module parameter to disable SDP over ROCEE
Since ROCEE added, SDP always try to connect, even
if link layer of type Ethernet. This make 'both' mode
in libsdp useless.
Added a module paramter to disable it by default
Signed-off-by: Amir Vadai <amirv-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
commit 9a6ef6793c83b1587984d919a84ed5c2ca19a0c0
Author: Eldad Zinger <eldadz-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
Date: Thu Apr 8 10:09:37 2010 +0300
sdp: added differentiation between bind failures of sdp.
When bind()ing in mode 'BOTH', bind(sdp_sock) might fail if:
1. the IP&port is already bounded.
2. the IP is not part of IB network.
previous implementation returned errno=EADDRINUSE either way.
Only the first case should fail the bind(), the second is legitimate
because the TCP socket will hanle the connection.
This fix corresponds to a fix in libsdp.
Signed-off-by: Eldad Zinger <eldadz-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
commit feaa7f15f651c35978f8c2a3e08642d2530fce0b
Author: Eldad Zinger <eldadz-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
Date: Thu Mar 18 10:53:56 2010 +0200
sdp: BUG1727 - there is no point of using zcopy when credits are not available.
if credits are not available, data won't be transferred anyway,
so it would be better to use bcopy.
Signed-off-by: Eldad Zinger <eldadz-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
commit 8f358b2307d4eaa816fc8bec3dda365399754530
Author: Eldad Zinger <eldadz-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
Date: Wed Mar 31 16:02:03 2010 +0300
sdp: BUG1992 - enable transmission of credits update when tx_credits == 1
According to spec, if one credit is available, we can only send messages that
provide additional credits and also do not contain data payload.
Signed-off-by: Eldad Zinger <eldadz-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
commit 2ac23ff8ac947397ad6866cdf215809eb2511e93
Author: Eldad Zinger <eldadz-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
Date: Wed Mar 31 15:27:38 2010 +0300
sdp: unnecessary 'if' statement canceled.
Signed-off-by: Eldad Zinger <eldadz-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
commit 1c2405885e0e5595cdc531a84400458c4e0c70bb
Author: Eldad Zinger <eldadz-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
Date: Thu Mar 18 13:31:08 2010 +0200
sdp: BUG1727 - fixed select(2) behavior on a new nonblocking socket.
when calling for select(2) after socket(2) on a nonblocking socket,
select(2) should return 'writable'.
Signed-off-by: Eldad Zinger <eldadz-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
commit f9f0b4d008fa53407d9a64a30c2789c6ce4cdf6e
Author: Eldad Zinger <eldadz-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
Date: Wed Mar 17 15:24:28 2010 +0200
sdp: BUG1727 - sdp_destroy_work() and sdp_connect() interfere with each other.
This fix was done in order to make sure that sdp_destroy_work() is done
before sdp_connect() is active.
Signed-off-by: Eldad Zinger <eldadz-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
commit 88e95a2bf83596150919a077ea2e371f08812dbb
Author: Amir Vadai <amirv-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
Date: Sun Mar 14 16:36:32 2010 +0200
sdp: support iovlen > 1 in zcopy
Signed-off-by: Amir Vadai <amirv-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
commit 9cb9abb8501c96633d6ab144ccb1ff3880e12f74
Author: Amir Vadai <amirv-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
Date: Sun Mar 14 16:35:38 2010 +0200
sdp: make sdp_socket.h available to user applications
Signed-off-by: Amir Vadai <amirv-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
commit 7d39db269d341c8faf846b4ade0413c5dbb543f0
Author: Amir Vadai <amirv-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
Date: Thu Apr 1 10:28:37 2010 +0300
sdp: enable FMR pool cache
Signed-off-by: Amir Vadai <amirv-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
commit 7b1e5589a307f0783ff0b360a778bc6d32ab710a
Author: Chien Tung <chien.tin.tung-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Date: Thu Mar 25 10:33:30 2010 -0500
RDMA/nes: correct cap.max_inline_data assignment in nes_query_qp
Signed-off-by: Chien Tung <chien.tin.tung-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
commit 888aeb78804be9c0e89acb18622f49ff8cc40073
Author: Chien Tung <chien.tin.tung-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Date: Thu Mar 25 09:09:36 2010 -0500
RDMA/nes: disable dynamic interrupt moderation by default
Signed-off-by: Chien Tung <chien.tin.tung-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
commit 63079c7ac1161a090cdd9feef757118c9d56cc72
Merge: 17badd7 bf34b58
Author: Vladimir Sokolovsky <vlad-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
Date: Thu Mar 25 11:01:55 2010 +0200
Merge remote branch 'vu/ofed_kernel_1_5' into ofed_kernel_1_5
commit bf34b58acf19fdbef467aa9bb70b37d840f3bdc7
Author: Vu Pham <vu-OnC3O1emZK9LiOotz6YBiwC/G2K4zDHf@public.gmane.org>
Date: Wed Mar 24 16:36:53 2010 -0700
srp: solving dead_lock issue during unload/reboot system
Signed-off-by: Vu Pham <vu-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: bugs-2010-05-03.csv --]
[-- Type: text/csv; name="bugs-2010-05-03.csv", Size: 1252 bytes --]
bug_id,"bug_severity","priority","op_sys","assigned_to","bug_status","resolution","short_short_desc"
1727,"normal","P3","SLES 11","eldadz@mellanox.co.il","RESOLVED","FIXED","Select fails after connect to invalid address"
1847,"normal","P3","All","amirv@mellanox.co.il","RESOLVED","FIXED","zcopy send with iovlen > 1 is not supported"
1872,"normal","P3","All","amirv@mellanox.co.il","RESOLVED","FIXED","failed to send packets less than 4080 byte by zcopy"
1909,"normal","P3","All","amirv@mellanox.co.il","RESOLVED","FIXED","Enable a trivial way to include sdp_socket.h"
1965,"major","P3","RHEL 5","yevgenyp@mellanox.co.il","CLOSED","FIXED","Bonding mlx4_en: ping does not resume after failover between 10G ports"
1984,"normal","P3","All","eldadz@mellanox.co.il","RESOLVED","FIXED","Bug in accept()"
1988,"major","P2","All","swise@opengridcomputing.com","RESOLVED","FIXED","[OFED-1.5.1-rc4] - Softirq seen on one of the nodes after ~48 hours while running 2 instances each of IMB-MPI1+ OSU+Presta test together on 64 bit platforms"
2027,"normal","P3","RHEL 5","amirv@mellanox.co.il","CLOSED","FIXED","SDP not respecting # SGEs as reported from HW"
2041,"normal","P3","Other","jeremy.brown@qlogic.com","RESOLVED","FIXED","QIB compilation fails on RHEL5.5"
^ permalink raw reply
* Re: [PATCH] RDMA/cxgb3: Shrink .text with compile-time init of handlers arrays
From: Steve Wise @ 2010-05-03 14:32 UTC (permalink / raw)
To: Roland Dreier; +Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA
In-Reply-To: <ada7hnr1bwc.fsf-BjVyx320WGW9gfZ95n9DRSW4+XlvGpQz@public.gmane.org>
Roland Dreier wrote:
> > Looks ok to me. Functionally it doesn't change anything I guess.
>
> That's the hope at least...
>
If you push this into your for-next branch, I'll regression test it.
Steve.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH 0/2] RDMA/core: add support for iWarp multicast acceleration over IB_QPT_RAW_ETY QP type
From: Steve Wise @ 2010-05-03 14:35 UTC (permalink / raw)
To: Mirek Walukiewicz
Cc: rdreier-FYB4Gu1CFyUAvxtiuMwx3w, linux-rdma-u79uwXL29TY76Z2rM5mHXA
In-Reply-To: <20100430165249.1386.73960.stgit-dAdtdUp2yJRU7keBU/FxOFDQ4js95KgL@public.gmane.org>
BTW: I agree with the core changes proposed in this series as well as
the patch adding the RAW_ETY QP type for user mode.
Steve.
Mirek Walukiewicz wrote:
> This patch series implements iWarp multicast acceleration over raw eth QP type.
>
> In ibv_attach_mcast and ibv_detach_mcast, new case is added for RDMA_TRANSPORT_IWARP to allow multicast
> on IB_QPT_RAW_ETY QP type. This change is for iWarp only and does not affect IB.
>
> Changes to nes demenstrate how IB_QPT_RAW_ETY QP is used to accelerate multicast traffic.
>
> Miroslaw Walukiewicz
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH] RDMA/cxgb3: Shrink .text with compile-time init of handlers arrays
From: Roland Dreier @ 2010-05-03 14:37 UTC (permalink / raw)
To: Steve Wise; +Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA
In-Reply-To: <4BDEDE62.8010905-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW@public.gmane.org>
> If you push this into your for-next branch, I'll regression test it.
It should be there... yep, just checked and I see it on git.kernel.org gitweb.
--
Roland Dreier <rolandd-FYB4Gu1CFyUAvxtiuMwx3w@public.gmane.org> || For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/index.html
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Hang in ib_umad when attempting to unregister.
From: Mike Heinz @ 2010-05-03 15:15 UTC (permalink / raw)
To: LINUX-RDMA
Hello, all.
In my company's test labs, we've been seeing nodes occasionally hang when a process tries to disconnect from the ib_mad interface. Can anyone suggest what might be causing this?
Here's a typical example:
Apr 29 10:01:37 st2139 kernel: qlgc_dsc D ffffffff80148c54 0 5478
1 5497 5477 (NOTLB)
Apr 29 10:01:37 st2139 kernel: ffff81042b785dd8 0000000000000082 000000000062f388 00000000437b2038
Apr 29 10:01:37 st2139 kernel: 0000000000000000 000000000000000a ffff81043fa3f040 ffff81043fb6e100
Apr 29 10:01:37 st2139 kernel: 00003463ec0fbcd0 0000000000003720 ffff81043fa3f228 000000080062f388
Apr 29 10:01:37 st2139 kernel: Call Trace:
Apr 29 10:01:37 st2139 kernel: [<ffffffff8003dd13>] do_futex+0x282/0xc3f
Apr 29 10:01:37 st2139 kernel: [<ffffffff80063206>] wait_for_completion+0x79/0xa2
Apr 29 10:01:37 st2139 kernel: [<ffffffff8008a461>] default_wake_function+0x0/0xe
Apr 29 10:01:37 st2139 kernel: [<ffffffff88318399>]:ib_mad:ib_cancel_rmpp_recvs+0xa6/0xe9
Apr 29 10:01:37 st2139 kernel: [<ffffffff883155f1>]:ib_mad:ib_unregister_mad_agent+0x30d/0x424
Apr 29 10:01:37 st2139 kernel: [<ffffffff8850d24e>]:ib_umad:ib_umad_unreg_agent+0x6f/0x94
Apr 29 10:01:37 st2139 kernel: [<ffffffff8850db71>]:ib_umad:ib_umad_ioctl+0x4a/0x5d
Apr 29 10:01:37 st2139 kernel: [<ffffffff80041b2e>] do_ioctl+0x21/0x6b
Apr 29 10:01:37 st2139 kernel: [<ffffffff8002fd1e>] vfs_ioctl+0x248/0x261
Apr 29 10:01:37 st2139 kernel: [<ffffffff8004c0a3>] sys_ioctl+0x59/0x78
Apr 29 10:01:37 st2139 kernel: [<ffffffff8005d28d>] tracesys+0xd5/0xe0
Reviewing the code, the problem is that, basically, ib_cancel_rmpp_recvs is waiting for a completion() to occur, but the completion() is never getting invoked, presumably because the reference count is wrong on one of the rmpp structures:
static inline void deref_rmpp_recv(struct mad_rmpp_recv *rmpp_recv)
{
if (atomic_dec_and_test(&rmpp_recv->refcount))
complete(&rmpp_recv->comp);
}
static void destroy_rmpp_recv(struct mad_rmpp_recv *rmpp_recv)
{
deref_rmpp_recv(rmpp_recv);
wait_for_completion(&rmpp_recv->comp);
ib_destroy_ah(rmpp_recv->ah);
kfree(rmpp_recv);
}
Reviewing our internal bugs database, I actually found that this problem has actually been around for several years, but we were never able to reproduce it under controlled circumstances. Most frequently, the problem occurred when trying to unload a module. Here's an example that was captured in 2007:
rmmod D ffff81003af6fd60 0 22020 21962
ffff81003b017c68 0000000000000082 ffffffff813a22a8 ffff81003b017c88
ffff81003b017c90 ffff81003ab39800 ffff81003fba6800 ffff81003ab39a68
000000013b017c58 ffffffff8126b945 0000000000000001 ffffffff81042433
Call Trace:
[<ffffffff8126b945>] wait_for_completion+0xa0/0xb3
[<ffffffff81042433>] flush_cpu_workqueue+0x29/0x6f
[<ffffffff8102def5>] default_wake_function+0x0/0xe
[<ffffffff8126b92f>] wait_for_completion+0x8a/0xb3
[<ffffffff8102def5>] default_wake_function+0x0/0xe
[<ffffffff881271d7>] :ib_mad:ib_cancel_rmpp_recvs+0x8a/0xdf
[<ffffffff88124475>] :ib_mad:ib_unregister_mad_agent+0x333/0x445
[<ffffffff8812f0d0>] :ib_sa:free_sm_ah+0x0/0x17
[<ffffffff88125e90>] :ib_mad:ib_agent_port_close+0x7c/0x8b
[<ffffffff8812245b>] :ib_mad:ib_mad_remove_device+0x38/0x85
[<ffffffff880fbf20>] :ib_core:ib_unregister_device+0x30/0xc4
[<ffffffff8817033c>] :ib_ipath:ipath_unregister_ib_device+0x59/0x282
[<ffffffff88152e69>] :ib_ipath:ipath_remove_one+0x75/0x474
[<ffffffff81122d01>] pci_device_remove+0x24/0x48
[<ffffffff811885aa>] __device_release_driver+0x8e/0xb0
[<ffffffff81188ae8>] driver_detach+0xce/0x10e
[<ffffffff81188053>] bus_remove_driver+0x6d/0x90
[<ffffffff81122f53>] pci_unregister_driver+0x10/0x5f
[<ffffffff8817da5f>] :ib_ipath:infinipath_cleanup+0x3f/0x4c
[<ffffffff81050d23>] sys_delete_module+0x196/0x1c5
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* dropped packets when IP forwarding
From: Eli Cohen @ 2010-05-03 15:31 UTC (permalink / raw)
To: linux-net, Linux RDMA list
Hi,
I am trying to check the forwarding rate between two net devices on my
machine. What I see is that when the rate gets higher than some
thresohld, the reception rate can still increase but the trasnmit rate
reaches a maximum. At this point I assume packets get dropped. When
looking at the CPU utilization I see that they're around 50% so the
CPUs can do work. I also see that the trasmit queue is not stopped so
I think that maybe the IP layer drops them (lack of buffers?). Does
anyone have an idea?
^ permalink raw reply
* RE: Hang in ib_umad when attempting to unregister.
From: Sean Hefty @ 2010-05-03 15:47 UTC (permalink / raw)
To: 'Mike Heinz', LINUX-RDMA
In-Reply-To: <4C2744E8AD2982428C5BFE523DF8CDCB49A47409EA-amwN6d8PyQWXx9kJd3VG2h2eb7JE58TQ@public.gmane.org>
>In my company's test labs, we've been seeing nodes occasionally hang when a
>process tries to disconnect from the ib_mad interface. Can anyone suggest what
>might be causing this?
What kernel are you running?
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* RE: Hang in ib_umad when attempting to unregister.
From: Mike Heinz @ 2010-05-03 15:58 UTC (permalink / raw)
To: Sean Hefty, LINUX-RDMA
In-Reply-To: <57C080F9018D4C488FD7F575E2CE231F-Zpru7NauK7drdx17CPfAsdBPR1lH4CV8@public.gmane.org>
Sean,
This has happened on a couple of different systems in our test fabric, but the most recent occurrences are with RHEL5 and SLES11:
RHEL5.3 2.6.18-128.el5
SLES11 2.6.27-19-5-default
-----Original Message-----
From: Sean Hefty [mailto:sean.hefty-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org]
Sent: Monday, May 03, 2010 11:48 AM
To: Mike Heinz; LINUX-RDMA
Subject: RE: Hang in ib_umad when attempting to unregister.
>In my company's test labs, we've been seeing nodes occasionally hang when a
>process tries to disconnect from the ib_mad interface. Can anyone suggest what
>might be causing this?
What kernel are you running?
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* RE: Hang in ib_umad when attempting to unregister.
From: Sean Hefty @ 2010-05-03 16:08 UTC (permalink / raw)
To: 'Mike Heinz', LINUX-RDMA
In-Reply-To: <4C2744E8AD2982428C5BFE523DF8CDCB49A47409F6-amwN6d8PyQWXx9kJd3VG2h2eb7JE58TQ@public.gmane.org>
>This has happened on a couple of different systems in our test fabric, but the
>most recent occurrences are with RHEL5 and SLES11:
>
>RHEL5.3 2.6.18-128.el5
>SLES11 2.6.27-19-5-default
Here are a couple commits that may be relevant. I didn't see what version they
were added, but they're both from Sept. 2009.
commit 6b2eef8fd78ff909c3396b8671d57c42559cc51d
commit 0e442afd92fcdde2cc63b6f25556b8934e42b7d2
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* RE: Hang in ib_umad when attempting to unregister.
From: Mike Heinz @ 2010-05-03 17:06 UTC (permalink / raw)
To: Hefty, Sean; +Cc: LINUX-RDMA
In-Reply-To: <CF9C39F99A89134C9CF9C4CCB68B8DDF254C321EFA-osO9UTpF0USkrb+BlOpmy7fspsVTdybXVpNB7YpNyf8@public.gmane.org>
Ah. Got it. Thanks.
They do seem to be related. 0e442afd92fcdde2cc63b6f25556b8934e42b7d2 seems to be directly related - but I think that fix is already in OFED 1.5:
core_0310-IB-mad-Fix-lock-lock-timer-deadlock-in-RMPP-code.patch
seems to be the same patch as 0e442afd92fcdde2cc63b6f25556b8934e42b7d2.
-----Original Message-----
From: Hefty, Sean [mailto:sean.hefty-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org]
Sent: Monday, May 03, 2010 12:40 PM
To: Mike Heinz
Subject: RE: Hang in ib_umad when attempting to unregister.
>Where did you get those commit #s? I looked in my local copy of
>
>git://git.openfabrics.org/ofed_1_5/linux-2.6
>
>and they don't seem to be valid objects for that repo. Am I pulling from the
>wrong place?
These are from the upstream kernel.
>commit 6b2eef8fd78ff909c3396b8671d57c42559cc51d
>commit 0e442afd92fcdde2cc63b6f25556b8934e42b7d2
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox