* [PATCH 01/10] IB/hfi1, qib: Add ieth to the packet header definitions
[not found] ` <20160524194746.19706.42976.stgit-9QXIwq+3FY+1XWohqUldA0EOCMrvLtNR@public.gmane.org>
@ 2016-05-24 19:50 ` Dennis Dalessandro
2016-05-24 19:50 ` [PATCH 02/10] IB/hfi1: Add tracing support for send with invalidate opcode Dennis Dalessandro
` (9 subsequent siblings)
10 siblings, 0 replies; 18+ messages in thread
From: Dennis Dalessandro @ 2016-05-24 19:50 UTC (permalink / raw)
To: dledford-H+wXaHxf7aLQT0dZR+AlfA
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA, Mike Marciniszyn,
Jianxin Xiong
From: Jianxin Xiong <jianxin.xiong-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
A new union member "ieth" (Invalidate Extended Transport Header) is
added to the packet header definition in preparation of supporting
the send with invalidate opcode.
Reviewed-by: Mike Marciniszyn <mike.marciniszyn-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Reviewed-by: Dennis Dalessandro <dennis.dalessandro-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Jianxin Xiong <jianxin.xiong-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
drivers/infiniband/hw/hfi1/verbs.h | 1 +
drivers/infiniband/hw/qib/qib_verbs.h | 1 +
2 files changed, 2 insertions(+), 0 deletions(-)
diff --git a/drivers/infiniband/hw/hfi1/verbs.h b/drivers/infiniband/hw/hfi1/verbs.h
index 3ee2239..4883567 100644
--- a/drivers/infiniband/hw/hfi1/verbs.h
+++ b/drivers/infiniband/hw/hfi1/verbs.h
@@ -152,6 +152,7 @@ union ib_ehdrs {
} at;
__be32 imm_data;
__be32 aeth;
+ __be32 ieth;
struct ib_atomic_eth atomic_eth;
} __packed;
diff --git a/drivers/infiniband/hw/qib/qib_verbs.h b/drivers/infiniband/hw/qib/qib_verbs.h
index 6888f03..4f87815 100644
--- a/drivers/infiniband/hw/qib/qib_verbs.h
+++ b/drivers/infiniband/hw/qib/qib_verbs.h
@@ -159,6 +159,7 @@ struct qib_other_headers {
} at;
__be32 imm_data;
__be32 aeth;
+ __be32 ieth;
struct ib_atomic_eth atomic_eth;
} u;
} __packed;
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related [flat|nested] 18+ messages in thread* [PATCH 02/10] IB/hfi1: Add tracing support for send with invalidate opcode
[not found] ` <20160524194746.19706.42976.stgit-9QXIwq+3FY+1XWohqUldA0EOCMrvLtNR@public.gmane.org>
2016-05-24 19:50 ` [PATCH 01/10] IB/hfi1, qib: Add ieth to the packet header definitions Dennis Dalessandro
@ 2016-05-24 19:50 ` Dennis Dalessandro
2016-05-24 19:50 ` [PATCH 03/10] IB/hfi1: Fix hard lockup due to not using save/restore spin lock Dennis Dalessandro
` (8 subsequent siblings)
10 siblings, 0 replies; 18+ messages in thread
From: Dennis Dalessandro @ 2016-05-24 19:50 UTC (permalink / raw)
To: dledford-H+wXaHxf7aLQT0dZR+AlfA
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA, Mike Marciniszyn,
Jianxin Xiong
From: Jianxin Xiong <jianxin.xiong-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Enable trace generation for packets with the "Send Last with
Invalidate" and "Send Only with Invalidate" opcodes.
Reviewed-by: Mike Marciniszyn <mike.marciniszyn-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Reviewed-by: Dennis Dalessandro <dennis.dalessandro-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Jianxin Xiong <jianxin.xiong-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
drivers/infiniband/hw/hfi1/trace.c | 7 +++++++
drivers/infiniband/hw/hfi1/trace.h | 2 ++
drivers/infiniband/hw/hfi1/verbs.c | 2 ++
3 files changed, 11 insertions(+), 0 deletions(-)
diff --git a/drivers/infiniband/hw/hfi1/trace.c b/drivers/infiniband/hw/hfi1/trace.c
index caddb2a..79b2952 100644
--- a/drivers/infiniband/hw/hfi1/trace.c
+++ b/drivers/infiniband/hw/hfi1/trace.c
@@ -66,6 +66,7 @@ u8 ibhdr_exhdr_len(struct hfi1_ib_header *hdr)
#define RETH_PRN "reth vaddr 0x%.16llx rkey 0x%.8x dlen 0x%.8x"
#define AETH_PRN "aeth syn 0x%.2x %s msn 0x%.8x"
#define DETH_PRN "deth qkey 0x%.8x sqpn 0x%.6x"
+#define IETH_PRN "ieth rkey 0x%.8x"
#define ATOMICACKETH_PRN "origdata %lld"
#define ATOMICETH_PRN "vaddr 0x%llx rkey 0x%.8x sdata %lld cdata %lld"
@@ -166,6 +167,12 @@ const char *parse_everbs_hdrs(
be32_to_cpu(eh->ud.deth[0]),
be32_to_cpu(eh->ud.deth[1]) & RVT_QPN_MASK);
break;
+ /* ieth */
+ case OP(RC, SEND_LAST_WITH_INVALIDATE):
+ case OP(RC, SEND_ONLY_WITH_INVALIDATE):
+ trace_seq_printf(p, IETH_PRN,
+ be32_to_cpu(eh->ieth));
+ break;
}
trace_seq_putc(p, 0);
return ret;
diff --git a/drivers/infiniband/hw/hfi1/trace.h b/drivers/infiniband/hw/hfi1/trace.h
index a6c1adf..28c1d08 100644
--- a/drivers/infiniband/hw/hfi1/trace.h
+++ b/drivers/infiniband/hw/hfi1/trace.h
@@ -392,6 +392,8 @@ __print_symbolic(opcode, \
ib_opcode_name(RC_ATOMIC_ACKNOWLEDGE), \
ib_opcode_name(RC_COMPARE_SWAP), \
ib_opcode_name(RC_FETCH_ADD), \
+ ib_opcode_name(RC_SEND_LAST_WITH_INVALIDATE), \
+ ib_opcode_name(RC_SEND_ONLY_WITH_INVALIDATE), \
ib_opcode_name(UC_SEND_FIRST), \
ib_opcode_name(UC_SEND_MIDDLE), \
ib_opcode_name(UC_SEND_LAST), \
diff --git a/drivers/infiniband/hw/hfi1/verbs.c b/drivers/infiniband/hw/hfi1/verbs.c
index ae92ac5..849c4b9 100644
--- a/drivers/infiniband/hw/hfi1/verbs.c
+++ b/drivers/infiniband/hw/hfi1/verbs.c
@@ -335,6 +335,8 @@ const u8 hdr_len_by_opcode[256] = {
[IB_OPCODE_RC_ATOMIC_ACKNOWLEDGE] = 12 + 8 + 4,
[IB_OPCODE_RC_COMPARE_SWAP] = 12 + 8 + 28,
[IB_OPCODE_RC_FETCH_ADD] = 12 + 8 + 28,
+ [IB_OPCODE_RC_SEND_LAST_WITH_INVALIDATE] = 12 + 8 + 4,
+ [IB_OPCODE_RC_SEND_ONLY_WITH_INVALIDATE] = 12 + 8 + 4,
/* UC */
[IB_OPCODE_UC_SEND_FIRST] = 12 + 8,
[IB_OPCODE_UC_SEND_MIDDLE] = 12 + 8,
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related [flat|nested] 18+ messages in thread* [PATCH 03/10] IB/hfi1: Fix hard lockup due to not using save/restore spin lock
[not found] ` <20160524194746.19706.42976.stgit-9QXIwq+3FY+1XWohqUldA0EOCMrvLtNR@public.gmane.org>
2016-05-24 19:50 ` [PATCH 01/10] IB/hfi1, qib: Add ieth to the packet header definitions Dennis Dalessandro
2016-05-24 19:50 ` [PATCH 02/10] IB/hfi1: Add tracing support for send with invalidate opcode Dennis Dalessandro
@ 2016-05-24 19:50 ` Dennis Dalessandro
2016-05-24 19:50 ` [PATCH 04/10] IB/rdmavt: Max atomic value should be a u8 Dennis Dalessandro
` (7 subsequent siblings)
10 siblings, 0 replies; 18+ messages in thread
From: Dennis Dalessandro @ 2016-05-24 19:50 UTC (permalink / raw)
To: dledford-H+wXaHxf7aLQT0dZR+AlfA
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA, Mike Marciniszyn,
Sebastian Sanchez, stable-u79uwXL29TY76Z2rM5mHXA
From: Mike Marciniszyn <mike.marciniszyn-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Commit b9b06cb6feda
("IB/hfi1: Fix missing lock/unlock in verbs drain callback")
added a spin lock.
Unfortunately, the new lock code can be called from a base
level interrupt state, and an interrupt that can get stacked
will attempt to get the same lock.
Fix by using the flag save/restore spin lock variation.
Cc: stable-u79uwXL29TY76Z2rM5mHXA@public.gmane.org # 4.6+
Reviewed-by: Sebastian Sanchez <sebastian.sanchez-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Mike Marciniszyn <mike.marciniszyn-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Dennis Dalessandro <dennis.dalessandro-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
drivers/infiniband/hw/hfi1/qp.c | 5 +++--
1 files changed, 3 insertions(+), 2 deletions(-)
diff --git a/drivers/infiniband/hw/hfi1/qp.c b/drivers/infiniband/hw/hfi1/qp.c
index 14f889e..1a942ff 100644
--- a/drivers/infiniband/hw/hfi1/qp.c
+++ b/drivers/infiniband/hw/hfi1/qp.c
@@ -512,6 +512,7 @@ static void iowait_wakeup(struct iowait *wait, int reason)
static void iowait_sdma_drained(struct iowait *wait)
{
struct rvt_qp *qp = iowait_to_qp(wait);
+ unsigned long flags;
/*
* This happens when the send engine notes
@@ -519,12 +520,12 @@ static void iowait_sdma_drained(struct iowait *wait)
* do the flush work until that QP's
* sdma work has finished.
*/
- spin_lock(&qp->s_lock);
+ spin_lock_irqsave(&qp->s_lock, flags);
if (qp->s_flags & RVT_S_WAIT_DMA) {
qp->s_flags &= ~RVT_S_WAIT_DMA;
hfi1_schedule_send(qp);
}
- spin_unlock(&qp->s_lock);
+ spin_unlock_irqrestore(&qp->s_lock, flags);
}
/**
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related [flat|nested] 18+ messages in thread* [PATCH 04/10] IB/rdmavt: Max atomic value should be a u8
[not found] ` <20160524194746.19706.42976.stgit-9QXIwq+3FY+1XWohqUldA0EOCMrvLtNR@public.gmane.org>
` (2 preceding siblings ...)
2016-05-24 19:50 ` [PATCH 03/10] IB/hfi1: Fix hard lockup due to not using save/restore spin lock Dennis Dalessandro
@ 2016-05-24 19:50 ` Dennis Dalessandro
2016-05-24 19:50 ` [PATCH 05/10] IB/rdamvt: Fix rdmavt s_ack_queue sizing Dennis Dalessandro
` (6 subsequent siblings)
10 siblings, 0 replies; 18+ messages in thread
From: Dennis Dalessandro @ 2016-05-24 19:50 UTC (permalink / raw)
To: dledford-H+wXaHxf7aLQT0dZR+AlfA
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA, Mitko Haralanov,
Mike Marciniszyn
From: Mike Marciniszyn <mike.marciniszyn-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
This matches the ib_qp_attr size and
avoids a extremely large value when the lower level
driver registers.
As part of the patch, the u8 ordinals are moved to the
end of the struct to reduce pahole noted excesses.
Reviewed-by: Mitko Haralanov <mitko.haralanov-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Reviewed-by: Dennis Dalessandro <dennis.dalessandro-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Mike Marciniszyn <mike.marciniszyn-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
include/rdma/rdma_vt.h | 4 ++--
1 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/include/rdma/rdma_vt.h b/include/rdma/rdma_vt.h
index d57ceee..3f12da9 100644
--- a/include/rdma/rdma_vt.h
+++ b/include/rdma/rdma_vt.h
@@ -149,15 +149,15 @@ struct rvt_driver_params {
int qpn_res_end;
int nports;
int npkeys;
- u8 qos_shift;
char cq_name[RVT_CQN_MAX];
int node;
- int max_rdma_atomic;
int psn_mask;
int psn_shift;
int psn_modify_mask;
u32 core_cap_flags;
u32 max_mad_size;
+ u8 qos_shift;
+ u8 max_rdma_atomic;
};
/* Protection domain */
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related [flat|nested] 18+ messages in thread* [PATCH 05/10] IB/rdamvt: Fix rdmavt s_ack_queue sizing
[not found] ` <20160524194746.19706.42976.stgit-9QXIwq+3FY+1XWohqUldA0EOCMrvLtNR@public.gmane.org>
` (3 preceding siblings ...)
2016-05-24 19:50 ` [PATCH 04/10] IB/rdmavt: Max atomic value should be a u8 Dennis Dalessandro
@ 2016-05-24 19:50 ` Dennis Dalessandro
[not found] ` <20160524195039.19706.66866.stgit-9QXIwq+3FY+1XWohqUldA0EOCMrvLtNR@public.gmane.org>
2016-05-24 19:50 ` [PATCH 06/10] IB/hfi1: Update pkey table properly after link down or FM start Dennis Dalessandro
` (5 subsequent siblings)
10 siblings, 1 reply; 18+ messages in thread
From: Dennis Dalessandro @ 2016-05-24 19:50 UTC (permalink / raw)
To: dledford-H+wXaHxf7aLQT0dZR+AlfA
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA, Mitko Haralanov,
Mike Marciniszyn
From: Mike Marciniszyn <mike.marciniszyn-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
rdmavt allows the driver to specify the size of the ack queue, but
only uses it for the modify QP limit testing for setting the atomic
limit value.
The driver dependent size is now used to size the s_ack_queue ring
dynamicially.
Since the driver knows its size, the driver will use its define
for any ring size dependent code.
Reviewed-by: Mitko Haralanov <mitko.haralanov-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Mike Marciniszyn <mike.marciniszyn-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
drivers/infiniband/sw/rdmavt/qp.c | 22 ++++++++++++++++++++--
include/rdma/rdma_vt.h | 9 +++++++++
include/rdma/rdmavt_qp.h | 5 +----
3 files changed, 30 insertions(+), 6 deletions(-)
diff --git a/drivers/infiniband/sw/rdmavt/qp.c b/drivers/infiniband/sw/rdmavt/qp.c
index dd03b23..5fa4d4d 100644
--- a/drivers/infiniband/sw/rdmavt/qp.c
+++ b/drivers/infiniband/sw/rdmavt/qp.c
@@ -397,6 +397,7 @@ static void free_qpn(struct rvt_qpn_table *qpt, u32 qpn)
static void rvt_clear_mr_refs(struct rvt_qp *qp, int clr_sends)
{
unsigned n;
+ struct rvt_dev_info *rdi = ib_to_rvt(qp->ibqp.device);
if (test_and_clear_bit(RVT_R_REWIND_SGE, &qp->r_aflags))
rvt_put_ss(&qp->s_rdma_read_sge);
@@ -431,7 +432,7 @@ static void rvt_clear_mr_refs(struct rvt_qp *qp, int clr_sends)
if (qp->ibqp.qp_type != IB_QPT_RC)
return;
- for (n = 0; n < ARRAY_SIZE(qp->s_ack_queue); n++) {
+ for (n = 0; n < rvt_max_atomic(rdi); n++) {
struct rvt_ack_entry *e = &qp->s_ack_queue[n];
if (e->opcode == IB_OPCODE_RC_RDMA_READ_REQUEST &&
@@ -569,7 +570,12 @@ static void rvt_reset_qp(struct rvt_dev_info *rdi, struct rvt_qp *qp,
qp->s_ssn = 1;
qp->s_lsn = 0;
qp->s_mig_state = IB_MIG_MIGRATED;
- memset(qp->s_ack_queue, 0, sizeof(qp->s_ack_queue));
+ if (qp->s_ack_queue)
+ memset(
+ qp->s_ack_queue,
+ 0,
+ rvt_max_atomic(rdi) *
+ sizeof(*qp->s_ack_queue));
qp->r_head_ack_queue = 0;
qp->s_tail_ack_queue = 0;
qp->s_num_rd_atomic = 0;
@@ -677,6 +683,16 @@ struct ib_qp *rvt_create_qp(struct ib_pd *ibpd,
goto bail_swq;
RCU_INIT_POINTER(qp->next, NULL);
+ if (init_attr->qp_type == IB_QPT_RC) {
+ qp->s_ack_queue =
+ kzalloc_node(
+ sizeof(*qp->s_ack_queue) *
+ rvt_max_atomic(rdi),
+ gfp,
+ rdi->dparms.node);
+ if (!qp->s_ack_queue)
+ goto bail_qp;
+ }
/*
* Driver needs to set up it's private QP structure and do any
@@ -857,6 +873,7 @@ bail_driver_priv:
rdi->driver_f.qp_priv_free(rdi, qp);
bail_qp:
+ kfree(qp->s_ack_queue);
kfree(qp);
bail_swq:
@@ -1284,6 +1301,7 @@ int rvt_destroy_qp(struct ib_qp *ibqp)
vfree(qp->r_rq.wq);
vfree(qp->s_wq);
rdi->driver_f.qp_priv_free(rdi, qp);
+ kfree(qp->s_ack_queue);
kfree(qp);
return 0;
}
diff --git a/include/rdma/rdma_vt.h b/include/rdma/rdma_vt.h
index 3f12da9..16274e2 100644
--- a/include/rdma/rdma_vt.h
+++ b/include/rdma/rdma_vt.h
@@ -426,6 +426,15 @@ static inline unsigned rvt_get_npkeys(struct rvt_dev_info *rdi)
}
/*
+ * Return the max atomic suitable for determining
+ * the size of the ack ring buffer in a QP.
+ */
+static inline unsigned int rvt_max_atomic(struct rvt_dev_info *rdi)
+{
+ return rdi->dparms.max_rdma_atomic + 1;
+}
+
+/*
* Return the indexed PKEY from the port PKEY table.
*/
static inline u16 rvt_get_pkey(struct rvt_dev_info *rdi,
diff --git a/include/rdma/rdmavt_qp.h b/include/rdma/rdmavt_qp.h
index 0e1ff2a..6d23b87 100644
--- a/include/rdma/rdmavt_qp.h
+++ b/include/rdma/rdmavt_qp.h
@@ -211,8 +211,6 @@ struct rvt_mmap_info {
unsigned size;
};
-#define RVT_MAX_RDMA_ATOMIC 16
-
/*
* This structure holds the information that the send tasklet needs
* to send a RDMA read response or atomic operation.
@@ -282,8 +280,7 @@ struct rvt_qp {
atomic_t refcount ____cacheline_aligned_in_smp;
wait_queue_head_t wait;
- struct rvt_ack_entry s_ack_queue[RVT_MAX_RDMA_ATOMIC + 1]
- ____cacheline_aligned_in_smp;
+ struct rvt_ack_entry *s_ack_queue;
struct rvt_sge_state s_rdma_read_sge;
spinlock_t r_lock ____cacheline_aligned_in_smp; /* used for APM */
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related [flat|nested] 18+ messages in thread* [PATCH 06/10] IB/hfi1: Update pkey table properly after link down or FM start
[not found] ` <20160524194746.19706.42976.stgit-9QXIwq+3FY+1XWohqUldA0EOCMrvLtNR@public.gmane.org>
` (4 preceding siblings ...)
2016-05-24 19:50 ` [PATCH 05/10] IB/rdamvt: Fix rdmavt s_ack_queue sizing Dennis Dalessandro
@ 2016-05-24 19:50 ` Dennis Dalessandro
2016-05-24 19:50 ` [PATCH 07/10] IB/hfi1: Add a retry for the first-time QSFP access Dennis Dalessandro
` (4 subsequent siblings)
10 siblings, 0 replies; 18+ messages in thread
From: Dennis Dalessandro @ 2016-05-24 19:50 UTC (permalink / raw)
To: dledford-H+wXaHxf7aLQT0dZR+AlfA
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA, Dean Luick, Sebastian Sanchez
From: Sebastian Sanchez <sebastian.sanchez-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
When FM is disabled, and the HFI port on the switch is
changed from MgmtAllowed=YES to MgmtAllowed=NO and the
link is bounced, FULL_MGMT_P_KEY doesn't get cleared
from the pkey table. This also occurs when the QSFP
cable is moved from a switch port with MgmtAllowed=YES
to a MgmtAllowed=NO port. Clear pkey entry properly.
Also, when the driver is loaded and the switch port is
set to MgmtAllowed=NO, FULL_MGMT_P_KEY shouldn't be added
to pkey table after FM is started. Only set FULL_MGMT_P_KEY
in the pkey table if switch port is configured to
MgmtAllowed=YES.
Reviewed-by: Dean Luick <dean.luick-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Sebastian Sanchez <sebastian.sanchez-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
drivers/infiniband/hw/hfi1/chip.c | 9 +++++++++
drivers/infiniband/hw/hfi1/mad.c | 6 ++++++
2 files changed, 15 insertions(+), 0 deletions(-)
diff --git a/drivers/infiniband/hw/hfi1/chip.c b/drivers/infiniband/hw/hfi1/chip.c
index d8cc329..3b876da 100644
--- a/drivers/infiniband/hw/hfi1/chip.c
+++ b/drivers/infiniband/hw/hfi1/chip.c
@@ -1037,6 +1037,7 @@ static void dc_shutdown(struct hfi1_devdata *);
static void dc_start(struct hfi1_devdata *);
static int qos_rmt_entries(struct hfi1_devdata *dd, unsigned int *mp,
unsigned int *np);
+static void remove_full_mgmt_pkey(struct hfi1_pportdata *ppd);
/*
* Error interrupt table entry. This is used as input to the interrupt
@@ -6961,6 +6962,8 @@ void handle_link_down(struct work_struct *work)
}
reset_neighbor_info(ppd);
+ if (ppd->mgmt_allowed)
+ remove_full_mgmt_pkey(ppd);
/* disable the port */
clear_rcvctrl(ppd->dd, RCV_CTRL_RCV_PORT_ENABLE_SMASK);
@@ -7069,6 +7072,12 @@ static void add_full_mgmt_pkey(struct hfi1_pportdata *ppd)
(void)hfi1_set_ib_cfg(ppd, HFI1_IB_CFG_PKEYS, 0);
}
+static void remove_full_mgmt_pkey(struct hfi1_pportdata *ppd)
+{
+ ppd->pkeys[2] = 0;
+ (void)hfi1_set_ib_cfg(ppd, HFI1_IB_CFG_PKEYS, 0);
+}
+
/*
* Convert the given link width to the OPA link width bitmask.
*/
diff --git a/drivers/infiniband/hw/hfi1/mad.c b/drivers/infiniband/hw/hfi1/mad.c
index 17882dc..2190295 100644
--- a/drivers/infiniband/hw/hfi1/mad.c
+++ b/drivers/infiniband/hw/hfi1/mad.c
@@ -1403,6 +1403,12 @@ static int set_pkeys(struct hfi1_devdata *dd, u8 port, u16 *pkeys)
if (key == okey)
continue;
/*
+ * Don't update pkeys[2], if an HFI port without MgmtAllowed
+ * by neighbor is a switch.
+ */
+ if (i == 2 && !ppd->mgmt_allowed && ppd->neighbor_type == 1)
+ continue;
+ /*
* The SM gives us the complete PKey table. We have
* to ensure that we put the PKeys in the matching
* slots.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related [flat|nested] 18+ messages in thread* [PATCH 07/10] IB/hfi1: Add a retry for the first-time QSFP access
[not found] ` <20160524194746.19706.42976.stgit-9QXIwq+3FY+1XWohqUldA0EOCMrvLtNR@public.gmane.org>
` (5 preceding siblings ...)
2016-05-24 19:50 ` [PATCH 06/10] IB/hfi1: Update pkey table properly after link down or FM start Dennis Dalessandro
@ 2016-05-24 19:50 ` Dennis Dalessandro
[not found] ` <6395227d-617c-bd70-2822-6b53843f8611@redhat.com>
2016-05-24 19:51 ` [PATCH 08/10] IB/hfi1: Correct 8051 link parameter settings Dennis Dalessandro
` (3 subsequent siblings)
10 siblings, 1 reply; 18+ messages in thread
From: Dennis Dalessandro @ 2016-05-24 19:50 UTC (permalink / raw)
To: dledford-H+wXaHxf7aLQT0dZR+AlfA
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA, Dean Luick
From: Dean Luick <dean.luick-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Some QSFPs do not respond within the expected time, but later
appear fine. Add a limited retry on the first access.
Reviewed-by: Dennis Dalessandro <dennis.dalessandro-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Dean Luick <dean.luick-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
drivers/infiniband/hw/hfi1/qsfp.c | 16 ++++++++++++++++
1 files changed, 16 insertions(+), 0 deletions(-)
diff --git a/drivers/infiniband/hw/hfi1/qsfp.c b/drivers/infiniband/hw/hfi1/qsfp.c
index 2441669..aa7ed23 100644
--- a/drivers/infiniband/hw/hfi1/qsfp.c
+++ b/drivers/infiniband/hw/hfi1/qsfp.c
@@ -362,6 +362,7 @@ int refresh_qsfp_cache(struct hfi1_pportdata *ppd, struct qsfp_data *cp)
{
u32 target = ppd->dd->hfi1_id;
int ret;
+ int retry_count;
unsigned long flags;
u8 *cache = &cp->cache[0];
@@ -376,8 +377,23 @@ int refresh_qsfp_cache(struct hfi1_pportdata *ppd, struct qsfp_data *cp)
goto bail;
}
+ retry_count = 0;
+retry:
ret = qsfp_read(ppd, target, 0, cache, QSFP_PAGESIZE);
if (ret != QSFP_PAGESIZE) {
+ /*
+ * This is the first QSFP access the driver makes.
+ * Some QSFPs don't respond within the expected time,
+ * but later appear fine. Retry at 2s intervals for up
+ * to 10s.
+ */
+ if (ret < 0 && retry_count < 5) {
+ retry_count++;
+ dd_dev_info(ppd->dd, "%s: QSFP not responding, waiting and retrying %d\n",
+ __func__, retry_count);
+ msleep(2000);
+ goto retry;
+ }
dd_dev_info(ppd->dd,
"%s: Page 0 read failed, expected %d, got %d\n",
__func__, QSFP_PAGESIZE, ret);
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related [flat|nested] 18+ messages in thread* [PATCH 08/10] IB/hfi1: Correct 8051 link parameter settings
[not found] ` <20160524194746.19706.42976.stgit-9QXIwq+3FY+1XWohqUldA0EOCMrvLtNR@public.gmane.org>
` (6 preceding siblings ...)
2016-05-24 19:50 ` [PATCH 07/10] IB/hfi1: Add a retry for the first-time QSFP access Dennis Dalessandro
@ 2016-05-24 19:51 ` Dennis Dalessandro
2016-05-24 19:51 ` [PATCH 09/10] IB/hfi1: Fix pio map initialization Dennis Dalessandro
` (2 subsequent siblings)
10 siblings, 0 replies; 18+ messages in thread
From: Dennis Dalessandro @ 2016-05-24 19:51 UTC (permalink / raw)
To: dledford-H+wXaHxf7aLQT0dZR+AlfA
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA, Dean Luick
From: Dean Luick <dean.luick-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Two 8051 link settings, external device config and tuning method,
were written in the wrong location and the previous settings were
not cleared. For both, clear the old value and write the new
value.
Fixes: 8ebd4cf1852a ("staging/rdma/hfi1: Add active and optical cable support")
Reviewed-by: Dennis Dalessandro <dennis.dalessandro-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Dean Luick <dean.luick-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
drivers/infiniband/hw/hfi1/chip.h | 6 ++++++
drivers/infiniband/hw/hfi1/platform.c | 6 ++++--
2 files changed, 10 insertions(+), 2 deletions(-)
diff --git a/drivers/infiniband/hw/hfi1/chip.h b/drivers/infiniband/hw/hfi1/chip.h
index 1948706..66a3279 100644
--- a/drivers/infiniband/hw/hfi1/chip.h
+++ b/drivers/infiniband/hw/hfi1/chip.h
@@ -398,6 +398,12 @@
/* Lane ID for general configuration registers */
#define GENERAL_CONFIG 4
+/* LINK_TUNING_PARAMETERS fields */
+#define TUNING_METHOD_SHIFT 24
+
+/* LINK_OPTIMIZATION_SETTINGS fields */
+#define ENABLE_EXT_DEV_CONFIG_SHIFT 24
+
/* LOAD_DATA 8051 command shifts and fields */
#define LOAD_DATA_FIELD_ID_SHIFT 40
#define LOAD_DATA_FIELD_ID_MASK 0xfull
diff --git a/drivers/infiniband/hw/hfi1/platform.c b/drivers/infiniband/hw/hfi1/platform.c
index ae0e498..03df932 100644
--- a/drivers/infiniband/hw/hfi1/platform.c
+++ b/drivers/infiniband/hw/hfi1/platform.c
@@ -540,7 +540,8 @@ static void apply_tunings(
/* Enable external device config if channel is limiting active */
read_8051_config(ppd->dd, LINK_OPTIMIZATION_SETTINGS,
GENERAL_CONFIG, &config_data);
- config_data |= limiting_active;
+ config_data &= ~(0xff << ENABLE_EXT_DEV_CONFIG_SHIFT);
+ config_data |= ((u32)limiting_active << ENABLE_EXT_DEV_CONFIG_SHIFT);
ret = load_8051_config(ppd->dd, LINK_OPTIMIZATION_SETTINGS,
GENERAL_CONFIG, config_data);
if (ret != HCMD_SUCCESS)
@@ -553,7 +554,8 @@ static void apply_tunings(
/* Pass tuning method to 8051 */
read_8051_config(ppd->dd, LINK_TUNING_PARAMETERS, GENERAL_CONFIG,
&config_data);
- config_data |= tuning_method;
+ config_data &= ~(0xff << TUNING_METHOD_SHIFT);
+ config_data |= ((u32)tuning_method << TUNING_METHOD_SHIFT);
ret = load_8051_config(ppd->dd, LINK_TUNING_PARAMETERS, GENERAL_CONFIG,
config_data);
if (ret != HCMD_SUCCESS)
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related [flat|nested] 18+ messages in thread* [PATCH 09/10] IB/hfi1: Fix pio map initialization
[not found] ` <20160524194746.19706.42976.stgit-9QXIwq+3FY+1XWohqUldA0EOCMrvLtNR@public.gmane.org>
` (7 preceding siblings ...)
2016-05-24 19:51 ` [PATCH 08/10] IB/hfi1: Correct 8051 link parameter settings Dennis Dalessandro
@ 2016-05-24 19:51 ` Dennis Dalessandro
2016-05-24 19:51 ` [PATCH 10/10] IB/hfi1: Fix credit return threshold adjustment Dennis Dalessandro
2016-05-26 16:25 ` [PATCH 00/10] IB/hfi1,rdmavt: Another set of fixes for 4.7 Doug Ledford
10 siblings, 0 replies; 18+ messages in thread
From: Dennis Dalessandro @ 2016-05-24 19:51 UTC (permalink / raw)
To: dledford-H+wXaHxf7aLQT0dZR+AlfA
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA, Mike Marciniszyn, Jubin John,
Jianxin Xiong
From: Jubin John <jubin.john-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
The pio map initialization function is off by 1 causing the last
kernel send context that is allocated to not get mapped into the
pio map which leads to the last kernel send context not being used
by any of the qps.
The send context reserved for VL15 is taken care of by setting the
scontext variable that is used as the index into the kernel send
context array to 1 and does not need to be accounted for in the
kernel send context counting loop as it is currently done.
Fix the kernel send context counting loop to account for all the
allocated send contexts and map all of them to the different VLs.
Reviewed-by: Dennis Dalessandro <dennis.dalessandro-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Reviewed-by: Mike Marciniszyn <mike.marciniszyn-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Reviewed-by: Jianxin Xiong <jianxin.xiong-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Jubin John <jubin.john-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
drivers/infiniband/hw/hfi1/pio.c | 3 +--
1 files changed, 1 insertions(+), 2 deletions(-)
diff --git a/drivers/infiniband/hw/hfi1/pio.c b/drivers/infiniband/hw/hfi1/pio.c
index c67b9ad..d5edb1a 100644
--- a/drivers/infiniband/hw/hfi1/pio.c
+++ b/drivers/infiniband/hw/hfi1/pio.c
@@ -1835,8 +1835,7 @@ int pio_map_init(struct hfi1_devdata *dd, u8 port, u8 num_vls, u8 *vl_scontexts)
struct pio_vl_map *oldmap, *newmap;
if (!vl_scontexts) {
- /* send context 0 reserved for VL15 */
- for (i = 1; i < dd->num_send_contexts; i++)
+ for (i = 0; i < dd->num_send_contexts; i++)
if (dd->send_contexts[i].type == SC_KERNEL)
num_kernel_send_contexts++;
/* truncate divide */
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related [flat|nested] 18+ messages in thread* [PATCH 10/10] IB/hfi1: Fix credit return threshold adjustment
[not found] ` <20160524194746.19706.42976.stgit-9QXIwq+3FY+1XWohqUldA0EOCMrvLtNR@public.gmane.org>
` (8 preceding siblings ...)
2016-05-24 19:51 ` [PATCH 09/10] IB/hfi1: Fix pio map initialization Dennis Dalessandro
@ 2016-05-24 19:51 ` Dennis Dalessandro
[not found] ` <20160524195111.19706.76829.stgit-9QXIwq+3FY+1XWohqUldA0EOCMrvLtNR@public.gmane.org>
2016-05-26 16:25 ` [PATCH 00/10] IB/hfi1,rdmavt: Another set of fixes for 4.7 Doug Ledford
10 siblings, 1 reply; 18+ messages in thread
From: Dennis Dalessandro @ 2016-05-24 19:51 UTC (permalink / raw)
To: dledford-H+wXaHxf7aLQT0dZR+AlfA
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA, Mike Marciniszyn, Jubin John,
Jianxin Xiong
From: Jubin John <jubin.john-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
The credit return threshold adjustment on mtu change algorithm does not
take into account all the kernel send contexts that are assigned per VL.
Use the pio send context map to adjust the credit return thresholds for
all the allocated and assigned kernel send contexts based on the MTU
adjustment per VL.
The pio send context map can be changed dynamically based on the actual
number of operational vls which is set by the fabric manager. When this
happens update the credit return threshold values for all the remapped
kernel send contexts.
Reviewed-by: Dennis Dalessandro <dennis.dalessandro-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Reviewed-by: Mike Marciniszyn <mike.marciniszyn-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Reviewed-by: Jianxin Xiong <jianxin.xiong-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Jubin John <jubin.john-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
drivers/infiniband/hw/hfi1/chip.c | 7 +++++--
drivers/infiniband/hw/hfi1/pio.c | 21 +++++++++++++++++++--
2 files changed, 24 insertions(+), 4 deletions(-)
diff --git a/drivers/infiniband/hw/hfi1/chip.c b/drivers/infiniband/hw/hfi1/chip.c
index 3b876da..be4fd3e 100644
--- a/drivers/infiniband/hw/hfi1/chip.c
+++ b/drivers/infiniband/hw/hfi1/chip.c
@@ -9777,7 +9777,7 @@ static void set_send_length(struct hfi1_pportdata *ppd)
u64 len1 = 0, len2 = (((dd->vld[15].mtu + max_hb) >> 2)
& SEND_LEN_CHECK1_LEN_VL15_MASK) <<
SEND_LEN_CHECK1_LEN_VL15_SHIFT;
- int i;
+ int i, j;
u32 thres;
for (i = 0; i < ppd->vls_supported; i++) {
@@ -9801,7 +9801,10 @@ static void set_send_length(struct hfi1_pportdata *ppd)
sc_mtu_to_threshold(dd->vld[i].sc,
dd->vld[i].mtu,
dd->rcd[0]->rcvhdrqentsize));
- sc_set_cr_threshold(dd->vld[i].sc, thres);
+ for (j = 0; j < INIT_SC_PER_VL; j++)
+ sc_set_cr_threshold(
+ pio_select_send_context_vl(dd, j, i),
+ thres);
}
thres = min(sc_percent_to_threshold(dd->vld[15].sc, 50),
sc_mtu_to_threshold(dd->vld[15].sc,
diff --git a/drivers/infiniband/hw/hfi1/pio.c b/drivers/infiniband/hw/hfi1/pio.c
index d5edb1a..d5a9a5c 100644
--- a/drivers/infiniband/hw/hfi1/pio.c
+++ b/drivers/infiniband/hw/hfi1/pio.c
@@ -1832,6 +1832,7 @@ int pio_map_init(struct hfi1_devdata *dd, u8 port, u8 num_vls, u8 *vl_scontexts)
int scontext = 1;
int num_kernel_send_contexts = 0;
u8 lvl_scontexts[OPA_MAX_VLS];
+ u32 thres;
struct pio_vl_map *oldmap, *newmap;
if (!vl_scontexts) {
@@ -1872,11 +1873,27 @@ int pio_map_init(struct hfi1_devdata *dd, u8 port, u8 num_vls, u8 *vl_scontexts)
if (!newmap->map[i])
goto bail;
newmap->map[i]->mask = (1 << ilog2(sz)) - 1;
- /* assign send contexts */
+ /*
+ * assign send contexts and
+ * adjust credit return threshold
+ */
for (j = 0; j < sz; j++) {
- if (dd->kernel_send_context[scontext])
+ if (dd->kernel_send_context[scontext]) {
newmap->map[i]->ksc[j] =
dd->kernel_send_context[scontext];
+ thres = min(sc_percent_to_threshold(
+ dd->kernel_send_context
+ [scontext], 50),
+ sc_mtu_to_threshold(
+ dd->kernel_send_context
+ [scontext], dd->vld[i].mtu,
+ dd->rcd[0]->rcvhdrqentsize)
+ );
+ sc_set_cr_threshold(
+ dd->
+ kernel_send_context[scontext],
+ thres);
+ }
if (++scontext >= first_scontext +
vl_scontexts[i])
/* wrap back to first send context */
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related [flat|nested] 18+ messages in thread* Re: [PATCH 00/10] IB/hfi1,rdmavt: Another set of fixes for 4.7
[not found] ` <20160524194746.19706.42976.stgit-9QXIwq+3FY+1XWohqUldA0EOCMrvLtNR@public.gmane.org>
` (9 preceding siblings ...)
2016-05-24 19:51 ` [PATCH 10/10] IB/hfi1: Fix credit return threshold adjustment Dennis Dalessandro
@ 2016-05-26 16:25 ` Doug Ledford
10 siblings, 0 replies; 18+ messages in thread
From: Doug Ledford @ 2016-05-26 16:25 UTC (permalink / raw)
To: Dennis Dalessandro; +Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
[-- Attachment #1: Type: text/plain, Size: 2351 bytes --]
On 05/24/2016 03:50 PM, Dennis Dalessandro wrote:
> Doug, here are a few more fixes that we would like to get into 4.7 if possible.
> These apply on top of the series to get the driver out of staging. I can always
> respin if needed.
>
> Included here is one patch which has a stable tag for 4.6 since it fixes a hard
> lockup. The others are minor bug fixes.
>
> Patches can also be viewed in my repo at:
> https://github.com/ddalessa/kernel/tree/for-4.7
>
> ---
>
> Dean Luick (2):
> IB/hfi1: Add a retry for the first-time QSFP access
> IB/hfi1: Correct 8051 link parameter settings
>
> Jianxin Xiong (2):
> IB/hfi1,qib: Add ieth to the packet header definitions
> IB/hfi1: Add tracing support for send with invalidate opcode
>
> Jubin John (2):
> IB/hfi1: Fix pio map initialization
> IB/hfi1: Fix credit return threshold adjustment
>
> Mike Marciniszyn (3):
> IB/hfi1: Fix hard lockup due to not using save/restore spin lock
> IB/rdmavt: Max atomic value should be a u8
> IB/rdamvt: Fix rdmavt s_ack_queue sizing
>
> Sebastian Sanchez (1):
> IB/hfi1: Update pkey table properly after link down or FM start
>
>
> drivers/infiniband/hw/hfi1/chip.c | 16 ++++++++++++++--
> drivers/infiniband/hw/hfi1/chip.h | 6 ++++++
> drivers/infiniband/hw/hfi1/mad.c | 6 ++++++
> drivers/infiniband/hw/hfi1/pio.c | 24 ++++++++++++++++++++----
> drivers/infiniband/hw/hfi1/platform.c | 6 ++++--
> drivers/infiniband/hw/hfi1/qp.c | 5 +++--
> drivers/infiniband/hw/hfi1/qsfp.c | 16 ++++++++++++++++
> drivers/infiniband/hw/hfi1/trace.c | 7 +++++++
> drivers/infiniband/hw/hfi1/trace.h | 2 ++
> drivers/infiniband/hw/hfi1/verbs.c | 2 ++
> drivers/infiniband/hw/hfi1/verbs.h | 1 +
> drivers/infiniband/hw/qib/qib_verbs.h | 1 +
> drivers/infiniband/sw/rdmavt/qp.c | 22 ++++++++++++++++++++--
> include/rdma/rdma_vt.h | 13 +++++++++++--
> include/rdma/rdmavt_qp.h | 5 +----
> 15 files changed, 114 insertions(+), 18 deletions(-)
>
I took everything but 7 and 10. Please address the comments on those two.
--
Doug Ledford <dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
GPG KeyID: 0E572FDD
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 884 bytes --]
^ permalink raw reply [flat|nested] 18+ messages in thread