* [PATCH rdma-core 1/5] libhns: Add new PCI device matching for hip08
[not found] ` <1508245946-145697-1-git-send-email-oulijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
@ 2017-10-17 13:12 ` Lijun Ou
2017-10-17 13:12 ` [PATCH rdma-core 2/5] libhns: Introduce QP operations refered to hip08 RoCE device Lijun Ou
` (4 subsequent siblings)
5 siblings, 0 replies; 8+ messages in thread
From: Lijun Ou @ 2017-10-17 13:12 UTC (permalink / raw)
To: dledford-H+wXaHxf7aLQT0dZR+AlfA, leon-DgEjT+Ai2ygdnm+yROfE0A,
jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA
Init userspace frame for matching hip08 RoCE device.
Signed-off-by: Lijun Ou <oulijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
Signed-off-by: Wei Hu <xavier.huwei-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
---
providers/hns/hns_roce_u.c | 12 +++++++++++-
providers/hns/hns_roce_u.h | 5 ++++-
providers/hns/hns_roce_u_hw_v2.c | 41 ++++++++++++++++++++++++++++++++++++++++
3 files changed, 56 insertions(+), 2 deletions(-)
create mode 100644 providers/hns/hns_roce_u_hw_v2.c
diff --git a/providers/hns/hns_roce_u.c b/providers/hns/hns_roce_u.c
index c27212a..ababd9c 100644
--- a/providers/hns/hns_roce_u.c
+++ b/providers/hns/hns_roce_u.c
@@ -1,5 +1,5 @@
/*
- * Copyright (c) 2016 Hisilicon Limited.
+ * Copyright (c) 2016-2017 Hisilicon Limited.
*
* This software is available to you under a choice of one of two
* licenses. You may choose to be licensed under the terms of the GNU
@@ -44,10 +44,20 @@
#define HID_LEN 15
#define DEV_MATCH_LEN 128
+#ifndef PCI_VENDOR_ID_HUAWEI
+#define PCI_VENDOR_ID_HUAWEI 0x19E5
+#endif
+
static const struct verbs_match_ent hca_table[] = {
VERBS_MODALIAS_MATCH("acpi*:HISI00D1:*", &hns_roce_u_hw_v1),
VERBS_MODALIAS_MATCH("of:N*T*Chisilicon,hns-roce-v1C*", &hns_roce_u_hw_v1),
VERBS_MODALIAS_MATCH("of:N*T*Chisilicon,hns-roce-v1", &hns_roce_u_hw_v1),
+ VERBS_PCI_MATCH(PCI_VENDOR_ID_HUAWEI, 0xA222, &hns_roce_u_hw_v2),
+ VERBS_PCI_MATCH(PCI_VENDOR_ID_HUAWEI, 0xA223, &hns_roce_u_hw_v2),
+ VERBS_PCI_MATCH(PCI_VENDOR_ID_HUAWEI, 0xA224, &hns_roce_u_hw_v2),
+ VERBS_PCI_MATCH(PCI_VENDOR_ID_HUAWEI, 0xA225, &hns_roce_u_hw_v2),
+ VERBS_PCI_MATCH(PCI_VENDOR_ID_HUAWEI, 0xA226, &hns_roce_u_hw_v2),
+ VERBS_PCI_MATCH(PCI_VENDOR_ID_HUAWEI, 0xA227, &hns_roce_u_hw_v2),
{}
};
diff --git a/providers/hns/hns_roce_u.h b/providers/hns/hns_roce_u.h
index d0e2351..568bfc3 100644
--- a/providers/hns/hns_roce_u.h
+++ b/providers/hns/hns_roce_u.h
@@ -1,5 +1,5 @@
/*
- * Copyright (c) 2016 Hisilicon Limited.
+ * Copyright (c) 2016-2017 Hisilicon Limited.
*
* This software is available to you under a choice of one of two
* licenses. You may choose to be licensed under the terms of the GNU
@@ -50,6 +50,8 @@
#define HNS_ROCE_TPTR_OFFSET 0x1000
#define HNS_ROCE_HW_VER1 ('h' << 24 | 'i' << 16 | '0' << 8 | '6')
+#define HNS_ROCE_HW_VER2 ('h' << 24 | 'i' << 16 | '0' << 8 | '8')
+
#define PFX "hns: "
#define roce_get_field(origin, mask, shift) \
@@ -249,5 +251,6 @@ void hns_roce_free_buf(struct hns_roce_buf *buf);
void hns_roce_init_qp_indices(struct hns_roce_qp *qp);
extern struct hns_roce_u_hw hns_roce_u_hw_v1;
+extern struct hns_roce_u_hw hns_roce_u_hw_v2;
#endif /* _HNS_ROCE_U_H */
diff --git a/providers/hns/hns_roce_u_hw_v2.c b/providers/hns/hns_roce_u_hw_v2.c
new file mode 100644
index 0000000..a580bad
--- /dev/null
+++ b/providers/hns/hns_roce_u_hw_v2.c
@@ -0,0 +1,41 @@
+/*
+ * Copyright (c) 2016-2017 Hisilicon Limited.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses. You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#include <stdio.h>
+#include <string.h>
+#include <malloc.h>
+#include "hns_roce_u.h"
+#include "hns_roce_u_db.h"
+
+struct hns_roce_u_hw hns_roce_u_hw_v2 = {
+ .hw_version = HNS_ROCE_HW_VER2,
+};
--
1.9.1
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related [flat|nested] 8+ messages in thread* [PATCH rdma-core 2/5] libhns: Introduce QP operations refered to hip08 RoCE device
[not found] ` <1508245946-145697-1-git-send-email-oulijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
2017-10-17 13:12 ` [PATCH rdma-core 1/5] libhns: Add new PCI device matching for hip08 Lijun Ou
@ 2017-10-17 13:12 ` Lijun Ou
2017-10-17 13:12 ` [PATCH rdma-core 3/5] libhns: Introduce CQ operations refered to hip08 device Lijun Ou
` (3 subsequent siblings)
5 siblings, 0 replies; 8+ messages in thread
From: Lijun Ou @ 2017-10-17 13:12 UTC (permalink / raw)
To: dledford-H+wXaHxf7aLQT0dZR+AlfA, leon-DgEjT+Ai2ygdnm+yROfE0A,
jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA
QP APIs need to operate doorbell and cqe. the definition of
doorbell and cqe in The different hardware is discrepant.
Hence, This patch introduces the QP operations of hip08
RoCE hardware.
Signed-off-by: Lijun Ou <oulijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
Signed-off-by: Wei Hu <xavier.huwei-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
---
providers/hns/hns_roce_u_hw_v2.c | 190 +++++++++++++++++++++++++++++++++++++++
providers/hns/hns_roce_u_hw_v2.h | 136 ++++++++++++++++++++++++++++
providers/hns/hns_roce_u_verbs.c | 31 ++++---
3 files changed, 344 insertions(+), 13 deletions(-)
create mode 100644 providers/hns/hns_roce_u_hw_v2.h
diff --git a/providers/hns/hns_roce_u_hw_v2.c b/providers/hns/hns_roce_u_hw_v2.c
index a580bad..bf1c3f3 100644
--- a/providers/hns/hns_roce_u_hw_v2.c
+++ b/providers/hns/hns_roce_u_hw_v2.c
@@ -35,7 +35,197 @@
#include <malloc.h>
#include "hns_roce_u.h"
#include "hns_roce_u_db.h"
+#include "hns_roce_u_hw_v2.h"
+
+static struct hns_roce_v2_cqe *get_cqe_v2(struct hns_roce_cq *cq, int entry)
+{
+ return cq->buf.buf + entry * HNS_ROCE_CQE_ENTRY_SIZE;
+}
+
+static void *get_sw_cqe_v2(struct hns_roce_cq *cq, int n)
+{
+ struct hns_roce_v2_cqe *cqe = get_cqe_v2(cq, n & cq->ibv_cq.cqe);
+
+ return (!!(roce_get_bit(cqe->byte_4, CQE_BYTE_4_OWNER_S)) ^
+ !!(n & (cq->ibv_cq.cqe + 1))) ? cqe : NULL;
+}
+
+static void hns_roce_v2_update_cq_cons_index(struct hns_roce_context *ctx,
+ struct hns_roce_cq *cq)
+{
+ struct hns_roce_v2_cq_db cq_db;
+
+ cq_db.byte_4 = 0;
+ cq_db.parameter = 0;
+
+ roce_set_field(cq_db.byte_4, DB_BYTE_4_TAG_M, DB_BYTE_4_TAG_S, cq->cqn);
+ roce_set_field(cq_db.byte_4, DB_BYTE_4_CMD_M, DB_BYTE_4_CMD_S, 0x3);
+
+ roce_set_field(cq_db.parameter, CQ_DB_PARAMETER_CQ_CONSUMER_IDX_M,
+ CQ_DB_PARAMETER_CQ_CONSUMER_IDX_S,
+ cq->cons_index & ((cq->cq_depth << 1) - 1));
+ roce_set_field(cq_db.parameter, CQ_DB_PARAMETER_CMD_SN_M,
+ CQ_DB_PARAMETER_CMD_SN_S, 1);
+ roce_set_bit(cq_db.parameter, CQ_DB_PARAMETER_NOTIFY_S, 0);
+
+ hns_roce_write64((uint32_t *)&cq_db, ctx, ROCEE_VF_DB_CFG0_OFFSET);
+}
+
+static void hns_roce_v2_clear_qp(struct hns_roce_context *ctx, uint32_t qpn)
+{
+ int tind = (qpn & (ctx->num_qps - 1)) >> ctx->qp_table_shift;
+
+ if (!--ctx->qp_table[tind].refcnt)
+ free(ctx->qp_table[tind].table);
+ else
+ ctx->qp_table[tind].table[qpn & ctx->qp_table_mask] = NULL;
+}
+
+static void __hns_roce_v2_cq_clean(struct hns_roce_cq *cq, uint32_t qpn,
+ struct hns_roce_srq *srq)
+{
+ int nfreed = 0;
+ uint32_t prod_index;
+ uint8_t owner_bit = 0;
+ struct hns_roce_v2_cqe *cqe, *dest;
+ struct hns_roce_context *ctx = to_hr_ctx(cq->ibv_cq.context);
+
+ for (prod_index = cq->cons_index; get_sw_cqe_v2(cq, prod_index);
+ ++prod_index)
+ if (prod_index == cq->cons_index + cq->ibv_cq.cqe)
+ break;
+
+ while ((int) --prod_index - (int) cq->cons_index >= 0) {
+ cqe = get_cqe_v2(cq, prod_index & cq->ibv_cq.cqe);
+ if ((roce_get_field(cqe->byte_16, CQE_BYTE_16_LCL_QPN_M,
+ CQE_BYTE_16_LCL_QPN_S) & 0xffffff) == qpn) {
+ ++nfreed;
+ } else if (nfreed) {
+ dest = get_cqe_v2(cq,
+ (prod_index + nfreed) & cq->ibv_cq.cqe);
+ owner_bit = roce_get_bit(dest->byte_4,
+ CQE_BYTE_4_OWNER_S);
+ memcpy(dest, cqe, sizeof(*cqe));
+ roce_set_bit(dest->byte_4, CQE_BYTE_4_OWNER_S,
+ owner_bit);
+ }
+ }
+
+ if (nfreed) {
+ cq->cons_index += nfreed;
+ udma_to_device_barrier();
+ hns_roce_v2_update_cq_cons_index(ctx, cq);
+ }
+}
+
+static void hns_roce_v2_cq_clean(struct hns_roce_cq *cq, unsigned int qpn,
+ struct hns_roce_srq *srq)
+{
+ pthread_spin_lock(&cq->lock);
+ __hns_roce_v2_cq_clean(cq, qpn, srq);
+ pthread_spin_unlock(&cq->lock);
+}
+
+static int hns_roce_u_v2_modify_qp(struct ibv_qp *qp, struct ibv_qp_attr *attr,
+ int attr_mask)
+{
+ int ret;
+ struct ibv_modify_qp cmd;
+ struct hns_roce_qp *hr_qp = to_hr_qp(qp);
+
+ ret = ibv_cmd_modify_qp(qp, attr, attr_mask, &cmd, sizeof(cmd));
+
+ if (!ret && (attr_mask & IBV_QP_STATE) &&
+ attr->qp_state == IBV_QPS_RESET) {
+ hns_roce_v2_cq_clean(to_hr_cq(qp->recv_cq), qp->qp_num,
+ qp->srq ? to_hr_srq(qp->srq) : NULL);
+ if (qp->send_cq != qp->recv_cq)
+ hns_roce_v2_cq_clean(to_hr_cq(qp->send_cq), qp->qp_num,
+ NULL);
+
+ hns_roce_init_qp_indices(to_hr_qp(qp));
+ }
+
+ if (!ret && (attr_mask & IBV_QP_PORT)) {
+ hr_qp->port_num = attr->port_num;
+ printf("hr_qp->port_num= 0x%x\n", hr_qp->port_num);
+ }
+
+ hr_qp->sl = attr->ah_attr.sl;
+
+ return ret;
+}
+
+static void hns_roce_lock_cqs(struct ibv_qp *qp)
+{
+ struct hns_roce_cq *send_cq = to_hr_cq(qp->send_cq);
+ struct hns_roce_cq *recv_cq = to_hr_cq(qp->recv_cq);
+
+ if (send_cq == recv_cq) {
+ pthread_spin_lock(&send_cq->lock);
+ } else if (send_cq->cqn < recv_cq->cqn) {
+ pthread_spin_lock(&send_cq->lock);
+ pthread_spin_lock(&recv_cq->lock);
+ } else {
+ pthread_spin_lock(&recv_cq->lock);
+ pthread_spin_lock(&send_cq->lock);
+ }
+}
+
+static void hns_roce_unlock_cqs(struct ibv_qp *qp)
+{
+ struct hns_roce_cq *send_cq = to_hr_cq(qp->send_cq);
+ struct hns_roce_cq *recv_cq = to_hr_cq(qp->recv_cq);
+
+ if (send_cq == recv_cq) {
+ pthread_spin_unlock(&send_cq->lock);
+ } else if (send_cq->cqn < recv_cq->cqn) {
+ pthread_spin_unlock(&recv_cq->lock);
+ pthread_spin_unlock(&send_cq->lock);
+ } else {
+ pthread_spin_unlock(&send_cq->lock);
+ pthread_spin_unlock(&recv_cq->lock);
+ }
+}
+
+static int hns_roce_u_v2_destroy_qp(struct ibv_qp *ibqp)
+{
+ int ret;
+ struct hns_roce_qp *qp = to_hr_qp(ibqp);
+
+ pthread_mutex_lock(&to_hr_ctx(ibqp->context)->qp_table_mutex);
+ ret = ibv_cmd_destroy_qp(ibqp);
+ if (ret) {
+ pthread_mutex_unlock(&to_hr_ctx(ibqp->context)->qp_table_mutex);
+ return ret;
+ }
+
+ hns_roce_lock_cqs(ibqp);
+
+ __hns_roce_v2_cq_clean(to_hr_cq(ibqp->recv_cq), ibqp->qp_num,
+ ibqp->srq ? to_hr_srq(ibqp->srq) : NULL);
+
+ if (ibqp->send_cq != ibqp->recv_cq)
+ __hns_roce_v2_cq_clean(to_hr_cq(ibqp->send_cq), ibqp->qp_num,
+ NULL);
+
+ hns_roce_v2_clear_qp(to_hr_ctx(ibqp->context), ibqp->qp_num);
+
+ hns_roce_unlock_cqs(ibqp);
+ pthread_mutex_unlock(&to_hr_ctx(ibqp->context)->qp_table_mutex);
+
+ free(qp->sq.wrid);
+ if (qp->rq.wqe_cnt)
+ free(qp->rq.wrid);
+
+ hns_roce_free_buf(&qp->buf);
+ free(qp);
+
+ return ret;
+}
struct hns_roce_u_hw hns_roce_u_hw_v2 = {
.hw_version = HNS_ROCE_HW_VER2,
+ .modify_qp = hns_roce_u_v2_modify_qp,
+ .destroy_qp = hns_roce_u_v2_destroy_qp,
};
diff --git a/providers/hns/hns_roce_u_hw_v2.h b/providers/hns/hns_roce_u_hw_v2.h
new file mode 100644
index 0000000..d7fcf94
--- /dev/null
+++ b/providers/hns/hns_roce_u_hw_v2.h
@@ -0,0 +1,136 @@
+/*
+ * Copyright (c) 2016-2017 Hisilicon Limited.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses. You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#ifndef _HNS_ROCE_U_HW_V2_H
+#define _HNS_ROCE_U_HW_V2_H
+
+/* V2 REG DEFINITION */
+#define ROCEE_VF_DB_CFG0_OFFSET 0x0230
+
+struct hns_roce_db {
+ unsigned int byte_4;
+ unsigned int parameter;
+};
+#define DB_BYTE_4_TAG_S 0
+#define DB_BYTE_4_TAG_M (((1UL << 23) - 1) << DB_BYTE_4_TAG_S)
+
+#define DB_BYTE_4_CMD_S 24
+#define DB_BYTE_4_CMD_M (((1UL << 4) - 1) << DB_BYTE_4_CMD_S)
+
+#define DB_PARAM_SQ_PRODUCER_IDX_S 0
+#define DB_PARAM_SQ_PRODUCER_IDX_M \
+ (((1UL << 16) - 1) << DB_PARAM_SQ_PRODUCER_IDX_S)
+
+#define DB_PARAM_RQ_PRODUCER_IDX_S 0
+#define DB_PARAM_RQ_PRODUCER_IDX_M \
+ (((1UL << 16) - 1) << DB_PARAM_RQ_PRODUCER_IDX_S)
+
+#define DB_PARAM_SRQ_PRODUCER_COUNTER_S 0
+#define DB_PARAM_SRQ_PRODUCER_COUNTER_M \
+ (((1UL << 16) - 1) << DB_PARAM_SRQ_PRODUCER_COUNTER_S)
+
+#define DB_PARAM_SL_S 16
+#define DB_PARAM_SL_M \
+ (((1UL << 3) - 1) << DB_PARAM_SL_S)
+
+struct hns_roce_v2_cq_db {
+ unsigned int byte_4;
+ unsigned int parameter;
+};
+
+#define CQ_DB_BYTE_4_TAG_S 0
+#define CQ_DB_BYTE_4_TAG_M (((1UL << 23) - 1) << CQ_DB_BYTE_4_TAG_S)
+
+#define CQ_DB_BYTE_4_CMD_S 24
+#define CQ_DB_BYTE_4_CMD_M (((1UL << 4) - 1) << CQ_DB_BYTE_4_CMD_S)
+
+#define CQ_DB_PARAMETER_CQ_CONSUMER_IDX_S 0
+#define CQ_DB_PARAMETER_CQ_CONSUMER_IDX_M \
+ (((1UL << 24) - 1) << CQ_DB_PARAMETER_CQ_CONSUMER_IDX_S)
+
+#define CQ_DB_PARAMETER_NOTIFY_S 24
+
+#define CQ_DB_PARAMETER_CMD_SN_S 25
+#define CQ_DB_PARAMETER_CMD_SN_M \
+ (((1UL << 2) - 1) << CQ_DB_PARAMETER_CMD_SN_S)
+
+struct hns_roce_v2_cqe {
+ unsigned int byte_4;
+ unsigned int rkey_immtdata;
+ unsigned int byte_12;
+ unsigned int byte_16;
+ unsigned int byte_cnt;
+ unsigned int smac;
+ unsigned int byte_28;
+ unsigned int byte_32;
+};
+
+#define CQE_BYTE_4_OPCODE_S 0
+#define CQE_BYTE_4_OPCODE_M (((1UL << 5) - 1) << CQE_BYTE_4_OPCODE_S)
+
+#define CQE_BYTE_4_RQ_INLINE_S 5
+
+#define CQE_BYTE_4_S_R_S 6
+#define CQE_BYTE_4_OWNER_S 7
+
+#define CQE_BYTE_4_STATUS_S 8
+#define CQE_BYTE_4_STATUS_M (((1UL << 8) - 1) << CQE_BYTE_4_STATUS_S)
+
+#define CQE_BYTE_4_WQE_IDX_S 16
+#define CQE_BYTE_4_WQE_IDX_M (((1UL << 16) - 1) << CQE_BYTE_4_WQE_IDX_S)
+
+#define CQE_BYTE_12_XRC_SRQN_S 0
+#define CQE_BYTE_12_XRC_SRQN_M (((1UL << 24) - 1) << CQE_BYTE_12_XRC_SRQN_S)
+
+#define CQE_BYTE_16_LCL_QPN_S 0
+#define CQE_BYTE_16_LCL_QPN_M (((1UL << 24) - 1) << CQE_BYTE_16_LCL_QPN_S)
+
+#define CQE_BYTE_28_SMAC_S 0
+#define CQE_BYTE_28_SMAC_M (((1UL << 16) - 1) << CQE_BYTE_28_SMAC_S)
+
+#define CQE_BYTE_28_PORT_TYPE_S 16
+#define CQE_BYTE_28_PORT_TYPE_M (((1UL << 2) - 1) << CQE_BYTE_28_PORT_TYPE_S)
+
+#define CQE_BYTE_32_RMT_QPN_S 0
+#define CQE_BYTE_32_RMT_QPN_M (((1UL << 24) - 1) << CQE_BYTE_32_RMT_QPN_S)
+
+#define CQE_BYTE_32_SL_S 24
+#define CQE_BYTE_32_SL_M (((1UL << 3) - 1) << CQE_BYTE_32_SL_S)
+
+#define CQE_BYTE_32_PORTN_S 27
+#define CQE_BYTE_32_PORTN_M (((1UL << 3) - 1) << CQE_BYTE_32_PORTN_S)
+
+#define CQE_BYTE_32_GLH_S 30
+
+#define CQE_BYTE_32_LPK_S 31
+
+#endif /* _HNS_ROCE_U_HW_V2_H */
diff --git a/providers/hns/hns_roce_u_verbs.c b/providers/hns/hns_roce_u_verbs.c
index 0b8f444..8f6c666 100644
--- a/providers/hns/hns_roce_u_verbs.c
+++ b/providers/hns/hns_roce_u_verbs.c
@@ -1,5 +1,5 @@
/*
- * Copyright (c) 2016 Hisilicon Limited.
+ * Copyright (c) 2016-2017 Hisilicon Limited.
*
* This software is available to you under a choice of one of two
* licenses. You may choose to be licensed under the terms of the GNU
@@ -42,6 +42,7 @@
#include "hns_roce_u.h"
#include "hns_roce_u_abi.h"
#include "hns_roce_u_hw_v1.h"
+#include "hns_roce_u_hw_v2.h"
void hns_roce_init_qp_indices(struct hns_roce_qp *qp)
{
@@ -272,7 +273,7 @@ struct ibv_cq *hns_roce_u_create_cq(struct ibv_context *context, int cqe,
cq->set_ci_db = to_hr_ctx(context)->cq_tptr_base + cq->cqn * 2;
else
cq->set_ci_db = to_hr_ctx(context)->uar +
- ROCEE_DB_OTHERS_L_0_REG;
+ ROCEE_VF_DB_CFG0_OFFSET;
cq->arm_db = cq->set_ci_db;
cq->arm_sn = 1;
@@ -312,18 +313,22 @@ int hns_roce_u_destroy_cq(struct ibv_cq *cq)
static int hns_roce_verify_qp(struct ibv_qp_init_attr *attr,
struct hns_roce_context *context)
{
- if (attr->cap.max_send_wr < HNS_ROCE_MIN_WQE_NUM) {
- fprintf(stderr,
- "max_send_wr = %d, less than minimum WQE number.\n",
- attr->cap.max_send_wr);
- attr->cap.max_send_wr = HNS_ROCE_MIN_WQE_NUM;
- }
+ struct hns_roce_device *hr_dev = to_hr_dev(context->ibv_ctx.device);
+
+ if (hr_dev->hw_version == HNS_ROCE_HW_VER1) {
+ if (attr->cap.max_send_wr < HNS_ROCE_MIN_WQE_NUM) {
+ fprintf(stderr,
+ "max_send_wr = %d, less than minimum WQE number.\n",
+ attr->cap.max_send_wr);
+ attr->cap.max_send_wr = HNS_ROCE_MIN_WQE_NUM;
+ }
- if (attr->cap.max_recv_wr < HNS_ROCE_MIN_WQE_NUM) {
- fprintf(stderr,
- "max_recv_wr = %d, less than minimum WQE number.\n",
- attr->cap.max_recv_wr);
- attr->cap.max_recv_wr = HNS_ROCE_MIN_WQE_NUM;
+ if (attr->cap.max_recv_wr < HNS_ROCE_MIN_WQE_NUM) {
+ fprintf(stderr,
+ "max_recv_wr = %d, less than minimum WQE number.\n",
+ attr->cap.max_recv_wr);
+ attr->cap.max_recv_wr = HNS_ROCE_MIN_WQE_NUM;
+ }
}
if (attr->cap.max_recv_sge < 1)
--
1.9.1
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related [flat|nested] 8+ messages in thread* [PATCH rdma-core 3/5] libhns: Introduce CQ operations refered to hip08 device
[not found] ` <1508245946-145697-1-git-send-email-oulijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
2017-10-17 13:12 ` [PATCH rdma-core 1/5] libhns: Add new PCI device matching for hip08 Lijun Ou
2017-10-17 13:12 ` [PATCH rdma-core 2/5] libhns: Introduce QP operations refered to hip08 RoCE device Lijun Ou
@ 2017-10-17 13:12 ` Lijun Ou
2017-10-17 13:12 ` [PATCH rdma-core 4/5] libhns: Reimplement verbs of post_send and post_recv for hip08 RoCE Lijun Ou
` (2 subsequent siblings)
5 siblings, 0 replies; 8+ messages in thread
From: Lijun Ou @ 2017-10-17 13:12 UTC (permalink / raw)
To: dledford-H+wXaHxf7aLQT0dZR+AlfA, leon-DgEjT+Ai2ygdnm+yROfE0A,
jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA
CQ APIs need to operate doorbell and cqe. the design of doorbell
and cqe in The different hardware is discrepant. Hence, This patch
introduces the CQ operations of hip08 hardware.
Signed-off-by: Lijun Ou <oulijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
Signed-off-by: Wei Hu <xavier.huwei-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
---
providers/hns/hns_roce_u_hw_v2.c | 305 +++++++++++++++++++++++++++++++++++++++
providers/hns/hns_roce_u_hw_v2.h | 75 ++++++++++
providers/hns/hns_roce_u_verbs.c | 14 +-
3 files changed, 389 insertions(+), 5 deletions(-)
diff --git a/providers/hns/hns_roce_u_hw_v2.c b/providers/hns/hns_roce_u_hw_v2.c
index bf1c3f3..33f7973 100644
--- a/providers/hns/hns_roce_u_hw_v2.c
+++ b/providers/hns/hns_roce_u_hw_v2.c
@@ -37,6 +37,58 @@
#include "hns_roce_u_db.h"
#include "hns_roce_u_hw_v2.h"
+static void hns_roce_v2_handle_error_cqe(struct hns_roce_v2_cqe *cqe,
+ struct ibv_wc *wc)
+{
+ unsigned int status = roce_get_field(cqe->byte_4, CQE_BYTE_4_STATUS_M,
+ CQE_BYTE_4_STATUS_S);
+ switch (status & HNS_ROCE_V2_CQE_STATUS_MASK) {
+ fprintf(stderr, PFX "error cqe!\n");
+ case HNS_ROCE_V2_CQE_LOCAL_LENGTH_ERR:
+ wc->status = IBV_WC_LOC_LEN_ERR;
+ break;
+ case HNS_ROCE_V2_CQE_LOCAL_QP_OP_ERR:
+ wc->status = IBV_WC_LOC_QP_OP_ERR;
+ break;
+ case HNS_ROCE_V2_CQE_LOCAL_PROT_ERR:
+ wc->status = IBV_WC_LOC_PROT_ERR;
+ break;
+ case HNS_ROCE_V2_CQE_WR_FLUSH_ERR:
+ wc->status = IBV_WC_WR_FLUSH_ERR;
+ break;
+ case HNS_ROCE_V2_CQE_MEM_MANAGERENT_OP_ERR:
+ wc->status = IBV_WC_MW_BIND_ERR;
+ break;
+ case HNS_ROCE_V2_CQE_BAD_RESP_ERR:
+ wc->status = IBV_WC_BAD_RESP_ERR;
+ break;
+ case HNS_ROCE_V2_CQE_LOCAL_ACCESS_ERR:
+ wc->status = IBV_WC_LOC_ACCESS_ERR;
+ break;
+ case HNS_ROCE_V2_CQE_REMOTE_INVAL_REQ_ERR:
+ wc->status = IBV_WC_REM_INV_REQ_ERR;
+ break;
+ case HNS_ROCE_V2_CQE_REMOTE_ACCESS_ERR:
+ wc->status = IBV_WC_REM_ACCESS_ERR;
+ break;
+ case HNS_ROCE_V2_CQE_REMOTE_OP_ERR:
+ wc->status = IBV_WC_REM_OP_ERR;
+ break;
+ case HNS_ROCE_V2_CQE_TRANSPORT_RETRY_EXC_ERR:
+ wc->status = IBV_WC_RETRY_EXC_ERR;
+ break;
+ case HNS_ROCE_V2_CQE_RNR_RETRY_EXC_ERR:
+ wc->status = IBV_WC_RNR_RETRY_EXC_ERR;
+ break;
+ case HNS_ROCE_V2_CQE_REMOTE_ABORTED_ERR:
+ wc->status = IBV_WC_REM_ABORT_ERR;
+ break;
+ default:
+ wc->status = IBV_WC_GENERAL_ERR;
+ break;
+ }
+}
+
static struct hns_roce_v2_cqe *get_cqe_v2(struct hns_roce_cq *cq, int entry)
{
return cq->buf.buf + entry * HNS_ROCE_CQE_ENTRY_SIZE;
@@ -50,6 +102,11 @@ static void *get_sw_cqe_v2(struct hns_roce_cq *cq, int n)
!!(n & (cq->ibv_cq.cqe + 1))) ? cqe : NULL;
}
+static struct hns_roce_v2_cqe *next_cqe_sw(struct hns_roce_cq *cq)
+{
+ return get_sw_cqe_v2(cq, cq->cons_index);
+}
+
static void hns_roce_v2_update_cq_cons_index(struct hns_roce_context *ctx,
struct hns_roce_cq *cq)
{
@@ -71,6 +128,19 @@ static void hns_roce_v2_update_cq_cons_index(struct hns_roce_context *ctx,
hns_roce_write64((uint32_t *)&cq_db, ctx, ROCEE_VF_DB_CFG0_OFFSET);
}
+static struct hns_roce_qp *hns_roce_v2_find_qp(struct hns_roce_context *ctx,
+ uint32_t qpn)
+{
+ int tind = (qpn & (ctx->num_qps - 1)) >> ctx->qp_table_shift;
+
+ if (ctx->qp_table[tind].refcnt) {
+ return ctx->qp_table[tind].table[qpn & ctx->qp_table_mask];
+ } else {
+ printf("hns_roce_v2_find_qp fail!\n");
+ return NULL;
+ }
+}
+
static void hns_roce_v2_clear_qp(struct hns_roce_context *ctx, uint32_t qpn)
{
int tind = (qpn & (ctx->num_qps - 1)) >> ctx->qp_table_shift;
@@ -81,6 +151,239 @@ static void hns_roce_v2_clear_qp(struct hns_roce_context *ctx, uint32_t qpn)
ctx->qp_table[tind].table[qpn & ctx->qp_table_mask] = NULL;
}
+static int hns_roce_v2_poll_one(struct hns_roce_cq *cq,
+ struct hns_roce_qp **cur_qp, struct ibv_wc *wc)
+{
+ uint32_t qpn;
+ int is_send;
+ uint16_t wqe_ctr;
+ uint32_t local_qpn;
+ struct hns_roce_wq *wq = NULL;
+ struct hns_roce_v2_cqe *cqe = NULL;
+
+ /* According to CI, find the relative cqe */
+ cqe = next_cqe_sw(cq);
+ if (!cqe)
+ return V2_CQ_EMPTY;
+
+ /* Get the next cqe, CI will be added gradually */
+ ++cq->cons_index;
+
+ udma_from_device_barrier();
+
+ qpn = roce_get_field(cqe->byte_16, CQE_BYTE_16_LCL_QPN_M,
+ CQE_BYTE_16_LCL_QPN_S);
+
+ is_send = (roce_get_bit(cqe->byte_4, CQE_BYTE_4_S_R_S) ==
+ HNS_ROCE_V2_CQE_IS_SQ);
+
+ local_qpn = roce_get_field(cqe->byte_16, CQE_BYTE_16_LCL_QPN_M,
+ CQE_BYTE_16_LCL_QPN_S);
+
+ /* if qp is zero, it will not get the correct qpn */
+ if (!*cur_qp ||
+ (local_qpn & HNS_ROCE_V2_CQE_QPN_MASK) != (*cur_qp)->ibv_qp.qp_num) {
+
+ *cur_qp = hns_roce_v2_find_qp(to_hr_ctx(cq->ibv_cq.context),
+ qpn & 0xffffff);
+ if (!*cur_qp) {
+ fprintf(stderr, PFX "can't find qp!\n");
+ return V2_CQ_POLL_ERR;
+ }
+ }
+ wc->qp_num = qpn & 0xffffff;
+
+ if (is_send) {
+ wq = &(*cur_qp)->sq;
+ /*
+ * if sq_signal_bits is 1, the tail pointer first update to
+ * the wqe corresponding the current cqe
+ */
+ if ((*cur_qp)->sq_signal_bits) {
+ wqe_ctr = (uint16_t)(roce_get_field(cqe->byte_4,
+ CQE_BYTE_4_WQE_IDX_M,
+ CQE_BYTE_4_WQE_IDX_S));
+ /*
+ * wq->tail will plus a positive number every time,
+ * when wq->tail exceeds 32b, it is 0 and acc
+ */
+ wq->tail += (wqe_ctr - (uint16_t) wq->tail) &
+ (wq->wqe_cnt - 1);
+ }
+ /* write the wr_id of wq into the wc */
+ wc->wr_id = wq->wrid[wq->tail & (wq->wqe_cnt - 1)];
+ ++wq->tail;
+ } else {
+ wq = &(*cur_qp)->rq;
+ wc->wr_id = wq->wrid[wq->tail & (wq->wqe_cnt - 1)];
+ ++wq->tail;
+ }
+
+ /*
+ * HW maintains wc status, set the err type and directly return, after
+ * generated the incorrect CQE
+ */
+ if (roce_get_field(cqe->byte_4, CQE_BYTE_4_STATUS_M,
+ CQE_BYTE_4_STATUS_S) != HNS_ROCE_V2_CQE_SUCCESS) {
+ hns_roce_v2_handle_error_cqe(cqe, wc);
+ return V2_CQ_OK;
+ }
+
+ wc->status = IBV_WC_SUCCESS;
+
+ /*
+ * According to the opcode type of cqe, mark the opcode and other
+ * information of wc
+ */
+ if (is_send) {
+ /* Get opcode and flag before update the tail point for send */
+ switch (roce_get_field(cqe->byte_4, CQE_BYTE_4_OPCODE_M,
+ CQE_BYTE_4_OPCODE_S) & HNS_ROCE_V2_CQE_OPCODE_MASK) {
+ case HNS_ROCE_SQ_OP_SEND:
+ wc->opcode = IBV_WC_SEND;
+ wc->wc_flags = 0;
+ break;
+
+ case HNS_ROCE_SQ_OP_SEND_WITH_IMM:
+ wc->opcode = IBV_WC_SEND;
+ wc->wc_flags = IBV_WC_WITH_IMM;
+ break;
+
+ case HNS_ROCE_SQ_OP_SEND_WITH_INV:
+ wc->opcode = IBV_WC_SEND;
+ break;
+
+ case HNS_ROCE_SQ_OP_RDMA_READ:
+ wc->opcode = IBV_WC_RDMA_READ;
+ wc->byte_len = cqe->byte_cnt;
+ wc->wc_flags = 0;
+ break;
+
+ case HNS_ROCE_SQ_OP_RDMA_WRITE:
+ wc->opcode = IBV_WC_RDMA_WRITE;
+ wc->wc_flags = 0;
+ break;
+
+ case HNS_ROCE_SQ_OP_RDMA_WRITE_WITH_IMM:
+ wc->opcode = IBV_WC_RDMA_WRITE;
+ wc->wc_flags = IBV_WC_WITH_IMM;
+ break;
+ case HNS_ROCE_SQ_OP_LOCAL_INV:
+ wc->opcode = IBV_WC_LOCAL_INV;
+ wc->wc_flags = IBV_WC_WITH_INV;
+ break;
+ case HNS_ROCE_SQ_OP_ATOMIC_COMP_AND_SWAP:
+ wc->opcode = IBV_WC_COMP_SWAP;
+ wc->byte_len = 8;
+ wc->wc_flags = 0;
+ break;
+ case HNS_ROCE_SQ_OP_ATOMIC_FETCH_AND_ADD:
+ wc->opcode = IBV_WC_FETCH_ADD;
+ wc->byte_len = 8;
+ wc->wc_flags = 0;
+ break;
+ case HNS_ROCE_SQ_OP_BIND_MW:
+ wc->opcode = IBV_WC_BIND_MW;
+ wc->wc_flags = 0;
+ break;
+ default:
+ wc->status = IBV_WC_GENERAL_ERR;
+ wc->wc_flags = 0;
+ break;
+ }
+ } else {
+ /* Get opcode and flag in rq&srq */
+ wc->byte_len = cqe->byte_cnt;
+ switch (roce_get_field(cqe->byte_4, CQE_BYTE_4_OPCODE_M,
+ CQE_BYTE_4_OPCODE_S) & HNS_ROCE_V2_CQE_OPCODE_MASK) {
+ case HNS_ROCE_RECV_OP_RDMA_WRITE_IMM:
+ wc->opcode = IBV_WC_RECV_RDMA_WITH_IMM;
+ wc->wc_flags = IBV_WC_WITH_IMM;
+ wc->imm_data = cqe->rkey_immtdata;
+ break;
+
+ case HNS_ROCE_RECV_OP_SEND:
+ wc->opcode = IBV_WC_RECV;
+ wc->wc_flags = 0;
+ break;
+
+ case HNS_ROCE_RECV_OP_SEND_WITH_IMM:
+ wc->opcode = IBV_WC_RECV;
+ wc->wc_flags = IBV_WC_WITH_IMM;
+ wc->imm_data = cqe->rkey_immtdata;
+ break;
+
+ case HNS_ROCE_RECV_OP_SEND_WITH_INV:
+ wc->opcode = IBV_WC_RECV;
+ wc->wc_flags = IBV_WC_WITH_INV;
+ wc->imm_data = cqe->rkey_immtdata;
+ break;
+ default:
+ wc->status = IBV_WC_GENERAL_ERR;
+ break;
+ }
+ }
+
+ return V2_CQ_OK;
+}
+
+static int hns_roce_u_v2_poll_cq(struct ibv_cq *ibvcq, int ne,
+ struct ibv_wc *wc)
+{
+ int npolled;
+ int err = V2_CQ_OK;
+ struct hns_roce_qp *qp = NULL;
+ struct hns_roce_cq *cq = to_hr_cq(ibvcq);
+ struct hns_roce_context *ctx = to_hr_ctx(ibvcq->context);
+
+ pthread_spin_lock(&cq->lock);
+
+ for (npolled = 0; npolled < ne; ++npolled) {
+ err = hns_roce_v2_poll_one(cq, &qp, wc + npolled);
+ if (err != V2_CQ_OK)
+ break;
+ }
+
+ if (npolled) {
+ mmio_ordered_writes_hack();
+
+ hns_roce_v2_update_cq_cons_index(ctx, cq);
+ }
+
+ pthread_spin_unlock(&cq->lock);
+
+ return err == V2_CQ_POLL_ERR ? err : npolled;
+}
+
+static int hns_roce_u_v2_arm_cq(struct ibv_cq *ibvcq, int solicited)
+{
+ uint32_t ci;
+ uint32_t solicited_flag;
+ struct hns_roce_v2_cq_db cq_db;
+ struct hns_roce_cq *cq = to_hr_cq(ibvcq);
+
+ ci = cq->cons_index & ((cq->cq_depth << 1) - 1);
+ solicited_flag = solicited ? HNS_ROCE_V2_CQ_DB_REQ_SOL :
+ HNS_ROCE_V2_CQ_DB_REQ_NEXT;
+
+ cq_db.byte_4 = 0;
+ cq_db.parameter = 0;
+
+ roce_set_field(cq_db.byte_4, DB_BYTE_4_TAG_M, DB_BYTE_4_TAG_S, cq->cqn);
+ roce_set_field(cq_db.byte_4, DB_BYTE_4_CMD_M, DB_BYTE_4_CMD_S, 0x4);
+
+ roce_set_field(cq_db.parameter, CQ_DB_PARAMETER_CQ_CONSUMER_IDX_M,
+ CQ_DB_PARAMETER_CQ_CONSUMER_IDX_S, ci);
+
+ roce_set_field(cq_db.parameter, CQ_DB_PARAMETER_CMD_SN_M,
+ CQ_DB_PARAMETER_CMD_SN_S, 1);
+ roce_set_bit(cq_db.parameter, CQ_DB_PARAMETER_NOTIFY_S, solicited_flag);
+
+ hns_roce_write64((uint32_t *)&cq_db, to_hr_ctx(ibvcq->context),
+ ROCEE_VF_DB_CFG0_OFFSET);
+ return 0;
+}
+
static void __hns_roce_v2_cq_clean(struct hns_roce_cq *cq, uint32_t qpn,
struct hns_roce_srq *srq)
{
@@ -226,6 +529,8 @@ static int hns_roce_u_v2_destroy_qp(struct ibv_qp *ibqp)
struct hns_roce_u_hw hns_roce_u_hw_v2 = {
.hw_version = HNS_ROCE_HW_VER2,
+ .poll_cq = hns_roce_u_v2_poll_cq,
+ .arm_cq = hns_roce_u_v2_arm_cq,
.modify_qp = hns_roce_u_v2_modify_qp,
.destroy_qp = hns_roce_u_v2_destroy_qp,
};
diff --git a/providers/hns/hns_roce_u_hw_v2.h b/providers/hns/hns_roce_u_hw_v2.h
index d7fcf94..238bebf 100644
--- a/providers/hns/hns_roce_u_hw_v2.h
+++ b/providers/hns/hns_roce_u_hw_v2.h
@@ -33,9 +33,84 @@
#ifndef _HNS_ROCE_U_HW_V2_H
#define _HNS_ROCE_U_HW_V2_H
+#define HNS_ROCE_V2_CQE_IS_SQ 0
+
+#define HNS_ROCE_V2_CQ_DB_REQ_SOL 1
+#define HNS_ROCE_V2_CQ_DB_REQ_NEXT 0
+
/* V2 REG DEFINITION */
#define ROCEE_VF_DB_CFG0_OFFSET 0x0230
+enum {
+ HNS_ROCE_WQE_OP_SEND = 0x0,
+ HNS_ROCE_WQE_OP_SEND_WITH_INV = 0x1,
+ HNS_ROCE_WQE_OP_SEND_WITH_IMM = 0x2,
+ HNS_ROCE_WQE_OP_RDMA_WRITE = 0x3,
+ HNS_ROCE_WQE_OP_RDMA_WRITE_WITH_IMM = 0x4,
+ HNS_ROCE_WQE_OP_RDMA_READ = 0x5,
+ HNS_ROCE_WQE_OP_ATOMIC_COM_AND_SWAP = 0x6,
+ HNS_ROCE_WQE_OP_ATOMIC_FETCH_AND_ADD = 0x7,
+ HNS_ROCE_WQE_OP_ATOMIC_MASK_COMP_AND_SWAP = 0x8,
+ HNS_ROCE_WQE_OP_ATOMIC_MASK_FETCH_AND_ADD = 0x9,
+ HNS_ROCE_WQE_OP_FAST_REG_PMR = 0xa,
+ HNS_ROCE_WQE_OP_LOCAL_INV = 0xb,
+ HNS_ROCE_WQE_OP_BIND_MW_TYPE = 0xc,
+ HNS_ROCE_WQE_OP_MASK = 0x1f
+};
+
+enum {
+ /* rq operations */
+ HNS_ROCE_RECV_OP_RDMA_WRITE_IMM = 0x0,
+ HNS_ROCE_RECV_OP_SEND = 0x1,
+ HNS_ROCE_RECV_OP_SEND_WITH_IMM = 0x2,
+ HNS_ROCE_RECV_OP_SEND_WITH_INV = 0x3,
+};
+
+enum {
+ HNS_ROCE_SQ_OP_SEND = 0x0,
+ HNS_ROCE_SQ_OP_SEND_WITH_INV = 0x1,
+ HNS_ROCE_SQ_OP_SEND_WITH_IMM = 0x2,
+ HNS_ROCE_SQ_OP_RDMA_WRITE = 0x3,
+ HNS_ROCE_SQ_OP_RDMA_WRITE_WITH_IMM = 0x4,
+ HNS_ROCE_SQ_OP_RDMA_READ = 0x5,
+ HNS_ROCE_SQ_OP_ATOMIC_COMP_AND_SWAP = 0x6,
+ HNS_ROCE_SQ_OP_ATOMIC_FETCH_AND_ADD = 0x7,
+ HNS_ROCE_SQ_OP_ATOMIC_MASK_COMP_AND_SWAP = 0x8,
+ HNS_ROCE_SQ_OP_ATOMIC_MASK_FETCH_AND_ADD = 0x9,
+ HNS_ROCE_SQ_OP_FAST_REG_PMR = 0xa,
+ HNS_ROCE_SQ_OP_LOCAL_INV = 0xb,
+ HNS_ROCE_SQ_OP_BIND_MW = 0xc,
+};
+
+enum {
+ V2_CQ_OK = 0,
+ V2_CQ_EMPTY = -1,
+ V2_CQ_POLL_ERR = -2,
+};
+
+enum {
+ HNS_ROCE_V2_CQE_QPN_MASK = 0x3ffff,
+ HNS_ROCE_V2_CQE_STATUS_MASK = 0xff,
+ HNS_ROCE_V2_CQE_OPCODE_MASK = 0x1f,
+};
+
+enum {
+ HNS_ROCE_V2_CQE_SUCCESS = 0x00,
+ HNS_ROCE_V2_CQE_LOCAL_LENGTH_ERR = 0x01,
+ HNS_ROCE_V2_CQE_LOCAL_QP_OP_ERR = 0x02,
+ HNS_ROCE_V2_CQE_LOCAL_PROT_ERR = 0x04,
+ HNS_ROCE_V2_CQE_WR_FLUSH_ERR = 0x05,
+ HNS_ROCE_V2_CQE_MEM_MANAGERENT_OP_ERR = 0x06,
+ HNS_ROCE_V2_CQE_BAD_RESP_ERR = 0x10,
+ HNS_ROCE_V2_CQE_LOCAL_ACCESS_ERR = 0x11,
+ HNS_ROCE_V2_CQE_REMOTE_INVAL_REQ_ERR = 0x12,
+ HNS_ROCE_V2_CQE_REMOTE_ACCESS_ERR = 0x13,
+ HNS_ROCE_V2_CQE_REMOTE_OP_ERR = 0x14,
+ HNS_ROCE_V2_CQE_TRANSPORT_RETRY_EXC_ERR = 0x15,
+ HNS_ROCE_V2_CQE_RNR_RETRY_EXC_ERR = 0x16,
+ HNS_ROCE_V2_CQE_REMOTE_ABORTED_ERR = 0x22,
+};
+
struct hns_roce_db {
unsigned int byte_4;
unsigned int parameter;
diff --git a/providers/hns/hns_roce_u_verbs.c b/providers/hns/hns_roce_u_verbs.c
index 8f6c666..64a4ac3 100644
--- a/providers/hns/hns_roce_u_verbs.c
+++ b/providers/hns/hns_roce_u_verbs.c
@@ -197,11 +197,15 @@ static void hns_roce_set_sq_sizes(struct hns_roce_qp *qp,
static int hns_roce_verify_cq(int *cqe, struct hns_roce_context *context)
{
- if (*cqe < HNS_ROCE_MIN_CQE_NUM) {
- fprintf(stderr, "cqe = %d, less than minimum CQE number.\n",
- *cqe);
- *cqe = HNS_ROCE_MIN_CQE_NUM;
- }
+ struct hns_roce_device *hr_dev = to_hr_dev(context->ibv_ctx.device);
+
+ if (hr_dev->hw_version == HNS_ROCE_HW_VER1)
+ if (*cqe < HNS_ROCE_MIN_CQE_NUM) {
+ fprintf(stderr,
+ "cqe = %d, less than minimum CQE number.\n",
+ *cqe);
+ *cqe = HNS_ROCE_MIN_CQE_NUM;
+ }
if (*cqe > context->max_cqe)
return -1;
--
1.9.1
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related [flat|nested] 8+ messages in thread* [PATCH rdma-core 4/5] libhns: Reimplement verbs of post_send and post_recv for hip08 RoCE
[not found] ` <1508245946-145697-1-git-send-email-oulijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
` (2 preceding siblings ...)
2017-10-17 13:12 ` [PATCH rdma-core 3/5] libhns: Introduce CQ operations refered to hip08 device Lijun Ou
@ 2017-10-17 13:12 ` Lijun Ou
2017-10-17 13:12 ` [PATCH rdma-core 5/5] libhns: Update consolidate repo for userspace library of " Lijun Ou
2017-10-23 17:27 ` [PATCH rdma-core 0/5] Userspace library matching for " Leon Romanovsky
5 siblings, 0 replies; 8+ messages in thread
From: Lijun Ou @ 2017-10-17 13:12 UTC (permalink / raw)
To: dledford-H+wXaHxf7aLQT0dZR+AlfA, leon-DgEjT+Ai2ygdnm+yROfE0A,
jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA
The verbs of post_send and post recv need to fill sqwqe and rqwqe and
issue the doorbell cmd. The sqwqe&rqwqe&doorbell structures are
different between hip06 and hip08 etc. Hence, It needs to rewrite
the verbs of post_send and post_recv.
Signed-off-by: Lijun Ou <oulijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
Signed-off-by: Wei Hu <xavier.huwei-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
---
providers/hns/hns_roce_u.h | 8 +
providers/hns/hns_roce_u_hw_v2.c | 384 ++++++++++++++++++++++++++++++++++++++-
providers/hns/hns_roce_u_hw_v2.h | 50 +++++
providers/hns/hns_roce_u_verbs.c | 76 ++++++--
4 files changed, 501 insertions(+), 17 deletions(-)
diff --git a/providers/hns/hns_roce_u.h b/providers/hns/hns_roce_u.h
index 568bfc3..ea645be 100644
--- a/providers/hns/hns_roce_u.h
+++ b/providers/hns/hns_roce_u.h
@@ -159,6 +159,12 @@ struct hns_roce_wq {
int offset;
};
+struct hns_roce_sge_ex {
+ int offset;
+ unsigned int sge_cnt;
+ int sge_shift;
+};
+
struct hns_roce_qp {
struct ibv_qp ibv_qp;
struct hns_roce_buf buf;
@@ -167,6 +173,8 @@ struct hns_roce_qp {
unsigned int sq_signal_bits;
struct hns_roce_wq sq;
struct hns_roce_wq rq;
+ struct hns_roce_sge_ex sge;
+ unsigned int next_sge;
int port_num;
int sl;
};
diff --git a/providers/hns/hns_roce_u_hw_v2.c b/providers/hns/hns_roce_u_hw_v2.c
index 33f7973..b124813 100644
--- a/providers/hns/hns_roce_u_hw_v2.c
+++ b/providers/hns/hns_roce_u_hw_v2.c
@@ -37,6 +37,22 @@
#include "hns_roce_u_db.h"
#include "hns_roce_u_hw_v2.h"
+static inline void set_raddr_seg_v2(struct hns_roce_v2_wqe_raddr_seg *rseg,
+ uint64_t remote_addr, uint32_t rkey)
+{
+ rseg->raddr = remote_addr;
+ rseg->rkey = rkey;
+ rseg->len = 0;
+}
+
+static void set_data_seg_v2(struct hns_roce_v2_wqe_data_seg *dseg,
+ struct ibv_sge *sg)
+{
+ dseg->lkey = sg->lkey;
+ dseg->addr = sg->addr;
+ dseg->len = sg->length;
+}
+
static void hns_roce_v2_handle_error_cqe(struct hns_roce_v2_cqe *cqe,
struct ibv_wc *wc)
{
@@ -102,11 +118,92 @@ static void *get_sw_cqe_v2(struct hns_roce_cq *cq, int n)
!!(n & (cq->ibv_cq.cqe + 1))) ? cqe : NULL;
}
-static struct hns_roce_v2_cqe *next_cqe_sw(struct hns_roce_cq *cq)
+static struct hns_roce_v2_cqe *next_cqe_sw_v2(struct hns_roce_cq *cq)
{
return get_sw_cqe_v2(cq, cq->cons_index);
}
+static void *get_recv_wqe_v2(struct hns_roce_qp *qp, int n)
+{
+ if ((n < 0) || (n > qp->rq.wqe_cnt)) {
+ printf("rq wqe index:%d,rq wqe cnt:%d\r\n", n, qp->rq.wqe_cnt);
+ return NULL;
+ }
+
+ return qp->buf.buf + qp->rq.offset + (n << qp->rq.wqe_shift);
+}
+
+static void *get_send_wqe(struct hns_roce_qp *qp, int n)
+{
+ return (void *)((uint64_t)(qp->buf.buf) + qp->sq.offset +
+ (n << qp->sq.wqe_shift));
+}
+
+static void *get_send_sge_ex(struct hns_roce_qp *qp, int n)
+{
+ return (void *)((uint64_t)(qp->buf.buf) + qp->sge.offset +
+ (n << qp->sge.sge_shift));
+}
+
+static int hns_roce_v2_wq_overflow(struct hns_roce_wq *wq, int nreq,
+ struct hns_roce_cq *cq)
+{
+ unsigned int cur;
+
+ cur = wq->head - wq->tail;
+ if (cur + nreq < wq->max_post)
+ return 0;
+
+ /* While the num of wqe exceeds cap of the device, cq will be locked */
+ pthread_spin_lock(&cq->lock);
+ cur = wq->head - wq->tail;
+ pthread_spin_unlock(&cq->lock);
+
+ return cur + nreq >= wq->max_post;
+}
+
+static void hns_roce_update_rq_db(struct hns_roce_context *ctx,
+ unsigned int qpn, unsigned int rq_head)
+{
+ struct hns_roce_db rq_db;
+
+ rq_db.byte_4 = 0;
+ rq_db.parameter = 0;
+
+ roce_set_field(rq_db.byte_4, DB_BYTE_4_TAG_M, DB_BYTE_4_TAG_S, qpn);
+ roce_set_field(rq_db.byte_4, DB_BYTE_4_CMD_M, DB_BYTE_4_CMD_S, 0x1);
+ roce_set_field(rq_db.parameter, DB_PARAM_RQ_PRODUCER_IDX_M,
+ DB_PARAM_RQ_PRODUCER_IDX_S, rq_head);
+
+ udma_to_device_barrier();
+
+ hns_roce_write64((uint32_t *)&rq_db, ctx, ROCEE_VF_DB_CFG0_OFFSET);
+}
+
+static void hns_roce_update_sq_db(struct hns_roce_context *ctx,
+ unsigned int qpn, unsigned int sl,
+ unsigned int sq_head)
+{
+ struct hns_roce_db sq_db;
+
+ sq_db.byte_4 = 0;
+
+ /* In fact, the sq_head bits should be 15bit */
+ sq_db.parameter = 0;
+
+ /* cmd: 0 sq db; 1 rq db; 2; 2 srq db; 3 cq db ptr; 4 cq db ntr */
+ roce_set_field(sq_db.byte_4, DB_BYTE_4_CMD_M, DB_BYTE_4_CMD_S, 0);
+ roce_set_field(sq_db.byte_4, DB_BYTE_4_TAG_M, DB_BYTE_4_TAG_S, qpn);
+
+ roce_set_field(sq_db.parameter, DB_PARAM_SQ_PRODUCER_IDX_M,
+ DB_PARAM_SQ_PRODUCER_IDX_S, sq_head);
+ roce_set_field(sq_db.parameter, DB_PARAM_SL_M, DB_PARAM_SL_S, sl);
+
+ udma_to_device_barrier();
+
+ hns_roce_write64((uint32_t *)&sq_db, ctx, ROCEE_VF_DB_CFG0_OFFSET);
+}
+
static void hns_roce_v2_update_cq_cons_index(struct hns_roce_context *ctx,
struct hns_roce_cq *cq)
{
@@ -162,7 +259,7 @@ static int hns_roce_v2_poll_one(struct hns_roce_cq *cq,
struct hns_roce_v2_cqe *cqe = NULL;
/* According to CI, find the relative cqe */
- cqe = next_cqe_sw(cq);
+ cqe = next_cqe_sw_v2(cq);
if (!cqe)
return V2_CQ_EMPTY;
@@ -384,6 +481,287 @@ static int hns_roce_u_v2_arm_cq(struct ibv_cq *ibvcq, int solicited)
return 0;
}
+static int hns_roce_u_v2_post_send(struct ibv_qp *ibvqp, struct ibv_send_wr *wr,
+ struct ibv_send_wr **bad_wr)
+{
+ unsigned int ind_sge;
+ unsigned int ind;
+ int nreq;
+ int i;
+ void *wqe;
+ int ret = 0;
+ struct hns_roce_qp *qp = to_hr_qp(ibvqp);
+ struct hns_roce_context *ctx = to_hr_ctx(ibvqp->context);
+ struct hns_roce_rc_sq_wqe *rc_sq_wqe;
+ struct hns_roce_v2_wqe_data_seg *dseg;
+
+ pthread_spin_lock(&qp->sq.lock);
+
+ /* check that state is OK to post send */
+ ind = qp->sq.head;
+ ind_sge = qp->next_sge;
+
+ if (ibvqp->state != IBV_QPS_RTS && ibvqp->state != IBV_QPS_SQD) {
+ pthread_spin_unlock(&qp->sq.lock);
+ *bad_wr = wr;
+ return EINVAL;
+ }
+
+ for (nreq = 0; wr; ++nreq, wr = wr->next) {
+ if (hns_roce_v2_wq_overflow(&qp->sq, nreq,
+ to_hr_cq(qp->ibv_qp.send_cq))) {
+ ret = -1;
+ *bad_wr = wr;
+ goto out;
+ }
+
+ if (wr->num_sge > qp->sq.max_gs) {
+ ret = -1;
+ *bad_wr = wr;
+ goto out;
+ }
+
+ wqe = get_send_wqe(qp, ind & (qp->sq.wqe_cnt - 1));
+ rc_sq_wqe = wqe;
+
+ memset(rc_sq_wqe, 0, sizeof(struct hns_roce_rc_sq_wqe));
+
+ qp->sq.wrid[ind & (qp->sq.wqe_cnt - 1)] = wr->wr_id;
+ for (i = 0; i < wr->num_sge; i++)
+ rc_sq_wqe->msg_len += wr->sg_list[i].length;
+
+ if (wr->opcode == IBV_WR_SEND_WITH_IMM ||
+ wr->opcode == IBV_WR_RDMA_WRITE_WITH_IMM)
+ rc_sq_wqe->inv_key_immtdata = wr->imm_data;
+
+ roce_set_field(rc_sq_wqe->byte_16, RC_SQ_WQE_BYTE_16_SGE_NUM_M,
+ RC_SQ_WQE_BYTE_16_SGE_NUM_S, wr->num_sge);
+
+ roce_set_field(rc_sq_wqe->byte_20,
+ RC_SQ_WQE_BYTE_20_MSG_START_SGE_IDX_S,
+ RC_SQ_WQE_BYTE_20_MSG_START_SGE_IDX_S,
+ 0);
+
+ roce_set_bit(rc_sq_wqe->byte_4, RC_SQ_WQE_BYTE_4_CQE_S,
+ (wr->send_flags & IBV_SEND_SIGNALED) ? 1 : 0);
+
+ /* Set fence attr */
+ roce_set_bit(rc_sq_wqe->byte_4, RC_SQ_WQE_BYTE_4_FENCE_S,
+ (wr->send_flags & IBV_SEND_FENCE) ? 1 : 0);
+
+ /* Set solicited attr */
+ roce_set_bit(rc_sq_wqe->byte_4, RC_SQ_WQE_BYTE_4_SE_S,
+ (wr->send_flags & IBV_SEND_SOLICITED) ? 1 : 0);
+
+ wqe += sizeof(struct hns_roce_rc_sq_wqe);
+ /* set remote addr segment */
+ switch (ibvqp->qp_type) {
+ case IBV_QPT_RC:
+ switch (wr->opcode) {
+ case IBV_WR_RDMA_READ:
+ roce_set_field(rc_sq_wqe->byte_4,
+ RC_SQ_WQE_BYTE_4_OPCODE_M,
+ RC_SQ_WQE_BYTE_4_OPCODE_S,
+ HNS_ROCE_WQE_OP_RDMA_READ);
+ rc_sq_wqe->va = wr->wr.rdma.remote_addr;
+ rc_sq_wqe->rkey = wr->wr.rdma.rkey;
+ break;
+
+ case IBV_WR_RDMA_WRITE:
+ roce_set_field(rc_sq_wqe->byte_4,
+ RC_SQ_WQE_BYTE_4_OPCODE_M,
+ RC_SQ_WQE_BYTE_4_OPCODE_S,
+ HNS_ROCE_WQE_OP_RDMA_WRITE);
+ rc_sq_wqe->va = wr->wr.rdma.remote_addr;
+ rc_sq_wqe->rkey = wr->wr.rdma.rkey;
+ break;
+
+ case IBV_WR_RDMA_WRITE_WITH_IMM:
+ roce_set_field(rc_sq_wqe->byte_4,
+ RC_SQ_WQE_BYTE_4_OPCODE_M,
+ RC_SQ_WQE_BYTE_4_OPCODE_S,
+ HNS_ROCE_WQE_OP_RDMA_WRITE_WITH_IMM);
+ rc_sq_wqe->va = wr->wr.rdma.remote_addr;
+ rc_sq_wqe->rkey = wr->wr.rdma.rkey;
+ break;
+
+ case IBV_WR_SEND:
+ roce_set_field(rc_sq_wqe->byte_4,
+ RC_SQ_WQE_BYTE_4_OPCODE_M,
+ RC_SQ_WQE_BYTE_4_OPCODE_S,
+ HNS_ROCE_WQE_OP_SEND);
+ break;
+ case IBV_WR_SEND_WITH_INV:
+ roce_set_field(rc_sq_wqe->byte_4,
+ RC_SQ_WQE_BYTE_4_OPCODE_M,
+ RC_SQ_WQE_BYTE_4_OPCODE_S,
+ HNS_ROCE_WQE_OP_SEND_WITH_INV);
+ rc_sq_wqe->inv_key_immtdata = wr->imm_data;
+ break;
+ case IBV_WR_SEND_WITH_IMM:
+ roce_set_field(rc_sq_wqe->byte_4,
+ RC_SQ_WQE_BYTE_4_OPCODE_M,
+ RC_SQ_WQE_BYTE_4_OPCODE_S,
+ HNS_ROCE_WQE_OP_SEND_WITH_IMM);
+ break;
+
+ case IBV_WR_ATOMIC_CMP_AND_SWP:
+ roce_set_field(rc_sq_wqe->byte_4,
+ RC_SQ_WQE_BYTE_4_OPCODE_M,
+ RC_SQ_WQE_BYTE_4_OPCODE_S,
+ HNS_ROCE_WQE_OP_ATOMIC_COM_AND_SWAP);
+ break;
+
+ case IBV_WR_ATOMIC_FETCH_AND_ADD:
+ roce_set_field(rc_sq_wqe->byte_4,
+ RC_SQ_WQE_BYTE_4_OPCODE_M,
+ RC_SQ_WQE_BYTE_4_OPCODE_S,
+ HNS_ROCE_WQE_OP_ATOMIC_FETCH_AND_ADD);
+ break;
+ default:
+ roce_set_field(rc_sq_wqe->byte_4,
+ RC_SQ_WQE_BYTE_4_OPCODE_M,
+ RC_SQ_WQE_BYTE_4_OPCODE_S,
+ HNS_ROCE_WQE_OP_MASK);
+ printf("Not supported transport opcode %d\n",
+ wr->opcode);
+ break;
+ }
+
+ break;
+ case IBV_QPT_UC:
+ case IBV_QPT_UD:
+ default:
+ break;
+ }
+
+ dseg = wqe;
+
+ /* Inline */
+ if (wr->send_flags & IBV_SEND_INLINE && wr->num_sge) {
+ if (rc_sq_wqe->msg_len > qp->max_inline_data) {
+ ret = -1;
+ *bad_wr = wr;
+ printf("inline data len(1-32)=%d, send_flags = 0x%x, check failed!\r\n",
+ wr->send_flags, rc_sq_wqe->msg_len);
+ return ret;
+ }
+
+ for (i = 0; i < wr->num_sge; i++) {
+ memcpy(wqe,
+ ((void *) (uintptr_t) wr->sg_list[i].addr),
+ wr->sg_list[i].length);
+ wqe = wqe + wr->sg_list[i].length;
+ }
+
+ roce_set_bit(rc_sq_wqe->byte_4,
+ RC_SQ_WQE_BYTE_4_INLINE_S, 1);
+ } else {
+ /* set sge */
+ if (wr->num_sge <= 2) {
+ for (i = 0; i < wr->num_sge; i++)
+ set_data_seg_v2(dseg + i,
+ wr->sg_list + i);
+ } else {
+ roce_set_field(rc_sq_wqe->byte_20,
+ RC_SQ_WQE_BYTE_20_MSG_START_SGE_IDX_M,
+ RC_SQ_WQE_BYTE_20_MSG_START_SGE_IDX_S,
+ ind_sge & (qp->sge.sge_cnt - 1));
+
+ for (i = 0; i < 2; i++)
+ set_data_seg_v2(dseg + i,
+ wr->sg_list + i);
+
+ dseg = get_send_sge_ex(qp, ind_sge &
+ (qp->sge.sge_cnt - 1));
+
+ for (i = 0; i < wr->num_sge - 2; i++) {
+ set_data_seg_v2(dseg + i,
+ wr->sg_list + 2 + i);
+ ind_sge++;
+ }
+ }
+ }
+
+ ind++;
+ }
+
+out:
+ if (likely(nreq)) {
+ qp->sq.head += nreq;
+
+ hns_roce_update_sq_db(ctx, qp->ibv_qp.qp_num, qp->sl,
+ qp->sq.head & ((qp->sq.wqe_cnt << 1) - 1));
+
+ qp->next_sge = ind_sge;
+ }
+
+ pthread_spin_unlock(&qp->sq.lock);
+
+ return ret;
+}
+
+static int hns_roce_u_v2_post_recv(struct ibv_qp *ibvqp, struct ibv_recv_wr *wr,
+ struct ibv_recv_wr **bad_wr)
+{
+ int ret = 0;
+ int nreq;
+ int ind;
+ struct hns_roce_qp *qp = to_hr_qp(ibvqp);
+ struct hns_roce_context *ctx = to_hr_ctx(ibvqp->context);
+ struct hns_roce_v2_wqe_data_seg *dseg;
+ void *wqe;
+ int i;
+
+ pthread_spin_lock(&qp->rq.lock);
+
+ /* check that state is OK to post receive */
+ ind = qp->rq.head & (qp->rq.wqe_cnt - 1);
+
+ if (ibvqp->state == IBV_QPS_RESET || ibvqp->state == IBV_QPS_ERR) {
+ pthread_spin_unlock(&qp->rq.lock);
+ *bad_wr = wr;
+ return -1;
+ }
+
+ for (nreq = 0; wr; ++nreq, wr = wr->next) {
+ if (hns_roce_v2_wq_overflow(&qp->rq, nreq,
+ to_hr_cq(qp->ibv_qp.recv_cq))) {
+ ret = -1;
+ *bad_wr = wr;
+ goto out;
+ }
+
+ if (wr->num_sge > qp->rq.max_gs) {
+ ret = -1;
+ *bad_wr = wr;
+ goto out;
+ }
+
+ wqe = get_recv_wqe_v2(qp, ind);
+ dseg = (struct hns_roce_v2_wqe_data_seg *)wqe;
+
+ for (i = 0; i < wr->num_sge; i++)
+ set_data_seg_v2(dseg + i, wr->sg_list + i);
+
+ qp->rq.wrid[ind] = wr->wr_id;
+
+ ind = (ind + 1) & (qp->rq.wqe_cnt - 1);
+ }
+
+out:
+ if (nreq) {
+ qp->rq.head += nreq;
+
+ hns_roce_update_rq_db(ctx, qp->ibv_qp.qp_num,
+ qp->rq.head & ((qp->rq.wqe_cnt << 1) - 1));
+ }
+
+ pthread_spin_unlock(&qp->rq.lock);
+
+ return ret;
+}
+
static void __hns_roce_v2_cq_clean(struct hns_roce_cq *cq, uint32_t qpn,
struct hns_roce_srq *srq)
{
@@ -531,6 +909,8 @@ struct hns_roce_u_hw hns_roce_u_hw_v2 = {
.hw_version = HNS_ROCE_HW_VER2,
.poll_cq = hns_roce_u_v2_poll_cq,
.arm_cq = hns_roce_u_v2_arm_cq,
+ .post_send = hns_roce_u_v2_post_send,
+ .post_recv = hns_roce_u_v2_post_recv,
.modify_qp = hns_roce_u_v2_modify_qp,
.destroy_qp = hns_roce_u_v2_destroy_qp,
};
diff --git a/providers/hns/hns_roce_u_hw_v2.h b/providers/hns/hns_roce_u_hw_v2.h
index 238bebf..28aab60 100644
--- a/providers/hns/hns_roce_u_hw_v2.h
+++ b/providers/hns/hns_roce_u_hw_v2.h
@@ -208,4 +208,54 @@ struct hns_roce_v2_cqe {
#define CQE_BYTE_32_LPK_S 31
+struct hns_roce_rc_sq_wqe {
+ unsigned int byte_4;
+ unsigned int msg_len;
+ unsigned int inv_key_immtdata;
+ unsigned int byte_16;
+ unsigned int byte_20;
+ unsigned int rkey;
+ uint64_t va;
+};
+
+#define RC_SQ_WQE_BYTE_4_OPCODE_S 0
+#define RC_SQ_WQE_BYTE_4_OPCODE_M \
+ (((1UL << 5) - 1) << RC_SQ_WQE_BYTE_4_OPCODE_S)
+
+#define RC_SQ_WQE_BYTE_4_OWNER_S 7
+
+#define RC_SQ_WQE_BYTE_4_CQE_S 8
+
+#define RC_SQ_WQE_BYTE_4_FENCE_S 9
+
+#define RC_SQ_WQE_BYTE_4_SO_S 10
+
+#define RC_SQ_WQE_BYTE_4_SE_S 11
+
+#define RC_SQ_WQE_BYTE_4_INLINE_S 12
+
+#define RC_SQ_WQE_BYTE_16_XRC_SRQN_S 0
+#define RC_SQ_WQE_BYTE_16_XRC_SRQN_M \
+ (((1UL << 24) - 1) << RC_SQ_WQE_BYTE_16_XRC_SRQN_S)
+
+#define RC_SQ_WQE_BYTE_16_SGE_NUM_S 24
+#define RC_SQ_WQE_BYTE_16_SGE_NUM_M \
+ (((1UL << 8) - 1) << RC_SQ_WQE_BYTE_16_SGE_NUM_S)
+
+#define RC_SQ_WQE_BYTE_20_MSG_START_SGE_IDX_S 0
+#define RC_SQ_WQE_BYTE_20_MSG_START_SGE_IDX_M \
+ (((1UL << 24) - 1) << RC_SQ_WQE_BYTE_20_MSG_START_SGE_IDX_S)
+
+struct hns_roce_v2_wqe_data_seg {
+ __be32 len;
+ __be32 lkey;
+ __be64 addr;
+};
+
+struct hns_roce_v2_wqe_raddr_seg {
+ __be32 rkey;
+ __be32 len;
+ __be64 raddr;
+};
+
#endif /* _HNS_ROCE_U_HW_V2_H */
diff --git a/providers/hns/hns_roce_u_verbs.c b/providers/hns/hns_roce_u_verbs.c
index 64a4ac3..7dc643c 100644
--- a/providers/hns/hns_roce_u_verbs.c
+++ b/providers/hns/hns_roce_u_verbs.c
@@ -50,6 +50,7 @@ void hns_roce_init_qp_indices(struct hns_roce_qp *qp)
qp->sq.tail = 0;
qp->rq.head = 0;
qp->rq.tail = 0;
+ qp->next_sge = 0;
}
int hns_roce_u_query_device(struct ibv_context *context,
@@ -187,7 +188,6 @@ static void hns_roce_set_sq_sizes(struct hns_roce_qp *qp,
{
struct hns_roce_context *ctx = to_hr_ctx(qp->ibv_qp.context);
- qp->sq.max_gs = 2;
cap->max_send_sge = min(ctx->max_sge, qp->sq.max_gs);
qp->sq.max_post = min(ctx->max_qp_wr, qp->sq.wqe_cnt);
cap->max_send_wr = qp->sq.max_post;
@@ -372,21 +372,52 @@ static int hns_roce_alloc_qp_buf(struct ibv_pd *pd, struct ibv_qp_cap *cap,
}
}
- for (qp->rq.wqe_shift = 4;
- 1 << qp->rq.wqe_shift < sizeof(struct hns_roce_rc_send_wqe);
- qp->rq.wqe_shift++)
- ;
-
- qp->buf_size = align((qp->sq.wqe_cnt << qp->sq.wqe_shift), 0x1000) +
- (qp->rq.wqe_cnt << qp->rq.wqe_shift);
-
- if (qp->rq.wqe_shift > qp->sq.wqe_shift) {
- qp->rq.offset = 0;
- qp->sq.offset = qp->rq.wqe_cnt << qp->rq.wqe_shift;
+ if (to_hr_dev(pd->context->device)->hw_version == HNS_ROCE_HW_VER1) {
+ for (qp->rq.wqe_shift = 4; 1 << qp->rq.wqe_shift <
+ sizeof(struct hns_roce_rc_send_wqe); qp->rq.wqe_shift++)
+ ;
+
+ qp->buf_size = align((qp->sq.wqe_cnt << qp->sq.wqe_shift),
+ 0x1000) +
+ (qp->rq.wqe_cnt << qp->rq.wqe_shift);
+
+ if (qp->rq.wqe_shift > qp->sq.wqe_shift) {
+ qp->rq.offset = 0;
+ qp->sq.offset = qp->rq.wqe_cnt << qp->rq.wqe_shift;
+ } else {
+ qp->rq.offset = align((qp->sq.wqe_cnt <<
+ qp->sq.wqe_shift), 0x1000);
+ qp->sq.offset = 0;
+ }
} else {
- qp->rq.offset = align((qp->sq.wqe_cnt << qp->sq.wqe_shift),
- 0x1000);
- qp->sq.offset = 0;
+ for (qp->rq.wqe_shift = 4; 1 << qp->rq.wqe_shift < 16 *
+ cap->max_recv_sge; qp->rq.wqe_shift++)
+ ;
+
+ if (qp->sq.max_gs > 2)
+ qp->sge.sge_shift = 4;
+ else
+ qp->sge.sge_shift = 0;
+
+ qp->buf_size = align((qp->sq.wqe_cnt << qp->sq.wqe_shift),
+ 0x1000) +
+ align((qp->sge.sge_cnt << qp->sge.sge_shift),
+ 0x1000) +
+ (qp->rq.wqe_cnt << qp->rq.wqe_shift);
+
+ if (qp->sge.sge_cnt) {
+ qp->sq.offset = 0;
+ qp->sge.offset = align((qp->sq.wqe_cnt <<
+ qp->sq.wqe_shift), 0x1000);
+ qp->rq.offset = qp->sge.offset +
+ align((qp->sge.sge_cnt <<
+ qp->sge.sge_shift), 0x1000);
+ } else {
+ qp->sq.offset = 0;
+ qp->sge.offset = 0;
+ qp->rq.offset = align((qp->sq.wqe_cnt <<
+ qp->sq.wqe_shift), 0x1000);
+ }
}
if (hns_roce_alloc_buf(&qp->buf, align(qp->buf_size, 0x1000),
@@ -427,6 +458,7 @@ struct ibv_qp *hns_roce_u_create_qp(struct ibv_pd *pd,
struct hns_roce_create_qp cmd;
struct ibv_create_qp_resp resp;
struct hns_roce_context *context = to_hr_ctx(pd->context);
+ unsigned int sge_ex_count;
if (hns_roce_verify_qp(attr, context)) {
fprintf(stderr, "hns_roce_verify_sizes failed!\n");
@@ -443,6 +475,20 @@ struct ibv_qp *hns_roce_u_create_qp(struct ibv_pd *pd,
qp->sq.wqe_cnt = align_qp_size(attr->cap.max_send_wr);
qp->rq.wqe_cnt = align_qp_size(attr->cap.max_recv_wr);
+ if (to_hr_dev(pd->context->device)->hw_version == HNS_ROCE_HW_VER1) {
+ qp->sq.max_gs = 2;
+ } else {
+ qp->sq.max_gs = attr->cap.max_send_sge;
+ if (qp->sq.max_gs > 2) {
+ sge_ex_count = qp->sq.wqe_cnt * (qp->sq.max_gs - 2);
+ for (qp->sge.sge_cnt = 1; qp->sge.sge_cnt <
+ sge_ex_count; qp->sge.sge_cnt <<= 1)
+ ;
+ } else {
+ qp->sge.sge_cnt = 0;
+ }
+ }
+
if (hns_roce_alloc_qp_buf(pd, &attr->cap, attr->qp_type, qp)) {
fprintf(stderr, "hns_roce_alloc_qp_buf failed!\n");
goto err;
--
1.9.1
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related [flat|nested] 8+ messages in thread* [PATCH rdma-core 5/5] libhns: Update consolidate repo for userspace library of hip08 RoCE
[not found] ` <1508245946-145697-1-git-send-email-oulijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
` (3 preceding siblings ...)
2017-10-17 13:12 ` [PATCH rdma-core 4/5] libhns: Reimplement verbs of post_send and post_recv for hip08 RoCE Lijun Ou
@ 2017-10-17 13:12 ` Lijun Ou
2017-10-23 17:27 ` [PATCH rdma-core 0/5] Userspace library matching for " Leon Romanovsky
5 siblings, 0 replies; 8+ messages in thread
From: Lijun Ou @ 2017-10-17 13:12 UTC (permalink / raw)
To: dledford-H+wXaHxf7aLQT0dZR+AlfA, leon-DgEjT+Ai2ygdnm+yROfE0A,
jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA
This patch configure the consolidated repo to build user driver
of hip08 RoCE.
Signed-off-by: Lijun Ou <oulijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
---
providers/hns/CMakeLists.txt | 1 +
1 file changed, 1 insertion(+)
diff --git a/providers/hns/CMakeLists.txt b/providers/hns/CMakeLists.txt
index 19a793e..f136151 100644
--- a/providers/hns/CMakeLists.txt
+++ b/providers/hns/CMakeLists.txt
@@ -2,5 +2,6 @@ rdma_provider(hns
hns_roce_u.c
hns_roce_u_buf.c
hns_roce_u_hw_v1.c
+ hns_roce_u_hw_v2.c
hns_roce_u_verbs.c
)
--
1.9.1
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related [flat|nested] 8+ messages in thread* Re: [PATCH rdma-core 0/5] Userspace library matching for hip08 RoCE
[not found] ` <1508245946-145697-1-git-send-email-oulijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
` (4 preceding siblings ...)
2017-10-17 13:12 ` [PATCH rdma-core 5/5] libhns: Update consolidate repo for userspace library of " Lijun Ou
@ 2017-10-23 17:27 ` Leon Romanovsky
[not found] ` <20171023172725.GN2106-U/DQcQFIOTAAJjI8aNfphQ@public.gmane.org>
5 siblings, 1 reply; 8+ messages in thread
From: Leon Romanovsky @ 2017-10-23 17:27 UTC (permalink / raw)
To: Lijun Ou
Cc: dledford-H+wXaHxf7aLQT0dZR+AlfA,
jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/,
linux-rdma-u79uwXL29TY76Z2rM5mHXA
[-- Attachment #1: Type: text/plain, Size: 1105 bytes --]
On Tue, Oct 17, 2017 at 09:12:21PM +0800, Lijun Ou wrote:
> This patch series add userspace library for hip08 RoCE driver. It will
> work with the kernel driver of hip08 RoCE(hns-roce-hw-v2.ko).
>
> Lijun Ou (5):
> libhns: Add new PCI device matching for hip08
> libhns: Introduce QP operations refered to hip08 RoCE device
> libhns: Introduce CQ operations refered to hip08 device
> libhns: Reimplement verbs of post_send and post_recv for hip08 RoCE
> libhns: Update consolidate repo for userspace library of hip08 RoCE
>
Hi Lijun,
sorry for my slow response, your code causes to warnings during
compilation:
-- Build files have been written to: /home/leonro/src/rdma-core/build
[81/190] Building C object providers/hns/CMakeFiles/hns-rdmav16.dir/hns_roce_u_hw_v2.c.o
../providers/hns/hns_roce_u_hw_v2.c: In function ‘hns_roce_v2_handle_error_cqe’:
../providers/hns/hns_roce_u_hw_v2.c:62:3: warning: statement will never be executed [-Wswitch-unreachable]
fprintf(stderr, PFX "error cqe!\n");
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Please fix it.
Thanks
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 8+ messages in thread