* [PATCH mlx5-next 1/3] net/mlx5: Expose packet based credit mode
2018-11-30 11:22 [PATCH rdma-next 0/3] Packet based credit mode Leon Romanovsky
@ 2018-11-30 11:22 ` Leon Romanovsky
2018-11-30 11:22 ` [PATCH rdma-next 2/3] IB/mlx5: Add packet based credit mode support Leon Romanovsky
` (2 subsequent siblings)
3 siblings, 0 replies; 7+ messages in thread
From: Leon Romanovsky @ 2018-11-30 11:22 UTC (permalink / raw)
To: Doug Ledford, Jason Gunthorpe
Cc: Leon Romanovsky, RDMA mailing list, Danit Goldberg, Majd Dibbiny,
Saeed Mahameed, linux-netdev
From: Danit Goldberg <danitg@mellanox.com>
Packet based credit mode bit determines whether the credit mode
is done per message or packet. Expose the QP creation flag and
the HCA capability.
Signed-off-by: Danit Goldberg <danitg@mellanox.com>
Reviewed-by: Majd Dibbiny <majd@mellanox.com>
Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
---
include/linux/mlx5/mlx5_ifc.h | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/include/linux/mlx5/mlx5_ifc.h b/include/linux/mlx5/mlx5_ifc.h
index 171d68663640..52393fbcf3b4 100644
--- a/include/linux/mlx5/mlx5_ifc.h
+++ b/include/linux/mlx5/mlx5_ifc.h
@@ -1057,7 +1057,8 @@ struct mlx5_ifc_cmd_hca_cap_bits {
u8 vector_calc[0x1];
u8 umr_ptr_rlky[0x1];
u8 imaicl[0x1];
- u8 reserved_at_232[0x4];
+ u8 qp_packet_based[0x1];
+ u8 reserved_at_233[0x3];
u8 qkv[0x1];
u8 pkv[0x1];
u8 set_deth_sqpn[0x1];
@@ -2269,7 +2270,8 @@ struct mlx5_ifc_qpc_bits {
u8 st[0x8];
u8 reserved_at_10[0x3];
u8 pm_state[0x2];
- u8 reserved_at_15[0x3];
+ u8 reserved_at_15[0x1];
+ u8 req_e2e_credit_mode[0x2];
u8 offload_type[0x4];
u8 end_padding_mode[0x2];
u8 reserved_at_1e[0x2];
--
2.19.1
^ permalink raw reply related [flat|nested] 7+ messages in thread* [PATCH rdma-next 2/3] IB/mlx5: Add packet based credit mode support
2018-11-30 11:22 [PATCH rdma-next 0/3] Packet based credit mode Leon Romanovsky
2018-11-30 11:22 ` [PATCH mlx5-next 1/3] net/mlx5: Expose packet " Leon Romanovsky
@ 2018-11-30 11:22 ` Leon Romanovsky
2018-11-30 11:22 ` [PATCH rdma-next 3/3] IB/mlx5: Report packet based credit mode device capability Leon Romanovsky
2018-12-07 3:27 ` [PATCH rdma-next 0/3] Packet based credit mode Jason Gunthorpe
3 siblings, 0 replies; 7+ messages in thread
From: Leon Romanovsky @ 2018-11-30 11:22 UTC (permalink / raw)
To: Doug Ledford, Jason Gunthorpe
Cc: Leon Romanovsky, RDMA mailing list, Danit Goldberg, Majd Dibbiny,
Saeed Mahameed, linux-netdev
From: Danit Goldberg <danitg@mellanox.com>
The device can support two credit modes, message based (default) and
packet based. In order to enable packet based mode, the QP should be
created with special flag that indicates this.
This patch adds support for the new DV QP creation flag that can be used
for RC QPs in order to change the credit mode.
Signed-off-by: Danit Goldberg <danitg@mellanox.com>
Reviewed-by: Majd Dibbiny <majd@mellanox.com>
Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
---
drivers/infiniband/hw/mlx5/mlx5_ib.h | 1 +
drivers/infiniband/hw/mlx5/qp.c | 15 +++++++++++++--
include/uapi/rdma/mlx5-abi.h | 1 +
3 files changed, 15 insertions(+), 2 deletions(-)
diff --git a/drivers/infiniband/hw/mlx5/mlx5_ib.h b/drivers/infiniband/hw/mlx5/mlx5_ib.h
index 4ad493f581de..95652ef8d89b 100644
--- a/drivers/infiniband/hw/mlx5/mlx5_ib.h
+++ b/drivers/infiniband/hw/mlx5/mlx5_ib.h
@@ -462,6 +462,7 @@ enum mlx5_ib_qp_flags {
MLX5_IB_QP_UNDERLAY = 1 << 10,
MLX5_IB_QP_PCI_WRITE_END_PADDING = 1 << 11,
MLX5_IB_QP_TUNNEL_OFFLOAD = 1 << 12,
+ MLX5_IB_QP_PACKET_BASED_CREDIT = 1 << 13,
};
struct mlx5_umr_wr {
diff --git a/drivers/infiniband/hw/mlx5/qp.c b/drivers/infiniband/hw/mlx5/qp.c
index 369db954edbe..485d3c838bf2 100644
--- a/drivers/infiniband/hw/mlx5/qp.c
+++ b/drivers/infiniband/hw/mlx5/qp.c
@@ -1917,7 +1917,8 @@ static int create_qp_common(struct mlx5_ib_dev *dev, struct ib_pd *pd,
MLX5_QP_FLAG_BFREG_INDEX |
MLX5_QP_FLAG_TYPE_DCT |
MLX5_QP_FLAG_TYPE_DCI |
- MLX5_QP_FLAG_ALLOW_SCATTER_CQE))
+ MLX5_QP_FLAG_ALLOW_SCATTER_CQE |
+ MLX5_QP_FLAG_PACKET_BASED_CREDIT_MODE))
return -EINVAL;
err = get_qp_user_index(to_mucontext(pd->uobject->context),
@@ -1953,6 +1954,15 @@ static int create_qp_common(struct mlx5_ib_dev *dev, struct ib_pd *pd,
qp->flags_en |= MLX5_QP_FLAG_TIR_ALLOW_SELF_LB_MC;
}
+ if (ucmd.flags & MLX5_QP_FLAG_PACKET_BASED_CREDIT_MODE) {
+ if (init_attr->qp_type != IB_QPT_RC ||
+ !MLX5_CAP_GEN(dev->mdev, qp_packet_based)) {
+ mlx5_ib_dbg(dev, "packet based credit mode isn't supported\n");
+ return -EOPNOTSUPP;
+ }
+ qp->flags |= MLX5_IB_QP_PACKET_BASED_CREDIT;
+ }
+
if (init_attr->create_flags & IB_QP_CREATE_SOURCE_QPN) {
if (init_attr->qp_type != IB_QPT_UD ||
(MLX5_CAP_GEN(dev->mdev, port_type) !=
@@ -2049,7 +2059,8 @@ static int create_qp_common(struct mlx5_ib_dev *dev, struct ib_pd *pd,
MLX5_SET(qpc, qpc, cd_slave_send, 1);
if (qp->flags & MLX5_IB_QP_MANAGED_RECV)
MLX5_SET(qpc, qpc, cd_slave_receive, 1);
-
+ if (qp->flags & MLX5_IB_QP_PACKET_BASED_CREDIT)
+ MLX5_SET(qpc, qpc, req_e2e_credit_mode, 1);
if (qp->scat_cqe && is_connected(init_attr->qp_type)) {
configure_responder_scat_cqe(init_attr, qpc);
configure_requester_scat_cqe(dev, init_attr,
diff --git a/include/uapi/rdma/mlx5-abi.h b/include/uapi/rdma/mlx5-abi.h
index 8fa9f90e2bb1..4af581170f62 100644
--- a/include/uapi/rdma/mlx5-abi.h
+++ b/include/uapi/rdma/mlx5-abi.h
@@ -48,6 +48,7 @@ enum {
MLX5_QP_FLAG_TIR_ALLOW_SELF_LB_UC = 1 << 6,
MLX5_QP_FLAG_TIR_ALLOW_SELF_LB_MC = 1 << 7,
MLX5_QP_FLAG_ALLOW_SCATTER_CQE = 1 << 8,
+ MLX5_QP_FLAG_PACKET_BASED_CREDIT_MODE = 1 << 9,
};
enum {
--
2.19.1
^ permalink raw reply related [flat|nested] 7+ messages in thread* [PATCH rdma-next 3/3] IB/mlx5: Report packet based credit mode device capability
2018-11-30 11:22 [PATCH rdma-next 0/3] Packet based credit mode Leon Romanovsky
2018-11-30 11:22 ` [PATCH mlx5-next 1/3] net/mlx5: Expose packet " Leon Romanovsky
2018-11-30 11:22 ` [PATCH rdma-next 2/3] IB/mlx5: Add packet based credit mode support Leon Romanovsky
@ 2018-11-30 11:22 ` Leon Romanovsky
2018-12-07 3:27 ` [PATCH rdma-next 0/3] Packet based credit mode Jason Gunthorpe
3 siblings, 0 replies; 7+ messages in thread
From: Leon Romanovsky @ 2018-11-30 11:22 UTC (permalink / raw)
To: Doug Ledford, Jason Gunthorpe
Cc: Leon Romanovsky, RDMA mailing list, Danit Goldberg, Majd Dibbiny,
Saeed Mahameed, linux-netdev
From: Danit Goldberg <danitg@mellanox.com>
Report packet based credit mode capability via the mlx5 DV interface.
Signed-off-by: Danit Goldberg <danitg@mellanox.com>
Reviewed-by: Majd Dibbiny <majd@mellanox.com>
Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
---
drivers/infiniband/hw/mlx5/main.c | 3 +++
include/uapi/rdma/mlx5-abi.h | 1 +
2 files changed, 4 insertions(+)
diff --git a/drivers/infiniband/hw/mlx5/main.c b/drivers/infiniband/hw/mlx5/main.c
index 0acf6a4d8f4b..1ad5e35663af 100644
--- a/drivers/infiniband/hw/mlx5/main.c
+++ b/drivers/infiniband/hw/mlx5/main.c
@@ -1018,6 +1018,9 @@ static int mlx5_ib_query_device(struct ib_device *ibdev,
if (MLX5_CAP_GEN(mdev, cqe_128_always))
resp.flags |= MLX5_IB_QUERY_DEV_RESP_FLAGS_CQE_128B_PAD;
+ if (MLX5_CAP_GEN(mdev, qp_packet_based))
+ resp.flags |=
+ MLX5_IB_QUERY_DEV_RESP_PACKET_BASED_CREDIT_MODE;
}
if (field_avail(typeof(resp), sw_parsing_caps,
diff --git a/include/uapi/rdma/mlx5-abi.h b/include/uapi/rdma/mlx5-abi.h
index 4af581170f62..87b3198f4b5d 100644
--- a/include/uapi/rdma/mlx5-abi.h
+++ b/include/uapi/rdma/mlx5-abi.h
@@ -237,6 +237,7 @@ enum mlx5_ib_query_dev_resp_flags {
/* Support 128B CQE compression */
MLX5_IB_QUERY_DEV_RESP_FLAGS_CQE_128B_COMP = 1 << 0,
MLX5_IB_QUERY_DEV_RESP_FLAGS_CQE_128B_PAD = 1 << 1,
+ MLX5_IB_QUERY_DEV_RESP_PACKET_BASED_CREDIT_MODE = 1 << 2,
};
enum mlx5_ib_tunnel_offloads {
--
2.19.1
^ permalink raw reply related [flat|nested] 7+ messages in thread* Re: [PATCH rdma-next 0/3] Packet based credit mode
2018-11-30 11:22 [PATCH rdma-next 0/3] Packet based credit mode Leon Romanovsky
` (2 preceding siblings ...)
2018-11-30 11:22 ` [PATCH rdma-next 3/3] IB/mlx5: Report packet based credit mode device capability Leon Romanovsky
@ 2018-12-07 3:27 ` Jason Gunthorpe
2018-12-07 6:04 ` Leon Romanovsky
3 siblings, 1 reply; 7+ messages in thread
From: Jason Gunthorpe @ 2018-12-07 3:27 UTC (permalink / raw)
To: Leon Romanovsky
Cc: Doug Ledford, Leon Romanovsky, RDMA mailing list, Danit Goldberg,
Majd Dibbiny, Saeed Mahameed, linux-netdev
On Fri, Nov 30, 2018 at 01:22:03PM +0200, Leon Romanovsky wrote:
> From: Leon Romanovsky <leonro@mellanox.com>
>
> >From Danit,
>
> Packet based credit mode is an alternative end-to-end credit mode for QPs
> set during their creation. Credits are transported from the responder
> to the requester to optimize the use of its receive resources.
> In packet-based credit mode, credits are issued on a per packet basis.
>
> The advantage of this feature comes while sending large RDMA messages
> through switches that are short in memory.
>
> The first commit exposes QP creation flag and the HCA capability. The
> second commit adds support for a new DV QP creation flag. The last
> commit report packet based credit mode capability via the MLX5DV device
> capabilities.
>
> Thanks
>
> Danit Goldberg (3):
> net/mlx5: Expose packet based credit mode
> IB/mlx5: Add packet based credit mode support
> IB/mlx5: Report packet based credit mode device capability
This looks fine to me, can you update the shared branch please
Thanks,
Jason
^ permalink raw reply [flat|nested] 7+ messages in thread* Re: [PATCH rdma-next 0/3] Packet based credit mode
2018-12-07 3:27 ` [PATCH rdma-next 0/3] Packet based credit mode Jason Gunthorpe
@ 2018-12-07 6:04 ` Leon Romanovsky
2018-12-07 20:29 ` Jason Gunthorpe
0 siblings, 1 reply; 7+ messages in thread
From: Leon Romanovsky @ 2018-12-07 6:04 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Doug Ledford, RDMA mailing list, Danit Goldberg, Majd Dibbiny,
Saeed Mahameed, linux-netdev
[-- Attachment #1: Type: text/plain, Size: 1232 bytes --]
On Thu, Dec 06, 2018 at 08:27:06PM -0700, Jason Gunthorpe wrote:
> On Fri, Nov 30, 2018 at 01:22:03PM +0200, Leon Romanovsky wrote:
> > From: Leon Romanovsky <leonro@mellanox.com>
> >
> > >From Danit,
> >
> > Packet based credit mode is an alternative end-to-end credit mode for QPs
> > set during their creation. Credits are transported from the responder
> > to the requester to optimize the use of its receive resources.
> > In packet-based credit mode, credits are issued on a per packet basis.
> >
> > The advantage of this feature comes while sending large RDMA messages
> > through switches that are short in memory.
> >
> > The first commit exposes QP creation flag and the HCA capability. The
> > second commit adds support for a new DV QP creation flag. The last
> > commit report packet based credit mode capability via the MLX5DV device
> > capabilities.
> >
> > Thanks
> >
> > Danit Goldberg (3):
> > net/mlx5: Expose packet based credit mode
> > IB/mlx5: Add packet based credit mode support
> > IB/mlx5: Report packet based credit mode device capability
>
> This looks fine to me, can you update the shared branch please
Done, thanks
3fd3c80acc17 net/mlx5: Expose packet based credit mode
>
> Thanks,
> Jason
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 801 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH rdma-next 0/3] Packet based credit mode
2018-12-07 6:04 ` Leon Romanovsky
@ 2018-12-07 20:29 ` Jason Gunthorpe
0 siblings, 0 replies; 7+ messages in thread
From: Jason Gunthorpe @ 2018-12-07 20:29 UTC (permalink / raw)
To: Leon Romanovsky
Cc: Doug Ledford, RDMA mailing list, Danit Goldberg, Majd Dibbiny,
Saeed Mahameed, linux-netdev
On Fri, Dec 07, 2018 at 08:04:26AM +0200, Leon Romanovsky wrote:
> On Thu, Dec 06, 2018 at 08:27:06PM -0700, Jason Gunthorpe wrote:
> > On Fri, Nov 30, 2018 at 01:22:03PM +0200, Leon Romanovsky wrote:
> > > From: Leon Romanovsky <leonro@mellanox.com>
> > >
> > > >From Danit,
> > >
> > > Packet based credit mode is an alternative end-to-end credit mode for QPs
> > > set during their creation. Credits are transported from the responder
> > > to the requester to optimize the use of its receive resources.
> > > In packet-based credit mode, credits are issued on a per packet basis.
> > >
> > > The advantage of this feature comes while sending large RDMA messages
> > > through switches that are short in memory.
> > >
> > > The first commit exposes QP creation flag and the HCA capability. The
> > > second commit adds support for a new DV QP creation flag. The last
> > > commit report packet based credit mode capability via the MLX5DV device
> > > capabilities.
> > >
> > > Thanks
> > >
> > > Danit Goldberg (3):
> > > net/mlx5: Expose packet based credit mode
> > > IB/mlx5: Add packet based credit mode support
> > > IB/mlx5: Report packet based credit mode device capability
> >
> > This looks fine to me, can you update the shared branch please
>
> Done, thanks
> 3fd3c80acc17 net/mlx5: Expose packet based credit mode
Applied to for-next
Thanks,
Jason
^ permalink raw reply [flat|nested] 7+ messages in thread