* [PATCH rdma-next 0/2] Extend mlx5 CQ creation with large UAR page index
@ 2024-06-16 16:15 Leon Romanovsky
2024-06-16 16:15 ` [PATCH rdma-next 1/2] RDMA: Pass entire uverbs attr bundle to create cq function Leon Romanovsky
` (2 more replies)
0 siblings, 3 replies; 14+ messages in thread
From: Leon Romanovsky @ 2024-06-16 16:15 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Leon Romanovsky, Ajay Sharma, Akiva Goldberger, Bernard Metzler,
Chengchang Tang, Cheng Xu, Junxian Huang, Kai Shen, linux-kernel,
linux-rdma, Long Li, Michael Margolin, Mustafa Ismail,
Potnuri Bharat Teja, Selvin Xavier, Shiraz Saleem, Yishai Hadas,
Zhu Yanjun
From: Leon Romanovsky <leonro@nvidia.com>
Hi,
This series from Akiva extends the mlx5 private field with the UAR page index
which is larger than 16 bits as was before.
As this is first time, we extend ioctl API with private data field after
it already has UHW object, we need to change create CQ API signature to
support it.
Thanks
Akiva Goldberger (2):
RDMA: Pass entire uverbs attr bundle to create cq function
RDMA/mlx5: Send UAR page index as ioctl attribute
drivers/infiniband/core/uverbs_cmd.c | 2 +-
drivers/infiniband/core/uverbs_std_types_cq.c | 2 +-
drivers/infiniband/hw/bnxt_re/ib_verbs.c | 3 +-
drivers/infiniband/hw/bnxt_re/ib_verbs.h | 2 +-
drivers/infiniband/hw/cxgb4/cq.c | 3 +-
drivers/infiniband/hw/cxgb4/iw_cxgb4.h | 2 +-
drivers/infiniband/hw/efa/efa.h | 2 +-
drivers/infiniband/hw/efa/efa_verbs.c | 3 +-
drivers/infiniband/hw/erdma/erdma_verbs.c | 3 +-
drivers/infiniband/hw/erdma/erdma_verbs.h | 2 +-
drivers/infiniband/hw/hns/hns_roce_cq.c | 3 +-
drivers/infiniband/hw/hns/hns_roce_device.h | 2 +-
drivers/infiniband/hw/irdma/verbs.c | 5 +--
drivers/infiniband/hw/mana/cq.c | 2 +-
drivers/infiniband/hw/mana/mana_ib.h | 2 +-
drivers/infiniband/hw/mlx4/cq.c | 3 +-
drivers/infiniband/hw/mlx4/mlx4_ib.h | 2 +-
drivers/infiniband/hw/mlx5/cq.c | 31 ++++++++++++++++---
drivers/infiniband/hw/mlx5/main.c | 1 +
drivers/infiniband/hw/mlx5/mlx5_ib.h | 3 +-
drivers/infiniband/hw/mthca/mthca_provider.c | 3 +-
drivers/infiniband/sw/rxe/rxe_verbs.c | 3 +-
drivers/infiniband/sw/siw/siw_verbs.c | 2 +-
drivers/infiniband/sw/siw/siw_verbs.h | 2 +-
include/rdma/ib_verbs.h | 2 +-
include/uapi/rdma/ib_user_ioctl_cmds.h | 1 +
include/uapi/rdma/mlx5_user_ioctl_cmds.h | 4 +++
27 files changed, 67 insertions(+), 28 deletions(-)
--
2.45.2
^ permalink raw reply [flat|nested] 14+ messages in thread* [PATCH rdma-next 1/2] RDMA: Pass entire uverbs attr bundle to create cq function 2024-06-16 16:15 [PATCH rdma-next 0/2] Extend mlx5 CQ creation with large UAR page index Leon Romanovsky @ 2024-06-16 16:15 ` Leon Romanovsky 2024-06-17 6:17 ` Leon Romanovsky ` (2 more replies) 2024-06-16 16:15 ` [PATCH rdma-next 2/2] RDMA/mlx5: Send UAR page index as ioctl attribute Leon Romanovsky 2024-06-16 21:47 ` [PATCH rdma-next 0/2] Extend mlx5 CQ creation with large UAR page index Zhu Yanjun 2 siblings, 3 replies; 14+ messages in thread From: Leon Romanovsky @ 2024-06-16 16:15 UTC (permalink / raw) To: Jason Gunthorpe Cc: Akiva Goldberger, Ajay Sharma, Bernard Metzler, Chengchang Tang, Cheng Xu, Junxian Huang, Kai Shen, linux-rdma, Long Li, Michael Margolin, Mustafa Ismail, Potnuri Bharat Teja, Selvin Xavier, Shiraz Saleem, Yishai Hadas, Zhu Yanjun From: Akiva Goldberger <agoldberger@nvidia.com> Changes the create_cq verb signature by sending the entire uverbs attr bundle as a parameter. This allows drivers to send driver specific attrs through ioctl for the create_cq verb and access them in their driver specific code. Also adds a new enum value for driver specific ioctl attritbutes for methods already supporting UHW. Signed-off-by: Akiva Goldberger <agoldberger@nvidia.com> Signed-off-by: Leon Romanovsky <leonro@nvidia.com> --- drivers/infiniband/core/uverbs_cmd.c | 2 +- drivers/infiniband/core/uverbs_std_types_cq.c | 2 +- drivers/infiniband/hw/bnxt_re/ib_verbs.c | 3 ++- drivers/infiniband/hw/bnxt_re/ib_verbs.h | 2 +- drivers/infiniband/hw/cxgb4/cq.c | 3 ++- drivers/infiniband/hw/cxgb4/iw_cxgb4.h | 2 +- drivers/infiniband/hw/efa/efa.h | 2 +- drivers/infiniband/hw/efa/efa_verbs.c | 3 ++- drivers/infiniband/hw/erdma/erdma_verbs.c | 3 ++- drivers/infiniband/hw/erdma/erdma_verbs.h | 2 +- drivers/infiniband/hw/hns/hns_roce_cq.c | 3 ++- drivers/infiniband/hw/hns/hns_roce_device.h | 2 +- drivers/infiniband/hw/irdma/verbs.c | 5 +++-- drivers/infiniband/hw/mana/cq.c | 2 +- drivers/infiniband/hw/mana/mana_ib.h | 2 +- drivers/infiniband/hw/mlx4/cq.c | 3 ++- drivers/infiniband/hw/mlx4/mlx4_ib.h | 2 +- drivers/infiniband/hw/mlx5/cq.c | 3 ++- drivers/infiniband/hw/mlx5/mlx5_ib.h | 2 +- drivers/infiniband/hw/mthca/mthca_provider.c | 3 ++- drivers/infiniband/sw/rxe/rxe_verbs.c | 3 ++- drivers/infiniband/sw/siw/siw_verbs.c | 2 +- drivers/infiniband/sw/siw/siw_verbs.h | 2 +- include/rdma/ib_verbs.h | 2 +- include/uapi/rdma/ib_user_ioctl_cmds.h | 1 + 25 files changed, 36 insertions(+), 25 deletions(-) diff --git a/drivers/infiniband/core/uverbs_cmd.c b/drivers/infiniband/core/uverbs_cmd.c index 3d3ee3eca983..1b3ea71f2c33 100644 --- a/drivers/infiniband/core/uverbs_cmd.c +++ b/drivers/infiniband/core/uverbs_cmd.c @@ -1051,7 +1051,7 @@ static int create_cq(struct uverbs_attr_bundle *attrs, rdma_restrack_new(&cq->res, RDMA_RESTRACK_CQ); rdma_restrack_set_name(&cq->res, NULL); - ret = ib_dev->ops.create_cq(cq, &attr, &attrs->driver_udata); + ret = ib_dev->ops.create_cq(cq, &attr, attrs); if (ret) goto err_free; rdma_restrack_add(&cq->res); diff --git a/drivers/infiniband/core/uverbs_std_types_cq.c b/drivers/infiniband/core/uverbs_std_types_cq.c index 370ad7c83f88..432054f0a8a4 100644 --- a/drivers/infiniband/core/uverbs_std_types_cq.c +++ b/drivers/infiniband/core/uverbs_std_types_cq.c @@ -128,7 +128,7 @@ static int UVERBS_HANDLER(UVERBS_METHOD_CQ_CREATE)( rdma_restrack_new(&cq->res, RDMA_RESTRACK_CQ); rdma_restrack_set_name(&cq->res, NULL); - ret = ib_dev->ops.create_cq(cq, &attr, &attrs->driver_udata); + ret = ib_dev->ops.create_cq(cq, &attr, attrs); if (ret) goto err_free; diff --git a/drivers/infiniband/hw/bnxt_re/ib_verbs.c b/drivers/infiniband/hw/bnxt_re/ib_verbs.c index d261b09025ca..e453ca701e87 100644 --- a/drivers/infiniband/hw/bnxt_re/ib_verbs.c +++ b/drivers/infiniband/hw/bnxt_re/ib_verbs.c @@ -2948,10 +2948,11 @@ int bnxt_re_destroy_cq(struct ib_cq *ib_cq, struct ib_udata *udata) } int bnxt_re_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr, - struct ib_udata *udata) + struct uverbs_attr_bundle *attrs) { struct bnxt_re_cq *cq = container_of(ibcq, struct bnxt_re_cq, ib_cq); struct bnxt_re_dev *rdev = to_bnxt_re_dev(ibcq->device, ibdev); + struct ib_udata *udata = &attrs->driver_udata; struct bnxt_re_ucontext *uctx = rdma_udata_to_drv_context(udata, struct bnxt_re_ucontext, ib_uctx); struct bnxt_qplib_dev_attr *dev_attr = &rdev->dev_attr; diff --git a/drivers/infiniband/hw/bnxt_re/ib_verbs.h b/drivers/infiniband/hw/bnxt_re/ib_verbs.h index b267d6d5975f..e98cb1717338 100644 --- a/drivers/infiniband/hw/bnxt_re/ib_verbs.h +++ b/drivers/infiniband/hw/bnxt_re/ib_verbs.h @@ -221,7 +221,7 @@ int bnxt_re_post_send(struct ib_qp *qp, const struct ib_send_wr *send_wr, int bnxt_re_post_recv(struct ib_qp *qp, const struct ib_recv_wr *recv_wr, const struct ib_recv_wr **bad_recv_wr); int bnxt_re_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr, - struct ib_udata *udata); + struct uverbs_attr_bundle *attrs); int bnxt_re_resize_cq(struct ib_cq *ibcq, int cqe, struct ib_udata *udata); int bnxt_re_destroy_cq(struct ib_cq *cq, struct ib_udata *udata); int bnxt_re_poll_cq(struct ib_cq *cq, int num_entries, struct ib_wc *wc); diff --git a/drivers/infiniband/hw/cxgb4/cq.c b/drivers/infiniband/hw/cxgb4/cq.c index 7e2835dcbc1c..5111421f9473 100644 --- a/drivers/infiniband/hw/cxgb4/cq.c +++ b/drivers/infiniband/hw/cxgb4/cq.c @@ -995,8 +995,9 @@ int c4iw_destroy_cq(struct ib_cq *ib_cq, struct ib_udata *udata) } int c4iw_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr, - struct ib_udata *udata) + struct uverbs_attr_bundle *attrs) { + struct ib_udata *udata = &attrs->driver_udata; struct ib_device *ibdev = ibcq->device; int entries = attr->cqe; int vector = attr->comp_vector; diff --git a/drivers/infiniband/hw/cxgb4/iw_cxgb4.h b/drivers/infiniband/hw/cxgb4/iw_cxgb4.h index fb8a0c248866..f838bb6718af 100644 --- a/drivers/infiniband/hw/cxgb4/iw_cxgb4.h +++ b/drivers/infiniband/hw/cxgb4/iw_cxgb4.h @@ -978,7 +978,7 @@ int c4iw_dereg_mr(struct ib_mr *ib_mr, struct ib_udata *udata); int c4iw_destroy_cq(struct ib_cq *ib_cq, struct ib_udata *udata); void c4iw_cq_rem_ref(struct c4iw_cq *chp); int c4iw_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr, - struct ib_udata *udata); + struct uverbs_attr_bundle *attrs); int c4iw_arm_cq(struct ib_cq *ibcq, enum ib_cq_notify_flags flags); int c4iw_modify_srq(struct ib_srq *ib_srq, struct ib_srq_attr *attr, enum ib_srq_attr_mask srq_attr_mask, diff --git a/drivers/infiniband/hw/efa/efa.h b/drivers/infiniband/hw/efa/efa.h index 926f9ff1f60f..e580e087e9da 100644 --- a/drivers/infiniband/hw/efa/efa.h +++ b/drivers/infiniband/hw/efa/efa.h @@ -161,7 +161,7 @@ int efa_create_qp(struct ib_qp *ibqp, struct ib_qp_init_attr *init_attr, struct ib_udata *udata); int efa_destroy_cq(struct ib_cq *ibcq, struct ib_udata *udata); int efa_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr, - struct ib_udata *udata); + struct uverbs_attr_bundle *attrs); struct ib_mr *efa_reg_mr(struct ib_pd *ibpd, u64 start, u64 length, u64 virt_addr, int access_flags, struct ib_udata *udata); diff --git a/drivers/infiniband/hw/efa/efa_verbs.c b/drivers/infiniband/hw/efa/efa_verbs.c index 8f7a13b79cdc..9ced560d7f42 100644 --- a/drivers/infiniband/hw/efa/efa_verbs.c +++ b/drivers/infiniband/hw/efa/efa_verbs.c @@ -1084,8 +1084,9 @@ static int cq_mmap_entries_setup(struct efa_dev *dev, struct efa_cq *cq, } int efa_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr, - struct ib_udata *udata) + struct uverbs_attr_bundle *attrs) { + struct ib_udata *udata = &attrs->driver_udata; struct efa_ucontext *ucontext = rdma_udata_to_drv_context( udata, struct efa_ucontext, ibucontext); struct efa_com_create_cq_params params = {}; diff --git a/drivers/infiniband/hw/erdma/erdma_verbs.c b/drivers/infiniband/hw/erdma/erdma_verbs.c index 40c9b6e46b82..d7e1cbf9f5c2 100644 --- a/drivers/infiniband/hw/erdma/erdma_verbs.c +++ b/drivers/infiniband/hw/erdma/erdma_verbs.c @@ -1628,8 +1628,9 @@ static int erdma_init_kernel_cq(struct erdma_cq *cq) } int erdma_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr, - struct ib_udata *udata) + struct uverbs_attr_bundle *attrs) { + struct ib_udata *udata = &attrs->driver_udata; struct erdma_cq *cq = to_ecq(ibcq); struct erdma_dev *dev = to_edev(ibcq->device); unsigned int depth = attr->cqe; diff --git a/drivers/infiniband/hw/erdma/erdma_verbs.h b/drivers/infiniband/hw/erdma/erdma_verbs.h index 4f02ba06b210..6afdc02f5869 100644 --- a/drivers/infiniband/hw/erdma/erdma_verbs.h +++ b/drivers/infiniband/hw/erdma/erdma_verbs.h @@ -329,7 +329,7 @@ int erdma_query_device(struct ib_device *dev, struct ib_device_attr *attr, int erdma_get_port_immutable(struct ib_device *dev, u32 port, struct ib_port_immutable *ib_port_immutable); int erdma_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr, - struct ib_udata *data); + struct uverbs_attr_bundle *attrs); int erdma_query_port(struct ib_device *dev, u32 port, struct ib_port_attr *attr); int erdma_query_gid(struct ib_device *dev, u32 port, int idx, diff --git a/drivers/infiniband/hw/hns/hns_roce_cq.c b/drivers/infiniband/hw/hns/hns_roce_cq.c index 56dc3908da2f..4ec66611a143 100644 --- a/drivers/infiniband/hw/hns/hns_roce_cq.c +++ b/drivers/infiniband/hw/hns/hns_roce_cq.c @@ -353,9 +353,10 @@ static int set_cqe_size(struct hns_roce_cq *hr_cq, struct ib_udata *udata, } int hns_roce_create_cq(struct ib_cq *ib_cq, const struct ib_cq_init_attr *attr, - struct ib_udata *udata) + struct uverbs_attr_bundle *attrs) { struct hns_roce_dev *hr_dev = to_hr_dev(ib_cq->device); + struct ib_udata *udata = &attrs->driver_udata; struct hns_roce_ib_create_cq_resp resp = {}; struct hns_roce_cq *hr_cq = to_hr_cq(ib_cq); struct ib_device *ibdev = &hr_dev->ib_dev; diff --git a/drivers/infiniband/hw/hns/hns_roce_device.h b/drivers/infiniband/hw/hns/hns_roce_device.h index ff0b3f68ee3a..ef50cd03f489 100644 --- a/drivers/infiniband/hw/hns/hns_roce_device.h +++ b/drivers/infiniband/hw/hns/hns_roce_device.h @@ -1267,7 +1267,7 @@ __be32 send_ieth(const struct ib_send_wr *wr); int to_hr_qp_type(int qp_type); int hns_roce_create_cq(struct ib_cq *ib_cq, const struct ib_cq_init_attr *attr, - struct ib_udata *udata); + struct uverbs_attr_bundle *attrs); int hns_roce_destroy_cq(struct ib_cq *ib_cq, struct ib_udata *udata); int hns_roce_db_map_user(struct hns_roce_ucontext *context, unsigned long virt, diff --git a/drivers/infiniband/hw/irdma/verbs.c b/drivers/infiniband/hw/irdma/verbs.c index 12704efb7b19..fc0ce35da14e 100644 --- a/drivers/infiniband/hw/irdma/verbs.c +++ b/drivers/infiniband/hw/irdma/verbs.c @@ -2035,14 +2035,15 @@ static inline int cq_validate_flags(u32 flags, u8 hw_rev) * irdma_create_cq - create cq * @ibcq: CQ allocated * @attr: attributes for cq - * @udata: user data + * @attrs: uverbs attribute bundle */ static int irdma_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr, - struct ib_udata *udata) + struct uverbs_attr_bundle *attrs) { #define IRDMA_CREATE_CQ_MIN_REQ_LEN offsetofend(struct irdma_create_cq_req, user_cq_buf) #define IRDMA_CREATE_CQ_MIN_RESP_LEN offsetofend(struct irdma_create_cq_resp, cq_size) + struct ib_udata *udata = &attrs->driver_udata; struct ib_device *ibdev = ibcq->device; struct irdma_device *iwdev = to_iwdev(ibdev); struct irdma_pci_f *rf = iwdev->rf; diff --git a/drivers/infiniband/hw/mana/cq.c b/drivers/infiniband/hw/mana/cq.c index c6a3fd57a196..4bf95df30eb6 100644 --- a/drivers/infiniband/hw/mana/cq.c +++ b/drivers/infiniband/hw/mana/cq.c @@ -6,7 +6,7 @@ #include "mana_ib.h" int mana_ib_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr, - struct ib_udata *udata) + struct ib_udata *udata, struct uverbs_attr_bundle *attrs) { struct mana_ib_cq *cq = container_of(ibcq, struct mana_ib_cq, ibcq); struct mana_ib_create_cq_resp resp = {}; diff --git a/drivers/infiniband/hw/mana/mana_ib.h b/drivers/infiniband/hw/mana/mana_ib.h index 977da9569701..522e2d79eae8 100644 --- a/drivers/infiniband/hw/mana/mana_ib.h +++ b/drivers/infiniband/hw/mana/mana_ib.h @@ -429,7 +429,7 @@ void mana_ib_uncfg_vport(struct mana_ib_dev *dev, struct mana_ib_pd *pd, u32 port); int mana_ib_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr, - struct ib_udata *udata); + struct ib_udata *udata, struct uverbs_attr_bundle *attrs); int mana_ib_destroy_cq(struct ib_cq *ibcq, struct ib_udata *udata); diff --git a/drivers/infiniband/hw/mlx4/cq.c b/drivers/infiniband/hw/mlx4/cq.c index 4cd738aae53c..aa9ea6ba26e5 100644 --- a/drivers/infiniband/hw/mlx4/cq.c +++ b/drivers/infiniband/hw/mlx4/cq.c @@ -172,8 +172,9 @@ static int mlx4_ib_get_cq_umem(struct mlx4_ib_dev *dev, #define CQ_CREATE_FLAGS_SUPPORTED IB_UVERBS_CQ_FLAGS_TIMESTAMP_COMPLETION int mlx4_ib_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr, - struct ib_udata *udata) + struct uverbs_attr_bundle *attrs) { + struct ib_udata *udata = &attrs->driver_udata; struct ib_device *ibdev = ibcq->device; int entries = attr->cqe; int vector = attr->comp_vector; diff --git a/drivers/infiniband/hw/mlx4/mlx4_ib.h b/drivers/infiniband/hw/mlx4/mlx4_ib.h index 41ca1114a995..b52bceff7d97 100644 --- a/drivers/infiniband/hw/mlx4/mlx4_ib.h +++ b/drivers/infiniband/hw/mlx4/mlx4_ib.h @@ -767,7 +767,7 @@ int mlx4_ib_map_mr_sg(struct ib_mr *ibmr, struct scatterlist *sg, int sg_nents, int mlx4_ib_modify_cq(struct ib_cq *cq, u16 cq_count, u16 cq_period); int mlx4_ib_resize_cq(struct ib_cq *ibcq, int entries, struct ib_udata *udata); int mlx4_ib_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr, - struct ib_udata *udata); + struct uverbs_attr_bundle *attrs); int mlx4_ib_destroy_cq(struct ib_cq *cq, struct ib_udata *udata); int mlx4_ib_poll_cq(struct ib_cq *ibcq, int num_entries, struct ib_wc *wc); int mlx4_ib_arm_cq(struct ib_cq *cq, enum ib_cq_notify_flags flags); diff --git a/drivers/infiniband/hw/mlx5/cq.c b/drivers/infiniband/hw/mlx5/cq.c index 4429bf7c746b..172f3987fc87 100644 --- a/drivers/infiniband/hw/mlx5/cq.c +++ b/drivers/infiniband/hw/mlx5/cq.c @@ -942,8 +942,9 @@ static void notify_soft_wc_handler(struct work_struct *work) } int mlx5_ib_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr, - struct ib_udata *udata) + struct uverbs_attr_bundle *attrs) { + struct ib_udata *udata = &attrs->driver_udata; struct ib_device *ibdev = ibcq->device; int entries = attr->cqe; int vector = attr->comp_vector; diff --git a/drivers/infiniband/hw/mlx5/mlx5_ib.h b/drivers/infiniband/hw/mlx5/mlx5_ib.h index bf25ddb17bce..2b03e607561e 100644 --- a/drivers/infiniband/hw/mlx5/mlx5_ib.h +++ b/drivers/infiniband/hw/mlx5/mlx5_ib.h @@ -1332,7 +1332,7 @@ int mlx5_ib_read_wqe_rq(struct mlx5_ib_qp *qp, int wqe_index, void *buffer, int mlx5_ib_read_wqe_srq(struct mlx5_ib_srq *srq, int wqe_index, void *buffer, size_t buflen, size_t *bc); int mlx5_ib_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr, - struct ib_udata *udata); + struct uverbs_attr_bundle *attrs); int mlx5_ib_destroy_cq(struct ib_cq *cq, struct ib_udata *udata); int mlx5_ib_poll_cq(struct ib_cq *ibcq, int num_entries, struct ib_wc *wc); int mlx5_ib_arm_cq(struct ib_cq *ibcq, enum ib_cq_notify_flags flags); diff --git a/drivers/infiniband/hw/mthca/mthca_provider.c b/drivers/infiniband/hw/mthca/mthca_provider.c index e1325f2927d6..677ebb145dbf 100644 --- a/drivers/infiniband/hw/mthca/mthca_provider.c +++ b/drivers/infiniband/hw/mthca/mthca_provider.c @@ -574,7 +574,8 @@ static int mthca_destroy_qp(struct ib_qp *qp, struct ib_udata *udata) static int mthca_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr, - struct ib_udata *udata) + struct ib_udata *udata, + struct uverbs_attr_bundle *attrs) { struct ib_device *ibdev = ibcq->device; int entries = attr->cqe; diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.c b/drivers/infiniband/sw/rxe/rxe_verbs.c index c7d4d8ab5a09..82bb7f33290c 100644 --- a/drivers/infiniband/sw/rxe/rxe_verbs.c +++ b/drivers/infiniband/sw/rxe/rxe_verbs.c @@ -1053,8 +1053,9 @@ static int rxe_post_recv(struct ib_qp *ibqp, const struct ib_recv_wr *wr, /* cq */ static int rxe_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr, - struct ib_udata *udata) + struct uverbs_attr_bundle *attrs) { + struct ib_udata *udata = &attrs->driver_udata; struct ib_device *dev = ibcq->device; struct rxe_dev *rxe = to_rdev(dev); struct rxe_cq *cq = to_rcq(ibcq); diff --git a/drivers/infiniband/sw/siw/siw_verbs.c b/drivers/infiniband/sw/siw/siw_verbs.c index ecf0444666b4..ce7ef5b5ef89 100644 --- a/drivers/infiniband/sw/siw/siw_verbs.c +++ b/drivers/infiniband/sw/siw/siw_verbs.c @@ -1128,7 +1128,7 @@ int siw_destroy_cq(struct ib_cq *base_cq, struct ib_udata *udata) */ int siw_create_cq(struct ib_cq *base_cq, const struct ib_cq_init_attr *attr, - struct ib_udata *udata) + struct ib_udata *udata, struct uverbs_attr_bundle *attrs) { struct siw_device *sdev = to_siw_dev(base_cq->device); struct siw_cq *cq = to_siw_cq(base_cq); diff --git a/drivers/infiniband/sw/siw/siw_verbs.h b/drivers/infiniband/sw/siw/siw_verbs.h index 4b57a4fb7237..5c4c7074eef0 100644 --- a/drivers/infiniband/sw/siw/siw_verbs.h +++ b/drivers/infiniband/sw/siw/siw_verbs.h @@ -43,7 +43,7 @@ int siw_get_port_immutable(struct ib_device *base_dev, u32 port, int siw_query_device(struct ib_device *base_dev, struct ib_device_attr *attr, struct ib_udata *udata); int siw_create_cq(struct ib_cq *base_cq, const struct ib_cq_init_attr *attr, - struct ib_udata *udata); + struct ib_udata *udata, struct uverbs_attr_bundle *attrs); int siw_query_port(struct ib_device *base_dev, u32 port, struct ib_port_attr *attr); int siw_query_gid(struct ib_device *base_dev, u32 port, int idx, diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index c20571618798..432a81c8c7b7 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -2463,7 +2463,7 @@ struct ib_device_ops { int qp_attr_mask, struct ib_qp_init_attr *qp_init_attr); int (*destroy_qp)(struct ib_qp *qp, struct ib_udata *udata); int (*create_cq)(struct ib_cq *cq, const struct ib_cq_init_attr *attr, - struct ib_udata *udata); + struct uverbs_attr_bundle *attrs); int (*modify_cq)(struct ib_cq *cq, u16 cq_count, u16 cq_period); int (*destroy_cq)(struct ib_cq *cq, struct ib_udata *udata); int (*resize_cq)(struct ib_cq *cq, int cqe, struct ib_udata *udata); diff --git a/include/uapi/rdma/ib_user_ioctl_cmds.h b/include/uapi/rdma/ib_user_ioctl_cmds.h index dafc7ebe545b..bfb34c23cf1c 100644 --- a/include/uapi/rdma/ib_user_ioctl_cmds.h +++ b/include/uapi/rdma/ib_user_ioctl_cmds.h @@ -63,6 +63,7 @@ enum uverbs_default_objects { enum { UVERBS_ATTR_UHW_IN = UVERBS_UDATA_DRIVER_DATA_FLAG, UVERBS_ATTR_UHW_OUT, + UVERBS_ATTR_UHW_DRIVER_DATA, }; enum uverbs_methods_device { -- 2.45.2 ^ permalink raw reply related [flat|nested] 14+ messages in thread
* Re: [PATCH rdma-next 1/2] RDMA: Pass entire uverbs attr bundle to create cq function 2024-06-16 16:15 ` [PATCH rdma-next 1/2] RDMA: Pass entire uverbs attr bundle to create cq function Leon Romanovsky @ 2024-06-17 6:17 ` Leon Romanovsky 2024-06-17 13:44 ` Jason Gunthorpe 2024-06-17 16:00 ` Zhu Yanjun 2 siblings, 0 replies; 14+ messages in thread From: Leon Romanovsky @ 2024-06-17 6:17 UTC (permalink / raw) To: Jason Gunthorpe Cc: Akiva Goldberger, Ajay Sharma, Bernard Metzler, Chengchang Tang, Cheng Xu, Junxian Huang, Kai Shen, linux-rdma, Long Li, Michael Margolin, Mustafa Ismail, Potnuri Bharat Teja, Selvin Xavier, Shiraz Saleem, Yishai Hadas, Zhu Yanjun On Sun, Jun 16, 2024 at 07:15:57PM +0300, Leon Romanovsky wrote: > From: Akiva Goldberger <agoldberger@nvidia.com> > > Changes the create_cq verb signature by sending the entire uverbs attr > bundle as a parameter. This allows drivers to send driver specific attrs > through ioctl for the create_cq verb and access them in their driver > specific code. > > Also adds a new enum value for driver specific ioctl attritbutes for > methods already supporting UHW. > > Signed-off-by: Akiva Goldberger <agoldberger@nvidia.com> > Signed-off-by: Leon Romanovsky <leonro@nvidia.com> > --- > drivers/infiniband/core/uverbs_cmd.c | 2 +- > drivers/infiniband/core/uverbs_std_types_cq.c | 2 +- > drivers/infiniband/hw/bnxt_re/ib_verbs.c | 3 ++- > drivers/infiniband/hw/bnxt_re/ib_verbs.h | 2 +- > drivers/infiniband/hw/cxgb4/cq.c | 3 ++- > drivers/infiniband/hw/cxgb4/iw_cxgb4.h | 2 +- > drivers/infiniband/hw/efa/efa.h | 2 +- > drivers/infiniband/hw/efa/efa_verbs.c | 3 ++- > drivers/infiniband/hw/erdma/erdma_verbs.c | 3 ++- > drivers/infiniband/hw/erdma/erdma_verbs.h | 2 +- > drivers/infiniband/hw/hns/hns_roce_cq.c | 3 ++- > drivers/infiniband/hw/hns/hns_roce_device.h | 2 +- > drivers/infiniband/hw/irdma/verbs.c | 5 +++-- > drivers/infiniband/hw/mana/cq.c | 2 +- > drivers/infiniband/hw/mana/mana_ib.h | 2 +- > drivers/infiniband/hw/mlx4/cq.c | 3 ++- > drivers/infiniband/hw/mlx4/mlx4_ib.h | 2 +- > drivers/infiniband/hw/mlx5/cq.c | 3 ++- > drivers/infiniband/hw/mlx5/mlx5_ib.h | 2 +- > drivers/infiniband/hw/mthca/mthca_provider.c | 3 ++- > drivers/infiniband/sw/rxe/rxe_verbs.c | 3 ++- > drivers/infiniband/sw/siw/siw_verbs.c | 2 +- > drivers/infiniband/sw/siw/siw_verbs.h | 2 +- > include/rdma/ib_verbs.h | 2 +- > include/uapi/rdma/ib_user_ioctl_cmds.h | 1 + > 25 files changed, 36 insertions(+), 25 deletions(-) <...> > --- a/drivers/infiniband/hw/mthca/mthca_provider.c > +++ b/drivers/infiniband/hw/mthca/mthca_provider.c > @@ -574,7 +574,8 @@ static int mthca_destroy_qp(struct ib_qp *qp, struct ib_udata *udata) > > static int mthca_create_cq(struct ib_cq *ibcq, > const struct ib_cq_init_attr *attr, > - struct ib_udata *udata) > + struct ib_udata *udata, > + struct uverbs_attr_bundle *attrs) > { This hunk needs to be fixed with the following patch: diff --git a/drivers/infiniband/hw/mthca/mthca_provider.c b/drivers/infiniband/hw/mthca/mthca_provider.c index 677ebb145dbf..6a1e2e79ddc3 100644 --- a/drivers/infiniband/hw/mthca/mthca_provider.c +++ b/drivers/infiniband/hw/mthca/mthca_provider.c @@ -574,9 +574,9 @@ static int mthca_destroy_qp(struct ib_qp *qp, struct ib_udata *udata) static int mthca_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr, - struct ib_udata *udata, struct uverbs_attr_bundle *attrs) { + struct ib_udata *udata = &attrs->driver_udata; struct ib_device *ibdev = ibcq->device; int entries = attr->cqe; struct mthca_create_cq ucmd; Thanks ^ permalink raw reply related [flat|nested] 14+ messages in thread
* Re: [PATCH rdma-next 1/2] RDMA: Pass entire uverbs attr bundle to create cq function 2024-06-16 16:15 ` [PATCH rdma-next 1/2] RDMA: Pass entire uverbs attr bundle to create cq function Leon Romanovsky 2024-06-17 6:17 ` Leon Romanovsky @ 2024-06-17 13:44 ` Jason Gunthorpe 2024-06-17 15:49 ` Leon Romanovsky 2024-06-17 16:00 ` Zhu Yanjun 2 siblings, 1 reply; 14+ messages in thread From: Jason Gunthorpe @ 2024-06-17 13:44 UTC (permalink / raw) To: Leon Romanovsky Cc: Akiva Goldberger, Ajay Sharma, Bernard Metzler, Chengchang Tang, Cheng Xu, Junxian Huang, Kai Shen, linux-rdma, Long Li, Michael Margolin, Mustafa Ismail, Potnuri Bharat Teja, Selvin Xavier, Shiraz Saleem, Yishai Hadas, Zhu Yanjun On Sun, Jun 16, 2024 at 07:15:57PM +0300, Leon Romanovsky wrote: > @@ -63,6 +63,7 @@ enum uverbs_default_objects { > enum { > UVERBS_ATTR_UHW_IN = UVERBS_UDATA_DRIVER_DATA_FLAG, > UVERBS_ATTR_UHW_OUT, > + UVERBS_ATTR_UHW_DRIVER_DATA, The start of the driver's attributes is not a "UHW", the UHW is only the old structs. Something like UVERBS_ATTR_DRIVER_NS_WITH_UHW Jason ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH rdma-next 1/2] RDMA: Pass entire uverbs attr bundle to create cq function 2024-06-17 13:44 ` Jason Gunthorpe @ 2024-06-17 15:49 ` Leon Romanovsky 2024-06-17 20:10 ` Jason Gunthorpe 0 siblings, 1 reply; 14+ messages in thread From: Leon Romanovsky @ 2024-06-17 15:49 UTC (permalink / raw) To: Jason Gunthorpe Cc: Akiva Goldberger, Ajay Sharma, Bernard Metzler, Chengchang Tang, Cheng Xu, Junxian Huang, Kai Shen, linux-rdma, Long Li, Michael Margolin, Mustafa Ismail, Potnuri Bharat Teja, Selvin Xavier, Shiraz Saleem, Yishai Hadas, Zhu Yanjun On Mon, Jun 17, 2024 at 10:44:09AM -0300, Jason Gunthorpe wrote: > On Sun, Jun 16, 2024 at 07:15:57PM +0300, Leon Romanovsky wrote: > > > @@ -63,6 +63,7 @@ enum uverbs_default_objects { > > enum { > > UVERBS_ATTR_UHW_IN = UVERBS_UDATA_DRIVER_DATA_FLAG, > > UVERBS_ATTR_UHW_OUT, > > + UVERBS_ATTR_UHW_DRIVER_DATA, > > The start of the driver's attributes is not a "UHW", the UHW is only > the old structs. I asked from Akiva to keep existing naming convention UVERBS_ATTR_UHW_XXX to emphasize the namespace and the position of this attribute as relevant for existing UHW calls. > > Something like UVERBS_ATTR_DRIVER_NS_WITH_UHW I think that my proposed name is better. Thanks > > Jason ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH rdma-next 1/2] RDMA: Pass entire uverbs attr bundle to create cq function 2024-06-17 15:49 ` Leon Romanovsky @ 2024-06-17 20:10 ` Jason Gunthorpe 2024-06-18 5:05 ` Leon Romanovsky 0 siblings, 1 reply; 14+ messages in thread From: Jason Gunthorpe @ 2024-06-17 20:10 UTC (permalink / raw) To: Leon Romanovsky Cc: Akiva Goldberger, Ajay Sharma, Bernard Metzler, Chengchang Tang, Cheng Xu, Junxian Huang, Kai Shen, linux-rdma, Long Li, Michael Margolin, Mustafa Ismail, Potnuri Bharat Teja, Selvin Xavier, Shiraz Saleem, Yishai Hadas, Zhu Yanjun On Mon, Jun 17, 2024 at 06:49:47PM +0300, Leon Romanovsky wrote: > On Mon, Jun 17, 2024 at 10:44:09AM -0300, Jason Gunthorpe wrote: > > On Sun, Jun 16, 2024 at 07:15:57PM +0300, Leon Romanovsky wrote: > > > > > @@ -63,6 +63,7 @@ enum uverbs_default_objects { > > > enum { > > > UVERBS_ATTR_UHW_IN = UVERBS_UDATA_DRIVER_DATA_FLAG, > > > UVERBS_ATTR_UHW_OUT, > > > + UVERBS_ATTR_UHW_DRIVER_DATA, > > > > The start of the driver's attributes is not a "UHW", the UHW is only > > the old structs. > > I asked from Akiva to keep existing naming convention UVERBS_ATTR_UHW_XXX > to emphasize the namespace and the position of this attribute as > relevant for existing UHW calls. Well, calling it DRIVER_DATA and UHW is very confusing when it is really the start of the indexing for drivers that use UHW. A better name is needed Jason ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH rdma-next 1/2] RDMA: Pass entire uverbs attr bundle to create cq function 2024-06-17 20:10 ` Jason Gunthorpe @ 2024-06-18 5:05 ` Leon Romanovsky 2024-06-18 13:08 ` Jason Gunthorpe 0 siblings, 1 reply; 14+ messages in thread From: Leon Romanovsky @ 2024-06-18 5:05 UTC (permalink / raw) To: Jason Gunthorpe Cc: Akiva Goldberger, Ajay Sharma, Bernard Metzler, Chengchang Tang, Cheng Xu, Junxian Huang, Kai Shen, linux-rdma, Long Li, Michael Margolin, Mustafa Ismail, Potnuri Bharat Teja, Selvin Xavier, Shiraz Saleem, Yishai Hadas, Zhu Yanjun On Mon, Jun 17, 2024 at 05:10:03PM -0300, Jason Gunthorpe wrote: > On Mon, Jun 17, 2024 at 06:49:47PM +0300, Leon Romanovsky wrote: > > On Mon, Jun 17, 2024 at 10:44:09AM -0300, Jason Gunthorpe wrote: > > > On Sun, Jun 16, 2024 at 07:15:57PM +0300, Leon Romanovsky wrote: > > > > > > > @@ -63,6 +63,7 @@ enum uverbs_default_objects { > > > > enum { > > > > UVERBS_ATTR_UHW_IN = UVERBS_UDATA_DRIVER_DATA_FLAG, > > > > UVERBS_ATTR_UHW_OUT, > > > > + UVERBS_ATTR_UHW_DRIVER_DATA, > > > > > > The start of the driver's attributes is not a "UHW", the UHW is only > > > the old structs. > > > > I asked from Akiva to keep existing naming convention UVERBS_ATTR_UHW_XXX > > to emphasize the namespace and the position of this attribute as > > relevant for existing UHW calls. > > Well, calling it DRIVER_DATA and UHW is very confusing when it is > really the start of the indexing for drivers that use UHW. > > A better name is needed UVERBS_ATTR_UHW_PRIVATE ???? > > Jason > ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH rdma-next 1/2] RDMA: Pass entire uverbs attr bundle to create cq function 2024-06-18 5:05 ` Leon Romanovsky @ 2024-06-18 13:08 ` Jason Gunthorpe 2024-06-18 16:05 ` Leon Romanovsky 0 siblings, 1 reply; 14+ messages in thread From: Jason Gunthorpe @ 2024-06-18 13:08 UTC (permalink / raw) To: Leon Romanovsky Cc: Akiva Goldberger, Ajay Sharma, Bernard Metzler, Chengchang Tang, Cheng Xu, Junxian Huang, Kai Shen, linux-rdma, Long Li, Michael Margolin, Mustafa Ismail, Potnuri Bharat Teja, Selvin Xavier, Shiraz Saleem, Yishai Hadas, Zhu Yanjun On Tue, Jun 18, 2024 at 08:05:57AM +0300, Leon Romanovsky wrote: > On Mon, Jun 17, 2024 at 05:10:03PM -0300, Jason Gunthorpe wrote: > > On Mon, Jun 17, 2024 at 06:49:47PM +0300, Leon Romanovsky wrote: > > > On Mon, Jun 17, 2024 at 10:44:09AM -0300, Jason Gunthorpe wrote: > > > > On Sun, Jun 16, 2024 at 07:15:57PM +0300, Leon Romanovsky wrote: > > > > > > > > > @@ -63,6 +63,7 @@ enum uverbs_default_objects { > > > > > enum { > > > > > UVERBS_ATTR_UHW_IN = UVERBS_UDATA_DRIVER_DATA_FLAG, > > > > > UVERBS_ATTR_UHW_OUT, > > > > > + UVERBS_ATTR_UHW_DRIVER_DATA, > > > > > > > > The start of the driver's attributes is not a "UHW", the UHW is only > > > > the old structs. > > > > > > I asked from Akiva to keep existing naming convention UVERBS_ATTR_UHW_XXX > > > to emphasize the namespace and the position of this attribute as > > > relevant for existing UHW calls. > > > > Well, calling it DRIVER_DATA and UHW is very confusing when it is > > really the start of the indexing for drivers that use UHW. > > > > A better name is needed > > UVERBS_ATTR_UHW_PRIVATE ???? I think it need to have the word "start" in it, because it is the start of numbers, not an actual number itself. It is also not PRIVATE at all, this is just in the device specific space number space, not the core space. Jason ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH rdma-next 1/2] RDMA: Pass entire uverbs attr bundle to create cq function 2024-06-18 13:08 ` Jason Gunthorpe @ 2024-06-18 16:05 ` Leon Romanovsky 2024-06-18 16:13 ` Jason Gunthorpe 0 siblings, 1 reply; 14+ messages in thread From: Leon Romanovsky @ 2024-06-18 16:05 UTC (permalink / raw) To: Jason Gunthorpe Cc: Akiva Goldberger, Ajay Sharma, Bernard Metzler, Chengchang Tang, Cheng Xu, Junxian Huang, Kai Shen, linux-rdma, Long Li, Michael Margolin, Mustafa Ismail, Potnuri Bharat Teja, Selvin Xavier, Shiraz Saleem, Yishai Hadas, Zhu Yanjun On Tue, Jun 18, 2024 at 10:08:54AM -0300, Jason Gunthorpe wrote: > On Tue, Jun 18, 2024 at 08:05:57AM +0300, Leon Romanovsky wrote: > > On Mon, Jun 17, 2024 at 05:10:03PM -0300, Jason Gunthorpe wrote: > > > On Mon, Jun 17, 2024 at 06:49:47PM +0300, Leon Romanovsky wrote: > > > > On Mon, Jun 17, 2024 at 10:44:09AM -0300, Jason Gunthorpe wrote: > > > > > On Sun, Jun 16, 2024 at 07:15:57PM +0300, Leon Romanovsky wrote: > > > > > > > > > > > @@ -63,6 +63,7 @@ enum uverbs_default_objects { > > > > > > enum { > > > > > > UVERBS_ATTR_UHW_IN = UVERBS_UDATA_DRIVER_DATA_FLAG, > > > > > > UVERBS_ATTR_UHW_OUT, > > > > > > + UVERBS_ATTR_UHW_DRIVER_DATA, > > > > > > > > > > The start of the driver's attributes is not a "UHW", the UHW is only > > > > > the old structs. > > > > > > > > I asked from Akiva to keep existing naming convention UVERBS_ATTR_UHW_XXX > > > > to emphasize the namespace and the position of this attribute as > > > > relevant for existing UHW calls. > > > > > > Well, calling it DRIVER_DATA and UHW is very confusing when it is > > > really the start of the indexing for drivers that use UHW. > > > > > > A better name is needed > > > > UVERBS_ATTR_UHW_PRIVATE ???? > > I think it need to have the word "start" in it, because it is the > start of numbers, not an actual number itself. UVERBS_ATTR_UHW_DRIVER_DATA_START ???? What do you suggest instead? > > It is also not PRIVATE at all, this is just in the device specific > space number space, not the core space. Private in the sense of driver specific, like net_priv(). > > Jason ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH rdma-next 1/2] RDMA: Pass entire uverbs attr bundle to create cq function 2024-06-18 16:05 ` Leon Romanovsky @ 2024-06-18 16:13 ` Jason Gunthorpe 2024-06-19 8:57 ` Leon Romanovsky 0 siblings, 1 reply; 14+ messages in thread From: Jason Gunthorpe @ 2024-06-18 16:13 UTC (permalink / raw) To: Leon Romanovsky Cc: Akiva Goldberger, Ajay Sharma, Bernard Metzler, Chengchang Tang, Cheng Xu, Junxian Huang, Kai Shen, linux-rdma, Long Li, Michael Margolin, Mustafa Ismail, Potnuri Bharat Teja, Selvin Xavier, Shiraz Saleem, Yishai Hadas, Zhu Yanjun On Tue, Jun 18, 2024 at 07:05:59PM +0300, Leon Romanovsky wrote: > On Tue, Jun 18, 2024 at 10:08:54AM -0300, Jason Gunthorpe wrote: > > On Tue, Jun 18, 2024 at 08:05:57AM +0300, Leon Romanovsky wrote: > > > On Mon, Jun 17, 2024 at 05:10:03PM -0300, Jason Gunthorpe wrote: > > > > On Mon, Jun 17, 2024 at 06:49:47PM +0300, Leon Romanovsky wrote: > > > > > On Mon, Jun 17, 2024 at 10:44:09AM -0300, Jason Gunthorpe wrote: > > > > > > On Sun, Jun 16, 2024 at 07:15:57PM +0300, Leon Romanovsky wrote: > > > > > > > > > > > > > @@ -63,6 +63,7 @@ enum uverbs_default_objects { > > > > > > > enum { > > > > > > > UVERBS_ATTR_UHW_IN = UVERBS_UDATA_DRIVER_DATA_FLAG, > > > > > > > UVERBS_ATTR_UHW_OUT, > > > > > > > + UVERBS_ATTR_UHW_DRIVER_DATA, > > > > > > > > > > > > The start of the driver's attributes is not a "UHW", the UHW is only > > > > > > the old structs. > > > > > > > > > > I asked from Akiva to keep existing naming convention UVERBS_ATTR_UHW_XXX > > > > > to emphasize the namespace and the position of this attribute as > > > > > relevant for existing UHW calls. > > > > > > > > Well, calling it DRIVER_DATA and UHW is very confusing when it is > > > > really the start of the indexing for drivers that use UHW. > > > > > > > > A better name is needed > > > > > > UVERBS_ATTR_UHW_PRIVATE ???? > > > > I think it need to have the word "start" in it, because it is the > > start of numbers, not an actual number itself. > > UVERBS_ATTR_UHW_DRIVER_DATA_START ???? > What do you suggest instead? How about: diff --git a/include/uapi/rdma/ib_user_ioctl_cmds.h b/include/uapi/rdma/ib_user_ioctl_cmds.h index dafc7ebe545b8d..e9322f66cd2dec 100644 --- a/include/uapi/rdma/ib_user_ioctl_cmds.h +++ b/include/uapi/rdma/ib_user_ioctl_cmds.h @@ -37,9 +37,6 @@ #define UVERBS_ID_NS_MASK 0xF000 #define UVERBS_ID_NS_SHIFT 12 -#define UVERBS_UDATA_DRIVER_DATA_NS 1 -#define UVERBS_UDATA_DRIVER_DATA_FLAG (1UL << UVERBS_ID_NS_SHIFT) - enum uverbs_default_objects { UVERBS_OBJECT_DEVICE, /* No instances of DEVICE are allowed */ UVERBS_OBJECT_PD, @@ -61,8 +58,10 @@ enum uverbs_default_objects { }; enum { - UVERBS_ATTR_UHW_IN = UVERBS_UDATA_DRIVER_DATA_FLAG, + UVERBS_ID_DRIVER_NS = 1U << UVERBS_ID_NS_SHIFT, + UVERBS_ATTR_UHW_IN = UVERBS_ID_DRIVER_NS, UVERBS_ATTR_UHW_OUT, + UVERBS_ID_DRIVER_NS_WITH_UHW, }; enum uverbs_methods_device { And recommend replacing the open coded UVERBS_ID_DRIVER_NS all over the place. > > It is also not PRIVATE at all, this is just in the device specific > > space number space, not the core space. > > Private in the sense of driver specific, like net_priv(). It is not a private, it is a namespace, that is the naming that was used here. Jason ^ permalink raw reply related [flat|nested] 14+ messages in thread
* Re: [PATCH rdma-next 1/2] RDMA: Pass entire uverbs attr bundle to create cq function 2024-06-18 16:13 ` Jason Gunthorpe @ 2024-06-19 8:57 ` Leon Romanovsky 0 siblings, 0 replies; 14+ messages in thread From: Leon Romanovsky @ 2024-06-19 8:57 UTC (permalink / raw) To: Jason Gunthorpe Cc: Akiva Goldberger, Ajay Sharma, Bernard Metzler, Chengchang Tang, Cheng Xu, Junxian Huang, Kai Shen, linux-rdma, Long Li, Michael Margolin, Mustafa Ismail, Potnuri Bharat Teja, Selvin Xavier, Shiraz Saleem, Yishai Hadas, Zhu Yanjun On Tue, Jun 18, 2024 at 01:13:45PM -0300, Jason Gunthorpe wrote: > On Tue, Jun 18, 2024 at 07:05:59PM +0300, Leon Romanovsky wrote: > > On Tue, Jun 18, 2024 at 10:08:54AM -0300, Jason Gunthorpe wrote: > > > On Tue, Jun 18, 2024 at 08:05:57AM +0300, Leon Romanovsky wrote: > > > > On Mon, Jun 17, 2024 at 05:10:03PM -0300, Jason Gunthorpe wrote: > > > > > On Mon, Jun 17, 2024 at 06:49:47PM +0300, Leon Romanovsky wrote: > > > > > > On Mon, Jun 17, 2024 at 10:44:09AM -0300, Jason Gunthorpe wrote: > > > > > > > On Sun, Jun 16, 2024 at 07:15:57PM +0300, Leon Romanovsky wrote: > > > > > > > > > > > > > > > @@ -63,6 +63,7 @@ enum uverbs_default_objects { > > > > > > > > enum { > > > > > > > > UVERBS_ATTR_UHW_IN = UVERBS_UDATA_DRIVER_DATA_FLAG, > > > > > > > > UVERBS_ATTR_UHW_OUT, > > > > > > > > + UVERBS_ATTR_UHW_DRIVER_DATA, > > > > > > > > > > > > > > The start of the driver's attributes is not a "UHW", the UHW is only > > > > > > > the old structs. > > > > > > > > > > > > I asked from Akiva to keep existing naming convention UVERBS_ATTR_UHW_XXX > > > > > > to emphasize the namespace and the position of this attribute as > > > > > > relevant for existing UHW calls. > > > > > > > > > > Well, calling it DRIVER_DATA and UHW is very confusing when it is > > > > > really the start of the indexing for drivers that use UHW. > > > > > > > > > > A better name is needed > > > > > > > > UVERBS_ATTR_UHW_PRIVATE ???? > > > > > > I think it need to have the word "start" in it, because it is the > > > start of numbers, not an actual number itself. > > > > UVERBS_ATTR_UHW_DRIVER_DATA_START ???? > > What do you suggest instead? > > How about: ok, let's take this variant. Thanks > > diff --git a/include/uapi/rdma/ib_user_ioctl_cmds.h b/include/uapi/rdma/ib_user_ioctl_cmds.h > index dafc7ebe545b8d..e9322f66cd2dec 100644 > --- a/include/uapi/rdma/ib_user_ioctl_cmds.h > +++ b/include/uapi/rdma/ib_user_ioctl_cmds.h > @@ -37,9 +37,6 @@ > #define UVERBS_ID_NS_MASK 0xF000 > #define UVERBS_ID_NS_SHIFT 12 > > -#define UVERBS_UDATA_DRIVER_DATA_NS 1 > -#define UVERBS_UDATA_DRIVER_DATA_FLAG (1UL << UVERBS_ID_NS_SHIFT) > - > enum uverbs_default_objects { > UVERBS_OBJECT_DEVICE, /* No instances of DEVICE are allowed */ > UVERBS_OBJECT_PD, > @@ -61,8 +58,10 @@ enum uverbs_default_objects { > }; > > enum { > - UVERBS_ATTR_UHW_IN = UVERBS_UDATA_DRIVER_DATA_FLAG, > + UVERBS_ID_DRIVER_NS = 1U << UVERBS_ID_NS_SHIFT, > + UVERBS_ATTR_UHW_IN = UVERBS_ID_DRIVER_NS, > UVERBS_ATTR_UHW_OUT, > + UVERBS_ID_DRIVER_NS_WITH_UHW, > }; > > enum uverbs_methods_device { > > And recommend replacing the open coded UVERBS_ID_DRIVER_NS all over > the place. > > > > It is also not PRIVATE at all, this is just in the device specific > > > space number space, not the core space. > > > > Private in the sense of driver specific, like net_priv(). > > It is not a private, it is a namespace, that is the naming that was > used here. > > Jason ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH rdma-next 1/2] RDMA: Pass entire uverbs attr bundle to create cq function 2024-06-16 16:15 ` [PATCH rdma-next 1/2] RDMA: Pass entire uverbs attr bundle to create cq function Leon Romanovsky 2024-06-17 6:17 ` Leon Romanovsky 2024-06-17 13:44 ` Jason Gunthorpe @ 2024-06-17 16:00 ` Zhu Yanjun 2 siblings, 0 replies; 14+ messages in thread From: Zhu Yanjun @ 2024-06-17 16:00 UTC (permalink / raw) To: Leon Romanovsky, Jason Gunthorpe Cc: Akiva Goldberger, Ajay Sharma, Bernard Metzler, Chengchang Tang, Cheng Xu, Junxian Huang, Kai Shen, linux-rdma, Long Li, Michael Margolin, Mustafa Ismail, Potnuri Bharat Teja, Selvin Xavier, Shiraz Saleem, Yishai Hadas, Zhu Yanjun 在 2024/6/17 0:15, Leon Romanovsky 写道: > From: Akiva Goldberger <agoldberger@nvidia.com> > > Changes the create_cq verb signature by sending the entire uverbs attr > bundle as a parameter. This allows drivers to send driver specific attrs > through ioctl for the create_cq verb and access them in their driver > specific code. > > Also adds a new enum value for driver specific ioctl attritbutes for > methods already supporting UHW. > > Signed-off-by: Akiva Goldberger <agoldberger@nvidia.com> > Signed-off-by: Leon Romanovsky <leonro@nvidia.com> > --- > drivers/infiniband/core/uverbs_cmd.c | 2 +- > drivers/infiniband/core/uverbs_std_types_cq.c | 2 +- > drivers/infiniband/hw/bnxt_re/ib_verbs.c | 3 ++- > drivers/infiniband/hw/bnxt_re/ib_verbs.h | 2 +- > drivers/infiniband/hw/cxgb4/cq.c | 3 ++- > drivers/infiniband/hw/cxgb4/iw_cxgb4.h | 2 +- > drivers/infiniband/hw/efa/efa.h | 2 +- > drivers/infiniband/hw/efa/efa_verbs.c | 3 ++- > drivers/infiniband/hw/erdma/erdma_verbs.c | 3 ++- > drivers/infiniband/hw/erdma/erdma_verbs.h | 2 +- > drivers/infiniband/hw/hns/hns_roce_cq.c | 3 ++- > drivers/infiniband/hw/hns/hns_roce_device.h | 2 +- > drivers/infiniband/hw/irdma/verbs.c | 5 +++-- > drivers/infiniband/hw/mana/cq.c | 2 +- > drivers/infiniband/hw/mana/mana_ib.h | 2 +- > drivers/infiniband/hw/mlx4/cq.c | 3 ++- > drivers/infiniband/hw/mlx4/mlx4_ib.h | 2 +- > drivers/infiniband/hw/mlx5/cq.c | 3 ++- > drivers/infiniband/hw/mlx5/mlx5_ib.h | 2 +- > drivers/infiniband/hw/mthca/mthca_provider.c | 3 ++- > drivers/infiniband/sw/rxe/rxe_verbs.c | 3 ++- > drivers/infiniband/sw/siw/siw_verbs.c | 2 +- > drivers/infiniband/sw/siw/siw_verbs.h | 2 +- > include/rdma/ib_verbs.h | 2 +- > include/uapi/rdma/ib_user_ioctl_cmds.h | 1 + > 25 files changed, 36 insertions(+), 25 deletions(-) > > diff --git a/drivers/infiniband/core/uverbs_cmd.c b/drivers/infiniband/core/uverbs_cmd.c > index 3d3ee3eca983..1b3ea71f2c33 100644 > --- a/drivers/infiniband/core/uverbs_cmd.c > +++ b/drivers/infiniband/core/uverbs_cmd.c > @@ -1051,7 +1051,7 @@ static int create_cq(struct uverbs_attr_bundle *attrs, > rdma_restrack_new(&cq->res, RDMA_RESTRACK_CQ); > rdma_restrack_set_name(&cq->res, NULL); > > - ret = ib_dev->ops.create_cq(cq, &attr, &attrs->driver_udata); > + ret = ib_dev->ops.create_cq(cq, &attr, attrs); > if (ret) > goto err_free; > rdma_restrack_add(&cq->res); > diff --git a/drivers/infiniband/core/uverbs_std_types_cq.c b/drivers/infiniband/core/uverbs_std_types_cq.c > index 370ad7c83f88..432054f0a8a4 100644 > --- a/drivers/infiniband/core/uverbs_std_types_cq.c > +++ b/drivers/infiniband/core/uverbs_std_types_cq.c > @@ -128,7 +128,7 @@ static int UVERBS_HANDLER(UVERBS_METHOD_CQ_CREATE)( > rdma_restrack_new(&cq->res, RDMA_RESTRACK_CQ); > rdma_restrack_set_name(&cq->res, NULL); > > - ret = ib_dev->ops.create_cq(cq, &attr, &attrs->driver_udata); > + ret = ib_dev->ops.create_cq(cq, &attr, attrs); > if (ret) > goto err_free; > > diff --git a/drivers/infiniband/hw/bnxt_re/ib_verbs.c b/drivers/infiniband/hw/bnxt_re/ib_verbs.c > index d261b09025ca..e453ca701e87 100644 > --- a/drivers/infiniband/hw/bnxt_re/ib_verbs.c > +++ b/drivers/infiniband/hw/bnxt_re/ib_verbs.c > @@ -2948,10 +2948,11 @@ int bnxt_re_destroy_cq(struct ib_cq *ib_cq, struct ib_udata *udata) > } > > int bnxt_re_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr, > - struct ib_udata *udata) > + struct uverbs_attr_bundle *attrs) > { > struct bnxt_re_cq *cq = container_of(ibcq, struct bnxt_re_cq, ib_cq); > struct bnxt_re_dev *rdev = to_bnxt_re_dev(ibcq->device, ibdev); > + struct ib_udata *udata = &attrs->driver_udata; > struct bnxt_re_ucontext *uctx = > rdma_udata_to_drv_context(udata, struct bnxt_re_ucontext, ib_uctx); > struct bnxt_qplib_dev_attr *dev_attr = &rdev->dev_attr; > diff --git a/drivers/infiniband/hw/bnxt_re/ib_verbs.h b/drivers/infiniband/hw/bnxt_re/ib_verbs.h > index b267d6d5975f..e98cb1717338 100644 > --- a/drivers/infiniband/hw/bnxt_re/ib_verbs.h > +++ b/drivers/infiniband/hw/bnxt_re/ib_verbs.h > @@ -221,7 +221,7 @@ int bnxt_re_post_send(struct ib_qp *qp, const struct ib_send_wr *send_wr, > int bnxt_re_post_recv(struct ib_qp *qp, const struct ib_recv_wr *recv_wr, > const struct ib_recv_wr **bad_recv_wr); > int bnxt_re_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr, > - struct ib_udata *udata); > + struct uverbs_attr_bundle *attrs); > int bnxt_re_resize_cq(struct ib_cq *ibcq, int cqe, struct ib_udata *udata); > int bnxt_re_destroy_cq(struct ib_cq *cq, struct ib_udata *udata); > int bnxt_re_poll_cq(struct ib_cq *cq, int num_entries, struct ib_wc *wc); > diff --git a/drivers/infiniband/hw/cxgb4/cq.c b/drivers/infiniband/hw/cxgb4/cq.c > index 7e2835dcbc1c..5111421f9473 100644 > --- a/drivers/infiniband/hw/cxgb4/cq.c > +++ b/drivers/infiniband/hw/cxgb4/cq.c > @@ -995,8 +995,9 @@ int c4iw_destroy_cq(struct ib_cq *ib_cq, struct ib_udata *udata) > } > > int c4iw_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr, > - struct ib_udata *udata) > + struct uverbs_attr_bundle *attrs) > { > + struct ib_udata *udata = &attrs->driver_udata; > struct ib_device *ibdev = ibcq->device; > int entries = attr->cqe; > int vector = attr->comp_vector; > diff --git a/drivers/infiniband/hw/cxgb4/iw_cxgb4.h b/drivers/infiniband/hw/cxgb4/iw_cxgb4.h > index fb8a0c248866..f838bb6718af 100644 > --- a/drivers/infiniband/hw/cxgb4/iw_cxgb4.h > +++ b/drivers/infiniband/hw/cxgb4/iw_cxgb4.h > @@ -978,7 +978,7 @@ int c4iw_dereg_mr(struct ib_mr *ib_mr, struct ib_udata *udata); > int c4iw_destroy_cq(struct ib_cq *ib_cq, struct ib_udata *udata); > void c4iw_cq_rem_ref(struct c4iw_cq *chp); > int c4iw_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr, > - struct ib_udata *udata); > + struct uverbs_attr_bundle *attrs); > int c4iw_arm_cq(struct ib_cq *ibcq, enum ib_cq_notify_flags flags); > int c4iw_modify_srq(struct ib_srq *ib_srq, struct ib_srq_attr *attr, > enum ib_srq_attr_mask srq_attr_mask, > diff --git a/drivers/infiniband/hw/efa/efa.h b/drivers/infiniband/hw/efa/efa.h > index 926f9ff1f60f..e580e087e9da 100644 > --- a/drivers/infiniband/hw/efa/efa.h > +++ b/drivers/infiniband/hw/efa/efa.h > @@ -161,7 +161,7 @@ int efa_create_qp(struct ib_qp *ibqp, struct ib_qp_init_attr *init_attr, > struct ib_udata *udata); > int efa_destroy_cq(struct ib_cq *ibcq, struct ib_udata *udata); > int efa_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr, > - struct ib_udata *udata); > + struct uverbs_attr_bundle *attrs); > struct ib_mr *efa_reg_mr(struct ib_pd *ibpd, u64 start, u64 length, > u64 virt_addr, int access_flags, > struct ib_udata *udata); > diff --git a/drivers/infiniband/hw/efa/efa_verbs.c b/drivers/infiniband/hw/efa/efa_verbs.c > index 8f7a13b79cdc..9ced560d7f42 100644 > --- a/drivers/infiniband/hw/efa/efa_verbs.c > +++ b/drivers/infiniband/hw/efa/efa_verbs.c > @@ -1084,8 +1084,9 @@ static int cq_mmap_entries_setup(struct efa_dev *dev, struct efa_cq *cq, > } > > int efa_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr, > - struct ib_udata *udata) > + struct uverbs_attr_bundle *attrs) > { > + struct ib_udata *udata = &attrs->driver_udata; > struct efa_ucontext *ucontext = rdma_udata_to_drv_context( > udata, struct efa_ucontext, ibucontext); > struct efa_com_create_cq_params params = {}; > diff --git a/drivers/infiniband/hw/erdma/erdma_verbs.c b/drivers/infiniband/hw/erdma/erdma_verbs.c > index 40c9b6e46b82..d7e1cbf9f5c2 100644 > --- a/drivers/infiniband/hw/erdma/erdma_verbs.c > +++ b/drivers/infiniband/hw/erdma/erdma_verbs.c > @@ -1628,8 +1628,9 @@ static int erdma_init_kernel_cq(struct erdma_cq *cq) > } > > int erdma_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr, > - struct ib_udata *udata) > + struct uverbs_attr_bundle *attrs) > { > + struct ib_udata *udata = &attrs->driver_udata; > struct erdma_cq *cq = to_ecq(ibcq); > struct erdma_dev *dev = to_edev(ibcq->device); > unsigned int depth = attr->cqe; > diff --git a/drivers/infiniband/hw/erdma/erdma_verbs.h b/drivers/infiniband/hw/erdma/erdma_verbs.h > index 4f02ba06b210..6afdc02f5869 100644 > --- a/drivers/infiniband/hw/erdma/erdma_verbs.h > +++ b/drivers/infiniband/hw/erdma/erdma_verbs.h > @@ -329,7 +329,7 @@ int erdma_query_device(struct ib_device *dev, struct ib_device_attr *attr, > int erdma_get_port_immutable(struct ib_device *dev, u32 port, > struct ib_port_immutable *ib_port_immutable); > int erdma_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr, > - struct ib_udata *data); > + struct uverbs_attr_bundle *attrs); > int erdma_query_port(struct ib_device *dev, u32 port, > struct ib_port_attr *attr); > int erdma_query_gid(struct ib_device *dev, u32 port, int idx, > diff --git a/drivers/infiniband/hw/hns/hns_roce_cq.c b/drivers/infiniband/hw/hns/hns_roce_cq.c > index 56dc3908da2f..4ec66611a143 100644 > --- a/drivers/infiniband/hw/hns/hns_roce_cq.c > +++ b/drivers/infiniband/hw/hns/hns_roce_cq.c > @@ -353,9 +353,10 @@ static int set_cqe_size(struct hns_roce_cq *hr_cq, struct ib_udata *udata, > } > > int hns_roce_create_cq(struct ib_cq *ib_cq, const struct ib_cq_init_attr *attr, > - struct ib_udata *udata) > + struct uverbs_attr_bundle *attrs) > { > struct hns_roce_dev *hr_dev = to_hr_dev(ib_cq->device); > + struct ib_udata *udata = &attrs->driver_udata; > struct hns_roce_ib_create_cq_resp resp = {}; > struct hns_roce_cq *hr_cq = to_hr_cq(ib_cq); > struct ib_device *ibdev = &hr_dev->ib_dev; > diff --git a/drivers/infiniband/hw/hns/hns_roce_device.h b/drivers/infiniband/hw/hns/hns_roce_device.h > index ff0b3f68ee3a..ef50cd03f489 100644 > --- a/drivers/infiniband/hw/hns/hns_roce_device.h > +++ b/drivers/infiniband/hw/hns/hns_roce_device.h > @@ -1267,7 +1267,7 @@ __be32 send_ieth(const struct ib_send_wr *wr); > int to_hr_qp_type(int qp_type); > > int hns_roce_create_cq(struct ib_cq *ib_cq, const struct ib_cq_init_attr *attr, > - struct ib_udata *udata); > + struct uverbs_attr_bundle *attrs); > > int hns_roce_destroy_cq(struct ib_cq *ib_cq, struct ib_udata *udata); > int hns_roce_db_map_user(struct hns_roce_ucontext *context, unsigned long virt, > diff --git a/drivers/infiniband/hw/irdma/verbs.c b/drivers/infiniband/hw/irdma/verbs.c > index 12704efb7b19..fc0ce35da14e 100644 > --- a/drivers/infiniband/hw/irdma/verbs.c > +++ b/drivers/infiniband/hw/irdma/verbs.c > @@ -2035,14 +2035,15 @@ static inline int cq_validate_flags(u32 flags, u8 hw_rev) > * irdma_create_cq - create cq > * @ibcq: CQ allocated > * @attr: attributes for cq > - * @udata: user data > + * @attrs: uverbs attribute bundle > */ > static int irdma_create_cq(struct ib_cq *ibcq, > const struct ib_cq_init_attr *attr, > - struct ib_udata *udata) > + struct uverbs_attr_bundle *attrs) > { > #define IRDMA_CREATE_CQ_MIN_REQ_LEN offsetofend(struct irdma_create_cq_req, user_cq_buf) > #define IRDMA_CREATE_CQ_MIN_RESP_LEN offsetofend(struct irdma_create_cq_resp, cq_size) > + struct ib_udata *udata = &attrs->driver_udata; > struct ib_device *ibdev = ibcq->device; > struct irdma_device *iwdev = to_iwdev(ibdev); > struct irdma_pci_f *rf = iwdev->rf; > diff --git a/drivers/infiniband/hw/mana/cq.c b/drivers/infiniband/hw/mana/cq.c > index c6a3fd57a196..4bf95df30eb6 100644 > --- a/drivers/infiniband/hw/mana/cq.c > +++ b/drivers/infiniband/hw/mana/cq.c > @@ -6,7 +6,7 @@ > #include "mana_ib.h" > > int mana_ib_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr, > - struct ib_udata *udata) > + struct ib_udata *udata, struct uverbs_attr_bundle *attrs) > { > struct mana_ib_cq *cq = container_of(ibcq, struct mana_ib_cq, ibcq); > struct mana_ib_create_cq_resp resp = {}; > diff --git a/drivers/infiniband/hw/mana/mana_ib.h b/drivers/infiniband/hw/mana/mana_ib.h > index 977da9569701..522e2d79eae8 100644 > --- a/drivers/infiniband/hw/mana/mana_ib.h > +++ b/drivers/infiniband/hw/mana/mana_ib.h > @@ -429,7 +429,7 @@ void mana_ib_uncfg_vport(struct mana_ib_dev *dev, struct mana_ib_pd *pd, > u32 port); > > int mana_ib_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr, > - struct ib_udata *udata); > + struct ib_udata *udata, struct uverbs_attr_bundle *attrs); > > int mana_ib_destroy_cq(struct ib_cq *ibcq, struct ib_udata *udata); > > diff --git a/drivers/infiniband/hw/mlx4/cq.c b/drivers/infiniband/hw/mlx4/cq.c > index 4cd738aae53c..aa9ea6ba26e5 100644 > --- a/drivers/infiniband/hw/mlx4/cq.c > +++ b/drivers/infiniband/hw/mlx4/cq.c > @@ -172,8 +172,9 @@ static int mlx4_ib_get_cq_umem(struct mlx4_ib_dev *dev, > > #define CQ_CREATE_FLAGS_SUPPORTED IB_UVERBS_CQ_FLAGS_TIMESTAMP_COMPLETION > int mlx4_ib_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr, > - struct ib_udata *udata) > + struct uverbs_attr_bundle *attrs) > { > + struct ib_udata *udata = &attrs->driver_udata; > struct ib_device *ibdev = ibcq->device; > int entries = attr->cqe; > int vector = attr->comp_vector; > diff --git a/drivers/infiniband/hw/mlx4/mlx4_ib.h b/drivers/infiniband/hw/mlx4/mlx4_ib.h > index 41ca1114a995..b52bceff7d97 100644 > --- a/drivers/infiniband/hw/mlx4/mlx4_ib.h > +++ b/drivers/infiniband/hw/mlx4/mlx4_ib.h > @@ -767,7 +767,7 @@ int mlx4_ib_map_mr_sg(struct ib_mr *ibmr, struct scatterlist *sg, int sg_nents, > int mlx4_ib_modify_cq(struct ib_cq *cq, u16 cq_count, u16 cq_period); > int mlx4_ib_resize_cq(struct ib_cq *ibcq, int entries, struct ib_udata *udata); > int mlx4_ib_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr, > - struct ib_udata *udata); > + struct uverbs_attr_bundle *attrs); > int mlx4_ib_destroy_cq(struct ib_cq *cq, struct ib_udata *udata); > int mlx4_ib_poll_cq(struct ib_cq *ibcq, int num_entries, struct ib_wc *wc); > int mlx4_ib_arm_cq(struct ib_cq *cq, enum ib_cq_notify_flags flags); > diff --git a/drivers/infiniband/hw/mlx5/cq.c b/drivers/infiniband/hw/mlx5/cq.c > index 4429bf7c746b..172f3987fc87 100644 > --- a/drivers/infiniband/hw/mlx5/cq.c > +++ b/drivers/infiniband/hw/mlx5/cq.c > @@ -942,8 +942,9 @@ static void notify_soft_wc_handler(struct work_struct *work) > } > > int mlx5_ib_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr, > - struct ib_udata *udata) > + struct uverbs_attr_bundle *attrs) > { > + struct ib_udata *udata = &attrs->driver_udata; > struct ib_device *ibdev = ibcq->device; > int entries = attr->cqe; > int vector = attr->comp_vector; > diff --git a/drivers/infiniband/hw/mlx5/mlx5_ib.h b/drivers/infiniband/hw/mlx5/mlx5_ib.h > index bf25ddb17bce..2b03e607561e 100644 > --- a/drivers/infiniband/hw/mlx5/mlx5_ib.h > +++ b/drivers/infiniband/hw/mlx5/mlx5_ib.h > @@ -1332,7 +1332,7 @@ int mlx5_ib_read_wqe_rq(struct mlx5_ib_qp *qp, int wqe_index, void *buffer, > int mlx5_ib_read_wqe_srq(struct mlx5_ib_srq *srq, int wqe_index, void *buffer, > size_t buflen, size_t *bc); > int mlx5_ib_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr, > - struct ib_udata *udata); > + struct uverbs_attr_bundle *attrs); > int mlx5_ib_destroy_cq(struct ib_cq *cq, struct ib_udata *udata); > int mlx5_ib_poll_cq(struct ib_cq *ibcq, int num_entries, struct ib_wc *wc); > int mlx5_ib_arm_cq(struct ib_cq *ibcq, enum ib_cq_notify_flags flags); > diff --git a/drivers/infiniband/hw/mthca/mthca_provider.c b/drivers/infiniband/hw/mthca/mthca_provider.c > index e1325f2927d6..677ebb145dbf 100644 > --- a/drivers/infiniband/hw/mthca/mthca_provider.c > +++ b/drivers/infiniband/hw/mthca/mthca_provider.c > @@ -574,7 +574,8 @@ static int mthca_destroy_qp(struct ib_qp *qp, struct ib_udata *udata) > > static int mthca_create_cq(struct ib_cq *ibcq, > const struct ib_cq_init_attr *attr, > - struct ib_udata *udata) > + struct ib_udata *udata, > + struct uverbs_attr_bundle *attrs) > { > struct ib_device *ibdev = ibcq->device; > int entries = attr->cqe; > diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.c b/drivers/infiniband/sw/rxe/rxe_verbs.c > index c7d4d8ab5a09..82bb7f33290c 100644 > --- a/drivers/infiniband/sw/rxe/rxe_verbs.c > +++ b/drivers/infiniband/sw/rxe/rxe_verbs.c > @@ -1053,8 +1053,9 @@ static int rxe_post_recv(struct ib_qp *ibqp, const struct ib_recv_wr *wr, > > /* cq */ > static int rxe_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr, > - struct ib_udata *udata) > + struct uverbs_attr_bundle *attrs) > { > + struct ib_udata *udata = &attrs->driver_udata; > struct ib_device *dev = ibcq->device; > struct rxe_dev *rxe = to_rdev(dev); > struct rxe_cq *cq = to_rcq(ibcq); Reviewed-by: Zhu Yanjun <yanjun.zhu@linux.dev> Zhu Yanjun ^ permalink raw reply [flat|nested] 14+ messages in thread
* [PATCH rdma-next 2/2] RDMA/mlx5: Send UAR page index as ioctl attribute 2024-06-16 16:15 [PATCH rdma-next 0/2] Extend mlx5 CQ creation with large UAR page index Leon Romanovsky 2024-06-16 16:15 ` [PATCH rdma-next 1/2] RDMA: Pass entire uverbs attr bundle to create cq function Leon Romanovsky @ 2024-06-16 16:15 ` Leon Romanovsky 2024-06-16 21:47 ` [PATCH rdma-next 0/2] Extend mlx5 CQ creation with large UAR page index Zhu Yanjun 2 siblings, 0 replies; 14+ messages in thread From: Leon Romanovsky @ 2024-06-16 16:15 UTC (permalink / raw) To: Jason Gunthorpe Cc: Akiva Goldberger, Ajay Sharma, Bernard Metzler, Chengchang Tang, Cheng Xu, Junxian Huang, Kai Shen, linux-rdma, Long Li, Michael Margolin, Mustafa Ismail, Potnuri Bharat Teja, Selvin Xavier, Shiraz Saleem, Yishai Hadas, Zhu Yanjun From: Akiva Goldberger <agoldberger@nvidia.com> Add UAR page index as a driver ioctl attribute to increase the number of supported indices, previously limited to 16 bits by mlx5_ib_create_cq struct. Signed-off-by: Akiva Goldberger <agoldberger@nvidia.com> Signed-off-by: Leon Romanovsky <leonro@nvidia.com> --- drivers/infiniband/hw/mlx5/cq.c | 28 +++++++++++++++++++++--- drivers/infiniband/hw/mlx5/main.c | 1 + drivers/infiniband/hw/mlx5/mlx5_ib.h | 1 + include/uapi/rdma/mlx5_user_ioctl_cmds.h | 4 ++++ 4 files changed, 31 insertions(+), 3 deletions(-) diff --git a/drivers/infiniband/hw/mlx5/cq.c b/drivers/infiniband/hw/mlx5/cq.c index 172f3987fc87..74fc78ee147a 100644 --- a/drivers/infiniband/hw/mlx5/cq.c +++ b/drivers/infiniband/hw/mlx5/cq.c @@ -38,6 +38,9 @@ #include "srq.h" #include "qp.h" +#define UVERBS_MODULE_NAME mlx5_ib +#include <rdma/uverbs_named_ioctl.h> + static void mlx5_ib_cq_comp(struct mlx5_core_cq *cq, struct mlx5_eqe *eqe) { struct ib_cq *ibcq = &to_mibcq(cq)->ibcq; @@ -714,7 +717,8 @@ static int mini_cqe_res_format_to_hw(struct mlx5_ib_dev *dev, u8 format) static int create_cq_user(struct mlx5_ib_dev *dev, struct ib_udata *udata, struct mlx5_ib_cq *cq, int entries, u32 **cqb, - int *cqe_size, int *index, int *inlen) + int *cqe_size, int *index, int *inlen, + struct uverbs_attr_bundle *attrs) { struct mlx5_ib_create_cq ucmd = {}; unsigned long page_size; @@ -788,7 +792,11 @@ static int create_cq_user(struct mlx5_ib_dev *dev, struct ib_udata *udata, order_base_2(page_size) - MLX5_ADAPTER_PAGE_SHIFT); MLX5_SET(cqc, cqc, page_offset, page_offset_quantized); - if (ucmd.flags & MLX5_IB_CREATE_CQ_FLAGS_UAR_PAGE_INDEX) { + if (uverbs_attr_is_valid(attrs, MLX5_IB_ATTR_CREATE_CQ_UAR_INDEX)) { + err = uverbs_copy_from(index, attrs, MLX5_IB_ATTR_CREATE_CQ_UAR_INDEX); + if (err) + goto err_cqb; + } else if (ucmd.flags & MLX5_IB_CREATE_CQ_FLAGS_UAR_PAGE_INDEX) { *index = ucmd.uar_page_index; } else if (context->bfregi.lib_uar_dyn) { err = -EINVAL; @@ -981,7 +989,7 @@ int mlx5_ib_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr, if (udata) { err = create_cq_user(dev, udata, cq, entries, &cqb, &cqe_size, - &index, &inlen); + &index, &inlen, attrs); if (err) return err; } else { @@ -1443,3 +1451,17 @@ int mlx5_ib_generate_wc(struct ib_cq *ibcq, struct ib_wc *wc) return 0; } + +ADD_UVERBS_ATTRIBUTES_SIMPLE( + mlx5_ib_cq_create, + UVERBS_OBJECT_CQ, + UVERBS_METHOD_CQ_CREATE, + UVERBS_ATTR_PTR_IN( + MLX5_IB_ATTR_CREATE_CQ_UAR_INDEX, + UVERBS_ATTR_TYPE(u32), + UA_OPTIONAL)); + +const struct uapi_definition mlx5_ib_create_cq_defs[] = { + UAPI_DEF_CHAIN_OBJ_TREE(UVERBS_OBJECT_CQ, &mlx5_ib_cq_create), + {}, +}; diff --git a/drivers/infiniband/hw/mlx5/main.c b/drivers/infiniband/hw/mlx5/main.c index 4a0380e711ea..89083f454952 100644 --- a/drivers/infiniband/hw/mlx5/main.c +++ b/drivers/infiniband/hw/mlx5/main.c @@ -3833,6 +3833,7 @@ static const struct uapi_definition mlx5_ib_defs[] = { UAPI_DEF_CHAIN(mlx5_ib_qos_defs), UAPI_DEF_CHAIN(mlx5_ib_std_types_defs), UAPI_DEF_CHAIN(mlx5_ib_dm_defs), + UAPI_DEF_CHAIN(mlx5_ib_create_cq_defs), UAPI_DEF_CHAIN_OBJ_TREE(UVERBS_OBJECT_DEVICE, &mlx5_ib_query_context), UAPI_DEF_CHAIN_OBJ_TREE_NAMED(MLX5_IB_OBJECT_VAR, diff --git a/drivers/infiniband/hw/mlx5/mlx5_ib.h b/drivers/infiniband/hw/mlx5/mlx5_ib.h index 2b03e607561e..c718c2cfffb8 100644 --- a/drivers/infiniband/hw/mlx5/mlx5_ib.h +++ b/drivers/infiniband/hw/mlx5/mlx5_ib.h @@ -1532,6 +1532,7 @@ extern const struct uapi_definition mlx5_ib_devx_defs[]; extern const struct uapi_definition mlx5_ib_flow_defs[]; extern const struct uapi_definition mlx5_ib_qos_defs[]; extern const struct uapi_definition mlx5_ib_std_types_defs[]; +extern const struct uapi_definition mlx5_ib_create_cq_defs[]; static inline int is_qp1(enum ib_qp_type qp_type) { diff --git a/include/uapi/rdma/mlx5_user_ioctl_cmds.h b/include/uapi/rdma/mlx5_user_ioctl_cmds.h index 595edad03dfe..d58556a019e4 100644 --- a/include/uapi/rdma/mlx5_user_ioctl_cmds.h +++ b/include/uapi/rdma/mlx5_user_ioctl_cmds.h @@ -270,6 +270,10 @@ enum mlx5_ib_device_query_context_attrs { MLX5_IB_ATTR_QUERY_CONTEXT_RESP_UCTX = (1U << UVERBS_ID_NS_SHIFT), }; +enum mlx5_ib_create_cq_attrs { + MLX5_IB_ATTR_CREATE_CQ_UAR_INDEX = UVERBS_ATTR_UHW_DRIVER_DATA, +}; + #define MLX5_IB_DW_MATCH_PARAM 0xA0 struct mlx5_ib_match_params { -- 2.45.2 ^ permalink raw reply related [flat|nested] 14+ messages in thread
* Re: [PATCH rdma-next 0/2] Extend mlx5 CQ creation with large UAR page index 2024-06-16 16:15 [PATCH rdma-next 0/2] Extend mlx5 CQ creation with large UAR page index Leon Romanovsky 2024-06-16 16:15 ` [PATCH rdma-next 1/2] RDMA: Pass entire uverbs attr bundle to create cq function Leon Romanovsky 2024-06-16 16:15 ` [PATCH rdma-next 2/2] RDMA/mlx5: Send UAR page index as ioctl attribute Leon Romanovsky @ 2024-06-16 21:47 ` Zhu Yanjun 2 siblings, 0 replies; 14+ messages in thread From: Zhu Yanjun @ 2024-06-16 21:47 UTC (permalink / raw) To: Leon Romanovsky, Jason Gunthorpe Cc: Leon Romanovsky, Ajay Sharma, Akiva Goldberger, Bernard Metzler, Chengchang Tang, Cheng Xu, Junxian Huang, Kai Shen, linux-kernel, linux-rdma, Long Li, Michael Margolin, Mustafa Ismail, Potnuri Bharat Teja, Selvin Xavier, Shiraz Saleem, Yishai Hadas, Zhu Yanjun 在 2024/6/17 0:15, Leon Romanovsky 写道: > From: Leon Romanovsky <leonro@nvidia.com> > > Hi, > > This series from Akiva extends the mlx5 private field with the UAR page index > which is larger than 16 bits as was before. UAR: User Access Region Zhu Yanjun > > As this is first time, we extend ioctl API with private data field after > it already has UHW object, we need to change create CQ API signature to > support it. > > Thanks > > Akiva Goldberger (2): > RDMA: Pass entire uverbs attr bundle to create cq function > RDMA/mlx5: Send UAR page index as ioctl attribute > > drivers/infiniband/core/uverbs_cmd.c | 2 +- > drivers/infiniband/core/uverbs_std_types_cq.c | 2 +- > drivers/infiniband/hw/bnxt_re/ib_verbs.c | 3 +- > drivers/infiniband/hw/bnxt_re/ib_verbs.h | 2 +- > drivers/infiniband/hw/cxgb4/cq.c | 3 +- > drivers/infiniband/hw/cxgb4/iw_cxgb4.h | 2 +- > drivers/infiniband/hw/efa/efa.h | 2 +- > drivers/infiniband/hw/efa/efa_verbs.c | 3 +- > drivers/infiniband/hw/erdma/erdma_verbs.c | 3 +- > drivers/infiniband/hw/erdma/erdma_verbs.h | 2 +- > drivers/infiniband/hw/hns/hns_roce_cq.c | 3 +- > drivers/infiniband/hw/hns/hns_roce_device.h | 2 +- > drivers/infiniband/hw/irdma/verbs.c | 5 +-- > drivers/infiniband/hw/mana/cq.c | 2 +- > drivers/infiniband/hw/mana/mana_ib.h | 2 +- > drivers/infiniband/hw/mlx4/cq.c | 3 +- > drivers/infiniband/hw/mlx4/mlx4_ib.h | 2 +- > drivers/infiniband/hw/mlx5/cq.c | 31 ++++++++++++++++--- > drivers/infiniband/hw/mlx5/main.c | 1 + > drivers/infiniband/hw/mlx5/mlx5_ib.h | 3 +- > drivers/infiniband/hw/mthca/mthca_provider.c | 3 +- > drivers/infiniband/sw/rxe/rxe_verbs.c | 3 +- > drivers/infiniband/sw/siw/siw_verbs.c | 2 +- > drivers/infiniband/sw/siw/siw_verbs.h | 2 +- > include/rdma/ib_verbs.h | 2 +- > include/uapi/rdma/ib_user_ioctl_cmds.h | 1 + > include/uapi/rdma/mlx5_user_ioctl_cmds.h | 4 +++ > 27 files changed, 67 insertions(+), 28 deletions(-) > ^ permalink raw reply [flat|nested] 14+ messages in thread
end of thread, other threads:[~2024-06-19 8:57 UTC | newest] Thread overview: 14+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2024-06-16 16:15 [PATCH rdma-next 0/2] Extend mlx5 CQ creation with large UAR page index Leon Romanovsky 2024-06-16 16:15 ` [PATCH rdma-next 1/2] RDMA: Pass entire uverbs attr bundle to create cq function Leon Romanovsky 2024-06-17 6:17 ` Leon Romanovsky 2024-06-17 13:44 ` Jason Gunthorpe 2024-06-17 15:49 ` Leon Romanovsky 2024-06-17 20:10 ` Jason Gunthorpe 2024-06-18 5:05 ` Leon Romanovsky 2024-06-18 13:08 ` Jason Gunthorpe 2024-06-18 16:05 ` Leon Romanovsky 2024-06-18 16:13 ` Jason Gunthorpe 2024-06-19 8:57 ` Leon Romanovsky 2024-06-17 16:00 ` Zhu Yanjun 2024-06-16 16:15 ` [PATCH rdma-next 2/2] RDMA/mlx5: Send UAR page index as ioctl attribute Leon Romanovsky 2024-06-16 21:47 ` [PATCH rdma-next 0/2] Extend mlx5 CQ creation with large UAR page index Zhu Yanjun
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).