[PATCH rdma-core 0/6] verbs: Custom parent-domain allocators

[PATCH rdma-core 0/6] verbs: Custom parent-domain allocators

[Yishai Hadas]

This series extends the parent domain object with custom allocation callbacks
that can be used by user-applications to override the provider allocation.

This can be used for example to add NUMA aware allocation.

The API was introduced in the mailing list by the below RFC [1] and was
implemented by mlx5 provider.

A detailed man page exists as part of the series to describe the usage and the
behavior.

A PR was sent as well [2].

[1] https://www.spinics.net/lists/linux-rdma/msg84590.html
[2] https://github.com/linux-rdma/rdma-core/pull/596

Haggai Eran (1):
  verbs: custom parent-domain allocators

Yishai Hadas (5):
  Update kernel headers
  mlx5: Extend mlx5_alloc_parent_domain() to support custom allocator
  mlx5: Add custom allocation support for QP and RWQ buffers
  mlx5: Add custom allocation support for DBR
  mlx5: Add custom allocation support for SRQ buffer

 kernel-headers/rdma/ib_user_ioctl_verbs.h  |  22 ++++++
 kernel-headers/rdma/rdma_user_ioctl_cmds.h |  22 ------
 libibverbs/man/ibv_alloc_parent_domain.3   |  54 +++++++++++++++
 libibverbs/verbs.h                         |  12 ++++
 providers/mlx5/buf.c                       |  59 +++++++++++++++++
 providers/mlx5/cq.c                        |   2 +-
 providers/mlx5/dbrec.c                     |  34 +++++++++-
 providers/mlx5/mlx5.h                      |  23 ++++++-
 providers/mlx5/mlx5dv.h                    |   6 ++
 providers/mlx5/srq.c                       |  25 +++++--
 providers/mlx5/verbs.c                     | 103 ++++++++++++++++++++---------
 11 files changed, 297 insertions(+), 65 deletions(-)

-- 
1.8.3.1

[PATCH rdma-core 1/6] Update kernel headers

[Yishai Hadas]

To commit 45b268543a8d ("RDMA/uapi: Fix and re-organize the usage of
rdma_driver_id")

Signed-off-by: Yishai Hadas <yishaih@mellanox.com>
---
 kernel-headers/rdma/ib_user_ioctl_verbs.h  | 22 ++++++++++++++++++++++
 kernel-headers/rdma/rdma_user_ioctl_cmds.h | 22 ----------------------
 2 files changed, 22 insertions(+), 22 deletions(-)

diff --git a/kernel-headers/rdma/ib_user_ioctl_verbs.h b/kernel-headers/rdma/ib_user_ioctl_verbs.h
index 72c7fc7..9019b2d 100644
--- a/kernel-headers/rdma/ib_user_ioctl_verbs.h
+++ b/kernel-headers/rdma/ib_user_ioctl_verbs.h
@@ -173,4 +173,26 @@ struct ib_uverbs_query_port_resp_ex {
 	__u8  reserved[6];
 };
 
+enum rdma_driver_id {
+	RDMA_DRIVER_UNKNOWN,
+	RDMA_DRIVER_MLX5,
+	RDMA_DRIVER_MLX4,
+	RDMA_DRIVER_CXGB3,
+	RDMA_DRIVER_CXGB4,
+	RDMA_DRIVER_MTHCA,
+	RDMA_DRIVER_BNXT_RE,
+	RDMA_DRIVER_OCRDMA,
+	RDMA_DRIVER_NES,
+	RDMA_DRIVER_I40IW,
+	RDMA_DRIVER_VMW_PVRDMA,
+	RDMA_DRIVER_QEDR,
+	RDMA_DRIVER_HNS,
+	RDMA_DRIVER_USNIC,
+	RDMA_DRIVER_RXE,
+	RDMA_DRIVER_HFI1,
+	RDMA_DRIVER_QIB,
+	RDMA_DRIVER_EFA,
+	RDMA_DRIVER_SIW,
+};
+
 #endif
diff --git a/kernel-headers/rdma/rdma_user_ioctl_cmds.h b/kernel-headers/rdma/rdma_user_ioctl_cmds.h
index b8bb285..7b1ec80 100644
--- a/kernel-headers/rdma/rdma_user_ioctl_cmds.h
+++ b/kernel-headers/rdma/rdma_user_ioctl_cmds.h
@@ -84,26 +84,4 @@ struct ib_uverbs_ioctl_hdr {
 	struct ib_uverbs_attr  attrs[0];
 };
 
-enum rdma_driver_id {
-	RDMA_DRIVER_UNKNOWN,
-	RDMA_DRIVER_MLX5,
-	RDMA_DRIVER_MLX4,
-	RDMA_DRIVER_CXGB3,
-	RDMA_DRIVER_CXGB4,
-	RDMA_DRIVER_MTHCA,
-	RDMA_DRIVER_BNXT_RE,
-	RDMA_DRIVER_OCRDMA,
-	RDMA_DRIVER_NES,
-	RDMA_DRIVER_I40IW,
-	RDMA_DRIVER_VMW_PVRDMA,
-	RDMA_DRIVER_QEDR,
-	RDMA_DRIVER_HNS,
-	RDMA_DRIVER_USNIC,
-	RDMA_DRIVER_RXE,
-	RDMA_DRIVER_HFI1,
-	RDMA_DRIVER_QIB,
-	RDMA_DRIVER_EFA,
-	RDMA_DRIVER_SIW,
-};
-
 #endif
-- 
1.8.3.1

[PATCH rdma-core 2/6] verbs: custom parent-domain allocators

[Yishai Hadas]

From: Haggai Eran <haggaie@mellanox.com>

Extend the parent domain object with custom allocation callbacks that
can be used by user-applications to override the provider allocation.

This can be used for example to add NUMA aware allocation.

The new allocator receives context information about the parent domain,
as well as the requested size and alignment of the buffer. It also
receives a vendor-specific resource type code to allow customizing it
for specific resources.

The allocator then allocates the memory or returns an
IBV_ALLOCATOR_USE_DEFAULT value to request that the provider driver use
its own allocation method.

Signed-off-by: Haggai Eran <haggaie@mellanox.com>
Signed-off-by: Yishai Hadas <yishaih@mellanox.com>
---
 libibverbs/man/ibv_alloc_parent_domain.3 | 54 ++++++++++++++++++++++++++++++++
 libibverbs/verbs.h                       | 12 +++++++
 2 files changed, 66 insertions(+)

diff --git a/libibverbs/man/ibv_alloc_parent_domain.3 b/libibverbs/man/ibv_alloc_parent_domain.3
index 92b6058..6e2f356 100644
--- a/libibverbs/man/ibv_alloc_parent_domain.3
+++ b/libibverbs/man/ibv_alloc_parent_domain.3
@@ -41,11 +41,23 @@ The
 argument specifies the following:
 .PP
 .nf
+enum ibv_parent_domain_init_attr_mask {
+.in +8
+IBV_PARENT_DOMAIN_INIT_ATTR_ALLOCATORS = 1 << 0,
+IBV_PARENT_DOMAIN_INIT_ATTR_PD_CONTEXT = 1 << 1,
+.in -8
+};
+
 struct ibv_parent_domain_init_attr {
 .in +8
 struct ibv_pd *pd; /* referance to a protection domain, can't be NULL */
 struct ibv_td *td; /* referance to a thread domain, or NULL */
 uint32_t comp_mask;
+void *(*alloc)(struct ibv_pd *pd, void *pd_context, size_t size,
+               size_t alignment, uint64_t resource_type);
+void (*free)(struct ibv_pd *pd, void *pd_context, void *ptr,
+             uint64_t resource_type);
+void *pd_context;
 .in -8
 };
 .fi
@@ -56,6 +68,48 @@ will deallocate the parent domain as its exposed as an ibv_pd
 .I pd\fR.
 All resources created with the parent domain
 should be destroyed prior to deallocating the parent domain\fR.
+.SH "ARGUMENTS"
+.B pd
+Reference to the protection domain that this parent domain uses.
+.PP
+.B td
+An optional thread domain that the parent domain uses.
+.PP
+.B comp_mask
+Bit-mask of optional fields in the ibv_parent_domain_init_attr struct.
+.PP
+.B alloc
+Custom memory allocation function for this parent domain. Provider
+memory allocations will use this function to allocate the needed memory.
+The allocation function is passed the parent domain
+.B pd
+and the user-specified context
+.B pd_context.
+In addition, the callback receives the
+.B size
+and the
+.B alignment
+of the requested buffer, as well a vendor-specific
+.B resource_type
+, which is derived from the rdma_driver_id enum (upper 32 bits) and a vendor
+specific resource code.
+The function returns the pointer to the allocated buffer, or NULL to
+designate an error.  It may also return
+.B IBV_ALLOCATOR_USE_DEFAULT
+asking the callee to allocate the buffer using the default allocator.
+
+The callback makes sure the allocated buffer is initialized with zeros. It is
+also the responsibility of the callback to make sure the memory cannot be
+COWed, e.g. by using madvise(MADV_DONTFORK) or by allocating anonymous shared
+memory.
+.PP
+.B free
+Callback to free memory buffers that were allocated using a successful
+alloc().
+.PP
+.B pd_context
+A pointer for additional user-specific data to be associated with this
+parent domain. The pointer is passed back to the custom allocator functions.
 .SH "RETURN VALUE"
 .B ibv_alloc_parent_domain()
 returns a pointer to the allocated struct
diff --git a/libibverbs/verbs.h b/libibverbs/verbs.h
index 12a33a9..d873f6d 100644
--- a/libibverbs/verbs.h
+++ b/libibverbs/verbs.h
@@ -1976,10 +1976,22 @@ struct ibv_cq_init_attr_ex {
 	uint32_t		flags;
 };
 
+enum ibv_parent_domain_init_attr_mask {
+	IBV_PARENT_DOMAIN_INIT_ATTR_ALLOCATORS = 1 << 0,
+	IBV_PARENT_DOMAIN_INIT_ATTR_PD_CONTEXT = 1 << 1,
+};
+
+#define IBV_ALLOCATOR_USE_DEFAULT ((void *)-1)
+
 struct ibv_parent_domain_init_attr {
 	struct ibv_pd *pd; /* referance to a protection domain object, can't be NULL */
 	struct ibv_td *td; /* referance to a thread domain object, or NULL */
 	uint32_t comp_mask;
+	void *(*alloc)(struct ibv_pd *pd, void *pd_context, size_t size,
+		       size_t alignment, uint64_t resource_type);
+	void (*free)(struct ibv_pd *pd, void *pd_context, void *ptr,
+		     uint64_t resource_type);
+	void *pd_context;
 };
 
 struct ibv_counters_init_attr {
-- 
1.8.3.1

Re: [PATCH rdma-core 2/6] verbs: custom parent-domain allocators

[Leon Romanovsky]

On Mon, Oct 28, 2019 at 11:14:55AM +0200, Yishai Hadas wrote:
> From: Haggai Eran <haggaie@mellanox.com>
>
> Extend the parent domain object with custom allocation callbacks that
> can be used by user-applications to override the provider allocation.
>
> This can be used for example to add NUMA aware allocation.
>
> The new allocator receives context information about the parent domain,
> as well as the requested size and alignment of the buffer. It also
> receives a vendor-specific resource type code to allow customizing it
> for specific resources.
>
> The allocator then allocates the memory or returns an
> IBV_ALLOCATOR_USE_DEFAULT value to request that the provider driver use
> its own allocation method.
>
> Signed-off-by: Haggai Eran <haggaie@mellanox.com>
> Signed-off-by: Yishai Hadas <yishaih@mellanox.com>
> ---
>  libibverbs/man/ibv_alloc_parent_domain.3 | 54 ++++++++++++++++++++++++++++++++
>  libibverbs/verbs.h                       | 12 +++++++
>  2 files changed, 66 insertions(+)
>

It is unclear to me how and maybe it is not possible for this API. but I
would expect any changes in public API be accompanied by relevant tests.

Thanks

Re: [PATCH rdma-core 2/6] verbs: custom parent-domain allocators

[Yishai Hadas]

On 10/28/2019 11:48 AM, Leon Romanovsky wrote:
> On Mon, Oct 28, 2019 at 11:14:55AM +0200, Yishai Hadas wrote:
>> From: Haggai Eran <haggaie@mellanox.com>
>>
>> Extend the parent domain object with custom allocation callbacks that
>> can be used by user-applications to override the provider allocation.
>>
>> This can be used for example to add NUMA aware allocation.
>>
>> The new allocator receives context information about the parent domain,
>> as well as the requested size and alignment of the buffer. It also
>> receives a vendor-specific resource type code to allow customizing it
>> for specific resources.
>>
>> The allocator then allocates the memory or returns an
>> IBV_ALLOCATOR_USE_DEFAULT value to request that the provider driver use
>> its own allocation method.
>>
>> Signed-off-by: Haggai Eran <haggaie@mellanox.com>
>> Signed-off-by: Yishai Hadas <yishaih@mellanox.com>
>> ---
>>   libibverbs/man/ibv_alloc_parent_domain.3 | 54 ++++++++++++++++++++++++++++++++
>>   libibverbs/verbs.h                       | 12 +++++++
>>   2 files changed, 66 insertions(+)
>>
> 
> It is unclear to me how and maybe it is not possible for this API. but I
> would expect any changes in public API be accompanied by relevant tests.
> 

The current pyverbs infrastructure has some lack in this area to enable 
adding a simple test over.
The infrastructure team is working to find a solution and may push it 
once be ready later on.

Yishai

[PATCH rdma-core 3/6] mlx5: Extend mlx5_alloc_parent_domain() to support custom allocator

[Yishai Hadas]

Extend mlx5_alloc_parent_domain() to support custom allocator,
downstream patches from this series will use this functionality.

Signed-off-by: Yishai Hadas <yishaih@mellanox.com>
---
 providers/mlx5/mlx5.h  |  5 +++++
 providers/mlx5/verbs.c | 12 +++++++++++-
 2 files changed, 16 insertions(+), 1 deletion(-)

diff --git a/providers/mlx5/mlx5.h b/providers/mlx5/mlx5.h
index e63319e..e960e6f 100644
--- a/providers/mlx5/mlx5.h
+++ b/providers/mlx5/mlx5.h
@@ -352,6 +352,11 @@ struct mlx5_pd {
 struct mlx5_parent_domain {
 	struct mlx5_pd mpd;
 	struct mlx5_td *mtd;
+	void *(*alloc)(struct ibv_pd *pd, void *pd_context, size_t size,
+		       size_t alignment, uint64_t resource_type);
+	void (*free)(struct ibv_pd *pd, void *pd_context, void *ptr,
+		     uint64_t resource_type);
+	void *pd_context;
 };
 
 enum {
diff --git a/providers/mlx5/verbs.c b/providers/mlx5/verbs.c
index 0a66a33..166ea4f 100644
--- a/providers/mlx5/verbs.c
+++ b/providers/mlx5/verbs.c
@@ -326,7 +326,9 @@ mlx5_alloc_parent_domain(struct ibv_context *context,
 	if (ibv_check_alloc_parent_domain(attr))
 		return NULL;
 
-	if (attr->comp_mask) {
+	if (!check_comp_mask(attr->comp_mask,
+			     IBV_PARENT_DOMAIN_INIT_ATTR_ALLOCATORS |
+			     IBV_PARENT_DOMAIN_INIT_ATTR_PD_CONTEXT)) {
 		errno = EINVAL;
 		return NULL;
 	}
@@ -350,6 +352,14 @@ mlx5_alloc_parent_domain(struct ibv_context *context,
 	    &mparent_domain->mpd.ibv_pd,
 	    &mparent_domain->mpd.mprotection_domain->ibv_pd);
 
+	if (attr->comp_mask & IBV_PARENT_DOMAIN_INIT_ATTR_ALLOCATORS) {
+		mparent_domain->alloc = attr->alloc;
+		mparent_domain->free = attr->free;
+	}
+
+	if (attr->comp_mask & IBV_PARENT_DOMAIN_INIT_ATTR_PD_CONTEXT)
+		mparent_domain->pd_context = attr->pd_context;
+
 	return &mparent_domain->mpd.ibv_pd;
 }
 
-- 
1.8.3.1

[PATCH rdma-core 4/6] mlx5: Add custom allocation support for QP and RWQ buffers

[Yishai Hadas]

Add custom allocation support for QP and RWQ buffers by extending the
internal allocation flows to consider this option.

Signed-off-by: Yishai Hadas <yishaih@mellanox.com>
---
 providers/mlx5/buf.c    | 59 +++++++++++++++++++++++++++++++++++++++++++++++++
 providers/mlx5/cq.c     |  2 +-
 providers/mlx5/mlx5.h   |  6 +++++
 providers/mlx5/mlx5dv.h |  4 ++++
 providers/mlx5/verbs.c  | 34 +++++++++++++++++++++++-----
 5 files changed, 98 insertions(+), 7 deletions(-)

diff --git a/providers/mlx5/buf.c b/providers/mlx5/buf.c
index aa2fe02..b5cf391 100644
--- a/providers/mlx5/buf.c
+++ b/providers/mlx5/buf.c
@@ -356,6 +356,40 @@ int mlx5_alloc_buf_extern(struct mlx5_context *ctx, struct mlx5_buf *buf,
 	return -1;
 }
 
+static void mlx5_free_buf_custom(struct mlx5_context *ctx,
+			  struct mlx5_buf *buf)
+{
+	struct mlx5_parent_domain *mparent_domain = buf->mparent_domain;
+
+	mparent_domain->free(&mparent_domain->mpd.ibv_pd,
+			     mparent_domain->pd_context,
+			     buf->buf,
+			     buf->resource_type);
+}
+
+static int mlx5_alloc_buf_custom(struct mlx5_context *ctx,
+			  struct mlx5_buf *buf, size_t size)
+{
+	struct mlx5_parent_domain *mparent_domain = buf->mparent_domain;
+	void *addr;
+
+	addr = mparent_domain->alloc(&mparent_domain->mpd.ibv_pd,
+				   mparent_domain->pd_context, size,
+				   buf->req_alignment,
+				   buf->resource_type);
+	if (addr == IBV_ALLOCATOR_USE_DEFAULT)
+		return 1;
+
+	if (addr || size == 0) {
+		buf->buf = addr;
+		buf->length = size;
+		buf->type = MLX5_ALLOC_TYPE_CUSTOM;
+		return 0;
+	}
+
+	return -1;
+}
+
 int mlx5_alloc_prefered_buf(struct mlx5_context *mctx,
 			    struct mlx5_buf *buf,
 			    size_t size, int page_size,
@@ -364,6 +398,14 @@ int mlx5_alloc_prefered_buf(struct mlx5_context *mctx,
 {
 	int ret;
 
+	if (type == MLX5_ALLOC_TYPE_CUSTOM) {
+		ret = mlx5_alloc_buf_custom(mctx, buf, size);
+		if (ret <= 0)
+			return ret;
+
+		/* Fallback - default allocation is required */
+	}
+
 	/*
 	 * Fallback mechanism priority:
 	 *	huge pages
@@ -426,6 +468,10 @@ int mlx5_free_actual_buf(struct mlx5_context *ctx, struct mlx5_buf *buf)
 		mlx5_free_buf_extern(ctx, buf);
 		break;
 
+	case MLX5_ALLOC_TYPE_CUSTOM:
+		mlx5_free_buf_custom(ctx, buf);
+		break;
+
 	default:
 		fprintf(stderr, "Bad allocation type\n");
 	}
@@ -458,12 +504,20 @@ static uint32_t mlx5_get_block_order(uint32_t v)
 	return r;
 }
 
+bool mlx5_is_custom_alloc(struct ibv_pd *pd)
+{
+	struct mlx5_parent_domain *mparent_domain = to_mparent_domain(pd);
+
+	return (mparent_domain && mparent_domain->alloc && mparent_domain->free);
+}
+
 bool mlx5_is_extern_alloc(struct mlx5_context *context)
 {
 	return context->extern_alloc.alloc && context->extern_alloc.free;
 }
 
 void mlx5_get_alloc_type(struct mlx5_context *context,
+			 struct ibv_pd *pd,
 			 const char *component,
 			 enum mlx5_alloc_type *alloc_type,
 			 enum mlx5_alloc_type default_type)
@@ -472,6 +526,11 @@ void mlx5_get_alloc_type(struct mlx5_context *context,
 	char *env_value;
 	char name[128];
 
+	if (mlx5_is_custom_alloc(pd)) {
+		*alloc_type = MLX5_ALLOC_TYPE_CUSTOM;
+		return;
+	}
+
 	if (mlx5_is_extern_alloc(context)) {
 		*alloc_type = MLX5_ALLOC_TYPE_EXTERNAL;
 		return;
diff --git a/providers/mlx5/cq.c b/providers/mlx5/cq.c
index b9b47df..26edd86 100644
--- a/providers/mlx5/cq.c
+++ b/providers/mlx5/cq.c
@@ -1861,7 +1861,7 @@ int mlx5_alloc_cq_buf(struct mlx5_context *mctx, struct mlx5_cq *cq,
 	if (mlx5_use_huge("HUGE_CQ"))
 		default_type = MLX5_ALLOC_TYPE_HUGE;
 
-	mlx5_get_alloc_type(mctx, MLX5_CQ_PREFIX, &type, default_type);
+	mlx5_get_alloc_type(mctx, NULL, MLX5_CQ_PREFIX, &type, default_type);
 
 	ret = mlx5_alloc_prefered_buf(mctx, buf,
 				      align(nent * cqe_sz, dev->page_size),
diff --git a/providers/mlx5/mlx5.h b/providers/mlx5/mlx5.h
index e960e6f..1c13390 100644
--- a/providers/mlx5/mlx5.h
+++ b/providers/mlx5/mlx5.h
@@ -173,6 +173,7 @@ enum mlx5_alloc_type {
 	MLX5_ALLOC_TYPE_PREFER_HUGE,
 	MLX5_ALLOC_TYPE_PREFER_CONTIG,
 	MLX5_ALLOC_TYPE_EXTERNAL,
+	MLX5_ALLOC_TYPE_CUSTOM,
 	MLX5_ALLOC_TYPE_ALL
 };
 
@@ -334,6 +335,9 @@ struct mlx5_buf {
 	int                             base;
 	struct mlx5_hugetlb_mem	       *hmem;
 	enum mlx5_alloc_type		type;
+	uint64_t			resource_type;
+	size_t				req_alignment;
+	struct mlx5_parent_domain	*mparent_domain;
 };
 
 struct mlx5_td {
@@ -780,10 +784,12 @@ int mlx5_alloc_prefered_buf(struct mlx5_context *mctx,
 			    const char *component);
 int mlx5_free_actual_buf(struct mlx5_context *ctx, struct mlx5_buf *buf);
 void mlx5_get_alloc_type(struct mlx5_context *context,
+			 struct ibv_pd *pd,
 			 const char *component,
 			 enum mlx5_alloc_type *alloc_type,
 			 enum mlx5_alloc_type default_alloc_type);
 int mlx5_use_huge(const char *key);
+bool mlx5_is_custom_alloc(struct ibv_pd *pd);
 bool mlx5_is_extern_alloc(struct mlx5_context *context);
 int mlx5_alloc_buf_extern(struct mlx5_context *ctx, struct mlx5_buf *buf,
 			  size_t size);
diff --git a/providers/mlx5/mlx5dv.h b/providers/mlx5/mlx5dv.h
index d5e8e0c..0ad9768 100644
--- a/providers/mlx5/mlx5dv.h
+++ b/providers/mlx5/mlx5dv.h
@@ -59,6 +59,10 @@ extern "C" {
 #define MLX5DV_ALWAYS_INLINE inline
 #endif
 
+
+#define MLX5DV_RES_TYPE_QP ((uint64_t)RDMA_DRIVER_MLX5 << 32 | 1)
+#define MLX5DV_RES_TYPE_RWQ ((uint64_t)RDMA_DRIVER_MLX5 << 32 | 2)
+
 enum {
 	MLX5_RCV_DBR	= 0,
 	MLX5_SND_DBR	= 1,
diff --git a/providers/mlx5/verbs.c b/providers/mlx5/verbs.c
index 166ea4f..513af5e 100644
--- a/providers/mlx5/verbs.c
+++ b/providers/mlx5/verbs.c
@@ -1554,8 +1554,14 @@ static int mlx5_alloc_qp_buf(struct ibv_context *context,
 	if (mlx5_use_huge(qp_huge_key))
 		default_alloc_type = MLX5_ALLOC_TYPE_HUGE;
 
-	mlx5_get_alloc_type(to_mctx(context), MLX5_QP_PREFIX, &alloc_type,
-			    default_alloc_type);
+	mlx5_get_alloc_type(to_mctx(context), attr->pd, MLX5_QP_PREFIX,
+			    &alloc_type, default_alloc_type);
+
+	if (alloc_type == MLX5_ALLOC_TYPE_CUSTOM) {
+		qp->buf.mparent_domain = to_mparent_domain(attr->pd);
+		qp->buf.req_alignment = to_mdev(context->device)->page_size;
+		qp->buf.resource_type = MLX5DV_RES_TYPE_QP;
+	}
 
 	err = mlx5_alloc_prefered_buf(to_mctx(context), &qp->buf,
 				      align(qp->buf_size, to_mdev
@@ -1569,12 +1575,20 @@ static int mlx5_alloc_qp_buf(struct ibv_context *context,
 		goto ex_wrid;
 	}
 
-	memset(qp->buf.buf, 0, qp->buf_size);
+	if (qp->buf.type != MLX5_ALLOC_TYPE_CUSTOM)
+		memset(qp->buf.buf, 0, qp->buf_size);
 
 	if (attr->qp_type == IBV_QPT_RAW_PACKET ||
 	    qp->flags & MLX5_QP_FLAGS_USE_UNDERLAY) {
 		size_t aligned_sq_buf_size = align(qp->sq_buf_size,
 						   to_mdev(context->device)->page_size);
+
+		if (alloc_type == MLX5_ALLOC_TYPE_CUSTOM) {
+			qp->sq_buf.mparent_domain = to_mparent_domain(attr->pd);
+			qp->sq_buf.req_alignment = to_mdev(context->device)->page_size;
+			qp->sq_buf.resource_type = MLX5DV_RES_TYPE_QP;
+		}
+
 		/* For Raw Packet QP, allocate a separate buffer for the SQ */
 		err = mlx5_alloc_prefered_buf(to_mctx(context), &qp->sq_buf,
 					      aligned_sq_buf_size,
@@ -1586,7 +1600,8 @@ static int mlx5_alloc_qp_buf(struct ibv_context *context,
 			goto rq_buf;
 		}
 
-		memset(qp->sq_buf.buf, 0, aligned_sq_buf_size);
+		if (qp->sq_buf.type != MLX5_ALLOC_TYPE_CUSTOM)
+			memset(qp->sq_buf.buf, 0, aligned_sq_buf_size);
 	}
 
 	return 0;
@@ -3121,13 +3136,14 @@ static void mlx5_free_rwq_buf(struct mlx5_rwq *rwq, struct ibv_context *context)
 }
 
 static int mlx5_alloc_rwq_buf(struct ibv_context *context,
+			      struct ibv_pd *pd,
 			      struct mlx5_rwq *rwq,
 			      int size)
 {
 	int err;
 	enum mlx5_alloc_type alloc_type;
 
-	mlx5_get_alloc_type(to_mctx(context), MLX5_RWQ_PREFIX,
+	mlx5_get_alloc_type(to_mctx(context), pd, MLX5_RWQ_PREFIX,
 			    &alloc_type, MLX5_ALLOC_TYPE_ANON);
 
 	rwq->rq.wrid = malloc(rwq->rq.wqe_cnt * sizeof(uint64_t));
@@ -3136,6 +3152,12 @@ static int mlx5_alloc_rwq_buf(struct ibv_context *context,
 		return -1;
 	}
 
+	if (alloc_type == MLX5_ALLOC_TYPE_CUSTOM) {
+		rwq->buf.mparent_domain = to_mparent_domain(pd);
+		rwq->buf.req_alignment = to_mdev(context->device)->page_size;
+		rwq->buf.resource_type = MLX5DV_RES_TYPE_RWQ;
+	}
+
 	err = mlx5_alloc_prefered_buf(to_mctx(context), &rwq->buf,
 				      align(rwq->buf_size, to_mdev
 				      (context->device)->page_size),
@@ -3186,7 +3208,7 @@ static struct ibv_wq *create_wq(struct ibv_context *context,
 	}
 
 	rwq->buf_size = ret;
-	if (mlx5_alloc_rwq_buf(context, rwq, ret))
+	if (mlx5_alloc_rwq_buf(context, attr->pd, rwq, ret))
 		goto err;
 
 	mlx5_init_rwq_indices(rwq);
-- 
1.8.3.1

[PATCH rdma-core 5/6] mlx5: Add custom allocation support for DBR

[Yishai Hadas]

Add custom allocation support for DBR by extending the internal
allocation/free flows to consider this option.

Signed-off-by: Yishai Hadas <yishaih@mellanox.com>
---
 providers/mlx5/dbrec.c  | 34 ++++++++++++++++++++++++++++++++--
 providers/mlx5/mlx5.h   |  9 +++++++--
 providers/mlx5/mlx5dv.h |  1 +
 providers/mlx5/verbs.c  | 47 +++++++++++++++++++++++++++--------------------
 4 files changed, 67 insertions(+), 24 deletions(-)

diff --git a/providers/mlx5/dbrec.c b/providers/mlx5/dbrec.c
index d6a69a5..5ef3d16 100644
--- a/providers/mlx5/dbrec.c
+++ b/providers/mlx5/dbrec.c
@@ -85,12 +85,31 @@ static struct mlx5_db_page *__add_page(struct mlx5_context *context)
 	return page;
 }
 
-__be32 *mlx5_alloc_dbrec(struct mlx5_context *context)
+__be32 *mlx5_alloc_dbrec(struct mlx5_context *context, struct ibv_pd *pd,
+			 bool *custom_alloc)
 {
 	struct mlx5_db_page *page;
 	__be32 *db = NULL;
 	int i, j;
 
+	if (mlx5_is_custom_alloc(pd)) {
+		struct mlx5_parent_domain *mparent_domain = to_mparent_domain(pd);
+
+		db = mparent_domain->alloc(&mparent_domain->mpd.ibv_pd,
+				   mparent_domain->pd_context, 8, 8,
+				   MLX5DV_RES_TYPE_DBR);
+
+		if (db == IBV_ALLOCATOR_USE_DEFAULT)
+			goto default_alloc;
+
+		if (!db)
+			return NULL;
+
+		*custom_alloc = true;
+		return db;
+	}
+
+default_alloc:
 	pthread_mutex_lock(&context->db_list_mutex);
 
 	for (page = context->db_list; page; page = page->next)
@@ -118,12 +137,23 @@ out:
 	return db;
 }
 
-void mlx5_free_db(struct mlx5_context *context, __be32 *db)
+void mlx5_free_db(struct mlx5_context *context, __be32 *db, struct ibv_pd *pd,
+		  bool custom_alloc)
 {
 	struct mlx5_db_page *page;
 	uintptr_t ps = to_mdev(context->ibv_ctx.context.device)->page_size;
 	int i;
 
+	if (custom_alloc) {
+		struct mlx5_parent_domain *mparent_domain = to_mparent_domain(pd);
+
+		mparent_domain->free(&mparent_domain->mpd.ibv_pd,
+				     mparent_domain->pd_context,
+				     db,
+				     MLX5DV_RES_TYPE_DBR);
+		return;
+	}
+
 	pthread_mutex_lock(&context->db_list_mutex);
 
 	for (page = context->db_list; page; page = page->next)
diff --git a/providers/mlx5/mlx5.h b/providers/mlx5/mlx5.h
index 1c13390..9a5cd6b 100644
--- a/providers/mlx5/mlx5.h
+++ b/providers/mlx5/mlx5.h
@@ -438,6 +438,7 @@ struct mlx5_srq {
 	int				waitq_head;
 	int				waitq_tail;
 	__be32			       *db;
+	bool				custom_db;
 	uint16_t			counter;
 	int				wq_sig;
 	struct ibv_qp		       *cmd_qp;
@@ -553,6 +554,7 @@ struct mlx5_qp {
 	struct mlx5_wq                  sq;
 
 	__be32                         *db;
+	bool				custom_db;
 	struct mlx5_wq                  rq;
 	int                             wq_sig;
 	uint32_t			qp_cap_cache;
@@ -582,6 +584,7 @@ struct mlx5_rwq {
 	int buf_size;
 	struct mlx5_wq rq;
 	__be32  *db;
+	bool	custom_db;
 	void	*pbuff;
 	__be32	*recv_db;
 	int wq_sig;
@@ -795,8 +798,10 @@ int mlx5_alloc_buf_extern(struct mlx5_context *ctx, struct mlx5_buf *buf,
 			  size_t size);
 void mlx5_free_buf_extern(struct mlx5_context *ctx, struct mlx5_buf *buf);
 
-__be32 *mlx5_alloc_dbrec(struct mlx5_context *context);
-void mlx5_free_db(struct mlx5_context *context, __be32 *db);
+__be32 *mlx5_alloc_dbrec(struct mlx5_context *context, struct ibv_pd *pd,
+			 bool *custom_alloc);
+void mlx5_free_db(struct mlx5_context *context, __be32 *db, struct ibv_pd *pd,
+		  bool custom_alloc);
 
 int mlx5_query_device(struct ibv_context *context,
 		       struct ibv_device_attr *attr);
diff --git a/providers/mlx5/mlx5dv.h b/providers/mlx5/mlx5dv.h
index 0ad9768..ac291eb 100644
--- a/providers/mlx5/mlx5dv.h
+++ b/providers/mlx5/mlx5dv.h
@@ -62,6 +62,7 @@ extern "C" {
 
 #define MLX5DV_RES_TYPE_QP ((uint64_t)RDMA_DRIVER_MLX5 << 32 | 1)
 #define MLX5DV_RES_TYPE_RWQ ((uint64_t)RDMA_DRIVER_MLX5 << 32 | 2)
+#define MLX5DV_RES_TYPE_DBR ((uint64_t)RDMA_DRIVER_MLX5 << 32 | 3)
 
 enum {
 	MLX5_RCV_DBR	= 0,
diff --git a/providers/mlx5/verbs.c b/providers/mlx5/verbs.c
index 513af5e..1e026af 100644
--- a/providers/mlx5/verbs.c
+++ b/providers/mlx5/verbs.c
@@ -749,7 +749,7 @@ static struct ibv_cq_ex *create_cq(struct ibv_context *context,
 		goto err_spl;
 	}
 
-	cq->dbrec  = mlx5_alloc_dbrec(to_mctx(context));
+	cq->dbrec  = mlx5_alloc_dbrec(to_mctx(context), NULL, NULL);
 	if (!cq->dbrec) {
 		mlx5_dbg(fp, MLX5_DBG_CQ, "\n");
 		goto err_buf;
@@ -834,7 +834,7 @@ static struct ibv_cq_ex *create_cq(struct ibv_context *context,
 	return &cq->ibv_cq;
 
 err_db:
-	mlx5_free_db(to_mctx(context), cq->dbrec);
+	mlx5_free_db(to_mctx(context), cq->dbrec, NULL, 0);
 
 err_buf:
 	mlx5_free_cq_buf(to_mctx(context), &cq->buf_a);
@@ -968,7 +968,7 @@ int mlx5_destroy_cq(struct ibv_cq *cq)
 	if (ret)
 		return ret;
 
-	mlx5_free_db(to_mctx(cq->context), to_mcq(cq)->dbrec);
+	mlx5_free_db(to_mctx(cq->context), to_mcq(cq)->dbrec, NULL, 0);
 	mlx5_free_cq_buf(to_mctx(cq->context), to_mcq(cq)->active_buf);
 	free(to_mcq(cq));
 
@@ -1028,13 +1028,14 @@ struct ibv_srq *mlx5_create_srq(struct ibv_pd *pd,
 		goto err;
 	}
 
-	srq->db = mlx5_alloc_dbrec(to_mctx(pd->context));
+	srq->db = mlx5_alloc_dbrec(to_mctx(pd->context), pd, &srq->custom_db);
 	if (!srq->db) {
 		fprintf(stderr, "%s-%d:\n", __func__, __LINE__);
 		goto err_free;
 	}
 
-	*srq->db = 0;
+	if (!srq->custom_db)
+		*srq->db = 0;
 
 	cmd.buf_addr = (uintptr_t) srq->buf.buf;
 	cmd.db_addr  = (uintptr_t) srq->db;
@@ -1077,7 +1078,7 @@ err_destroy:
 
 err_db:
 	pthread_mutex_unlock(&ctx->srq_table_mutex);
-	mlx5_free_db(to_mctx(pd->context), srq->db);
+	mlx5_free_db(to_mctx(pd->context), srq->db, pd, srq->custom_db);
 
 err_free:
 	free(srq->wrid);
@@ -1128,7 +1129,7 @@ int mlx5_destroy_srq(struct ibv_srq *srq)
 	else
 		mlx5_clear_srq(ctx, msrq->srqn);
 
-	mlx5_free_db(ctx, msrq->db);
+	mlx5_free_db(ctx, msrq->db, srq->pd, msrq->custom_db);
 	mlx5_free_buf(&msrq->buf);
 	free(msrq->tm_list);
 	free(msrq->wrid);
@@ -2033,14 +2034,16 @@ static struct ibv_qp *create_qp(struct ibv_context *context,
 			mlx5_spinlock_init_pd(&qp->rq.lock, attr->pd))
 		goto err_free_qp_buf;
 
-	qp->db = mlx5_alloc_dbrec(ctx);
+	qp->db = mlx5_alloc_dbrec(ctx, attr->pd, &qp->custom_db);
 	if (!qp->db) {
 		mlx5_dbg(fp, MLX5_DBG_QP, "\n");
 		goto err_free_qp_buf;
 	}
 
-	qp->db[MLX5_RCV_DBR] = 0;
-	qp->db[MLX5_SND_DBR] = 0;
+	if (!qp->custom_db) {
+		qp->db[MLX5_RCV_DBR] = 0;
+		qp->db[MLX5_SND_DBR] = 0;
+	}
 
 	cmd.buf_addr = (uintptr_t) qp->buf.buf;
 	cmd.sq_buf_addr = (attr->qp_type == IBV_QPT_RAW_PACKET ||
@@ -2147,7 +2150,7 @@ err_free_uidx:
 		mlx5_clear_uidx(ctx, usr_idx);
 
 err_rq_db:
-	mlx5_free_db(to_mctx(context), qp->db);
+	mlx5_free_db(to_mctx(context), qp->db, attr->pd, qp->custom_db);
 
 err_free_qp_buf:
 	mlx5_free_qp_buf(ctx, qp);
@@ -2270,7 +2273,7 @@ int mlx5_destroy_qp(struct ibv_qp *ibqp)
 		mlx5_clear_uidx(ctx, qp->rsc.rsn);
 
 	if (qp->dc_type != MLX5DV_DCTYPE_DCT) {
-		mlx5_free_db(ctx, qp->db);
+		mlx5_free_db(ctx, qp->db, ibqp->pd, qp->custom_db);
 		mlx5_free_qp_buf(ctx, qp);
 	}
 free:
@@ -2892,13 +2895,14 @@ struct ibv_srq *mlx5_create_srq_ex(struct ibv_context *context,
 		goto err;
 	}
 
-	msrq->db = mlx5_alloc_dbrec(ctx);
+	msrq->db = mlx5_alloc_dbrec(ctx, attr->pd, &msrq->custom_db);
 	if (!msrq->db) {
 		fprintf(stderr, "%s-%d:\n", __func__, __LINE__);
 		goto err_free;
 	}
 
-	*msrq->db = 0;
+	if (!msrq->custom_db)
+		*msrq->db = 0;
 
 	cmd.buf_addr = (uintptr_t)msrq->buf.buf;
 	cmd.db_addr  = (uintptr_t)msrq->db;
@@ -2990,7 +2994,7 @@ err_free_uidx:
 		pthread_mutex_unlock(&ctx->srq_table_mutex);
 
 err_free_db:
-	mlx5_free_db(ctx, msrq->db);
+	mlx5_free_db(ctx, msrq->db, attr->pd, msrq->custom_db);
 
 err_free:
 	free(msrq->wrid);
@@ -3216,12 +3220,15 @@ static struct ibv_wq *create_wq(struct ibv_context *context,
 	if (mlx5_spinlock_init_pd(&rwq->rq.lock, attr->pd))
 		goto err_free_rwq_buf;
 
-	rwq->db = mlx5_alloc_dbrec(ctx);
+	rwq->db = mlx5_alloc_dbrec(ctx, attr->pd, &rwq->custom_db);
 	if (!rwq->db)
 		goto err_free_rwq_buf;
 
-	rwq->db[MLX5_RCV_DBR] = 0;
-	rwq->db[MLX5_SND_DBR] = 0;
+	if (!rwq->custom_db) {
+		rwq->db[MLX5_RCV_DBR] = 0;
+		rwq->db[MLX5_SND_DBR] = 0;
+	}
+
 	rwq->pbuff = rwq->buf.buf + rwq->rq.offset;
 	rwq->recv_db =  &rwq->db[MLX5_RCV_DBR];
 	cmd.buf_addr = (uintptr_t)rwq->buf.buf;
@@ -3278,7 +3285,7 @@ static struct ibv_wq *create_wq(struct ibv_context *context,
 err_create:
 	mlx5_clear_uidx(ctx, cmd.user_index);
 err_free_db_rec:
-	mlx5_free_db(to_mctx(context), rwq->db);
+	mlx5_free_db(to_mctx(context), rwq->db, attr->pd, rwq->custom_db);
 err_free_rwq_buf:
 	mlx5_free_rwq_buf(rwq, context);
 err:
@@ -3342,7 +3349,7 @@ int mlx5_destroy_wq(struct ibv_wq *wq)
 	__mlx5_cq_clean(to_mcq(wq->cq), rwq->rsc.rsn, NULL);
 	mlx5_spin_unlock(&to_mcq(wq->cq)->lock);
 	mlx5_clear_uidx(to_mctx(wq->context), rwq->rsc.rsn);
-	mlx5_free_db(to_mctx(wq->context), rwq->db);
+	mlx5_free_db(to_mctx(wq->context), rwq->db, wq->pd, rwq->custom_db);
 	mlx5_free_rwq_buf(rwq, wq->context);
 	free(rwq);
 
-- 
1.8.3.1

[PATCH rdma-core 6/6] mlx5: Add custom allocation support for SRQ buffer

[Yishai Hadas]

Add custom allocation support for SRQ buffer by extending the internal
allocation flow to consider this option.

As part of this change other options that were missed as of "extern
allocator" became applicable as well.

Signed-off-by: Yishai Hadas <yishaih@mellanox.com>
---
 providers/mlx5/mlx5.h   |  3 ++-
 providers/mlx5/mlx5dv.h |  1 +
 providers/mlx5/srq.c    | 25 ++++++++++++++++++++-----
 providers/mlx5/verbs.c  | 10 +++++-----
 4 files changed, 28 insertions(+), 11 deletions(-)

diff --git a/providers/mlx5/mlx5.h b/providers/mlx5/mlx5.h
index 9a5cd6b..953abe2 100644
--- a/providers/mlx5/mlx5.h
+++ b/providers/mlx5/mlx5.h
@@ -71,6 +71,7 @@ enum {
 #define MLX5_QP_PREFIX "MLX_QP"
 #define MLX5_MR_PREFIX "MLX_MR"
 #define MLX5_RWQ_PREFIX "MLX_RWQ"
+#define MLX5_SRQ_PREFIX "MLX_SRQ"
 #define MLX5_MAX_LOG2_CONTIG_BLOCK_SIZE 23
 #define MLX5_MIN_LOG2_CONTIG_BLOCK_SIZE 12
 
@@ -863,7 +864,7 @@ int mlx5_query_srq(struct ibv_srq *srq,
 			   struct ibv_srq_attr *attr);
 int mlx5_destroy_srq(struct ibv_srq *srq);
 int mlx5_alloc_srq_buf(struct ibv_context *context, struct mlx5_srq *srq,
-		       uint32_t nwr);
+		       uint32_t nwr, struct ibv_pd *pd);
 void mlx5_complete_odp_fault(struct mlx5_srq *srq, int ind);
 void mlx5_free_srq_wqe(struct mlx5_srq *srq, int ind);
 int mlx5_post_srq_recv(struct ibv_srq *ibsrq,
diff --git a/providers/mlx5/mlx5dv.h b/providers/mlx5/mlx5dv.h
index ac291eb..1f49352 100644
--- a/providers/mlx5/mlx5dv.h
+++ b/providers/mlx5/mlx5dv.h
@@ -63,6 +63,7 @@ extern "C" {
 #define MLX5DV_RES_TYPE_QP ((uint64_t)RDMA_DRIVER_MLX5 << 32 | 1)
 #define MLX5DV_RES_TYPE_RWQ ((uint64_t)RDMA_DRIVER_MLX5 << 32 | 2)
 #define MLX5DV_RES_TYPE_DBR ((uint64_t)RDMA_DRIVER_MLX5 << 32 | 3)
+#define MLX5DV_RES_TYPE_SRQ ((uint64_t)RDMA_DRIVER_MLX5 << 32 | 4)
 
 enum {
 	MLX5_RCV_DBR	= 0,
diff --git a/providers/mlx5/srq.c b/providers/mlx5/srq.c
index 1c15656..e9568c6 100644
--- a/providers/mlx5/srq.c
+++ b/providers/mlx5/srq.c
@@ -250,13 +250,14 @@ static void set_srq_buf_ll(struct mlx5_srq *srq, int start, int end)
 }
 
 int mlx5_alloc_srq_buf(struct ibv_context *context, struct mlx5_srq *srq,
-		       uint32_t max_wr)
+		       uint32_t max_wr, struct ibv_pd *pd)
 {
 	int size;
 	int buf_size;
 	struct mlx5_context	   *ctx;
 	uint32_t orig_max_wr = max_wr;
 	bool have_wq = true;
+	enum mlx5_alloc_type alloc_type;
 
 	ctx = to_mctx(context);
 
@@ -296,11 +297,25 @@ int mlx5_alloc_srq_buf(struct ibv_context *context, struct mlx5_srq *srq,
 	srq->max = align_queue_size(max_wr);
 	buf_size = srq->max * size;
 
-	if (mlx5_alloc_buf(&srq->buf, buf_size,
-			   to_mdev(context->device)->page_size))
+	mlx5_get_alloc_type(ctx, pd, MLX5_SRQ_PREFIX, &alloc_type,
+			    MLX5_ALLOC_TYPE_ANON);
+
+	if (alloc_type == MLX5_ALLOC_TYPE_CUSTOM) {
+		srq->buf.mparent_domain = to_mparent_domain(pd);
+		srq->buf.req_alignment = to_mdev(context->device)->page_size;
+		srq->buf.resource_type = MLX5DV_RES_TYPE_SRQ;
+	}
+
+	if (mlx5_alloc_prefered_buf(ctx,
+				    &srq->buf, buf_size,
+				    to_mdev(context->device)->page_size,
+				    alloc_type,
+				    MLX5_SRQ_PREFIX))
 		return -1;
 
-	memset(srq->buf.buf, 0, buf_size);
+	if (srq->buf.type != MLX5_ALLOC_TYPE_CUSTOM)
+		memset(srq->buf.buf, 0, buf_size);
+
 	srq->head = 0;
 	srq->tail = align_queue_size(orig_max_wr + 1) - 1;
 	if (have_wq)  {
@@ -313,7 +328,7 @@ int mlx5_alloc_srq_buf(struct ibv_context *context, struct mlx5_srq *srq,
 
 	srq->wrid = malloc(srq->max * sizeof(*srq->wrid));
 	if (!srq->wrid) {
-		mlx5_free_buf(&srq->buf);
+		mlx5_free_actual_buf(ctx, &srq->buf);
 		return -1;
 	}
 
diff --git a/providers/mlx5/verbs.c b/providers/mlx5/verbs.c
index 1e026af..786a75e 100644
--- a/providers/mlx5/verbs.c
+++ b/providers/mlx5/verbs.c
@@ -1023,7 +1023,7 @@ struct ibv_srq *mlx5_create_srq(struct ibv_pd *pd,
 	srq->max_gs  = attr->attr.max_sge;
 	srq->counter = 0;
 
-	if (mlx5_alloc_srq_buf(pd->context, srq, attr->attr.max_wr)) {
+	if (mlx5_alloc_srq_buf(pd->context, srq, attr->attr.max_wr, pd)) {
 		fprintf(stderr, "%s-%d:\n", __func__, __LINE__);
 		goto err;
 	}
@@ -1082,7 +1082,7 @@ err_db:
 
 err_free:
 	free(srq->wrid);
-	mlx5_free_buf(&srq->buf);
+	mlx5_free_actual_buf(ctx, &srq->buf);
 
 err:
 	free(srq);
@@ -1130,7 +1130,7 @@ int mlx5_destroy_srq(struct ibv_srq *srq)
 		mlx5_clear_srq(ctx, msrq->srqn);
 
 	mlx5_free_db(ctx, msrq->db, srq->pd, msrq->custom_db);
-	mlx5_free_buf(&msrq->buf);
+	mlx5_free_actual_buf(ctx, &msrq->buf);
 	free(msrq->tm_list);
 	free(msrq->wrid);
 	free(msrq->op);
@@ -2890,7 +2890,7 @@ struct ibv_srq *mlx5_create_srq_ex(struct ibv_context *context,
 	msrq->max_gs  = attr->attr.max_sge;
 	msrq->counter = 0;
 
-	if (mlx5_alloc_srq_buf(context, msrq, attr->attr.max_wr)) {
+	if (mlx5_alloc_srq_buf(context, msrq, attr->attr.max_wr, attr->pd)) {
 		fprintf(stderr, "%s-%d:\n", __func__, __LINE__);
 		goto err;
 	}
@@ -2998,7 +2998,7 @@ err_free_db:
 
 err_free:
 	free(msrq->wrid);
-	mlx5_free_buf(&msrq->buf);
+	mlx5_free_actual_buf(ctx, &msrq->buf);
 
 err:
 	free(msrq);
-- 
1.8.3.1

Re: [PATCH rdma-core 0/6] verbs: Custom parent-domain allocators

[Yishai Hadas]

On 10/28/2019 11:14 AM, Yishai Hadas wrote:
> This series extends the parent domain object with custom allocation callbacks
> that can be used by user-applications to override the provider allocation.
> 
> This can be used for example to add NUMA aware allocation.
> 
> The API was introduced in the mailing list by the below RFC [1] and was
> implemented by mlx5 provider.
> 
> A detailed man page exists as part of the series to describe the usage and the
> behavior.
> 
> A PR was sent as well [2].
> 
> [1] https://www.spinics.net/lists/linux-rdma/msg84590.html
> [2] https://github.com/linux-rdma/rdma-core/pull/596
> 
> Haggai Eran (1):
>    verbs: custom parent-domain allocators
> 
> Yishai Hadas (5):
>    Update kernel headers
>    mlx5: Extend mlx5_alloc_parent_domain() to support custom allocator
>    mlx5: Add custom allocation support for QP and RWQ buffers
>    mlx5: Add custom allocation support for DBR
>    mlx5: Add custom allocation support for SRQ buffer
> 
>   kernel-headers/rdma/ib_user_ioctl_verbs.h  |  22 ++++++
>   kernel-headers/rdma/rdma_user_ioctl_cmds.h |  22 ------
>   libibverbs/man/ibv_alloc_parent_domain.3   |  54 +++++++++++++++
>   libibverbs/verbs.h                         |  12 ++++
>   providers/mlx5/buf.c                       |  59 +++++++++++++++++
>   providers/mlx5/cq.c                        |   2 +-
>   providers/mlx5/dbrec.c                     |  34 +++++++++-
>   providers/mlx5/mlx5.h                      |  23 ++++++-
>   providers/mlx5/mlx5dv.h                    |   6 ++
>   providers/mlx5/srq.c                       |  25 +++++--
>   providers/mlx5/verbs.c                     | 103 ++++++++++++++++++++---------
>   11 files changed, 297 insertions(+), 65 deletions(-)
> 

The PR was merged, thanks.

Yishai