* [PATCH rdma-next v2 1/3] RDMA/mana_ib: Access remote atomic for MRs
2025-04-14 9:00 [PATCH rdma-next v2 0/3] RDMA/mana_ib: extend MR support Konstantin Taranov
@ 2025-04-14 9:00 ` Konstantin Taranov
2025-04-16 18:02 ` Long Li
2025-04-14 9:00 ` [PATCH rdma-next v2 2/3] RDMA/mana_ib: support of the zero based MRs Konstantin Taranov
` (2 subsequent siblings)
3 siblings, 1 reply; 8+ messages in thread
From: Konstantin Taranov @ 2025-04-14 9:00 UTC (permalink / raw)
To: kotaranov, pabeni, haiyangz, kys, edumazet, kuba, davem, decui,
wei.liu, longli, jgg, leon
Cc: linux-rdma, linux-kernel, netdev
From: Konstantin Taranov <kotaranov@microsoft.com>
Add IB_ACCESS_REMOTE_ATOMIC to the valid flags for MRs and use
the corresponding flag bit during MR creation in the HW.
Signed-off-by: Konstantin Taranov <kotaranov@microsoft.com>
---
drivers/infiniband/hw/mana/mr.c | 7 +++++--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/drivers/infiniband/hw/mana/mr.c b/drivers/infiniband/hw/mana/mr.c
index f99557e..e4a9f53 100644
--- a/drivers/infiniband/hw/mana/mr.c
+++ b/drivers/infiniband/hw/mana/mr.c
@@ -5,8 +5,8 @@
#include "mana_ib.h"
-#define VALID_MR_FLAGS \
- (IB_ACCESS_LOCAL_WRITE | IB_ACCESS_REMOTE_WRITE | IB_ACCESS_REMOTE_READ)
+#define VALID_MR_FLAGS (IB_ACCESS_LOCAL_WRITE | IB_ACCESS_REMOTE_WRITE | IB_ACCESS_REMOTE_READ |\
+ IB_ACCESS_REMOTE_ATOMIC)
#define VALID_DMA_MR_FLAGS (IB_ACCESS_LOCAL_WRITE)
@@ -24,6 +24,9 @@ mana_ib_verbs_to_gdma_access_flags(int access_flags)
if (access_flags & IB_ACCESS_REMOTE_READ)
flags |= GDMA_ACCESS_FLAG_REMOTE_READ;
+ if (access_flags & IB_ACCESS_REMOTE_ATOMIC)
+ flags |= GDMA_ACCESS_FLAG_REMOTE_ATOMIC;
+
return flags;
}
--
2.43.0
^ permalink raw reply related [flat|nested] 8+ messages in thread* RE: [PATCH rdma-next v2 1/3] RDMA/mana_ib: Access remote atomic for MRs
2025-04-14 9:00 ` [PATCH rdma-next v2 1/3] RDMA/mana_ib: Access remote atomic for MRs Konstantin Taranov
@ 2025-04-16 18:02 ` Long Li
0 siblings, 0 replies; 8+ messages in thread
From: Long Li @ 2025-04-16 18:02 UTC (permalink / raw)
To: Konstantin Taranov, Konstantin Taranov, pabeni@redhat.com,
Haiyang Zhang, KY Srinivasan, edumazet@google.com,
kuba@kernel.org, davem@davemloft.net, Dexuan Cui,
wei.liu@kernel.org, jgg@ziepe.ca, leon@kernel.org
Cc: linux-rdma@vger.kernel.org, linux-kernel@vger.kernel.org,
netdev@vger.kernel.org
> Subject: [PATCH rdma-next v2 1/3] RDMA/mana_ib: Access remote atomic for
> MRs
>
> From: Konstantin Taranov <kotaranov@microsoft.com>
>
> Add IB_ACCESS_REMOTE_ATOMIC to the valid flags for MRs and use the
> corresponding flag bit during MR creation in the HW.
>
> Signed-off-by: Konstantin Taranov <kotaranov@microsoft.com>
Reviewed-by: Long Li <longli@microsoft.com>
> ---
> drivers/infiniband/hw/mana/mr.c | 7 +++++--
> 1 file changed, 5 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/infiniband/hw/mana/mr.c b/drivers/infiniband/hw/mana/mr.c
> index f99557e..e4a9f53 100644
> --- a/drivers/infiniband/hw/mana/mr.c
> +++ b/drivers/infiniband/hw/mana/mr.c
> @@ -5,8 +5,8 @@
>
> #include "mana_ib.h"
>
> -#define VALID_MR_FLAGS \
> - (IB_ACCESS_LOCAL_WRITE | IB_ACCESS_REMOTE_WRITE |
> IB_ACCESS_REMOTE_READ)
> +#define VALID_MR_FLAGS (IB_ACCESS_LOCAL_WRITE |
> IB_ACCESS_REMOTE_WRITE | IB_ACCESS_REMOTE_READ |\
> + IB_ACCESS_REMOTE_ATOMIC)
>
> #define VALID_DMA_MR_FLAGS (IB_ACCESS_LOCAL_WRITE)
>
> @@ -24,6 +24,9 @@ mana_ib_verbs_to_gdma_access_flags(int access_flags)
> if (access_flags & IB_ACCESS_REMOTE_READ)
> flags |= GDMA_ACCESS_FLAG_REMOTE_READ;
>
> + if (access_flags & IB_ACCESS_REMOTE_ATOMIC)
> + flags |= GDMA_ACCESS_FLAG_REMOTE_ATOMIC;
> +
> return flags;
> }
>
> --
> 2.43.0
^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH rdma-next v2 2/3] RDMA/mana_ib: support of the zero based MRs
2025-04-14 9:00 [PATCH rdma-next v2 0/3] RDMA/mana_ib: extend MR support Konstantin Taranov
2025-04-14 9:00 ` [PATCH rdma-next v2 1/3] RDMA/mana_ib: Access remote atomic for MRs Konstantin Taranov
@ 2025-04-14 9:00 ` Konstantin Taranov
2025-04-16 18:09 ` Long Li
2025-04-14 9:00 ` [PATCH rdma-next v2 3/3] RDMA/mana_ib: Add support of 4M, 1G, and 2G pages Konstantin Taranov
2025-04-20 10:36 ` [PATCH rdma-next v2 0/3] RDMA/mana_ib: extend MR support Leon Romanovsky
3 siblings, 1 reply; 8+ messages in thread
From: Konstantin Taranov @ 2025-04-14 9:00 UTC (permalink / raw)
To: kotaranov, pabeni, haiyangz, kys, edumazet, kuba, davem, decui,
wei.liu, longli, jgg, leon
Cc: linux-rdma, linux-kernel, netdev
From: Konstantin Taranov <kotaranov@microsoft.com>
Add IB_ZERO_BASED to the valid flags and use
the corresponding MR creation request for the zero
based memory.
Signed-off-by: Konstantin Taranov <kotaranov@microsoft.com>
---
drivers/infiniband/hw/mana/mr.c | 24 +++++++++++++++++-------
include/net/mana/gdma.h | 11 ++++++++++-
2 files changed, 27 insertions(+), 8 deletions(-)
diff --git a/drivers/infiniband/hw/mana/mr.c b/drivers/infiniband/hw/mana/mr.c
index e4a9f53..6d974d0 100644
--- a/drivers/infiniband/hw/mana/mr.c
+++ b/drivers/infiniband/hw/mana/mr.c
@@ -6,7 +6,7 @@
#include "mana_ib.h"
#define VALID_MR_FLAGS (IB_ACCESS_LOCAL_WRITE | IB_ACCESS_REMOTE_WRITE | IB_ACCESS_REMOTE_READ |\
- IB_ACCESS_REMOTE_ATOMIC)
+ IB_ACCESS_REMOTE_ATOMIC | IB_ZERO_BASED)
#define VALID_DMA_MR_FLAGS (IB_ACCESS_LOCAL_WRITE)
@@ -51,7 +51,10 @@ static int mana_ib_gd_create_mr(struct mana_ib_dev *dev, struct mana_ib_mr *mr,
req.gva.virtual_address = mr_params->gva.virtual_address;
req.gva.access_flags = mr_params->gva.access_flags;
break;
-
+ case GDMA_MR_TYPE_ZBVA:
+ req.zbva.dma_region_handle = mr_params->zbva.dma_region_handle;
+ req.zbva.access_flags = mr_params->zbva.access_flags;
+ break;
default:
ibdev_dbg(&dev->ib_dev,
"invalid param (GDMA_MR_TYPE) passed, type %d\n",
@@ -147,11 +150,18 @@ struct ib_mr *mana_ib_reg_user_mr(struct ib_pd *ibpd, u64 start, u64 length,
dma_region_handle);
mr_params.pd_handle = pd->pd_handle;
- mr_params.mr_type = GDMA_MR_TYPE_GVA;
- mr_params.gva.dma_region_handle = dma_region_handle;
- mr_params.gva.virtual_address = iova;
- mr_params.gva.access_flags =
- mana_ib_verbs_to_gdma_access_flags(access_flags);
+ if (access_flags & IB_ZERO_BASED) {
+ mr_params.mr_type = GDMA_MR_TYPE_ZBVA;
+ mr_params.zbva.dma_region_handle = dma_region_handle;
+ mr_params.zbva.access_flags =
+ mana_ib_verbs_to_gdma_access_flags(access_flags);
+ } else {
+ mr_params.mr_type = GDMA_MR_TYPE_GVA;
+ mr_params.gva.dma_region_handle = dma_region_handle;
+ mr_params.gva.virtual_address = iova;
+ mr_params.gva.access_flags =
+ mana_ib_verbs_to_gdma_access_flags(access_flags);
+ }
err = mana_ib_gd_create_mr(dev, mr, &mr_params);
if (err)
diff --git a/include/net/mana/gdma.h b/include/net/mana/gdma.h
index 50ffbc4..3db506d 100644
--- a/include/net/mana/gdma.h
+++ b/include/net/mana/gdma.h
@@ -812,6 +812,8 @@ enum gdma_mr_type {
* address that is set up in the MST
*/
GDMA_MR_TYPE_GVA = 2,
+ /* Guest zero-based address MRs */
+ GDMA_MR_TYPE_ZBVA = 4,
};
struct gdma_create_mr_params {
@@ -823,6 +825,10 @@ struct gdma_create_mr_params {
u64 virtual_address;
enum gdma_mr_access_flags access_flags;
} gva;
+ struct {
+ u64 dma_region_handle;
+ enum gdma_mr_access_flags access_flags;
+ } zbva;
};
};
@@ -838,7 +844,10 @@ struct gdma_create_mr_request {
u64 virtual_address;
enum gdma_mr_access_flags access_flags;
} gva;
-
+ struct {
+ u64 dma_region_handle;
+ enum gdma_mr_access_flags access_flags;
+ } zbva;
};
u32 reserved_2;
};/* HW DATA */
--
2.43.0
^ permalink raw reply related [flat|nested] 8+ messages in thread* RE: [PATCH rdma-next v2 2/3] RDMA/mana_ib: support of the zero based MRs
2025-04-14 9:00 ` [PATCH rdma-next v2 2/3] RDMA/mana_ib: support of the zero based MRs Konstantin Taranov
@ 2025-04-16 18:09 ` Long Li
0 siblings, 0 replies; 8+ messages in thread
From: Long Li @ 2025-04-16 18:09 UTC (permalink / raw)
To: Konstantin Taranov, Konstantin Taranov, pabeni@redhat.com,
Haiyang Zhang, KY Srinivasan, edumazet@google.com,
kuba@kernel.org, davem@davemloft.net, Dexuan Cui,
wei.liu@kernel.org, jgg@ziepe.ca, leon@kernel.org
Cc: linux-rdma@vger.kernel.org, linux-kernel@vger.kernel.org,
netdev@vger.kernel.org
> Subject: [PATCH rdma-next v2 2/3] RDMA/mana_ib: support of the zero based
> MRs
>
> From: Konstantin Taranov <kotaranov@microsoft.com>
>
> Add IB_ZERO_BASED to the valid flags and use the corresponding MR creation
> request for the zero based memory.
>
> Signed-off-by: Konstantin Taranov <kotaranov@microsoft.com>
Reviewed-by: Long Li <longli@microsoft.com>
> ---
> drivers/infiniband/hw/mana/mr.c | 24 +++++++++++++++++-------
> include/net/mana/gdma.h | 11 ++++++++++-
> 2 files changed, 27 insertions(+), 8 deletions(-)
>
> diff --git a/drivers/infiniband/hw/mana/mr.c b/drivers/infiniband/hw/mana/mr.c
> index e4a9f53..6d974d0 100644
> --- a/drivers/infiniband/hw/mana/mr.c
> +++ b/drivers/infiniband/hw/mana/mr.c
> @@ -6,7 +6,7 @@
> #include "mana_ib.h"
>
> #define VALID_MR_FLAGS (IB_ACCESS_LOCAL_WRITE |
> IB_ACCESS_REMOTE_WRITE | IB_ACCESS_REMOTE_READ |\
> - IB_ACCESS_REMOTE_ATOMIC)
> + IB_ACCESS_REMOTE_ATOMIC | IB_ZERO_BASED)
>
> #define VALID_DMA_MR_FLAGS (IB_ACCESS_LOCAL_WRITE)
>
> @@ -51,7 +51,10 @@ static int mana_ib_gd_create_mr(struct mana_ib_dev
> *dev, struct mana_ib_mr *mr,
> req.gva.virtual_address = mr_params->gva.virtual_address;
> req.gva.access_flags = mr_params->gva.access_flags;
> break;
> -
> + case GDMA_MR_TYPE_ZBVA:
> + req.zbva.dma_region_handle = mr_params-
> >zbva.dma_region_handle;
> + req.zbva.access_flags = mr_params->zbva.access_flags;
> + break;
> default:
> ibdev_dbg(&dev->ib_dev,
> "invalid param (GDMA_MR_TYPE) passed, type %d\n",
> @@ -147,11 +150,18 @@ struct ib_mr *mana_ib_reg_user_mr(struct ib_pd
> *ibpd, u64 start, u64 length,
> dma_region_handle);
>
> mr_params.pd_handle = pd->pd_handle;
> - mr_params.mr_type = GDMA_MR_TYPE_GVA;
> - mr_params.gva.dma_region_handle = dma_region_handle;
> - mr_params.gva.virtual_address = iova;
> - mr_params.gva.access_flags =
> - mana_ib_verbs_to_gdma_access_flags(access_flags);
> + if (access_flags & IB_ZERO_BASED) {
> + mr_params.mr_type = GDMA_MR_TYPE_ZBVA;
> + mr_params.zbva.dma_region_handle = dma_region_handle;
> + mr_params.zbva.access_flags =
> + mana_ib_verbs_to_gdma_access_flags(access_flags);
> + } else {
> + mr_params.mr_type = GDMA_MR_TYPE_GVA;
> + mr_params.gva.dma_region_handle = dma_region_handle;
> + mr_params.gva.virtual_address = iova;
> + mr_params.gva.access_flags =
> + mana_ib_verbs_to_gdma_access_flags(access_flags);
> + }
>
> err = mana_ib_gd_create_mr(dev, mr, &mr_params);
> if (err)
> diff --git a/include/net/mana/gdma.h b/include/net/mana/gdma.h index
> 50ffbc4..3db506d 100644
> --- a/include/net/mana/gdma.h
> +++ b/include/net/mana/gdma.h
> @@ -812,6 +812,8 @@ enum gdma_mr_type {
> * address that is set up in the MST
> */
> GDMA_MR_TYPE_GVA = 2,
> + /* Guest zero-based address MRs */
> + GDMA_MR_TYPE_ZBVA = 4,
> };
>
> struct gdma_create_mr_params {
> @@ -823,6 +825,10 @@ struct gdma_create_mr_params {
> u64 virtual_address;
> enum gdma_mr_access_flags access_flags;
> } gva;
> + struct {
> + u64 dma_region_handle;
> + enum gdma_mr_access_flags access_flags;
> + } zbva;
> };
> };
>
> @@ -838,7 +844,10 @@ struct gdma_create_mr_request {
> u64 virtual_address;
> enum gdma_mr_access_flags access_flags;
> } gva;
> -
> + struct {
> + u64 dma_region_handle;
> + enum gdma_mr_access_flags access_flags;
> + } zbva;
> };
> u32 reserved_2;
> };/* HW DATA */
> --
> 2.43.0
^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH rdma-next v2 3/3] RDMA/mana_ib: Add support of 4M, 1G, and 2G pages
2025-04-14 9:00 [PATCH rdma-next v2 0/3] RDMA/mana_ib: extend MR support Konstantin Taranov
2025-04-14 9:00 ` [PATCH rdma-next v2 1/3] RDMA/mana_ib: Access remote atomic for MRs Konstantin Taranov
2025-04-14 9:00 ` [PATCH rdma-next v2 2/3] RDMA/mana_ib: support of the zero based MRs Konstantin Taranov
@ 2025-04-14 9:00 ` Konstantin Taranov
2025-04-16 18:31 ` Long Li
2025-04-20 10:36 ` [PATCH rdma-next v2 0/3] RDMA/mana_ib: extend MR support Leon Romanovsky
3 siblings, 1 reply; 8+ messages in thread
From: Konstantin Taranov @ 2025-04-14 9:00 UTC (permalink / raw)
To: kotaranov, pabeni, haiyangz, kys, edumazet, kuba, davem, decui,
wei.liu, longli, jgg, leon
Cc: linux-rdma, linux-kernel, netdev
From: Konstantin Taranov <kotaranov@microsoft.com>
Check PF capability flag whether the 4M, 1G, and 2G pages are
supported. Add these pages sizes to mana_ib, if supported.
Define possible page sizes in enum gdma_page_type and
remove unused enum atb_page_size.
Signed-off-by: Konstantin Taranov <kotaranov@microsoft.com>
---
drivers/infiniband/hw/mana/main.c | 10 +++++++---
drivers/infiniband/hw/mana/mana_ib.h | 1 +
drivers/net/ethernet/microsoft/mana/gdma_main.c | 1 +
include/net/mana/gdma.h | 17 +++--------------
4 files changed, 12 insertions(+), 17 deletions(-)
diff --git a/drivers/infiniband/hw/mana/main.c b/drivers/infiniband/hw/mana/main.c
index 730f958..a28b712 100644
--- a/drivers/infiniband/hw/mana/main.c
+++ b/drivers/infiniband/hw/mana/main.c
@@ -479,7 +479,7 @@ int mana_ib_create_dma_region(struct mana_ib_dev *dev, struct ib_umem *umem,
{
unsigned long page_sz;
- page_sz = ib_umem_find_best_pgsz(umem, PAGE_SZ_BM, virt);
+ page_sz = ib_umem_find_best_pgsz(umem, dev->adapter_caps.page_size_cap, virt);
if (!page_sz) {
ibdev_dbg(&dev->ib_dev, "Failed to find page size.\n");
return -EINVAL;
@@ -494,7 +494,7 @@ int mana_ib_create_zero_offset_dma_region(struct mana_ib_dev *dev, struct ib_ume
unsigned long page_sz;
/* Hardware requires dma region to align to chosen page size */
- page_sz = ib_umem_find_best_pgoff(umem, PAGE_SZ_BM, 0);
+ page_sz = ib_umem_find_best_pgoff(umem, dev->adapter_caps.page_size_cap, 0);
if (!page_sz) {
ibdev_dbg(&dev->ib_dev, "Failed to find page size.\n");
return -EINVAL;
@@ -577,7 +577,7 @@ int mana_ib_query_device(struct ib_device *ibdev, struct ib_device_attr *props,
memset(props, 0, sizeof(*props));
props->max_mr_size = MANA_IB_MAX_MR_SIZE;
- props->page_size_cap = PAGE_SZ_BM;
+ props->page_size_cap = dev->adapter_caps.page_size_cap;
props->max_qp = dev->adapter_caps.max_qp_count;
props->max_qp_wr = dev->adapter_caps.max_qp_wr;
props->device_cap_flags = IB_DEVICE_RC_RNR_NAK_GEN;
@@ -696,6 +696,10 @@ int mana_ib_gd_query_adapter_caps(struct mana_ib_dev *dev)
caps->max_recv_sge_count = resp.max_recv_sge_count;
caps->feature_flags = resp.feature_flags;
+ caps->page_size_cap = PAGE_SZ_BM;
+ if (mdev_to_gc(dev)->pf_cap_flags1 & GDMA_DRV_CAP_FLAG_1_GDMA_PAGES_4MB_1GB_2GB)
+ caps->page_size_cap |= (SZ_4M | SZ_1G | SZ_2G);
+
return 0;
}
diff --git a/drivers/infiniband/hw/mana/mana_ib.h b/drivers/infiniband/hw/mana/mana_ib.h
index 6903946..f0dbd90 100644
--- a/drivers/infiniband/hw/mana/mana_ib.h
+++ b/drivers/infiniband/hw/mana/mana_ib.h
@@ -60,6 +60,7 @@ struct mana_ib_adapter_caps {
u32 max_recv_sge_count;
u32 max_inline_data_size;
u64 feature_flags;
+ u64 page_size_cap;
};
struct mana_ib_queue {
diff --git a/drivers/net/ethernet/microsoft/mana/gdma_main.c b/drivers/net/ethernet/microsoft/mana/gdma_main.c
index 4a2b17f..b5156d4 100644
--- a/drivers/net/ethernet/microsoft/mana/gdma_main.c
+++ b/drivers/net/ethernet/microsoft/mana/gdma_main.c
@@ -937,6 +937,7 @@ int mana_gd_verify_vf_version(struct pci_dev *pdev)
err, resp.hdr.status);
return err ? err : -EPROTO;
}
+ gc->pf_cap_flags1 = resp.pf_cap_flags1;
if (resp.pf_cap_flags1 & GDMA_DRV_CAP_FLAG_1_HWC_TIMEOUT_RECONFIG) {
err = mana_gd_query_hwc_timeout(pdev, &hwc->hwc_timeout);
if (err) {
diff --git a/include/net/mana/gdma.h b/include/net/mana/gdma.h
index 3db506d..89abf98 100644
--- a/include/net/mana/gdma.h
+++ b/include/net/mana/gdma.h
@@ -407,6 +407,8 @@ struct gdma_context {
/* Azure RDMA adapter */
struct gdma_dev mana_ib;
+
+ u64 pf_cap_flags1;
};
#define MAX_NUM_GDMA_DEVICES 4
@@ -556,6 +558,7 @@ enum {
#define GDMA_DRV_CAP_FLAG_1_NAPI_WKDONE_FIX BIT(2)
#define GDMA_DRV_CAP_FLAG_1_HWC_TIMEOUT_RECONFIG BIT(3)
#define GDMA_DRV_CAP_FLAG_1_VARIABLE_INDIRECTION_TABLE_SUPPORT BIT(5)
+#define GDMA_DRV_CAP_FLAG_1_GDMA_PAGES_4MB_1GB_2GB BIT(4)
#define GDMA_DRV_CAP_FLAGS1 \
(GDMA_DRV_CAP_FLAG_1_EQ_SHARING_MULTI_VPORT | \
@@ -704,20 +707,6 @@ struct gdma_query_hwc_timeout_resp {
u32 reserved;
};
-enum atb_page_size {
- ATB_PAGE_SIZE_4K,
- ATB_PAGE_SIZE_8K,
- ATB_PAGE_SIZE_16K,
- ATB_PAGE_SIZE_32K,
- ATB_PAGE_SIZE_64K,
- ATB_PAGE_SIZE_128K,
- ATB_PAGE_SIZE_256K,
- ATB_PAGE_SIZE_512K,
- ATB_PAGE_SIZE_1M,
- ATB_PAGE_SIZE_2M,
- ATB_PAGE_SIZE_MAX,
-};
-
enum gdma_mr_access_flags {
GDMA_ACCESS_FLAG_LOCAL_READ = BIT_ULL(0),
GDMA_ACCESS_FLAG_LOCAL_WRITE = BIT_ULL(1),
--
2.43.0
^ permalink raw reply related [flat|nested] 8+ messages in thread* RE: [PATCH rdma-next v2 3/3] RDMA/mana_ib: Add support of 4M, 1G, and 2G pages
2025-04-14 9:00 ` [PATCH rdma-next v2 3/3] RDMA/mana_ib: Add support of 4M, 1G, and 2G pages Konstantin Taranov
@ 2025-04-16 18:31 ` Long Li
0 siblings, 0 replies; 8+ messages in thread
From: Long Li @ 2025-04-16 18:31 UTC (permalink / raw)
To: Konstantin Taranov, Konstantin Taranov, pabeni@redhat.com,
Haiyang Zhang, KY Srinivasan, edumazet@google.com,
kuba@kernel.org, davem@davemloft.net, Dexuan Cui,
wei.liu@kernel.org, jgg@ziepe.ca, leon@kernel.org
Cc: linux-rdma@vger.kernel.org, linux-kernel@vger.kernel.org,
netdev@vger.kernel.org
> Subject: [PATCH rdma-next v2 3/3] RDMA/mana_ib: Add support of 4M, 1G, and
> 2G pages
>
> From: Konstantin Taranov <kotaranov@microsoft.com>
>
> Check PF capability flag whether the 4M, 1G, and 2G pages are supported. Add
> these pages sizes to mana_ib, if supported.
>
> Define possible page sizes in enum gdma_page_type and remove unused enum
> atb_page_size.
>
> Signed-off-by: Konstantin Taranov <kotaranov@microsoft.com>
Reviewed-by: Long Li <longli@microsoft.com>
> ---
> drivers/infiniband/hw/mana/main.c | 10 +++++++---
> drivers/infiniband/hw/mana/mana_ib.h | 1 +
> drivers/net/ethernet/microsoft/mana/gdma_main.c | 1 +
> include/net/mana/gdma.h | 17 +++--------------
> 4 files changed, 12 insertions(+), 17 deletions(-)
>
> diff --git a/drivers/infiniband/hw/mana/main.c
> b/drivers/infiniband/hw/mana/main.c
> index 730f958..a28b712 100644
> --- a/drivers/infiniband/hw/mana/main.c
> +++ b/drivers/infiniband/hw/mana/main.c
> @@ -479,7 +479,7 @@ int mana_ib_create_dma_region(struct mana_ib_dev
> *dev, struct ib_umem *umem, {
> unsigned long page_sz;
>
> - page_sz = ib_umem_find_best_pgsz(umem, PAGE_SZ_BM, virt);
> + page_sz = ib_umem_find_best_pgsz(umem,
> +dev->adapter_caps.page_size_cap, virt);
> if (!page_sz) {
> ibdev_dbg(&dev->ib_dev, "Failed to find page size.\n");
> return -EINVAL;
> @@ -494,7 +494,7 @@ int mana_ib_create_zero_offset_dma_region(struct
> mana_ib_dev *dev, struct ib_ume
> unsigned long page_sz;
>
> /* Hardware requires dma region to align to chosen page size */
> - page_sz = ib_umem_find_best_pgoff(umem, PAGE_SZ_BM, 0);
> + page_sz = ib_umem_find_best_pgoff(umem,
> +dev->adapter_caps.page_size_cap, 0);
> if (!page_sz) {
> ibdev_dbg(&dev->ib_dev, "Failed to find page size.\n");
> return -EINVAL;
> @@ -577,7 +577,7 @@ int mana_ib_query_device(struct ib_device *ibdev, struct
> ib_device_attr *props,
>
> memset(props, 0, sizeof(*props));
> props->max_mr_size = MANA_IB_MAX_MR_SIZE;
> - props->page_size_cap = PAGE_SZ_BM;
> + props->page_size_cap = dev->adapter_caps.page_size_cap;
> props->max_qp = dev->adapter_caps.max_qp_count;
> props->max_qp_wr = dev->adapter_caps.max_qp_wr;
> props->device_cap_flags = IB_DEVICE_RC_RNR_NAK_GEN; @@ -696,6
> +696,10 @@ int mana_ib_gd_query_adapter_caps(struct mana_ib_dev *dev)
> caps->max_recv_sge_count = resp.max_recv_sge_count;
> caps->feature_flags = resp.feature_flags;
>
> + caps->page_size_cap = PAGE_SZ_BM;
> + if (mdev_to_gc(dev)->pf_cap_flags1 &
> GDMA_DRV_CAP_FLAG_1_GDMA_PAGES_4MB_1GB_2GB)
> + caps->page_size_cap |= (SZ_4M | SZ_1G | SZ_2G);
> +
> return 0;
> }
>
> diff --git a/drivers/infiniband/hw/mana/mana_ib.h
> b/drivers/infiniband/hw/mana/mana_ib.h
> index 6903946..f0dbd90 100644
> --- a/drivers/infiniband/hw/mana/mana_ib.h
> +++ b/drivers/infiniband/hw/mana/mana_ib.h
> @@ -60,6 +60,7 @@ struct mana_ib_adapter_caps {
> u32 max_recv_sge_count;
> u32 max_inline_data_size;
> u64 feature_flags;
> + u64 page_size_cap;
> };
>
> struct mana_ib_queue {
> diff --git a/drivers/net/ethernet/microsoft/mana/gdma_main.c
> b/drivers/net/ethernet/microsoft/mana/gdma_main.c
> index 4a2b17f..b5156d4 100644
> --- a/drivers/net/ethernet/microsoft/mana/gdma_main.c
> +++ b/drivers/net/ethernet/microsoft/mana/gdma_main.c
> @@ -937,6 +937,7 @@ int mana_gd_verify_vf_version(struct pci_dev *pdev)
> err, resp.hdr.status);
> return err ? err : -EPROTO;
> }
> + gc->pf_cap_flags1 = resp.pf_cap_flags1;
> if (resp.pf_cap_flags1 &
> GDMA_DRV_CAP_FLAG_1_HWC_TIMEOUT_RECONFIG) {
> err = mana_gd_query_hwc_timeout(pdev, &hwc-
> >hwc_timeout);
> if (err) {
> diff --git a/include/net/mana/gdma.h b/include/net/mana/gdma.h index
> 3db506d..89abf98 100644
> --- a/include/net/mana/gdma.h
> +++ b/include/net/mana/gdma.h
> @@ -407,6 +407,8 @@ struct gdma_context {
>
> /* Azure RDMA adapter */
> struct gdma_dev mana_ib;
> +
> + u64 pf_cap_flags1;
> };
>
> #define MAX_NUM_GDMA_DEVICES 4
> @@ -556,6 +558,7 @@ enum {
> #define GDMA_DRV_CAP_FLAG_1_NAPI_WKDONE_FIX BIT(2) #define
> GDMA_DRV_CAP_FLAG_1_HWC_TIMEOUT_RECONFIG BIT(3) #define
> GDMA_DRV_CAP_FLAG_1_VARIABLE_INDIRECTION_TABLE_SUPPORT BIT(5)
> +#define GDMA_DRV_CAP_FLAG_1_GDMA_PAGES_4MB_1GB_2GB BIT(4)
>
> #define GDMA_DRV_CAP_FLAGS1 \
> (GDMA_DRV_CAP_FLAG_1_EQ_SHARING_MULTI_VPORT | \ @@ -
> 704,20 +707,6 @@ struct gdma_query_hwc_timeout_resp {
> u32 reserved;
> };
>
> -enum atb_page_size {
> - ATB_PAGE_SIZE_4K,
> - ATB_PAGE_SIZE_8K,
> - ATB_PAGE_SIZE_16K,
> - ATB_PAGE_SIZE_32K,
> - ATB_PAGE_SIZE_64K,
> - ATB_PAGE_SIZE_128K,
> - ATB_PAGE_SIZE_256K,
> - ATB_PAGE_SIZE_512K,
> - ATB_PAGE_SIZE_1M,
> - ATB_PAGE_SIZE_2M,
> - ATB_PAGE_SIZE_MAX,
> -};
> -
> enum gdma_mr_access_flags {
> GDMA_ACCESS_FLAG_LOCAL_READ = BIT_ULL(0),
> GDMA_ACCESS_FLAG_LOCAL_WRITE = BIT_ULL(1),
> --
> 2.43.0
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH rdma-next v2 0/3] RDMA/mana_ib: extend MR support
2025-04-14 9:00 [PATCH rdma-next v2 0/3] RDMA/mana_ib: extend MR support Konstantin Taranov
` (2 preceding siblings ...)
2025-04-14 9:00 ` [PATCH rdma-next v2 3/3] RDMA/mana_ib: Add support of 4M, 1G, and 2G pages Konstantin Taranov
@ 2025-04-20 10:36 ` Leon Romanovsky
3 siblings, 0 replies; 8+ messages in thread
From: Leon Romanovsky @ 2025-04-20 10:36 UTC (permalink / raw)
To: kotaranov, pabeni, haiyangz, kys, edumazet, kuba, davem, decui,
wei.liu, longli, jgg, Konstantin Taranov
Cc: linux-rdma, linux-kernel, netdev
On Mon, 14 Apr 2025 02:00:31 -0700, Konstantin Taranov wrote:
> This patch series extends MR support for mana.
> It implements two flags for MRs: REMOTE_ATOMIC and ZERO_BASED
> It also adds support for large page sizes.
>
> v1->v2:
> - removed unused enum values for page sizes
>
> [...]
Applied, thanks!
[1/3] RDMA/mana_ib: Access remote atomic for MRs
https://git.kernel.org/rdma/rdma/c/7d40ccf018694a
[2/3] RDMA/mana_ib: support of the zero based MRs
https://git.kernel.org/rdma/rdma/c/8f49682d94f3a1
[3/3] RDMA/mana_ib: Add support of 4M, 1G, and 2G pages
https://git.kernel.org/rdma/rdma/c/f1652d76f4c51b
Best regards,
--
Leon Romanovsky <leon@kernel.org>
^ permalink raw reply [flat|nested] 8+ messages in thread