[PATCH rdma-next 0/3] RDMA/mana_ib: extend MR support

[PATCH rdma-next 0/3] RDMA/mana_ib: extend MR support

[Konstantin Taranov]

From: Konstantin Taranov <kotaranov@microsoft.com>

This patch series extends MR support for mana.
It implements two flags for MRs: REMOTE_ATOMIC and ZERO_BASED
It also adds support for large page sizes.

Konstantin Taranov (3):
  RDMA/mana_ib: Access remote atomic for MRs
  RDMA/mana_ib: support of the zero based MRs
  RDMA/mana_ib: Add support of 4M, 1G, and 2G pages

 drivers/infiniband/hw/mana/main.c             | 10 +++--
 drivers/infiniband/hw/mana/mana_ib.h          |  1 +
 drivers/infiniband/hw/mana/mr.c               | 29 +++++++++----
 .../net/ethernet/microsoft/mana/gdma_main.c   |  1 +
 include/net/mana/gdma.h                       | 41 ++++++++++++-------
 5 files changed, 56 insertions(+), 26 deletions(-)

-- 
2.43.0

[PATCH rdma-next 1/3] RDMA/mana_ib: Access remote atomic for MRs

[Konstantin Taranov]

From: Konstantin Taranov <kotaranov@microsoft.com>

Add IB_ACCESS_REMOTE_ATOMIC to the valid flags for MRs and use
the corresponding flag bit during MR creation in the HW.

Signed-off-by: Konstantin Taranov <kotaranov@microsoft.com>
---
 drivers/infiniband/hw/mana/mr.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/drivers/infiniband/hw/mana/mr.c b/drivers/infiniband/hw/mana/mr.c
index f99557e..e4a9f53 100644
--- a/drivers/infiniband/hw/mana/mr.c
+++ b/drivers/infiniband/hw/mana/mr.c
@@ -5,8 +5,8 @@
 
 #include "mana_ib.h"
 
-#define VALID_MR_FLAGS                                                         \
-	(IB_ACCESS_LOCAL_WRITE | IB_ACCESS_REMOTE_WRITE | IB_ACCESS_REMOTE_READ)
+#define VALID_MR_FLAGS (IB_ACCESS_LOCAL_WRITE | IB_ACCESS_REMOTE_WRITE | IB_ACCESS_REMOTE_READ |\
+			IB_ACCESS_REMOTE_ATOMIC)
 
 #define VALID_DMA_MR_FLAGS (IB_ACCESS_LOCAL_WRITE)
 
@@ -24,6 +24,9 @@ mana_ib_verbs_to_gdma_access_flags(int access_flags)
 	if (access_flags & IB_ACCESS_REMOTE_READ)
 		flags |= GDMA_ACCESS_FLAG_REMOTE_READ;
 
+	if (access_flags & IB_ACCESS_REMOTE_ATOMIC)
+		flags |= GDMA_ACCESS_FLAG_REMOTE_ATOMIC;
+
 	return flags;
 }
 
-- 
2.43.0

Re: [PATCH rdma-next 1/3] RDMA/mana_ib: Access remote atomic for MRs

[Leon Romanovsky]

On Fri, Apr 04, 2025 at 07:45:53AM -0700, Konstantin Taranov wrote:
> From: Konstantin Taranov <kotaranov@microsoft.com>
> 
> Add IB_ACCESS_REMOTE_ATOMIC to the valid flags for MRs and use
> the corresponding flag bit during MR creation in the HW.
> 
> Signed-off-by: Konstantin Taranov <kotaranov@microsoft.com>
> ---
>  drivers/infiniband/hw/mana/mr.c | 7 +++++--
>  1 file changed, 5 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/infiniband/hw/mana/mr.c b/drivers/infiniband/hw/mana/mr.c
> index f99557e..e4a9f53 100644
> --- a/drivers/infiniband/hw/mana/mr.c
> +++ b/drivers/infiniband/hw/mana/mr.c
> @@ -5,8 +5,8 @@
>  
>  #include "mana_ib.h"
>  
> -#define VALID_MR_FLAGS                                                         \
> -	(IB_ACCESS_LOCAL_WRITE | IB_ACCESS_REMOTE_WRITE | IB_ACCESS_REMOTE_READ)
> +#define VALID_MR_FLAGS (IB_ACCESS_LOCAL_WRITE | IB_ACCESS_REMOTE_WRITE | IB_ACCESS_REMOTE_READ |\
> +			IB_ACCESS_REMOTE_ATOMIC)
>  
>  #define VALID_DMA_MR_FLAGS (IB_ACCESS_LOCAL_WRITE)
>  
> @@ -24,6 +24,9 @@ mana_ib_verbs_to_gdma_access_flags(int access_flags)
>  	if (access_flags & IB_ACCESS_REMOTE_READ)
>  		flags |= GDMA_ACCESS_FLAG_REMOTE_READ;
>  
> +	if (access_flags & IB_ACCESS_REMOTE_ATOMIC)
> +		flags |= GDMA_ACCESS_FLAG_REMOTE_ATOMIC;

Can you enable this flag unconditionally without relation to running RW?

Thanks

> +
>  	return flags;
>  }
>  
> -- 
> 2.43.0
> 

RE: [EXTERNAL] Re: [PATCH rdma-next 1/3] RDMA/mana_ib: Access remote atomic for MRs

[Konstantin Taranov]

> > @@ -24,6 +24,9 @@ mana_ib_verbs_to_gdma_access_flags(int
> access_flags)
> >  	if (access_flags & IB_ACCESS_REMOTE_READ)
> >  		flags |= GDMA_ACCESS_FLAG_REMOTE_READ;
> >
> > +	if (access_flags & IB_ACCESS_REMOTE_ATOMIC)
> > +		flags |= GDMA_ACCESS_FLAG_REMOTE_ATOMIC;
> 
> Can you enable this flag unconditionally without relation to running RW?

Yes, ATOMIC access does not depend on Remote Read and Remote Write.
I also do not see any conditions in other drivers.

- Konstantin

> 
> Thanks
> 
> > +
> >  	return flags;
> >  }
> >
> > --
> > 2.43.0
> >

Re: [EXTERNAL] Re: [PATCH rdma-next 1/3] RDMA/mana_ib: Access remote atomic for MRs

[Leon Romanovsky]

On Thu, Apr 10, 2025 at 01:37:00PM +0000, Konstantin Taranov wrote:
> > > @@ -24,6 +24,9 @@ mana_ib_verbs_to_gdma_access_flags(int
> > access_flags)
> > >  	if (access_flags & IB_ACCESS_REMOTE_READ)
> > >  		flags |= GDMA_ACCESS_FLAG_REMOTE_READ;
> > >
> > > +	if (access_flags & IB_ACCESS_REMOTE_ATOMIC)
> > > +		flags |= GDMA_ACCESS_FLAG_REMOTE_ATOMIC;
> > 
> > Can you enable this flag unconditionally without relation to running RW?
> 
> Yes, ATOMIC access does not depend on Remote Read and Remote Write.

The question is "do you have FW which doesn't support
GDMA_ACCESS_FLAG_REMOTE_ATOMIC? and what will happen if such flag used
for such FW?"

> I also do not see any conditions in other drivers.

At least for mlx5, we are checking FW capability if it is supported.
See get_unchangeable_access_flags().

> 
> - Konstantin
> 
> > 
> > Thanks
> > 
> > > +
> > >  	return flags;
> > >  }
> > >
> > > --
> > > 2.43.0
> > >

RE: [EXTERNAL] Re: [PATCH rdma-next 1/3] RDMA/mana_ib: Access remote atomic for MRs

[Konstantin Taranov]

> On Thu, Apr 10, 2025 at 01:37:00PM +0000, Konstantin Taranov wrote:
> > > > @@ -24,6 +24,9 @@ mana_ib_verbs_to_gdma_access_flags(int
> > > access_flags)
> > > >  	if (access_flags & IB_ACCESS_REMOTE_READ)
> > > >  		flags |= GDMA_ACCESS_FLAG_REMOTE_READ;
> > > >
> > > > +	if (access_flags & IB_ACCESS_REMOTE_ATOMIC)
> > > > +		flags |= GDMA_ACCESS_FLAG_REMOTE_ATOMIC;
> > >
> > > Can you enable this flag unconditionally without relation to running RW?
> >
> > Yes, ATOMIC access does not depend on Remote Read and Remote Write.
> 
> The question is "do you have FW which doesn't support
> GDMA_ACCESS_FLAG_REMOTE_ATOMIC? and what will happen if such flag
> used for such FW?"

As far as I am aware, all FW support this flag.
Nonetheless, if FW does not support it, the HW request to create MR will fail.

> 
> > I also do not see any conditions in other drivers.
> 
> At least for mlx5, we are checking FW capability if it is supported.
> See get_unchangeable_access_flags().
> 
> >
> > - Konstantin
> >
> > >
> > > Thanks
> > >
> > > > +
> > > >  	return flags;
> > > >  }
> > > >
> > > > --
> > > > 2.43.0
> > > >

[PATCH rdma-next 2/3] RDMA/mana_ib: support of the zero based MRs

[Konstantin Taranov]

From: Konstantin Taranov <kotaranov@microsoft.com>

Add IB_ZERO_BASED to the valid flags and use
the corresponding MR creation request for the zero
based memory.

Signed-off-by: Konstantin Taranov <kotaranov@microsoft.com>
---
 drivers/infiniband/hw/mana/mr.c | 24 +++++++++++++++++-------
 include/net/mana/gdma.h         | 11 ++++++++++-
 2 files changed, 27 insertions(+), 8 deletions(-)

diff --git a/drivers/infiniband/hw/mana/mr.c b/drivers/infiniband/hw/mana/mr.c
index e4a9f53..6d974d0 100644
--- a/drivers/infiniband/hw/mana/mr.c
+++ b/drivers/infiniband/hw/mana/mr.c
@@ -6,7 +6,7 @@
 #include "mana_ib.h"
 
 #define VALID_MR_FLAGS (IB_ACCESS_LOCAL_WRITE | IB_ACCESS_REMOTE_WRITE | IB_ACCESS_REMOTE_READ |\
-			IB_ACCESS_REMOTE_ATOMIC)
+			IB_ACCESS_REMOTE_ATOMIC | IB_ZERO_BASED)
 
 #define VALID_DMA_MR_FLAGS (IB_ACCESS_LOCAL_WRITE)
 
@@ -51,7 +51,10 @@ static int mana_ib_gd_create_mr(struct mana_ib_dev *dev, struct mana_ib_mr *mr,
 		req.gva.virtual_address = mr_params->gva.virtual_address;
 		req.gva.access_flags = mr_params->gva.access_flags;
 		break;
-
+	case GDMA_MR_TYPE_ZBVA:
+		req.zbva.dma_region_handle = mr_params->zbva.dma_region_handle;
+		req.zbva.access_flags = mr_params->zbva.access_flags;
+		break;
 	default:
 		ibdev_dbg(&dev->ib_dev,
 			  "invalid param (GDMA_MR_TYPE) passed, type %d\n",
@@ -147,11 +150,18 @@ struct ib_mr *mana_ib_reg_user_mr(struct ib_pd *ibpd, u64 start, u64 length,
 		  dma_region_handle);
 
 	mr_params.pd_handle = pd->pd_handle;
-	mr_params.mr_type = GDMA_MR_TYPE_GVA;
-	mr_params.gva.dma_region_handle = dma_region_handle;
-	mr_params.gva.virtual_address = iova;
-	mr_params.gva.access_flags =
-		mana_ib_verbs_to_gdma_access_flags(access_flags);
+	if (access_flags & IB_ZERO_BASED) {
+		mr_params.mr_type = GDMA_MR_TYPE_ZBVA;
+		mr_params.zbva.dma_region_handle = dma_region_handle;
+		mr_params.zbva.access_flags =
+			mana_ib_verbs_to_gdma_access_flags(access_flags);
+	} else {
+		mr_params.mr_type = GDMA_MR_TYPE_GVA;
+		mr_params.gva.dma_region_handle = dma_region_handle;
+		mr_params.gva.virtual_address = iova;
+		mr_params.gva.access_flags =
+			mana_ib_verbs_to_gdma_access_flags(access_flags);
+	}
 
 	err = mana_ib_gd_create_mr(dev, mr, &mr_params);
 	if (err)
diff --git a/include/net/mana/gdma.h b/include/net/mana/gdma.h
index 50ffbc4..3db506d 100644
--- a/include/net/mana/gdma.h
+++ b/include/net/mana/gdma.h
@@ -812,6 +812,8 @@ enum gdma_mr_type {
 	 * address that is set up in the MST
 	 */
 	GDMA_MR_TYPE_GVA = 2,
+	/* Guest zero-based address MRs */
+	GDMA_MR_TYPE_ZBVA = 4,
 };
 
 struct gdma_create_mr_params {
@@ -823,6 +825,10 @@ struct gdma_create_mr_params {
 			u64 virtual_address;
 			enum gdma_mr_access_flags access_flags;
 		} gva;
+		struct {
+			u64 dma_region_handle;
+			enum gdma_mr_access_flags access_flags;
+		} zbva;
 	};
 };
 
@@ -838,7 +844,10 @@ struct gdma_create_mr_request {
 			u64 virtual_address;
 			enum gdma_mr_access_flags access_flags;
 		} gva;
-
+		struct {
+			u64 dma_region_handle;
+			enum gdma_mr_access_flags access_flags;
+		} zbva;
 	};
 	u32 reserved_2;
 };/* HW DATA */
-- 
2.43.0

[PATCH rdma-next 3/3] RDMA/mana_ib: Add support of 4M, 1G, and 2G pages

[Konstantin Taranov]

From: Konstantin Taranov <kotaranov@microsoft.com>

Check PF capability flag whether the 4M, 1G, and 2G pages are
supported. Add these pages sizes to mana_ib, if supported.

Define possible page sizes in enum gdma_page_type and
remove unused enum atb_page_size.

Signed-off-by: Konstantin Taranov <kotaranov@microsoft.com>
---
 drivers/infiniband/hw/mana/main.c             | 10 +++++--
 drivers/infiniband/hw/mana/mana_ib.h          |  1 +
 .../net/ethernet/microsoft/mana/gdma_main.c   |  1 +
 include/net/mana/gdma.h                       | 30 ++++++++++---------
 4 files changed, 25 insertions(+), 17 deletions(-)

diff --git a/drivers/infiniband/hw/mana/main.c b/drivers/infiniband/hw/mana/main.c
index 730f958..a28b712 100644
--- a/drivers/infiniband/hw/mana/main.c
+++ b/drivers/infiniband/hw/mana/main.c
@@ -479,7 +479,7 @@ int mana_ib_create_dma_region(struct mana_ib_dev *dev, struct ib_umem *umem,
 {
 	unsigned long page_sz;
 
-	page_sz = ib_umem_find_best_pgsz(umem, PAGE_SZ_BM, virt);
+	page_sz = ib_umem_find_best_pgsz(umem, dev->adapter_caps.page_size_cap, virt);
 	if (!page_sz) {
 		ibdev_dbg(&dev->ib_dev, "Failed to find page size.\n");
 		return -EINVAL;
@@ -494,7 +494,7 @@ int mana_ib_create_zero_offset_dma_region(struct mana_ib_dev *dev, struct ib_ume
 	unsigned long page_sz;
 
 	/* Hardware requires dma region to align to chosen page size */
-	page_sz = ib_umem_find_best_pgoff(umem, PAGE_SZ_BM, 0);
+	page_sz = ib_umem_find_best_pgoff(umem, dev->adapter_caps.page_size_cap, 0);
 	if (!page_sz) {
 		ibdev_dbg(&dev->ib_dev, "Failed to find page size.\n");
 		return -EINVAL;
@@ -577,7 +577,7 @@ int mana_ib_query_device(struct ib_device *ibdev, struct ib_device_attr *props,
 
 	memset(props, 0, sizeof(*props));
 	props->max_mr_size = MANA_IB_MAX_MR_SIZE;
-	props->page_size_cap = PAGE_SZ_BM;
+	props->page_size_cap = dev->adapter_caps.page_size_cap;
 	props->max_qp = dev->adapter_caps.max_qp_count;
 	props->max_qp_wr = dev->adapter_caps.max_qp_wr;
 	props->device_cap_flags = IB_DEVICE_RC_RNR_NAK_GEN;
@@ -696,6 +696,10 @@ int mana_ib_gd_query_adapter_caps(struct mana_ib_dev *dev)
 	caps->max_recv_sge_count = resp.max_recv_sge_count;
 	caps->feature_flags = resp.feature_flags;
 
+	caps->page_size_cap = PAGE_SZ_BM;
+	if (mdev_to_gc(dev)->pf_cap_flags1 & GDMA_DRV_CAP_FLAG_1_GDMA_PAGES_4MB_1GB_2GB)
+		caps->page_size_cap |= (SZ_4M | SZ_1G | SZ_2G);
+
 	return 0;
 }
 
diff --git a/drivers/infiniband/hw/mana/mana_ib.h b/drivers/infiniband/hw/mana/mana_ib.h
index 6903946..f0dbd90 100644
--- a/drivers/infiniband/hw/mana/mana_ib.h
+++ b/drivers/infiniband/hw/mana/mana_ib.h
@@ -60,6 +60,7 @@ struct mana_ib_adapter_caps {
 	u32 max_recv_sge_count;
 	u32 max_inline_data_size;
 	u64 feature_flags;
+	u64 page_size_cap;
 };
 
 struct mana_ib_queue {
diff --git a/drivers/net/ethernet/microsoft/mana/gdma_main.c b/drivers/net/ethernet/microsoft/mana/gdma_main.c
index 4a2b17f..b5156d4 100644
--- a/drivers/net/ethernet/microsoft/mana/gdma_main.c
+++ b/drivers/net/ethernet/microsoft/mana/gdma_main.c
@@ -937,6 +937,7 @@ int mana_gd_verify_vf_version(struct pci_dev *pdev)
 			err, resp.hdr.status);
 		return err ? err : -EPROTO;
 	}
+	gc->pf_cap_flags1 = resp.pf_cap_flags1;
 	if (resp.pf_cap_flags1 & GDMA_DRV_CAP_FLAG_1_HWC_TIMEOUT_RECONFIG) {
 		err = mana_gd_query_hwc_timeout(pdev, &hwc->hwc_timeout);
 		if (err) {
diff --git a/include/net/mana/gdma.h b/include/net/mana/gdma.h
index 3db506d..fd37507 100644
--- a/include/net/mana/gdma.h
+++ b/include/net/mana/gdma.h
@@ -194,6 +194,19 @@ struct gdma_wqe_request {
 
 enum gdma_page_type {
 	GDMA_PAGE_TYPE_4K,
+	GDMA_PAGE_SIZE_8K,
+	GDMA_PAGE_SIZE_16K,
+	GDMA_PAGE_SIZE_32K,
+	GDMA_PAGE_SIZE_64K,
+	GDMA_PAGE_SIZE_128K,
+	GDMA_PAGE_SIZE_256K,
+	GDMA_PAGE_SIZE_512K,
+	GDMA_PAGE_SIZE_1M,
+	GDMA_PAGE_SIZE_2M,
+	/* Only when GDMA_DRV_CAP_FLAG_1_GDMA_PAGES_4MB_1GB_2GB is set */
+	GDMA_PAGE_SIZE_4M,
+	GDMA_PAGE_SIZE_1G = 18,
+	GDMA_PAGE_SIZE_2G
 };
 
 #define GDMA_INVALID_DMA_REGION 0
@@ -407,6 +420,8 @@ struct gdma_context {
 
 	/* Azure RDMA adapter */
 	struct gdma_dev		mana_ib;
+
+	u64 pf_cap_flags1;
 };
 
 #define MAX_NUM_GDMA_DEVICES	4
@@ -556,6 +571,7 @@ enum {
 #define GDMA_DRV_CAP_FLAG_1_NAPI_WKDONE_FIX BIT(2)
 #define GDMA_DRV_CAP_FLAG_1_HWC_TIMEOUT_RECONFIG BIT(3)
 #define GDMA_DRV_CAP_FLAG_1_VARIABLE_INDIRECTION_TABLE_SUPPORT BIT(5)
+#define GDMA_DRV_CAP_FLAG_1_GDMA_PAGES_4MB_1GB_2GB BIT(4)
 
 #define GDMA_DRV_CAP_FLAGS1 \
 	(GDMA_DRV_CAP_FLAG_1_EQ_SHARING_MULTI_VPORT | \
@@ -704,20 +720,6 @@ struct gdma_query_hwc_timeout_resp {
 	u32 reserved;
 };
 
-enum atb_page_size {
-	ATB_PAGE_SIZE_4K,
-	ATB_PAGE_SIZE_8K,
-	ATB_PAGE_SIZE_16K,
-	ATB_PAGE_SIZE_32K,
-	ATB_PAGE_SIZE_64K,
-	ATB_PAGE_SIZE_128K,
-	ATB_PAGE_SIZE_256K,
-	ATB_PAGE_SIZE_512K,
-	ATB_PAGE_SIZE_1M,
-	ATB_PAGE_SIZE_2M,
-	ATB_PAGE_SIZE_MAX,
-};
-
 enum gdma_mr_access_flags {
 	GDMA_ACCESS_FLAG_LOCAL_READ = BIT_ULL(0),
 	GDMA_ACCESS_FLAG_LOCAL_WRITE = BIT_ULL(1),
-- 
2.43.0

Re: [PATCH rdma-next 3/3] RDMA/mana_ib: Add support of 4M, 1G, and 2G pages

[Leon Romanovsky]

On Fri, Apr 04, 2025 at 07:45:55AM -0700, Konstantin Taranov wrote:
> From: Konstantin Taranov <kotaranov@microsoft.com>
> 
> Check PF capability flag whether the 4M, 1G, and 2G pages are
> supported. Add these pages sizes to mana_ib, if supported.
> 
> Define possible page sizes in enum gdma_page_type and
> remove unused enum atb_page_size.
> 
> Signed-off-by: Konstantin Taranov <kotaranov@microsoft.com>
> ---
>  drivers/infiniband/hw/mana/main.c             | 10 +++++--
>  drivers/infiniband/hw/mana/mana_ib.h          |  1 +
>  .../net/ethernet/microsoft/mana/gdma_main.c   |  1 +
>  include/net/mana/gdma.h                       | 30 ++++++++++---------
>  4 files changed, 25 insertions(+), 17 deletions(-)

<...>

>  enum gdma_page_type {
>  	GDMA_PAGE_TYPE_4K,
> +	GDMA_PAGE_SIZE_8K,
> +	GDMA_PAGE_SIZE_16K,
> +	GDMA_PAGE_SIZE_32K,
> +	GDMA_PAGE_SIZE_64K,
> +	GDMA_PAGE_SIZE_128K,
> +	GDMA_PAGE_SIZE_256K,
> +	GDMA_PAGE_SIZE_512K,
> +	GDMA_PAGE_SIZE_1M,
> +	GDMA_PAGE_SIZE_2M,
> +	/* Only when GDMA_DRV_CAP_FLAG_1_GDMA_PAGES_4MB_1GB_2GB is set */
> +	GDMA_PAGE_SIZE_4M,
> +	GDMA_PAGE_SIZE_1G = 18,
> +	GDMA_PAGE_SIZE_2G

Where are all these defines used?

Thanks

RE: [EXTERNAL] Re: [PATCH rdma-next 3/3] RDMA/mana_ib: Add support of 4M, 1G, and 2G pages

[Konstantin Taranov]

> 
> >  enum gdma_page_type {
> >  	GDMA_PAGE_TYPE_4K,
> > +	GDMA_PAGE_SIZE_8K,
> > +	GDMA_PAGE_SIZE_16K,
> > +	GDMA_PAGE_SIZE_32K,
> > +	GDMA_PAGE_SIZE_64K,
> > +	GDMA_PAGE_SIZE_128K,
> > +	GDMA_PAGE_SIZE_256K,
> > +	GDMA_PAGE_SIZE_512K,
> > +	GDMA_PAGE_SIZE_1M,
> > +	GDMA_PAGE_SIZE_2M,
> > +	/* Only when
> GDMA_DRV_CAP_FLAG_1_GDMA_PAGES_4MB_1GB_2GB is set */
> > +	GDMA_PAGE_SIZE_4M,
> > +	GDMA_PAGE_SIZE_1G = 18,
> > +	GDMA_PAGE_SIZE_2G
> 
> Where are all these defines used?

There are not used explicitly in this patch, but they can be used in theory.
I can remove unused defines in V2.

> 
> Thanks

Re: [EXTERNAL] Re: [PATCH rdma-next 3/3] RDMA/mana_ib: Add support of 4M, 1G, and 2G pages

[Leon Romanovsky]

On Thu, Apr 10, 2025 at 01:38:39PM +0000, Konstantin Taranov wrote:
> > 
> > >  enum gdma_page_type {
> > >  	GDMA_PAGE_TYPE_4K,
> > > +	GDMA_PAGE_SIZE_8K,
> > > +	GDMA_PAGE_SIZE_16K,
> > > +	GDMA_PAGE_SIZE_32K,
> > > +	GDMA_PAGE_SIZE_64K,
> > > +	GDMA_PAGE_SIZE_128K,
> > > +	GDMA_PAGE_SIZE_256K,
> > > +	GDMA_PAGE_SIZE_512K,
> > > +	GDMA_PAGE_SIZE_1M,
> > > +	GDMA_PAGE_SIZE_2M,
> > > +	/* Only when
> > GDMA_DRV_CAP_FLAG_1_GDMA_PAGES_4MB_1GB_2GB is set */
> > > +	GDMA_PAGE_SIZE_4M,
> > > +	GDMA_PAGE_SIZE_1G = 18,
> > > +	GDMA_PAGE_SIZE_2G
> > 
> > Where are all these defines used?
> 
> There are not used explicitly in this patch, but they can be used in theory.

Yes, please.

> I can remove unused defines in V2.
> 
> > 
> > Thanks