[PATCH rdma-next 0/2] Followup changes to mlx5-next branch

[PATCH rdma-next 0/2] Followup changes to mlx5-next branch

[Leon Romanovsky]

From: Leon Romanovsky <leonro@mellanox.com>

Hi,

There are two small cleanups needed after prefetch MR code was merged.

Thanks

Leon Romanovsky (2):
  net/mlx5: Align ODP capability function with netdev coding style
  net/mlx5: Factor out HCA capabilities functions

 .../net/ethernet/mellanox/mlx5/core/main.c    | 62 ++++++++++++-------
 1 file changed, 39 insertions(+), 23 deletions(-)

[PATCH rdma-next 0/2] Followup changes to mlx5-next branch

[Leon Romanovsky]

From: Leon Romanovsky <leonro@mellanox.com>

Hi,

There are two small cleanups needed after prefetch MR code was merged.

Thanks

Leon Romanovsky (2):
  net/mlx5: Align ODP capability function with netdev coding style
  net/mlx5: Factor out HCA capabilities functions

 .../net/ethernet/mellanox/mlx5/core/main.c    | 62 ++++++++++++-------
 1 file changed, 39 insertions(+), 23 deletions(-)

--
2.19.1

[PATCH mlx5-next 1/2] net/mlx5: Align ODP capability function with netdev coding style

[Leon Romanovsky]

From: Leon Romanovsky <leonro@mellanox.com>

Update newly introduced function to be aligned to netdev coding style.

Fixes: 46861e3e88be ("net/mlx5: Set ODP SRQ support in firmware")
Reported-by: Saeed Mahameed <saeedm@mellanox.com>
Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
---
 drivers/net/ethernet/mellanox/mlx5/core/main.c | 15 ++++++---------
 1 file changed, 6 insertions(+), 9 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/main.c b/drivers/net/ethernet/mellanox/mlx5/core/main.c
index e38aa206ab6d..6d45518edbdc 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/main.c
@@ -461,9 +461,9 @@ static int handle_hca_cap_atomic(struct mlx5_core_dev *dev)
 
 static int handle_hca_cap_odp(struct mlx5_core_dev *dev)
 {
-	void *set_ctx;
 	void *set_hca_cap;
-	int set_sz = MLX5_ST_SZ_BYTES(set_hca_cap_in);
+	void *set_ctx;
+	int set_sz;
 	int err;
 
 	if (!MLX5_CAP_GEN(dev, pg))
@@ -473,15 +473,12 @@ static int handle_hca_cap_odp(struct mlx5_core_dev *dev)
 	if (err)
 		return err;
 
-	/**
-	 * If all bits are cleared we shouldn't try to set it
-	 * or we might fail while trying to access a reserved bit.
-	 */
 	if (!(MLX5_CAP_ODP_MAX(dev, ud_odp_caps.srq_receive) ||
 	      MLX5_CAP_ODP_MAX(dev, rc_odp_caps.srq_receive) ||
 	      MLX5_CAP_ODP_MAX(dev, xrc_odp_caps.srq_receive)))
 		return 0;
 
+	set_sz = MLX5_ST_SZ_BYTES(set_hca_cap_in);
 	set_ctx = kzalloc(set_sz, GFP_KERNEL);
 	if (!set_ctx)
 		return -ENOMEM;
@@ -492,13 +489,13 @@ static int handle_hca_cap_odp(struct mlx5_core_dev *dev)
 
 	/* set ODP SRQ support for RC/UD and XRC transports */
 	MLX5_SET(odp_cap, set_hca_cap, ud_odp_caps.srq_receive,
-		 (MLX5_CAP_ODP_MAX(dev, ud_odp_caps.srq_receive)));
+		 MLX5_CAP_ODP_MAX(dev, ud_odp_caps.srq_receive));
 
 	MLX5_SET(odp_cap, set_hca_cap, rc_odp_caps.srq_receive,
-		 (MLX5_CAP_ODP_MAX(dev, rc_odp_caps.srq_receive)));
+		 MLX5_CAP_ODP_MAX(dev, rc_odp_caps.srq_receive));
 
 	MLX5_SET(odp_cap, set_hca_cap, xrc_odp_caps.srq_receive,
-		 (MLX5_CAP_ODP_MAX(dev, xrc_odp_caps.srq_receive)));
+		 MLX5_CAP_ODP_MAX(dev, xrc_odp_caps.srq_receive));
 
 	err = set_caps(dev, set_ctx, set_sz, MLX5_SET_HCA_CAP_OP_MOD_ODP);
 
-- 
2.19.1

[PATCH mlx5-next 2/2] net/mlx5: Factor out HCA capabilities functions

[Leon Romanovsky]

From: Leon Romanovsky <leonro@mellanox.com>

Combine all HCA capabilities setters under one function
and compile out the ODP related function in case kernel
was compiled without ODP support.

Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
---
 .../net/ethernet/mellanox/mlx5/core/main.c    | 47 +++++++++++++------
 1 file changed, 33 insertions(+), 14 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/main.c b/drivers/net/ethernet/mellanox/mlx5/core/main.c
index 6d45518edbdc..d7145ab6105d 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/main.c
@@ -459,6 +459,7 @@ static int handle_hca_cap_atomic(struct mlx5_core_dev *dev)
 	return err;
 }
 
+#ifdef CONFIG_INFINIBAND_ON_DEMAND_PAGING
 static int handle_hca_cap_odp(struct mlx5_core_dev *dev)
 {
 	void *set_hca_cap;
@@ -502,6 +503,7 @@ static int handle_hca_cap_odp(struct mlx5_core_dev *dev)
 	kfree(set_ctx);
 	return err;
 }
+#endif
 
 static int handle_hca_cap(struct mlx5_core_dev *dev)
 {
@@ -576,6 +578,35 @@ static int handle_hca_cap(struct mlx5_core_dev *dev)
 	return err;
 }
 
+static int set_hca_cap(struct mlx5_core_dev *dev)
+{
+	struct pci_dev *pdev = dev->pdev;
+	int err;
+
+	err = handle_hca_cap(dev);
+	if (err) {
+		dev_err(&pdev->dev, "handle_hca_cap failed\n");
+		goto out;
+	}
+
+	err = handle_hca_cap_atomic(dev);
+	if (err) {
+		dev_err(&pdev->dev, "handle_hca_cap_atomic failed\n");
+		goto out;
+	}
+
+#ifdef CONFIG_INFINIBAND_ON_DEMAND_PAGING
+	err = handle_hca_cap_odp(dev);
+	if (err) {
+		dev_err(&pdev->dev, "handle_hca_cap_odp failed\n");
+		goto out;
+	}
+#endif
+
+out:
+	return err;
+}
+
 static int set_hca_ctrl(struct mlx5_core_dev *dev)
 {
 	struct mlx5_reg_host_endianness he_in;
@@ -963,21 +994,9 @@ static int mlx5_load_one(struct mlx5_core_dev *dev, struct mlx5_priv *priv,
 		goto reclaim_boot_pages;
 	}
 
-	err = handle_hca_cap(dev);
+	err = set_hca_cap(dev);
 	if (err) {
-		dev_err(&pdev->dev, "handle_hca_cap failed\n");
-		goto reclaim_boot_pages;
-	}
-
-	err = handle_hca_cap_atomic(dev);
-	if (err) {
-		dev_err(&pdev->dev, "handle_hca_cap_atomic failed\n");
-		goto reclaim_boot_pages;
-	}
-
-	err = handle_hca_cap_odp(dev);
-	if (err) {
-		dev_err(&pdev->dev, "handle_hca_cap_odp failed\n");
+		dev_err(&pdev->dev, "set_hca_cap failed\n");
 		goto reclaim_boot_pages;
 	}
 
-- 
2.19.1

Re: [PATCH mlx5-next 2/2] net/mlx5: Factor out HCA capabilities functions

[Jason Gunthorpe]

On Mon, Feb 11, 2019 at 01:56:08PM +0200, Leon Romanovsky wrote:
> From: Leon Romanovsky <leonro@mellanox.com>
> 
> Combine all HCA capabilities setters under one function
> and compile out the ODP related function in case kernel
> was compiled without ODP support.
> 
> Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
>  .../net/ethernet/mellanox/mlx5/core/main.c    | 47 +++++++++++++------
>  1 file changed, 33 insertions(+), 14 deletions(-)
> 
> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/main.c b/drivers/net/ethernet/mellanox/mlx5/core/main.c
> index 6d45518edbdc..d7145ab6105d 100644
> +++ b/drivers/net/ethernet/mellanox/mlx5/core/main.c
> @@ -459,6 +459,7 @@ static int handle_hca_cap_atomic(struct mlx5_core_dev *dev)
>  	return err;
>  }
>  
> +#ifdef CONFIG_INFINIBAND_ON_DEMAND_PAGING
>  static int handle_hca_cap_odp(struct mlx5_core_dev *dev)
>  {
>  	void *set_hca_cap;
> @@ -502,6 +503,7 @@ static int handle_hca_cap_odp(struct mlx5_core_dev *dev)
>  	kfree(set_ctx);
>  	return err;
>  }
> +#endif
>  
>  static int handle_hca_cap(struct mlx5_core_dev *dev)
>  {
> @@ -576,6 +578,35 @@ static int handle_hca_cap(struct mlx5_core_dev *dev)
>  	return err;
>  }
>  
> +static int set_hca_cap(struct mlx5_core_dev *dev)
> +{
> +	struct pci_dev *pdev = dev->pdev;
> +	int err;
> +
> +	err = handle_hca_cap(dev);
> +	if (err) {
> +		dev_err(&pdev->dev, "handle_hca_cap failed\n");
> +		goto out;
> +	}
> +
> +	err = handle_hca_cap_atomic(dev);
> +	if (err) {
> +		dev_err(&pdev->dev, "handle_hca_cap_atomic failed\n");
> +		goto out;
> +	}
> +
> +#ifdef CONFIG_INFINIBAND_ON_DEMAND_PAGING
> +	err = handle_hca_cap_odp(dev);
> +	if (err) {
> +		dev_err(&pdev->dev, "handle_hca_cap_odp failed\n");
> +		goto out;
> +	}
> +#endif

Adding 
  if (IS_ENABLED..) 
    return 0;

To the top of handle_hca_cap_odp is alot better.

Jason

Re: [PATCH mlx5-next 2/2] net/mlx5: Factor out HCA capabilities functions

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 2235 bytes --]

On Mon, Feb 11, 2019 at 07:50:55PM +0000, Jason Gunthorpe wrote:
> On Mon, Feb 11, 2019 at 01:56:08PM +0200, Leon Romanovsky wrote:
> > From: Leon Romanovsky <leonro@mellanox.com>
> >
> > Combine all HCA capabilities setters under one function
> > and compile out the ODP related function in case kernel
> > was compiled without ODP support.
> >
> > Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
> >  .../net/ethernet/mellanox/mlx5/core/main.c    | 47 +++++++++++++------
> >  1 file changed, 33 insertions(+), 14 deletions(-)
> >
> > diff --git a/drivers/net/ethernet/mellanox/mlx5/core/main.c b/drivers/net/ethernet/mellanox/mlx5/core/main.c
> > index 6d45518edbdc..d7145ab6105d 100644
> > +++ b/drivers/net/ethernet/mellanox/mlx5/core/main.c
> > @@ -459,6 +459,7 @@ static int handle_hca_cap_atomic(struct mlx5_core_dev *dev)
> >  	return err;
> >  }
> >
> > +#ifdef CONFIG_INFINIBAND_ON_DEMAND_PAGING
> >  static int handle_hca_cap_odp(struct mlx5_core_dev *dev)
> >  {
> >  	void *set_hca_cap;
> > @@ -502,6 +503,7 @@ static int handle_hca_cap_odp(struct mlx5_core_dev *dev)
> >  	kfree(set_ctx);
> >  	return err;
> >  }
> > +#endif
> >
> >  static int handle_hca_cap(struct mlx5_core_dev *dev)
> >  {
> > @@ -576,6 +578,35 @@ static int handle_hca_cap(struct mlx5_core_dev *dev)
> >  	return err;
> >  }
> >
> > +static int set_hca_cap(struct mlx5_core_dev *dev)
> > +{
> > +	struct pci_dev *pdev = dev->pdev;
> > +	int err;
> > +
> > +	err = handle_hca_cap(dev);
> > +	if (err) {
> > +		dev_err(&pdev->dev, "handle_hca_cap failed\n");
> > +		goto out;
> > +	}
> > +
> > +	err = handle_hca_cap_atomic(dev);
> > +	if (err) {
> > +		dev_err(&pdev->dev, "handle_hca_cap_atomic failed\n");
> > +		goto out;
> > +	}
> > +
> > +#ifdef CONFIG_INFINIBAND_ON_DEMAND_PAGING
> > +	err = handle_hca_cap_odp(dev);
> > +	if (err) {
> > +		dev_err(&pdev->dev, "handle_hca_cap_odp failed\n");
> > +		goto out;
> > +	}
> > +#endif
>
> Adding
>   if (IS_ENABLED..)
>     return 0;
>
> To the top of handle_hca_cap_odp is alot better.

Saeed gave comment that he prefers code to be compiled-out in case
config is not set. In you suggestion, the code will exist and only
with some optimizations enabled it will be thrown.

Thanks

>
> Jason

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: [PATCH mlx5-next 2/2] net/mlx5: Factor out HCA capabilities functions

[Jason Gunthorpe]

On Mon, Feb 11, 2019 at 10:02:07PM +0200, Leon Romanovsky wrote:
> On Mon, Feb 11, 2019 at 07:50:55PM +0000, Jason Gunthorpe wrote:
> > On Mon, Feb 11, 2019 at 01:56:08PM +0200, Leon Romanovsky wrote:
> > > From: Leon Romanovsky <leonro@mellanox.com>
> > >
> > > Combine all HCA capabilities setters under one function
> > > and compile out the ODP related function in case kernel
> > > was compiled without ODP support.
> > >
> > > Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
> > >  .../net/ethernet/mellanox/mlx5/core/main.c    | 47 +++++++++++++------
> > >  1 file changed, 33 insertions(+), 14 deletions(-)
> > >
> > > diff --git a/drivers/net/ethernet/mellanox/mlx5/core/main.c b/drivers/net/ethernet/mellanox/mlx5/core/main.c
> > > index 6d45518edbdc..d7145ab6105d 100644
> > > +++ b/drivers/net/ethernet/mellanox/mlx5/core/main.c
> > > @@ -459,6 +459,7 @@ static int handle_hca_cap_atomic(struct mlx5_core_dev *dev)
> > >  	return err;
> > >  }
> > >
> > > +#ifdef CONFIG_INFINIBAND_ON_DEMAND_PAGING
> > >  static int handle_hca_cap_odp(struct mlx5_core_dev *dev)
> > >  {
> > >  	void *set_hca_cap;
> > > @@ -502,6 +503,7 @@ static int handle_hca_cap_odp(struct mlx5_core_dev *dev)
> > >  	kfree(set_ctx);
> > >  	return err;
> > >  }
> > > +#endif
> > >
> > >  static int handle_hca_cap(struct mlx5_core_dev *dev)
> > >  {
> > > @@ -576,6 +578,35 @@ static int handle_hca_cap(struct mlx5_core_dev *dev)
> > >  	return err;
> > >  }
> > >
> > > +static int set_hca_cap(struct mlx5_core_dev *dev)
> > > +{
> > > +	struct pci_dev *pdev = dev->pdev;
> > > +	int err;
> > > +
> > > +	err = handle_hca_cap(dev);
> > > +	if (err) {
> > > +		dev_err(&pdev->dev, "handle_hca_cap failed\n");
> > > +		goto out;
> > > +	}
> > > +
> > > +	err = handle_hca_cap_atomic(dev);
> > > +	if (err) {
> > > +		dev_err(&pdev->dev, "handle_hca_cap_atomic failed\n");
> > > +		goto out;
> > > +	}
> > > +
> > > +#ifdef CONFIG_INFINIBAND_ON_DEMAND_PAGING
> > > +	err = handle_hca_cap_odp(dev);
> > > +	if (err) {
> > > +		dev_err(&pdev->dev, "handle_hca_cap_odp failed\n");
> > > +		goto out;
> > > +	}
> > > +#endif
> >
> > Adding
> >   if (IS_ENABLED..)
> >     return 0;
> >
> > To the top of handle_hca_cap_odp is alot better.
> 
> Saeed gave comment that he prefers code to be compiled-out in case
> config is not set. In you suggestion, the code will exist and only
> with some optimizations enabled it will be thrown.

the kernel always compiles with optimizations that will throw away
this code, it is a widely used pattern.

#ifdef creates a compilation test matrix that is very undesired.

Jason

Re: [PATCH mlx5-next 2/2] net/mlx5: Factor out HCA capabilities functions

[Saeed Mahameed]

On Mon, 2019-02-11 at 20:32 +0000, Jason Gunthorpe wrote:
> On Mon, Feb 11, 2019 at 10:02:07PM +0200, Leon Romanovsky wrote:
> > On Mon, Feb 11, 2019 at 07:50:55PM +0000, Jason Gunthorpe wrote:
> > > On Mon, Feb 11, 2019 at 01:56:08PM +0200, Leon Romanovsky wrote:
> > > > From: Leon Romanovsky <leonro@mellanox.com>
> > > > 
> > > > Combine all HCA capabilities setters under one function
> > > > and compile out the ODP related function in case kernel
> > > > was compiled without ODP support.
> > > > 
> > > > Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
> > > >  .../net/ethernet/mellanox/mlx5/core/main.c    | 47
> > > > +++++++++++++------
> > > >  1 file changed, 33 insertions(+), 14 deletions(-)
> > > > 
> > > > diff --git a/drivers/net/ethernet/mellanox/mlx5/core/main.c
> > > > b/drivers/net/ethernet/mellanox/mlx5/core/main.c
> > > > index 6d45518edbdc..d7145ab6105d 100644
> > > > +++ b/drivers/net/ethernet/mellanox/mlx5/core/main.c
> > > > @@ -459,6 +459,7 @@ static int handle_hca_cap_atomic(struct
> > > > mlx5_core_dev *dev)
> > > >  	return err;
> > > >  }
> > > > 
> > > > +#ifdef CONFIG_INFINIBAND_ON_DEMAND_PAGING
> > > >  static int handle_hca_cap_odp(struct mlx5_core_dev *dev)
> > > >  {
> > > >  	void *set_hca_cap;
> > > > @@ -502,6 +503,7 @@ static int handle_hca_cap_odp(struct
> > > > mlx5_core_dev *dev)
> > > >  	kfree(set_ctx);
> > > >  	return err;
> > > >  }
> > > > +#endif
> > > > 
> > > >  static int handle_hca_cap(struct mlx5_core_dev *dev)
> > > >  {
> > > > @@ -576,6 +578,35 @@ static int handle_hca_cap(struct
> > > > mlx5_core_dev *dev)
> > > >  	return err;
> > > >  }
> > > > 
> > > > +static int set_hca_cap(struct mlx5_core_dev *dev)
> > > > +{
> > > > +	struct pci_dev *pdev = dev->pdev;
> > > > +	int err;
> > > > +
> > > > +	err = handle_hca_cap(dev);
> > > > +	if (err) {
> > > > +		dev_err(&pdev->dev, "handle_hca_cap failed\n");
> > > > +		goto out;
> > > > +	}
> > > > +
> > > > +	err = handle_hca_cap_atomic(dev);
> > > > +	if (err) {
> > > > +		dev_err(&pdev->dev, "handle_hca_cap_atomic
> > > > failed\n");
> > > > +		goto out;
> > > > +	}
> > > > +
> > > > +#ifdef CONFIG_INFINIBAND_ON_DEMAND_PAGING
> > > > +	err = handle_hca_cap_odp(dev);
> > > > +	if (err) {
> > > > +		dev_err(&pdev->dev, "handle_hca_cap_odp
> > > > failed\n");
> > > > +		goto out;
> > > > +	}
> > > > +#endif
> > > 
> > > Adding
> > >   if (IS_ENABLED..)
> > >     return 0;
> > > 
> > > To the top of handle_hca_cap_odp is alot better.
> > 
> > Saeed gave comment that he prefers code to be compiled-out in case
> > config is not set. In you suggestion, the code will exist and only
> > with some optimizations enabled it will be thrown.
> 
> the kernel always compiles with optimizations that will throw away
> this code, it is a widely used pattern.
> 
> #ifdef creates a compilation test matrix that is very undesired.
> 

It is more about code separation and readability rather than compiler
optimizations, optimally i want to have a file(s) that will include all
IB stuff and compile out  directly from Makefile, of course we still
need stub functions in a header file which will yield the same
compilation test matrix, but code readability and separation will
become more clear.

-Saeed.

> Jason

Re: [PATCH rdma-next 0/2] Followup changes to mlx5-next branch

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 546 bytes --]

On Mon, Feb 11, 2019 at 01:56:06PM +0200, Leon Romanovsky wrote:
> From: Leon Romanovsky <leonro@mellanox.com>
>
> Hi,
>
> There are two small cleanups needed after prefetch MR code was merged.
>
> Thanks
>
> Leon Romanovsky (2):
>   net/mlx5: Align ODP capability function with netdev coding style

This patch was applied.

>   net/mlx5: Factor out HCA capabilities functions

This will be resent.
>
>  .../net/ethernet/mellanox/mlx5/core/main.c    | 62 ++++++++++++-------
>  1 file changed, 39 insertions(+), 23 deletions(-)
>
> --
> 2.19.1
>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 801 bytes --]