[PATCH net-next v2 0/2] Provide master netdev to mana_ib

[PATCH net-next v2 0/2] Provide master netdev to mana_ib

[Konstantin Taranov]

From: Konstantin Taranov <kotaranov@microsoft.com>

This patch series aims to allow mana_ib to get master netdev
from mana. In the netvsc deployment, the VF netdev is enslaved
by netvsc and the upper device should be considered for the network
state. In the baremetal case, the VF netdev is a master device.

I would appreciate if I could get Acks on it from:
* netvsc maintainers (e.g., Haiyang)
* net maintainers (e.g., Jakub, David, Eric, Paolo)

v1 -> v2:
Leon Romanovsky asked to make a helper in the net/mana and get
acks from net maintainers.

Konstantin Taranov (2):
  net: mana: introduce helper to get a master netdev
  RDMA/mana_ib: Set correct device into ib

 drivers/infiniband/hw/mana/device.c           | 16 ++++++++--------
 drivers/net/ethernet/microsoft/mana/mana_en.c | 18 ++++++++++++++++++
 include/net/mana/mana.h                       |  2 ++
 3 files changed, 28 insertions(+), 8 deletions(-)

-- 
2.43.0

[PATCH net-next v2 1/2] net: mana: introduce helper to get a master netdev

[Konstantin Taranov]

From: Konstantin Taranov <kotaranov@microsoft.com>

Add helper to get a master netdevice for a given port.
When mana is used with netvsc, the VF netdev is controlled
by an upper netvsc device. In a baremetal case, the VF
netdev is the master device.

Signed-off-by: Konstantin Taranov <kotaranov@microsoft.com>
---
 drivers/net/ethernet/microsoft/mana/mana_en.c | 18 ++++++++++++++++++
 include/net/mana/mana.h                       |  2 ++
 2 files changed, 20 insertions(+)

diff --git a/drivers/net/ethernet/microsoft/mana/mana_en.c b/drivers/net/ethernet/microsoft/mana/mana_en.c
index d087cf9..b893339 100644
--- a/drivers/net/ethernet/microsoft/mana/mana_en.c
+++ b/drivers/net/ethernet/microsoft/mana/mana_en.c
@@ -2948,3 +2948,21 @@ out:
 	gd->gdma_context = NULL;
 	kfree(ac);
 }
+
+/* the caller should hold rcu_read_lock */
+struct net_device *mana_get_master_netdev_rcu(struct mana_context *ac, u32 port_index)
+{
+	struct net_device *ndev;
+
+	if (port_index >= ac->num_ports)
+		return NULL;
+
+	/* When mana is used in netvsc, the upper netdevice should be returned. */
+	if (ac->ports[port_index]->flags & IFF_SLAVE)
+		ndev = netdev_master_upper_dev_get_rcu(ac->ports[port_index]);
+	else
+		ndev = ac->ports[port_index];
+
+	return ndev;
+}
+EXPORT_SYMBOL_NS(mana_get_master_netdev_rcu, NET_MANA);
diff --git a/include/net/mana/mana.h b/include/net/mana/mana.h
index 46f741e..2d3625c 100644
--- a/include/net/mana/mana.h
+++ b/include/net/mana/mana.h
@@ -797,4 +797,6 @@ void mana_destroy_wq_obj(struct mana_port_context *apc, u32 wq_type,
 int mana_cfg_vport(struct mana_port_context *apc, u32 protection_dom_id,
 		   u32 doorbell_pg_id);
 void mana_uncfg_vport(struct mana_port_context *apc);
+
+struct net_device *mana_get_master_netdev_rcu(struct mana_context *ac, u32 port_index);
 #endif /* _MANA_H */
-- 
2.43.0

Re: [PATCH net-next v2 1/2] net: mana: introduce helper to get a master netdev

[Zhu Yanjun]

在 2024/7/1 20:58, Konstantin Taranov 写道:
> From: Konstantin Taranov <kotaranov@microsoft.com>
> 
> Add helper to get a master netdevice for a given port.
> When mana is used with netvsc, the VF netdev is controlled
> by an upper netvsc device. In a baremetal case, the VF
> netdev is the master device.
> 
> Signed-off-by: Konstantin Taranov <kotaranov@microsoft.com>
> ---
>   drivers/net/ethernet/microsoft/mana/mana_en.c | 18 ++++++++++++++++++
>   include/net/mana/mana.h                       |  2 ++
>   2 files changed, 20 insertions(+)
> 
> diff --git a/drivers/net/ethernet/microsoft/mana/mana_en.c b/drivers/net/ethernet/microsoft/mana/mana_en.c
> index d087cf9..b893339 100644
> --- a/drivers/net/ethernet/microsoft/mana/mana_en.c
> +++ b/drivers/net/ethernet/microsoft/mana/mana_en.c
> @@ -2948,3 +2948,21 @@ out:
>   	gd->gdma_context = NULL;
>   	kfree(ac);
>   }
> +
> +/* the caller should hold rcu_read_lock */
> +struct net_device *mana_get_master_netdev_rcu(struct mana_context *ac, u32 port_index)
> +{
> +	struct net_device *ndev;

 From the comments, this function requires rcu_read_lock. Can the 
following be added in this function?

RCU_LOCKDEP_WARN(!rcu_read_lock_held(), "no rcu read lock held");

If rcu_read_lock is not held, this function will notify the caller.
This can explicitly ensure that rcu_read_lock should be held before this 
fuction is called.

Best Regards,

Zhu Yanjun

> +
> +	if (port_index >= ac->num_ports)
> +		return NULL;
> +
> +	/* When mana is used in netvsc, the upper netdevice should be returned. */
> +	if (ac->ports[port_index]->flags & IFF_SLAVE)
> +		ndev = netdev_master_upper_dev_get_rcu(ac->ports[port_index]);
> +	else
> +		ndev = ac->ports[port_index];
> +
> +	return ndev;
> +}
> +EXPORT_SYMBOL_NS(mana_get_master_netdev_rcu, NET_MANA);
> diff --git a/include/net/mana/mana.h b/include/net/mana/mana.h
> index 46f741e..2d3625c 100644
> --- a/include/net/mana/mana.h
> +++ b/include/net/mana/mana.h
> @@ -797,4 +797,6 @@ void mana_destroy_wq_obj(struct mana_port_context *apc, u32 wq_type,
>   int mana_cfg_vport(struct mana_port_context *apc, u32 protection_dom_id,
>   		   u32 doorbell_pg_id);
>   void mana_uncfg_vport(struct mana_port_context *apc);
> +
> +struct net_device *mana_get_master_netdev_rcu(struct mana_context *ac, u32 port_index);
>   #endif /* _MANA_H */

Re: [PATCH net-next v2 1/2] net: mana: introduce helper to get a master netdev

[Konstantin Taranov]

> > +/* the caller should hold rcu_read_lock */ struct net_device
> > +*mana_get_master_netdev_rcu(struct mana_context *ac, u32 port_index)
> > +{
> > +	struct net_device *ndev;
> 
>  From the comments, this function requires rcu_read_lock. Can the following
> be added in this function?
> 
> RCU_LOCKDEP_WARN(!rcu_read_lock_held(), "no rcu read lock held");
> 
> If rcu_read_lock is not held, this function will notify the caller.
> This can explicitly ensure that rcu_read_lock should be held before this
> fuction is called.

Thanks! Great idea!

> 
> Best Regards,
> 
> Zhu Yanjun
> 

[PATCH net-next v2 2/2] RDMA/mana_ib: Set correct device into ib

[Konstantin Taranov]

From: Konstantin Taranov <kotaranov@microsoft.com>

Use the mana_get_master_netdev_rcu() helper to get
a master netdevice for querying network states.
The helper allows the mana_ib transparently
support baremetal and netvsc deployment cases.

Fixes: 8b184e4f1c32 ("RDMA/mana_ib: Enable RoCE on port 1")
Signed-off-by: Konstantin Taranov <kotaranov@microsoft.com>
---
 drivers/infiniband/hw/mana/device.c | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/drivers/infiniband/hw/mana/device.c b/drivers/infiniband/hw/mana/device.c
index b07a8e2..ec849ac 100644
--- a/drivers/infiniband/hw/mana/device.c
+++ b/drivers/infiniband/hw/mana/device.c
@@ -56,7 +56,7 @@ static int mana_ib_probe(struct auxiliary_device *adev,
 {
 	struct mana_adev *madev = container_of(adev, struct mana_adev, adev);
 	struct gdma_dev *mdev = madev->mdev;
-	struct net_device *upper_ndev;
+	struct net_device *ndev;
 	struct mana_context *mc;
 	struct mana_ib_dev *dev;
 	u8 mac_addr[ETH_ALEN];
@@ -84,17 +84,17 @@ static int mana_ib_probe(struct auxiliary_device *adev,
 	dev->ib_dev.num_comp_vectors = mdev->gdma_context->max_num_queues;
 	dev->ib_dev.dev.parent = mdev->gdma_context->dev;
 
-	rcu_read_lock(); /* required to get upper dev */
-	upper_ndev = netdev_master_upper_dev_get_rcu(mc->ports[0]);
-	if (!upper_ndev) {
+	rcu_read_lock(); /* required to get master dev */
+	ndev = mana_get_master_netdev_rcu(mc, 0);
+	if (!ndev) {
 		rcu_read_unlock();
 		ret = -ENODEV;
-		ibdev_err(&dev->ib_dev, "Failed to get master netdev");
+		ibdev_err(&dev->ib_dev, "Failed to get netdev for IB port 1");
 		goto free_ib_device;
 	}
-	ether_addr_copy(mac_addr, upper_ndev->dev_addr);
-	addrconf_addr_eui48((u8 *)&dev->ib_dev.node_guid, upper_ndev->dev_addr);
-	ret = ib_device_set_netdev(&dev->ib_dev, upper_ndev, 1);
+	ether_addr_copy(mac_addr, ndev->dev_addr);
+	addrconf_addr_eui48((u8 *)&dev->ib_dev.node_guid, ndev->dev_addr);
+	ret = ib_device_set_netdev(&dev->ib_dev, ndev, 1);
 	rcu_read_unlock();
 	if (ret) {
 		ibdev_err(&dev->ib_dev, "Failed to set ib netdev, ret %d", ret);
-- 
2.43.0

Re: [PATCH net-next v2 2/2] RDMA/mana_ib: Set correct device into ib

[Leon Romanovsky]

On Mon, Jul 01, 2024 at 05:58:56AM -0700, Konstantin Taranov wrote:
> From: Konstantin Taranov <kotaranov@microsoft.com>
> 
> Use the mana_get_master_netdev_rcu() helper to get
> a master netdevice for querying network states.
> The helper allows the mana_ib transparently
> support baremetal and netvsc deployment cases.
> 
> Fixes: 8b184e4f1c32 ("RDMA/mana_ib: Enable RoCE on port 1")
> Signed-off-by: Konstantin Taranov <kotaranov@microsoft.com>
> ---
>  drivers/infiniband/hw/mana/device.c | 16 ++++++++--------
>  1 file changed, 8 insertions(+), 8 deletions(-)
> 

Thanks,
Acked-by: Leon Romanovsky <leon@kernel.org>

Re: [PATCH net-next v2 0/2] Provide master netdev to mana_ib

[Jakub Kicinski]

On Mon,  1 Jul 2024 05:58:54 -0700 Konstantin Taranov wrote:
> This patch series aims to allow mana_ib to get master netdev
> from mana. In the netvsc deployment, the VF netdev is enslaved
> by netvsc and the upper device should be considered for the network
> state. In the baremetal case, the VF netdev is a master device.
> 
> I would appreciate if I could get Acks on it from:
> * netvsc maintainers (e.g., Haiyang)
> * net maintainers (e.g., Jakub, David, Eric, Paolo)

Would you like us to take this via net-next?
 - it doesn't apply cleanly
 - the patches should be squashed, they aren't separate logical changes
 - please find a better word than master