[PATCH net 0/2] Fix issues when PF sets MAC address for VF

[PATCH net 0/2] Fix issues when PF sets MAC address for VF

[Wei Fang]

The ENETC PF driver provides enetc_pf_set_vf_mac() to configure the MAC
address for the ENETC VF, but there are two issues when configuring the
MAC address of the VF through this interface. For specific issues, please
refer to the commit message of the following two patches. Therefore, this
patch set is used to fix these two issues.

Wei Fang (2):
  net: enetc: allocate vf_state during PF probes
  net: enetc: prevent PF from configuring MAC address for an enabled VF

 .../net/ethernet/freescale/enetc/enetc_pf.c   | 25 ++++++++++++-------
 1 file changed, 16 insertions(+), 9 deletions(-)

-- 
2.34.1

[PATCH net 1/2] net: enetc: allocate vf_state during PF probes

[Wei Fang]

In the previous implementation, vf_state is allocated memory only when VF
is enabled. However, net_device_ops::ndo_set_vf_mac() may be called before
VF is enabled to configure the MAC address of VF. If this is the case,
enetc_pf_set_vf_mac() will access vf_state, resulting in access to a null
pointer. The simplified error log is as follows.

root@ls1028ardb:~# ip link set eno0 vf 1 mac 00:0c:e7:66:77:89
[  173.543315] Unable to handle kernel NULL pointer dereference at virtual address 0000000000000004
[  173.637254] pc : enetc_pf_set_vf_mac+0x3c/0x80 Message from sy
[  173.641973] lr : do_setlink+0x4a8/0xec8
[  173.732292] Call trace:
[  173.734740]  enetc_pf_set_vf_mac+0x3c/0x80
[  173.738847]  __rtnl_newlink+0x530/0x89c
[  173.742692]  rtnl_newlink+0x50/0x7c
[  173.746189]  rtnetlink_rcv_msg+0x128/0x390
[  173.750298]  netlink_rcv_skb+0x60/0x130
[  173.754145]  rtnetlink_rcv+0x18/0x24
[  173.757731]  netlink_unicast+0x318/0x380
[  173.761665]  netlink_sendmsg+0x17c/0x3c8

Fixes: d4fd0404c1c9 ("enetc: Introduce basic PF and VF ENETC ethernet drivers")
Signed-off-by: Wei Fang <wei.fang@nxp.com>
---
 .../net/ethernet/freescale/enetc/enetc_pf.c    | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)

diff --git a/drivers/net/ethernet/freescale/enetc/enetc_pf.c b/drivers/net/ethernet/freescale/enetc/enetc_pf.c
index 8f6b0bf48139..c95a7c083b0f 100644
--- a/drivers/net/ethernet/freescale/enetc/enetc_pf.c
+++ b/drivers/net/ethernet/freescale/enetc/enetc_pf.c
@@ -665,19 +665,11 @@ static int enetc_sriov_configure(struct pci_dev *pdev, int num_vfs)
 
 	if (!num_vfs) {
 		enetc_msg_psi_free(pf);
-		kfree(pf->vf_state);
 		pf->num_vfs = 0;
 		pci_disable_sriov(pdev);
 	} else {
 		pf->num_vfs = num_vfs;
 
-		pf->vf_state = kcalloc(num_vfs, sizeof(struct enetc_vf_state),
-				       GFP_KERNEL);
-		if (!pf->vf_state) {
-			pf->num_vfs = 0;
-			return -ENOMEM;
-		}
-
 		err = enetc_msg_psi_init(pf);
 		if (err) {
 			dev_err(&pdev->dev, "enetc_msg_psi_init (%d)\n", err);
@@ -696,7 +688,6 @@ static int enetc_sriov_configure(struct pci_dev *pdev, int num_vfs)
 err_en_sriov:
 	enetc_msg_psi_free(pf);
 err_msg_psi:
-	kfree(pf->vf_state);
 	pf->num_vfs = 0;
 
 	return err;
@@ -1286,6 +1277,12 @@ static int enetc_pf_probe(struct pci_dev *pdev,
 	pf = enetc_si_priv(si);
 	pf->si = si;
 	pf->total_vfs = pci_sriov_get_totalvfs(pdev);
+	if (pf->total_vfs) {
+		pf->vf_state = kcalloc(pf->total_vfs, sizeof(struct enetc_vf_state),
+				       GFP_KERNEL);
+		if (!pf->vf_state)
+			goto err_alloc_vf_state;
+	}
 
 	err = enetc_setup_mac_addresses(node, pf);
 	if (err)
@@ -1363,6 +1360,8 @@ static int enetc_pf_probe(struct pci_dev *pdev,
 	free_netdev(ndev);
 err_alloc_netdev:
 err_setup_mac_addresses:
+	kfree(pf->vf_state);
+err_alloc_vf_state:
 	enetc_psi_destroy(pdev);
 err_psi_create:
 	return err;
@@ -1389,6 +1388,7 @@ static void enetc_pf_remove(struct pci_dev *pdev)
 	enetc_free_si_resources(priv);
 
 	free_netdev(si->ndev);
+	kfree(pf->vf_state);
 
 	enetc_psi_destroy(pdev);
 }
-- 
2.34.1

[PATCH net 2/2] net: enetc: prevent PF from configuring MAC address for an enabled VF

[Wei Fang]

If PF changes the MAC address of VF after VF is enabled, VF cannot sense
the change of its own MAC address, and the MAC address in VF's net_device
is still the original MAC address, which will cause the VF's network to
not work properly. Therefore, we should restrict PF to configure VF's MAC
address only when VF is disabled.

Of course, another solution is to notify VF of this event when PF changes
VF's MAC address, but the PSI-to-VSI messaging is not implemented in the
current PF and VF drivers, so this solution is not suitable at present,
If PSI-to-VSI messaging is supported in the future, we can remove the
current restriction to enhance the PF's ability to configure the VF's MAC
address.

Fixes: d4fd0404c1c9 ("enetc: Introduce basic PF and VF ENETC ethernet drivers")
Signed-off-by: Wei Fang <wei.fang@nxp.com>
---
 drivers/net/ethernet/freescale/enetc/enetc_pf.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/drivers/net/ethernet/freescale/enetc/enetc_pf.c b/drivers/net/ethernet/freescale/enetc/enetc_pf.c
index c95a7c083b0f..a295236cd931 100644
--- a/drivers/net/ethernet/freescale/enetc/enetc_pf.c
+++ b/drivers/net/ethernet/freescale/enetc/enetc_pf.c
@@ -343,11 +343,18 @@ static int enetc_pf_set_vf_mac(struct net_device *ndev, int vf, u8 *mac)
 {
 	struct enetc_ndev_priv *priv = netdev_priv(ndev);
 	struct enetc_pf *pf = enetc_si_priv(priv->si);
+	struct device *dev = &pf->si->pdev->dev;
 	struct enetc_vf_state *vf_state;
 
 	if (vf >= pf->total_vfs)
 		return -EINVAL;
 
+	if (vf + 1 <= pf->num_vfs) {
+		dev_err(dev, "Cannot set MAC address for an enabled VF\n");
+
+		return -EPERM;
+	}
+
 	if (!is_valid_ether_addr(mac))
 		return -EADDRNOTAVAIL;
 
-- 
2.34.1

Re: [PATCH net 1/2] net: enetc: allocate vf_state during PF probes

[Vladimir Oltean]

On Thu, Oct 31, 2024 at 02:02:46PM +0800, Wei Fang wrote:
> In the previous implementation, vf_state is allocated memory only when VF
> is enabled. However, net_device_ops::ndo_set_vf_mac() may be called before
> VF is enabled to configure the MAC address of VF. If this is the case,
> enetc_pf_set_vf_mac() will access vf_state, resulting in access to a null
> pointer. The simplified error log is as follows.
> 
> root@ls1028ardb:~# ip link set eno0 vf 1 mac 00:0c:e7:66:77:89
> [  173.543315] Unable to handle kernel NULL pointer dereference at virtual address 0000000000000004
> [  173.637254] pc : enetc_pf_set_vf_mac+0x3c/0x80 Message from sy
> [  173.641973] lr : do_setlink+0x4a8/0xec8
> [  173.732292] Call trace:
> [  173.734740]  enetc_pf_set_vf_mac+0x3c/0x80
> [  173.738847]  __rtnl_newlink+0x530/0x89c
> [  173.742692]  rtnl_newlink+0x50/0x7c
> [  173.746189]  rtnetlink_rcv_msg+0x128/0x390
> [  173.750298]  netlink_rcv_skb+0x60/0x130
> [  173.754145]  rtnetlink_rcv+0x18/0x24
> [  173.757731]  netlink_unicast+0x318/0x380
> [  173.761665]  netlink_sendmsg+0x17c/0x3c8
> 
> Fixes: d4fd0404c1c9 ("enetc: Introduce basic PF and VF ENETC ethernet drivers")
> Signed-off-by: Wei Fang <wei.fang@nxp.com>
> ---

Reviewed-by: Vladimir Oltean <vladimir.oltean@nxp.com>
Tested-by: Vladimir Oltean <vladimir.oltean@nxp.com>

Re: [PATCH net 2/2] net: enetc: prevent PF from configuring MAC address for an enabled VF

[Vladimir Oltean]

On Thu, Oct 31, 2024 at 02:02:47PM +0800, Wei Fang wrote:
> If PF changes the MAC address of VF after VF is enabled, VF cannot sense
> the change of its own MAC address, and the MAC address in VF's net_device
> is still the original MAC address, which will cause the VF's network to
> not work properly. Therefore, we should restrict PF to configure VF's MAC
> address only when VF is disabled.
> 
> Of course, another solution is to notify VF of this event when PF changes
> VF's MAC address, but the PSI-to-VSI messaging is not implemented in the
> current PF and VF drivers, so this solution is not suitable at present,
> If PSI-to-VSI messaging is supported in the future, we can remove the
> current restriction to enhance the PF's ability to configure the VF's MAC
> address.
> 
> Fixes: d4fd0404c1c9 ("enetc: Introduce basic PF and VF ENETC ethernet drivers")
> Signed-off-by: Wei Fang <wei.fang@nxp.com>
> ---

Reviewed-by: Vladimir Oltean <vladimir.oltean@nxp.com>
Tested-by: Vladimir Oltean <vladimir.oltean@nxp.com>

Re: [PATCH net 0/2] Fix issues when PF sets MAC address for VF

[Jakub Kicinski]

On Thu, 31 Oct 2024 14:02:45 +0800 Wei Fang wrote:
>   net: enetc: allocate vf_state during PF probes
>   net: enetc: prevent PF from configuring MAC address for an enabled VF

This combination of changes would imply that nobody sets the MAC
address on VFs via this driver, correct? Patch 1 fixes a crash
if address is set before VFs are enabled, patch 2 forces setting
the MAC before VFs are enabled (which would previously crash).
Which leads me to believe this will cause regressions to all users,
if such users exist.

The fact that the MAC address is not picked up by a running VM is
normal, I'd say even maybe expected. IIUC hypervisor will enable 
SRIOV at the start of day, then allocate, configure and assign VFs
to VMs. It will FLR the VF after configuration.

Your change will make it impossible to reconfigure VF with a MAC
of a new VM, if any other VF is in use.

Long story short, I don't believe the patch 2 is necessary at all,
maybe you can print a warning to the logs, if you really want.

Re: [PATCH net 0/2] Fix issues when PF sets MAC address for VF

[patchwork-bot+netdevbpf]

Hello:

This series was applied to netdev/net.git (main)
by Jakub Kicinski <kuba@kernel.org>:

On Thu, 31 Oct 2024 14:02:45 +0800 you wrote:
> The ENETC PF driver provides enetc_pf_set_vf_mac() to configure the MAC
> address for the ENETC VF, but there are two issues when configuring the
> MAC address of the VF through this interface. For specific issues, please
> refer to the commit message of the following two patches. Therefore, this
> patch set is used to fix these two issues.
> 
> Wei Fang (2):
>   net: enetc: allocate vf_state during PF probes
>   net: enetc: prevent PF from configuring MAC address for an enabled VF
> 
> [...]

Here is the summary with links:
  - [net,1/2] net: enetc: allocate vf_state during PF probes
    https://git.kernel.org/netdev/net/c/e15c5506dd39
  - [net,2/2] net: enetc: prevent PF from configuring MAC address for an enabled VF
    (no matching commit)

You are awesome, thank you!
-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html

RE: [PATCH net 0/2] Fix issues when PF sets MAC address for VF

[Wei Fang]

> -----Original Message-----
> From: Jakub Kicinski <kuba@kernel.org>
> Sent: 2024年11月4日 4:51
> To: Wei Fang <wei.fang@nxp.com>
> Cc: davem@davemloft.net; edumazet@google.com; pabeni@redhat.com;
> andrew+netdev@lunn.ch; Claudiu Manoil <claudiu.manoil@nxp.com>;
> Vladimir Oltean <vladimir.oltean@nxp.com>; netdev@vger.kernel.org;
> linux-kernel@vger.kernel.org; imx@lists.linux.dev
> Subject: Re: [PATCH net 0/2] Fix issues when PF sets MAC address for VF
> 
> On Thu, 31 Oct 2024 14:02:45 +0800 Wei Fang wrote:
> >   net: enetc: allocate vf_state during PF probes
> >   net: enetc: prevent PF from configuring MAC address for an enabled VF
> 
> This combination of changes would imply that nobody sets the MAC
> address on VFs via this driver, correct? Patch 1 fixes a crash
> if address is set before VFs are enabled, patch 2 forces setting
> the MAC before VFs are enabled (which would previously crash).
> Which leads me to believe this will cause regressions to all users,
> if such users exist.

To be honest, I don't know much about the LS1028A platform. Apparently,
Before this patch, users could only modify the MAC address of VF through
the PF driver after the VF is enabled. However, the VF driver only obtains
the MAC address from the register and set it into net_device during probe.
Since VF has been enabled, the VF driver has already completed the probe.
Therefore, if users want to the VF network to work properly, they need to
re-probe the VF driver. If users do disable/enable SRIOV or unbind/bind
VF driver to re-probe the VF driver, such as you mentioned below, assign
the VF to a VM, patch 2 will indeed cause a regression issue.

> 
> The fact that the MAC address is not picked up by a running VM is
> normal, I'd say even maybe expected. IIUC hypervisor will enable
> SRIOV at the start of day, then allocate, configure and assign VFs
> to VMs. It will FLR the VF after configuration.
> 
> Your change will make it impossible to reconfigure VF with a MAC
> of a new VM, if any other VF is in use.
> 
> Long story short, I don't believe the patch 2 is necessary at all,
> maybe you can print a warning to the logs, if you really want.

I think your concern is reasonable, the best way is that we add the
PSI-to-VSI messaging in the future, so that is wont cause regression
issue. Thanks.