[PATCH net-next] ixgbe: make macvlan on PF working when SRIOV is enabled

[PATCH net-next] ixgbe: make macvlan on PF working when SRIOV is enabled

[Shirley Ma]

When SRIOV is enabled, if we create macvlan on PF and load the VF driver in host 
domain, when macvlan/PF and VF interfaces are all up, the macvlan incoming network
traffics are directed to VF interface not PF interface. This patch has fixed this
issue.

Signed-off-by: Shirley Ma <xma@us.ibm.com>
--- 
 drivers/net/ixgbe/ixgbe_common.c |   26 ++++++++++++++++----------
 1 files changed, 16 insertions(+), 10 deletions(-)

diff --git a/drivers/net/ixgbe/ixgbe_common.c b/drivers/net/ixgbe/ixgbe_common.c
index 1159d91..591dd19 100644
--- a/drivers/net/ixgbe/ixgbe_common.c
+++ b/drivers/net/ixgbe/ixgbe_common.c
@@ -1271,6 +1271,7 @@ s32 ixgbe_init_rx_addrs_generic(struct ixgbe_hw *hw)
 {
 	u32 i;
 	u32 rar_entries = hw->mac.num_rar_entries;
+	struct ixgbe_adapter *adapter = hw->back;
 
 	/*
 	 * If the current mac address is valid, assume it is a software override
@@ -1288,15 +1289,18 @@ s32 ixgbe_init_rx_addrs_generic(struct ixgbe_hw *hw)
 		hw_dbg(hw, "Overriding MAC Address in RAR[0]\n");
 		hw_dbg(hw, " New MAC Addr =%pM\n", hw->mac.addr);
 
-		hw->mac.ops.set_rar(hw, 0, hw->mac.addr, 0, IXGBE_RAH_AV);
+		hw->mac.ops.set_rar(hw, 0, hw->mac.addr, adapter->num_vfs,
+				    IXGBE_RAH_AV);
 	}
 	hw->addr_ctrl.overflow_promisc = 0;
 
-	hw->addr_ctrl.rar_used_count = 1;
+	/* reserve VFs receive address entries if any */
+	hw->addr_ctrl.rar_used_count = adapter->num_vfs + 1;
 
-	/* Zero out the other receive addresses. */
-	hw_dbg(hw, "Clearing RAR[1-%d]\n", rar_entries - 1);
-	for (i = 1; i < rar_entries; i++) {
+	/* Zero out the other receive addresses except VFs if any */
+	hw_dbg(hw, "Clearing RAR[%d-%d]\n",
+	       adapter->num_vfs + 1, rar_entries - 1);
+	for (i = adapter->num_vfs + 1; i < rar_entries; i++) {
 		IXGBE_WRITE_REG(hw, IXGBE_RAL(i), 0);
 		IXGBE_WRITE_REG(hw, IXGBE_RAH(i), 0);
 	}
@@ -1368,18 +1372,20 @@ s32 ixgbe_update_uc_addr_list_generic(struct ixgbe_hw *hw,
 	u32 uc_addr_in_use;
 	u32 fctrl;
 	struct netdev_hw_addr *ha;
+	struct ixgbe_adapter *adapter = hw->back;
 
 	/*
 	 * Clear accounting of old secondary address list,
 	 * don't count RAR[0]
 	 */
-	uc_addr_in_use = hw->addr_ctrl.rar_used_count - 1;
+	uc_addr_in_use = hw->addr_ctrl.rar_used_count - 1 - adapter->num_vfs;
 	hw->addr_ctrl.rar_used_count -= uc_addr_in_use;
 	hw->addr_ctrl.overflow_promisc = 0;
 
-	/* Zero out the other receive addresses */
-	hw_dbg(hw, "Clearing RAR[1-%d]\n", uc_addr_in_use + 1);
-	for (i = 0; i < uc_addr_in_use; i++) {
+	/* Zero out the other receive addresses except VFs if any */
+	hw_dbg(hw, "Clearing RAR[%d-%d]\n",
+	       adapter->num_vfs + 1, uc_addr_in_use + adapter->num_vfs + 1);
+	for (i = adapter->num_vfs; i < uc_addr_in_use + adapter->num_vfs; i++) {
 		IXGBE_WRITE_REG(hw, IXGBE_RAL(1+i), 0);
 		IXGBE_WRITE_REG(hw, IXGBE_RAH(1+i), 0);
 	}
@@ -1387,7 +1393,7 @@ s32 ixgbe_update_uc_addr_list_generic(struct ixgbe_hw *hw,
 	/* Add the new addresses */
 	netdev_for_each_uc_addr(ha, netdev) {
 		hw_dbg(hw, " Adding the secondary addresses:\n");
-		ixgbe_add_uc_addr(hw, ha->addr, 0);
+		ixgbe_add_uc_addr(hw, ha->addr, adapter->num_vfs);
 	}
 
 	if (hw->addr_ctrl.overflow_promisc) {




------------------------------------------------------------------------------

_______________________________________________
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel&#174; Ethernet, visit http://communities.intel.com/community/wired

Re: [PATCH net-next] ixgbe: make macvlan on PF working when SRIOV is enabled

[Shirley Ma]

To produce this problem:

1. modprobe ixgbe max_vfs=2 
   eth4 is PF, eth5 is VF
2. ip link set eth4 up
3. ip link add link eth4 address 54:52:00:35:e3:20 macvlan2 type macvlan
4. ip addr add 192.168.7.74/24 dev macvlan2
5. ping macvlan2 from remote host, works
6. ip link set eth5 up
7. ping macvlan2 from remote host failed.

Based on my understanding, the problem is:
1. PF set_rar use rar index is 0, and vmdq index is adapter->num_vfs, 
2. when macvlan2 is created, rar index is based rar_used_count, which
would be 1.
3. later when VF is up, the rar index is vf+1, and vmdq index is vf, so
VF0 will overwrite macvlan2 rar entry.

The fix here:
1. make sure PF uses vmdq index = adapter->num_vfs during
initialization, reset.
2. reserve rar index for all VFs from 1 to num_vfs + 1.


Please let me know whether my understanding is correct or not.

Thanks
Shirley


------------------------------------------------------------------------------

_______________________________________________
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel® Ethernet, visit http://communities.intel.com/community/wired

RE: [PATCH net-next] ixgbe: make macvlan on PF working when SRIOV is enabled

[Rose, Gregory V]

>-----Original Message-----
>From: Shirley Ma [mailto:mashirle@us.ibm.com]
>Sent: Tuesday, May 25, 2010 10:16 AM
>To: Kirsher, Jeffrey T
>Cc: Rose, Gregory V; davem@davemloft.net; kvm@vger.kernel.org;
>netdev@vger.kernel.org; e1000-devel@lists.sourceforge.net
>Subject: Re: [PATCH net-next] ixgbe: make macvlan on PF working when
>SRIOV is enabled
>
>To produce this problem:
>
>1. modprobe ixgbe max_vfs=2
>   eth4 is PF, eth5 is VF
>2. ip link set eth4 up
>3. ip link add link eth4 address 54:52:00:35:e3:20 macvlan2 type macvlan
>4. ip addr add 192.168.7.74/24 dev macvlan2
>5. ping macvlan2 from remote host, works
>6. ip link set eth5 up
>7. ping macvlan2 from remote host failed.
>
>Based on my understanding, the problem is:
>1. PF set_rar use rar index is 0, and vmdq index is adapter->num_vfs,
>2. when macvlan2 is created, rar index is based rar_used_count, which
>would be 1.
>3. later when VF is up, the rar index is vf+1, and vmdq index is vf, so
>VF0 will overwrite macvlan2 rar entry.
>
>The fix here:
>1. make sure PF uses vmdq index = adapter->num_vfs during
>initialization, reset.
>2. reserve rar index for all VFs from 1 to num_vfs + 1.
>
>
>Please let me know whether my understanding is correct or not.

Yes, that appears to be correct.

We'll test your patch but I think you're on the right track.

- Greg

Re: [PATCH net-next] ixgbe: make macvlan on PF working when SRIOV is enabled

[Jeff Kirsher]

On Tue, May 25, 2010 at 09:55, Shirley Ma <mashirle@us.ibm.com> wrote:
> When SRIOV is enabled, if we create macvlan on PF and load the VF driver in host
> domain, when macvlan/PF and VF interfaces are all up, the macvlan incoming network
> traffics are directed to VF interface not PF interface. This patch has fixed this
> issue.
>
> Signed-off-by: Shirley Ma <xma@us.ibm.com>
> ---
>  drivers/net/ixgbe/ixgbe_common.c |   26 ++++++++++++++++----------
>  1 files changed, 16 insertions(+), 10 deletions(-)
>

Thanks, I have added the patch to my queue.

-- 
Cheers,
Jeff