From: Simon Horman <horms@kernel.org>
To: Ivan Vecera <ivecera@redhat.com>
Cc: Wojciech Drewek <wojciech.drewek@intel.com>,
netdev@vger.kernel.org,
Jesse Brandeburg <jesse.brandeburg@intel.com>,
linux-kernel@vger.kernel.org, Eric Dumazet <edumazet@google.com>,
Tony Nguyen <anthony.l.nguyen@intel.com>,
Jakub Kicinski <kuba@kernel.org>,
Jacob Keller <jacob.e.keller@intel.com>,
intel-wired-lan@lists.osuosl.org, Paolo Abeni <pabeni@redhat.com>,
"David S. Miller" <davem@davemloft.net>
Subject: Re: [Intel-wired-lan] [PATCH v5 5/5] i40e: Remove VEB recursion
Date: Sun, 3 Dec 2023 16:22:48 +0000 [thread overview]
Message-ID: <20231203162248.GJ50400@kernel.org> (raw)
In-Reply-To: <20231124150343.81520-6-ivecera@redhat.com>
On Fri, Nov 24, 2023 at 04:03:43PM +0100, Ivan Vecera wrote:
> The VEB (virtual embedded switch) as a switch element can be
> connected according datasheet though its uplink to:
> - Physical port
> - Port Virtualizer (not used directly by i40e driver but can
> be present in MFP mode where the physical port is shared
> between PFs)
> - No uplink (aka floating VEB)
>
> But VEB uplink cannot be connected to another VEB and any attempt
> to do so results in:
>
> "i40e 0000:02:00.0: couldn't add VEB, err -EIO aq_err I40E_AQ_RC_ENOENT"
>
> that indicates "the uplink SEID does not point to valid element".
>
> Remove this logic from the driver code this way:
>
> 1) For debugfs only allow to build floating VEB (uplink_seid == 0)
> or main VEB (uplink_seid == mac_seid)
> 2) Do not recurse in i40e_veb_link_event() as no VEB cannot have
> sub-VEBs
> 3) Ditto for i40e_veb_rebuild() + simplify the function as we know
> that the VEB for rebuild can be only the main LAN VEB or some
> of the floating VEBs
> 4) In i40e_rebuild() there is no need to check veb->uplink_seid
> as the possible ones are 0 and MAC SEID
> 5) In i40e_vsi_release() do not take into account VEBs whose
> uplink is another VEB as this is not possible
> 6) Remove veb_idx field from i40e_veb as a VEB cannot have
> sub-VEBs
>
> Tested using i40e debugfs interface:
> 1) Initial state
> [root@cnb-03 net-next]# CMD="/sys/kernel/debug/i40e/0000:02:00.0/command"
> [root@cnb-03 net-next]# echo dump switch > $CMD
> [root@cnb-03 net-next]# dmesg -c
> [ 98.440641] i40e 0000:02:00.0: header: 3 reported 3 total
> [ 98.446053] i40e 0000:02:00.0: type=19 seid=392 uplink=160 downlink=16
> [ 98.452593] i40e 0000:02:00.0: type=17 seid=160 uplink=2 downlink=0
> [ 98.458856] i40e 0000:02:00.0: type=19 seid=390 uplink=160 downlink=16
>
> 2) Add floating VEB
> [root@cnb-03 net-next]# echo add relay > $CMD
> [root@cnb-03 net-next]# dmesg -c
> [ 122.745630] i40e 0000:02:00.0: added relay 162
> [root@cnb-03 net-next]# echo dump switch > $CMD
> [root@cnb-03 net-next]# dmesg -c
> [ 136.650049] i40e 0000:02:00.0: header: 4 reported 4 total
> [ 136.655466] i40e 0000:02:00.0: type=19 seid=392 uplink=160 downlink=16
> [ 136.661994] i40e 0000:02:00.0: type=17 seid=160 uplink=2 downlink=0
> [ 136.668264] i40e 0000:02:00.0: type=19 seid=390 uplink=160 downlink=16
> [ 136.674787] i40e 0000:02:00.0: type=17 seid=162 uplink=0 downlink=0
>
> 3) Add VMDQ2 VSI to this new VEB
> [root@cnb-03 net-next]# dmesg -c
> [ 168.351763] i40e 0000:02:00.0: added VSI 394 to relay 162
> [ 168.374652] enp2s0f0np0v0: NIC Link is Up, 40 Gbps Full Duplex, Flow Control: None
> [root@cnb-03 net-next]# echo dump switch > $CMD
> [root@cnb-03 net-next]# dmesg -c
> [ 195.683204] i40e 0000:02:00.0: header: 5 reported 5 total
> [ 195.688611] i40e 0000:02:00.0: type=19 seid=394 uplink=162 downlink=16
> [ 195.695143] i40e 0000:02:00.0: type=17 seid=162 uplink=0 downlink=0
> [ 195.701410] i40e 0000:02:00.0: type=19 seid=392 uplink=160 downlink=16
> [ 195.707935] i40e 0000:02:00.0: type=17 seid=160 uplink=2 downlink=0
> [ 195.714201] i40e 0000:02:00.0: type=19 seid=390 uplink=160 downlink=16
>
> 4) Try to delete the VEB
> [root@cnb-03 net-next]# echo del relay 162 > $CMD
> [root@cnb-03 net-next]# dmesg -c
> [ 239.260901] i40e 0000:02:00.0: deleting relay 162
> [ 239.265621] i40e 0000:02:00.0: can't remove VEB 162 with 1 VSIs left
>
> 5) Do PF reset and check switch status after rebuild
> [root@cnb-03 net-next]# echo pfr > $CMD
> [root@cnb-03 net-next]# echo dump switch > $CMD
> [root@cnb-03 net-next]# dmesg -c
> ...
> [ 272.333655] i40e 0000:02:00.0: header: 5 reported 5 total
> [ 272.339066] i40e 0000:02:00.0: type=19 seid=394 uplink=162 downlink=16
> [ 272.345599] i40e 0000:02:00.0: type=17 seid=162 uplink=0 downlink=0
> [ 272.351862] i40e 0000:02:00.0: type=19 seid=392 uplink=160 downlink=16
> [ 272.358387] i40e 0000:02:00.0: type=17 seid=160 uplink=2 downlink=0
> [ 272.364654] i40e 0000:02:00.0: type=19 seid=390 uplink=160 downlink=16
>
> 6) Delete VSI and delete VEB
> [ 297.199116] i40e 0000:02:00.0: deleting VSI 394
> [ 299.807580] i40e 0000:02:00.0: deleting relay 162
> [ 309.767905] i40e 0000:02:00.0: header: 3 reported 3 total
> [ 309.773318] i40e 0000:02:00.0: type=19 seid=392 uplink=160 downlink=16
> [ 309.779845] i40e 0000:02:00.0: type=17 seid=160 uplink=2 downlink=0
> [ 309.786111] i40e 0000:02:00.0: type=19 seid=390 uplink=160 downlink=16
>
> Reviewed-by: Wojciech Drewek <wojciech.drewek@intel.com>
> Signed-off-by: Ivan Vecera <ivecera@redhat.com>
Reviewed-by: Simon Horman <horms@kernel.org>
_______________________________________________
Intel-wired-lan mailing list
Intel-wired-lan@osuosl.org
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
WARNING: multiple messages have this Message-ID (diff)
From: Simon Horman <horms@kernel.org>
To: Ivan Vecera <ivecera@redhat.com>
Cc: intel-wired-lan@lists.osuosl.org,
Jesse Brandeburg <jesse.brandeburg@intel.com>,
Tony Nguyen <anthony.l.nguyen@intel.com>,
"David S. Miller" <davem@davemloft.net>,
Eric Dumazet <edumazet@google.com>,
Jakub Kicinski <kuba@kernel.org>, Paolo Abeni <pabeni@redhat.com>,
linux-kernel@vger.kernel.org,
Jacob Keller <jacob.e.keller@intel.com>,
Wojciech Drewek <wojciech.drewek@intel.com>,
mschmidt@redhat.com, netdev@vger.kernel.org
Subject: Re: [PATCH v5 5/5] i40e: Remove VEB recursion
Date: Sun, 3 Dec 2023 16:22:48 +0000 [thread overview]
Message-ID: <20231203162248.GJ50400@kernel.org> (raw)
In-Reply-To: <20231124150343.81520-6-ivecera@redhat.com>
On Fri, Nov 24, 2023 at 04:03:43PM +0100, Ivan Vecera wrote:
> The VEB (virtual embedded switch) as a switch element can be
> connected according datasheet though its uplink to:
> - Physical port
> - Port Virtualizer (not used directly by i40e driver but can
> be present in MFP mode where the physical port is shared
> between PFs)
> - No uplink (aka floating VEB)
>
> But VEB uplink cannot be connected to another VEB and any attempt
> to do so results in:
>
> "i40e 0000:02:00.0: couldn't add VEB, err -EIO aq_err I40E_AQ_RC_ENOENT"
>
> that indicates "the uplink SEID does not point to valid element".
>
> Remove this logic from the driver code this way:
>
> 1) For debugfs only allow to build floating VEB (uplink_seid == 0)
> or main VEB (uplink_seid == mac_seid)
> 2) Do not recurse in i40e_veb_link_event() as no VEB cannot have
> sub-VEBs
> 3) Ditto for i40e_veb_rebuild() + simplify the function as we know
> that the VEB for rebuild can be only the main LAN VEB or some
> of the floating VEBs
> 4) In i40e_rebuild() there is no need to check veb->uplink_seid
> as the possible ones are 0 and MAC SEID
> 5) In i40e_vsi_release() do not take into account VEBs whose
> uplink is another VEB as this is not possible
> 6) Remove veb_idx field from i40e_veb as a VEB cannot have
> sub-VEBs
>
> Tested using i40e debugfs interface:
> 1) Initial state
> [root@cnb-03 net-next]# CMD="/sys/kernel/debug/i40e/0000:02:00.0/command"
> [root@cnb-03 net-next]# echo dump switch > $CMD
> [root@cnb-03 net-next]# dmesg -c
> [ 98.440641] i40e 0000:02:00.0: header: 3 reported 3 total
> [ 98.446053] i40e 0000:02:00.0: type=19 seid=392 uplink=160 downlink=16
> [ 98.452593] i40e 0000:02:00.0: type=17 seid=160 uplink=2 downlink=0
> [ 98.458856] i40e 0000:02:00.0: type=19 seid=390 uplink=160 downlink=16
>
> 2) Add floating VEB
> [root@cnb-03 net-next]# echo add relay > $CMD
> [root@cnb-03 net-next]# dmesg -c
> [ 122.745630] i40e 0000:02:00.0: added relay 162
> [root@cnb-03 net-next]# echo dump switch > $CMD
> [root@cnb-03 net-next]# dmesg -c
> [ 136.650049] i40e 0000:02:00.0: header: 4 reported 4 total
> [ 136.655466] i40e 0000:02:00.0: type=19 seid=392 uplink=160 downlink=16
> [ 136.661994] i40e 0000:02:00.0: type=17 seid=160 uplink=2 downlink=0
> [ 136.668264] i40e 0000:02:00.0: type=19 seid=390 uplink=160 downlink=16
> [ 136.674787] i40e 0000:02:00.0: type=17 seid=162 uplink=0 downlink=0
>
> 3) Add VMDQ2 VSI to this new VEB
> [root@cnb-03 net-next]# dmesg -c
> [ 168.351763] i40e 0000:02:00.0: added VSI 394 to relay 162
> [ 168.374652] enp2s0f0np0v0: NIC Link is Up, 40 Gbps Full Duplex, Flow Control: None
> [root@cnb-03 net-next]# echo dump switch > $CMD
> [root@cnb-03 net-next]# dmesg -c
> [ 195.683204] i40e 0000:02:00.0: header: 5 reported 5 total
> [ 195.688611] i40e 0000:02:00.0: type=19 seid=394 uplink=162 downlink=16
> [ 195.695143] i40e 0000:02:00.0: type=17 seid=162 uplink=0 downlink=0
> [ 195.701410] i40e 0000:02:00.0: type=19 seid=392 uplink=160 downlink=16
> [ 195.707935] i40e 0000:02:00.0: type=17 seid=160 uplink=2 downlink=0
> [ 195.714201] i40e 0000:02:00.0: type=19 seid=390 uplink=160 downlink=16
>
> 4) Try to delete the VEB
> [root@cnb-03 net-next]# echo del relay 162 > $CMD
> [root@cnb-03 net-next]# dmesg -c
> [ 239.260901] i40e 0000:02:00.0: deleting relay 162
> [ 239.265621] i40e 0000:02:00.0: can't remove VEB 162 with 1 VSIs left
>
> 5) Do PF reset and check switch status after rebuild
> [root@cnb-03 net-next]# echo pfr > $CMD
> [root@cnb-03 net-next]# echo dump switch > $CMD
> [root@cnb-03 net-next]# dmesg -c
> ...
> [ 272.333655] i40e 0000:02:00.0: header: 5 reported 5 total
> [ 272.339066] i40e 0000:02:00.0: type=19 seid=394 uplink=162 downlink=16
> [ 272.345599] i40e 0000:02:00.0: type=17 seid=162 uplink=0 downlink=0
> [ 272.351862] i40e 0000:02:00.0: type=19 seid=392 uplink=160 downlink=16
> [ 272.358387] i40e 0000:02:00.0: type=17 seid=160 uplink=2 downlink=0
> [ 272.364654] i40e 0000:02:00.0: type=19 seid=390 uplink=160 downlink=16
>
> 6) Delete VSI and delete VEB
> [ 297.199116] i40e 0000:02:00.0: deleting VSI 394
> [ 299.807580] i40e 0000:02:00.0: deleting relay 162
> [ 309.767905] i40e 0000:02:00.0: header: 3 reported 3 total
> [ 309.773318] i40e 0000:02:00.0: type=19 seid=392 uplink=160 downlink=16
> [ 309.779845] i40e 0000:02:00.0: type=17 seid=160 uplink=2 downlink=0
> [ 309.786111] i40e 0000:02:00.0: type=19 seid=390 uplink=160 downlink=16
>
> Reviewed-by: Wojciech Drewek <wojciech.drewek@intel.com>
> Signed-off-by: Ivan Vecera <ivecera@redhat.com>
Reviewed-by: Simon Horman <horms@kernel.org>
next prev parent reply other threads:[~2023-12-03 16:23 UTC|newest]
Thread overview: 36+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-11-24 15:03 [Intel-wired-lan] [PATCH iwl-next v5 0/5] i40e: Simplify VSI and VEB handling Ivan Vecera
2023-11-24 15:03 ` Ivan Vecera
2023-11-24 15:03 ` [Intel-wired-lan] [PATCH v5 1/5] i40e: Use existing helper to find flow director VSI Ivan Vecera
2023-11-24 15:03 ` Ivan Vecera
2023-11-30 5:23 ` [Intel-wired-lan] " Pucha, HimasekharX Reddy
2023-11-30 5:23 ` Pucha, HimasekharX Reddy
2023-12-03 11:41 ` Simon Horman
2023-12-03 11:41 ` Simon Horman
2023-11-24 15:03 ` [Intel-wired-lan] [PATCH v5 2/5] i40e: Introduce and use macros for iterating VSIs and VEBs Ivan Vecera
2023-11-24 15:03 ` Ivan Vecera
2023-11-30 5:23 ` [Intel-wired-lan] " Pucha, HimasekharX Reddy
2023-11-30 5:23 ` Pucha, HimasekharX Reddy
2023-12-03 11:41 ` Simon Horman
2023-12-03 11:41 ` Simon Horman
2023-11-24 15:03 ` [Intel-wired-lan] [PATCH v5 3/5] i40e: Add helpers to find VSI and VEB by SEID and use them Ivan Vecera
2023-11-24 15:03 ` Ivan Vecera
2023-11-30 5:23 ` [Intel-wired-lan] " Pucha, HimasekharX Reddy
2023-11-30 5:23 ` Pucha, HimasekharX Reddy
2023-12-03 11:41 ` Simon Horman
2023-12-03 11:41 ` Simon Horman
2023-11-24 15:03 ` [Intel-wired-lan] [PATCH v5 4/5] i40e: Fix broken support for floating VEBs Ivan Vecera
2023-11-24 15:03 ` Ivan Vecera
2023-11-27 8:43 ` [Intel-wired-lan] " Dan Carpenter
2023-11-27 8:43 ` Dan Carpenter
2023-11-29 8:38 ` [Intel-wired-lan] " Ivan Vecera
2023-11-29 8:38 ` Ivan Vecera
2023-11-30 5:23 ` [Intel-wired-lan] " Pucha, HimasekharX Reddy
2023-11-30 5:23 ` Pucha, HimasekharX Reddy
2023-12-03 16:22 ` Simon Horman
2023-12-03 16:22 ` Simon Horman
2023-11-24 15:03 ` [Intel-wired-lan] [PATCH v5 5/5] i40e: Remove VEB recursion Ivan Vecera
2023-11-24 15:03 ` Ivan Vecera
2023-11-30 5:23 ` [Intel-wired-lan] " Pucha, HimasekharX Reddy
2023-11-30 5:23 ` Pucha, HimasekharX Reddy
2023-12-03 16:22 ` Simon Horman [this message]
2023-12-03 16:22 ` Simon Horman
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20231203162248.GJ50400@kernel.org \
--to=horms@kernel.org \
--cc=anthony.l.nguyen@intel.com \
--cc=davem@davemloft.net \
--cc=edumazet@google.com \
--cc=intel-wired-lan@lists.osuosl.org \
--cc=ivecera@redhat.com \
--cc=jacob.e.keller@intel.com \
--cc=jesse.brandeburg@intel.com \
--cc=kuba@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=netdev@vger.kernel.org \
--cc=pabeni@redhat.com \
--cc=wojciech.drewek@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.