* [PATCH net-next 1/3] i40e: optimize for XDP_REDIRECT in xsk path
2021-03-16 16:42 [PATCH net-next 0/3][pull request] 40GbE Intel Wired LAN Driver Updates 2021-03-16 Tony Nguyen
@ 2021-03-16 16:42 ` Tony Nguyen
2021-03-16 16:42 ` [PATCH net-next 2/3] ixgbe: " Tony Nguyen
` (3 subsequent siblings)
4 siblings, 0 replies; 7+ messages in thread
From: Tony Nguyen @ 2021-03-16 16:42 UTC (permalink / raw)
To: davem, kuba
Cc: Magnus Karlsson, netdev, sassmann, anthony.l.nguyen, bjorn.topel,
maciej.fijalkowski, George Kuruvinakunnel
From: Magnus Karlsson <magnus.karlsson@intel.com>
Optimize i40e_run_xdp_zc() for the XDP program verdict being
XDP_REDIRECT in the xsk zero-copy path. This path is only used when
having AF_XDP zero-copy on and in that case most packets will be
directed to user space. This provides a little over 100k extra packets
in throughput on my server when running l2fwd in xdpsock.
Signed-off-by: Magnus Karlsson <magnus.karlsson@intel.com>
Tested-by: George Kuruvinakunnel <george.kuruvinakunnel@intel.com>
Signed-off-by: Tony Nguyen <anthony.l.nguyen@intel.com>
---
drivers/net/ethernet/intel/i40e/i40e_xsk.c | 11 +++++++----
1 file changed, 7 insertions(+), 4 deletions(-)
diff --git a/drivers/net/ethernet/intel/i40e/i40e_xsk.c b/drivers/net/ethernet/intel/i40e/i40e_xsk.c
index fc32c5019b0f..edddaf034638 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_xsk.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_xsk.c
@@ -160,6 +160,13 @@ static int i40e_run_xdp_zc(struct i40e_ring *rx_ring, struct xdp_buff *xdp)
xdp_prog = READ_ONCE(rx_ring->xdp_prog);
act = bpf_prog_run_xdp(xdp_prog, xdp);
+ if (likely(act == XDP_REDIRECT)) {
+ err = xdp_do_redirect(rx_ring->netdev, xdp, xdp_prog);
+ result = !err ? I40E_XDP_REDIR : I40E_XDP_CONSUMED;
+ rcu_read_unlock();
+ return result;
+ }
+
switch (act) {
case XDP_PASS:
break;
@@ -167,10 +174,6 @@ static int i40e_run_xdp_zc(struct i40e_ring *rx_ring, struct xdp_buff *xdp)
xdp_ring = rx_ring->vsi->xdp_rings[rx_ring->queue_index];
result = i40e_xmit_xdp_tx_ring(xdp, xdp_ring);
break;
- case XDP_REDIRECT:
- err = xdp_do_redirect(rx_ring->netdev, xdp, xdp_prog);
- result = !err ? I40E_XDP_REDIR : I40E_XDP_CONSUMED;
- break;
default:
bpf_warn_invalid_xdp_action(act);
fallthrough;
--
2.26.2
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH net-next 2/3] ixgbe: optimize for XDP_REDIRECT in xsk path
2021-03-16 16:42 [PATCH net-next 0/3][pull request] 40GbE Intel Wired LAN Driver Updates 2021-03-16 Tony Nguyen
2021-03-16 16:42 ` [PATCH net-next 1/3] i40e: optimize for XDP_REDIRECT in xsk path Tony Nguyen
@ 2021-03-16 16:42 ` Tony Nguyen
2021-03-16 16:42 ` [PATCH net-next 3/3] ice: " Tony Nguyen
` (2 subsequent siblings)
4 siblings, 0 replies; 7+ messages in thread
From: Tony Nguyen @ 2021-03-16 16:42 UTC (permalink / raw)
To: davem, kuba
Cc: Magnus Karlsson, netdev, sassmann, anthony.l.nguyen, bjorn.topel,
maciej.fijalkowski, Vishakha Jambekar
From: Magnus Karlsson <magnus.karlsson@intel.com>
Optimize ixgbe_run_xdp_zc() for the XDP program verdict being
XDP_REDIRECT in the xsk zero-copy path. This path is only used when
having AF_XDP zero-copy on and in that case most packets will be
directed to user space. This provides a little under 100k extra
packets in throughput on my server when running l2fwd in xdpsock.
Signed-off-by: Magnus Karlsson <magnus.karlsson@intel.com>
Tested-by: Vishakha Jambekar <vishakha.jambekar@intel.com>
Signed-off-by: Tony Nguyen <anthony.l.nguyen@intel.com>
---
drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c | 11 +++++++----
1 file changed, 7 insertions(+), 4 deletions(-)
diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c
index 3771857cf887..91ad5b902673 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c
@@ -104,6 +104,13 @@ static int ixgbe_run_xdp_zc(struct ixgbe_adapter *adapter,
xdp_prog = READ_ONCE(rx_ring->xdp_prog);
act = bpf_prog_run_xdp(xdp_prog, xdp);
+ if (likely(act == XDP_REDIRECT)) {
+ err = xdp_do_redirect(rx_ring->netdev, xdp, xdp_prog);
+ result = !err ? IXGBE_XDP_REDIR : IXGBE_XDP_CONSUMED;
+ rcu_read_unlock();
+ return result;
+ }
+
switch (act) {
case XDP_PASS:
break;
@@ -115,10 +122,6 @@ static int ixgbe_run_xdp_zc(struct ixgbe_adapter *adapter,
}
result = ixgbe_xmit_xdp_ring(adapter, xdpf);
break;
- case XDP_REDIRECT:
- err = xdp_do_redirect(rx_ring->netdev, xdp, xdp_prog);
- result = !err ? IXGBE_XDP_REDIR : IXGBE_XDP_CONSUMED;
- break;
default:
bpf_warn_invalid_xdp_action(act);
fallthrough;
--
2.26.2
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH net-next 3/3] ice: optimize for XDP_REDIRECT in xsk path
2021-03-16 16:42 [PATCH net-next 0/3][pull request] 40GbE Intel Wired LAN Driver Updates 2021-03-16 Tony Nguyen
2021-03-16 16:42 ` [PATCH net-next 1/3] i40e: optimize for XDP_REDIRECT in xsk path Tony Nguyen
2021-03-16 16:42 ` [PATCH net-next 2/3] ixgbe: " Tony Nguyen
@ 2021-03-16 16:42 ` Tony Nguyen
2021-03-16 21:43 ` [PATCH net-next 0/3][pull request] 40GbE Intel Wired LAN Driver Updates 2021-03-16 Jakub Kicinski
2021-03-16 22:40 ` patchwork-bot+netdevbpf
4 siblings, 0 replies; 7+ messages in thread
From: Tony Nguyen @ 2021-03-16 16:42 UTC (permalink / raw)
To: davem, kuba
Cc: Magnus Karlsson, netdev, sassmann, anthony.l.nguyen, bjorn.topel,
maciej.fijalkowski, George Kuruvinakunnel
From: Magnus Karlsson <magnus.karlsson@intel.com>
Optimize ice_run_xdp_zc() for the XDP program verdict being
XDP_REDIRECT in the xsk zero-copy path. This path is only used when
having AF_XDP zero-copy on and in that case most packets will be
directed to user space. This provides a little over 100k extra packets
in throughput on my server when running l2fwd in xdpsock.
Signed-off-by: Magnus Karlsson <magnus.karlsson@intel.com>
Tested-by: George Kuruvinakunnel <george.kuruvinakunnel@intel.com>
Signed-off-by: Tony Nguyen <anthony.l.nguyen@intel.com>
---
drivers/net/ethernet/intel/ice/ice_xsk.c | 12 ++++++++----
1 file changed, 8 insertions(+), 4 deletions(-)
diff --git a/drivers/net/ethernet/intel/ice/ice_xsk.c b/drivers/net/ethernet/intel/ice/ice_xsk.c
index 83f3c9574ed1..727f277e9d75 100644
--- a/drivers/net/ethernet/intel/ice/ice_xsk.c
+++ b/drivers/net/ethernet/intel/ice/ice_xsk.c
@@ -473,6 +473,14 @@ ice_run_xdp_zc(struct ice_ring *rx_ring, struct xdp_buff *xdp)
xdp_prog = READ_ONCE(rx_ring->xdp_prog);
act = bpf_prog_run_xdp(xdp_prog, xdp);
+
+ if (likely(act == XDP_REDIRECT)) {
+ err = xdp_do_redirect(rx_ring->netdev, xdp, xdp_prog);
+ result = !err ? ICE_XDP_REDIR : ICE_XDP_CONSUMED;
+ rcu_read_unlock();
+ return result;
+ }
+
switch (act) {
case XDP_PASS:
break;
@@ -480,10 +488,6 @@ ice_run_xdp_zc(struct ice_ring *rx_ring, struct xdp_buff *xdp)
xdp_ring = rx_ring->vsi->xdp_rings[rx_ring->q_index];
result = ice_xmit_xdp_buff(xdp, xdp_ring);
break;
- case XDP_REDIRECT:
- err = xdp_do_redirect(rx_ring->netdev, xdp, xdp_prog);
- result = !err ? ICE_XDP_REDIR : ICE_XDP_CONSUMED;
- break;
default:
bpf_warn_invalid_xdp_action(act);
fallthrough;
--
2.26.2
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH net-next 0/3][pull request] 40GbE Intel Wired LAN Driver Updates 2021-03-16
2021-03-16 16:42 [PATCH net-next 0/3][pull request] 40GbE Intel Wired LAN Driver Updates 2021-03-16 Tony Nguyen
` (2 preceding siblings ...)
2021-03-16 16:42 ` [PATCH net-next 3/3] ice: " Tony Nguyen
@ 2021-03-16 21:43 ` Jakub Kicinski
2021-03-16 22:40 ` patchwork-bot+netdevbpf
4 siblings, 0 replies; 7+ messages in thread
From: Jakub Kicinski @ 2021-03-16 21:43 UTC (permalink / raw)
To: Tony Nguyen
Cc: davem, netdev, sassmann, bjorn.topel, maciej.fijalkowski,
magnus.karlsson
On Tue, 16 Mar 2021 09:42:51 -0700 Tony Nguyen wrote:
> Optimize run_xdp_zc() for the XDP program verdict being XDP_REDIRECT
> in the xsk zero-copy path. This path is only used when having AF_XDP
> zero-copy on and in that case most packets will be directed to user
> space. This provides around 100k extra packets in throughput on my
> server when running l2fwd in xdpsock.
Looks reasonable:
Reviewed-by: Jakub Kicinski <kuba@kernel.org>
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH net-next 0/3][pull request] 40GbE Intel Wired LAN Driver Updates 2021-03-16
2021-03-16 16:42 [PATCH net-next 0/3][pull request] 40GbE Intel Wired LAN Driver Updates 2021-03-16 Tony Nguyen
` (3 preceding siblings ...)
2021-03-16 21:43 ` [PATCH net-next 0/3][pull request] 40GbE Intel Wired LAN Driver Updates 2021-03-16 Jakub Kicinski
@ 2021-03-16 22:40 ` patchwork-bot+netdevbpf
4 siblings, 0 replies; 7+ messages in thread
From: patchwork-bot+netdevbpf @ 2021-03-16 22:40 UTC (permalink / raw)
To: Tony Nguyen
Cc: davem, kuba, netdev, sassmann, bjorn.topel, maciej.fijalkowski,
magnus.karlsson
Hello:
This series was applied to netdev/net-next.git (refs/heads/master):
On Tue, 16 Mar 2021 09:42:51 -0700 you wrote:
> This series contains updates to i40e, ixgbe, and ice drivers.
>
> Magnus Karlsson says:
>
> Optimize run_xdp_zc() for the XDP program verdict being XDP_REDIRECT
> in the xsk zero-copy path. This path is only used when having AF_XDP
> zero-copy on and in that case most packets will be directed to user
> space. This provides around 100k extra packets in throughput on my
> server when running l2fwd in xdpsock.
>
> [...]
Here is the summary with links:
- [net-next,1/3] i40e: optimize for XDP_REDIRECT in xsk path
https://git.kernel.org/netdev/net-next/c/346497c78d15
- [net-next,2/3] ixgbe: optimize for XDP_REDIRECT in xsk path
https://git.kernel.org/netdev/net-next/c/7d52fe2eaddf
- [net-next,3/3] ice: optimize for XDP_REDIRECT in xsk path
https://git.kernel.org/netdev/net-next/c/bb52073645a6
You are awesome, thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html
^ permalink raw reply [flat|nested] 7+ messages in thread