From: "Toke Høiland-Jørgensen" <toke@redhat.com>
To: bpf@vger.kernel.org, netdev@vger.kernel.org
Cc: "Martin KaFai Lau" <kafai@fb.com>,
"Hangbin Liu" <liuhangbin@gmail.com>,
"Jesper Dangaard Brouer" <brouer@redhat.com>,
"Magnus Karlsson" <magnus.karlsson@gmail.com>,
"Paul E . McKenney" <paulmck@kernel.org>,
"Jakub Kicinski" <kuba@kernel.org>,
"Toke Høiland-Jørgensen" <toke@redhat.com>,
"Jesse Brandeburg" <jesse.brandeburg@intel.com>,
"Tony Nguyen" <anthony.l.nguyen@intel.com>,
intel-wired-lan@lists.osuosl.org
Subject: [PATCH bpf-next v5 11/19] net: intel: remove rcu_read_lock() around XDP program invocation
Date: Thu, 24 Jun 2021 18:06:01 +0200 [thread overview]
Message-ID: <20210624160609.292325-12-toke@redhat.com> (raw)
In-Reply-To: <20210624160609.292325-1-toke@redhat.com>
The Intel drivers all have rcu_read_lock()/rcu_read_unlock() pairs around
XDP program invocations. However, the actual lifetime of the objects
referred by the XDP program invocation is longer, all the way through to
the call to xdp_do_flush(), making the scope of the rcu_read_lock() too
small. This turns out to be harmless because it all happens in a single
NAPI poll cycle (and thus under local_bh_disable()), but it makes the
rcu_read_lock() misleading.
Rather than extend the scope of the rcu_read_lock(), just get rid of it
entirely. With the addition of RCU annotations to the XDP_REDIRECT map
types that take bh execution into account, lockdep even understands this to
be safe, so there's really no reason to keep it around.
Cc: Jesse Brandeburg <jesse.brandeburg@intel.com>
Cc: Tony Nguyen <anthony.l.nguyen@intel.com>
Cc: intel-wired-lan@lists.osuosl.org
Tested-by: Jesper Dangaard Brouer <brouer@redhat.com> # i40e
Signed-off-by: Toke Høiland-Jørgensen <toke@redhat.com>
---
drivers/net/ethernet/intel/i40e/i40e_txrx.c | 2 --
drivers/net/ethernet/intel/i40e/i40e_xsk.c | 3 ---
drivers/net/ethernet/intel/ice/ice_txrx.c | 6 +-----
drivers/net/ethernet/intel/ice/ice_xsk.c | 3 ---
drivers/net/ethernet/intel/igb/igb_main.c | 2 --
drivers/net/ethernet/intel/igc/igc_main.c | 7 ++-----
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 2 --
drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c | 3 ---
drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c | 2 --
9 files changed, 3 insertions(+), 27 deletions(-)
diff --git a/drivers/net/ethernet/intel/i40e/i40e_txrx.c b/drivers/net/ethernet/intel/i40e/i40e_txrx.c
index b883ab809df3..38eb8151ee9a 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_txrx.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_txrx.c
@@ -2298,7 +2298,6 @@ static int i40e_run_xdp(struct i40e_ring *rx_ring, struct xdp_buff *xdp)
struct bpf_prog *xdp_prog;
u32 act;
- rcu_read_lock();
xdp_prog = READ_ONCE(rx_ring->xdp_prog);
if (!xdp_prog)
@@ -2334,7 +2333,6 @@ static int i40e_run_xdp(struct i40e_ring *rx_ring, struct xdp_buff *xdp)
break;
}
xdp_out:
- rcu_read_unlock();
return result;
}
diff --git a/drivers/net/ethernet/intel/i40e/i40e_xsk.c b/drivers/net/ethernet/intel/i40e/i40e_xsk.c
index 68f177a86403..e7e778ca074c 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_xsk.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_xsk.c
@@ -153,7 +153,6 @@ static int i40e_run_xdp_zc(struct i40e_ring *rx_ring, struct xdp_buff *xdp)
struct bpf_prog *xdp_prog;
u32 act;
- rcu_read_lock();
/* NB! xdp_prog will always be !NULL, due to the fact that
* this path is enabled by setting an XDP program.
*/
@@ -164,7 +163,6 @@ static int i40e_run_xdp_zc(struct i40e_ring *rx_ring, struct xdp_buff *xdp)
err = xdp_do_redirect(rx_ring->netdev, xdp, xdp_prog);
if (err)
goto out_failure;
- rcu_read_unlock();
return I40E_XDP_REDIR;
}
@@ -188,7 +186,6 @@ static int i40e_run_xdp_zc(struct i40e_ring *rx_ring, struct xdp_buff *xdp)
result = I40E_XDP_CONSUMED;
break;
}
- rcu_read_unlock();
return result;
}
diff --git a/drivers/net/ethernet/intel/ice/ice_txrx.c b/drivers/net/ethernet/intel/ice/ice_txrx.c
index 917eba7fdd0c..dd791ca34fab 100644
--- a/drivers/net/ethernet/intel/ice/ice_txrx.c
+++ b/drivers/net/ethernet/intel/ice/ice_txrx.c
@@ -1135,15 +1135,11 @@ int ice_clean_rx_irq(struct ice_ring *rx_ring, int budget)
xdp.frame_sz = ice_rx_frame_truesize(rx_ring, size);
#endif
- rcu_read_lock();
xdp_prog = READ_ONCE(rx_ring->xdp_prog);
- if (!xdp_prog) {
- rcu_read_unlock();
+ if (!xdp_prog)
goto construct_skb;
- }
xdp_res = ice_run_xdp(rx_ring, &xdp, xdp_prog);
- rcu_read_unlock();
if (!xdp_res)
goto construct_skb;
if (xdp_res & (ICE_XDP_TX | ICE_XDP_REDIR)) {
diff --git a/drivers/net/ethernet/intel/ice/ice_xsk.c b/drivers/net/ethernet/intel/ice/ice_xsk.c
index 239b9bf10794..8a093368f631 100644
--- a/drivers/net/ethernet/intel/ice/ice_xsk.c
+++ b/drivers/net/ethernet/intel/ice/ice_xsk.c
@@ -466,7 +466,6 @@ ice_run_xdp_zc(struct ice_ring *rx_ring, struct xdp_buff *xdp)
struct ice_ring *xdp_ring;
u32 act;
- rcu_read_lock();
/* ZC patch is enabled only when XDP program is set,
* so here it can not be NULL
*/
@@ -478,7 +477,6 @@ ice_run_xdp_zc(struct ice_ring *rx_ring, struct xdp_buff *xdp)
err = xdp_do_redirect(rx_ring->netdev, xdp, xdp_prog);
if (err)
goto out_failure;
- rcu_read_unlock();
return ICE_XDP_REDIR;
}
@@ -503,7 +501,6 @@ ice_run_xdp_zc(struct ice_ring *rx_ring, struct xdp_buff *xdp)
break;
}
- rcu_read_unlock();
return result;
}
diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c
index 5db303d64d14..7e6435dc7e80 100644
--- a/drivers/net/ethernet/intel/igb/igb_main.c
+++ b/drivers/net/ethernet/intel/igb/igb_main.c
@@ -8381,7 +8381,6 @@ static struct sk_buff *igb_run_xdp(struct igb_adapter *adapter,
struct bpf_prog *xdp_prog;
u32 act;
- rcu_read_lock();
xdp_prog = READ_ONCE(rx_ring->xdp_prog);
if (!xdp_prog)
@@ -8416,7 +8415,6 @@ static struct sk_buff *igb_run_xdp(struct igb_adapter *adapter,
break;
}
xdp_out:
- rcu_read_unlock();
return ERR_PTR(-result);
}
diff --git a/drivers/net/ethernet/intel/igc/igc_main.c b/drivers/net/ethernet/intel/igc/igc_main.c
index 3f6b6d4543a8..95323095094d 100644
--- a/drivers/net/ethernet/intel/igc/igc_main.c
+++ b/drivers/net/ethernet/intel/igc/igc_main.c
@@ -2240,18 +2240,15 @@ static struct sk_buff *igc_xdp_run_prog(struct igc_adapter *adapter,
struct bpf_prog *prog;
int res;
- rcu_read_lock();
-
prog = READ_ONCE(adapter->xdp_prog);
if (!prog) {
res = IGC_XDP_PASS;
- goto unlock;
+ goto out;
}
res = __igc_xdp_run_prog(adapter, prog, xdp);
-unlock:
- rcu_read_unlock();
+out:
return ERR_PTR(-res);
}
diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
index 2ac5b82676f3..ffff69efd78a 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
@@ -2199,7 +2199,6 @@ static struct sk_buff *ixgbe_run_xdp(struct ixgbe_adapter *adapter,
struct xdp_frame *xdpf;
u32 act;
- rcu_read_lock();
xdp_prog = READ_ONCE(rx_ring->xdp_prog);
if (!xdp_prog)
@@ -2237,7 +2236,6 @@ static struct sk_buff *ixgbe_run_xdp(struct ixgbe_adapter *adapter,
break;
}
xdp_out:
- rcu_read_unlock();
return ERR_PTR(-result);
}
diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c
index f72d2978263b..96dd1a4f956a 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c
@@ -100,7 +100,6 @@ static int ixgbe_run_xdp_zc(struct ixgbe_adapter *adapter,
struct xdp_frame *xdpf;
u32 act;
- rcu_read_lock();
xdp_prog = READ_ONCE(rx_ring->xdp_prog);
act = bpf_prog_run_xdp(xdp_prog, xdp);
@@ -108,7 +107,6 @@ static int ixgbe_run_xdp_zc(struct ixgbe_adapter *adapter,
err = xdp_do_redirect(rx_ring->netdev, xdp, xdp_prog);
if (err)
goto out_failure;
- rcu_read_unlock();
return IXGBE_XDP_REDIR;
}
@@ -134,7 +132,6 @@ static int ixgbe_run_xdp_zc(struct ixgbe_adapter *adapter,
result = IXGBE_XDP_CONSUMED;
break;
}
- rcu_read_unlock();
return result;
}
diff --git a/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c b/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c
index dc56931fc1dc..c714e1ecd308 100644
--- a/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c
+++ b/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c
@@ -1054,7 +1054,6 @@ static struct sk_buff *ixgbevf_run_xdp(struct ixgbevf_adapter *adapter,
struct bpf_prog *xdp_prog;
u32 act;
- rcu_read_lock();
xdp_prog = READ_ONCE(rx_ring->xdp_prog);
if (!xdp_prog)
@@ -1082,7 +1081,6 @@ static struct sk_buff *ixgbevf_run_xdp(struct ixgbevf_adapter *adapter,
break;
}
xdp_out:
- rcu_read_unlock();
return ERR_PTR(-result);
}
--
2.32.0
next prev parent reply other threads:[~2021-06-24 16:06 UTC|newest]
Thread overview: 21+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-06-24 16:05 [PATCH bpf-next v5 00/19] Clean up and document RCU-based object protection for XDP and TC BPF Toke Høiland-Jørgensen
2021-06-24 16:05 ` [PATCH bpf-next v5 01/19] rcu: Create an unrcu_pointer() to remove __rcu from a pointer Toke Høiland-Jørgensen
2021-06-24 16:05 ` [PATCH bpf-next v5 02/19] doc: Clarify and expand RCU updaters and corresponding readers Toke Høiland-Jørgensen
2021-06-24 16:05 ` [PATCH bpf-next v5 03/19] doc: Give XDP as example of non-obvious RCU reader/updater pairing Toke Høiland-Jørgensen
2021-06-24 16:05 ` [PATCH bpf-next v5 04/19] bpf: allow RCU-protected lookups to happen from bh context Toke Høiland-Jørgensen
2021-06-24 16:05 ` [PATCH bpf-next v5 05/19] xdp: add proper __rcu annotations to redirect map entries Toke Høiland-Jørgensen
2021-06-24 16:05 ` [PATCH bpf-next v5 06/19] sched: remove unneeded rcu_read_lock() around BPF program invocation Toke Høiland-Jørgensen
2021-06-24 16:05 ` [PATCH bpf-next v5 07/19] ena: remove rcu_read_lock() around XDP " Toke Høiland-Jørgensen
2021-06-24 16:05 ` [PATCH bpf-next v5 08/19] bnxt: " Toke Høiland-Jørgensen
2021-06-24 16:05 ` [PATCH bpf-next v5 09/19] thunderx: " Toke Høiland-Jørgensen
2021-06-24 16:06 ` [PATCH bpf-next v5 10/19] freescale: " Toke Høiland-Jørgensen
2021-06-24 16:06 ` Toke Høiland-Jørgensen [this message]
2021-06-24 16:06 ` [PATCH bpf-next v5 12/19] marvell: " Toke Høiland-Jørgensen
2021-06-24 16:06 ` [PATCH bpf-next v5 13/19] mlx4: " Toke Høiland-Jørgensen
2021-06-24 16:06 ` [PATCH bpf-next v5 14/19] nfp: " Toke Høiland-Jørgensen
2021-06-24 16:06 ` [PATCH bpf-next v5 15/19] qede: " Toke Høiland-Jørgensen
2021-06-24 16:06 ` [PATCH bpf-next v5 16/19] sfc: " Toke Høiland-Jørgensen
2021-06-24 16:06 ` [PATCH bpf-next v5 17/19] netsec: " Toke Høiland-Jørgensen
2021-06-24 16:06 ` [PATCH bpf-next v5 18/19] stmmac: " Toke Høiland-Jørgensen
2021-06-24 16:06 ` [PATCH bpf-next v5 19/19] net: ti: " Toke Høiland-Jørgensen
2021-06-24 18:00 ` [PATCH bpf-next v5 00/19] Clean up and document RCU-based object protection for XDP and TC BPF patchwork-bot+netdevbpf
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210624160609.292325-12-toke@redhat.com \
--to=toke@redhat.com \
--cc=anthony.l.nguyen@intel.com \
--cc=bpf@vger.kernel.org \
--cc=brouer@redhat.com \
--cc=intel-wired-lan@lists.osuosl.org \
--cc=jesse.brandeburg@intel.com \
--cc=kafai@fb.com \
--cc=kuba@kernel.org \
--cc=liuhangbin@gmail.com \
--cc=magnus.karlsson@gmail.com \
--cc=netdev@vger.kernel.org \
--cc=paulmck@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).