Netdev List
 help / color / mirror / Atom feed
* [PATCH net v3] wireguard: drain per-peer queues with sentinels
@ 2026-11-17 19:41 Chris J Arges
  2026-11-17 19:47 ` netdev-bot+sinfo
  0 siblings, 1 reply; 2+ messages in thread
From: Chris J Arges @ 2026-11-17 19:41 UTC (permalink / raw)
  To: Jason A. Donenfeld, Andrew Lunn, David S. Miller, Jakub Kicinski,
	Paolo Abeni, Eric Dumazet
  Cc: wireguard, netdev, linux-kernel, kernel-team, Chris J Arges

peer_remove_after_dead() flushes device-wide packet crypto and handshake
workqueues while holding RTNL. Unrelated peers can keep those workqueues
busy, delaying removal and other users of RTNL.

After closing packet admission, append an internal sentinel to each of
the peer's TX and RX queues and wait for their completions. The serial
consumers reach each sentinel only after handling all preceding packets.
Reserve an extra queue slot for each sentinel so full queues can drain.

A crypto helper can publish a packet's final state before queuing TX work
or scheduling NAPI. An already-running consumer can process the packet
and its following sentinel during that interval. Guard publication and
notification in RCU read-side sections, then wait for a grace period after
both completions. This finishes those notification calls before flushing
the peer's TX work and disabling NAPI.

Keep the sentinels on the remover's stack until the consumers finish, and
recognize them before normal packet, keypair and reference handling.
Flush only this peer's handshake work; timer teardown already flushes its
key-clearing work.

Signed-off-by: Chris J Arges <carges@cloudflare.com>
---
Changes in v3:
- Use sentinel approach and not use atomic counts for every RX/TX packet.
- Link to v2: https://patch.msgid.link/20260913-fix-wg-peer-removal-v2-1-0cade985245a@cloudflare.com

Changes in v2:
- wait for RX packets to drain from peer’s NAPI queue
- Link to v1: https://patch.msgid.link/20260902-fix-wg-peer-removal-v1-1-7a0190f5cdb1@cloudflare.com
---
 drivers/net/wireguard/peer.c     | 42 ++++++++++++++++++++++++----------------
 drivers/net/wireguard/queueing.c | 10 ++++++++++
 drivers/net/wireguard/queueing.h | 22 ++++++++++++++++-----
 drivers/net/wireguard/receive.c  |  4 ++++
 drivers/net/wireguard/send.c     |  4 ++++
 5 files changed, 60 insertions(+), 22 deletions(-)

diff --git a/drivers/net/wireguard/peer.c b/drivers/net/wireguard/peer.c
index 1cb502a932e0..25d87509df93 100644
--- a/drivers/net/wireguard/peer.c
+++ b/drivers/net/wireguard/peer.c
@@ -93,6 +93,10 @@ static void peer_make_dead(struct wg_peer *peer)
 
 static void peer_remove_after_dead(struct wg_peer *peer)
 {
+	struct sk_buff tx = { }, rx = { };
+	DECLARE_COMPLETION_ONSTACK(tx_done);
+	DECLARE_COMPLETION_ONSTACK(rx_done);
+
 	WARN_ON(!peer->is_dead);
 
 	/* No more keypairs can be created for this peer, since is_dead protects
@@ -105,28 +109,32 @@ static void peer_remove_after_dead(struct wg_peer *peer)
 	 */
 	wg_timers_stop(peer);
 
-	/* The transition between packet encryption/decryption queues isn't
-	 * guarded by is_dead, but each reference's life is strictly bounded by
-	 * two generations: once for parallel crypto and once for serial
-	 * ingestion, so we can simply flush twice, and be sure that we no
-	 * longer have references inside these queues.
+	/* While is_dead guards against new TX/RX, there may be existing items
+	 * in encryption/decryption queues. To address that, queue a sentinel
+	 * on those queues and wait for completion for both tx/rx queues.
+	 */
+	wg_prev_queue_enqueue_sentinel(&peer->tx_queue, &tx, &tx_done);
+	wg_prev_queue_enqueue_sentinel(&peer->rx_queue, &rx, &rx_done);
+	queue_work_on(wg_cpumask_choose_online(&peer->serial_work_cpu, peer->internal_id),
+		      peer->device->packet_crypt_wq, &peer->transmit_packet_work);
+	napi_schedule(&peer->napi);
+	wait_for_completion(&tx_done);
+	wait_for_completion(&rx_done);
+
+	/* Handle edge cases where a consumer can finish a packet before crypto
+	 * helper schedules the consumer again. Use RCU sync to ensure that all
+	 * notification helpers finish.
 	 */
+	synchronize_rcu();
 
-	/* a) For encrypt/decrypt. */
-	flush_workqueue(peer->device->packet_crypt_wq);
-	/* b.1) For send (but not receive, since that's napi). */
-	flush_workqueue(peer->device->packet_crypt_wq);
-	/* b.2.1) For receive (but not send, since that's wq). */
+	/* Keep the sentinels alive until both consumers have finished. */
+	flush_work(&peer->transmit_packet_work);
 	napi_disable(&peer->napi);
-	/* b.2.1) It's now safe to remove the napi struct, which must be done
-	 * here from process context.
-	 */
+	/* netif_napi_del() must run in process context. */
 	netif_napi_del(&peer->napi);
 
-	/* Ensure any workstructs we own (like transmit_handshake_work or
-	 * clear_peer_work) no longer are in use.
-	 */
-	flush_workqueue(peer->device->handshake_send_wq);
+	/* wg_timers_stop() already flushed clear_peer_work. */
+	flush_work(&peer->transmit_handshake_work);
 
 	/* After the above flushes, a peer might still be active in a few
 	 * different contexts: 1) from xmit(), before hitting is_dead and
diff --git a/drivers/net/wireguard/queueing.c b/drivers/net/wireguard/queueing.c
index 26d235d15235..db3b94e57db7 100644
--- a/drivers/net/wireguard/queueing.c
+++ b/drivers/net/wireguard/queueing.c
@@ -77,6 +77,16 @@ bool wg_prev_queue_enqueue(struct prev_queue *queue, struct sk_buff *skb)
 	return true;
 }
 
+void wg_prev_queue_enqueue_sentinel(struct prev_queue *queue, struct sk_buff *skb,
+				    struct completion *completion)
+{
+	PACKET_CB(skb)->completion = completion;
+	atomic_set_release(&PACKET_CB(skb)->state, PACKET_STATE_DRAIN);
+	/* Reserve an extra slot even if the queue is full. */
+	atomic_inc(&queue->count);
+	__wg_prev_queue_enqueue(queue, skb);
+}
+
 struct sk_buff *wg_prev_queue_dequeue(struct prev_queue *queue)
 {
 	struct sk_buff *tail = queue->tail, *next = smp_load_acquire(&NEXT(tail));
diff --git a/drivers/net/wireguard/queueing.h b/drivers/net/wireguard/queueing.h
index 663b19b78395..e8d912f36ed8 100644
--- a/drivers/net/wireguard/queueing.h
+++ b/drivers/net/wireguard/queueing.h
@@ -7,6 +7,7 @@
 #define _WG_QUEUEING_H
 
 #include "peer.h"
+#include <linux/completion.h>
 #include <linux/types.h>
 #include <linux/skbuff.h>
 #include <linux/ip.h>
@@ -53,12 +54,16 @@ void wg_packet_encrypt_worker(struct work_struct *work);
 enum packet_state {
 	PACKET_STATE_UNCRYPTED,
 	PACKET_STATE_CRYPTED,
-	PACKET_STATE_DEAD
+	PACKET_STATE_DEAD,
+	PACKET_STATE_DRAIN
 };
 
 struct packet_cb {
 	u64 nonce;
-	struct noise_keypair *keypair;
+	union {
+		struct noise_keypair *keypair;
+		struct completion *completion;
+	};
 	atomic_t state;
 	u32 mtu;
 	u8 ds;
@@ -133,6 +138,10 @@ void wg_prev_queue_init(struct prev_queue *queue);
 /* Multi producer */
 bool wg_prev_queue_enqueue(struct prev_queue *queue, struct sk_buff *skb);
 
+/* Called only after packet admission has stopped. */
+void wg_prev_queue_enqueue_sentinel(struct prev_queue *queue, struct sk_buff *skb,
+				    struct completion *completion);
+
 /* Single consumer */
 struct sk_buff *wg_prev_queue_dequeue(struct prev_queue *queue);
 
@@ -176,15 +185,16 @@ static inline int wg_queue_enqueue_per_device_and_peer(
 
 static inline void wg_queue_enqueue_per_peer_tx(struct sk_buff *skb, enum packet_state state)
 {
-	/* We take a reference, because as soon as we call atomic_set, the
-	 * peer can be freed from below us.
-	 */
+	/* Keep the peer alive across state publication and TX scheduling. */
 	struct wg_peer *peer = wg_peer_get(PACKET_PEER(skb));
 
+	/* Teardown waits for this section after the drain sentinel completes. */
+	rcu_read_lock();
 	atomic_set_release(&PACKET_CB(skb)->state, state);
 	queue_work_on(wg_cpumask_choose_online(&peer->serial_work_cpu, peer->internal_id),
 		      peer->device->packet_crypt_wq, &peer->transmit_packet_work);
 	wg_peer_put(peer);
+	rcu_read_unlock();
 }
 
 static inline void wg_queue_enqueue_per_peer_rx(struct sk_buff *skb, enum packet_state state)
@@ -194,9 +204,11 @@ static inline void wg_queue_enqueue_per_peer_rx(struct sk_buff *skb, enum packet
 	 */
 	struct wg_peer *peer = wg_peer_get(PACKET_PEER(skb));
 
+	rcu_read_lock();
 	atomic_set_release(&PACKET_CB(skb)->state, state);
 	napi_schedule(&peer->napi);
 	wg_peer_put(peer);
+	rcu_read_unlock();
 }
 
 #ifdef DEBUG
diff --git a/drivers/net/wireguard/receive.c b/drivers/net/wireguard/receive.c
index 824bbefce61c..beca1e6308dd 100644
--- a/drivers/net/wireguard/receive.c
+++ b/drivers/net/wireguard/receive.c
@@ -452,6 +452,10 @@ int wg_packet_rx_poll(struct napi_struct *napi, int budget)
 	       (state = atomic_read_acquire(&PACKET_CB(skb)->state)) !=
 		       PACKET_STATE_UNCRYPTED) {
 		wg_prev_queue_drop_peeked(&peer->rx_queue);
+		if (unlikely(state == PACKET_STATE_DRAIN)) {
+			complete(PACKET_CB(skb)->completion);
+			break;
+		}
 		keypair = PACKET_CB(skb)->keypair;
 		free = true;
 
diff --git a/drivers/net/wireguard/send.c b/drivers/net/wireguard/send.c
index 67d01478eb76..e72264b64e72 100644
--- a/drivers/net/wireguard/send.c
+++ b/drivers/net/wireguard/send.c
@@ -270,6 +270,10 @@ void wg_packet_tx_worker(struct work_struct *work)
 	       (state = atomic_read_acquire(&PACKET_CB(first)->state)) !=
 		       PACKET_STATE_UNCRYPTED) {
 		wg_prev_queue_drop_peeked(&peer->tx_queue);
+		if (unlikely(state == PACKET_STATE_DRAIN)) {
+			complete(PACKET_CB(first)->completion);
+			break;
+		}
 		keypair = PACKET_CB(first)->keypair;
 
 		if (likely(state == PACKET_STATE_CRYPTED))

---
base-commit: 67869597fab06abaa5b977cb7004532269fc960b
change-id: 20260901-fix-wg-peer-removal-43fc390d0311

Best regards,
--  
Chris J Arges <carges@cloudflare.com>


^ permalink raw reply related	[flat|nested] 2+ messages in thread

* Re: [PATCH net v3] wireguard: drain per-peer queues with sentinels
  2026-11-17 19:41 [PATCH net v3] wireguard: drain per-peer queues with sentinels Chris J Arges
@ 2026-11-17 19:47 ` netdev-bot+sinfo
  0 siblings, 0 replies; 2+ messages in thread
From: netdev-bot+sinfo @ 2026-11-17 19:47 UTC (permalink / raw)
  To: Chris J Arges
  Cc: Jason A. Donenfeld, Andrew Lunn, David S. Miller, Jakub Kicinski,
	Paolo Abeni, Eric Dumazet, wireguard, netdev, linux-kernel,
	kernel-team

Hi!

This is an automated message. This series looks like a fix, but its
commit messages seem to be missing some information:

 - How the issue was discovered, e.g. hit in production, hit during
   development, syzbot report, manual code inspection, LLM or static
   analysis tool scan.

 - Whether the issue was actually triggered, or is only theoretical
   (e.g. found by LLM and not proven at runtime). If it was triggered
   please include the symptoms, like the stack trace or kernel messages.

Please do not repost the series just to address the above. Instead,
reply to this email with the missing information, so that reviewers
can take it into account.

*IMPORTANT*: If you do repost (for other reasons) - please include
the relevant info in the cover letter or commit messages.
-- 
The evaluation is done by an LLM so it may be wrong, if you think
that is the case please reply and explain.

^ permalink raw reply	[flat|nested] 2+ messages in thread

end of thread, other threads:[~2026-11-17 19:47 UTC | newest]

Thread overview: 2+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-17 19:41 [PATCH net v3] wireguard: drain per-peer queues with sentinels Chris J Arges
2026-11-17 19:47 ` netdev-bot+sinfo

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox