[PATCH mptcp-next v3 0/4] mptcp: sched: penalise a slow subflow

[PATCH mptcp-next v3 0/4] mptcp: sched: penalise a slow subflow

[Shardul Bankar]

A subflow that is slow relative to the others can soak up connection
resources and head-of-line-block the aggregate stream. This series lets the
packet scheduler send less than such a subflow's congestion window by halving
it, and leaves the congestion control to grow it back. It implements
issue #345 [1]. This is a deliberately simple first form.

- Patch 1: a prerequisite fix. The scheduler truncates the pacing rate to a
  u32, which skews subflow selection on fast paths (Fixes).
- Patch 2: penalise a slow subflow by halving its cwnd, rate-triggered and
  gated on the fast path being cwnd-limited.
- Patch 3: skip the penalty when the transfer is receive-window-limited,
  where it would otherwise slow the transfer.
- Patch 4: MPTcpExt counters and a dedicated mptcp_subflow_penalise
  tracepoint for observability.

Validation, baseline versus this series (test helper, its run_matrix.sh
base-vs-patch driver, and run steps are in [2]):
- No regression in the mptcp selftests.
- No regression versus baseline across the buffer-limited transfers
  (send-buffer, receive-window, both, sndbuf>rcvbuf): the penalised two-path
  completion time matches the unpatched kernel. Where the slow subflow is
  beneficial (send-buffer-limited), the two-path transfer stays faster than the
  fast path alone.
- Patch 3's gate keeps the penalty from firing when receive-window-limited,
  where halving cwnd would slow the transfer.
- With a bufferbloated slow path, the penalty roughly halves the slow path's
  queueing latency (subflow srtt), with the transfer time unchanged. It does
  not reduce the aggregate out-of-order volume or move data off the slow path;
  that is the role of #332, and the two are complementary.
- One regime, unbounded buffers with a purely delay-slow path, shows a
  throughput cost that grows with the delay; the current gates do not catch it.

On patch 4: observability is a dedicated mptcp_subflow_penalise tracepoint
rather than a field on the existing mptcp_subflow_get_send. The penalise verdict
is only known after the selection loop, so a field there would need a second,
post-decision emit plus a marker to tell it apart from the per-candidate emits,
and would make existing get_send consumers count an extra event per call. A
dedicated tracepoint avoids the double emit and leaves mptcp_subflow_get_send
untouched. Happy to use a field instead if you prefer.

[1] https://github.com/multipath-tcp/mptcp_net-next/issues/345
[2] https://github.com/shardulsdk-mpiric/linux/blob/161a4bd556eb/tools/testing/selftests/net/mptcp/mptcp_sched_penalise.sh
    (run_matrix.sh + runs.conf are alongside it in the same directory)

Changes in v3:
- Dropped the RFC tag (Matthieu Baerts (NGI0)). No changes to the patches themselves.
- Rebased on current export.
- Link to v2: https://patch.msgid.link/20260815-mptcp_penalise_send_v2-v2-0-3e5049a73681@mpiricsoftware.com

Changes in v2
- Patch 1 (new, not in v1): fix the scheduler's pre-existing u32 pacing-rate
  truncation (Fixes: 3ce0852c86b9).
- Patch 2 (v1's 1/3): also skip a subflow already at the cwnd floor
  (MPTCP_PENALISE_MIN_CWND); clear the flag when returning without send-window
  room; re-check TCP_CA_Open in the apply path; express the rate trigger as a
  division (avg_pacing_rate < max_pace / ratio, using the widened pace from
  patch 1) instead of a u64-cast multiply.
- Patch 3 (v1's 2/3): make the receive-window test wrap-safe
  (after64, like tcp_snd_wnd_test()); same condition otherwise.
- Patch 4 (v1's 3/3): real MPTcpExt counters (dropped DO-NOT-MERGE) plus
  a dedicated mptcp_subflow_penalise tracepoint (not a field on
  mptcp_subflow_get_send).
- Rebased on current export.
- Link to v1:
  https://lore.kernel.org/mptcp/20260726-mptcp_penalise_send-v1-0-84485e0e995b@mpiricsoftware.com

Signed-off-by: Shardul Bankar <shardul.b@mpiricsoftware.com>
---
Shardul Bankar (4):
      mptcp: sched: avoid truncating the pacing rate in the scheduler
      mptcp: sched: penalise a slow subflow by halving its cwnd
      mptcp: sched: do not penalise when receive-window-limited
      mptcp: sched: add penalise counters and tracepoint

 include/trace/events/mptcp.h | 30 ++++++++++++++++
 net/mptcp/mib.c              |  2 ++
 net/mptcp/mib.h              |  2 ++
 net/mptcp/protocol.c         | 85 ++++++++++++++++++++++++++++++++++++++++----
 net/mptcp/protocol.h         |  2 ++
 5 files changed, 115 insertions(+), 6 deletions(-)
---
base-commit: 19c6bf13fd58c0d98668764de2f3d68d975685c9
change-id: 20260815-mptcp_penalise_send_v2-ed6206f70a84

Best regards,
--  
Shardul Bankar <shardul.b@mpiricsoftware.com>

[PATCH mptcp-next v3 1/4] mptcp: sched: avoid truncating the pacing rate in the scheduler

[Shardul Bankar]

mptcp_subflow_get_send() derives each subflow's linger time and a running
pacing-rate average through a u32 "pace", while sk_pacing_rate and
avg_pacing_rate are unsigned long.

On 64-bit systems the u32 truncates pacing rates above ~4.29 GB/s, skewing
the linger-time based subflow selection on very fast paths. On 32-bit
systems, where unsigned long is also 32-bit, the sk_pacing_rate * burst
product overflows: with burst up to ~65428 it wraps once the pacing rate
exceeds ~526 kbit/s, misestimating the running average at ordinary rates.

Widen "pace" to unsigned long, divide the linger time with a 64-bit
divisor, and cast the pacing-rate average product to u64.

Fixes: 3ce0852c86b9 ("mptcp: enforce HoL-blocking estimation")
Suggested-by: Matthieu Baerts (NGI0) <matttbe@kernel.org>
Signed-off-by: Shardul Bankar <shardul.b@mpiricsoftware.com>
---
 net/mptcp/protocol.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/net/mptcp/protocol.c b/net/mptcp/protocol.c
index c2762d74f29d..46a49ec75e78 100644
--- a/net/mptcp/protocol.c
+++ b/net/mptcp/protocol.c
@@ -1626,7 +1626,8 @@ struct sock *mptcp_subflow_get_send(struct mptcp_sock *msk)
 	struct subflow_send_info send_info[SSK_MODE_MAX];
 	struct mptcp_subflow_context *subflow;
 	struct sock *sk = (struct sock *)msk;
-	u32 pace, burst, wmem;
+	unsigned long pace;
+	u32 burst, wmem;
 	int i, nr_active = 0;
 	struct sock *ssk;
 	u64 linger_time;
@@ -1657,7 +1658,7 @@ struct sock *mptcp_subflow_get_send(struct mptcp_sock *msk)
 				continue;
 		}
 
-		linger_time = div_u64((u64)READ_ONCE(ssk->sk_wmem_queued) << 32, pace);
+		linger_time = div64_u64((u64)READ_ONCE(ssk->sk_wmem_queued) << 32, pace);
 		if (linger_time < send_info[backup].linger_time) {
 			send_info[backup].ssk = ssk;
 			send_info[backup].linger_time = linger_time;
@@ -1691,7 +1692,7 @@ struct sock *mptcp_subflow_get_send(struct mptcp_sock *msk)
 
 	subflow = mptcp_subflow_ctx(ssk);
 	subflow->avg_pacing_rate = div_u64((u64)subflow->avg_pacing_rate * wmem +
-					   READ_ONCE(ssk->sk_pacing_rate) * burst,
+					   (u64)READ_ONCE(ssk->sk_pacing_rate) * burst,
 					   burst + wmem);
 	msk->snd_burst = burst;
 	return ssk;

-- 
2.34.1

Re: [PATCH mptcp-next v3 1/4] mptcp: sched: avoid truncating the pacing rate in the scheduler

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [High] The weighted average pacing rate computation silently overflows `u64` on high-speed networks with large send buffers, artificially clamping the calculated rate.
--

--- Patch [1]: [PATCH mptcp-next v3 1/4] mptcp: sched: avoid truncating the pacing rate in the scheduler ---
commit b2e45abf072b6d7db21560b1c0ae59fd43742be5
Author: Shardul Bankar <shardul.b@mpiricsoftware.com>

mptcp: sched: avoid truncating the pacing rate in the scheduler

This commit addresses truncation and overflow issues in the MPTCP
scheduler's pacing rate computations. It widens the pace variable to
unsigned long and casts the pacing-rate average product to u64 to
prevent 32-bit integer overflow. It also uses div64_u64 for the linger
time calculation.

> diff --git a/net/mptcp/protocol.c b/net/mptcp/protocol.c
> index c2762d74f29d7..46a49ec75e784 100644
> --- a/net/mptcp/protocol.c
> +++ b/net/mptcp/protocol.c

[ ... ]

> @@ -1691,7 +1692,7 @@ struct sock *mptcp_subflow_get_send(struct mptcp_sock *msk)
>  
>  	subflow = mptcp_subflow_ctx(ssk);
>  	subflow->avg_pacing_rate = div_u64((u64)subflow->avg_pacing_rate * wmem +
> -					   READ_ONCE(ssk->sk_pacing_rate) * burst,
> +					   (u64)READ_ONCE(ssk->sk_pacing_rate) * burst,
>  					   burst + wmem);

[Severity: High]
This is a pre-existing issue, but is it possible for the weighted average
pacing rate computation to still silently overflow a u64 on high-speed
networks with large send buffers?

If mptcp_subflow_get_send() computes the moving average of the pacing rate
for a connection with a large send buffer (e.g., 2 GB) combined with a high
or unlimited pacing rate, the numerator ((u64)subflow->avg_pacing_rate *
wmem + (u64)READ_ONCE(ssk->sk_pacing_rate) * burst) can exceed 64 bits
before division.

This could artificially clamp the calculated rate, drastically reducing it.
For example, an unlimited pacing rate (~0UL) with a 2 GB send buffer might
be incorrectly throttled to ~73 Gbps, and a 100 Gbps paced connection with
a 2 GB send buffer could be throttled to ~26 Gbps.

>  	msk->snd_burst = burst;
>  	return ssk;

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260817-mptcp_penalise_send_v2-v3-0-e6a2ad2f1b82@mpiricsoftware.com?part=1

[PATCH mptcp-next v3 2/4] mptcp: sched: penalise a slow subflow by halving its cwnd

[Shardul Bankar]

A poorly-performing but usable subflow (high latency, loss, bufferbloat)
can soak up connection resources and cause head-of-line blocking of the
aggregate stream. Give the default packet scheduler a way to send less
than such a subflow's full congestion window.

Once a subflow has been picked for transmission, flag it for penalisation
when:
- its smoothed delivery rate (avg_pacing_rate) is below half that of the
  fastest path, keying on rate, not RTT, so a slow-but-high-throughput
  path is left alone;
- the fastest path is cwnd-limited (saturated), so shifting load off the
  slow path is worthwhile;
- the subflow is in TCP_CA_Open, so its cwnd is not already being reduced
  by loss recovery;
- its cwnd is still above MPTCP_PENALISE_MIN_CWND, so a subflow already at
  the floor is left to recover instead of being churned by a halving that
  can no longer reduce it;
- it has not been penalised in the last RTT.

The reduction halves tcp_snd_cwnd, floored at MPTCP_PENALISE_MIN_CWND, and
ssthresh if cwnd is past it. It is applied in the push path under the
subflow socket lock, which protects snd_cwnd (the scheduler runs under the
msk lock). The congestion control grows the window back, ACK-clocked; that
regrowth is the built-in probe, so no explicit MPTCP-side probing is
needed.

Closes: https://github.com/multipath-tcp/mptcp_net-next/issues/345
Co-developed-by: Matthieu Baerts (NGI0) <matttbe@kernel.org>
Signed-off-by: Shardul Bankar <shardul.b@mpiricsoftware.com>
---
 net/mptcp/protocol.c | 65 ++++++++++++++++++++++++++++++++++++++++++++++++----
 net/mptcp/protocol.h |  2 ++
 2 files changed, 63 insertions(+), 4 deletions(-)

diff --git a/net/mptcp/protocol.c b/net/mptcp/protocol.c
index 46a49ec75e78..63cc6cb33044 100644
--- a/net/mptcp/protocol.c
+++ b/net/mptcp/protocol.c
@@ -1617,6 +1617,41 @@ bool mptcp_subflow_active(struct mptcp_subflow_context *subflow)
 #define SSK_MODE_BACKUP	1
 #define SSK_MODE_MAX	2
 
+/* Penalise a subflow pacing below the fastest path's rate / this ratio */
+#define MPTCP_PENALISE_RATE_RATIO	2
+/* cwnd floor for the penalty: skip a subflow already there, and never reduce
+ * below it (halving at the floor only churns congestion control).
+ */
+#define MPTCP_PENALISE_MIN_CWND		2
+
+/* Rate-limit the penalty to at most once per subflow RTT, so the congestion
+ * control can grow the window back between reductions.
+ */
+static bool mptcp_penalise_throttle_ok(struct mptcp_subflow_context *subflow)
+{
+	struct sock *ssk = mptcp_subflow_tcp_sock(subflow);
+	u32 rtt = usecs_to_jiffies(READ_ONCE(tcp_sk(ssk)->srtt_us) >> 3);
+
+	return tcp_jiffies32 - subflow->last_penalise >= max_t(u32, rtt, 1);
+}
+
+/* Halve cwnd (and ssthresh if past it) under the subflow socket lock. */
+static void mptcp_penalise_cwnd(struct sock *ssk)
+{
+	struct mptcp_subflow_context *subflow = mptcp_subflow_ctx(ssk);
+	struct tcp_sock *tp = tcp_sk(ssk);
+	u32 cwnd = tcp_snd_cwnd(tp);
+
+	subflow->penalise = false;
+	/* CA state may have left TCP_CA_Open since get_send flagged this */
+	if (inet_csk(ssk)->icsk_ca_state != TCP_CA_Open)
+		return;
+	subflow->last_penalise = tcp_jiffies32;
+	tcp_snd_cwnd_set(tp, max_t(u32, cwnd >> 1, MPTCP_PENALISE_MIN_CWND));
+	if (cwnd >= tp->snd_ssthresh)
+		tp->snd_ssthresh = max_t(u32, tp->snd_ssthresh >> 1, 2);
+}
+
 /* implement the mptcp packet scheduler;
  * returns the subflow that will transmit the next DSS
  * additionally updates the rtx timeout
@@ -1626,10 +1661,11 @@ struct sock *mptcp_subflow_get_send(struct mptcp_sock *msk)
 	struct subflow_send_info send_info[SSK_MODE_MAX];
 	struct mptcp_subflow_context *subflow;
 	struct sock *sk = (struct sock *)msk;
-	unsigned long pace;
+	unsigned long pace, max_pace = 0;
 	u32 burst, wmem;
+	bool penal_cand;
 	int i, nr_active = 0;
-	struct sock *ssk;
+	struct sock *ssk, *fastest = NULL;
 	u64 linger_time;
 	long tout = 0;
 
@@ -1658,6 +1694,12 @@ struct sock *mptcp_subflow_get_send(struct mptcp_sock *msk)
 				continue;
 		}
 
+		/* track the fastest path; slower ones get throttled below */
+		if (pace > max_pace) {
+			max_pace = pace;
+			fastest = ssk;
+		}
+
 		linger_time = div64_u64((u64)READ_ONCE(ssk->sk_wmem_queued) << 32, pace);
 		if (linger_time < send_info[backup].linger_time) {
 			send_info[backup].ssk = ssk;
@@ -1685,12 +1727,24 @@ struct sock *mptcp_subflow_get_send(struct mptcp_sock *msk)
 	if (!ssk || !sk_stream_memory_free(ssk))
 		return NULL;
 
+	/* Flag a slow subflow for cwnd halving, applied in the push path */
+	subflow = mptcp_subflow_ctx(ssk);
+	penal_cand = fastest && ssk != fastest &&
+		     subflow->avg_pacing_rate < max_pace / MPTCP_PENALISE_RATE_RATIO;
+	subflow->penalise = penal_cand &&
+			    tcp_snd_cwnd(tcp_sk(ssk)) > MPTCP_PENALISE_MIN_CWND &&
+			    inet_csk(ssk)->icsk_ca_state == TCP_CA_Open &&
+			    tcp_is_cwnd_limited(fastest) &&
+			    mptcp_penalise_throttle_ok(subflow);
+
 	burst = min(MPTCP_SEND_BURST_SIZE, mptcp_wnd_end(msk) - msk->snd_nxt);
 	wmem = READ_ONCE(ssk->sk_wmem_queued);
-	if (!burst)
+	if (!burst) {
+		/* no window room to send; skip the penalty */
+		subflow->penalise = false;
 		return ssk;
+	}
 
-	subflow = mptcp_subflow_ctx(ssk);
 	subflow->avg_pacing_rate = div_u64((u64)subflow->avg_pacing_rate * wmem +
 					   (u64)READ_ONCE(ssk->sk_pacing_rate) * burst,
 					   burst + wmem);
@@ -1747,6 +1801,9 @@ static int __subflow_push_pending(struct sock *sk, struct sock *ssk,
 	struct mptcp_data_frag *dfrag;
 	int len, copied = 0, err = 0;
 
+	if (mptcp_subflow_ctx(ssk)->penalise)
+		mptcp_penalise_cwnd(ssk);
+
 	while ((dfrag = mptcp_send_head(sk))) {
 		info->sent = dfrag->already_sent;
 		info->limit = dfrag->data_len;
diff --git a/net/mptcp/protocol.h b/net/mptcp/protocol.h
index 7e168e450fb0..ab9ba6699ccd 100644
--- a/net/mptcp/protocol.h
+++ b/net/mptcp/protocol.h
@@ -588,6 +588,7 @@ struct mptcp_subflow_context {
 		__unused : 9;
 	bool	data_avail;
 	bool	scheduled;
+	bool	penalise;	    /* scheduler flagged this subflow for cwnd halving */
 	bool	pm_listener;	    /* a listener managed by the kernel PM? */
 	bool	fully_established;  /* path validated */
 	u32	lent_mem_frag;
@@ -611,6 +612,7 @@ struct mptcp_subflow_context {
 				     */
 
 	u32	subflow_id;
+	u32	last_penalise;	    /* tcp_jiffies32 of the last cwnd penalty */
 
 	long	delegated_status;
 	unsigned long	fail_tout;

-- 
2.34.1

Re: [PATCH mptcp-next v3 2/4] mptcp: sched: penalise a slow subflow by halving its cwnd

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
- [High] A 64-bit integer overflow in the EWMA `avg_pacing_rate` calculation incorrectly cripples highly performant subflows.
- [Medium] Bypassing TCP Congestion Control API corrupts internal state and renders the penalty ineffective.
- [Medium] Unsynchronized lockless reads of TCP subflow internal states and bitfields cause data races.
--

--- Patch [2]: [PATCH mptcp-next v3 2/4] mptcp: sched: penalise a slow subflow by halving its cwnd ---
commit 07a5e01113da906e5797c2902f8c9a2e4065ed23
Author: Shardul Bankar <shardul.b@mpiricsoftware.com>
Subject: mptcp: sched: penalise a slow subflow by halving its cwnd

This commit introduces a mechanism to penalize poorly performing subflows
by halving their congestion window. It flags subflows for penalization in
the scheduler when their pacing rate falls significantly behind the fastest
path, and applies the reduction under the subflow socket lock.

Closes: https://github.com/multipath-tcp/mptcp_net-next/issues/345

> --- a/net/mptcp/protocol.c
> +++ b/net/mptcp/protocol.c
[ ... ]
> +static void mptcp_penalise_cwnd(struct sock *ssk)
> +{
> +	struct mptcp_subflow_context *subflow = mptcp_subflow_ctx(ssk);
> +	struct tcp_sock *tp = tcp_sk(ssk);
> +	u32 cwnd = tcp_snd_cwnd(tp);
> +
> +	subflow->penalise = false;
> +	/* CA state may have left TCP_CA_Open since get_send flagged this */
> +	if (inet_csk(ssk)->icsk_ca_state != TCP_CA_Open)
> +		return;
> +	subflow->last_penalise = tcp_jiffies32;
> +	tcp_snd_cwnd_set(tp, max_t(u32, cwnd >> 1, MPTCP_PENALISE_MIN_CWND));
> +	if (cwnd >= tp->snd_ssthresh)
> +		tp->snd_ssthresh = max_t(u32, tp->snd_ssthresh >> 1, 2);

[Severity: Medium]
Will bypassing the TCP congestion control API work for modern algorithms?

Algorithms like BBR or CUBIC explicitly ignore external modifications to
snd_cwnd and might immediately overwrite or inflate the window back up on the
next ACK. 

BBR tracks its own window internally, so the penalty here might be instantly
undone and rendered ineffective.

> +}
[ ... ]
> @@ -1685,12 +1727,24 @@ struct sock *mptcp_subflow_get_send(struct mptcp_sock *msk)
>  	if (!ssk || !sk_stream_memory_free(ssk))
>  		return NULL;
>  
> +	/* Flag a slow subflow for cwnd halving, applied in the push path */
> +	subflow = mptcp_subflow_ctx(ssk);
> +	penal_cand = fastest && ssk != fastest &&
> +		     subflow->avg_pacing_rate < max_pace / MPTCP_PENALISE_RATE_RATIO;
> +	subflow->penalise = penal_cand &&
> +			    tcp_snd_cwnd(tcp_sk(ssk)) > MPTCP_PENALISE_MIN_CWND &&
> +			    inet_csk(ssk)->icsk_ca_state == TCP_CA_Open &&
> +			    tcp_is_cwnd_limited(fastest) &&

[Severity: Medium]
Are we safe from data races when reading tcp_is_cwnd_limited(fastest)
locklessly?

The scheduler is running under the MPTCP connection lock, but does not hold
the individual subflow socket locks. tcp_is_cwnd_limited is a bitfield
sharing a byte with other heavily modified TCP flags.

Concurrent updates by the TCP stack to adjacent flags in the bitfield during
ACK processing require Read-Modify-Write operations. This lockless read
might trigger KCSAN splats.

> +			    mptcp_penalise_throttle_ok(subflow);
> +
>  	burst = min(MPTCP_SEND_BURST_SIZE, mptcp_wnd_end(msk) - msk->snd_nxt);
>  	wmem = READ_ONCE(ssk->sk_wmem_queued);
> -	if (!burst)
> +	if (!burst) {
> +		/* no window room to send; skip the penalty */
> +		subflow->penalise = false;
>  		return ssk;
> +	}
>  
> -	subflow = mptcp_subflow_ctx(ssk);
>  	subflow->avg_pacing_rate = div_u64((u64)subflow->avg_pacing_rate * wmem +

[Severity: High]
Could this EWMA calculation overflow a 64-bit integer on high-throughput
links?

If a highly performant subflow, like 100Gbps (around 12.5 GB/s), has a large
write queue (wmem around 1.5 GB), the product of (u64)avg_pacing_rate * wmem
might exceed the u64 maximum of ~18.44 * 10^18.

This would wrap around modulo 2^64 and drastically reduce avg_pacing_rate.
The scheduler might then falsely flag this highly performant subflow as
a penalty candidate and repeatedly halve its congestion window.

>  					   (u64)READ_ONCE(ssk->sk_pacing_rate) * burst,
>  					   burst + wmem);

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260817-mptcp_penalise_send_v2-v3-0-e6a2ad2f1b82@mpiricsoftware.com?part=2

[PATCH mptcp-next v3 3/4] mptcp: sched: do not penalise when receive-window-limited

[Shardul Bankar]

The penalty in the previous patch shifts load off a slow subflow onto the
fastest one, which only helps if the fastest path can absorb it. When the
connection is receive-window-limited (the receiver's advertised window,
not our congestion window, is the bottleneck), the fastest path is capped
by that shared window too and cannot send more, so halving the slow path's
cwnd just sheds its throughput. In a receive-window-limited transfer this
was measured roughly 2x slower than baseline.

Gate on the application's queued data fitting within the send window:
penalise only while write_seq <= wnd_end. If the application has queued
past the window edge the receive window is the binding constraint, so skip
the penalty. Neither write_seq (application demand) nor wnd_end (peer
window) is derived from cwnd, so the test is not made circular by the
window itself suppressing cwnd.

Co-developed-by: Matthieu Baerts (NGI0) <matttbe@kernel.org>
Signed-off-by: Shardul Bankar <shardul.b@mpiricsoftware.com>
---
 net/mptcp/protocol.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/net/mptcp/protocol.c b/net/mptcp/protocol.c
index 63cc6cb33044..e4d26c2dce1e 100644
--- a/net/mptcp/protocol.c
+++ b/net/mptcp/protocol.c
@@ -1635,6 +1635,14 @@ static bool mptcp_penalise_throttle_ok(struct mptcp_subflow_context *subflow)
 	return tcp_jiffies32 - subflow->last_penalise >= max_t(u32, rtt, 1);
 }
 
+/* Like tcp_snd_wnd_test() but without an skb: true while queued data still fits
+ * the send window, i.e. not receive-window-limited.
+ */
+static bool mptcp_snd_wnd_test(const struct mptcp_sock *msk)
+{
+	return !after64(msk->write_seq, mptcp_wnd_end(msk));
+}
+
 /* Halve cwnd (and ssthresh if past it) under the subflow socket lock. */
 static void mptcp_penalise_cwnd(struct sock *ssk)
 {
@@ -1735,6 +1743,7 @@ struct sock *mptcp_subflow_get_send(struct mptcp_sock *msk)
 			    tcp_snd_cwnd(tcp_sk(ssk)) > MPTCP_PENALISE_MIN_CWND &&
 			    inet_csk(ssk)->icsk_ca_state == TCP_CA_Open &&
 			    tcp_is_cwnd_limited(fastest) &&
+			    mptcp_snd_wnd_test(msk) &&
 			    mptcp_penalise_throttle_ok(subflow);
 
 	burst = min(MPTCP_SEND_BURST_SIZE, mptcp_wnd_end(msk) - msk->snd_nxt);

-- 
2.34.1

Re: [PATCH mptcp-next v3 3/4] mptcp: sched: do not penalise when receive-window-limited

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] The `mptcp_snd_wnd_test()` heuristic incorrectly disables the slow-subflow penalty for standard bulk transfers by equating application queue depth with receive-window limitation.
--

--- Patch [3]: [PATCH mptcp-next v3 3/4] mptcp: sched: do not penalise when receive-window-limited ---
commit 97d5176c170e6dd87e10d2ec2eefd7e6a38340c4
Author: Shardul Bankar <shardul.b@mpiricsoftware.com>

mptcp: sched: do not penalise when receive-window-limited

This commit disables the slow-subflow cwnd penalty when the connection
is receive-window limited. It introduces mptcp_snd_wnd_test to skip the
penalty if the application has queued past the window edge.

> diff --git a/net/mptcp/protocol.c b/net/mptcp/protocol.c
> --- a/net/mptcp/protocol.c
> +++ b/net/mptcp/protocol.c
> @@ -1635,6 +1635,14 @@ static bool mptcp_penalise_throttle_ok(struct mptcp_subflow_context *subflow)
>  	return tcp_jiffies32 - subflow->last_penalise >= max_t(u32, rtt, 1);
>  }
>  
> +/* Like tcp_snd_wnd_test() but without an skb: true while queued data still fits
> + * the send window, i.e. not receive-window-limited.
> + */
> +static bool mptcp_snd_wnd_test(const struct mptcp_sock *msk)
> +{
> +	return !after64(msk->write_seq, mptcp_wnd_end(msk));

[Severity: High]
During a standard bulk data transfer, the application typically pushes data
into the socket buffer up to sk_sndbuf, which is often much larger than the
peer's advertised receive window.

Because of this, won't msk->write_seq naturally exceed mptcp_wnd_end(msk),
causing this test to return false even when the network is bottlenecked by
a small congestion window rather than the receive window?

If this evaluates to false during bulk transfers, doesn't it unconditionally
disable the slow-subflow penalty and prevent the intended shift of load to
faster subflows?

> +}
> +
>  /* Halve cwnd (and ssthresh if past it) under the subflow socket lock. */
>  static void mptcp_penalise_cwnd(struct sock *ssk)

[ ... ]

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260817-mptcp_penalise_send_v2-v3-0-e6a2ad2f1b82@mpiricsoftware.com?part=3

[PATCH mptcp-next v3 4/4] mptcp: sched: add penalise counters and tracepoint

[Shardul Bankar]

Add observability for the cwnd-halving penalty introduced in the previous
patches.

Two MPTcpExt SNMP counters:
- CwndPenalised: times a subflow cwnd was actually halved;
- PenalCandidate: times the rate trigger picked a slow subflow.

The two show whether the rate trigger fired at all and how many of those
candidates were actually halved; the gap reflects the gates and the
once-per-RTT limiter collectively, not any single condition.

Add a mptcp_subflow_penalise tracepoint. The penalise verdict is decided
after the subflow-selection loop, once the fastest path is known, so it is
exposed through a dedicated tracepoint rather than the per-candidate
mptcp_subflow_get_send emit. It reports the selected subflow's pace, the
fastest path's pace and its cwnd alongside the verdict, so a slow-but-gated
subflow and its window can be watched from a single line.

Co-developed-by: Matthieu Baerts (NGI0) <matttbe@kernel.org>
Signed-off-by: Shardul Bankar <shardul.b@mpiricsoftware.com>
---
 include/trace/events/mptcp.h | 30 ++++++++++++++++++++++++++++++
 net/mptcp/mib.c              |  2 ++
 net/mptcp/mib.h              |  2 ++
 net/mptcp/protocol.c         |  6 ++++++
 4 files changed, 40 insertions(+)

diff --git a/include/trace/events/mptcp.h b/include/trace/events/mptcp.h
index 22882bd03459..fe13f0c8a34d 100644
--- a/include/trace/events/mptcp.h
+++ b/include/trace/events/mptcp.h
@@ -68,6 +68,36 @@ TRACE_EVENT(mptcp_subflow_get_send,
 		  __entry->backup, __entry->ratio)
 );
 
+TRACE_EVENT(mptcp_subflow_penalise,
+
+	TP_PROTO(struct mptcp_subflow_context *subflow, unsigned long max_pace),
+
+	TP_ARGS(subflow, max_pace),
+
+	TP_STRUCT__entry(
+		__field(u64, pace)
+		__field(u64, max_pace)
+		__field(u32, cwnd)
+		__field(bool, penalise)
+	),
+
+	TP_fast_assign(
+		struct sock *ssk = mptcp_subflow_tcp_sock(subflow);
+
+		__entry->pace = subflow->avg_pacing_rate;
+		__entry->max_pace = max_pace;
+		if (ssk && sk_fullsock(ssk))
+			__entry->cwnd = tcp_snd_cwnd(tcp_sk(ssk));
+		else
+			__entry->cwnd = 0;
+		__entry->penalise = subflow->penalise;
+	),
+
+	TP_printk("pace=%llu max_pace=%llu cwnd=%u penalise=%d",
+		  __entry->pace, __entry->max_pace,
+		  __entry->cwnd, __entry->penalise)
+);
+
 DECLARE_EVENT_CLASS(mptcp_dump_mpext,
 
 	TP_PROTO(struct mptcp_ext *mpext),
diff --git a/net/mptcp/mib.c b/net/mptcp/mib.c
index 608cb568897c..b39b5511345d 100644
--- a/net/mptcp/mib.c
+++ b/net/mptcp/mib.c
@@ -95,6 +95,8 @@ static const struct snmp_mib mptcp_snmp_list[] = {
 	SNMP_MIB_ITEM("BacklogDrop", MPTCP_MIB_BACKLOGDROP),
 	SNMP_MIB_ITEM("RcvPruned", MPTCP_MIB_RCVPRUNED),
 	SNMP_MIB_ITEM("OFOPruned", MPTCP_MIB_OFOPRUNED),
+	SNMP_MIB_ITEM("CwndPenalised", MPTCP_MIB_CWNDPENALISED),
+	SNMP_MIB_ITEM("PenalCandidate", MPTCP_MIB_PENALCAND),
 };
 
 /* mptcp_mib_alloc - allocate percpu mib counters
diff --git a/net/mptcp/mib.h b/net/mptcp/mib.h
index 1ebdb55e9534..a7b5bf684af7 100644
--- a/net/mptcp/mib.h
+++ b/net/mptcp/mib.h
@@ -98,6 +98,8 @@ enum linux_mptcp_mib_field {
 	MPTCP_MIB_BACKLOGDROP,		/* Backlog over memory limit */
 	MPTCP_MIB_RCVPRUNED,		/* Dropped due to memory constraints */
 	MPTCP_MIB_OFOPRUNED,		/* MPTCP-level OoO queue pruned */
+	MPTCP_MIB_CWNDPENALISED,	/* subflow cwnd halved by the scheduler */
+	MPTCP_MIB_PENALCAND,		/* scheduler picked a slow (low-rate) subflow */
 	__MPTCP_MIB_MAX
 };
 
diff --git a/net/mptcp/protocol.c b/net/mptcp/protocol.c
index e4d26c2dce1e..0cd18705ab40 100644
--- a/net/mptcp/protocol.c
+++ b/net/mptcp/protocol.c
@@ -1656,6 +1656,7 @@ static void mptcp_penalise_cwnd(struct sock *ssk)
 		return;
 	subflow->last_penalise = tcp_jiffies32;
 	tcp_snd_cwnd_set(tp, max_t(u32, cwnd >> 1, MPTCP_PENALISE_MIN_CWND));
+	MPTCP_INC_STATS(sock_net(ssk), MPTCP_MIB_CWNDPENALISED);
 	if (cwnd >= tp->snd_ssthresh)
 		tp->snd_ssthresh = max_t(u32, tp->snd_ssthresh >> 1, 2);
 }
@@ -1739,6 +1740,8 @@ struct sock *mptcp_subflow_get_send(struct mptcp_sock *msk)
 	subflow = mptcp_subflow_ctx(ssk);
 	penal_cand = fastest && ssk != fastest &&
 		     subflow->avg_pacing_rate < max_pace / MPTCP_PENALISE_RATE_RATIO;
+	if (penal_cand)
+		MPTCP_INC_STATS(sock_net(ssk), MPTCP_MIB_PENALCAND);
 	subflow->penalise = penal_cand &&
 			    tcp_snd_cwnd(tcp_sk(ssk)) > MPTCP_PENALISE_MIN_CWND &&
 			    inet_csk(ssk)->icsk_ca_state == TCP_CA_Open &&
@@ -1746,6 +1749,9 @@ struct sock *mptcp_subflow_get_send(struct mptcp_sock *msk)
 			    mptcp_snd_wnd_test(msk) &&
 			    mptcp_penalise_throttle_ok(subflow);
 
+	/* trace the penalise verdict, decided here after the selection loop */
+	trace_mptcp_subflow_penalise(subflow, max_pace);
+
 	burst = min(MPTCP_SEND_BURST_SIZE, mptcp_wnd_end(msk) - msk->snd_nxt);
 	wmem = READ_ONCE(ssk->sk_wmem_queued);
 	if (!burst) {

-- 
2.34.1

Re: [PATCH mptcp-next v3 4/4] mptcp: sched: add penalise counters and tracepoint

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [Medium] The `mptcp_subflow_penalise` tracepoint logs an inaccurate verdict because it is emitted before the scheduler finalizes the decision to penalize a subflow.
- [Medium] Unprotected lockless plain reads and writes of TCP and subflow state violate concurrency rules, which will trigger KCSAN warnings.
--

--- Patch [4]: [PATCH mptcp-next v3 4/4] mptcp: sched: add penalise counters and tracepoint ---
commit 5337879700eb732d117ee91fe901b615a12833e3
Author: Shardul Bankar <shardul.b@mpiricsoftware.com>

mptcp: sched: add penalise counters and tracepoint

Add observability for the cwnd-halving penalty introduced in the previous
patches. It adds MPTcpExt SNMP counters and an mptcp_subflow_penalise
tracepoint.

> diff --git a/net/mptcp/protocol.c b/net/mptcp/protocol.c
> index e4d26c2dce1ee..0cd18705ab404 100644
> --- a/net/mptcp/protocol.c
> +++ b/net/mptcp/protocol.c
> @@ -1656,6 +1656,7 @@ static void mptcp_penalise_cwnd(struct sock *ssk)
>  		return;
>  	subflow->last_penalise = tcp_jiffies32;

[Severity: Medium]
Is this unprotected write safe from load tearing?

Since subflow->last_penalise is concurrently read locklessly in
mptcp_penalise_throttle_ok(), should this be updated using WRITE_ONCE()
to avoid KCSAN warnings?

> +	MPTCP_INC_STATS(sock_net(ssk), MPTCP_MIB_CWNDPENALISED);
>  	if (cwnd >= tp->snd_ssthresh)
>  		tp->snd_ssthresh = max_t(u32, tp->snd_ssthresh >> 1, 2);
>  }

[ ... ]

> @@ -1739,6 +1740,8 @@ struct sock *mptcp_subflow_get_send(struct mptcp_sock *msk)
>  	subflow = mptcp_subflow_ctx(ssk);
>  	penal_cand = fastest && ssk != fastest &&
>  		     subflow->avg_pacing_rate < max_pace / MPTCP_PENALISE_RATE_RATIO;
> +	if (penal_cand)
> +		MPTCP_INC_STATS(sock_net(ssk), MPTCP_MIB_PENALCAND);

[Severity: Medium]
Does this block trigger KCSAN data race warnings?

These lockless plain reads of TCP socket fields (tcp_snd_cwnd, icsk_ca_state,
tcp_is_cwnd_limited) execute under the MPTCP connection lock, while softirqs
may concurrently process incoming ACKs and update the states.

Do these need data_race() or READ_ONCE() annotations to meet kernel
concurrency standards?

>  	subflow->penalise = penal_cand &&
>  			    tcp_snd_cwnd(tcp_sk(ssk)) > MPTCP_PENALISE_MIN_CWND &&
>  			    inet_csk(ssk)->icsk_ca_state == TCP_CA_Open &&
> @@ -1746,6 +1749,9 @@ struct sock *mptcp_subflow_get_send(struct mptcp_sock *msk)
>  			    mptcp_snd_wnd_test(msk) &&
>  			    mptcp_penalise_throttle_ok(subflow);
>  
> +	/* trace the penalise verdict, decided here after the selection loop */
> +	trace_mptcp_subflow_penalise(subflow, max_pace);
> +

[Severity: Medium]
Will this tracepoint log an inaccurate verdict?

If there is no window room to send (!burst), the code immediately below
sets subflow->penalise = false.

Since the tracepoint is emitted before this window limitation check, could
it log penalise=1 even when the penalty is later canceled?

>  	burst = min(MPTCP_SEND_BURST_SIZE, mptcp_wnd_end(msk) - msk->snd_nxt);
>  	wmem = READ_ONCE(ssk->sk_wmem_queued);
>  	if (!burst) {

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260817-mptcp_penalise_send_v2-v3-0-e6a2ad2f1b82@mpiricsoftware.com?part=4