All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH mptcp-next v15 00/11] Reduce the differences between TCP and MPTCP for TLS usage
@ 2026-10-23  1:58 Geliang Tang
  2026-10-23  1:58 ` [PATCH mptcp-next v15 01/11] mptcp: drop the mptcp_ooo_try_coalesce() helper Geliang Tang
                   ` (11 more replies)
  0 siblings, 12 replies; 19+ messages in thread
From: Geliang Tang @ 2026-10-23  1:58 UTC (permalink / raw)
  To: mptcp; +Cc: Geliang Tang

From: Geliang Tang <tanggeliang@kylinos.cn>

v15:
 - Patch 2: drop mptcp_data_lock() in mptcp_inq_hint(), use READ_ONCE()
   and __test_and_clear_bit() instead. This issue is unrelated to this
   series and will be addressed in a separate patch later.
 - Patch 5: Guard mptcp_cleanup_rbuf() with read_copied > 0. Initialize/
   reset msk->read_copied. Drop redundant mptcp_recv_skb() in
   __mptcp_read_sock().
 - Patch 6: simplify memory ordering, drop smp_mb()/smp_rmb(), keep
   existing smp_mb__before_atomic(). Check SOCK_DONE inline.
 - Patch 9: drop tcp_sock_rate_check_app_limited() split, call
   tcp_rate_check_app_limited() directly as Paolo suggested. Add
   write_seq != snd_nxt early bail-out.
 - Patch 10: wrap assertion behind expect_all_features().

v14:
 - v13 can no longer be applied to the export branch, so it has been
   rebased here with the following corrections:
 - Patch 2: move locking in mptcp_inq_hint() to avoid races with ack_seq
   change from patch 3 to here. Update commit log.
 - Patch 3: add mptcp_data_lock() in __mptcp_move_skb() around copied_seq
   update. Remove local delta variable in __mptcp_ofo_queue(). Initialize
   msk->copied_seq to 0 in mptcp_propagate_state() fallback path.
 - Patch 4: add map_seq64 comparison in __mptcp_try_coalesce() for 64-bit
   sequence consistency.
 - New patch 11: rename mptcp_check_readable to mptcp_stream_is_readable,
   and align with TCP by calling sk_is_readable() for KTLS support.
 - https://patchwork.kernel.org/project/mptcp/cover/cover.1790734083.git.tanggeliang@kylinos.cn/

v13:
 - use smp_mb instead of smp_mb__before_atomic in mptcp_check_data_fin.
 - https://patchwork.kernel.org/project/mptcp/cover/cover.1788245079.git.tanggeliang@kylinos.cn/

v12:
 - Move sock_set_flag(SOCK_DONE) to after smp_mb__before_atomic() in
   mptcp_check_data_fin() and reuse the existing barrier rather than adding
   a new smp_wmb().
 - Sample SOCK_DONE before smp_rmb() in mptcp_inq_hint() and mptcp_inq(),
   so the smp_rmb() only orders SOCK_DONE read relative to the subsequent
   ack_seq/copied_seq reads.
 - https://patchwork.kernel.org/project/mptcp/cover/cover.1788171508.git.tanggeliang@kylinos.cn/

v11:
 - __mptcp_move_skb(): gate __mptcp_sync_rcv_sequence() on
   test_and_clear_bit(MPTCP_SYNC_SEQ) so the dummy mapping is only re-based
   after ack_seq = iasn.
 - a new patch to set and test SOCK_DONE.
 - use SOCK_DONE in mptcp_inq_hint() and mptcp_inq().
 - https://patchwork.kernel.org/project/mptcp/cover/cover.1788099847.git.tanggeliang@kylinos.cn/

v10:
 - Drop "mptcp: use atomic64_t for locklessly accessed u64 fields"
   (v9 patch 1) and the squash to bpf_burst test (v9 patch 11).
   Series reduced from 11 to 9 patches.
 - Revert all atomic64_read()/atomic64_set() on msk->ack_seq back to
   plain reads and WRITE_ONCE() in patches 1-4. Remove the local u64
   ack_seq variables that only existed to hold atomic64_read() results.
 - mptcp_inq_hint: move test_bit(MPTCP_SYNC_SEQ) inside mptcp_data_lock()
   to close the race between the flag check and the ack_seq read. Revert
   atomic64_read() to plain msk->ack_seq under the lock.
 - mptcp_inq (patch 6): use READ_ONCE() for both ack_seq and copied_seq
   instead of atomic64_read(). Add comment for the FIN subtraction,
   mirroring tcp_inq(). Update commit log.
 - https://patchwork.kernel.org/project/mptcp/cover/cover.1787888115.git.tanggeliang@kylinos.cn/

v9:
 - Extend atomic64_t conversion from only msk->ack_seq to all locklessly
   accessed u64 fields: write_seq, snd_nxt, snd_una, wnd_end,
   bytes_received, bytes_consumed.
 - Add mptcp_data_lock() in mptcp_inq_hint() to read ack_seq and
   copied_seq atomically.
 - Add read_copied field and drain it under mptcp_data_lock() in
   mptcp_read_complete() to fix the BH vs worker data race.
 - New patches: mptcp_inq/peek_len and sendmsg_locked proto_ops.
 - Squash to "selftests/bpf: Add bpf_burst scheduler & test".
 - https://patchwork.kernel.org/project/mptcp/cover/cover.1787827525.git.tanggeliang@kylinos.cn/

v8:
 - check MPTCP_SYNC_SEQ in mptcp_inq_hint:
       if (hint_val >= INT_MAX) {
               if (test_bit(MPTCP_SYNC_SEQ, &msk->cb_flags))
                       return 0;
               return INT_MAX;
       }
 - rename mptcp_sock_rate_check_app_limited to
   mptcp_rate_check_app_limited.
 - https://patchwork.kernel.org/project/mptcp/cover/cover.1787644449.git.tanggeliang@kylinos.cn/

v7:
 - Patch 1 is a new one to fix the existing issue Sashiko mentioned -
   using atomic64_t for msk->ack_seq.
 - Fix the map_subflow_seq setting in fallback mode in patch 4.
 - https://patchwork.kernel.org/project/mptcp/cover/cover.1787537436.git.tanggeliang@kylinos.cn/

v6:
 - Memory ordering for MPTCP_SYNC_SEQ: Add smp_wmb() before set_bit() in
   subflow_set_remote_key() and smp_rmb() after test_and_clear_bit() in
   __mptcp_move_skb() and mptcp_release_cb() to prevent data races on
   weakly-ordered architectures, ensuring ack_seq updates are visible
   before the flag is set and readers see the updated sequence after
   observing the flag.
 - Lockdep nested locking: Use lock_sock_fast_nested(ssk) instead of
   lock_sock_fast(ssk) in mptcp_sock_rate_check_app_limited() to suppress
   false positive recursive locking warnings when acquiring subflow socket
   locks while holding the parent MPTCP socket lock.
 - Receive window advertisement order: Swap mptcp_rcv_space_adjust() and
   mptcp_cleanup_rbuf() in mptcp_read_complete() to expand the receive
   buffer before evaluating window updates, ensuring ACKs advertise the
   new (larger) window size instead of delaying the advertisement until
   the next cycle.
 - https://patchwork.kernel.org/project/mptcp/cover/cover.1787446274.git.tanggeliang@kylinos.cn/

v5:
 - only patch 3 changed.
 - Fallback offset underflow: Initialize MPTCP sequence space to 0 in
   mptcp_propagate_state() when mp_opt == NULL, ensuring SKB's map_seq
   starts from 0 to match msk->copied_seq and prevent offset calculation
   underflow in fallback mode.
 - Stale peek_seq: Move peek_seq recomputation into the mptcp_move_skbs()
   continue path and add another recomputation after sk_wait_data(),
   ensuring peek_seq stays synchronized with msk->copied_seq updates from
   MPTCP_SYNC_SEQ processing in both paths.
 - https://patchwork.kernel.org/project/mptcp/cover/cover.1787368526.git.tanggeliang@kylinos.cn/

v4:
 - mptcp_recvmsg: move the peek_seq recompute from after sk_wait_data()
   to before the mptcp_move_skbs() continue check.
   mptcp_move_skbs() -> __mptcp_move_skb() may shift the head skb's
   map_seq to the IASN frame via __mptcp_sync_rcv_sequence(); if
   'continue' then exits the loop, the recompute is skipped, and the
   next __mptcp_recvmsg_mskq() computes offset with peek_seq in the old
   frame minus map_seq in the new frame. The underflow makes the offset
   test fail and the skb is skipped (and mptcp_recv_skb() actually frees
   the TFO skb via mptcp_eat_recv_skb()).
 - Make all three MPTCP_SYNC_SEQ bit accesses atomic:
   - subflow_set_remote_key:   __set_bit            -> set_bit
   - __mptcp_move_skb:         __test_and_clear_bit -> test_and_clear_bit
   - mptcp_release_cb:         __test_and_clear_bit -> test_and_clear_bit
   The flag is set in BH under mptcp_data_lock() (msk slock held) and
   cleared in user context under lock_sock() slow path (only owned=1,
   slock not held). The non-atomic RMW on the two sides races, which can
   lose the IASN sync.
 - https://patchwork.kernel.org/project/mptcp/cover/cover.1787295147.git.tanggeliang@kylinos.cn/

v3:
 - Patch 2, handle "offset" in __mptcp_sync_rcv_sequence().
 - Patch 3, update __mptcp_move_skb() in response to Sashiko comments:

 	if (__test_and_clear_bit(MPTCP_SYNC_SEQ, &msk->cb_flags))
		msk->copied_seq += mptcp_iasn(msk);

 - Patch 4, replace after64() to after() in mptcp_prune_ofo_queue(). The
   pre-existing issue raised by Sashiko regarding changing the type of
   ack_seq to atomic64_t is not addressed in this series.
 - https://patchwork.kernel.org/project/mptcp/cover/cover.1786445142.git.tanggeliang@kylinos.cn/

v2:
 - Patch 3, updated in response to Sashiko comments:

        if (unlikely(msk->rcvd_dummy_seq)) {
                msk->copied_seq += mptcp_iasn(msk);
                __mptcp_sync_rcv_sequence(sk);
                /* Release cb() would otherwise re-base copied_seq
                 * again.
                 */
                test_and_clear_bit(MPTCP_SYNC_SEQ, &msk->cb_flags);
        }

        /* Skip the already peeked data. */
        if (offset >= skb->len) {
                *last = skb;
                continue;
        }
 - Patch 5, replaced with Paolo's patch.
 - Patch 6, 7, new patches addressing app-limited conditions.
 - The patch "trim the duplicated skb head at receive enqueue" has been
   dropped, as it is no longer needed after Paolo updated the "mptcp:
   out-of-order queue pruning" series.
 - https://patchwork.kernel.org/project/mptcp/cover/cover.1786158416.git.tanggeliang@kylinos.cn/

v1:
 - https://patchwork.kernel.org/project/mptcp/cover/cover.1785150300.git.tanggeliang@kylinos.cn/

The goal of this series is to reduce the differences between TCP and MPTCP
for TLS usage, in preparation for adding TLS over MPTCP support in the
future.

In previous versions [1], a struct tls_prot_ops was defined to represent
the interface differences between TCP and MPTCP, which contained the
following callbacks:

	struct sk_buff *(*recv_skb)(struct sock *sk, u32 *off);
	bool (*lock_is_held)(struct sock *sk);
	void (*read_done)(struct sock *sk, size_t len);
	u32 (*get_skb_seq)(struct sk_buff *skb);
	int (*skb_get_header)(const struct sk_buff *skb, int offset,
	                      void *to, int len);
	bool (*epollin_ready)(const struct sock *sk);
	void (*check_app_limited)(struct sock *sk);

In reality, some of these callbacks are unnecessary. This series aims to
eliminate the get_skb_seq(), skb_get_header(), and lock_is_held()
callbacks.

The first four patches come from Paolo's "mptcp: address stall under memory
pressure" series v5 [2], with only minor cleanup from my side.

They remove the CB offset field and sync the MPTCP skb CB layout with the
TCP one, so that we can obtain the TCP or MPTCP sequence number in a
unified way, e.g.:

	struct tls_skb_cb {
	    u32 seq;
	};

	#define TLS_SKB_CB(__skb) ((struct tls_skb_cb *)&((__skb)->cb[0]))

This eliminates the need for a separate get_skb_seq() callback.

Building on the removal of the CB offset field, I also added patch 5 that
trims the duplicated skb head at receive enqueue. With that in place, KTLS
can retrieve the record header via skb_copy_bits() directly, so there is no
longer any need for a dedicated MPTCP helper like mptcp_skb_get_header().
The skb_get_header() callback can thus be removed.

Patch 6 defers sk_data_ready to the worker, which avoids recursive locking
when TLS calls back into MPTCP under mptcp_data_lock(). With this change,
the lock_is_held() callback is no longer needed and can be removed.

[1]
https://patchwork.kernel.org/project/mptcp/cover/cover.1782123118.git.tanggeliang@kylinos.cn/
[2]
https://patchwork.kernel.org/project/mptcp/cover/cover.1778446731.git.pabeni@redhat.com/

Gang Yan (1):
  mptcp: update mptcp_check_readable

Geliang Tang (5):
  mptcp: align FIN handling with TCP via SOCK_DONE
  mptcp: implement peek_len for proto_ops
  mptcp: add sendmsg_locked to proto_ops
  mptcp: track app-limited state in mptcp_sendmsg
  selftests: mptcp: sockopt: check app_limited

Paolo Abeni (5):
  mptcp: drop the mptcp_ooo_try_coalesce() helper
  mptcp: drop the cant_coalesce CB field
  mptcp: remove CB offset field
  mptcp: sync mptcp skb cb layout with tcp one
  mptcp: defer read_sock cleanup to mptcp_worker

 net/mptcp/fastopen.c                          |  17 +-
 net/mptcp/protocol.c                          | 365 ++++++++++++------
 net/mptcp/protocol.h                          |  21 +-
 net/mptcp/subflow.c                           |  23 ++
 .../selftests/net/mptcp/mptcp_sockopt.c       |   5 +
 5 files changed, 309 insertions(+), 122 deletions(-)

-- 
2.55.0


^ permalink raw reply	[flat|nested] 19+ messages in thread
* Re: [PATCH mptcp-next v15 05/11] mptcp: defer read_sock cleanup to mptcp_worker
@ 2026-10-24 16:26 netdev-bot+sashiko
  0 siblings, 0 replies; 19+ messages in thread
From: netdev-bot+sashiko @ 2026-10-24 16:26 UTC (permalink / raw)
  To: geliang; +Cc: mptcp, yangang, kuba

Thank you for your contribution! Sashiko AI review found more severe
issue(s) in other patches of this series. This mail is only an FYI, in case
you want to address the nit picks below when you respin.

Low priority / nit picks for your consideration:

- [Low] msk->read_copied (struct mptcp_sock, protocol.h:330) is a signed
  int.
- [Low] The patch removes msk_owned_by_me() from __mptcp_read_sock() on
  the grounds that 'the BH caller does not own the msk lock'.

--

mptcp: defer read_sock cleanup to mptcp_worker

This commit prepares for kTLS-over-MPTCP, where __mptcp_read_sock() can be
reached from BH context. mptcp_rcv_space_adjust() and mptcp_cleanup_rbuf()
move out of __mptcp_read_sock() and into mptcp_worker(). The consumed byte
count goes into a new msk->read_copied field, and the worker is signalled
through MPTCP_WORK_READ_COMPLETE. The commit also removes the backlog flush
from mptcp_recv_skb() and the msk_owned_by_me() assertion from
__mptcp_read_sock().

> diff --git a/net/mptcp/protocol.c b/net/mptcp/protocol.c
> index 01eb7fb314e76..3c3ad8991799a 100644
> --- a/net/mptcp/protocol.c
> +++ b/net/mptcp/protocol.c

[ ... ]

> @@ -4655,6 +4672,7 @@ static struct sk_buff *mptcp_recv_skb(struct sock *sk, u32 *off)
>  /*
>   * Note:
>   *	- It is assumed that the socket was locked by the caller.
> + *	- Can be invoked in BH scope.
>   */
>  static int __mptcp_read_sock(struct sock *sk, read_descriptor_t *desc,
>  			     sk_read_actor_t recv_actor, bool noack)
> @@ -4664,8 +4682,6 @@ static int __mptcp_read_sock(struct sock *sk, read_descriptor_t *desc,
>  	int copied = 0;
>  	u32 offset;
>  
> -	msk_owned_by_me(msk);
> -
>  	if (sk->sk_state == TCP_LISTEN)
>  		return -ENOTCONN;
>  	while ((skb = mptcp_recv_skb(sk, &offset)) != NULL) {

[Severity: Low]
Is the reason given for dropping this assertion accurate? The commit
message says:

  "drop the msk_owned_by_me() assertion from __mptcp_read_sock() (the BH
  caller does not own the msk lock)"

msk_owned_by_me() ends up in lockdep_sock_is_held(), and that function
also accepts the socket spinlock:

include/net/sock.h:lockdep_sock_is_held() {
	return lockdep_is_held(&sk->sk_lock) ||
	       lockdep_is_held(&sk->sk_lock.slock);
}

The expected BH entry point is sk_data_ready, called from
mptcp_data_ready(). That path runs under mptcp_data_lock() and checks
ownership:

net/mptcp/protocol.c:mptcp_data_ready() {
	...
	mptcp_data_lock(sk);
	mptcp_rcv_rtt_update(msk, subflow);
	if (!sock_owned_by_user(sk)) {
	...
}

A caller like that would pass the assertion, so removing it only loses
lockdep coverage.

If the planned BH caller holds neither lock, __mptcp_read_sock() would do
several things without serialization:

  - unlink skbs from sk_receive_queue through mptcp_eat_recv_skb()
  - update copied_seq and bytes_consumed
  - do a plain "msk->read_copied += copied"

These would race with __mptcp_move_skbs_from_subflow(). They would also
race with mptcp_read_complete(), which reads and clears read_copied under
mptcp_data_lock().

The only current in-tree caller, mptcp_splice_read(), holds lock_sock, so
there is no live race today.

The header comment now says both "It is assumed that the socket was
locked by the caller" and "Can be invoked in BH scope". The new
read_copied field also has no annotation saying which lock protects it.

Could the assertion stay, with the locking rules for read_copied written
down?

[ ... ]

> @@ -4696,11 +4712,14 @@ static int __mptcp_read_sock(struct sock *sk, read_descriptor_t *desc,
>  	if (noack)
>  		goto out;
>  
> -	mptcp_rcv_space_adjust(msk, copied);
> -
> +	/* The backlog flushing is only needed when some data is actually
> +	 * moved and will take place in mptcp_release_cb() via
> +	 * release_sock() in the worker.
> +	 */
>  	if (copied > 0) {
> -		mptcp_recv_skb(sk, &offset);
> -		mptcp_cleanup_rbuf(msk, copied);
> +		msk->read_copied += copied;

[Severity: Low]
Can msk->read_copied overflow here? It is a signed int, and nothing
resets it until mptcp_worker() runs mptcp_read_complete(). If more than
INT_MAX bytes are consumed through read_sock/splice before the worker
drains the counter, this addition wraps.

The commit message says:

  "To prevent integer overflow when read_copied wraps to negative values,
  mptcp_read_complete() only calls mptcp_cleanup_rbuf() when
  read_copied > 0."

That check runs on the consumer side, after the wrap has already
happened:

net/mptcp/protocol.c:mptcp_read_complete() {
	...
	mptcp_rcv_space_adjust(msk, read_copied);
	if (read_copied > 0)
		mptcp_cleanup_rbuf(msk, read_copied);
}

With a negative value, mptcp_rcv_space_adjust() returns early:

	if (copied <= 0)
		return;

and mptcp_cleanup_rbuf() is skipped. The autotune sample and the window
update for the whole batch are dropped without any warning. If the value
wraps past zero again, both functions get an arbitrary positive count.

Hitting this needs the worker to stay off the CPU for more than 2 GiB of
reads. That could be a starved system_wq kworker, or a splicer that keeps
winning lock_sock. Each read_sock call is limited by what is in the
receive queue, so the wrap only happens across many calls.
CONFIG_UBSAN_SIGNED_WRAP would also report the wrap.

Would a u64 accumulator, or saturating before the add, avoid the wrap?
The commit message would then need updating, since the read_copied > 0
check does not prevent the overflow.

> +		set_bit(MPTCP_WORK_READ_COMPLETE, &msk->flags);
> +		mptcp_schedule_work(sk);
>  	}
>  out:
>  	return copied;

[ ... ]

-- 
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/cover.1792718041.git.tanggeliang%40kylinos.cn

^ permalink raw reply	[flat|nested] 19+ messages in thread
end of thread, other threads:[~2026-10-24 16:26 UTC | newest]

Thread overview: 19+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-10-23  1:58 [PATCH mptcp-next v15 00/11] Reduce the differences between TCP and MPTCP for TLS usage Geliang Tang
2026-10-23  1:58 ` [PATCH mptcp-next v15 01/11] mptcp: drop the mptcp_ooo_try_coalesce() helper Geliang Tang
2026-10-23  1:58 ` [PATCH mptcp-next v15 02/11] mptcp: drop the cant_coalesce CB field Geliang Tang
2026-10-23  2:35   ` sashiko-bot
2026-10-23  1:58 ` [PATCH mptcp-next v15 03/11] mptcp: remove CB offset field Geliang Tang
2026-10-23  1:58 ` [PATCH mptcp-next v15 04/11] mptcp: sync mptcp skb cb layout with tcp one Geliang Tang
2026-10-23  2:35   ` sashiko-bot
2026-10-23  1:58 ` [PATCH mptcp-next v15 05/11] mptcp: defer read_sock cleanup to mptcp_worker Geliang Tang
2026-10-23  2:29   ` sashiko-bot
2026-10-23  1:58 ` [PATCH mptcp-next v15 06/11] mptcp: align FIN handling with TCP via SOCK_DONE Geliang Tang
2026-10-23  2:29   ` sashiko-bot
2026-10-23  1:58 ` [PATCH mptcp-next v15 07/11] mptcp: implement peek_len for proto_ops Geliang Tang
2026-10-23  2:19   ` sashiko-bot
2026-10-23  1:58 ` [PATCH mptcp-next v15 08/11] mptcp: add sendmsg_locked to proto_ops Geliang Tang
2026-10-23  1:58 ` [PATCH mptcp-next v15 09/11] mptcp: track app-limited state in mptcp_sendmsg Geliang Tang
2026-10-23  1:58 ` [PATCH mptcp-next v15 10/11] selftests: mptcp: sockopt: check app_limited Geliang Tang
2026-10-23  1:58 ` [PATCH mptcp-next v15 11/11] mptcp: update mptcp_check_readable Geliang Tang
2026-10-23  3:11 ` [PATCH mptcp-next v15 00/11] Reduce the differences between TCP and MPTCP for TLS usage MPTCP CI
  -- strict thread matches above, loose matches on Subject: below --
2026-10-24 16:26 [PATCH mptcp-next v15 05/11] mptcp: defer read_sock cleanup to mptcp_worker netdev-bot+sashiko

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.